CRAFT: Forgetting-Aware Intervention-Based Adaptation for Continual Learning

Md Anwar Hossen, Fatema Siddika, Juan Pablo Muñoz, Tanya Roosta, and Ali Jannesari

Under submission, 2026

TL;DR. CRAFT adapts a model to a stream of new tasks by intervening directly on the representations that matter for each task, while explicitly protecting the knowledge needed for earlier tasks, reducing catastrophic forgetting without storing or replaying old data.

Motivation

Continually adapting large models to new tasks usually comes at a cost: each update risks overwriting what the model already knew, a problem known as catastrophic forgetting. Standard fine-tuning and many parameter-efficient methods still drift, especially when tasks arrive sequentially and prior data is unavailable for replay.

Approach

CRAFT frames adaptation as a targeted intervention on hidden representations rather than a global parameter overwrite. It identifies the representation directions that a new task needs, applies a controlled adjustment along those directions, and constrains updates so that directions important to previously learned tasks remain stable. The result is a forgetting-aware update rule that integrates new knowledge while preserving old.

Intervention-based adaptation localized to task-relevant representations.
Forgetting-aware constraints that shield previously acquired knowledge.
No replay buffer or access to prior task data required.

Importance

For deployed foundation models that must keep learning across domains, users, or time CRAFT offers a path to continual adaptation that is both data-efficient and stable, an important property for privacy-sensitive and resource-constrained settings.