Learning to Prompt for Continual Learning
L2P introduces a prompt pool with instance-wise query for continual learning without task labels, outperforming buffer-based methods.
Key Findings
Methodology
L2P constructs a prompt pool of learnable parameters stored in a shared memory, dynamically queried via a similarity measure (cosine distance) based on input features extracted from a frozen pre-trained vision transformer. The selected prompts are prepended to input embeddings, guiding the model's predictions. The training optimizes both classification loss and prompt matching distance, enabling task-invariant and task-specific knowledge separation. This approach avoids task labels and buffers, effectively mitigating catastrophic forgetting across diverse benchmarks, including class-incremental, domain-incremental, and task-agnostic scenarios.
Key Results
- On Split CIFAR-100, L2P achieves 83.83% average accuracy without rehearsal buffer, surpassing EWC (47.01%) and LwF (60.69%). With a buffer of 5 datasets, accuracy reaches 81.14%, outperforming many buffer-based methods. In CORe50 domain-incremental tasks, it attains 78.33%, demonstrating robustness across scenarios.
- In the high-difficulty Gaussian scheduled CIFAR-100, L2P reaches 88.34% accuracy, outperforming DER++ (85.24%) and other buffer methods. Ablation studies show that prompt pool size and instance-wise query are critical for performance, validating the design choices.
- Across multiple benchmarks, L2P maintains competitive or superior performance even with minimal or no rehearsal buffer, highlighting its potential for privacy-preserving and resource-constrained applications.
Significance
L2P shifts the paradigm from buffer-dependent rehearsal to parameter-efficient prompt-based knowledge storage, enabling task-agnostic continual learning. It addresses core issues of memory cost, privacy, and scalability, offering a flexible, scalable solution for real-world applications like autonomous driving, medical diagnosis, and edge AI. Its ability to dynamically select relevant knowledge per instance enhances generalization and reduces catastrophic forgetting, marking a significant step forward in continual learning research.
Technical Contribution
The paper introduces a novel prompt pool mechanism with instance-wise query, decoupling shared and task-specific knowledge. It employs a matching distance-based prompt selection, integrated into a pre-trained frozen backbone, enabling efficient knowledge transfer without task labels or buffers. The optimization jointly minimizes classification loss and prompt matching, providing theoretical and empirical guarantees of reduced interference and improved plasticity. This framework broadens the scope of prompt tuning from transfer learning to continual learning, opening new avenues for model compression and fast adaptation.
Novelty
This work is the first to embed prompt learning into continual learning, proposing a task-agnostic, buffer-free framework that dynamically selects prompts per instance. Unlike prior methods relying on explicit task labels or large memory buffers, L2P leverages a learned prompt pool and similarity-based retrieval, enabling flexible, scalable, and privacy-preserving continual learning. Its innovative use of prompt mechanisms for knowledge management marks a new frontier in the field.
Limitations
- The prompt pool size and retrieval mechanism may face scalability issues in extremely large or highly diverse task environments, potentially impacting efficiency.
- Performance depends on the quality of input features extracted from the frozen backbone; in low-resource or noisy scenarios, effectiveness may decline.
- The method assumes a pre-trained model with strong generalization; transferability to less capable models or domains remains to be validated.
Future Work
Future directions include expanding prompt pool capacity with hierarchical or modular structures, integrating meta-learning for prompt adaptation, and extending to multi-modal and multi-task settings. Additionally, optimizing retrieval efficiency and exploring adaptive prompt update strategies could further enhance scalability. Combining this approach with continual meta-learning frameworks may enable even faster adaptation to new tasks, broadening its applicability in real-world, resource-constrained environments.
AI Executive Summary
Continual learning aims to enable models to acquire new knowledge over time without forgetting previous information. Traditional approaches often rely on rehearsal buffers or explicit task labels, which pose challenges in privacy, scalability, and real-world deployment. This paper introduces Learning to Prompt (L2P), a novel framework that leverages a shared prompt pool and an instance-wise query mechanism to dynamically select task-relevant prompts, guiding a frozen pre-trained vision transformer to perform sequential tasks. Unlike conventional methods, L2P does not require task labels or buffers, significantly reducing memory overhead and privacy concerns.
The core idea is to store multiple small, learnable prompts in a shared pool, which are retrieved based on input features through a similarity measure. These prompts act as task-specific instructions, enabling the model to adapt flexibly to new tasks while maintaining high plasticity. The prompts are prepended to input embeddings, effectively instructing the model without modifying the backbone parameters. The training optimizes both classification accuracy and prompt relevance, ensuring shared knowledge transfer and task-specific adaptation.
Extensive experiments on benchmarks like CIFAR-100, CORe50, and Gaussian scheduled CIFAR-100 demonstrate that L2P outperforms state-of-the-art rehearsal-based and architecture-based methods, especially in buffer-free and task-agnostic settings. Results show accuracy improvements of over 30% compared to regularization methods and competitive performance against buffer-dependent approaches, highlighting its practical value.
This approach fundamentally advances continual learning by offering a scalable, memory-efficient, and task-agnostic solution. It opens new avenues for deploying models in privacy-sensitive and resource-limited environments, such as edge devices and autonomous systems. Despite its strengths, challenges remain in scaling prompt pools for extremely large task spaces and ensuring robustness in noisy or low-resource scenarios. Future work will focus on hierarchical prompt structures, meta-learning integration, and multi-modal extensions, aiming to further enhance adaptability and efficiency in real-world applications.
Deep Dive
Plain Language Accessible to non-experts
想象你在一个厨房里做饭。每次做不同的菜，你都需要用不同的调料和工具。有时候，厨房里有很多调料瓶，但你不想每次都重新记住所有的调料用法。于是，你把常用的调料放在一个调料盒里，每次做菜时，从调料盒里挑出合适的调料贴在菜上，告诉厨师该用哪个。厨师看了调料，就知道怎么做菜了。这个方法就像L2P，用一个调料池存放不同的提示（调料），根据菜的不同自动挑选出合适的调料，帮助厨师（模型）在不断变化的菜谱中学习，既节省空间，又能灵活应对各种菜肴。
ELI14 Explained like you're 14
想象你在学校里学不同的科目，比如数学、英语和科学。每个科目都有自己的老师和教材，但你不想每次都重新学一遍所有内容。于是，老师给你一些小卡片（提示），上面写着“数学问题”、“英语句子”或者“科学实验”。每次考试前，老师会根据题目选择合适的卡片，把它贴在你的书上，告诉你该用哪个方法解题。这样，你就不用记住所有的知识点，只需要看一看卡片，就知道怎么做。L2P就像这个老师，用一组小卡片存储不同任务的“指令”，根据需要自动选择，帮助模型在连续学习中记住新知识，又不忘旧的，就像你在学校里灵活应对各种科目一样！
Abstract
The mainstream paradigm behind continual learning has been to adapt the model parameters to non-stationary data distributions, where catastrophic forgetting is the central challenge. Typical methods rely on a rehearsal buffer or known task identity at test time to retrieve learned knowledge and address forgetting, while this work presents a new paradigm for continual learning that aims to train a more succinct memory system without accessing task identity at test time. Our method learns to dynamically prompt (L2P) a pre-trained model to learn tasks sequentially under different task transitions. In our proposed framework, prompts are small learnable parameters, which are maintained in a memory space. The objective is to optimize prompts to instruct the model prediction and explicitly manage task-invariant and task-specific knowledge while maintaining model plasticity. We conduct comprehensive experiments under popular image classification benchmarks with different challenging continual learning settings, where L2P consistently outperforms prior state-of-the-art methods. Surprisingly, L2P achieves competitive results against rehearsal-based methods even without a rehearsal buffer and is directly applicable to challenging task-agnostic continual learning. Source code is available at https://github.com/google-research/l2p.