ContinualAI Seminar: "Prompting-based Continual Learning"
Abstract: The mainstream paradigm behind continual learning has been to adapt the model parameters to non-stationary data distributions, where catastrophic forgetting is the central challenge. In this talk, we present a new continual learning paradigm – Prompting-based Continual Learning, which learns a tiny set of parameters, called prompts, to properly instruct a pre-trained model to learn tasks arriving sequentially. In our CVPR 22 work, Learning to Prompt (L2P), we design a key-value paired pool of prompts to dynamically instruct the backbone to learn coming tasks. In our ECCV 22 work, DualPrompt, we further improve the design of prompts by decoupling them into complementary “General” and “Expert” prompts to learn task-invariant and task-specific instructions, respectively. Both methods set new state-of-the-art performance on multiple challenging benchmarks, even without buffering examples. We hope that Prompting-based Continual Learning can provide a different perspective for solving frontier challenges in continual learning.
You can discuss about this seminar in our forum: https://continualai.discourse.group/c/reading-group
------------------------ About ContinualAI ------------------------
ContinualAI is an official non-profit research organization and the largest open community on Continual Learning for AI. We aim at connecting people and working better together on this fascinating topic we consider fundamental for the future of AI.
• Official website: https://www.continualai.org
• Join us now at https://www.continualai.org/about-us/join-us
Please consider supporting us with a small donation at: https://www.continualai.org/about-us/supporters
It's thanks to people like you that we are making ContinualAI a reality!
0 Comments