Introduction
Continual learning research field gathered all together all machine learning solution that aim at dealing with not i.i.d. data. This can of solution can be used for incremental classification learning, or for agent learning in a evolving environment or for algorithms that aims at learning from continual stream of data from network activities.
All of those settings have been collected into one single big field: Continual Learning. Indeed, they are all linked to a commune objective, learning new informations/skills without loosing old ones: Learning without forgetting.
However, I am writing this blog post because for some times already I did not make the clear distintion between all settings because I thought that all of them have the same objective, so why would it matter to make distinction.
The fact is, even if the global objectives is the same for all, it does not mean exactly the same thing for all and all settings can make different assumption and may have different criterion of success. In order to go forward, it is probably more effective to split continual learning into several fields and address them separately rather than trying to find a global solution that work for everything: some says that we will never find any free lunch.
So maybe the pletory of similar terms for continual learning as: learning without forgetting, incremental learning, online learning, lifelong learning, never ending learning, open-ended learning … might be have a good use. It seems to me that they are used a bit ramdomly depending on the author believes and a priori. I’d like to propose a continual subdivision to have clearer idea on which continual learning algorithms is done for what: “divide and conquer”
First, lets take two examples of continual algorithms problems that would be very hard for an algorithms to deal with both. I hope to convince you of the need of a more precise terminology.
- robot learning vs incremental classification
Terminology proposition: I will try to use already existing terms to not create more confusion.
lifelong learning: an agent that learn a specific skill in a continual fashion, the examples is the robot that learns to packs christmas gift. The skill to learn is always the same but the gift may change their shape and the paper craft miht change to. Forcing the algorithms to learn continually without forgetting.
Incremental learning: For me this is linked to stacking knowledge into a model, like for classification, the model learn knowledge one after one and stack them into its “skills library”. It is to accumulate knowledge rather than improving them continuously like it is the case for lifelong learning.
Online learning: Is maybe the terms with the more agreement in the community, an algorithm learning by receiving data point one by one. However, for me it is linked to an intrinsec characteristics of the data stream and to the action that should taken. It is not a design choice. Online learning is an extrem case were all new data point should immediately integrate into the knowledge base to be used. In the case where the algorithms has time to gather data and creating batches, this is not really a online learning approaches in my opinon.
Never-ending learning: Could be a composition of incremental and lifelong learning, there is a set of specific objective that evolves through time and that should be after improves by repetitions.
Open-ended learning: is maybe the more biological inspired family, the goal is to learn anything the it is possible to learn, it is really linked to curiosity I think. It is a bit similar to a baby who born and which should learn stuf. The algorithms should then learn from specific objectives that are given to its but maybe learning also to set it’s own objectives, and maybe learning to learn more efficiently. So, maybe an algorithms with a very high level objective like “survive!”. Like us human maybe.
Learning without forgetting is the spirit of continual learning not a family of approaches, I would say …
Forgetting assumption
Now, we have our distinction between several family of continual learning approaches. We can use them to make assumptions: For example, it is not always clear what is acceptable to forget or not. Baby forget a lot of things but at the end they often converge to a high amount of knowledge without having the restart the process from zero.
That is what we would like to realize in fact in CL, not really learning without forgetting at all but learning and converging to a bigger ammount of knowledge than in the begining. This really true for all, the real life inspired continual learning settings like, lifelong-learning/ never-ending learnin and open-ended learning. Considering incremental learning, it is more clear, the learning model have some knowledge, we want to add new one, we don’t want to forget at all.
I would say that the need of absolut remembering is classified in this order: Incremental learning > lifelong learning > never-ending > Open-ended learning
The more large the field of learning is the more acceptable forgetting is. Maybe a more interesting subject to study is not the capacity at remembering but the capacity of reactivation of memory. As a human, I don’t really care to remember exactly how to put a life-jacket but I would like to learn fast how to put it if I need to.
Future assumption
The environment change a lot the input are varied and entangled the goals will change through time Is it autorized to forget How much knowledge will the model need to remember ? -> do I need a dynamic architecture ? What degree of generalization do I need ?
Forgetting assumption
Precise terminology helps to target specific problems and find a solution and helps to communicate between researchers. The big problem is often to find an agreement on the terminology … Tell if you have better idea or other proposition of a more precise terminology for continual learnig, put a comment or send me a PM.