Regularization Shortcomings for Continual Learning

Abstract

In most machine learning algorithms, training data are assumed independent and identically distributed (iid). Otherwise, the algorithms’ performances are challenged. A famous phenomenon with non-iid data distribution is known as catastrophic forgetting. Algorithms dealing with it are gathered in the Continual Learning research field. In this article, we study the regularization based approaches to continual learning. We show that those approaches can not learn to discriminate classes from different tasks in an elemental continual benchmark: class-incremental setting. We make theoretical reasoning to prove this shortcoming and illustrate it with examples and experiments.

Arxiv PDF

Bibtex

@misc{lesort2020regularization,
      title={Regularization Shortcomings for Continual Learning}, 
      author={Timothée Lesort and Andrei Stoian and David Filliat},
      year={2020},
      eprint={1912.03049},
      archivePrefix={arXiv},
      primaryClass={cs.LG}
}