[CVPR 2022] Research Trend of Continual Learning

In 2016, AlphaGo, a program utilizing artificial neural networks, defeated human player Sedol Lee in a game of Go, which had been previously regarded as being untouchable by artificial intelligence. Since then, artificial neural networks have continued to advance and outperform humans in a variety of fields including image classification or produce unexpected outcomes in the natural language processing fields, etc. In the context of learning a single task, the artificial neural network-based learning method is achieving excellent results. However, there is still a significant gap between humans and machines in their capacity for “continual learning,” or the ability to learn multiple tasks in succession. Even if a human learns a new task while simultaneously learning a continuous task, the information on the previously learned task is not completely forgotten. However, catastrophic forgetting occurs in artificial neural networks, where the majority of information about previously learned tasks is lost. In an effort to narrow this gap, the field of continual learning research has received considerable attention in the past, resulting in the publication of numerous research findings. This post will briefly introduce two research directions that represent the most recent research trends among continual learning-related studies presented at CVPR 2022.

Research Direction 1: Research on Continual Learning Algorithm Using Transformer Model

Recent applications of transformer-based models to computer vision have demonstrated comparable performance to that of Convolutional Neural Network (CNN)-based models in a variety of tasks. Consequently, continual learning researchers, who have previously only studied conventional CNN (or Fully Connected Neural Network)-based models, are beginning to develop an interest in continual learning algorithms for transformer-based models. The following is a summary of a representative related paper[1] that was presented at CVPR 2022.

(Figure 1) DyTox Example

Using Transformer model, the paper [1] proposed a method called dynamic token expansion (DyTox) for continual learning. Existing research has demonstrated the effectiveness of an algorithm that performs continual learning while dynamically extending a CNN-based model in overcoming catastrophic forgetting; however, each algorithm has the drawback of being difficult to implement in practice or requiring the tuning of numerous hyperparameters. This paper, however, proposes an algorithm called DyTox that can outperform existing algorithms without requiring complex processes by utilizing the Transformer model's characteristics with relatively small model expansion. As shown in [Figure 1], when the encoder and decoder of Transformer are shared in a continual learning scenario, DyTox proposed a method for learning a classifier (clf) via binary cross-entropy utilizing task-specific embedding ($e_i$) generated from task tokens (${\theta }_t$) and encoder output ($x_5$). As a result, it was demonstrated that the proposed algorithm achieves excellent results in experiments with different datasets (e.g., Imagenet-100, ImageNet-1000, and CIFAR-100) starting with the same random initialization.

Research Direction 2: Research on Algorithms for Unsupervised (or Self-Supervised) Continual Learning

Recent research results on self-supervised learning indicate that it is possible to learn high-quality representations even in unsupervised learning situations in which classifiers are learned without label information. Based on these findings, research on unsupervised (or self-supervised) continual learning algorithms is commencing in earnest. Two papers^[2,3] were presented at the conference, the contents of which are summarized below.

Interestingly, both papers demonstrate experimentally that continual learning with self-supervised contrastive learning suffers less catastrophic forgetting than continual learning with conventional supervised learning using cross entropy. However, considering self-supervised and supervised situations in various experimental and continual learning scenarios, [2] demonstrated that the contrastive learning-based loss function experiences less catastrophic forgetting in continual learning situations. In contrast, [3] focuses on self-supervised learning and compares the results of continual learning using different types of self-supervised learning algorithms. In addition, it proposes a simple idea (CaSSLe) that can assist in overcoming catastrophic forgetting in continual learning situations using self-supervised learning, and experimental results indicate that higher performance can be attained than with supervised fine-tuning.

Research Direction 3: Research on the Proper Application of Batch Normalization in Continual Learning Situations (CLVISION workshop)

Among various workshops held at CVPR, workshops on continual learning (CLVISION) have been held continuously over the past few years. A draft of the study was introduced in this year's workshop. The study was motivated by the fact that the majority of continual learning studies in classification aim to propose a new methodological algorithm for continual learning while utilizing the majority of the same elements (e.g., model, optimizer) as in the case of learning a single task, while posing the question, “Are all constant elements suitable for a situation of continual learning?” In the context of exemplar-based continual learning using a CNN-based model with batch normalization (BN), it was shown in this paper that the conventional BN does not play an appropriate role, and a simple solution — task-balanced batch normalization — was proposed to address this issue^[7].

(Figure 2) Task-balanced mean, var calculation

(Figure 3) Less-biased gamma, beta learning

As shown in (Figure 2), in the continual learning scenario using exemplar, each mini-batch is created by sampling data for the current task ($B_c$) and data for the previous task ($B_p$). As a result of the fact that the composition of the data in the mini-batch is heavily biased towards the current task data, the learning process encounters issues such as catastrophic forgetting and biased prediction. Therefore, a variety of solutions have been proposed to overcome such problems. In this paper, we also demonstrated that the mean, variance, gamma, and beta of the BN layer are highly biased towards the current task and proposed a straightforward solution. First, as shown in (Figure 2), task-balanced mean and variance can be calculated when the current task data is reshaped and the previous task data is repeated based on task-adaptively determined r. In addition, as depicted in (Figure 3), when the reshape operation is implemented in the BN process, gamma and beta can be learned with less bias toward the current task. In the case of using each SOTA algorithm for continual learning, when the same continual learning was performed by simply replacing the BN layer in the existing CNN model with the proposed method (TBBN), it has been demonstrated that performance can be improved in nearly all cases compared to when using the existing BN layer.

(Figure 4) Poster Presentation at CVPR 2022

Lastly, at CVPR 2022, not only the continual learning algorithm for the classification task, which has been in the limelight to date but also the continual learning algorithm for more diverse domains such as visual search^[4] and semantic segmentation^[5,6] were presented, meaning that the field of continual learning research is being expanded. However, because there is still a performance gap between the outcomes of multi-task learning and the outcomes of continual learning, we believe that it is still necessary to consider novel approaches that could close this gap.

▶[CVPR 2022] LG AI Research joins CVPR 2022, the Mecca of Vision Research (Link)
▶[CVPR 2022] NeRF - Representing scenes as neural radiance fields for view synthesis (Link)