[ECCV 2022] Review of ECCV 2022 and Computer Vision Research Trends

European Conference on Computer Vision (ECCV) 2022 was held in Tel Aviv, Israel from October 23 to 27, 2022. ECCV, which is held for the 17th time this year, is organized on even years. It is known as one of the most influential AI image recognition conferences with Computer Vision and Pattern Recognition Conference (CVPR) and International Conference on Computer Vision (ICCV).

The main conference was held for three days from October 25 in Expo Tel Aviv. Numerous researchers made presentations on their research and interacted with one another.

Image 1. ECCV 2022 Hosting Place (Expo Tel Aviv, Israel)

Image 2. Main Conference (Oral Session)

Research Trends

Researchers made presentations on a variety of topics related to computer vision such as 2D/3D object detection, image synthesis and generation, and low level vision at ECCV 2022. Recently, large-scale vision models for text-to-image such as LG AI Research EXAONE Vision, Google Imagen, OpenAI DALL-E 2, and Stability Stable Diffusion AI are being launched. Accordingly, a lot of multimodal AI-associated researches were presented in this year's ECCV. It shows that multimodal AI researches are not limited to text and image, but are expanded to audio and video.

In particular, video-associated studies were noteworthy. It was found that participants had a lot of interest in 3D image generation with NeRF and self-supervised learning. Through this blog post, we will introduce two papers published by LG AI Research Vision Lab presented at ECCV 2022 and another two recently published papers about self-supervised representation learning.

LG AI Research in ECCV 2022

Research paper conducted by LG AI Research interns Hyounguk Shon and Juseung Yun and their mentor Janghyeon Lee from Vision Lab were presented during the Poster Session.

[ECCV 2022] DLCFT: Deep Linear Continual Fine-Tuning for General Incremental Learning (Link)

The paper written by Hyounguk Shon suggests continual fine-tuning that conducts continual learning from a pre-trained model, unlike existing continual learning method that starts from scratch. Shon verified continual fine-tuning under a variety of conditions such as data incremental, task incremental, and class incremental learning scenarios by utilizing the linearization of a pre-trained model.

[ECCV 2022] On the Angular Update and Hyperparameter Tuning of a Scale-Invariant Network (Link)

The paper written by Juseung Yun is about the relation between the hyperparameters and angular update of a scale-invariant network that contains normalization layers. It suggests an effective hyperparameter tuning method which can find quality hyperparameters only with a small number of learning data.

Many researchers paid attention to Shon and Yun’s presentations and shared their opinion through Q&A during Poster Session.

Image 3. Presentations during Poster Session, Hyounguk Shon (left) and Juseung Yun (right)

Self-supervised Classification Network

Recently, self-supervised contrastive learning is widely suggested. In previous related works, two views are generated through two different augmentations in an instance and then it conducts learning so that embedding vectors passing the backbone network become similar to each other^[4]. In addition, its learning approach aims to lower the similarity of embedding vectors with other instances in the same batch. As this self-supervised representation learning is for the backbone network, it should separately train the classifier network in order to learn models that conduct tasks such as classification. Previous self-supervised representation learning approaches should be conducted in 2-stage manner. In addition, it requires labeled data for the classifier network. The paper titled Self-Supervised Classification Network^[1] presented at ECCV trained the backbone and classifier networks at the same time as shown in Figure 1.

Figure 1. Self-Classifier architecture^[1]

 

In this paper, performance was yielded by clustering evaluation index on the ImageNet dataset, and the proposed approach is compared with other representation learning methods, and clustering algorithms. It was announced that the highest level of clustering accuracy could be obtained as shown in Table 1. 

 

SPot-the-Difference

As a researcher studying anomaly detection models through the vision inspection project(Link), SPot-the-Difference^[2] had an interesting topic.

In the study on anomaly detection^[3] based on self-supervised contrastive learning, transforms maintaining the data distribution and transforms making it deviated from the data distribution are separated when contrastive learning is conducted in the process of augmenting normal data. In other words, learning is conducted so that the embedding vectors are attracted in case of instances applied with the weak augmentation maintaining the data distribution. On the other hand, learning is conducted so that the embedding vectors are repelled in case of instances applied with the strong augmentation making it deviated from the data distribution. In addition, the anomaly detection task is conducted through learning that categorizes normal data and type of strong augmentation. 

The paper^[2] follows the approach of anomaly detection based on existing self-supervised contrastive learning. The key idea is that learning is conducted so that it responds insensitively to global changes and sensitively to local changes as shown in Figure 2.

Figure 2. The contrastive spot-the-difference learning^[2]

In manufacturing vision inspection, defects are occurred locally in the product image. Thus, it is more important to detect local changes than global changes. In this paper, local perturbations were created through SmoothBlend similar to CutPaste and learning was conducted so that it repels embedding vectors between transformed images. It induces to reduce the difference caused by global augmentation through the regularization LSPD. Leaning is conducted so that representations different from normal images are extracted for images that contain local perturbations by making the model concentrate on the local perturbations. Lastly, the classifier network is attached to the trained backbone network (f) and the anomaly detection task is performed to categorize normal images and local augmentation-applied images.

The method suggested in this paper is not outstanding in terms of performance when evaluated with MVTec dataset compared to other latest anomaly detection models. However, the paper is meaningful in that it suggests a regularization method that is able to control augmentation in self-supervised learning-based anomaly detection. 

Industrial Trends

We could take a look at the latest trends through the industrial exhibition held in the same hall where Poster Session was held. It was noticeable that human action recognition was not just applied to abnormal action detection in CCTVs but also expanded to other areas such as autonomous drivind and sports analysis. There was still a great interest in applications in which more information is provided by adding virtual reality to the real world through smart glasses.  

Mobileye exhibited self-driving cars and provided an outdoor autonomous driving experience through reservations. GM presented video demos of their technology controlling self-driving cars by recognizing human hand signals as well as videos of their system that recognizes objects affecting driving such as lanes, traffic lights, and traffic signs. Looking around AI applied in different industrial fields, we realized that AI should be able to communicate with humans and understand the intentions of human actions as well as our languages in order for vehicles and robots equipped with AI to coexist with humans in the future. 

Image 4. GM Exhibition Booth

Meta presented their ARIA Project(Link) and exhibited Augmented Reality (AR) glasses. Google Research demonstrated sports content creation with its system that displays analysis results in a video by recognizing and analyzing players’ actions and the traces of the ball. The exhibition was filled with a wide range of technologies such as 4D scanning for 3D data creation and a development platform for an efficient application of image recognition to embedded hardware. 

Image 5. Industrial Exhibition: 4D Scanning (left), Embedded Development Platform (right)

Conclusion

We could gain a lot of information, listening to computer vision presentations made by researchers of different countries and asking them questions at ECCV 2022. Multimodal AI represented by text-to-image is expanding to videos. More and more researchers tend to approach a variety of computer vision tasks with self-supervised learning. We need to pay continuous attention to these topics in line with the recent research trend.  

LG AI Research has been continuously publishing its research outcome, including three academic papers selected at ECCV 2022. We will actively communicate with the academic circles and put forth continuous efforts to become a global leading AI research institute.