[NeurIPS 2023] Introduction to Key Research Topics and Notable Papers

LG joined the world’s largest AI conference, “Neural Information Processing Systems (NeurIPS) 2023,” and gained broad insights into the latest trends in AI research. LG's major affiliates participated together at NeurIPS 2023, operating an integrated booth. Also we could understand the latest trends in AI research and engage with various researchers. I attended the conference as a representative of LG Energy Solution and conducted a demonstration on 'essential vision inspection technology for implementing smart factories.'

A host of innovative ideas and research findings were presented for various fields at NeurIPS 2023. We’ll take a look at some highlights from five topics covered at the conference: Computer Vision (CV), Large Language Models (LLM), Large Multimodal Models (LMM), Diffusion Models, and Reinforcement Learning (RL), along with some notable research papers.

Topic 1. Computer Vision (CV)

Let’s start with computer vision (CV). CV refers to models that handle images as input. Traditionally, CV models have been organized separately for different types of output, such as image classification, object detection, and segmentation. For example, a typical model for object detection is the YOLO^[1] series and a typical model for segmentation is U-Net^[2].

However, in recent years, CV models have been moving toward the creation of so-called foundation models that can handle various output types. Typical models include MAE^[3], DINO^[4], and more. NeurIPS 2023 continued this trend and presented ways to develop a better-performing vision foundation model

1. Towards In-context Scene MUnderstanding^[5]

The first paper we will look at is Hummingbird^[5]입니다. Traditional foundation models handled a wide range of vision task outputs but were limited by poor performance on dense outputs. This is because most foundation models used a method to learn the alignment between image-text.

Accordingly, Hummingbird aims to develop a vision foundation model that performs well on dense output. One of the biggest features of the Hummingbird model is the decoder.

Figure 1. How the Hummingbird Decoder Works

Figure 1 above shows the output of the segmentation map from the Hummingbird decoder. The process of determining the class of a pixel uses the nearest neighbor, which is based on the assumption that the encoded features are distributed close to the same class in the feature space.

To do this, an encoder that takes context into account and encodes similar features to be distributed close together is required.

Figure 2. Hummingbird Architecture

Figure 2 above shows how the Hummingbird encoder works. Notably, it uses data from the memory bank for encoding. It also uses previous batch data as a memory bank during the learning process. It then considers information from this memory bank data together and learns in a self-supervised learning fashion, similar to BYOL^[6].

Figure 3. Comparing the Semantic Segmentation Performance of Hummingbird and Traditional Vision Models

Figure 3 above shows a comparison of the semantic segmentation performance of Hummingbird and traditional vision models trained with this method. This shows the results of pre-training with their respective methods and then finetuning with a small dataset. Results showed that Hummingbird performed better than the traditional methods.

1. Siamese Masked Autoencoders^[7]

SiamMAE^[7] is our next paper to cover. One of the traditional vision foundation model’s typical learning methods is contrastive learning, which has the advantage of being able to learn good features by contrasting large amounts of data. However, it is difficult to use in the video domain, which made it inapplicable to a wide variety of problems that required dealing with time information. Therefore, SiamMAE proposes a way to learn videos using the Siamese Encoder.

 

Figure 4. SiamMAE Architecture

 

Figure 4 above is a representation of SiamMAE architecture. The two input images are adjacent frames extracted from one video. For example, think of Frame 2 as the frame 3 seconds after Frame 1. SiamMAE will only mask the 95% patch on Frame 2, which is the latter frame. The model then learns to restore Frame 2.

Figure 5. Comparing SiamMAE and Traditional Vision Model Performance

 

Figure 5 above compares the performance of SiamMAE trained with traditional models. DAVIS is Video Object Segmentation, JHMDB is Human Pose Propagation, and VIP is Semantic Part Propagation Task. SiamMAE outperforms the traditional models in all tasks.

 

Topic 2. Large Language Model (LLM)

Let’s next take a look at Large Language Models (LLMs). LLMs are one of the two horses of the AI carriage. Whereas CV deals with images as input, LLMs deal with language as input. Since the release of Transformer^[8] in 2017, LLMs have continued to innovate and grow, with a number of competing LLMs including BERT^[30], GPT series^[9,10,11], LLaMA^[12].

However, traditional LLMs also have limitations. For example, they are less able to acquire up-to-date information, tell plausible lies (hallucination), are weak in math calculations, and have poor time-lapse recognition. At NeurIPS 2023, various attempts were made to overcome these limitations of traditional LLMs.

 

1. SToolformer: Language Models Can Teach Themselves to Use Tools^[13]

Our first paper to discuss covers Toolformer^[13]. Toolformer shows an attempt to overcome the limitations of traditional LLMs by using external tool APIs. To achieve this, an external tool is used to configure an answer dataset and fine-tune the language model.

 

Figure 6. How to Create a Toolformer Dataset

 

Figure 6 shows the process of configuring a dataset to answer using these external tools. First, the traditional language model is used to construct a question, and then the API is used to generate the API’s answer to this question. It shows the process of filtering out the relevant data from the questions and API answers, then fine-tuning the model.

 

Figure 7. Comparing Toolformer and Traditional Model Performance

 

The table in Figure 7 compares the performance of this trained Toolformer and the traditional model on math problems. It outperforms traditional LLMs that do not use external tools.

 

1. ToolkenGPT: Augmenting Frozen Language Models with Massive Tools via Tool Embeddings^[14]

Similarly, there is ToolkenGPT^[14], which aims to overcome the limitations of traditional LLMs by using external tools.

 

Figure 8. How ToolkenGPT Works

 

Figure 8 is an illustration of how ToolkenGPT functions. Traditional language models work by predicting the token that corresponds to a word. But what if we could make it predict tool tokens (Toolken) instead of word tokens when we need to use external tools? Once Toolken is predicted, it is possible to use the corresponding external tool to find the correct answer and then return to the traditional language model mode.

 

Figure 9. Comparing ToolkenGPT and Traditional Model Performance

 

The table in Figure 9 compares the performance of this configuration of ToolkenGPT and traditional LLMs. Even on math problem datasets such as GSM and FuncQA, ToolkenGPT outperforms traditional LLMs on all datasets.

 

Topic 3. Large Multimodal Model (LMM)

Recently, active research has looked into Large Multimodal Models (LMMs), which integrate traditional CV and language models. Typical examples include Flamingo^[15], and GPT4^[16]. These models can learn image and text data, allowing them to compute with both image and text as input. Such LMMs have been actively researched in recent years, and a number of them were presented at NeurIPS 2023.

 

1. Visual Instruction Tuning^[17]

Our first paper to discuss covers LLaVA^[17]. While GPT4 is a powerful LMM, it is not a publicly available model. LLaVA proposes a simple way to create an LMM using an LLM that takes only text as input, which requires first creating a dataset consisting of images, instructions, and answers.

 

Figure 10. Creating an LMM Dataset from a Traditional LLM

 

To do this, LLaVA utilizes the image-caption dataset. This method generates questions and answers using a public LLM (GPT4 in the figure) based on the text describing the image.

 

Figure 11. Final Configured LMM Dataset

 

You can then create a dataset for the LMM with paired images, instructions, and answers, as shown in Figure 11 above. With the dataset ready, the next step is to train the LMM. LLaVA suggests a simple method to construct an LMM using an existing pre-trained LLM and a pre-trained vision encoder.

 

Figure 12. LLaVA Architecture

 

Figure 12 shows the LMM Architecture proposed by LLaVA. If you notice, only the projection layer is learned while the weights of all models are fixed.

 

Figure 13. Performance Comparison of LLaVA and Traditional Models

 

This simple configuration of aligning traditional image embedding and text embedding outperforms traditional vision language models as shown in the table above. Although not covered in detail in this post, various LMMs were proposed for NeurIPS 2023, including InstructBLIP^[18], and KOSMOS-1^[19].

 

Topic 4. Diffusion Model

Recently, diffusion models have became famous in the field of image generation. At NeurIPS 2023, various diffusion models were also presented, which can be broadly categorized into three separate research directions.

1. ImageBrush: Learning Visual In-Context Instructions for Exemplar-Based Image Manipulation^[20]

The first direction is to extend the functionality of the traditional diffusion model. For example, rather than just creating an image, you can edit it or give it instructions. A typical model is ImageBrush^[20]. When generating images using traditional diffusion models such as Stable Diffusion^[21], it is difficult to describe exactly what you want the image to look like. ImageBrush proposes a method of giving instructions with example images rather than describing what you want the image to look like in language.

 

Figure 14. Example of How ImageBrush Works

 

As shown in Figure 14, you can provide a before/after image via instruction and then enter a query image to see the image edited to match the instruction.

 

Figure 15. ImageBrush Architecture

 

To do this, ImageBrush uses a method that creates a grid of four images that are fed into the diffusion model at once. This method configures the two instruction images, the input image, and the image that needs to be generated as the output into a single input. To calculate the correlation between the instruction and the query, Transformer is used in tandem. As a result, it learns how to transform the input image by properly configuring self-attention and cross-attention.

 

1. The Surprising Effectiveness of Diffusion Models for Optical Flow and Monocular Depth Estimation^[22]

Extending the diffusion model to the vision model is the second direction. This trend existed even before NeurIPS 2023. A typical example is DiffusionDet^[23], which shows how to use the diffusion model itself as an object detection model.

Continuing with this trend, the DDVM^[22] paper was presented at NeurIPS 2023. DDVM shows how to use a diffusion model as a depth/flow estimation model.

 

Figure 16. DDVM Architecture

 

Figure 16 depicts the DDVM architecture. It receives a noisy map with depth, flow, etc. as input and an RGB image as a condition. The output is a ground truth map. In other words, it is not a diffusion model that generates an image, but rather a diffusion model that uses an image to generate depth/flow.

 

Figure 17. Performance Comparison of DDVM and Traditional Models

 

The trained DDVM shows performance comparable to traditional depth/flow estimation models. Figure 17 above compares depth estimation results.

 

1. StableRep: Synthetic Images from Text-to-Image Models Make Strong Visual Representation Learners^[24]

The third direction is to utilize diffusion models for existing tasks. One example is to generate images with diffusion models and use them to train traditional vision models; a paper proposing such a method is StableRep^[24].

 

Figure 18. Comparing Model Learning with Real and Synthetic Images

 

As shown in Figure 18, it proposes synthesizing images using a stable diffusion model and utilizing them as learning data.

 

Figure 19. StableRep Learning Method

 

It also proposes a self-supervised learning method to efficiently learn these synthesized images. Unlike CLIP^[25] which used captions to learn images and alignments, captions are fed into stable diffusion to synthesize images. The synthesized images are then learned with a contrastive learning method similar to SimCLR^[26].

 

Topic 5. Reinforcement Learning (RL)

Reinforcement Learning (RL) is our final topic for today. RL is an area that is rapidly evolving as more and more research is being conducted. Research is being actively carried out ever since Reinforcement Learning by Human Feedback (RLHF), which uses RL to learn human preferences, was introduced by InstructGPT^[27].

 

1. Direct Preference Optimization : Your Language Model is Secretly a Reward Model^[28]

One paper depicting this trend is DPO^[28]. InstructGPT proposed a way to use RL to learn human preferences, but it was achieved through a complicated process.

 

Figure 20. Comparing RLHF and DPO

 

Figure 20 shows a comparison between the traditional RLHF and DPO processes. RLHF first constructs a human preference dataset, then constructs a reward model learned from it, then finally uses the reward model to learn an LM policy that reflects human preference. Basically, it requires a complex two-step learning process. On the other hand, the DPO on the right fine-tunes the LM directly using the human preference dataset, streamlining RLHF’s two-step learning process into a single step.

 

1. DPOK : Reinforcement Learning for Fine-tuning Text-to-Image Diffusion Models^[29]

Utilization of RL has been proposed not only for LLM but also for diffusion models. DPOK^[29] proposes to use RL for finetuning a text-to-image diffusion model.

 

Figure 21. Comparing Supervised Finetuning and RL Finetuning

 

In traditional supervised fine-tuning methods, a pre-trained diffusion model is first used to generate images of different shapes. It then configures a reward function that reflects human preferences for these images. This method uses the reward function to ultimately fine-tune the diffusion model.

However, this method is limited in that it only learns preferences for datasets that are already configured. In response, DPOK proposes an iterative approach to organizing image generation, evaluation, and feedback. This allows the model to continuously learn from the preferences and perform better.

This post covered five topics and related research papers presented at NeurIPS 2023. Each of these topics is advancing the field of AI in its own unique way. The competition between vision foundation models and language models is making it possible to develop more efficient and powerful AI models. Furthermore, developments in multimodal models allow for the integration of image and text data, making AI applicable to a wider range of applications. Diffusion models and reinforcement learning are providing innovative ideas for visual work and learning methods, and this research is expected to be applied to a wide range of fields in the future.

Through analyzing the current trends in research, it can be observed that AI research is progressing towards developing a Multi-Modality Model capable of handling all modalities. I plan to apply the insights gained from this to my own research. Currently, at LG Energy Solution, I am involved in the development of algorithms for image defect detection, recommending optimal battery designs, and predicting demand for electric vehicles. Up until now, I have been developing algorithms using only single-modality data. However, I plan to progress my research by inputting domain knowledge in text format to the model and developing algorithms in a multimodal approach. We can expect AI research to continue to advance and make a difference in the world.

 

▶ Want to learn more about the NeurIPS 2023 Research Blog series? (Link)