[ICLR 2024]The Latest Research Trends of Foundation Model

The International Conference on Learning Representations (ICLR) is an international AI conference that has attracted the attention of many AI researchers since it began in 2013. As a testament to this interest, a total of 7,404 papers were submitted to ICLR 2024, which was held in Vienna, Austria, in early May, and 2,260 papers were ultimately presented. This post will introduce the research papers presented by LG AI Research at ICLR 2024 and the latest research trends on large-scale AI models, which have recently attracted great attention, from three perspectives: (1)Understandable AI, (2)Sustainable AI, and (3)Safe AI.

As AI technology advances, the use of AI models to address real-world issues is becoming more common. Since AI models frequently encounter unseen data during the learning process, it is critical for them to have the ability to make reasonable inferences about unfamiliar data. In fact, a model’s effectiveness in practical applications hinges on its ability to generalize beyond the data it was trained on.

To enhance a model’s generalization performance, a straightforward method is to train it on as much diverse data as possible. Recently, a prevalent approach involves a two-step approach: initially training a model on a vast amount of data, followed by finetuning it on  the model on the problem data to be solved. These pre-trained models, often referred to as foundation models (FMs) due to their wide-ranging usefulness, have been shown in studies to tackle new problems with minimal examples, with no additional training. Thanks to this impressive ability to generalize, foundation models are quickly integrating into our daily lives. Examples like EXAONE from LG AI Research and OpenAI's GPT demonstrate how foundation models can greatly enhance modern life.

As foundation models become more prevalent in daily tasks, various research efforts are underway to leverage the rich knowledge that learned by foundation models. These studies seek to answer three questions: (1)Why foundation models have high generalization performance, (2)How to make foundation models cost-effective, and (3)How to ensure the safety of large-scale models. Research organizations around the world are conducting research on this in various fields. At ICLR 2024, there were a number of studies and workshops related to the foundation model. This post will summarize the latest trends in foundation model research under three main topics.

1. Interpretability: Understanding the generalization ability of the foundation model

The studies that attracted the most attention at ICLR 2024 were papers that called for a theoretical and experimental understanding of the foundation model. Specifically, researchers sought to explain why foundation models have the ability to generalize when solving problems that they have not learned.

The first paper in this post is “Robust Agents Learn Causal World Models^[1]”, which interprets the generalization ability of foundation models from the perspective of the causal inference.

Image 1. Three levels of Pearl Causal Hierarchy (PCH)^[4]

Intelligent entities understand the causal relationships in the world in different ways. From this perspective of causal inference, statistical AI models are positioned at the first level among the tree levels1 of the 3 levels depicted in the above Image 1. Causal inference researchers aim to teach these AI models how to think like humans.

Causal inference is a field of research that aims to theoretically understand the causal relationships between causes and effects^[3]. Causal inference is widely used in disciplines such as medicine, policy-making, and economics, where quantifying of causes and effects is essential. Recently, many AI researchers have been exploring new avenues for AI using concepts from causal inference^[2]. LG AI Research has also joined this trend and research AI models based on causal inference.

Image 2. Causal Models and Statistical Models^[2]

Statistical models assume that the distribution of data is fixed. While this is very simple and efficient, it makes them vulnerable to changes in the distribution. In contrast, causal models assume that the distribution of data is transformed under a causal system. From this perspective, causal models try to become robust to distributional changes by learning the causal system behind the data.

Causal models differ significantly from existing statistical AI models. Traditional statistical models assume that the distribution of the training data is fixed, and aim to learn the distribution that best fits the distribution of the training data (Image 2, left). The problem with these models is that while they work well for data with similar distributions, they struggle when the distribution changes even slightly.

In contrast, causal models do not believe that distributions are fixed. They assume that there is an underlying causal system (the data generating process) that generated the data, and they interpret the distribution of the training data as the result of this causal structure under certain controlled circumstances (Image 2, right). From this perspective, causal models become robust to changes in distribution.

However, due to this difference, it has been difficult to interpret current AI models, which are built from a statistical perspective, from a causal perspective. While many causal inference researchers have hypothesized that statistical-based AI models have inherent causal thinking, but there has long been insufficient evidence to support this hypothesis.

The paper “Robust Agents Learn Causal World Models^[1]”, presented at ICLR 2024, reveals that AI models understand the world causally. The authors argue that AI models learn the causal systems underlying data generation during their reasoning processes. Specifically, the authors confirm that AI models with high generalization performance (e.g., foundation models) are robust to various controlled conditions of causal systems. They demonstrate that, based on these models' reasoning processes, it is possible to reconstruct the causal systems that generate the data. This paper is significant in that it breaks down the high barrier between causal inference and AI, and in recognition of this achievement, this paper was selected as the oral presentation paper with the highest score among ICLR submissions.

While the previous paper sought to understand AI models from a theoretical perspective, the next paper, “Understanding In-Context Learning in Transformers and LLMs by learning to learn discrete functions^[5]”, takes an experimental approach to exploring the generalization ability of AI models. Specifically, the paper explores why foundation models can infer solutions to problems they have never seen before.

Image 3. Prompt Engineering^[6].

By simply showing AI models how to approach a problem or providing examples of answers, they can perform faster and more accurate inferences. Since this ability was discovered, researchers have been working to develop various techniques to extract as much knowledge as possible from AI models.

When using large language models (LLMs) such as LG's EXAONE, there is a technique known as prompt engineering that can dramatically enhance performance. Prompt engineering is a technique that provides AI models with additional information such as problem-solving methods, examples, and context, enabling the models to deliver solutions tailored to the user’s needs. In the example in Image 3, the user guides the AI model by directly presenting the method to solve the problem, leading the AI model (assistant) to solve problems it has never seen before. This allows the model to solve problems with just a few lines of instructions that it wouldn't have been able to solve without context. Since this technique involves providing AI models with the context of the problem, it is often referred to as In-Context Learning (ICL). ICL is significant because it draws out the knowledge of foundation models without additional training. However, much remains unknown about why language models possess this ability and the processes through which they perform such tasks.

The authors of this paper study how LLMs operates by experimenting with their ability to solve discrete functions in-context. Through extensive experiments, the authors demonstrate that LLMs learn general patterns from language data and further reveal that LLMs implement some kind of learning algorithm inside the model during inference. While previous researches on in-context learning has focused on continuous functions or language tasks, this paper brings a new perspective by addressing discrete Boolean functions.

2. Sustainability: Compressing and Accelerating foundation models

As foundation models are frequently used to solve real-world problems, the industry is becoming more concerned about the cost of running large models. Reflecting this trend, a variety of researches aimed at reducing the cost of using models were presented at this year’s ICLR. In this chapter, we will explore two papers on lightweighting and acceleration foundation models.

“Batched Low-Rank Adaptation of Foundation Models^[7]”, selected as an oral presentation paper at ICLR 2024, proposes a lightweight approach to reduce the training cost of large models. Recently, to minimize the cost of retraining large language models, techniques such as Parameter-eEfficient Fine-Tuning (PEFT) have emerged, which involve training only subset of layers in large language models. Among these techniques, Low-Rank Adaptation (LoRA) is prominent, where most parameters of the large model are frozen, and only a small layer approximating the model is trained. The authors of “Batched Low-Rank Adaptation of Foundation Models” target LoRA, arguing that existing LoRA is not optimized for batch-based computation. This means that simply using LoRA is performance penalized when different tasks are interleaved within a batch. To address this issue, the paper proposes Fast LoRA (FLoRA), a technique optimized for batch-based computation.

Image 4. FLoRA^[7] assigns a unique Low Rank Adapter to each sample within a batch, making batch-level training more efficient.

FLoRA enables fast processing at a batch level by assigning a unique low-rank approximation weights to each input example in the batch. The authors experimentally demonstrate that FLoRA maintains the performance of LoRA and achieves competitive results in code generation for 8 languages using MultiPL-E and multilingual speech recognition tasks spanning 6 languages. This paper is significant as it addresses a number of real-world challenges faced by large language models.

From a different perspective, we also saw acceleration research aimed at making models run faster. “ReLU Strikes Back: Exploiting Activation Sparsity in Large Language Models^[8]”, which was presented orally this year, is a paper on a very important discovery about large language models.

In general, large language models (LLMs) do not use ReLU activation function because there has been a belief that ReLU may cause performance degradation. Therefore, existing language models have used improved versions of ReLU, such as GELU or SiLU. However, the authors of this paper experimentally prove that this belief mistaken and show that ReLU activation function do not significantly affect the performance of large language models. Moreover, the authors demonstrate that the sparse nature of ReLU function is highly advantageous in terms of both model acceleration and being lightweight.

Image 5. Architectural surgeries for ReLUfication^[8]

Furthermore, the authors propose a technique called ReLUfication, which is a simple method to dramatically lightweight pre-trained language models. ReLUfication involves replacing the activation function of a model trained with a non-ReLU activation function with ReLU, followed by retraining. The authors announce that models adjusted with ReLUfication not only quickly regain their original performance, but also show up to three times cost savings in the inference process. This paper is impactful as it provides ample experimental results on large language models and proposes a very simple method to accelerate and lightweight them.

"ShERPA: Leveraging Neuron Alignment for Knowledge-preserving Fine-tuning^[9]", presented at  ICLR 2024 Workshop on Mathematical and Empirical Understanding of Foundation Models (ME-FoMo), is another paper that aims to accelerate the training and generalization of foundation models.

Image 6. ShERPA^[9]

ShERPA aligns untrained models into the loss basin of trained models, thereby preventing information loss during model training. This simple technique reduces information loss during the learning process, and contributes to the model’s higher generalization performance.

Recently, published studies have shown that the generalization performance of a trained model depends on the structure of its loss landscape. From this perspective, techniques such as neuron alignment, which aligns different models in the loss landscape, have been proposed. Our paper demonstrates that neuron alignment techniques aligning two models in the loss landscape yield effects similar to transfer learning. Specifically, models aligned by neuron alignment before learning have higher generalization performance, leading us to propose a robust fine-tuning technique that preserves existing information more effectively during fine-tuning.

3. Safety: Ensuring the safety of foundation models

Foundation models are increasingly being used widely among the general tasks, not just by experts, and efforts to ensure their safety continue. In fact, such research is crucial to ensure that AI models are not misused as dangerous tools for humanity. At ICLR 2024, several papers were presented focusing on the safety of foundation models.

The topic of model safety is a complex one that combines many different issues. One of the most relevant to real life is ensuring that large language models do not provide harmful information. Many researchers are working on model alignment techniques to ensure that language models do not provide information that is harmful to children, or information that is overly violent or can be used for criminal purposes.

Image 7. Self Alignment with Instruction Backtranslation^[10].
To ensure model safety, the authors propose a technique to allow a model to generate its own data and verify its safety.

The first paper, “Self-alignment with Instruction Backtranslation^[10]”, explores research on model alignment techniques. The paper proposes a methodology that utilizes the model’s own knowledge to ensure the safety of large language models. Specifically, the authors propose a backtranslation technique that allows them to generate instructions backward from the resulting text. Importantly, the quality of the data can be increased by having the model once again censor the data it generates. With these techniques, the authors show that it is possible to minimize human intervention and increase the safety of the model.

However, there are clear limitations to these model alignment techniques. The last paper discussed in this post, “Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!^[11]” shows that such model alignment can be invalidated very easily.

The authors of this paper argue that even if a model is secure with a high-performance model alignment technique, its safety can be removed when approximately 10 new pieces of harmful information are introduced as additional training data. Specifically, the authors feed 10 harmful samples into GPT and then attempt to fine-tune it using this data. They experimentally demonstrate that the GPT trained this manner produces harmful outputs, unlike its behavior before the fine-tuning. More importantly, this task cost only $0.2 (about KRW 275). Considering that many AI companies invest significantly more in model alignment, this finding carries important implications: existing AI alignment techniques can be easily neutralized.

Image 8. “Jailbreaking GPT with just $0.2!^[11]”.
The authors show that existing expensive model alignment techniques can be neutralized with very small cost.

Based on these experimental results, the authors of the paper report the vulnerability of existing model alignment techniques. They also express concerns about fine-tuning powerful models. Depending on how the findings of this paper are interpreted, it is believed that there may be a division of opinion about the steps that should be taken to release powerful AI models. LG AI Research also strives to develop safe AI models, taking cues from such research efforts.

The Future Research of LG AI Research

Public interest in AI has never been higher, especially as large-scale AI models, such as conversational AI models, are becoming more accessible to the general public. This is creating new problems as well. Reflecting this trend, this year’s ICLR 2024 conference featured a number of studies that addressed these concerns. As a leading company in AI model development in South Korea, LG AI Research is striving to create greater synergy between AI technology and people. We appreciate your continued interest in the secure AI ecosystem that LG AI Research is building.

 

▶[ICLR 2024] Compression of Neural Networks Research Trends