[ACL 2024] Research Trends in LLM Evaluation Methods for Faithfulness and LLM Efficiency

The Association for Computational Linguistics (ACL) Conference is one of the most prestigious international conferences in the field of natural language processing (NLP) and is an important platform to see the latest trends in NLP research every year. At ACL 2024, held this year in Bangkok, Thailand, one of the biggest topics of discussion was large language models (LLMs).

More than 319 of the 943 main conference papers presented at the conference were related to LLMs, which shows how active LLM research is as a field. The three main keynote sessions also centered on LLMs, addressing the current capabilities, limitations, and reliability of the models.

For example, the session “Can LLMs Reason and Plan?” expressed caution against viewing LLMs as a tool that can solve every problem. The presenter pointed out the limitations of current LLM inference capabilities and suggested utilizing LLM with additional resources or validation. The “Are LLMs Narrowing Our Horizon? Let’s Embrace Variation in NLP!” session discussed faithfulness issues as LLMs develop and the various methodologies proposed to address them. The gap between human and model judgment is interpreted as the domain of faithfulness, and its causes and solutions are explored through the perspective of data and social science.

 

Image 1. ACL 2024 Main Conference Papers Topic
(Based on https://aclanthology.org/events/acl-2024/)

Interest in the capabilities, limitations, reliability, and utilization of LLMs was evident throughout the keynote sessions and workshop topics. Below, I will dive further into two topics centered around LLMs.

1. LLM Evaluation & Faithfulness

2. LLM Efficiency: PEFT

LLM Evaluation & Faithfulness

Research on proposing new benchmarks and evaluation methodologies is currently a hot topic. About 70 benchmark-related papers and 120 evaluation-related papers have been published, and various approaches have been attempted to objectively evaluate the LLM performance. Many benchmarks that were published at ACL go beyond simple tasks to target more complex and challenging tasks or tasks that require higher reasoning ability.

As the capabilities of LLMs have evolved, the need for more detailed and specialized evaluation methodologies has been raised, There has also been recent research into utilizing LLMs as evaluators to solve the problem of expensive human evaluators. LG AI Research is also actively conducting research into LLM evaluation and recently archived papers on PROMETHEUS 2^[1] and The BiGGen Bench^[2]. 

Additionally, a workshop was held at this ACL on data contamination related to the reliability of benchmarks, based on concerns that publicly available data on the web may have been included in the model training data.

Below are ACL papers on the complex task benchmark, LLM faithfulness metric, and data contamination.

1) AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agents^[3]

Summary: This paper goes beyond simple API call tasks in existing benchmarks and proposes a benchmark aimed at advanced/complex tasks that reflect the interactive coding environment to apply LLMs as an agent to the real world.

 

이미지 2. AppWorld benchmark overview^[3]

Existing tool-based LLM evaluation benchmarks have the limitation that they only cover simple API call sequences. To evaluate whether an LLM can serve as an agent in a real-world environment, a test environment that reflects the operation, environment, and user interaction of various programs/apps is required.

The authors propose the AppWorld Engine: a high-quality execution environment consisting of 60,000 lines of code to build a simulated environment in which nine everyday apps can be manipulated through 457 APIs. They also proposed the AppWorld Benchmark, which was created by simulating the lives of 106 virtual users in that environment to collect realistic digital activities. The benchmark is a set of 750 natural, diverse, and challenging autonomous agent tasks comprising 40,000 lines of code, requiring rich and interactive code generation.

It also supports robust program evaluation with state-based unit tests, which allow for multiple ways to complete a task while also checking for unexpected changes (i.e., collateral damage).

Experimental Result & Conclusion

This paper measures the performance of GPT-4o, GPT-4 Turbo, LLaMA3 70B, and Deepseek Coder on the AppWorld Benchmark to achieve SoTA on existing code generation benchmarks, and evaluates the best performance using four different methodologies (ReAct^[4], PlanExec^[5], FullCodeRefl^[6], Iterative Parallel Function Calling).

 

Image 3. Performance graph of different LLMs on AppWorld^[3]

Even GPT-4o can only solve about 49% of the “normal” tasks and about 30% of the “challenge” tasks, with other models performing at least 16% worse than that. The contribution of this paper is that the number of APIs, number of calls, and lines of code required are more challenging than existing code generation benchmarks or tool-based benchmarks, making them suitable as benchmarks for evaluating the potential of interactive coding agents.

2) The Probabilities Also Matter: A More Faithful Metric for Faithfulness of Free-Text Explanations in Large Language Models^[7]

Summary: This paper is an attempt to analyze the relationship between the explanations/rationales explicitly produced by LLMs and their final judgments, and thereby directly or indirectly assess LLM reliability.

LLMs generated by methodologies such as CoT sound plausible for natural language explanations or inference processes, but it is unclear how well the explanations produced by the model match the factors that actually influence the model’s final judgment.This paper proposes a new evaluation metric called Correlational Explanatory Faithfulness (CEF) to evaluate model judgments and explanations in a more precise way.

One prior work is the Counterfactual Test (CT) [8] by Atanasova et al. (2023). CT uses a methodology of inserting some text into the input query (interventional addition (IA)). CT-Unfaithfulness is a metric for model explanation. An explanation is considered to be unfaithful if, when the model prediction changes due to some inserted text, the explanation generated by the model does not mention the added content. In other words, (1) the IA is considered to be valid only if the model’s final prediction changes, and (2) whether the explanation mentions the IA is measured binary (assuming the classification threshold = 0.5). The downside of this is that if the model generates an explanation with the entire context, the explanation will be judged faithful despite its low explanation quality. Additionally, it does not take into account the variability of model predictions.

This paper judges that it is, therefore, necessary to incorporate the correlation of model-generated explanations to actual predictions into CT methodologies and propose an evaluation metric called CEF. Correlational Explanatory Faithfulness (CEF) is a methodology that measures the relevance of intervention* impact** and explanation mentions*** using Pearson correlation.

* Intervention: modifying the input query by inserting a word
** Intervention Impact: Intervention impact: how effective the intervention was on the model’s prediction
*** Explanation Mention: Explanation mention: how important the model treats the intervention in the explanation

Image 4. Example of a CEF metric: Measuring changes in model predictions and explanations before and after intervention^[7]

CEF’s intervention impact is measured by the total variation distance (TVD) of the model’s pre- and post-intervention predictions. The TVD can be used to quantify how much the intervention affected the model predictions(Image 5). Mention importance uses the same binary of presence/absence criteria used in the CT methodology. As a result, CEF can be measured as a point-by-point correlation coefficient.

Image 5. The correlation between TVD values and the impact of inserted text on the model^[7]

Experimental Result & Conclusion

Applying the above CEF metrics and CT methodology, the authors proposed a Correlational Counterfactual Test (CCT) and tested the Llama2 model against three representative classification benchmarks: e-SNLI, ComVE, and ECQA.

Image 6. Comparison of Faithfulness performance measured by traditional CT methodologies and CCT across three benchmarks^[7]

In Image 6, ECQA shows that the model’s explanation tends to mention the intervention, but the intervention has little effect on the model’s predictions. This can be interpreted as the explanation not reflecting what was important to the model at all, and CCT can express this in a numerical way.

3) Quantifying Contamination in Evaluating Code Generation Capabilities of Language Models^[9]

This paper addresses the data leakage potential of representative code generation benchmarks (HumanEval, MBPP). It proposes a methodology to evaluate the similarity between training data and benchmark data at the surface level and semantic level and analyze the data contamination on two large-scale training datasets, The PILE, and The STACK.

1. The similarity measure at the surface level uses the Levenshtein similarity score. The Levenshtein edit distance is an algorithm that computes the morphological similarity between two strings, calculating the minimum number of operations for one string to be equal to the other. The Levenshtein similarity score has been utilized when deduplicating existing datasets on a file-by-file basis, but in this paper, similarity is measured on a substring-by-string basis rather than on a file-by-file basis.

2. The semantic level measures similarity based on abstract syntax trees (ASTs) using the dolos toolkit in terms of what the code actually does. This methodology is often used to detect code plagiarism. Code that is functionally exactly the same but differs only in function names and variable names may have a high semantic level similarity.

Image 7. Comparison of Data Contamination rates in MBPP and HumanEval benchmarks across different training sets,
and performance differences between problem sets with high vs. low contamination likelihood^[9]

By classifying contaminated and non-contaminated evaluation data within the benchmarks and comparing LLM performance between the two subsets, the paper challenged the reliability of LLM’s code generation performance reported to date(Image 7). It can be seen that the accuracy of benchmark problems that are likely to have been leaked into the training set is lower than the accuracy of problem sets in which the model did not see similar content when training. The authors analyzed whether or not the model saw similar code when training is directly related to its accuracy, regardless of the length or difficulty of the code problem.

LLM Efficiency : Parameter-Efficient Fine-Tuning (PEFT)

As an extension of the use of LLMs, there is also a growing body of research on the efficiency and optimization of LLMs due to their very high learning and delivery costs. At this ACL, about 64 papers addressed this topic, notably including many from global tech companies and ACL sponsors. 

Parameter-Efficient Fine-Tuning (PEFT) methods that learn only a small number of parameters have attracted much attention and are being actively researched. Low-Rank Adaptation (LoRA)^[10], which learns only low-rank matrices, is a typical example of this.The following are a few examples of PEFT research presented at ACL 2024.

1) Multimodal Instruction Tuning with Conditional Mixture of LoRA^[11]

This paper proposed that methods that learn a relatively small number of parameters, such as LoRA^[10] are affected by task interference when performing multimodal instruction tuning. It further proposed how to address these issues. Task interference is a phenomenon that occurs when multi-task learning. To be more precise, it refers to the phenomenon that when learning various tasks simultaneously, the gradients between the tasks are directed in different directions, resulting in poor learning.

Using the Task Interference Score^[12], the authors showed that task interference actually occurs when multimodal instruction tuning, which can be seen in the following data.

Image 8. Task Interference Score heatmap of LoRA matrices A and B^[11]

When the LLaVa^[13] model is trained on various tasks with LoRAs, results show that high scores appear regardless of the depth of the model. To address this, the authors propose a Conditional Mixture-of-LoRA, inspired by MoE^[14], which dynamically combines multiple low-rank factors.

Image 9. A schematic diagram of LoRA and the proposed MixLoRA in this paper^[11]

As seen in the figure above, there are a number of a and b Low-Rank Factors on the right, replacing the A and B matrices of the existing LoRA with a sum of Low-Rank Factors that are dynamically selected based on input. The algorithm for selecting Low-Rank Factors consists of Independent Factor Selection (IFS), which selects a and b factors independently, and Conditional Factor Selection (CFS), which helps select b factors based on the selected a factors.

2) MELoRA: Mini-Ensemble Low-Rank Adapters for Parameter-Efficient Fine-Tuning^[15]

This paper shows that LoRA^[10], which learns by approximating parameters at low ranks, may not perform as well as full fine-tuning, which learns all parameters, and considers how to approximate at higher ranks but still learn efficiently. As increasing the rank of LoRA is ultimately equivalent to increasing the number of parameters, how can we approximate High-Rank while learning a small number of parameters?

The authors proposed the idea of maintaining a high rank while keeping a small number of parameters by exploiting the property that when matrices are connected diagonally, the sum of the ranks of each matrix is equal to the rank of the connected matrix.

Imgae 10. nUsing n relatively small partial LoRA matrices to construct the complete sparse LoRA matrices A and B
(with 3 partial matrices shown in the diagram)^[15]

3) PRoLoRA: Partial Rotation Empowers More Parameter-Efficient LoRA^[16]

In situations such as personal customization and multi-tasking, it may be necessary to learn and maintain multiple LoRA^[10]. This paper argues that a more parameter-efficient method is needed and proposes a new methodology called Partially Rotation Enhanced Low-Rank Adaptation (PRoLoRA).

Image 11. LA schematic diagram of LoRA, PRoLoRA, and the intermediate stages leading up to PRoLoRA (CLoRA, RoLoRA).
PRoLoRA consists of four elements^[16]

1. Broadcast Reduction: To reduce the number of parameters, the existing LoRA matrices A and B are cut into smaller chunks of the same size to share parameters via broadcasting. (CLoRA)

2. Rotation Enhancement: Although Broadcast Reduction reduces the number of parameters, the same values are duplicated from chunk to chunk, which is less expressive when learning. To solve this problem, each chunk is rotated according to a fixed stride. (RoLoRA)

3. Partially-Sharing Refinement: Rotation alone is not enough to increase expressiveness, so some ranks should not share parameters when first cut into chunks. (PRoLoRA)

4. Rectified Initialization Strategy: Matrix B is initialized to 0 like an existing LoRA, but in the case of A, a slightly modified Kaiming initialization^[17] is performed to initialize the shared and non-shared parameters.

At ACL 2024, we were able to see various studies related to the capabilities, limitations, and reliability of LLM and have in-depth discussions with many researchers. LG AI Research announced EXAONE 3.0 with enhanced performance and cost-effectiveness in August and released 7.8B instruction tuned model as open source for research purposes. We plan to continue research and development of more reliable and performance-enhanced models based on EXAONE 3.0 and to apply EXAONE 3.0 to actual industrial sites to create usage cases for AI.

ACL Trends Review Series

#1. [ACL 2024] Emerging Trends and Key Insights in LLM Research
#2. [ACL 2024] A New Approach to Chart Understanding and Reasoning
#3. [ACL 2024] Towards efficient Large Language Models
#4. [ACL 2024] AI-generated Text Detection Research Trends
#6. [ACL 2024] Developing a Large Language Model with Cross-Cultural Considerations