[ACL 2022] A Search-Based Knowledge Augmentation for Natural Language Understanding

The Annual Meeting of the ACL (Association for Computational Linguistics), which celebrated its 60th anniversary this year, is considered one of the best international conferences on natural language processing. In this blog, I will review the research trend of the “language model that combines the knowledge information of natural language processing based on knowledge NLP tutorial,” which was introduced at ACL 2022 held in May 2022.

In general, language models are trained using basic corpora. However, what if knowledge information such as the relationship between specific words and general common sense were embedded into the language model? We have collected search-based studies among those based on this question.

In this blog, we will introduce six studies, including REALM and DPR, which are researches related to vector retrieval, RETRO, and WebGPT, as well as REINA and Kformer.

REALM: Retrieval-Augmented Language Model Pre-Training

The study on REALM introduced a knowledge retriever in the process of training a language model to combine external information with the language model. Unlike previous studies, it trained the knowledge retriever in unsupervised learning mode.

Figure 1. Overall schematic for training the REALM model. The model trains the retriever and encoder simultaneously by combining the retrieved documents with the language model learning.

It configured a new input value by finding the relevant information and suffixing it to the input value to include knowledge information when training a language model. In this process, it combined a model for finding the relevant information (Neural Knowledge Retriever) and a language model (Knowledge-Augmented Encoder) into the structure shown in Figure 1 and trained them simultaneously. The fact that the retrieval model was included in the language model training enabled the retrieval model to be trained in unsupervised learning.

REALM was the highest performing model at the time of its introduction. It showed a higher performance difference of about 5.9% to 7.1% compared to ORQA, an existing open domain question answering model. It also drew attention by showing 5.9% performance gap with the T5 (11b) model, the largest language model available at the time.

Dense Passage Retrieval for Open-Domain Question Answering

DPR study is still widely mentioned in studies on vector retrieval. The study used two encoder models to solve the task of finding the most relevant paragraph and the correct answer when a question is given. It also adopted a method of efficiently utilizing data within a batch.

It utilized two encoders for the first time in the question-answering task. In this case, the encoders learned through initialization with the transformer-based encoder model BERT, which is well-known in natural language processing. Each encoder was dedicated to questions and paragraphs, respectively, to learn and find the relevant paragraphs during inference.

An important feature of this study is the use of in-batch negatives to train the presented dense passage retrieval (DPR) model. Using Equation 1 below as a loss term, it learned to ensure that the passage most relevant to the sentence given as data of all passages in the batch has a more similar vector. Its purpose is to train two encoders more efficiently by utilizing a passage that can find the answer to the question, a passage that does not have an answer but is highly relevant (hard negative passage), and passages to other questions.

Equation 1. Negative log-likelihood loss term using the similarity between positive passage and n negative passages for the i^th question

This study is significant since it recorded the highest performance in open-domain question answering at the time and showed a deep learning search performance that surpassed TF-IDF/BM25 for open-domain QA.

Improving Language Models by Retrieving from Trillions of Tokens

This study started with ideas from large language models created by increasing parameters. Instead of increasing parameters for training, however, it created a Retrieval Enhanced TRansfOrmers (RETRO) model that used data containing trillions of words.

Similar to the REALM introduced earlier, the RETRO model uses the idea of adding the relevant knowledge information to the input but handles longer document chunks. Unlike the previous two encoder-based models, the RETRO model uses encoder-decoder-based models (auto-regressive language models).

Figure 2. (Left) RETRO architecture, (Right) Structure of cross-attention

As shown in Figure 2, in the RETRO model, a chunked cross-attention (CCA) structure is added between the self-attention (ATTN) and the feed-forward network (FFW). This cross-attention adds knowledge information through the structure. The relevant data are vectorized through an already trained language model, and input values for each document are sequentially combined with the vectorized information.

This model has the advantage of continuously receiving data required for training. This study showed that the performance of the transformer model could be significantly improved by varying data application method even with the same parameters.

WebGPT: Browser-assisted question-answering with human feedback

WebGPT is a study on a fine-tuned GPT-3 by getting hints from people finding information by asking questions on the Internet, accessing related pages, and scrolling the page.

In this study, the model’s input is created by adding descriptions of specific contents such as “Search: ...”, “Find in Page: ...”, and “Cited: ...” to the contents found using search. In some cases, supervised learning is performed with man-made correct answers (behavior cloning) or by applying a reward in reinforcement learning (rejection sampling) after generating several correct answers.

While the RETRO model searches for information in its own database, WebGPT searches through Microsoft's search engine — Bing — to secure information. It is a meaningful study as most retrieved documents are up-to-date and it focuses only on finding the right answer as the role of the existing search model is left to the web browser.

Training Data is More Valuable than You Think: A Simple and Effective Method by Retrieving from Training Data

This study was conducted with ideas from prompt learning, one of the methods of tuning large models. Prompt learning places data on the problem to be solved in the inference process. The study devised a method of adding more data related to supervised learning and tested it with 11 types of data.

The study found that it can improve performance by retrieving the relevant data from the training data and inputting it into the model. It introduced such method under the name of REINA (REtrieving from the traINing datA).

Figure 3. Indexing structure of training data with REINA

Figure 4. Schematic diagram showing the input value changed by REINA

As shown in Figure 3, for summarization REINA reconstructs the training data by extracting similar documents' summary statements. The method inserts summaries of similar documents into the input value to the encoder during model training, as shown in Figure 4.

Unlike other studies that focus on one type of problem among NLU (natural-language understanding) and NLG (natural language generation), REINA showed high performance for both types of problems and recorded the highest performance for three data types — XSum, Big-Patent, and CommonsenseQA. By comparing it with the BART model for the summarization, BART-base showed comparable performance to BART-large when using the REINA model.

The study confirmed with several data that are simply inputting similar data from the training data could bring about the effect of adding more required information in the model.

Kformer: Knowledge Injection in Transformer Feed-Forward Layers

The last model to be introduced is the most recently published study “Kformer”, which is a model injecting knowledge information at the feed-forward layer. The study introduced a method of more effectively combining expertise in a specific field with a pre-trained language model through Kformer.

Since language models based on transformer encoders that have been published so far are designed to understand general languages, they do not include expertise in a specific field. Therefore, when it is necessary to add additional knowledge to the language model, most of the existing studies have tried to concatenate knowledge to the input value or insert information into the model’s attention structure.

Figure 5. Structure diagram of Kformer, a model that combines knowledge through a feed-forward network layer

Based on the study wherein the feed-forward network emulates neural memory, it created Kformer by changing the model structure to inject knowledge into the feed-forward structure as shown in Figure 5. By using Kformer, it was confirmed to have shown better performance in SocialIQA dataset, a question-answering-based social commonsense reasoning benchmark, and MedQA-USMLE, a medical question-answering data.

The advantage of Kformer as presented in this study is that it can be applied to various pre-trained language models and inject the necessary external knowledge as needed.

This blog reviewed studies that tried to inject external knowledge by introducing retrieval to the language model. In the course of the study, we have shown that the model structure was changed to be more effective by adding the retrieved information. Although retrieval has been included in deep learning and trained end-to-end in just a few years, studies on retrieval and deep learning, or combining the two, have been ongoing. Synergy is expected to be revealed in more diverse tasks through the interaction of the two in the future.