KIISE Transactions on Computing Practices(KCTP) (2021)
Janghoon Han, Youngjoong Ko (Sungkyunkwan University), Jungyun Seo (Sogang University)
Abstract
Developing dialogue systems is a crucial aspect of artificial intelligence. Various studies have been advanced to improve the performance of the dialogue systems using external knowledge. However, well-structured external knowledge is required in order to be applied to the dialogue systems, and it needs a considerable amount of time and resources to be generated. From this point of view, this study proposes a model that uses unstructured texts as external knowledge for retrieval-based dialogue systems. We use a pre-trained language model, BERT as a base model for the response selection task, and then we train BERT with the external knowledge through post-training. Eventually, the post-trained model is fine-tuned for the dialogue response selection task. Compared to existing BERT models, the performance of the post-trained model with external knowledge is improved by 1.3% in on ubuntu corpus.