Paper Accepted at ACL, a Top-Tier AI Conference in Natural Language Processing

Research by Bang Na-mo and Lee Ji-hyun, master's students in the first cohort of the graduate Department of Artificial Intelligence, who majored in journalism and broadcasting and in linguistics respectively as undergraduates

Corresponding author Professor Koo Myoung-wan of the Department of Computer Science & Engineering: "A case of turning the strengths of humanities graduates into convergence research"

A dialogue systems paper by the research team of the Intelligent Spoken Dialogue Systems Laboratory (ISDS) in the Department of Computer Science & Engineering has been accepted as an 'ACL Findings Paper' at The 61st Annual Meeting of the Association for Computational Linguistics (ACL 2023). ACL is one of the world's three leading natural language processing conferences and is listed at IF 4, the highest impact factor grade, among the outstanding international conferences in computer science under the BK21 Plus programme.

▲ (From left) Koo Myoung-wan (Professor, Department of Computer Science & Engineering), Bang Na-mo (master's student, Department of Artificial Intelligence), Lee Ji-hyun (master's student, Department of Artificial Intelligence)

The paper by master's students Bang Na-mo and Lee Ji-hyun of the graduate Department of Artificial Intelligence (3rd semester, joint first authors) and Professor Koo Myoung-wan of the Department of Computer Science & Engineering (corresponding author), titled 'Task-Optimized Adapters for an End-to-End Task-Oriented Dialogue System', will be published in the proceedings of ACL 2023, held from 10 to 12 July.

The team proposed a task-oriented dialogue system that adds adapter modules to a large pre-trained language model (LLM) with frozen parameters and improves performance through reinforcement learning. The adapters optimize the dialogue system according to the purpose of each speech act — understanding user utterances, dialogue state tracking, generating system responses — thereby improving the performance of each function. In this study too, they were applied to raise the performance of the dialogue system.

▲ The task-oriented dialogue system architecture proposed in 'Task-Optimized Adapter for an end-to-end Task-Oriented Dialogue System'

The team showed that the idea of adding adapters to an existing LLM can also minimize the computational cost of improving dialogue system performance. Building a dialogue system with an LLM so that it performs a variety of speech-act tasks requires fine-tuning the parameters for each task. Until now, fine-tuning an LLM has required sharing all the parameters used in pre-training, which is computationally costly, and catastrophic forgetting — losing what was learned in pre-training — has been raised as a problem. By training only adapters made up of a small number of parameters instead of the LLM's full parameter set during fine-tuning, the team resolved both the training cost problem and the forgetting problem. In terms of the training method, they also demonstrated that reinforcement learning can optimize the system for each speech-act task.

▲ The detailed adapter structure proposed in 'Task-Optimized Adapter for an end-to-end Task-Oriented Dialogue System'

The research is an achievement by students who majored in the humanities and social sciences as undergraduates and went on to the first cohort of the University's interdisciplinary programme in Artificial Intelligence. Bang Na-mo comes from the University's School of Communication, where she majored in journalism and broadcasting with a convergence major in Big Data Science. Lee Ji-hyun graduated in linguistics from Korea University before entering the University's master's programme. Both are DHE (Digital Human Entertainment) scholarship holders sponsored by Smilegate.

Professor Koo said: "This is a case where the insight of humanities graduates has been turned into a strength in artificial intelligence, a convergent discipline," adding, "I will continue to guide students so that they can create synergy by combining engineering with domain knowledge."