The research team of the Intelligent Spoken Dialogue Systems Laboratory (ISDS),

Paper Accepted at ACL, a Top-Tier AI Conference in Natural Language Processing

- A research result by Bang Na-mo and Lee Ji-hyun, master's students in the first cohort of the Graduate Department of Artificial Intelligence, who majored in journalism and broadcasting and in linguistics respectively as undergraduates -

- Corresponding author Professor Koo Myoung-wan of the Department of Computer Science & Engineering: "A case of drawing on the strengths of convergence research by students from humanities backgrounds" -

▲ (From left) Professor Koo Myoung-wan, Department of Computer Science & Engineering; Bang Na-mo and Lee Ji-hyun, master's students in the Department of Artificial Intelligence

A paper on dialogue systems by the research team of the Intelligent Spoken Dialogue Systems Laboratory (ISDS) in the Department of Computer Science & Engineering has been accepted as an ACL Findings Paper at The 61st Annual Meeting of the Association for Computational Linguistics (ACL 2023). ACL is one of the world's three leading natural language processing conferences and is listed at IF 4, the highest impact factor grade, among the outstanding international conferences in computer science under the BK21 Plus programme.

The paper by master's students Bang Na-mo and Lee Ji-hyun of the Graduate Department of Artificial Intelligence (third semester, joint first authors) and Professor Koo Myoung-wan of the Department of Computer Science & Engineering (corresponding author), titled 'Task-Optimized Adapters for an End-to-End Task-Oriented Dialogue System', will appear in the proceedings of ACL 2023, held from 10 to 12 July.

The team proposed a task-oriented dialogue system that adds adapter modules to a large pre-trained language model (LLM) with frozen parameters and improves performance through reinforcement learning. The adapters optimize the dialogue system according to the speech-act objective — user utterance understanding, dialogue state tracking, system response generation and so on — thereby improving the performance of each function. In this study too they were applied to improve the performance of the dialogue system to which they were added.

▲ The task-oriented dialogue system architecture proposed in 'Task-Optimized Adapters for an End-to-End Task-Oriented Dialogue System'

The team showed that the idea of adding adapters to an existing LLM can also minimize the computational cost of improving dialogue system performance. To build a dialogue system with an LLM and have it perform various speech-act tasks, parameters must be fine-tuned for each task. Until now, fine-tuning an LLM has required sharing all the parameters used in pre-training, incurring high computational costs, and catastrophic forgetting — the loss of what was learned in pre-training — has been raised as a problem. By training only adapters composed of a small number of parameters during fine-tuning, rather than all of the LLM's parameters, the team resolved both the training cost problem and the problem of forgetting. In terms of the training method, they also demonstrated that optimization for each speech-act task is possible through reinforcement learning.

▲ The detailed adapter architecture proposed in 'Task-Optimized Adapters for an End-to-End Task-Oriented Dialogue System'

This research carries particular significance as an achievement by students who majored in the humanities and social sciences as undergraduates before entering the first cohort of the University's interdisciplinary programme in artificial intelligence. Bang Na-mo comes from the University's School of Communication, where she majored in journalism and broadcasting with a convergence major in Big Data Science. Lee Ji-hyun graduated in linguistics from Korea University before entering the University's master's programme. Both students are DHE (Digital Human Entertainment) scholarship holders sponsored by Smilegate.

Professor Koo Myoung-wan said: "This is a case of drawing on the insight of students from humanities backgrounds as a strength in artificial intelligence, a convergence discipline," adding: "I will continue to guide students so that they can create synergy by combining engineering with domain knowledge."