The research team of Professor Seo Jung-yun of Sogang University's Department of Computer Science & Engineering (doctoral student Kim Byung-jae*, master's students Jung Young-hoon* and Lee In-kwon, and undergraduate interns Jung Yeon-seok and Lee Se-jin) took first place in the Situated Interactive Multi-Modal Conversational AI category of the Ninth Dialogue System Technology Challenge (DSTC9), a leading international competition in AI dialogue systems (* joint contribution). DSTC9 was co-hosted by IBM Research, Google Research, Amazon Alexa AI and the Technical University of Madrid, and ran for about three months following the release of the dataset in June. Track 4, in which the team competed, concerned developing a next-generation shopping system in which users converse with an AI shop assistant using visual information such as AR/VR while shopping for furniture or clothes. The SIMMC (Situated and Interactive Multimodal Conversations) data used was collected by Facebook AI and newly released this year.

(Reference: https://ai.facebook.com/blog/simmc-a-data-set-for-developing-next-generation-shopping-assistants/) The sub-tasks were 1) API calls (dialogue policy), 2) system utterance generation (natural language generation; NLG) and 3) dialogue state tracking (DST), with winning teams selected for each. The team took the honour of first place in tasks 2 and 3 and second place in task 1. For tasks 2 and 3 they designed a single deep neural network model based on the language generation model GPT-2 that performs dialogue state tracking and utterance generation together, and greatly improved performance through preprocessing. For task 1 they proposed a model based on the long short-term memory (LSTM) architecture that applies three gates to fuse visual information useful for shopping with the user's utterance information. The research will be presented at the Dialogue System Technology Challenge workshop of the 2021 Association for the Advancement of Artificial Intelligence (AAAI) conference. An additional invited presentation to researchers and developers of the Facebook Conversational AI team is also planned.

References

DSTC9 official website https://dstc9.dstc.community/

▲ (From left) Jung Young-hoon (master's student), Kim Byung-jae (doctoral student) and Lee In-kwon (master's student), Department of Computer Science & Engineering

▲ (From left) Jung Yeon-seok and Lee Se-jin, undergraduate interns, Department of Computer Science & Engineering

▲ (From left) Jung Yeon-seok and Lee Se-jin (undergraduate interns), Kim Byung-jae (doctoral student), supervisor Professor Seo Jung-yun, Jung Young-hoon and Lee In-kwon (master's students), Department of Computer Science & Engineering