Best Paper Award at the Korea Software Congress 2022 (KSC 2022)

- Data-Centric Computing and Systems Laboratory -

The paper 'A Technique for Accelerating Distributed Deep Learning by Minimizing Data Imbalance', by graduate student Maeng San-ha of the Department of Computer Science & Engineering (supervisor: Professor Park Sung-yong), received the Best Paper Award at the 2022 Korea Software Congress (KSC 2022), held over four days from 20 to 23 December.

▲ From left: Maeng San-ha (master's, 4th semester) and Professor Park Sung-yong (supervisor)

▲ Certificate of the Best Paper Award at the Korea Software Congress 2022 (KSC 2022)

Because deep learning is computationally intensive and time-consuming, distributed deep learning using GPU clusters is widely used to accelerate training. Training delays in distributed deep learning are heavily influenced by the slowest straggler. Recent studies have proposed various training policies to address the straggler problem, but because they were designed on the assumption that all training samples are of uniform size — as with image data — they failed to recognize the training delays caused by data with imbalanced sample sizes, such as video or audio.

This paper recognizes the imbalance in training sample sizes and presents scheduling and shuffling policies to address the resulting straggler problem.

Evaluation of the data-imbalance-aware scheduler showed training time reduced by up to 53% and convergence accelerated by up to 24% compared with the general-purpose machine learning framework TensorFlow.