Regular Paper Accepted at the Outstanding International Conference, the 23rd IEEE/ACM International Symposium on Cluster,

Cloud and Internet Computing

 

 

The paper 'Chronica: A Data-Imbalance-Aware Scheduler for Distributed Deep Learning', written by Maeng San-ha of the Department of Computer Science & Engineering (Data-Centric Computing and Systems Laboratory) and Professor Moon Eui-hyun (Machine Learning Systems Laboratory) with corresponding author Professor Park Sung-yong (Data-Centric Computing and Systems Laboratory), has been accepted for publication at the IEEE/ACM International Symposium on Cluster, Cloud and Internet Computing (CCGrid), listed at an adjusted IF of 1 among the outstanding international conferences in computer science under the BK21 Plus programme.

▲ (From left) Maeng San-ha, Professor Moon Eui-hyun and Professor Park Sung-yong

The aim of the research is to improve training speed and scalability when distributing the training of deep learning models over data — such as video, audio or text — where the number of operations required differs from sample to sample; the proposed system is named Chronica.

 

Chronica infers the performance of each process from the size of the data trained on and the time taken. It thereby equalizes training time across all processes, accelerating training by minimizing synchronization delays between processes. Beyond shortening training time, Chronica further accelerates training by resolving the problem of gradient weight imbalance arising from differences in data sample size.

 

Chronica was evaluated on a cluster of 32 GPU instances on Amazon's cloud using four deep learning models and data with six different degrees of imbalance, and showed training up to 3.19 times faster than the comparison systems.

 

IEEE/ACM CCGrid is a distinguished international conference covering a wide range of topics in distributed systems. It will be held in Bengaluru, India, from 1 to 4 May this year.

 

 

References

 

CCGrid 2023 homepage: https://ccgrid2023.iisc.ac.in/

Chronica paper: https://discos.sogang.ac.kr/file/2023/intl_conf/ccgrid23_chronica.pdf