Outstanding International Conference International Conference on Parallel Processing (ICPP) 2025

Regular Paper Accepted

From left: Byun Hong-su (combined master's–doctoral), Yoo Hong-hyun (combined master's–doctoral)

 

The paper 'Revisiting Multi-threaded Compaction in LSM-trees: Enabling Compaction Pipelining', written by Byun Hong-su (joint first author) and Yoo Hong-hyun (joint first author) of the Data-Centric Computing and AI Systems Laboratory (DISCOS) with Professor Park Sung-yong (corresponding author), has been accepted for publication at the 54th International Conference on Parallel Processing (ICPP 2025).

 

The volume of data generated worldwide is forecast to grow to hundreds of zettabytes (ZB) within a few years. Accordingly, the capacity of a single SSD is being researched and developed beyond the terabyte (TB) level towards the petabyte (PB) level. As SSDs scale to petabytes, however, various difficulties arise in using the existing SSD software stack as it is — the limits of internal SSD memory size, the file system, and growing complexity in metadata management. Techniques for managing data directly in key-value form in user space are therefore attracting attention, and against this background LSM-tree based NoSQL key-value databases are drawing interest.

LSM-tree based stores continuously append data for fast write performance, and the process of periodically merging and reorganizing it is called compaction. Compaction is essential for removing duplicate data and maintaining the efficiency of the store. Existing LSM-tree based key-value stores have supported multi-threaded compaction in order to exploit the parallelism of modern storage devices, but close analysis of the actual performance gains of multi-threaded compaction has been surprisingly lacking.

 

The team analysed the parallelization limits of existing multi-threaded compaction and proposed DownForce, a compaction parallelization strategy to resolve them. Through pipelined compaction DownForce allows compaction work across levels to proceed simultaneously, eliminating thread imbalance and realizing fully multi-threaded compaction.

 

Figure 1. Step-by-step description of DownForce compaction. (a) Step 1: conditions for running DownForce compaction. (b) Step 2: managing the execution of DownForce compaction. (c) Step 3: the internal DownForce compaction process. (d) Step 4: commit guaranteeing consistency

 

DownForce was implemented in RocksDB, a representative LSM-tree based key-value database; in experiments it improved thread parallelism by an average of 1.44 times and throughput by up to 1.81 times over the existing method. By demonstrating that the performance bottleneck of LSM-trees can be effectively improved through a purely software approach, the research is highly significant for systems software research.

 

Byun Hong-su and Yoo Hong-hyun, both on the combined master's–doctoral course, said: "There were many challenges in analysing and overcoming the limits of existing multi-threaded compaction, but it is deeply meaningful to have developed an effective strategy in DownForce and demonstrated the performance gains. This research too had its share of obstacles, but that made the reward and sense of achievement all the greater when we solved the problems and delivered real performance improvements. We hope many students will experience the appeal and satisfaction of systems software research through challenges like this."

 

ICPP is a prestigious international conference for discussing the latest research trends and technological innovation in parallel computing; this year it will be held in San Diego, United States, from 8 to 11 September 2025. ICPP is also listed among the outstanding conferences in the software field of the Korean Institute of Information Scientists and Engineers and among the outstanding international conferences in computer science under the BK21 Plus programme (BK IF=2).

 

References:

- The 54th International Conference on Parallel Processing (ICPP 2025)

- Website : https://icpp2025.sdsc.edu/   

- Paper : https://discos.sogang.ac.kr/file/2025/intl_conf/ICPP_2025_H_Byun.pdf