Regular Paper Accepted at the Top International Conference Design Automation Conference (DAC) 2025

Professor Lee Young-min

A paper written with Professor Lee Young-min as corresponding author was selected for the 'Design Automation Conference' (DAC 2025), a conference on hardware–software design automation (a top conference of the Korean Institute of Information Scientists and Engineers; BK recognized IF=3), and was presented at DAC 2025 in San Francisco, United States, from 22 to 25 June 2025.

The research, presented under the title 'Grasp: Group-based Prediction of Activation Sparsity for Fast LLM Inference', proposes Grasp, a new method for efficiently predicting output activation sparsity in order to accelerate LLM inference.

Whereas input activation sparsity allows only the computation of individual elements to be skipped, output activation sparsity allows all the memory loads and computation for the entire set of input vectors involved to be skipped, making it highly effective for LLM inference, where memory loading is the main performance bottleneck. Predicting output activation sparsity accurately, however, is a difficult problem. By predicting it accurately and efficiently, this research shortened the MLP (FFN) of an LLM by a factor of 3.5 and total LLM inference time by a factor of 1.9 (with an upper bound of 2.1) with no loss of accuracy.

Professor Lee Young-min said: "LLM inference has many memory bottlenecks, so simply increasing the number of GPU cores does not solve it; demand for expensive GPUs with HBM is rising precisely to relieve this memory bottleneck. An approach that exploits output activation sparsity relieves both computation and memory bottlenecks at once, which makes it very promising. Beyond the recently published SparseInfer and this paper (Grasp), I expect to continue publishing a series of research results on accelerating LLM inference with this approach."

References:

Conference: https://www.dac.com/ Laboratory: https://aisys.sogang.ac.kr https://aisys.sogang.ac.kr/