Professor Lee Young-min

A paper written with Professor Lee Young-min as corresponding author has been selected for 'Design Automation and Test in Europe' (DATE 2025), a conference on hardware–software design automation (an outstanding conference of the Korean Institute of Information Scientists and Engineers; BK recognized IF=2), and will be presented at DATE 2025 in Lyon, France, from 31 March to 2 April 2025.

 

The research, to be presented under the title 'SparseInfer: Training-free Prediction of Activation Sparsity for Fast LLM Inference', proposes SparseInfer, a new method for efficiently predicting output activation sparsity in order to accelerate LLM inference.

 

Whereas input activation sparsity allows only the computation of individual elements to be skipped, output activation sparsity is highly efficient in that it allows all the memory operations and computation for the entire set of input vectors involved to be skipped. Predicting it in advance, however, is a difficult problem. By predicting output activation sparsity accurately and efficiently without any separate training, the research accelerated the MLP of an LLM by 3.5 times and full LLM inference by 1.8 times with no loss of accuracy.

Professor Lee Young-min said: "This research shows that for any given LLM, output activation sparsity can be predicted relatively accurately without separate training, allowing LLMs to be greatly accelerated with no loss of accuracy. LLMs suffer many memory bottlenecks, and an approach that exploits output activation sparsity is very promising because it relieves memory bottlenecks as well as computation; I expect this paper to be the start of a series of research results."

 

References:

Conference: https://www.date-conference.com/

Paper: https://arxiv.org/abs/2411.12692