GreenLLM: SLO-Aware Dynamic Frequency Scaling for Energy-Efficient LLM Serving

Today's Large Language Models (LLMs) inference costs are dominated by GPU energy consumption. LLM inference consists of two stages: prefill and decode, each exhibiting different characteristics. However, current GPU power governors disregard these differences and are not aware of runtime performance, leading to suboptimal DVFS configurations, head-of-line blocking, and excessive energy consumption. We introduce GreenLLM, a service-level objectives (SLO)aware serving framework that minimizes GPU energy by explicitly separating prefill and decode control with a runtime optimization algorithm for end-to-end energy reduction without sacrificing performance. For prefill, GreenLLM solves a queueing-aware optimization to select the energyminimal frequency for the target GPU. During decode, a dual-loop controller tracks throughput (tokens-per-second) and adjusts frequency with hysteresis during runtime, using fine grained steps to maintain tail TBT within target bounds. Across Alibaba and Azure trace replays, GreenLLM achieves up to 34% reduction in total energy consumption compared to the default DVFS baseline, with no loss of throughput and only less than 3.5% increase in SLO violations.