It is estimated that due to the rapid development of artificial intelligence, data centers will consume resources to 12 percent of total U.S. electricity by 2028– reports Lawrence Berkeley National Laboratory. Improving the energy efficiency of data centers is one way researchers are trying to make AI more sustainable.
To achieve this goal, researchers at MIT and the MIT-IBM Watson AI Lab have developed a rapid prediction tool that tells data center operators how much power will be consumed by running a specific AI workload on a specific processor or AI accelerator chip.
Their method produces reliable power estimates in seconds, unlike conventional modeling techniques that can take hours or even days to produce results. Moreover, their prediction tool can be applied to a wide range of hardware configurations – even up-to-date designs that have not yet been implemented.
Data center operators could employ these estimates to efficiently allocate constrained resources among multiple models and AI processors, improving energy efficiency. Additionally, this tool could enable algorithm developers and model providers to assess the potential energy consumption of a up-to-date model before deploying it.
“The AI sustainability challenge is an urgent question we need to answer. Because our estimation method is fast, convenient, and provides direct feedback, we hope it will make algorithm developers and data center operators think more about reducing energy consumption,” says Kyungmi Lee, an MIT postdoc and lead author of the book article about this technique.
She is joined in the article by Zhiye Song, an electrical engineering and computer science (EECS) graduate; Eun Kyung Lee and Xin Zhang, research managers at IBM Research and MIT-IBM Watson AI Lab; Tamar Eilam, IBM Fellow, principal scientist for sustainable computing at IBM Research and member of the MIT-IBM Watson AI Lab; and senior author Anantha P. Chandrakasan, MIT chancellor, Vannevar Bush Professor of Electrical Engineering and Computer Science, and member of the MIT-IBM Watson AI Lab. The study’s results will be presented this week at the IEEE International Symposium on System and Software Performance Analysis.
Speeding up energy estimation
In the data center, thousands of powerful graphics processing units (GPUs) perform the operations of training and deploying artificial intelligence models. The power consumption of a specific GPU will vary depending on its configuration and the workload it supports.
Many conventional methods of predicting energy consumption involve dividing the workload into individual stages and emulating how each GPU module is used step by step. However, AI workloads such as model training and data preprocessing are extremely high, and simulating in this way can take many hours or even days.
“If as an operator I want to compare different algorithms or configurations to find the most energy-efficient way to proceed, and a single emulation takes several days, this becomes very impractical,” says Lee.
To speed up the prediction process, MIT researchers tried to employ less detailed information that could be estimated more quickly. They found that AI workloads often have many repeating patterns. They could employ these patterns to generate the information needed to estimate power reliably but quickly.
In many cases, algorithm developers write programs to run as efficiently as possible on the GPU. For example, they employ well-organized optimizations to distribute work among parallel processing cores and move pieces of data in the most proficient way.
“Optimizations from software developers create a regular structure, and that’s what we’re trying to leverage,” Lee explains.
The researchers developed a lightweight estimation model called EnergAIzer that captures GPU power consumption patterns based on optimization.
Correct assessment
While the estimates were quick, the researchers found they did not take into account all energy costs. For example, every time the GPU runs a program, there is a constant energy cost to set up and configure that program. Then, each time the GPU performs an operation on a piece of data, an additional energy cost must be paid.
Due to hardware fluctuations or conflicts in accessing or moving data, the GPU may not be able to employ all of its available bandwidth, slowing down performance and drawing more power over time.
To account for these additional costs and variations, the researchers collected actual measurements from GPUs to generate correction factors that they applied to their estimation model.
“This way we can get a quick and very accurate estimate,” he says.
Finally, the user can provide information about the workload, such as the AI model they want to run and the number and length of user input to process, and EnergAIzer will display an estimate of energy consumption within seconds.
The user can also change the GPU configuration or adjust the operating speed to see how such design choices affect the overall power consumption.
When researchers tested EnergAIzer using real AI workload information from real GPUs, they were able to estimate power consumption with an error of only about 8 percent, which is comparable to conventional methods that can take hours to obtain results.
Their method can also be used to predict the power consumption of future GPUs and up-to-date device configurations, as long as the hardware does not change drastically in the tiny term.
In the future, researchers want to test EnergAIzer on the latest GPU configurations and scale the model so that it can be applied to multiple GPUs working together to execute bulky workloads.
“To truly make an impact on sustainability, we need a tool that provides a rapid, stack-wide energy estimation solution for hardware designers, data center operators, and algorithm developers so that everyone can be more energy conscious. With this tool, we have taken one step toward that goal,” says Lee.
This research was funded in part by the MIT-IBM Watson AI Lab.