As part of our goal of building increasingly productive and general artificial intelligence (AI) systems, we are working to create AI tools with a broader understanding of the world. This can enable the transfer of useful knowledge across many different types of tasks.
Using reinforcement learning, our AlphaZero and MuZero AI systems achieved superhuman gaming performance. We’ve since expanded their capabilities to lend a hand design better computer chips, optimize data centers, and compress video. And our specialized version of AlphaZero, called AlphaDev, has also discovered recent algorithms to accelerate the software at the heart of our digital society.
Early results have shown the transformative potential of more versatile AI tools. Here, we explain how these advances are shaping the future of computing—and are already helping billions of people and the planet.
Designing Better Integrated Circuits
Specialized hardware is needed to make today’s AI systems resource-efficient for large-scale users. But designing and manufacturing recent computer chips can take years of work.
Our researchers have developed an AI-based approach to designing more productive and effective circuits. By treating a circuit like a neural network, we have found a way to speed up IC design and push performance to recent heights.
Neural networks are often designed to take input from a user and produce outputs such as images, text, or video. Inside a neural network, edges connect to nodes in a graph-like structure.
To create the circuit design, our team proposed “neural networks” – a recent type of neural network that turns edges into wires and nodes into logic gates and learns how to connect them together.
Animated illustration of a circuit neural network learning the design of a circuit. It determines which edges (wires) connect to which nodes (logic gates) to improve the overall circuit design.
We optimized the learned circuit for computational speed, energy efficiency, and size while maintaining its functionality. Using “simulated annealing,” a classic search technique that looks a step into the future, we also tested various options to find its optimal configuration.
Thanks to this technique we won IWLS 2023 Programming Competition — offering the best solution to 82% of competitor circuit design problems.
Our team also used the AlphaZero tool, which allows us to think ahead multiple steps, to refine the circuit design by treating the challenge as a game to be solved.
So far, our research combining neural networks with the reward function of reinforcement learning has produced very promising results for building even more advanced integrated circuits.
Data Center Resource Optimization
Data centers manage everything from delivering search results to processing data sets. Like a game of multi-dimensional Tetris, a system called Credit manages and optimizes workloads across Google’s huge data centers.
To schedule tasks, Borg relies on hand-coded rules. However, at Google’s scale, hand-coded rules can’t cover the variety of constantly changing workload distributions. So they’re designed as one size fits all.
This is where machine learning technologies like AlphaZero come in handy: they can operate at scale, automatically creating individual rules that are optimally suited to different load distributions.
During the training, AlphaZero learned to recognize patterns in the workloads coming into its data centers, as well as how to best manage capacity and make decisions that deliver the best long-term outcomes.
When we applied AlphaZero to Borg in experimental testing, we found that we could reduce the percentage of unused hardware in the data center by up to 19%.
Animated visualization of organized, optimized data storage versus cluttered, unoptimized storage.
Competent video compression
Streaming video makes up the majority of internet traffic. So finding ways to make streaming, no matter how large or miniature, more productive will have a huge impact on the millions of people who watch videos every day.
We worked with YouTube to compress and stream the video, leveraging MuZero’s problem-solving capabilities. By reducing the bitrate by 4%, MuZero has improved the overall YouTube experience — without compromising image quality.
We initially used MuZero to optimize the compression of each individual video frame. We have now extended this work to lend a hand make decisions about how to group and reference frames during encoding, leading to greater bitrate savings.
The results obtained in the first two steps give great hope that MuZero will become a more universal tool, helping to find optimal solutions in the entire video compression process.
Visualization showing how MuZero compresses video files. It defines groups of images with visual similarities to compress. A single keyframe is compressed. MuZero then compresses the other frames using the keyframe as a reference. The process is repeated for the rest of the video until compression is complete.
Discovering faster algorithms
AlfaDevversion of AlphaZero, made a breakthrough in computing when it discovered faster sorting and hashing algorithms. These basic processes are used trillions of times a day to sort, store, and retrieve data.
AlphaDev Sorting Algorithms
Sorting algorithms lend a hand digital devices process and display information, from ranking online search results and social media posts to user recommendations.
AlphaDev has discovered an algorithm that improves sorting performance for compact sequences of elements by 70%, and by about 1.7% for sequences with more than 250,000 elements, compared to the algorithms in the C++ library. This means that results generated from user queries can be sorted much faster. This saves a huge amount of time and energy when used on a enormous scale.
AlphaDev Hashing Algorithms
Hashing algorithms are often used to store and retrieve data, such as in a customer database. They typically operate a key (e.g., the username “Jane Doe”) to generate a unique hash that corresponds to the data values to be retrieved (e.g., “order number 164335-87”).
Like a librarian who uses a classification system to quickly find a specific book, the hashing system already lets the computer know what it’s looking for and where to find it. When applied to the 9-16 byte hashing range in data centers, AlphaDev’s algorithm improved performance by 30%.
The impact of these algorithms
We have added sorting algorithms to Standard C++ LLVM Library — replacing routines that had been in operate for over a decade. And he contributed to the creation of AlphaDev’s hashing algorithms Abseil’s library.
Since then, they have been used by millions of developers and companies in industries as diverse as cloud computing, online shopping, and supply chain management.
Universal tools to support our digital future
Our AI tools are already saving billions of people time and energy. This is just the beginning. We envision a future where universal AI tools can lend a hand optimize the global computing ecosystem.
We are not there yet – we still need a faster, more productive and more sustainable digital infrastructure.
Many more theoretical and technological breakthroughs are needed to create fully generalized AI tools. But the potential of these tools—in technology, science, and medicine—makes us excited about what’s on the horizon.