MIT researchers have created a periodic table that shows how more than 20 classic machine learning algorithms are connected. The fresh framework sheds delicate on how scientists can combine strategies from various methods of improving existing AI models or inventing fresh ones.
For example, scientists used their frames to combine elements of two different algorithms to create a fresh algorithm of image classification, which reached 8 percent better than the current most newfangled approaches.
The periodic table results from one key idea: all these algorithms learn a specific type of relationship between data points. While each algorithm can achieve this in a slightly different way, the basic mathematics standing behind each approach is the same.
Based on these observations, scientists identified the uniting equation, which underlies many classic AI algorithms. They used this equation to reformulate popular methods and arrange them in the table, categorizing each of them on the basis of approximate relationships.
Like the periodic table of chemical elements, which initially contained empty squares, which were later filled with scientists, the periodic machine learning table also has empty spaces. These spaces predict where algorithms should exist, but which have not yet been discovered.
The table gives scientists a set of tools for designing fresh algorithms without the need to discover ideas from previous approaches, says Shaden Alshammari, a graduate of MIT and main author A paper with this new frame.
“It’s not just a metaphor,” adds Alshammari. “We begin to perceive machine learning as a system with a structure that is a space that we can discover, not guess.”
It is joined by the article by John Hershey, a researcher in Google AI perception; Axel Feldmann, MIT graduate; William Freeman, Thomas and Gerd Perkins professor of electrical and computer science and a member of the IT laboratory and artificial intelligence (CSIL); and senior author Mark Hamilton, a graduate of MIT and senior engineering manager at Microsoft. The research will be presented at an international conference on the representation of learning.
Accidental equation
Scientists did not decide to create a periodic machine learning table.
After joining Freeman Lab, Alshammari began to study the cluster, machine learning technique that classifies images, learning to organize similar images into nearby clusters.
She realized that the grouping algorithm she studied was similar to another classic machine learning algorithm, called contrasting learning, and began to dig deeper into mathematics. Alshammari stated that these two different algorithms can be converted using the same equation underlying.
“We almost reached this uniting equation. When Shaden discovered that he connects two methods, we just started to dream of new methods of bringing to this frame. Almost everyone from which we tried can be added,” says Hamilton.
The framework they created, Information Contrastive Learning (I-Con) shows how you can view various algorithms through the lens of this uniting equation. It covers everything from classification algorithms that can detect spam to deep learning algorithms that LLM supply.
The equation describes how such algorithms find connections between real data points, and then present these connections internally.
Each algorithm aims to minimize the deviation between the connections that it learns to bring closer and real connections in their training data.
They decided to organize the I-Con in the periodic table to categorize algorithms based on how to combine points in real data sets, and the main ways in which algorithms can bring these connections closer.
“The works lasted gradually, but when we identified the general structure of this equation, it was easier to add more methods to our frames,” says Alshammari.
Tool to discover
When they organized a table, scientists began to see gaps in which algorithms could exist, but which have not yet been invented.
Scientists have filled one gap, lending ideas from machine learning technique called contrasting learning and using them for grouping images. This resulted in a fresh algorithm that could classify unknown images 8 percent better than the other most newfangled approach.
They also used I-Con to show how you can utilize the technique of debating data for contrasting learning to raise the accuracy of clustering algorithms.
In addition, a versatile periodic table allows researchers to add fresh rows and columns to present additional types of connections with the data point.
Ultimately, having i-con as a guide can facilitate scientists to learn about machine learning outside the box, encouraging them to combine ideas in a way they would not necessarily think that differently, says Hamilton.
“We have shown that only one very elegant equation, rooted in science of information, gives rich algorithms covering 100 years of machine learning research. This opens many new discovery opportunities,” he adds.
“Perhaps the most tough aspect of being a scientist of machine learning is currently the seemingly unlimited number of articles that appear every year. In this context, articles that unite and combine existing algorithms are of great importance, but they are extremely sporadic. I-con is an excellent example Jerusalem, who was not involved in this research.
These studies were partly financed by Air Force Artificial Intelligence Accelerator, National Science Foundation AI Institute for Artificial Intelligence and Fundamental Interactions and Quanta Computer.