If you think an artificial intelligence model running on thousands of cutting-edge computer chips is clever, let me introduce you to the concept of a 1-year-old.
OK, so kids may not be able to write computer programs, solve advanced math problems, or debate philosophical concepts. But unlike today’s artificial intelligence models, which consume an ocean of training data and as much energy as a tiny country, children learn to understand the world with astonishing efficiency. They identify modern objects after seeing them once or twice and learn through fleeting observation and physical interaction.
When it comes to improving AI, children – and the architecture of their brains – may hold key insights. Creating a more child-like version of AI could make pioneering models less pricey and less energy-intensive, but it could also be valuable if AI-powered robots are to learn about their environment in a more natural way.
To explore this bold modern frontier, scientists from Meta, Stanford University, the University of Tokyo and France’s École Normale Supérieure developed a modern test that highlights children’s learning skills and encourages artificial intelligence researchers to design algorithms that match them.
The EgoBabyVLM Challenge assesses how well visual language models, or VLMs, that learn from both text and images can make sense of the world a child sees. It requires a model that describes the world after it has been digested a thousand hours of movies collected from cameras strapped to the heads of infants and youthful children. (Yes, really.)
It turns out that state-of-the-art models fail when fed with realistic and sloppy material, suggesting that there may be something different about the design of a child’s brain that allows it to learn so quickly from so little information.
Instead of using selected data sets, children learn from a kaleidoscopic point of view: parents talk about objects that are no longer observable, point to things by sight or gesture, or discuss events in the past or future rather than what is happening right now. Babies learn not only through language but also through luxurious multimodal and tactile experiences, says Michael Frank, a cognitive scientist at Stanford University who specializes in language learning and was involved in the development of EgoBabyVLM.
The test shows that when it comes to artificial intelligence, “it is clear that there is more of it [than just language] it’s necessary,” says Frank.
Learning a language
EgoBabyVLM is the latest example of how scientists are using artificial intelligence to study human intelligence. The so-called challenge ChildLMintroduced in 2023, tasked AI models to learn the syntax of a language using roughly the same amount of data as a 10-year-old – tens of millions of words compared to trillions for AI models. Remarkably, it turns out that transformer-based AI models – which process language by paying attention to the relationships between words in different sentences – do this quite well, which is a challenge for Ideas of Noam Chomsky regarding how syntax may be hard-wired into the human brain.
Ryan Cotterell, a linguist at ETH Zurich who first developed BabyLM, says the situation is different when it comes to understanding the physical world. “There won’t be a large set of human interactions – there’s no Internet of human interactions,” he says.
Joshua Tenenbaum, a cognitive scientist at the Massachusetts Institute of Technology, notes that BabyLM showed that models do not acquire “common sense” about the physical world, social dynamics, or theory of mind.
“Transformers are very good at finding patterns in data,” Tenenbaum says. “But regular pattern learning systems don’t seem to be able to take the data the baby receives and learn everything they do.”