For natural language to be an effective form of communication, the parties involved must be able to understand the words and their context, assume that the content is largely provided in good faith and is trustworthy, justify the sharing of information, and then apply it to real-world scenarios . MIT graduate students interning at the MIT-IBM Watson AI Lab—Athul Paul Jacob SM ’22, Maohao Shen SM ’23, Victor Butoi, and Andi Peng SM ’23—are working to attack each step of this process written in natural language models, thus AI systems can be more reliable and right for users.
To achieve this, Jacob’s research goes to the heart of existing natural language models to improve results, using game theory. His interests, he says, are twofold: “One is understanding how humans behave, using the lens of multi-agent systems and understanding language, and the other is how do we use this as knowledge to create better artificial intelligence systems?” ” His work stems from the board game Diplomacy, in which his research team developed a system that can learn and predict human behavior and strategically negotiate to achieve a desired, optimal outcome.
“It was a game where you had to build trust; you have to communicate using language. You also have to play with six other players at the same time, which is very different from any task domain that people have tackled in the past,” says Jacob, referring to other games like poker and GO that researchers have embedded in neural networks. “As a result, many research challenges have arisen. One of them was: “How to model people?” How do you know if people are behaving irrationally?” Jacob and his research mentors—including Associate Professor Jacob Andreas and Assistant Professor Gabriele Farina of MIT’s Department of Electrical Engineering and Computer Science (EECS) and MIT-IBM Watson Yikang Shen of the AI Lab—present the problem of generating language in a two-player game.
Using “generator” and “discriminator” models, Jacob’s team developed a natural language system that allows you to create answers to questions, then observe the answers and determine whether they are correct. If so, the AI system receives a point; if not, no point will be awarded. Language models are notoriously prone to hallucinations, which makes them less trustworthy; This no-regret learning algorithm collectively adopts a natural language model and encourages the system’s responses to be more truthful and reliable, while keeping solutions close to the priorities of the pre-trained language model. Jacob says that using this technique in conjunction with a smaller language model could likely make it competitive with a model many times larger with the same performance.
Once a language model has produced an output, ideally researchers want its confidence in producing it to match its accuracy, but this is often not the case. Hallucinations can occur when a model reports a high level of confidence when it should be low. Maohao Shen and his group, along with mentors Gregory Wornell, Sumitomo Professor of Engineering at EECS, and IBM Research lab researchers Subhro Das, Prasanna Sattigeri and Soumya Ghosh — want to solve this problem through uncertainty quantification (UQ). “Our project aims to calibrate language models when they are poorly calibrated,” says Shen. In particular, they deal with the problem of classification. To do this, Shen allows the language model to generate arbitrary text, which is then converted into a multiple-choice classification task. For example, they can ask the model to solve a math problem and then ask whether the generated answer is correct with “yes, no, or maybe.” This helps determine whether the model is overconfident or underconfident.
By automating this, the team developed a technique that helps tune the confidence score using a pre-trained language model. The researchers trained an auxiliary model using background information so that their system could correct the language model. “If your model is overconfident in its predictions, we can detect that and reduce the confidence, and vice versa,” Shen explains. The team evaluated their technique on a number of popular benchmark datasets to demonstrate how well it generalizes from unseen tasks to re-adjust the accuracy and confidence of a language model’s predictions. “After training, you can simply plug in the technique and apply it to new tasks without any other supervision,” Shen says. “All you need is data for this new task.”
Victor Butoi is also expanding the capabilities of the models, but instead his lab team—which includes John Guttag, the Dugald C. Jackson Professor of Computer Science and Electrical Engineering at EECS; lab researchers Leonid Karlinsky and Rogerio Feris of IBM Research; and lab staff Hilde Kühne from the University of Bonn and Wei Lin from the Technical University of Graz – create techniques that enable visual-linguistic models to reason about what they see, and design prompts that unlock novel learning opportunities and understand key phrases.
Compositional reasoning is just another aspect of the decision-making process that we ask machine learning models to perform to be helpful in real-world situations, Butoi explains. “You need to be able to think about problems compositionally and solve subtasks,” says Butoi, “for example, if you say a chair is on the left side of a person, you need to recognize both the chair and the person. You need to understand the directions. And once the model understands “left,” the research team wants the model to be able to answer other questions about “left.”
Surprisingly, vision language models don’t handle composition well, Butoi explains, but they can be helped with this by using a model that can “guide the witness,” if you will. The team developed a model that was refined using a technique called vast low-rank language model adaptation (LoRA) and trained on an annotated dataset called Visual Genome, which contains objects in the image and arrows denoting relationships such as directions. In this case, the trained LoRA model will be instructed to say something about the “left” relationships, and the resulting signature will then be used to provide context and hints for the visual language model, making it “a much easier task,” Butoi says.
In the world of robotics, artificial intelligence systems also communicate with the environment using computer vision and language. Settings can range from warehouses to home. Andi Peng and mentors HN Slater, professor of aeronautics and astronautics at MIT, Julie Shah and Chuang Gan at the lab and the University of Massachusetts at Amherst, are focused on helping people with physical limitations using virtual worlds. To this end, Peng’s group is developing two embodied artificial intelligence models – a “human” in need of support and a supporting agent – in a simulated environment called ThreeDWorld. Focusing on human-robot interactions, the team uses semantic priorities captured in vast language models to lend a hand artificial intelligence infer what capabilities a “human” agent may not be able to perform and the motivation behind “human” actions, leveraging natural Tongue. The team wants to strengthen the midfielder’s sequential decision-making, two-way communication, ability to understand the physical situation and how best to contribute.
“Many people believe that artificial intelligence programs should be autonomous, but in my opinion an important part of this process is building robots and systems for humans and being willing to transfer human knowledge,” Peng says. “We don’t want the system to do something in a weird way; we want them to do it in a human way that we can understand.”