Have you ever been asked a question that you only knew part of the answer to? To give a more informed answer, it might be best to call a friend who knows more about the subject.
This collaborative process can also aid immense language models (LLMs) improve their accuracy. It has been challenging to teach LLMs to recognize when they should work with another model to come up with an answer. Instead of using complicated formulas or immense amounts of labeled data to determine where models should work together, researchers at MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) imagined a more organic approach.
Their modern algorithm, called “Co-LLM,” can combine LLM’s universal base model with a more specialized model and aid them work together. When the former generates an answer, Co-LLM examines each word (or token) in its answer to see where it can reference a more precise answer from the expert model. This process leads to more precise answers for things like medical clues and math and reasoning problems. Because the expert model isn’t needed in every iteration, it also leads to more productive answer generation.
To decide when the base model needs aid from the expert model, the framework uses machine learning to train a “switch variable,” or a tool that can indicate the competence of each word in the answers of the two LLMs. The switch is like a project manager who figures out where to call in a specialist. If you asked the Co-LLM to give you some examples of extinct bear species, for example, the two models would work out the answers together. The universal LLM starts putting together the answer, and the switch variable steps in on the parts where it can put in a better token from the expert model, such as adding the year the bear species went extinct.
“With Co-LLM, we’re essentially training a generic LLM to ‘phone’ an expert model when needed,” says Shannon Shen, an MIT doctoral candidate in electrical engineering and computer science and a CSAIL associate who is the lead author new article about the approach“We use domain-specific data to teach a baseline model its counterpart’s expert knowledge in areas such as biomedical tasks and mathematical and reasoning questions. This process automatically finds parts of the data that are difficult for the baseline model to generate, and then instructs the baseline model to switch to an expert LLM that has been pretrained on data from a similar domain. The general-purpose model provides the “scaffolding” for generating, and when it calls on a specialized LLM, it prompts the expert to generate the desired tokens. Our findings indicate that LLMs learn collaborative patterns organically, resembling the way humans recognize when to call on an expert to fill in the gaps.”
Combining flexibility and facts
Imagine asking an LLM to provide the ingredients of a specific prescription drug. They may not answer correctly, requiring a specialized model.
To demonstrate the flexibility of the Co-LLM program, the researchers used data such as BioASQ a medical kit combining the basic LLM programme with expert LLM programmes in various fields, such as: Meditron Modelwhich is pre-trained on unlabeled medical data. This enabled the algorithm to aid answer questions that a biomedical expert is typically asked, such as naming the mechanisms that cause a given disease.
For example, if you ask a basic LLM to list the ingredients of a particular prescription drug, it might not answer correctly. With the added expertise of a model specializing in biomedical data, you get a more precise answer. Co-LLM also notifies users where to double-check their answers.
Another example of how Co-LLM improved performance: When given the task of solving a math problem such as “a3 a2 if a=5,” the general-purpose model incorrectly calculated the answer to 125. As Co-LLM trained the model to work better with a immense mathematical LLM called LemmaTogether they determined that the correct solution was 3.125.
Co-LLM produced more precise responses than both tuned basic LLM and untuned specialized models working independently. Co-LLM can guide two models that have been trained differently to work together, whereas other successful collaborative LLM approaches, such as “Proxy tuning,”require all of their component models to be trained in a similar manner. Furthermore, this baseline requires that each model be used simultaneously to generate an answer, whereas the MIT algorithm simply activates its expert model for specific tokens, leading to more productive generation.
When to ask an expert
The MIT researchers’ algorithm highlights that more faithfully mimicking human teamwork could improve accuracy in multilevel collaborative LLM. To further improve its factual accuracy, the team could employ human self-correction: They are considering a more tough deferral approach that could backtrack when the expert model doesn’t give the right answer. This upgrade would allow the Co-LLM to course-correct so that the algorithm could still give a satisfactory answer.
The team would also like to update the expert model (by training only the baseline model) as modern information becomes available, keeping the answers as up-to-date as possible. This would allow the Co-LLM to combine the most up-to-date information with mighty reasoning power. Ultimately, the model could aid with enterprise documents by using the latest information it has to update them accordingly. The Co-LLM could also train diminutive, private models to work with the more productive LLM to improve documents that need to remain on the server.
“Co-LLM presents an interesting approach to learning to choose between two models to improve performance and efficiency,” says Colin Raffel, an associate professor at the University of Toronto and deputy director of research at the Vector Institute, who was not involved in the study. “Because routing decisions are made at the token level, Co-LLM provides a granular way to defer difficult generation steps to a more efficient model. The unique combination of model-level and token-level routing also provides a great deal of flexibility that similar methods lack. Co-LLM contributes to an important line of work that aims to develop ecosystems of specialized models to outperform expensive monolithic AI systems.”
Shen wrote the paper with four other CSAIL members: graduate student Hunter Lang ’17, MEng ’18; former postdoctoral fellow and Apple AI/ML researcher Bailin Wang; MIT assistant professor of electrical engineering and computer science Yoon Kim; and professor and Jameel Clinic member David Sontag PhD ’10, who are part of the MIT-IBM Watson AI Lab. Their research was supported in part by the National Science Foundation, the National Defense Science and Engineering Graduate (NDSEG) Fellowship, the MIT-IBM Watson AI Lab, and Amazon. Their work was presented at the annual meeting of the Association for Computational Linguistics.