Q: Language is a prosperous ecosystem full of subtle nuances that people operate to communicate with each other—sarcasm, irony, and other forms of figurative language. There are many ways to convey meaning beyond the literal. Can vast language models understand the intricacies of context? What does it mean for a model to achieve “learning in context”? Moreover, how do multilingual transformers process varieties and dialects of different languages beyond English?
AND: When we think about linguistic contexts, these models are able to reason about much, much longer documents and text fragments with greater scope than anything we have been able to build before. But that’s only one kind of context. In humans, language production and understanding takes place in a well-established context. For example, I know I’m sitting at this table. There are objects I can refer to, and the language models we have today typically don’t recognize any of them when interacting with the user.
There is a broader social context that largely influences our operate of language to which these models are, at least not immediately, sensitive and of which they are unaware. It is unclear how to provide them with information about the social context in which their language generation and modeling takes place. Another essential thing is the time context. We’re making this movie at a certain point in time, when certain facts are true. The models we have today were trained on a snapshot of the Internet that stopped at a specific point in time – for most models we have today, probably several years ago – and they don’t know about anything that has happened since then. They don’t even know at what point they are generating the text. Figuring out how to provide all these different types of contexts is also an intriguing question.
Perhaps one of the most surprising elements is a phenomenon called contextual learning. If I take a miniature ML [machine learning] a set of data and feed it into a model, for example a movie review and the star rating assigned to the movie by a critic, you’re just giving a few examples of these things, language models generate the ability to both generate credible-sounding movie reviews and predict star ratings. More generally, if I have a machine learning problem, I have my inputs and my outputs. When you give a model input, give it another input, and ask it to predict the outcome, models often do it really well.
This is a very intriguing, fundamentally different way of machine learning where I have one vast general purpose model that I can insert many miniature machine learning datasets into, and at the same time without having to train a up-to-date model, classifier or generator or anything else that specializes in my specific task. This is actually something that we’ve been thinking about a lot in my group, and also in some of our work with colleagues at Google – trying to understand exactly how this learning phenomenon actually arises in context.
Q: We like to believe that people (at least to some extent) strive for what is objectively and morally recognized as truth. Immense language models, perhaps with poorly defined or misunderstood “moral compasses”, are not related to truth. Why do vast language models tend to hallucinate facts or confidently state inaccuracies? Does this limit its usefulness in applications where factual accuracy is critical? Is there a leading theory on how to solve this problem?
AND: It is well documented that these models hallucinate facts and that they are not always reliable. I recently asked ChatGPT to describe some of our group’s research. Five articles are listed, four of which are not actually existing documents and one is a real article written by a colleague of mine living in the UK with whom I have never co-authored. Facts are still a huge issue. Even beyond that, things that require reasoning in a very general sense, requiring complicated calculations and complicated conclusions, still tend to be really complex with these models. There may even be fundamental limitations to this transformer architecture, and I believe a lot more modeling work is needed to make things better.
Why this happens is still partly an open question, but perhaps, precisely from an architectural point of view, there are reasons why it is complex for these models to build coherent models of the world. They can do it a bit. You can ask them factual or trivia questions and most of the time they will answer correctly, maybe even more often than the average street user. But unlike the average user, it’s really not clear that there’s anything hidden in this language model that corresponds to a belief about the state of the world. I think this is both for architectural reasons, since transformers obviously have nowhere to put this belief, and also for training data, that these models are trained on the Internet, and the author of this is a bunch of different people at different times who believe different things on the state of the world. So it’s challenging to expect models to represent these things consistently.
All things considered, I don’t think this is a fundamental limitation of neural models of language or even more general models of language in general, but something that is true of today’s language models. We’re already seeing that models are getting closer to creating representations of facts, representations of the state of the world, so I think there’s still a lot of work to be done.
Q: The pace of progress from GPT-2 to GPT-3 to GPT-4 was dizzying. What does the trajectory rate look like from here? Will it be an exponential phenomenon or an S-curve that will decrease in the near future? If so, are there any limiting factors in terms of scale, computation, data or architecture?
AND: Certainly in the low term, my biggest concern is the issue of veracity and consistency that I mentioned earlier, which is that even the best models we have today produce incorrect facts. They generate code with errors, and because of the way these models work, they do so in a way that is particularly complex for humans to detect because the model output contains all the right surface statistics. When we think about code, it’s still an open question whether it’s less work for someone to write a function by hand, or to ask a language model to generate that function and then ask that person to verify that the implementation of that function was done properly.
There is a danger in rushing to implement these tools right away, and we end up in a world where everything is a little worse, but it’s very complex for people to reliably check the results of these models. That being said, these are problems that can be overcome. Especially given the pace at which things are moving, there is plenty of room to address issues of factuality and the consistency and correctness of generated code over the long term. These really are tools, tools that we can operate to free ourselves as a society from a lot of unpleasant tasks, responsibilities and challenging work that have been complex to automate – and that’s something to get excited about.