To do this, scientists from Stanford and the University of Washington used a method known as distillation – which allows smaller models to draw on the answers resulting from the larger ones – to improve S1 by means of answering from the AI Google reasoning model, Flash Experimental thinking. Google Service conditions Remember that you cannot operate the APi Gemini interface to “develop models competing with” AI models of the company. The Verge He contacted Google with a request for comment, but I will not immediately know.
Scientists based S1 on Qwen2.5, the Open Source model from Alibaba Cloud. Initially, they started with a pool of 59,000 questions to train the model, but they found that a larger set of data did not offer “significant benefits” in relation to the bankrupt set only 1000. Scientists say that they trained the model of only 16 GPU NVIDIA H100.
The S1 model also uses a technique called test time, enabling the model “thinking” for a long time before creating an answer. As noted in the article, scientists forced the model to continue reasoning, adding “waiting” to the model response. “This can lead the model to a double response, often setting incorrect steps of reasoning,” says the article.
The OPENAI O1 reasoning model uses a similar approach, something that the AI Deepseek startup tried to repeat with the launch of the R1 model, which, he claims, was trained for a fraction of costs. Since then, Opeli has accused Deepseek of distillation of information from models to build a competitor, violating his conditions for providing services. As for S1, scientists say that S1 “exceeds O1 review on mathematical questions of competition by up to 27%”.
The escalate in smaller and cheaper AI models threatens to raise the entire industry. They can prove that the main companies such as Opeli, Microsoft, Meta and Google do not have to spend billions of dollars on AI training, while building huge data centers filled with thousands of NVIDIA graphic processors.