The company relies on artificial intelligence systems that can understand and generate text, called language models. According to questionnaire According to John Snow Labs, technology leaders’ budgets for AI language technologies increased by at least 10% in 2020. One vendor, OpenAI, says its most vital language model, GPT-3, is used by tens of thousands of developers.
In recent years, novel vendors have emerged eager to get a piece of the pie, claiming to offer unique language modeling capabilities. In addition to well-resourced startups like OpenAI, Cohere, and Hugging Face, there are crop providers creating services based on open source artificial intelligence models. Sitting somewhere in the middle is there AI21 Labsan Israeli company that developed the model — Jurassic-1 Jumbo, which is about the size of GPT-3 — and slowly built products around it, including an “AI-as-a-service” platform called AI21 Studio that allows customers to create virtual assistants, chatbots, content moderation tools and more.
Investors clearly sense an opportunity. AI21 Labs today closed a $64 million Series B round that values the company at $664 million. Led by Ahren Innovation Capital Fund with participation from Mobileye CEO and co-founder Amnon Shashua, Walden Catalyst, Pitango, TPY Capital and Mark Leslie, this tranche brings A21Labs’ total capital to $118.5 million.
Co-founder and CEO Ori Goshen said the novel money will go toward research and development, particularly the development of larger and more sophisticated language models and talent recruitment. AI21 Labs currently employs 120 employees and plans to employ approximately 50 more by the end of the year, contrary to the macroeconomic trend.
“Fortunately, the pandemic had a positive impact on business — as more companies transitioned to remote work, individuals had to communicate in writing what they would normally share orally,” Goshen told TechCrunch in an email interview. “[Our] Proprietary core capabilities of multi-language models enable the processing of massive amounts of enterprise data used to… create custom content, summaries and classifications.”
AI21 Labs was founded in 2017 by Goshen, Shashua, and Stanford University professor Yoav Shoham. Corporate the first product was Wordtune, an artificial intelligence writing tool intended to compete with Grammarly, which suggests rephrasing text wherever the user writes. AI21 Studio was released last August with a pay-as-you-go service that allows developers to apply for access to custom models tailored to datasets unique to their requirements.
In AI21 Studio, AI21 Labs’ family of Jurassic-1 models can be used for paraphrasing (e.g. generating compact product names based on a product description), extracting numbers from text, and tagging emails and notes by subject or category. Models can also summarize content using a Wordtune feature called Wordtune Read, which includes excerpts from articles, reports, and PDF files.
Because they are trained on vast amounts of data from the Internet, including social media, language models are able to generate toxic and biased text based on similar language they have encountered during training. AI21 Labs models are no different; in early tests, one researcher managed to do this hint tell them that “people who love Jews are narrow-minded.” While AI21 Labs requires customers to agree to terms of utilize and usage guidelines, it has not implemented filters for potentially toxic content generated by its APIs.
AI21 Labs, which says it manually reviews requests for tuned models to combat abuse, says its models are “marginally less biased” than GPT-3.
Regardless, according to Goshen, models have the advantage of being supplemented with external sources of knowledge, such as Wikipedia. The latest version of AI21 Labs’ Jurassic-1 model, Jurassic-X, uses what Goshen calls a “modular knowledge system” to refine its answers using “discrete reasoning experts” such as online calculators and currency converters. As a result, Jurassic-X can answer “non-trivial” mathematical operations expressed in natural language, Goshen says, as well as simplify “complex” questions that other language models might encounter.
Of course, it’s worth noting that AI21 Labs hasn’t commissioned a comparison of its Jurassic-X models to other commercial models, so we’re left with nothing but claims.
The company’s questionable recent marketing stunt doesn’t inspire much confidence. In June, AI21 Labs launched chatbot were based on legal opinions by the slow Supreme Court Justice Ruth Bader Ginsburg, which several experts in the field of artificial intelligence technology have called misleading. In response to the criticism, AI21 Labs said the chatbot is “just an experiment” and admitted it may give misleading answers that should be taken “with a pinch of salt.”
When asked, Goshen declined to reveal company revenue figures or even growth estimates. But he said Studio has “hundreds” of paying customers and design partners – none of whom he declined to name – in addition to more than 10,000 users of the free plan, while Wordtune has “millions” of users.
Given the costs of training advanced models, there is likely significant investor pressure to develop. Property of AI21 Labs tests pegs the expense of developing a text generation model with 1.5 billion parameters (i.e., the variables the model uses to generate and analyze text) at as much as $1.6 million. Jurassic-1 Jumbo contains 178 billion parameters. This does not take into account hosting costs to host the models; AI21 Labs says it uses “several” third-party cloud service providers both in the U.S. and abroad.
“[There’s a lack] market intelligence because the language model technology is so youthful and just starting to gain traction,” Goshen said. “With this new funding, AI21 Labs will continue its mission to build artificial intelligence systems with unprecedented ability to understand and generate natural language.”