Are language models a commodity?

# Entry

What to do things what do electricity, wheat, cell phones and the Internet have in common? They have probably become what we call them for many commoditythat is, “a resource or good that has full or significant interchangeability (or is simply considered essential to modern lifestyles).”

Most of the examples mentioned became commodities at specific turning points in history. For example, in the delayed 1990s, cell phones were still something of an emerging novelty – at least as far as I remember. However, in this century, who can imagine life without it?

Rapid forward to the present day and look at the wave of language model technology that has swept through our lives over the last 3-4 years, this article examines a set of objective facts. I add a pinch of personal thoughts to the following question: are language models a recent commodity that we can no longer live without?

# Fact review

Just a few years ago, an advanced language model was considered a luxury technology resource, today it has become a ubiquitous solution that many organizations can no longer do without.

There are several facts about the current market reality that explain this wide-ranging availability of language models:

• Falling costs: This may seem counterintuitive in today’s global context of rising prices for almost everything, but one exception to this norm is the cost of “raw intelligence” solutions. One example is the cost of processing one million tokens (about 750,000 words) in frontier models, which cost tens of dollars a few years ago, but can now cost tens of dollars. cents.
• The open-access revolution: open-weight models have helped break the barrier of exclusivity. Language model families such as Meta’s Lama and Mistral have demonstrated in public benchmarks that they can match or even outperform many commercial alternatives.
• Zero cost: Today, any user can download free language modeling tools like Ollama and execute high-performance models locally on their own computer. This completely eliminates the need for paid subscriptions or API cap usage, and also eliminates the dependency on third-party services. As a result, access to AI as a basic and free resource has become the recent standard.

“For example, in the late 1990s, cell phones were still something of an emerging novelty – at least as far as I remember. But in this century, who can imagine life without them?”

# Discovering evidence-based views

Basic AI capabilities may have become a commodity, but having a model with “its own personality” that can handle intricate, detailed tasks cannot yet be viewed that way. Many basic models – which are fundamental from an initial general purpose point of view and which represent the initial phase of large-scale initial training before the model is adapted for specific tasks – can generate text responses or code for free, but there is still a noticeable difference between the natural human narrative and the conversational style of the model, with it still being somewhat robotic and predictable at times in terms of the words and phrases used. Many of the results generated by these models still require final refinement and this is a differentiating factor in various fields.

On the other hand, many users do not invest in the models themselves, but in the experiences associated with them. Having a free text generation engine on your own computer sounds great, but companies have to charge for solutions where this model has been refined to allow you to interact with your documents, code, or workflow in a specific way. We’re still relatively far from the point where everyone would fully accept delegating (often paid) language model solutions to do this task, rather than doing it themselves.

# Issuance of judgment

Based on facts revealing how the role and access to language models have evolved in recent years to become almost casual, it can be argued that these factors have made such models the recent commodity of the decade. However, there are other aspects such as reliability, fully guaranteed privacy and adaptation to niche application areas (such as medical or legal reasoning) that are still not within everyone’s reach because they are still bonus goods and makes the term “goods” somewhat moot in the language model landscape.

Ivan Palomares Carrascosa is a thought leader, writer, speaker and advisor in the fields of Artificial Intelligence, Machine Learning, Deep Learning and LLM. Trains and advises others on the employ of artificial intelligence in the real world.