Artificial Intelligence—known in English as Artificial Intelligence—is currently the most talked-about topic in technology. Not only scientists and technologists, but its usage has now reached our homes. In fields like medicine, education, banking, or insurance—you name it, AI is being used everywhere. However, the biggest barrier to its widespread adoption is cost. While some companies offer free usage, buying any paid or professional package can be surprisingly expensive. In many cases, this high price is the main reason people aren’t opting for the paid packages. The good news is that the cost of using AI is rapidly falling over time. As a result, it will now be possible to build various businesses and applications using this technology. Recently, OpenAI set the price of its GPT-4o model at only $4 per million tokens, whereas in March 2023, the GPT-4 model cost $36 per million tokens. This is nearly a 79% reduction in cost over about 17 months. Furthermore, using a new Batch API, where results are available within 24 hours, the cost can go down to as low as $2, making it even more affordable.
In an AI system, a "token" is a basic unit or element used for storing and processing various types of information. Generally, tokens often represent words, letters, or symbols within text. For example, in a language model, “Bangladesh is my country” would count each word or symbol as a token, meaning there are 3 tokens here. Simply put, a token is a fundamental part of information that helps an AI system understand and process data. It’s important to remember that the size and type of a token can vary: it could be a single word, part of a word, or even a whole sentence.
Why Are Token Prices Dropping?
One of the main reasons behind the drop in token prices is the arrival of new and open-source models, like Llama 3.1. Startups such as Anyscale, Fireworks, Together.ai, and other large organizations—who don’t have to pay to develop their own models—are now able to compete on price and several other fronts. This increased competition is driving token prices down.
Additionally, innovation in hardware is also playing a part in lowering prices. Companies like Groq are leading with rapid token generation, and Samba Nova is producing 114 tokens per second. Corporations like Cerebras and semiconductor giants such as NVIDIA, AMD, Intel, and Qualcomm are providing advanced hardware, which will help reduce costs even further in the future.
The Future
Technology is changing rapidly, so when building applications, it’s important to look not just at current trends but also toward the future. With the roadmaps for advanced semiconductors, smaller models, and algorithmic innovations, it’s easy to predict that token costs will continue to decrease.
As a result of these changes, even if running an application today seems a little expensive, it may become quite affordable in the future. For example, if you create an app that uses 100 tokens per second, at current rates, the hourly cost is only $1.44, which is much less than the minimum wage in the United States.
How Can You Prepare to Build Products?
When developing a product or application using artificial intelligence, how can organizations prepare themselves for this new landscape? Here are a few suggestions:
1. Gain a Clear Understanding of Costs: Many people find that, when launching their projects, the actual cost of using LLMs is much lower than expected. Therefore, it’s better to focus on creating simple and effective applications instead of overcomplicating things in an effort to reduce spending.
2. Prepare in Advance: If an application seems a bit expensive at today’s prices, it might still be wise to launch it, considering that costs will likely drop in the future.
3. Make Room for Model Switching: When new models launch, it might be possible to switch from an older model to a newer one, such as moving from GPT-4 to the new GPT-4o-2024-08-06. Additionally, switching between different model providers could be an option, especially since open-source models are becoming available from various suppliers.
Conclusion
The rapid reduction in token prices is making it much easier and more affordable to build and deploy AI applications. This progress in artificial intelligence is opening new doors of possibility, and now it is crucial for AI-focused organizations to plan their strategies with the future technology roadmap in mind.
Leave a comment