The AI cycle and it's token usage market
The way generative AI is totally different than how it should be. Mostly, one of the bigger problems is the gap between understanding value by output and actually using tokens in production, or using more tokens. They measure it by the amount of tokens spent, not by the total value it creates in the end.
This is called “Goodhart’s Law,” which means that when measures become the target, they cease to be good measures.
The actual quality to users remains unsatisfied, as a report by MIT says that many of these companies themselves say that 95% of these generative AI deployments show zero return because the market is highly subsidized and pricing does not reflect true costs.
Standard transformer attention has quadratic time and memory complexity with respect to sequence length. Doubling the context length roughly quadruples the amount of attention computation and the memory required for the attention matrix.
Many of these systems use “thinking” and high reasoning tokens that users cannot see but still pay for. These can grow from 33% to 60% of a task’s total tokens in complex workflows. Sometimes it also prepends context of conversation history or irrelevant data to every request, creating “structural waste”.
There is a “circularity problem” in which tech giants invest in AI startups that then spend that same money on the investors’ own cloud services. This boom is increasingly financed by debt rather than cash flow, and it’s creating systemic risk if revenue growth fails to materialize.
Enterprises are having problems in facing financial volatility because token-based AI costs are nonlinear and unpredictable. If you have an unconstrained agentic loop, it can continue consuming tokens again and again while appearing normal until the first sign of failure is a massive invoice bill. Uber burned through its entire 2026 coding budget in just four months.
These kinds of things become “Day 2” problems that become massive cost overruns and inconsistent performance, with 72% of teams reporting that running agents actually costs more than building them.
The reason these Agentic AI systems consume more tokens is because it work rather than normal chat-based AI. They operate through complex, multi-step workflows rather than single-pass interactions. Rather than giving one answer per query it’s goes through many layers of planning and a long sequence of actions to do certain works.
Many agentic architectures use a “naïve harness” that simply prepends the entire conversation history, execution traces, and retrieved data to every new request in the loop. This creates “context bloat,” where the system carries forward thousands of tokens of state and intermediate outputs at every turn.
The problem with absolute institutional purity and the total elimination of error or “informality” is actively destructive. By suppressing all deviation from prescribed behavior, systems destroy their own adaptive variance, the capacity to learn, evolve, and correct mistakes in a changing world.
AI proponents often frame algorithms as objective, but they actually extract and amplify human prejudices, shortcuts, and biases from their training data. Because these patterns operate at machine speed, they make isolated human errors into systematic “algorithmic oppression” that appears neutral and inevitable.
Large Language Models (LLMs) are probabilistic, not deterministic; they are trained to predict the “next token,” not to tell the truth. The input text is first tokenized into subword units and converted into a numerical vector using embeddings, and it includes positional encodings to track sequence order.
This has problems with “hallucinations”: plausible sounding but factually incorrect or nonsensical outputs. In medical settings, these outputs are conveyed with high confidence and can lead to dangerous misinformation regarding diagnoses or treatments. This happens a lot of the time with AI: you ask a question, and then it gives wrong information with full confidence.
AI systems often inherit and amplify the prejudices found in their training data. For example, Amazon’s recruiting AI learned to penalize resumes containing the word “women’s” because it was trained on a decade of predominantly male hiring data.
The continuous process cycle of building larger and more capable models, where each new release surpasses the last one, has created a high-stakes “Capex arms race” driven by the fear of missing out on a potential monopoly by these AI companies.
With these models scaling and growing, with training compute doubling every week and months, and power use doubling, the performance gap between models annually is actually shrinking.
The difference in benchmark scores between the top-ranked model and the 10th-ranked model fell from 11.9% to just 5.4% in a single year, which makes the frontier becoming “increasingly competitive and increasingly crowded”. This means that all the AI companies should spend exponentially more on compute to achieve only marginal gains over their predecessors.
There is a “Catch-22” thing where the more great and “better” AI becomes, the less users may need to spend on it to produce the same result. If token prices continue to drop faster than usage increases, providers may never achieve the monopoly-level profits they are chasing.
But there is also this theoretical problem: when improved AI becomes great, and falling token prices theoretically reduce the cost per result, historical data shows that aggregate spending doubles as unit costs drop, because cheaper intelligence allows entirely new workflows and autonomous agents that consume exponentially more tokens.
The “low-hanging fruit” of training data (freely available text like Wikipedia) has been exhaustively utilized. More progress now depends on high-quality, specialized data that is operationally complex and costly to produce, because it requires experts paid hundreds of dollars an hour to curate training sets. This creates a constraint from the quantity of data to the quality and cost of producing it.
As models become more powerful in “leaps, not steps,” main players’ labs face a “Cinderella Glass Slipper” effect. Users only stay with a model if it perfectly fits a high value workload (the “glass slipper”). Because technical progress is fast and a model can be rendered obsolete almost immediately upon the release of a slightly better alternative, making sure companies from building the long-term customer loyalty or “moats” required for profitability.
Many predict an “intelligence explosion” or infinite exponential growth; most technologies follow an S-shaped curve, where progress naturally slows as it approaches the physical and economic limits of what the technology can do.
Training and running these models requires a staggering amount of resources that mostly remain “under the radar”. For instance, training GPT-3 consumed roughly 700,000 liters of clean freshwater for cooling, and global AI demand is expected to require as much water withdrawal as half of the United Kingdom by 2027.
AI in software development, particularly full-stack, creates pressure on developer roles, reflecting how these systems were built and where the “low-hanging fruit” of data lies. With the majority of AI companies promising a future of medical breakthroughs, the reality on the ground is shaped by economic incentives and technical constraints.
Large language models (LLMs) are trained on massive open-source knowledge repositories, with GitHub being a primary source. This provides a huge, structured, and high-quality dataset that doesn’t exist in the same way for proprietary cloud architectures or sensitive medical records.
Programming follows rigid logical rules that current transformer architectures are great at predicting. This creates a high demand and high consumption of coding-related AI usage; programming queries on platforms like OpenRouter grew from 11% to exceeding 50% of total volume in just one year.
AI is making it a high place in allowing individual developers to work as “full stack” practitioners because it can work with the “boilerplate” and specialized syntax of both front-end and back-end work.
Cloud computing is currently being overwhelmed by the “AI token crisis”. Organizations are finding that “Day 2” operational issues like unexpected $500 million bills are now more common than actual productivity gains, which leads to a pullback in some cloud-based AI work.
Research labs like Anthropic need agentic workflows to succeed because if they cannot sustainably sell this vision to other companies, their own company will collapse. Management usually orders teams to promote and adopt AI “at all costs” because they fear that if they don’t force adoption now, they will lack the infrastructure and expertise when the inevitable industry-wide shift eventually happens.
These companies market their AI and say “AI will “take over mundane, repetitive tasks” so humans can focus on “big picture, creative thinking”. But data from the retail industry shows the reverse: 88% of organizations use generative AI for “creative ideation/concepting” and 74% for content creation, with only 10% to 23% use it for mundane tasks like generating variations by channel or geography.
They say that AI will or is getting cheaper to run, with per-token pricing for some tasks decreasing dozens of times over time. But as tokens become cheaper, usage rises so exponentially that total costs often go up, a phenomenon known as the Jevons Paradox. It is an economic phenomenon said to occur when Technological change that increases the efficiency of a resource’s use leads to a rise, rather than a fall, in total consumption of that resource.
Algorithms are object and it work as a clean alternative than normal biased human decision making. But in reality, AI extracts and amplifies patterns of prejudice found in its training data. This results in “algorithmic oppression,” where systems like Amazon’s recruiting AI learned to penalize resumes containing the word “women’s” because it was trained on a decade of predominantly male hiring decisions.
There is a huge difference between true open-source and open-weights models. Companies release “open” weights while keeping the training data and pipelines secret. This “open-washing” makes models appear transparent while they remain opaque to independent audits.
This creates a problem in understanding the “black box”. It refers to AI systems, particularly deep neural networks, whose internal workings are opaque, unobservable, and unexplainable even to their designers.
Because these models are founded on complex statistical correlations rather than direct cause-and-effect relationships, it is often impossible to determine exactly why a specific decision was reached. This lack of transparency is particularly dangerous in high-stakes fields like medicine, finance, and justice, as it can hide algorithmic bias and lead to a diffusion of responsibility where harmful outcomes are blamed on the algorithm rather than a human actor.
The field of Explainable AI (XAI) works here to supplement machine learning with techniques that provide insight into algorithmic correlations. Specific technical methods include:
SHAP and LIME: These tools help visualize the contribution of individual features to an output or approximate complex models with simpler, interpretable ones.
LLM Self-Explanations: Large Language Models can be prompted to provide qualitative, human-like explanations for their logic, which can bridge the gap in understandability for businesses and researchers.
Feature Activation Mapping: Techniques like dictionary learning can associate patterns of neuron activations with human-understandable concepts.
The Explanation Gap: Large Language Models (LLMs) can be prompted to provide “self-explanations”; these qualitative justifications do not necessarily decode the model’s underlying billions of parameters, and it’s creating a “veneer of reasoning” rather than true transparency.
Even though matching human performance on PhD-level questions, the top models still struggle with basic logic tasks, such as correctly reading an analog clock, which they fail roughly 50% of the time.
Architectural Diversification is when Transformer models remain dominant; the market is exploring alternatives like Mamba (state space models) and hybrid architectures that work with linear scaling for long contexts, which are more computationally efficient than traditional quadratic-scaling Transformers.
Governments are showing increased urgency in passing AI-related laws (a ninefold increase since 2016), with a rising focus on AI sovereignty, developing domestic supercomputing and model ecosystems to avoid dependence on foreign Big Tech.
A new market is forming around “monetization infrastructure”: unified systems that track model load and usage in real time to connect AI product behavior directly with financial outcomes for better understanding the costs.
 | A guest post by
|
 | A guest post by
|