Agentic workflows can consume 700x more tokens per user query than a chatbot, erasing the savings from falling model prices and pushing some vendors into negative gross margins.
DeepSeek slashed prices on its V4-Pro model by 75%, but for many enterprise AI vendors the cut has not translated into healthier margins — because agent-based systems are consuming tokens far faster than prices are falling [1].
The core issue, which VentureBeat’s sources describe as the “100x problem,” is that a single user-visible request routed through an agentic workflow can cost orders of magnitude more to serve than the same request handled by a simple chatbot or retrieval-augmented generation (RAG) system [1]. RAG is a technique that supplements a model’s response with text retrieved from an external knowledge base.
How token amplification works
In a standard single-turn chatbot, one user message produces roughly one model call, yielding an input-to-billed-token ratio of about 1:5 [1]. In a multi-step agent deployed across customer support, sales, finance, legal, or engineering, that ratio routinely reaches 1:700 or higher, according to the analysis [1].
The reason is structural: every loop iteration in an agent carries forward the full cumulative conversation, tool outputs, and reasoning traces, and each step appends rather than replaces [1]. A query as straightforward as “What did our top customer ask about last week?” can touch seven separately priced operations — including multiple model calls, a retrieval step, and tool execution — before returning an answer, totaling roughly 35,000 input tokens billed [1].
At frontier-model pricing, that single query costs somewhere between $0.10 and $0.40 [1]. At one million queries a month — described as “table-stakes volume for any enterprise B2B feature” — the line item reaches six figures [1].
The seat-based SaaS model under pressure
The dominant enterprise AI pricing structure has been seat-based software-as-a-service (SaaS): a flat per-user monthly fee in exchange for agent capability [1]. That model assumes a reasonably bounded cost per user, an assumption token amplification breaks [1].
A power user running 50 agent invocations a day on a $40-per-seat plan can generate more in inference costs than the plan charges, pushing vendor gross margins negative [1]. Several vendors are now privately reporting exactly that outcome on heavy users, mirroring cloud expenditure data from the Bessemer “Supernova” cohort, where the correlation between AI-agent adoption and gross margin contraction has moved from a theoretical risk to a primary profit-and-loss headwind, according to the report [1].
The dynamic creates a counterintuitive problem: the customers extracting the most value from a product are often the ones generating the highest inference costs, meaning the most engaged users can contribute the least profit [1].
Bryan Catanzaro, VP of Applied Deep Learning at Nvidia, put the pressure in stark terms: “For my team, the cost of compute is far beyond the costs of the employees.” [1]
Bloomberg separately documented a widening gap between Salesforce’s Agentforce marketing demos and the capabilities actually shipping to customers — the kind of gap, the VentureBeat analysis argues, that opens when promised functionality is technically possible but uneconomical to serve at the price a seat plan implies [1].
Technical responses and what enterprises should do
Several mitigation techniques are available, though none eliminates the underlying structural problem [1]. Cost-aware routing — using a small classifier model to direct each query to the appropriate model tier — can cut inference bills by around 60% without quality degradation, according to the analysis [1]. Prompt caching, now offered by Anthropic, OpenAI, and Google at discounts of 75 to 90% on cached prefixes, is another lever [1]. Context discipline — truncating tool outputs, pruning reasoning traces, and capping tool depth — prevents agents from running unbounded loops [1].
IBM research cited in the piece found that “organizations using orchestration-led governance report stronger productivity gains — a holistic orchestration layer is associated with six times greater productivity impact than compliance-only approaches.” [1]
The analysis recommends that enterprise leaders treat inference cost as a first-class metric tracked per feature, per tenant, and per query class; set hard cost-per-thousand-queries ceilings; treat the routing layer as core infrastructure rather than an optional optimization; audit system prompts quarterly; and negotiate volume-commit deals with model providers, since list price is described as “the worst price any enterprise will ever pay” [1].
Frontier inference unit costs are dropping roughly 3x per year and the curve is not slowing, the authors note — but amplification is outrunning those cuts, meaning a 75% price reduction does not help a company whose agents are performing 700x more tokens per query than its pricing model assumed [1].
Sources
This article was drafted with AI from the cited sources and checked against them before publication. Spot an error? Let us know.
