The price of a token keeps falling, and enterprise AI bills keep rising. A single request now fans out into retrieval, tool calls, reasoning loops, and multi-step execution, so the number of tokens per task climbs faster than the price per token drops.
On a recent All-In episode, @Chamath Palihapitiya described token costs "doubling every 45 days" against a productivity lift of "5%, max." In the same episode, @DavidSacks said enterprises want cheaper and more diverse models. DoorDash has described reserving a frontier model for its hardest code-review work while routing lower-level tasks to cheaper ones, and Decagon reportedly sends 90% of mature customer-support work to open models. It echoes what Palo Alto Networks CEO @NikeshArora told CNBCrecently: AI pricing has to fall as much as 90% before the technology is genuinely useful across the enterprise.
This gap is not a token pricing problem. It's an architecture problem.
Organizations should stop measuring cost per token and start measuring cost per successful task. A cheap model that produces rework is expensive. A capable model used at exactly the right step is efficient. The right question is where expensive intelligence earns its price. Answering that question well is an engineering discipline, and it’s where competitive advantage is emerging.
We started building for this at @Glean seven years ago, before the market agreed it mattered. Three parts of the architecture matter most.
Context comes first
A model on its own does not understand your company. It does not know which documents are current, which projects are active, who owns what, or what any given employee is allowed to see. In most enterprises that knowledge is scattered across dozens of systems. If you can’t assemble it in a permission-aware way, the model reasons from partial information and burns tokens rediscovering your business on every call.
In abenchmark we ran in Claude Cowork, we held the model constant and swapped only the context layer, comparing Glean's index against off-the-shelf MCP servers. Glean-powered responses were preferred about 2.5 times as often, and the off-the-shelf tools used about 30% more tokens to do the same work. In the cases where those tools did reach a correct answer, they burned nearly double the tokens to get there because they had to brute-force the search the context layer should have handled. The advantage grew as tasks got harder, with Glean's context layer win rate rising from 66% on simple work to 73% on complex, multi-step work.
Better context raises quality and lowers cost at the same time.
Control, and real model choice
Enterprises cannot standardize on one model family. Some tasks need a frontier model. Many do not, and can be handled well by faster or cheaper ones. But few companies have the capacity to continuously evaluate models, manage access across providers, design fallback logic, and watch quality, cost, and latency as the market shifts every week.
What’s underappreciated is that the open models are no longer the cheap, weaker option. On specific high-volume work like coding and agents, they are now matching or beating the frontier at a fraction of the cost. GLM-5.2 has landed within about a point of the best closed models on long-horizon coding while running at roughlya sixth of the cost, and Kimi K3 arrived as the largest open-weight model yet, debuting at number one on a leading coding benchmark ahead of the best systems from Anthropic and OpenAI. These are frontier-class systems that happen to be open, not budget compromises. Glean co-founder @TonyGentilcore has written more about why enterprises shouldstop treating that as off-limits.
Through Glean'sModel Hub, customers reach 30-plus commercial and open models through one governed layer, choosing which model runs by user, department, or by application, with access controls and audit applied consistently across every one. The model layer can change underneath you without forcing a rebuild of the stack above it. That is what turns a fast-moving, commoditizing model market from a risk into leverage.
Economics follows
Chamath's point about cost outrunning value is the one every CFO will eventually force the organization to confront. The answer is not rationing access. It is routing work.Waldo, our specialized agentic search model built on open NVIDIA Nemotron, runs ahead of the frontier model on every query to plan the search and hand off clean context, cutting token use by about 25% and latency by about half at the same quality. The frontier model then does only the part it is uniquely good at. Token efficiency, not token count, is the metric that matters.
Where this leaves us
Few companies have been building this layer as long, or measuring it as rigorously, as we have. The payoff for the enterprise is not access to the latest model. It is strategic choice: which models to use, where AI shows up across the business, and how to trade off intelligence, speed, and cost, with the freedom to adapt as the market changes. The models will keep leapfrogging each other. The architecture around them is what becomes the moat.
