For most of the software era, the cost of serving one more customer was close to nothing. You wrote the product once, and the marginal cost of the next signup was a little storage and a little bandwidth. That is the reason SaaS businesses have carried gross margins above 80%. The arithmetic held for a long time.
Products built on large models do not work that way. When your product calls a model, every request costs money, and the cost rises with how much text goes in and comes back. The bill is no longer a fixed platform expense. It is a variable input that scales with usage, which is the textbook definition of a cost of goods sold.
Why agents changed the shape of the bill
A single question to a model is cheap. The change came when the unit of work grew. Agents now run for many steps to finish a task. Coding tools loop over a repository. Assistants hold conversations that carry on for hours. The work is a session, not a message, and a session sends far more tokens than a one-off answer ever did.
The mechanism is worth being precise about. At each step, the model has no memory of the last one, so the whole context goes back in: the system prompt, the running history, the tool definitions, the documents that were retrieved. A task that takes forty steps pays for that context forty times, and the context grows as the task goes on. Research on long agent sessions finds that as much as 90% of what gets sent can be context that was already sent earlier in the same session. You are paying, over and over, to resend text the model has effectively already seen.
What the numbers look like
This is not a rounding error on the income statement. Enterprise spend on model APIs reached $8.4B in 2025, up 2.4 times in a single year (Menlo Ventures). Industry analysts expect the inference market to grow into roughly $105B by 2030. At the company level the pressure shows up in margin: AI-native software is running at 30 to 50% gross margin, against the 80%-plus that traditional SaaS has enjoyed (ICONIQ and public reporting, 2026). In one reported period a category leader was spending $1.23 of inference cost for every dollar of revenue it brought in (public reporting, 2026).
When a cost grows faster than the revenue it supports, it sets the ceiling on the business. A team can win more customers and still watch margin slip, because each new customer brings more sessions, and each session brings more resent context. The pricing page holds steady while the cost underneath it climbs.
What a company can do about it
The useful part is that a cost of goods can be managed like one. Manufacturing businesses have spent a century learning to hold a unit cost inside a target while keeping the product good enough to sell. The same discipline applies here, and it starts with looking at the whole session rather than the single call.
Most spend goes through one default model for everything, which means the hardest step and the most routine step are billed at the same rate. Looking across the session lets you match each step to a model that can handle it, and send the routine steps somewhere cheaper without the result getting worse. Providers already cache repeated context at a fraction of the price, so the same work does not have to be paid for at full rate every time.
Two controls turn that into something you can rely on. The first is a budget: a ceiling on what a session, or a day, is allowed to cost, fixed before the work runs rather than read off an invoice at the end of the month. The second is a quality floor: a standard the routing is not allowed to drop below, so cost never comes at the expense of the answer. Set both, route the whole session inside them, and the bill stops being a number you discover and becomes one you decide.
That is the work we do at Moji Router: most of the bill is context sent more than once, and routing the whole session inside a budget and a quality floor is how we bring it down.