The hidden cost of complexity: your AI bill

AI coding assistants have changed how many of us work. For a flat monthly fee in the range of $20 to $50, you get something that reasons about code, explains unfamiliar systems, and helps debug problems that would otherwise take hours or days. Most developers I know consider this money well spent.

But this is about to change. If you have been reading reports that in some tech companies, AI tooling costs are beginning to exceed salary costs for human developers. That number will surprise some people. It shouldn't. The pricing we've enjoyed is adoption or "beta test" pricing, not sustainable pricing. The prices will go up, and I think that will happen sooner rather than later.

When the prices start going up, engineering teams will start asking the same question they ask about cloud spend: where is this going, and what are we getting for the money?

What you're actually paying for

AI models work with context. The more context a model needs to reason about your software development, debugging or architecture problem, the more tokens are consumed, and the higher is the cost. This is straightforward in principle. What is less obvious is how much of that context has nothing to do with your actual problem.

Consider a typical debugging session. Before the model can reason about the bug, it needs to understand the framework you are using, the abstraction layers around the bug, why your code is structured the way it is, and what the surrounding conventions mean. A significant fraction of every AI interaction in a complex codebase is spent establishing context that exists because of architectural choices made long before the problem appeared. All that turns easily to millions and millions of tokens.

The bug itself might be in twenty lines of code. The context required to reason about it might be tens of thousands.

Complexity is a tax, and AI makes it visible

Despite the classic "KISS" principle, as an industry we've grown accustomed to complex software stacks. There are framework over frameworks, abstraction on abstraction, and then there's the infrastructure on which all this runs. Each layer is justified locally but rarely questioned in aggregate. The operational costs of this complexity, like slower onboarding, more frustrating debugging, fragile deployments, are all real "benefits". They're spread across enough people and enough time that they're easy to normalize.

AI token costs make complexity expensive in a different way: immediately, measurably, on an monthly invoice.

A simpler system requires less context to reason about. Less context means fewer tokens. Fewer tokens means lower cost: for every interaction, every day, for the lifetime of the system. The architectural decisions made at the start have a compounding effect on AI costs that won't appear in any initial estimate. AI isn't going to go away, so the costs will stay with you and the project.

The model size question

There's a related point worth considering. The largest, most capable AI models are currently cloud-only. They have hundreds of billions of parameters. Running them on-premise isn't realistic for most organizations. That's fine when the pricing is comfortable, as it currently is. But when pricing corrects to match costs and profit targets, the on-premise question becomes more interesting.

Smaller models, in the range that can run on a capable workstation, are already surprisingly good at small, coherent systems. The constraint in these systems is not the model's capability for regular development tasks. It's the signal-to-noise ratio of what you give the model to work with.

If your entire application stack (everything the model needs to understand to help you effectively) fits in a context window, a smaller on-premise model may serve you better than a larger cloud model working with a noisier, more complex codebase. The architectural choice and the AI sovereignty question are connected in a way that isn't obvious until you think it through.

If your software were simpler, you might get away with a smaller model. If you can use a smaller model, you could run it yourself. If you ran it yourself, you'd be immune to whatever pricing comes next.

This isn't an argument against AI tools

I'm not at all against AI or cloud-based models. I use them daily and find them superbly useful. My argument is for thinking carefully about what you're feeding those models, and what the long-term cost structure looks like.

The next time someone proposes adding a framework, a layer, or a service to a system, the question isn't only "what does this cost to build and operate?" There's a third cost now: what does this add to every AI interaction for the lifecycle of this system?

That cost is real. It's just not on the invoice yet.