AI Doesn’t Have a Cost Problem. Enterprise AI Architecture Does.
Over the last few months, it’s been hard not to notice a growing narrative that AI is becoming too expensive for enterprise adoption, particularly as more AI providers are switching to usage-based billing to try to start trying to turn a profit.
Usually the logic goes something like this:
“We gave everyone access to the latest frontier model.”
“Sometimes we even tell them their performance review depends on how many tokens they use.”
“They use that frontier model all day without a lot of intentionality.”
“Our AI bill is enormous.”
Then comes the inevitable conclusion: “AI economics don’t work.”
The dominant enterprise adoption strategy has been surprisingly simplistic: Hand every employee a chatbot or AI agent powered by the most capable model available and let them figure out how to use it. Every request — no matter whether it’s summarizing an email, extracting fields from a PDF, writing code, or answering a complex research question — gets routed to the same expensive general-purpose model.
That’s the AI equivalent of running every application in your company on the biggest server money can buy.
AI Is Another Building Block
We learned decades ago that good software architecture doesn’t work that way. Instead, we decompose systems into modular components. We use deterministic software where appropriate. We mix “build” and “buy” systems. We automate repetitive workflows. We choose the cheapest technology that meets the requirements.
AI should be treated exactly the same way.
Most enterprise workflows don’t need frontier intelligence at every step. They need the right model for each task, the right context supplied automatically, and integration into the surrounding business process.
A document classification task almost certainly doesn’t need Anthropic’s most expensive model. A coding assistant should be leveraging traditional programming language tooling instead of making the AI learn the design on its own from scratch. A customer support workflow should invoke AI only where human judgment actually adds value.
When AI becomes another building block in your architecture (alongside cloud, SaaS, your legacy applications, etc.) rather than another application on every employee’s desktop, two things happen:
First, costs fall dramatically because you’re no longer paying frontier-model prices for plain-vanilla commodity work. Simple tasks get routed to smaller or specialized models, deterministic software handles deterministic problems, and expensive reasoning is reserved for the handful of cases where it actually creates value.
Second, the system (and here by “system” I mean the true system including the humans in the loop) becomes more reliable. Users no longer have to craft elaborate prompts or remember which documents to attach. The application supplies the relevant context automatically, enforces business rules, and limits AI to the decisions it’s actually qualified to make. Instead of relying on every employee to become a level-6 prompt engineer, you’ve built a workflow that is naturally more predictable, secure, and governable.
That’s what good software architecture has always been about. AI doesn’t change that principle, It lives alongside and reinforces it.
My Own AI Bill
I can tell you first-hand, that this isn’t just theory. I spend a significant amount of my day using AI, and a surprising amount of it costs me absolutely nothing.
My local stack includes tools like Ollama, LiteLLM, Open WebUI, Onyx, and n8n running on hardware I already own and carry around in my backpack. I use these tools for knowledge management, semantic search across my own content, code assistance, workflow automation, and dozens of small tasks that save me time every single day.
Of course, I also use frontier models. Some of what I do — particularly complex reasoning, coding in real systems, software architecture, and technical research — benefits enormously from the latest models, and sometimes they’re the only practical choice. But that’s exactly the point:
I don’t reach for a frontier model because AI is involved. I reach for one because that particular task requires frontier intelligence.
Everything else gets handled by smaller models, local infrastructure, and/or traditional software. That’s how we’ve always built scalable systems. AI shouldn’t be any different.
Better Architecture Is More Affordable
The organizations that benefit most from AI won’t simply race to deploy the most powerful models. They’ll build systems that automatically route work to the right combination of deterministic software, retrieval, specialized models, frontier models, and humans.
As technical leaders, we shouldn’t be asking, “How do we get everyone using AI?” We should be asking, “Where can we intelligently use AI to optimize our workflows?”
Answer that well, and AI starts looking a lot less expensive. Answer it poorly, and we’ll end up convincing ourselves that AI failed because it’s inherently too costly, when in reality we deployed it with the wrong architecture or (arguably) no architecture at all.
That would be an expensive mistake, not just for individual companies, but for the industry as a whole: Every major technology experiences a backlash after the initial excitement fades. AI is arguably even more susceptible to this than earlier tech. There are already plenty of skeptics waiting for enterprise AI initiatives to fail, whether because they question the economics, distrust the technology, or simply prefer the status quo.
If organizations continue approaching AI as though it’s an autonomous employee instead of another software component, they’ll give those critics exactly the evidence they’re looking for.
The next phase of AI adoption won’t be bigger models or better prompts. It will be better engineering.
