·4 min read
AI app architecture cheat sheet
The handful of components every production AI app ends up with, and how they fit together.
architectureAI
The shape most apps converge on
- An API layer that takes user input and enforces auth
- A model gateway that handles provider routing, retries, and cost limits
- A retrieval layer (vector + structured) for grounding
- A queue for anything longer than a request cycle
- An observability layer logging prompts, tool calls, and outcomes
What changes per project
The tools. The shape stays remarkably similar across an internal assistant, a customer-facing chatbot, and a back-office automation.
What we optimise for
Boring days in production. Fast iteration on prompts and tools. Cost you can predict a month out. None of that is glamorous, all of it is the difference between a demo and a product.
