What Actually Goes Inside a Production AI Agent

By Punta del Este Labs · 9 min readThere is a moment, early in every AI project, where someone wires a language model to a couple of API calls, types a question, and watches it do something genuinely impressive. That moment is real, and it is also a trap — because the distance between that demo and something you can put in front of users is most of the actual work. A demo answers once and hopes. An agent runs, checks itself, recovers, and knows when to stop.

The whole difference in one picture. A demo is a straight line that ends in hope; an agent is a governed loop that ends in a checked result.

We have shipped enough of these to have opinions about what has to be there. What follows is the anatomy — the five systems we build around a model before we would call the result an agent. It is the same skeleton behind Hermes, the runtime we use so we stop rebuilding it every time.

An agent is a loop, not a prompt

The single idea that separates a real agent from a chatbot is the loop. Instead of asking the model for the answer, you ask it for the next step. It proposes an action, your code runs that action, you feed the result back, and you ask again — until the goal is met or a budget says stop. The model never touches the world directly; it only ever proposes, and your loop decides.

Plan, act, observe, decide. The loop is what lets an agent recover from a bad step instead of confidently shipping it — and the “done / budget” exit is what stops it running forever.

This is also where most naïve agents fail. If the loop has no way to notice a tool returned an error, it barrels ahead on a false assumption. If it has no budget, a small misunderstanding becomes an expensive infinite loop. The loop is not glamorous, but it is the control structure everything else hangs off.

Tools: the only way it touches the world

An agent is only as capable as the tools you give it, and only as safe as the way you define them. Every capability — read a record, send an email, charge a card — is a typed, validated function with a schema the model has to fill in correctly. The model’s output is never trusted raw; it is parsed against that schema, and a malformed call is rejected and retried rather than executed on a guess.The important tools are also idempotent and scoped. Idempotent, so that a retried step doesn’t send the same email twice. Scoped, so that the tool which reads invoices physically cannot delete them. The permissions live in the tool, not in the prompt — because a prompt is a suggestion, and a permission check is not.

Memory: what survives between steps

A model has no memory of its own; it sees only what you put in front of it. So an agent needs two kinds. Working memory is the running context of the current task — what it has tried, what came back — kept small on purpose, because a bloated context is both slower and dumber. Long-term memory is durable recall across runs, usually retrieval over your data, so the agent can pull in the one relevant document instead of being handed all of them. Deciding what the model is allowed and able to see is half of building a good agent.

Guardrails: the difference between can and may

Autonomy is only useful when it is bounded. Guardrails are the boundary: budgets on tokens, steps and money so a run cannot spiral; timeouts so it cannot hang; approval gates where a human signs off before anything irreversible; and reversible operations so the worst realistic outcome is an undo, not an incident. There is also the adversarial side — a document the agent reads can try to hijack it, so untrusted input is treated as data, never as instructions.

Evals: how you know it still works tomorrow

This is the system teams skip, and the one that decides whether the project survives contact with a second month. An agent has no compiler telling you a change broke something; a reworded prompt can quietly wreck a case that used to pass. So quality has to become a number. We build a golden set of real cases with known-good outcomes, score new versions against it — often with a model acting as judge on a rubric — and gate every change on it in CI. Without evals, you are not improving an agent so much as disturbing it and hoping.

The five systems, together

None of these is exotic on its own. What makes an agent production-grade is that all five are present and wired together: context feeding the loop, tools letting it act, guardrails bounding it, evals judging it. Take any one away and you are back to a demo — impressive in the room, unpredictable everywhere else.

The anatomy. The loop is the engine; context, tools, guardrails and evals are the four systems that make it safe to let run.

This is exactly the scaffolding Hermes gives us for free, so a project starts at the interesting problem — your workflow, your data, your definition of a good outcome — instead of at the plumbing every agent needs and no demo bothers with.

Have a workflow where an agent would earn its keep? We are good at telling you which ones will and which ones won’t.

What actually goes inside a production AI agent