The AI trust stack: observability, guardrails and safety for production LLM apps

Every production outage we have responded to in the last twelve months that involved an LLM had one thing in common: the team could not answer "what did the model see, what did it say, and why?" within fifteen minutes. That is the problem the AI trust stack solves.

Why "just log it" is not enough

Traditional observability measures binary questions: did the request succeed, how long did it take, what was the status code. LLM systems ask a harder question: was the answer good? And "good" is a distribution, not a threshold.

You cannot A/B test your way out of an AI incident if you cannot reconstruct, after the fact, exactly what was in the prompt.

The five layers of the trust stack

1. Tracing

For every user-facing call, capture:

• The full prompt (system + user + any retrieved context)
• Every tool call, input, output and latency
• The final model output, token count and cost
• The user action after the response (accepted, edited, rejected, ignored)

This is expensive storage-wise and non-negotiable operationally. Sample aggressively for volume, but never skip structured fields.

2. Evals

Two flavours, both required:

• Offline evals — golden sets, adversarial sets, and regression suites you run before every prompt or model change.
• Online evals — cheap heuristics (length, toxicity, JSON validity) running on every production request, flagging outliers.

A team without a golden set is not running an AI product. They are running a prayer.

3. Guardrails

Policy-as-code that sits between the user and the model and between the model and the outside world:

• Input — prompt injection detection, PII redaction, scope enforcement.
• Output — toxicity, hallucination checks for factual claims, brand/tone filters.
• Action — what the model is allowed to do via tools, under what conditions.

Guardrails are not a third-party SaaS you bolt on at the end. They are a product surface. Invest accordingly.

4. Human-in-the-loop

Not "every response goes to a human" — that is a scaling disaster. But for high-risk actions, explicit approval, with the full trace surfaced to the reviewer. The goal is to make human review easy and fast, not ceremonial.

5. Rollback

Prompts, models, tools and guardrails should all be versioned and flag-controlled. If a new prompt is degrading quality, you should be able to roll back in seconds, not hours. Treat prompts like deployable artefacts, because they are.

A real incident, de-identified

A client shipped a new system prompt that scored +3% on their golden set. Two days later, enterprise customers began complaining that the assistant's answers were "oddly flippant." Our trace logs showed the new prompt had accidentally introduced a casual register that the golden set (which skewed technical) did not penalise. We rolled back the prompt in 90 seconds. Without versioning and tracing, this would have been a three-day investigation.

Compliance is a byproduct, not a goal

If you build the trust stack well, SOC 2, ISO 27001, GDPR and the EU AI Act become mostly evidence-gathering exercises rather than engineering work. That is the right order to do these things in. Build the trust stack because you are a serious engineering team, and compliance falls out naturally.

What we recommend starting tomorrow

1. Pick one production LLM feature. Instrument every call end-to-end.
2. Extract a 50-item golden set from real user logs. Automate running it on every change.
3. Add one guardrail — the one that would have prevented your most recent bad output.
4. Version your prompts in a separate file, shippable independent of code.
5. Set a cost and latency budget per request, and alert on deviation.

TL;DR

• Production LLM apps need a dedicated trust stack: tracing, evals, guardrails, HITL, rollback.
• If you cannot reconstruct a single bad response in under 15 minutes, you are flying blind.
• Compliance comes for free when the engineering is right. The reverse is not true.