Most AI builds we are asked to harden look great in a demo and fall over the first weekend in production. The difference is not the model, the framework, or the prompt. The difference is the dozen unglamorous things the demo did not need and the production system absolutely does. Here is the list we use when we take over a vibe-coded build and ship it.
Auth and authorisation, not auth alone
A logged-in user is not automatically allowed to do everything the system can do. Every AI endpoint, every tool call, every retrieval needs an explicit authorisation policy keyed to the user's role and tenant. The check belongs on the server, in the request handler, before the model is invoked.
Rate limiting on the expensive endpoints
Per-user token bucket, per-IP daily cap, per-tenant monthly cap. If your LLM endpoint can be called without an authenticated user, add Turnstile or equivalent. The first time you ship without this, you will learn about it via your provider bill.
Structured error handling and graceful degradation
The model will return malformed JSON. The provider will return 429s, 500s, and the occasional silent truncation. Every model call needs a timeout, a retry policy with jitter, and a graceful fallback — either to a smaller model, a cached response, or a clear user-facing error. "Try again" is not a fallback.
Observability that is not just logs
Per-call: model, prompt hash, input token count, output token count, latency, cost, tool calls invoked, retrieval IDs returned. Per-user and per-tenant aggregates of the same. Alerting on latency p95, error rate, cost per session, and unusual tool call patterns.
Without these you cannot debug a regression. You cannot prove SLA. You cannot defend a security incident.
Secrets management
Never in client bundles, never in repo, never in unencrypted env files. Use a secrets manager, scope keys per service, rotate on a schedule. Have a script that audits the codebase for the prefixes of every secret you own.
Human-in-the-loop escalation paths
AI will be wrong. The product needs to know how to hand off — to a human agent, to a deterministic fallback, to a "we are not sure, please confirm" experience. The escalation path is part of the product, not a bug-fix lane.
Evaluation that runs on every deploy
A small, growing set of canonical inputs that the model has to handle correctly before you ship. Run it in CI. Block deploys when it regresses. The set grows every time something breaks in production — every incident contributes a test.
The line between a demo and production
A demo proves the idea. A production system survives a Saturday with no operator on call, a malicious user trying to exfiltrate data, a provider outage, and a customer who is angry that the answer was wrong. Everything on this list exists to handle those four cases.
If you have a working prototype and need to cross this line without rebuilding from scratch, that is the work our development practice does — we take prompt-generated code and harden it to the bar above, then hand it back with the runbook to operate it.