Most AI features fail in production not because the model produces wrong outputs — they fail because nobody specified what a wrong output looks like at the service boundary. The notebook works. The demo works. The first week in production works. Then a prompt change shifts the output distribution, a model update changes a behavior that was never documented, a budget overrun appears in the billing report three weeks after it started. None of these are packaging failures. They are contracts failures: the service was deployed without specifying what it guarantees, what it costs, and what happens when it degrades.
Notebook-to-service is harder for AI features than for deterministic services because the failure modes are probabilistic and the success criteria are often implicit. A traditional service either returns a 200 or it doesn't. An AI service can return a 200 with a response that is confidently wrong, semantically degraded from last week, or correct for ninety percent of inputs and reliably wrong for a specific user cohort. The monitoring surface that catches these failures is different from the monitoring surface that catches infrastructure failures — and most teams build only the latter.
The research question is not whether AI features can be deployed reliably. They can. The question is what the minimum operational surface looks like — the smallest set of contracts, traces, gates, and procedures that makes a feature genuinely ownable rather than merely running. The production readiness framework at FRM/04 defines the readiness dimensions. This page investigates what each dimension requires to be operational rather than aspirational.
Traditional software deployment has a mature operational playbook. The question is which parts of that playbook apply directly to AI-backed services, which require adaptation because the failure modes are different, and which produce false confidence because the underlying mechanism does not transfer.
| Concept | Transfers? | Adaptation Required | Confidence |
|---|---|---|---|
| Canary deployment | Yes | Applies cleanly as a traffic-split mechanism. The interpretation of canary metrics is harder — a 5% error rate increase is unambiguous in a deterministic service; a 5% quality degradation in an LLM-backed feature requires a defined quality metric to detect. Canary deployment requires eval infrastructure to be useful, not just traffic routing. | High |
| Feature flag rollout | Yes | Feature flags work. The AI-specific consideration is that a flag controlling a model-backed feature may have observable side effects that differ by user cohort in ways that are hard to predict — a prompt that works for one user population may degrade for another due to input distribution differences. Flag rollouts need per-cohort quality monitoring, not just error rate monitoring. | High |
| SLA contracts at service boundary | Partially | Latency and availability SLAs transfer directly. Quality SLAs require defining what quality means at the boundary — which requires the same scored evaluation infrastructure as the eval pipeline. Most AI services ship without a quality SLA because the measurement is hard. The absence of a quality SLA means the service has no contract on the property that matters most. | Medium |
| Rollback semantics | Partially | Code rollback is binary and deterministic. Model version rollback changes the output distribution of every call — a rollback that improves one class of inputs may degrade another. Prompt version rollback has the same property. Whether a rollback is an improvement requires the eval infrastructure to verify. Rollback without eval is a guess. | Medium |
| Distributed tracing | Yes | Trace spans for LLM calls require additional fields: model version, prompt hash, token counts, latency by stage (prefill vs. decode), and the full input/output payload if cost permits. Standard distributed tracing instrumentation captures latency and errors; it does not capture the semantic content required to diagnose quality failures. | High |
| Budget enforcement | No | Token budget enforcement has no direct analog in traditional service deployment. A service that consumes more CPU than expected triggers an autoscaling event. A service that consumes more tokens than expected consumes more money — silently, until the billing cycle. Budget contracts must be defined per call, per user, and per time window, enforced before the model call fires rather than detected after. | Medium |
| Schema validation at service boundary | Yes | Input schema validation is standard practice and applies directly. Output schema validation is the AI-specific addition — LLM outputs must be validated against the expected structure before they are passed to downstream services. An invalid output that passes the service boundary is an incident waiting to happen at the consumer. | High |
| On-call ownership model | Contested | Traditional on-call: someone owns the service, gets paged when it errors, and can fix it. AI on-call: the service may produce wrong outputs that do not error — no page fires, but quality has degraded. On-call for AI services requires quality alerting, not just error alerting. Whether the on-call responder can fix a quality regression during an incident — by adjusting a prompt, switching a model, or rolling back — without breaking other behavior is not established. | Low |
State of Understanding — RES/DEP · Deployment Primitive Transfer
Ownership of an AI feature in production is concentric. The innermost layer is the model call itself — a function that accepts a prompt and returns a completion. Owning just this layer means the feature runs. It does not mean the feature is ownable under operational conditions. Each outer layer adds a requirement that must be satisfied before the layer beyond it can be meaningfully operated.
The diagram below shows the minimum envelope. The model call at the center cannot be owned without a prompt contract that defines valid inputs and expected output structure. The prompt contract cannot be operated without a trace surface that makes what happened observable. Traces are not actionable without a budget gate that prevents runaway cost from degrading the system. Budgets require an SLA boundary that defines what the service guarantees. And nothing in the envelope is recoverable without a rollback path that is defined before it is needed.
Fig. RES/DEP — Production Ownership Envelope · Concentric ownership layers · inner rings must be owned before outer rings are operable
The most common deployment failure is attempting to operate an outer ring without the inner rings in place. A team that defines an SLA without a trace surface cannot measure whether it is being met. A team that implements a rollback procedure without an eval gate cannot verify whether the rollback improved or degraded quality. The envelope is not a checklist of optional features — it is a dependency structure. The inner rings are prerequisites for the outer ones.
A contract is a specification of what the service guarantees at its boundary — not what it tries to do, but what it will reliably produce, within what bounds, and what happens when it cannot. Traditional services have contracts: input schema, output schema, latency SLA, error codes and their meanings. AI services often have only a partial contract: input schema (sometimes), latency target (sometimes), and an implicit output contract of "something reasonable."
Three contract types are required for a production AI service. The schema contract specifies valid input structure and expected output structure. Output schema validation — checking that the model's response matches the expected structure before it is passed downstream — is the most commonly missing contract. An invalid output that passes the service boundary propagates to every consumer of that service. Output validation is cheap and prevents the most common class of production incident from AI services.
The budget contract specifies the maximum token consumption per call, per user session, and per billing period. Budget contracts must be enforced before the model call fires, not detected after the billing cycle closes. A service without a budget contract is a service whose cost is bounded only by its traffic — which means any traffic spike is also a cost spike, and any prompt change that increases token use does so silently until the bill arrives.
The quality SLA specifies what minimum quality level the service guarantees and how quality is measured. This is the hardest contract to define because it requires a scoring function — and agreeing on the scoring function is itself a product decision. The alternative — shipping without a quality SLA — means the implicit SLA is user complaints. That is the worst possible detection mechanism for quality regressions, as documented in the evaluation work at BLD/EVA.
Infrastructure observability — latency, error rate, throughput — tells you whether the service is running. It does not tell you whether the service is working. The distinction is the difference between a deployment that can be monitored and one that can be diagnosed. A service that is running and returning 200s while producing degraded quality outputs is invisible to infrastructure observability. It is only visible to quality observability — which requires semantic content in the trace, not just status codes and latency.
The minimum trace fields for an AI service extend standard distributed tracing. Model version and prompt version must be present to distinguish a model behavior change from a prompt regression. Token counts and per-stage latency (prefill, decode, total) are required for cost attribution and performance diagnosis. The input hash — not the full input, but a hash that allows correlation across runs — enables reproducibility testing: if the same input produces different outputs, the trace must be able to surface that. Output hash enables regression detection without storing the full payload.
The trace fields required to diagnose a quality failure are different from the fields required to diagnose an infrastructure failure. An infrastructure failure diagnosis needs latency breakdown and error codes. A quality failure diagnosis needs model version, prompt version, input class, and the eval score for this call if the eval pipeline ran inline. Building one without the other means the system can be diagnosed for only one class of failure. The relationship between trace design and the evaluation pipeline is explored in RES/REL.
Rollback in deterministic systems is a binary operation: the previous version either works or it does not. Rollback in AI systems changes a probability distribution — the previous model version or prompt version produced a different output distribution, and whether that distribution is better or worse for the current workload requires measurement, not assumption. A rollback that improves median quality may degrade quality for a specific input class. A rollback that reduces a quality regression may introduce a latency regression. Rollback without eval verification is a change without a check.
Three rollback primitives are available at different layers. Model version rollback reverts to a prior model checkpoint — the highest-impact change, affecting every call. It is only safe when the prior version has a current eval baseline against the current prompt and fixture set. Prompt version rollback reverts the system prompt or few-shot examples — lower impact than a model change but still affects every call that uses that prompt. Both require the version history that is currently missing from most production AI systems. Feature flag rollback disables the AI feature entirely and falls back to a deterministic path — the safest option under time pressure, requiring no eval verification, at the cost of the feature's functionality.
The practical guidance that emerges from this analysis: feature flag rollback should always be available as a last-resort option, implemented before deployment, never added reactively during an incident. Model and prompt version rollback require the version control and eval infrastructure to be useful. Attempting rollback without that infrastructure means making a change under pressure with no way to verify it was correct — which is the same error that caused the incident in the first place. The evaluation system at SYS-03 is the prerequisite infrastructure for model and prompt rollback to be operationally safe.
These are the questions where current understanding runs out. They are first-class content, not an appendix.
What is the minimum eval surface required before an AI feature is on-call-ready?
A feature is on-call-ready when the on-call responder can detect a quality regression, diagnose its cause, and take a corrective action — all under incident pressure. This requires: a quality signal that fires without manual inspection, enough trace context to diagnose the failure class, and a set of corrective actions (rollback, flag disable, prompt swap) that are safe to execute under pressure. What the minimum scoring coverage is, what trace fields are required for diagnosis, and which corrective actions are safe to execute without a full eval run — none of these are specified as a standard.
How should quality SLAs be defined for AI services where output quality is subjective or domain-specific?
Latency and availability SLAs have objective, measurable definitions. Quality SLAs require a scoring function — which requires ground truth, a judge model, or human evaluation — each with different cost and latency profiles. For AI services where quality is domain-specific, the scoring function must be agreed upon before the SLA is defined. Most teams skip this step, which means the service has an implicit quality SLA of "complaints from users" — which is the worst possible detection mechanism.
When is a model version rollback safe to execute without a full eval run?
A model version rollback changes the output distribution of every call. Some rollbacks are clearly safe: reverting to a version that was previously stable with a known eval baseline. Others are not: reverting to a version whose eval baseline is stale relative to the current prompt and fixture set. Specifying the conditions under which a rollback is safe to execute without a full eval run — and encoding those conditions in an incident playbook — requires agreement on what 'safe' means before the incident happens.
How should token budget enforcement compose with multi-step pipelines where per-step token use is variable?
A per-call token budget is straightforward for single-step features. For multi-step pipelines, the total token budget must be allocated across steps — but step-level token use varies based on input content and model behavior. A step that generates a list of items to process downstream can produce 2 items or 20, each requiring subsequent model calls. Pre-allocating budget across steps requires predicting token use per step before execution, which depends on the same output cardinality problem as backpressure in orchestration. The per-step budget allocation problem and the backpressure problem are structurally equivalent.
What trace fields are strictly required to diagnose a quality regression vs. an infrastructure failure at the service boundary?
The distinction between a quality regression and an infrastructure failure determines the corrective action. An infrastructure failure (high latency, timeout, provider error) is fixed by operational means. A quality regression (output distribution shift, prompt sensitivity change, model behavior change) requires eval-informed intervention. The trace fields required to distinguish the two at the point of an alert have not been formally specified. Current practice is to look at error rates first and quality metrics second — which means quality regressions that do not produce errors are diagnosed late or not at all.
The resolution order follows the ownership envelope inward. The trace field specification for quality diagnosis is the most immediately resolvable — it requires design decisions, not infrastructure that does not yet exist. Trace fields gate everything else: the on-call readiness standard cannot be defined without knowing what the trace surface must contain; the rollback safety conditions cannot be specified without knowing which trace fields distinguish a quality regression from an infrastructure failure. The quality SLA definition problem and the on-call readiness standard are parallel work streams that both depend on the trace specification. The budget enforcement problem for multi-step pipelines is the last to resolve because it depends on the same output cardinality measurement that the backpressure questions in RES/ORC require — and that data only exists after the pipeline runs at sufficient volume under realistic conditions. Until then, the budget allocation question is unanswerable by design. That is what makes this research rather than engineering.