GenAI in Enterprise Operations: Patterns That Actually Work

Enterprise teams are running GenAI pilots, declaring success on demos, and struggling to move to production. The gap is almost never the model — it is the operational wrapper: observability, guardrails, cost controls, fallback logic, and the feedback loops that let the system improve without human intervention. This talk is a walkthrough of five patterns that have cleared that gap, with the architecture, the failure mode to watch for, and a real cost-per-call estimate for each.

The Pilot-to-Production Gap

Most GenAI pilots are demos with a good day. They run against a curated dataset, a friendly prompt, and no adversarial input. Production is none of those things — it's noisy telemetry, ambiguous tickets, edge-case PRs, and a cost ceiling that finance will ask about within the first billing cycle. Closing that gap doesn't require a better model. It requires treating the LLM as one component inside a system that has to fail predictably, cost a known amount, and get better over time without a human rewriting the prompt every week.

The five patterns below are the ones Naval has seen clear that bar at enterprise scale. Each includes the architecture, the prompt engineering approach, the failure mode to watch for, and a cost-per-call estimate — the four things a platform team actually needs before they'll sign off on a production rollout.

Five Patterns That Actually Work

1. AI-Assisted Incident Triage

Architecture: alert stream → context retrieval (recent deploys, related incidents, on-call runbook) → LLM summarization and severity suggestion → human confirms before paging. Prompt approach: structured extraction, not free-form chat — the model returns a fixed schema (likely cause, affected services, suggested severity, confidence) so downstream tooling can act on it deterministically. Failure mode to watch: confidence inflation — models are consistently overconfident about root cause with partial telemetry, so treat every suggestion as a hypothesis, not a verdict. Cost-per-call: roughly $0.01–0.03 per triage using a mid-tier model with a trimmed context window.

2. LLM-Backed Runbook Generation

Architecture: retrieval-augmented generation over your actual runbooks and past incident postmortems — never a bare model, always grounded in internal documentation. Prompt approach: few-shot with your team's existing runbook format so output matches house style instead of generic AI prose. Failure mode to watch: stale grounding — a runbook generated from six-month-old documentation is worse than no runbook, because it's confidently wrong. Re-index on every documentation merge. Cost-per-call: $0.05–0.15 per generated runbook, dominated by retrieval context size.

3. Cost Anomaly Narration

Architecture: your existing anomaly detector fires on a cost spike → LLM translates the raw delta (service, resource, magnitude, time window) into a plain-English narrative an engineering manager can act on without reading a billing export. Prompt approach: template-constrained generation — the numbers come from your data pipeline, the model only writes the sentence around them, which eliminates hallucinated figures. Failure mode to watch: narrating noise as signal — pair this with a minimum-magnitude threshold or you'll page people about $4 anomalies. Cost-per-call: under $0.01, since the model only ever writes one paragraph from structured input.

4. Automated PR Review and Policy Enforcement

Architecture: diff-scoped review — the model only ever sees the changed lines plus immediate surrounding context, not the whole repository, both for cost and for focus. Prompt approach: a fixed policy checklist (secrets, dependency changes, breaking API changes, missing tests) rather than an open-ended "review this code" prompt, which produces inconsistent, low-signal comments. Failure mode to watch: review fatigue — if the bot comments on everything, engineers start ignoring it within two weeks. Tune it to flag only policy violations, not style preferences. Cost-per-call: $0.02–0.08 per PR depending on diff size.

5. Intelligent Capacity Forecasting

Architecture: time-series forecasting model handles the numeric prediction; the LLM's job is strictly to explain the forecast and flag what's driving it (seasonality, a recent traffic pattern change, an upcoming release) — never to do the numeric forecasting itself. Prompt approach: the model receives the forecast output and the top contributing features, and writes the explanation — it never sees raw time-series data. Failure mode to watch: letting the LLM freelance a number when the forecasting model has low confidence — always surface the confidence interval, and instruct the model to say so explicitly. Cost-per-call: $0.01–0.02, run on a scheduled batch rather than per-request.

Measuring Whether It's Actually Working

Conclusion

None of these five patterns require a frontier model or a research team. What they require is discipline about scope — the LLM does one narrow job, grounded in real data, wrapped in guardrails that make its failure modes boring instead of catastrophic. That discipline is the actual skill enterprises are missing, and it's the difference between a GenAI pilot that gets a case study and one that gets decommissioned after the demo.