Prompts Are Config That Can Take Down Prod

Two lessons ago the system prompt was a string. Now it's a structured contract plus a curated fixture set plus an assembly function with cache-sensitive ordering. The stack map made a promise about exactly this artifact: prompts are hot-path production config that happens to be written in English, and hardcoded prompt strings are the new hardcoded connection strings. This lesson pays that promise mechanically.

The claim earns its urgency from an asymmetry: a one-line wording change can shift output quality more than a model upgrade, and nothing in your toolchain notices. Change a SQL query and tests fail; change "be concise" to "be brief and precise" and every test passes while your feature's tone, length, and refusal rate quietly move. English is a programming language your CI can't parse. The only defense is process: version the asset, review changes, deploy them observably, and keep rollback one step away.

The connection-string test

The test for whether config is being managed or merely used: can you answer what value was live at 14:32 last Tuesday, and who changed it? Connection strings pass because a decade of incidents taught everyone to centralize them (the twelve-factor config rule is that lesson fossilized). Prompt strings scattered as literals across forty services fail it completely.

Before you can version the asset, be honest about what the asset is. A prompt version is not the template text alone. It's the tuple that produces behavior:

• The template (contract clauses, structure, fences) and its variables.
• The fixture set, since a changed example changes behavior as surely as a changed clause.
• Sampling parameters: temperature, max tokens, stop sequences.
• The model pin it was tuned against. Module 2's versioning lesson taught you to pin models; the abstraction lesson warned that prompts don't port across them. A prompt version without a model pin is half a version: "v4 of the words, tuned against nobody in particular."

Bump the version when any element changes. The unit that ships is the tuple, not the text.

The template is a function signature

Once the prompt is data, the call site needs a way to fill its slots, and that filling deserves the same rigor as any function call. A template with two variables is a function with two parameters: render() should validate them the way a signature would, at render time, in your process, where failure is cheap and loud, instead of at the provider, where failure is a well-formed garbage response that costs money and reads as a quality regression.

// One versioned entry in the prompt registry. The unit is the tuple:
// template + fixtures + params + the model pin it was tuned against.
export const ticketTriage = definePrompt({
  id: "ticket-triage",
  version: 4,
  modelPin: pins.triage,                  // tuned against this snapshot
  params: { temperature: 0, maxTokens: 256 },
  examples: TRIAGE_FIXTURES_V3,
  vars: { ticketBody: { required: true, maxChars: 8_000 } },
  render({ ticketBody }) {
    return {
      system: renderSystemPrompt(triageContract),
      messages: withExamples(TRIAGE_FIXTURES_V3, ticketBody),
    };
  },
});
// render() throws on a missing or oversized variable: fail here, in-process,
// not as a confusing model response 900ms and $0.004 later.

Render-time validation also enforces last lesson's injection discipline in exactly one place: the template owns the fencing and escaping, so no call site can forget it. And a template change that adds a variable is a signature change: every caller is affected, and the diff should be reviewed like one.

Where prompt versions live

Three storage options, in ascending order of infrastructure:

The trigger for leaving the repo is organizational, not technical: the moment non-engineers legitimately need to iterate on wording (support leads tuning tone, localization), a git workflow becomes the bottleneck and a database or a platform like Langfuse earns its keep. Until that moment, the repo wins on the strength of everything you get for free: review, blame, and the atomicity of shipping the prompt with the code that consumes its output. That atomicity matters more in this module than ever, because lesson 6 couples every prompt to a validation schema, and a prompt deployed without its matching schema is a coordinated-deploy bug you've met before in API versioning.

A prompt change is a deploy

Process, not storage, is what actually prevents the 2am incident. The pipeline for a prompt change is the pipeline for any risky config change, with one LLM-specific insertion:

Three stations deserve emphasis:

The eval gate. Reviewers can catch a contradictory clause; nobody can eyeball whether new wording drops extraction accuracy from 96% to 89%. That's a measurement, the harness for it is module 6's subject, and it sits in this pipeline exactly where tests sit in a code pipeline: a prompt PR that drops the pass rate gets blocked like a PR that breaks the build.

The log line. Every call already logs model, tokens, cost, and request ID. Add promptId@version. This one field turns "quality feels off since Tuesday" from vibes into a query: group outcomes by prompt version, see the cliff, read the diff. Without it, prompt changes are invisible in your observability stack, which is what "English is config your tooling can't see" means operationally.

const rendered = ticketTriage.render({ ticketBody: ticket.body });
const res = await llm.complete({ ...rendered, ...ticketTriage.params });
 
log.info("llm.call", {
  prompt: "ticket-triage@4",        // the field that makes incidents queryable
  model: res.model,
  usage: res.usage,
  requestId: res.requestId,
});
 
// Rollback, the whole procedure:
//   import { ticketTriage } from "./prompts/ticket-triage/v3";
// One step, like re-pointing a config value. If getting back to last
// Tuesday's behavior takes more than this, versioning isn't done yet.

The canary. Non-determinism means an eval pass is a distribution statement, not a guarantee, so the last gate is real traffic at small percentage with the refusal-rate and quality metrics from module 2 on a dashboard. Google's SRE workbook treats config changes as a leading outage cause and prescribes exactly this: progressive rollout, observable impact, fast rollback. Prompts inherit the prescription because prompts are config; the only novelty is why they're risky (semantic blast radius your parsers can't lint).

The request side, closed out

The request side of the trust problem is now governed end to end: a contract states behavior, fixtures pin the boundaries, a template validates its inputs and owns the fencing, and the whole tuple is versioned, eval-gated, canaried, logged, and one step from rollback. Everything the model receives from you is reviewed, reproducible, and attributable.

Everything it returns is still a string you hope parses. The response side gets the same treatment next: first the mechanisms that make model output take a shape you dictate (JSON mode, structured outputs, and the tool-call trick), then schema design, then validation at the boundary. By the capstone, both directions of the payload run under contract.

Key takeaways

• A prompt version is a tuple: template + fixtures + sampling params + the model pin it was tuned against. Bump the version when any element changes; ship them together.
• Hardcoded prompt strings fail the connection-string test (what was live at 14:32 Tuesday, who changed it?). Centralize prompts in a registry, wherever it's stored.
• Templates are function signatures: validate variables at render time in-process, own fencing/escaping in one place, and review a new variable like a signature change.
• Store versions in the repo by default (review, blame, atomic deploy with the consuming code); move to DB-plus-flag or a platform when non-engineers need to edit, and accept the drift risk you take on.
• A prompt change is a config deploy: PR review, eval gate, canary, and one-step rollback, per the SRE playbook for config-caused outages.
• Log promptId@version on every call next to model, cost, and request ID. It's the field that turns "quality feels off" into a query.