Abstract the Provider, Not the Prompt

Count what this module has handed you so far: a model pin registry with prices and retirement dates, transcript assembly and trimming, stream consumption with abort wiring, error classification and translation, rate-limit budgets, and the cost log line from module 1. Now ask the only architecture question that matters: how many places in your codebase should know about all of that?

One. If forty services each hold a provider SDK, every rule in this module is a convention you hope everyone re-implements correctly. If every model call flows through one seam, each rule is code that exists once. The stack-map lesson called this box the LLM gateway and promised module 2 would build it; this is that lesson. Go's database/sql is the exact prior art: one interface, swappable drivers, and the drivers stay thin because they only translate, never think.

The two dialects, for real this time

Your first API call noted the ecosystem has two request shapes and called the differences cosmetic. For sending one request by hand, they are. An adapter has to sweat the cosmetics, because every one of them is a bug when normalization misses it:

Every row is the same idea wearing different serialization, which is exactly the situation database/sql was built for: Postgres and MySQL don't agree on the wire either, and your application code neither knows nor cares. (The stream grammar differences are the deepest row; normalizing them means exposing one delta-iterator shape regardless of who's underneath.)

Design the seam: normalize the envelope, pass through the rest

The interface your services see should be boring, and it should speak your vocabulary, not either provider's:

// The only LLM types the rest of the codebase imports.
export type CompletionResult = {
  text: string;
  stopReason: "complete" | "truncated" | "refused" | "tool_use";  // yours, not theirs
  usage: { inputTokens: number; outputTokens: number };
  model: string;          // echoed snapshot that actually served
  requestId: string;      // provider trace handle, into every log line
  raw?: unknown;          // escape hatch: the untouched provider response
};
 
export interface LLMClient {
  complete(req: CompletionRequest, opts?: { signal?: AbortSignal }): Promise<CompletionResult>;
  stream(req: CompletionRequest, onDelta: (t: string) => void,
         opts?: { signal?: AbortSignal }): Promise<CompletionResult>;
}

And a driver is small precisely because it only translates:

export class AnthropicClient implements LLMClient {
  constructor(private sdk = new Anthropic()) {}
 
  async complete(req: CompletionRequest, opts?: { signal?: AbortSignal }) {
    const res = await this.sdk.messages.create({
      model: req.model,
      system: req.system,                      // top-level here, message-role in the OpenAI driver
      messages: req.messages,
      max_tokens: req.maxTokens,               // required by this dialect; the seam makes it required for all
    }, { signal: opts?.signal });
 
    return {
      text: res.content[0].type === "text" ? res.content[0].text : "",
      stopReason: ({ end_turn: "complete", max_tokens: "truncated",
                     refusal: "refused", tool_use: "tool_use" } as const)[res.stop_reason] ?? "complete",
      usage: { inputTokens: res.usage.input_tokens, outputTokens: res.usage.output_tokens },
      model: res.model,
      requestId: res._request_id ?? "",
      raw: res,
    };
  }
  // stream(): same translation over the event grammar from the streaming lesson
}

Three design rules keep this from growing into a framework:

• Normalize the envelope, not the intelligence. Text, stop reason, usage, IDs: normalize ruthlessly. Prompt content, sampling parameters you actually tune, provider-specific features you deliberately use: pass through. The moment your abstraction needs a plugin system, you've rebuilt LangChain with fewer tests.
• Adopt the strictest dialect's rules. Anthropic requires max_tokens; therefore your interface requires it, and the OpenAI driver simply always sends it. Requiring the union of strictness costs nothing and caps runaway output everywhere for free.
• Keep the escape hatch. The raw field means the one feature that needs a provider-specific response tomorrow doesn't fork your whole abstraction; it reads raw and owns the coupling locally.

Don't unify tool calling on day one. The two dialects diverge most there (schemas, streaming deltas, parallel calls), you'll get the abstraction wrong before you've used both, and module 3 hasn't even defined a tool yet. Add that surface when a real second-provider tool workload exists.

The trap that names this lesson

Here's the failure mode that catches teams who build a beautiful seam: the adapter makes a provider swap compile, so they believe it makes the swap safe. Then they flip a fallback to the other provider during an incident and quality craters, because the interface ports but the prompts don't. Your prompts were tuned against one model's habits: its verbosity, its adherence to the system prompt, its JSON discipline, its refusal style. The other provider's model reads the same prompt and behaves differently, per exactly the mechanics the non-determinism lesson laid out.

So the rule from the versioning lesson applies with no discount: a provider swap is a model migration. Eval-gate it, canary it, log which side served. The seam's honest value is that it makes the mechanical cost of trying near zero, so the eval is the only cost left. That's a huge win; it's just not the same claim as "we can switch providers transparently." Cross-provider fallback chains (the availability play from the playbook) live behind this same seam, with the same eval requirement on every link.

Build it, buy it, or skip it

The middle option deserves its honest trade-off stated: a gateway product (LiteLLM self-hosted, OpenRouter-style hosted routers) is infrastructure on your hottest path. It adds a network hop, it can lag provider feature releases, it has its own rate limits and incidents, and it becomes load-bearing the moment it works. Sometimes that's the right buy (genuinely many providers, platform teams serving dozens of internal customers). For a typical product with one primary and one fallback provider, the 150-line adapter is less system to operate. A useful middle path while migrating: Anthropic ships an OpenAI-SDK compatibility layer (point the OpenAI SDK's base URL at Anthropic), which is a bridge for evaluation, not an architecture.

The module, assembled

Look at what fits inside that one seam now. The pin registry resolves features to snapshots and prices. The transcript assembler builds and trims the messages. The stream consumer aggregates deltas and never loses usage. The failure translator turns provider chaos into your API's contract, with the degradation ladder underneath. The capacity budgets make features queue instead of starving each other. Every one of them written once, reviewed once, living where every call already flows.

That's the client layer, and it's the part of the stack most teams get wrong by not noticing they were building it. What it transports is still raw text in both directions, though: module 3 is about making the payloads trustworthy, starting with prompts as versioned contracts and outputs as validated, schema-shaped data instead of strings you hope parse.

Key takeaways

• Everything this module built (pins, transcripts, stream handling, error translation, budgets) should exist once, at one seam every model call flows through. The gateway box from the stack map is this lesson.
• The two dialects differ in mechanical, enumerable ways: system prompt location, max_tokens requiredness, response paths, usage field names, stop vocabularies, error shapes, stream grammars. Adapters translate exactly that list and nothing more.
• Normalize the envelope (text, stop reason, usage, IDs) into your own vocabulary, adopt the strictest dialect's rules for everyone, and keep a raw escape hatch. Don't abstract tool calling before module 3 gives you a reason.
• Interfaces port, prompts don't: a provider swap that compiles is not a swap that's safe. Every swap and every fallback link is a model migration with an eval gate.
• Build the thin adapter for 1-2 providers; buy a gateway only when provider breadth is the actual requirement (it's a new hot-path dependency to operate); single-provider teams still get one wrapper module so the seam exists early.
• The chokepoint property is the point: one place holding keys, logs, budgets, and translation. Module 3 makes what flows through it trustworthy.