Working with LLM APIs

The LLM API as a backend dependency: chat completions, streaming, retries and timeouts, rate limits, provider abstraction, and treating model calls like any other flaky upstream service.

1. 01·Pin Your Model Like You Pin Your PackagesModel IDs are versions, aliases move under you, and deprecations have deadlines. How to pin, monitor, and upgrade your LLM dependency like any other.7 min
2. 02·The API Is Stateless. Your Chat Isn't.The LLM API keeps no session state, so your chat feature needs a transcript store, a token budget, and a truncation strategy. Build all three.7 min
3. 03·What's Actually in the StreamWhat an LLM stream actually sends: OpenAI chunks vs Anthropic events, delta aggregation, mid-stream errors, and canceling generations nobody is reading.7 min
4. 04·When the Model Has a Bad DayLLM calls fail three ways: real errors, hangs, and 200s that lie. Map each failure to what your API serves instead, from a cached copy to a hidden feature.8 min
5. 05·Rate Limits Are a Capacity ContractRPM and TPM are capacity contracts, not suggestions. Read the rate-limit headers, throttle client-side, and plan token headroom before the 429s arrive.7 min
6. 06·Abstract the Provider, Not the PromptDesign the one seam every LLM call goes through: a normalized client interface over OpenAI and Anthropic dialects, and when to buy a gateway instead.8 min