AI Engineering for Backend Developers

A working mental model of LLMs for people who think in systems: tokens, context windows, model selection, non-determinism, and the production AI stack.

The LLM API as a backend dependency: streaming, retries and timeouts, rate limits, provider abstraction, and treating model calls as a flaky upstream.

Prompts as API contracts: system prompt design, few-shot patterns, JSON schema outputs, tool definitions, and treating model output as untrusted input.

Retrieval-augmented generation end to end: embeddings, chunking strategies, vector databases vs pgvector, hybrid search, reranking, and the architecture of a production RAG service.

From single tool calls to agent loops: designing tools, orchestration patterns, state and memory, guardrails, human-in-the-loop, and when an agent is the wrong answer.

The skill that separates demos from products: building an eval harness, golden datasets, LLM-as-judge, regression testing prompts, tracing, and monitoring LLM systems in production.

LLM performance engineering: token economics, caching strategies, model routing, batching, self-hosting vs API trade-offs, and keeping p95 latency and unit costs under control.

Three production-grade builds that tie the track together: a RAG service with hybrid search, an eval harness for a real prompt, and an observable agent with guardrails.