Production AI Systems Engineering

TODO: provisional, author to rewrite. Six layers, written in the open

TODO: provisional, author to rewrite. A book on the engineering of production AI systems, written in the open, one section at a time.

TODO: provisional, author to rewrite. For engineers who have shipped something with a model in it and found that the demo was the easy part.

Layers

  1. Layer 0

    Foundations

    0/4

    TODO: provisional, author to rewrite. The groundwork beneath the stack. The concepts and constraints every layer above depends on, established before any specific system is built.

    Sections in progress

  2. Layer 1

    Infrastructure

    0/4

    TODO: provisional, author to rewrite. Compute platforms, GPU clusters, container orchestration and networking. The substrate an AI system runs on.

    Sections in progress

  3. Layer 2

    Data

    1/5

    TODO: provisional, author to rewrite. Ingestion pipelines, document processing, feature stores, indexing and governance. What an AI system knows and how it comes to know it.

  4. Layer 3

    Intelligence

    0/6

    TODO: provisional, author to rewrite. Models, RAG pipelines, retrieval, context assembly, guardrails and evaluation. The engineering around the model, not the design of the model.

    Sections in progress

  5. Layer 4

    Service and API

    0/5

    TODO: provisional, author to rewrite. LLM gateways, model routing, caching, prompt management and tracing. The service boundary through which everything above reaches the model.

    Sections in progress

  6. Layer 5

    Application

    0/4

    TODO: provisional, author to rewrite. User-facing AI applications, copilots, agents and workflow automation. Where the stack meets a person doing a job.

    Sections in progress

4% complete

Subscribe

Updates when a new section of the book is published.

Subscribe