Chapter 4

Service and API

TODO: provisional, author to rewrite. LLM gateways, model routing, caching, prompt management and tracing. The service boundary through which everything above reaches the model.

Sections

Sections are in progress. The chapter is part of the book’s declared structure and will fill in.