Tree — 09

serving/4 tuning/4 orchestration/5

Tooling

The rest of the book is about mechanisms. This shelf is about the software people actually type — the runtimes that serve a model, the libraries that fine-tune one, the frameworks that wire several together. Entries here are deliberately shallow on internals and specific about purpose: what a tool does, what it does not, and which mechanism elsewhere in the book it is an implementation of. Versions move faster than anything else here, so nothing depends on a flag or an API signature.

13 entries across three branches, none of them stubs.

Serving and runtimes

getting weights to answer a request

llama.cppstandard

A dependency-free C++ inference engine, and the GGUF format everything local uses.

A local model runner: pull a name, get a server, forget the flags.

A serving runtime built around prefix reuse and constrained decoding.

The throughput server: paged KV cache, continuous batching, one endpoint.

Training and tuning

libraries for adapting a model you did not train

Config-Driven Fine-Tuningcommon

Axolotl, LLaMA-Factory and the case for a YAML file instead of a script.

PEFT and TRLstandard

The adapter library and the preference-training library, and how they compose.

Transformersstandard

The reference implementation of nearly every architecture, and the format everything reads.

Hand-written kernels and manual backward passes for fine-tuning on one GPU.

Orchestration

orchestration/ ›

wiring models to tools, data and each other

Agent Frameworkscommon

Loops with tools: what the frameworks add, and how little of it is the hard part.

LangChaincommon

The composition framework everyone starts with, and the argument about whether to keep it.

LlamaIndexcommon

A retrieval framework: ingestion, chunking, indexing and query pipelines.

Model Context Protocolcommon

A standard interface between a model host and the tools it can call.

Vector Databasescommon

Approximate nearest-neighbour search, and whether you need a separate system for it.