Transformers
Not the fastest way to run a model or the most efficient way to train one. It is the one that has every architecture, and the one whose checkpoint format every other tool consumes.
Standing
Current standardPart of the default recipe. Present in most frontier models trained today, and the thing a new design departs from rather than argues for.
The lingua franca. A model that is not in this library is, for most practical purposes, a model nobody can use.
judged as of 2026-09 · what the labels mean
Theory
Every other tool on this shelf either wraps this library, consumes checkpoints saved by it, or is measured against it. That position — not speed, not elegance — is what makes it worth an entry.
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained(
"meta-llama/Llama-3.1-8B-Instruct",
dtype="bfloat16", # the default is fp32; say what you mean
device_map="auto", # shard across whatever GPUs exist
attn_implementation="flash_attention_2",
)
tok = AutoTokenizer.from_pretrained("meta-llama/Llama-3.1-8B-Instruct")
chat = [{"role": "user", "content": "What is a capacity factor?"}]
ids = tok.apply_chat_template(chat, add_generation_prompt=True, return_tensors="pt")
print(tok.decode(model.generate(ids.cuda(), max_new_tokens=200)[0]))What it is for
- Loading any architecture. Several hundred of them, each a readable Python file. When the paper and the code disagree, this is usually the code people mean.
- The
from_pretrainedcontract. A name resolves to weights, tokeniser and config, cached locally. Nearly every tool in this book accepts one of these identifiers. - Reading how something works.
modeling_llama.pyis a better description of the architecture than most papers, and it is the one that runs.
What it is not for
Production serving. generate has no continuous batching and no paged cache.
vLLM exists for this and is an order of magnitude faster under
load.
Memory-optimal training. The Trainer works and is not tuned;
Unsloth and the config-driven wrappers beat it substantially
on a single GPU.
Edge deployment. Python, PyTorch and CUDA. llama.cpp is the answer there.
Two things worth knowing
apply_chat_template. Every instruction-tuned model has its own turn
formatting, and getting it wrong degrades output in a way that looks like the
model being bad. The template ships in the tokeniser config, so the model tells
you its own format — hand-writing [INST] markers is a mistake with a silent
failure mode.
safetensors. The successor to pickled .bin checkpoints, which could execute
arbitrary code on load. safetensors is a flat, memory-mappable format that cannot
carry code, so loading an untrusted checkpoint is safe. A repository still
offering only .bin is a reason for suspicion, not merely for inconvenience.