AI Grimoire
04 — siblings
kv-cache/decoding/
quantisation/
kernels/serving/

Inference / quantisation

Quantisation

Weights carry more precision than inference needs. The interesting part is not the rounding but the error compensation — deciding which weights to round first and how to adjust the rest so the layer’s output survives.

1 entry.

Entries

04.03.1
GPTQcommon
Layer-wise second-order weight quantisation with Hessian-guided rounding.
O(d³)