GPTQ

Frame quantisation per layer as minimising output error on a calibration set rather than weight error. The objective is quadratic in the weight perturbation, with Hessian $H = 2XX^{\top}$ given by the layer’s input activations.

Optimal Brain Surgeon gives the compensation for a single rounded weight in closed form. GPTQ applies it greedily in a fixed column order, so the inverse Hessian can be Cholesky-factorised once and reused, dropping the cost from $O(d^4)$ to $O(d^3)$.

import torch
from torch import Tensor


def gptq_column_loop(W: Tensor, Hinv: Tensor, bits: int = 4) -> Tensor:
    """W: [out, in] rows quantised jointly, columns in order.
    Hinv: upper-Cholesky inverse Hessian [in, in]."""
    W = W.clone()
    n_cols = W.size(1)
    scale = W.abs().max(dim=1, keepdim=True).values / (2 ** (bits - 1) - 1)

    for q in range(n_cols):
        w = W[:, q]
        dq = (w / scale.squeeze(1)).round().clamp(
            -(2 ** (bits - 1)), 2 ** (bits - 1) - 1
        ) * scale.squeeze(1)
        err = (w - dq) / Hinv[q, q]
        W[:, q] = dq
        # push the error onto columns not yet visited
        W[:, q + 1:] -= err[:, None] * Hinv[q, q + 1:][None, :]
    return W

Dampen the Hessian diagonal by about 1% of its mean before inverting; activation Hessians from a small calibration set are routinely singular, and an unregularised Cholesky will fail or, worse, succeed with garbage. Quantising in blocks of 128 columns with a re-derived scale per block is what the released implementations actually do — the single global scale above is for legibility.