AI Grimoire

Inference / kernels

Kernels

Some of the largest speedups of the last few years changed no mathematics at all. They changed what was written to high-bandwidth memory and what stayed in on-chip SRAM.

1 entry.

Entries

04.04.1
FlashAttentionstandard
Tiled, IO-aware attention kernel; recomputes softmax statistics online.
O(n²·d)