Memory

Backpropagation needs the forward pass’s activations, and at depth that dominates memory. The techniques here recompute what they refused to store, which is almost always the right trade when the alternative is not fitting at all.