Grokking
Also known as: delayed generalization, grok, late generalization
In training, you'd expect a model that gets 100% accuracy on its training examples to either generalize (do well on new examples) or not. Grokking is the weird case where generalization is severely delayed: training accuracy hits 100% early, but test accuracy (performance on new examples) stays near random for thousands of additional training steps, then abruptly jumps to near-perfect. The model was memorizing first, then suddenly learned the underlying rule.
This was documented in a 2022 paper from DeepMind using simple arithmetic tasks on small transformers. It's been studied since to understand what triggers the transition. One hypothesis: the model first learns a brute-force memorization solution, then during continued training discovers a more efficient 'algorithmic' solution that also generalizes, and switches to it.
For most builders, grokking is more of an intellectual curiosity than a daily concern. It shows up in discussions about why models sometimes improve unexpectedly with longer training, and why stopping training early based on apparent convergence might miss later gains. It's also cited in debates about what 'understanding' vs. 'memorization' means in AI systems.