
Meta research reveals that large language models have a strict memory ceiling: GPT-class models store approximately 3.6 bits per parameter. Despite training on 7TB of data, models retain only about one-two-thousandth. When capacity is exceeded, models shift from memorization to learning generalized patterns. The paper received an Outstanding Paper Honorable Mention at ICML 2026.

