The Forgetting Curve of AI: Why Neural Networks Lose Memories

by

Author: Denis Avetisyan

New research reveals the underlying mechanisms driving catastrophic forgetting in artificial intelligence, offering insights into how to build more stable and adaptable learning systems.

The observed structural collapse within neural networks-manifesting as progressively limited representational diversity starting in early layers-demonstrates that standard training methods like SGD and LwF induce a loss of feature richness, ultimately causing decision boundary drift and differing failure modes across network architectures.
The observed structural collapse within neural networks-manifesting as progressively limited representational diversity starting in early layers-demonstrates that standard training methods like SGD and LwF induce a loss of feature richness, ultimately causing decision boundary drift and differing failure modes across network architectures.

Representational and structural collapse are key drivers of forgetting in continual learning, and monitoring effective rank can help mitigate this phenomenon.

Despite advances in machine learning, neural networks still struggle to learn sequentially without catastrophically forgetting previously acquired knowledge. This phenomenon is explored in ‘Why Do Neural Networks Forget: A Study of Collapse in Continual Learning’, which investigates the link between forgetting and representational collapse-a loss of model capacity reflected in decreasing effective rank (eRank). The authors demonstrate that forgetting correlates strongly with this collapse across multiple architectures and benchmarks, and that continual learning strategies like Experience Replay can mitigate both performance loss and capacity reduction. But can monitoring eRank serve as a reliable early indicator of potential forgetting, enabling more proactive regularization techniques?

The Fragile Architecture of Memory

Conventional artificial neural networks face a significant hurdle when tasked with continual learning – the ability to acquire new knowledge without overwriting previously learned information. This limitation manifests as “catastrophic forgetting,” wherein performance on older tasks plummets rapidly as the network adapts to new ones. Unlike human learning, which builds upon existing knowledge, these networks often treat each new task as entirely separate, adjusting internal parameters in ways that disrupt established representations. This isn’t simply a matter of imperfect recall; the very patterns the network used to solve prior problems are effectively erased, hindering its ability to retain a cumulative understanding of the world. Consequently, developing methods to mitigate catastrophic forgetting is crucial for creating truly adaptable and intelligent systems capable of lifelong learning.

The phenomenon of catastrophic forgetting arises from a neural network’s tendency to overwrite previously learned information when exposed to new data, ultimately due to a constricted representational space. Rather than integrating novel knowledge, the network adjusts its internal parameters to strongly reflect the most recent task, effectively diminishing the distinctiveness of earlier learnings. This isn’t merely a loss of specific weights, but a fundamental reshaping of the network’s internal landscape, leading to a reduction in the diversity of feature representations. Consequently, the network struggles to maintain a stable and comprehensive understanding of all encountered tasks, hindering its ability to generalize and retain knowledge across a sequence of learning experiences. The stability of this representation space is therefore critical; a robust network must be able to accommodate new information without sacrificing the integrity of existing knowledge structures.

To rigorously assess a neural network’s susceptibility to catastrophic forgetting – the tendency to abruptly lose previously learned information when exposed to new data – researchers frequently employ benchmark datasets like Split MNIST and Split CIFAR-100. These datasets are specifically constructed to simulate sequential learning scenarios, presenting tasks one after another to challenge a model’s memory retention. A recent study utilizing these benchmarks revealed a crucial link between catastrophic forgetting and a decline in the complexity of the model’s internal representations. As networks learn new tasks, they often simplify their understanding of all tasks, leading to a collapse in both the diversity of features the network extracts – its representational complexity – and the intricate relationships between those features – its structural complexity. This simplification, the study suggests, is a core mechanism driving the forgetting phenomenon, highlighting the need for strategies that preserve both richness and organization within a network’s learned knowledge.

The model successfully performs sequential binary classification on the Split MNIST dataset, demonstrating its ability to learn and adapt across five consecutive tasks.
The model successfully performs sequential binary classification on the Split MNIST dataset, demonstrating its ability to learn and adapt across five consecutive tasks.

The Erosion of Representation

Representational collapse, frequently observed alongside catastrophic forgetting in neural networks, describes the phenomenon where the model’s internal feature space diminishes in size during continual learning. This shrinking of the feature space restricts the model’s capacity to represent new information, as previously learned features become overly similar or are overwritten. Consequently, the model struggles to effectively encode and retain knowledge from subsequent tasks, leading to performance degradation. The reduction in feature space diversity limits the model’s ability to discriminate between different inputs and hinders its generalization capabilities.

Effective Rank (eRank) provides a quantifiable metric for assessing representational collapse during continual learning. This measure assesses the diversity and richness of a model’s learned representations by evaluating the rank of its feature activation matrix; a lower eRank indicates reduced representational capacity. Research demonstrated that models trained using Stochastic Gradient Descent (SGD) and the Learning without Forgetting (LwF) method both experienced a consistent decline in eRank as they were exposed to sequential tasks. This observed decrease in eRank directly correlates with the loss of a model’s ability to effectively encode new information and is indicative of representational collapse during continual learning scenarios.

A decline in Effective Rank (eRank) during continual learning indicates a reduction in the model’s representational capacity, directly contributing to catastrophic forgetting as the feature space becomes less diverse. Research demonstrates that Experience Replay (ER) effectively counteracts this phenomenon; models utilizing ER consistently maintain or even improve their eRank throughout the training process. This preservation of representational diversity through ER suggests that retaining previously learned information within the replay buffer helps the model avoid representational collapse and subsequently, mitigate catastrophic forgetting by preserving a richer and more robust feature space.

Catastrophic forgetting compresses previously learned features into a low-dimensional subspace, reducing activation <span class="katex-eq" data-katex-display="false">eRank</span> and hindering long-term learning, a problem partially mitigated by Experience Replay (ER) but limited by structural collapse in Learning without Forgetting (LwF).
Catastrophic forgetting compresses previously learned features into a low-dimensional subspace, reducing activation eRank and hindering long-term learning, a problem partially mitigated by Experience Replay (ER) but limited by structural collapse in Learning without Forgetting (LwF).

The Architecture’s Internal Fracture

Continual learning models are susceptible to degradation beyond changes in activation values; the model’s weights can also experience structural collapse. This phenomenon indicates a reduction in the model’s effective complexity, where the weight matrices lose their capacity to represent diverse features. Structural collapse is characterized by a decrease in the rank of weight matrices, signifying a loss of representational power and contributing to catastrophic forgetting. Unlike traditional degradation metrics focused on activations, assessing weight rank provides a direct measurement of this loss in model complexity and offers insight into the model’s ability to retain knowledge over time.

Weight eRank functions as a quantifiable metric for assessing structural collapse within neural networks during continual learning, effectively indicating changes in model capacity. This value is derived from the singular values of the weight matrix and reflects the distribution of learned feature importance. Experimental results demonstrate that employing Experience Replay (ER) consistently mitigated the decline in weight eRank compared to standard Stochastic Gradient Descent (SGD) and Learning without Forgetting (LwF) methodologies. The slower rate of decline in weight eRank under ER suggests a greater preservation of model complexity and representational capacity throughout the continual learning process, contributing to improved performance and reduced catastrophic forgetting.

Activation eRank and Weight eRank provide quantifiable metrics for assessing representational and structural collapse during continual learning. Empirical results demonstrate that the Elastic Weight Consolidation (EWC) method, utilizing these eRanks, consistently outperforms standard Stochastic Gradient Descent (SGD) and Learning without Forgetting (LwF) across diverse neural network architectures – including Multilayer Perceptrons (MLP), Convolutional GRUs (ConvGRU), ResNet-18, and Bi-ConvGRUs – and datasets, specifically Split MNIST and Split CIFAR-100. This performance translates to higher overall accuracy and a substantial reduction in catastrophic forgetting, indicating EWC’s ability to preserve knowledge from previously learned tasks while adapting to new information.

Experience replay (ER) demonstrably mitigates structural collapse across all layers of both architectures, while learning without forgetting (LwF) provides comparatively minor improvements, especially in early and mid-level layers.
Experience replay (ER) demonstrably mitigates structural collapse across all layers of both architectures, while learning without forgetting (LwF) provides comparatively minor improvements, especially in early and mid-level layers.

The pursuit of continual learning, as detailed in this study, reveals a fascinating fragility within neural networks – a tendency towards representational collapse. It’s a process akin to dismantling a carefully constructed edifice, brick by brick, with each new task. Donald Knuth observed, “The best computer costs nothing-it’s the one you build yourself.” This resonates deeply; the network, too, must actively ‘build’ and maintain its internal structure to resist forgetting. Monitoring effective rank (eRank) offers a means to observe this structural integrity, revealing when the foundations begin to crumble under the weight of new information. Experience Replay, then, isn’t simply a mitigation strategy, but a deliberate act of reconstruction, reinforcing those critical internal supports.

The Architecture of Forgetting

The identification of representational and structural collapse as drivers of catastrophic forgetting feels less like a solution and more like a precise articulation of the problem. It clarifies how networks forget, but does little to suggest why such elegant systems are so prone to this fundamental instability. The effective rank, as a diagnostic, offers a glimpse under the hood, a metric for observing the tightening grip of collapse, but it remains a descriptive tool. Future work must move beyond monitoring these collapses to actively engineering resilience into the network architecture itself.

Experience replay demonstrably mitigates the issue, yet it feels…unsatisfactory. A system that learns by constantly revisiting the past hints at a deeper limitation – an inability to truly synthesize information. Is the brain itself not more efficient, distilling experience rather than endlessly replaying it? The field needs to explore methods that promote genuine knowledge consolidation, perhaps by mimicking biological mechanisms of synaptic pruning and long-term potentiation with greater fidelity.

Ultimately, the persistent struggle against forgetting highlights a crucial point: intelligence isn’t about accumulating data, but about distilling meaningful structure from chaos. The architecture of a learning system, therefore, should not merely be judged by its ability to store information, but by its capacity to ignore it – to identify and discard the irrelevant, leaving only the essential framework of understanding intact. The true challenge lies not in preventing collapse, but in orchestrating it.

Original article: https://arxiv.org/pdf/2603.04580.pdf

Contact the author: https://www.linkedin.com/in/avetisyan/

See also:

2026-03-07 11:10