Memory-Augmented Architecture for Long-Term Context Handling in Large Language Models

Abstract

Large language models (LLMs) struggle to maintain coherent, contextually-aware interactions over long dialogues due to the inherent limitations of fixed-length context windows. This leads to fragmented conversations, loss of essential information, and diminished user experience. We address this challenge by introducing a memory-augmented architecture with a dynamic, relevance-based pruning mechanism that evaluates and manages stored information based on its semantic importance to the ongoing conversation, rather than relying on simplistic temporal heuristics. Our approach enables intelligent retrieval, updating, and pruning of memory, ensuring that contextually vital information is retained. Extensive experiments across eight benchmarks demonstrate that our method improves long-term contextual coherence, achieves a 7.6% relative accuracy gain, and reduces memory overhead by 24.6% compared with state-of-the-art baselines.