Latent Personal Memory: Represent personal memory as dynamic soft prompts
LPM uses dynamic soft prompts for personalization, improving accuracy by 8.8% on PersonaMem v1, reducing KV-cache by 64x.
Key Findings
Methodology
LPM represents user history as a compact matrix of N latent slots, using a shared cross-attention projection network to map these slots into dynamic input-conditioned soft prompts, prepended to a frozen LLM input. This allows personalization without modifying the base model.
Key Results
- On PersonaMem v1, LPM outperforms LoRA by 8.8% and Prompt Tuning by 54.4%, while reducing KV-cache usage by 64x.
- On LoCOMO, LPM matches LoRA's accuracy with 120x fewer trainable parameters.
- LPM outperforms full-context methods in efficiency at 128K context length.
Significance
LPM is significant for academia and industry, addressing computational efficiency and storage demands in large-scale personalization while maintaining model generality. It offers a scalable approach to personalizing large language models.
Technical Contribution
LPM compresses user history into latent memory slots, avoiding base model modification, and offers a new parameter-efficient fine-tuning method. It significantly reduces KV-cache usage without increasing training complexity.
Novelty
LPM is the first to represent user history as latent memory slots and achieve personalization through dynamic soft prompts. This fundamentally differs from traditional text-space memory methods, providing a more efficient personalization solution.
Limitations
- LPM performs poorly in memorizing precise facts, potentially requiring integration with text-based memory methods.
- The ability of memory slots to capture long- and short-term information may need further optimization.
Future Work
Future research can explore hybrid memory methods, combining explicit fact caches to enhance precise memory capabilities, and optimize memory slots for better long- and short-term user information capture.
AI Executive Summary
Personalizing large language models requires encoding user-specific long-term behavioral patterns while maintaining computational efficiency and model generality. Existing methods struggle to meet these requirements simultaneously. We introduce Latent Personal Memory (LPM), which represents user history as a compact matrix of latent slots and uses a shared cross-attention projection network to map them into dynamic soft prompts, prepended to a frozen LLM input. Experiments show that LPM performs excellently on PersonaMem v1 and LoCOMO benchmarks, significantly reducing KV-cache usage and trainable parameters. LPM also outperforms full-context methods in efficiency at long context lengths. This research provides a scalable and efficient approach to personalizing large language models, with significant academic and industrial implications. Future research can explore hybrid memory methods to enhance precise memory capabilities and optimize memory slots for better user information capture.
Deep Analysis
Background
Personalizing large language models requires encoding user-specific long-term behavioral patterns while maintaining computational efficiency and model generality. Existing methods like LoRA and Prompt Tuning have limitations in parameter efficiency and expressivity.
Core Problem
Existing personalization methods struggle to meet the requirements of computational efficiency, scalability, and model generality. Especially when handling long contexts and user histories, existing methods face high computational costs and storage demands.
Innovation
LPM represents user history as a compact matrix of latent slots and uses a shared cross-attention projection network to map them into dynamic soft prompts, achieving efficient personalization. This method avoids modifying the base model and significantly reduces KV-cache usage.
Methodology
- �� Represent user history as a compact matrix of latent slots
- �� Use a shared cross-attention projection network
- �� Map latent slots into dynamic soft prompts
- �� Prepend to a frozen LLM input
Experiments
Evaluated on PersonaMem v1 and LoCOMO benchmarks using Qwen3-1.7B, 4B, 8B models. Baselines include LoRA and Prompt Tuning, with evaluation metrics including accuracy and KV-cache usage.
Results
LPM outperforms LoRA by 8.8% and Prompt Tuning by 54.4% on PersonaMem v1, reducing KV-cache usage by 64x. On LoCOMO, LPM matches LoRA's accuracy with 120x fewer trainable parameters.
Applications
LPM is suitable for applications requiring efficient personalization of large language models, such as personalized recommendations and intelligent assistants. Its low storage demands and high computational efficiency make it suitable for large-scale deployment.
Limitations & Outlook
LPM performs poorly in memorizing precise facts, potentially requiring integration with text-based memory methods. The ability of memory slots to capture long- and short-term information may need further optimization.
Plain Language Accessible to non-experts
Imagine a library where LPM acts like a smart librarian who can quickly find the books you need without having to search the entire library each time. It's like having a small notebook that records the titles and locations of the books you most frequently read. When you need a book, the librarian uses this notebook to find the relevant books based on your needs and quickly hands them to you. This method saves time and doesn't require much space.
ELI14 Explained like you're 14
Imagine you have a super memory friend at school who remembers all your likes and habits. Every time you have a question, they can give the best advice based on your past performance without relearning. That's how LPM works! It's like a smart assistant that quickly understands your needs and provides the best answers. Isn't that cool?
Glossary
Latent Personal Memory
A method that represents user history as a compact matrix of latent slots, enabling efficient personalization.
Used in the paper to represent user-specific long-term behavioral patterns.
Dynamic Soft Prompts
Prompts generated dynamically based on input conditions, prepended to a large language model input.
A key component for achieving personalization.
Cross-attention Projection Network
A network used to map latent slots into dynamic soft prompts.
Used in LPM to achieve dynamic mapping of user history.
KV-cache
A cache storing key-value pairs used in self-attention mechanisms.
Reduced in usage by LPM through latent slots.
LoRA
A parameter-efficient fine-tuning method that trains small user-specific weights for personalization.
Used as a baseline comparison method in LPM.
Open Questions Unanswered questions from this research
- 1 How to integrate explicit fact caches to enhance LPM's precise memory capabilities?
- 2 How to optimize memory slots for better long- and short-term user information capture?
Applications
Immediate Applications
Personalized Recommendations
LPM can be used in personalized recommendation systems, improving the accuracy and relevance of recommendations through dynamic soft prompts.
Long-term Vision
Intelligent Assistants
LPM can be applied to intelligent assistants, enhancing user interaction experiences through efficient personalization.
Abstract
Personalizing large language models (LLMs) requires encoding long-term, user-specific behavioral patterns in a way that is computationally efficient, scalable, and compatible with a frozen base model. We present Latent Personal Memory (LPM), a scalable framework that represents user-specific history as a compact, persistent matrix of N latent slots, that are interpretable. A shared cross-attention projection network maps these slots into dynamic, input-conditioned soft prompts that are prepended to the input of a frozen LLM. We evaluate LPM on PersonaMem v1 and LoCOMO benchmarks across Qwen3-1.7B, 4B, and 8B backbones. Results demonstrate that LPM outperforms LoRA and Prompt Tuning by up to 8.8% and 54.4% in overall accuracy respectively on PersonaMem v1, while reducing KV-cache usage by over 64x. On LoCoMo, LPM matches LoRA accuracy with 120x fewer trainable parameters. We also show that the efficiency of LPM grows with context length and outperforms full-context at 128K context length.