Personalized Generation In Large Model Era: A Survey

Personalized Generation (PGen) is an important direction in content generation in the era of large models, aiming to tailor content to individual preferences and needs. Recent advancements in large generative models have shifted content generation from generic, one-size-fits-all approaches to personalized generation, attracting significant attention in fields such as natural language processing, computer vision, and information retrieval. Despite independent developments within different communities, there is a lack of a unified cross-community framework.

The core problem of personalized generation is effectively understanding and meeting personalized user needs. Due to distinct data structures and challenges across modalities, personalized generation requires innovations in user modeling and guidance mechanisms for generative models to achieve high-quality, instruction-aligned, and personalized content generation.

The core innovations of this paper include introducing a unified user-centric perspective that conceptualizes personalized generation, systematically formalizing its key components, core objectives, and abstract workflows. Based on this perspective, the paper proposes a multi-level taxonomy that integrates personalized generation research across modalities, offering new theoretical guarantees and engineering possibilities.

• �� User Modeling: Captures user preferences and specific content needs using techniques such as representation learning, prompt engineering, and retrieval-augmented generation.

• �� Generative Modeling: Selects appropriate foundation models (e.g., LLMs, MLLMs, DMs) and employs guidance mechanisms and optimization strategies to generate personalized content.

• �� Guidance Mechanism: Integrates personalized signals using instruction guidance (e.g., in-context learning, instruction tuning) and structural guidance (e.g., adapters, cross-attention).

• �� Optimization Strategy: Includes tuning-free methods, supervised fine-tuning, and preference-based optimization strategies.

The experimental design includes comparisons using various datasets and baseline models to evaluate the performance of personalized generation models across different modalities and tasks. Key hyperparameters and ablation studies are also discussed in detail to validate the effectiveness and robustness of the models.

Experimental results demonstrate that personalized generation models perform exceptionally well in text, image, and multimodal generation tasks. In text generation, models achieve a 15% accuracy improvement; in image generation, generated images show a 20% increase in alignment with user historical queries; in multimodal generation, user satisfaction scores improved by 25%.

Application scenarios for personalized generation include customized product images in e-commerce, personalized advertisements in marketing campaigns, and personalized AI assistants. These applications enhance user experience and engagement, with broad industry impact.

Personalized generation models face high computational complexity when handling multimodal data. Additionally, accuracy in certain domains requiring high precision remains a challenge. Future research needs to address these issues and establish standardized evaluation frameworks.