LifeSentence: Language models can encode human life course trajectories from longitudinal panel data
LifeSentence leverages pretrained language models with structured event data to predict and analyze human life trajectories, outperforming baselines with 35.3% joint accuracy.
Key Findings
Methodology
LifeSentence employs a pretrained 24-billion-parameter language model (Mistral Small 3.1), fine-tuned via instruction-based multi-task learning on 18 tasks including prediction, robustness, and reasoning. Input comprises structured natural language event sequences, task instructions, and prompts, outputting structured JSON predictions. Data from the German Socio-Economic Panel (SOEP) with 65,000 individuals provides the training corpus. The model leverages semantic knowledge embedded during pretraining to enhance predictive accuracy, capturing complex temporal dependencies and social patterns. Multi-task training enables event forecasting, trajectory generation, anomaly detection, and sequence reordering, demonstrating comprehensive understanding of life-course structure.
Key Results
- LifeSentence achieves a joint accuracy of 35.3% in next-event prediction, with a temporal mean absolute error of 1.15 years, surpassing all baselines by over three times. Its ability to reconstruct chronological order yields a Kendall's tau of 91.2%, indicating strong structural understanding. The model reproduces known social stratification patterns such as education premium, gender wage gap, and motherhood penalty without explicit supervision.
- In long-term trajectory generation, LifeSentence attains a Jaccard similarity of 0.627 and Levenshtein distance of 8.10, outperforming XGBoost and Transformer, maintaining event diversity and realistic timing. It satisfies over 99% of constraints in targeted generation tasks (e.g., predicting to a specific year or event). The model also excels in anomaly detection (accuracy 93.9%) and event ordering, confirming its grasp of event causality and sequence logic.
- Ablation studies show key events like military service and education serve as structural anchors; removing them significantly degrades prediction accuracy. The model accurately recovers socioeconomic patterns, such as income stratification by education and gender, and captures the motherhood penalty, demonstrating deep internalization of social phenomena.
Significance
This work advances human life trajectory modeling by integrating large-scale pretrained language models with structured event data, overcoming data scarcity and enhancing interpretability. It addresses long-standing challenges in social science prediction, enabling nuanced understanding of social stratification, causal relationships, and counterfactual scenarios. The approach broadens AI applications in social sciences, public health, and policy-making, providing a versatile tool for analyzing complex human behaviors and societal patterns with minimal supervision. Its multi-task framework and natural language interface set new standards for interpretability and flexibility in modeling human lives.
Technical Contribution
The key innovation lies in adapting pretrained language models for sequential life data through structured natural language event encoding and instruction tuning across multiple tasks. This approach leverages semantic embeddings learned during pretraining, reducing the need for large labeled datasets. The multi-task training framework incorporates prediction, robustness, and reasoning tasks, enabling the model to understand causal and temporal dependencies. The use of instruction-based fine-tuning with a structured input-output format enhances model generalization and interpretability. This methodology bridges NLP advances with longitudinal life data, opening new avenues for scalable, interpretable human trajectory modeling.
Novelty
This is the first application of large-scale pretrained language models to long-term human life trajectory prediction, utilizing structured natural language event descriptions to achieve multi-task reasoning. Unlike prior models requiring extensive labeled data, LifeSentence exploits pretraining knowledge, enabling high performance with limited data. Its ability to reproduce complex social stratification patterns without explicit supervision demonstrates a profound understanding of social and temporal structures, marking a significant departure from traditional statistical or sequence models.
Limitations
- The model's performance diminishes with extremely sparse or anomalous events, especially when key structural events are missing, indicating sensitivity to data completeness.
- Transferability across different cultural or social contexts remains untested; adaptation to other countries or systems may require retraining or fine-tuning.
- Computational costs for training and inference are substantial, limiting real-time deployment and large-scale applications.
Future Work
Future research will explore integrating multimodal data (images, sensor data) to enrich life-course modeling, incorporating causal inference for counterfactual analysis, and extending the framework to diverse cultural contexts. Enhancing model efficiency and interpretability, as well as developing user-friendly interfaces for social scientists and policymakers, are also key directions. These efforts aim to make LifeSentence a versatile, scalable tool for understanding and shaping human life trajectories globally.
AI Executive Summary
Deep Dive
Plain Language Accessible to non-experts
Imagine you have a big, detailed diary of a person's life, filled with notes about school, jobs, family, health, and more. Now, you want a smart robot that can read this diary and understand the story of that person's life. This robot learns from thousands of such diaries, understanding how different events are connected—like how finishing school leads to a job, or how having children affects health. Once it learns, it can predict what might happen next in someone's life, or tell you how different choices could change their future. It can even find patterns, like how people with higher education tend to earn more, or how becoming a parent might lower a woman's income temporarily. This way, the robot helps us see the big picture of human lives, making predictions and exploring 'what-if' scenarios, just like a wise friend who knows a lot about life stories.
ELI14 Explained like you're 14
Think of a super-smart friend who has read thousands of people's life stories. This friend can look at what someone has done so far—like going to school, working, getting married, having kids—and then guess what might happen next. Not only that, but this friend can also tell you how different choices might change the future. For example, if someone keeps studying, they might get a better job later. Or if a woman has a baby, her income might drop for a while. This friend understands the patterns in people's lives without needing to be told explicitly—it's like they have a special sense for how life usually goes. They can also spot unusual events, like a sudden illness, and tell you how likely it is to happen. Basically, this is a very clever robot that learns from many stories and helps us understand and predict human lives better than ever before.
Glossary
Pretrained Language Model (预训练语言模型)
A model trained on large text data to understand and generate language, such as GPT or BERT; it captures semantic relationships.
In this paper, it provides background knowledge to improve life trajectory predictions.
Structured Event Description (结构化事件描述)
A detailed natural language record of life events, including time, type, and details.
Used to encode individual biographies for the model.
Instruction Tuning (指令微调)
Fine-tuning a pretrained model on specific tasks using explicit instructions to improve task performance.
Enables multi-task capabilities in LifeSentence.
Kendall's Tau (肯德尔相关系数)
A statistic measuring the agreement between two rankings, from -1 to 1.
Evaluates how well the model reconstructs chronological order.
Jaccard Similarity (Jaccard相似度)
A measure of similarity between two sets, calculated as intersection over union.
Used to compare generated and real life event sequences.
Open Questions Unanswered questions from this research
- 1 如何让模型更好地理解极端稀疏或异常事件，特别是在关键事件缺失时的表现。
- 2 模型在不同文化背景下的迁移能力未充分验证，跨国应用仍需探索。
- 3 模型训练和推理成本较高，限制了大规模部署和实时应用。
Abstract
Forecasting human life outcomes is important to gain insights into how individuals attain long and healthy lives. Conventional statistical approaches yield limited accuracy, potentially due to discarding the sequential structure of the life course. Modern methods such as transformer architectures require large scale training data that most longitudinal panel studies lack. Here we introduce LifeSentence, a model for life-course reasoning that bridges large language models with longitudinal panel data. By representing each life event as a structured natural-language record and instruction-tuning a pretrained 24-billion-parameter language model across an 18-task evaluation taxonomy spanning prediction, robustness and reasoning, LifeSentence supplements panel data with distributional knowledge already encoded during pretraining. Trained on approximately 65,000 individuals from the German Socio-Economic Panel - roughly 45 times fewer than prior transformer-based approaches - LifeSentence outperforms classical and deep learning baselines across all task families, achieving a threefold improvement in joint event-and-timing prediction from best baselines and 91.2% Kendall's tau when reconstructing chronological order from timestamp-stripped event sets. Without explicit supervision, the model recovers documented patterns of social stratification, including the education premium, the gender wage gap and the motherhood penalty, from discrete event sequences alone. A natural-language interface further enables qualitatively new research queries, such as connecting an early-life history to a specified late-life endpoint, establishing LifeSentence as both a predictive tool and a probe for counterfactual exploration of human biographies.