Enhancing LLMs in Predictive Political QA with Semi-Structured Data
Proposed PSL framework combines semi-structured political records' stance and structural signals, significantly improving predictive political QA.
Key Findings
Methodology
The dual-view PSL framework transforms semi-structured political records into inference evidence. The semantic view extracts issue-specific stance signals from relevant profiles, while the vector view constructs an actor interaction graph to learn high-order dependency signals. These signals are integrated via co-embedding and lightweight fine-tuning, enhancing large language models’ reasoning for predictive tasks. The process involves building actor profiles, retrieving relevant records through ladder retrieval, distilling stance signals with a trained small language model, constructing and propagating in the interaction graph, and finally fusing signals into the LLM for inference.
Key Results
- Across three datasets (RCVP, ICEWS, StaId) and multiple LLMs (e.g., Llama-3.1-8B, Mistral-7B), PSL consistently outperforms baselines, with improvements of 10-20% in macro F1 scores. For instance, on RCVP, PSL achieves 55.92, surpassing PEGexp_GT R (41.21).
- Ablation studies show removing stance signals drops performance by ~15%, removing structure signals drops ~10%, confirming their complementary roles. Increasing retrieved records up to 15 and two propagation layers yields optimal results, indicating the importance of rich context and structural information.
- Compared to pure knowledge graph or profile-based methods, PSL leverages the detailed semi-structured data to capture implicit preferences and indirect dependencies, significantly boosting future event prediction accuracy and demonstrating strong practical potential.
Significance
This work advances political AI by integrating multi-level signals from semi-structured data, overcoming limitations of static knowledge bases. Its dual-view design and signal fusion significantly improve reasoning in predictive political tasks, offering new avenues for policy analysis, electoral forecasting, and decision support. The approach enhances the interpretability and robustness of large models in complex, real-world scenarios, marking a step toward more intelligent political systems.
Technical Contribution
The paper introduces a novel dual-view framework that systematically extracts and fuses semantic stance signals and high-order structural signals from semi-structured political records. It employs ladder retrieval, graph convolution, and collaborative embedding techniques, combined with lightweight fine-tuning (LoRA), to embed structural and behavioral cues into large models. This integrated approach differs from prior methods by jointly modeling issue-specific preferences and indirect dependencies, providing a comprehensive evidence base for inference.
Novelty
This is the first work to unify semi-structured political data's dual signals—semantic and structural—within a single framework for predictive reasoning. Unlike previous approaches focusing solely on knowledge graphs or profile simulation, PSL leverages detailed data to extract implicit preferences and high-order relationships, offering a more complete and nuanced inference mechanism.
Limitations
- Dependence on high-quality semi-structured data; noisy or incomplete records can impair performance.
- Graph propagation complexity limits scalability in very large networks, affecting efficiency.
- Handling dynamic political environments and multi-modal data remains challenging, requiring further research.
Future Work
Future directions include integrating multi-modal data such as images and videos, developing dynamic graph models to adapt to evolving political contexts, and scaling the approach for real-time prediction. Additionally, exploring unsupervised or semi-supervised signal extraction could reduce reliance on labeled data, broadening applicability.
AI Executive Summary
Predictive political question answering (QA) is a complex task that requires understanding not only explicit facts but also implicit preferences and indirect dependencies among political actors. Traditional approaches rely heavily on factual retrieval or knowledge graphs, which often fall short in capturing the nuanced signals needed for accurate future behavior prediction.
This paper introduces PSL, a dual-view framework that leverages semi-structured political records to enhance large language models’ reasoning capabilities. The core idea is to extract two complementary signals: actor stance signals, which reflect issue-specific preferences, and high-order structure signals, which encode indirect dependencies within political networks. The semantic view retrieves relevant records and distills stance signals via a trained small language model, while the vector view constructs an interaction graph among actors and propagates information through graph convolution layers.
By integrating these signals through co-embedding and lightweight fine-tuning, PSL provides a richer, more context-aware evidence base for models. Extensive experiments across three real-world datasets and multiple LLMs demonstrate that PSL outperforms existing baselines by significant margins, with improvements up to 20% in macro F1 scores. Ablation studies confirm the importance of both signals, highlighting their mutual complementarity.
This approach marks a significant step forward in political AI, addressing the limitations of static knowledge bases and single-source signals. Its ability to incorporate detailed behavioral and relational information offers promising applications in policy analysis, electoral prediction, and automated decision support. Despite current challenges related to data quality and scalability, future work aims to extend PSL to multi-modal, dynamic, and real-time scenarios, pushing the boundaries of intelligent political reasoning.
Deep Dive
Abstract
Predictive political question answering (QA), such as predicting how a political actor will vote, goes beyond factual lookup. External political resources offer rich historical evidence, but rarely contain the answer itself. Existing LLM augmentation methods, including actor-profile-based simulation and knowledge graph evidence injection, improve political reasoning but largely treat external resources as knowledge-based evidence, leaving prediction-relevant signals under-modeled. We identify two complementary signals for predictive political QA: actor stances that capture issue-specific preferences, and high-order structure signals that capture indirect dependencies among political actors. We propose PSL, a dual-view framework that converts semi-structured political records into inference-oriented evidence for LLMs. PSL extracts stance signals from question-relevant actor records in a semantic view, and learns structure-aware actor representations from an actor interaction graph in a vector view. Across three real-world datasets and multiple LLMs, PSL consistently outperforms baselines, with ablations confirming the complementary gains of stance and structure signals.