Syntactic Belief Update as the Driver of Garden Path Processing Difficulty

Psycholinguistics has long sought to model sentence comprehension, with surprisal theory (Hale, 2001; Levy, 2008) emphasizing lexical prediction as a key factor. These models successfully predict general reading times and neural responses but struggle with syntactic ambiguities like garden path sentences, where initial interpretations are later revised. Recent advances incorporate syntactic structures, yet most focus on static probabilities or lexical features, neglecting the dynamic process of reanalysis. The need for a model that explicitly captures the cognitive effort involved in syntactic restructuring has driven research toward belief update mechanisms, integrating information theory with syntactic parsing. This paper builds on these foundations, proposing a belief update framework grounded in dependency parsing and Rényi divergence, aiming to better reflect human processing costs during syntactic reanalysis.

Existing models, primarily based on lexical surprisal, underestimate the cognitive load associated with garden path sentences. They fail to account for the reanalysis process, which involves updating internal syntactic beliefs when encountering disambiguation cues. This gap limits our understanding of the temporal dynamics of sentence processing and hampers the development of more accurate predictive models. The core challenge is to quantify the cognitive effort involved in restructuring syntactic hypotheses, which is crucial for explaining the pronounced slowdowns observed in human reading data. Addressing this requires a shift from static probability measures to dynamic belief update metrics that directly reflect syntactic restructuring costs.

The key innovation is the introduction of a syntactic belief update metric based on Rényi divergence, which measures the change in probability distribution over syntactic trees after each word. This approach emphasizes structural shifts rather than lexical probabilities, capturing the cognitive cost of reanalysis more faithfully. The model employs a non-projective dependency parser trained on multiple English treebanks, leveraging the matrix-tree theorem for efficient likelihood computation. By tuning the α parameter, the model can emphasize different aspects of belief change, with α=2 providing optimal correlation with human data. This framework extends traditional surprisal models, offering a more granular and structure-aware measure of processing difficulty.

• �� Develop a probabilistic model over dependency trees using a non-projective parser trained on EWT, GUM, and other treebanks.
• �� Implement Bayesian updates to the syntactic belief distribution after each word, reflecting the human reanalysis process.
• �� Calculate belief update magnitude using Rényi divergence Dα(p||q), with q as prior and p as posterior distribution.
• �� Tune α parameter to balance sensitivity to structural shifts, with α=2 yielding best results.
• �� Simulate incremental reading by masking future words, mirroring self-paced reading experiments.
• �� Evaluate the model against human reading time data from SAP, focusing on critical disambiguation regions, and compare with baseline surprisal metrics.

The experimental setup uses the SAP dataset, comprising self-paced reading times from 2000 participants on sentences with NP/S, NP/Z, and MV/RR ambiguities. The model computes belief update metrics at each disambiguation point, correlating these with observed reading delays. Baselines include lexical surprisal from RoBERTa and syntactic surprisal based on supertag predictions. Linear mixed-effects models control for confounding factors like word length and frequency, assessing the predictive power of belief updates. Results show that the Rényi divergence-based metric, especially at α=2, achieves a correlation of 0.65 with human data, outperforming traditional surprisal by 20%. The analysis confirms the importance of structural belief shifts in processing difficulty.

The belief update metric correlates strongly with human reading times across different sentence types, with a maximum correlation of 0.65 at α=2. The model captures the hierarchy of garden path difficulty, with higher divergence at disambiguation points. Parameter tuning reveals that emphasizing structural shifts improves prediction accuracy. The results demonstrate that syntactic belief changes, rather than lexical predictability, better explain the cognitive load during reanalysis, providing a more precise tool for modeling sentence processing.

This framework can enhance NLP applications such as syntactic parsing, language modeling, and real-time language understanding systems. It offers insights for designing cognitive models that mimic human reanalysis, useful in educational tools and language disorder diagnostics. Additionally, it can inform the development of more human-like AI systems capable of handling syntactic ambiguities in natural language understanding tasks.

The model relies on dependency parsing accuracy, which may be limited in complex or low-resource languages. Computational costs are significant, especially for large-scale or real-time applications. Validation is currently restricted to English, requiring adaptation for other languages with different syntactic structures. Future work must address these issues to improve scalability and cross-linguistic applicability.