When Only Time Will Tell: Interpreting How Transformers Process Local Ambiguities Through the Lens of Restart-Incrementality

The evolution of NLP models from RNNs to Transformers has significantly improved language understanding. Early models like LSTM-based RNNs introduced incremental processing but lacked effective mechanisms for revising earlier outputs. The advent of Transformer architectures, especially BERT and RoBERTa, enabled deep bidirectional context integration, boosting performance in tasks like question answering and dependency parsing. However, these models are often treated as black boxes, with limited insight into their internal state dynamics during incremental processing. Recent efforts, such as Ulmer et al. (2019) and Ferrando et al. (2023), have begun exploring how internal representations evolve, but primarily focus on static or layer-wise attention analysis. The gap remains in understanding how models internally revise their interpretations when encountering local ambiguities, a process crucial for cognitive plausibility and robustness. This paper addresses this gap by proposing a structured, state-based analysis framework, aiming to elucidate the internal revision mechanisms of bidirectional transformers during incremental language understanding.

Despite their success, the internal mechanisms enabling bidirectional models to revise earlier interpretations remain poorly understood. Existing analyses are limited to output probabilities, attention weights, or layer activations, which do not fully capture the dynamic state adjustments during ambiguity resolution. This opacity hampers interpretability, trust, and targeted improvements. The core challenge is developing methods to visualize and quantify how internal states evolve at ambiguity points, especially in real-time, and how these updates correlate with output corrections. Without such insights, the models' cognitive plausibility and capacity for self-correction remain speculative, limiting their deployment in sensitive applications requiring transparency and reliability.

The key innovation is the formalization of a structured state sequence framework that captures the internal dynamics of restart-incremental bidirectional transformers. This involves constructing a 3D matrix representing states across layers, tokens, and time steps, allowing detailed tracking of state evolution. The framework employs cosine similarity and entropy metrics to quantify how states change at ambiguity points, revealing the internal path of reanalysis. Unlike prior work that focuses on softmax outputs or attention distributions, this approach directly visualizes the internal representation shifts, providing a transparent view of the revision process. It bridges the gap between cognitive theories of language processing and neural network mechanisms, offering a new tool for interpretability and model diagnostics.

• �� Extract hidden states from pretrained bidirectional models (e.g., BERT, RoBERTa) at each layer for each token during incremental input processing.
• �� Organize these states into a three-dimensional matrix, with dimensions corresponding to layers, tokens, and time steps.
• �� Compute pairwise cosine distances between states at successive time steps to measure how much each token's representation changes as new context arrives.
• �� Calculate entropy of attention scores and dependency arcs to assess structural updates.
• �� Use controlled stimuli with known local ambiguities (NP/S, MVRR) and baseline unambiguous sentences for comparison.
• �� Analyze the temporal evolution of states, focusing on disambiguation points, to identify patterns of revision and reanalysis.

The experiments utilize datasets containing local ambiguities, such as Huang et al. (2023), with sentences designed to trigger garden path effects. The models evaluated include BERT, RoBERTa, and GPT-2, with internal states extracted at each layer during incremental processing. The analysis involves comparing state trajectories at disambiguation points, measuring cosine distances and entropy changes. Baseline sentences without ambiguities serve as controls. The study examines how state revisions correlate with output corrections, focusing on the layers where the most significant changes occur. Results consistently show that bidirectional models exhibit larger state adjustments at disambiguation points compared to causal models, confirming their superior reanalysis capabilities.

The findings demonstrate that bidirectional models dynamically update their internal states during ambiguity resolution, with cosine distance increases exceeding 0.15 at disambiguation points. These state changes are strongly correlated with output revisions, especially in dependency parsing accuracy improvements of 3-5%. The analysis reveals that state adjustments are layer-dependent, with middle and upper layers showing more pronounced revisions. The results validate that internal state dynamics are integral to the models' ability to reanalyze and correct earlier interpretations, providing a mechanistic explanation for their superior performance in ambiguous contexts.

This interpretability framework can be employed to diagnose and improve NLP systems, particularly in dialogue, translation, and question answering tasks where ambiguity is common. It enables developers to visualize how models internally revise their understanding, facilitating targeted enhancements. Additionally, the insights can inform cognitive models of language processing, bridging AI and human cognition. In industry, this approach can enhance transparency and trustworthiness of AI systems, especially in high-stakes applications like legal or medical NLP.

The current approach relies on internal state extraction, which may vary across architectures and training regimes, limiting generalizability. Computational costs increase with sentence length and model size, restricting real-time deployment. The analysis is primarily validated on English, and its applicability to other languages with different syntactic structures remains to be tested. Moreover, high-dimensional state visualization can be complex, necessitating more intuitive tools for broader adoption.