Transition-Matrix Regularization for Next Dialogue Act Prediction in Counselling Conversations

Next Dialogue Act Prediction (NDAP) is a critical task in dialogue systems, aiming to forecast the communicative function of the next utterance based on dialogue history. In traditional dialogue research, classical dialogue managers explicitly modeled these transitions using structures like Markov models or Conditional Random Fields (CRFs). However, with the advent of neural network technologies, modern dialogue systems have shifted towards end-to-end architectures, attempting to infer discourse structure implicitly. While this shift increases model flexibility, it also removes the inductive bias of dialogue act transitions, leading to limited signals when multiple next acts are plausible.

In counseling and other highly structured domains, dialogue acts often follow consistent pragmatic patterns, such as greetings typically preceding problem statements, exploration preceding intervention, and closing behaviors following resolution. These patterns were explicitly modeled in traditional dialogue managers but are often overlooked in modern neural systems. Neural models typically see only a single gold next-act label per instance, which is a common situation in counseling dialogues. The gold label in NDAP is inherently under-specified, representing one observed continuation among many valid possibilities. As a result, standard cross-entropy supervision penalizes the model for predicting other plausible acts.

The core innovation of this paper is the proposal of a transition-matrix regularization method to improve next dialogue act prediction in counseling conversations. Specifically, the method incorporates a KL divergence regularization term into the loss function, aligning predicted act distributions with transition patterns derived from the corpus. This approach not only enhances prediction accuracy but also improves dialogue-flow alignment. Unlike traditional CRF or HMM models, this method does not require sequence decoding, and unlike posterior regularization approaches, the structural prior is directly grounded in observed dialogue act transitions.

• �� The method incorporates a KL divergence regularization term into the loss function, aligning predicted act distributions with transition patterns derived from the corpus.
• �� Evaluated in German text-based counseling, where communicative actions are fine-grained and governed by psychosocial norms.
• �� The dataset uses a five-level taxonomy with 60 dialogue act categories.
• �� NDAP is performed across all speaker transitions.
• �� To exploit the taxonomy structure, category history-augmented architectures are introduced.
• �� Results show that transition-based regularization provides consistent gains and disproportionately benefits weaker models.

Experiments were conducted on a 60-class German counseling taxonomy using 5-fold cross-validation to evaluate the model's performance. To validate the method's transferability across languages and counseling domains, cross-dataset validation was performed on the HOPE dataset. The experimental design included comparisons with multiple baseline models, including simple RNNs, the architecture proposed by Tanaka et al., and zero-shot LLM baselines. To validate the robustness of different pretrained language models, all neural baselines and history-aware models were tested using 7 different German BERT variants.

The experimental results show that the transition-matrix regularization method improved macro-F1 scores by 9-42% relative to baseline models, with specific improvements depending on the encoder used. Cross-dataset validation on the HOPE dataset demonstrated that the proposed method transfers well across languages and counseling domains, with a 3.2% improvement in macro-F1 and a 33% reduction in JS divergence. Systematic ablation studies indicate that transition regularization provides consistent gains, especially benefiting weaker baseline models in data-sparse dialogue tasks.

The method can be directly applied to the development of counseling dialogue systems, especially in scenarios requiring precise prediction of the next dialogue act. By introducing lightweight discourse-flow priors, the method complements pretrained encoders, improving the robustness and adaptability of dialogue systems. Additionally, the method can be applied to other highly structured dialogue domains, such as medical dialogues and educational dialogues.

Despite the method's strong performance in multiple experiments, it may face increased computational complexity when dealing with a large number of dialogue act categories, especially on large-scale datasets. Additionally, since the construction of the transition matrix relies on the statistical information of the training dataset, the model's generalization ability may be affected when there is a significant change in data distribution. Future research directions include exploring the application of this method on larger datasets and investigating how to integrate other types of dialogue priors to further enhance prediction performance.