GptGet

LLM as a Broken Telephone: Iterative Generation Distorts Information

TL;DR

This study uses translation chains to reveal iterative generation causes information distortion, with key metrics showing increased deviation over rounds.

cs.CL 🔴 Advanced 2025-02-28 51 views
Amr Mohamed Mingmeng Geng Michalis Vazirgiannis Guokan Shang
AI Reader Arxiv Page Download PDF
Large Language Models Information Distortion Translation Chains Iterative Generation Model Robustness

Key Findings

Methodology

The research employs multi-round translation experiments using Llama-3.1-8B-Instruct and Mistral-7B, involving multiple languages (e.g., French, German, Thai). The process simulates iterative information transfer, measuring relevance and factuality with BLEU, ROUGE, CHR-F, METEOR, BERTScore, and FActScore. Experiments include single-model self-loops, multi-model collaborations, and increased chain complexity, systematically analyzing how information degrades across rounds. The methodology integrates temperature control and prompt restrictions to evaluate mitigation strategies.

Key Results

  • Results show that information distortion accumulates over rounds, with Latin-script languages like EN-FR maintaining higher factuality (FActScore gradient ~-0.004), while non-Latin scripts like Thai degrade faster (gradient ~-0.04). Increasing chain complexity (more languages/models) amplifies degradation, with the highest complexity reaching a FActScore decline rate of -0.038. Temperature adjustments and prompt constraints slow but do not eliminate distortion.
  • Multi-model cooperation sometimes worsens or slightly improves factuality depending on language pair, with collaborative chains in French showing minimal additional degradation (gradient ~-0.007), whereas Thai chains degrade faster (gradient ~-0.035).
  • Overall, higher chain complexity correlates with faster information loss, emphasizing the importance of prompt strategies and model design to mitigate cumulative errors.

Significance

This work highlights fundamental limitations of current LLMs in multi-round information propagation, raising awareness of risks in AI-mediated content dissemination. It informs future development of robust models and prompts, crucial for applications like cross-lingual communication, content verification, and multi-agent systems. The findings underscore the necessity of designing mechanisms to preserve factuality and relevance over iterative workflows, addressing long-standing challenges in AI reliability and trustworthiness.

Technical Contribution

The paper introduces a comprehensive framework for analyzing information degradation through multi-layered translation chains, combining quantitative metrics and experimental validation. It innovates by integrating multi-model and multi-language scenarios, revealing how chain complexity and model collaboration influence information fidelity. The approach offers a systematic way to quantify and understand the dynamics of iterative distortion, providing a foundation for developing more resilient AI systems.

Novelty

This is the first systematic study quantifying how multi-round translation and model collaboration affect information fidelity in large language models. It uniquely combines multi-language, multi-model, and complexity analyses, demonstrating that iterative processes inherently induce cumulative errors. The work advances beyond prior single-task or single-model assessments, offering new insights into the stability and reliability of AI-generated content in complex workflows.

Limitations

  • The experiments focus on specific models (Llama, Mistral) and selected languages; broader generalization requires testing additional models and languages.
  • The study centers on translation tasks, leaving other generative tasks like summarization or Q&A less explored in the context of iterative distortion.
  • While prompt tuning mitigates some degradation, it does not solve the fundamental issue; future work should explore model training and architecture improvements.

Future Work

Future research will extend analysis to multi-modal and multi-task scenarios, aiming to develop models with built-in robustness against information drift. Incorporating human feedback and reinforcement learning could enhance factuality preservation. Additionally, exploring training strategies that explicitly penalize divergence over multiple rounds will be crucial for building trustworthy AI systems capable of long-term information consistency.

AI Executive Summary

The rapid adoption of large language models (LLMs) in content creation and information dissemination has raised critical questions about their reliability over multiple rounds of generation. This study investigates how iterative processes, such as multi-language translation chains, influence the fidelity of information. Using models like Llama-3.1-8B and Mistral-7B, researchers designed experiments involving multiple languages—French, German, Thai—and varied chain complexities to simulate real-world scenarios like cross-lingual news transmission.

The core methodology involved translating documents through sequences of languages and models, measuring how relevance and factuality deteriorate over rounds. Metrics like BLEU, ROUGE, CHR-F, METEOR, BERTScore, and FActScore provided a comprehensive evaluation of semantic and factual integrity. Results consistently showed that with each iteration, small deviations accumulate, leading to significant information loss, especially in non-Latin scripts like Thai. Chain complexity and model collaboration further exacerbate this degradation.

These findings carry profound implications for AI deployment in critical fields. They highlight the necessity of strategic prompt design and model architecture improvements to mitigate long-term information drift. While simple prompt tuning offers some relief, it cannot fully prevent the inherent instability of iterative generation. The research underscores the importance of developing more robust, fact-aware models to ensure trustworthy AI outputs in multi-round workflows.

Despite its insights, the study has limitations, including a focus on translation tasks and specific models. Future work aims to explore broader tasks, multi-modal scenarios, and training techniques to enhance information stability. Overall, this work advances our understanding of the risks and mechanisms of information distortion in AI systems, guiding the development of more reliable and trustworthy language models for complex, real-world applications.

Deep Dive

⚠️

Limitations & Outlook

What gaps remain?

While the study provides valuable insights, it primarily focuses on specific models (Llama, Mistral) and languages, which may limit generalizability. The experiments are centered on translation tasks, and other generative scenarios like summarization or dialogue are less explored. Although prompt tuning can reduce distortion, it does not address the root causes inherent in model architecture and training data. Further research is needed to develop inherently robust models capable of maintaining information fidelity over multiple iterations.

Abstract

As large language models are increasingly responsible for online content, concerns arise about the impact of repeatedly processing their own outputs. Inspired by the "broken telephone" effect in chained human communication, this study investigates whether LLMs similarly distort information through iterative generation. Through translation-based experiments, we find that distortion accumulates over time, influenced by language choice and chain complexity. While degradation is inevitable, it can be mitigated through strategic prompting techniques. These findings contribute to discussions on the long-term effects of AI-mediated information propagation, raising important questions about the reliability of LLM-generated content in iterative workflows.

cs.CL cs.AI