XBridge: Entity-Grounded Latent Bridge for Heterogeneous LLM Communication
XBRIDGE combines lexical anchor mapping and latent enrichment bridge for efficient heterogeneous LLM communication, reducing latency by 11×.
Key Findings
Methodology
This paper introduces XBRIDGE, integrating Lexical Anchor Mapping (LAM) and Latent Enrichment Bridge (LEB). LAM maps source model tokens to target vocabularies, providing discrete entity anchors, while LEB employs gated cross-attention to query source hidden states for contextual enrichment. Both operate without autoregressive decoding, with only 264M trainable parameters, enabling rapid adaptation across diverse model pairs. The system ensures entity fidelity and contextual richness, overcoming limitations of continuous projection methods.
Key Results
- Across three model families (Llama, Qwen, Mistral) and seven benchmarks, XBRIDGE outperforms text-based communication, with an average F1 score increase of 21.4 points and 11× latency reduction. It excels particularly in multi-hop reasoning tasks like Countries and MuSiQue, surpassing FullComm performance. In same-architecture scenarios, it exceeds KV cache sharing by approximately 7.9 points.
- Entity fidelity evaluations show that XBRIDGE maintains high cosine similarity and entity ranking metrics, significantly better than baseline methods, confirming effective entity grounding.
- Training takes less than 10 minutes on a small dataset of 587 samples, demonstrating fast convergence and strong generalization across model pairs and tasks.
Significance
This work addresses the critical bottleneck in heterogeneous LLM systems—effective, low-latency communication. By solving the entity grounding collapse and representation mismatch issues, it enables diverse models to collaborate efficiently. The dual-channel approach sets a new standard for multi-model reasoning, paving the way for scalable, flexible AI ecosystems capable of complex reasoning and knowledge sharing across different architectures and modalities.
Technical Contribution
The paper formalizes the entity grounding problem, revealing the rare-token collapse in continuous bridges. It introduces a novel dual-channel protocol combining discrete lexical anchors with continuous contextual queries. The architecture employs lightweight, trainable cross-attention modules, preserving pretraining benefits while enabling cross-model information flow. The training process is efficient, requiring minimal data and time, yet achieves robust performance.
Novelty
This is the first work to explicitly combine discrete entity anchors with continuous contextual enrichment in a unified communication protocol for heterogeneous models. Unlike prior approaches relying solely on continuous projections or textual summaries, XBRIDGE offers a hybrid solution that maintains entity fidelity and contextual depth simultaneously, filling a significant gap in multi-model AI research.
Limitations
- The approach depends on precomputed token mappings, which may struggle with unseen entities or extreme vocabulary divergence. Dynamic mapping strategies are needed for better robustness.
- LEB training on small datasets may limit scalability; larger, more diverse datasets could improve generalization.
- Handling very long texts or integrating multi-modal data remains challenging, requiring further architectural enhancements.
Future Work
Future directions include developing adaptive, dynamic vocabulary mapping techniques, extending the framework to multi-modal data (images, audio), and scaling training to larger datasets for broader generalization. Exploring real-time, online adaptation mechanisms and applying the method to real-world multi-agent AI systems are also promising avenues.
AI Executive Summary
Multi-agent large language models (LLMs) hold promise for complex reasoning tasks, but their effectiveness hinges on efficient communication across diverse architectures. Traditional methods like text summaries or KV cache sharing either sacrifice information richness or require architectural compatibility, limiting scalability and flexibility. This paper introduces XBRIDGE, a novel communication protocol that combines discrete lexical anchors with a continuous contextual enrichment mechanism, enabling heterogeneous models to exchange information effectively without decoding or strict architectural alignment.
The core innovation lies in two mechanisms: Lexical Anchor Mapping (LAM) and Latent Enrichment Bridge (LEB). LAM maps source model tokens to target vocabularies, providing explicit entity anchors that preserve entity identities. LEB employs gated cross-attention modules inserted at multiple layers of the receiver, allowing it to query the source model’s hidden states directly in its native space, thereby enriching the context with deep semantic information. This dual-channel setup ensures that entities are grounded correctly while maintaining rich contextual understanding.
Experimental results demonstrate that XBRIDGE outperforms traditional text-based communication across various model pairs and benchmarks, achieving an average F1 score improvement of 21.4 points and reducing latency by a factor of 11. It surpasses even full direct communication methods in many cases, especially on multi-hop reasoning tasks. The training process is lightweight, requiring less than 10 minutes on a small dataset, and the approach generalizes well across different model architectures and tasks.
This work significantly advances the field of multi-model AI, addressing fundamental challenges in heterogeneous communication. It opens pathways for scalable, flexible, and efficient AI systems capable of complex reasoning, knowledge sharing, and multi-modal integration. Future work will focus on dynamic vocabulary adaptation, multi-modal extension, and large-scale deployment, promising a new era of collaborative AI ecosystems.
Deep Analysis
Background
近年来，预训练大模型（如GPT、BERT、LLaMA）在自然语言处理领域取得巨大突破，但其单一架构限制了多智能体系统的灵活性。多模型协作能提升推理能力，但模型间的异构性带来信息传递瓶颈。早期方案如KV缓存共享和文本摘要，虽然提升效率，却牺牲了信息丰富性或架构兼容性。连续投影技术试图跨模型映射，但在实体识别中表现不佳。研究逐步转向如何在保持模型多样性的同时，实现高效、精确的通信。
Core Problem
异构模型通信面临三大挑战：实体归属（确保实体在目标模型中的正确识别）、上下文丰富（传递深层语义信息）以及表示空间兼容（不同模型的隐藏状态难以直接融合）。连续桥在传递上下文时容易丢失实体的离散信息，导致识别失败；而直接投影则面临分布不匹配的问题，影响信息的有效利用。这些问题限制了多模型协作的效率和准确性，亟需创新方案解决。
Innovation
本文提出两大创新：1）词汇锚定映射（LAM），实现实体词的离散映射，确保实体识别的准确性；2）潜在丰富桥（LEB），利用门控交叉注意力机制，从源模型查询隐藏状态，丰富上下文信息。两机制结合，突破了连续投影的实体崩溃和表示空间不匹配的难题。该方案无需模型架构对齐，训练快速，参数少，适应性强，显著提升异构模型间的通信效率。
Methodology
- �� 词汇映射：预计算源模型词表到目标模型词表的映射，确保实体词的离散锚定。
- �� 词汇嵌入：将映射后的词ID转换为目标模型的词向量，作为输入。
- �� 上下文丰富：源模型最后一层隐藏状态通过门控交叉注意力模块，动态查询源模型的隐藏状态，获取上下文信息。
- �� 交叉注意力：在模型多个层中插入门控交叉注意力模块，利用源模型隐藏状态作为键值，目标模型表示作为查询。
- �� 训练：用标准的下一词预测损失，参数调优仅在LEB模块，训练时间少于10分钟。
- �� 结合：实体锚点保证实体识别，潜在桥提供深层语义，二者协同实现高效异构通信。
Experiments
采用Llama、Qwen、Mistral三大模型家族，覆盖不同架构和词表，评估七个任务（HotpotQA、MuSiQue等）。比较方法包括文本摘要（NLComm）、KV缓存共享（KVComm）和FullComm。指标为F1得分和延迟。训练采用587样本，参数调优在单GPU下完成，验证模型在不同模型对和任务中的泛化能力。实验还包括实体扰动分析，验证实体归属的鲁棒性。
Results
XBRIDGE在所有模型对和任务中均优于文本通信方案，平均提升21.4个百分点，延迟降低11倍。多跳推理任务中表现尤为突出，实体识别保持高精度。训练参数少，训练时间短，模型泛化能力强。实体归属指标显示，XBRIDGE在余弦相似度和实体排名方面显著优于对比方法，验证了实体锚定的有效性。
Applications
该技术适用于多智能体协作、跨模态信息融合、复杂推理任务等场景。可用于智能问答、协作式搜索、知识图谱构建等领域，提升系统的响应速度和准确性。实现条件包括不同模型间的接口设计和预训练模型的适配，未来可结合多模态数据，拓展应用范围。
Limitations & Outlook
当前方法依赖预定义词汇映射，面对新实体或极端词表差异时可能表现不足。LEB训练在小样本集上，泛化到大规模复杂场景时仍需验证。模型在超长文本或多模态信息融合场景中的适应性尚未充分探索。
Plain Language Accessible to non-experts
想象一群来自不同国家的学生，每个人用不同的语言和方式交流。老师想让他们合作完成一个大项目，但每个学生的表达方式都不同，交流困难。XBRIDGE就像给每个学生准备了一个“翻译卡片”，让他们用自己熟悉的词表达实体（比如“老师”、“书”），同时还提供一个“信息查询器”，让学生可以在需要时向其他班级“问”隐藏的想法。这样，大家既能用自己熟悉的词汇表达，又能分享深层的想法，合作变得更顺畅。这种方式避免了繁琐的翻译，也保证了信息的准确传递。
ELI14 Explained like you're 14
你知道在学校里，不同班级的学生可能用不同的语言交流，想合作做个大项目就很难。XBRIDGE就像给每个学生准备了一个特殊的“翻译卡片”，让他们用自己熟悉的词说话，还能偷偷问老师隐藏的想法。这样，不管用什么语言，大家都能明白对方在说什么，还能分享更深的想法。就像用不同的密码交流，但每个人都知道密码的规则，合作就变得简单多了。这种方法让不同“语言”的模型也能像朋友一样合作，效率更高，误会更少。
Glossary
Entity Grounding (实体归属)
确保模型识别的实体在目标模型中对应正确的词汇，是信息传递的基础。
论文中强调实体归属对信息传递的关键作用。
Latent Enrichment Bridge (潜在丰富桥)
利用门控交叉注意力机制，从源模型查询隐藏状态，丰富上下文信息。
作为核心技术之一，用于增强异构模型间的上下文传递。
Lexical Anchor Mapping (词汇锚定映射)
将源模型的实体词映射到目标模型词表，提供离散的实体锚点。
解决异构模型中实体识别和归属问题。
Cross-attention (交叉注意力)
在模型层中实现源模型隐藏状态的动态查询，融合不同模型的表示空间。
LEB模块的关键机制。
Rare-token compression collapse (稀有词压缩崩溃)
连续表示在传递过程中丢失稀有实体的识别信息的现象。
揭示连续桥在实体识别中的局限。
Open Questions Unanswered questions from this research
- 1 如何在极端词表差异或新实体出现时保持映射的鲁棒性仍需研究。
- 2 大规模模型中LEB的训练策略和泛化能力尚未充分验证。
- 3 多模态信息融合对异构模型通信的影响是未来研究方向。
Applications
Immediate Applications
多智能体问答系统
实现不同模型间高效信息交流，提升问答准确率和响应速度，适用于智能客服和知识问答平台。
跨模态信息融合
结合视觉、文本等多模态数据，增强模型理解和推理能力，推动多模态AI的发展。
Long-term Vision
多模型协作平台
构建大规模异构模型协作生态，支持复杂任务如自动驾驶、智能制造等，推动AI系统的普及和智能化。
Abstract
Heterogeneous multi-agent LLM systems, where agents are powered by different model families, can outperform homogeneous configurations by reducing redundant reasoning patterns. Yet existing communication protocols either operate through text, discarding the sender's internal representations, or require architectural homogeneity for latent-level transfer. We identify the entity grounding problem in cross-architecture communication: cross-attention bridges that transfer continuous representations across different LLM families suffer from rare-token compression collapse, where entity identity is lost in the continuous bottleneck (bridge-only F1 ~30%). We propose XBRIDGE, a decode-free communication protocol that addresses this through two mechanisms. Lexical Anchor Mapping (LAM) maps the sender's original context tokens to the receiver's vocabulary, providing discrete entity anchors. A Latent Enrichment Bridge (LEB) lets the receiver query the sender's hidden states for contextual enrichment. The entity anchors ground the bridge's contextual signals to specific entities through the receiver's own self-attention. Across three model families (Llama, Qwen, and Mistral), seven benchmarks, and both communication directions, XBRIDGE outperforms text-based communication on all seven tasks for each model pair while achieving 11x lower latency, and in a same-architecture setting it also exceeds a KV-sharing baseline on six of seven tasks. LEB requires only 264M trainable parameters (3.8% of the receiver), is trained on a small balanced sample set, and adds negligible inference overhead.