Ling and Ring 2.6 Technical Report: Efficient and Instant Agentic Intelligence at Trillion-Parameter Scale
Proposes Ling-2.6 and Ring-2.6 models using hybrid linear attention and transfer pretraining, boosting long-context efficiency and reasoning.
Key Findings
Methodology
The study combines architecture migration pretraining with large-scale post-training, employing Lightning Attention and MLA in a 7:1 ratio to optimize long-context processing. It introduces Evolutionary Chain-of-Thought and Linguistic Unit Policy Optimization to enhance token-level reasoning capacity. The KPop reinforcement learning framework stabilizes training of trillion-parameter models through asynchronous scheduling and binary KL divergence, enabling scalable environment-grounded learning. Extensive experiments demonstrate superior performance in long-context tasks, agentic benchmarks, and multi-domain generalization, with models achieving scores comparable to or surpassing GPT-5.4 in reasoning tasks.
Key Results
- Ling-2.6-1T achieves an Artificial Analysis Intelligence Index score of 34 with only 16 million output tokens, nearing GPT-5.4 performance. Ring-2.6-1T scores 87.60 on PinchBench and 63.82 on ClawEval, outperforming many baselines. Long-context processing efficiency improves 3-4x, with deep reasoning capabilities and 20% better training stability. The hybrid attention reduces inference FLOPs by over 50%, enabling practical deployment.
- Architectural innovations, including hybrid attention and transfer learning, allow the model to generalize across tasks and domains effectively. The token optimization strategies increase reasoning density, reducing redundancy. The KPop RL framework stabilizes training at scale, leading to more reliable agent behaviors in complex environments.
- Experimental results confirm that the models excel in tool use, multi-turn dialogue, and long document understanding, with significant improvements over prior state-of-the-art. The models demonstrate robustness and scalability, suitable for real-world agentic applications.
Significance
This work addresses critical bottlenecks in large-scale models—long-context processing and agentic reasoning—by integrating architectural and training innovations. It paves the way for practical, efficient, and scalable intelligent agents capable of complex decision-making and multi-task learning. The open-source release fosters community development, accelerating progress in AI research and deployment. By enabling models to handle ultra-long contexts with high efficiency and reasoning depth, it significantly advances the state-of-the-art in AI capabilities, bridging the gap between research and real-world applications.
Technical Contribution
The core technical contributions include: 1) a hybrid linear attention architecture combining Lightning Attention and MLA, drastically reducing long-context computational costs; 2) a transfer pretraining pipeline that upgrades existing models without retraining from scratch; 3) novel token-level optimization methods (Evo-CoT, LPO) that enhance reasoning density; 4) the KPop reinforcement learning framework for stable, scalable training of trillion-parameter models in environment-grounded settings. These innovations collectively enable high-performance, efficient large models suitable for practical deployment.
Novelty
This research is the first to combine Lightning Attention with MLA for ultra-long context models at trillion scale, achieving significant efficiency gains. It innovatively applies transfer pretraining to large models, avoiding costly from-scratch training. The integration of evolutionary reasoning and linguistic policy optimization for token efficiency is novel, as is the design of the KPop RL framework for large-scale environment interaction. These combined innovations set new benchmarks in model scalability, efficiency, and agentic capabilities.
Limitations
- Despite improvements, the models still face challenges in multi-modal integration and reasoning consistency, especially in highly complex or multi-modal scenarios.
- Training at trillion scale remains resource-intensive, limiting accessibility and deployment in resource-constrained environments.
- Ultra-long context processing, while improved, still encounters hardware bottlenecks at extremely large token lengths, restricting real-time applications.
Future Work
Future directions include optimizing model architectures for multi-modal data, reducing computational costs further, and enhancing robustness in multi-task environments. Exploring self-supervised learning and few-shot adaptation could improve domain-specific performance. Additionally, integrating safety and interpretability mechanisms will be critical for deploying these models in sensitive applications.
AI Executive Summary
This paper introduces Ling-2.6 and Ring-2.6, a family of large-scale models designed to excel in long-context processing and agentic reasoning. By integrating a hybrid linear attention architecture—combining Lightning Attention and MLA—the models achieve significant reductions in computational complexity, enabling efficient handling of contexts up to 262,144 tokens. Building upon the Ling-2.0 checkpoint, the authors employ a transfer pretraining strategy, supplemented with large-scale post-training, to upgrade the models without incurring the prohibitive costs of training from scratch. To further enhance reasoning capabilities, they introduce Evolutionary Chain-of-Thought and Linguistic Unit Policy Optimization, which improve token efficiency and reasoning density. These methods collectively enable the models to perform complex tasks with fewer tokens, achieving scores comparable to or better than GPT-5.4 in reasoning benchmarks.
A key innovation is the KPop reinforcement learning framework, which stabilizes training of trillion-parameter models through asynchronous scheduling and binary KL divergence. This approach allows the models to learn from complex, environment-grounded interactions at scale, resulting in highly reliable agentic behaviors. Experimental results demonstrate that Ling-2.6-1T attains an Artificial Analysis Intelligence Index score of 34 with only 16 million output tokens, while Ring-2.6-1T scores 87.60 on PinchBench and 63.82 on ClawEval, outperforming many existing models in deep reasoning and multi-task performance.
The architectural and training innovations presented in this work significantly advance the state-of-the-art in large language models, particularly for applications requiring ultra-long contexts and robust agentic capabilities. The models are open-sourced, fostering further research and development. Overall, this work marks a substantial step toward practical, scalable, and intelligent agent systems capable of complex reasoning, tool use, and environment interaction, with broad implications for AI deployment in industry and academia.
Deep Analysis
Background
The evolution of large language models (LLMs) has seen rapid progress, with architectures like Transformer and GQA enabling impressive performance in NLP tasks. However, their scalability is limited by quadratic attention complexity, especially for ultra-long contexts. Recent innovations such as Lightning Attention and MLA aim to address these issues by reducing computational costs. Despite these advances, deploying trillion-parameter models with deep reasoning and long-context processing remains challenging due to high resource demands and inference latency. Prior works like GPT series, PaLM, and Chinchilla have focused on scaling and efficiency, but often at the expense of long-context handling or agentic reasoning. This research builds on these foundations, seeking to unify efficiency and capability at unprecedented scale.
Core Problem
The core challenge is to develop models that can process ultra-long contexts efficiently while maintaining deep reasoning and agentic functions. Existing models either sacrifice speed for capability or vice versa. Long-context attention mechanisms like GQA become computationally prohibitive beyond certain token lengths. Additionally, training stability and scalability of trillion-parameter models in environment-grounded tasks are unresolved issues. Addressing these bottlenecks requires architectural innovations, efficient training strategies, and robust reinforcement learning frameworks, which are currently lacking in an integrated manner.
Innovation
The key innovations include: 1) a hybrid attention architecture combining Lightning Attention and MLA, reducing long-context compute from quadratic to linear complexity; 2) transfer pretraining from Ling-2.0 with architectural retrofit, avoiding costly from-scratch training; 3) token-level optimization via Evo-CoT and LPO, improving reasoning density; 4) the KPop RL framework, stabilizing large-scale environment-grounded training through asynchronous scheduling and KL divergence regularization. These innovations collectively enable the models to handle ultra-long contexts efficiently, perform deep reasoning, and learn reliably in complex environments.
Methodology
- �� Start from Ling-2.0 checkpoint, perform architectural migration by replacing GQA layers with Lightning Attention and MLA in a 7:1 ratio. • Conduct multi-stage pretraining, including Lightning Attention conversion, MLA warmup, and continued training on 9.6 trillion tokens across diverse datasets. • Use hybrid attention to balance computational cost and modeling quality, optimizing layer ratios via scaling law experiments. • Enhance token efficiency through Evo-CoT to remove redundant reasoning steps, and LPO to shift optimization to linguistic units. • Develop KPop reinforcement learning with asynchronous rollout collection and binary KL divergence to stabilize training of trillion-scale models. • Evaluate on long-context benchmarks, agentic tasks, and multi-domain datasets, comparing with baselines to validate improvements.
Experiments
Models were tested on benchmarks like Artificial Analysis Intelligence Index, PinchBench, and ClawEval, focusing on long-context reasoning, tool use, and multi-turn interactions. The experiments compared different attention ratios, migration strategies, and training methods. Ablation studies assessed the impact of each innovation, confirming that the hybrid attention architecture and token optimization significantly improved efficiency and reasoning depth. The models demonstrated robustness in environment-grounded tasks, with training stability improved by KPop. Data included ultra-long web corpora, agentic tool use, and multi-domain datasets, with metrics covering accuracy, FLOPs, latency, and stability.
Results
Ling-2.6-1T scored 34 on the Artificial Analysis Index with only 16 million tokens, nearing GPT-5.4 performance. Ring-2.6-1T achieved 87.60 on PinchBench and 63.82 on ClawEval, outperforming prior models in reasoning and multi-task benchmarks. The hybrid attention architecture reduced inference FLOPs by over 50%, enabling practical deployment. Token efficiency improvements yielded 4× higher reasoning capability per token. The KPop RL framework stabilized training at trillion scale, reducing instability by 20% and improving agentic behavior reliability. These results demonstrate the effectiveness of architectural and training innovations in scaling model capabilities efficiently.
Applications
The models are suited for intelligent assistants, automated coding, long document understanding, and multi-turn dialogue systems. Their ultra-long context handling supports complex content analysis, while deep reasoning enables decision-making in dynamic environments. They can be deployed in cloud or edge settings for industry applications like enterprise knowledge management, autonomous agents, and AI-powered research tools. The ability to process multi-modal data and interact with tools makes them versatile for real-world AI solutions.
Limitations & Outlook
Despite progress, models still face challenges in multi-modal integration, reasoning consistency, and resource demands. Ultra-long context processing requires significant hardware, limiting real-time deployment. Data biases from pretraining corpora may affect generalization. Future work should focus on reducing costs, improving robustness, and expanding multi-modal capabilities to ensure broader applicability and safety.
Plain Language Accessible to non-experts
想象你在一个超级大的厨房里工作，里面有很多厨师和工具。以前，厨师只能处理简单的菜肴，速度慢，还容易出错。现在，这个新厨房引入了一套聪明的系统，能让厨师同时处理更长、更复杂的菜谱，而且速度快得多。这个系统像是给厨师们装上了大脑，能记住很多步骤，还能用各种工具帮忙，比如切菜、调味。厨师们学会了拆解复杂的菜谱，把大任务变成小步骤，然后一步步完成。结果，厨房的效率大大提高，能做出更复杂、更美味的菜肴。这就像给厨房装上了超级智能的机器人，让它变得又快又聪明，还能帮你做很多事情。
ELI14 Explained like you're 14
想象你有个超级聪明的哥哥，他不仅能快速回答你的问题，还能理解你说的长长的故事。以前的哥哥只能处理简单的问题，回答慢，还容易出错。现在，他用了一些特别的方法，学会了用工具查资料、写代码，还能拆解复杂的问题，把它变成简单的步骤，然后帮你解决。这样一来，不管你问的问题多长、多难，哥哥都能很快帮你答出来，而且还很靠谱！这个哥哥就像是一个超级智能的机器人，既快又聪明，还能帮你做很多事情。
Glossary
Lightning Attention（闪电注意力）
一种高效的线性注意力机制，降低长文本处理的计算复杂度，支持超长上下文。技术结合稀疏和低秩压缩，提升效率。
在论文中用于替代传统GQA注意力，优化长上下文训练和推理。
MLA（Low-rank Memory Attention）
一种低秩记忆注意力机制，通过将KV缓存压缩到低秩空间，减少存储和计算成本，支持超长文本处理。
用于模型中的长上下文注意力部分，提升效率。
Evolutionary Chain-of-Thought（进化式思维链）
一种优化策略，通过演化算法去除冗余推理步骤，增强推理密度，提高Token利用效率。
在后训练中应用，提升模型推理能力。
Linguistic Unit Policy Optimization（语言单元策略优化）
一种基于语义单元的优化方法，将策略从Token级提升到语义单元级，改善信用分配和减少重复。
用于提升推理密度和信息利用率。
KPop（强化学习框架）
一种支持大规模环境交互的强化学习算法，通过异步调度和二元KL散度，稳定训练超大模型。
用于训练Ring-2.6-1T模型，改善环境适应性。
Open Questions Unanswered questions from this research
- 1 如何进一步降低超长上下文模型的硬件资源需求，尤其是在边缘设备上的部署难题。
- 2 多模态融合中的推理一致性和逻辑可靠性仍需突破，特别是在多模态数据交互场景。
- 3 模型在极端复杂环境中的自主决策能力和安全性保障仍待深入研究。
Applications
Immediate Applications
智能助理与自动编程
利用长上下文和深度推理能力，支持复杂任务的自动化处理，提升工作效率。模型可部署于云端或企业内部系统，帮助自动生成代码、分析长文档。
知识管理与内容生成
在企业和科研中实现高效内容整理、长篇摘要和多轮交互，改善信息检索和决策支持。
Long-term Vision
自主智能系统
未来模型将具备更强的自主学习和决策能力，支持多模态、多任务的复杂环境交互，推动自动化、智能化产业升级。
Abstract
Efficient and scalable agentic intelligence requires models that can deliver both low-latency responses and strong reasoning capabilities while remaining practical to train, serve, and deploy. In this report, we present Ling-2.6 and Ring-2.6, a family of models designed to address this challenge at scale. Ling-2.6 is optimized for instant response generation and high capability per output token, whereas Ring-2.6 is tailored for deeper reasoning and more advanced agentic workflows. Instead of training from scratch, we upgrade the Ling-2.0 base model through architectural migration pre-training and large-scale post-training. This upgrade is guided by a unified co-design of model architecture, optimization objectives, serving systems, and agent training environments, enabling improvements in both model capability and deployment efficiency. At the architectural level, we introduce a hybrid linear attention design that integrates Lightning Attention with MLA, improving the efficiency of long-context training and decoding. To further enhance token efficiency, we optimize capability per output token through Evolutionary Chain-of-Thought, Linguistic Unit Policy Optimization, bidirectional preference alignment, and shortest-correct-response distillation. For agentic capabilities, we propose KPop, a reinforcement learning framework designed to support stable training of Ring-2.6-1T on large-scale environment-grounded data. KPop improves training efficiency through asynchronous scheduling across coding, search, tool use, and workflow execution, enabling scalable learning from complex agent-environment interactions. Together, Ling-2.6 and Ring-2.6 provide a practical pathway toward efficient, scalable, and open agentic systems. We open-source all checkpoints in the 2.6 family to support further research and development in practical agentic intelligence.