OpenClaw-Skill: Collective Skill Tree Search for Agentic Large Language Models
OpenClaw-Skill enhances large language models in tool use and multi-step reasoning via Collective Skill Tree Search.
Key Findings
Methodology
This study introduces a novel framework called Collective Skill Tree Search (CSTS) for automatically constructing reusable skills to enhance large language models in complex tasks. CSTS leverages collective intelligence through two iterative phases: Collective Skill Node Generation (CSN-Gen) and Collective Skill Node Assessment (CSN-Assess) to explore and evaluate skill nodes using multiple models.
Key Results
- In the QwenClawBench benchmark, OpenClaw-Skill improved the overall score from 34.5 to 44.9 on the Qwen3.5-9B model, significantly enhancing long-horizon planning and tool use capabilities.
- In the PinchBench test, OpenClaw-Skill 9B improved the best score from 61.1 to 68.2 and the average score from 47.1 to 53.6 in the 123-task setting.
- Ablation studies show that adding CSN-Gen and CSN-Assess improved performance by 5.3 and 3.0 points, respectively, with CSRL further enhancing by 2.1 points.
Significance
This research addresses the fragmentation and limited generalization of skills in large language models for complex tasks through the CSTS framework. CSTS not only enhances tool use and multi-step reasoning capabilities but also provides an efficient automated method for skill construction, reducing manual design costs and time.
Technical Contribution
Technical contributions include introducing the CSTS framework, which enhances skill diversity and transferability through collective intelligence. The proposed CSRL method further optimizes strategy by using skill-conditioned rollout groups, improving model flexibility and robustness.
Novelty
CSTS is the first framework to utilize collective intelligence for skill construction, significantly improving skill structuring and generalization compared to existing methods through tree search and multi-model evaluation mechanisms.
Limitations
- In certain complex tasks, skill node generation and evaluation may still be influenced by model preferences, leading to insufficient skill diversity.
- The CSTS framework has high computational demands, which may limit its application in resource-constrained environments.
Future Work
Future research directions include optimizing the computational efficiency of the CSTS framework, exploring more model integration strategies, and expanding applications across different domain tasks.
AI Executive Summary
In complex tasks, traditional large language models often face issues of skill fragmentation and limited generalization. Existing methods rely heavily on manual design, which is costly and hard to scale. To address these challenges, researchers have proposed a novel framework called Collective Skill Tree Search (CSTS). CSTS uses collective intelligence to automatically generate and evaluate skill nodes, constructing structured, diverse, and generalizable skill trees. Experimental results show that OpenClaw-Skill performs exceptionally well across multiple benchmarks, particularly in long-horizon planning and tool use. Although CSTS demands significant computational resources, its innovations in skill construction open new pathways for applying large language models. Future research will focus on optimizing the framework's efficiency and exploring its potential applications in various fields.
Deep Analysis
Background
Large language models have made significant advances in natural language processing, especially in interactive environments. However, as task complexity increases, issues of skill fragmentation and limited generalization become more pronounced. Traditional methods rely heavily on manually designed skills, which are costly and difficult to scale.
Core Problem
The core problem is how to automatically construct structured and generalizable skills to enhance large language models in complex tasks. Existing methods often lack effective organization of skill sequences, making it difficult to handle long-term dependencies and multi-step execution.
Innovation
The CSTS framework innovatively addresses skill fragmentation and limited generalization through collective intelligence and tree search methods. The CSN-Gen and CSN-Assess phases are responsible for generating and evaluating skill nodes, ensuring diversity and transferability.
Methodology
- �� Collective Skill Node Generation (CSN-Gen): Utilizes collective knowledge from multiple models to generate diverse skill candidates. • Collective Skill Node Assessment (CSN-Assess): Uses multi-model evaluation mechanisms to select effective and transferable skill nodes. • Collective Skill Reinforcement Learning (CSRL): Optimizes strategy through skill-conditioned rollout groups, enhancing model flexibility.
Experiments
Experiments were conducted on the QwenClawBench and PinchBench benchmarks using Qwen3 and Qwen3.5 series models. Skill-augmented training data were generated using CSTS, followed by supervised fine-tuning and reinforcement learning optimization.
Results
In QwenClawBench, OpenClaw-Skill significantly improved long-horizon planning and tool use capabilities. In the PinchBench test, the model performed exceptionally well across multiple task settings, demonstrating the effectiveness of the CSTS framework.
Applications
The CSTS framework can be applied to tasks requiring complex skill combinations, such as automated tool use, long-horizon planning, and multi-step reasoning. It has broad application potential in industrial automation and intelligent assistants.
Limitations & Outlook
Although CSTS excels in skill construction, its computational resource demands are high, potentially limiting its application in resource-constrained environments. Additionally, skill node diversity may still be influenced by model preferences.
Plain Language Accessible to non-experts
Imagine you're cooking in a kitchen. You need different tools and steps to complete a complex dish. Traditional methods are like figuring it out on your own, while CSTS is like having an experienced team of chefs who automatically generate the best cooking plan based on different ingredients and steps, and evaluate each step's effectiveness. This way, you not only make delicious dishes but also navigate different kitchen environments with ease. The CSTS framework uses collective intelligence to help large language models find the best solutions in complex tasks, just like the chef team helps you in the kitchen.
ELI14 Explained like you're 14
Imagine you're playing a puzzle game. Each puzzle needs different skills and tools. Traditional methods are like figuring it out alone, while CSTS is like having a team that automatically generates solutions and evaluates each step's effectiveness. This way, you solve puzzles faster and perform well in different game levels. The CSTS framework is like a super assistant in the game, helping large language models find the best solutions in complex tasks. Isn't that cool?
Glossary
Collective Skill Tree Search (CSTS)
A framework that automatically generates and evaluates skills using collective intelligence to enhance large language models in complex tasks.
Used to construct structured, diverse, and generalizable skill trees.
Collective Skill Node Generation (CSN-Gen)
Utilizes collective knowledge from multiple models to generate diverse skill candidates.
A key phase in the CSTS framework.
Collective Skill Node Assessment (CSN-Assess)
Uses multi-model evaluation mechanisms to select effective and transferable skill nodes.
A key phase in the CSTS framework.
Collective Skill Reinforcement Learning (CSRL)
Optimizes strategy through skill-conditioned rollout groups, enhancing model flexibility.
Used to further optimize CSTS-generated skills.
Skill Fragmentation
Refers to skills being limited to local tasks, lacking overall structure and long-term dependencies.
One of the issues the CSTS framework aims to solve.
Open Questions Unanswered questions from this research
- 1 How to efficiently apply the CSTS framework in resource-constrained environments?
- 2 How to further enhance the diversity and generalization of skill nodes?
- 3 What is the potential for application in more domain tasks?
Applications
Immediate Applications
Automated Tool Use
Enhance large language models' performance in automated tool use through the CSTS framework, applicable in industrial automation and intelligent assistants.
Long-term Vision
Intelligent Assistants
Develop smarter assistants through the CSTS framework, capable of providing more effective solutions in complex tasks, advancing human-computer interaction.
Abstract
Equipping Large Language Model (LLM) agents with effective skills is crucial for solving complex tasks in real-world systems like OpenClaw. In this work, we aim to develop a framework that automatically constructs such reusable skills to enhance LLMs in tool use, multi-step reasoning, and dynamic environment interaction. To this end, we propose Collective Skill Tree Search (CSTS), a novel tree-search-based skill construction framework that constructs structured, diverse and generalizable tree of skills. The core idea of CSTS is to leverage collective intelligence to jointly search, identify and compose effective skills via two iterative phases: Collective Skill Node Generation (CSN-Gen) and Collective Skill Node Assessment (CSN-Assess). CSN-Gen exploits collective knowledge from multiple models to explore diverse candidate skills for each subtask, enabling comprehensive skill exploration. CSN-Assess employs multiple models as judges to evaluate and select skill nodes with two scoring mechanisms: (1) collective quality scoring that aggregates independent evaluations to produce a robust estimate of skill effectiveness, and (2) collective transferability scoring that explicitly verifies whether a skill generalizes well across different models. With CSTS, we construct a set of comprehensive tree of skills along with skill-augmented training data, enabling models to effectively learn and utilize skills. Besides, we introduce Collective Skill Reinforcement Learning, which actively selects multiple relevant skills from the tree to broaden solution-space exploration, avoid being trapped by a single skill and its resulting homogeneous or suboptimal solutions. As a result, our trained model, OpenClaw-Skill, exhibits outstanding agentic capabilities in long-horizon planning, tool use and generalization over challenging benchmarks.