SkillNet: Create, Evaluate, and Connect AI Skills
SkillNet builds a unified skill ontology with multi-source creation and multi-dimensional evaluation, enhancing scalable AI skill management.
Key Findings
Methodology
SkillNet employs a multi-layered ontology comprising taxonomy, relation graph, and package layers. It integrates automated information extraction from diverse sources—logs, GitHub, documents, prompts—using large language models like GPT-4. Skills are filtered via rule-based and model-based multi-dimensional evaluation covering safety, completeness, executability, maintainability, and cost. The relation graph models dependencies, similarities, and compositions, enabling reasoning. Experiments on ALFWorld, WebShop, and ScienceWorld show a 40% increase in average reward and 30% reduction in steps, validating the framework’s effectiveness.
Key Results
- Across multiple backbone models, SkillNet improves average task rewards by over 40% and reduces interaction steps by more than 30%. It outperforms traditional skill management approaches in efficiency and robustness.
- SkillNet-Gym benchmarks demonstrate superior skill retrieval, utilization, and composition capabilities. SkillNet-Fabric enables task-specific skill routing, significantly boosting task success rates.
- Automated multi-dimensional evaluation correlates strongly with human judgments, with MAE below 0.03 and QWK above 0.90, confirming high reliability.
Significance
This work addresses the core challenge of skill fragmentation and quality assurance in AI systems. By systematically organizing and evaluating skills at scale, SkillNet facilitates knowledge reuse, transfer, and long-term evolution of intelligent agents. It bridges the gap between episodic experience and durable mastery, enabling AI to learn continuously and adapt across tasks and domains, thus pushing forward the frontier of autonomous, scalable AI capabilities.
Technical Contribution
SkillNet introduces a comprehensive multi-layered ontology, automated skill extraction pipeline, and multi-dimensional evaluation framework. The relation graph captures complex dependencies and similarities, supporting reasoning and composition. This integrated approach offers a scalable, reliable, and dynamic skill management system, surpassing prior static repositories and ad hoc methods, and enabling large-scale knowledge reuse.
Novelty
This is the first framework to unify multi-source skill extraction, multi-dimensional quality assessment, and relation graph modeling into a coherent, scalable system. Unlike existing repositories, SkillNet emphasizes automated, multi-faceted evaluation and dynamic relation inference, establishing a new paradigm for AI skill management that supports continuous evolution and large-scale deployment.
Limitations
- The automated evaluation relies heavily on large language models, which may introduce biases or inaccuracies in specialized domains. Further validation and hybrid human-AI approaches are needed.
- Coverage in niche or highly specialized fields remains limited; ongoing expansion and domain adaptation are required.
- Static relation graphs may not adapt well to environmental or task changes; future work should incorporate dynamic learning mechanisms.
Future Work
Future efforts will focus on integrating reinforcement learning for dynamic skill updating, expanding multi-modal skill representations, and developing adaptive relation graphs. Enhancing cross-domain transferability and reducing reliance on large models for evaluation are also key directions.
AI Executive Summary
SkillNet represents a significant advancement in large-scale AI skill management, addressing the fragmentation and quality assurance challenges that have hindered knowledge reuse. Traditional AI systems often rely on isolated, manually curated skills, which are difficult to organize, evaluate, and scale. This fragmentation leads to inefficiencies, redundant efforts, and limited transferability across tasks and domains.
The core innovation of SkillNet lies in its multi-layered ontology, which structures skills into taxonomy, relation graph, and package layers. By leveraging large language models like GPT-4, it automates the extraction of skills from diverse sources—such as execution logs, open-source repositories, and documents—and filters them through a multi-dimensional evaluation framework. This framework assesses safety, completeness, executability, maintainability, and cost, ensuring only high-quality skills are retained.
The relation graph models complex inter-skill dependencies, similarities, and compositions, enabling reasoning and efficient skill reuse. Extensive experiments across environments like ALFWorld, WebShop, and ScienceWorld demonstrate that agents equipped with SkillNet outperform baselines, achieving 40% higher rewards and 30% fewer steps. The SkillNet-Gym benchmark further validates the system’s capabilities in skill retrieval and composition, while SkillNet-Fabric provides task-specific routing, improving overall task success.
This work has profound implications for scalable, reliable AI development. By formalizing skills as evolving, composable assets, SkillNet paves the way for autonomous agents capable of continuous learning and adaptation. Future directions include dynamic relation learning, multi-modal skill integration, and broader domain coverage, promising a new era of intelligent, scalable systems.
Deep Analysis
Background
随着深度学习和符号逻辑的发展，AI逐渐从规则驱动走向数据驱动，技能成为连接知识与操作的关键。早期符号系统强调可解释性，但扩展性不足；深度学习提供强大表现，但缺乏结构化知识管理。近年来，研究集中在技能的系统组织与迁移能力，如SkillGraph、OpenAI插件体系等，但仍存在碎片化、验证难题。大规模、多源、多维的技能管理体系亟待建立，以实现知识的高效复用与持续演化，为智能体提供长效学习能力。
Core Problem
现有AI技能多为零散资源，缺乏统一的组织架构与质量保障机制，导致技能难以系统存储、检索和迁移。碎片化严重，难以实现跨任务、跨领域的迁移学习。同时，缺少多维度的自动评估体系，难以保证技能的安全性、完整性和可执行性。这些问题限制了智能体的长效学习和知识积累，亟需建立系统化、动态的技能管理平台。
Innovation
SkillNet创新点在于：1）多层次本体模型，划分技能分类、关系和包层，支持动态扩展；2）自动化信息抽取，从多源数据中提取技能，减少人工干预；3）多维自动评估体系，确保技能质量，涵盖安全、完整、可执行、可维护和成本；4）关系图模型，捕捉技能间的依赖、相似与组合关系，支持推理与复用。这些创新实现了技能的系统化、动态演化与高效管理。
Methodology
- �� 利用大模型（如GPT-4）从执行轨迹、GitHub、文档和自然语言提示中自动抽取技能。
- �� 根据预定义类别和语义标签，将技能归类，建立技能分类层。
- �� 通过语义嵌入和关系推理，构建技能相似、依赖、组合关系，形成关系图。
- �� 采用多维度评估模型，自动打分技能的安全性、完整性、可执行性、可维护性和成本，筛选优质技能。
- �� 利用关系图进行技能推理，支持技能组合与任务规划。
- �� 结合人工验证校准自动评估模型，确保高可靠性。
Experiments
在ALFWorld、WebShop和ScienceWorld环境中，采用多种基础模型（如DeepSeek V3、Gemini 2.5 Pro）进行任务测试。指标包括平均奖励、交互步骤、技能检索准确率等。通过对比传统技能管理方法，验证SkillNet提升奖励40%以上，步骤减少30%。还进行了技能质量的人工标注验证，确保自动评估的准确性。
Results
实验显示，SkillNet显著提升智能体在复杂任务中的表现，奖励平均提升40%，交互步骤减少30%。技能检索和组合能力优于基线，关系图模型支持技能推理和复用。自动评估模型与人工标注高度一致，验证了体系的可靠性。多维度评估确保技能质量，为大规模应用提供保障。
Applications
该体系可广泛应用于智能客服、自动化办公、机器人控制等场景，支持跨任务技能迁移与复用。通过构建高质量技能库，提升系统的自主学习与适应能力。未来结合强化学习实现技能的动态更新与优化，推动智能体在实际环境中的自主演化。
Limitations & Outlook
目前体系主要依赖大模型进行自动评估，可能在专业领域存在偏差。技能库虽大，但在某些专业领域覆盖不足。关系图模型静态，难以应对环境变化，未来应引入动态学习机制。此外，构建高质量技能仍需大量标注与验证，成本较高。
Plain Language Accessible to non-experts
想象你在一个大型厨房里，厨师们有各种不同的菜谱（技能），每个菜谱都详细说明了做菜的步骤和所需材料。以前，这些菜谱散落在不同的地方，有的在纸上，有的在手机里，厨师们很难找到合适的菜谱，也不知道哪个菜谱更安全、更完整。现在，SkillNet就像建立了一个超级厨房管理系统，把所有菜谱都整理到一个大数据库里，不仅分类清楚，还能自动检测菜谱是否完整、安全。厨师可以快速找到需要的菜谱，组合不同的菜肴，甚至根据不同的需求定制新的菜谱。这让厨房变得更高效，菜肴也更好吃。这个系统的核心是用智能算法自动整理、评估和连接所有菜谱，让厨房的每个厨师都能用上最好的工具，做出最棒的菜肴。
ELI14 Explained like you're 14
想象你在学校里学做菜，你有很多不同的菜谱。有的菜谱写得很详细，有的只写了大概的步骤。以前，你要自己去找这些菜谱，有时候还会忘记步骤或者用错材料。现在，有一个超级智能的厨房助手，它把所有的菜谱都整理在一起，还会帮你检查菜谱是不是完整、安全。比如，它会告诉你这个菜谱是不是缺少重要的步骤，或者用的材料是不是安全。你只要告诉它你想做什么菜，它就会帮你找到最合适的菜谱，还能帮你组合不同的菜肴，甚至帮你设计新的菜谱。这样，你做菜就变得又快又好，还能学到很多新菜。这个助手就像一个聪明的厨师，帮你把所有的菜谱变成一个有序、可靠的宝库，让你随时都能做出美味的菜肴。
Glossary
Skill (技能)
一种模块化、可复用的知识单元，包含操作步骤和资源，用于扩展AI的能力。
在论文中，Skill指AI系统中的操作能力单元，用于任务执行。
Ontology (本体)
一种层级化的知识结构，用于组织和描述技能之间的关系。
SkillNet利用本体模型组织技能分类与关系。
Multi-dimensional evaluation (多维评估)
从安全、完整性、可执行性、可维护性和成本五个方面对技能进行自动化评估。
确保技能质量和可靠性。
Relation graph (关系图)
描述技能之间依赖、相似和组合关系的结构化网络。
用于技能推理和组合。
Knowledge reuse (知识复用)
将已有技能应用到新任务中，减少重复开发。
SkillNet促进技能的系统存储与复用。
Open Questions Unanswered questions from this research
- 1 如何动态更新技能关系以适应环境变化仍未充分解决，未来需结合在线学习机制。
- 2 跨模态技能的组织与融合尚未实现，支持多模态信息的技能建模是未来方向。
- 3 技能自动生成的质量保障机制仍需完善，结合人类专家验证与强化学习可能是突破点。
Applications
Immediate Applications
智能客服系统
利用SkillNet构建客服技能库，实现自动应答、问题诊断和流程引导，提升响应速度与准确性。
自动化办公
集成技能库支持文档处理、数据分析和流程自动化，减少人工操作，提高效率。
Long-term Vision
自主学习智能体
通过持续扩展与优化技能体系，实现智能体在复杂环境中的自主学习与长效演化，推动AI普适化。
Abstract
Current AI agents can flexibly invoke tools and execute complex tasks, yet their long-term advancement is hindered by the lack of systematic accumulation and transfer of skills. Without a unified mechanism for skill consolidation, agents frequently ``reinvent the wheel'', rediscovering solutions in isolated contexts without leveraging prior strategies. To address this challenge, we introduce SkillNet, an open infrastructure for creating, evaluating, and organizing AI skills at scale. SkillNet structures skills within a unified ontology that supports creating skills from heterogeneous sources, establishing rich relational connections, and performing multi-dimensional evaluation across Safety, Completeness, Executability, Maintainability, and Cost-awareness. Our infrastructure integrates a repository of over 600,000 skills, an interactive platform, and a versatile Python toolkit. Experiments on ALFWorld, WebShop, and ScienceWorld show 40% higher average rewards and 30% fewer execution steps across multiple backbone models. Furthermore, SkillNet-Gym benchmarks skill retrieval, utilization, and composition, while SkillNet-Fabric enables task-specific skill routing through lightweight Wikis. By formalizing skills as evolving, composable assets, SkillNet provides a robust foundation for agents to move from transient experience to durable mastery.