Don't Adapt Small Language Models for Tools; Adapt Tool Schemas to the Models
Proposes PA-Tool, using peakedness to align tool schemas with pretrained models, boosting accuracy by 17% without retraining.
Key Findings
Methodology
This approach leverages peakedness, a signal from contamination detection, to assess a model’s familiarity with tool component names. By generating multiple candidate names from descriptions, computing their peakedness via character-level similarity, and selecting the most peaked candidate, it aligns tool schemas with pretrained model biases. The process involves: providing descriptions, sampling candidates, calculating peakedness scores, and choosing the highest, forming a mapping for schema adjustment. Experiments on MetaTool and RoTBench show up to 17% performance gains and 80% error reduction in schema misalignment.
Key Results
- In MetaTool, PA-Tool improves reliability task accuracy by up to 17% (e.g., Llama3.2-3B from 43.6% to 60.6%), and multi-tool selection by 9.6%. On RoTBench, single-turn tool selection improves by 10%, parameter identification by 4.3%. Errors related to non-existent tool names decrease by 80%, indicating better schema alignment. Different models, including Qwen and Llama, benefit significantly, especially resource-constrained small models.
- Compared to fine-tuning, PA-Tool achieves similar or better performance without training, and complements existing description enhancement and retrieval correction methods. The peakness metric increases with training epochs, supporting its role as a familiarity indicator. The approach enhances robustness in API invocation and parameter extraction tasks, demonstrating its practical value in real-world scenarios.
- The experimental results confirm that schema misalignment is a primary failure mode. Correcting naming biases via peakedness-based selection reduces errors and improves overall tool use. The method’s simplicity, efficiency, and transferability make it a promising solution for deploying resource-efficient models in complex multi-tool environments.
Significance
This work addresses a fundamental bottleneck in resource-limited models’ ability to correctly invoke external tools. By aligning tool schemas with pretrained biases, it offers a training-free, scalable solution that significantly enhances tool-use accuracy. The approach reduces reliance on costly fine-tuning, lowers deployment barriers, and improves robustness across diverse tasks. It provides a new paradigm for schema-level intervention, enabling models to better understand and utilize external tools, which is crucial for building reliable, multi-modal AI systems. The method’s broad applicability and effectiveness suggest it can accelerate the development of intelligent, autonomous agents capable of complex reasoning and external interaction.
Technical Contribution
The paper introduces PA-Tool, a novel schema alignment framework that exploits peakedness signals from contamination detection to identify models’ internalized naming patterns. Unlike traditional fine-tuning or description-based methods, PA-Tool performs zero-shot schema optimization by generating multiple candidate names, computing their peakedness via character-level edit distances, and selecting the most familiar pattern. This process creates a direct, architecture-level alignment between tool schemas and model biases, significantly reducing naming errors and improving tool invocation accuracy. The approach is validated across multiple models and benchmarks, demonstrating its generality and efficiency, and providing a new avenue for schema-level model adaptation without retraining.
Novelty
This work is the first to adapt tool schemas to pretrained model biases using contamination detection signals, specifically peakedness, rather than relying on retraining or manual renaming. It shifts the paradigm from model-centric adaptation to schema-centric optimization, enabling resource-efficient models to better leverage external tools. The core innovation lies in repurposing contamination detection metrics for schema alignment, offering a simple yet powerful method that outperforms existing description-based or fine-tuning approaches. This approach opens new directions in model schema design and external tool integration, emphasizing architecture-level interventions over parameter updates.
Limitations
- The method depends on the model’s exposure to certain naming patterns during pretraining; in cases where the model’s training data lacks relevant schemas, effectiveness diminishes.
- Peakedness may not fully capture complex or highly domain-specific naming conventions, limiting its robustness in diverse application areas.
- The current approach is validated mainly on English datasets and benchmarks; multilingual or cross-domain scenarios require further adaptation and testing.
Future Work
Future research will explore extending PA-Tool to multi-language settings and more complex multi-tool environments. Integrating dynamic learning mechanisms could enable models to adapt schemas in real-time as new tools emerge. Additionally, combining schema alignment with reinforcement learning could further improve robustness and flexibility. Investigating the interaction with multimodal data and reasoning tasks will broaden its applicability, aiming toward fully autonomous, adaptable AI agents capable of seamless external tool integration in diverse real-world applications.
AI Executive Summary
The rapid evolution of AI models has highlighted the importance of external tool integration for complex reasoning and task execution. However, resource-efficient small language models (SLMs) often struggle with tool invocation, especially in selecting appropriate tools and parameters. A key challenge is schema misalignment—models hallucinate plausible tool names that do not match the provided schemas, primarily due to internalized naming conventions from pretraining. Existing solutions rely heavily on retraining or manual schema adjustments, which are costly and inflexible.
This paper introduces PA-Tool, a training-free method that leverages peakedness, a signal from contamination detection, to identify models’ internalized naming patterns. By generating multiple candidate names from descriptions, calculating their peakedness via character-level similarity, and selecting the most peaked candidate, PA-Tool aligns tool schemas with the model’s pretrained biases. This approach significantly reduces schema misalignment errors, with experiments on MetaTool and RoTBench showing up to 17% performance improvements and 80% error reductions.
The core innovation lies in repurposing contamination detection signals for schema optimization, enabling resource-efficient models to better utilize external tools without retraining. The method is simple, scalable, and broadly applicable, offering a new pathway for improving tool use in AI systems. Results demonstrate that schema-level interventions can unlock the full potential of resource-constrained models, making them more reliable and effective in real-world multi-tool scenarios. Future work aims to extend this approach to multilingual, multimodal, and dynamic environments, fostering more autonomous and adaptable AI agents.
Deep Analysis
Background
近年来，随着大规模预训练模型（如GPT、LLaMA）的广泛应用，模型在自然语言理解和生成方面取得了巨大突破。多模态、多任务系统逐渐成为研究热点，工具增强技术成为提升模型实用性的重要途径。现有研究主要集中在微调（如监督微调）或描述优化（如描述增强、检索校正）上，以改善模型的工具调用能力。然而，模型在实际应用中仍面临架构偏差、命名不一致等问题，导致工具调用错误频发，尤其在资源有限的小模型中表现尤为明显。这些偏差源自预训练中内化的命名习惯，难以通过微调解决，限制了模型的推广应用。
Core Problem
核心问题在于模型对工具组件名称的偏好与实际工具架构存在偏差，导致命名偏差和调用错误。这种偏差源于预训练中内化的命名习惯，难以通过微调解决，且微调成本高昂。模型在面对不同工具架构时，容易生成不存在的工具名或参数名，影响任务完成率。解决这一问题对于提升小模型的工具利用效率、降低部署门槛具有重要意义，但现有方法缺乏高效、无训练的解决方案。
Innovation
本研究提出PA-Tool，创新点在于：1）利用污染检测中的峰值性指标，评估模型对工具命名的熟悉程度；2）通过多次采样生成候选名称，计算峰值性，自动识别模型偏好的命名模式；3）无需模型微调，直接进行架构层面的工具命名对齐。该方法突破了传统微调依赖的限制，提供了一种高效、可扩展的工具架构优化方案。其核心创新在于将污染检测的偏好信号迁移至工具命名，开启了模型偏好识别与架构优化的新路径。
Methodology
- �� 提供工具组件描述，输入模型，生成多个候选名称。• 采样N次候选（N=32），利用温度t=0.4探索模型偏好。• 计算每个候选名称的Levenshtein距离，定义字符相似性阈值（α=0.2）。• 统计每个候选的峰值性，即在候选集中邻近的相似名称数。• 选择峰值性最高的候选作为偏好名称，构建映射字典。• 将此过程应用于所有工具组件，完成架构对齐。整个流程确保模型偏好与工具架构一致，提升调用准确性。
Experiments
在MetaTool和RoTBench两个基准上验证，采用不同模型（Qwen、Llama）和多配置（原始、贪婪、多候选、人工、PA-Tool），比较准确率提升。候选生成数N=32，温度t=0.4，峰值阈值α=0.2。通过错误分析和消融实验验证峰值性指标的有效性。多轮交互和多工具场景下的性能也得到验证，确保方法的鲁棒性和迁移性。
Results
PA-Tool在MetaTool中，工具选择任务最高提升17%，多工具任务提升9.6%；在RoTBench中，单轮和多轮任务最高提升10%，参数识别提升4.3%。错误类型中，命名偏差错误减少80%，模型对工具偏好的理解显著改善。不同模型规模均获益，尤其在资源有限的小模型中效果最明显。峰值性指标在训练过程中逐步增强，验证其作为偏好信号的有效性。整体表现优于传统微调和描述增强方法。
Applications
该方法适用于多模态、多任务系统中的工具调用优化，尤其在边缘计算、嵌入式设备等资源有限场景。无需模型微调，快速部署即可显著提升工具调用准确率。未来还可结合动态学习机制，实现模型在不断变化的任务环境中的自适应架构调整，推动智能系统的自主演化。
Limitations & Outlook
该方法依赖模型在预训练中接触到的命名模式，若模型未曾遇到相关工具或偏离预训练知识，效果有限。峰值性指标在复杂多工具场景中的鲁棒性有待验证。当前验证主要在英语数据集上，跨语言、多领域迁移仍需深入研究。未来需增强多模态、多语言的适应能力，提升广泛应用的实用性。
Plain Language Accessible to non-experts
想象你在厨房做饭，每个调料瓶都有名字，比如‘盐’、‘糖’、‘胡椒’。有时候，你会忘记某个调料的名字，或者用自己习惯的叫法，比如叫‘白色粉末’。这就像模型在用工具时遇到的问题：它们记得一些工具的名字，但面对新或不同的名字时，会胡乱猜测，甚至叫出不存在的工具。PA-Tool就像一个聪明的助手，它会观察你平时怎么叫这些调料，找到你最常用的名字，然后用这些名字替换掉那些不太熟悉的名字。这样，无论你用什么名字，它都能帮你找到正确的调料，做出美味的菜肴。这种方法不用重新学习，只是调整名字的标签，让厨房变得更顺畅。它让模型像个懂得你习惯的厨师一样，能更准确地调用工具，做出更好的结果。
ELI14 Explained like you're 14
想象你在玩一个游戏，你的朋友给你一堆装备，比如‘剑’、‘盾’、‘魔法棒’，但你有时候会用自己习惯的名字，比如叫‘长刀’或‘防御牌’。如果你用的名字和朋友给的名字不一样，你可能会找不到装备，或者用错装备。PA-Tool就像一个超级聪明的朋友，它会观察你平时怎么叫这些装备，然后帮你把你习惯的名字换成朋友的正式名字。这样一来，不管你用什么名字，它都能帮你找到正确的装备，帮你赢得比赛。这就像让你的游戏角色变得更聪明，更懂你的习惯，不用重新学东西，只是调整名字，让一切变得更顺畅。它让模型更像个懂得你偏好的助手，能更快更准地帮你完成任务！
Abstract
Small language models (SLMs) enable scalable tool-augmented multi-agent systems where multiple SLMs handle subtasks orchestrated by a powerful coordinator. However, they struggle with tool-use tasks, particularly in selecting appropriate tools and identifying correct parameters. A common failure mode is \textit{schema misalignment}: models hallucinate plausible tool names that are absent from the provided tool schema, due to different naming conventions internalized during pretraining. Rather than training models to adapt to unfamiliar schemas, we propose adapting schemas to align with models' pretrained knowledge. We introduce \textbf{PA-Tool} (Pretraining-Aligned Tool Schema Generation), a training-free method that leverages peakedness, a signal used in contamination detection that indicates pretraining familiarity, to rename tool components. By generating multiple candidates and selecting the candidate with the highest peakedness, PA-Tool identifies pretraining-aligned naming patterns. Experiments on MetaTool and RoTBench show improvements of up to 17\%, with schema misalignment errors reduced by 80\%. PA-Tool enables small models to substantially improve tool-use accuracy without retraining, showing that schema-level interventions can unlock the tool-use potential of resource-efficient models. Our code is available at https://github.com/holi-lab/PA-Tool.