SciVisAgentSkills: Design and Evaluation of Agent Skills for Scientific Data Analysis and Visualization
SciVisAgentSkills enhances coding agents for scientific data analysis and visualization efficiency.
Key Findings
Methodology
SciVisAgentSkills augments coding agents by encoding environment assumptions, tool usage patterns, and domain heuristics, applicable to scientific tools like ParaView, napari, VMD, and TTK. Evaluated using SciVisAgentBench, these skills improve task scores and show varied token efficiency across different agent harnesses and tool settings.
Key Results
- Agents with skills perform better in task scores on SciVisAgentBench, with Claude Code showing about 60% improvement in topology visualization tasks.
- Skills improve output quality but may introduce new execution paths, leading to a drop in completion rate in some cases.
- Token usage decreases in some tasks, especially noticeable in Claude Code.
Significance
The study highlights the importance of structured procedural knowledge for reliable long-horizon SciVis workflows, suggesting skills should be studied alongside the execution harness. It lays the foundation for expanding the agent skill ecosystem for scientific data analysis and visualization.
Technical Contribution
SciVisAgentSkills provides a reusable layer that enhances general capabilities of foundation models, manages context and tool execution, and injects task- and domain-specific expertise. It improves performance across multiple tools and task types, demonstrating the value of procedural guidance in complex scientific settings.
Novelty
This is the first application of reusable agent skills to core SciVis workflows, validated across various tools and task types. Compared to existing work, it offers broader tool applicability and task coverage.
Limitations
- Gains may be limited when the base model already handles well-documented tools effectively, as seen in VMD for molecular visualization.
- The interaction between skills and the agent harness is model-dependent, with varying token usage across different models.
- Skill benefits vary substantially across task suites, especially in complex multi-step tasks.
Future Work
Future work should explore joint design of skills and agent harnesses to support more complex long-horizon workflows and develop new skills for additional scientific tools and task types.
AI Executive Summary
Scientific data analysis and visualization are crucial tools for interdisciplinary research, yet existing solutions often fall short in handling complex data. SciVisAgentSkills offers a novel solution by enhancing coding agents' skills. These skills augment agent capabilities by encoding environment assumptions, tool usage patterns, and domain heuristics, applicable to scientific tools like ParaView, napari, VMD, and TTK. Experimental results show that agents with these skills perform better in task scores, particularly in complex multi-step tasks. While skills introduce new execution paths that may lead to a drop in completion rate, they generally improve output quality. The study emphasizes the importance of structured procedural knowledge for reliable long-horizon SciVis workflows and calls for community collaboration to expand the agent skill ecosystem for scientific data analysis and visualization. Future work should explore joint design of skills and agent harnesses to support more complex long-horizon workflows and develop new skills for additional scientific tools and task types.
Deep Analysis
Background
Scientific data analysis and visualization are essential tools for understanding complex scientific data. As data complexity increases, traditional visualization methods struggle to meet demands. Recently, agentic visualization systems have demonstrated the feasibility of long-horizon workflows by converting natural language requests into concrete visualization actions. However, these systems are often tailored to specific tools or applications, lacking generality.
Core Problem
Existing coding agents often lack tool-specific expertise when handling scientific data analysis and visualization tasks. This leads to inefficiency in executing complex tasks, especially those requiring long-horizon, multi-step workflows.
Innovation
SciVisAgentSkills provides a reusable layer of agent skills that enhance the general capabilities of foundation models. These skills encode environment assumptions, tool usage patterns, and domain heuristics, applicable to multiple scientific tools, offering broader task coverage.
Methodology
- �� Encode environment assumptions to reduce redundant exploration
- �� Provide tool usage patterns to guide agent behavior
- �� Include representative code snippets to reduce trial-and-error
- �� Incorporate empirical observations to optimize output generation
Experiments
Evaluated using SciVisAgentBench, which includes 108 expert-designed multi-step tasks covering diverse domains, data types, and visualization operations. The effectiveness of skills is assessed by comparing agent performance with and without skills, focusing on task scores and token usage.
Results
Agents with skills perform better in task scores, particularly in complex multi-step tasks. Claude Code shows about 60% improvement in topology visualization tasks. While skills introduce new execution paths that may lead to a drop in completion rate, they generally improve output quality.
Applications
SciVisAgentSkills can be applied in various fields of scientific data analysis and visualization, including physics, biology, climate science, and materials science. They provide more reliable long-horizon workflows, improving efficiency in interdisciplinary research.
Limitations & Outlook
Skill benefits vary substantially across task suites, especially in complex multi-step tasks. The interaction between skills and the agent harness is model-dependent, with varying token usage across different models.
Plain Language Accessible to non-experts
Imagine you're in a kitchen cooking a meal. Existing coding agents are like novice chefs who know how to chop ingredients but aren't familiar with the specific cooking steps. SciVisAgentSkills are like a detailed recipe, guiding the chef on how to use different tools and ingredients to complete a complex dish. By providing clear steps and techniques, these skills help the chef complete tasks more efficiently.
ELI14 Explained like you're 14
Imagine you're playing a complex video game. You need to complete multiple tasks to win the game. Existing coding agents are like new players who know basic controls but aren't familiar with the game rules. SciVisAgentSkills are like a game guide, showing players how to use different items and skills to complete tasks. By providing clear steps and techniques, these skills help players complete tasks more efficiently.
Glossary
SciVisAgentSkills
A set of reusable agent skills that enhance coding agents' capabilities for scientific data analysis and visualization.
Used in the paper to improve agent performance in scientific tools.
Codex
A coding agent capable of executing commands, inspecting files, writing code, and iteratively refining outputs.
Used in the paper to evaluate the effectiveness of agent skills.
Claude Code
A coding agent capable of executing commands, inspecting files, writing code, and iteratively refining outputs.
Used in the paper to evaluate the effectiveness of agent skills.
SciVisAgentBench
A benchmark for evaluating scientific data analysis and visualization agents, comprising 108 expert-designed multi-step tasks.
Used in the paper to assess the effectiveness of agent skills.
ParaView
An end-user tool for large-data visualization.
Used in the paper as one of the tools for evaluating agent skills.
Open Questions Unanswered questions from this research
- 1 How to apply agent skills to more scientific tools to support more complex tasks and workflows.
- 2 How to optimize the interaction between skills and agent harnesses to improve token efficiency.
Applications
Immediate Applications
Scientific Data Analysis
Enhancing coding agents' skills to improve efficiency and accuracy in scientific data analysis. Applicable in fields like physics and biology.
Long-term Vision
Interdisciplinary Research
Expanding the agent skill ecosystem to support more complex interdisciplinary research, improving scientific discovery efficiency.
Abstract
Recent advances in agentic visualization have enabled the translation of natural language into executable scientific visualization (SciVis) workflows. While general-purpose coding agents show strong capabilities, they often lack the tool-specific expertise required for SciVis tasks. In this work, we present SciVisAgentSkills, a collection of reusable agent skills that augment coding agents for scientific data analysis and visualization by encoding environment assumptions, tool usage patterns, and domain heuristics across scientific tools such as ParaView, napari, VMD, and TTK. We evaluate these skills on Codex and Claude Code using SciVisAgentBench, a benchmark of 108 expert-designed multi-step tasks. Results show that agent skills improve mean task scores across the evaluated suites, with token-efficiency benefits that depend on the agent harness and tool setting. These findings highlight the importance of structured procedural knowledge for enabling reliable, long-horizon SciVis workflows, while also showing that skills should be studied alongside the execution harness that loads and applies them. The skills are available at https://github.com/KuangshiAi/SciVisAgentSkills.