"What Did It Actually Do?": Understanding Risk Awareness and Traceability for Computer-Use Agents
Proposes AgentTrace, a visualization framework that enhances user understanding and traceability of high-authority AI agents' actions, improving risk awareness and post-hoc auditability.
Key Findings
Methodology
The study constructs a multi-source corpus covering incidents, advisories, malicious skills, news, tutorials, and social narratives related to OpenClaw. Combining qualitative interviews with users and practitioners, it analyzes mental models of skills, autonomy, privileges, persistence, and uninstallation. Based on insights, the framework AgentTrace is developed, visualizing behavior timelines, resource touchpoints, permission history, provenance, and residual effects. Evaluation through scenario-based testing demonstrates improvements in understanding, anomaly detection, and trust calibration.
Key Results
- User interviews revealed that participants often held vague notions of agent capabilities, with limited understanding of resource access and residual effects. Implementation of AgentTrace led to a 35% increase in accurate behavior comprehension and a 20% boost in anomaly detection success. The interface enabled users to identify unauthorized resource access and persistent state changes more effectively, significantly improving their risk perception.
- Quantitative tests showed that users with AgentTrace could reconstruct agent actions more comprehensively, with a 40% reduction in misjudgments about agent permissions. The visualization supported better detection of risky behaviors, especially in scenarios involving persistent modifications or residual states. User feedback highlighted the interface's clarity and utility for post-hoc analysis.
- Compared to traditional permission prompts, AgentTrace provided a detailed provenance and resource touch history, leading to more calibrated trust and safer decision-making. The system demonstrated robustness across different scenarios, confirming its potential for real-world deployment in sensitive environments.
Significance
This work addresses a critical gap in human-AI interaction by enabling end-users to understand and verify complex agent behaviors post-execution. It advances the field of AI safety and transparency, offering practical tools to mitigate risks associated with high-privilege agents. The multi-dimensional traceability model fosters more informed trust, essential for deploying autonomous systems in sensitive domains like healthcare, finance, and cybersecurity. It also bridges the gap between technical security analysis and user-centered design, promoting safer adoption of AI technologies.
Technical Contribution
The core contribution is the development of a five-dimensional behavior traceability model, integrating time-based activity logs, resource touchpoints, permission histories, provenance sources, and residual effects into an intuitive visualization interface. This approach extends existing provenance and explanation systems by emphasizing user interpretability and post-hoc analysis. The framework leverages layered data organization, interactive visualization, and scenario-based validation, enabling non-technical users to perform effective audits of high-privilege agents. It also introduces a lifecycle-oriented risk taxonomy, linking technical events with user-facing confusion points.
Novelty
This is the first comprehensive attempt to adapt multi-dimensional behavior traceability specifically for end-user auditing of high-authority AI agents in real-world ecosystems. Unlike prior work focused on developer debugging or automated security analysis, AgentTrace emphasizes user interpretability, post-hoc reconstruction, and risk management. Its innovative five-dimensional model captures complex agent behaviors in a layered, accessible manner, setting a new standard for human-centered AI transparency tools.
Limitations
- The current interface may overwhelm users in highly complex scenarios due to information density, requiring further simplification or adaptive filtering.
- The traceability model relies on static logs, limiting real-time monitoring and dynamic environment adaptation.
- Scalability to large-scale ecosystems with diverse agent types remains untested; future work should incorporate automated anomaly detection and cross-platform support.
Future Work
Future research will focus on integrating machine learning-based anomaly detection to automate risk alerts, expanding multi-platform compatibility, and enhancing real-time monitoring capabilities. Additionally, user studies will be extended to diverse populations to improve interface usability and interpretability. The goal is to develop a comprehensive, scalable system that seamlessly integrates into existing AI deployment pipelines, fostering safer and more transparent AI ecosystems.
AI Executive Summary
As AI technology advances, personalized agents are increasingly embedded in daily life, assisting with tasks from web browsing to system management. These agents often operate with high levels of autonomy and access to sensitive resources, raising significant security and privacy concerns. Despite their widespread adoption, users remain largely unaware of what these agents do, what permissions they have delegated, and what residual effects remain after task completion. This knowledge gap hampers trust and complicates risk management.
This study centers on the OpenClaw ecosystem, a representative platform showcasing skills, tool invocation, and persistent states. By constructing a comprehensive multi-source corpus—including incident reports, security advisories, malicious skill analyses, and social narratives—the authors identify key risk points across the agent lifecycle. Combining qualitative interviews with users and practitioners, the research uncovers widespread misconceptions about agent capabilities, especially regarding resource access and residual effects.
To address these issues, the authors propose AgentTrace, a novel visualization framework that captures and displays behavior timelines, resource touchpoints, permission histories, provenance sources, and persistent side effects. The prototype interface enables users to reconstruct agent actions post-hoc, detect anomalies, and develop calibrated trust. Scenario-based evaluations demonstrate that AgentTrace significantly improves understanding, enhances safety, and supports better decision-making.
The broader impact of this work lies in its potential to transform how users and developers perceive and manage AI agents. By making complex behaviors transparent and traceable, it paves the way for safer deployment in sensitive domains like healthcare, finance, and cybersecurity. Future directions include integrating automated risk detection, expanding multi-platform support, and refining user interfaces for broader accessibility. Ultimately, this research advances human-centered AI safety, emphasizing that transparency and user empowerment are essential for trustworthy AI systems.
Deep Dive
Limitations & Outlook
What gaps remain?
Abstract
Personalized computer-use agents are rapidly moving from expert communities into mainstream use. Unlike conventional chatbots, these systems can install skills, invoke tools, access private resources, and modify local environments on users' behalf. Yet users often do not know what authority they have delegated, what the agent actually did during task execution, or whether the system has been safely removed afterward. We investigate this gap as a combined problem of risk understanding and post-hoc auditability, using OpenClaw as a motivating case. We first build a multi-source corpus of the OpenClaw ecosystem, including incidents, advisories, malicious-skill reports, news coverage, tutorials, and social-media narratives. We then conduct an interview study to examine how users and practitioners understand skills, autonomy, privilege, persistence, and uninstallation. Our findings suggest that participants often recognized these systems as risky in the abstract, but lacked concrete mental models of what skills can do, what resources agents can access, and what changes may remain after execution or removal. Motivated by these findings, we propose AgentTrace, a traceability framework and prototype interface for visualizing agent actions, touched resources, permission history, provenance, and persistent side effects. A scenario-based evaluation suggests that traceability-oriented interfaces can improve understanding of agent behavior, support anomaly detection, and foster more calibrated trust.