MyPCBench: A Benchmark for Personally Intelligent Computer-Use Agents
MyPCBench tests personal intelligent computing agents; Claude Opus 4.6 solves 55.4% of tasks.
Key Findings
Methodology
MyPCBench simulates 17 real-world web applications on a Linux desktop to test personal intelligent computing agents. Each task is based on real requests, evaluated with a uniform computer and bash tool interface.
Key Results
- Claude Opus 4.6 fully solves 55.4% of 184 tasks, the only model above 50%.
- Model failures cluster on tasks spanning many applications and long trajectories.
- In tasks involving 7+ applications, Claude Opus 4.6's perfect rate drops to 36%.
Significance
MyPCBench bridges the gap between existing benchmarks and real-world deployment, especially in web tasks requiring personalization and login information. It sets a new standard for evaluating personal assistants across a user's digital life.
Technical Contribution
MyPCBench introduces a reproducible evaluation platform with consistent user identity and cross-application history in a full Linux desktop environment, allowing more realistic testing of personalized agents.
Novelty
This is the first benchmark to seed a coherent user identity at the scale of a full personal computer, filling a gap in existing evaluation methods.
Limitations
- Models perform poorly on tasks spanning multiple applications, especially long trajectories.
- The current environment is limited to Linux desktops, which may not apply to other OS.
Future Work
Future work can expand to other operating systems and improve model performance on multi-application tasks.
AI Executive Summary
MyPCBench is a novel benchmark focused on evaluating personal intelligent computing agents in real-world scenarios. Existing benchmarks often operate in non-personalized environments, failing to accurately reflect the challenges of real-world use cases. MyPCBench provides a more realistic testing environment by simulating 17 real-world web applications on a Linux desktop.
In experiments, Claude Opus 4.6 fully solved 55.4% of 184 tasks, demonstrating strong capabilities in personalized tasks. However, models performed poorly on tasks spanning multiple applications, especially long trajectories. This indicates that current models still have room for improvement in handling complex personalization needs.
The release of MyPCBench provides researchers and developers with a new tool to better evaluate and improve the performance of personal intelligent computing agents. Future research can expand to other operating systems and further optimize model performance on multi-application tasks.
Deep Analysis
Background
With the rise of personal intelligent computing agents, evaluating their performance in real-world scenarios has become increasingly important. Existing benchmarks often operate in non-personalized environments, failing to accurately reflect the challenges of real-world use cases. MyPCBench provides a more realistic testing environment by simulating 17 real-world web applications on a Linux desktop.
Core Problem
Existing benchmarks operate in non-personalized environments, failing to accurately evaluate personal assistants across a user's digital life. This gap is particularly evident in web tasks requiring personalization and login information.
Innovation
MyPCBench introduces a reproducible evaluation platform with consistent user identity and cross-application history in a full Linux desktop environment, allowing more realistic testing of personalized agents.
Methodology
- �� Simulate 17 real-world web applications on a Linux desktop.
- �� Evaluate using a uniform computer and bash tool interface.
- �� Each task is based on real requests, testing model performance in personalized tasks.
Experiments
Experiments are conducted in a full Linux desktop environment, featuring 17 simulated web applications and 184 tasks. A uniform computer and bash tool interface is used to evaluate six models on personalized tasks.
Results
Claude Opus 4.6 fully solves 55.4% of 184 tasks, the only model above 50%. Models perform poorly on tasks spanning multiple applications, especially long trajectories.
Applications
MyPCBench sets a new standard for evaluating personal intelligent computing agents across a user's digital life, applicable to researchers and developers.
Limitations & Outlook
The current environment is limited to Linux desktops, which may not apply to other OS. Models perform poorly on tasks spanning multiple applications, especially long trajectories.
Plain Language Accessible to non-experts
Imagine you have a super assistant that helps you with all your computer tasks. MyPCBench is like a testing ground for this assistant. In this world, there are 17 different applications, like banking, email, and calendar. Each application has data from your life, like bank transactions and emails. MyPCBench tests if these assistants can complete tasks in this complex environment, like paying bills or scheduling meetings. Through this test, we can see which assistant is the smartest and can help you the best.
ELI14 Explained like you're 14
Imagine you have a super-smart computer assistant that can help you with everything! MyPCBench is like a virtual playground with 17 different apps, like banking, email, and calendar. Each app has data from your life, like bank transactions and emails. MyPCBench tests if these assistants can complete tasks in this complex environment, like paying bills or scheduling meetings. Through this test, we can see which assistant is the smartest and can help you the best.
Glossary
Benchmark
A standardized test method for evaluating system or model performance.
Used to evaluate personal intelligent computing agents in MyPCBench.
Personal Assistant
A software agent designed to help users complete personal tasks.
MyPCBench tests personal assistants in complex tasks.
Linux Desktop
A desktop environment based on the Linux operating system.
MyPCBench is conducted on a Linux desktop.
Claude Opus 4.6
A high-performance personal intelligent computing agent model.
The best-performing model in MyPCBench.
Multi-application
Tasks or environments involving multiple applications.
MyPCBench tests model performance in multi-application tasks.
Open Questions Unanswered questions from this research
- 1 How to improve model performance on tasks spanning multiple applications?
- 2 How to expand MyPCBench to support other operating systems?
Applications
Immediate Applications
Personal Assistant Evaluation
Researchers can use MyPCBench to evaluate and improve the performance of personal intelligent computing agents.
Long-term Vision
Cross-platform Support
Future expansions of MyPCBench could support other operating systems, increasing its applicability.
Abstract
Current benchmarks for computer-use agents evaluate models in impersonal environments. This leaves a gap between evaluation and deployment where personal assistants are expected to work across a user's whole digital life, including their context, historical data, and logged-in accounts. This gap is widest on web tasks, where live web evaluations cannot exercise sites that require logging in or personal information, the kind of site a real personal assistant has to drive. We introduce MyPCBench, which tests computer-use agents as personal assistants on a Linux desktop populated with 17 simulated real-world web applications and a full desktop stack, all seeded for one canonical persona, Michael Scott from The Office. We define 184 tasks in this environment, each inspired by a real request drawn from the OpenClaw community, and benchmark six closed and open-weight models with a uniform computer+bash tool surface. We find that the best model, Claude Opus 4.6, fully solves 55.4\% of the tasks, the only model above 50\%. Model failures cluster on tasks that span many applications and on long trajectories, where personalization stresses an assistant the most. We release the environment, task set, and agent harness at https://mypcbench.com.