Simulating Teams with LLM Agents: Interactive 2D Environments for Studying Human-AI Dynamics
VirT-Lab simulates customizable LLM teams in 2D worlds, validated through rescue alignment, ablations, a 12-person user study, and scaling tests.
Key Findings
Methodology
VirT-Lab combines natural-language scenario authoring, LLM agents, a 2D spatial world, and event scheduling. Users specify demographics, roles, personalities, backstories, memories, tasks, entities, and maps. An Initialization Module creates the initial state; the Agent–Environment Interface supports perception and action; and a Simulation Manager advances time, location, actions, and communication while preserving short- and long-term memory.
Key Results
- The paper evaluates VirT-Lab through component ablations, a rescue-mission simulation compared with empirical teams, a 12-participant user study, and numerical studies varying agent count and environmental complexity. The abstract reports alignment with empirical evaluations, but the supplied text contains no accuracy, effect-size, or performance figures.
- Table 1 compares OASIS, Generative Agents, AutoGen, MindAgent, AgentCoord, and AgentSociety. VirT-Lab is the only listed framework marked as supporting all five capabilities: Web UI, customized simulation environment, customized scenario, task solving, and spatial management.
- Outputs include team- and agent-level performance metrics, action and communication logs, agent–environment traces, and post-simulation interviews. These support analysis of both process and outcome variables, although the excerpt gives no numerical ablation results.
Significance
VirT-Lab addresses the cost, access, ethical constraints, and weak counterfactual control of studying real teams. It also targets the rigidity of rule-based Agent-Based Modeling, the training burden of Reinforcement Learning, and the conversational or task-specific scope of many LLM frameworks. By connecting team cognition with scalable agent simulation, it offers researchers, practitioners, and non-experts a controlled way to construct repeatable experiments.
Technical Contribution
The contribution is primarily a full-stack simulation architecture rather than a new LLM training algorithm. It integrates natural-language configuration, role and memory initialization, 2D spatial state, event scheduling, dyadic and multi-party dialogue, visualization, and structured evaluation. Actions, rationales, movement, communication, and environmental changes are recorded in a unified trace, making experiments inspectable and enabling iterative editing of roles, environments, and tasks.
Novelty
Compared with dialogue-centered AutoGen, Generative Agents, and map-specific AgentCoord, VirT-Lab combines user-authored 2D environments, natural-language scenario construction, spatial movement, and team-level evaluation. The paper does not establish universal superiority over baselines; its central novelty is therefore integrative and human-centered: making spatial, customizable team simulation accessible through a web interface rather than introducing a new foundation-model algorithm.
Limitations
- LLM agents can hallucinate, drift from assigned roles, or produce inconsistent reasoning during long interactions. The paper explicitly identifies social norms, organizational structures, and stable relationships as difficult to model reliably.
- The supplied manuscript does not report the model names, prompts, map sizes, costs, repetitions, metric formulas, or numerical scores. This prevents rigorous reproduction and makes the strength of real-team alignment impossible to assess.
- A 2D representation and textual memory may simplify emotion, power, culture, embodiment, and accountability. Simulated behavior can therefore reflect model priors and author prompts more than human team psychology.
Future Work
Future work should release code, model versions, prompts, seeds, and raw traces; construct cross-domain benchmarks; and preregister behavioral-validity metrics. Important extensions include uncertainty-aware agents, dynamic role reassignment, human-in-the-loop teams, long-term memory evaluation, cost optimization, and systematic study of bias, privacy, misuse, and accountability in high-stakes applications.
AI Executive Summary
Teamwork is central to disaster response, healthcare, and science, yet observing real teams is expensive, unpredictable, ethically constrained, and poorly suited to counterfactual experiments. Traditional Agent-Based Modeling uses hand-written rules, Reinforcement Learning requires costly training, and many LLM systems remain tied to fixed roles, linear workflows, dialogue-only interaction, or specialized maps.
The authors introduce Virtual Teaming Laboratory, or VirT-Lab. Through a web interface and natural-language prompts, users create agents, roles, tasks, entities, and 2D environments. The Initialization Module converts descriptions into an initial state; LLM agents perceive the world and select actions through an Agent–Environment Interface; and the Simulation Manager schedules time, movement, operations, and dyadic or multi-party communication. Conversations update short- and long-term memory. The interface exposes maps, actions, rationales, and dialogue, while final outputs include team metrics, individual metrics, interaction logs, and post-simulation interviews.
Evaluation includes ablations, a rescue mission compared with empirical teams, a 12-person user study, and scaling experiments with varying agent numbers and environmental complexity. The paper reports alignment with empirical evaluations, but the supplied text contains no numerical scores, percentages, datasets, or model details. VirT-Lab’s strongest contribution is consequently an accessible, auditable, and customizable research platform—not a new reasoning algorithm. Hallucination, role instability, limited behavioral validation, and computational cost remain substantial open challenges.
Deep Analysis
Background
Team simulation has evolved from rule-based Agent-Based Modeling and Reinforcement Learning toward LLM-based agents. KABOOM models cognitive styles through exploration and exploitation; Meimandi et al. study trust with RL; Leibo et al. examine cooperation and competition. Generative Agents, AutoGen, and AgentCoord add memory, dialogue, and coordination, but commonly lack open scenario authoring, general spatial environments, or integrated visual analytics.
Core Problem
The central problem is how to simulate adaptive teamwork without hard-coding every behavior, while representing movement, communication, memory, role change, and environmental constraints. The system must also maintain event order, reduce hallucination-driven incoherence, support meaningful process metrics, and remain usable by researchers who cannot build a multi-agent backend from scratch.
Innovation
- ��Natural-language authoring lets users define agents, roles, tasks, entities, and maps.
- ��A customizable 2D world makes location, movement, and object interaction part of coordination.
- ��The Simulation Manager synchronizes time, actions, positions, and communication.
- ��Memory-aware dialogue supports dyadic and multi-party interaction over time.
- ��A Web UI visualizes behavior and lowers adoption barriers.
- ��Structured team and agent metrics connect observable processes to outcomes.
Methodology
- ��Input: natural-language scenario, agent attributes, objectives, entities, and map parameters.
- ��Initialization: the Initialization Module instantiates agents, memories, objects, and spatial layout.
- ��Decision: each LLM agent receives goals, state, observations, history, and dialogue context, then proposes an action or message.
- ��Execution: the Agent–Environment Interface applies valid actions and updates locations, objects, and state.
- ��Scheduling: the Simulation Manager orders events and coordinates conversations.
- ��Memory: interactions modify short- and long-term memory, preserving relevant history.
- ��Termination: the run ends after a specified duration or objective completion.
- ��Analysis: the system exports rationales, actions, communication, traces, team metrics, agent metrics, and post-hoc interviews.
Experiments
The evaluation has four parts: systematic component ablations; a rescue-mission simulation aligned with real-team empirical evaluation; a user study with 12 participants assessing usability, interpretability, and perceived realism; and numerical scaling studies varying agent count and environmental complexity. Table 1 provides a capability comparison with OASIS, Generative Agents, AutoGen, MindAgent, AgentCoord, and AgentSociety. The provided text does not specify datasets, LLMs, prompts, hyperparameters, repetitions, or metric equations.
Results
The authors report that simulated outcomes align with empirical evaluations and argue that the architecture provides realistic, customizable team dynamics. In Table 1, VirT-Lab uniquely checks every listed capability: Web UI, customized environment, customized scenario, task solving, and spatial management. The 12-person study supports claims of usability and interpretability, but the excerpt provides no means, significance tests, baseline scores, or ablation deltas. Numerical cross-scenario conclusions therefore cannot be independently quantified from the supplied material.
Applications
Researchers can manipulate map layout, information access, role assignments, communication norms, and task demands to study coordination. Training organizations can prototype rescue, healthcare, or project-team exercises and inspect traces for bottlenecks. Deployment requires domain-reviewed scenarios, explicit success metrics, model-access budgets, privacy controls, and human validation; generated behavior should be treated as a hypothesis generator rather than a forecast of real personnel.
Limitations & Outlook
VirT-Lab inherits LLM weaknesses: hallucinations, repetitive actions, role drift, prompt sensitivity, and unstable long-horizon reasoning. A 2D world cannot fully encode embodiment, emotion, hierarchy, culture, or real consequences. The excerpt lacks key reproducibility details and does not establish external validity across models, languages, or high-stakes domains. Future work should use longitudinal real-team comparisons, standardized benchmarks, uncertainty estimates, and cost–fidelity analyses.
Plain Language Accessible to non-experts
Think of VirT-Lab as a rehearsal room for a rescue team. A researcher builds a map, places several virtual teammates inside it, and gives each one a different background: one knows the routes, one handles medical care, and one manages radio calls. Instead of following a completely fixed script, each teammate watches what happens, listens to others, remembers important events, and chooses what to do next.
A backstage coordinator keeps the rehearsal orderly. It decides when someone moves, discovers an object, sends a message, or changes the situation. The screen shows where everyone is and records what they say and do. Afterward, the system reports whether the group met its goal, how much each member contributed, and which steps helped or hurt cooperation.
Researchers can replay the same rehearsal while changing only the map, number of teammates, or information available. This creates a safe way to ask “what if?” questions that would be expensive or risky with real people. But the virtual teammates are still generated by language models: they may sound confident while being wrong, forget duties, or behave unlike humans. The tool is best for testing ideas before confirming them with real teams.
ELI14 Explained like you're 14
Imagine a co-op rescue game where your teammates are AI characters instead of classmates. You design the level: where the walls are, what objects are hidden, who is the medic, who is good at navigation, and what the team must accomplish. Then the characters chat, walk around, search for things, and make plans on their own. Cool, right?
VirT-Lab is like the game’s director. It keeps track of time, places, actions, and messages so the story does not become total chaos. If one character hears that a hallway is dangerous, that information can affect what they remember and do later. You can watch everything on a 2D map instead of digging through endless chat logs.
Scientists can run the same mission many times and change one thing: the map, team size, job assignments, or available information. They can then ask why one team succeeded and another failed. The paper tested rescue simulations, removed system components, involved 12 users, and varied the number of agents and environmental complexity. However, the supplied text does not include exact scores or a named dataset.
There is an important catch! AI teammates are not automatically realistic humans. They can confidently invent facts, forget their jobs, or choose a plan that sounds smart but fails in the real world. So VirT-Lab is useful for experiments, practice, and discovering questions—not for replacing real people or making high-stakes decisions by itself.
Glossary
Large Language Model (LLM)
A model that generates language and action proposals from context. Technically, it predicts continuations conditioned on prompts, state descriptions, and interaction history.
VirT-Lab uses an LLM as the decision and communication engine for each simulated agent.
Agent-Based Modeling (ABM)
A modeling approach that represents a system as interacting individuals with attributes and rules. Collective behavior emerges from those local interactions.
The paper contrasts VirT-Lab’s flexible LLM behavior with conventional rule-based ABMs.
Reinforcement Learning (RL)
A learning paradigm in which an agent improves actions through trial, error, and reward. It can adapt, but often requires substantial training and parameter tuning.
RL is discussed as a major predecessor for adaptive team simulation.
Simulation Manager
The orchestration component that advances simulation time and orders events. It coordinates locations, actions, environment updates, and communication.
VirT-Lab uses it to maintain temporal consistency across multiple agents.
Short- and Long-term Memory
Short-term memory stores recent context, while long-term memory retains more durable experiences. Both influence later decisions.
Agent conversations update these memories to support continuity across the simulation.
Ablation Study
An experiment that removes or disables one component at a time to measure its contribution. It helps distinguish essential mechanisms from incidental features.
VirT-Lab uses ablations to assess the importance of architectural components.
Open Questions Unanswered questions from this research
- 1 How closely do VirT-Lab behaviors match human teams? The paper claims empirical alignment, but the supplied material lacks validity metrics, confidence intervals, and cross-task evidence.
- 2 Are findings stable across LLMs, prompts, seeds, and runs? Missing model, cost, repetition, and reproducibility details prevent robust assessment of transferability.
- 3 Can a 2D textual world represent hierarchy, emotion, culture, and embodiment? Longitudinal studies with real teams and richer multimodal environments are needed.
Applications
Immediate Applications
Rescue-team rehearsal
Emergency organizations can configure maps, obstacles, roles, and communication constraints, then compare alternative coordination policies. Experts should review scenarios and metrics; outputs can reveal bottlenecks but should not replace certified field training.
Organizational collaboration research
Companies and research groups can simulate projects involving missing information, resource conflict, and changing responsibilities. Action and dialogue traces can expose coordination breakdowns and generate hypotheses for subsequent human-subject studies.
Long-term Vision
Infrastructure for human–AI team science
VirT-Lab could become a cross-domain, multilingual benchmark for transparency, adaptation, fairness, and role coordination in AI teammates. Major obstacles include behavioral validity, privacy, bias control, reproducibility, and the cost of large-scale simulation.
Abstract
Enabling users to create their own simulations offers a powerful way to study team dynamics and performance. We introduce VirTLab, a system that allows researchers and practitioners to design interactive, customizable simulations of team dynamics with LLM-based agents situated in 2D spatial environments. Unlike prior frameworks that restrict scenarios to predefined or static tasks, our approach enables users to build scenarios, assign roles, and observe how agents coordinate, move, and adapt over time. By bridging team cognition behaviors with scalable agent-based modeling, our system provides a testbed for investigating how environments influence coordination, collaboration, and emergent team behaviors. We demonstrate its utility by aligning simulated outcomes with empirical evaluations and a user study, underscoring the importance of customizable environments for advancing research on multi-agent simulations. This work contributes to making simulations accessible to both technical and non-technical users, supporting the design, execution, and analysis of complex multi-agent experiments.