Gricea: An Open Science Platform for Conversational AI Research
Gricea platform supports CAI study construction, replication, and extension through shared research artifacts.
Key Findings
Methodology
Gricea platform supports CAI research through configurable research artifacts. Researchers can design, run, inspect, and share studies on the platform. It couples study procedures, participant systems, and conversational task behavior, simplifying complex interactive task configurations through visual tools.
Key Results
- In a replication study using Gricea, 93% of eligible CUI 2026 paper configurations were successfully replicated, while missing information was flagged in 96% of papers.
- User studies showed researchers from diverse backgrounds successfully constructed runnable studies addressing various open-ended research questions.
- Gricea supports constructing, reproducing, and extending CAI studies through shared research artifacts.
Significance
Gricea platform addresses the fragmentation issue in CAI research systems and configurations, promoting knowledge accumulation. It provides researchers a common starting point to reproduce conditions, systematically vary design choices, and relate new findings to prior work.
Technical Contribution
Gricea offers a shared, executable representation of CAI studies connecting study procedure, task behavior, participant conditions, and instrumentation, enabling researchers to inspect, reproduce, and extend experimental designs. The platform lowers technical barriers, making individual studies resources for subsequent research.
Novelty
Gricea is the first platform to represent CAI study designs as configurable, deployable research artifacts. It simplifies complex interactive task configurations through visual tools, offering broader research infrastructure compared to existing systems.
Limitations
- Gricea may require additional technical support for certain complex task configurations.
- The platform's visual tools may face performance issues when handling very large-scale studies.
Future Work
Future work includes expanding Gricea to support more types of CAI studies and improving the platform's visual tools to handle more complex study configurations.
AI Executive Summary
Gricea is an open science platform aimed at addressing fragmentation in conversational AI research. Existing CAI studies often face replication and extension challenges due to fragmented systems and study configurations. Gricea facilitates knowledge accumulation by allowing researchers to design, run, inspect, and share studies through shared research artifacts.
The Gricea platform supports CAI research through configurable research artifacts. Researchers can design, run, inspect, and share studies on the platform. It couples study procedures, participant systems, and conversational task behavior, simplifying complex interactive task configurations through visual tools.
In a replication study using Gricea, 93% of eligible CUI 2026 paper configurations were successfully replicated, while missing information was flagged in 96% of papers. User studies showed researchers from diverse backgrounds successfully constructed runnable studies addressing various open-ended research questions.
Deep Dive
Abstract
We need studies on conversational AI (CAI) at scale to understand human behavior and shape CAI design. However, fragmented reporting of systems and study configurations hinders replication, extension, and knowledge accumulation. We present Gricea, an open-science platform representing studies as configurable, deployable research artifacts that researchers can run, inspect, share, and reuse. Informed by a formative analysis of prior CAI research, Gricea couples study procedures, participant-facing systems, and conversational task behavior in. In a replication study using Gricea, we replicated configurations 93% of eligible CUI 2026 papers; while also flagging missing information in 96% of papers that hinder faithful replication --- further motivating Gricea's need. In a user study, researchers and practitioners from diverse backgrounds successfully constructed runnable studies addressing various open-ended research questions. Together, these findings demonstrate Gricea's support for constructing, reproducing, and extending CAI studies through shared research artifacts, enabling cumulative knowledge building through open science.