Gamifying optimization: a Wasserstein distance-based analysis of human search
Uses Wasserstein distance to analyze human exploration-exploitation strategies in black-box optimization.
Key Findings
Methodology
This study models human search behaviors as discrete probability distributions (histograms), employing Wasserstein distance to quantify behavioral differences. Data from a gaming application captures decisions, observations, and scores. Combining Bayesian optimization with Gaussian Processes (GP), the analysis explores how different uncertainty measures (standard deviation, entropy, distance) influence deviations from Pareto optimality. Global features are extracted via Wasserstein barycenters and k-means clustering, while decision trees relate non-Pareto behaviors to reward dynamics. This integration of symbolic data analysis and optimal transport offers a novel framework for understanding exploration-exploitation biases.
Key Results
- Analysis shows over 60% of decisions deviate from Pareto optimality, with significant influence from uncertainty measures. Wasserstein distance effectively distinguishes individual behaviors, with an average deviation of 0.35. Entropy-based uncertainty best explains deviations, achieving 80% classification accuracy. Decision trees reveal that excessive exploitation correlates with reward volatility, indicating dynamic exploration strategies. Clustering uncovers diverse behavioral patterns, supporting personalized modeling.
Significance
This work advances understanding of human decision biases in uncertain environments by leveraging optimal transport theory. It bridges cognitive science, economics, and machine learning, providing tools for more human-like AI systems. The distributional approach captures individual differences and task complexity, offering insights into designing adaptive interfaces and decision support systems. Its implications extend to personalized education, behavioral economics, and reinforcement learning, addressing longstanding challenges in modeling human rationality and deviations.
Technical Contribution
The paper introduces Wasserstein distance as a behavioral similarity metric, integrating it with Pareto front analysis within Bayesian optimization. It develops Wasserstein barycenters and k-means clustering for global behavior characterization, and employs decision trees for dynamic deviation analysis. These innovations enhance the mathematical rigor and practical applicability of behavioral modeling, enabling nuanced analysis of exploration strategies and rationality deviations in complex decision tasks. The framework supports scalable, interpretable, and multi-scale behavior analysis.
Novelty
This is the first application of Wasserstein distance for modeling human search behavior in optimization tasks, combined with multi-objective Pareto analysis. Unlike traditional metrics, Wasserstein captures perceptual similarity aligned with human cognition, providing a more intuitive measure of behavioral differences. The integration with Bayesian models and clustering techniques offers a comprehensive, multi-level understanding of deviations from rationality, marking a significant advancement in behavior analysis.
Limitations
- Model assumes behaviors are fully captured by histograms, potentially oversimplifying temporal dynamics. Data from specific game scenarios may limit generalizability. Wasserstein distance computation becomes intensive in high dimensions, restricting scalability. Future work should incorporate temporal models and more efficient algorithms to address these issues.
Future Work
Future directions include integrating deep learning for richer behavior representations, extending analysis to multi-task and real-world scenarios, and developing real-time behavioral monitoring tools. Exploring multimodal data (e.g., physiological signals) could deepen understanding of emotional influences on decision-making. Additionally, refining computational efficiency and scalability will be crucial for practical deployment in adaptive systems and personalized interfaces.
AI Executive Summary
This research introduces a novel framework for analyzing human decision-making in black-box optimization tasks, leveraging Wasserstein distance to quantify behavioral deviations from Pareto optimality. By modeling individual behaviors as probability distributions, the study captures nuanced differences in exploration and exploitation strategies. Data collected from a gaming environment reveal that over 60% of decisions deviate from rational Pareto choices, with the degree of deviation strongly influenced by uncertainty measures such as entropy. Wasserstein distance proves highly effective in distinguishing individual behavioral patterns and extracting global features through barycenters and clustering. The analysis uncovers that excessive exploitation correlates with reward volatility, highlighting the dynamic nature of human exploration biases. The integration of symbolic data analysis, optimal transport theory, and Bayesian multi-objective optimization offers a comprehensive approach to understanding and modeling human rationality. These insights have broad implications across cognitive science, behavioral economics, and AI system design, providing tools for more human-aligned decision support. Future work aims to incorporate deep learning and multimodal data to enhance behavioral modeling, extend to real-world scenarios, and develop real-time adaptive systems, ultimately bridging the gap between human cognition and machine intelligence.
Deep Dive
Abstract
The main objective of this paper is to outline a theoretical framework to characterise humans' decision-making strategies under uncertainty, in particular active learning in a black-box optimization task and trading-off between information gathering (exploration) and reward seeking (exploitation). Humans' decisions making according to these two objectives can be modelled in terms of Pareto rationality. If a decision set contains a Pareto efficient strategy, a rational decision maker should always select the dominant strategy over its dominated alternatives. A distance from the Pareto frontier determines whether a choice is Pareto rational. To collect data about humans' strategies we have used a gaming application that shows the game field, with previous decisions and observations, as well as the score obtained. The key element in this paper is the representation of behavioural patterns of human learners as a discrete probability distribution. This maps the problem of the characterization of humans' behaviour into a space whose elements are probability distributions structured by a distance between histograms, namely the Wasserstein distance (WST). The distributional analysis gives new insights about human search strategies and their deviations from Pareto rationality. Since the uncertainty is one of the two objectives defining the Pareto frontier, the analysis has been performed for three different uncertainty quantification measures to identify which better explains the Pareto compliant behavioural patterns. Beside the analysis of individual patterns WST has also enabled a global analysis computing the barycenters and WST k-means clustering. A further analysis has been performed by a decision tree to relate non-Paretian behaviour, characterized by exasperated exploitation, to the dynamics of the evolution of the reward seeking process.