Estimation of discrete choice models with hybrid stochastic adaptive batch size algorithms
Proposes HAMABS, a hybrid adaptive batch size stochastic algorithm, achieving 23x speedup in discrete choice model estimation.
Key Findings
Methodology
This paper introduces a hybrid framework combining stochastic Hessian approximation, adaptive batch size adjustment via window moving average, and dynamic algorithm switching based on batch scale. The stochastic Hessian leverages low-dimensional parameter space for efficient second-order information. Batch size adapts during optimization to balance precision and efficiency, with the switch from first-order to second-order methods triggered by batch size thresholds. Extensive testing across ten benchmark models with fifteen algorithms demonstrates HAMABS's superior performance, notably a 23-fold reduction in maximum model estimation time.
Key Results
- HAMABS accelerates the largest model's optimization by a factor of 23 compared to traditional methods, significantly reducing computation time while maintaining high parameter estimation accuracy.
- Across ten models, HAMABS consistently shows faster convergence and higher stability, with parameter estimates suitable for behavioral interpretation.
- Incorporating stochastic Hessian improves convergence precision, ensuring reliable behavioral indicators like Value of Time (VoT).
Significance
This work addresses the computational bottleneck in large-scale discrete choice model estimation. By integrating second-order information with adaptive batch strategies, it overcomes the limitations of classical methods that struggle with big data and complex models. The approach enhances both speed and accuracy, enabling practical deployment in transportation planning, market analysis, and policy modeling. Its ability to drastically cut estimation time opens new avenues for real-time decision support and large-scale behavioral simulations, marking a significant advance in the field.
Technical Contribution
The core innovation combines stochastic Hessian approximation, adaptive batch size control, and multi-strategy switching, creating a flexible, efficient optimization framework. Unlike standard quasi-Newton or gradient methods, this approach ensures high-precision parameter estimates with reduced computational costs. The theoretical guarantees include convergence under stochastic approximation conditions, and the engineering implementation allows seamless integration into existing estimation software. This framework broadens the scope of scalable, accurate discrete choice model estimation.
Novelty
This is the first systematic integration of stochastic second-order information, adaptive batch size, and algorithm switching tailored specifically for discrete choice models. While prior work focused separately on each aspect, this paper combines all three, resulting in a highly efficient, accurate estimation method. The approach addresses the unique needs of behavioral models—high precision and interpretability—unlike typical machine learning algorithms optimized solely for prediction. Its novelty lies in bridging the gap between large-scale data processing and behavioral modeling requirements.
Limitations
- The stochastic Hessian computation, while efficient for models with hundreds of parameters, may still be costly for extremely high-dimensional models (thousands of parameters).
- Batch size adjustment relies on window parameters that require tuning for different datasets and models, potentially affecting robustness.
- Current validation is limited to transportation and behavioral models; application to other domains like finance or healthcare needs further testing.
Future Work
Future research will explore multi-level Hessian approximations, deeper integration with deep learning architectures, and cross-domain validation. Enhancing robustness of batch size tuning and extending theoretical convergence guarantees under broader conditions are also planned. Additionally, real-time adaptive algorithms for streaming data environments are envisioned to broaden practical deployment.
AI Executive Summary
In the era of big data, estimating discrete choice models efficiently remains a challenge. Traditional methods like quasi-Newton and gradient descent struggle with large datasets and complex models, often taking hours or days to converge. To address this, the authors propose HAMABS, a hybrid stochastic optimization algorithm that combines stochastic Hessian approximation, adaptive batch size control, and dynamic switching between first- and second-order methods.
The core idea is to leverage the low-dimensional parameter space of choice models to compute a stochastic Hessian, providing second-order information that accelerates convergence. Simultaneously, the batch size adapts during the optimization process based on a window moving average of progress, allowing the algorithm to start with small batches for speed and gradually increase to full data for precision. The switching mechanism chooses the most suitable optimization strategy depending on the current batch size, ensuring both rapid initial progress and high-accuracy final estimates.
Extensive experiments across ten benchmark discrete choice models, including the largest with over 80,000 observations, demonstrate that HAMABS reduces estimation time by a factor of 23 compared to existing methods. This significant speedup enables researchers and practitioners to estimate complex models in a fraction of previous timeframes, opening new possibilities for behavioral analysis, policy simulation, and large-scale decision modeling.
The approach's broader impact lies in transforming how large-scale behavioral models are estimated, making high-precision, computationally feasible analysis accessible across industries. While promising, the method faces limitations in extremely high-dimensional settings and requires careful tuning of adaptive parameters. Future work aims to extend the framework's scalability and robustness, paving the way for real-time, adaptive decision support systems in various fields.
Deep Dive
Abstract
The emergence of Big Data has enabled new research perspectives in the discrete choice community. While the techniques to estimate Machine Learning models on a massive amount of data are well established, these have not yet been fully explored for the estimation of statistical Discrete Choice Models based on the random utility framework. In this article, we provide new ways of dealing with large datasets in the context of Discrete Choice Models. We achieve this by proposing new efficient stochastic optimization algorithms and extensively testing them alongside existing approaches. We develop these algorithms based on three main contributions: the use of a stochastic Hessian, the modification of the batch size, and a change of optimization algorithm depending on the batch size. A comprehensive experimental comparison of fifteen optimization algorithms is conducted across ten benchmark Discrete Choice Model cases. The results indicate that the HAMABS algorithm, a hybrid adaptive batch size stochastic method, is the best performing algorithm across the optimization benchmarks. This algorithm speeds up the optimization time by a factor of 23 on the largest model compared to existing algorithms used in practice. The integration of the new algorithms in Discrete Choice Models estimation software will significantly reduce the time required for model estimation and therefore enable researchers and practitioners to explore new approaches for the specification of choice models.