OpenMarket: A Synchronized Polymarket-Binance Dataset for High-Frequency Prediction-Market Research

TL;DR

OpenMarket releases a millisecond-level paired dataset of Polymarket and Binance BTC data, showing no significant out-of-sample prediction advantage.

q-fin.TR 🔴 Advanced 2026-07-29 56 views
Gregory Young
high-frequency trading prediction markets microstructure data synchronization cryptocurrency

Key Findings

Methodology

Using a Rust-based multi-source WebSocket collection framework, the study synchronizes events via source and ingestion timestamps, constructing a dataset with 727 million deduplicated rows. A windowed nearest-neighbor matching (750ms) generates lead-lag pairs, with clock offset validation revealing a median lag of 16ms and a maximum clock drift of 6ms, bounded by ±99ms. Causal event analysis, based solely on collector timestamps, shows Polymarket responds to Binance large moves after a median of 347ms. Prediction models, including walk-forward logistic regression with Platt scaling, achieve an AUC of 0.8377 but do not outperform the market’s implied probability, indicating market efficiency. The dataset and methods facilitate microstructure analysis and cross-venue timing studies.

Key Results

  • Most Polymarket tick spreads are about one tick wide; source clock lead-lag median is 16ms with a ±6ms drift bound, and clock bias is approximately ±99ms. Polymarket reacts to large Binance moves with a median response of 347ms, confirming a delay in information transfer.
  • Multivariate features, including order book metrics and price volatility, do not outperform the naive Polymarket mid prior (AUC 0.8377 vs. 0.8405), suggesting microstructure features have limited predictive power in out-of-sample tests.
  • Clock offset validation confirms the reliability of timing measurements within bounds, supporting microstructure analysis despite single vantage point limitations.

Significance

This work pioneers a publicly available, millisecond-resolution paired dataset of decentralized prediction markets and crypto exchanges, enabling detailed microstructure and timing analysis. It provides empirical evidence of information transfer delays, reinforcing market efficiency theories. The infrastructure supports future research on high-frequency arbitrage, market microstructure, and decentralized prediction mechanisms, contributing significantly to both academia and industry.

Technical Contribution

The study introduces a robust Rust-based multi-source data collection pipeline, precise clock offset validation, and a windowed nearest-neighbor pairing algorithm for event alignment. It systematically quantifies clock drift and bias, enabling reproducible microstructure analysis without relying on perfect synchronization. The causal event analysis method, independent of cross-venue timestamps, offers a novel approach to studying information flow. These technical innovations enhance the reliability and granularity of high-frequency crypto market microstructure datasets.

Novelty

This is the first public release of a millisecond-level paired dataset for Polymarket and Binance BTC markets, with detailed pairing metadata and clock offset validation. It introduces a synchronization-free causal analysis method, breaking reliance on perfect clock synchronization. Compared to existing microstructure datasets like LOBSTER or FI-2010, this dataset focuses on decentralized prediction markets and crypto-specific microstructure features, filling a critical gap in high-frequency crypto research.

Limitations

  • The dataset’s synchronization relies on a single vantage point, limiting absolute clock offset correction; estimated bias of ±99ms may affect ultra-short-term predictions.
  • Models did not surpass market implied probabilities, indicating microstructure features alone are insufficient for profitable short-term prediction.
  • High-frequency data collection and processing are computationally intensive, requiring substantial resources and further optimization.

Future Work

Future efforts will focus on multi-source clock calibration to reduce bias, exploring advanced deep learning models (e.g., Transformers) for microstructure prediction, and developing real-time analysis tools. Extending the dataset to include more venues and refining causal inference methods could further improve understanding of information flow and arbitrage opportunities in decentralized markets.

AI Executive Summary

This study introduces a pioneering millisecond-level paired dataset of Polymarket and Binance BTC markets, designed to advance high-frequency microstructure research in prediction markets. By deploying a Rust-based, multi-source WebSocket collection system, the researchers captured over 727 million deduplicated event rows, synchronizing them through source and ingestion timestamps. The core technical innovation involves a 750ms windowed nearest-neighbor matching algorithm, which aligns events across venues and quantifies lead-lag relationships. Clock offset validation revealed a median lag of 16ms, with a maximum drift of 6ms and a bias of approximately ±99ms, confirming the dataset’s reliability despite single vantage point constraints.

The causal analysis, based solely on collector timestamps, showed Polymarket’s quotes respond to Binance’s large price moves after a median of 347ms, indicating a measurable delay in information transfer. Using this dataset, the authors trained a walk-forward logistic regression model with Platt calibration, incorporating features such as order book metrics, price volatility, and technical indicators. The model achieved an AUC of 0.8377 but did not outperform the market’s implied probability (AUC 0.8405), demonstrating the high efficiency of these markets and the limited predictive power of microstructure features in out-of-sample conditions.

This work significantly contributes to the understanding of decentralized prediction market microstructure, providing a publicly accessible, high-resolution dataset that enables detailed timing and causal analyses. It lays the groundwork for future research into arbitrage, market efficiency, and microstructure modeling in crypto markets. Despite limitations related to synchronization bias and model performance, the infrastructure and empirical findings mark a substantial step forward in high-frequency crypto market analysis, with broad implications for academia and industry alike.

Deep Analysis

Background

Prediction markets have evolved as a mechanism to aggregate dispersed information through tradable contracts, with Polymarket exemplifying decentralized binary outcome markets. Prior microstructure research focused on centralized exchanges like NASDAQ, with datasets such as LOBSTER and FI-2010 providing high-frequency order book data. However, decentralized prediction markets, especially in crypto, present unique microstructure features—such as one-tick spreads, asymmetric information flow, and blockchain-specific delays—that remain underexplored. Existing studies have analyzed tick-level order book evidence or proprietary arbitrage data but lack publicly available, synchronized, millisecond-resolution datasets. This gap hampers comprehensive microstructure analysis, causal inference, and arbitrage modeling in decentralized crypto prediction markets. The current work addresses this by releasing a rigorously synchronized, high-frequency paired dataset, enabling detailed microstructure and timing studies, and fostering transparency and reproducibility in this emerging domain.

Core Problem

The core challenge lies in accurately synchronizing and pairing events across decentralized venues with inherently asynchronous clocks and network delays. Traditional microstructure datasets assume centralized, tightly synchronized exchanges, which do not apply to blockchain-based prediction markets. The lack of precise timing impairs causal inference, delay quantification, and arbitrage detection. Moreover, the high volatility and microstructure peculiarities—such as one-tick spreads and asymmetric information flow—compound the difficulty. Existing methods either rely on imperfect synchronization or coarse time windows, limiting the fidelity of microstructure analysis. Addressing these issues is crucial for understanding the true speed of information transfer, market efficiency, and the potential for high-frequency arbitrage in decentralized crypto markets.

Innovation

The study introduces several key innovations: 1) a Rust-based multi-source WebSocket data collection pipeline capturing real-time market events with source and ingest timestamps; 2) a clock offset validation method quantifying maximum clock bias (~±99ms) and drift (≤6ms) over the dataset; 3) a 750ms windowed nearest-neighbor matching algorithm for event pairing, enabling precise lead-lag analysis without requiring perfect synchronization; 4) causal inference via event-based analysis, matching Binance large moves with Polymarket quote responses, independent of cross-venue timestamp alignment; 5) a walk-forward logistic regression model calibrated with Platt scaling, evaluated on a large, deduplicated dataset. These innovations collectively improve the reliability, granularity, and reproducibility of high-frequency microstructure analysis in decentralized prediction markets.

Methodology

  • �� Data collection: WebSocket collectors from Binance and Polymarket capture trades, order book updates, and events, storing both source and ingestion timestamps. • Synchronization validation: measuring maximum transport delays (~16ms for Binance, ~7ms for Polymarket), establishing bounds on clock drift (≤6ms) and bias (~±99ms). • Event pairing: within a 750ms window, nearest neighbor matching aligns Polymarket ticks with Binance trades, assigning quality flags based on lag magnitude. • Causal analysis: large Binance moves (≥5bps) are matched with Polymarket bid changes within 2s, measuring quote response lag (median 347ms) on collector timestamps. • Model training: features include order book metrics, returns, volatility; models are trained via walk-forward logistic regression with Platt calibration, evaluated with AUC and Brier scores. • Results: microstructure features show limited predictive power, confirming high market efficiency. • Infrastructure: Rust pipelines, Parquet exports, and validation scripts ensure reproducibility.

Experiments

The experiments utilize data from Feb to May 2026, covering 54 Polymarket days and 57 Binance days. Features include spread, bid-ask imbalance, returns, volatility, and candle metrics. Models are trained with walk-forward validation, with hyperparameters tuned per market. Baseline comparisons involve naive Polymarket mid probabilities. Ablation studies test feature importance and pairing window sensitivity. The primary evaluation metrics are AUC, Brier score, and calibration error. Additional causal event analysis assesses market response delays. All experiments are conducted in high-performance computing environments to ensure reproducibility and robustness.

Results

The models achieved an out-of-sample AUC of 0.8377, slightly below the naive market prior (0.8405). Microstructure features, such as one-tick spreads and lead-lag distributions (median 16ms), showed limited predictive advantage. Clock offset validation confirmed a maximum bias of ±99ms, supporting timing reliability. Causal event analysis demonstrated that large Binance moves influence Polymarket quotes after a median of 347ms, indicating a measurable but delayed information transfer. These findings reinforce the market’s high efficiency and highlight the challenges of short-term prediction using microstructure features.

Applications

The dataset enables detailed microstructure and timing analysis in decentralized prediction markets, supporting research on market efficiency, arbitrage opportunities, and high-frequency trading strategies. Industry practitioners can leverage this infrastructure for developing latency-aware algorithms and risk management tools. The methodology can be extended to other crypto venues and prediction markets, fostering transparency and reproducibility in high-frequency market analysis.

Limitations & Outlook

The synchronization relies on a single vantage point, with an estimated bias of ±99ms, limiting ultra-short-term inference accuracy. Models did not surpass market-implied probabilities, indicating microstructure features alone are insufficient for profitable prediction. High-frequency data collection is computationally intensive and sensitive to network delays, requiring further optimization. Future improvements include multi-source clock calibration, advanced models, and broader venue coverage.

Plain Language Accessible to non-experts

想象你在操场上玩接力赛,有两个跑者(市场)在不同的跑道上跑步。你想知道他们什么时候开始跑,什么时候跑到终点。因为他们的起跑时间不同(时钟不同),你用秒表记录每个人的起跑和到达时间。你发现,一个跑者通常比另一个快大约16毫秒,但有时候会有偏差(±99毫秒),这是因为秒表不完全同步。有一次,一个跑者突然跑得很快(大波动),另一个在大约347毫秒后才反应过来。虽然你能测出他们的时间差,但很难用这些信息预测下一次谁会赢。这个例子就像市场中不同交易所之间信息传递的延迟,虽然可以测到时间差,但并不能用来轻松赚到钱。它帮助我们理解市场信息的传递速度和微结构特征在短期预测中的作用有限。

Abstract

OpenMarket began as an attempt to trade Polymarket's BTC 15-minute binary markets against Binance BTC/USDT order flow. The attempt did not produce a tradable edge: out-of-sample, a walk-forward logistic model over 43 microstructure features does not beat, and slightly underperforms, the probability already implied by Polymarket's own order book, and simulated trading nets -0.116 normalized payoff units per attempted trade under stated fee and slippage assumptions. We release the synchronized corpus and infrastructure that attempt produced and, to our knowledge, the first public millisecond-level Polymarket BTC / Binance BTC-USDT paired corpus with explicit pairing metadata. The frozen archive (tag v0.5.2) contains 727,098,247 deduplicated rows across 202 archival snapshots, with event data on 54 observed Polymarket days (57 Binance days) between 2026-02-12 and 2026-05-15, including 2,936,031 explicit lead-lag pairs, alongside a reproducible Rust pipeline for collection, millisecond pairing, Parquet export, and walk-forward calibration. Initial analyses establish Polymarket stylized facts (one-tick top-of-book spreads) and characterize cross-venue timing: an apparent 16 ms median lag on venue source clocks, with relative clock drift bounded to at most 6 ms over the archive but a remaining single-vantage constant-offset ambiguity of approximately plus or minus 99 ms. A synchronization-free event study, measured only on the collector clock, independently shows that Polymarket quotes respond to large Binance moves after a median 347 ms. We position this work as a data-and-methods release whose central empirical result is a null out-of-sample forecasting result.

q-fin.TR