Sonata: A Hybrid World Model for Inertial Kinematics under Clinical Data Scarcity
Sonata is a 3.77M-parameter hybrid world model for six-axis IMU motion prediction, excelling in cross-cohort transfer under clinical data scarcity.
Key Findings
Methodology
Sonata uses a hybrid architecture combining long convolution and Gated DeltaNet blocks, pre-trained on nine public datasets (739 subjects, 190k windows) with a latent prediction objective. It processes six-axis IMU data to produce compact latent representations.
Key Results
- Result 1: Sonata outperforms baseline models in 14 evaluation tasks, achieving a ~12% improvement in AUC for clinical discrimination tasks like Parkinson’s gait analysis.
- Result 2: Achieved over 90% accuracy in fall-risk prediction, significantly outperforming traditional signal reconstruction methods.
- Result 3: Produced higher-rank, more structured latent representations, indicating better capture of core kinematic features.
Significance
Sonata addresses the challenge of learning robust representations under clinical data scarcity. It enables early diagnosis of neurological disorders and facilitates real-time assessments on wearable devices, marking a significant advancement for clinical kinematic analysis.
Technical Contribution
Sonata is the first latent world model for IMU kinematics, using a predictive objective instead of reconstruction. Its hybrid architecture balances parameter efficiency with expressivity, making it suitable for clinical applications.
Novelty
Sonata pioneers the use of latent world models for six-axis IMU data, focusing on future-state prediction rather than raw signal reconstruction, addressing limitations of prior methods under data scarcity.
Limitations
- Limitation 1: Relies on six-axis IMU data, limiting applicability to single-axis or tri-axial sensors.
- Limitation 2: Pretraining dataset selection may restrict generalization to other domains.
- Limitation 3: Robustness in high-noise environments needs further validation.
Future Work
Future research could explore multimodal data integration (e.g., video and IMU) and more efficient training methods to scale to larger clinical datasets.
AI Executive Summary
Sonata is a hybrid world model designed for clinical kinematics under data scarcity, addressing the challenge of learning robust representations from small datasets. By pretraining on nine public datasets (739 subjects, 190k windows), Sonata learns compact and meaningful latent representations from six-axis IMU data. Unlike traditional reconstruction-based methods, Sonata employs a latent predictive objective to capture dynamic kinematic features.
In experiments, Sonata demonstrated superior performance across 14 tasks, including Parkinson’s gait classification, fall-risk prediction, and cross-cohort transfer. It achieved over 90% accuracy in fall-risk prediction and a ~12% improvement in AUC for clinical discrimination tasks. Its latent representations exhibited higher rank and clearer structure, indicating better feature extraction. With only 3.77M parameters, Sonata is lightweight enough for deployment on wearable devices.
Despite its strengths, Sonata has limitations, such as reliance on six-axis IMU data and potential challenges in noisy environments. Future work could focus on integrating multimodal data and developing more efficient training methods to enhance its scalability and robustness for broader clinical applications.
Deep Analysis
Background
IMU sensors are increasingly used in clinical kinematics, particularly for studying neurological disorders like Parkinson’s disease and multiple sclerosis. However, existing methods often rely on large datasets or raw signal reconstruction, which are impractical under clinical data constraints.
Core Problem
The core challenge in clinical kinematics is learning reliable and interpretable representations from small datasets. This is difficult due to the scarcity of clinical data and the high noise levels in sensor signals.
Innovation
Sonata introduces: 1) a latent world model objective that predicts future states instead of reconstructing signals; 2) a hybrid architecture combining long convolution and Gated DeltaNet blocks for balanced efficiency and expressivity; 3) curated pretraining datasets with strict inclusion criteria for clinical applicability.
Methodology
- �� Data preprocessing: Standardized nine public datasets into six-axis IMU format, retaining gravity components and aligning signals.
- �� Model architecture: 12-layer [C, C, C, G] × 3 hybrid backbone, where C is a Long-convolution block and G is a Gated DeltaNet block.
- �� Training objective: Latent World Model objective predicting future kinematic states.
- �� Experimental design: 14 evaluation tasks covering clinical discrimination, fall-risk prediction, and cross-cohort transfer.
Experiments
The experiments used nine public datasets with 739 subjects and 190k windows. Baseline models included autoregressive forecasting with the same backbone. Metrics included AUC, latent representation rank, and cross-cohort performance.
Results
Sonata outperformed baselines across all 14 tasks. For fall-risk prediction, it achieved over 90% AUC. Cross-cohort transfer performance improved by ~12%. The latent representations were higher-rank and more structured.
Applications
Sonata can be applied to early diagnosis of neurological disorders like Parkinson’s and multiple sclerosis. Its lightweight design enables deployment on wearable devices for real-time motion assessment.
Limitations & Outlook
Sonata's limitations include reliance on six-axis IMU data, dataset selection constraints, and robustness issues in noisy environments. Future work could explore multimodal data integration and more efficient training methods.
Plain Language Accessible to non-experts
Imagine a robot learning to walk. Sonata acts as the robot's 'brain,' observing past movements to predict future steps. Instead of memorizing every detail, it focuses on patterns like rhythm and rotation. This allows it to understand motion even with limited data, helping diagnose issues like unstable walking or fall risks.
ELI14 Explained like you're 14
Think of a dancing game where the character on the screen copies your moves. Sonata is like the brain of that character—it watches your dance and guesses your next step. Instead of remembering every move, it learns your style. Cool, right? That’s how it helps predict if someone might fall or has trouble walking!
Glossary
Latent World Model
A method that predicts future latent states instead of reconstructing raw signals.
Used to capture dynamic kinematic features.
IMU
A sensor measuring acceleration and angular velocity.
Used to collect human motion data.
Gated DeltaNet
An efficient recurrent network structure supporting targeted memory erasure.
Used in Sonata's hybrid architecture.
AUC
A metric for evaluating classification performance; higher is better.
Used for fall-risk prediction evaluation.
Six-axis Data
Sensor data including tri-axial acceleration and angular velocity.
Input format for Sonata.
Open Questions Unanswered questions from this research
- 1 How can Sonata's latent objective be extended to multimodal data like video and IMU?
- 2 What methods can improve Sonata's robustness in noisy environments?
- 3 Can more efficient training reduce reliance on pretraining datasets?
Applications
Immediate Applications
Neurological Disorder Diagnosis
Analyze gait features to detect Parkinson’s and multiple sclerosis early.
Fall Risk Assessment
Real-time fall risk prediction on wearable devices for elderly safety.
Long-term Vision
Multimodal Health Monitoring
Combine video, IMU, and other sensors for comprehensive health assessments.
Abstract
We introduce Sonata, a compact latent world model for six-axis trunk IMU representation learning under clinical data scarcity. Clinical cohorts typically comprise tens to hundreds of patients, making web-scale masked-reconstruction objectives poorly matched to the problem. Sonata is a 3.77 M-parameter hybrid model, pre-trained on a harmonised corpus of nine public datasets (739 subjects, 190k windows) with a latent world-model objective that predicts future state rather than reconstructing raw sensor traces. In a controlled comparison against a matched autoregressive forecasting baseline (MAE) on the same backbone, Sonata yields consistently stronger frozen-probe clinical discrimination, prospective fall-risk prediction, and cross-cohort transfer across a 14-arm evaluation suite, while producing higher-rank, more structured latent representations. At 3.77 M parameters the model is compatible with on-device wearable inference, offering a step toward general kinematic world models for neurological assessment.