GMoT: Gated Motion-Aware Tokenization for Fine-Grained Micro-Gesture Video Reasoning with Multimodal LLMs
GMoT module explicitly extracts sparse motion cues, boosting micro-gesture video reasoning accuracy to 67.32% Top-1, surpassing baselines.
Key Findings
Methodology
This paper introduces GMoT, a lightweight module combining spatial saliency scoring, adjacent-frame differencing, and a semantic gating mechanism. It dynamically highlights action-relevant regions, explicitly captures subtle motion energy, and adaptively fuses these cues into the visual stream. The training employs a four-stage strategy: warm-up, domain adaptation, chain-of-thought (CoT) supervision, and reward-guided policy refinement, supported by semi-supervised annotation for evidence-grounded explanations. Experiments on iMiGUE and SMG datasets show improvements of +6.80 and +3.11 points over the Qwen3-VL baseline, achieving 67.32% and 73.11% Top-1 accuracy respectively, with additional metrics like Body-Region Grounding (BRG) recall indicating enhanced interpretability and robustness.
Key Results
- GMoT significantly improves micro-gesture recognition accuracy, reaching 67.32% on iMiGUE and 73.11% on SMG, outperforming baseline models. Ablation studies confirm the importance of spatial saliency and temporal differencing. The reward-guided training reduces false reasoning and maintains performance under label corruption and domain shifts. BRG recall demonstrates better anatomical localization, enhancing interpretability.
- The model's robustness is validated through cross-domain transfer and corruption tests, showing stable accuracy and high grounding fidelity. The staged training strategy effectively aligns visual and language representations for fine-grained evidence reasoning, leading to superior performance in subtle micro-gesture tasks.
- Overall, GMoT advances the state-of-the-art in fine-grained micro-gesture video reasoning by explicitly modeling motion cues, integrating evidence-grounded explanations, and ensuring interpretability and robustness across scenarios.
Significance
This work addresses fundamental limitations of existing multimodal large models in capturing transient, localized micro-movements. By explicitly modeling sparse motion evidence, GMoT enhances the sensitivity and interpretability of micro-gesture recognition, crucial for applications like emotion understanding, psychological analysis, and deception detection. The integration of evidence-grounded reasoning and anatomical grounding metrics pushes the frontier of explainable AI in video understanding. Its robustness under domain shifts and corruptions demonstrates practical viability for real-world deployment, marking a significant step toward fine-grained, trustworthy behavioral AI systems.
Technical Contribution
The paper introduces GMoT, a novel module that combines spatial saliency scoring, explicit temporal differencing, and conservative semantic gating, enabling pretrained multimodal models to focus on subtle micro-movements without disrupting their core representations. The multi-stage training framework, including semi-supervised annotation and reward-guided policy refinement, systematically improves reasoning accuracy and evidence traceability. The incorporation of Body-Region Grounding (BRG) as an interpretability proxy and cross-domain transfer protocols further distinguishes this work from prior methods, offering a comprehensive solution for fine-grained micro-gesture reasoning with enhanced explainability.
Novelty
This is the first systematic integration of explicit motion cues via spatial saliency and temporal differencing into multimodal large language models for micro-gesture recognition. Unlike previous approaches relying solely on dense features or global attention, GMoT emphasizes localized, transient dynamics, enabling the model to detect subtle micro-movements effectively. Its multi-stage training with evidence-grounded supervision and reward optimization represents a novel paradigm for enhancing interpretability and robustness in fine-grained video understanding, setting a new benchmark in the field.
Limitations
- Despite improvements, the model's performance may degrade under extreme occlusion or complex backgrounds where motion cues are obscured or noisy. The multi-stage training process, involving semi-supervised annotation and reinforcement strategies, incurs high computational costs, limiting real-time deployment. Additionally, the reliance on high-quality annotations and datasets restricts generalization to unseen micro-gesture categories or domains, necessitating further research on unsupervised or weakly supervised methods.
Future Work
Future directions include developing more efficient, end-to-end training architectures that reduce computational overhead, exploring unsupervised motion cue extraction, and extending the framework to multi-modal inputs such as speech and physiological signals. Enhancing domain adaptation capabilities and real-time inference will facilitate deployment in practical scenarios like surveillance, human-computer interaction, and psychological assessment. Further, integrating self-supervised learning could improve generalization to new micro-gesture categories with limited annotations.
AI Executive Summary
Micro-gesture recognition is a challenging task due to the fleeting, localized, and subtle nature of these movements, which are often overwhelmed by static backgrounds and noise. Existing multimodal large language models (MLLMs) excel at broad video understanding but struggle with the fine-grained kinematic details necessary for accurate micro-gesture detection. This gap limits their applicability in sensitive domains such as emotion analysis, deception detection, and psychological assessment.
To address this, the authors propose GMoT, a Gated Motion-Aware Tokenization module that explicitly extracts sparse motion cues from short video clips. GMoT employs a spatial saliency scorer to dynamically highlight action-relevant regions, computes explicit adjacent-frame differences to capture precise motion energy, and fuses these cues into the visual stream via a conservative semantic gate. This design ensures the model remains stable while focusing on subtle movements. The training involves a four-stage process: initial warm-up, domain adaptation with micro-gesture datasets, chain-of-thought supervision for reasoning, and reward-guided policy refinement to enhance evidence grounding.
Experimental results on iMiGUE and SMG datasets demonstrate that GMoT significantly improves recognition accuracy, achieving 67.32% and 73.11% Top-1 accuracy respectively, surpassing baseline models by substantial margins. The introduction of Body-Region Grounding (BRG) recall further enhances interpretability, indicating the model's ability to localize relevant anatomical regions accurately. The approach also shows robustness under label corruption and cross-domain transfer scenarios, highlighting its practical potential.
This work marks a meaningful advancement in fine-grained video understanding, bridging the gap between static appearance reliance and dynamic motion sensitivity. By explicitly modeling sparse motion evidence and grounding explanations anatomically, GMoT paves the way for more trustworthy and interpretable AI systems capable of understanding subtle human behaviors. Future efforts will focus on improving efficiency, reducing annotation dependency, and extending multi-modal integration, aiming for real-time, scalable solutions applicable across diverse real-world settings.
Deep Analysis
Background
微表情和微动作作为人类非语言交流的重要组成部分，近年来受到广泛关注。早期研究主要依赖RGB视频和骨架信息，采用空间卷积和时序模型（如LSTM、Transformer）捕获微动作特征。代表性工作包括3D-CNN、图卷积网络（GCN）以及多模态融合方法，但在瞬态、局部微动作的细粒度识别上仍存在不足。随着大规模数据集和深度学习的发展，基于预训练模型的多模态大模型（如Qwen3-VL）逐渐成为主流，提升了整体理解能力。然而，这些模型在微动作细节捕获方面仍受限于全局注意和空间池化机制，难以有效识别瞬时微表情。近年来，强调局部区域动态变化和证据驱动的推理成为研究热点，推动了微动作识别的技术革新。
Core Problem
微动作识别面临的核心挑战在于其瞬时性和局部性，运动信号稀疏且易被背景噪声干扰。现有多模态大模型多依赖密集帧级特征提取，导致微细运动信号被稀释或遗漏，影响识别准确性。此外，模型缺乏对运动证据的显式建模，难以实现证据可追溯和解释。训练过程中，单纯追求分类准确率容易导致虚假推理和偏差，限制模型在实际应用中的鲁棒性。如何有效提取微动作的稀疏运动信息，并融合到预训练模型中，成为亟待解决的问题。
Innovation
本文提出GMoT模块，创新性地结合空间显著性评分、邻帧差分和门控融合机制，显式提取微动作的稀疏运动证据。空间评分器动态突出动作区域，邻帧差分捕获瞬时运动能量，门控机制确保运动信息的稳健融合，避免干扰静态背景。训练方面，采用多阶段策略，包括预热、微动作域适应、链式推理和奖励引导，逐步优化模型推理路径。引入身体区域对齐召回（BRG）指标，提升模型的可解释性。整体架构兼容预训练模型，显著改善微动作的细粒度识别和证据追溯能力。这些创新为微动作理解提供了新思路。
Methodology
- �� 预处理：利用预训练视觉编码器将视频帧转化为patch tokens。
- �� 空间显著性评分：线性投影每个patch，应用softmax获得权重，动态加权聚合特征。
- �� 邻帧差分：计算相邻帧差异的平均幅值和符号差，形成两个运动流。
- �� 运动标记：将两个差分流拼接，投影成紧凑的运动感知标记。
- �� 门控融合：用sigmoid门控将运动标记融合到原始视觉特征，初始化为接近关闭状态以保持稳定。
- �� 训练策略：分四阶段，包括预热（冻结模型训练GMoT）、微动作域适应（微调识别任务）、链式推理（生成可解释描述）、奖励引导（优化推理路径和证据追溯）。
- �� 评估：在iMiGUE和SMG数据集上进行准确率、BRG召回和跨域迁移测试，验证模型性能和鲁棒性。
Experiments
采用iMiGUE和SMG两个微动作数据集，分别进行训练和测试。模型基于Qwen3-VL预训练架构，加入GMoT模块。评估指标包括Top-1准确率、BRG召回率和跨域迁移性能。通过消融实验验证空间显著性评分器、邻帧差分和门控机制的贡献。参数设置包括学习率、批次大小和训练轮数，确保模型收敛。对比多种基线模型，验证GMoT在微动作识别中的优越性。
Results
GMoT模型在iMiGUE达67.32%，SMG达73.11%的Top-1准确率，超越Qwen3-VL基线+6.80和+3.11点。引入BRG指标，模型对微动作的解剖区域定位能力显著提升。消融研究显示空间评分器和邻帧差分对性能提升贡献最大。奖励策略有效减少虚假推理，模型在标签腐蚀和跨域迁移中表现稳定。整体验证了GMoT在微动作细粒度识别中的优越性和鲁棒性。
Applications
该方法适用于微表情分析、心理状态检测、安防监控等场景，特别是在需要细粒度动作识别和解释的应用中。依赖高质量数据和注释，能显著提升微动作的识别准确性和可解释性。未来结合多模态信息（如语音、文本）可进一步增强系统的智能水平，推动行业应用落地。
Limitations & Outlook
模型对极端复杂背景或遮挡条件下的微动作识别仍存在困难，主要由于运动证据提取受限于空间显著性评分器的准确性。训练复杂，计算成本高，难以实时部署。对类别的细粒度区分依赖高质量标注，泛化能力在少样本或新类别上仍需验证。未来需优化模型效率和鲁棒性。
Plain Language Accessible to non-experts
想象你在厨房做饭，微动作就像是你不经意间的小动作，比如轻轻搓手、皱眉或舔嘴唇。这些动作非常细微，只有在非常专注或用心观察时才能发现。传统的相机或视频分析就像用放大镜看整个厨房，很多微动作都被背景和大动作淹没了。本文提出的方法就像给放大镜装了一个智能识别器，它能专门找到那些微小的动作区域，快速捕捉到这些瞬间的变化。通过特殊的“门”机制，模型可以决定哪些运动证据值得关注，把这些微动作变得更清晰、更容易被识别。这样一来，无论是情感分析还是心理状态检测，都能更准确、更细腻地理解人的微表情和微动作，就像用放大镜看细节一样清楚。
ELI14 Explained like you're 14
想象你在学校里偷偷观察朋友的微小动作，比如轻轻皱眉或嘴角抽动。这些动作虽然很细微，但能透露很多情绪。可是普通的视频分析就像用普通相机拍照，很多细节都被背景和大动作盖住了。这个研究就像给视频装上了一个聪明的“放大镜”，它可以专门找到那些微小的动作区域，然后用特殊的“门”机制决定哪些运动证据最重要。这样一来，模型就能更好地捕捉到这些瞬间的细节，理解朋友的真实情绪。通过多阶段训练和奖励机制，模型学会了如何专注于微动作，避免被虚假的线索误导。最终，这项技术能帮助心理医生、安防人员更准确地识别微表情，甚至在社交媒体上分析人们的真实想法，就像拥有了一个超强的“微动作侦探”一样。
Abstract
Micro-gesture recognition demands the detection of fleeting, spatially localized movements that are frequently overwhelmed by dominant static appearances and background noise. While Multimodal Large Language Models (MLLMs) excel at general video understanding, they inherently struggle with subtle kinematics and often rely on static posture priors. To this end, we propose GMoT, a Gated Motion-Aware Tokenization module that explicitly distills sparse kinematic evidence into a compact sequence prior to temporal modeling. GMoT dynamically spotlights action-relevant regions via spatially weighted pooling, extracts adjacent-frame temporal differencing to capture precise motion energy, and adaptively fuses these cues into the visual stream using a conservatively initialized semantic gate. To transition from simple classification to evidence-grounded reasoning, we further introduce a progressive reward-guided policy refinement paradigm, supported by a semi-supervised annotation pipeline that generates anatomically focused captions. Beyond achieving the best Top-1 accuracy among the compared methods on iMiGUE (67.32\%) and SMG (73.11\%), improving the Qwen3-VL-8B baseline by +6.80 and +3.11 points, our framework introduces Body-Region Grounding (BRG) Recall as an anatomical-grounding proxy conditioned on correct predictions, together with an overlapping-label cross-domain transfer protocol between iMiGUE and SMG. Extensive evaluations demonstrate that our GMoT-augmented model improves in-domain accuracy, retains clear gains under label-preserving corruptions, and improves accuracy-oriented cross-domain transfer under explicit small-split caveats while maintaining high anatomical grounding in its generated rationales.