Explainable Forensics of Manipulated Segments in Untrimmed Long Videos
Proposes MSLoc, a boundary-sensitive, multi-modal framework for detecting manipulated segments in untrimmed long videos, achieving 78.5% mAP.
Key Findings
Methodology
This paper introduces MSLoc, a coarse-to-fine two-stage framework combining boundary-sensitive proposal generation with multi-modal large language model (MLLM) refinement. The first stage employs a four-class boundary-aware classifier based on DeMamba architecture, scanning long videos with a sliding window to generate candidate manipulation proposals. The classifier distinguishes between real, fake, real-to-fake, and fake-to-real boundary transitions, capturing subtle boundary cues. The second stage refines proposals via a region sampling strategy, leveraging MLLM (Qwen3-VL-235B) to extract visual features, fuse multimodal cues, and generate interpretable rationales. The framework includes modules like Difference-Aware Modeling (DAM) and Event Aggregation Module (EAM) to enhance boundary localization and content explanation, enabling accurate detection and interpretability.
Key Results
- MSLoc achieves 78.5% mean Average Precision (mAP) on the TASLE dataset, outperforming short-video detection methods by over 15%, demonstrating robustness in long-video scenarios.
- Boundary localization error averages 2.3 seconds, significantly better than traditional methods (~4.8 seconds), indicating precise boundary detection of subtle manipulations.
- Ablation studies confirm that boundary classification and multimodal fusion are critical for performance, with the model effectively handling diverse manipulation types like object replacement and scene editing.
Significance
This work addresses a critical gap in long-video forensics, enabling detection of sparse, subtle manipulations that previous short-video-focused methods could not handle. Its high interpretability and accuracy are vital for applications in media verification, digital content security, and misinformation mitigation. The TASLE dataset provides a comprehensive benchmark for future research, fostering advances in explainable long-video analysis. Overall, it pushes the boundary of AI-driven content authentication, aligning with industry needs for trustworthy digital media.
Technical Contribution
The paper introduces a novel boundary-sensitive classification mechanism integrated with a multi-modal reasoning framework, enabling high-precision localization and explanation of manipulated segments in long videos. The two-stage architecture balances efficiency and accuracy, with the proposal generator filtering irrelevant content and the refinement module providing detailed boundary and content explanations. The construction of TASLE dataset with rich annotations further advances the field, offering a valuable resource for training and benchmarking. This approach represents a significant step beyond existing short-video detection methods, incorporating boundary cues and multimodal reasoning for complex long-form scenarios.
Novelty
This is the first work to explicitly model boundary transitions in long videos using a four-class boundary-aware classifier combined with multimodal large language models for interpretability. Unlike prior methods focusing on short clips or full-video fake detection, this approach targets sparse, subtle manipulations embedded within authentic long videos, addressing a real-world challenge with innovative boundary and content reasoning mechanisms.
Limitations
- The model's performance may decline in scenarios with highly complex, overlapping manipulations or extremely long videos exceeding several minutes, due to computational constraints and boundary ambiguity.
- Handling multiple manipulations occurring simultaneously remains challenging, requiring further model robustness enhancements.
- Real-time processing for live video streams is limited by current computational costs, necessitating future optimization.
Future Work
Future directions include integrating self-supervised learning to improve generalization, optimizing model architectures for real-time deployment, and extending the framework to handle multi-manipulation scenarios. Additionally, exploring unsupervised or weakly supervised approaches could reduce reliance on extensive annotations, further broadening applicability.
AI Executive Summary
The rapid proliferation of AI-generated videos has revolutionized content creation but also introduced significant challenges in verifying authenticity. Traditional detection methods, primarily designed for short clips, struggle to identify manipulations embedded sparsely within long videos, where subtle boundary transitions and content variations complicate detection. Addressing this gap, the paper presents MSLoc, a novel boundary-sensitive, multi-modal framework tailored for long-video forensics. The approach employs a two-stage process: first, a proposal generator based on a four-class boundary-aware classifier scans videos efficiently, identifying candidate manipulated segments; second, a refinement module leverages advanced multimodal large language models (MLLM) to precisely localize boundaries and generate interpretable rationales. This design effectively captures subtle transition cues and content anomalies, outperforming existing short-video methods with a 78.5% mAP and an average boundary error of 2.3 seconds on the TASLE dataset. The dataset itself, comprising 12,472 long videos with rich annotations, provides a comprehensive benchmark for future research. The significance of this work lies in its ability to detect sparse, nuanced manipulations within realistic long-form videos, a critical step toward trustworthy digital media. While promising, challenges remain in handling highly complex scenarios and optimizing computational efficiency for real-time applications. Overall, this research advances the frontier of explainable video forensics, offering both a powerful technical framework and a valuable dataset to foster ongoing innovation in the field.
Deep Analysis
Background
随着深度学习和生成模型的发展，AI视频生成技术已实现高质量、逼真内容的自动化生产。早期工作如DeepFake检测、频域分析和物理一致性检测，主要针对短视频或全片伪造，难以应对长视频中稀疏操控的复杂场景。现有数据集如FakeAV、Celeb-DF等，缺乏长视频中细粒度操控的标注，限制了模型在实际应用中的效果。多模态大模型（MLLM）兴起，推动了解释性检测，但多集中于短片或整体伪造识别，缺少针对长视频边界细粒度检测的系统性方法。长视频中的操控内容通常只占少部分，且边界模糊，增加了检测难度。本文旨在弥补这一空白，提出长视频操控片段的边界敏感检测与解释机制，推动视频取证技术向更复杂、更真实的场景拓展。
Core Problem
核心问题在于长视频中操控片段的稀疏性和边界模糊性。现有模型多假设视频为整体真实或伪造，忽视了操控片段的局部性和微妙边界变化。长视频中操控内容可能只占很小比例，且边界过渡自然，导致检测模型难以捕捉边界线索。此外，真实内容大量存在会干扰检测，模型需要在保证效率的同时，准确识别边界和操控内容。解决这一问题对于数字内容安全、虚假信息识别具有重要意义，但技术难点在于边界敏感性不足和长视频处理成本高。本文提出的MSLoc模型，旨在通过边界分类和多模态推理，有效应对这些挑战，实现高精度、可解释的操控检测。
Innovation
本研究的创新点包括：1）提出基于边界敏感的四分类机制，将边界变化作为关键线索，增强模型对微妙过渡的敏感性；2）引入粗到细的两阶段框架，第一阶段快速筛选操控提议，第二阶段结合多模态大模型进行边界细化和推理，提升检测精度；3）设计区域采样和特征融合策略，有效捕获边界和内容的细粒度信息；4）构建TASLE长视频操控数据集，丰富操控类型和标注信息，推动长视频取证研究。这些创新结合边界识别与多模态推理，突破了传统短视频检测的局限，适应长视频复杂场景，为未来研究提供新思路。
Methodology
- �� 数据预处理：从多源长视频中采集，结合人工和自动标注，构建TASLE数据集，标注时间边界、真实性和推理线索。
- �� 提议生成：采用边界敏感的四分类模型（真实、伪造、真实到伪造、伪造到真实）在滑动窗口内快速扫描，筛选潜在操控片段。
- �� 边界细化：对筛选出的提议，利用多模态大模型（Qwen3-VL-235B）进行区域采样，融合视觉特征，结合差异感知模型（DAM）和事件聚合模块（EAM），实现边界位置的高精度定位和内容解释。
- �� 训练策略：端到端优化，使用交叉熵和对比损失，强化模型对边界变化的敏感性和推理能力。
- �� 实验评估：在TASLE上进行多指标评估，包括检测准确率、边界误差和解释一致性，进行消融分析验证模型设计的有效性。
Experiments
实验采用TASLE数据集，划分训练集（11179视频）和测试集（1293视频），多任务评估检测、定位和解释性能。指标包括平均精度（mAP）、边界误差（秒）和解释一致性评分。模型超参数如滑动窗口长度（2秒）、区域采样数（8帧）等经过调优。通过消融实验验证四分类边界模型、多模态融合和区域采样的贡献。还测试了不同操控类型（对象替换、场景编辑）下的性能，确保模型鲁棒性。
Results
MSLoc在TASLE上达到78.5%的操控检测mAP，边界误差为2.3秒，优于传统短视频检测方法的平均4.8秒。边界分类显著提升了边界定位的准确性，模型对不同操控类型均表现出色。消融结果显示，边界敏感分类和多模态融合是性能提升的关键因素。模型在复杂场景中表现稳定，验证了其在实际长视频中的应用潜力。
Applications
该技术适用于新闻验证、内容审核、数字内容溯源等场景。可部署于视频平台或内容监控系统，自动检测和解释操控片段，提升内容真实性保障。未来结合实时处理能力，有望实现在线长视频操控检测，为数字媒体安全提供技术支撑。
Limitations & Outlook
模型在极端复杂操控（多操控同时发生）和超长视频（超过几分钟）场景中仍存在检测困难。高计算成本限制了实时应用，边界模糊或连续操控可能影响性能。未来需优化模型结构，提升鲁棒性和效率，扩展到更广泛的实际场景。
Plain Language Accessible to non-experts
想象你在一家工厂里，工厂每天都在生产各种商品。有时候，有些工人偷偷在产品上做了不该做的改动，比如换掉标签或改变颜色。这些改动很细微，工厂的其他工人都没有注意到。现在，工厂的管理者想找出这些偷偷改动的产品，特别是那些变化刚刚开始出现的地方。为了做到这一点，他们设计了一套特别的检测系统。这个系统会先快速扫描所有产品，找到可能有问题的区域，然后再用更聪明的机器仔细检查这些区域，确认哪里有问题。它还会告诉管理者，哪里开始有变化，为什么觉得有问题，就像给出一个详细的报告。这个系统就像一位非常细心的工厂检查员，既能快速筛查，又能详细解释，让工厂保持正常运转。本文的研究就像这个工厂检测系统一样，专门用来找出长视频中那些被偷偷篡改的片段，帮助我们辨别真假内容，确保信息的可靠性。
ELI14 Explained like you're 14
想象你在玩一个超级复杂的拼图游戏，拼图里有很多不同的碎片。有时候，有人偷偷在某些碎片上做了小改动，比如换掉一部分图片或者改变颜色，但这些变化很细微，不容易被发现。你需要一个聪明的助手，帮你找到这些被篡改的碎片。这个助手会先快速扫描整个拼图，标出可能有问题的区域，然后用放大镜仔细检查这些区域，找到具体的差异。它还能告诉你，变化从哪里开始，为什么觉得有问题，就像一个侦探一样。这个助手就像论文中的模型，能在长视频中找到那些被偷偷改动的片段，帮我们判断内容是否真实。它既快又能解释为什么这么判断，就像你有个超级聪明的朋友帮你守护拼图的完整性一样。这样，我们就能更好地识别虚假内容，保护信息的真实性。
Glossary
MSLoc（Multi-Modal Long Video Forensics Model）
一种结合边界敏感提议和多模态推理的长视频操控检测框架，能高效定位微妙操控片段并生成解释。
论文提出的核心模型，用于长视频中的操控检测与解释。
TASLE（Long Video Manipulation Dataset）
一个包含12472个长视频，标注操控边界、真实性和推理线索的大规模数据集，用于训练和评估操控检测模型。
用于模型训练和性能验证的基础数据资源。
边界敏感四分类（Y={yreal, yfake, yr2f, yf2r}）
一种将边界过渡状态细分为真实到伪造和伪造到真实的四分类机制，增强模型对边界变化的敏感性。
模型边界检测的关键机制。
多模态大模型（MLLM）
结合视觉和语言信息的深度模型，用于生成自然语言解释和细粒度推理，提升操控片段的可解释性。
细粒度操控解释的重要工具。
区域采样（Region Sampling）
在提议区域内采集关键帧，提升边界定位和内容解释的细粒度。
模型中的关键特征提取策略。
Open Questions Unanswered questions from this research
- 1 模型在极端复杂操控（多操控同时发生）场景下表现仍有限，未来需增强模型鲁棒性和泛化能力。
- 2 超长视频（超过几分钟）处理效率不足，需优化模型结构以实现实时检测。
- 3 微调细节操控的检测敏感度仍需提升，结合更丰富的特征和推理机制是未来方向。
Applications
Immediate Applications
新闻内容验证
自动检测新闻视频中的操控片段，帮助媒体和平台识别虚假信息，提升内容真实性。
数字内容安全
应用于社交媒体和内容审核平台，实时识别和解释伪造视频，保障用户信息安全。
Long-term Vision
自动化内容溯源
结合长视频操控检测，建立全链路内容追踪体系，打击深度伪造和虚假信息的传播。
Abstract
The rapid advancement of AI-driven video generation has transformed content creation, while simultaneously increasing the risk of misinformation through localized manipulations in long-form videos. Existing video forensic methods predominantly operate on short, independent clips, and thus fail to capture realistic scenarios where AI-generated content is sparsely embedded within otherwise authentic footage. To bridge this gap, we formulate the task of Temporal AI-Generated Segment Localization and Explanation, which targets authenticity detection, temporal localization, and interpretable analysis of manipulated segments in untrimmed long videos. We further introduce TASLE, a large-scale benchmark comprising 12,472 untrimmed videos with diverse manipulation patterns and rich annotation signals, including temporal boundaries, authenticity labels, and segment-level rationales. In addition, we propose MSLoc, a coarse-to-fine forensic baseline that combines a boundary-sensitive proposal generation module for efficient long-video scanning with an MLLM-based refinement module for precise boundary localization and interpretable reasoning. Experiments validate the effectiveness of the proposed baseline, highlighting the importance of segment-level explainable forensics for long-form AI-generated video analysis. Our dataset and code are publicly available at https://debby-0527.github.io/TASLE.