Toward Real-world Text Image Forgery Localization: Structured and Interpretable Data Synthesis
Fourier series-based tampering synthesis (FSTS) models invisible tampering parameters, significantly improving real-world forgery localization generalization.
Key Findings
Methodology
This work collects 16,750 real tampered text images with multi-format logs, analyzing behavioral patterns to build a hierarchical Fourier series model. Each tampering parameter is represented as a linear combination of basis functions, with learned weights enabling interpretable sampling. This approach captures complex, invisible tampering behaviors, generating diverse, realistic training data that enhances model robustness against real-world forgeries.
Key Results
- Models trained with FSTS data outperform baseline synthetic datasets, with F1 scores improving by over 15% and AUC by more than 10% on real datasets across four evaluation protocols.
- Cross-domain tests on AFAC, CertificatePS, and other real-world datasets show significant performance gains, confirming the effectiveness of modeling invisible tampering parameters.
- Ablation studies demonstrate that frequency-based parameter modeling yields better diversity and generalization than direct sampling, leading to more accurate forgery localization.
Significance
This research addresses the critical gap in synthetic data realism by modeling hidden behavioral parameters of tampering. It advances forensic technology by providing more representative training data, enabling models to better detect subtle, complex forgeries in real-world scenarios. The approach bridges the distribution gap, facilitating practical deployment of forgery localization systems and setting a new standard for data synthesis in digital forensics.
Technical Contribution
The core innovation lies in integrating Fourier series principles into behavior-based tampering parameter modeling, creating an interpretable, hierarchical frequency domain framework. This surpasses prior deep generative models by emphasizing explainability and behavioral consistency, offering a new paradigm for realistic data synthesis that captures complex, invisible tampering traces.
Novelty
This is the first work to apply Fourier series decomposition to model the latent distribution of tampering parameters in text images. It systematically captures recurring behavioral patterns at both individual and population levels, providing a structured, interpretable approach that significantly improves the diversity and realism of synthetic training data compared to existing rule-based or GAN-based methods.
Limitations
- The approach relies on extensive collection of real tampering operation logs, which is labor-intensive and costly, limiting scalability to broader scenarios.
- The frequency domain model assumes certain linearity and periodicity, which may not fully capture highly nonlinear or atypical tampering behaviors.
- Synthetic samples, although diverse, are constrained by the predefined set of operation configurations, potentially missing rare or novel tampering strategies.
Future Work
Future directions include integrating multi-modal data (video, audio) for richer behavior modeling, extending the framework to nonlinear and non-periodic behaviors, and automating data collection to scale the approach. Additionally, exploring adaptive models that update with new tampering patterns will further enhance robustness.
AI Executive Summary
In the rapidly evolving field of digital forensics, the detection of manipulated text images remains a significant challenge. Existing datasets often focus on visible artifacts, neglecting the complex, invisible parameters underlying real-world forgery behaviors. This limits the ability of deep learning models to generalize effectively across diverse scenarios. To address this, the authors introduce Fourier series-based Tampering Synthesis (FSTS), a novel framework that models the hidden distribution of tampering parameters by analyzing extensive real-world operation logs. By decomposing these parameters into interpretable basis functions, the method captures recurring behavioral patterns at both individual and population levels. This hierarchical frequency domain model enables the generation of highly diverse and realistic tampered images, which serve as superior training data. Extensive experiments across multiple benchmarks demonstrate that models trained with FSTS data outperform traditional synthetic datasets, achieving over 15% improvement in F1 scores and 10% in AUC on real-world datasets. The approach effectively narrows the distribution gap, significantly enhancing the robustness and generalization of forgery localization systems. This work marks a substantial step toward practical, scalable solutions for digital document security, with promising avenues for future multi-modal behavior modeling and automated data collection. Despite its strengths, the method faces challenges in scalability and modeling highly nonlinear behaviors, suggesting directions for ongoing research and refinement.
Deep Analysis
Background
随着数字文本图像在金融、证件、新闻等领域的广泛应用,篡改检测成为信息安全的重要环节。早期方法依赖手工特征,如字符错位、字体不一致等,但随着篡改技术的提升,这些方法逐渐失效。深度学习模型虽提升检测能力,但受限于数据规模和多样性,难以应对复杂场景。现有数据多为人工合成,存在分布偏差,影响模型泛化。近年来,合成数据研究逐渐兴起,利用规则或生成模型模拟篡改,但多关注可见特征,忽略潜在行为参数,导致在真实环境中表现不足。因此,迫切需要结合行为分析和频域建模,提升数据的真实性和多样性。
Core Problem
核心问题在于如何模拟真实场景中隐性篡改参数的复杂分布。现有合成方法多关注表面特征,忽略篡改行为的潜在参数,导致模型在复杂场景下泛化能力不足。采集真实篡改轨迹成本高,难以大规模应用。如何在有限数据基础上,建立具有代表性和多样性的潜在参数模型,成为提升篡改检测效果的关键。
Innovation
本研究的创新在于引入基于傅里叶级数的潜在参数建模框架。首先,设计结构化采集流程,自动记录多格式篡改轨迹,分析行为模式。其次,提出层次化频域模型,将每个篡改参数表示为基础操作-参数配置的线性组合,结合频率学习实现高效采样。最后,通过频域逼近与行为复现,生成多样逼真的训练样本,显著提升模型泛化能力。这一方法区别于传统深度生成模型,强调可解释性和行为一致性。
Methodology
- �� 采集:通过多格式日志记录67名专家和志愿者的篡改操作,构建16750个实例。
- �� 行为分析:识别重复行为模式,统计操作参数频率。
- �� 模型建立:将每个篡改参数表示为基础操作-参数配置的线性组合,借鉴傅里叶级数思想,建立频域模型。
- �� 频域逼近:学习基础函数及其权重,实现参数的高效采样。
- �� 样本生成:从模型中采样参数,应用于原始图像,合成逼真篡改样本。
- �� 训练:利用合成数据训练篡改定位模型,验证泛化能力。
Experiments
采用公开的合成数据集(如DocTamper)进行训练,设计四个评估协议,包括合成到合成、合成到真实、真实到真实、预训练加微调。模型性能以F1和AUC指标评估,比较传统合成方法与FSTS增强效果。跨域测试验证模型泛化能力,进行消融分析确认频域建模优势。实验结果充分证明了FSTS在提升真实场景检测能力方面的有效性。
Results
使用FSTS合成数据训练的模型在真实数据集上F1值平均提升15%以上,AUC提升10%以上。跨域测试中,性能优于传统方法,验证了潜在参数模拟的有效性。消融实验显示频域模型优于直接采样,样本多样性增强,模型在复杂篡改场景中表现更稳健。
Applications
该方法适用于数字取证、内容验证、反篡改系统。通过模拟真实潜在参数,提升模型在复杂环境中的检测能力。未来结合多模态信息,自动化生成更丰富的篡改样本,推动行业标准制定。
Limitations & Outlook
模型依赖大量真实操作轨迹,采集成本高,难以快速扩展。频域模型假设线性特性,可能在非线性篡改中表现不足。合成样本虽多样,但受基础操作库限制,未来需增强多样性和非线性表达能力。
Plain Language Accessible to non-experts
想象你在厨房做菜,每个人用不同的调料、火候和时间,做出来的菜味道都不一样。为了让别人也能做出和你一样的菜,你需要理解这些隐形的习惯。本文的方法就像用数学的“调味歌”来总结这些习惯,把它们变成一首歌,大家都可以用来做菜。通过分析很多真实的篡改操作,提取出常用的“调味音符”,用它们随机生成新的篡改样本。这样,电脑就能更好地理解和模拟复杂的篡改行为,提升检测的准确性。
ELI14 Explained like you're 14
想象你和朋友在玩拼图游戏,每个人拼的方式都不一样。有的人喜欢先拼边缘,有的人喜欢拼颜色块。为了让每个人都能拼出像你一样的拼图,你需要知道他们的习惯。这项研究就像用数学的“调味歌”总结这些拼图习惯,把它们变成一首歌,大家都可以用它来拼拼图。它用一种特别的数学方法,把每个人的拼图习惯变成一组“音符”,然后用这些“音符”随机生成新的拼图样本。这样,无论是谁用这个“歌”来拼,都能拼出和你类似的拼图,但每个拼图又有点不同。这个方法让电脑更聪明,能更好地理解和模拟那些看不见的拼图习惯,从而更准确地找到假拼图。
Abstract
Existing Text Image Forgery Localization (T-IFL) methods often suffer from poor generalization due to the limited scale of real-world datasets and the distribution gap caused by synthetic data that fails to capture the complexity of real-world tampering. To tackle this issue, we propose Fourier Series-based Tampering Synthesis (FSTS), a structured and interpretable framework for synthesizing tampered text images. FSTS first collects 16,750 real-world tampering instances from five representative tampering types, using a structured pipeline that records human-performed editing traces via multi-format logs (e.g., video, PSD, and editing logs). By analyzing these collected parameters and identifying recurring behavioral patterns at both individual and population levels, we formulate a hierarchical modeling framework. Specifically, each individual tampering parameter is represented as a compact combination of basis operation-parameter configurations, while the population-level distribution is constructed by aggregating these behaviors. Since this formulation draws inspiration from the Fourier series, it enables an interpretable approximation using basis functions and their learned weights. By sampling from this modeled distribution, FSTS synthesizes diverse and realistic training data that better reflect real-world forgery traces. Extensive experiments across four evaluation protocols demonstrate that models trained with FSTS data achieve significantly improved generalization on real-world datasets. Dataset is available at \href{https://github.com/ZeqinYu/FSTS}{Project Page}.