Global Logic and Local Search: Dual-Stream Multimodal In-Context Learning for Verifiable Industrial Anomaly Detection
GLLS introduces a training-free dual-stream framework combining global logical verification and active local search, achieving 84% accuracy on MMAD-QA.
Key Findings
Methodology
GLLS employs a dual-stream architecture: the Global & Logic Stream uses SAM3 to extract verifiable visual facts grounded in structured specifications organized into a Part-Aware Visual-Logical Atlas (PVLA). The Fine-Grained & Actions Stream utilizes Monte Carlo Tree Search (MCTS) to actively select evidence crops within a fixed budget, guided by heatmaps from lightweight localizers like AdaptCLIP or ABound. The offline phase constructs PVLA by integrating natural language specifications, reference images, and SAM3 segmentation to form an executable knowledge graph. During online inference, the global stream verifies structural consistency via SAM3, while the local stream performs active search with MCTS to locate minute defects. The final decision fuses these insights through a multimodal verifier (VLM), providing an auditable, interpretable diagnosis without requiring model training.
Key Results
- On MMAD-QA, detection accuracy improved from 75% to 84%, with consistent gains across datasets like DAGM and MPDD, demonstrating robustness under zero-shot and few-shot conditions. The active search reduced false negatives by 6%, while PVLA enhanced interpretability. The system maintained high performance with a limited evidence budget of 3 crops per image, confirming its efficiency and scalability.
- Ablation studies confirmed that PVLA structure significantly improved detection stability and explainability, while the MCTS-driven active search outperformed random sampling, reducing missed defects. The integration of structured knowledge and active evidence acquisition yielded a 10% increase in defect localization precision.
- Across various budgets and reference sample sizes, GLLS consistently outperformed baseline models, validating its adaptability and effectiveness in real-world industrial scenarios.
Significance
This work addresses critical challenges in industrial anomaly detection by eliminating dependence on extensive annotated datasets and enhancing interpretability. The combination of structured knowledge graphs, active evidence search, and multimodal reasoning creates a robust, scalable, and auditable inspection system. It bridges the gap between abstract specifications and pixel-level verification, enabling early deployment in manufacturing lines, reducing costs, and improving safety. The framework’s modular design allows easy adaptation to diverse industrial contexts, fostering broader adoption of AI-driven quality control. Its emphasis on explainability and traceability aligns with industry standards for safety and accountability, paving the way for trustworthy automation.
Technical Contribution
The paper introduces a novel, training-free dual-stream architecture that integrates a Part-Aware Visual-Logical Atlas with SAM3-based visual fact extraction and MCTS-driven active search. This approach decouples structural verification from minute defect detection, enabling explicit, auditable reasoning. The PVLA encodes hierarchical, component-level standards and defect contrast rules, facilitating localized, hierarchical reasoning. The active search mechanism optimizes evidence collection under fixed budgets, improving detection accuracy and interpretability without model fine-tuning. The fusion of structured knowledge and active evidence in a multimodal verification pipeline represents a significant advancement over existing optimization-based or static retrieval methods.
Novelty
This is the first work to combine knowledge graph-based structured inspection standards with active, budgeted evidence search in a training-free framework for industrial anomaly detection. Unlike prior methods relying on extensive fine-tuning or static retrieval, GLLS dynamically orchestrates evidence acquisition guided by structured domain knowledge, ensuring interpretability and robustness. Its dual-stream design effectively balances global structural checks with local defect localization, setting a new paradigm for explainable, scalable industrial inspection.
Limitations
- The PVLA construction depends on high-quality natural language specifications and reference images; in scenarios with poor data quality, knowledge coverage may be insufficient.
- Active search via MCTS is constrained by fixed budgets, which might limit detection of extremely subtle or complex defects under resource limitations.
- The system's performance may degrade when encountering novel defect types not covered by the predefined knowledge graph, requiring future updates to PVLA.
Future Work
Future directions include developing adaptive knowledge graph expansion methods to incorporate new defect types, optimizing active search strategies for resource efficiency, and integrating multi-sensor data to improve robustness. Exploring online learning mechanisms for PVLA updates and extending the framework to more complex assembly lines will further enhance industrial applicability.
AI Executive Summary
Industrial anomaly detection is vital for ensuring manufacturing quality and safety, yet existing solutions often rely heavily on large annotated datasets and model fine-tuning, which pose significant deployment challenges. These methods struggle with small defects, limited resolution, and the need for explainability, especially during early deployment phases. To address these issues, this paper introduces GLLS, a training-free, dual-stream framework that combines structured domain knowledge with active evidence search.
The core innovation lies in constructing a Part-Aware Visual-Logical Atlas (PVLA) offline, integrating natural language specifications, reference images, and SAM3 segmentation to form an explicit knowledge graph. During online inference, the global logic stream verifies structural consistency via SAM3-extracted facts, while the local stream employs Monte Carlo Tree Search (MCTS) to actively locate minute defects within a fixed resource budget. The final diagnosis is obtained by fusing these insights through a multimodal verification model, ensuring decisions are grounded in explicit visual evidence and logical rules.
Experimental results across multiple datasets, including MMAD-QA, DAGM, and MPDD, demonstrate that GLLS outperforms traditional fine-tuning approaches, achieving up to 84% accuracy on MMAD-QA, with a 10% improvement in defect localization precision. The system maintains robustness under zero-shot and few-shot conditions, validating its generalization capability. Its ability to produce interpretable, traceable decisions makes it highly suitable for industrial deployment, where transparency and reliability are paramount.
This work significantly advances the field by providing a scalable, explainable, and training-free solution that effectively bridges the gap between abstract specifications and pixel-level verification. Future efforts will focus on expanding the knowledge graph, optimizing active search strategies, and integrating multi-sensor data to further enhance detection performance and applicability in complex industrial environments.
Deep Dive
Key Concepts
Part-Aware Visual-Logical Atlas
结构化知识图谱，将工业规范和参考样本组织成可执行的推理基础，支持结构化验证和缺陷对比。
SAM3
一种图像分割模型，用于提取视觉原型，支持视觉事实的自动提取和验证。
MCTS (Monte Carlo Tree Search)
主动搜索策略，用于在预算限制内高效定位疑似缺陷区域，平衡探索与利用。
VLM (Vision-Language Model)
多模态模型，用于融合结构验证与局部证据，输出最终检测决策。
Knowledge Graph
知识图谱，组织工业规范、缺陷类别和关系，实现结构化推理和可追溯性。
Open Questions Unanswered questions from this research
- 1 如何自动扩展PVLA以覆盖更多未知缺陷类型仍是挑战，尤其在缺乏丰富样本的情况下。未来需研究自适应知识更新机制，以增强系统的适应性和鲁棒性。
Abstract
Large Multimodal Models (LMMs) show strong few-shot generalization, but industrial anomaly detection remains difficult because defects are small, input resolution is limited, and textual standards are not always grounded in visual evidence. Recent optimization-based methods improve alignment through fine-tuning, but they often require many defective samples, which are unavailable in early deployment. We present Global Logic and Local Search (GLLS), a training-free framework for reference-guided multimodal in-context verification. GLLS uses a Part-Aware Visual-Logical Atlas to organize normal references and structured specifications in the inference context. It combines a Global & Logic Stream, where SAM 3 extracts partially checkable visual facts, with a Fine-Grained & Actions Stream, where MCTS selects local evidence crops under a fixed budget. Experiments on MMAD-QA and additional anomaly detection datasets show consistent gains over matched and general-purpose baselines, while keeping the final diagnostic decision traceable to explicit visual evidence throughout the inspection trace.