ARC-CT: Anatomy-Routed Contrastive Vision-Language Learning for 3D Chest CT
ARC-CT employs anatomy-guided contrastive learning, combining regional features to achieve 0.86 macro AUC on chest CT abnormalities.
Key Findings
Methodology
ARC-CT introduces a region-aware contrastive framework with AnatomyQFormer, which uses automatically generated organ masks to localize evidence. It combines a label-Jaccard soft InfoNCE loss to mitigate false negatives and employs offline large language models to extract report text for organ-level alignment. Training occurs in two stages: initial weak supervision with 18 abnormality labels, followed by region-guided contrastive fine-tuning. The architecture leverages a compact 3D ResNet-18 backbone, enabling efficient training and inference without manual bounding boxes. The approach emphasizes local feature representation, reducing the dilution effect of global pooling and addressing shared abnormalities across scans.
Key Results
- On the CT-RATE dataset, ARC-CT achieves a macro AUC of 0.86, outperforming MPS-CT (0.84) and GreenRFM (0.85) with the same backbone, especially excelling in small lesions like lung nodules (0.72) and opacities (0.72). External validation on RAD-ChestCT yields 0.73 macro AUC without fine-tuning, demonstrating good generalization. Ablation studies confirm the importance of weak supervision, anatomy routing, and soft label strategies, with performance gains of 0.03-0.04 in macro AUC.
- The model also excels in multi-modal retrieval, with image-to-report MAP of 71.3 and report-to-image R@5 of 11.4, indicating its potential for integrated diagnostic support. Results highlight that local, anatomy-guided features significantly improve detection of subtle abnormalities, addressing limitations of previous global pooling methods.
- Overall, ARC-CT's innovative use of automatic organ masks and soft contrastive targets enhances local feature sensitivity, leading to state-of-the-art performance in chest abnormality detection and cross-modal retrieval, with promising clinical implications.
Significance
This work advances medical AI by providing a highly efficient, annotation-free framework that emphasizes local anatomical evidence, crucial for early detection of subtle pathologies. It addresses long-standing challenges of false negatives and feature dilution in volumetric imaging, offering a scalable solution for large-scale clinical deployment. The integration of vision and language models enables more accurate, interpretable, and robust diagnostics, potentially transforming radiology workflows. Its lightweight architecture facilitates deployment in resource-constrained settings, broadening access to AI-powered diagnostics. The approach also opens new avenues for multi-modal medical AI research, emphasizing regional and anatomical priors without manual annotations.
Technical Contribution
The core technical innovation lies in the AnatomyQFormer, which employs role-specific queries constrained by organ masks to localize evidence without bounding boxes. The soft label-Jaccard InfoNCE loss effectively reduces false negatives by weighting shared labels, improving contrastive learning in multi-abnormal scenarios. The multi-task training combines global and organ-specific alignment, query supervision, and contrastive objectives within a unified framework. The architecture’s efficiency stems from using a compact 3D ResNet-18 backbone, enabling high performance with limited computational resources. This design bridges the gap between large-scale models and practical deployment, setting a new standard for lightweight, region-aware medical contrastive learning.
Novelty
This is the first work to incorporate automatic organ masks into a contrastive learning framework for 3D chest CT, enabling region-specific evidence localization without manual annotations. The soft Jaccard-based contrastive target addresses the false-negative issue inherent in batch-based contrastive loss, especially relevant for shared abnormalities. Unlike prior methods relying on region proposals or bounding boxes, ARC-CT leverages report-derived labels and organ masks for efficient, scalable training. Its multi-task design and anatomy-guided attention mechanism represent a significant step forward in local feature learning for volumetric medical imaging, opening new directions for annotation-free, region-aware AI models.
Limitations
- Model performance may degrade on scans from different institutions or protocols due to domain shift, requiring further robustness validation.
- Detection of extremely small or subtle lesions still poses challenges, especially when organ masks are imperfect.
- Dependence on automatic organ segmentation quality could limit accuracy if masks are inaccurate or incomplete.
Future Work
Future research will explore multi-source, multi-modal pretraining to improve generalization across diverse datasets. Incorporating multi-scale region guidance and self-supervised learning could enhance detection of microlesions. Additionally, integrating active learning and domain adaptation strategies will help extend robustness. The authors also plan to investigate replacing explicit organ masks with learned spatial priors, reducing reliance on segmentation quality, and deploying in real-world clinical settings for prospective validation.
AI Executive Summary
Deep Dive
Plain Language Accessible to non-experts
想象你在一家工厂工作，工厂里有许多不同的区域，每个区域负责生产不同的产品。有时候，工厂需要检查每个区域是否正常，但如果只看整个工厂的整体情况，可能会忽略某个小角落的问题。这个研究就像是给工厂装上了特殊的“放大镜”和“指示牌”，可以帮助工厂更好地找到那些微小但重要的问题。它用自动生成的“区域标签”引导模型专注于关键区域，不用人工标记，也不用边界框，只需依靠报告中的信息。这样，工厂就能更快、更准确地发现潜在的故障，特别是那些隐藏在角落里的细节。这个方法让工厂的检测变得更智能、更高效，也为未来的工厂自动化提供了新的思路。
ELI14 Explained like you're 14
想象你在学校里，有很多不同的教室，每个教室里都在上不同的课。有时候，老师会让你找出哪个教室有问题，比如说有个教室的灯坏了或者桌子不干净。可是如果你只看整个学校，可能就看不到这些细节。这个研究就像是给你一副特殊的眼镜，可以帮你专门看每个教室的细节。它会自动帮你找到每个教室的重点区域，比如灯坏了的地方，然后告诉你哪里需要修理。而且，它还会根据老师的报告，帮你确认哪些问题是真的，哪些是误会。这样一来，你就能更快、更准确地找到学校里的问题，特别是那些很小、很难发现的细节。这就像是给你装上了超级放大镜和智能助手，让你成为学校里最棒的“问题侦探”。
Abstract
Contrastive vision-language learning uses paired chest CT volumes and radiology reports to learn abnormality classifiers without manually annotated labels. However, two characteristics of chest CT challenge conventional global contrastive learning. First, many critical abnormalities are small or anatomically localized, and pooling an en- tire volume into a single embedding may dilute their visual evidence. Second, the standard contrastive objective treats every other scan in a batch as a negative. Because many chest CTs share abnormalities, this objective incorrectly pushes co-positive pairs apart. We propose Anatomy-Routed Contrastive Learning for 3D Chest CT (ARC-CT), a region-aware framework that addresses these limitations using only la- bels extracted from reports by an LLM, with no manual annotations or bounding boxes. ARC-CT combines three components: (1) an Anato- myQFormer localizing evidence via queries constrained by automatically generated organ masks; (2) a label-Jaccard soft InfoNCE objective in- tegrating the standard one-hot target with the label-set overlap of each pair, which reduces false-negative penalties between studies that share clinical findings; and (3) an organ-level alignment loss connecting mask- pooled visual features to organ-specific report text extracted offline with a large language model. ARC-CT achieves a 0.86 mask-free macro AUC across 18 abnormalities using a compact 3D ResNet-18 backbone. Over- all, ARC-CT outperforms both comparable efficient baselines and sev- eral larger transformer models. Our code and weights are available at https://github.com/arc-ct/arc-ct.