R-FCN: Object Detection via Region-based Fully Convolutional Networks
R-FCN introduces position-sensitive score maps in a fully convolutional framework, achieving 83.6% mAP on VOC 2007 with 2.5× speedup over Faster R-CNN.
Key Findings
Methodology
This paper proposes R-FCN, a region-based fully convolutional network that encodes spatial information via position-sensitive score maps. The architecture integrates a shared convolutional backbone (e.g., ResNet-101), a bank of k×k score maps for each class, and a position-sensitive RoI pooling layer. This design eliminates the need for multiple region-wise subnetworks, drastically reducing computational cost while maintaining high accuracy. The entire system is trained end-to-end, combining RPN for proposals and the detection network, leveraging the deep features for precise localization and classification. The key innovation lies in encoding spatial position into the score maps, enabling the network to distinguish object parts without additional region-specific layers.
Key Results
- On PASCAL VOC 2007, R-FCN with ResNet-101 achieves 83.6% mAP, outperforming Faster R-CNN (76.4%) by a significant margin, with a test speed of 170ms per image, 2.5× faster. On COCO, the model reaches 51.5% AP with single-scale training, and 53.2% with multi-scale, demonstrating robustness across datasets. Ablation studies confirm that larger k (e.g., 7×7) enhances performance, and the position-sensitive approach effectively encodes spatial cues, especially for small objects.
- The speed advantage stems from removing per-region subnetworks, enabling shared computation across the entire image. The model maintains competitive accuracy while reducing inference time, making it suitable for real-time applications. The experiments validate that spatially sensitive score maps significantly improve localization without sacrificing speed, setting a new standard in efficient object detection.
Significance
This work addresses the longstanding challenge of balancing detection accuracy and computational efficiency in deep learning-based object detectors. By innovatively encoding spatial information directly into convolutional feature maps, R-FCN reduces redundant computations inherent in previous region-based methods. Its end-to-end trainability and compatibility with deep architectures like ResNet open new avenues for deploying high-performance detectors in real-world, resource-constrained environments. The approach's simplicity and effectiveness make it highly impactful for applications requiring fast, accurate detection, such as autonomous driving, surveillance, and robotics.
Technical Contribution
The core technical contribution is the introduction of position-sensitive score maps, which encode spatial information within the convolutional layers, enabling the network to distinguish object parts without additional region-specific layers. This design allows the entire detection pipeline to be fully convolutional, sharing features across the image and reducing computational redundancy. The position-sensitive RoI pooling operation aggregates responses from the score maps based on spatial bins, facilitating end-to-end training. This approach differs from prior methods by eliminating the need for multiple region-wise subnetworks, thus achieving higher speed without compromising accuracy. The theoretical guarantee lies in the spatial encoding capacity of the score maps, which preserves localization cues while leveraging deep convolutional features.
Novelty
This paper is the first to embed explicit spatial position encoding into convolutional score maps for object detection, creating a fully convolutional, end-to-end trainable system that encodes spatial cues without additional region-specific layers. Unlike previous approaches that rely on separate region-wise subnetworks or feature pyramids, R-FCN leverages a k×k grid of score maps to capture relative spatial positions, significantly reducing computation and complexity. This innovation bridges the gap between translation invariance in classification and translation variance needed for detection, setting a new paradigm for efficient, accurate detectors.
Limitations
- The effectiveness of the position-sensitive score maps depends on the choice of k; too small or too large k can degrade performance, requiring careful tuning. In extremely cluttered or occluded scenes, spatial encoding may be less discriminative. Additionally, the method's reliance on high-quality region proposals means that proposal quality directly impacts detection accuracy. While speed is improved, training on very large datasets or with complex multi-scale strategies still incurs substantial computational costs. Future work should explore adaptive k values and integration with multi-modal cues to further enhance robustness.
Future Work
Future directions include integrating multi-scale feature pyramids to better handle objects of varying sizes, exploring adaptive or learned grid partitioning for spatial encoding, and combining attention mechanisms to focus on salient regions. Extending the framework to video object detection and instance segmentation could further broaden its applicability. Additionally, optimizing the architecture for deployment on edge devices and exploring unsupervised or semi-supervised training paradigms are promising avenues to make this approach more scalable and versatile.
AI Executive Summary
Object detection remains a fundamental yet challenging task in computer vision, especially when balancing accuracy with real-time performance. Traditional methods like Faster R-CNN, despite其高精度，因区域子网络的重复计算而在速度上受到限制。为应对这一瓶颈，本文提出了R-FCN，一种基于全卷积架构的目标检测框架，核心创新为位置敏感得分图。通过在卷积层中编码空间位置信息，R-FCN实现了端到端训练，避免了多次区域子网络的重复计算，从而大幅提升检测速度。实验结果显示，使用ResNet-101作为骨干网络，R-FCN在VOC 2007上达到了83.6%的mAP，速度为170ms/图像，比Faster R-CNN快2.5倍，同时保持了优异的检测性能。在COCO数据集上，单尺度训练后，AP达51.5%，多尺度训练提升到53.2%。这些结果验证了空间敏感特征的有效性和模型的高效性。该方法的简洁设计和优越性能，为工业界的实时目标检测提供了新思路，特别是在自动驾驶、安防监控等应用场景中具有广泛潜力。未来，结合多尺度、多模态信息，将推动目标检测技术的进一步发展，向更高的速度和更强的鲁棒性迈进。
Deep Dive
Abstract
We present region-based, fully convolutional networks for accurate and efficient object detection. In contrast to previous region-based detectors such as Fast/Faster R-CNN that apply a costly per-region subnetwork hundreds of times, our region-based detector is fully convolutional with almost all computation shared on the entire image. To achieve this goal, we propose position-sensitive score maps to address a dilemma between translation-invariance in image classification and translation-variance in object detection. Our method can thus naturally adopt fully convolutional image classifier backbones, such as the latest Residual Networks (ResNets), for object detection. We show competitive results on the PASCAL VOC datasets (e.g., 83.6% mAP on the 2007 set) with the 101-layer ResNet. Meanwhile, our result is achieved at a test-time speed of 170ms per image, 2.5-20x faster than the Faster R-CNN counterpart. Code is made publicly available at: https://github.com/daijifeng001/r-fcn