MnasNet: Platform-Aware Neural Architecture Search for Mobile

The rapid growth of deep learning has led to increasingly complex models, achieving remarkable performance in tasks like image classification and object detection. However, deploying these models on mobile devices remains challenging due to limited computational resources, power constraints, and latency requirements. Early lightweight models such as SqueezeNet, MobileNet, and ShuffleNet introduced techniques like depthwise separable convolutions and channel shuffling to reduce model size and computation. Nonetheless, manual design of optimal architectures is labor-intensive and often suboptimal. Recent advances in neural architecture search (NAS) have automated this process, leading to models like NASNet and AmoebaNet that outperform hand-crafted counterparts. Yet, most NAS methods optimize proxy metrics like FLOPS or parameter count, which do not always correlate with real-world latency on mobile hardware. As hardware heterogeneity increases, there is a pressing need for hardware-aware NAS that directly considers actual inference speed, enabling models that are truly optimized for deployment in resource-constrained environments.

The core challenge addressed in this paper is to develop a neural architecture search method that explicitly incorporates real-world inference latency on mobile devices into the optimization process. Traditional proxy metrics such as FLOPS or model size often fail to accurately predict actual latency due to hardware-specific factors like memory bandwidth, parallelism, and software stack inefficiencies. This discrepancy leads to models that perform well in theory but poorly in practice. Moreover, existing NAS approaches tend to generate architectures with repetitive cell structures, limiting layer diversity and potential performance gains. Balancing high accuracy with low latency under strict hardware constraints remains a significant bottleneck, especially when models must be scaled across different devices and applications. The paper proposes a solution that directly measures latency during search and employs a multi-objective framework to find Pareto-optimal solutions tailored for mobile hardware.

The main innovations include: 1) integrating real-device latency measurements into the NAS process, ensuring models are optimized for actual hardware performance; 2) designing a factorized hierarchical search space that allows different blocks within the network to have distinct architectures, promoting layer diversity; 3) employing reinforcement learning with a multi-objective reward function that balances accuracy and latency, enabling the discovery of models that meet diverse deployment constraints; 4) using a controller trained via Proximal Policy Optimization to efficiently explore the large search space. These innovations collectively enable the automatic design of mobile neural networks that are both accurate and hardware-efficient, overcoming limitations of proxy-based methods and fixed cell stacking strategies.

• �� Construct a multi-objective reward function combining accuracy and real-device latency, with adjustable weights.
• �� Build a factorized hierarchical search space dividing the network into multiple blocks, each with independently optimized parameters such as convolution type, kernel size, and number of layers.
• �� Use a reinforcement learning controller, based on RNN, to generate architecture tokens for each block, sampling diverse models.
• �� For each sampled architecture, train on ImageNet or COCO, then measure inference latency directly on mobile phones.
• �� Calculate reward based on accuracy and latency, guiding the controller to favor Pareto-efficient models.
• �� Repeat sampling and evaluation until convergence, selecting top models for deployment.
• �� Validate models across different hardware and tasks, including classification and object detection, demonstrating robustness and efficiency.

The experimental setup involves searching on ImageNet with a limited number of epochs due to high computational cost, using a Pixel phone for latency measurement. About 8,000 models are sampled, with top candidates evaluated on full datasets. Hyperparameters include RMSProp optimizer, learning rate schedules, batch normalization, and dropout. The models are compared against MobileNetV2, NASNet, and other lightweight architectures, with metrics including Top-1 accuracy, parameters, multiply-adds, and real-device latency. Additional ablation studies examine the impact of different reward weights and search space configurations. For object detection, models are integrated into SSD frameworks and evaluated on COCO, confirming the generality of the approach.

MNASNet-A1 achieves 75.2% Top-1 accuracy with 78ms latency, outperforming MobileNetV2 (72.0%) and NASNet (74.0%) in speed and accuracy. The model uses only 3.9M parameters and 312M multiply-adds, representing a significant efficiency gain. In object detection, the feature extractor based on MNASNet attains 23.0% mAP with 42ms latency, surpassing MobileNets and matching state-of-the-art performance with fewer resources. The results demonstrate that directly optimizing for real-world latency yields models with superior practical performance, validating the multi-objective framework. Ablation studies confirm the importance of the hierarchical search space and hardware-aware reward in achieving these gains.

The proposed method is suitable for deploying real-time vision applications on smartphones, tablets, and embedded devices, including image classification, object detection, augmented reality, and autonomous systems. It enables developers to automatically generate models tailored to specific hardware constraints, reducing manual tuning efforts. The approach facilitates rapid prototyping and deployment of AI in resource-limited environments, broadening access to intelligent applications. In the long term, this framework can be extended to multi-task learning, multi-hardware optimization, and integrated with model compression techniques, further enhancing AI accessibility and efficiency in edge computing.

The reliance on real-device latency measurements makes the search process computationally intensive and time-consuming, limiting scalability. The predefined hierarchical search space, while flexible, may restrict exploration of more complex architectures. The approach's effectiveness in ultra-low latency scenarios (<20ms) remains to be validated. Hardware heterogeneity poses challenges for generalization, requiring multiple hardware-specific searches. Future work should focus on improving latency estimation efficiency, expanding architecture diversity, and automating multi-device adaptation to address these limitations.