Argus: Orchestrating Cross-Layer GPU Performance Measurements around Semantic Regions
Argus automates cross-layer GPU performance measurements, boosting AlphaEvolve's speedup from 5.4% to 8.9%.
Key Findings
Methodology
Argus is a region-centric measurement planner and runtime that automates cross-layer GPU performance measurements. Clients identify code regions with markers and select signals and execution scopes. Argus preserves region identity, constructs multi-run plans, and orchestrates transformations and profiling across backends.
Key Results
- Across 44 persistent-GEMM and attention configurations, Argus improved performance in 39 configurations, raising AlphaEvolve's geometric mean speedup from 5.4% to 8.9%.
- On a persistent TinyLlama-1.1B decode megakernel, an optimization agent reached 1.65 ms/token with Argus versus 4.92 ms/token without it, producing a kernel 2.1Γ faster than PyTorch.
- Argus-guided cross-level PGO improved compute-communication overlap, increasing throughput by 7% on average across five multi-GPU settings.
Significance
Argus addresses the fragmentation in GPU performance measurement by providing automated and unified cross-layer measurements, offering developers and automated optimizers more precise performance evidence. This tool not only enhances existing optimization methods but also provides new perspectives and tools for future GPU optimization research.
Technical Contribution
Argus's technical contributions include a region-centric observability model that preserves region identity across compilation and execution, providing a common attribution space across profiler views. It also develops an interference-aware measurement planner capable of constructing valid multi-run measurement plans from region-level requests.
Novelty
Argus is the first to automate cross-layer GPU performance measurement, solving the fragmentation issue present in existing tools. Compared to existing methods, Argus better coordinates performance data across different layers, providing more comprehensive performance analysis.
Limitations
- Argus may encounter performance bottlenecks when handling very complex GPU kernels, as its measurement plans can introduce additional overhead.
- In some scenarios, Argus may not fully eliminate measurement interference, especially in highly concurrent execution environments.
- The current implementation of Argus only supports NVIDIA Hopper GPUs, limiting its applicability to other hardware platforms.
Future Work
Future work includes extending Argus's applicability to more hardware platforms and optimizing its measurement plans to reduce overhead. Additionally, exploring how Argus can be integrated with other performance optimization tools is an important direction.
AI Executive Summary
Argus is an innovative tool designed to address the fragmentation in GPU performance measurement. Existing performance analysis tools often provide only a single-layer view, making it difficult for developers to obtain comprehensive performance data. Argus offers a unified solution by automating cross-layer measurements.
The core of Argus lies in its region-centric measurement planner and runtime. Users can define code regions of interest through simple markers and signal selections, and Argus automatically preserves these regions' identities, coordinating transformations and analyses across different backends. This process significantly reduces the workload of performance measurement.
In experiments, Argus significantly improved the performance of various GPU configurations. For instance, on a persistent TinyLlama-1.1B decode megakernel, Argus increased the optimization agent's speed by 2.1 times. By improving compute-communication overlap, Argus increased throughput by 7% on average in multi-GPU settings.
Deep Dive
Abstract
GPU developers and automated optimizers need performance evidence for semantic code regions--such as neural-network operator implementations and pipeline stages--but this evidence is fragmented across profiling tools. Answering a region-level question can require manually constructing probes and program variants, isolating interfering measurements, and mapping evidence to regions and execution contexts. We present Argus, a region-centric measurement planner and runtime that automates this workflow. Clients identify regions with boundary markers and select signals and execution scopes. Argus preserves region identity across compilation, execution, and measurement variants, constructs interference-aware multi-run plans, and orchestrates transformations and profiling across backends. It joins compiler-, hardware-, and system-level evidence using region identity and dynamic execution context, producing reports that record measurement origins and attribution ambiguity. We evaluate Argus across agentic kernel optimization, persistent megakernel optimization, and cross-level PGO. Across 44 persistent-GEMM and attention configurations, Argus improves 39/44 cases and raises AlphaEvolve's geometric-mean speedup from 5.4% to 8.9%. On a persistent TinyLlama-1.1B decode megakernel, an optimization agent reaches 1.65 ms/token with Argus versus 4.92 ms/token without it, producing a kernel $2.1\times$ faster than PyTorch with CUDA Graphs. Finally, Argus-guided cross-level PGO improves compute--communication overlap, increasing throughput by 7% on average across five multi-GPU settings.