Label-invariant Augmentation for Semi-Supervised Graph Classification
Proposes label-invariant augmentation in representation space, generating hardest samples to improve semi-supervised graph classification.
Key Findings
Methodology
This paper introduces Graph Label-invariant Augmentation (GLA), perturbing graph representations within the embedding space to produce the most challenging samples while preserving labels. Random candidate perturbations are generated, and the one closest to the decision boundary is selected to maximize model robustness. The framework integrates a GNN encoder (ResGCN), a classifier, and a contrastive head, trained jointly with a combined contrastive and classification loss. The augmentation avoids explicit graph modifications, reducing complexity and leveraging label information to guide the process, ensuring label consistency and enhancing generalization.
Key Results
- On 8 benchmark datasets, GLA outperforms classical GNNs and recent contrastive methods, with accuracy improvements of 2-3%. In particular, on MUTAG, accuracy reaches 90.00%, surpassing GraphCL and JOAOv2. The method maintains high performance across label ratios (30%, 50%, 70%), demonstrating robustness. Ablation studies confirm the importance of perturbation magnitude η and candidate number, with performance gains attributed to the difficulty-maximizing strategy. The results show GLA's adaptability across biological and social network data, confirming its broad applicability.
- Across datasets, GLA ranks best overall, especially at higher label ratios, with significant gains over baselines. Analysis of the difficulty distribution of augmented samples reveals enhanced boundary robustness. The method's stability is validated through extensive ablations, indicating its effectiveness in diverse semi-supervised scenarios. The experimental evidence underscores the potential of embedding-space augmentation for improving graph representation learning.
- Furthermore, GLA demonstrates consistent improvements when combined with traditional data augmentation, indicating synergy. Its ability to generate label-preserving, challenging samples without explicit graph modifications offers a scalable, robust alternative to conventional augmentation strategies, promising for large-scale and real-world applications.
Significance
This work addresses a core limitation in graph contrastive learning—the reliance on handcrafted augmentations that risk damaging graph structures or mislabeling. By shifting augmentation into the embedding space guided by label information, GLA ensures label consistency and improves robustness, especially under limited labels. This approach simplifies augmentation design, reduces dependency on domain expertise, and enhances model generalization. Its broad applicability across biological, social, and technological networks highlights its potential to revolutionize semi-supervised graph learning, making it more scalable and reliable for real-world tasks. The method paves the way for future integration with active and meta-learning frameworks, further boosting performance and adaptability.
Technical Contribution
The core technical innovation lies in performing label-preserving perturbations directly in the graph embedding space, avoiding explicit graph modifications. The method employs a stochastic candidate generation process, selecting the most challenging sample based on proximity to the decision boundary, thus maximizing model robustness. It combines contrastive loss (NT-Xent) with classification loss in an end-to-end training pipeline, enabling simultaneous representation learning and label-guided augmentation. This approach offers theoretical guarantees of label invariance and robustness, reduces complexity compared to graph-structure augmentations, and enhances the ability to generalize from limited labeled data. The framework can be integrated with various GNN architectures, broadening its applicability.
Novelty
This research is the first to introduce label-invariant augmentation directly in the representation space for graph data, circumventing issues associated with traditional graph augmentations that may alter structure or labels. Unlike prior methods relying on handcrafted graph operations, GLA leverages model boundary information to generate the most challenging samples, effectively maximizing the margin and robustness. Its end-to-end design that jointly optimizes contrastive and classification objectives, guided by label consistency, marks a significant departure from existing semi-supervised and contrastive learning frameworks. This innovation opens new avenues for scalable, label-preserving data augmentation in graph learning.
Limitations
- The effectiveness heavily depends on accurate boundary estimation; with very limited labels, boundary approximation may be unreliable, reducing augmentation quality.
- Parameter choices such as perturbation magnitude η and candidate number influence performance, requiring careful tuning.
- In high-noise or highly imbalanced datasets, the method may generate less meaningful challenging samples, impacting robustness.
Future Work
Future directions include integrating active learning to adaptively refine boundary estimation, extending the approach to dynamic and heterogeneous graphs, and exploring multi-task settings. Combining GLA with meta-learning could further improve adaptability to diverse data distributions. Additionally, scaling the method for large graphs with millions of nodes remains an open challenge, requiring efficient approximation techniques.
AI Executive Summary
Graph neural networks (GNNs) have revolutionized the analysis of graph-structured data, enabling breakthroughs in fields like bioinformatics and social network analysis. However, their performance in semi-supervised settings remains constrained by limited labeled data and reliance on handcrafted data augmentation strategies. Traditional augmentation methods—such as node dropping or edge perturbation—often risk damaging the graph's intrinsic structure or introducing label noise, which hampers model robustness and generalization.
This paper introduces a novel approach called Graph Label-invariant Augmentation (GLA), which shifts the augmentation process from the graph structure to the representation space. Instead of physically altering the graph, GLA perturbs the learned graph embeddings to generate the most challenging samples near the decision boundary, while ensuring label consistency. This is achieved by randomly sampling candidate perturbations and selecting the one that maximizes the model's difficulty, thus effectively training the model to handle hard cases.
The framework integrates a GNN encoder (ResGCN), a classifier, and a contrastive head, trained jointly with a combined contrastive and classification loss. The method leverages label information to guide the augmentation, ensuring that the generated samples are both challenging and label-preserving. Extensive experiments on eight benchmark datasets—including MUTAG, PROTEINS, and social network graphs—demonstrate that GLA consistently outperforms state-of-the-art methods, achieving accuracy improvements of up to 3% and surpassing baselines across all label ratios.
The significance of this work lies in its ability to enhance model robustness without complex graph manipulations, making it scalable and applicable to diverse real-world scenarios. It addresses a fundamental challenge in graph learning—how to generate meaningful, label-preserving augmented data—by innovatively operating in the embedding space guided by model boundaries. While promising, future work should focus on boundary estimation in extremely low-label regimes, extending to dynamic graphs, and improving scalability for large-scale applications. Overall, GLA offers a powerful new tool for advancing semi-supervised graph representation learning, with broad implications for both academia and industry.
Deep Dive
Abstract
Recently, contrastiveness-based augmentation surges a new climax in the computer vision domain, where some operations, including rotation, crop, and flip, combined with dedicated algorithms, dramatically increase the model generalization and robustness. Following this trend, some pioneering attempts employ the similar idea to graph data. Nevertheless, unlike images, it is much more difficult to design reasonable augmentations without changing the nature of graphs. Although exciting, the current graph contrastive learning does not achieve as promising performance as visual contrastive learning. We conjecture the current performance of graph contrastive learning might be limited by the violation of the label-invariant augmentation assumption. In light of this, we propose a label-invariant augmentation for graph-structured data to address this challenge. Different from the node/edge modification and subgraph extraction, we conduct the augmentation in the representation space and generate the augmented samples in the most difficult direction while keeping the label of augmented data the same as the original samples. In the semi-supervised scenario, we demonstrate our proposed method outperforms the classical graph neural network based methods and recent graph contrastive learning on eight benchmark graph-structured data, followed by several in-depth experiments to further explore the label-invariant augmentation in several aspects.