Manifold Alignment with Label Information
MALI leverages label-guided diffusion maps and optimal transport for manifold alignment, outperforming state-of-the-art methods.
Key Findings
Methodology
MALI employs diffusion maps to capture data geometry, constructs cross-domain similarity matrices incorporating class labels, and solves entropic optimal transport for sample matching. It avoids known correspondences, enabling both sample pairing and shared representation learning. The approach integrates graph-based diffusion with label aggregation, producing robust, nonlinear alignments suitable for complex datasets.
Key Results
- On MNIST-D, Helix, STL10, and RNA-ATAC datasets, MALI surpasses KEMA by over 30% in FOSCTTM scores, especially at low label ratios (1-5%). Label transfer accuracy reaches up to 97%, demonstrating superior matching and classification performance.
- In low-dimensional spaces (3-10D), MALI maintains stable, high-quality alignments. Soft matching with entropy regularization further improves robustness, capturing complex sample relationships effectively.
- Experiments show MALI's advantage in biological data integration and multi-modal vision tasks, outperforming linear and deep models, with better class separability and generalization.
Significance
This work advances manifold alignment by integrating label information into diffusion and optimal transport frameworks, addressing the challenge of aligning non-corresponding, nonlinear, multi-modal data. It provides a scalable, robust tool for data fusion, domain adaptation, and multi-omics analysis, impacting both theoretical research and practical applications across biology, computer vision, and machine learning.
Technical Contribution
The paper introduces a novel semi-supervised framework combining diffusion maps with label-guided similarity aggregation and entropy-regularized optimal transport. It offers theoretical guarantees on convergence, robustness to nonlinearities, and computational efficiency. The method extends existing manifold alignment techniques by enabling flexible, soft sample matching without known correspondences, a significant step forward in unsupervised and semi-supervised learning.
Novelty
This is the first approach to embed class labels directly into a diffusion-OT pipeline for manifold alignment, enabling effective, nonlinear, label-guided cross-domain matching without prior known correspondences. It bridges the gap between unsupervised and supervised methods, offering a new paradigm for multi-domain data integration.
Limitations
- The reliance on accurate class labels makes the method sensitive to label noise and imbalance, potentially degrading alignment quality. In highly noisy or unlabelled scenarios, performance may decline.
- Computational complexity scales with data size, especially in high-dimensional spaces, requiring further optimization for large-scale applications.
- The method's effectiveness diminishes with extremely complex nonlinear deformations or when the underlying manifold assumptions are violated.
Future Work
Future research will focus on extending MALI to fully unsupervised settings, improving scalability with stochastic or approximate algorithms, and integrating deep neural networks for end-to-end nonlinear embedding. Exploring robustness to noisy labels and unlabelled data, as well as applications in real-time multi-modal systems, are promising directions.
AI Executive Summary
The integration of multi-domain data remains a central challenge in modern data science, especially when datasets lack explicit correspondences or exhibit complex nonlinear relationships. Traditional methods like Canonical Correlation Analysis (CCA) and kernel-based approaches have made progress but often fall short in handling unpaired, highly nonlinear, or multi-modal data. Recent advances in manifold learning, such as Diffusion Maps and Laplacian Eigenmaps, have enabled better geometric understanding, yet aligning disparate datasets without prior correspondence remains difficult.
This paper introduces MALI, a novel semi-supervised manifold alignment framework that leverages label information to guide the alignment process. Building upon diffusion processes and optimal transport, MALI constructs graph-based representations of each domain, captures their global geometry, and aggregates class labels to compute cross-domain similarities. The core innovation lies in solving an entropic optimal transport problem to find soft, robust correspondences between samples, even in the absence of known pairs. This approach effectively balances local geometric fidelity with global class consistency.
Extensive experiments on datasets like MNIST-D, Helix, STL10, and RNA-ATAC demonstrate MALI’s superior performance over existing methods such as KEMA. Notably, in low-label regimes, MALI achieves over 30% improvement in FOSCTTM scores and up to 97% accuracy in label transfer tasks. The method’s stability across different embedding dimensions and its ability to produce meaningful, class-separated embeddings highlight its robustness and practical utility.
The significance of this work extends beyond technical novelty. By enabling effective, scalable, and flexible alignment of complex, multi-modal datasets, MALI paves the way for breakthroughs in biological data integration, computer vision, and transfer learning. Its ability to handle nonlinearities, partial labels, and soft correspondences makes it a versatile tool for real-world applications. Looking ahead, future research will aim to extend MALI to fully unsupervised settings, improve computational efficiency, and incorporate deep learning for end-to-end nonlinear embedding, broadening its impact across scientific disciplines.
Deep Dive
Abstract
Multi-domain data is becoming increasingly common and presents both challenges and opportunities in the data science community. The integration of distinct data-views can be used for exploratory data analysis, and benefit downstream analysis including machine learning related tasks. With this in mind, we present a novel manifold alignment method called MALI (Manifold alignment with label information) that learns a correspondence between two distinct domains. MALI can be considered as belonging to a middle ground between the more commonly addressed semi-supervised manifold alignment problem with some known correspondences between the two domains, and the purely unsupervised case, where no known correspondences are provided. To do this, MALI learns the manifold structure in both domains via a diffusion process and then leverages discrete class labels to guide the alignment. By aligning two distinct domains, MALI recovers a pairing and a common representation that reveals related samples in both domains. Additionally, MALI can be used for the transfer learning problem known as domain adaptation. We show that MALI outperforms the current state-of-the-art manifold alignment methods across multiple datasets.