You Only Train Once: Differentiable Subset Selection for Omics Data
YOTO framework combines differentiable subset selection and multi-task learning for efficient gene selection and prediction in single-cell transcriptomics.
Key Findings
Methodology
YOTO is an end-to-end framework combining differentiable subset selection and multi-task learning. It uses Gumbel-Softmax and Plackett-Luce models for sparse, differentiable gene selection, with a shared encoder for task-specific predictions. Multi-task learning enables shared representations across tasks, improving generalization.
Key Results
- Result 1: On the COVID-PBMC dataset, YOTO achieved a 5% F1-score and 8% AUPRC improvement over PERSIST.
- Result 2: On the VISp dataset, YOTO outperformed baselines across all subset sizes (16-256 genes), with 3%-7% F1-score gains.
- Result 3: Ablation studies demonstrated that YOTO's sparse selection mechanism significantly improves performance, especially for small subsets.
Significance
YOTO addresses the decoupling of feature selection and prediction in traditional methods. Its multi-task learning capability allows it to leverage shared information across tasks, making it particularly suitable for biomarker discovery and disease mechanism research in single-cell data. It improves predictive performance while reducing computational overhead.
Technical Contribution
Key contributions include: 1) a differentiable ranking mechanism using Gumbel-Softmax and Plackett-Luce models for sparse, discrete gene selection; 2) a multi-task learning framework supporting joint optimization across tasks; 3) eliminating the need for separate classifiers, directly evaluating selected subsets' predictive utility.
Novelty
YOTO is the first fully end-to-end sparse gene selection and prediction framework for single-cell transcriptomics, with a fully discrete selection mechanism. Unlike PERSIST, it avoids reliance on non-sparse masks, ensuring direct evaluation of selected subsets.
Limitations
- Limitation 1: YOTO relies on shared structure across tasks, which may limit performance in unrelated tasks.
- Limitation 2: Sensitivity to hyperparameters (e.g., k and τ) requires additional tuning.
- Limitation 3: Computational cost may increase for extremely large datasets.
Future Work
Future work includes extending YOTO to other omics data types (e.g., proteomics) and improving computational efficiency for large-scale datasets. Enhancing performance in unrelated tasks is another key direction.
AI Executive Summary
Single-cell transcriptomics faces challenges of high dimensionality, sparsity, and noise. Traditional feature selection methods decouple selection from prediction, limiting their effectiveness. YOTO introduces an end-to-end framework that integrates differentiable subset selection with multi-task learning. Its key mechanisms include a Gumbel-Softmax-based sparse selection mask and a Plackett-Luce-based ranking model, ensuring differentiability and sparsity.
Experiments on COVID-PBMC and VISp datasets show YOTO outperforms baselines like PERSIST in F1-score, AUPRC, and other metrics, especially for small gene subsets (e.g., 16 or 32 genes). Its multi-task learning capability enables information sharing across tasks, identifying biologically meaningful gene subsets.
While YOTO excels in performance and efficiency, it is sensitive to task correlations and hyperparameters. Future research can optimize its robustness and scalability. Overall, YOTO provides a powerful tool for biomarker discovery and disease research in single-cell transcriptomics.
Deep Analysis
Background
Single-cell transcriptomics has become a cornerstone in biomedical research. Traditional methods like mRMR and HSIC Lasso rely on statistical or information-theoretic measures but struggle with high-dimensional, sparse data. Deep learning approaches like PERSIST integrate feature selection and prediction but lack fully sparse selection, requiring additional classifier training.
Core Problem
High dimensionality and sparsity in single-cell transcriptomics make selecting biologically meaningful gene subsets challenging. Traditional methods fail to tightly couple selection and prediction, leading to suboptimal performance. Multi-task learning in single-cell data remains underexplored, particularly for leveraging partially labeled datasets.
Innovation
YOTO's innovations include: 1) a differentiable ranking mechanism using Gumbel-Softmax and Plackett-Luce models for sparse, discrete gene selection; 2) a multi-task learning framework for joint optimization across tasks; 3) direct evaluation of selected subsets without separate classifiers.
Methodology
- �� Uses Gumbel-Softmax and Plackett-Luce models to generate differentiable gene scores.
- �� Applies a sparse selection mask to retain the top-k genes.
- �� Shared encoder extracts latent representations from selected genes.
- �� Task-specific heads perform classification or regression.
- �� Multi-task loss function jointly optimizes all tasks.
Experiments
Experiments on COVID-PBMC and VISp datasets evaluate YOTO's performance in biomarker discovery and cell type classification. Baselines include Seurat, mRMR, and PERSIST. Metrics include F1-score, AUPRC, and AUROC. Subset sizes range from 16 to 256 genes, covering diverse scenarios.
Results
YOTO achieved a 5% F1-score and 8% AUPRC improvement on COVID-PBMC. On VISp, it outperformed baselines across all subset sizes, with 3%-7% F1-score gains. Ablation studies confirmed its sparse selection mechanism's critical role.
Applications
YOTO is suited for biomarker discovery, disease classification, and cell type identification in single-cell data. Its sparse selection makes it ideal for resource-constrained experimental settings.
Limitations & Outlook
YOTO's reliance on shared task structure may limit its applicability to unrelated tasks. It is sensitive to hyperparameters and computationally intensive for large datasets.
Plain Language Accessible to non-experts
Imagine YOTO as a smart shopping assistant. You need to pick a few items from a massive store to make a recipe. Traditional methods might pick items first and then try cooking, but YOTO selects items while testing them in real-time to ensure the best results. It even considers the preferences of all your family members (multi-task learning) to find ingredients that work for everyone, completing the task in one go.
ELI14 Explained like you're 14
Imagine you're playing a game where you need to pick a few skills to defeat a boss. Old methods just pick random skills and hope they work. YOTO is like a super-smart game assistant. It tests the skills while you're picking them to make sure they're the best combo. Plus, it can help you fight multiple bosses at the same time! Cool, right?
Glossary
Gumbel-Softmax
A differentiable sampling method used to sample from discrete distributions.
Used in YOTO's differentiable ranking mechanism.
Plackett-Luce model
A probabilistic ranking model that selects items based on scores.
Used for ranking genes during subset selection.
Multi-task learning
A learning paradigm that optimizes multiple related tasks simultaneously.
YOTO uses multi-task learning to share gene subsets across tasks.
COVID-PBMC dataset
A single-cell RNA-seq dataset of PBMCs from COVID-19 patients.
Used to evaluate YOTO's biomarker discovery capabilities.
VISp dataset
Single-cell transcriptomics data from the mouse primary visual cortex.
Used to evaluate YOTO's cell type classification performance.
Open Questions Unanswered questions from this research
- 1 How can YOTO perform well in tasks with no shared biological structure?
- 2 How can YOTO's computational efficiency be improved for large-scale datasets?
Applications
Immediate Applications
Biomarker Discovery
YOTO can quickly identify disease-relevant genes, aiding precision medicine research.
Cell Type Classification
Efficiently classifies cell types in single-cell data, supporting biological studies.
Long-term Vision
Cross-omics Analysis
Extending YOTO to proteomics and other omics data to enable integrative analysis.
Abstract
Selecting compact and informative gene subsets from single-cell transcriptomic data is essential for biomarker discovery, improving interpretability, and cost-effective profiling. However, most existing feature selection approaches either operate as multi-stage pipelines or rely on post hoc feature attribution, making selection and prediction weakly coupled. In this work, we present YOTO (you only train once), an end-to-end framework that jointly identifies discrete gene subsets and performs prediction within a single differentiable architecture. In our model, the prediction task directly guides which genes are selected, while the learned subsets, in turn, shape the predictive representation. This closed feedback loop enables the model to iteratively refine both what it selects and how it predicts during training. Unlike existing approaches, YOTO enforces sparsity so that only the selected genes contribute to inference, eliminating the need to train additional downstream classifiers. Through a multi-task learning design, the model learns shared representations across related objectives, allowing partially labeled datasets to inform one another, and discovering gene subsets that generalize across tasks without additional training steps. We evaluate YOTO on two representative single-cell RNA-seq datasets, showing that it consistently outperforms state-of-the-art baselines. These results demonstrate that sparse, end-to-end, multi-task gene subset selection improves predictive performance and yields compact and meaningful gene subsets, advancing biomarker discovery and single-cell analysis.