Do we need equivariant models for molecule generation?
This study evaluates whether non-equivariant CNNs trained with rotation augmentation can learn equivariance, showing comparable performance to equivariant models in molecule tasks.
Key Findings
Methodology
The research employs rotation augmentation during training of CNN models, combined with a loss decomposition technique that separates prediction error from equivariance error. By comparing models like VoxMol and E3NN across tasks—denoising, molecule generation, and property prediction—the study quantifies how model size, dataset size, and training epochs influence learned equivariance. The loss decomposition involves Taylor expansion to measure the discrepancy between rotated inputs and outputs, enabling precise evaluation of approximate equivariance in non-architectural models.
Key Results
- CNNs trained with rotation augmentation quickly learn approximate equivariance in denoising tasks, achieving low equivariance error (<0.05) even with small models (7M parameters) and limited data (10%). For molecule generation, larger models (111M) maintain rotation-invariant outputs, with generated molecules showing similar chemical property distributions (KL divergence <0.02) regardless of seed rotation. Smaller models or less data lead to decreased stability and increased diversity, indicating incomplete equivariance. Latent space analysis reveals significant differences in embeddings for rotated molecules, suggesting redundant representations rather than true invariance.
- In property prediction tasks, models trained with reconstruction loss exhibit higher robustness to rotations, with Spearman correlations exceeding 0.9 and MAE differences minimal. Early training epochs suffice for learning approximate equivariance, with marginal gains from longer training. The study demonstrates that data augmentation alone can induce strong equivariance learning, reducing reliance on complex architectures.
- Experimental results highlight that model capacity and data volume influence the generalization of equivariance during generation, with larger models and more data producing more rotation-consistent molecules. The approach offers a scalable alternative to explicit equivariant architectures, enabling efficient molecular design workflows while maintaining high-quality outputs across orientations.
Significance
This work challenges the prevailing notion that explicit equivariant architectures are necessary for robust molecular generation. By showing that CNNs with data augmentation can learn near-equivariance, it opens pathways for simpler, more scalable models in drug discovery and materials science. The findings suggest that training strategies emphasizing data diversity can compensate for architectural complexity, reducing computational costs and implementation barriers. This paradigm shift enhances the accessibility of high-performance generative models, fostering broader adoption and faster innovation in molecular design workflows.
Technical Contribution
The paper introduces a formal loss decomposition framework that isolates prediction and equivariance errors, providing a quantitative measure of learned invariance. It empirically demonstrates that non-equivariant CNNs can approximate equivariance through data augmentation, with theoretical backing from Taylor expansion analysis. The study systematically explores the impact of model size, dataset scale, and training duration, offering practical guidelines for designing efficient molecular generative models without relying solely on equivariant architectures. This advances the understanding of how invariance properties emerge during training, bridging the gap between architectural design and data-driven learning.
Novelty
This is the first comprehensive analysis of learned equivariance in generative molecular models trained with data augmentation, contrasting with prior focus on supervised classification. It demonstrates that non-equivariant CNNs can achieve near-equivariance in reconstruction and generation tasks, challenging the assumption that explicit equivariant layers are indispensable. The study's theoretical and empirical insights provide a new perspective on how invariance properties develop during training, emphasizing the role of data diversity over architectural constraints.
Limitations
- While CNNs can learn approximate equivariance in denoising, their performance during molecule generation under seed rotations is less consistent, especially for smaller models or limited data, indicating incomplete invariance.
- Latent space analysis reveals redundant representations, which may hinder interpretability and efficiency, necessitating further regularization or alignment strategies.
- The experiments focus on the GEOM-Drugs dataset; applicability to larger, more diverse molecular datasets remains to be validated, and the approach may require adaptation for complex structures.
Future Work
Future research could incorporate contrastive learning or explicit regularization to align latent embeddings across rotations, further improving invariance. Extending experiments to larger, more diverse datasets and more complex molecular structures will test scalability. Additionally, integrating active invariance constraints during generation could enhance robustness, enabling more reliable applications in drug discovery pipelines.
AI Executive Summary
The rapid advancement of deep generative models has revolutionized molecular discovery, yet the reliance on complex equivariant architectures remains a bottleneck. Traditional models like SE(3)-equivariant GNNs offer high fidelity but are computationally intensive and difficult to scale. This study explores an alternative approach—training non-equivariant CNNs with rotation data augmentation—to assess their ability to learn approximate equivariance. Using a rigorous loss decomposition framework, the authors quantify how prediction and equivariance errors evolve during training across tasks such as denoising, molecule generation, and property prediction.
Results show that CNNs, even with limited parameters and data, can quickly acquire near-invariance in denoising tasks, indicating that explicit architectural equivariance may not be strictly necessary. Larger models trained on more data maintain rotational consistency during molecule generation, producing chemically valid and property-distribution-preserving outputs regardless of seed orientation. Conversely, smaller models or less data lead to degraded robustness, highlighting the importance of model capacity and data diversity.
A key insight from latent space analysis reveals that models learn redundant representations for rotated molecules, suggesting inefficiencies that could be mitigated through auxiliary regularization or contrastive learning. Overall, the findings challenge the dogma that explicit equivariant architectures are essential, proposing instead that data augmentation combined with loss-based quantification can achieve comparable robustness.
This work broadens the toolkit for molecular modeling, offering scalable, flexible alternatives that reduce complexity without sacrificing performance. It paves the way for more accessible, efficient drug discovery workflows, emphasizing the power of training strategies over architectural constraints. Future directions include enhancing latent alignment, testing on larger datasets, and integrating active invariance during generation, promising a new paradigm in molecular AI.
Deep Dive
Glossary
Equivariance (等变性)
函数f在变换R作用下满足f(Rx) = Rf(x)，确保模型对输入变换保持一致性。
定义3.1中描述模型对旋转的等变性特性。
Loss decomposition (损失分解)
将模型预测误差与等变误差拆分，量化模型学习到的等变性程度。
定理3.2中提出的关键技术，用于分析模型的等变性学习。
SE(3) group (SE(3)群)
包含三维空间中的旋转和平移变换的连续李群，描述空间刚体运动。
论文中关注的空间变换群，涉及旋转和位移的等变性。
Voxel representation (体素表示)
用三维像素网格表示分子结构，每个体素存储原子密度信息。
用于分子生成的空间表示方法。
WJS (Walk Jump Sampling)
基于扩散思想的采样方法，通过随机漫步和去噪实现分子生成。
论文中介绍的高效生成技术。
Open Questions Unanswered questions from this research
- 1 如何进一步提升非等变模型在生成任务中的旋转一致性，特别是在极端角度或复杂分子结构下的表现尚未充分理解。
- 2 现有数据增强策略在不同任务中的效果差异，缺乏统一理论框架解释其机制和局限。
- 3 潜在空间的冗余问题未被充分解决，影响模型的解释性和效率，未来需探索更有效的正则化策略。
Abstract
Deep generative models are increasingly used for molecular discovery, with most recent approaches relying on equivariant graph neural networks (GNNs) under the assumption that explicit equivariance is essential for generating high-quality 3D molecules. However, these models are complex, difficult to train, and scale poorly. We investigate whether non-equivariant convolutional neural networks (CNNs) trained with rotation augmentations can learn equivariance and match the performance of equivariant models. We derive a loss decomposition that separates prediction error from equivariance error, and evaluate how model size, dataset size, and training duration affect performance across denoising, molecule generation, and property prediction. To our knowledge, this is the first study to analyze learned equivariance in generative tasks.