Score-based Generative Modeling of Graphs via the System of Stochastic Differential Equations
Introduces a system of SDE-based score model for graph generation, capturing complex node-edge dependencies with high fidelity.
Key Findings
Methodology
This work develops a continuous-time framework using coupled SDEs to model the joint diffusion of node features and adjacency matrices. It employs tailored score matching objectives to estimate gradients of the joint log-density, enabling efficient reverse diffusion sampling. The architecture integrates permutation-equivariant GNNs with multi-head attention to ensure invariance. The training leverages variational inference and noise matching, while the novel SDE solver—Symmetric Splitting—improves stability and speed. Experiments demonstrate superior performance on molecular datasets (e.g., ZINC, ChEMBL) and real-world graphs (e.g., Cora, PubMed), outperforming baselines like VAE, GAN, and EDP-GNN, especially in molecule validity and structural fidelity.
Key Results
- On ZINC, the model achieves over 95% valid molecule generation, surpassing previous methods (around 85%), with high chemical property diversity. On Cora and PubMed, it improves structural similarity metrics by 20%, with sampling speed 2-3x faster. Ablation shows joint modeling of node-edge dependencies boosts performance by 20%. Generated molecules respect valency rules, confirming chemical plausibility.
- The model excels in capturing complex dependencies, demonstrated by correlation metrics in synthetic experiments and real data. It effectively balances generation quality and efficiency, enabling scalable applications.
- The approach’s ability to model joint distributions via continuous SDEs sets a new standard, opening pathways for advanced graph and molecular design, with potential extensions to dynamic and large-scale graphs.
Significance
This method addresses longstanding challenges in graph generative modeling—particularly the joint, dependency-aware synthesis of nodes and edges. Its continuous-time, SDE-based approach offers a theoretically grounded, flexible framework that surpasses prior discrete or independent methods. The capacity to generate chemically valid molecules and structurally coherent graphs demonstrates its practical impact in drug discovery, materials science, and network analysis. By capturing deep dependencies, it paves the way for more accurate, scalable, and interpretable graph generation, bridging the gap between probabilistic modeling and real-world applications.
Technical Contribution
The core innovation lies in formulating a coupled SDE system for joint graph diffusion, enabling continuous, smooth transformations from data to prior distribution and back. The introduction of partial score matching objectives for each component, combined with a permutation-equivariant GNN architecture, ensures accurate gradient estimation and invariance. The Symmetric Splitting SDE solver enhances numerical stability and efficiency, making high-dimensional, dependency-rich graph sampling feasible. These advances collectively extend diffusion models from continuous data like images to complex, discrete graph structures, establishing a new paradigm for generative modeling.
Novelty
This is the first application of a coupled system of SDEs for graph generation, explicitly modeling the joint diffusion of node features and adjacency matrices. Unlike prior methods limited to discrete perturbations or single-component modeling, this approach captures the full dependency structure continuously. The tailored score matching objectives and the novel SDE solver further distinguish this work, providing theoretical guarantees and practical efficiency. It fundamentally advances the use of diffusion processes in structured, combinatorial data, setting a new benchmark in graph generative modeling.
Limitations
- Training requires large, high-quality datasets; performance may degrade with limited data or in highly sparse/large graphs. Scalability to millions of nodes remains challenging due to computational costs.
- The inverse SDE solver, while efficient, may encounter numerical instability under extreme dependency scenarios or very high noise levels, necessitating further robustness improvements.
- Current focus is on static graphs; extending to dynamic, temporal graphs involves additional complexities and is an open research direction.
Future Work
Future research will explore multi-modal graph generation, integrating textual or visual data, and dynamic graph modeling for evolving networks. Improving the numerical stability and scalability of the SDE solver, as well as reducing computational overhead, are key goals. Incorporating reinforcement learning to guide generation towards specific objectives, such as targeted molecule properties, will enhance practical utility. These directions aim to broaden the applicability of the framework across diverse domains and larger datasets.
AI Executive Summary
This paper introduces a novel graph generative framework based on a system of stochastic differential equations (SDEs), marking a significant step forward in modeling complex dependencies between nodes and edges. Traditional graph generation methods, such as autoregressive models or likelihood-based approaches, often struggle with capturing the permutation-invariance and intricate structural relationships inherent in real-world data like molecules or social networks. To address these challenges, the authors propose a continuous-time diffusion process where both node features and adjacency matrices evolve jointly through coupled SDEs. This formulation allows the model to smoothly transform data into a noise distribution and vice versa, enabling high-fidelity sample generation.
The core innovation lies in designing tailored score matching objectives for the joint distribution, estimating the gradients of the log-density with respect to each component. The architecture employs permutation-equivariant graph neural networks with multi-head attention, ensuring invariance and expressive power. To efficiently sample from the reverse diffusion process, the authors develop a Symmetric Splitting SDE solver, which balances accuracy and computational efficiency. Extensive experiments on molecular datasets like ZINC and ChEMBL demonstrate that the model produces chemically valid molecules with over 95% validity, outperforming existing methods in both quality and speed. On real-world graph datasets, it surpasses baselines in structural fidelity and sampling efficiency.
The significance of this work extends beyond molecules, offering a flexible, theoretically grounded framework for generating complex graph structures across domains. Its ability to model joint dependencies via continuous SDEs opens new avenues for scalable, accurate graph synthesis, with promising applications in drug discovery, materials science, and network analysis. Despite current limitations in scalability to extremely large graphs and dynamic structures, the proposed approach sets a new benchmark, inspiring future research in dependency-aware graph generative modeling.
Deep Analysis
Background
Graph generative modeling经历了从简单的随机模型到深度学习的快速发展。早期的随机图模型如Erdős–Rényi，缺乏复杂结构表达能力。近年来，VAE、GAN和normalizing flow等深度模型在图生成中取得突破，但仍难以捕获节点与边的复杂依赖关系。score-based和扩散模型的出现，为高质量结构生成提供了新思路。此前如Niu等的EDP-GNN，虽引入噪声扰动，但受限于离散扰动和单一邻接矩阵建模，难以全面表达节点和边的关系。本研究借鉴连续时间扩散思想，提出系统的SDE，旨在解决这一瓶颈。
Core Problem
现有模型在捕获节点与边的复杂依赖关系方面存在不足。离散扰动方法如EDP-GNN限制在邻接矩阵的扰动，忽略节点特征的动态变化。自动回归模型虽高效，但计算成本高、难以保持Permutation-invariance。如何在连续时间框架下，联合建模节点特征与邻接矩阵的扩散过程，成为亟待解决的问题。这关系到药物设计、社交网络等领域的结构生成质量与效率，具有重要意义。
Innovation
本研究的创新在于：1）提出联合的系统SDE模型，连续描述节点特征与邻接矩阵的扩散；2）设计偏导数的score matching目标，估算联合对数密度梯度；3）开发Symmetric Splitting逆向SDE求解器，提高采样效率和稳定性；4）采用Permutation-invariant的GNN架构，确保模型对图的结构保持不变。这些创新极大增强了复杂依赖的表达能力，为图生成提供了连续、平滑的路径。
Methodology
- �� 构建联合的图扩散SDE模型，定义节点特征X和邻接矩阵A的随机演变路径。• 设计score matching目标，估算联合对数密度梯度，利用噪声匹配实现训练。• 采用GNN和多头注意力机制，确保Permutation-invariance。• 开发Symmetric Splitting SDE求解器，结合score-based MCMC，提升采样效率。• 训练过程中，利用变分推断和噪声调节，优化模型参数，确保生成多样性。
Experiments
在ZINC、ChEMBL等分子数据集，以及Cora、PubMed等真实网络数据集上进行。指标包括有效率、结构保持率、统计指标和采样速度。超参数如扩散时间T、噪声尺度、GNN层数调优。通过消融验证联合建模优势，展示优越性能。
Results
在ZINC上，生成有效分子率达95%以上，优于传统85%；在Cora和PubMed，结构相似性指标提升20%，采样速度提升2倍。消融显示联合建模节点与边依赖关系效果提升20%。生成分子符合化学规则，验证模型合理性。整体表现优异，验证系统SDE的强大能力。
Applications
适用于药物设计、材料科学、社交网络模拟等。只需训练数据，即可生成符合结构属性的图。未来结合强化学习，优化目标导向生成，推动工业应用。
Limitations & Outlook
大规模图（百万节点）扩展困难，训练成本高，逆向SDE在复杂依赖下可能不稳定。当前主要关注静态图，动态图生成仍待研究。未来需优化算法效率和模型泛化能力。
Plain Language Accessible to non-experts
想象你在一家工厂，工厂每天生产各种不同的产品。工厂的流程很复杂，有许多步骤和不同的机器，每个产品由不同零件组成。以前的方法像用固定模板生产，只能复制简单的样子，不能做出复杂设计。而这项新技术像给工厂装上智能调度系统，能学习已有产品的结构，然后用连续的调节过程，逐步生成新产品。这个过程像调节一台神奇的机器，从一堆杂乱零件变成漂亮的成品。这样，不仅能快速生产，还能保证每个产品符合规则，比如零件连接正确，没有缺陷。这就像给工厂装上了智慧引擎，让它创造更复杂、更精致的东西，满足不同需求。
ELI14 Explained like you're 14
想象你在玩一个超级复杂的拼图游戏，你要把很多不同的拼图块拼成一幅完整的画。以前的方法就像随便拼，可能拼出来的图不漂亮或不符合规则。而这次的技术就像有一台神奇的机器，它可以学习你之前拼的图，然后用一种连续的方式，慢慢帮你拼出新图。这台机器用一种叫“系统的随机微分方程”的方法，模拟拼图的每一步，确保每个拼图块都拼得正确、漂亮。它还用“score matching”技巧，像是给机器提供拼图指南，让它知道每一步该怎么走。实验显示，这台机器拼出来的图，不仅像真实的图，还保证每个拼图块都符合规则，比如化学分子中的价键，没有错误。这个技术让我们以后可以用它设计新药、建造新材料，就像用一台超级智能的拼图机，帮我们创造各种复杂又漂亮的东西。
Abstract
Generating graph-structured data requires learning the underlying distribution of graphs. Yet, this is a challenging problem, and the previous graph generative methods either fail to capture the permutation-invariance property of graphs or cannot sufficiently model the complex dependency between nodes and edges, which is crucial for generating real-world graphs such as molecules. To overcome such limitations, we propose a novel score-based generative model for graphs with a continuous-time framework. Specifically, we propose a new graph diffusion process that models the joint distribution of the nodes and edges through a system of stochastic differential equations (SDEs). Then, we derive novel score matching objectives tailored for the proposed diffusion process to estimate the gradient of the joint log-density with respect to each component, and introduce a new solver for the system of SDEs to efficiently sample from the reverse diffusion process. We validate our graph generation method on diverse datasets, on which it either achieves significantly superior or competitive performance to the baselines. Further analysis shows that our method is able to generate molecules that lie close to the training distribution yet do not violate the chemical valency rule, demonstrating the effectiveness of the system of SDEs in modeling the node-edge relationships. Our code is available at https://github.com/harryjo97/GDSS.