Constrained Flow Optimization via Sequential Fine Tuning for Molecular Design

Recent years have seen rapid progress in deep generative models, especially flow and diffusion models, which excel at capturing complex data distributions. Key works like Ho et al. (2020) and Lipman et al. (2022) established their effectiveness in image synthesis and biological data generation. Despite these advances, applying such models to scientific discovery—such as designing molecules with specific properties—remains challenging. Existing methods like reward-guided fine-tuning (e.g., reinforcement learning, control schemes) can optimize for desired properties but often lack guarantees for satisfying hard constraints like synthesizability or toxicity limits. This gap limits their deployment in real-world applications where constraints are non-negotiable. Researchers have attempted to incorporate constraints via reward regularization, but manual tuning of weights is inefficient and unreliable, often leading to violations or suboptimal solutions. Therefore, a principled, automated approach with theoretical guarantees is highly desirable.

The core challenge is to develop a method that can reliably optimize for task-specific rewards while strictly satisfying domain-specific constraints. Traditional reward-based fine-tuning struggles with balancing these objectives, especially in high-dimensional spaces. Fixed-weight penalty methods are sensitive to hyperparameter choices and often require extensive trial-and-error. Moreover, they cannot guarantee that the generated samples will meet the constraints, which is critical in scientific contexts like drug design, where violations can render candidates unusable. The problem becomes more complex when constraints are hard (must be satisfied) rather than soft (preferably satisfied). Addressing this requires a framework that can adaptively and theoretically guarantee the satisfaction of constraints while maximizing rewards.

This work introduces CFO, an algorithm that leverages the augmented Lagrangian method within the flow model fine-tuning process. Its key innovations include: 1) transforming the constrained optimization into a sequence of unconstrained subproblems with adaptive penalties; 2) dynamically updating dual variables—penalty and Lagrange multipliers—based on constraint violations; 3) integrating these updates into existing flow model fine-tuning algorithms like Adjoint Matching, ensuring scalability and efficiency. The approach guarantees convergence to feasible solutions and, under ideal conditions, to global optima. It bridges classical optimization theory with modern deep generative modeling, enabling automatic, reliable balancing of reward and constraints.

• �� Define the constrained optimization problem: maximize reward r(x) subject to constraint c(x) ≤ B, with KL regularization to preserve prior knowledge.
• �� Introduce dual variables (penalty ρ and multiplier λ) to embed constraints into the objective, forming an augmented Lagrangian.
• �� At each iteration, construct an augmented objective fk(x) combining reward and penalty terms, with λ and ρ updated based on the current constraint violation Gk.
• �� Use a fine-tuning solver (e.g., Adjoint Matching) to optimize the flow model parameters with fk, ensuring reward enhancement and constraint adherence.
• �� Update dual variables: decrease λ when constraints are satisfied, increase penalty ρ when violations persist, with adaptive rules to ensure convergence.
• �� Theoretically, under bounded approximation errors, the sequence of models converges to a feasible, near-optimal solution, as proven by the authors.

Experiments include synthetic 2D examples with known distributions and molecular design tasks using datasets like ZINC. Metrics involve reward improvement, constraint violation rate, and diversity. Baselines include reward-only fine-tuning and inference-time constrained generation. Hyperparameters such as initial penalty and iteration count are tuned for stability. Ablation studies analyze the impact of adaptive penalty updates. Results show CFO achieves 15-20% reward gains with violation rates below 5%, outperforming baselines. Visualizations confirm the model’s ability to generate samples within constraints while optimizing properties. The experiments demonstrate scalability from simple to complex, high-dimensional problems.

CFO consistently improves reward metrics by approximately 20% over baseline methods, with constraint violations reduced to below 5%. In molecular tasks, it maintains high validity and synthesizability rates, significantly outperforming reward-only approaches. The adaptive penalty mechanism is crucial, as ablation studies show fixed penalties lead to either violations or suboptimal rewards. Theoretical analysis aligns with empirical findings, confirming convergence and feasibility under realistic assumptions. These results validate CFO’s effectiveness for real-world scientific optimization.

CFO can be directly applied to drug discovery, protein engineering, and materials design, where strict constraints are essential. It enables automated, goal-oriented generation of candidates that meet domain-specific criteria, accelerating research workflows. The framework is adaptable to various property functions and constraints, making it versatile for multiple scientific fields. Long-term, CFO could underpin AI-driven design pipelines, enabling rapid, reliable exploration of vast design spaces with guaranteed constraint satisfaction, transforming how scientific innovation is conducted.

The method relies on the quality of approximate solvers; poor solver accuracy can lead to violations or suboptimal solutions. High-dimensional, complex problems may require many iterations, increasing computational cost. The current validation is limited to molecular design; broader applications need further testing. Additionally, the approach assumes the ability to evaluate constraints and rewards efficiently, which may not hold in all scenarios. Future work should focus on improving solver robustness, reducing computational overhead, and extending to multi-objective and dynamic constraints.