Curating Same-Family Neural Networks for LLM-Guided Model Improvement: A Controlled Case Study
Proposes TuneNNGen, leveraging source models and LLMs to boost CIFAR-10 accuracy from 23.98% to 50.49%.
Key Findings
Methodology
This study uses a curated neural network database containing evaluated models, training recipes, and performance metrics. The core algorithm, TuneNNGen, extends NN-GPT by adding a source-guided path, where an LLM generates candidate models based on a target and a selected source model. Experiments on CIFAR-10 and SVHN compare target-only versus source-guided generation under equal budgets, assessing accuracy, reliability, and consistency. Source selection strategies include choosing the highest accuracy model or the structurally most similar one. The approach involves deterministic compatibility checks, candidate training, and evaluation, with multiple seeds and epochs to verify performance gains.
Key Results
- On CIFAR-10, accuracy improved from 23.98% to 50.49%, and on SVHN from 22.54% to 78.80%, outperforming target-only methods. Multi-epoch validation showed gains of 40.94, 18.83, and 7.27 points across datasets. Structural similarity-based source models yielded better transfer effects than highest accuracy models, emphasizing the importance of architecture alignment.
- Different LLMs (DeepSeek-Coder-6.7B, Qwen-2.5B, OlympicCoder-7B) exhibited varying transfer performance, with structurally similar sources consistently providing superior guidance. Source-guided generation increased candidate success rates and top accuracy, confirming the effectiveness of the transfer strategy.
- Direct copying and negative control analyses demonstrated that transfer gains depend on source-target compatibility and LLM adaptation, not just stored source accuracy. The approach was validated across multiple datasets and architecture families, showing broad applicability.
Significance
This work advances AutoML by systematically utilizing curated model repositories combined with LLMs for efficient model improvement. It addresses the challenge of selecting effective transfer sources, emphasizing structural similarity over performance metrics alone. The methodology reduces computational costs and accelerates model tuning, with potential applications in automated neural architecture design, transfer learning, and model compression. It also provides theoretical insights into the importance of architecture alignment in transfer success, influencing future research directions in model automation.
Technical Contribution
The paper introduces a novel transfer strategy combining model structure similarity with LLM-based candidate generation, extending the NN-GPT framework. It proposes multiple source selection rules and a comprehensive evaluation pipeline, demonstrating significant performance improvements under limited budgets. The approach bridges the gap between curated experiment repositories and automated model synthesis, offering a systematic way to leverage prior knowledge for neural network optimization.
Novelty
This is the first systematic study integrating curated neural network repositories with LLM-guided candidate generation, emphasizing structural similarity as a key transfer metric. Unlike traditional hyperparameter tuning or architecture search, it leverages existing models as guidance sources, validated across multiple datasets and architectures. The method introduces a new paradigm for model transfer, combining data-driven and structure-aware strategies, filling a critical gap in AutoML research.
Limitations
- The effectiveness heavily depends on the quality and diversity of the stored models; poor source models limit transfer gains. Structural similarity metrics may not generalize well to complex or novel architectures. The current experiments focus on image classification, requiring validation in other domains. Computational costs for candidate generation and evaluation remain high, especially with large models. Further automation in source selection and adaptation strategies is needed to improve scalability.
Future Work
Future directions include developing automated, learned source selection mechanisms, integrating meta-learning for better transfer prediction, and extending the framework to other tasks like NLP and object detection. Combining multiple sources and exploring multi-task transfer could further enhance performance. Additionally, optimizing computational efficiency and incorporating model compression techniques will facilitate real-world deployment. The integration of reinforcement learning to adapt transfer strategies dynamically is also a promising avenue.
AI Executive Summary
In the rapidly evolving field of deep learning, automating model design and optimization remains a central challenge. Traditional methods like neural architecture search (NAS) and hyperparameter tuning, while effective, often demand enormous computational resources. Recently, leveraging curated repositories of evaluated models combined with large language models (LLMs) has emerged as a promising approach to accelerate this process. This paper introduces TuneNNGen, a novel framework that incorporates source models from a curated database to guide the generation of improved neural network candidates under strict resource constraints.
The core idea is to select a high-quality, structurally similar source model and feed its information into an LLM, which then proposes candidate architectures or training recipes for a target, low-performing model. This approach contrasts with traditional target-only generation, which relies solely on the target model's information. The experiments, conducted on CIFAR-10 and SVHN datasets, demonstrate that this source-guided method significantly boosts accuracy — from 23.98% to 50.49% on CIFAR-10 and from 22.54% to 78.80% on SVHN — outperforming target-only baselines.
A key insight from the study is that structural similarity between source and target models is a better predictor of transfer success than raw performance metrics. Multiple LLMs were tested, revealing consistent advantages when using structurally aligned sources. The analysis also confirmed that transfer gains depend on source-target compatibility and LLM adaptation, not merely on stored source accuracy. These findings suggest a paradigm shift in AutoML, emphasizing architecture-aware transfer strategies.
Beyond performance metrics, the research underscores the importance of model structure in transfer learning, providing a systematic framework for selecting sources based on structural similarity. The methodology’s scalability and effectiveness across different datasets and architectures highlight its potential for broad application, including automated neural architecture design, transfer learning, and model compression.
Despite its success, the approach faces limitations such as reliance on the quality of stored models and computational costs. Future work aims to automate source selection further, extend to other domains, and optimize efficiency. Overall, this study paves the way for smarter, resource-efficient model optimization, leveraging the synergy of curated experiment repositories and advanced language models.
Deep Dive
Abstract
Neural-network repositories contain executable models, recipes, input transformations, and measured accuracies. We study whether one same-family experiment can be curated as prompt guidance for LLM-based improvement of a low-performing target under equal generation and evaluation budgets. TuneNNGen extends NNGPT with a source-guided route and compares it with target-only generation on one CIFAR-10 target, two fixed source-selection rules, three code LLMs, and an additional SVHN target. Under the historical one-epoch search protocol, best-of-budget accuracy on the available evaluation split rises from 23.98% to 50.49% on CIFAR-10 and from 22.54% to 78.80% on SVHN. Selected five-epoch, three-seed means on train-derived validation splits retain gains of 40.94 points on CIFAR-10, 18.83 on Imagenette, and 7.27 on CIFAR-100. Direct-copy and negative-control analyses show that gains depend on source-target compatibility and LLM adaptation; stored source accuracy alone does not predict transferability.