Rome was Not Built in a Single Step: Hierarchical Prompting for LLM-based Chip Design
Hierarchical prompting with ROME boosts complex HDL generation success from 0 to 78%+ in LLMs, reducing time by 30%.
Key Findings
Methodology
This paper introduces a hierarchical prompting framework combined with a multi-stage feedback loop called ROME, enabling fully automated HDL generation for complex hardware modules. The approach involves extracting submodule hierarchies from natural language descriptions or user input, then sequentially generating Verilog code for each submodule with integrated simulation and error correction. The process iterates until submodules pass verification tests, after which they are integrated into top-level modules. The prompts are structured in multiple layers, guiding the LLM step-by-step, with relay prompts to manage output length constraints. Experiments compare open-source models (Llama, Code Llama-Verilog) and proprietary models (GPT-4), demonstrating significant improvements in success rates and efficiency, especially on complex benchmarks like AES, UART, and barrel shifters.
Key Results
- Hierarchical prompting elevates pass@1 success rates from near zero to over 78% for complex modules such as AES and UART, with average generation times reduced by over 30%. For example, Llama 2 13B, which fails on complex tasks with flat prompts, achieves a success rate of 0.78 with hierarchy. The method enables smaller open-source models to compete with large proprietary models like GPT-4, making HDL generation more accessible and cost-effective.
- Generated designs for challenging modules exhibit correct functionality in simulation, validating the effectiveness of the hierarchical approach. The success rates and time savings highlight the method’s robustness across different hardware structures.
- This work significantly lowers the barrier for automated chip design, allowing resource-constrained models to produce high-quality hardware modules, thus democratizing AI-assisted hardware development and accelerating innovation in chip manufacturing.
Significance
This research addresses fundamental challenges in automating HDL generation for complex hardware, overcoming issues of output length, structural complexity, and verification. By mimicking human hierarchical design processes, the proposed framework enhances success rates and reduces costs, pushing the boundary of what small and medium-sized LLMs can achieve in hardware synthesis. It offers a scalable, automated pathway to chip design, promising to transform industry practices and enable rapid prototyping of sophisticated hardware architectures. The approach also opens avenues for integrating AI into existing EDA workflows, fostering a new era of intelligent chip development.
Technical Contribution
The core innovation lies in the hierarchical prompting architecture combined with a multi-stage feedback loop (ROME), which systematically decomposes complex modules, generates subcomponents, and iteratively corrects errors through simulation. This contrasts with prior flat prompting methods that struggle with long outputs and structural complexity. The framework supports automatic submodule extraction, hierarchical code generation, and seamless integration, providing a scalable solution for complex hardware design. It also demonstrates that resource-limited open-source models can perform at industry-relevant levels when guided hierarchically, broadening the applicability of LLMs in hardware engineering.
Novelty
This is the first comprehensive application of hierarchical prompting combined with iterative error correction for fully automated HDL synthesis of complex hardware modules. Unlike previous works limited to simple modules or manual intervention, this approach systematically decomposes design tasks, enabling small models to produce high-quality, verified hardware descriptions. The integration of relay prompts and multi-stage feedback distinguishes this work, establishing a new paradigm for AI-driven chip design that bridges the gap between research and industrial deployment.
Limitations
- Despite significant improvements, the method still faces challenges with extremely novel or highly intricate modules where submodule extraction or error correction may fail due to limited model understanding or training data gaps.
- The current verification relies on open-source simulation tools, which may not fully capture industrial validation requirements, necessitating further integration with formal verification and physical design tools.
- Computational costs, especially for large hierarchies, remain non-trivial, requiring optimization for practical deployment in industrial environments.
Future Work
Future directions include integrating physical layout and timing optimization into the automated pipeline, expanding multi-modal prompts (e.g., combining schematic images with text), and improving submodule extraction algorithms. Additionally, scaling the framework to handle larger, more diverse hardware architectures and refining verification methods for industrial-grade validation will be key. Collaborations with EDA tool vendors could facilitate seamless integration, enabling end-to-end AI-assisted chip design workflows that are both efficient and reliable.
AI Executive Summary
Deep Dive
Plain Language Accessible to non-experts
想象你在一家工厂里，负责组装一辆复杂的汽车。单纯按照说明书一步步拼装，可能会很慢，还容易出错。于是，你决定把汽车拆成几个部分，比如引擎、车身、轮子，然后逐个组装。每完成一部分，你还会检查是否正确，确保没有出错。最后，把所有部分拼在一起，就变成了一辆完整的汽车。这个过程就像论文里的层次化提示，把复杂的芯片拆成小块，逐步生成和验证。这样做比一次性拼装整个复杂结构更容易，也更可靠。通过这种方式，AI可以像人一样，先拼好每个部分，再组合成完整的芯片，大大提高效率和成功率。
ELI14 Explained like you're 14
你知道拼乐高积木时，如果一次拼出一座超级复杂的城堡，可能会拼错或者忘记细节，对吧？所以聪明的做法是，把城堡拆成几个小部分，比如城墙、门、塔楼，然后一个一个拼好，再把它们组合起来。每拼完一部分，还会检查有没有拼错，有问题就重新调整。这样一步步拆解、拼装、检查，最后才能得到一座完美的城堡。论文里的方法也是一样，把复杂的芯片设计拆成很多小模块，让AI一步步帮你拼好、检查，最后合成完整的芯片。这样比一次性拼整个复杂结构要容易得多，也更靠谱。
Abstract
Large Language Models (LLMs) are effective in computer hardware synthesis via hardware description language (HDL) generation. However, LLM-assisted approaches for HDL generation struggle when handling complex tasks. We introduce a suite of hierarchical prompting techniques which facilitate efficient stepwise design methods, and develop a generalizable automation pipeline for the process. To evaluate these techniques, we present a benchmark set of hardware designs which have solutions with or without architectural hierarchy. Using these benchmarks, we compare various open-source and proprietary LLMs, including our own fine-tuned Code Llama-Verilog model. Our hierarchical methods automatically produce successful designs for complex hardware modules that standard flat prompting methods cannot achieve, allowing smaller open-source LLMs to compete with large proprietary models. Hierarchical prompting reduces HDL generation time and yields savings on LLM costs. Our experiments detail which LLMs are capable of which applications, and how to apply hierarchical methods in various modes. We explore case studies of generating complex cores using automatic scripted hierarchical prompts, including the first-ever LLM-designed processor with no human feedback. Tools for the Recurrent Optimization via Machine Editing (ROME) method can be found at https://github.com/ajn313/ROME-LLM