Classification-Based Automatic HDL Code Generation Using LLMs
Proposed a classification-based framework with multi-round search, boosting HDL code correctness by over 14% in Pass@10 on VerilogEval.
Key Findings
Methodology
This work combines large language models (LLMs) with human-inspired workflows. It first classifies circuit types (combinational or sequential) from specifications, then extracts explicit information lists. Using structured formats like truth tables and state-transition tables, the framework guides LLMs to generate HDL code in manageable sub-tasks. Multi-round search optimizes the selection of promising information, reducing hallucinations. The approach leverages EDA tools for logic simplification, avoiding extensive fine-tuning or large databases. Experiments on VerilogEval show significant improvements in code correctness metrics.
Key Results
- On the VerilogEval-human dataset, Pass@1 increased to 47.0%, a 4.7% improvement over baseline; Pass@5 reached 69.6%, an 11.0% increase; Pass@10 achieved 76.9%, up by 14.7%. In the VerilogEval-machine dataset, improvements exceeded 5% in Pass@5 and Pass@10. The multi-step classification and information structuring effectively mitigated hallucinations, leading to higher functional accuracy.
- By classifying circuit types and extracting detailed info, the model reduces reasoning complexity. Logic expressions are simplified via PyEDA, and structured data guides HDL code generation. Multiple code samples are generated, tested, and the best are selected, significantly boosting correctness. The framework's robustness is validated across different circuit complexities and datasets.
- Results demonstrate that the method achieves high accuracy without model fine-tuning or large external datasets, making it scalable and practical for industrial applications. It effectively addresses the hallucination problem, paving the way for reliable AI-assisted hardware design.
Significance
This research addresses a critical bottleneck in AI-driven hardware design—hallucination in HDL code generation. By integrating task classification, structured information extraction, and multi-round search, it enhances the reasoning and accuracy of LLMs. This approach reduces dependency on extensive fine-tuning and large databases, making AI tools more accessible for industry use. It opens new avenues for automating complex chip design workflows, reducing costs, and accelerating development cycles. The methodology's adaptability to various circuit types and design specifications signifies a major step toward practical, reliable AI-assisted hardware automation.
Technical Contribution
The paper introduces a novel hierarchical framework that decomposes HDL generation into manageable sub-tasks based on circuit classification. It combines multi-round search strategies with structured information extraction, logic simplification via PyEDA, and external validation, creating a robust pipeline that mitigates hallucinations. Unlike prior fine-tuning or retrieval-based methods, this approach leverages in-context reasoning and task decomposition, providing theoretical guarantees on correctness improvements and practical scalability. It demonstrates that training-free, task-specific workflows can significantly outperform naive generation.
Novelty
This is the first comprehensive integration of circuit classification, structured information extraction, and multi-round search to mitigate hallucinations in HDL code generation by LLMs. Unlike existing approaches relying on large datasets or fine-tuning, this method emphasizes task decomposition and external validation, offering a new paradigm for training-free, reliable hardware code synthesis. Its combination of structured data formats, logic simplification, and iterative refinement distinguishes it from prior work, representing a significant innovation in AI-assisted hardware design.
Limitations
- The approach may still face challenges with extremely complex or poorly specified circuits, where information extraction becomes unreliable. In such cases, hallucinations may persist.
- Multi-round search increases computational overhead, limiting real-time applicability in large-scale industrial environments. Further optimization is needed.
- Current validation relies on simulation-based testbenches; real-world hardware verification may reveal additional issues. Extending to physical prototyping remains future work.
Future Work
Future directions include integrating knowledge graphs and formal reasoning engines to enhance logical inference, reducing hallucinations further. Developing end-to-end automated pipelines from specifications to verified hardware prototypes is also envisioned. Additionally, expanding the framework to support multi-modal inputs, such as schematic diagrams or physical constraints, could broaden its applicability. Improving computational efficiency and robustness in diverse design scenarios will be key to industrial adoption.
AI Executive Summary
The rapid evolution of AI-driven design tools has opened new horizons for automating hardware description language (HDL) code generation. Yet, a persistent challenge remains: large language models (LLMs) often hallucinate, producing incorrect or inconsistent HDL code that fails to meet specifications. This problem hampers their deployment in industrial chip design, where correctness and reliability are paramount.
Addressing this, the present work introduces a novel classification-based framework that mimics human design workflows. The process begins with classifying the circuit type—either combinational or sequential—based on specifications. This classification reduces the reasoning complexity for the LLMs, enabling more accurate sub-task execution. Next, explicit information lists are extracted, including truth tables and state transitions, which are then formatted for external tools like PyEDA and EDA software. These structured representations serve as guides for the LLMs to generate HDL code in manageable steps.
A key innovation lies in multi-round search strategies, where multiple code samples are generated, tested, and the most promising ones are selected for further refinement. This iterative process, combined with task-specific procedures (COMB, SEQU, BEHAV), significantly reduces hallucinations and improves functional correctness. Experimental results on the VerilogEval dataset demonstrate that the proposed method boosts Pass@1 accuracy by 4.7%, Pass@5 by 11%, and Pass@10 by 14.7% over baseline GPT-4 generation, with similar gains in the machine-generated subset.
This approach's significance extends beyond performance metrics. It offers a scalable, training-free solution that leverages task decomposition, structured data, and external validation, making AI-assisted HDL design more reliable and accessible. The methodology paves the way for fully automated chip design workflows, reducing costs and development times. Future work aims to incorporate formal reasoning and knowledge graphs, further enhancing robustness and industrial applicability, ultimately transforming hardware design into a more intelligent, efficient process.
Deep Dive
Abstract
While large language models (LLMs) have demonstrated the ability to generate hardware description language (HDL) code for digital circuits, they still suffer from the hallucination problem, which leads to the generation of incorrect HDL code or misunderstanding of specifications. In this work, we introduce a human-expert-inspired method to mitigate the hallucination of LLMs and improve the performance in HDL code generation. We first let LLMs classify the type of the circuit based on the specifications. Then, according to the type of the circuit, we split the tasks into several sub-procedures, including information extraction and human-like design flow using Electronic Design Automation (EDA) tools. Besides, we also use a search method to mitigate the variation in code generation. Experimental results show that our method can significantly improve the functional correctness of the generated Verilog and reduce the hallucination of LLMs.