A Taxonomy of Inefficiencies in LLM-Generated Python Code

Recent advances in LLMs like GPT-4, Codex, and Llama have significantly improved automated code generation, enabling rapid prototyping and assisting developers. Prior work primarily focused on correctness, security, and bug detection, with models like DeepMind's AlphaCode and OpenAI's Codex showing notable success. However, as models scale, inefficiencies such as redundant computations, poor algorithm choices, and maintenance challenges become prominent. Existing studies lack a systematic taxonomy to categorize these issues comprehensively, limiting targeted improvements. The gap between high-level performance and practical efficiency necessitates detailed analysis to guide future model training and tooling.

Despite high functional accuracy, LLM-generated code often suffers from inefficiencies that degrade runtime performance, increase resource consumption, and complicate maintenance. These issues include poorly optimized algorithms, excessive redundancy, and unclear code structure, which collectively hinder real-world deployment. The absence of a unified framework to identify and address these inefficiencies hampers progress. Practitioners face difficulty in diagnosing and fixing these problems, leading to manual, time-consuming efforts. Addressing this gap requires a systematic, empirical approach to classify and validate efficiency issues, providing actionable insights for model developers and users.

This work introduces a comprehensive taxonomy of inefficiency patterns in LLM-generated Python code, covering five main categories and 19 subcategories. Unlike prior bug-focused studies, it emphasizes efficiency aspects like performance bottlenecks, code readability, and maintainability. The methodology combines automated judgment from GPT-4o-mini with manual open coding, ensuring robustness and depth. Validation through expert surveys confirms the taxonomy's practical relevance. The approach bridges the gap between theoretical model capabilities and real-world deployment needs, offering a structured framework for future research and tooling development.

• �� Generate code snippets from HumanEval+ using CodeLlama, DeepSeek-Coder, and CodeGemma with fixed prompts and temperature=0 for determinism.
• �� Use GPT-4o-mini as an automated judge, inputting code and task descriptions to identify inefficiencies.
• �� Manually analyze a pilot set (196 samples) with open coding, iteratively defining categories and subcategories.
• �� Expand to full dataset (492 samples), with two raters independently labeling, resolving discrepancies via discussion.
• �� Validate taxonomy through a survey with 58 practitioners, collecting feedback on relevance and prevalence.
• �� Refine categories based on feedback, achieving high inter-rater agreement (Cohen’s Kappa > 0.7).

The experimental setup involves generating code for 164 tasks across three models, with fixed prompts to ensure comparability. GPT-4o-mini evaluates each snippet for inefficiency patterns, supplemented by manual analysis to refine categories. The survey assesses practitioners’ perceptions of these inefficiencies, measuring frequency and importance via Likert scales. Data analysis includes frequency distribution, co-occurrence patterns, and correlation with model types. The robustness is tested through inter-rater agreement and cross-model comparisons, providing comprehensive insights into prevalent inefficiency patterns.

Analysis shows that logic errors (35%) and performance issues (28%) are most common, often co-occurring with readability and maintainability problems. Multiple inefficiencies appear in over one-third of samples, indicating complex interactions. Experts validated the taxonomy's comprehensiveness and relevance, confirming that these inefficiencies reflect real-world challenges. The study highlights specific issues like unnecessary steps, sub-optimal algorithms, and poor variable naming, which can be targeted for future model improvements. The results guide both model developers and tool creators towards more efficient code generation.

The taxonomy can be integrated into automated code review tools, enabling real-time detection of inefficiencies. It also informs training objectives for future models to prioritize efficiency alongside correctness. Industry practitioners can use it to diagnose and improve LLM outputs, reducing manual effort. Long-term, the framework supports developing self-healing systems that automatically optimize generated code, accelerating the adoption of AI-assisted software engineering and reducing resource costs.

The study relies on open-source models and specific datasets, which may not fully capture commercial or domain-specific scenarios. The judgment model (GPT-4o-mini) might introduce biases or miss subtle issues. The survey sample, though expert, is limited in scope, possibly affecting generalizability. Future work should include broader datasets, diverse model architectures, and automated detection tools to enhance robustness and applicability across different contexts.