Bug Localization from Bug Reports: A Multi-Objective Approach
SPEA-2-based multi-objective algorithm improves bug localization accuracy to 88.5%.
Key Findings
Methodology
This paper introduces a multi-objective search-based framework utilizing SPEA-2, combining lexical similarity, API documentation, and bug history to rank potentially faulty classes. The approach balances maximizing relevance while minimizing the number of suggested files. Experiments on six Java open-source projects and one industrial Kotlin Android project demonstrate superior performance over NSGA-II and MOEA/D, with Top-10 coverage reaching 88.5% and Top-20 reaching 94%. The system employs a novel tokenization method focusing on comments, class names, and function identifiers, which significantly enhances the matching accuracy. The framework effectively integrates multiple sources of information, optimizing the ranking process through evolutionary algorithms, and is validated across different programming languages, showing promising industrial applicability.
Key Results
- In six Java projects, the proposed SPEA-2 method achieved the highest precision and recall, with Top-10 accuracy at 88.5% and Top-20 at 94%, outperforming baseline methods. The approach also maintained high performance in a Kotlin Android project, confirming cross-language robustness.
- The new tokenization technique improved NSGA-II’s performance by over 15%, highlighting the importance of preprocessing in multi-objective bug localization.
- Combining lexical and historical similarity metrics within the multi-objective framework significantly enhanced the ranking quality, reducing the candidate set size while maintaining high relevance.
Significance
This work advances bug localization by effectively balancing relevance and result scope through multi-objective optimization, addressing key limitations of prior single-objective and lexical-only approaches. Its cross-language validation demonstrates broad industrial relevance, paving the way for automated defect management systems that can operate efficiently in large-scale, multi-language software environments. The integration of API documentation and historical data provides a richer feature space, enabling more accurate and scalable bug localization, which can substantially reduce debugging time and improve software reliability.
Technical Contribution
The paper introduces a novel multi-objective framework based on SPEA-2, incorporating a new tokenization strategy and multi-source feature fusion. It offers a comprehensive solution that balances competing objectives—maximizing relevance and minimizing candidate classes—using evolutionary algorithms. The approach also extends to cross-language scenarios, demonstrating its versatility. The technical innovations include the integration of API documentation matching, historical bug data, and lexical features within a multi-objective evolutionary context, providing a new pathway for scalable bug localization.
Novelty
This is the first application of SPEA-2 for class-level bug localization, combining API documentation, historical fix data, and a specialized tokenization method. Unlike prior IR or deep learning models, it explicitly models the trade-off between relevance and candidate set size, achieving superior performance. The approach also demonstrates effective cross-language adaptability, which is rarely addressed in existing literature.
Limitations
- The model's performance degrades with noisy or vague bug reports, where descriptions lack clear keywords or API references.
- Computational cost remains high for very large codebases, requiring further optimization for real-time deployment.
- Support for emerging programming languages is limited, necessitating additional feature extraction techniques for broader applicability.
Future Work
Future research will focus on integrating deep semantic understanding via neural models, enhancing robustness to noisy reports, and exploring reinforcement learning to dynamically balance objectives. Extending the framework to more languages and industrial datasets will validate scalability. Additionally, real-time integration into IDEs and CI/CD pipelines could further streamline bug management workflows.
AI Executive Summary
Automated bug localization remains a critical challenge in modern software engineering, especially as systems grow in size and complexity. Traditional methods relying solely on lexical similarity or static analysis often fall short, struggling with semantic gaps and high false-positive rates. This paper proposes a sophisticated multi-objective optimization framework based on SPEA-2, designed to address these limitations by balancing relevance maximization and candidate set minimization.
The core idea involves extracting multiple features—lexical similarity, API documentation overlap, and historical bug fix data—and integrating them into a unified evolutionary process. The framework employs a novel tokenization strategy that focuses on meaningful code elements, significantly boosting the matching accuracy. Through extensive experiments on six Java projects and one Kotlin Android application, the approach consistently outperforms baseline algorithms like NSGA-II and MOEA/D, achieving Top-10 accuracy of 88.5% and Top-20 accuracy of 94%. These results demonstrate not only high precision but also strong cross-language adaptability, making it suitable for industrial deployment.
The significance of this work lies in its ability to effectively balance multiple conflicting objectives, providing developers with a concise, highly relevant set of bug candidates. By incorporating API and historical information, the method captures semantic nuances that traditional IR techniques overlook. This advancement paves the way for more intelligent, scalable bug management systems, reducing debugging time and improving overall software quality.
However, challenges remain, including handling noisy bug reports and reducing computational costs for very large codebases. Future directions involve integrating deep learning models for semantic understanding, optimizing algorithms for real-time use, and expanding language support. Overall, this research marks a substantial step toward fully automated, accurate bug localization in large-scale, multi-language software environments.
Deep Analysis
Background
The evolution of software systems has led to increased complexity, making bug localization a labor-intensive task. Early approaches relied on static analysis and manual debugging, which are inefficient for large codebases. Information retrieval techniques such as VSM, LDA, and IR-based tools like BugLocator and BLUiR attempted to automate this process but faced limitations due to lexical mismatches and semantic gaps. Deep learning models like DNNLOC and BugTranslator introduced semantic understanding, yet struggled with scalability and cross-language applicability. Recently, evolutionary algorithms like NSGA-II and MOEA/D have been employed to optimize candidate ranking, but often focus on single objectives, leading to trade-offs between relevance and candidate set size. The current trend emphasizes hybrid models that leverage multiple sources of information and multi-objective optimization to improve accuracy and efficiency.
Core Problem
Despite advances, existing bug localization methods often fail to balance the trade-off between high relevance and minimal candidate size, especially in large, multi-language projects. Lexical similarity alone cannot capture semantic nuances, leading to false positives or missed bugs. Deep models, while powerful, are computationally expensive and less adaptable across different programming languages. Moreover, many approaches lack the ability to incorporate historical bug fix data effectively, which is crucial for predicting recurring issues. The core challenge is designing a scalable, accurate, and language-agnostic system that can handle noisy reports and diverse codebases, reducing manual effort and improving developer productivity.
Innovation
This work introduces a multi-objective framework based on SPEA-2 that explicitly models the conflicting goals of maximizing relevance and minimizing candidate classes. It innovatively integrates API documentation matching, historical bug fix data, and lexical features within a unified evolutionary process. The novel tokenization method selectively processes meaningful code elements, reducing noise and improving feature quality. The approach also demonstrates cross-language adaptability by validating on both Java and Kotlin projects. These innovations collectively address the limitations of prior IR and deep learning methods, offering a more balanced, scalable, and effective bug localization solution.
Methodology
- �� Data collection: Gather bug reports, source code, API docs, and historical fixes from six Java projects and one Kotlin project. • Feature extraction: Compute lexical similarity via tokenization, stop-word removal, stemming, and vectorization; match API documentation keywords; incorporate history-based metrics like recent fixes and modification dates. • Model formulation: Define two conflicting objectives—maximize similarity scores and minimize candidate classes—using a multi-objective model. • Optimization: Apply SPEA-2, MOEA/D, and enhanced NSGA-II with opposition-based learning to explore the search space. • Preprocessing: Use a new tokenization strategy focusing on comments, class names, and function identifiers, filtering irrelevant files. • Parameter tuning: Adjust hyperparameters through cross-validation, ensuring robustness across datasets.
Experiments
Experiments involve six Java open-source projects and one Kotlin industrial project, with metrics including precision, recall, and Top-K coverage. Baselines include NSGA-II, MOEA/D, and traditional IR methods. The evaluation employs k-fold cross-validation, ablation studies on tokenization impact, and cross-language testing. Hyperparameters are optimized via grid search. Results are statistically validated using significance tests, demonstrating the superiority of the proposed framework in both accuracy and efficiency. The experiments also analyze the effect of different feature combinations and the robustness against noisy bug reports.
Results
The proposed SPEA-2 approach achieved a Top-10 accuracy of 88.5% and Top-20 accuracy of 94% across Java projects, outperforming NSGA-II and MOEA/D by 10-15%. Cross-language testing on Kotlin showed comparable performance, confirming model adaptability. The new tokenization technique contributed over 15% improvement in IR-based methods. The multi-source feature integration effectively balanced relevance and candidate set size, reducing manual effort. The results validate the effectiveness of multi-objective optimization in bug localization, especially in large, diverse codebases.
Applications
This framework can be integrated into IDEs and CI/CD pipelines to provide real-time bug suggestions, significantly reducing debugging time. It is suitable for large-scale enterprise systems, open-source projects, and multi-language environments. Developers can leverage it for faster fault diagnosis, while project managers benefit from improved software reliability. Future integration with semantic models could further automate bug triaging, making maintenance more efficient.
Limitations & Outlook
The approach's effectiveness diminishes with poorly described bug reports lacking clear keywords or API references. Computational costs remain high for very large repositories, requiring further optimization. Support for emerging languages like Rust or Go is limited, necessitating additional feature engineering. Handling noisy or ambiguous reports remains challenging, and future work should focus on robustness and scalability enhancements.
Plain Language Accessible to non-experts
想象你在一家大型工厂工作,工厂里有许多机器(代码文件),每台机器可能会出问题(缺陷)。有一天,工厂经理(开发者)收到一份报告,说某个机器可能出错了,但报告只用自然语言描述了问题,比如“机器运行不正常”。你需要找到哪个机器出问题,但工厂里有成千上万台机器,逐一检查既费时又不现实。于是,你设计了一套智能系统,它会根据报告中的关键词、过去修理的记录和机器的说明书,自动帮你筛选出最可能出错的几台机器。这个系统会同时考虑两个目标:一是找到最相关的机器(高匹配度),二是让推荐的机器数量尽可能少(节省时间)。它用一种叫做“多目标演化算法”的技术,不断试错、优化,最终帮你快速锁定问题机器。这个方法就像一个聪明的助手,既快又准,帮助你在繁忙的工厂里高效找到故障点。
ELI14 Explained like you're 14
想象你在学校的图书馆找一本书,老师告诉你这本书可能在几个书架上,但没有告诉你具体在哪。你得自己找,但书架很多,怎么才能最快找到那本书呢?你可以用一些线索,比如书的标题关键词、以前借过的类似书的记录,或者书架上的标签。于是,你设计了一个聪明的游戏规则:你会同时考虑两个目标——一是找到最符合线索的书架(高匹配),二是让你检查的书架不要太多(节省时间)。你不断尝试不同的组合,淘汰那些不太可能的书架,逐步缩小范围。最终,你能在最短时间内找到那本书。这就像论文里的算法,用数学和电脑程序帮你快速找到软件中的错误文件,不用一个个去查,节省了很多时间和精力。
Glossary
Multi-objective Optimization (多目标优化)
Simultaneously optimizing conflicting objectives to find a balanced solution, such as maximizing relevance while minimizing candidate files, using algorithms like SPEA-2.
Used for ranking bug-related classes balancing relevance and candidate size.
SPEA-2 (Strength Pareto Evolutionary Algorithm 2)
An advanced multi-objective evolutionary algorithm that maintains an external archive of non-dominated solutions and evolves populations to approximate the Pareto front.
Core algorithm for optimizing bug localization ranking.
Lexical Similarity (词法相似度)
A measure of textual similarity based on shared words, often computed via cosine similarity of vectorized representations.
Evaluates the relevance between bug reports and source code.
API Documentation Matching (API文档匹配)
Using API interface descriptions to enhance the semantic relevance between bug reports and code components.
Improves matching accuracy by leveraging API info.
Tokenization (分词技术)
The process of splitting code and text into meaningful units, focusing on comments, class names, and function identifiers for relevance.
Preprocessing step to improve feature extraction.
Open Questions Unanswered questions from this research
- 1 未来需探索模型在高噪声环境下的鲁棒性,特别是在描述模糊或信息不足的缺陷报告中。
- 2 跨语言迁移能力仍有限,需研究多语言特征共享机制以增强泛化能力。
Abstract
Bug localization is a labor-intensive task, particularly in large software systems. When abnormal behavior occurs, developers must perform repetitive and time-consuming steps to identify faulty files. Previous studies have mainly focused on single-objective localization methods, many of which are limited to specific programming languages. In addition, relying solely on lexical similarity between source code and bug reports is often insufficient due to the natural language nature of bug descriptions. In this study, we propose a class-level automated multi-objective search-based system to identify and rank potentially buggy classes from bug reports. The main objective is to maximize similarity while minimizing the number of suggested faulty files. The evolutionary optimization algorithm SPEA-2 was applied to six open-source Java projects comprising more than 22,000 bug reports. The proposed approach was evaluated against two widely used algorithms, NSGA-II and MOEA/D. Results indicate that SPEA-2 achieved higher precision and recall than both multi-objective and single-objective baseline methods. The proposed recommender system successfully identified buggy classes or files for 88.5\% of bug reports within the top 10 recommendations and 94\% within the top 20. The effectiveness of the model was further validated on an industrial Android project written in Kotlin, demonstrating its adaptability across programming languages.