ESG-Bench: Benchmarking Long-Context ESG Reports for Hallucination Mitigation

In recent years, as corporate responsibility increasingly incorporates environmental, social, and governance (ESG) criteria, ESG reporting has become a legal requirement in many regions and a key channel for documenting sustainability practices and assessing firms' long-term and ethical performance. ESG provides a framework for assessing how companies manage sustainability-related risks across environmental, social, and governance pillars. Once largely voluntary, ESG disclosure has become a legal requirement in many regions, most notably through EU regulations such as the Corporate Sustainability Reporting Directive and the Sustainable Finance Disclosure Regulation. This shift reflects growing expectations for transparency in corporate impacts on society and the environment. ESG reporting therefore plays a critical role in enabling compliance and supporting stakeholders' evaluation of long-term performance. Corporations now publish extensive ESG reports for investors, regulators, and the public. However, the usefulness of these disclosures depends on their credibility and comparability. Third-party ESG rating agencies such as Sustainalytics and MSCI have been widely criticized for methodological opacity and inconsistency, with studies showing that their scores often diverge substantially even for the same company due to differences in indicator selection, weighting schemes, and data sources. These controversies undermine stakeholder trust and highlight that ESG assessments are far from standardized. Combined with the growing length and complexity of sustainability reports, this inconsistency increases the need for scalable, transparent tools that can support reliable and evidence-grounded interpretation.

The complexity and diversity of ESG reports pose significant challenges for reliable LLM deployment: Companies may engage in greenwashing to appear more sustainable, misleading investors and stakeholders about their true ESG impact. ESG reports require deep contextual understanding, industry-specific knowledge, and familiarity with regulatory frameworks, barriers that LLMs may struggle with due to their reliance on general knowledge. ESG reports involve a mix of text, tables, and graphics. Retrieving and analyzing these documents often span hundreds of pages. LLMs remain limited in efficient document parsing, robust memory recall, and cross-sectional understanding in lengthy reports. LLMs struggle with these demands due to limitations in document parsing, retrieval, and cross-sectional understanding, and also because they rely heavily on parametric knowledge that may conflict with the factual content of ESG reports. This misalignment frequently leads to hallucinations, answers that are not grounded in the source document. We classify hallucinations into two types: (1) where the model introduces unsupported information, and (2) omissive hallucinations, where the model fails to answer despite relevant evidence.

ESG-Bench frames ESG report analysis as a QA task with verifiability constraints, enabling systematic evaluation of LLMs' ability to extract and reason over ESG content and providing a new use case: mitigating hallucinations in socially sensitive, compliance-critical settings. We design task-specific Chain-of-Thought (CoT) prompting strategies and fine-tune multiple state-of-the-art LLMs on ESG-Bench using CoT-annotated rationales. Our experiments show that these CoT-based methods substantially outperform standard prompting and direct fine-tuning in reducing hallucinations, and that the gains transfer to existing QA benchmarks beyond the ESG domain.

• �� ESG-Bench dataset construction: Built through a model–then–annotator pipeline, establishing a taxonomy of hallucination types.

• �� Task-specific Chain-of-Thought strategy: Designed task-specific CoT prompting strategies and fine-tuned multiple state-of-the-art LLMs using CoT-annotated reasoning paths.

• �� Systematic evaluation: Evaluated multiple LLMs on ESG-Bench, focusing on hallucination mitigation.

• �� Experimental validation: Compared different fine-tuning strategies, validating the effectiveness of CoT strategies in reducing hallucinations.

The experimental design includes evaluating multiple large language models using the ESG-Bench dataset, focusing on hallucination mitigation. We selected several state-of-the-art LLMs, including Llama-3.2-3B Instruct, Gemma-2-2B-it, and Mistral-7B-Instruct-v0.3. These models are tested on ESG-Bench, HaluEval, and BioASQ benchmarks, assessing their ability to generate responses while identifying hallucinations. The evaluation metrics used include WA (With Answer) and WoA (Without Answer) accuracy to fairly assess the models' ability to generate accurate answers and appropriately abstain when sufficient information is unavailable.

The experimental results show that models using CoT strategies significantly outperform standard prompting and direct fine-tuning in reducing hallucinations, with a reduction rate of over 30%. On ESG-Bench, models using CoT strategies significantly outperform standard prompting and direct fine-tuning in reducing hallucinations, with a reduction rate of over 30%. Experiments demonstrate that CoT strategies are effective not only in the ESG domain but also transferable to other QA benchmarks like HaluEval and BioASQ, showing higher accuracy and reliability. Comparative analysis of different fine-tuning strategies reveals that CoT fine-tuning enhances reasoning consistency and factual accuracy in long-text contexts.

The application scenarios of ESG-Bench include corporate ESG audits and compliance verification, as well as providing a valuable resource for training summarization models on long ESG documents. Annotator-corrected responses enable fine-tuning of ESG-specific QA models for improved factual grounding, while hallucination labels aid in developing mitigation strategies. The dataset also serves as a benchmarking tool for evaluating answer accuracy, retrieval robustness, and format-specific performance.

ESG-Bench currently focuses on English ESG reports, not covering multilingual and cross-cultural ESG report analysis. Due to the complexity and diversity of ESG reports, models may still have limitations when handling reports from specific industries or domains. The current Chain-of-Thought strategy may perform suboptimally in extremely long texts or highly complex reports. Future research directions include expanding ESG-Bench to cover multilingual and cross-cultural ESG reports, developing more general hallucination mitigation strategies, and exploring how to enhance models' reasoning capabilities in extremely long texts.