LLM-Based Examination of Eligibility Criteria from Securities Prospectuses at the German Central Bank

The financial industry’s compliance and eligibility verification processes rely heavily on analyzing lengthy prospectuses, which contain critical legal and financial information. Traditional methods employ rule-based systems and manual review, which are slow, costly, and prone to human error. With the advent of NLP, early efforts focused on rule-based extraction and classical machine learning models, but these approaches faced challenges in handling the semi-structured, multilingual, and noisy nature of financial documents. The emergence of pre-trained language models like BERT, FinBERT, and GPT has revolutionized NLP, enabling more flexible and context-aware information extraction. Recent studies have explored domain-specific fine-tuning and prompt-based techniques, but their application to complex, real-world securities documents remains limited. This research builds on these developments, aiming to leverage the zero-shot and instruction-following capabilities of large general-purpose models to address the unique challenges of securities eligibility screening, including bilingual content, OCR artifacts, and complex conditional logic.

The core challenge is to accurately identify and interpret eligibility criteria within lengthy, semi-structured prospectuses that often contain bilingual text, OCR noise, and complex conditional logic. Existing span-based NER models struggle with the variability of expressions, noise, and layout differences, leading to high error rates and limited robustness. Moreover, traditional methods lack flexibility in handling complex logical conditions, such as determining whether a bond's coupon structure or status meets specific legal criteria. Automating this process requires a system that can understand nuanced language, normalize diverse expressions into standard formats, and evaluate conditions semantically, all while maintaining high precision to prevent false approvals that could pose financial risks.

This work introduces several key innovations: 1) A multi-stage generative IE pipeline that employs instruction-tuned LLMs to perform extraction, normalization, and interpretation, overcoming the rigidity of span-based methods; 2) A value-based evaluation framework where LLMs act as semantic judges, assessing the correctness of extracted entities based on evidence and contextual understanding; 3) Robust handling of bilingual, noisy, and complex documents through prompt engineering and multi-modal normalization strategies. These innovations enable the system to operate effectively in real-world, high-noise environments, providing a flexible and scalable solution for securities eligibility screening that surpasses prior rule-based and discriminative models.

• �� PDF preprocessing: Convert prospectus PDFs into Markdown format using Docling, normalizing text to reduce OCR artifacts and layout inconsistencies.
• �� Text extraction: Use LLMs (Llama-3.3-70B-Instruct, Cohere Command-R) with prompts designed to identify entities such as currency, principal amount, and status, outputting structured JSON data.
• �� Normalization: Convert extracted entities into standardized formats (e.g., EUR for currency, fixed amounts for principal) using rule-based and model-guided methods.
• �� Conditional interpretation: In the second inference step, models evaluate whether the extracted entities satisfy eligibility criteria, such as fixed amounts or non-subordination, based on rules and context.
• �� Evidence collection: Extract quotes and context from the source document to support decision-making.
• �� Final decision: Aggregate the interpreted results, master data, and base prospectus annotations to determine overall eligibility, ensuring high precision and low false positive rate.
• �� Evaluation: Use a combination of fuzzy string matching and LLM-as-a-judge to assess extraction accuracy and semantic correctness, with thresholds set at 80% similarity for true positives.

The dataset comprises 413 prospectuses from the FinCorpus-DE corpus, split into training (268) and test sets (145), with detailed annotations for eligibility criteria. The models were evaluated on their ability to classify entire documents as eligible or not, and to accurately extract key entities. Metrics include accuracy, precision, recall, and F1-score, with special emphasis on the semantic evaluation of extracted values. Hyperparameters such as temperature (0.1) and frequency penalty (0.05) were tuned to optimize output stability. The experiments involved testing different models (Llama-3.3-70B-Instruct, Cohere Command-R, Mistral Small) under various noise conditions, including OCR artifacts and bilingual layouts. Ablation studies assessed the impact of multi-stage normalization and value-based judgment, demonstrating significant improvements over baseline span-based methods.

The pipeline achieved a document-level accuracy of up to 91% in eligibility classification, with key entity extraction F1-scores exceeding 0.97 for principal amount and redemption at maturity. The value-based evaluation via LLM-as-a-judge outperformed fuzzy string matching, especially in noisy, bilingual contexts, with similarity scores surpassing 0.80 for most critical fields. The system’s conservative bias minimized false positives, with 90% of predicted eligible securities truly valid, significantly reducing risk exposure. Moreover, the multi-stage approach demonstrated robustness against OCR noise, layout variations, and linguistic differences, validating its applicability in real-world settings. The experiments confirmed that the generative, instruction-guided methodology outperforms traditional span-based models in complex, noisy environments.

This system can be directly integrated into regulatory workflows for automated securities eligibility verification, reducing manual review time and operational costs. Financial institutions can use it for rapid compliance checks, risk assessment, and document processing. Its robustness in bilingual and noisy environments makes it suitable for international markets and multilingual jurisdictions. Long-term, the approach can be expanded to automate other financial document analyses, such as financial statements, credit reports, and legal contracts, fostering a comprehensive AI-driven financial compliance ecosystem. The modular design allows easy adaptation to evolving regulations and document formats, supporting industry-wide digital transformation.

While highly accurate, the current inference speed limits real-time deployment, as each document requires multiple LLM requests, taking tens of seconds. Handling extremely long or complex documents remains challenging, especially with multi-language layouts and OCR artifacts. The reliance on prompt engineering and predefined instructions may limit adaptability to unforeseen scenarios, necessitating ongoing refinement. Additionally, the black-box nature of LLMs poses interpretability challenges, which are critical in regulatory contexts. Future work should focus on optimizing inference efficiency, integrating multi-modal data, and developing explainability tools to enhance transparency and trustworthiness.