How Can We Know What Language Models Know?

The rapid development of pre-trained language models (PLMs) such as BERT, RoBERTa, and GPT has significantly advanced NLP tasks. Early efforts focused on using these models for text generation and classification, but recent work emphasizes probing their internal knowledge. The LAMA benchmark and related datasets have been instrumental in quantifying factual knowledge stored within models. Prior methods relied on manually designed prompts, which are limited by human intuition and scope. Automated prompt generation techniques, including relation extraction and paraphrasing, have emerged to address these limitations, enabling more comprehensive and scalable knowledge assessment. However, challenges remain in ensuring prompt diversity, reducing noise, and generalizing across relations and languages.

Despite progress, current probing methods depend heavily on manual prompts, which are often suboptimal and limited in coverage. This results in underestimating the true knowledge contained in models. The core challenge is to develop automated, scalable techniques that generate diverse, high-quality prompts capable of eliciting a broader range of factual information. Additionally, integrating multiple prompts effectively remains an open problem, as different prompts may perform variably depending on the entity pair or relation. Addressing these issues is crucial for accurately assessing and leveraging the knowledge embedded in large-scale language models.

The paper introduces two key innovations: 1) a relation mining-based prompt generation method that extracts relation-specific prompts from Wikipedia sentences using middle words and dependency paths, capturing diverse linguistic expressions; 2) a paraphrasing approach employing back-translation to create multiple semantically similar prompts, increasing lexical diversity. These prompts are then combined via an ensemble framework, including a learned weighting scheme that optimizes prompt contributions based on training data. This integrated approach surpasses manual prompts in accuracy, providing a scalable, data-driven solution for probing model knowledge across multiple relations and models.

• �� Relation mining: Collect sentences from Wikipedia containing subject-object pairs for a specific relation; extract middle words or dependency paths as prompts.
• �� Paraphrasing: Use back-translation (English-German-English) to generate multiple prompt variants; rank by round-trip probability, select top candidates.
• �� Prompt evaluation: Measure each prompt’s accuracy on a training set of subject-object pairs; select high-performing prompts.
• �� Ensemble strategies: Average log probabilities from top prompts or learn optimal weights via gradient-based optimization.
• �� Model probing: Input prompts into models like BERT, compute token probabilities for missing objects, and evaluate correctness.
• �� Ablation: Analyze effects of prompt types, diversity, and ensemble methods on accuracy.
• �� Dataset: Use Wikidata-derived subject-object pairs, evaluate on T-REx subset of LAMA, with hyperparameters T=40 prompts, B=7 back-translation candidates.

Experiments involve probing BERT-base, BERT-large, ERNIE, and KnowBERT on the T-REx subset of LAMA, covering 41 relations. The evaluation metrics include micro- and macro-accuracy, with training data for prompt selection and ensemble weight optimization. Baselines include manual prompts and majority-class predictions. The study compares single prompt, multiple prompt averaging, and learned weighted ensemble approaches. Hyperparameters T=40 prompts, B=7 back-translation candidates, optimized via Adam. Ablation studies assess the impact of prompt types, diversity, and ensemble strategies. Results demonstrate that automated prompts significantly outperform manual prompts, with ensemble methods further boosting accuracy.

Automated prompt generation improves BERT-base accuracy from 31.1% to 34.1%, and ensemble methods reach 39.6%. BERT-large improves from 32.3% to 43.7%. For relations like religion, prompt optimization boosts accuracy by over 60%. The learned ensemble weights reveal that diverse prompts contribute complementarily. Ablation confirms that relation-specific prompts and ensemble strategies are critical. The approach generalizes well across models and relation types, with consistent improvements over manual prompts and simple averaging.

This methodology facilitates automatic knowledge probing, aiding in constructing comprehensive knowledge graphs, enhancing question-answering systems, and verifying factual information. It reduces reliance on manual prompt design, enabling scalable, domain-agnostic knowledge extraction. The techniques are applicable in industry scenarios requiring factual consistency, such as virtual assistants, information retrieval, and AI explainability. Future integration with multi-modal data could further expand its utility, supporting richer knowledge bases and more robust AI systems.

The approach depends on large textual corpora and syntactic parsing, which can be computationally intensive. Noisy prompts from mining may introduce errors, requiring filtering. Its effectiveness across diverse languages, complex relations, and multi-modal data remains to be validated. Scalability to real-time applications is limited by resource demands. Future work should focus on improving prompt filtering, reducing costs, and extending to broader domains.