EXCISE: Query-Side Exclusion for Late-Interaction Retrieval
EXCISE employs query-side modules to solve exclusion inversion in late-interaction retrieval, boosting exclusion success@10 to 0.691 and Boolean NOT accuracy to 0.92.
Key Findings
Methodology
This work analyzes the failure modes of late-interaction models like ColBERT in handling exclusion queries, identifying the core issue as exclusion inversion caused by the additive MaxSim scoring. The authors propose EXCISE, a query-time framework comprising two lightweight modules: a detector that identifies exclusion topics via contrastive learning, and an exclusion adapter that re-embeds the top-k candidate documents. The adapter employs a parameter-free rule to demote candidates matching the excluded topic by measuring their match strength and applying adaptive soft penalties, with a hard cap to prevent over-penalization. Crucially, the index remains frozen; all modifications are confined to the query side. Extensive experiments across six datasets and three backbone models demonstrate that EXCISE significantly outperforms baseline models, achieving a success@10 of 0.691 and Boolean NOT accuracy of 0.92, surpassing both frozen and fine-tuned baselines.
Key Results
- On the ExcluIR dataset, EXCISE raises exclusion success@10 from 0.058 (frozen baseline) to 0.691, and Boolean NOT accuracy from 0.25-0.29 to 0.90-0.92. It consistently outperforms all baselines across 18 model-dataset combinations, including fine-tuned cross-encoders, with no harm to ordinary retrieval metrics like nDCG@10.
- In a pooled evaluation of 1,860 queries, EXCISE surpasses all fine-tuned models, demonstrating its ability to improve exclusion without sacrificing overall relevance ranking. The system’s design ensures robustness even in out-of-domain zero-shot settings, maintaining high performance across diverse datasets.
- Ablation studies confirm the importance of each component: the detector’s high accuracy in identifying exclusion queries, the re-embedding step’s role in distinguishing confusable documents, and the adaptive demotion rule’s effectiveness in maintaining ranking stability. The system’s efficiency, with minimal overhead, makes it practical for real-world deployment.
Significance
This research addresses a fundamental challenge in deep retrieval systems—accurately handling exclusion queries without costly index re-encoding or performance degradation. By confining learning to the query side and introducing a dynamic, lightweight correction mechanism, EXCISE offers a scalable, efficient solution that can be integrated into existing retrieval pipelines. Its ability to dramatically improve exclusion success rates while preserving relevance ranking paves the way for more controllable, explainable retrieval systems, especially critical in sensitive domains like legal discovery, medical information retrieval, and regulatory compliance. The framework’s generality and strong empirical results suggest it will influence future research on retrieval fairness, robustness, and user-controlled search.
Technical Contribution
The core technical contribution lies in the novel identification and mitigation of exclusion inversion within late-interaction models. The authors formalize the problem as an identity of MaxSim scoring, proving that no readout of frozen vectors can recover the exclusion constraint. They then design a query-side detection mechanism based on contrastive learning, coupled with a re-embedding module that learns to distinguish confusable documents. The demotion rule, which measures the match strength to the excluded topic and applies an adaptive soft penalty, ensures stable ranking adjustments. The addition of a hard cap on candidate removal guarantees safety against misclassification. This approach enables effective exclusion correction without re-encoding the index, offering a new paradigm for controllable retrieval.
Novelty
This work is the first to explicitly analyze and address the exclusion inversion phenomenon in late-interaction models, providing a theoretical foundation that explains why existing readouts fail. The innovative use of a lightweight, query-side detection and re-embedding modules, combined with a parameter-free, adaptive demotion rule, distinguishes it from prior methods relying on costly fine-tuning or index modifications. The approach achieves significant improvements in exclusion success while maintaining relevance performance, representing a substantial advancement in controllable deep retrieval systems.
Limitations
- The system's effectiveness depends heavily on the detection accuracy of the exclusion detector; misclassification can lead to either missed exclusions or unnecessary demotions, affecting overall performance.
- Candidate re-embedding is limited to the top-k shortlist; if the target document is not initially retrieved, the system cannot recover it, potentially reducing recall in long-tail or highly diverse corpora.
- In very noisy or extremely long documents, the Evidence scoring and adaptive demotion may become less stable, necessitating further robustness enhancements for deployment in complex real-world scenarios.
Future Work
Future research could focus on improving the robustness of the exclusion detector, possibly through multi-modal cues or richer contextual understanding. Extending the framework to handle multiple simultaneous exclusions and more complex logical combinations remains an open challenge. Additionally, integrating index-level adaptivity, such as dynamic restructuring or partial re-encoding, could further enhance performance and scalability. Exploring applications in multilingual and multi-domain environments, as well as real-time interactive systems, will be essential to realize the full potential of this approach in practical, large-scale retrieval systems.
AI Executive Summary
In the realm of information retrieval, the ability to accurately exclude certain topics from search results is crucial, especially in sensitive domains like legal, medical, and financial applications. Traditional late-interaction models such as ColBERT excel at relevance ranking but falter when handling exclusion queries, suffering from a phenomenon known as exclusion inversion. This occurs because the additive MaxSim scoring mechanism inadvertently promotes documents containing the excluded topics, leading to significant retrieval errors.
Addressing this challenge, Mohammed Ali and colleagues introduce EXCISE, a novel query-side framework designed to correct exclusion inversion dynamically at query time. Unlike conventional methods that rely on re-encoding the entire corpus or fine-tuning the model—both costly and inflexible—EXCISE operates solely on the query side, leaving the index untouched. It employs a lightweight exclusion detector trained via contrastive learning to identify whether a query contains an exclusion intent. When an exclusion is detected, a re-embedding module re-processes the top candidate documents, which are then evaluated for their match strength to the excluded topic.
The core innovation lies in the soft demotion rule: a parameter-free, adaptive mechanism that penalizes candidates based on their similarity to the excluded topic. This rule ensures that documents matching the exclusion are ranked lower, effectively reversing the exclusion inversion problem. To prevent over-penalization, a hard cap limits the number of candidates removed, preserving the overall ranking stability. The entire system is designed for efficiency, adding minimal latency overhead (~12.7 ms per query) and no index modifications.
Extensive experiments across six datasets—including ExcluIR, FiQA, TREC-COVID, ESGenius, EDGAR, and EUR-Lex—and three backbone models demonstrate the system’s robustness and generality. Results show a dramatic increase in exclusion success rate, with success@10 reaching 0.691, and Boolean negation accuracy soaring to 0.92. Notably, EXCISE outperforms all fine-tuned models in zero-shot settings, maintaining relevance metrics such as nDCG@10, and offers a practical, scalable solution for real-world retrieval systems.
Furthermore, the authors introduce X-BENCH, a comprehensive benchmark suite that evaluates exclusion performance across explicit, implicit, and compound scenarios, with controls for no-harm and Boolean logic. This benchmark facilitates future research and development in controllable retrieval. Looking ahead, potential improvements include enhancing detection robustness, supporting multiple simultaneous exclusions, and integrating index-level adaptivity. Overall, EXCISE marks a significant step toward more precise, controllable, and efficient deep retrieval systems, with broad implications for industry and academia alike.
Deep Dive
Plain Language Accessible to non-experts
想象你在图书馆里找书，你想找关于“电动汽车”的资料，但又不想看到关于“特斯拉”的内容。以前的方法就像你用眼睛扫一遍所有书名，然后把那些写着“特斯拉”的书放到一边，但有时候那些书名会让你误以为它们是你要找的内容，或者你会漏掉一些真正关于电动汽车的书。现在，EXCISE就像有个聪明的助手，他能听懂你说的话，知道你不想看“特斯拉”的内容，然后帮你把那些书自动放到一边，让你只看到你真正感兴趣的资料。这种助手不用重新整理所有书，只是在你问的时候，临时帮你筛选，既快又准。它的秘密在于：它能理解你说的“不要特斯拉”，并且知道哪些书是关于“特斯拉”的，然后在你浏览的时候，把那些书的排名降低，确保你不会被它们迷惑。这样一来，你就能更轻松、更准确地找到你需要的资料，而不用担心被不相关的内容干扰。
ELI14 Explained like you're 14
想象你在学校图书馆找资料，你想找到关于“足球”的书，但又不想看到关于“梅西”的内容。以前的方法就像你随便翻书，把你不想看的“梅西”相关的书也翻到前面，让你误以为它们是你要找的。现在，有个聪明的机器人助手，它能听懂你说：‘我想找足球，但不要梅西的资料’，然后它会帮你把那些关于梅西的书放到后面，或者降低它们的排名。这样，你就能更快找到真正关于足球的书，而不用担心被那些你不想看的内容迷惑。这个机器人助手不用重新整理所有书，只是在你问的时候，临时帮你筛选，把不想看的内容排到后面。它的秘密在于：它能理解你说的“不要梅西”，知道哪些内容是你不想要的，然后在你浏览时，把那些内容的排名降低，让你更容易找到想要的资料。这样一来，你的搜索变得更聪明、更贴心，也更省事。
Abstract
Late-interaction retrievers handle exclusion queries poorly. When a user asks for X but not Z, the additive MaxSim score promotes documents covering Z, a problem we call exclusion inversion. We show that no readout of the frozen vectors recovers the constraint, because the difficulty lies in identifying the excluded topic, which depends on the query alone. EXCISE operates at query time and corrects the inversion while leaving the index frozen. Two query-side modules totalling 1.5M parameters identify the topic and re-embed a 100-document shortlist, and a parameter-free rule demotes candidates matching that topic. Across six collections and three backbones, EXCISE is the strongest system in all eighteen backbone-collection cells against that backbone's own frozen and fine-tuned baselines. It raises exclusion success@10 on ExcluIR from 0.058 to 0.691 and raises Boolean NOT accuracy from 0.25-0.29 to 0.90-0.92. Pooled over 1,860 queries, it outperforms every fine-tuned cross-encoder, each of which loses no-harm nDCG@10, whereas EXCISE matches its frozen baseline on its strongest backbone. We release X-BENCH, a tiered benchmark of explicit, implicit, and compound exclusions with no-harm and Boolean controls.
References (20)
EDGAR-CORPUS: Billions of Tokens Make The World Go Round
Lefteris Loukas, Manos Fergadiotis, Ion Androutsopoulos et al.
WWW'18 Open Challenge: Financial Opinion Mining and Question Answering
Macedo Maia, S. Handschuh, A. Freitas et al.
Towards General Text Embeddings with Multi-stage Contrastive Learning
Zehan Li, Xin Zhang, Yanzhao Zhang et al.
C-Pack: Packed Resources For General Chinese Embeddings
Shitao Xiao, Zheng Liu, Peitian Zhang et al.
Fact or Fiction: Verifying Scientific Claims
David Wadden, Kyle Lo, Lucy Lu Wang et al.
Relevance feedback in information retrieval
J. Rocchio
Promptriever: Instruction-Trained Retrievers Can Be Prompted Like Language Models
Orion Weller, Benjamin Van Durme, Dawn J. Lawrie et al.
MultiEURLEX - A multi-lingual and multi-label legal document classification dataset for zero-shot cross-lingual transfer
Ilias Chalkidis, Manos Fergadiotis, Ion Androutsopoulos
Retrieval of the Best Counterargument without Prior Topic Knowledge
Henning Wachsmuth, S. Syed, Benno Stein
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Jacob Devlin, Ming-Wei Chang, Kenton Lee et al.
Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval
Lee Xiong, Chenyan Xiong, Ye Li et al.
INSTRUCTIR: A Benchmark for Instruction Following of Information Retrieval Models
Hanseok Oh, Hyunji Lee, Seonghyeon Ye et al.
Dense Passage Retrieval for Open-Domain Question Answering
Vladimir Karpukhin, Barlas Oğuz, Sewon Min et al.
What BERT Is Not: Lessons from a New Suite of Psycholinguistic Diagnostics for Language Models
Allyson Ettinger
Reproducing NevIR: Negation in Neural Information Retrieval
Coen van den Elsen, Francien Barkhof, Thijmen Nijdam et al.
Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference
Benjamin Warner, Antoine Chaffin, Benjamin Clavié et al.
Task-aware Retrieval with Instructions
Akari Asai, Timo Schick, Patrick Lewis et al.
PLAID: An Efficient Engine for Late Interaction Retrieval
Keshav Santhanam, O. Khattab, Christopher Potts et al.
Document Ranking with a Pretrained Sequence-to-Sequence Model
Rodrigo Nogueira, Zhiying Jiang, Ronak Pradeep et al.