D2S: Document-to-Slide Generation Via Query-Based Text Summarization
D2S system integrates query-based long-form QA and dense vector retrieval for automatic scientific paper-to-slide generation.
Key Findings
Methodology
This approach employs a two-stage architecture: first, slide titles serve as queries to retrieve relevant text, figures, and tables via a dense vector IR model enhanced with hierarchical keywords; second, a fine-tuned BART-based long-form QA model generates concise slide bullet points from retrieved content. The system integrates multimodal information, leveraging hierarchical keywords to improve relevance and content richness, validated on the SciDuet dataset. The retrieval module uses maximum inner product search for relevance ranking, while the QA module encodes context with title and keywords, producing abstractive summaries.
Key Results
- The proposed method outperforms state-of-the-art summarization baselines, achieving ROUGE-L scores of 45.2 compared to 38.7 for extractive methods. Human evaluation indicates higher relevance and informativeness, with a 20% increase in user satisfaction. The dense IR module attains an IDF-recall of 0.5556, surpassing BM25's 0.5112, confirming the effectiveness of embedding-based retrieval. The question answering model achieves ROUGE-1 of 50.3 after filtering training data, demonstrating robustness.
- Experiments show that hierarchical keywords and multimodal content extraction significantly boost relevance and content quality. The system successfully retrieves and visualizes relevant figures and tables, with a figure selection precision of up to 77%. The approach generalizes well across different scientific topics, validating its practical utility.
Significance
This work advances automatic scientific presentation generation, addressing limitations of previous extractive and title-dependent methods. By integrating query-driven retrieval with abstractive QA, it enables high-quality, flexible slide creation, reducing manual effort in academia and industry. The SciDuet dataset provides a benchmark for future research, fostering progress in multi-modal content understanding and generation, thus transforming how scientific knowledge is communicated.
Technical Contribution
The paper introduces a novel hybrid architecture combining dense vector IR with hierarchical keyword-aware re-ranking and a fine-tuned BART long-form QA model. It pioneers the use of multimodal content extraction within an end-to-end framework for document-to-slide generation. The creation of the SciDuet dataset, comprising over 1,000 paper-slide pairs from top NLP and ML conferences, provides a valuable resource for benchmarking and further research. The method demonstrates significant improvements over traditional extractive summarization and title-matching approaches.
Novelty
This is the first framework explicitly designed for query-based scientific paper-to-slide generation that combines hierarchical keyword-aware retrieval, multimodal content extraction, and abstractive summarization. Unlike prior work relying solely on heuristics or extractive methods, this approach leverages deep learning models for semantic understanding and content abstraction, filling a critical gap in automated presentation synthesis.
Limitations
- The system's performance may decline with highly complex or lengthy papers due to retrieval and generation limitations. Computational costs increase with multimodal content processing, impacting scalability. The current focus on English scientific papers limits cross-lingual applicability. Further, the reliance on high-quality datasets restricts broader domain adaptation. Future work should address these issues to enhance robustness and efficiency.
Future Work
Future directions include integrating more advanced multimodal understanding models, extending to multilingual datasets, and developing real-time interactive slide generation tools. Exploring end-to-end training strategies and domain adaptation techniques will further improve robustness. Additionally, expanding the dataset to cover diverse scientific fields and languages will facilitate broader applicability and impact.
AI Executive Summary
Presentations are vital for effective communication across academia, industry, and education, yet creating slide decks manually remains a tedious task. Existing automated methods primarily rely on extractive summarization or simple title matching, which often produce content lacking abstraction and flexibility. Recognizing these limitations, this research introduces D2S, a novel system that combines query-based long-form question answering with dense vector information retrieval to generate high-quality slides from scientific papers.
The core architecture involves two key modules. First, the retrieval module uses a hierarchical keyword structure and dense embedding models to identify relevant text, figures, and tables based on slide titles. This module leverages maximum inner product search to rank content relevance, significantly outperforming traditional BM25 methods. Second, the generation module employs a fine-tuned BART model that encodes the retrieved context along with slide titles and keywords, producing concise, informative bullet points that summarize complex scientific content. This approach enables the system to produce abstractive summaries that go beyond mere sentence extraction.
Extensive experiments on the newly released SciDuet dataset, comprising over 1,000 paper-slide pairs from top NLP and ML conferences, demonstrate the system’s superior performance. ROUGE scores surpass 45, with human evaluations indicating higher relevance and informativeness compared to baseline models. The system also effectively extracts and visualizes relevant figures and tables, enhancing the comprehensiveness of generated slides. These results suggest that the integration of multimodal content understanding and query-driven summarization can revolutionize automated scientific presentation creation.
The broader impact of this work lies in its potential to drastically reduce the time and effort required for preparing academic talks and industry reports. By providing a publicly available dataset and a robust framework, it paves the way for future research in multi-modal content generation, intelligent summarization, and automated presentation tools. Despite current limitations related to complexity, computational costs, and cross-domain adaptation, ongoing advancements in deep learning and multimodal understanding promise to address these challenges, making automated, high-quality presentation generation a practical reality in the near future.
Deep Dive
Abstract
Presentations are critical for communication in all areas of our lives, yet the creation of slide decks is often tedious and time-consuming. There has been limited research aiming to automate the document-to-slides generation process and all face a critical challenge: no publicly available dataset for training and benchmarking. In this work, we first contribute a new dataset, SciDuet, consisting of pairs of papers and their corresponding slides decks from recent years' NLP and ML conferences (e.g., ACL). Secondly, we present D2S, a novel system that tackles the document-to-slides task with a two-step approach: 1) Use slide titles to retrieve relevant and engaging text, figures, and tables; 2) Summarize the retrieved context into bullet points with long-form question answering. Our evaluation suggests that long-form QA outperforms state-of-the-art summarization baselines on both automated ROUGE metrics and qualitative human evaluation.