FeTaQA: Free-form Table Question Answering

The evolution of question answering (QA) systems has transitioned from simple fact extraction to complex reasoning over structured and unstructured data. Early datasets like WikiTableQuestions and Spider focused on SQL-based retrieval, while recent models such as TAPAS and T5 have improved understanding of tables. However, these systems mainly produce short, factual answers, limiting their ability to generate detailed explanations. Generative QA datasets like NarrativeQA and ELI5 introduced long-form answers but lacked structured data grounding. FeTaQA bridges this gap by emphasizing multi-fact reasoning and natural language generation over Wikipedia tables, fostering advances in deep reasoning and expressive answers.

Current table QA models predominantly perform extractive tasks, which are insufficient for questions requiring multiple facts, relations, or abstract reasoning. The core challenge lies in retrieving relevant, diverse facts from tables, understanding their interrelations, and synthesizing coherent, informative responses. Existing datasets lack the complexity needed to train models for such tasks, resulting in limited reasoning capabilities. Moreover, generating long, faithful answers that accurately reflect the data remains difficult, especially when facts are discontinuous or involve high-level relations.

This work introduces several innovations: 1) a large-scale dataset with 10,330 Wikipedia tables annotated with human-generated, free-text answers emphasizing multi-fact reasoning; 2) a hybrid modeling framework combining semantic parsing (TAPAS) for fact retrieval and sequence generation (T5) for natural language synthesis; 3) comprehensive evaluation metrics integrating automatic scores and human judgments to assess fluency, correctness, and faithfulness. These innovations enable models to handle more complex reasoning tasks, bridging structured data understanding with expressive language generation, a significant step beyond prior short-answer datasets.

• �� Data collection: Migrate from ToTTo, select tables with multiple highlighted regions, and annotate questions and answers via human judges.
• �� Model architecture: Develop a pipeline combining TAPAS for logical form generation and T5 for text realization, fine-tuned on FeTaQA.
• �� Training: Use weak supervision with annotated denotations, fine-tune models on the dataset, and validate with human evaluations.
• �� End-to-end approach: Linearize tables and questions into sequences, then train T5 models to directly generate answers.
• �� Evaluation: Use BLEU, ROUGE, BERTScore, and human ratings to compare model outputs against references.
• �� Ablation: Test impact of retrieval accuracy, reasoning depth, and data diversity on performance.

The dataset is split into training, validation, and testing sets, with careful similarity-based partitioning to challenge generalization. Models are trained with hyperparameters tuned for optimal performance, including batch sizes, learning rates, and epochs. Automatic metrics evaluate n-gram overlap and semantic similarity, while human judges rate fluency, correctness, and faithfulness. Ablation studies analyze the contribution of retrieval accuracy and reasoning complexity. Results show the end-to-end T5-large surpasses pipeline models in all metrics, but both still lag behind human performance, especially in multi-fact reasoning and coherence.

The T5-large end-to-end model achieves BLEU 30.54 and ROUGE-L 0.53, with human correctness at 54.8%. Fine-tuning improves pipeline BLEU by nearly 2 points, yet retrieved facts remain fewer than human annotations, indicating room for improvement in retrieval and reasoning modules. Human evaluation confirms the superiority of end-to-end models in fluency and faithfulness, but challenges in multi-fact reasoning persist. These findings demonstrate the feasibility of complex generative QA over tables, while highlighting the need for better retrieval, reasoning, and data coverage.

FeTaQA enables advanced question answering in domains requiring detailed explanations, such as scientific research, financial analysis, and educational tools. It supports intelligent virtual assistants capable of deep reasoning over structured data, automated report writing, and knowledge management systems. The dataset's emphasis on multi-fact inference and natural language synthesis makes it suitable for developing human-like AI systems that can handle complex queries and generate comprehensive responses, transforming how structured information is accessed and utilized.

Despite progress, models still struggle with retrieving multiple relevant facts and understanding complex relations, often underperforming compared to human annotations. The dataset size, while large, limits domain coverage, and scaling to broader contexts remains challenging. Computational costs are high, especially for large models like T5-large, and long-text generation can suffer from semantic drift. Future work must focus on improving retrieval accuracy, reasoning depth, and efficiency, while expanding dataset diversity across languages and domains.