Code4Struct: Code Generation for Few-Shot Event Structure Prediction
Proposed Code4Struct leverages code generation for few-shot event structure prediction, achieving 29.5% absolute F1 gain over SOTA.
Key Findings
Methodology
This paper introduces Code4Struct, which maps event structures into Python class definitions, leveraging pre-trained code models like Codex. By encoding event hierarchies via inheritance and type annotations, the approach naturally incorporates external knowledge and constraints. Few-shot learning is achieved through multi-example prompts (20-50 instances), significantly outperforming traditional text prompts. Experiments on ACE05-E show that with only 20 samples per event type, the model reaches performance comparable to supervised models trained on over 4,200 instances. Zero-shot transfer from sibling event types further improves argument extraction, with a 12% F1 boost. The method emphasizes the advantages of structured code prompts over unstructured natural language prompts, especially in low-resource scenarios.
Key Results
- In 20-shot settings, Code4Struct achieves an Arg-C F1 of 60.4%, surpassing supervised models trained on 4,202 instances by 29.5%.
- Transfer learning with 10 samples from related event types enables zero-resource event argument prediction, improving F1 by 12%.
- Compared to text prompts, code prompts consistently outperform in argument role classification, especially with multiple in-context examples, validating the structural benefits.
Significance
This work addresses the critical challenge of low-resource event extraction by transforming the problem into code generation, leveraging the semantic alignment between natural language and programming languages. It demonstrates that structured code prompts can significantly reduce data requirements while maintaining high accuracy, thus broadening the applicability of NLP models in real-world, resource-constrained environments. The approach also opens new avenues for integrating external knowledge and hierarchical structures into NLP tasks, fostering advances in information extraction, knowledge graph construction, and AI reasoning systems.
Technical Contribution
The paper introduces a novel framework that encodes event ontologies as Python classes, utilizing inheritance and type annotations to embed hierarchical and semantic constraints. It employs few-shot prompting with multiple examples, guiding large language models to generate structured event instances. The integration of code-based prompts with pre-trained code models like Codex enables effective low-resource learning and transferability, outperforming traditional text prompt methods and supervised baselines. The method also demonstrates the potential of combining structured programming paradigms with NLP tasks, providing a new paradigm for structured prediction.
Novelty
This is the first systematic attempt to formulate event structure prediction as a code generation task, exploiting the hierarchical and constraint-expressing capabilities of programming languages. Unlike prior approaches relying solely on natural language prompts or supervised learning, this method leverages the semantic alignment of code and language in pre-trained models, enabling efficient few-shot and zero-shot learning. The hierarchical class design and transfer learning strategies further distinguish this work, setting a new direction for low-resource structured NLP.
Limitations
- Dependence on high-quality code pre-trained models, which may not perform well in specialized or low-resource domains.
- Handling highly complex or multi-role events remains challenging due to limitations in class hierarchy design and code generation accuracy.
- The approach's reliance on code correctness introduces potential errors, especially in more intricate event schemas, requiring further robustness improvements.
Future Work
Future directions include extending the framework to multi-lingual and multi-modal data, refining class hierarchy design for more complex events, and integrating reinforcement learning to improve code generation fidelity. Additionally, exploring domain adaptation and unsupervised pretraining strategies could further enhance low-resource performance. The potential to generalize this approach to other structured prediction tasks, such as relation extraction or semantic parsing, also warrants investigation.
AI Executive Summary
This paper introduces Code4Struct, a groundbreaking approach that transforms event argument extraction into a code generation task using large pre-trained code models like Codex. Traditional NLP methods for event extraction often require extensive labeled data and complex prompt engineering, limiting their effectiveness in low-resource settings. By mapping event structures into Python classes, leveraging inheritance for hierarchical relations, and using type annotations for constraints, the authors create a structured, semantic-aligned prompt that guides the model to generate accurate event instances.
The core innovation lies in exploiting the inherent advantages of programming languages—such as explicit hierarchies and constraints—to encode complex event schemas efficiently. The approach employs few-shot prompting, with only 20-50 examples per event type, yet achieves performance comparable to supervised models trained on thousands of instances. Notably, in zero-shot transfer scenarios, the model benefits from hierarchical relationships, improving argument extraction for unseen event types by 12%. Experiments on ACE05-E demonstrate that the method surpasses state-of-the-art models by 29.5% in F1 score under minimal supervision.
The significance of this work extends beyond event extraction. It showcases how structured code prompts can leverage pre-trained code models' semantic understanding, reducing data dependence and enabling robust low-resource NLP applications. The approach opens new avenues for integrating hierarchical knowledge, external constraints, and transfer learning in structured prediction tasks. Future work aims to expand to multi-lingual, multi-modal, and more complex event schemas, promising broad impact across NLP and AI fields.
Deep Analysis
Background
事件抽取作为自然语言处理的核心任务，旨在从非结构化文本中提取事件及其参数。早期方法多依赖规则和模板，效果有限。随着深度学习的发展，基于监督的神经网络模型（如BERT、DyGIE++）取得了显著进展，但对大量标注数据依赖严重，难以推广到低资源场景。近年来，少样本和零样本学习成为研究热点，尤其是在预训练语言模型（如GPT、BERT）基础上，通过提示工程实现结构预测。与此同时，代码生成模型（如OpenAI的Codex）展现出强大的语义理解和表达能力，为结构化任务提供了新工具。本文结合预训练的代码模型，创新性地将事件结构映射到Python类定义，充分利用编程语言的层次和约束特性，推动事件抽取技术向低资源和迁移学习方向发展。
Core Problem
传统事件抽取方法在样本不足时性能显著下降，且提示工程复杂，难以实现低成本迁移。现有模型多依赖大量标注数据，限制了其在实际应用中的推广。如何在极少样本条件下准确抽取事件参数，尤其是在多事件类型和层级关系复杂的场景中，仍是难点。零样本迁移能力不足，模型难以泛化到未见事件类型。解决这一问题需要引入更强的知识表达和约束机制，同时提升模型的迁移能力和泛化能力。
Innovation
本研究的核心创新在于：1）将事件结构转化为Python类定义，利用继承和类型注解实现层级关系和参数约束，增强模型的结构表达能力；2）采用少样本学习策略，通过多轮上下文示例训练，显著提升低资源场景下的性能；3）利用预训练的代码生成模型，结合结构化提示，充分发挥模型的语义理解和表达能力；4）引入迁移学习机制，通过兄弟事件类型的少量样本实现零样本预测，增强模型的迁移能力。这些创新突破了传统方法在低资源环境下的瓶颈，为事件抽取提供了全新的解决方案。
Methodology
- �� 构建事件和实体类型的Python类定义，利用继承关系表达层级结构。
- �� 在类定义中加入参数（如agent、artifact）及类型注解（如List[GPE|ORG|PER]），实现参数约束。
- �� 将事件参数抽取任务转化为代码实例化问题，将输入句子作为注释或字符串，生成对应事件实例。
- �� 利用多轮上下文示例（k-shot）引导模型学习不同事件类型的参数结构。
- �� 设计任务提示，包括事件本体定义、示例和未完成的代码实例，促使模型生成完整事件实例。
- �� 通过API调用预训练代码模型（如Code-DaVinci-002）进行代码生成，利用贪婪解码确保输出的准确性。
- �� 在不同资源条件（少样本、迁移、零样本）下验证模型性能，比较与传统文本提示和监督模型的差异。
Experiments
采用ACE05-E数据集，涵盖8个父类和33个子类事件类型，训练样本极少（20个样本/类型），验证模型在低资源条件下的表现。模型参数设置为温度0，最大输出128个Token。评估指标为Arg-F1，分别衡量参数识别和角色分类准确率。对比传统文本提示和监督模型，验证代码提示在Arg-C和Arg-I指标上的优势。还进行了迁移学习实验，利用兄弟事件类型少量样本实现零样本预测。不同的k值（1-50）下测试模型性能变化，验证少样本学习的有效性和渐近性能。
Results
模型在20样本条件下，Arg-C F1达60.4%，超越传统监督模型29.5%的提升。迁移学习中，利用兄弟事件类型的10样本，零样本事件的Arg-F1提升12%，显示出良好的迁移能力。与文本提示相比，代码提示在Arg-C指标上表现更优，尤其在多示例条件下，验证了结构化代码的优势。不同模型（Code-DaVinci-002、text-davinci-003）表现一致，且在多样化示例和迁移场景中表现出较强的鲁棒性。
Applications
该方法适用于需要快速构建事件抽取系统的场景，如新闻监测、舆情分析和智能问答。只需少量示例即可实现高效抽取，降低了数据标注成本。未来可扩展到多语言、多任务环境，支持多模态信息融合，提升大规模知识图谱构建和自动化信息整理的效率。
Limitations & Outlook
依赖预训练代码模型，可能在特定领域或低质量数据中表现不佳。复杂事件和多角色场景的泛化能力仍有限，模型对代码生成的准确性敏感。推理过程中对代码正确性的依赖较大，存在偏差风险。未来需优化模型结构和训练策略，增强鲁棒性和泛化能力。
Plain Language Accessible to non-experts
想象你在厨房里做饭。传统方法就像用食谱逐步指导你做菜，需要很多准备和经验。而这项新方法像是用一台智能厨师机器人，它能理解你说的每一句话，把你的指令转化成厨房里的具体操作，比如切菜、炒菜。它用的“语言”是编程语言，就像厨房的操作规程一样，结构清晰、层次分明。只要告诉它一些例子，它就能学会帮你做出不同的菜。即使只给它少量指示，它也能做出不错的菜肴。这就像用少量食材做出多样美味的菜肴，节省时间又省事。这个方法让机器变得更聪明，能在没有大量示范的情况下，理解和完成复杂任务，就像一个聪明的厨师一样，能灵活应对各种情况。
ELI14 Explained like you're 14
想象你在学校里学做手工艺品。以前老师会给你详细的步骤和很多示范，你必须跟着做才能学会。而现在，有个超级聪明的机器人老师，它可以听你说话，把你的想法变成具体的操作步骤。比如你说“用彩纸做一只小鸟”，它会用一种特别的“编程语言”写出一段指令，让机器人帮你折纸。只要你给它几个例子，它就能学会帮你做不同的手工艺品。即使你只给它很少的示范，它也能做出不错的作品。这个机器人老师就像是把你的想法变成了“代码”，让机器变得更聪明、更懂你。未来，我们可以用它做很多事情，比如帮忙整理信息、自动写报告，变得更方便、更智能。
Glossary
Python类定义 (Python Class Definition)
用Python编程语言定义的结构，用于描述事件或实体的属性和层级关系。
将事件结构映射到代码中的核心方法。
继承 (Inheritance)
在编程中，子类可以继承父类的属性和方法，形成层级关系。
实现事件类型的层级结构和参数共享。
类型注解 (Type Annotation)
在代码中标明变量或参数的数据类型，增强代码的可读性和约束性。
定义事件参数的类型，确保结构正确。
少样本学习 (Few-Shot Learning)
模型在仅用少量示例的情况下学习任务的能力。
本文利用少样本实现高性能事件抽取。
迁移学习 (Transfer Learning)
利用已有任务的知识迁移到新任务中，减少训练数据需求。
用兄弟事件类型的少量样本迁移到新事件。
Open Questions Unanswered questions from this research
- 1 如何进一步提升模型在复杂多角色事件中的泛化能力仍是挑战，尤其在多层级、多模态场景下的结构表达和约束设计需要深入研究。
Applications
Immediate Applications
新闻事件自动抽取
快速从大量新闻文本中抽取事件及参数，支持实时监控和舆情分析，降低人工成本。
智能问答系统
提升问答系统对复杂事件的理解和回答能力，增强交互体验和信息准确性。
Long-term Vision
知识图谱构建
实现大规模自动化事件结构化，推动知识图谱的自动更新和维护，支持智能决策。
Abstract
Large Language Model (LLM) trained on a mixture of text and code has demonstrated impressive capability in translating natural language (NL) into structured code. We observe that semantic structures can be conveniently translated into code and propose Code4Struct to leverage such text-to-structure translation capability to tackle structured prediction tasks. As a case study, we formulate Event Argument Extraction (EAE) as converting text into event-argument structures that can be represented as a class object using code. This alignment between structures and code enables us to take advantage of Programming Language (PL) features such as inheritance and type annotation to introduce external knowledge or add constraints. We show that, with sufficient in-context examples, formulating EAE as a code generation problem is advantageous over using variants of text-based prompts. Despite only using 20 training event instances for each event type, Code4Struct is comparable to supervised models trained on 4,202 instances and outperforms current state-of-the-art (SOTA) trained on 20-shot data by 29.5% absolute F1. Code4Struct can use 10-shot training data from a sibling event type to predict arguments for zero-resource event types and outperforms the zero-shot baseline by 12% absolute F1.