Enhancing Diversity of LLM-Generated Educational Tasks

Recent advances in large language models (LLMs) like GPT-4 and CodeX have enabled automatic generation of educational content, including questions, exercises, and explanations. Prior works such as GPT-3-based question generators and CodeX for programming tasks have shown promising results. However, these models tend to produce repetitive, homogeneous outputs due to their training on large datasets with dominant patterns, leading to the 'artificial hivemind' phenomenon. Researchers have attempted fine-tuning, reinforcement learning, and multi-agent debate frameworks to promote diversity, but these often compromise content quality or require extensive data. The challenge remains to generate diverse yet high-utility educational tasks at scale, which is crucial for personalized learning, curriculum coverage, and engagement.

Despite the capabilities of modern LLMs, their tendency toward content homogeneity hampers the creation of varied educational materials. Existing prompt strategies often lead to similar outputs, limiting the scope of personalized and comprehensive education. The core problem is how to systematically enhance the diversity of generated tasks without sacrificing their relevance, correctness, or difficulty. This is particularly difficult because encouraging diversity can conflict with maintaining utility, and naive approaches may result in irrelevant or low-quality tasks. Developing a structured prompting framework that balances exploration and refinement is essential to overcome these bottlenecks.

The main innovation is the integration of a staged reasoning process—divergent and convergent thinking—within prompt design. First, the model explores a wide range of ideas related to the theme without constraints, promoting creative diversity. Then, it selects and refines the most promising ideas into valid tasks that meet input criteria. This approach mimics human creative processes and encourages broader exploration before focusing on feasibility. Additionally, the paper introduces the 'Effective Diversity' metric, combining automated embeddings-based scores and expert assessments, to evaluate the quality of diverse high-utility tasks. This systematic framework surpasses traditional single-step prompts, providing a scalable solution for generating varied educational content.

• �� Define a two-stage prompting process: first, instruct the model to brainstorm broadly around the theme, generating multiple diverse ideas without constraints. • Use theme and concept inputs to guide the initial exploration, encouraging unusual or surprising ideas. • In the second stage, prompt the model to select one idea from the brainstormed list and refine it into a valid programming task, ensuring it satisfies all input requirements and focuses on the specified concept. • Employ the Qwen-3-235B model with temperature set to 1.0 for generation. • Calculate the Vendi Score based on embeddings from Qwen-Embedding-0.6B to quantify diversity automatically. • Conduct expert evaluations on a subset of tasks, rating utility, novelty, and interest. • Combine automated metrics and expert assessments to compute 'Effective Diversity,' emphasizing high-utility, diverse tasks.

The experimental setup involved generating 50 tasks per context across 20 themes and concepts, comparing CreativeDC with baseline methods: direct prompt (BASE) and a thought-step prompt (COT). The evaluation used the Vendi Score for automated diversity measurement and expert panels for qualitative assessment of task relevance, novelty, and engagement. Model parameters included a temperature of 1.0, with multiple random seeds to ensure robustness. Statistical tests confirmed the significance of diversity improvements. The experiments demonstrated that CreativeDC consistently outperformed baselines in both automated and human evaluations, with a focus on high-utility task diversity and quality.

At K=50, CreativeDC achieved an average Vendi Score of 13, with expert evaluations indicating 16 distinct high-utility task clusters, both significantly higher than baseline methods (~8 clusters). Tasks generated by CreativeDC scored higher in novelty (0.81 vs. 0.50) and interest, with expert ratings confirming better relevance and engagement. The results validate that the staged prompting approach effectively broadens creative exploration while maintaining utility, leading to richer, more diverse educational content. The consistency across themes and concepts suggests strong generalization potential.

This method can be directly applied in automated question banks, personalized tutoring systems, and curriculum development platforms. It enables educators and AI systems to produce a wide array of engaging tasks tailored to different difficulty levels and learner profiles, fostering active learning and curiosity. Long-term, it supports scalable, adaptive education environments where content adapts dynamically to student needs, making learning more interactive and effective.

The current validation is limited to introductory Python tasks, and its effectiveness in more complex or interdisciplinary domains remains untested. Expert evaluations are based on small samples, which may not fully capture real-world variability. The two-stage process is not iterative, potentially limiting refinement; future work could incorporate multi-round feedback. Additionally, computational costs and model biases may affect scalability and fairness, requiring further investigation.