Automatic Chain of Thought Prompting in Large Language Models
Auto-CoT automatically generates diverse reasoning demonstrations, leveraging clustering and prompting to outperform manual methods in reasoning tasks.
Key Findings
Methodology
Auto-CoT employs question clustering via Sentence-BERT embeddings and k-means to ensure diverse problem sets. It then selects representative questions from each cluster and uses a 'let’s think step by step' prompt with GPT-3 to generate reasoning chains automatically. This process reduces manual effort, emphasizes diversity to mitigate misleading demonstrations, and constructs high-quality, varied exemplars for reasoning tasks. Experiments across ten benchmarks show that Auto-CoT matches or surpasses manual demonstration methods, especially in scenarios lacking annotated training data.
Key Results
- Auto-CoT achieves an average accuracy increase of 3.5% across ten reasoning datasets, reaching 48.0% on GSM8K, outperforming manual demonstrations. The method's robustness stems from diversity sampling, which reduces the propagation of errors caused by misleading similar questions. Ablation studies confirm that diversity and simplicity of demonstrations are critical for performance gains.
- Compared to similarity-based retrieval (Retrieval-Q-CoT), Auto-CoT maintains superior stability and accuracy, particularly in unannotated data settings, demonstrating its adaptability and effectiveness without relying on labeled exemplars.
- Ablation results indicate that selecting questions based on cluster diversity and limiting demonstration length improves reasoning accuracy, highlighting the importance of diverse, concise exemplars in chain-of-thought prompting.
Significance
This work addresses a key bottleneck in deploying large language models for complex reasoning: manual demonstration construction. By automating and diversifying the generation of reasoning exemplars, Auto-CoT reduces human effort, enhances model robustness, and broadens applicability in real-world tasks like automated tutoring, question answering, and reasoning-based AI systems. It paves the way for scalable, self-supervised reasoning capabilities, advancing AI towards more autonomous and intelligent systems.
Technical Contribution
The paper introduces a novel combination of question clustering via sentence embeddings and diversity-aware sampling to automatically generate reasoning demonstrations. The core innovation lies in integrating zero-shot prompting with clustering to produce varied, representative exemplars, effectively reducing misleading influence from similar questions. This approach significantly improves the robustness and scalability of chain-of-thought prompting, providing a new paradigm for automatic demonstration construction in large language models.
Novelty
This is the first systematic framework that combines clustering and diversity sampling to automatically generate high-quality reasoning demonstrations for LLMs. Unlike prior methods relying solely on manual design or similarity retrieval, Auto-CoT emphasizes diversity to mitigate error propagation, representing a fundamental shift in how automatic reasoning exemplars are constructed, especially in low-resource or unlabeled scenarios.
Limitations
- Auto-CoT's effectiveness depends on the quality of question embeddings and clustering accuracy; poor embeddings can lead to suboptimal diversity and demonstration quality.
- In highly ambiguous or complex tasks, automatic generation may still produce misleading or low-quality reasoning chains, requiring further refinement.
- Computational costs for clustering large datasets and generating multiple demonstrations may limit scalability without optimization.
Future Work
Future directions include integrating reinforcement learning to refine demonstration selection, extending the framework to multi-modal data (images, videos), and developing more efficient clustering algorithms. Additionally, exploring adaptive sampling strategies and incorporating feedback from downstream tasks could further enhance demonstration quality and reasoning robustness.
AI Executive Summary
Recent advances in large language models (LLMs) have demonstrated remarkable abilities in complex reasoning tasks, primarily through techniques like chain-of-thought (CoT) prompting. Traditional approaches rely heavily on manually crafted demonstrations, which are labor-intensive and task-specific, limiting scalability. Zero-shot CoT prompts, such as 'Let's think step by step,' enable models to generate intermediate reasoning without manual examples but often suffer from errors and lack of diversity, reducing overall performance.
This paper introduces Auto-CoT, an innovative framework that automates the construction of reasoning demonstrations. The core idea involves clustering questions based on semantic similarity using Sentence-BERT embeddings, ensuring diverse problem sets. From each cluster, representative questions are selected, and GPT-3 is prompted with a simple 'let’s think step by step' cue to generate reasoning chains automatically. This process reduces manual effort, enhances diversity, and mitigates the risk of misleading demonstrations propagating errors.
Experimental results across ten public reasoning benchmarks—including arithmetic, commonsense, and symbolic tasks—show that Auto-CoT consistently matches or exceeds the performance of manual demonstration methods. Notably, in scenarios lacking annotated training data, Auto-CoT demonstrates superior robustness and stability, with an average accuracy improvement of 3.5%. The ablation studies confirm that diversity and simplicity of demonstrations are key factors driving these gains.
The significance of this work lies in its potential to democratize and scale reasoning capabilities in LLMs. By automating demonstration generation, Auto-CoT reduces human labor, enhances model robustness, and broadens applicability in real-world AI tasks such as automated tutoring, question answering, and reasoning systems. Looking ahead, integrating reinforcement learning, multi-modal data, and more efficient clustering algorithms could further advance this promising approach, pushing AI toward more autonomous and intelligent reasoning systems.
Deep Dive
Abstract
Large language models (LLMs) can perform complex reasoning by generating intermediate reasoning steps. Providing these steps for prompting demonstrations is called chain-of-thought (CoT) prompting. CoT prompting has two major paradigms. One leverages a simple prompt like "Let's think step by step" to facilitate step-by-step thinking before answering a question. The other uses a few manual demonstrations one by one, each composed of a question and a reasoning chain that leads to an answer. The superior performance of the second paradigm hinges on the hand-crafting of task-specific demonstrations one by one. We show that such manual efforts may be eliminated by leveraging LLMs with the "Let's think step by step" prompt to generate reasoning chains for demonstrations one by one, i.e., let's think not just step by step, but also one by one. However, these generated chains often come with mistakes. To mitigate the effect of such mistakes, we find that diversity matters for automatically constructing demonstrations. We propose an automatic CoT prompting method: Auto-CoT. It samples questions with diversity and generates reasoning chains to construct demonstrations. On ten public benchmark reasoning tasks with GPT-3, Auto-CoT consistently matches or exceeds the performance of the CoT paradigm that requires manual designs of demonstrations. Code is available at https://github.com/amazon-research/auto-cot