Awesome GPT Prompt Engineering
Section: Papers
Entry
Appears in 6 awesome lists
Samples diverse reasoning paths and selects their most consistent answer to improve inference-time reliability. (ICLR 2023)
Section: Papers
Section: 2023
Section: Reasoning and Test-Time Compute · majority vote over sampled CoT chains.
Section: Foundations · Multi-path sampling + majority vote: GSM8K 57% → 74%
Section: Self-Verification & Self-Correction — Enabling Foundations · Samples diverse reasoning paths and selects their most consistent answer to improve inference-time reliability. (ICLR 2023)
Section: Recent advances in Prompt engineering
The foundational paper defining the Thought/Action/Observation loop structure that underlies virtually every agent harness. Required reading for understanding why the loop is structured the way it is and where each harness component maps onto the reasoning-acting cycle.
(AIAYN) - Introducing multi-head self-attention neural networks with positional encoding to do sentence-level NLP without any RNN nor CNN - this paper is a must-read (also see this explanation and this visualization of the paper).
foundational result; intermediate reasoning steps improve performance.
"Let's think step by step" — zero-shot CoT milestone
search over reasoning trees.
Uses written principles and model-generated critiques to scale supervision while retaining explicit behavioral constraints. (arXiv 2022)
by Tom B. Brown (OpenAI) et al. - "We train GPT-3, an autoregressive language model with 175 billion parameters :scream:, 10x more than any previous non-sparse language model, and test its performance in the few-shot setting."
(2025) - small policy model paired with a process preference model trained via MCTS rollouts; enables small LMs to bootstrap reasoning without distilling from larger models.