Awesome GPT Prompt Engineering
Section: Papers
Entry
Appears in 4 awesome lists
"Let's think step by step" — zero-shot CoT milestone
Section: Papers
Section: 2022
Section: Reasoning and Test-Time Compute · chain-of-thought for NLP reasoning tasks.
Section: Foundations · "Let's think step by step" — zero-shot CoT milestone
The foundational paper defining the Thought/Action/Observation loop structure that underlies virtually every agent harness. Required reading for understanding why the loop is structured the way it is and where each harness component maps onto the reasoning-acting cycle.
Samples diverse reasoning paths and selects their most consistent answer to improve inference-time reliability. (ICLR 2023)
(AIAYN) - Introducing multi-head self-attention neural networks with positional encoding to do sentence-level NLP without any RNN nor CNN - this paper is a must-read (also see this explanation and this visualization of the paper).
foundational result; intermediate reasoning steps improve performance.
search over reasoning trees.
Uses written principles and model-generated critiques to scale supervision while retaining explicit behavioral constraints. (arXiv 2022)
by Tom B. Brown (OpenAI) et al. - "We train GPT-3, an autoregressive language model with 175 billion parameters :scream:, 10x more than any previous non-sparse language model, and test its performance in the few-shot setting."
(2025) - small policy model paired with a process preference model trained via MCTS rollouts; enables small LMs to bootstrap reasoning without distilling from larger models.