ReAct: Reasoning and Acting

ReAct (Reasoning + Acting) is a prompting paradigm for large language models that interleaves chain-of-thought reasoning traces with discrete action calls to external tools, enabling agents to dynamically update their beliefs and behavior in response to environmental feedback. It matters because it closes the loop between thinking and doing — solving the critical limitation of pure chain-of-thought, which can hallucinate facts it cannot verify, and pure action-only agents, which cannot self-correct. The counterintuitive fact: ReAct agents frequently outperform fine-tuned task-specific models on multi-hop question answering despite using only prompting, because interleaved observations act as a grounding signal that prevents compounding reasoning errors.

💡 The Core Insight

A language model left to reason in isolation is like a scientist who must write the entire paper before running any experiment — errors compound silently with no opportunity for correction. ReAct breaks this by treating reasoning and acting as an alternating dialogue: Thought → Action → Observation → Thought → Action → … until a terminal answer is produced. Each observation from the real world (a search result, a database row, a calculator output) anchors the next reasoning step to ground truth, making the agent self-correcting by construction. The "aha moment" is realizing that the LLM's context window itself becomes a working scratchpad that accumulates evidence, turning a frozen snapshot model into a dynamic, evidence-gathering reasoner.

⚠️ Common Misconception

Practitioners often conflate ReAct with simple tool-calling APIs (e.g., OpenAI function calling) and assume the two are interchangeable. The critical difference is that ReAct mandates that reasoning traces are generated in the same token stream as action decisions — the Thought step is not post-hoc commentary but is causally upstream of the Action step. Stripping thoughts out and just emitting action calls degrades into a reactive policy that loses the chain-of-thought grounding; conversely, generating thoughts after actions (as some implementations mistakenly do for logging) reverses the causal order entirely. The correct mental model: Thought is a plan written before the action, not a summary written after it.

How It Works

ReAct operates by prompting the LLM to generate interleaved trajectories of three token types — Thought, Action, and Observation — within a single autoregressive decoding pass.

Formal Trajectory Definition

Let the agent's trajectory be a sequence of triples. At each step tt, the model generates a thought hth_t and action ata_t conditioned on the full history, then receives observation oto_t from the environment:

1
τ=(h1,a1,o1,  h2,a2,o2,  …,  hT,aT,oT)\quad \tau = (h_1, a_1, o_1,\; h_2, a_2, o_2,\; \ldots,\; h_T, a_T, o_T)

Explanation: The full ReAct trajectory is a time-ordered interleaving of thoughts, actions, and observations — the model only ever sees the prefix of this sequence when generating its next token. Here ht∈V∗h_t \in \mathcal{V}^* is a free-text reasoning string, at∈Aa_t \in \mathcal{A} is a structured action (e.g., `Search[entity]`, `Lookup[keyword]`, `Finish[answer]`), and ot∈V∗o_t \in \mathcal{V}^* is the environment's response appended to the context.

Autoregressive Factorization

The LLM parameterized by θ\theta produces the next token sequence by conditioning on the growing context ctc_t:

2
Pθ(ht,at∣ct)=∏i=1∣ht∣+∣at∣Pθ ⁣(xi∣ct,x1:i−1)\quad P_\theta(h_t, a_t \mid c_t) = \prod_{i=1}^{|h_t|+|a_t|} P_\theta\!\left(x_i \mid c_t, x_{1:i-1}\right)

Explanation: Both the reasoning trace and the action are generated token-by-token from the same autoregressive distribution — there is no separate "reasoning head"; thinking and acting share identical model weights. where ct=(q,h1,a1,o1,…,ht−1,at−1,ot−1)c_t = (q, h_1, a_1, o_1, \ldots, h_{t-1}, a_{t-1}, o_{t-1}) concatenates the original query qq with all prior steps.

Action Space Definition

The action set A\mathcal{A} is task-specific but typically includes a Wikipedia-style API:

3
A={ Search[e],  Lookup[k],  Finish[ans] }\quad \mathcal{A} = \{\,\texttt{Search}[e],\; \texttt{Lookup}[k],\; \texttt{Finish}[ans]\,\}

Explanation: The action vocabulary is deliberately small and deterministic so that each call returns a crisp, parseable observation that the LLM can incorporate without ambiguity.

Trajectory Likelihood and Few-Shot Prompting

ReAct uses in-context learning rather than gradient-based training. The model is conditioned on KK demonstration trajectories D={τ(1),…,τ(K)}D = \{\tau^{(1)}, \ldots, \tau^{(K)}\}:

4
Pθ(ans∣q)=∑τPθ(τ∣q,D)  1[Finish(τ)=ans]\quad P_\theta(ans \mid q) = \sum_{\tau} P_\theta(\tau \mid q, D)\; \mathbf{1}[\texttt{Finish}(\tau) = ans]

Explanation: The answer probability marginalizes over all possible reasoning-action trajectories, but in practice greedy or beam decoding collapses this to a single trajectory — the sum is conceptual, not computational.

Comparison with Chain-of-Thought

Pure CoT omits the action and observation:

5
PθCoT(ans∣q)=Pθ(h1:T,  ans∣q)\quad P_\theta^{\text{CoT}}(ans \mid q) = P_\theta(h_{1:T},\; ans \mid q)

Explanation: Without the oto_t observations, all reasoning is closed-loop hallucination over the model's frozen parametric knowledge — any factual error in h1h_1 propagates uncorrected through all subsequent steps. The gap between equations [4] and [5] is precisely the grounding benefit of ReAct: every oto_t is an external correction signal injected into the context. ---

researcher · Shunyu Yao | b. 1997 | Co-inventor of ReAct

Shunyu Yao is a PhD student at Princeton (later affiliated with Stanford) working at the intersection of NLP, cognitive science, and AI agents; his undergraduate background in philosophy informs his emphasis on the cognitive plausibility of LLM reasoning. His 2022 paper ReAct: Synergizing Reasoning and Acting in Language Models (with Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao) introduced the interleaved thought-action-observation paradigm that became foundational to modern LLM agent design. Surprisingly, the paper was partly inspired by Yao's reading of classical AI planning literature from the 1980s, drawing a direct line from STRIPS-style action theory to modern prompt engineering.

researcher · Karthik Narasimhan | b. 1989 | Grounding LLMs in Interactive Environments

Karthik Narasimhan is an assistant professor at Princeton whose research program focuses on language-grounded agents that learn to act in text-based and robotic environments. His earlier work on learning to navigate text-based games (2015–2018) established the conceptual scaffolding for action-conditioned language models that directly preceded ReAct. He is notable for bridging reinforcement learning theory with modern transformer-based agents at a time when most NLP researchers had not yet engaged seriously with agentic settings.

---

Figure
ReAct Trajectory Data Flow
ReAct: Thought → Action → Observation Loop LLM (frozen weights) P_θ(h_t, a_t | context) query q + few-shot demos D Thought hₜ "I need to find..." Action aₜ Search[entity] Observation oₜ "Wikipedia snippet..." External Tool / Env Wikipedia · Calc · DB appended to context Growing context window: [q | h₁ a₁ o₁ | h₂ a₂ o₂ | ...]

---

Figure
ReAct vs. CoT vs. Act-Only: Decision Boundary Comparison
Paradigm Comparison: Grounding vs. Reasoning Act Only Chain-of-Thought ReAct Action: Search[X] Obs: snippet... Action: Finish[Y] No reasoning trace Brittle to novel queries Cannot self-correct Thought: "X implies Y" Thought: "So Z..." Answer: Z Rich reasoning No grounding → hallucinates Errors compound silently Thought: "Need birth year" Action: Search[person] Obs: "Born 1943..." Thought: "1943, so age=80" Action: Finish[80] Grounded + reasoned ✓

Intuition

🧠 The Mental Model

Imagine a detective interrogating witnesses to solve a case. A pure chain-of-thought detective locks herself in a room and reasons entirely from memory: "The butler was angry, angry people commit crimes, therefore the butler did it." She may be brilliant, but if her memory of the butler's motive is wrong, she convicts an innocent man with perfect logical consistency. A pure action-only detective runs around randomly interviewing witnesses with no theory to guide which witness to visit next, wasting time and missing the crucial lead. The ReAct detective does neither — she forms a hypothesis (Thought), then tests it by interviewing a specific witness (Action), updates her theory based on what the witness actually says (Observation), then forms the next hypothesis. Each witness interview is targeted by the current theory, and each answer sharpens or refutes it. The context window is her case notebook: a running log of hypotheses and evidence that accumulates until the case is closed. The key insight is that the notebook itself is the reasoning — it's not a summary of thinking, it's the thinking, made legible enough to drive the next action.

Historical Development

⏳ History
1987: Allen Newell and Herbert Simon formalize the physical symbol system hypothesis and STRIPS planning, establishing the idea that intelligent action requires symbolic reasoning over a world model — the conceptual ancestor of thought-then-act.
2015: Narasimhan, Kulkarni, and Barzilay publish Language Understanding for Text-based Games using Deep Reinforcement Learning, showing that recurrent networks could learn action policies grounded in natural language observations — an early neural precursor to tool-augmented agents.
2020: Brown et al. publish GPT-3: Language Models are Few-Shot Learners, demonstrating that in-context learning with demonstrations could elicit complex behaviors from frozen LLMs, making prompt-only agent design tractable without fine-tuning.
2022 (January): Wei et al. publish Chain-of-Thought Prompting Elicits Reasoning in Large Language Models, proving that intermediate reasoning traces dramatically improve multi-step QA — but the traces remain ungrounded, creating the hallucination problem ReAct would later solve.
2022 (October): Yao et al. release the ReAct preprint on arXiv (ReAct: Synergizing Reasoning and Acting in Language Models), showing that interleaved thought-action-observation trajectories outperform CoT alone on HotpotQA, FEVER, and ALFWorld benchmarks with PaLM 540B.
2023 (March): Shinn et al. publish Reflexion: Language Agents with Verbal Reinforcement Learning, extending ReAct by adding a retrospective reflection step after task failure — a self-critiquing layer built directly on top of ReAct trajectories.
2023 (May): LangChain and LlamaIndex adopt ReAct as the default reasoning loop in their agent frameworks, accelerating industrial adoption and making ReAct the de-facto standard for production LLM agents.
2023 (October): Yao et al. publish Tree of Thoughts, generalizing ReAct's linear trajectory to a tree-search over multiple reasoning branches — explicitly framing it as overcoming ReAct's greedy left-to-right limitation.
2024: OpenAI's Assistants API and Anthropic's Claude tool-use implement architectures structurally equivalent to ReAct, with thought scratchpads exposed via extended thinking modes, validating the paradigm at commercial scale.

Why It Matters

🔬 Real-World Impact
LangChain AgentExecutorThe default `ZERO_SHOT_REACT_DESCRIPTION` agent in LangChain implements the Thought/Action/Observation loop verbatim, allowing developers to wire any Python function as a tool callable by the LLM — this became the most-starred open-source agent framework with over 80k GitHub stars by 2024.
OpenAI GPT-4 with Function CallingGPT-4's function-calling mode is architecturally ReAct with the thought trace implicit — the model decides which tool to call based on intermediate reasoning, then updates its response plan based on the tool's JSON return value, enabling applications like code interpreters and web browsing plugins.
Google Gemini with ExtensionsGoogle's Gemini agent extensions use a ReAct-style loop to orchestrate calls to Google Search, Google Maps, and Google Flights within a single conversational turn, grounding travel planning responses in live data.
Reflexion (Shinn et al., 2023)Builds a verbal reinforcement learning loop directly on ReAct trajectories — after a failed episode, the agent's full Thought/Action/Observation trace is fed to a reflector LLM which writes a natural-language critique, stored in an episodic memory buffer and prepended to the next episode's context, achieving state-of-the-art on HumanEval coding benchmarks.
AutoGPT and BabyAGIThese early autonomous agent frameworks (2023) popularized ReAct-style loops to a non-technical audience by running the Thought/Action/Observation cycle recursively with task memory, spawning an entire ecosystem of autonomous agent research.
Clinical Decision Support (experimental)Research groups at Stanford and MIT have applied ReAct-style agents to differential diagnosis — the LLM issues `OrderLab[test]` actions, receives simulated lab results as observations, and updates its diagnostic reasoning trace, achieving performance approaching attending physicians on USMLE-style cases while producing auditable reasoning chains.
SWE-agent (Princeton, 2024)A software engineering agent that wraps a shell environment as a ReAct-compatible tool set (`BashRun`, `FileEdit`, `FileView`), achieving 12.5% resolution rate on real GitHub issues from SWE-bench — demonstrating ReAct scaling to long-horizon, multi-file coding tasks.

Prerequisites

What This Leads To

See Also

Chain-of-Thought Prompting · Retrieval-Augmented Generation · Tool-Augmented Language Models · Reflexion · Tree of Thoughts · LLM Agents · Prompt Engineering

Before this

Where this leads