Reflexion: Language Agents with Verbal Reinforcement Learning

• Canonical ID: doi-10.52202-075280-0377
• Aliases: arXiv:2303.11366; NeurIPS 2023 proceedings article
• Canonical URL: https://doi.org/10.52202/075280-0377
• Full text audited: arXiv v4 HTML, updated 2023-10-10: https://arxiv.org/html/2303.11366
• Audit date: 2026-08-03
• Access: full text
• Verdict: PROMISING — UNREPLICATED (3/5), scoped to the reported ALFWorld, HotpotQA, HumanEval, MBPP, and LeetcodeHard protocols. The broader claim that verbal self-reflection itself is the causal ingredient is more fragile (2/5).

Identity, version, and integrity status

Crossref resolves the NeurIPS proceedings DOI and reports no correction/retraction relation: https://api.crossref.org/works/10.52202%2F075280-0377. arXiv metadata records initial submission 2023-03-20, v4 update 2023-10-10, and the comment “v4 contains a few additional experiments”: https://export.arxiv.org/api/query?id_list=2303.11366. No publisher correction, retraction, or formal misconduct finding was found in the bounded Crossref/arXiv checks. No integrity concern issued. This is not proof that none exists.

Load-bearing claims and trace

Claim	Population/system, comparator, outcome	Direct support	Audit judgment
A Reflexion/ReAct agent learns from verbal summaries of failed trials in ALFWorld	GPT-3 ReAct action generator; 134 ALFWorld tasks; ReAct reset-and-retry control; cumulative task success over 12 trials	Methods specify the same reset trigger, with only Reflexion writing a verbal memory; Figure 3 reports 130/134 tasks solved and a 22-point absolute gain over ReAct: https://arxiv.org/html/2303.11366#S4.SS1	SUPPORTED AS REPORTED; SINGLE-SOURCE. Same environment and trigger are useful controls, but no multi-seed interval or independent rerun is reported. Repeated attempts also change the quantity of environment interaction.
Reflexion improves HotpotQA reasoning/search	100 HotpotQA questions; CoT/ReAct baselines; exact-match feedback; up to three consecutive failures	Figure 4 reports gains up to 20 points and an episodic-memory ablation in which verbal reflection adds 8 points: https://arxiv.org/html/2303.11366#S4.SS2	SUPPORTED AS REPORTED; FRAGILE. The sample is 100, stopping/retry rules differ from a one-shot baseline, and uncertainty/seed analyses are absent. Ground-truth exact-match feedback is external information, so this does not isolate unaided “self-correction.”
Reflexion reaches 91.0% HumanEval Python pass@1, above GPT-4 at 80.1%	HumanEval; GPT-4 base completion versus test generation plus repeated reflection	Table 1 reports 80.1 and 91.0; Table 2 reports a 1.4% false-positive internal-test rate on HumanEval Python: https://arxiv.org/html/2303.11366#S4.T1 and https://arxiv.org/html/2303.11366#S4.T2	SUPPORTED AS REPORTED; SINGLE-SOURCE. It is not a matched compute/attempt comparison: Reflexion generates tests and revises, while the cited GPT-4 figure is a base pass@1. Benchmark contamination cannot be excluded for GPT-4/HumanEval.
Verbal reflection contributes beyond test execution	50 hardest HumanEval-to-Rust translations; base, test-only, reflection-only, full composite	Table 3 reports 0.60 base, 0.52 reflection without tests, 0.60 tests without reflection, and 0.68 full: https://arxiv.org/html/2303.11366#S4.T3	MIXED BUT INFORMATIVE. Full composition wins, but reflection alone harms performance and test execution alone does not improve it. The experiment supports an interaction, not a general claim that introspection independently improves answers.
Verbal traces are more interpretable/diagnosable	Proposed monitoring of reflections before tool use	Broader-impact discussion only: https://arxiv.org/html/2303.11366#S6	NOT DEMONSTRATED. No human study, faithfulness intervention, or causal trace test is reported.

Design, statistics, and reproducibility

• Design fit: strongest in ALFWorld, where reset conditions and feedback triggers are explicit. Coding adds executable tests but combines more inference and tool budget with reflection. HotpotQA uses exact-match feedback and repeated trials.
• Sampling/statistics: the paper gives task counts and point estimates but no confidence intervals, repeated seeds, or correction for repeated benchmark comparisons in the audited sections: https://arxiv.org/html/2303.11366#S4.
• Ablations: the Rust table separates test generation and reflection; HotpotQA compares raw episodic memory with verbal reflection. These narrow the mechanism but do not isolate compute/attempt budget.
• Leakage/contamination: LeetcodeHard is described as 40 post-2022 problems, but HumanEval and MBPP were public before GPT-4; the paper does not provide a contamination audit: https://arxiv.org/html/2303.11366#S4.SS3.
• Code/data provenance: the paper-linked repository now resolves at https://api.github.com/repos/noahshinn/reflexion, reports an MIT license, and contains task-specific code in its public tree: https://api.github.com/repos/noahshinn/reflexion/git/trees/main?recursive=1. Artifact presence is not reproduction.
• Safety: the authors explicitly advise isolated execution because generated code is not validated before execution: https://arxiv.org/html/2303.11366#S8.

Independent support, contradiction, and current status

OpenAlex reports 267 citing works for the canonical DOI as of 2026-08-03: https://api.openalex.org/works/https://doi.org/10.52202/075280-0377. The newest 100-work citation traversal contains many derivative self-reflection systems but no exact independent rerun of all original tables: https://api.openalex.org/works?filter=cites:W7133222210&sort=publication_date:desc&per-page=100. NOT FOUND IN BOUNDED SEARCH is not proof that none exists.

A current arXiv follow-up, Reflection or Re-Generation?, reports that two-pass revision without external information can have near-zero or negative information gain: https://export.arxiv.org/api/query?id_list=2607.28908. This is CONVERGENT COUNTEREVIDENCE to broad intrinsic-reflection claims, not a direct contradiction of Reflexion's externally scored/test-driven loops.

Verdict rationale

PROMISING — UNREPLICATED (3/5). The paper presents multiple environments, concrete baselines, executable feedback, and useful ablations. Confidence is capped by single-study results, unmatched attempt/compute budgets, absent interval/seed reporting, public benchmark contamination risk, and no exact independent reproduction located. The broad “verbal reflection” narrative overstates what the strongest ablation proves: in Rust, reflection without tests underperforms the base model.

Decisive next verification

Preregister a multi-seed reproduction on frozen ALFWorld and contamination-resistant code tasks. Match total model calls, environment interactions, test executions, stopping rules, and context budget across: regenerate-only, episodic-memory, external-error-only, verbal-reflection-only, and full composite conditions. Kill the reflection-specific claim if full Reflexion does not exceed the best budget-matched nonverbal/error-conditioned control by at least five absolute points with a confidence interval excluding zero on two task families.