AI agents and agentic AI — decision brief — 2026-08-02

What you should know

• Do not build another generic agent framework. ReAct is a useful design pattern, but the market already uses it widely. The research does not show a distinct product gap. Confidence: high on market crowding; medium on the exact performance claim.
• SafeKeep points to a narrower test worth running. Its authors report that tool-schema formatting can weaken model refusals. Their two-stage method improved safety scores across four models. No independent team has reproduced the effect. Confidence: medium-low.
• The best current idea is a schema-aware safety regression test, not a safety product. A fixed test could show whether changes to models, tools, or serializers create new safety failures. Demand and willingness to pay remain untested. Confidence: medium on technical feasibility; low on commercial value.
• No result supports deployment yet. Both audited papers rely on one research origin for their main benchmark effects. The next useful work is independent testing, not product development.

PROMISING — UNREPLICATED means the paper presents credible direct evidence, but no independent team has yet confirmed the main effect.

Recommendation

Do now

Reproduce SafeKeep's schema-versus-plain-text effect across current agent stacks. Use realistic tools, repeated runs, semantic task scoring, adaptive attacks, and fixed pass/fail thresholds.

Do not do

Do not build a generic ReAct orchestrator. Do not market SafeKeep's result as established safety protection. Do not infer buyer demand from benchmark gains.

What would change the call

Move the safety regression idea from verify paper to test only if an independent run improves refusal or injection-attack performance by at least 20 points on two of three model families while reducing benign task completion by no more than 5 points. Drop the mechanism-specific idea if it misses that threshold.

Evidence that changed the view

This was the first durable run, so there is no earlier report to compare.

Two findings shaped the decision:

1. ReAct showed that explicit reason/action loops can improve some bounded tasks. It did not establish a new product gap. The pattern, official code, and a large follow-up literature already exist.
2. SafeKeep identified a specific failure mode in tool formatting. The authors' ablations make the mechanism plausible. The result is three days old, lacks independent support, and does not yet measure real benign task completion well enough for deployment.

That combination rules out a broad agent product and supports one narrow verification test.

Evidence table

Subject	What the authors claim	Direct evidence	Independent support	Main limit	Scoped verdict
ReAct	Interleaving reasoning and actions improves performance, grounding, and interpretability.	ReAct reached 71% success on 134 ALFWorld games versus 45% for the best action-only trial. It reached 40.0% on 500 WebShop instructions versus 30.1% for action only (paper, Tables 3–4).	AgentBench also finds long-horizon agent failures. This supports the problem, not ReAct's exact gain.	The paper uses best trials for headline comparisons. It reports no WebShop intervals or seed variance. PaLM-540B is unavailable. ReAct trails chain-of-thought on HotpotQA.	3/5 — promising, not independently reproduced for the named benchmarks. Interpretability and trust claims: 2/5.
SafeKeep	Tool schemas suppress refusal signals; plain-text tool descriptions can restore them for a separate safety check.	Across Llama, Qwen, and Mistral, flattening improved harmful-versus-benign separation. Across four models, SafeKeep raised harmful refusal from 23.8% to 70.6% and cut injection attack success from 25.6% to 2.5% (paper, Tables 1–4).	AgentHarm and InjecAgent confirm that harmful tool use and prompt injection are real test problems. They do not confirm SafeKeep's method.	The study uses 400 paired diagnostic examples, point estimates without intervals, and some generated benign rewrites. Stronger interventions often caused invalid output.	3/5 — promising, not independently reproduced for the tested models, benchmarks, and templates. Broader mechanism claim: 2/5.

Paper notes

ReAct: Synergizing Reasoning and Acting in Language Models

The paper's strongest result is not that visible reasoning makes agents trustworthy. It is that a sparse reason/action loop improved some benchmark point estimates under the paper's prompt and model setup.

The same-trajectory ReAct-versus-action-only comparison and six ALFWorld prompt variants isolate some value from explicit reasoning. But the study does not show that the text trace faithfully explains the model's action. It also does not provide a controlled human study of trust or interpretability.

The OpenAlex citation graph reports 566 citing works. That proves attention, not replication. The bounded search found no direct independent reproduction.

Next decisive check: rerun ALFWorld and WebShop with frozen revisions, open or API models, multiple seeds, identical prompt selection, action-only and state-machine controls, confidence intervals, and contamination checks.

Dossier

Tool Specifications Matter: Uncovering and Mitigating Safety Risks in AI Agents

SafeKeep's core idea is simple: show tool semantics as plain text to a safety judge, then use the normal schema only for execution. Format and semantics ablations support the claim that representation matters.

The paper does not yet show that this works across current production stacks. It also does not fully establish benign task success, latency, cost, or resistance to adaptive attacks. No correction or formal notice appeared in the bounded search. ArXiv lists version 1 on 2026-07-31 (metadata).

Next decisive check: run a preregistered, multi-model reproduction with realistic tools, repeated decoding, semantic task completion, adaptive attacks, latency, and cost.

Dossier

Opportunity decisions

Opportunity	Evidence	Unexploredness	Technical	Operational	Value	Binding constraint	Next test	Kill criterion	Decision
Schema-aware safety regression test	3/5	2/5	4/5	3/5	2/5	Independent reproduction and buyer evidence	Fixed schema-versus-plain-text test across three model families	Drop if fewer than two families gain 20 points, or benign completion loses more than 5 points	Verify paper
Generic ReAct orchestrator	3/5	0/5	5/5	4/5	3/5	No distinct product gap	None	Existing pattern and implementations already occupy the idea	Occupied

Why the safety test is not yet a product

The technical path is ordinary engineering: freeze tool definitions, serializers, model versions, and expected safety outcomes. The commercial case is absent. No buyer interviews, incident review, budget check, procurement search, or willingness-to-pay test ran in this report.

Nearby work also weakens the novelty claim. SafeKeep released public code. AgentHarm, InjecAgent, and guardrail tools already cover general agent-safety evaluation. The bounded search found no separate product built around plain-text tool rendering, but search absence does not prove novelty. The correct label is partly occupied with a different possible wedge.

What remains unknown

1. Can anyone reproduce ReAct's exact gains? PaLM-540B is unavailable, and the search found no direct rerun. A budget-matched benchmark reproduction would answer this.
2. Do ReAct traces explain actions? Readable text is not proof of causal faithfulness. Intervention and counterfactual tests are needed.
3. Does SafeKeep generalize? The current evidence covers named models, benchmarks, and templates. An independent multi-stack run must test realistic schemas and adaptive attacks.
4. Does SafeKeep preserve useful work? Parseable output and benchmark safety are not the same as successful benign tasks. Semantic task scoring must measure both.
5. Will anyone pay for this? No market evidence exists.
6. Is the idea legally or commercially clear? Patent, product, and freedom-to-operate searches remain incomplete.

Filtered out or already occupied

• Generic ReAct orchestration: occupied by the published pattern, official code, and extensive follow-up work.
• Agentic AI in medicine (arXiv:2607.25489): a survey, not original evidence eligible for a full audit.
• Evaluation Scores Are Perishable Knowledge Claims (arXiv:2607.26191): a position paper, not original audited evidence.
• The broad archive queries returned much unrelated work. The next run will use tighter date bands rather than hide the poor query.

Coverage and method

• Scope: original research on autonomous or semi-autonomous agents, from the earliest searchable record through 2026-08-02. The baseline remains incomplete.
• Sources: OpenAlex, Crossref, arXiv, Semantic Scholar, GitHub metadata, and canonical paper or benchmark pages. The source run records scope, coverage, blind spots, and outputs.
• Work counted: 590 index or graph hits returned; 260 discovery hits title-screened; 15 in-scope records retained; 2 full texts read; 2 papers audited.
• Follow-up work: ReAct citation, criticism, and reproduction branches; SafeKeep correction, benchmark, repository, mechanism, and prior-art branches.
• Blind spots: OpenReview returned HTTP 403. Two Semantic Scholar calls returned HTTP 429. The run did not cover ACL Anthology, patents, procurement, buyer demand, or all result pages.
• Experiment: none. A toy test would not resolve either paper's main uncertainty.
• Coverage claim: this report covers records returned by the logged protocol. It does not cover every agent paper.

Next run

1. Start the new-paper search at 2026-08-01 with overlap for indexing lag.
2. Continue the archive at OpenAlex / 2023-04-01..2023-08-31 / planning-tool-use-memory-evaluation / page 1, then search the same band in arXiv.
3. Audit Reflexion first. Then inspect AgenticRepair, SecRespond, Addressable Recall Compaction, and Explanation-Bound Tool Execution.
4. Search for direct ReAct reproductions and later SafeKeep versions or replications.
5. Do not promote the SafeKeep opportunity before independent reproduction.

Sources

1. ReAct arXiv metadata — primary metadata, updated 2023-03-10, accessed 2026-08-03.
2. ReAct full text — primary paper, accessed 2026-08-03.
3. ReAct OpenAlex record — scholarly index, accessed 2026-08-03.
4. ReAct official repository — implementation metadata, accessed 2026-08-03.
5. ReAct citing graph — scholarly index, accessed 2026-08-03.
6. AgentBench metadata — independent benchmark context, accessed 2026-08-03.
7. SafeKeep arXiv metadata — primary metadata, published 2026-07-31, accessed 2026-08-03.
8. SafeKeep full text — primary paper, accessed 2026-08-03.
9. SafeKeep repository metadata — implementation metadata, accessed 2026-08-03.
10. SafeKeep repository tree — artifact inventory, accessed 2026-08-03.
11. AgentHarm metadata — independent benchmark, accessed 2026-08-03.
12. InjecAgent metadata — independent benchmark, accessed 2026-08-03.