Agent-Reflex: Training Qwen3-8B Agents to Recover From Broken Tools and User Corrections

Most tool-using agents are trained in tidy sandboxes where every API call returns clean JSON and the user never interrupts. Production is not like that: an MCP server times out, a permission is denied, a payload comes back malformed, and a person types "no, I meant the other account" halfway through a task. Agent-Reflex, released this week by the NLPIR lab at Renmin University of China, is a training recipe that targets exactly that gap. Built on Qwen3-8B, it scores 58.6 across five long-horizon tool-use benchmarks, 18.4 points above the base model, and 29.1 on a noise-injected benchmark where the base model scores 12.3 (paper page on Hugging Face, GitHub repo).

The paper page appeared on Hugging Face on October 14, 2026, and, more usefully for builders, it ships with code, data and weights. This post explains what the method does, what the numbers do and do not show, and how to try it.

TL;DR: the questions people ask first

Why noisy tools are an agent problem

The authors frame two gaps. First, training environments return clean feedback, so an agent never learns to detect that a tool call failed in a subtle way. Real MCP services time out, reject permissions or return broken results; the README groups tool errors into eight recurring families. Second, user simulators rarely correct the agent mid-task, even though real users do it constantly.

The cost is visible in the paper's own motivation figure: strong agents lose success rate when either tool noise or user noise is injected. If you have wired agents to external services through the Model Context Protocol, you have seen the pattern. The model keeps going with a stale assumption, retries the same failing call, or hallucinates a result. Our coverage of silent agent failures and of database-state reliability benchmarks describes the same family of problems from the evaluation side.

How Agent-Reflex works: three stages

The paper's central idea is to treat reflection as a learnable meta-skill, mined from failures and then internalized into the model rather than bolted on as a prompt.

Stage 1: Reflection Skill Discovery (RSD)

The team builds failure-aware executable environments by perturbing tool returns with real-world failure patterns. According to the README, this yields 557 MCP-anchored environments (the environment synthesis follows the earlier Agent-World work). A pool of LLMs with different capability levels attempts tasks in these environments, and the system mines verifier-matched success and failure pairs: the same task, one run that recovered, one that did not. From those contrasts it summarizes an offline library of 1,170 reusable reflection meta-skills, available in English and Chinese.

Stage 2: Reflective Supervised Fine-Tuning (ReSFT)

Each skill is converted into verifier-validated training data of three kinds: self-reflection (noticing a tool result is wrong), skill recovery (applying the right fix) and user-correction handling (accepting and acting on a mid-task correction). The skill prompts are stripped before training, so the model has to internalize the behavior. The released SFT set has 54,663 trajectories, and the recipe is full fine-tuning of Qwen3-8B with ZeRO-3 over three epochs (5,127 steps).

Stage 3: On-Policy Reflective Policy Optimization (ORPO)

The final stage combines agentic reinforcement learning under noisy interactions with same-token, skill-guided on-policy distillation. In plain terms, the model rolls out in noisy environments and earns reward for the task, while a skill-conditioned version of itself provides dense, token-level guidance on what a good reflection looks like. The RL set has 2,550 tasks with skills attached, and training is built on verl. Skills are used only in training: the deployed agent is skill-free.

A green ball following a curved track toward a goal, illustrating the reinforcement learning stage of Agent-Reflex

If the on-policy distillation idea is new to you, it is the same family of techniques behind several recent small-model results, and our write-up of Microsoft's Agent Lightning covers the harness side of agentic RL.

The numbers, as reported

All figures below come from the authors' README results table, using each benchmark's official metric. They are not independently reproduced.

A few readings of the table are worth making. The biggest absolute gain over the base is on tau2-Bench (26.2 to 62.1), a dual-control benchmark where the agent and a simulated user both act on shared state under a policy document, which is precisely where user corrections matter. The margin over the strongest prior environment-scaling baseline, EnvScaler-8B, is 7.7 points overall. And on AgentNoiseBench, which injects tool-side and user-side perturbations into tau2-Bench and VitaBench environments, Agent-Reflex more than doubles the base score, though 29.1 is still far from solved.

Note also what the table does not say. The plain Reflexion baseline, a prompt-level self-reflection method, actually drops VitaBench from 11.4 to 6.8. That fits the authors' argument that bolting reflection onto an untrained model is not the same as training it in.

What the benchmarks measure

The six evaluation suites cover three areas, per the README:

• General function calling: BFCL V4 (5,217 tasks including web search and multi-turn), ACEBench (when to call a tool and how to handle underspecified requests) and ToolSandbox (stateful conversational tool use, with deliberately unsolvable tasks).
• Long-horizon workflows: tau2-Bench (airline, retail, telecom) and VitaBench (delivery, in-store and travel domains, including cross-scenario tasks).
• Noisy interaction: AgentNoiseBench, built on the previous two.

For background on how tool-use training data gets built in the first place, our ToolGrad post is a useful companion.

How to try it

The repository is organized by stage: RSD/ for the skill library, LLaMA-Factory/ for ReSFT, ORPO/ for the verl-based RL stage and evaluation/ for the six benchmarks. If you only want to evaluate or use the model, you can skip training:

pip install -U huggingface_hub
hf download dongguanting/Agent-Reflex-8B --local-dir /path/to/Agent-Reflex-8B
cd evaluation
MODEL_PATH=/path/to/Agent-Reflex-8B RUN_NAME=agent-reflex bash run_all.sh

run_all.sh serves the model with vLLM and runs all six benchmarks; set BENCHES="Tau2 BFCL-v4" to run a subset. To reproduce training, download the SFT data (dongguanting/Agent-Reflex-SFT-54K) and RL data (dongguanting/Agent-Reflex-RL-2K) and run the ReSFT then ORPO scripts. The 1,170-skill library is a 22 MB JSONL file in the repo (RSD/environment_skills.jsonl), which is useful on its own as a catalogue of how tool-using agents fail.

Hardware is not trivial: the README asks for NVIDIA 80 GB GPUs with CUDA 12.8, using two nodes of eight for ReSFT and one node of eight for ORPO and evaluation. Training itself needs no API keys, but evaluation does: the user simulator (gpt-4.1) and judge (claude-sonnet-4-6) are reached through a third-party OpenAI-compatible gateway by default, and the README says you can point the base URL at official endpoints or your own gateway. BFCL web search also needs a Serper key.

Limitations and open questions

• One backbone. Results are for Qwen3-8B. The abstract says the framework is general, but the released evidence is one model size. We would want to see it on a larger or differently pretrained base before assuming the gains transfer.
• Simulated users and judges. Several benchmarks rely on LLM user simulators and LLM judges, so the "user correction" behavior is measured against a model's idea of how users interrupt, not real people.
• Synthetic noise. The failure families are injected perturbations over 557 environments. Production failures are messier, and the paper's own claim is about robustness gains, not elimination.
• Self-reported and brand new. The repo had 22 stars and one fork when we read it, and the paper is only days old. Independent replication will tell us how much of the gap survives.
• Compute. Even an 8B model needs a multi-node cluster to retrain, though inference on the released checkpoint is cheap by comparison.

What this means if you build agents

You do not have to train anything to benefit from the lesson. Three practical takeaways:

1. Test with injected failures. Wrap your tools so a fraction of calls time out, return empty results or deny permission, and measure whether your agent notices. If you only ever test the happy path, you are measuring the wrong thing.
2. Treat user corrections as a first-class scenario. Add test cases where the simulated user changes their mind or says the agent got something wrong.
3. Watch small-model viability. Agent-Reflex joins a growing list of results where a small, specialized model closes much of the gap on a measured task, alongside Coinbase's fine-tuned Qwen3.5 9B for fraud. For context management in long runs, see also Apple's SelfCompact.

If your agents act on real systems, pair training-time robustness with runtime guardrails. AgentBeam, the agent security platform from the explainx.ai team, stops AI agents before they take dangerous actions, which matters most exactly when a model misreads a failed tool result and tries something drastic.

Related reading

• What is MCP? The Model Context Protocol guide
• Silent agent failures and simulations
• Agent database-state reliability benchmark
• Agent Lightning v1: harnessed agentic RL
• Google Research ToolGrad: tool-use datasets
• Apple SelfCompact for long-running agents
• Official sources: arXiv 2610.17953, GitHub repository

Details reflect the paper page and repository as read on October 14, 2026; results, licenses and release assets may change as the authors update them.