Stop Asking Once: How AREX-2 Teaches AI Agents to Improve Themselves, Round After Round

Here's how most of us use AI agents: ask once, take the answer, move on. If the answer is wrong, we tweak the prompt and ask again — but every attempt starts from zero. The agent never learns from its own mistakes inside the run.

What if it did? What if the agent picked up where it left off, reviewed what failed, changed its approach, and came back stronger — automatically, round after round, with no human in the loop? That ability has a name in the research literature: self-improvement at test time. And a new paper, AREX-2, shows a concrete, reproducible path to it — one where even a modest 27-billion-parameter open model keeps getting better the more rounds you give it.

What the researchers did

A team of 14 researchers led by Hongjin Qian started by defining the target precisely. Self-improvement, they say, is the ability to iteratively refine a solution at test time — not after retraining, but during the run itself. They split it into two complementary skills:

• Reflection — producing a solution that is actually better than the current one, by diagnosing what went wrong.
• Long-horizon execution — keeping the iteration effective over many rounds, instead of the loop going stale after two or three tries.

Their central hypothesis: both skills are domain-agnostic. You don't need to teach reflection separately for coding, research, and math — you can learn it where supervision is easy and transfer it elsewhere. So they synthesized long-horizon improvement trajectories from machine learning tasks and algorithmic programming tasks — two domains where feedback is verifiable (tests pass or fail) and where sustained iteration is natural.

They trained an agent built on Qwen3.8-27B on this trajectory data. Then they tested it where it counts.

Why it matters

The numbers are striking for a 27B open model — not a frontier giant:

• MLE-bench Lite: 81.8 and Frontier-CS: 70.7 — strong on the machine-learning engineering tasks it trained on.
• Transfer to deep research it never trained on: 84.0 on BrowseComp, 52.6 on HLE, 92.2 on GAIA, 93.8 on DeepSearchQA.
• The kicker: performance keeps improving as the budget of rounds grows. Give it more iterations and it keeps climbing — the gains don't plateau after the second or third try the way most agent loops do.

That last point reframes how we think about scaling. We usually scale parameters, data, or context. AREX-2 says: scale test-time rounds — and give the agent the reflective skill to actually use them. The difference between "the agent gave a mediocre answer" and "the agent gave a mediocre answer, reviewed it, fixed it, and delivered something good" is no longer a bigger model. It's a better loop.

And the transfer results matter just as much. Reflection learned on programming tasks carried over to open-ended deep-research benchmarks like GAIA and BrowseComp. That suggests reflective iteration is a general cognitive skill, not a coding trick — which means the technique below should work on your writing, analysis, and research agents too.

How YOU can use it

You don't need AREX-2's weights to apply the insight today. The paper's recipe translates into four practical moves for anyone running agents — in Cursor, Claude Code, or a custom loop:

1. Budget rounds, not just tokens

Most agent setups allow 2–3 iterations before giving up. The research shows gains compound across many rounds. Set your max iterations to 10–20. The cost is linear; the quality curve in the paper keeps rising.

2. Force explicit reflection every round

Reflection doesn't happen by accident — agents default to "try something different" without diagnosing. Add a reflection step to your prompt or agent instructions:

After each attempt, BEFORE writing anything new:
1. REVIEW: Quote exactly what failed (the failing test, the error message, the wrong output).
2. DIAGNOSE: Why did it fail — wrong approach, missed edge case, or small bug?
3. PLAN: What is the single smallest change most likely to fix it?
Then implement ONLY that change. Do not rewrite from scratch.

The "do not rewrite from scratch" line matters: the paper's long-horizon execution depends on incremental refinement, not fresh starts.

3. Give it verifiable feedback

Reflection needs a ground truth to reflect against. Run the tests. Render the page. Execute the query. Then feed the actual error output back into the loop — not your summary of it. Verifiable feedback is what made the training domains in the paper work, and it's what makes your loop work too.

4. Keep one thread alive

Don't restart the conversation on each attempt. The agent needs its own history of failures to reflect on — that's the "trajectory" the paper talks about. One long thread with 15 reflective rounds beats 15 fresh one-shot attempts.

Try it tonight

Take a task your agent usually gets 70% right — a tricky function, a data pipeline, a research summary. Wrap it in a 15-round reflect-and-retry loop with tests (or a checklist) as the judge, using the template above. That is the AREX-2 pattern in miniature: reflection plus long-horizon execution, no new model required.

Key takeaways

• Self-improvement is trainable: reflection + long-horizon execution can be learned from synthesized improvement trajectories.
• 27B is enough: an open Qwen3.8-27B-based agent posted these numbers — the loop matters more than the parameter count.
• Reflection transfers: learned on ML and programming tasks, it carried over to deep-research benchmarks (BrowseComp 84.0, GAIA 92.2).
• More rounds = better answers: performance keeps climbing with the iteration budget instead of plateauing.
• Your move today: budget 10–20 rounds, enforce explicit reflection, feed real error output, keep one thread alive.

Read the original paper: "AREX-2: Advancing Self-Improving Agents through Long-Horizon Reflective Tasks" — Hongjin Qian et al. — arXiv:2609.38288 (cs.AI, submitted 29 Sep 2026).