Self-HPO: Microsoft Research's Coding Agent That Rewrites Its Own Harness

Microsoft Research has released Self-HPO, a method that lets a coding agent rewrite the harness around it and then trains the model inside that harness with reinforcement learning. Starting from the open Qwen3.5-35B-A3B model, the authors report that the average resolve rate over three SWE-bench variants rises from 57.8% to 64.8%, with no stronger teacher model involved. The paper, code and checkpoints are public, and the authors published a write-up on October 19, 2026.

If you have been following the idea that coding agents are as much about the scaffolding as the weights, this is a clean test of it. We covered the broader pattern in The Second Writer, and Self-HPO is a concrete, reproducible instance of it.

TL;DR: the questions most readers ask

What is a harness, and why optimize it?

A coding agent is a model, called the policy, plus the code around it, called the harness. The harness decides what the model sees, how its shell commands run, and when the run is declared finished. The same policy can solve an issue under one harness and fail it under another, which is why harness work has become its own discipline. Our agent harness explainer covers the basics.

The Self-HPO authors frame the existing options as one-sided. Reinforcement learning with verifiable rewards (RLVR) trains the policy inside a fixed harness. Self-Harness lets the policy rewrite its harness but never updates the weights, so it keeps proposing from the same blind spots. Recent joint methods train both, but still rely on an external model to write the harness, so the agent is only as good as what it is handed.

How does the Self-HPO loop work?

Each round has two steps, as the project README describes.

Self-HPO harness-policy optimization loop alternating an H-step and a P-step Image: Self-HPO authors (Microsoft Research)

H-step (harness). The current policy reads its own failed runs on a search set and proposes harness revisions, covering both prompts and runtime logic. It tries a few candidates and keeps the one that solves the most tasks. In round one the search set is 300 tasks sampled from the training data.

P-step (policy). Reinforcement learning updates the policy inside the winning harness, rewarded only by whether the tests pass. The tasks the policy still never solves during training become the next round's search set.

Because the policy that writes the next harness is the one that just got better, the loop is recursive. The authors run two rounds, following a stopping criterion. This is a small, bounded cousin of the ideas in our recursive self-improvement explainer, not an open-ended self-improving system.

What do the numbers show?

The authors train only on Python tasks and evaluate on SWE-bench Verified (in domain) plus SWE-bench Multilingual and SWE-bench Pro, which the loop never sees. All figures below are resolve rates averaged over three runs, taken from the authors' table.

Three takeaways stand out.

• Both halves matter. Self-HPO beats RLVR by 2.0 points and Self-Harness by 4.3 points on SWE-bench Verified. The abstract adds that the joint gain exceeds the sum of the individual gains.
• Gains transfer. Both harness search and RL use only Python tasks, yet SWE-bench Multilingual gains 5.5 points and SWE-bench Pro gains 6.9 points. Harness search alone barely moved Multilingual (+0.1).
• Put a strong model in the harness, not the weights. Distilling Opus 4.8 trajectories with SFT helped in domain but fell below the base agent out of domain. Letting Opus write the harness did better everywhere, 8.9 points above SFT on average.

For context on how fragile SWE-bench-style numbers can be, read our coverage of OpenAI's SWE-bench Pro audit, which found a large share of tasks broken. Treat any single resolve rate as a comparison within one paper, not an absolute ranking.

What did the agent actually write into its harness?

The most concrete part of the write-up is a case study. Almost everything the policy added happens at one moment: when it says it is done. The seed harness ends the run right there and sends the submission to the grader. The harness the policy wrote first checks the patch, tells the model what looks wrong, and gives it one more try.

The checks the policy invented, according to the authors, include:

• a coverage check on whether every file it worked on ended up in the patch
• a decorator reminder, for patches that use something like @cache without importing it
• a pull-request intent reminder, checking that the patch mentions the key terms of the issue

In one real case, sphinx-doc__sphinx-9658, the final policy had fixed the source but finished by printing a summary instead of a patch, so the submission was empty. The seed harness would have graded that as a failure. The learned harness noticed the empty patch, handed control back once, and the policy then submitted a valid patch that resolved the task. It is a modest fix, which is exactly the point: much of the gain looks like practical guardrails rather than a mysterious new capability.

Is it reliability or new capability?

Pass@k curve on SWE-Bench Verified comparing the base agent with Self-HPO Image: Self-HPO authors (Microsoft Research)

The authors measured both. On SWE-bench Verified with eight attempts per task, pass@8 rises from 80.0% to 84.8%, and Self-HPO stays above the base agent for every k. Most of the gain is reliability: tasks the base agent solved only some of the time (+17.2 points on the medium group, +12.9 on the easy group) are now solved most of the time. Hard tasks the base agent never solved in eight attempts gain +6.1, so there is some genuinely new capability too.

There is a cost. Tasks the base agent always solved dropped by 2.2 points. If your workload is mostly easy, well-covered tasks, the optimized agent is not a free upgrade.

What the ablations say about doing this yourself

• The search set should move with the policy. Searching on the same initial tasks every round costs 1.5 points, because the failures that mattered for an earlier policy are not always the useful ones for the next.
• A stronger proposer helps, but you do not need one. Swapping Opus 4.8 for the policy itself costs 2.1 points on SWE-bench Verified, and the result still beats every single-component baseline that does not use an external model.
• More harness-only rounds do not reliably help. A second Self-Harness round dropped SWE-bench Multilingual from 55.0% to 54.3%.

If you are building agents in production, the practical lesson overlaps with other Microsoft work we have covered: the Orchard agentic modeling framework (the sandbox service Self-HPO runs on), Agent Lightning v1.0 for training agents with the harness you ship, and FrogNano, a 4B coding agent trained with no frontier distillation.

A rounded shape held within a lightweight scaffold frame, illustrating an agent harness around a coding model

How to try it

The repo ships the seed, HPO and Self-HPO harnesses plus scripts for each stage. According to the README:

A minimal evaluation run, from the README, serves a released checkpoint and then generates and scores trajectories:

bash scripts/eval/serve_model.sh --model self-hpo/Self-HPO-Qwen3.5-35B-A3B --ngpu 8
bash scripts/eval/gen_traj.sh --method Self-HPO --benchmark sbv --harness self-hpo --ngpu 8

To use an API model as the proposer instead (the HPO variant), the README says to export PROPOSER=api, an Anthropic key and the model name. Even if you never train, reading the three harness folders is a cheap way to see what a model-written harness looks like next to the seed.

Limits and open questions

• Authors' numbers only. These are the team's own evaluations on one base model; we have not seen independent reproductions.
• Python-only training. Out-of-domain gains are encouraging, but the loop was never trained on other languages.
• Compute. Two rounds of RL on a 35B mixture-of-experts model with eight B200 GPUs and a sandbox fleet is not a weekend project for most teams.
• Benchmark caveats. SWE-bench variants differ in harness details and task quality, so absolute numbers are not comparable across papers. Compare the deltas.
• Related work moves fast. The authors cite other joint harness-and-weights methods, and the field is crowded; see how Sakana's MASS and Microsoft's SkillOpt approach self-improvement from other angles.

Why it matters

The result supports a simple engineering claim: for coding agents, the harness is trainable surface area, and letting the model that will live inside it write it, then training that model against it, beats improving either half alone. It also suggests a budget rule that many teams can act on today: spend frontier-model tokens on writing better scaffolding rather than on distilling trajectories into small models. For a skeptical counterpoint on how far self-improvement claims go, see our piece on DeepMind's view that self-improving AI is not close.

If you build or evaluate agents, the immediate to-do list is short: pull the released harnesses, diff them against your own, and test whether a pre-submit patch check helps your workflow.

Related reading

• The Second Writer: How Self-Evolving Coding Agents Actually Learn
• What Is an Agent Harness?
• What Is Recursive Self-Improvement (RSI) in AI?
• Microsoft Orchard: Open-Source Agentic Modeling Framework
• Agent Lightning v1.0
• FrogNano: Microsoft's 4B Coding Agent
• Sakana AI MASS
• Primary sources: paper on Hugging Face, arXiv listing, GitHub repo, author write-up

Details and numbers are accurate as of October 20, 2026, and come from the authors' own materials. Check the repository for later changes.