NVIDIA EdgeWise: A Local 9B Agent That Calls GPT-5.5 Only When Needed

Most coding agents burn money by asking their most expensive model to do everything. A new paper from NVIDIA argues that most of those calls are unnecessary. EdgeWise, described on its Hugging Face paper page and project page, runs a small model on your own device as the main solver and calls a cloud frontier model only for the steps that need it. The abstract reports that it "matches or outperforms pure-LLM execution while reducing inference cost up to 62% on SWE-Rebench and 34% on Terminal-Bench 2.1."

NVIDIA is a top lab and this is first-party research, so it is worth reading closely, with the usual caveat that the numbers are the authors' own and the code is not out yet.

TL;DR: EdgeWise at a glance

What is the core idea?

Long-horizon agent runs have dozens of steps: read a file, run a command, edit, run tests, read the error, edit again. A frontier model on every step is expensive, yet many steps are routine. The authors found that "allocating the LLM to only a small subset of key steps can achieve performance comparable to full LLM execution."

So EdgeWise inverts the usual setup. Instead of a cloud model that occasionally calls a small helper, a small local model drives, and it delegates "critical steps" upward. The project page notes a striking sanity check: randomly delegating about 40 percent of steps sometimes matched full-cloud performance. Its conclusion: "performance depends more on where cloud intelligence is used than on how often it is used."

How does the model learn when to delegate?

Two approaches are compared:

1. External judge. A judge model estimates each step's difficulty and importance and decides whether the small model or the cloud LLM should act. The authors use this strategy as a research tool to study design questions.
2. Self-delegation via reinforcement learning. After supervised fine-tuning, a step-level RL method jointly optimizes task solving and delegation, with fine-grained credit assignment for each delegation decision. The small model learns to delegate on its own, which means "no additional models at inference time."

The design questions they investigate are when to delegate, how delegation decisions should be made, what context to share, and how often to call the cloud model. Findings from the project page:

• When: critical steps. Writing or editing code and complex reading or search have the strongest link to answer quality, and cloud calls concentrate there.
• What context: the full interaction history generally works best, so context is a trade-off between completeness and cost.
• How often: a relatively small number of well-placed calls can be enough.

What do the numbers say?

The headline results on the project page, comparing self-delegation to an LLM-only run:

Averaged across the two cloud models and two benchmarks, cost falls 45.16 percent with Pass@3 essentially unchanged. The method was trained only on the Terminus2 harness yet transferred to OpenHands, Pi and mini-swe-agent with cost reductions of at least 25 percent; the average across four harnesses is 33.04 percent.

Note a small inconsistency: the abstract says up to 62 percent while the table shows 63 percent for one configuration. That likely reflects rounding or a different aggregation, and it is a reminder to read the paper's tables before quoting a number.

Does it work on a real workstation?

Yes, per the project page. On an NVIDIA DGX Spark with GPT-5.5 as the cloud model, self-delegation reached Pass@3 comparable to an A100 setup and cut inference cost by about 50 percent versus the vanilla cloud model. The page says it also transfers to RTX Spark. For the hardware background, see our coverage of the DGX Spark and of Nvidia's Nemotron 3.5 Bolt model built for 24GB machines. The pairing is natural: these are the machines the paper has in mind.

What are the caveats?

• Cost counts cloud API spend only. Local compute for the small model is excluded. If you pay for GPUs or power, the true saving is smaller.
• Coding and terminal tasks only. Results may not carry over to browsing, data analysis or customer support agents.
• Code is not out. The project page labels its code link "Soon," so no one outside NVIDIA has reproduced this, and there is no word on weights.
• Pass@3. Three attempts per task is a generous metric; single-attempt behavior may differ.
• Cost depends on pricing. Reductions depend on the cloud prices at test time, which change often.
• Training needs data. Self-delegation requires supervised fine-tuning and RL on trajectories; this is not a drop-in prompt trick.

How does it compare with other cost-cutting approaches?

Builders already try several routes to cheaper agents:

EdgeWise sits between the first two: the small model is the default, the large model is the exception, and the decision is made per step rather than per request.

What does a 45 percent saving look like in dollars?

The paper reports percentages, not dollar figures, so here is an illustration with made-up round numbers. Suppose a team runs 1,000 agent tasks a week and each task costs $2.00 in cloud API spend when a frontier model handles every step. That is $2,000 a week. A 45 percent reduction, the paper's average across two cloud models and two benchmarks, brings it to about $1,100, saving roughly $900 a week or $47,000 a year. At the paper's best case of 62 percent, the same workload costs about $760.

Those numbers are illustrative, but the structure is real. The saving scales with how much of your spend is routine steps, and with the price gap between the local and cloud models. It shrinks if you have to buy hardware to run the local model, which is why the DGX Spark result matters: a workstation that already exists for other work turns the local model into nearly free capacity. NVIDIA's page says the workstation run cut inference cost by about 50 percent versus the vanilla cloud model while matching the A100 setup on Pass@3.

How would you build a rough version today?

The trained self-delegation policy is the paper's contribution, and it is not available. But the structure can be approximated by hand:

1. Run a small local model as the default actor. A 9B-class open model in a standard agent harness is enough to try. Our posts on local inference engines and the Llama Windows app cover ways to serve one.
2. Define cheap escalation triggers. For example: the local model proposes an edit to more than one file, tests fail twice in a row, or the model reports low confidence. Escalate those steps to the frontier model with the full history, as the paper recommends.
3. Keep a log of every escalation and what it changed. If escalated steps rarely differ from what the local model would have done, tighten the triggers.
4. Measure on your own tasks. Compare pass rate and total cost, including local compute, against an all-cloud baseline over several runs per task. Agent outcomes vary between runs, which is why the paper reports Pass@3.
5. Only then consider training. Self-delegation via RL needs trajectories, reward design and compute. If the authors release code, that is the point to revisit.

One reason to prefer the learned approach over hand rules is credit assignment. A delegation decision at step 12 may only pay off at step 40. The paper's fine-grained credit assignment for individual delegation decisions addresses exactly that, and simple heuristics will miss it.

What would change our read of the result?

Three things would make us more confident: an independent reproduction once code is out, results on non-coding agents, and cost accounting that includes local compute. Three things would make us less confident: a failure to transfer across harnesses beyond the four tested, large differences between Pass@1 and Pass@3, or evidence that the cloud models' prices shift the savings sharply. For now, the safest summary is that a credible lab has shown, on two public coding benchmarks, that selective escalation can capture much of a frontier model's accuracy at roughly half the cloud bill.

What should builders take from this?

1. Instrument your agent. Log which steps in your own runs actually change outcomes. The paper's claim is that writing and complex search matter most.
2. Try a crude version now. Without any training, you can test an external-judge or rule-based split: let a local model handle file reads and simple commands, and call a frontier model for edits and planning. Measure pass rate and cost.
3. Share full history on delegation. The paper suggests that cutting context to save tokens often costs more in quality.
4. Watch for the release. If the code lands, the training recipe for self-delegation will be the useful part.
5. Track total cost. Include local hardware, not just API invoices.

Related reading

• LangChain Open SWE model router: 64% cheaper
• Coinbase's fine-tuned Qwen3.5 9B beats Opus 4.5 on fraud
• Apple SelfCompact: self-compacting agents
• Nemotron 3.5 Bolt 74B-A9B in 24GB
• DGX Spark price and memory change
• Magnitude local inference engine, 2x claim checked

Sources: the paper on arXiv (2610.17560), NVIDIA EdgeWise project page and the Hugging Face paper page.

Figures are as published by the authors on October 13 to 16, 2026 and have not been independently reproduced.