Apple Trains an 8B iOS UI Agent With Online RL on Live Apps: Planner, Grounder and GiGPPO

Apple Machine Learning Research has published a paper on teaching a small AI agent to operate real iPhone apps. The agent is an 8B-parameter model split into two parts, a planner and a grounding model, and it learns by doing: it is trained with online reinforcement learning on live iOS apps, running on hundreds of virtual iOS devices at once. The authors report that it beats Gemini-3.5-Flash and ties Gemini-3.1-Pro on real iOS applications. The paper is on the Apple Machine Learning Research page and on arXiv. The authors are Peitong Duan, Seyed Khashaiar Gatmiry, Jeff Nichols, Emmanuel Abbe and Samy Bengio, with the first two marked as equal contributors.

One caution up front: the Apple page gives qualitative claims and a few headline comparisons, but no benchmark tables and no code link. This post separates what is stated from what we cannot check.

TL;DR: what a builder should know

What problem is Apple trying to solve?

The paper starts from a cost problem. According to Apple's summary, frontier vision-language models are strong at navigating app screens, but too expensive to run for everyday use. The obvious answer is a smaller model. The trouble is how small models are usually trained.

Supervised fine-tuning (SFT) copies recorded human or model trajectories. Offline reinforcement learning learns from a fixed dataset. In both cases the model sees a static set of screens. Apple says such agents tend to memorize static trajectories, so they execute brittly and cannot recover when something unexpected appears: a pop-up, a slow load, a different layout.

Live apps are messy. Ads change, buttons move, networks lag. An agent trained only on frozen screenshots has never practiced recovering. Online RL puts the agent in the live environment and lets it try, fail and adjust. That is the core idea, and it mirrors how coding agents are now trained against real tools. Our coverage of Alibaba's OpenGCUA stack shows the open side of the same trend for desktop computer use.

How does the planning-grounding agent work?

The design splits the job in two.

1. A planning agent decides what to do next at a high level: open settings, find the toggle, confirm.
2. A UI grounding model turns that intent into a concrete place on the screen: which element to tap and where.

Splitting these helps because they fail in different ways. A planner can be wrong about the goal. A grounder can be right about the goal and still tap the wrong pixel. Training them as separate components makes errors easier to locate. Many recent GUI agents use a similar split, including the accessibility-tree approach we described in Vercel's agent-desktop post, though that one is a tool rather than a trained model.

What are dense rewards, and what does the LLM judge do?

A green ball following a track toward a cup, illustrating step-by-step rewards in an online RL UI navigation agent

The hardest part of training a UI agent with RL is the reward. If you only reward finishing the task, most attempts on a long task get zero, and the agent learns slowly. This is the sparse reward problem.

Apple's approach adds an LLM-as-a-Judge that scores intermediate progress at each step, plus a terminal reward for overall completion. That is the "dense" in the title. A step that moves closer to the goal gets credit even if the full task fails.

There is a known risk here: judge models can be fooled, and an agent can learn to please the judge rather than do the task. The summary page does not describe safeguards against this, such as judge audits or held-out verification. Because the terminal reward still anchors the result, the risk is reduced but not removed. Readers should look for this in the full paper.

What is GiGPPO?

The agent is trained with Proximal Policy Optimization (PPO), using step-level advantage estimation. PPO needs a critic, a value estimator that predicts how good a state is. In a noisy live environment that critic is unstable.

The authors introduce GiGPPO, described as a neighbor-conditioned value estimation method. It improves critic stability by using rewards from visually similar states across concurrent trajectories. In plain terms: if many parallel runs reach screens that look alike, they share information about how good that screen is. Hundreds of devices running at once make this possible, since there are many near-identical screens to compare.

We cannot judge the math from the summary alone. The practical claim is that this makes training more stable and convergence faster than terminal-only RL.

What infrastructure does it need?

Apple built a system that runs hundreds of concurrent virtual iOS devices. That is a real engineering cost and a quiet barrier for outside groups. Online RL on live apps needs simulators that boot fast, reset cleanly and can be parallelized. Few teams outside the large labs have that. It is also why the result is hard to reproduce without a release: no code is listed on the page.

What results do the authors report?

The page names three evaluation sets: UINavBench, iOSWorld and AndroidWorld. It does not give scores for them. The stated findings are:

• Online RL agents substantially outperform both SFT and terminal-only RL baselines.
• Training converges faster.
• The 8B RL agent shows nearly a sevenfold improvement over the SFT baseline in out-of-distribution generalization to Android.
• On real iOS applications the 8B agent outperforms Gemini-3.5-Flash and ties Gemini-3.1-Pro.

Treat these as the authors' claims. The Android number is about transfer: the agent was trained on iOS and tested on a platform it did not train on. A sevenfold gain over SFT may come from a low SFT baseline, so the absolute score matters, and we do not have it. The Gemini comparison depends on which tasks, which prompts and how the Gemini models were driven. Without the tables, we cannot verify those.

Claimed versus verified

You can check the claims by reading the full paper on arXiv and looking for per-task tables, the judge's prompt, and the baseline details.

Why does it matter outside Apple?

Three reasons.

Small agents are the economic path. If an 8B model can match a mid-tier frontier model on app navigation, running agents on-device or cheaply in the cloud becomes plausible. Apple's CEO has said the company will not ship AI for AI's sake; see our post on Ternus's Wired interview. A paper like this fits that stance: small, specific, useful.

It points at Siri. Apple is expected to let Siri delegate to other models, as in our report on Siri model delegation to Claude and ChatGPT. An agent that can operate apps would make such delegation more useful. The paper makes no such claim, and Apple has announced nothing, so this is our reading and not a fact.

The method generalizes. Dense rewards from a judge, plus sharing value information across similar states, can apply to desktop and web agents too. Android agents like the one in our Mika open-source agent post could benefit from the same training idea if someone reproduces it.

What should builders do with this?

You cannot download this agent today. You can still use the lessons:

1. Train in the live loop where you can. If your agent fails on layout changes, offline data will not fix it.
2. Reward progress, not only success. Step-level scoring speeds learning. Audit the judge so it cannot be gamed.
3. Split planning from grounding. Debug each part separately.
4. Measure transfer. Test on a second platform or app set, as Apple did with Android.
5. Wait for the paper details before copying hyperparameters.

Safety also deserves a note. An agent that taps through real apps can make purchases, send messages or change settings. Training on live apps raises the stakes of mistakes, so sandboxing and action limits matter. Our Claude background computer use post covers how a closed product handles permissions on the desktop.

What is still open

• Are the scores in the paper's tables as strong as the abstract suggests?
• How was the LLM judge protected from reward hacking?
• Will Apple release code, an environment or weights?
• How does the agent do on tasks with payments or logins, which live apps often block?
• Does it work offline and on-device, or only on a server?

We will update this post if Apple publishes more.

Related reading

• Alibaba OpenGCUA: open stack for computer-use agents
• Vercel agent-desktop and the accessibility tree
• Mika: open-source Android AI agent
• Apple Siri delegating to Claude and ChatGPT
• Apple CEO Ternus on not shipping AI for AI's sake
• Claude background computer use

Details are as stated on Apple's research page on November 2, 2026. Scores may differ in the full paper.