Your Phone's Next Assistant Won't Be a Giant — Apple's Tiny 8B Agent Just Matched Gemini Pro at Tapping Real Apps

Ask today's best AI to use your phone and it will probably do something impressive — until it taps the wrong button, gets stuck on an unexpected pop-up, and quietly gives up. The demo looked flawless because it was rehearsed. Real apps are not rehearsed: buttons move, dialogs appear out of nowhere, and yesterday's screen is never quite today's screen.

That gap is exactly what a new paper from Apple researchers attacks. Their answer is counterintuitive: instead of building a bigger model, they built a smaller one and trained it the hard way — letting it practice on live iPhone apps, hundreds of devices at a time, with a judge scoring every step. The result is an 8-billion-parameter agent that, on real iOS applications, outperformed Gemini-3.5-Flash and tied Gemini-3.1-Pro.

The paper is "Online Dense Reinforcement Learning with Planning-Grounding Agent for UI Navigation on Live iOS Apps" by Peitong Duan, Khashayar Gatmiry, Jeffrey Nichols, Samy Bengio and Emmanuel Abbe (arXiv:2610.39264, submitted 29 October 2026). Here is what they did, why it matters, and how you can steal the pattern for your own agents.

What the researchers did

Today's phone agents mostly come in two flavours, and both have a flaw. Giant frontier vision-language models are capable, but their inference cost makes them too expensive for everyday use — you cannot put a flagship model behind every tap. Smaller models, trained by supervised fine-tuning or offline reinforcement learning on recorded trajectories, are cheap — but they memorise the recordings. Show them a new screen, or let them make one wrong tap, and they fall apart because they never learned to recover.

Apple's team changed the training environment instead of just the model. Their framework trains UI navigation agents directly on live iOS applications, not on replays, and splits the agent into two specialised parts:

1. A Planning agent. It looks at the screen, remembers what it has already done, and reasons — with chain-of-thought — about both the immediate next step and the longer-term plan. It decides what to do: "open Settings, find the toggle, confirm."
2. A UI Grounding model. It turns that intention into a precise action: which element to tap, and exactly where. Planning and grounding are different skills — one is thinking, the other is pointing — and separating them lets each get good at its own job.

The training signal is the clever part. Instead of only rewarding the agent at the end ("did it finish the task?"), the team uses an LLM-as-a-Judge to give dense, step-level feedback on intermediate progress, combined with a terminal reward for overall completion. The agent is optimised with Proximal Policy Optimization (PPO) using step-level advantage estimation, so a good move halfway through a task gets credit even if the task later fails — and a bad move gets blamed immediately.

They also introduce GiGPPO, a neighbour-conditioned value estimation method that stabilises the critic by borrowing reward signals from visually similar states across concurrent trajectories. To run all of this at scale, the team built infrastructure orchestrating hundreds of concurrent virtual iOS devices — the first time, the authors say, UI navigation agents have been trained at scale entirely on live iOS devices and real iOS applications.

Why it matters

The headline result is about size versus skill. The paper's 8B RL agents achieved nearly a sevenfold improvement over the supervised fine-tuning baseline in out-of-distribution generalisation to Android environments — a platform the agent was never trained on — while outperforming Gemini-3.5-Flash and tying Gemini-3.1-Pro on real iOS applications.

Reported benchmark figures make the picture sharper. On UINavBench, a benchmark of real iOS applications, the RL-trained 8B model scored 32.4%, matching Gemini-3.1-Pro's 32.4% and beating Gemini-3.5-Flash's 28.4%. On AndroidWorld, the iOS-trained agent reached 31.9% task success against the fine-tuning baseline's 4.7%. But honesty matters too: on iOSWorld, the same system scored only 9.8% against Gemini-3.1-Pro's 39.5%. Parity on one real-app benchmark does not mean parity everywhere.

Still, the direction of travel is what matters for builders. The online RL agents substantially outperformed both fine-tuning and terminal-only RL baselines while converging faster. Three lessons fall out of that:

Small can win when training is live. An 8B model trained on live apps matched models many times its size on real iOS tasks — the difference between a feature you can ship and a demo you cannot afford.

Step-level feedback beats pass/fail. A terminal reward teaches what success looks like; dense feedback teaches how to get there and recover when it strays.

Generalisation is the real test. A model trained only on iOS transferred to a different operating system far better than its fine-tuned counterpart. If your agent only works on screens it trained on, it does not work — it remembers.

How YOU can use it

Apple's phone farm and judge are research infrastructure, not something you can download today. But the pattern behind them is directly usable in any agent you build — phone, web, or desktop. The code sketch below is illustrative, not code from the paper.

Step 1: Split the thinker from the pointer

Stop asking one prompt to do everything. Give your agent two roles: a planner that outputs the next intent in plain language ("tap the search icon, type the query"), and a grounder that resolves that intent to a concrete element on the current screen. When something goes wrong, you will instantly know which half failed — a bad plan or a bad tap.

ILLUSTRATIVE sketch — not code from the paper:

# Planner: decides WHAT to do next (intent, in words)
plan = planner.decide(screen=screenshot, history=actions_so_far)
# → "Open Settings, scroll to Privacy, toggle Location Services"
# Grounder: decides WHERE to act (element + coordinates)
action = grounder.resolve(intent=plan.next_step, screen=screenshot)
# → tap(element_id="privacy_toggle", x=842, y=315)
# Never let the planner emit coordinates, and never let the grounder invent the plan. Log both, separately, every step.

Step 2: Score every step, not just the finish

Most agent evaluations are pass/fail, which throws away the most useful information. Borrow the dense-reward idea in lightweight form: after each action, have a judge score whether the agent is closer to the goal, further away, or stuck (+1 for progress, 0 for a harmless no-op, −1 for moving away or repeating a failed action). Review the −1 steps weekly; they are your training data.

Step 3: Test on live apps and a foreign platform

Recorded test scripts flatter agents. Once a week, run your agent against the live version of the apps it must handle — real logins, real pop-ups, real delays. Then run the same tasks on a platform it was not built for (Android if you built for iOS). The gap between your home-platform score and the foreign-platform score is your generalisation gap, and closing it is worth more than any single benchmark win.

One guardrail before you ship anything like this: a phone agent that can tap anything can also buy, send, delete, and share anything. Keep a human-confirmation gate on irreversible actions (payments, messages, deletions) no matter how good the agent gets.

Key takeaways

Frontier phone agents are capable but expensive; small fine-tuned agents are cheap but brittle because they memorise recordings instead of learning to recover. Apple's 8B agent, trained with online RL on live iOS apps with a planner, a grounder, step-level judge rewards and GiGPPO, tied Gemini-3.1-Pro on real iOS apps (UINavBench 32.4%) and transferred to Android far better than its fine-tuned baseline (31.9% vs 4.7%) — while still trailing badly on iOSWorld, a reminder that no single benchmark tells the whole story.

The practical lesson is portable: split planning from grounding, score every step, train and test on live apps, and measure generalisation on a platform you did not train on. The next great phone assistant probably will not be the biggest model in the room. It will be the smallest one that learned by actually using the phone.

Read the original paper: Online Dense Reinforcement Learning with Planning-Grounding Agent for UI Navigation on Live iOS Apps (arXiv:2610.39264) by Peitong Duan, Khashayar Gatmiry, Jeffrey Nichols, Samy Bengio and Emmanuel Abbe. Link: https://arxiv.org/abs/2610.39264