OpenGCUA: Alibaba Opens the Stack for Generalist Computer-Use Agents

Alibaba's Tongyi MAI team has released OpenGCUA, an open stack for what it calls generalist computer-use agents. The pitch, in the project's own words: "GPT-6 made computer use the frontier. We make it open." The first release is a research preview, and on the team's numbers it lifts eight frontier models by an average of 19.6 points on OSWorld-v2 while cutting the steps they need by 53 percent. Those figures are self-reported, so read them as the authors' claim until others reproduce them.

Quick reference

What is a generalist computer-use agent?

First-generation computer-use agents follow a loop: take a screenshot, pick a click or keystroke, repeat. That is slow and brittle. OpenGCUA argues the question has shifted "from how well a model operates a GUI to how well a system completes work on a real computer by whatever means works."

A generalist computer-use agent (GCUA) therefore uses more channels than a person would:

• It reads the accessibility tree instead of guessing from pixels.
• It calls app APIs and runs shell commands when those are faster.
• It operates the GUI only when nothing else will do.
• It writes GUI operations as code, combined with all of the above in a single program that has loops, branches and intermediate results.
• It can work in background windows so it does not interrupt what you are doing.

That mirrors where the products have gone. OpenAI's agent in GPT-6 Astra's launch and its contracting-agent deployment lean on mixed interfaces, and Anthropic's background computer use in Cowork and Code on macOS works in a similar spirit. The README names GPT-6 Astra's system as "a typical GCUA".

A hand placing a tile into a tray, an image for how a computer-use agent performs GUI actions step by step

How does the design work?

The project describes two ideas.

Code execution. GUI actions are written as code and combined with accessibility, API and shell operations in one program. Instead of one model call per click, a single generated program can loop through a list, branch on what it finds and only then return control. This is where the step reduction comes from.

Harness-to-model decision transfer. In older agents the harness decides when to take a screenshot or check state, using fixed rules. In OpenGCUA, "the harness provides the mechanism, and the model decides when to use it", for example requesting a screenshot only when it needs one. The model controls how to observe, how to act and how to organise its work.

What do the results show?

The authors tested on OSWorld-v2 with eight frontier models, including GPT-6 Astra, Claude Opus 5 and Gemini 3.8 Flash, using the offline set, plus real Macs with their harness.

Source: OpenGCUA README. Per-model baselines appear only in the project's chart.

Several points deserve care.

Weaker models gain most. The spread shrinking by more than half suggests the harness does much of the work that a stronger model would otherwise do by itself. That is good news for cheaper models and also a warning: part of the gain is the baseline being a simple GUI-only agent with batched actions.

The baseline matters. A 19.6-point gain is against the authors' own GUI-only reference, not against each lab's best shipped agent. For a view of how the closed products compare, see our write-up of the GPT-6 Astra browser-use benchmark versus Claude Opus 5.

Real-device results are tiny. On four macOS tasks, Qwen3.8-Max used 19.4 times the reasoning tokens of GPT-6 Astra and finished 3 of 4 tasks, while GPT-6 Astra finished 4 of 4. Four tasks is a demonstration, not a benchmark. The team lists real-device environments and a benchmark as future work.

How to try it

The preview ships a macOS source harness. According to the repository's quick start:

1. Clone the repo, create a Python 3.12 environment with uv venv --python 3.12, and install the SDK and runtime packages with uv pip install -e ./packages/sdk -e ./packages/harness/runtime.
2. Run mu setup macos and mu doctor macos, then set up a Chromium engine with mu setup browser --engine chromium.
3. Copy packages/harness/configs/macos_demo.json to local.json and add your API key, model name, base URL and protocol.
4. Grant the CuaDriver app Accessibility and Screen Recording access and restart it.
5. Start the agent with the runtime's mac_agent.py or the demo studio script.

Requirements include macOS 14 or later, Python 3.11 to 3.14, Xcode Command Line Tools, uv, and Rust 1.89 or later unless you use prebuilt SDK wheels. The native menu-bar app is a separate Apple Silicon build and is currently a local development build. There is also a browser-based interactive lab with seven demos, such as taking a Mac apart in 3D, comparing four ways to act side by side, and replaying real-device trajectories from five models.

What it means for builders

• You can test harness effects yourself. The published trajectories for eight models make it possible to study where agents waste steps without paying for the runs.
• Cheaper models become more viable. If the harness narrows the gap between models, a mid-priced model may be enough for routine desktop tasks. Verify on your own workflows.
• Cross-device control is the new frontier. The preliminary SDK connects computers, browsers and phones through one programming interface, so a single program can coordinate several devices in one task.
• Permissions are the real risk. An agent with Accessibility and Screen Recording access, shell commands and API calls can do real damage if it misreads a page or follows an injected instruction. Sandbox it, scope the accounts it uses, and consider a guard such as AgentBeam, which stops AI agents before they take dangerous actions.

What are people likely to ask next?

Is this a competitor to Claude or GPT-6 Astra computer use? Not directly. OpenGCUA is a harness that sits around a model, and the headline test runs those same closed models inside it. It is better read as an open reference design for how a product like theirs might be built, plus the data to study it.

Do I need a Mac? For the real-device harness, yes: the quick start is macOS-only for now. The OSWorld integration example is the route for people who want to evaluate the design in a standard benchmark setting rather than on their own machine.

Is it free? The code is Apache-2.0, so you can use and modify it. You still pay for whichever model API you plug in, and the project's own real-device test shows costs can differ sharply: one model used 19.4 times the reasoning tokens of another on the same four tasks.

Why release trajectories? Evaluation trajectories show every observation and action an agent took. Researchers can look at where a model loops, misreads a window or recovers, which a bare benchmark score never reveals. Publishing them for eight frontier models is arguably the most reusable part of the release.

Why does Alibaba care? The MAI-UI team already works on GUI agents, and the roadmap points to a dedicated GCUA model that unifies computer use and browser use. Opening the harness first invites outside contributors, in the team's words, to "come build it with us", before the model arrives.

Limits and open questions

• The results are self-reported on a preview release, and the macOS harness and Cross-Device SDK are labelled preliminary.
• The roadmap promises a GCUA model, efficiency work, a real-device benchmark, cross-device reliability and a native macOS app, but the project says it commits to no dates.
• The README compares against proprietary systems whose full designs are not public, so "typical GCUA" is the authors' interpretation.
• Token cost is untested at scale. Fewer steps does not automatically mean fewer tokens, as the 19.4 times reasoning-token figure for one model shows.

For more on what agents built on frontier models can actually do, see our GPT-6 Astra best demos showcase, and for the model that sits behind many of these agents, the Claude Opus 5.5 launch.

Related reading

• MAI-Gym: Tongyi-MAI's open GUI agent gym and 9B model

• GPT-6 Astra launch, benchmarks and pricing

• GPT-6 Astra browser-use benchmark v2 versus Claude Opus 5

• Claude background computer use in Cowork and Code on macOS

• OpenAI contracting agents with computer use

• GPT-6 Astra best demos

• Claude Opus 5.5 launch

Details are accurate as of October 16, 2026; the repository is an active preview and may change.