Lab Notes: Post-training Nemotron 3.5 Super VL 120B‑A12B for computer use
Methods and findings from post-training the new NVIDIA 120B-A12B vision model.
Research · October 15, 2026 · 5 min read
By Aleix Cambray, Frederic Renard, Daniel Beechey, Vincent Coyette, Oliver Bolton, Mats Richter, Michael Eickenberg, Tony Wu
NVIDIA released Nemotron 3.5 Super VL today, a 120B-A12B Mixture-of-Experts model with a hybrid Mamba-attention backbone and native vision. As members of the NVIDIA Nemotron Coalition, we have spent the past few weeks post-training Nemotron 3.5 Super VL for computer use.
Our latest checkpoint, Holotron4-Super-Preview, improves on the base model across computer-use tasks involving GUI, MCP/API and code use: OSWorld (+25.1), OSWorld 2.1 (+12.0) and AutomationBench (+17.9).
Benchmarks
OSWorld
OSWorld 2.1
AutomationBench
Means of three runs per model, with ± 1 standard error; gains in percentage points. OSWorld 2.1 uses partial credit; AutomationBench uses the 600 public non-simple tasks pending evaluation on the official private set.
In this first edition of our new Lab Notes series, we share the initial recipe behind it (SFT, weight merging and RL) and early results from each stage. On the long-horizon OSWorld 2.1 benchmark, SFT and merging with the base model are complementary: the SFT checkpoint scores below the base model, but a merge of the two scores more than 10 points above both.
The recipe
(opens full size in a new tab)Our recipe: SFT, a merge with the base model, and two RL experts merged back in.
Our recipe starts with supervised fine-tuning (SFT) on agentic trajectories and multimodal data to strengthen the model's broad computer-use skills. We then merge the SFT checkpoint with the base model to improve long-horizon performance, taking the attention query and key projections from base (QK-restore). From that merge, we train two reinforcement-learning (RL) experts, one for GUI use and one for MCP tool calling, and merge the weight changes each expert learned back in.
SFT
Our SFT mixture holds about 109B tokens. Most of it is successful agentic trajectories across desktop, web, MCP/API tool use and coding. These come largely from our Agentic Task Factory, plus NVIDIA Nemotron post-training data and human demonstrations. The rest is multimodal data, such as GUI grounding, VQA and OCR.
We followed the NVIDIA Nemotron 3 Super SFT recipe: an auxiliary-loss-free load balancing strategy, a standard load balancing loss, and no weight decay on the Mamba state-space parameters. With these settings, scores improved on OSWorld, AutomationBench and our internal test sets, but not on OSWorld 2.1.
Merging with base
On the long-horizon OSWorld 2.1 benchmark, SFT fell 3.4 points below the base model, despite gaining 22.5 points on OSWorld and 19.7 on AutomationBench.
To recover the base model's long-horizon score, we linearly combine the two checkpoints, with no additional training: we average the SFT and base weights in equal parts, then take the attention query and key projections entirely from base (QK-restore).
Merging with base: every weight is averaged from base and SFT, except the attention Q/K projections, which come entirely from base.
This base merge scores 39.4 on OSWorld 2.1, more than 10 points above either checkpoint on its own. In exchange, merging loses some of SFT's gains on the shorter benchmarks.
Score after each step: SFT, then the merge of SFT with base. Means of up to three runs; OSWorld 2.1 uses partial credit.
RL experts
From the base merge, we train two experts as LoRA adapters: a GUI expert for desktop and web tasks, and an MCP expert for tool-calling tasks. Both use asynchronous RL with our internal fork of prime-rl, optimized for computer use, on long-horizon tasks from the SFT task pool. Splitting by domain lets each expert specialize, and lets us tune GUI and MCP training separately to maximize throughput: GUI rollouts include screenshots, so their contexts are far longer.
Because rollouts on virtual machines can last up to an hour, we continuously sample training batches from a large, persistent pool of in-flight rollouts. We train on unscaled group-relative advantages, as in Dr. GRPO, but average the loss within each rollout, as in the original GRPO, so that long rollouts do not dominate the gradient. To correct for drift between the rollout and training policies, we use importance sampling and enforce a trust region on their total variation distance, as in DPPO.
Rewards measure binary success in some domains and partial progress in others. Since groups with more varied rewards carry more signal, our curriculum samples each task in proportion to its reward variance over recent attempts.
Mean success rate across each expert's training tasks.
Merging the experts
Each RL expert improves on the benchmark in its domain, the GUI expert on OSWorld and the MCP expert on AutomationBench, but both score lower than the base merge on OSWorld 2.1.
To combine their capabilities, we merge both experts into one checkpoint: we add each expert's learned weight change to the base merge, the GUI expert's in full and the MCP expert's at half weight.
The resulting model, Holotron4-Super-Preview, outperforms the GUI expert on OSWorld (77.0), retains most of the MCP expert's gain on AutomationBench (40.9) and the base merge's score on OSWorld 2.1 (40.4).
Score of the base merge, each RL expert on its own, and the final merge of both experts. Means of three runs; OSWorld 2.1 uses partial credit.
Conclusion
Each stage of the recipe builds on the one before. SFT strengthens the model's computer-use skills, merging with the base model raises its long-horizon score above both, and the RL experts recover most of what that merge costs on the shorter benchmarks. Together, they produce a single checkpoint that improves on the base model across every benchmark we report.
Next, we are looking more closely at why a stage of training can raise scores on shorter tasks while lowering them on long-horizon ones, and how merging recovers them.
We want to congratulate the Nemotron team on the release, and thank them for the early access and the technical discussions along the way.
