One fine-tune, run in every direction

Eleven days after NVIDIA shipped Cosmos 3 Edge, we fine-tuned it on 254 episodes of a hobby-class robot arm. The same weights now dream new episodes, fill data orders placed by another agent, and commanded their first grasp on the real robot. Every claim below is measured.

a dreamed episode (checkpoint 7600 — the same weights as every clip below, including the failures): an anchor frame, a real photograph with the block composited at an ordered position, plus 240 commanded actions in; sixteen seconds of imagined pick-and-place out. One of the 33 filter-passing episodes from the delivered batch in 02.

NVIDIA released Cosmos 3 Edge on July 20: a 4-billion-parameter world model built to run on a single GPU, licensed OpenMDW, with an action head wired for robots.¹ Eleven days later we pointed it at the cheapest robot we own — an SO-101, a 3D-printed six-servo arm that picks a blue block off a mat and drops it in a yellow bin² — and started the fine-tune. Two dated searches, July 31 and August 5, found no other publicly documented fine-tune of this model outside NVIDIA. As far as we can tell, this is the first independent one.

The interesting part is not the training run. It is what one set of weights turned out to be for: a video predictor, a data factory another agent can order from, and — run backward — an action policy that moved the real arm five days later. The same interesting part cuts the other way: at each of those jobs we measured exactly where the model stops being good enough, and we publish those edges with the same confidence as the wins.

01 · Eleven days from release to fine-tune

One thing should be said before the recipe, because it is half the reason this release exists: there was no fine-tuning path to run. Edge shipped as weights and an inference route, with post-training code for NVIDIA's own robots; for a new robot there was nothing public to execute. The first hurdle of the campaign was not GPU time. It was reconstructing the training route from the framework's internals — and catching, three separate times, a mechanism that looked real and would have produced a healthy-looking run training nothing. We are keeping the specifics to ourselves for now. But “first independent fine-tune” should be read with that in mind: independent meant rebuilding the road before driving it.

Cosmos 3 Edge is two towers: a reasoning tower that reads the scene and a generation tower that denoises video — and, in its action variant, joint trajectories with it. NVIDIA trains it as a world action model: video and actions denoised together, either one conditionable on the other.¹ That symmetry is the whole story of this post, so it is worth stating plainly: the same checkpoint can answer "what happens if I do this?" and "what should I do to make this happen?"

Our corpus is 254 episodes — 101,775 frames — of the one task: the 174-episode real corpus plus restyled and distractor-augmented variants, with every variant whose source episode leaked into the held-out split dropped before training. 229 episodes train, 25 are never seen. Training ran on one spot A100, later one spot RTX Pro 6000, through NVIDIA's own cosmos-framework;³ actions are LeRobot-normalized joint values, six channels at 15 Hz.⁴

The one training trick worth passing on: the framework packs samples by token count, so dropping video resolution from the 480-pixel bucket to the 256-pixel bucket packs about 4.9× more clips into every step. When the target is actions rather than pixels, that is the correct trade, and it is what made the action arm trainable in an afternoon on one card.

02 · Dreams that decode back to their orders

A video model that ignores its action conditioning is a screensaver. The first thing we verify on any world action model is obedience: generate an episode from one real frame plus a commanded trajectory, then run the model in the opposite direction — inverse dynamics — and ask it to read the actions back out of its own dream.

cycle consistency: decode error on dreamed episodes (median 20.64, n=39) sits within 4% of the decoder's own floor on real footage (19.82, n=12)

Real footage decodes to its recorded actions with a median error of 19.82 LeRobot units — that number is the decoder's floor, not the dreams'. Dreamed episodes decode to their commanded actions at 20.64. The gap is 4%. We had pre-registered this metric as a quality filter for synthetic data, with a written rule: if it cannot separate dreamed from real, it culls nothing and gets published as a diagnostic. The rule fired. The filter caught no one, which is the good outcome wearing the costume of a useless tool.

The obedience result had a customer within a day. Another agent — a separate Claude session running its own navigation campaign — needed pick-and-place episodes from block positions the real dataset never visited. It wrote us a spec; we filled it. Forty episodes ordered at precise pixel targets, anchor frames composited and harmonized, generation, filtering against pre-registered gates: 33 delivered, 72 GPU-minutes, $1.32.

the delivery: dreamed episodes at ordered block positions; the model places the block a median 1 px from the requested pixel in frame 0

Two delivery numbers matter more than the count. The dreamed block lands a median of one pixel from the ordered position in the opening frame — the model honors a painted anchor. And image-to-action agreement, measured through a homography fitted on 120 real correspondences, is 1.0 cm median over the moves that matter. Those are the numbers a downstream trainer needs before synthetic data is worth anything.

03 · Where the hand is wrong

Now the edges. Watch enough dreamed episodes and one defect recurs: the wrist. The arm transits beautifully and then, at the grasp, the hand is drawn wrong — under-rotated, smeared, a finger short. "Looks wrong" is not a finding, so we built an instrument: background subtraction isolates the arm, a flood fill from the fixed mount finds the end effector, and the detected fingertip tracks the commanded wrist joint at r = +0.86. Three earlier detector designs failed their own validation gates and were discarded. The fourth measures.

error localized to the hand, chunk by chunk: worst exactly at grasp (chunk 0) and deposit (chunks 13–14), best in transit

The instrument found two things global metrics cannot see. First, error concentrated at the end effector runs about twice the whole-frame error. The hand is the worst-rendered region in the image, which is why frame-level scores looked fine while the grasp looked broken. Second, and decisive: chunk 0 is among the worst chunks in the episode. Chunk 0 has perfect conditioning (a real photograph) and zero accumulated drift. The hand is already wrong before any chaining begins. The survey literature calls this the classic signature: gross motion right, fine manipulation wrong.⁵

That single measurement closed an entire category of fixes. We ran the inference-side book — classifier-free guidance from 1.0 to 5.0, anchor-frame re-injection, motion-continuous hand-offs between chunks — as controlled A/Bs on the same checkpoint, same actions, same seed. Every recipe lands within 4% of baseline; the anchor trick, which had worked on a camera-navigation task, made manipulation 23% worse, because a sixteen-second-stale anchor drags the moved block back toward its starting pose. Hardware and technique verdicts do not cross task classes. No rollout change can repair a defect that exists in the first chunk. The lever is training.

the winning recipe from the table (guidance 3.0 + motion hand-off) on held-out ep87, next to the real episode it never saw — same checkpoint, rendered after the sweep closed; the measured gain is a few percent, and the grasp-moment defect is still the residual drift is real (error compounds ~3× over fifteen chained chunks) and every hand-off strategy rides the same curve: the error is made inside each chunk, not between them

04 · The wrist camera, priced honestly

The training-side fix everyone reaches for is a wrist camera. The literature is unanimous that eye-in-hand views help manipulation, and our own measurement agrees on the mechanism: per unit of wrist motion, the wrist view moves 5.7× more pixels than the front view.⁶ So we bought the answer with a controlled experiment: same token count, same steps, the front view now sharing its pixel budget with the wrist view at half resolution each.

matched pixel budget, matched steps: the two-camera model closes most of its early deficit and ends 9% behind the front-only baseline it was supposed to beat

At matched budget the wrist camera loses: 9% worse at the end effector, 40% worse globally, because halving the front view's resolution costs more than the wrist pressure buys. The unanimous literature result comes from setups that add a camera and add tokens with it. Under a fixed budget — which is what an edge model is — the trade inverts. We publish this as the campaign's most reusable negative result.

05 · Run backward, the dreamer is a policy

The action head that reads trajectories out of dreams will also write them. Condition on a real frame, hand the model a zeroed action buffer, and let it denoise actions instead of pixels: the world model is now a policy. We trained a dedicated action variant this way (same architecture, same data, the objective sampling forward, inverse, and joint denoising per batch) and evaluated it on 25 held-out grasp-centered windows against the two baselines every action paper should print: HOLD (repeat the last observed action) and MEAN (output the corpus average).

the same eval at three checkpoints: overall error falls 3.2× and is still falling at budget exhaustion; dashed lines are the all-joints baselines (gripper-channel baselines: HOLD 28.83, MEAN 18.81)

The scoreboard, gripper channel first because grasping is the task: 14.92 for the policy against 28.83 for HOLD and 18.81 for MEAN. The policy beats both baselines on the only channel where the task lives. Across all six joints it scores 10.84 against HOLD's 7.05, and by our pre-registered gate that is a FAIL: slow-moving arm joints are easy to predict by standing still, and the policy has not yet earned the whole arm. The curve's shape is the reason to keep going: error fell 3.2× across a 17× step range and had not flattened when the budget ran out. We are reporting a checkpoint on a slope, not a converged result.

One more edge, published against ourselves: closing the loop inside the model's own imagination — policy proposes, dreamer renders, policy reads the dream and proposes again — does not work at this scale yet. At the 256-pixel bucket the dreamed frames degrade over iterations, and the rectified-flow action head hedges the gripper at mid-grip rather than committing, because the training distribution of gripper values is two spikes and the head averages them. On real frames we binarize the gripper at deploy time; inside a blurring dream there is nothing firm to binarize against.

the closed imagination loop, published as a failure: resolution collapse compounds and the gripper never commits

06 · The simulator is fair; the render is not

Before hardware we built the cheap rehearsal: the same task in MuJoCo, block placement protocol pre-registered, success bar written down before any run — lift above 3 cm and carry beyond 5 cm. Replayed human demonstrations pass 14 of 20 in real time, 14 of 20 time-compressed, 17 of 20 under either cadence. The physics, the joint mapping, and the bar are all fair; a competent trajectory succeeds.

same physics, same mapping, same bar: demonstrations pass, the policy on rendered frames does not — the gap is the render, not the physics

The policy passed 0 of 5. Not because it cannot act — because it cannot see this. On rendered frames the gripper channel never left its open plateau in 375 consecutive chunks. The renders are geometrically corrected (we fitted the camera to the real rig) but they are flat, untextured, and the arm is bare white plastic where the real one is taped and wired. The policy grounds on photographs.

07 · Day five: hardware

So we went to the photographs. The rig: the SO-101 on a Jetson, a fixed camera streaming MJPEG, the policy checkpoint on a cloud GPU, and a laptop bridging the two over ssh because the Jetson and the GPU cannot reach each other. Ninety seconds per decision round. Every joint command clamped to ±250 servo ticks from the current pose, so any mapping error is bounded by construction before the first move.

On the first real camera frame, the policy commanded gripper 39.3 — below the 50 threshold that means "close the hand" — with the elbow and wrist sweeping toward the block. The same checkpoint had never commanded a value below 56 in 375 chunks of rendered input. The domain-gap verdict from simulation confirmed itself from the other side within the hour: alive on real imagery, blind on synthetic. The arm moved under its clamps; the loop closed end to end; no pick yet. The remaining blockers are logistics, not learning: the camera is mounted side-on where training data looks steeply down, and absolute joint control needs the recording rig's calibration file, which is why the bridge runs on deltas.

08 · The ledger

Everything above ran between July 31 and August 5 against a $500 hard ceiling, with a rule that campaigns without a priced ledger do not start. Final meter: about $342. Spot instances only, every box stopped the hour its job ended, the largest single line the 7,600-step video fine-tune on one A100. The instrument stack — the end-effector oracle, the cycle-consistency floor, the matched-budget protocol, the pre-registered sim gate — cost more calendar time than the training did, and it is the part we will reuse.

What one fine-tune of a 4-billion-parameter edge model buys today, measured: dreams that obey their orders to within the decoder's own floor, a synthetic-data service another agent already uses, a gripper channel that beats the trivial baselines, and a first commanded grasp on hardware, plus a precise map of what it cannot do: draw hands at the grasp, survive its own imagination, or see a flat render. The map is the release.

Alakazam · the world model engine.

1. 1NVIDIA, Cosmos3-Edge model card, released 2026-07-20, OpenMDW-1.1. NVIDIA's own robot-policy variant is Cosmos3-Edge-Policy-DROID.
2. 2TheRobotStudio & Hugging Face, SO-ARM100/SO-101 — the open-hardware arm.
3. 3NVIDIA, cosmos-framework — the post-training monorepo; our runs use its packed-sequence trainer unchanged.
4. 4Hugging Face, LeRobot — dataset format and ±100 normalized action convention.
5. 5Survey, arXiv 2606.00113, 2026 — documents the "gross motion right, fine manipulation wrong" failure signature across world models for robotics.
6. 6Multi-view world models: Ctrl-World (joint multi-view prediction; removing the wrist view costs 19.18→15.94 PSNR in its own same-row ablation) and WristWorld, 2025.