The gain is four trials per task

A new paper from researchers at the University of North Carolina at Chapel Hill, Carnegie Mellon University, Shanghai Jiao Tong University and Amazon AWS AI proposes DenseReward, a vision-language reward model trained to score robot progress at every frame. The authors assemble 26,579 episodes and 7,560,942 frame-level samples from DROID, three simulators and deliberately generated failures, then fine-tune Qwen3-VL-4B-Instruct to predict a scalar reward.[1]

The real-robot result is concrete but small. DenseReward steers a frozen π0 policy through DSRL, a method that changes a diffusion policy's latent noise without updating the base model. After about 20 training rollouts for stacking cups and 10 for putting a ball in a basket, the final policies were evaluated for 10 trials each. The project page reports success moving from 40% to 80% and from 30% to 70%. With 10 trials per condition, those percentages mean four additional successes in each test: four to eight and three to seven.[1,2,4]

That is evidence that the learned reward can guide low-budget adaptation on these two tabletop tasks. It is not yet a reliable estimate of general robot performance. The paper reports no confidence intervals for the 10-trial tests, no independent reproduction and no real-world comparison across longer-horizon tasks, tools or materially different failure distributions.[1,2]

Failure synthesis supplies the supervision

DenseReward's core move is to manufacture structured negative experience rather than wait for robots to fail in the lab. Tasks are split into five phases—reach, grasp, lift, move and place—and simulator scripts introduce collisions, misses, dropped objects, smooth or jittery motion and recoveries. Handwritten phase rules assign dense reward curves to the resulting trajectories. The model therefore learns from engineered notions of progress, not direct human preferences or an open-ended discovery of what counts as success.[1]

On the authors' held-out reward-prediction suite, DenseReward records mean absolute error of 0.081, compared with 0.230 for RoboReward-8B and 0.366 for VLM-RM. RoboReward, an earlier reward-model system, instead creates counterfactual failures by relabeling and temporally clipping mostly successful robot data. The comparison supports failure synthesis as a useful data strategy inside this evaluation, but both systems remain author-defined reward benchmarks rather than independent measures of downstream reliability.[1,3]

A separate model-predictive-control test is narrower than task completion. It covers three objects in Isaac Lab, fixes object orientation and evaluates 10 episodes per object. The reported score is the minimum final distance between the robot end effector and the object, not whether the object was successfully manipulated. DenseReward's average distance is 0.229, better than the listed baselines at 0.267 to 0.358, but it should not be read as a completed-task rate.[1]

The promised release is not downloadable yet

The paper and project page say the dataset, trained reward models and evaluation suite are released. The official page does not currently make those artifacts accessible: its arXiv, code and dataset buttons point to empty page anchors rather than a paper record, repository or download. That does not prove the artifacts do not exist elsewhere, but the authors' own release page cannot presently support a public rerun from its listed links.[1,2]

The next proof is measurable: activate versioned code, model and dataset links with licenses; report uncertainty over substantially more real trials; and test the reward model on new tasks, objects and failure modes against human feedback and independent baselines. Until then, DenseReward is a promising demonstration of synthetic-failure supervision, not evidence that dense reward learning has been solved for general-purpose robots.[1,2,3,4]