WORLD ACTION MODELS

ConGR-WAM Diagnosing Local Conflicts for Sparse Gradient Routing in World Action Models

Jointly learning visual dynamics and robot actions through sparse, conditional gradient routing.

Anonymous authors · Research manuscript

ROBOT MANIPULATION SHOWCASE02:06

LIBERO

98.8%

Average success +1.2 pp

ROBOTWIN 2.0

93.0%

Average success +1.2 pp

VIDEO BLOCKS ROUTED

2 / 30

Selected on LIBERO

ROUTED PARAMETERS

37.8M

10 video K/V tensors

Tables 1 and 2 · Gains over Fast-WAM in percentage points · RoboTwin average over clean and randomized settings

01 / MOTIVATION

Joint video and action learning

Global agreement can hide local conflict

Video and action gradients can agree across the model while conflicting within individual blocks. ConGR-WAM identifies a sparse subspace and adjusts its gradients only when conflict occurs.

Comparison of joint World Action Models and ConGR-WAM. Sparse conditional routing adjusts conflicting gradients in a selected subspace and retains the original sum elsewhere. — Figure 1 · Local conflict diagnosis and sparse routing in a joint World Action Model Click to enlarge

Abstract

Joint World Action Models learn visual dynamics and robot control together, yet global gradient alignment can conceal local conflicts between the two objectives. An analysis of video K/V parameters reveals a stable spatial pattern of conflict across training checkpoints and task suites, while conflict within a given subspace varies across optimizer steps. ConGR-WAM uses this structure to guide sparse gradient routing, separating where to intervene from when to intervene. Offline diagnosis constructs a conflict topology over block-pair subspaces with equal parameter budgets, yielding a sparse intervention subspace selected from gradient evidence through a tail-robust maximin criterion. During training, symmetric projection is applied only when the video and action gradients conflict within the selected subspace. Routing preserves the original gradient sum elsewhere and on nonconflicting steps. Controlled ablations show that routing in the diagnosed subspace outperforms random sparse and dense conditional alternatives, and that conditional activation improves over unconditional projection. On LIBERO and RoboTwin 2.0, ConGR-WAM achieves state-of-the-art performance among comparable World Action Models, while leaving the model architecture, parameter count, and inference procedure unchanged. Our project materials are available at the anonymous repository.

02 / METHOD

Offline diagnosis · Conditional training

Sparse gradient routing from local evidence

The routing subspace is selected before training. At each optimizer step, symmetric projection activates only if the video and action gradients conflict within that subspace.

ConGR-WAM pipeline with conflict diagnosis across block pairs, robust subspace selection and conditional gradient projection during training. — Figure 2 · The ConGR-WAM pipeline Click to enlarge

Diagnose

Compare 435 block pairs with equal parameter budgets using video and action gradients.

Select

Use a tail-robust maximin criterion across checkpoints and task suites to select one sparse subspace.

Route

Project conflicting gradients in the selected subspace. Preserve the original sum elsewhere and on aligned steps.

LIBERO selects video blocks 0 and 8. The subspace is selected independently for each benchmark.

Scatterplot showing globally aligned gradients that conflict within the selected block pair. — Figure 5 · From 1952 LIBERO diagnostic records

Local gradient alignment and conditional routing activity over sampled training steps. — Figure 8 · 498 activations in 2170 sampled steps

03 / IN MOTION

Recorded simulation rollouts

Simulation results

Single-arm and bimanual manipulation on LIBERO and RoboTwin 2.0

ConGR-WAM OURSSuccess

Transfers the microphone between grippers.

Fast-WAM BASELINEFailure

Does not complete the handover.

Full episodes · Original recording speed

Recording details

The main camera view is cropped from the original recording. The all-camera view shows the complete recording. Both views retain the full episode at its recorded speed. Each video ends at its own terminal frame.

All four comparisons show a successful ConGR-WAM episode on the left and a failed Fast-WAM episode on the right. The RoboTwin 2.0 recordings use similar scenes within the same setting. The LIBERO recordings use the same task and trial. Each panel is a separate recording. These selected examples do not estimate benchmark success rates. Playback duration does not measure inference latency.

Other recorded episodes

ConGR-WAM stacking Fast-WAM failed stacking ConGR-WAM drawer Fast-WAM drawer ConGR-WAM basket Fast-WAM basket

04 / RESULTS

Quantitative evaluation

Evaluation on two benchmarks

Architecture, parameter count and inference procedure remain unchanged.

40 tasks4 suites · 50 trials per task · 2,000 outcomes

98.8% average success

Table 1 · LIBERO success rate in percent
Method	Robot PT	Spatial	Object	Goal	Long	Average
OpenVLA	Yes	84.7	88.4	79.2	53.7	76.5
π₀	Yes	96.8	98.8	95.8	85.2	94.1
π_0.5	Yes	98.8	98.2	98.0	92.4	96.9
LingBot-VA	Yes	98.5	99.6	97.2	98.5	98.5
Motus	Yes	96.8	99.8	96.6	97.6	97.7
VITA	Yes	95.9	98.9	95.1	96.8	96.7
PFD	No	98.6	99.2	99.2	95.4	98.1
Fast-WAM baseline	No	98.2	100.0	97.0	95.2	97.6
ConGR-WAM ours	No	100.0	99.4	100.0	95.8	98.8

50 bimanual tasks2 settings · 100 episodes per task and setting · 10,000 episodes

93.0% average success

Table 2 · RoboTwin 2.0 success rate in percent
Method	Robot PT	Clean	Randomized	Average
π₀	Yes	65.9	58.4	62.2
π_0.5	Yes	82.7	76.8	79.8
Motus	Yes	88.7	87.0	87.8
Motus from WAN2.2	No	77.6	77.0	77.3
LingBot-VA	Yes	92.9	91.5	92.2
X-WAM	Yes	89.8	90.7	90.3
AttenA+WAM	No	93.1	91.9	92.5
Fast-WAM baseline	No	91.9	91.8	91.8
ConGR-WAM ours	No	93.2	92.7	93.0

Both clean and randomized settings use unseen language instructions. Average is calculated from unrounded clean and randomized rates, then reported to one decimal place.

Robot PT denotes robot pretraining before benchmark adaptation. Bold marks the best value in each metric column. Method references and full protocols are in the paper.

CONTROLLED ABLATIONS

Subspace and activation both matter

Diagnosed sparse routing reaches 98.8% on LIBERO. Random sparse pairs, dense conditional routing, and unconditional projection all yield lower success under the reported controls.

The selected subspace contains 37.8 million parameters across video blocks 0 and 8.

Table 4 · Routing controls on LIBERO
Routing variant	Parameters routed	Success
Random sparse R1 blocks 7 and 8	37.8M	96.7%
Random sparse R2 blocks 9 and 19	37.8M	97.4%
Dense conditional	566.5M	97.6%
Unconditional projection blocks 0 and 8	37.8M	96.9%
ConGR-WAM blocks 0 and 8 · on conflict	37.8M	98.8%

Training and inference cost COMPUTATIONAL COST

27.7 minutes

One-time diagnosis on four GPUs, including model loading and gradient collection on existing checkpoints; checkpoint training is excluded.

+13.7%

Training step overhead 8.3 s vs. 7.3 s under matched timing conditions.

6.0B

Policy-core parameters in both methods, excluding the frozen VAE.

Table 5 · Inference operators are identical. Measured inference batch runtime is 865.6 ms for ConGR-WAM and 863.3 ms for the baseline. Both measurements are inference batch runtimes.

PROJECT RESOURCES

ConGR-WAM

Project page assets, figures and recorded demonstrations.

Open repository