What Fast-WAM actually built, stated precisely

Fast-WAM (Tianyuan Yuan, Zibin Dong, Yicheng Liu, Hang Zhao; arXiv:2603.16666, v1 17 Mar 2026, v2 23 Mar) is a Mixture-of-Transformers built on the Wan2.2-5B video diffusion backbone with a 1B action expert of hidden width $d_a = 1024$, roughly 6B parameters in total. The token sequence carries three groups: clean latent tokens from the first observed frame, noisy latent tokens for the future video frames (training only), and action tokens.

Both branches are trained by flow matching. For a target variable y — either the action chunk $a_{1:H}$ or the future video latents $z_{1:T}$ — with interpolant $y_t = (1-t)y + t\varepsilon$,

$$L_{\mathrm{FM}}(y) = \mathbb{E}_{y,\varepsilon,t} \Big[\, \big\| f_\theta(y_t, t, o, l) - (\varepsilon - y) \big\|_2^2 \,\Big]$$

and the total objective is $L = L_{\mathrm{act}} + \lambda L_{\mathrm{vid}}$, where the numerical value of $\lambda$ is not disclosed anywhere in the paper.

The attention structure is the part that matters for everything downstream. In the authors' own words, "future noisy video tokens attend bidirectionally within the video branch and can access the clean first-frame tokens; action tokens attend bidirectionally within the action branch and can also access the clean first-frame tokens," and critically, "action tokens cannot attend to future video tokens, and the clean first-frame tokens do not attend to any other tokens." The stated rationale is that this "ensures that both video modeling and action prediction are grounded in the same visual context while preventing future information from leaking into the action branch." At inference the entire future-video branch is deleted, the video backbone runs one forward pass over the first frame, and the action expert denoises directly — an interface the authors describe as being like a VLA's.

The experimental design was a controlled four-way comparison. Fast-WAM (no test-time future), Fast-WAM-Joint (future video and action denoised together), and Fast-WAM-IDM (imagine, then infer the action) differ only in inference paradigm; a fourth variant keeps the architecture and inference identical but removes the video objective during training.

Variant	LIBERO avg	RoboTwin avg
Fast-WAM	97.6 (98.2 / 100.0 / 97.0 / 95.2)	91.8
Fast-WAM-Joint	98.5	90.6
Fast-WAM-IDM	98.0	91.3
Fast-WAM without video co-training	93.5	83.8

Latency on a single RTX 5090D V2 32GB was 190 ms for Fast-WAM against 810 ms for Fast-WAM-IDM. The real-world evaluation was towel folding on a Galaxea R1 Lite trained on 60 hours of teleoperation, where the no-co-training variant collapsed to 10% success. Training was 20k steps on LIBERO and 30k on RoboTwin, with action horizon 32, 4× temporal downsampling to 9 video frames per chunk, ten denoising steps, and classifier-free guidance at 1.0.

The asymmetry in that table is the whole argument: changing the inference paradigm moves the numbers by about a point, while removing the training objective costs four points on LIBERO, eight on RoboTwin, and essentially the whole real-world task.

---