Matrix/Policy·Imitation·Columbia University and Toyota Research Institute (Chi, Xu, Feng, Cousineau, Du, Burchfiel, Tedrake, Song)
Diffusion Policy
Visuomotor policy that represents the action distribution as a conditional denoising diffusion process, predicting a sequence of future actions from recent observations and executing it with receding-horizon control. LeRobot's implementation uses a ResNet-18 image encoder and a 1D convolutional U-Net denoiser.
open weightsverified 2026-10-24paper →
Architecture
Compute requirements
| Memory | Precision | Conditions | |
|---|---|---|---|
| train | ≥8 GB · ≤14 GBsrc | — | Peak training VRAM at batch size 8 with AdamW (LeRobot default), "Diffusion" group (diffusion, multi_task_dit) in the LeRobot compute hardware guide. The guide calls its figures indicative, order-of-magnitude numbers for picking hardware, not exact predictions. Precision not stated. · batch 8 · 5 epochs on a ~45k-frame dataset (640×480): ~2–4 h at batch 8 on a single RTX 4090/3090; ~30–60 min at batch 32 on a 4× H100 80 GB cluster (indicative, ±50%). Measured anchor in the same guide: 5000 steps, batch 32, 4× H100, 16m 17s. |
| inference | 1.12 GB measuredsrc | fp32 | Peak memory at inference measured by the LeRobot paper (Table 2): 1.12 GB on both RTX 4090 and A100, full fp32, 10 denoising steps. (Same table: 1.22GB CPU-only, 224MB on Apple MPS.) |
Training requirements: see also the LeRobot compute hardware guide, the authoritative source for LeRobot training envelopes.
Measured inference 0 runs · inference cost only, not task success
No measured runs yet. Submit one with its runtime, precision and camera count.
Robots running Diffusion Policy 20 of 24 with evidence
| Robot | State | Tier | Verified | Detail |
|---|---|---|---|---|
| AgileX PiPER | works with caveats | T3 · community reported | 2026-10-25 today | Community-reported only; not documented upstream or maintainer-tested. |
| ALOHA (stationary) | works with caveats | T3 · community reported | 2026-10-24 1d ago | Community-reported only; not documented upstream or maintainer-tested. |
| ARX5 | works with caveats | T3 · community reported | 2026-10-25 today | Third-party research paper (UMI on Legs, from the Diffusion Policy authors' lab), not documented in Diffusion Policy's own docs or in LeRobot; the ARX5 LeRobot plugin documents no Diffusion Policy run. |
| Bimanual SO-100/SO-101 | partial | T3 · community reported | 2026-10-24 1d ago | Training evidenced by public checkpoint(s); on-robot inference not documented by a primary source. |
| Franka Research 3 | partial | T3 · community reported | 2026-10-25 today | Training evidenced by public checkpoint(s); on-robot inference not documented by a primary source. |
| Franka Emika Panda | works with caveats | T1 · upstream documented | 2026-10-24 1d ago | The LeRobot route is community-reported only: the LeVR/LeFranX authors fine-tuned LeRobot's Diffusion Policy on a Panda (FER) with a RobotEra XHand, using the LeFranX fork, which has no Franka Hand gripper support. |
| I2RT YAM | partial | T3 · community reported | 2026-10-24 1d ago | Training evidenced by public checkpoint(s); on-robot inference not documented by a primary source. |
| Koch v1.1 | partial | T3 · community reported | 2026-10-24 1d ago | Training evidenced by public checkpoint(s); on-robot inference not documented by a primary source. |
| LeKiwi | partial | T3 · community reported | 2026-10-25 today | Training evidenced by public checkpoint(s); on-robot inference not documented by a primary source. |
| Mobile ALOHA | works with caveats | T1 · upstream documented | 2026-10-24 1d ago | With 50 demonstrations Diffusion Policy was less precise than ACT on Wipe Wine (65% success); the authors attribute this to too few demonstrations. |
| ROBOTIS OMX | partial | T3 · community reported | 2026-10-24 1d ago | Training evidenced by public checkpoint(s); on-robot inference not documented by a primary source. |
| OpenArm | works | T1 · upstream documented | 2026-10-24 1d ago | |
| SO-100 | works with caveats | T3 · community reported | 2026-10-25 today | Community-reported only; not documented upstream or maintainer-tested. |
| SO-101 | works with caveats | T3 · community reported | 2026-10-24 1d ago | Community-reported only; not documented upstream or maintainer-tested. |
| Trossen WidowX AI | works with caveats | T3 · community reported | 2026-10-24 1d ago | Community-reported only; not documented upstream or maintainer-tested. |
| Unitree G1 | works with caveats | T1 · upstream documented | 2026-10-24 1d ago | Documented by Unitree in its unitree_IL_lerobot repository, which pins LeRobot as a git submodule (commit a5b29d4, November 2025) and deploys with its own eval_robot/eval_g1.py script, not through LeRobot's native unitree_g1 robot or lerobot-rollout. |
| Universal Robots UR5e | works with caveats | T3 · community reported | 2026-10-24 1d ago | Community-reported only; not documented upstream or maintainer-tested. |
| WidowX 250 S | partial | T3 · community reported | 2026-10-24 1d ago | Training evidenced by public checkpoint(s); on-robot inference not documented by a primary source. |
| UFACTORY xArm | partial | T3 · community reported | 2026-10-24 1d ago | Training evidenced by public checkpoint(s); on-robot inference not documented by a primary source. |
| XLeRobot | partial | T3 · community reported | 2026-10-25 today | Training evidenced by public checkpoint(s); on-robot inference not documented by a primary source. |
No verified evidence, but both are available in LeRobot (ecosystem context): EarthRover Mini Plus, HopeJR, Reachy 2, reBot Arm B601-DM.
Framework support
| Framework | Support | Since | Type string | Docs |
|---|---|---|---|---|
| LeRobot | nativesrc | — | diffusion | docs |
| LeRobot: Present at v0.3.2, the earliest tagged LeRobot release. `since` left blank: the only earlier PyPI release (0.1.0, 2024-03, a prototype whose README mentions diffusion_policy) was not inspected. Defaults: horizon 64, n_action_steps 32, n_obs_steps 2. | ||||
Checkpoints
- lerobot/diffusion_pusht · PushT simulation checkpoint (Apache-2.0 on the Hub).
Related terms
Action chunking · Action horizon · Behavioral cloning · Diffusion policy · Imitation learning · LeRobotDataset
Sources
- T1 · upstream documentedpapersource 2023-03-07verified 2026-10-24 (1d ago)
Diffusion Policy models the visuomotor policy as a conditional denoising diffusion process over action sequences.
“This paper introduces Diffusion Policy, a new way of generating robot behavior by representing a robot’s visuomotor policy as a conditional denoising diffusion process.”
https://arxiv.org/abs/2303.04137 - T1 · upstream documentedsource-codesource 2026-08-03verified 2026-10-24 (1d ago)lerobot 0.6.1
Policy type string is `diffusion`; defaults resnet18 backbone, horizon 64, n_action_steps 32.
“@PreTrainedConfig.register_subclass("diffusion")”
https://github.com/huggingface/lerobot/blob/v0.6.1/src/lerobot/policies/diffusion/configuration_diffusion.py - T1 · upstream documentedsource-codesource 2026-08-03verified 2026-10-24 (1d ago)lerobot 0.6.1
Feature validation requires at least one image or an environment-state input.
“raise ValueError("You must provide at least one image or the environment state among the inputs.")”
https://github.com/huggingface/lerobot/blob/v0.6.1/src/lerobot/policies/diffusion/configuration_diffusion.py - T1 · upstream documentedofficial-docssource 2026-08-03verified 2026-10-24 (1d ago)lerobot 0.6.1
Compute hardware guide: Diffusion group (`diffusion`, `multi_task_dit`) peak VRAM ~8–14GB at BS 8 with AdamW; figures are indicative.
“| Diffusion | `diffusion`, `multi_task_dit` | ~8–14GB | RTX 4070+ / L4 / A10G |”
https://github.com/huggingface/lerobot/blob/v0.6.1/docs/source/hardware_guide.mdx - T1 · upstream documentedpapersource 2026-02-26verified 2026-10-24 (1d ago)
LeRobot paper Table 2: Diffusion Policy (263M) peak memory 1.22GB CPU, 224MB MPS, 1.12 GB RTX 4090, 1.12 GB A100, fp32.
“Table 2: Peak memory consumption of policy models currently supported by lerobot . All models are run in full precision (fp32). Diffusion and Flow Models are run with 10 denoising steps at inference.”
https://arxiv.org/abs/2602.22818