Matrix/Policy·Imitation·Stanford University, with Meta and UC Berkeley (Zhao, Kumar, Levine, Finn — ALOHA project)
ACT
Action Chunking with Transformers: a CVAE encoder-decoder transformer with a ResNet-18 image backbone that predicts chunks of future joint actions. Single-task, trained from scratch on ~50 demonstrations; LeRobot's recommended starting policy.
80M paramsopen weightsverified 2026-10-24paper →
Architecture
- Family
- bc
- Parameters
- 80,000,000 (Authors state "around 80M" for their configuration. LeRobot's default implementation is smaller: the LeRobot paper (Table 2) lists ACT at 52M parameters.)src
- Backbone
- ResNet-18 (ImageNet-pretrained) + transformer encoder-decoder (CVAE)src
- Action head
- regressionsrc
- Action chunking
- yes
- Code license
- Apache-2.0 (LeRobot implementation)
- Weights license
- Apache-2.0
- Language input
- unsupported
Compute requirements
| Memory | Precision | Conditions | |
|---|---|---|---|
| train | ≥2 GB · ≤6 GBsrc | — | Peak training VRAM at batch size 8 with AdamW (LeRobot default), "Light BC" group (act, vqbet, tdmpc) in the LeRobot compute hardware guide. The guide calls its figures indicative, order-of-magnitude numbers for picking hardware, not exact predictions. Precision not stated. · batch 8 · 5 epochs on a ~45k-frame dataset (640×480): ~30–60 min at batch 8 on a single RTX 4090/3090; ~1–2 h at batch 8 on a single L4/A10G; ~6–14 h at batch 4 on Apple M1/M2/M3 Max (MPS) (indicative, ±50%). |
| inference | 211 MB measuredsrc | fp32 | Peak memory at inference measured by the LeRobot paper (Table 2): 211.24 MB on both RTX 4090 and A100, full fp32. (Same table: 817.4MB CPU-only, 462MB on Apple MPS.) |
Training requirements: see also the LeRobot compute hardware guide, the authoritative source for LeRobot training envelopes.
Measured inference 0 runs · inference cost only, not task success
No measured runs yet. Submit one with its runtime, precision and camera count.
Robots running ACT 23 of 24 with evidence
| Robot | State | Tier | Verified | Detail |
|---|---|---|---|---|
| AgileX PiPER | works with caveats | T3 · community reported | 2026-10-24 1d ago | Community-reported only; not documented upstream or maintainer-tested. |
| ALOHA (stationary) | works with caveats | T1 · upstream documented | 2026-10-24 1d ago | LeRobot's built-in `aloha` robot type was removed in the hardware API redesign (PR #777, June 2025) and is not in v0.6.1; the documented LeRobot workflow needs a pre-redesign commit or a third-party driver. |
| ARX5 | partial | T3 · community reported | 2026-10-25 today | Training evidenced by public checkpoint(s); on-robot inference not documented by a primary source. |
| Bimanual SO-100/SO-101 | partial | T3 · community reported | 2026-10-24 1d ago | Training evidenced by public checkpoint(s); on-robot inference not documented by a primary source. |
| Franka Research 3 | works with caveats | T3 · community reported | 2026-10-25 today | Community-reported only (anonymous PhAIL benchmark under double-blind review); not documented upstream or maintainer-tested. |
| Franka Emika Panda | works with caveats | T3 · community reported | 2026-10-24 1d ago | Community-reported only; not documented upstream or maintainer-tested. |
| HopeJR | partial | T1 · upstream documented | 2026-10-24 1d ago | Upstream states that Record, Replay and Train with Hope-JR are still experimental. |
| I2RT YAM | partial | T3 · community reported | 2026-10-24 1d ago | Training evidenced by public checkpoint(s); on-robot inference not documented by a primary source. |
| Koch v1.1 | works | T1 · upstream documented | 2026-10-24 1d ago | |
| LeKiwi | works | T1 · upstream documented | 2026-10-24 1d ago | |
| Mobile ALOHA | works | T1 · upstream documented | 2026-10-24 1d ago | Documented with the Mobile ALOHA authors' codebase (MarkFzp/act-plus-plus, co-training with static ALOHA data), not LeRobot; LeRobot's converted lerobot/aloha_mobile_* datasets carry 14-dim arm actions only (no base velocity). |
| ROBOTIS OMX | works | T1 · upstream documented | 2026-10-24 1d ago | |
| OpenArm | works | T1 · upstream documented | 2026-10-24 1d ago | |
| Reachy 2 | works with caveats | T1 · upstream documented | 2026-10-25 today | On-robot inference is community-reported only. Upstream documents training and recording; its evaluation step uses lerobot-eval, which in v0.6.1 evaluates in a simulation env, so it does not document on-robot inference. |
| reBot Arm B601-DM | partial | T1 · upstream documented | 2026-10-25 today | Training on reBot B601-DM data is documented by the vendor (Seeed wiki 'B601-DM with LeRobot'). The wiki's on-robot evaluation command (lerobot-record with --policy.path and no teleoperator) is rejected by the Seeed LeRobot fork the wiki installs, which directs policy deployment to lerobot-rollout, so on-robot inference is not validly documented. Public checkpoints trained on B601-DM data also exist. |
| SO-100 | works | T1 · upstream documented | 2026-10-24 1d ago | |
| SO-101 | works | T1 · upstream documented | 2026-10-24 1d ago | |
| Trossen WidowX AI | works | T1 · upstream documented | 2026-10-24 1d ago | |
| Unitree G1 | works with caveats | T1 · upstream documented | 2026-10-24 1d ago | Documented by Unitree in its unitree_IL_lerobot repository, which pins LeRobot as a git submodule (commit a5b29d4, November 2025) and deploys with its own eval_robot/eval_g1.py script, not through LeRobot's native unitree_g1 robot or lerobot-rollout. |
| Universal Robots UR5e | works with caveats | T3 · community reported | 2026-10-24 1d ago | Community-reported only; not documented upstream or maintainer-tested. |
| WidowX 250 S | partial | T1 · upstream documented | 2026-10-24 1d ago | The BridgeData V2 evaluation used goal-image-conditioned ACT. bridge_data_v2 does not include the ACT implementation or a checkpoint. |
| UFACTORY xArm | partial | T3 · community reported | 2026-10-24 1d ago | Training evidenced by public checkpoint(s); on-robot inference not documented by a primary source. |
| XLeRobot | works with caveats | T1 · upstream documented | 2026-10-24 1d ago | The documented ACT demo is single-arm: one XLeRobot arm is recorded and run as robot.type=so101_follower, not the full xlerobot robot type. |
No verified evidence, but both are available in LeRobot (ecosystem context): EarthRover Mini Plus.
Framework support
Checkpoints
- lerobot/act_aloha_sim_transfer_cube_human · ALOHA sim transfer-cube checkpoint (Apache-2.0 on the Hub).
- lerobot/act_aloha_sim_insertion_human · ALOHA sim insertion checkpoint (Apache-2.0 on the Hub).
Related terms
Action chunking · Action horizon · Asynchronous inference · Behavioral cloning · Control rate · Imitation learning · Leader–follower arm · LeRobotDataset · Proprioception · Teleoperation
Sources
- T1 · upstream documentedpapersource 2023-04-23verified 2026-10-24 (1d ago)
ACT paper (Appendix D) states the model size, training time and inference time.
“The model has around 80M parameters, and we train from scratch for each task. The training takes around 5 hours on a single 11G RTX 2080 Ti GPU, and the inference time is around 0.01 seconds on the same machine.”
https://arxiv.org/abs/2304.13705 - T1 · upstream documentedofficial-docssource 2026-08-03verified 2026-10-24 (1d ago)lerobot 0.6.1
LeRobot ACT docs describe the ResNet-18 backbone and transformer encoder/decoder that outputs an action chunk.
“**Vision Backbone**: ResNet-18 processes images from multiple camera viewpoints”
https://github.com/huggingface/lerobot/blob/v0.6.1/docs/source/act.mdx - T1 · upstream documentedofficial-docssource 2026-08-03verified 2026-10-24 (1d ago)lerobot 0.6.1
ACT inputs are multi-view RGB images, joint positions and a latent z; the rollout example marks --task as skippable for ACT.
“--task="Your task description" \ # can be skipped for ACT”
https://github.com/huggingface/lerobot/blob/v0.6.1/docs/source/act.mdx - T1 · upstream documentedsource-codesource 2026-08-03verified 2026-10-24 (1d ago)lerobot 0.6.1
Policy type string is `act` (chunk_size 100 by default).
“@PreTrainedConfig.register_subclass("act")”
https://github.com/huggingface/lerobot/blob/v0.6.1/src/lerobot/policies/act/configuration_act.py - T1 · upstream documentedofficial-docssource 2026-08-03verified 2026-10-24 (1d ago)lerobot 0.6.1
Compute hardware guide: Light BC (`act`, `vqbet`, `tdmpc`) peak VRAM ~2–6GB at BS 8 with AdamW; figures are indicative.
“| Light BC | `act`, `vqbet`, `tdmpc` | ~2–6GB | Laptop GPU (RTX 3060), L4, A10G |”
https://github.com/huggingface/lerobot/blob/v0.6.1/docs/source/hardware_guide.mdx - T1 · upstream documentedpapersource 2026-02-26verified 2026-10-24 (1d ago)
LeRobot paper Table 2: ACT (52M) peak memory 817.4MB CPU, 462MB MPS, 211.24 MB RTX 4090, 211.24 MB A100, fp32.
“Table 2: Peak memory consumption of policy models currently supported by lerobot . All models are run in full precision (fp32). Diffusion and Flow Models are run with 10 denoising steps at inference.”
https://arxiv.org/abs/2602.22818