MolmoAct2
Ai2's action reasoning model: the Molmo2-ER embodied-reasoning VLM grafted onto a DiT-style flow-matching continuous action expert via per-layer key-value conditioning. Pre-training uses discrete MolmoAct2-FAST action tokens, so the policy can emit either continuous (flow-matching) or discrete-token actions.
open weightsPEFT / LoRAverified 2026-10-24paper →
Architecture
Compute requirements
| Memory | Precision | Conditions | |
|---|---|---|---|
| train | 48.3 GB measuredsrc | bf16 | LeRobot MolmoAct2 docs: full-model fine-tuning, both action modes, batch 8, peak 48.3 GiB (GiB as reported; about 51.9 GB), single H100 80GB, bf16 model loading, LIBERO with two RGB cameras, chunk_size 10, 8 flow timesteps, gradient_checkpointing on, includes optimizer state. Peak GPU memory sampled with nvidia-smi. Other rows in the same table: action-expert-only 16.5 GiB, LoRA VLM 20.2 GiB (bs 8); full model 49.8 GiB (bs 16), 60.1 GiB (bs 32). · batch 8 |
| inference | 12.1 GB measuredsrc | bf16 | LeRobot MolmoAct2 docs: inference, continuous action mode, CUDA graph enabled, batch 1, peak 12.1 GiB (GiB as reported), single H100 80GB, bf16 model loading, LIBERO with two RGB cameras, peak sampled with nvidia-smi. · batch 1 |
Training requirements: see also the LeRobot compute hardware guide, the authoritative source for LeRobot training envelopes.
Measured inference 1 run · inference cost only, not task success
| Compute | Runtime | Precision | Cams | p50 / p95 ms | Inference | Mem | Platform | Source |
|---|---|---|---|---|---|---|---|---|
| H100 80GB | PyTorch eager | bf16 | 2 | — | — | 13 GB | lerobot 0.6.1conditionsos: Ubuntu 22.04 (repo tested with) · memory_sampling: nvidia-smi peak Inference memory only (12.1 GiB peak GPU memory = 13.0 GB); no latency reported. LIBERO with two RGB cameras, chunk_size=10, 8 flow timesteps. Not task success. | T3 · community reported |
Robots running MolmoAct2 8 of 24 with evidence
| Robot | State | Tier | Verified | Detail |
|---|---|---|---|---|
| Bimanual SO-100/SO-101 | partial | T3 · community reported | 2026-10-25 today | Training evidenced by public checkpoint(s); on-robot inference not documented by a primary source. |
| Franka Emika Panda | works with caveats | T1 · upstream documented | 2026-10-24 1d ago | Evidence is for the DROID setup (Franka, external + wrist camera, 15-step chunks at DROID's 15 Hz). The paper says 'DROID setup' without naming the arm model. The DROID platform is Panda-based, but its software also accepts an FR3. |
| I2RT YAM | works | T1 · upstream documented | 2026-10-24 1d ago | |
| ROBOTIS OMX | partial | T3 · community reported | 2026-10-24 1d ago | Training evidenced by public checkpoint(s); on-robot inference not documented by a primary source. |
| OpenArm | partial | T3 · community reported | 2026-10-24 1d ago | Training evidenced by public checkpoint(s); on-robot inference not documented by a primary source. |
| reBot Arm B601-DM | partial | T3 · community reported | 2026-10-24 1d ago | Training evidenced by public checkpoint(s); on-robot inference not documented by a primary source. |
| SO-100 | works with caveats | T1 · upstream documented | 2026-10-24 1d ago | The MolmoAct2-SO100_101 weights expect the pre-LeRobot-0.5.0 joint calibration convention: lerobot/MolmoAct2-SO100_101-LeRobot includes the frame correction, but self-converted or fine-tuned checkpoints must set joint_signs/joint_offsets or the arm may move in the wrong direction. |
| SO-101 | works with caveats | T1 · upstream documented | 2026-10-24 1d ago | The MolmoAct2-SO100_101 weights expect the pre-LeRobot-0.5.0 joint calibration convention: lerobot/MolmoAct2-SO100_101-LeRobot includes the frame correction, but self-converted or fine-tuned checkpoints must set joint_signs/joint_offsets or the arm may move in the wrong direction. |
No verified evidence, but both are available in LeRobot (ecosystem context): AgileX PiPER, ARX5, EarthRover Mini Plus, HopeJR, Koch v1.1, LeKiwi, Reachy 2, Trossen WidowX AI, Unitree G1, Universal Robots UR5e, UFACTORY xArm, XLeRobot.
Framework support
| Framework | Support | Since | Type string | Docs |
|---|---|---|---|---|
| LeRobot | nativesrc | 0.6.0 | molmoact2 | docs |
| MolmoAct2 (allenai/molmoact2) | upstream-reposrc | — | — | repo |
| LeRobot: Covers the regular MolmoAct2 model; MolmoAct2-Think is not included yet (docs). Training target is chosen with `policy.action_mode` (continuous | discrete | both) and rollout requires `policy.inference_action_mode`. LoRA is a policy-specific flag (`policy.enable_lora_vlm`, uses the peft package), not the generic `--peft.*` path. | ||||
| MolmoAct2 (allenai/molmoact2): Original training code used for the paper's experiments (no tagged releases; pin by commit). | ||||
Checkpoints
- allenai/MolmoAct2 · Post-trained multi-embodiment base for fine-tuning (original HF format; load via policy.checkpoint_path).
- allenai/MolmoAct2-LIBERO
- allenai/MolmoAct2-LIBERO-LeRobot · LeRobot fine-tuned LIBERO checkpoint (98.25% avg in LeRobot docs).
- lerobot/MolmoAct2-SO100_101-LeRobot · LeRobot-format SO-100/101 checkpoint with the joint-frame correction for LeRobot >= 0.5.0 calibration.
- lerobot/MolmoAct2-LIBERO-LeRobot
- lerobot/MolmoAct2-DROID-LeRobot
- lerobot/MolmoAct2-BimanualYAM-LeRobot
Sources
- T1 · upstream documentedpapersource 2026-05-04verified 2026-10-24 (1d ago)
MolmoAct2 grafts a discrete-token VLM onto a flow-matching continuous action expert via per-layer KV conditioning.
“We design a new VLA architecture to graft the discrete-token VLM into the flow-matching continuous-action expert via per-layer key-value (KV) conditioning.”
https://arxiv.org/abs/2605.02881 - T1 · upstream documentedpapersource 2026-05-04verified 2026-10-24 (1d ago)
Molmo2-ER is built from the Molmo2-4B mid-training checkpoint.
“Starting from the Molmo2 –4B mid-training checkpoint”
https://arxiv.org/abs/2605.02881 - T1 · upstream documentedofficial-docssource 2026-08-03verified 2026-10-24 (1d ago)lerobot 0.6.1
LeRobot docs point to allenai/molmoact2 as the original training code.
“For the original MolmoAct2 training code used for the experiments reported in”
https://github.com/huggingface/lerobot/blob/v0.6.1/docs/source/molmoact2.mdx - T1 · upstream documentedofficial-docssource 2026-08-03verified 2026-10-24 (1d ago)lerobot 0.6.1
State and action use quantile normalisation by default.
“MolmoAct2 defaults to quantile normalization for state and action features.”
https://github.com/huggingface/lerobot/blob/v0.6.1/docs/source/molmoact2.mdx - T1 · upstream documentedofficial-docssource 2026-08-03verified 2026-10-24 (1d ago)lerobot 0.6.1
LeRobot MolmoAct2 exposes LoRA on the VLM (and optionally the action expert).
“- `policy.enable_lora_vlm`: enables LoRA on VLM linear layers.”
https://github.com/huggingface/lerobot/blob/v0.6.1/docs/source/molmoact2.mdx - T1 · upstream documentedofficial-docssource 2026-08-03verified 2026-10-24 (1d ago)lerobot 0.6.1
Peak memory table (H100 80GB, bf16, LIBERO, 2 cameras): inference 12.1 GiB (bs 1); full fine-tune 48.3 GiB (bs 8).
“| Fine-tuning, full model, both action modes | 48.3 GiB | 49.8 GiB | 60.1 GiB |”
https://github.com/huggingface/lerobot/blob/v0.6.1/docs/source/molmoact2.mdx - T1 · upstream documentedofficial-docssource 2026-08-03verified 2026-10-24 (1d ago)lerobot 0.6.1
Inference peak memory row and measurement setup.
“| Inference, continuous, CUDA graph enabled (bs=1) | 12.1 GiB | - | - |”
https://github.com/huggingface/lerobot/blob/v0.6.1/docs/source/molmoact2.mdx - T1 · upstream documentedsource-codesource 2026-08-03verified 2026-10-24 (1d ago)lerobot 0.6.1
Policy type string is `molmoact2`.
“@PreTrainedConfig.register_subclass("molmoact2")”
https://github.com/huggingface/lerobot/blob/v0.6.1/src/lerobot/policies/molmoact2/configuration_molmoact2.py - T1 · upstream documentedsource-codesource 2026-08-03verified 2026-10-24 (1d ago)lerobot 0.6.1
supports_rtc() is conditional on continuous inference mode.
“return self.config.inference_action_mode == "continuous"”
https://github.com/huggingface/lerobot/blob/v0.6.1/src/lerobot/policies/molmoact2/modeling_molmoact2.py - T1 · upstream documentedrelease-notessource 2026-07-06verified 2026-10-24 (1d ago)lerobot 0.6.0
MolmoAct2 was added in LeRobot v0.6.0 (PR
“Add MolmoAct2 policy by @hq-fang”
https://github.com/huggingface/lerobot/releases/tag/v0.6.0 - T1 · upstream documentedofficial-docssource 2026-10-02verified 2026-10-24 (1d ago)
Upstream README licenses the model under Apache 2.0 (research and educational use per Ai2 guidelines).
“This model is licensed under Apache 2.0.”
https://github.com/allenai/molmoact2/blob/6070080a20321b4f498ab30f28e1d09ac465edb7/README.md