{
  "schema_version": "1.0.0",
  "license": "CC-BY-4.0",
  "attribution": "CanIEmbody (https://caniembody.pages.dev)",
  "dataset_version": "1.0.0-seed",
  "generated": "2026-10-25",
  "data": [
    {
      "id": "action-chunking",
      "term": "Action chunking",
      "aka": [
        "chunked action prediction",
        "ACT-style chunking"
      ],
      "short": "Predicting a sequence of future actions per model call and executing them before querying the policy again, instead of predicting one action per control step.",
      "body": "Action chunking means a policy outputs a fixed-length sequence of future actions (a *chunk*) from one observation, and the controller executes some or all of those actions before calling the policy again. It was introduced to robot imitation learning by Zhao, Kumar, Levine and Finn in \"Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware\" (2023), together with the [ACT](/policy/act) policy and the ALOHA teleoperation rig. The paper takes the term from psychology and argues that predicting the next *k* steps \"reduces the effective horizon of the task by k-fold, mitigating compounding errors\".\n\nTwo numbers matter and are often confused: how many actions the model predicts, and how many the robot executes before the next call (see [action horizon](/glossary/action-horizon)). In LeRobot v0.6.1, ACT defaults to `chunk_size=100` and `n_action_steps=100`; [SmolVLA](/policy/smolvla) and [π0](/policy/pi0) default to 50 and 50. Executing fewer actions than were predicted makes the robot more reactive at the cost of more model calls.\n\nChunking is now the norm rather than the exception: [Diffusion Policy](/policy/diffusion), π0, [π0.5](/policy/pi05), [GR00T N1.7](/policy/groot-n1-7) and [OpenVLA-OFT](/policy/openvla-oft) all predict chunks.\n\nA common misconception is that chunking solves latency. It does not. A chunk is executed open-loop, so the robot ignores new observations until the next call, and under synchronous inference it pauses while that call runs. ACT's temporal ensembling, [asynchronous inference](/glossary/async-inference) and [real-time chunking](/glossary/real-time-chunking) each address a different part of this problem. Chunking also makes [control rate](/glossary/control-rate) and model call rate two different numbers.\n",
      "related": [
        "action-horizon",
        "real-time-chunking",
        "async-inference",
        "control-rate",
        "inference-latency",
        "behavioral-cloning"
      ],
      "policies": [
        "act",
        "diffusion",
        "smolvla",
        "pi0",
        "pi05",
        "groot-n1-7",
        "openvla-oft"
      ],
      "sources": [
        {
          "id": "ev-act-paper",
          "tier": "upstream-documented",
          "type": "paper",
          "url": "https://arxiv.org/abs/2304.13705",
          "date": "2023-04-23",
          "verified_on": "2026-10-24",
          "summary": "Zhao et al. introduce Action Chunking with Transformers (ACT).",
          "quote": "we develop a simple yet novel algorithm, Action Chunking with Transformers (ACT), which learns a generative model over action sequences."
        },
        {
          "id": "ev-act-paper-full",
          "tier": "upstream-documented",
          "type": "paper",
          "url": "https://arxiv.org/html/2304.13705",
          "date": "2023-04-23",
          "verified_on": "2026-10-24",
          "summary": "ACT paper: chunking reduces the effective task horizon k-fold.",
          "quote": "rather than just one step at a time. This reduces the effective horizon of the task by"
        },
        {
          "id": "ev-lerobot-act-config",
          "tier": "upstream-documented",
          "type": "source-code",
          "url": "https://github.com/huggingface/lerobot/blob/v0.6.1/src/lerobot/policies/act/configuration_act.py",
          "date": "2026-08-03",
          "verified_on": "2026-10-24",
          "versions": {
            "lerobot": "0.6.1"
          },
          "summary": "LeRobot v0.6.1 ACT default chunk size is 100.",
          "quote": "chunk_size: int = 100"
        },
        {
          "id": "ev-lerobot-smolvla-config",
          "tier": "upstream-documented",
          "type": "source-code",
          "url": "https://github.com/huggingface/lerobot/blob/v0.6.1/src/lerobot/policies/smolvla/configuration_smolvla.py",
          "date": "2026-08-03",
          "verified_on": "2026-10-24",
          "versions": {
            "lerobot": "0.6.1"
          },
          "summary": "LeRobot v0.6.1 SmolVLA default chunk size is 50.",
          "quote": "chunk_size: int = 50"
        }
      ]
    },
    {
      "id": "action-horizon",
      "term": "Action horizon",
      "aka": [
        "prediction horizon",
        "execution horizon",
        "chunk size",
        "n_action_steps"
      ],
      "short": "How many future action steps a policy predicts per call (prediction horizon) and how many it executes before replanning (execution horizon).",
      "body": "\"Action horizon\" names two different quantities, and published tables often mix them. The *prediction horizon* is the number of future actions the model outputs per inference call, which is the length of the [action chunk](/glossary/action-chunking). The *execution horizon* is how many of those actions the robot executes before the policy is called again. Chi et al.'s Diffusion Policy paper (2023) defines both: \"T_o as the observation horizon, T_p as the action prediction horizon and T_a as the action execution horizon\". It found an execution horizon of 8 steps optimal for most of its tasks, trading temporal consistency against reaction time.\n\nNames differ by codebase. In LeRobot v0.6.1, [ACT](/policy/act), [SmolVLA](/policy/smolvla) and [π0](/policy/pi0) call the prediction horizon `chunk_size` and the execution horizon `n_action_steps`; [Diffusion Policy](/policy/diffusion) calls the prediction horizon `horizon`. The π0 configuration notes that `chunk_size` is what openpi calls `action_horizon`. Defaults are 100/100 for ACT, 50/50 for SmolVLA and π0, 64/32 for Diffusion Policy, and 40/40 for [GR00T N1.7](/policy/groot-n1-7).\n\nA horizon in steps means nothing without the [control rate](/glossary/control-rate). Fifty steps is one second at 50 Hz and about 1.7 seconds at 30 Hz. In time units, the execution horizon is the longest the robot runs open-loop between observations under synchronous inference.\n\nA common misconception is that longer horizons are simply better or worse. Longer chunks mean fewer model calls and smoother motion but slower reaction to change. LeRobot's async-inference docs also warn that larger chunks \"might also result in less precise actions\". The right value depends on the task, the control rate and [inference latency](/glossary/inference-latency).\n",
      "related": [
        "action-chunking",
        "control-rate",
        "inference-latency",
        "diffusion-policy",
        "real-time-chunking"
      ],
      "policies": [
        "act",
        "smolvla",
        "pi0",
        "diffusion",
        "groot-n1-7"
      ],
      "sources": [
        {
          "id": "ev-chi-2023-full",
          "tier": "upstream-documented",
          "type": "paper",
          "url": "https://arxiv.org/html/2303.04137",
          "date": "2023-03-07",
          "verified_on": "2026-10-24",
          "summary": "Diffusion Policy defines prediction and execution horizons; 8-step execution optimal for most tasks.",
          "quote": "found the action horizon of 8 steps to be optimal for most tasks that we tested."
        },
        {
          "id": "ev-lerobot-pi0-config",
          "tier": "upstream-documented",
          "type": "source-code",
          "url": "https://github.com/huggingface/lerobot/blob/v0.6.1/src/lerobot/policies/pi0/configuration_pi0.py",
          "date": "2026-08-03",
          "verified_on": "2026-10-24",
          "versions": {
            "lerobot": "0.6.1"
          },
          "summary": "LeRobot π0 chunk_size equals openpi action_horizon.",
          "quote": "chunk_size: int = 50  # Number of action steps to predict, in openpi called \"action_horizon\""
        },
        {
          "id": "ev-lerobot-diffusion-config",
          "tier": "upstream-documented",
          "type": "source-code",
          "url": "https://github.com/huggingface/lerobot/blob/v0.6.1/src/lerobot/policies/diffusion/configuration_diffusion.py",
          "date": "2026-08-03",
          "verified_on": "2026-10-24",
          "versions": {
            "lerobot": "0.6.1"
          },
          "summary": "LeRobot Diffusion Policy default horizon 64, n_action_steps 32.",
          "quote": "horizon: int = 64"
        },
        {
          "id": "ev-lerobot-groot-config",
          "tier": "upstream-documented",
          "type": "source-code",
          "url": "https://github.com/huggingface/lerobot/blob/v0.6.1/src/lerobot/policies/groot/configuration_groot.py",
          "date": "2026-08-03",
          "verified_on": "2026-10-24",
          "versions": {
            "lerobot": "0.6.1"
          },
          "summary": "LeRobot GR00T N1.7 default chunk_size 40.",
          "quote": "chunk_size: int = 40"
        },
        {
          "id": "ev-lerobot-async-doc",
          "tier": "upstream-documented",
          "type": "official-docs",
          "url": "https://github.com/huggingface/lerobot/blob/v0.6.1/docs/source/async.mdx",
          "date": "2026-08-03",
          "verified_on": "2026-10-24",
          "versions": {
            "lerobot": "0.6.1"
          },
          "summary": "LeRobot async docs: larger chunks may reduce precision.",
          "quote": "However, larger values of `actions_per_chunk` might also result in less precise actions, due to the compounding errors consequent to predicting actions over longer timespans."
        }
      ]
    },
    {
      "id": "action-tokenization",
      "term": "Action tokenization",
      "aka": [
        "action discretization",
        "action tokens"
      ],
      "short": "Converting continuous robot actions into discrete tokens so that an autoregressive transformer can predict them the way it predicts text.",
      "body": "Action tokenization maps continuous action vectors to a sequence of discrete symbols from a fixed vocabulary, so that an autoregressive model, typically a language-model backbone, can predict actions with the same next-token objective it uses for text. RT-2 (Brohan et al., 2023) established the approach for [VLAs](/glossary/vla) by expressing robot actions \"as text tokens\". The simplest scheme bins each action dimension independently: [OpenVLA](/policy/openvla) discretizes \"each dimension of the robot actions separately into one of 256 bins\", with bin edges set from the 1st and 99th percentiles of the training actions.\n\nPer-dimension, per-timestep binning degrades at high control frequency. FAST (Pertsch et al., \"FAST: Efficient Action Tokenization for Vision-Language-Action Models\", 2025) found that such schemes \"typically perform poorly when learning dexterous skills from high-frequency robot data\". FAST instead compresses an [action chunk](/glossary/action-chunking) with the discrete cosine transform before tokenizing. [π0-FAST](/policy/pi0-fast) is π0 trained with this tokenizer. [VQ-BeT](/policy/vqbet) learns its vocabulary instead, \"tokenizing continuous actions with a hierarchical vector quantization module\".\n\nTokenization sets decoding cost. An autoregressive VLA emits tokens one forward step at a time, so latency grows with the number of tokens per chunk. Continuous heads, such as [flow matching](/glossary/flow-matching), diffusion or the L1 regression with parallel decoding used by [OpenVLA-OFT](/policy/openvla-oft), produce the whole chunk in a fixed number of passes.\n\nA common misreading concerns FAST's headline speed-up. The FAST paper reports matching diffusion VLAs \"while reducing training time by up to 5x\". That figure is about training, not inference speed, and should not be quoted as a latency advantage of π0-FAST over [π0](/policy/pi0).\n",
      "related": [
        "vla",
        "action-chunking",
        "flow-matching",
        "inference-latency"
      ],
      "policies": [
        "openvla",
        "pi0-fast",
        "vqbet",
        "openvla-oft",
        "pi0"
      ],
      "sources": [
        {
          "id": "ev-rt2-paper",
          "tier": "upstream-documented",
          "type": "paper",
          "url": "https://arxiv.org/abs/2307.15818",
          "date": "2023-07-28",
          "verified_on": "2026-10-24",
          "summary": "RT-2 expresses robot actions as text tokens.",
          "quote": "we express the actions as text tokens and incorporate them directly into the training set of the model in the same way as natural language tokens."
        },
        {
          "id": "ev-openvla-paper-full",
          "tier": "upstream-documented",
          "type": "paper",
          "url": "https://arxiv.org/html/2406.09246",
          "date": "2024-06-13",
          "verified_on": "2026-10-24",
          "summary": "OpenVLA bins each action dimension into 256 bins.",
          "quote": "we discretize each dimension of the robot actions separately into one of 256 bins."
        },
        {
          "id": "ev-fast-paper",
          "tier": "upstream-documented",
          "type": "paper",
          "url": "https://arxiv.org/abs/2501.09747",
          "date": "2025-01-16",
          "verified_on": "2026-10-24",
          "summary": "FAST: DCT-based action tokenization; training up to 5x faster.",
          "quote": "we propose a new compression-based tokenization scheme for robot actions, based on the discrete cosine transform."
        },
        {
          "id": "ev-vqbet-paper",
          "tier": "upstream-documented",
          "type": "paper",
          "url": "https://arxiv.org/abs/2403.03181",
          "date": "2024-03-05",
          "verified_on": "2026-10-24",
          "summary": "VQ-BeT tokenizes actions with hierarchical vector quantization.",
          "quote": "VQ-BeT augments BeT by tokenizing continuous actions with a hierarchical vector quantization module."
        }
      ]
    },
    {
      "id": "affordance",
      "term": "Affordance",
      "aka": [
        "action possibility",
        "affordance map"
      ],
      "short": "What actions an object or scene makes possible for a particular agent; in robot learning, a learned estimate of which actions are feasible and useful in the current state.",
      "body": "An affordance is a possibility for action that the environment offers to a particular agent: a handle affords pulling, a mug affords grasping by its handle or rim. The concept comes from the psychologist James J. Gibson, who set it out in *The Ecological Approach to Visual Perception* (1979). The point most often lost is that affordances are relational. They depend on the agent's body as well as on the object: a shelf at shoulder height affords reaching to a humanoid but not to a tabletop arm whose workspace ends below it.\n\nRobot learning uses the word in two concrete senses. The first is a learned function that scores whether a skill will succeed from the current state. SayCan (Ahn et al., \"Do As I Can, Not As I Say: Grounding Language in Robotic Affordances\", 2022) combined a language model's proposed steps with value functions of pretrained skills, which \"provide the grounding necessary to connect this knowledge to a particular physical environment\". The second is a spatial prediction, such as grasp points, contact regions or keypoints in an image, used as an intermediate representation between perception and control.\n\nEnd-to-end [VLAs](/glossary/vla) such as [SmolVLA](/policy/smolvla) and [π0](/policy/pi0) do not output an explicit affordance map. Whatever they encode about affordances is implicit in the action distribution learned from demonstrations on specific robots.\n\nA common misconception is that recognizing an object implies knowing how to act on it. Because affordances depend on kinematics, reach and gripper geometry, a policy can identify an object correctly and still fail to manipulate it on a different robot. That is one face of the [embodiment gap](/glossary/embodiment-gap), and it is why compatibility in this matrix is recorded per robot rather than per policy.\n",
      "related": [
        "embodiment-gap",
        "vla",
        "reward-model",
        "degrees-of-freedom"
      ],
      "policies": [],
      "sources": [
        {
          "id": "ev-saycan-paper",
          "tier": "upstream-documented",
          "type": "paper",
          "url": "https://arxiv.org/abs/2204.01691",
          "date": "2022-04-04",
          "verified_on": "2026-10-24",
          "summary": "SayCan grounds language-model plans in the affordances (value functions) of robot skills.",
          "quote": "while value functions associated with these skills provide the grounding necessary to connect this knowledge to a particular physical environment."
        }
      ]
    },
    {
      "id": "async-inference",
      "term": "Asynchronous inference",
      "aka": [
        "async inference",
        "PolicyServer / RobotClient"
      ],
      "short": "Running policy inference in parallel with action execution, so the robot keeps acting from an action queue while the next chunk is computed.",
      "body": "Asynchronous inference decouples computing [action chunks](/glossary/action-chunking) from executing them. Under synchronous inference the robot executes a chunk, stops, sends an observation, waits for the model and resumes, idling for the full [inference latency](/glossary/inference-latency) at every chunk boundary. Under asynchronous inference the robot keeps executing from an action queue and sends a new observation before the queue runs out, so the next chunk is ready when needed.\n\nLeRobot's implementation was introduced with [SmolVLA](/policy/smolvla) (Shukor et al., 2025) as \"an asynchronous inference stack decoupling perception and action prediction from action execution, allowing higher control rates with chunked action generation\". It is split into a `PolicyServer`, which can run on a remote GPU such as an [RTX 4090](/compute/rtx-4090), and a `RobotClient` on the computer attached to the robot, for example an [SO-101](/robot/so101). Two parameters govern it: `actions_per_chunk`, and `chunk_size_threshold` (*g* in the paper), the queue fraction at which the client sends a fresh observation. The paper shows that the queue does not run dry when g ≥ (E[ℓ_S]/Δt)/n, where ℓ_S is server latency, Δt the control period and n the chunk length. The LeRobot docs report values around 0.5–0.6 working well; values near 0 collapse to synchronous behavior.\n\nTwo misconceptions recur. Async inference does not reduce latency; it hides it. Each chunk is still computed from an observation that is at least ℓ_S old by the time its actions execute. And async inference alone does not make chunk transitions smooth. LeRobot aggregates overlapping actions, whereas [real-time chunking](/glossary/real-time-chunking) conditions the new chunk on actions already committed, and the two can be combined.\n",
      "related": [
        "real-time-chunking",
        "action-chunking",
        "inference-latency",
        "control-rate"
      ],
      "policies": [
        "smolvla",
        "pi0",
        "act"
      ],
      "sources": [
        {
          "id": "ev-smolvla-paper",
          "tier": "upstream-documented",
          "type": "paper",
          "url": "https://arxiv.org/abs/2506.01844",
          "date": "2025-06-02",
          "verified_on": "2026-10-24",
          "summary": "SmolVLA introduces an asynchronous inference stack.",
          "quote": "we introduce an asynchronous inference stack decoupling perception and action prediction from action execution, allowing higher control rates with chunked action generation."
        },
        {
          "id": "ev-lerobot-async-doc",
          "tier": "upstream-documented",
          "type": "official-docs",
          "url": "https://github.com/huggingface/lerobot/blob/v0.6.1/docs/source/async.mdx",
          "date": "2026-08-03",
          "verified_on": "2026-10-24",
          "versions": {
            "lerobot": "0.6.1"
          },
          "summary": "LeRobot v0.6.1 async inference guide.",
          "quote": "This is fundamentally different from synchronous inference (sync), where the robot stays idle while the policy computes the next chunk of actions."
        },
        {
          "id": "ev-lerobot-async-threshold",
          "tier": "upstream-documented",
          "type": "official-docs",
          "url": "https://github.com/huggingface/lerobot/blob/v0.6.1/docs/source/async.mdx",
          "date": "2026-08-03",
          "verified_on": "2026-10-24",
          "versions": {
            "lerobot": "0.6.1"
          },
          "summary": "LeRobot: chunk_size_threshold of about 0.5-0.6 works well.",
          "quote": "We found values around 0.5-0.6 to work well."
        }
      ]
    },
    {
      "id": "behavioral-cloning",
      "term": "Behavioral cloning",
      "aka": [
        "BC",
        "behavior cloning"
      ],
      "short": "Supervised learning of a policy that maps observations to the actions a demonstrator took, with no reward signal or environment interaction during training.",
      "body": "Behavioral cloning (BC) is the simplest form of [imitation learning](/glossary/imitation-learning): record observation–action pairs from a demonstrator and fit a policy to them with supervised learning. It needs no reward function, simulator or online interaction. An early, widely cited example is ALVINN (Pomerleau, NIPS 1988), a neural network that mapped camera and laser range-finder input to the direction a vehicle should travel to follow a road.\n\nMost manipulation policies in this matrix are trained by behavioral cloning on teleoperated demonstrations, including [ACT](/policy/act), [Diffusion Policy](/policy/diffusion), [SmolVLA](/policy/smolvla), and the fine-tuning stage of large VLAs such as [π0](/policy/pi0). They differ in the model used to represent the action distribution (a conditional VAE in ACT, a diffusion model, [flow matching](/glossary/flow-matching) or discrete [action tokens](/glossary/action-tokenization)) and in how much pretraining comes before the cloning step. Modern BC is therefore not plain regression. Generative action heads exist because demonstrations are multimodal, and regressing to their mean can produce an action no demonstrator took.\n\nBC's known weakness is distribution shift. The policy only sees states the demonstrator visited; at deployment its own small errors take it elsewhere, and errors accumulate. The ACT paper (Zhao et al., 2023) calls this the \"compounding error\" problem of imitation learning. Remedies include [action chunking](/glossary/action-chunking), [DAgger](/glossary/dagger)-style corrective data, and combining BC with reinforcement learning, as in [HIL-SERL](/policy/hil-serl).\n\nA common misconception is that more demonstrations always help. Demonstrations that differ in strategy or speed make the target distribution harder to model; the ACT abstract names non-stationary human demonstrations as a challenge alongside compounding error.\n",
      "related": [
        "imitation-learning",
        "dagger",
        "action-chunking",
        "teleoperation",
        "diffusion-policy"
      ],
      "policies": [
        "act",
        "diffusion",
        "smolvla",
        "pi0",
        "hil-serl"
      ],
      "sources": [
        {
          "id": "ev-alvinn-1988",
          "tier": "upstream-documented",
          "type": "paper",
          "url": "https://papers.nips.cc/paper_files/paper/1988/hash/812b4ba287f5ee0bc9d43bbf5bbe87fb-Abstract.html",
          "verified_on": "2026-10-24",
          "summary": "Pomerleau (NIPS 1988): ALVINN, a neural network trained to map sensor input to steering direction.",
          "quote": "ALVINN (Autonomous Land Vehicle In a Neural Network) is a 3-layer back-propagation network designed for the task of road following."
        },
        {
          "id": "ev-act-paper",
          "tier": "upstream-documented",
          "type": "paper",
          "url": "https://arxiv.org/abs/2304.13705",
          "date": "2023-04-23",
          "verified_on": "2026-10-24",
          "summary": "ACT paper names compounding errors and non-stationary demonstrations as challenges.",
          "quote": "errors in the policy can compound over time, and human demonstrations can be non-stationary."
        }
      ]
    },
    {
      "id": "control-rate",
      "term": "Control rate",
      "aka": [
        "control frequency",
        "control loop rate",
        "control_rate_hz"
      ],
      "short": "How many actions per second the robot's controller executes, in Hz; set by the robot and dataset, and distinct from how often the policy model is called.",
      "body": "Control rate is the frequency, in hertz, at which actions are sent to the robot's actuators: the inverse of the control period Δt. It is a property of the robot and recording setup, not of the model. A dataset recorded at 30 fps trains a policy whose actions are spaced 1/30 s apart, and the policy should be deployed at that rate. The ALOHA setup in the ACT paper (Zhao et al., 2023) teleoperated and recorded at 50 Hz; LeRobot's recording examples for the [SO-101](/robot/so101) use 30 fps.\n\nWith [action chunking](/glossary/action-chunking), control rate and model call rate come apart. The π0 paper (Black et al., 2024) ran its 50 Hz robots by calling the model \"every 0.5 seconds (after executing 25 actions)\": a 50 Hz control rate from a 2 Hz call rate. This matrix stores the two separately. `control_rate_hz` is executed actions per second after chunking; `inference_hz` is model calls per second. Neither may be derived from the other without stating the number of actions executed per call.\n\nThe deployment rate should match the training data. If a policy trained on 30 fps data is run at 50 Hz, each chunk plays back faster than the motion it imitates, which changes the dynamics the policy was trained on.\n\nA common misconception is that a high control rate makes a policy reactive at that rate. A [SmolVLA](/policy/smolvla) chunk executed open-loop at 30 Hz reacts to new observations only when the next chunk arrives; reactivity is bounded by [inference latency](/glossary/inference-latency) and the execution [horizon](/glossary/action-horizon), and is improved by [async inference](/glossary/async-inference). Rate matters for demonstrations too: in the ACT paper's user study, cutting teleoperation from 50 Hz to 5 Hz caused \"a 62% slowdown in completion time\".\n",
      "related": [
        "inference-latency",
        "action-chunking",
        "action-horizon",
        "async-inference",
        "lerobot-dataset"
      ],
      "policies": [
        "act",
        "pi0",
        "smolvla"
      ],
      "sources": [
        {
          "id": "ev-act-paper-full",
          "tier": "upstream-documented",
          "type": "paper",
          "url": "https://arxiv.org/html/2304.13705",
          "date": "2023-04-23",
          "verified_on": "2026-10-24",
          "summary": "ACT/ALOHA teleoperates and records at 50 Hz.",
          "quote": "Both the teleoperation and data recording happen at 50Hz."
        },
        {
          "id": "ev-pi0-paper-full",
          "tier": "upstream-documented",
          "type": "paper",
          "url": "https://arxiv.org/html/2410.24164",
          "date": "2024-10-31",
          "verified_on": "2026-10-24",
          "summary": "π0 runs 50 Hz robots with inference every 25 actions.",
          "quote": "for all other robots, which run at 50Hz, we run inference every 0.5 seconds (after executing 25 actions)."
        },
        {
          "id": "ev-lerobot-il-robots",
          "tier": "upstream-documented",
          "type": "official-docs",
          "url": "https://github.com/huggingface/lerobot/blob/v0.6.1/docs/source/il_robots.mdx",
          "date": "2026-08-03",
          "verified_on": "2026-10-24",
          "versions": {
            "lerobot": "0.6.1"
          },
          "summary": "LeRobot recording example uses 30 fps.",
          "quote": "FPS = 30"
        }
      ]
    },
    {
      "id": "dagger",
      "term": "DAgger",
      "aka": [
        "Dataset Aggregation"
      ],
      "short": "Dataset Aggregation: an iterative imitation-learning algorithm that runs the learned policy, has an expert label the states it visits, and retrains on the combined data.",
      "body": "DAgger (Dataset Aggregation) is an iterative imitation-learning algorithm introduced by Ross, Gordon and Bagnell in \"A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning\" (AISTATS 2011). It targets the main failure of [behavioral cloning](/glossary/behavioral-cloning). A cloned policy is trained on states the expert visited, but at deployment its own small errors carry it into states the expert never demonstrated, where errors compound. The paper frames this as a violation of the i.i.d. assumption in problems \"where future observations depend on previous predictions (actions)\".\n\nThe algorithm runs in rounds:\n\n1. Train a policy on expert demonstrations.\n2. Run that policy and record the states it actually visits.\n3. Ask the expert which action it would have taken in each of those states.\n4. Add the labeled states to the dataset, retrain on the aggregate, and repeat.\n\nRoss et al. show that this yields a policy with good performance \"under the distribution of observations it induces\".\n\nOn real robots, labeling every visited state is impractical, so practical variants let a human take over only when the policy goes wrong and record those corrections. LeRobot's [HIL-SERL](/policy/hil-serl) workflow records such interventions during on-robot training, but uses them inside a reinforcement-learning loop rather than for pure imitation (see [human-in-the-loop RL](/glossary/human-in-the-loop-rl)).\n\nA common misconception is that DAgger just means collecting more demonstrations. Extra demonstrations still come from states the expert chose to visit. DAgger's data comes from the *learner's* state distribution, and that is what corrects the mismatch. [Action chunking](/glossary/action-chunking) attacks the same compounding-error problem from the other side, by shortening the effective horizon instead of changing the data.\n",
      "related": [
        "behavioral-cloning",
        "imitation-learning",
        "human-in-the-loop-rl",
        "action-chunking"
      ],
      "policies": [
        "hil-serl"
      ],
      "sources": [
        {
          "id": "ev-ross-2011",
          "tier": "upstream-documented",
          "type": "paper",
          "url": "https://proceedings.mlr.press/v15/ross11a.html",
          "verified_on": "2026-10-24",
          "summary": "Ross, Gordon and Bagnell (AISTATS 2011) introduce DAgger.",
          "quote": "In this paper, we propose a new iterative algorithm, which trains a stationary deterministic policy, that can be seen as a no regret algorithm in an online learning setting."
        }
      ]
    },
    {
      "id": "degrees-of-freedom",
      "term": "Degrees of freedom",
      "aka": [
        "DoF",
        "DOF"
      ],
      "short": "The number of independent coordinates needed to specify a mechanism's configuration; for a serial robot arm, usually its number of actuated joints.",
      "body": "The degrees of freedom (DoF) of a mechanism are the number of independent coordinates needed to specify its configuration. For a serial arm built from revolute or prismatic joints, this equals the number of joints. A rigid body in 3D space has six DoF, three translational and three rotational, so an arm needs at least six joints to place its end-effector at an arbitrary position *and* orientation within its workspace. Arms with more than six joints are kinematically redundant.\n\nDoF figures in product descriptions are inconsistent about the gripper. The [SO-101](/robot/so101) has six STS3215 motors: five arm joints (shoulder pan, shoulder lift, elbow flex, wrist flex, wrist roll) and one gripper. It is often called \"6-DoF\", but its arm has five DoF, so it cannot reach every orientation at every position. LeRobot's action-representation guide, by contrast, uses \"a 6-DOF arm with a gripper\" for an arm with six joints plus a gripper. A precise statement separates the two, for example \"5-DoF arm + 1-DoF gripper\".\n\nDoF fixes the size of the action and [proprioceptive](/glossary/proprioception) state vectors. An SO-101 policy outputs six numbers per step, a bimanual [bi-SO](/robot/bi-so) setup twelve, and a humanoid such as the [Unitree G1](/robot/unitree-g1) many more. Cross-embodiment VLAs therefore pad or project to a fixed width ([SmolVLA](/policy/smolvla) uses 32 in LeRobot v0.6.1), and a checkpoint pretrained on other robots still has to be fine-tuned to learn a new robot's action mapping (see [embodiment gap](/glossary/embodiment-gap)).\n\nA common misconception is that more DoF is always better. Extra joints add reach and dexterity, but each one enlarges the action space that a policy must learn from the same number of demonstrations.\n",
      "related": [
        "proprioception",
        "embodiment-gap",
        "leader-follower-arm",
        "affordance"
      ],
      "policies": [
        "smolvla"
      ],
      "sources": [
        {
          "id": "ev-lerobot-so101-doc",
          "tier": "upstream-documented",
          "type": "official-docs",
          "url": "https://github.com/huggingface/lerobot/blob/v0.6.1/docs/source/so101.mdx",
          "date": "2026-08-03",
          "verified_on": "2026-10-24",
          "versions": {
            "lerobot": "0.6.1"
          },
          "summary": "SO-101 follower has six STS3215 motors (five arm joints plus gripper).",
          "quote": "The follower arm uses 6x STS3215 motors with 1/345 gearing."
        },
        {
          "id": "ev-lerobot-action-repr-doc",
          "tier": "upstream-documented",
          "type": "official-docs",
          "url": "https://github.com/huggingface/lerobot/blob/v0.6.1/docs/source/action_representations.mdx",
          "date": "2026-08-03",
          "verified_on": "2026-10-24",
          "versions": {
            "lerobot": "0.6.1"
          },
          "summary": "LeRobot action-representation guide: joint-space action for a 6-DOF arm with a gripper.",
          "quote": "For a 6-DOF arm with a gripper, a joint-space action might look like:"
        },
        {
          "id": "ev-lerobot-smolvla-config",
          "tier": "upstream-documented",
          "type": "source-code",
          "url": "https://github.com/huggingface/lerobot/blob/v0.6.1/src/lerobot/policies/smolvla/configuration_smolvla.py",
          "date": "2026-08-03",
          "verified_on": "2026-10-24",
          "versions": {
            "lerobot": "0.6.1"
          },
          "summary": "SmolVLA pads actions to 32 dimensions.",
          "quote": "max_action_dim: int = 32"
        }
      ]
    },
    {
      "id": "diffusion-policy",
      "term": "Diffusion policy",
      "aka": [
        "Diffusion Policy",
        "action diffusion"
      ],
      "short": "A policy that generates action sequences by iteratively denoising random noise with a conditional diffusion model, introduced by Chi et al. in 2023.",
      "body": "A diffusion policy represents a robot's visuomotor policy as a conditional denoising diffusion process. Starting from Gaussian noise, a network conditioned on recent observations removes noise over a series of steps until a sequence of actions remains. The technique was introduced by Chi et al. in \"Diffusion Policy: Visuomotor Policy Learning via Action Diffusion\" (2023). The paper's central argument is that diffusion handles multimodal action distributions, such as two equally valid ways around an obstacle, where regressing to the mean would produce an action no demonstrator took.\n\nThe paper defines three horizons: observation horizon T_o, action prediction horizon T_p, and action execution horizon T_a. The robot executes T_a of the T_p predicted steps before re-planning (see [action horizon](/glossary/action-horizon)). LeRobot's [Diffusion Policy](/policy/diffusion) maps these to `n_obs_steps`, `horizon` and `n_action_steps`, with v0.6.1 defaults of 2, 64 and 32, and uses a DDPM noise scheduler with 100 training timesteps; inference uses the same number of steps unless `num_inference_steps` is set. [Multitask DiT](/policy/multitask-dit) extends the design with a diffusion transformer and language conditioning.\n\nThe term is overloaded. As a technique, \"diffusion policy\" means any policy whose action head is a diffusion model. As a model, it means Chi et al.'s architecture, trained from scratch per task. VLAs with diffusion or [flow-matching](/glossary/flow-matching) action heads are diffusion policies in the first sense, not the second.\n\nEvery denoising step is a full forward pass of the action network, so the step count dominates [inference latency](/glossary/inference-latency) on small compute such as a [Jetson Orin Nano](/compute/jetson-orin-nano-8gb).\n",
      "related": [
        "flow-matching",
        "action-horizon",
        "action-chunking",
        "behavioral-cloning",
        "inference-latency"
      ],
      "policies": [
        "diffusion",
        "multitask-dit"
      ],
      "sources": [
        {
          "id": "ev-chi-2023",
          "tier": "upstream-documented",
          "type": "paper",
          "url": "https://arxiv.org/abs/2303.04137",
          "date": "2023-03-07",
          "verified_on": "2026-10-24",
          "summary": "Chi et al. introduce Diffusion Policy.",
          "quote": "This paper introduces Diffusion Policy, a new way of generating robot behavior by representing a robot's visuomotor policy as a conditional denoising diffusion process."
        },
        {
          "id": "ev-chi-2023-full",
          "tier": "upstream-documented",
          "type": "paper",
          "url": "https://arxiv.org/html/2303.04137",
          "date": "2023-03-07",
          "verified_on": "2026-10-24",
          "summary": "Diffusion Policy executes part of each predicted action sequence before re-planning (observation, prediction and execution horizons).",
          "quote": "This encourages temporal action consistency while remaining responsive."
        },
        {
          "id": "ev-lerobot-diffusion-config",
          "tier": "upstream-documented",
          "type": "source-code",
          "url": "https://github.com/huggingface/lerobot/blob/v0.6.1/src/lerobot/policies/diffusion/configuration_diffusion.py",
          "date": "2026-08-03",
          "verified_on": "2026-10-24",
          "versions": {
            "lerobot": "0.6.1"
          },
          "summary": "LeRobot v0.6.1 Diffusion Policy defaults to DDPM with 100 training timesteps.",
          "quote": "num_train_timesteps: int = 100"
        }
      ]
    },
    {
      "id": "embodiment-gap",
      "term": "Embodiment gap",
      "aka": [
        "cross-embodiment gap",
        "embodiment mismatch"
      ],
      "short": "The differences in kinematics, action space, sensors and dynamics between the robots a policy was trained on and the robot it is deployed on.",
      "body": "The embodiment gap is the set of differences between the robot that produced a policy's training data and the robot the policy is asked to control. It covers the number and arrangement of joints ([degrees of freedom](/glossary/degrees-of-freedom)), the action representation (joint positions or end-effector deltas), gripper type, camera count and placement, [control rate](/glossary/control-rate), and actuator dynamics. A policy has no built-in way to map actions across these differences; it learns mappings only for the embodiments in its data.\n\nCross-embodiment pretraining tries to narrow the gap through data diversity. The Open X-Embodiment collaboration (2023) assembled \"a dataset from 22 different robots collected through a collaboration between 21 institutions\" and reported positive transfer for its RT-X models. Later VLAs make the embodiment explicit. LeRobot's X-VLA documentation starts from the problem that \"Robot data is fragmented across platforms, sensors, embodiments, and collection protocols\", and [X-VLA](/policy/xvla) learns soft-prompt embeddings per hardware setup. [GR00T N1.7](/policy/groot-n1-7) is described as a cross-embodiment policy. [SmolVLA](/policy/smolvla) pads state and action vectors to a fixed width of 32 in LeRobot v0.6.1 so that different robots fit one network.\n\nA common misconception is that a pretrained generalist VLA runs zero-shot on any robot. The OpenVLA-OFT paper (Kim et al., 2025) observes that VLAs \"struggle with novel robot setups and require fine-tuning to achieve good performance\". Deploying a pretrained checkpoint on a robot absent from its pretraining data, such as an [SO-101](/robot/so101), normally means collecting demonstrations on that robot and fine-tuning. The gap is distinct from [sim-to-real](/glossary/sim-to-real) transfer, which concerns one robot in two worlds.\n",
      "related": [
        "degrees-of-freedom",
        "vla",
        "sim-to-real",
        "affordance",
        "peft-lora"
      ],
      "policies": [
        "xvla",
        "groot-n1-7",
        "smolvla",
        "openvla-oft"
      ],
      "sources": [
        {
          "id": "ev-oxe-paper",
          "tier": "upstream-documented",
          "type": "paper",
          "url": "https://arxiv.org/abs/2310.08864",
          "date": "2023-10-13",
          "verified_on": "2026-10-24",
          "summary": "Open X-Embodiment: dataset from 22 robots; RT-X shows positive transfer.",
          "quote": "We assemble a dataset from 22 different robots collected through a collaboration between 21 institutions, demonstrating 527 skills (160266 tasks)."
        },
        {
          "id": "ev-oft-paper",
          "tier": "upstream-documented",
          "type": "paper",
          "url": "https://arxiv.org/abs/2502.19645",
          "date": "2025-02-27",
          "verified_on": "2026-10-24",
          "summary": "OpenVLA-OFT: VLAs need fine-tuning for novel robot setups.",
          "quote": "VLAs struggle with novel robot setups and require fine-tuning to achieve good performance"
        },
        {
          "id": "ev-lerobot-xvla-doc",
          "tier": "upstream-documented",
          "type": "official-docs",
          "url": "https://github.com/huggingface/lerobot/blob/v0.6.1/docs/source/xvla.mdx",
          "date": "2026-08-03",
          "verified_on": "2026-10-24",
          "versions": {
            "lerobot": "0.6.1"
          },
          "summary": "LeRobot X-VLA docs on embodiment heterogeneity.",
          "quote": "Robot data is fragmented across platforms, sensors, embodiments, and collection protocols."
        },
        {
          "id": "ev-lerobot-groot-doc",
          "tier": "upstream-documented",
          "type": "official-docs",
          "url": "https://github.com/huggingface/lerobot/blob/v0.6.1/docs/source/groot.mdx",
          "date": "2026-08-03",
          "verified_on": "2026-10-24",
          "versions": {
            "lerobot": "0.6.1"
          },
          "summary": "LeRobot describes GR00T as a cross-embodiment policy.",
          "quote": "It is a cross-embodiment policy that accepts multimodal input, including language, images, and proprioception"
        },
        {
          "id": "ev-lerobot-smolvla-config",
          "tier": "upstream-documented",
          "type": "source-code",
          "url": "https://github.com/huggingface/lerobot/blob/v0.6.1/src/lerobot/policies/smolvla/configuration_smolvla.py",
          "date": "2026-08-03",
          "verified_on": "2026-10-24",
          "versions": {
            "lerobot": "0.6.1"
          },
          "summary": "SmolVLA pads actions to 32 dimensions in LeRobot v0.6.1.",
          "quote": "max_action_dim: int = 32"
        }
      ]
    },
    {
      "id": "flow-matching",
      "term": "Flow matching",
      "aka": [
        "conditional flow matching",
        "CFM"
      ],
      "short": "A generative training method that regresses a velocity field carrying noise to data; VLAs such as π0 and SmolVLA use it to generate continuous action chunks.",
      "body": "Flow matching trains a network to predict a velocity field that transports samples from a simple noise distribution to the data distribution along a chosen probability path. At inference the field is integrated with an ODE solver over a fixed number of steps. It was introduced by Lipman, Chen, Ben-Hamu, Nickel and Le in \"Flow Matching for Generative Modeling\" (2022), which describes it as \"a simulation-free approach for training CNFs based on regressing vector fields of fixed conditional probability paths\". Diffusion paths are a special case; straight optimal-transport paths are the usual choice.\n\nIn robot policies the generated sample is an [action chunk](/glossary/action-chunking), conditioned on images, language and [proprioceptive](/glossary/proprioception) state. [π0](/policy/pi0) (Black et al., 2024) attaches a flow-matching action expert to a pretrained vision-language model. LeRobot documents flow-matching action heads for [π0.5](/policy/pi05), [SmolVLA](/policy/smolvla), [GR00T N1.7](/policy/groot-n1-7), [X-VLA](/policy/xvla), [EO-1](/policy/eo1) and [EVO-1](/policy/evo1). In LeRobot v0.6.1, π0 defaults to `num_inference_steps=10` and SmolVLA to `num_steps=10`: each chunk costs ten passes through the action network, so the step count belongs in any [inference latency](/glossary/inference-latency) figure.\n\nA common misconception is that flow matching and diffusion are unrelated families. The π0 paper itself calls flow matching \"a variant of diffusion\". The practical differences are the training target (a velocity rather than noise) and, typically, fewer integration steps at inference. Both are iterative samplers, so both pay a per-step cost on the robot, and [real-time chunking](/glossary/real-time-chunking) was designed for policies of either kind.\n",
      "related": [
        "diffusion-policy",
        "action-chunking",
        "vla",
        "inference-latency",
        "real-time-chunking"
      ],
      "policies": [
        "pi0",
        "pi05",
        "smolvla",
        "groot-n1-7",
        "xvla",
        "eo1",
        "evo1"
      ],
      "sources": [
        {
          "id": "ev-lipman-2022",
          "tier": "upstream-documented",
          "type": "paper",
          "url": "https://arxiv.org/abs/2210.02747",
          "date": "2022-10-06",
          "verified_on": "2026-10-24",
          "summary": "Lipman et al. introduce Flow Matching.",
          "quote": "we present the notion of Flow Matching (FM), a simulation-free approach for training CNFs based on regressing vector fields of fixed conditional probability paths."
        },
        {
          "id": "ev-pi0-paper",
          "tier": "upstream-documented",
          "type": "paper",
          "url": "https://arxiv.org/abs/2410.24164",
          "date": "2024-10-31",
          "verified_on": "2026-10-24",
          "summary": "π0 uses a flow-matching architecture on top of a pretrained VLM.",
          "quote": "We propose a novel flow matching architecture built on top of a pre-trained vision-language model (VLM) to inherit Internet-scale semantic knowledge."
        },
        {
          "id": "ev-pi0-paper-full",
          "tier": "upstream-documented",
          "type": "paper",
          "url": "https://arxiv.org/html/2410.24164",
          "date": "2024-10-31",
          "verified_on": "2026-10-24",
          "summary": "π0 paper describes flow matching as a variant of diffusion.",
          "quote": "flow matching (a variant of diffusion) to represent complex continuous action distributions"
        },
        {
          "id": "ev-lerobot-rtc-doc",
          "tier": "upstream-documented",
          "type": "official-docs",
          "url": "https://github.com/huggingface/lerobot/blob/v0.6.1/docs/source/rtc.mdx",
          "date": "2026-08-03",
          "verified_on": "2026-10-24",
          "versions": {
            "lerobot": "0.6.1"
          },
          "summary": "LeRobot RTC docs name Pi0, Pi0.5 and SmolVLA as flow-matching policies.",
          "quote": "large, flow-matching based robotic policies, such as [Pi0](./pi0), [Pi0.5](./pi05), and [SmolVLA](./smolvla)"
        },
        {
          "id": "ev-lerobot-pi0-config",
          "tier": "upstream-documented",
          "type": "source-code",
          "url": "https://github.com/huggingface/lerobot/blob/v0.6.1/src/lerobot/policies/pi0/configuration_pi0.py",
          "date": "2026-08-03",
          "verified_on": "2026-10-24",
          "versions": {
            "lerobot": "0.6.1"
          },
          "summary": "LeRobot v0.6.1 π0 uses 10 denoising steps by default.",
          "quote": "num_inference_steps: int = 10  # Number of denoising steps during inference"
        }
      ]
    },
    {
      "id": "human-in-the-loop-rl",
      "term": "Human-in-the-loop reinforcement learning",
      "aka": [
        "HIL-RL",
        "HIL-SERL",
        "human-in-the-loop RL"
      ],
      "short": "Real-robot reinforcement learning in which a human supplies demonstrations and takes over during training to correct the policy, making on-hardware RL practical.",
      "body": "Human-in-the-loop reinforcement learning (HIL-RL) is reinforcement learning on a physical robot in which a person takes part during training: providing initial demonstrations, intervening to correct or stop the policy, and sometimes labeling success. Pure RL on hardware is usually impractical because random exploration is slow and unsafe; human input shapes exploration so that a policy can be learned in hours.\n\nThe reference system is HIL-SERL, from Luo, Xu, Wu and Levine, \"Precise and Dexterous Robotic Manipulation via Human-in-the-Loop Reinforcement Learning\" (2024). It \"integrates demonstrations and human corrections, efficient RL algorithms, and other system-level design choices\", and reports near-perfect success rates within 1 to 2.5 hours of training, with an average 2x improvement in success rate and 1.8x faster execution over imitation-learning baselines.\n\nLeRobot v0.6.1 implements it as [HIL-SERL](/policy/hil-serl). The workflow starts from a few teleoperated demonstrations, trains a vision-based [reward model](/glossary/reward-model) (a success classifier) on them, then runs an actor on the robot and a Soft Actor-Critic learner in parallel. The operator can take over at any time with a gamepad, keyboard or [leader arm](/glossary/leader-follower-arm), and those interventions are recorded. Joint or end-effector bounds and image cropping keep the hardware safe and the data useful.\n\nA common misconception is that HIL-RL is [DAgger](/glossary/dagger) with extra steps. DAgger uses expert labels as supervised targets, so the result imitates the expert. HIL-SERL optimizes a reward, so the final policy can be faster and more reliable than [behavioral cloning](/glossary/behavioral-cloning) on the same demonstrations. It is also task-specific: each run trains one policy for one task on one setup, unlike a pretrained [VLA](/glossary/vla).\n",
      "related": [
        "reward-model",
        "dagger",
        "imitation-learning",
        "leader-follower-arm",
        "teleoperation"
      ],
      "policies": [
        "hil-serl"
      ],
      "sources": [
        {
          "id": "ev-hilserl-paper",
          "tier": "upstream-documented",
          "type": "paper",
          "url": "https://arxiv.org/abs/2410.21845",
          "date": "2024-10-29",
          "verified_on": "2026-10-24",
          "summary": "Luo et al. introduce HIL-SERL.",
          "quote": "Our approach integrates demonstrations and human corrections, efficient RL algorithms, and other system-level design choices to learn policies that achieve near-perfect success rates and fast cycle times within just 1 to 2.5 hours of training."
        },
        {
          "id": "ev-lerobot-hilserl-doc",
          "tier": "upstream-documented",
          "type": "official-docs",
          "url": "https://github.com/huggingface/lerobot/blob/v0.6.1/docs/source/hilserl.mdx",
          "date": "2026-08-03",
          "verified_on": "2026-10-24",
          "versions": {
            "lerobot": "0.6.1"
          },
          "summary": "LeRobot v0.6.1 HIL-SERL workflow.",
          "quote": "HIL-SERL is a sample-efficient reinforcement learning algorithm that combines human demonstrations with online learning and human interventions."
        },
        {
          "id": "ev-lerobot-hilserl-teleop",
          "tier": "upstream-documented",
          "type": "official-docs",
          "url": "https://github.com/huggingface/lerobot/blob/v0.6.1/docs/source/hilserl.mdx",
          "date": "2026-08-03",
          "verified_on": "2026-10-24",
          "versions": {
            "lerobot": "0.6.1"
          },
          "summary": "LeRobot HIL-SERL supports gamepad, keyboard or leader arm interventions.",
          "quote": "We support using a gamepad or a keyboard or the leader arm of the robot."
        }
      ]
    },
    {
      "id": "imitation-learning",
      "term": "Imitation learning",
      "aka": [
        "IL",
        "learning from demonstration",
        "LfD"
      ],
      "short": "Learning a policy from demonstrations of the desired behavior rather than from a hand-written controller or a reward signal.",
      "body": "Imitation learning (IL), also called learning from demonstration, trains a policy from examples of an expert performing the task. Osa et al.'s survey \"An Algorithmic Perspective on Imitation Learning\" (2018) defines it directly: \"This process of learning from demonstrations, and the study of algorithms to do so, is called imitation learning.\" The alternatives are hand-engineering a controller and reinforcement learning, which optimizes a reward signal through trial and error.\n\nIL methods fall into three broad groups. [Behavioral cloning](/glossary/behavioral-cloning) treats the problem as supervised learning on recorded observation–action pairs. Interactive methods such as [DAgger](/glossary/dagger) query the expert on states the learner itself visits. Inverse reinforcement learning infers a reward function from demonstrations and then optimizes it.\n\nNearly every manipulation policy in this matrix, including [ACT](/policy/act), [Diffusion Policy](/policy/diffusion), [SmolVLA](/policy/smolvla) and the fine-tuning stage of [π0](/policy/pi0), is trained by behavioral cloning on demonstrations collected through [teleoperation](/glossary/teleoperation), typically with a [leader arm](/glossary/leader-follower-arm), and stored as a [LeRobotDataset](/glossary/lerobot-dataset). The data requirement can be small: the ACT paper's ALOHA tasks used 50 demonstrations each for most tasks, \"around 10-20 minutes of data for each task\".\n\nA common misconception is that IL is limited only by the demonstrator's skill. The deeper limit is distribution shift: a cloned policy fails in states its demonstrations never covered, and its own errors lead it there. Pure IL also has no mechanism to improve on the demonstrations. Methods such as [HIL-SERL](/policy/hil-serl) add reinforcement learning with a [reward model](/glossary/reward-model) and report higher success and faster execution than imitation-only baselines (see [human-in-the-loop RL](/glossary/human-in-the-loop-rl)).\n",
      "related": [
        "behavioral-cloning",
        "dagger",
        "teleoperation",
        "human-in-the-loop-rl",
        "reward-model"
      ],
      "policies": [
        "act",
        "diffusion",
        "smolvla",
        "pi0",
        "hil-serl"
      ],
      "sources": [
        {
          "id": "ev-osa-2018",
          "tier": "upstream-documented",
          "type": "paper",
          "url": "https://arxiv.org/abs/1811.06711",
          "date": "2018-11-16",
          "verified_on": "2026-10-24",
          "summary": "Osa et al. define imitation learning.",
          "quote": "This process of learning from demonstrations, and the study of algorithms to do so, is called imitation learning."
        },
        {
          "id": "ev-act-paper-full",
          "tier": "upstream-documented",
          "type": "paper",
          "url": "https://arxiv.org/html/2304.13705",
          "date": "2023-04-23",
          "verified_on": "2026-10-24",
          "summary": "ACT: about 10-20 minutes of demonstrations per task.",
          "quote": "The total amount for demonstrations is thus around 10-20 minutes of data for each task"
        }
      ]
    },
    {
      "id": "inference-latency",
      "term": "Inference latency",
      "aka": [
        "policy latency",
        "inference time",
        "inference_latency_ms"
      ],
      "short": "Wall-clock time for one policy call from observation to action chunk. Not the same as control rate: action chunking decouples the two.",
      "body": "Inference latency is the wall-clock time from handing an observation to the policy to receiving its output; for chunked policies, that output is one full [action chunk](/glossary/action-chunking). It includes image encoding, the language-model forward pass, every denoising or decoding step and, if the model runs off-board, network transfer. It is measured in milliseconds and should be reported as a distribution (p50 and p95), not a single best run.\n\nThe most common error in published comparison tables is treating latency and [control rate](/glossary/control-rate) as if one implied the other. Chunking decouples them. The π0 paper (Black et al., 2024) measured 73 ms of on-board inference on an [RTX 4090](/compute/rtx-4090) with three camera images, and ran its robots at 50 Hz by calling the model every 25 actions. 73 ms does not mean \"13.7 Hz control\", and \"50 Hz\" does not mean a 20 ms model. Converting between them requires the number of actions executed per call.\n\nA latency figure is only comparable if it states the conditions that dominate it: hardware, runtime (PyTorch eager, `torch.compile`, TensorRT), numeric precision (see [quantization](/glossary/quantization)), camera count and resolution, and the number of denoising steps. This matrix rejects measurements that omit runtime, precision or camera count.\n\nLatency matters even when throughput is sufficient. Under synchronous inference the robot idles for the full latency at every chunk boundary. [Async inference](/glossary/async-inference) hides that idle time, and [real-time chunking](/glossary/real-time-chunking) keeps motion continuous when latency is a large fraction of the chunk; the RTC paper reports success with delays above 300 ms. On small devices such as the [Jetson Orin Nano](/compute/jetson-orin-nano-8gb), latency often decides whether a policy is usable at all.\n",
      "related": [
        "control-rate",
        "action-chunking",
        "quantization",
        "async-inference",
        "real-time-chunking",
        "action-horizon"
      ],
      "policies": [
        "pi0",
        "smolvla",
        "pi05"
      ],
      "sources": [
        {
          "id": "ev-pi0-paper-full",
          "tier": "upstream-documented",
          "type": "paper",
          "url": "https://arxiv.org/html/2410.24164",
          "date": "2024-10-31",
          "verified_on": "2026-10-24",
          "summary": "π0 inference timed on an RTX 4090 with 3 camera images (73 ms on-board total, Table I).",
          "quote": "Table I summarizes the computation time for this operation with 3 camera images. The operations were timed on an NVIDIA GeForce RTX 4090 consumer-grade GPU."
        },
        {
          "id": "ev-pi0-paper-full-rate",
          "tier": "upstream-documented",
          "type": "paper",
          "url": "https://arxiv.org/html/2410.24164",
          "date": "2024-10-31",
          "verified_on": "2026-10-24",
          "summary": "π0 runs 50 Hz robots with inference every 25 actions.",
          "quote": "for all other robots, which run at 50Hz, we run inference every 0.5 seconds (after executing 25 actions)."
        },
        {
          "id": "ev-rtc-paper-full",
          "tier": "upstream-documented",
          "type": "paper",
          "url": "https://arxiv.org/html/2506.07339",
          "date": "2025-06-09",
          "verified_on": "2026-10-24",
          "summary": "RTC robust to inference delays over 300 ms.",
          "quote": "even in the presence of inference delays in excess of 300 milliseconds"
        }
      ]
    },
    {
      "id": "leader-follower-arm",
      "term": "Leader–follower arm",
      "aka": [
        "leader arm",
        "follower arm",
        "puppeteering",
        "joint-space teleoperation"
      ],
      "short": "A teleoperation setup in which a human moves a passive 'leader' arm and a kinematically matched 'follower' robot copies its joint positions in real time.",
      "body": "A leader–follower arm pair is a joint-space [teleoperation](/glossary/teleoperation) device. The operator physically moves the leader, an arm with the same kinematic structure as the robot, and the follower is commanded to match the leader's joint positions in real time. Because joints map one to one, no inverse kinematics is needed, and the operator directly controls the arm's full configuration, including wrist orientation.\n\nThe design became standard in low-cost robot learning with ALOHA, described by Zhao et al. (2023): \"The user teleoperates by backdriving the leader robots, with the follower robots mirroring the motion.\" GELLO (Wu et al., 2023) generalized it to industrial arms, building 3D-printed controllers with \"the same kinematic structure as the target arm\".\n\nIn this matrix, the [SO-100](/robot/so100), [SO-101](/robot/so101) and [Koch](/robot/koch) arms are used as leader–follower pairs in LeRobot, and [bi-SO](/robot/bi-so) is two pairs. On the SO-101, the follower uses six STS3215 servos with 1/345 gearing, while the leader mixes 1/191, 1/345 and 1/147 ratios so that it can hold its own weight yet \"be moved without requiring much force\". LeRobot's HIL-SERL guide notes that the SO-101 leader's gearing lets it track the follower during exploration, which makes human takeover smoother than with the SO-100.\n\nTwo misconceptions recur. A leader is not plug-and-play with any follower: joint zero positions, directions and limits must be calibrated so that leader readings map to follower targets, and LeRobot stores a calibration per device `id`. And a passive leader records only what the operator does; it provides no force feedback, so contact-rich demonstrations rely on the operator's vision.\n",
      "related": [
        "teleoperation",
        "degrees-of-freedom",
        "imitation-learning",
        "human-in-the-loop-rl"
      ],
      "policies": [
        "act",
        "smolvla",
        "hil-serl"
      ],
      "sources": [
        {
          "id": "ev-act-paper-full",
          "tier": "upstream-documented",
          "type": "paper",
          "url": "https://arxiv.org/html/2304.13705",
          "date": "2023-04-23",
          "verified_on": "2026-10-24",
          "summary": "ALOHA: leader robots are backdriven, followers mirror them.",
          "quote": "The user teleoperates by backdriving the leader robots, with the follower robots mirroring the motion."
        },
        {
          "id": "ev-gello-paper",
          "tier": "upstream-documented",
          "type": "paper",
          "url": "https://arxiv.org/abs/2309.13037",
          "date": "2023-09-22",
          "verified_on": "2026-10-24",
          "summary": "GELLO builds controllers kinematically matched to the target arm.",
          "quote": "we build a GELLO controller device that has the same kinematic structure as the target arm"
        },
        {
          "id": "ev-lerobot-so101-doc",
          "tier": "upstream-documented",
          "type": "official-docs",
          "url": "https://github.com/huggingface/lerobot/blob/v0.6.1/docs/source/so101.mdx",
          "date": "2026-08-03",
          "verified_on": "2026-10-24",
          "versions": {
            "lerobot": "0.6.1"
          },
          "summary": "LeRobot SO-101 docs: follower and leader motor gearing.",
          "quote": "The follower arm uses 6x STS3215 motors with 1/345 gearing. The leader, however, uses three differently geared motors to make sure it can both sustain its own weight and it can be moved without requiring much force."
        },
        {
          "id": "ev-lerobot-hilserl-doc",
          "tier": "upstream-documented",
          "type": "official-docs",
          "url": "https://github.com/huggingface/lerobot/blob/v0.6.1/docs/source/hilserl.mdx",
          "date": "2026-08-03",
          "verified_on": "2026-10-24",
          "versions": {
            "lerobot": "0.6.1"
          },
          "summary": "LeRobot HIL-SERL: SO-101 leader tracks the follower, smoother takeover than SO-100.",
          "quote": "The SO101 leader arm has reduced gears that allows it to move and track the follower arm during exploration."
        },
        {
          "id": "ev-lerobot-il-robots",
          "tier": "upstream-documented",
          "type": "official-docs",
          "url": "https://github.com/huggingface/lerobot/blob/v0.6.1/docs/source/il_robots.mdx",
          "date": "2026-08-03",
          "verified_on": "2026-10-24",
          "versions": {
            "lerobot": "0.6.1"
          },
          "summary": "LeRobot stores calibration per robot id.",
          "quote": "Note that the `id` associated with a robot is used to store the calibration file."
        }
      ]
    },
    {
      "id": "lerobot-dataset",
      "term": "LeRobotDataset",
      "aka": [
        "LeRobotDataset v3.0",
        "LeRobot dataset format"
      ],
      "short": "Hugging Face LeRobot's standard format for robot demonstration data: Parquet for states and actions, MP4 for camera streams, plus episode metadata.",
      "body": "LeRobotDataset is the data format and loader used by Hugging Face's LeRobot library to record, share and train on robot demonstrations. The current format, v3.0, has three parts. Low-dimensional, high-frequency signals (\"states, actions, timestamps\") are stored in Apache Parquet. Camera frames are encoded into MP4, with videos sharded per camera. Metadata in JSON and Parquet records feature names, dtypes, shapes, frame rates, normalization statistics, and the start and end offsets of each episode.\n\nThe main change from v2.1 is file-based storage: v3 packs \"many episodes per Parquet/MP4 file\" where v2 used one file per episode, and episodes are reconstructed from metadata rather than from filenames. Datasets can be streamed from the Hugging Face Hub with `StreamingLeRobotDataset` without downloading them first. The docs state that v3.0 ships with lerobot 0.4.0 and later, and provide a converter for v2.1 datasets.\n\nEach frame exposes standard keys: `observation.state` ([proprioception](/glossary/proprioception)), one `observation.images.<camera>` key per camera, and `action`. For an [SO-101](/robot/so101) recorded with `lerobot-record`, state and action are six joint positions. The frame rate is fixed per dataset and is the [control rate](/glossary/control-rate) the trained policy should run at.\n\nA common misconception is that \"LeRobot format\" makes any dataset usable on any robot. The format standardizes storage, not semantics. Two datasets can both be valid LeRobotDatasets yet have different joint orders, units, camera names or action spaces, and a policy such as [ACT](/policy/act), [SmolVLA](/policy/smolvla) or [π0](/policy/pi0) trained on one will not run on the other's robot without matching features (see [embodiment gap](/glossary/embodiment-gap)).\n",
      "related": [
        "teleoperation",
        "proprioception",
        "control-rate",
        "imitation-learning"
      ],
      "policies": [
        "act",
        "smolvla",
        "pi0",
        "diffusion"
      ],
      "sources": [
        {
          "id": "ev-lerobot-dataset-v3-doc",
          "tier": "upstream-documented",
          "type": "official-docs",
          "url": "https://github.com/huggingface/lerobot/blob/v0.6.1/docs/source/lerobot-dataset-v3.mdx",
          "date": "2026-08-03",
          "verified_on": "2026-10-24",
          "versions": {
            "lerobot": "0.6.1"
          },
          "summary": "LeRobotDataset v3.0 layout: Parquet tabular data, MP4 video, metadata.",
          "quote": "Low‑dimensional, high‑frequency signals (states, actions, timestamps) stored in **Apache Parquet**."
        },
        {
          "id": "ev-lerobot-dataset-v3-files",
          "tier": "upstream-documented",
          "type": "official-docs",
          "url": "https://github.com/huggingface/lerobot/blob/v0.6.1/docs/source/lerobot-dataset-v3.mdx",
          "date": "2026-08-03",
          "verified_on": "2026-10-24",
          "versions": {
            "lerobot": "0.6.1"
          },
          "summary": "v3 stores many episodes per file.",
          "quote": "Many episodes per Parquet/MP4 file (v2 used one file per episode)."
        }
      ]
    },
    {
      "id": "peft-lora",
      "term": "Parameter-efficient fine-tuning (PEFT) and LoRA",
      "aka": [
        "PEFT",
        "LoRA",
        "Low-Rank Adaptation"
      ],
      "short": "Adapting a pretrained model by training a small set of added parameters while the original weights stay frozen; LoRA learns low-rank update matrices.",
      "body": "Parameter-efficient fine-tuning (PEFT) adapts a large pretrained model to a new task by training only a small number of parameters while the original weights stay frozen. Low-Rank Adaptation (LoRA) is the most widely used method. Hu et al. introduced it in \"LoRA: Low-Rank Adaptation of Large Language Models\" (2021): it \"freezes the pre-trained model weights and injects trainable rank decomposition matrices into each layer of the Transformer architecture\". For a weight matrix W, LoRA learns an update BA whose inner rank r is small. After training, BA can be merged into W, so a merged adapter adds no inference latency.\n\nFor VLAs, LoRA cuts the memory needed to fine-tune, because gradients and optimizer state exist only for the adapter. The OpenVLA paper (Kim et al., 2024) reports fine-tuning its 7B model \"on consumer GPUs via modern low-rank adaptation methods\". LeRobot v0.6.1 integrates the Hugging Face PEFT library as `lerobot[peft]`; its example fine-tunes [SmolVLA](/policy/smolvla) on LIBERO with `--peft.method_type=LORA --peft.r=64 --peft.lora_alpha=64`, where the scaling factor is `lora_alpha / r`.\n\nTwo misconceptions recur. First, LoRA reduces *training* memory, not deployment memory: inference still loads the full base weights, so a LoRA-tuned [π0](/policy/pi0) needs the same GPU memory at run time as the base model. Lower deployment memory comes from [quantization](/glossary/quantization), not LoRA. Second, LoRA is not free in quality; the LeRobot docs note that the higher the rank, the closer the result gets to full fine-tuning. Whether a result used LoRA or full fine-tuning is part of what makes it reproducible on a given GPU, such as an [RTX 4090](/compute/rtx-4090).\n",
      "related": [
        "quantization",
        "vla",
        "embodiment-gap"
      ],
      "policies": [
        "smolvla",
        "pi0",
        "openvla"
      ],
      "sources": [
        {
          "id": "ev-lora-paper",
          "tier": "upstream-documented",
          "type": "paper",
          "url": "https://arxiv.org/abs/2106.09685",
          "date": "2021-06-17",
          "verified_on": "2026-10-24",
          "summary": "Hu et al. introduce LoRA.",
          "quote": "We propose Low-Rank Adaptation, or LoRA, which freezes the pre-trained model weights and injects trainable rank decomposition matrices into each layer of the Transformer architecture"
        },
        {
          "id": "ev-openvla-paper",
          "tier": "upstream-documented",
          "type": "paper",
          "url": "https://arxiv.org/abs/2406.09246",
          "date": "2024-06-13",
          "verified_on": "2026-10-24",
          "summary": "OpenVLA can be fine-tuned on consumer GPUs with low-rank adaptation.",
          "quote": "we show that OpenVLA can be fine-tuned on consumer GPUs via modern low-rank adaptation methods"
        },
        {
          "id": "ev-lerobot-peft-doc",
          "tier": "upstream-documented",
          "type": "official-docs",
          "url": "https://github.com/huggingface/lerobot/blob/v0.6.1/docs/source/peft_training.mdx",
          "date": "2026-08-03",
          "verified_on": "2026-10-24",
          "versions": {
            "lerobot": "0.6.1"
          },
          "summary": "LeRobot v0.6.1 PEFT/LoRA fine-tuning guide.",
          "quote": "This rank can be specified using the `--peft.r` parameter, and the LoRA scaling factor with\n`--peft.lora_alpha` (where `scaling = lora_alpha / r`)."
        }
      ]
    },
    {
      "id": "proprioception",
      "term": "Proprioception",
      "aka": [
        "proprioceptive state",
        "robot state",
        "observation.state"
      ],
      "short": "The robot's measurement of its own body state, such as joint positions, velocities and gripper opening, given to a policy alongside camera images.",
      "body": "Proprioception is a robot's measurement of its own configuration: joint positions and, depending on the hardware, joint velocities, motor currents or torques, and gripper state. The word is borrowed from physiology, where it names the body's sense of its own position and movement. In robot learning, the proprioceptive vector is the low-dimensional \"state\" input that accompanies camera images.\n\nThe π0 paper (Black et al., 2024) defines its observation as \"multiple RGB images, a language command, and the robot's proprioceptive state\". In a [LeRobotDataset](/glossary/lerobot-dataset) it is the `observation.state` feature, stored per frame next to `action`. For an [SO-101](/robot/so101) driven by LeRobot's `so_follower` code, state is one position reading per motor, `{motor}.pos`, for its six motors (five arm joints and the gripper). Its dimensionality therefore follows the robot's [degrees of freedom](/glossary/degrees-of-freedom).\n\nPolicies treat state differently. [SmolVLA](/policy/smolvla) pads it to `max_state_dim=32` in LeRobot v0.6.1 so that robots with different joint counts share one network. LeRobot describes [GR00T N1.7](/policy/groot-n1-7) as accepting \"language, images, and proprioception\". [ACT](/policy/act) feeds joint positions to its transformer alongside image features.\n\nTwo misconceptions are common. First, proprioception is not touch. Joint encoders report where the arm is; contact forces at the fingertips require tactile or force-torque sensing, or must be inferred indirectly from motor current. Second, state values are not portable between robots, or even between calibrations of the same model. They depend on joint zero positions, units and normalization, which is one reason a checkpoint trained on one arm does not transfer unchanged to another (see [embodiment gap](/glossary/embodiment-gap)).\n",
      "related": [
        "degrees-of-freedom",
        "lerobot-dataset",
        "vla",
        "embodiment-gap"
      ],
      "policies": [
        "pi0",
        "smolvla",
        "groot-n1-7",
        "act"
      ],
      "sources": [
        {
          "id": "ev-pi0-paper-full",
          "tier": "upstream-documented",
          "type": "paper",
          "url": "https://arxiv.org/html/2410.24164",
          "date": "2024-10-31",
          "verified_on": "2026-10-24",
          "summary": "π0 observations include proprioceptive state.",
          "quote": "The observation consists of multiple RGB images, a language command, and the robot’s proprioceptive state"
        },
        {
          "id": "ev-lerobot-so-follower",
          "tier": "upstream-documented",
          "type": "source-code",
          "url": "https://github.com/huggingface/lerobot/blob/v0.6.1/src/lerobot/robots/so_follower/so_follower.py",
          "date": "2026-08-03",
          "verified_on": "2026-10-24",
          "versions": {
            "lerobot": "0.6.1"
          },
          "summary": "LeRobot SO follower exposes one position feature per motor.",
          "quote": "return {f\"{motor}.pos\": float for motor in self.bus.motors}"
        },
        {
          "id": "ev-lerobot-smolvla-config",
          "tier": "upstream-documented",
          "type": "source-code",
          "url": "https://github.com/huggingface/lerobot/blob/v0.6.1/src/lerobot/policies/smolvla/configuration_smolvla.py",
          "date": "2026-08-03",
          "verified_on": "2026-10-24",
          "versions": {
            "lerobot": "0.6.1"
          },
          "summary": "SmolVLA pads state to 32 dimensions in LeRobot v0.6.1.",
          "quote": "max_state_dim: int = 32"
        },
        {
          "id": "ev-lerobot-groot-doc",
          "tier": "upstream-documented",
          "type": "official-docs",
          "url": "https://github.com/huggingface/lerobot/blob/v0.6.1/docs/source/groot.mdx",
          "date": "2026-08-03",
          "verified_on": "2026-10-24",
          "versions": {
            "lerobot": "0.6.1"
          },
          "summary": "LeRobot: GR00T accepts language, images and proprioception.",
          "quote": "It is a cross-embodiment policy that accepts multimodal input, including language, images, and proprioception"
        }
      ]
    },
    {
      "id": "quantization",
      "term": "Quantization and numeric precision",
      "aka": [
        "fp32",
        "bf16",
        "fp16",
        "int8",
        "int4",
        "reduced precision"
      ],
      "short": "Running a model's weights or activations at lower numeric precision (bf16, fp16, int8, int4) to save memory and often time; every latency number must state it.",
      "body": "Quantization, in the broad sense, is running a neural network at lower numeric precision than the fp32 it was trained in. Strictly, *quantization* maps values to integers (int8, int4) plus scale factors; casting to 16-bit floating point (fp16, bf16) is reduced precision rather than quantization, but the trade-offs are discussed together. Jacob et al. (2017), \"Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference\", established the standard integer scheme.\n\nThe formats differ in more than size. fp32 uses 32 bits. fp16 and bf16 both use 16 bits, but bf16 keeps fp32's exponent width, so \"the range of values it can represent is the same as that of IEEE 754 floating-point format (FP32)\" with less mantissa precision; fp16 has a narrower range. int8 and int4 store weights in 8 or 4 bits plus scales.\n\nPrecision changes speed, memory and success, and not always in the expected direction. The OpenVLA paper (Kim et al., 2024) served its 7B model on BridgeData V2 tasks at three precisions: bf16 used 16.8 GB at 71.3% success, int8 10.2 GB at 58.1%, and int4 7.0 GB at 71.9%. It observed that \"8-bit quantization slows down inference across most GPUs, due to the overhead of the added quantization operations\". Lower precision is not automatically faster.\n\nThis is why a latency or memory figure without precision is unusable. LeRobot v0.6.1 loads [π0](/policy/pi0) and [π0.5](/policy/pi05) in float32 by default, with bfloat16 as an option, so \"π0 on an [RTX 4090](/compute/rtx-4090)\" can mean two different measurements. On memory-limited devices such as the [Jetson Orin Nano](/compute/jetson-orin-nano-8gb), precision can decide whether a model loads at all. Every measurement in this matrix must state precision, runtime and camera count (see [inference latency](/glossary/inference-latency)).\n",
      "related": [
        "inference-latency",
        "peft-lora",
        "vla",
        "control-rate"
      ],
      "policies": [
        "openvla",
        "pi0",
        "pi05",
        "smolvla"
      ],
      "sources": [
        {
          "id": "ev-jacob-2017",
          "tier": "upstream-documented",
          "type": "paper",
          "url": "https://arxiv.org/abs/1712.05877",
          "date": "2017-12-15",
          "verified_on": "2026-10-24",
          "summary": "Jacob et al. introduce integer-arithmetic-only quantized inference.",
          "quote": "We propose a quantization scheme that allows inference to be carried out using integer-only arithmetic, which can be implemented more efficiently than floating point inference on commonly available integer-only hardware."
        },
        {
          "id": "ev-bf16-paper",
          "tier": "upstream-documented",
          "type": "paper",
          "url": "https://arxiv.org/abs/1905.12322",
          "date": "2019-05-29",
          "verified_on": "2026-10-24",
          "summary": "BFLOAT16 has the same range as FP32.",
          "quote": "the range of values it can represent is the same as that of IEEE 754 floating-point format (FP32) and conversion to/from FP32 is simple."
        },
        {
          "id": "ev-openvla-paper-full",
          "tier": "upstream-documented",
          "type": "paper",
          "url": "https://arxiv.org/html/2406.09246",
          "date": "2024-06-13",
          "verified_on": "2026-10-24",
          "summary": "OpenVLA quantized inference: bf16 16.8 GB 71.3%, int8 10.2 GB 58.1%, int4 7.0 GB 71.9% (Table 2); int8 slower on most GPUs.",
          "quote": "We observe that 8-bit quantization slows down inference across most GPUs, due to the overhead of the added quantization operations."
        },
        {
          "id": "ev-lerobot-pi0-config",
          "tier": "upstream-documented",
          "type": "source-code",
          "url": "https://github.com/huggingface/lerobot/blob/v0.6.1/src/lerobot/policies/pi0/configuration_pi0.py",
          "date": "2026-08-03",
          "verified_on": "2026-10-24",
          "versions": {
            "lerobot": "0.6.1"
          },
          "summary": "LeRobot v0.6.1 π0 defaults to float32, bfloat16 optional.",
          "quote": "dtype: str = \"float32\"  # Options: \"bfloat16\", \"float32\""
        }
      ]
    },
    {
      "id": "real-time-chunking",
      "term": "Real-time chunking",
      "aka": [
        "RTC"
      ],
      "short": "An inference-time method that generates the next action chunk while the current one executes, freezing actions that will run anyway and inpainting the rest.",
      "body": "Real-time chunking (RTC) is an inference-time algorithm for executing [action chunks](/glossary/action-chunking) from diffusion- or flow-based policies without pauses or jumps at chunk boundaries. It was introduced by Black, Galliker and Levine of Physical Intelligence in \"Real-Time Execution of Action Chunking Flow Policies\" (2025).\n\nThe policy starts generating the next chunk while the robot is still executing the current one. Because of [inference latency](/glossary/inference-latency), the first actions of the new chunk will already have been executed by the time it arrives. RTC \"freezes\" those actions to match the previous chunk and treats the rest as an inpainting problem, guiding the denoising so that the new chunk joins the old one smoothly. It needs no retraining. The paper reports precise tasks such as lighting a match succeeding \"even in the presence of inference delays in excess of 300 milliseconds, corresponding to more than 30% of the model's prediction horizon\".\n\nLeRobot v0.6.1 documents RTC for flow-matching policies such as [π0](/policy/pi0), [π0.5](/policy/pi05) and [SmolVLA](/policy/smolvla). Measurements in this matrix record it with an `rtc_enabled` flag, because the same model on the same hardware behaves differently with and without it.\n\nRTC is often confused with [asynchronous inference](/glossary/async-inference). Async inference decides *when* to request a new chunk so that the action queue never empties. RTC changes *how* the new chunk is generated so that it agrees with what is already executing. Naive asynchronous execution without RTC can switch abruptly between chunks that disagree. RTC also differs from temporal ensembling, which averages overlapping chunks; the paper reports RTC as smoother than all compared methods, including temporal ensembling.\n",
      "related": [
        "action-chunking",
        "async-inference",
        "inference-latency",
        "flow-matching",
        "control-rate"
      ],
      "policies": [
        "pi0",
        "pi05",
        "smolvla"
      ],
      "sources": [
        {
          "id": "ev-rtc-paper",
          "tier": "upstream-documented",
          "type": "paper",
          "url": "https://arxiv.org/abs/2506.07339",
          "date": "2025-06-09",
          "verified_on": "2026-10-24",
          "summary": "Black et al. introduce real-time chunking (RTC).",
          "quote": "Our method, real-time chunking (RTC), is applicable to any diffusion- or flow-based VLA out of the box with no re-training."
        },
        {
          "id": "ev-rtc-paper-full",
          "tier": "upstream-documented",
          "type": "paper",
          "url": "https://arxiv.org/html/2506.07339",
          "date": "2025-06-09",
          "verified_on": "2026-10-24",
          "summary": "RTC paper: robust to inference delays over 300 ms.",
          "quote": "even in the presence of inference delays in excess of 300 milliseconds, corresponding to more than 30% of the model’s prediction horizon"
        },
        {
          "id": "ev-lerobot-rtc-doc",
          "tier": "upstream-documented",
          "type": "official-docs",
          "url": "https://github.com/huggingface/lerobot/blob/v0.6.1/docs/source/rtc.mdx",
          "date": "2026-08-03",
          "verified_on": "2026-10-24",
          "versions": {
            "lerobot": "0.6.1"
          },
          "summary": "LeRobot v0.6.1 documents RTC for Pi0, Pi0.5 and SmolVLA.",
          "quote": "Real-Time Chunking (RTC) is an inference-time method that allows large, flow-matching based robotic policies, such as [Pi0](./pi0), [Pi0.5](./pi05), and [SmolVLA](./smolvla), to produce smooth, continuous, and reactive motion despite having high inference latency."
        }
      ]
    },
    {
      "id": "reward-model",
      "term": "Reward model",
      "aka": [
        "learned reward",
        "reward classifier",
        "success detector"
      ],
      "short": "A learned function that scores how well a trajectory or state accomplishes a task, used in place of a hand-written reward for RL, data weighting or evaluation.",
      "body": "A reward model is a learned function that scores how well an agent's behavior accomplishes a task, either as a single number per episode or as a per-frame signal. It replaces a hand-engineered reward where one is impractical to write: \"is the towel folded?\" is easy to judge and hard to code. The modern template comes from Christiano et al., \"Deep Reinforcement Learning from Human Preferences\" (2017), which learned a reward model from human comparisons of trajectory segments and optimized a policy against it, solving tasks \"without access to the reward function\".\n\nIn robot learning, reward models serve three roles: the reward signal for real-world reinforcement learning, a success detector that ends episodes, and a score for weighting or filtering demonstration data. LeRobot v0.6.1 includes several. [HIL-SERL](/policy/hil-serl) trains a vision-based reward classifier from a handful of demonstrations. [SARM](/policy/sarm) is a video-based, stage-aware reward model for long-horizon manipulation, usable with Reward-Aligned Behavior Cloning. [Robometer](/policy/robometer) predicts \"dense, frame-level task progress and frame-level success from a trajectory video and a task description\". [TOPReward](/policy/topreward) is a zero-shot reward model that reads token log-probabilities from an off-the-shelf VLM.\n\nThese models appear in this matrix but are not robot policies: they never command actuators, so they are listed outside the compatibility grid and have no robot cells.\n\nA common misconception is that a reward model is ground truth. A policy optimized against a learned reward can exploit its errors, so success rates scored by the same model used for training overstate real performance. Independent evaluation, by people or by scripted checks, is still required.\n",
      "related": [
        "human-in-the-loop-rl",
        "imitation-learning",
        "behavioral-cloning",
        "world-model"
      ],
      "policies": [
        "hil-serl",
        "sarm",
        "robometer",
        "topreward"
      ],
      "sources": [
        {
          "id": "ev-christiano-2017",
          "tier": "upstream-documented",
          "type": "paper",
          "url": "https://arxiv.org/abs/1706.03741",
          "date": "2017-06-12",
          "verified_on": "2026-10-24",
          "summary": "Christiano et al. learn a reward model from human preferences.",
          "quote": "We show that this approach can effectively solve complex RL tasks without access to the reward function"
        },
        {
          "id": "ev-lerobot-sarm-doc",
          "tier": "upstream-documented",
          "type": "official-docs",
          "url": "https://github.com/huggingface/lerobot/blob/v0.6.1/docs/source/sarm.mdx",
          "date": "2026-08-03",
          "verified_on": "2026-10-24",
          "versions": {
            "lerobot": "0.6.1"
          },
          "summary": "LeRobot v0.6.1 SARM reward model docs.",
          "quote": "SARM (Stage-Aware Reward Modeling) is a video-based reward modeling framework for long-horizon robot manipulation tasks."
        },
        {
          "id": "ev-lerobot-robometer-doc",
          "tier": "upstream-documented",
          "type": "official-docs",
          "url": "https://github.com/huggingface/lerobot/blob/v0.6.1/docs/source/robometer.mdx",
          "date": "2026-08-03",
          "verified_on": "2026-10-24",
          "versions": {
            "lerobot": "0.6.1"
          },
          "summary": "LeRobot v0.6.1 Robometer docs.",
          "quote": "It predicts dense, frame-level task progress and frame-level success from a trajectory video and a task description."
        },
        {
          "id": "ev-lerobot-topreward-doc",
          "tier": "upstream-documented",
          "type": "official-docs",
          "url": "https://github.com/huggingface/lerobot/blob/v0.6.1/docs/source/topreward.mdx",
          "date": "2026-08-03",
          "verified_on": "2026-10-24",
          "versions": {
            "lerobot": "0.6.1"
          },
          "summary": "LeRobot v0.6.1 TOPReward docs.",
          "quote": "that extracts token log-probabilities from an off-the-shelf vision-language model (VLM) as a robotic reward signal."
        },
        {
          "id": "ev-lerobot-hilserl-doc",
          "tier": "upstream-documented",
          "type": "official-docs",
          "url": "https://github.com/huggingface/lerobot/blob/v0.6.1/docs/source/hilserl.mdx",
          "date": "2026-08-03",
          "verified_on": "2026-10-24",
          "versions": {
            "lerobot": "0.6.1"
          },
          "summary": "LeRobot HIL-SERL trains a reward classifier from demonstrations.",
          "quote": "The approach starts from a small set of human demonstrations, uses them to train a reward classifier"
        }
      ]
    },
    {
      "id": "sim-to-real",
      "term": "Sim-to-real transfer",
      "aka": [
        "sim2real",
        "reality gap"
      ],
      "short": "Training or validating a policy in simulation and deploying it on physical hardware; the gap between simulated and real physics and sensing is the central obstacle.",
      "body": "Sim-to-real transfer means training or pretraining a policy in a physics simulator and then running it on a real robot. Simulation offers abundant, safe, automatically labeled data. The cost is the *reality gap*: differences in rendering, contact dynamics, actuator response, timing and sensor noise between simulator and hardware.\n\nTwo 2017 papers define the standard toolkit. Tobin et al., \"Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World\", randomized rendering so that \"the real world may appear to the model as just another variation\". Peng et al., \"Sim-to-Real Transfer of Robotic Control with Dynamics Randomization\", randomized the simulator's dynamics during training so that policies adapt to dynamics they were not trained on.\n\nFor the imitation-learning policies in this matrix the pattern is usually different. Models are trained on real teleoperated data and *evaluated* in simulated benchmarks such as LIBERO and Meta-World, which LeRobot provides as environments. A high LIBERO success rate for [OpenVLA-OFT](/policy/openvla-oft) or [SmolVLA](/policy/smolvla) is a simulation result; it says nothing directly about an [SO-101](/robot/so101) on a desk. Compatibility cells therefore keep `inference_sim` and `inference_real` as separate capabilities.\n\nA common misconception is that a policy which works in simulation will transfer once the images look realistic. Appearance is only one part of the gap. Unmodeled dynamics, actuator delay and a mismatched [control rate](/glossary/control-rate) break transfer just as reliably, which is why Peng et al. randomized dynamics rather than visuals. The [embodiment gap](/glossary/embodiment-gap) is a separate problem: it concerns differences between robots, not between a robot and its simulation.\n",
      "related": [
        "embodiment-gap",
        "control-rate",
        "world-model",
        "teleoperation"
      ],
      "policies": [
        "openvla-oft",
        "smolvla"
      ],
      "sources": [
        {
          "id": "ev-tobin-2017",
          "tier": "upstream-documented",
          "type": "paper",
          "url": "https://arxiv.org/abs/1703.06907",
          "date": "2017-03-20",
          "verified_on": "2026-10-24",
          "summary": "Tobin et al. introduce domain randomization for sim-to-real transfer.",
          "quote": "With enough variability in the simulator, the real world may appear to the model as just another variation."
        },
        {
          "id": "ev-peng-2017",
          "tier": "upstream-documented",
          "type": "paper",
          "url": "https://arxiv.org/html/1710.06537",
          "date": "2017-10-18",
          "verified_on": "2026-10-24",
          "summary": "Peng et al. introduce dynamics randomization for sim-to-real transfer.",
          "quote": "By randomizing the dynamics of the simulator during training, we are able to develop policies that are capable of adapting to very different dynamics"
        },
        {
          "id": "ev-oft-paper",
          "tier": "upstream-documented",
          "type": "paper",
          "url": "https://arxiv.org/abs/2502.19645",
          "date": "2025-02-27",
          "verified_on": "2026-10-24",
          "summary": "OpenVLA-OFT reports results on the LIBERO simulation benchmark.",
          "quote": "We propose OpenVLA-OFT, an instantiation of this recipe, which sets a new state of the art on the LIBERO simulation benchmark"
        }
      ]
    },
    {
      "id": "teleoperation",
      "term": "Teleoperation",
      "aka": [
        "teleop",
        "remote operation"
      ],
      "short": "Real-time human control of a robot through an input device; in robot learning, the standard way to collect demonstrations for imitation learning.",
      "body": "Teleoperation is real-time control of a robot by a human through an input device. In robot learning it is primarily a data-collection method: the operator performs the task through the robot, and the robot's observations and commanded actions are recorded as demonstrations for [imitation learning](/glossary/imitation-learning). Demonstration quality bounds what a policy can learn. The GELLO paper (Wu et al., 2023) states it directly: \"the performance of the learned policies is bottlenecked by the quality, scale, and variety of the demonstration data\".\n\nInterfaces differ in how the operator's input maps to the robot. Joint-space devices, chiefly a [leader arm](/glossary/leader-follower-arm), command each joint directly and need no inverse kinematics. Task-space devices such as VR controllers, 3D mice and phones command an end-effector pose that must be converted to joint targets. The ALOHA system in Zhao et al. (2023) used joint-space mapping and recorded at 50 Hz.\n\nIn LeRobot v0.6.1, `lerobot-teleoperate` drives a robot from a teleoperator such as `so101_leader`, and `lerobot-record` writes episodes to a [LeRobotDataset](/glossary/lerobot-dataset). The resulting dataset is the training set for policies such as [ACT](/policy/act) and [SmolVLA](/policy/smolvla).\n\nThe teleoperation interface determines the action space in the dataset. A leader-arm recording on an [SO-101](/robot/so101) stores joint-position targets; an end-effector recording does not, and a policy trained on one cannot be run on the other without conversion. Teleoperation rate matters too: in the ACT paper's user study, lowering the teleoperation frequency from 50 Hz to 5 Hz made human operators 62% slower.\n",
      "related": [
        "leader-follower-arm",
        "imitation-learning",
        "behavioral-cloning",
        "lerobot-dataset",
        "control-rate"
      ],
      "policies": [
        "act",
        "smolvla",
        "hil-serl"
      ],
      "sources": [
        {
          "id": "ev-gello-paper",
          "tier": "upstream-documented",
          "type": "paper",
          "url": "https://arxiv.org/abs/2309.13037",
          "date": "2023-09-22",
          "verified_on": "2026-10-24",
          "summary": "GELLO: teleoperated demonstrations bound learned-policy performance.",
          "quote": "However, the performance of the learned policies is bottlenecked by the quality, scale, and variety of the demonstration data."
        },
        {
          "id": "ev-act-paper-full",
          "tier": "upstream-documented",
          "type": "paper",
          "url": "https://arxiv.org/html/2304.13705",
          "date": "2023-04-23",
          "verified_on": "2026-10-24",
          "summary": "ACT paper: ALOHA teleoperates and records at 50 Hz; 5 Hz teleoperation is 62% slower.",
          "quote": "Lowering the frequency results in a 62% slowdown in completion time."
        },
        {
          "id": "ev-lerobot-il-robots",
          "tier": "upstream-documented",
          "type": "official-docs",
          "url": "https://github.com/huggingface/lerobot/blob/v0.6.1/docs/source/il_robots.mdx",
          "date": "2026-08-03",
          "verified_on": "2026-10-24",
          "versions": {
            "lerobot": "0.6.1"
          },
          "summary": "LeRobot uses a teleoperation device such as a leader arm to record trajectories.",
          "quote": "such as a leader arm or keyboard to teleoperate the robot and record its motion trajectories."
        }
      ]
    },
    {
      "id": "vla",
      "term": "Vision-language-action model",
      "aka": [
        "VLA",
        "VLA model"
      ],
      "short": "A robot policy built on a pretrained vision-language model that maps camera images, a language instruction and robot state to actions.",
      "body": "A vision-language-action model (VLA) is a robot policy that takes camera images and a natural-language instruction, usually with [proprioceptive](/glossary/proprioception) state, and outputs robot actions. It is built by adapting a vision-language model (VLM) pretrained on Internet-scale data. The term comes from the RT-2 paper (Brohan et al., 2023): \"We refer to such category of models as vision-language-action models (VLA)\". RT-2 co-fine-tuned a VLM on robot trajectories and web data and emitted actions as text tokens.\n\nVLAs differ mainly in how they produce actions. Autoregressive VLAs such as [OpenVLA](/policy/openvla) and [π0-FAST](/policy/pi0-fast) predict discrete [action tokens](/glossary/action-tokenization). Others attach a continuous action head to the VLM: [π0](/policy/pi0), [π0.5](/policy/pi05), [SmolVLA](/policy/smolvla), [GR00T N1.7](/policy/groot-n1-7), [X-VLA](/policy/xvla), [EO-1](/policy/eo1) and [EVO-1](/policy/evo1) use [flow matching](/glossary/flow-matching), while [OpenVLA-OFT](/policy/openvla-oft) uses parallel decoding with L1 regression. Size varies widely, from OpenVLA's 7B parameters to SmolVLA, which is \"designed to be trained on a single GPU and deployed on consumer-grade GPUs or even CPUs\".\n\nTwo misconceptions recur. \"VLA\" names an architecture family, not a capability level. Language conditioning does not mean the model follows arbitrary instructions on a new robot, and deployment on a robot absent from pretraining generally requires fine-tuning (see [embodiment gap](/glossary/embodiment-gap)). Nor does the label imply a size: parameter count, action head and [precision](/glossary/quantization), not the acronym, decide whether a VLA fits on a [Jetson Orin Nano](/compute/jetson-orin-nano-8gb) or needs an [H100](/compute/h100-80gb).\n",
      "related": [
        "action-tokenization",
        "flow-matching",
        "embodiment-gap",
        "proprioception",
        "peft-lora"
      ],
      "policies": [
        "openvla",
        "openvla-oft",
        "pi0",
        "pi0-fast",
        "pi05",
        "smolvla",
        "groot-n1-7",
        "xvla",
        "eo1",
        "evo1"
      ],
      "sources": [
        {
          "id": "ev-rt2-paper",
          "tier": "upstream-documented",
          "type": "paper",
          "url": "https://arxiv.org/abs/2307.15818",
          "date": "2023-07-28",
          "verified_on": "2026-10-24",
          "summary": "RT-2 introduces the term vision-language-action model (VLA).",
          "quote": "We refer to such category of models as vision-language-action models (VLA) and instantiate an example of such a model, which we call RT-2."
        },
        {
          "id": "ev-openvla-paper",
          "tier": "upstream-documented",
          "type": "paper",
          "url": "https://arxiv.org/abs/2406.09246",
          "date": "2024-06-13",
          "verified_on": "2026-10-24",
          "summary": "OpenVLA is a 7B-parameter open-source VLA.",
          "quote": "we introduce OpenVLA, a 7B-parameter open-source VLA trained on a diverse collection of 970k real-world robot demonstrations."
        },
        {
          "id": "ev-smolvla-paper",
          "tier": "upstream-documented",
          "type": "paper",
          "url": "https://arxiv.org/abs/2506.01844",
          "date": "2025-06-02",
          "verified_on": "2026-10-24",
          "summary": "SmolVLA targets single-GPU training and consumer-grade deployment.",
          "quote": "SmolVLA is designed to be trained on a single GPU and deployed on consumer-grade GPUs or even CPUs."
        }
      ]
    },
    {
      "id": "world-model",
      "term": "World model",
      "aka": [
        "dynamics model",
        "learned simulator"
      ],
      "short": "A learned model that predicts how an environment's state or observations evolve in response to actions, used for planning, imagined rollouts or auxiliary training.",
      "body": "A world model is a learned predictive model of an environment. Given the current state or observation history and an action, it predicts the next state, observation or latent representation, and often a reward. An agent can then plan, imagine rollouts, or learn a policy inside the model instead of only in the real environment. Ha and Schmidhuber's \"World Models\" (2018) is the reference point for the modern term: they trained an agent \"entirely inside of its own hallucinated dream generated by its world model\" and transferred the policy back to the actual environment. Learning dynamics models for control is older, under the name model-based reinforcement learning.\n\nWorld models differ in what they predict. TD-MPC, available in LeRobot as [TD-MPC](/policy/tdmpc), plans \"in the latent space of a learned implicit (decoder-free) world model\" (Hansen et al.) and never generates images. Video world models predict future frames. Several policies in this matrix use them. LeRobot describes [LingBot-VA](/policy/lingbot-va) as an autoregressive video-action world-model policy that interleaves predictions of future video latents and robot actions. [FastWAM](/policy/fastwam) is a \"World Action Model\" that keeps video modeling during training but predicts actions directly at inference. [VLA-JEPA](/policy/vla-jepa) adds a V-JEPA2 video world model as a training-only loss.\n\nA common misconception is that \"world model\" implies photorealistic video generation, or that a policy trained with a world model runs one on the robot. Neither holds. FastWAM and VLA-JEPA drop the video-prediction path at inference, while LingBot-VA keeps it, and that difference matters for [inference latency](/glossary/inference-latency) and memory on a given [compute](/compute/rtx-4090) target. A world model also does not remove the [sim-to-real](/glossary/sim-to-real) problem; it is itself a learned simulator with its own errors.\n",
      "related": [
        "sim-to-real",
        "vla",
        "inference-latency",
        "reward-model"
      ],
      "policies": [
        "tdmpc",
        "lingbot-va",
        "fastwam",
        "vla-jepa"
      ],
      "sources": [
        {
          "id": "ev-ha-2018",
          "tier": "upstream-documented",
          "type": "paper",
          "url": "https://arxiv.org/abs/1803.10122",
          "date": "2018-03-27",
          "verified_on": "2026-10-24",
          "summary": "Ha and Schmidhuber, World Models.",
          "quote": "We can even train our agent entirely inside of its own hallucinated dream generated by its world model, and transfer this policy back into the actual environment."
        },
        {
          "id": "ev-tdmpc2-paper",
          "tier": "upstream-documented",
          "type": "paper",
          "url": "https://arxiv.org/abs/2310.16828",
          "date": "2023-10-25",
          "verified_on": "2026-10-24",
          "summary": "TD-MPC plans in the latent space of a decoder-free world model.",
          "quote": "TD-MPC is a model-based reinforcement learning (RL) algorithm that performs local trajectory optimization in the latent space of a learned implicit (decoder-free) world model."
        },
        {
          "id": "ev-lerobot-lingbot-va-doc",
          "tier": "upstream-documented",
          "type": "official-docs",
          "url": "https://github.com/huggingface/lerobot/blob/v0.6.1/docs/source/lingbot_va.mdx",
          "date": "2026-08-03",
          "verified_on": "2026-10-24",
          "versions": {
            "lerobot": "0.6.1"
          },
          "summary": "LeRobot describes LingBot-VA as a video-action world-model policy.",
          "quote": "It interleaves, in one autoregressive sequence, the prediction of\nfuture **video latents** and **robot actions**"
        },
        {
          "id": "ev-lerobot-fastwam-doc",
          "tier": "upstream-documented",
          "type": "official-docs",
          "url": "https://github.com/huggingface/lerobot/blob/v0.6.1/docs/source/fastwam.mdx",
          "date": "2026-08-03",
          "verified_on": "2026-10-24",
          "versions": {
            "lerobot": "0.6.1"
          },
          "summary": "LeRobot: FastWAM keeps video modeling only during training.",
          "quote": "FastWAM keeps video modeling during training, but uses direct action prediction at inference time instead of iteratively generating future observations."
        },
        {
          "id": "ev-lerobot-vla-jepa-doc",
          "tier": "upstream-documented",
          "type": "official-docs",
          "url": "https://github.com/huggingface/lerobot/blob/v0.6.1/docs/source/vla_jepa.mdx",
          "date": "2026-08-03",
          "verified_on": "2026-10-24",
          "versions": {
            "lerobot": "0.6.1"
          },
          "summary": "LeRobot: VLA-JEPA uses a V-JEPA2 world model for a training-only loss.",
          "quote": "Self-supervised video prediction loss (training only)"
        }
      ]
    }
  ]
}