Asynchronous inference decouples computing action chunks from executing them. Under synchronous inference the robot executes a chunk, stops, sends an observation, waits for the model and resumes, idling for the full inference latency at every chunk boundary. Under asynchronous inference the robot keeps executing from an action queue and sends a new observation before the queue runs out, so the next chunk is ready when needed.
LeRobot's implementation was introduced with SmolVLA (Shukor et al., 2025) as "an asynchronous inference stack decoupling perception and action prediction from action execution, allowing higher control rates with chunked action generation". It is split into a PolicyServer, which can run on a remote GPU such as an RTX 4090, and a RobotClient on the computer attached to the robot, for example an SO-101. Two parameters govern it: actions_per_chunk, and chunk_size_threshold (g in the paper), the queue fraction at which the client sends a fresh observation. The paper shows that the queue does not run dry when g ≥ (E[ℓ_S]/Δt)/n, where ℓ_S is server latency, Δt the control period and n the chunk length. The LeRobot docs report values around 0.5–0.6 working well; values near 0 collapse to synchronous behavior.
Two misconceptions recur. Async inference does not reduce latency; it hides it. Each chunk is still computed from an observation that is at least ℓ_S old by the time its actions execute. And async inference alone does not make chunk transitions smooth. LeRobot aggregates overlapping actions, whereas real-time chunking conditions the new chunk on actions already committed, and the two can be combined.