HIL-SERL
Human-in-the-loop, sample-efficient real-robot reinforcement learning: a few teleoperated demonstrations and an optional vision-based reward classifier bootstrap a distributed actor/learner SAC loop, with a human able to intervene during on-robot exploration. In LeRobot v0.6.x the trained policy is a `gaussian_actor` driven by the `sac` algorithm.
verified 2026-10-24paper →
Architecture
Compute requirements
| Memory | Precision | Conditions | |
|---|---|---|---|
| train | —src | — | The LeRobot compute hardware guide gives no VRAM figure for RL: its RL row (still labelled `sac`, the pre-v0.6.0 type name) reads "config-dep." and points to the HIL-SERL guide. The HIL-SERL guide lists "A Nvidia GPU" as a requirement and suggests 64×64 instead of 128×128 image inputs to save GPU memory. |
Training requirements: see also the LeRobot compute hardware guide, the authoritative source for LeRobot training envelopes.
Measured inference 0 runs · inference cost only, not task success
No measured runs yet. Submit one with its runtime, precision and camera count.
Robots running HIL-SERL 3 of 24 with evidence
| Robot | State | Tier | Verified | Detail |
|---|---|---|---|---|
| Franka Research 3 | works with caveats | T1 · upstream documented | 2026-10-25 today | Upstream names an FR3 only as one arm of its bimanual Panda + FR3 setup. It does not say whether any single-arm task ran on the FR3. |
| Franka Emika Panda | works with caveats | T1 · upstream documented | 2026-10-24 1d ago | Upstream names a Panda only as one arm of its bimanual Panda + FR3 setup. It does not say which single-arm tasks ran on which Franka model. |
| SO-100 | works with caveats | T1 · upstream documented | 2026-10-24 1d ago | The guide's documented intervention path is a gamepad or keyboard in end-effector space (needs a URDF); using an SO leader arm as the intervention device is reported broken in open issue #2952, not yet confirmed or fixed by maintainers. |
No verified evidence, but both are available in LeRobot (ecosystem context): AgileX PiPER, ARX5, Bimanual SO-100/SO-101, EarthRover Mini Plus, HopeJR, I2RT YAM, Koch v1.1, LeKiwi, ROBOTIS OMX, OpenArm, Reachy 2, reBot Arm B601-DM, SO-101, Trossen WidowX AI, Unitree G1, Universal Robots UR5e, UFACTORY xArm, XLeRobot.
Framework support
| Framework | Support | Since | Type string | Docs |
|---|---|---|---|---|
| LeRobot | nativesrc | 0.3.2 | gaussian_actor | docs |
| LeRobot: v0.6.0 rebuilt the RL stack: the policy is now `--policy.type=gaussian_actor` (policy.type="gaussian_actor" in the JSON train config) with a top-level `algorithm` block of type "sac"; from v0.3.2 through v0.5.1 the policy type was `sac`. Training runs as two processes, `python -m lerobot.rl.learner` and `python -m lerobot.rl.actor`. An optional reward classifier (type "reward_classifier", lerobot/rewards/classifier) predicts success from camera images. v0.3.2 is the earliest tagged LeRobot release and already ships the `sac` policy and HIL-SERL guide; the only earlier PyPI release (0.1.0, 2024-03) predates the HIL-SERL paper. | ||||
Checkpoints
None recorded.
Related terms
Behavioral cloning · DAgger · Human-in-the-loop reinforcement learning · Imitation learning · Leader–follower arm · Reward model · Teleoperation
Sources
- T1 · upstream documentedofficial-docssource 2026-08-03verified 2026-10-24 (1d ago)lerobot 0.6.1
HIL-SERL guide: the policy is configured with type gaussian_actor and the algorithm with type sac.
“1. Configure the policy settings (`type="gaussian_actor"`, `device`, etc.)”
https://github.com/huggingface/lerobot/blob/v0.6.1/docs/source/hilserl.mdx - T1 · upstream documentedofficial-docssource 2026-08-03verified 2026-10-24 (1d ago)lerobot 0.6.1
HIL-SERL guide recommends square 128×128 or 64×64 image inputs.
“Most vision-based policies have been validated on square inputs of either **128×128** (default) or **64×64** pixels.”
https://github.com/huggingface/lerobot/blob/v0.6.1/docs/source/hilserl.mdx - T1 · upstream documentedsource-codesource 2026-08-03verified 2026-10-24 (1d ago)lerobot 0.6.1
`gaussian_actor` is registered as a policy type; its docstring places critics/temperature on the SAC algorithm side.
“This configures the policy-side (actor + observation encoder) of a Gaussian policy, as used by SAC and related maximum-entropy continuous-control algorithms.”
https://github.com/huggingface/lerobot/blob/v0.6.1/src/lerobot/policies/gaussian_actor/configuration_gaussian_actor.py - T1 · upstream documentedrelease-notessource 2026-07-06verified 2026-10-24 (1d ago)lerobot 0.6.0
v0.6.0 renamed the RL policy type from sac to gaussian_actor.
“The RL stack was rebuilt: the `sac` policy type is now `gaussian_actor` under the new modular RL API.”
https://github.com/huggingface/lerobot/releases/tag/v0.6.0 - T1 · upstream documentedsource-codesource 2025-08-01verified 2026-10-24 (1d ago)lerobot 0.3.2
The `sac` policy type (used by HIL-SERL) is registered at v0.3.2, the earliest tagged release.
“@PreTrainedConfig.register_subclass("sac")”
https://github.com/huggingface/lerobot/blob/v0.3.2/src/lerobot/policies/sac/configuration_sac.py - T1 · upstream documentedpapersource 2024-10-29verified 2026-10-24 (1d ago)
HIL-SERL paper builds on RLPD as its core RL algorithm.
“the core underlying RL algorithm that we build on is RLPD”
https://arxiv.org/abs/2410.21845 - T1 · upstream documentedofficial-docssource 2026-08-03verified 2026-10-24 (1d ago)lerobot 0.6.1
Compute hardware guide RL row: `sac`, VRAM config-dependent, see HIL-SERL guide.
“| RL | `sac` | config-dep. | See [HIL-SERL guide](./hilserl) |”
https://github.com/huggingface/lerobot/blob/v0.6.1/docs/source/hardware_guide.mdx