Make robot agents work like coding agents: the model writes code to operate a robotic arm by calling 22 low-level capabilities (perception, geometric measurement, inverse kinematics-level motion) within a sandboxed Python session, reads back the textual receipt of each call, and decides on its own when to call finish() to end the episode. Training follows the recipe used for coding/web agents: first supervised fine-tuning on synthetic trajectories, followed by multiple rounds of reinforcement learning; to distribute the episode-end success/failure signal across dozens of requests, the authors added a "physics critic" that can read the simulator's true state (which the policy cannot see), scoring each request based on how much it changes the success probability.
Likes 18 · Code Stars 50
Reasons
- The authors report that Qwen3.5-4B achieves 0% zero-shot, 55.4% on the LIBERO held-out set after SFT, and 73.8% after RL; 78.5% on LIBERO-PRO, surpassing several frontier model agents; trained only on LIBERO, it achieves 83.3% zero-shot on SimplerEnv WidowX and runs successfully on real hardware.
- At test time, it does not use frontier large models, VLA, or privileged state access; it is just a 4B model.
- Fully open: CaPtain-4B / CaPtain-4B-SFT weights, SFT data, the evaluation harness CaPtain-Harness, SFT and RL training code, and streaming rollout infrastructure are all released, with code under MIT license.
Usage
- git clone --recursive github.com/waltstephen/CaPtain, enter harness/ and run scripts/setup_third_party.sh, then set up the LIBERO environment and the Python 3.11 sandbox environment for the agent according to the README (requires bubblewrap 0.5 or higher).
- Use harness/scripts/download_models.sh to download SAM 3, Molmo2-8B, and CaPtain-4B; use services/start_perception.sh to start the perception service, and services/serve_policy.sh YijiaFan/CaPtain-4B 8600 to serve the policy using vLLM.
- Run an episode: eval/run_episode.sh libero_spatial 0 11 runs/demo; the harness connects to any OpenAI-compatible interface, allowing you to test your own models or frontier models as well.
Risks
- Reproducing training is expensive: the README states that SFT uses 16 H100 GPUs for about 7.6 hours, and RL takes about 1,290 GPU hours; evaluation also requires one GPU for the policy and one for perception and rendering.
- SFT trajectories are synthesized by closed-source models; when transferring to RoboTwin, SimplerEnv, etc., more inputs than the official observations are used (depth, camera calibration, wrist force sensing, etc.), which are listed item by item in the README.
- SAM 3 weights are subject to the SAM License, and model weights follow Qwen3.5-4B (Apache-2.0); reported results are from the authors.
Open · Code · Paper Page