NVIDIA's real-time motion generation model: given a character's recent pose and a target pose to reach, it generates coherent motion between the two, used for human skeletal animation and reference motions for the Unitree G1 humanoid robot; the accompanying Smart Primitives turn target poses into reusable "blocks," allowing the assembly of walking and object interactions without retraining.
Upvotes 73 · Likes 74 · Downloads 0
Reasons
- Two sets of weights: SOMA (30-joint human skeleton, approx. 200M parameters, trained on 313,526 segments from the Bones RigPlay 1 motion capture data) and Unitree G1 (34 joints, approx. 195M parameters); the files for the two sets are not interchangeable.
- Each set consists of three networks: a pose VQ-VAE, a pose predictor, and a root motion predictor; it outputs motion at 30 Hz, generating 6–16 tokens per inference (24–64 frames); longer motions are generated by feeding the output back as context for repeated generation; it does not accept text or image inputs.
- The paper was published in ACM Transactions on Graphics; the weights are licensed under the NVIDIA Open Model Agreement, allowing commercial use and redistribution of modified versions, with accompanying code under Apache 2.0. Released on 23 10, it was not included in previous issues.
Usage
- To just see the results: directly use the NVlabs/motionbricks software on GitHub, which includes ONNX models and demos for SOMA and G1 (based on Newton and Warp); downloading the weights here is not required; the TensorRT demo requires an NVIDIA GPU at runtime.
- To export ONNX yourself or use other runtimes: run
hf download nvidia/MotionBricks --include "soma/\*"(or "g1/*"), maintain the directory structure, and export according to the guidelines in the software repository'stools/onnx; the exporter runs on CPU and requires Python 3.12 and PyTorch 2.14.0+cpu.
Risks
- It generates motion, not a robot controller: deploying on real hardware still requires a separate tracking controller, joint limit checks, and safety verification.
- The model card does not provide end-to-end evaluation scores, nor does it specify inference latency and throughput; the weights are not Transformers
from_pretrainedmodels and do not include the training pipeline or training data. - The included BONES motion samples are permitted for internal evaluation testing only (NVIDIA Sample Data Evaluation License); the G1 package does not include the motion library used in the original demo.
Editor's note: data.json classified this as a language model, but it actually performs motion generation, so it has been reclassified here as a generative model.