RPU — Robot Processing Unit
A Specialized Chip
for Humanoid Robotics.
The ChatGPT moment of robotics requires true, embodied intelligence that runs onboard with strict, deterministic latency. We are building the RPU (robot processing unit), a processor and compiler optimized for diffusion transformer inference.
Detail
Robot intelligence is growing faster than the compute robots can carry. Larger models offer more capable behavior, but many require datacenter GPUs.
We are building the RPU to bring more of that intelligence onboard. We design around the computation shared by diffusion transformers, with loadable models and schedules that can change as robotics evolves.
01 — The requirement
Embodied Intelligence
Realtime intelligence must run onboard because real world deployment is safety critical. Delayed reactions are dangerous. Remote inference cannot safely and reliably scale beyond the demo.
Detail
We must not have datacenter latency and network jitter in the real-time control loop. Robots must operate in safety critical environments like households, workplaces, and public spaces. Safety requires low latency, deterministic control with ultra high reliability.
Some amount of remote inference may be unavoidable, but we believe it should be relegated to System 2 (non-real-time planning). Remote inference for System 1 (real-time control) introduces additional engineering requirements, including low-latency observation and action streaming, coordination with local execution, and handling network delays and interruptions.
For System 1, the critical metric is reaction latency: the elapsed time between capturing an observation and executing the corresponding action. Human visuomotor reaction time is roughly 200 milliseconds, and delays substantially beyond this range are readily perceived as sluggishness or hesitation.
Remote inference introduces more than a fixed delay. By the time an action chunk reaches the robot, it was generated from observations that are already stale, and the amount of staleness changes from one cycle to the next.
Design goals
- 100 Wonboard power budget
- 200 msreaction latency
- 5×performance per watt over Thor, mature compiler
- MLIRcompile from PyTorch
Performance figures are targets to validate through simulation and prototyping.
02 — The RPU
Models Have Converged Enough
For Specialization
Robot policies have converged on diffusion transformers. RPU specializes in the repeated matrix and attention operations at their core. A robot has limited power and cooling, so running a more capable model means spending less energy on each inference.
Detail
Robots need low-latency, batch-1 inference, acting on their latest observations without waiting to assemble a batch. But batch 1 does not imply low compute intensity. A diffusion transformer processes thousands of latent tokens together, reusing its weights across those tokens. That demands far more arithmetic per byte fetched than generating one token at a time.
The compiler plans computation and data movement in advance, and the hardware follows that schedule. A systolic array, low-precision arithmetic, compiler-managed memory, and fused attention together reduce the energy spent moving data and coordinating execution. We target roughly 3× Thor’s performance per watt with first silicon and 5× as the compiler improves.
RPU is a companion accelerator. The robot’s existing computer continues to handle sensors, networking, independent safety functions, and conventional control.
03 — The horizon
Better models and better
hardware, designed together.
Our first chip preserves the flexibility to follow changing models. As architectures stabilize, we can specialize further.
Detail
The emergence of scaling laws suggests the generalist robotics policies will continue to grow in size, and robots that can fit more of that intelligence onboard will always have a competitive advantage.
The area advantage remains a hypothesis. Supporting fewer numerical formats and a narrower workload family should let us simplify precision handling and general-purpose control, leaving more die area for arithmetic and memory. Physical implementation will determine how much area this saves.
Looking toward the horizon, we will need to deploy embodied intelligence to achieve humanity’s most ambitious projects. Robots will build and operate the moon base. Remote inference is intractable at those distances, unless we put datacenters on the moon, but then it becomes a chicken-and-egg problem. How do we set up all that infrastructure without robots?