Asynchronous Robot Inference: Decoupling Action Prediction and Execution

AI Impact Summary

Asynchronous inference decouples action prediction from execution, addressing the issue of robotic policies being overly bulky and leading to significant idle periods during action waiting. This approach, exemplified by the SmolVLA system with a PolicyServer and RobotClient communicating via gRPC, achieves a ~2x speedup in task completion by overlapping computation and execution, eliminating latency and improving responsiveness. This architecture is particularly relevant when using computationally intensive models like ACT, OpenVLA, PI0, and SmolVLA.