Abstract

Mobile manipulator robots operating in complex domestic and industrial environments must effectively coordinate their base and arm motions while avoiding obstacles. While current reactive control methods gracefully achieve this coordination, they rely on simplified and idealised geometric representations of the environment to avoid collisions. This limits their performance in cluttered environments. To address this problem, we introduce RMMI, a reactive control framework that leverages the ability of neural signed distance fields (SDFs) to provide a continuous and differentiable representation of the environment's geometry. RMMI formulates a quadratic program that optimises jointly for robot base and arm motion, maximises manipulability, and avoids collisions through a set of inequality constraints. These constraints are constructed by querying the SDF for the distance and direction to the closest obstacle for a large number of sampling points on the robot. We evaluate RMMI both in simulation and a set of real-world experiments. For reaching in cluttered environments, we observe a 25\% increase in success rate.




Video

Robot representation
Primitives 9476 Points 2358 Points Spheres
Robot Mesh

Collision representation
Interactive
Real-World Experiments
Scenes
Table Bookshelf Cabinet
Approach
No Collision Avoidance Only Inequality Ours