Scalable Assembly of General Objects

Abstract

In this thesis, I present a scalable system towards fully automated and flexible robotic assembly that generalizes over diverse geometries and complex structures. Most real-world objects are assemblies composed of multiple parts. Assembly presents significant challenges for robots to execute long-horizon, contact-rich manipulation with both reliability and generalization. However, most manufacturing facilities today still rely heavily on manually programmed assembly lines, which require significant labor, time, and setup costs yet offer no flexibility to object variations. My proposed system synergizes global multi-step planning with local reactive learning-based control to enable generalizable and precise assembly. Such an integrated paradigm effectively leverages the best of both worlds, accomplishing results that neither planning nor learning could achieve alone. For planning, I leverage guidance from physical simulation and learned feasibility networks to efficiently search for part sequences, precise motions, and stable grasps for dual-arm robots over long horizons. For learningbased control, I train robust policies via reinforcement learning for submillimeter-level insertion across different part geometries, assembly paths, and grasp poses. I introduce and open-source the largest-scale assembly dataset to date and demonstrate my system’s generalization on thousands of simulated assemblies as well as through end-to-end real robot experiments. By integrating planning and learning, I showcase the first system to achieve complete and generalizable real-world multi-part assembly without domain knowledge or human demonstrations. Although the system plans and learns purely in simulation, it transfers zero-shot to the real world and achieves 80% successful steps. Finally, I will share insights that further scale up robotic assembly and opportunities to extend to general manipulation, and discuss future directions to equip general-purpose robots with multi-step, precise manipulation capabilities.