Approach-constrained Grasp Synthesis and Interactive Perception for Rigid and Deformable Objects

QC 20250514

Abstract

This thesis introduces methods for two robotic tasks: grasp synthesis and deformable object manipulation. These tasks are connected by interactive perception, where robots actively manipulate objects to improve sensory feed-back and task performance. Achieving a collision-free, successful grasp is essential for subsequent interaction, while effective manipulation of deformable objects broadens real-world applications. For robotic grasp synthesis, we address the challenge of approach-constrained grasping. We introduce two methods: GoNet and CAPGrasp. GoNet learns a grasp sampler that generates grasp poses with approach directions that lie in a selected discretized bin. In contrast, CAPGrasp enables sampling in a continuous space without requiring explicit approach direction annotations in the learning phase, improving the grasp success rate and providing more flexibility for imposing approach constraint. For robotic deformable object manipulation, we focus on manipulating deformable bags with handles—a common daily human activity. We first propose a method that captures scene dynamics and predicts future states in environments containing both rigid spheres and a deformable bag. Our approach employs an object-centric graph representation and an encoder-decoder framework to forecast future graph states. Additionally, we integrate an active camera into the system, explicitly considering the regularity and structure of motion to couple the camera with the manipulator for effective exploration.

To address the common data scarcity issue in both domains, we also develop simulation environments and propose annotated datasets for extensive benchmarking. Experimental results on both simulated and real-world platforms demonstrate the effectiveness of our methods compared to established baselines.

urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-363359