Skip to main content

Learning Structured Representations for Rigid and Deformable Object Manipulation

Time: Tue 2021-11-09 15.00

Location: Ångdomen, Osquars backe 31, Stockholm (English)

Doctoral student: Michael C. Welle , Robotik, perception och lärande, RPL, EECS/RPL

Opponent: Associate Professor Dimitry Berenson,

Supervisor: Danica Kragic, Robotik, perception och lärande, RPL


The performance of learning based algorithms largely depends on the given representation of data. Therefore the questions arise, i) how to obtain useful representations, ii) how to evaluate representations, and iii) how to leverage these representations in a real-world robotic setting. In this thesis, we aim to answer all three of this questions in order to learn structured representations for rigid and deformable object manipulation. We firstly take a look into how to learn structured representation and show that imposing structure, informed from task priors, into the representation space is beneficial for certain robotic tasks. Furthermore we discuss and present suitable evaluation practices for structured representations as well as a benchmark for bimanual cloth manipulation. Finally, we introduce the Latent SpaceRoadmap (LSR) framework for visual action planning, where raw observations are mapped into a lower-dimensional latent space. Those are connected via the LSR, and visual action plans are generated that are able to perform a wide range of tasks. The framework is validated on a simulated rigid box stacking task, a simulated hybrid rope-box manipulation task, and a T-shirt folding task performed on a real robotic system.