TR2023-048

Learning Generalizable Pivoting Skills with Object Feature Based State/Action Projections

    •  Zhang, X., Jain, S., Huang, B., Tomizuka, M., Romeres, D., "Learning Generalizable Pivoting Skills with Object Feature Based State/Action Projections", ICRA 2023 Workshop on Effective Representations, Abstractions, and Priors for Robot Learning (RAP4Robots), May 2023.
      BibTeX TR2023-048 PDF
      • @inproceedings{Zhang2023may5,
      • author = {Zhang, Xiang and Jain, Siddarth and Huang, Baichuan and Tomizuka, Masayoshi and Romeres, Diego},
      • title = {Learning Generalizable Pivoting Skills with Object Feature Based State/Action Projections},
      • booktitle = {ICRA 2023 Workshop on Effective Representations, Abstractions, and Priors for Robot Learning (RAP4Robots)},
      • year = 2023,
      • month = may,
      • url = {https://www.merl.com/publications/TR2023-048}
      • }
  • MERL Contacts:
  • Research Area:

    Robotics

Abstract:

The task of pivoting an object with a robotic manipulator is challenging due to the precise application of force required to maintain contact with the object. However, even if the robot is capable of pivoting a particular object, generalizing these skills across different objects presents a more complex challenge. In this paper, we propose a method for generalizing a single-object pivoting skill to other objects by utilizing object visual features. Specifically, we train an encoder to extract the kinematic properties of arbitrary objects from their depth images. Then, we learn projections based on these properties to adjust the state and action space to adapt the single-object pivoting skill to the new pivoting task. The proposed approach is entirely trained in simulation. It requires only one depth image of the object and can zero-shot transfer to real-world objects. We demonstrate robustness to sim-to-real transfer and generalization to multiple objects.