Authors: Guanxing Lu, Tengbo Yu, Haoyuan Deng, Season Si Chen, Yansong Tang, Ziwei Wang
Abstract: Performing general language-conditioned bimanual manipulation tasks is of
great importance for many applications ranging from household service to
industrial assembly. However, collecting bimanual manipulation data is
expensive due to the high-dimensional action space, which poses challenges for
conventional methods to handle general bimanual manipulation tasks. In
contrast, unimanual policy has recently demonstrated impressive
generalizability across a wide range of tasks because of scaled model
parameters and training data, which can provide sharable manipulation knowledge
for bimanual systems. To this end, we propose a plug-and-play method named
AnyBimanual, which transfers pre-trained unimanual policy to general bimanual
manipulation policy with few bimanual demonstrations. Specifically, we first
introduce a skill manager to dynamically schedule the skill representations
discovered from pre-trained unimanual policy for bimanual manipulation tasks,
which linearly combines skill primitives with task-oriented compensation to
represent the bimanual manipulation instruction. To mitigate the observation
discrepancy between unimanual and bimanual systems, we present a visual aligner
to generate soft masks for visual embedding of the workspace, which aims to
align visual input of unimanual policy model for each arm with those during
pretraining stage. AnyBimanual shows superiority on 12 simulated tasks from
RLBench2 with a sizable 12.67% improvement in success rate over previous
methods. Experiments on 9 real-world tasks further verify its practicality with
an average success rate of 84.62%.
Source: http://arxiv.org/abs/2412.06779v1
