Primitives generation policy learning without catastrophic forgetting for robotic manipulation

Catastrophic forgetting is a tough challenge when agent attempts to address different tasks sequentially without storing previous information, which gradually hinders the development of continual learning. Except for image classification tasks in continual learning, however, there are little reviews related to robotic manipulation. In this paper, we present a novel hierarchical architecture called Primitives Generation Policy Learning to enable continual learning. More specifically, a generative method by Variational Autoencoder is employed to generate state primitives from task space, then separate policy learning component is designed to learn torque control commands for different tasks sequentially. Furthermore, different task policies could be identified automatically by comparing reconstruction loss in the autoencoder. Experiment on robotic manipulation task shows that the proposed method exhibits substantially improved performance over some other continual learning methods.