Self-Supervised Learning for Point Clouds through Multi-crop Mutual Prediction

As the demand for high-level autonomous driving continues to grow, efficient processing of large amount of point cloud data from LiDAR sensors has become a severe issue. Despite its great success, it is extremely difficult, if not impossible, to apply supervised learning in many real-world applications with natural scenes lacking annotations. In addition, removing label dependency and improving algorithm generalization ability are the current key research challenges in 3D deep learning. We propose a crop-based self-supervised learning framework for point clouds called Multi-Crop Mutual Prediction (MCMP). By cropping the original point clouds into different number of multiscale patches and making patches from the same object as close as possible through dynamic updates of online prototypes, the encoder is well able to maintain consistency between both local and global features. MCMP framework is model-agnostic so it is convenient to integrate it into encoders of any architectures. We pre-train three commonly used point cloud feature extraction networks on a single object-level dataset MondelNet40 and evaluate MCMP on three downstream tasks, namely, object classification, partial segmentation and semantic segmentation. Experimental results show that MCMP outperforms the previous crop-based self-supervised approaches in the object classification and semantic segmentation tasks and produces comparable results with the supervised benchmark methods.