Spatiotemporal Self-supervised Learning for Point Clouds in the Wild

Self-supervised learning (SSL) has the potential to ben- efit many applications, particularly those where manually annotating data is cumbersome. One such situation is the semantic segmentation of point clouds. In this context, ex- isting methods employ contrastive learning strategies and define positive pairs by performing various augmentation of point clusters in a single frame. As such, these meth- ods do not exploit the temporal nature of LiDAR data. In this paper, we introduce an SSL strategy that leverages pos- itive pairs in both the spatial and temporal domain. To this end, we design (i) a point-to-cluster learning strategy that aggregates spatial information to distinguish objects; and (ii) a cluster-to-cluster learning strategy based on unsu- pervised object tracking that exploits temporal correspon- dences. We demonstrate the benefits of our approach via extensive experiments performed by self-supervised train- ing on two large-scale LiDAR datasets and transferring the resulting models to other point cloud segmentation bench- marks. Our results evidence that our method outperforms the state-of-the-art point cloud SSL methods.