Unified skeleton representation learning for autonomous driving

How can a model learn human skeleton representations that generalize across multiple tasks in autonomous driving and intelligent transportation systems? Traditional self-supervised methods fall short by entangling spatial and temporal features, limiting their adaptability to downstream perception and behavior understanding tasks. In this paper, we introduce MotionMoCo, a novel self-supervised framework designed to decouple spatial and temporal information for robust, transferable skeleton representations. MotionMoCo consists of two branches, a query branch and a momentum-updated key branch, each of which separately encodes the spatial, temporal, and combined features from the input motion sequence. Unlike previous works, we contrast these features independently across the two branches, thus preserving their complementary information content. Further, we propose a Motion-Aware Frame Masking strategy that selectively removes fast-evolving frames in the query branch to encourage robust feature extraction. MotionMoCo serves as a generic pretraining stage that can be fine-tuned for a range of autonomous driving applications, including pedestrian intention prediction, skeleton-based action recognition, 2D-to-3D pose lifting, and human mesh recovery, demonstrating that spatiotemporal disentanglement enhances the quality and adaptability of skeleton representations.