From wild to lab: learning pose, identity and behavior across animals with deep learning

Understanding animal behavior is a critical goal for neuroscience. Computer vision contributes to neuroscience by providing automated, scalable analysis for animal movement and interaction. In controlled laboratory settings, pose estimation has proven highly effective in capturing the abstract structure and movement of animal bodies. This has led to the development of multi-animal pose estimation models to support research on social behavior. We present a novel architecture DLCRNet for multi-animal pose estimation on four multi-animal benchmarks in laboratory settings, targeting social interactions where animals are frequently occluded. Furthermore, we introduce BUCTD, a novel two-stage method that achieves state-of-the-art performance on human and animal benchmarks designed for crowded scenarios. While these advancements improve our understanding of behavior in controlled settings, studying animals in the wild require individual-level behavioral understanding over time and space, where re-identification (ReID) becomes crucial for tracking individuals across diverse environments and time periods. To address this, we develop PoseSwin, a novel ReID architecture that incorporates bodypart-aware features to distinguish individual Alaskan brown bears for longitudinal monitoring across years, habitats and viewpoints. Finally, to bridge the gap between foundational research and real-world application, we apply computer vision techniques to zebrafish welfare assessment. We aim to build a reliable fish behavioral dataset and a robust model for automated welfare monitoring. Although full behavioral dataset is still being collected, we present a data pre-processing pipeline and tool for annotation and conduct simulation experiments for abnormal behavior detection on a public dataset. These efforts provide practical insights into data collection and foundational tools for developing supervised methods in future studies.