Self-Supervised Hyperbolic Spectro-Temporal Graph Convolution Network for Early 3D Behavior Prediction

Abstract

3D human behavior is a highly nonlinear spatiotemporal interaction process. Therefore, early behavior prediction is a challenging task, especially prediction with low observation rates in unsupervised mode. To this end, we propose a novel self-supervised early 3D behavior prediction frame-work that learns graph structures on hyperbolic manifold. Firstly, we employ the sequence construction of multi-dynamic key information to enlarge the key details of spatio-temporal behavior sequences, addressing the high redundancy between frames of spatio-temporal interaction. Secondly, for capturing dependencies among long-distance joints, we explore a unique graph Laplacian on hyperbolic manifold to perceive the subtle local difference within frames. Finally, we leverage the learned spatio-temporal features under different observation rates for progressive contrast, forming self-supervised signals. This facilitates the extraction of more discriminative global and local spatio-temporal information from early behavior sequences in unsupervised mode. Extensive experiments on three behavior datasets have demonstrated the superiority of our approach at low to medium observation rates.