RDGFormer: Rough Domain Grassmann Transformer for Intention Prediction

Abstract

The modeling of highly nonlinear dynamic relation-ships among humans, objects, and the environment from videos is a complex task. Moreover, extrapolating human intentions from these relationships presents an even greater challenge. To effectively parse such intentions, we propose the concept of intention prediction, which means to anticipate future interaction trends. In this paper, a novel Transformer-based intention prediction framework on Grassmann manifold has been proposed. Specifically, to decompose ambiguous video spatio-temporal dynamic information, a separation method based on rough sets has been proposed to decouple critical cues in distinct geometric spaces. To effectively capture topological-geometric information across diverse spaces, we introduce Grassmann manifold projection with learnable orthogonal bases for multi-scale topology learning. Finally, a homeomorphic evolution module with adaptive residual connections is proposed to refine the parametric embedding space. The experimental results on three datasets demonstrate that our proposed method not only achieves state-of-the-art (SOTA) performance in prediction accuracy, but also significantly outperforms other comparative methods in intention prediction.