Fusionformer: A Novel Adversarial Transformer Utilizing Fusion Attention for Multivariate Anomaly Detection

Abstract

Multivariate time series forecasting (MTSF) is of significant importance in the enhancement and optimization of real-world applications. The task of MTSF poses substantial challenges due to the unpredictability of temporal patterns and the complexity in modeling the influence of all nonpredictive sequences on the target sequence at different time stages. Recent research has demonstrated the potential held by the Transformer algorithm to augment long-term forecasting capability. However, certain obstacles considerably obstruct the direct application of the Transformer to MTSF, such as an unsuitable embedding method, inadequate consideration of intervariable associations, and the intrinsic restriction of the point-wise objective function. To overcome these challenges, the Fusionformer, an effective Transformer-based forecasting model, is put forth in this article, which is characterized by three distinctive features: 1) the introduction of a segment-wise sequence embedding (SWSE) method allows for the conversion of the input sequence into multiple informative segments; 2) the implementation of a fusion attention mechanism (FAM), designed to capture predominant features across the time dimension and to model intricate intervariable dependencies; and 3) the development of an adversarial learning method, equipped with an auxiliary discriminator, facilitates the learning of data distribution, instead of progressively correcting the prediction error, thus substantially enhancing the MTSF’s accuracy. Furthermore, a Fusionformer-based risk assessment (FRA) method is structured for open-pit mine slope failure early warning issue (SFEW), which aims to prevent potential disasters by accurately predicting future slope movement trends and assessing the probabilities of landslide occurrences. Experimental outcomes validate that Fusionformer outperforms existing forecasting methods, while the FRA framework provides valuable insights and practical guidance for real-world applications.