Learning With Noisy Labels for Industrial Time Series Outlier Detection: A Transformer-Embedded Contrastive Learning Framework

Abstract

In many real-world industrial scenarios, acquiring accurately labeled data are often challenging due to limited resources or unexpected errors. Learning with noisy labels (LNL) has emerged as a significant research topic, aiming to develop reliable deep learning models using noisy-labeled training data. In this article, a novel Transformer-embedded LNL framework with fuzzy-clustering-assisted contrastive learning is developed for industrial time series outlier detection under noisy labels. Specifically, a fuzzy-clustering-assisted contrastive learning strategy is proposed to enhance the robustness of the Transformer encoder against noisy labels by leveraging the intrinsic characteristics of raw data. Furthermore, a dynamic two-stage training scheme is introduced to train the outlier detector. In the first training stage, the Transformer encoder is pretrained through data reconstruction to improve feature extraction capabilities for industrial time series. In the second stage, the outlier detector is jointly trained with the Transformer encoder, incorporating a joint learning strategy. Furthermore, a label-consistency regularization term is designed to enhance the robustness of the outlier detector against noisy labels by minimizing the discrepancy between the outputs of the outlier detector and the clustering algorithm. The proposed framework is applied to industrial time series data collected from a real-world wire arc additive manufacturing (WAAM) process. Experimental results demonstrate that the developed framework outperforms selected representative LNL approaches in WAAM outlier detection under both low and high noise ratios.