XAI-DSCSA: explainable-AI-based deep semi-supervised convolutional sparse autoencoder for facial expression recognition

Abstract

Facial expression recognition (FER) continues to be a vibrant research field, driven by the increasing need for its practical applications in areas such as e-learning, healthcare, candidate interview analysis, and more. Most deep learning approaches in supervised FER systems heavily rely on large, labeled datasets. Implementing FER in Convolutional Neural Networks (CNNs) often requires many layers, leading to extended training times and difficulties in finding optimal parameters. This can result in challenges in creating distinct facial expression patterns for classification, leading to poor real-time emotion classification In this paper, we propose a novel approach known as the Deep Semi-supervised Convolutional Sparse Autoencoder to address the aforementioned issues and enhance FER performance and prediction accuracy. This approach comprises two parts: (i) Initially, a deep convolutional sparse autoencoder is trained with unlabeled samples of facial expressions. Here, sparsity is introduced in the convolutional block to enforce penalties, focusing on more relevant features for feature representation in the latent space. (ii) A trained encoder with a feature map is connected to a fully connected layer with softmax for final fine-tuning with learned weights and labeled facial expression samples in a semi-supervised approach for emotion classification. This approach was experimented with two benchmark datasets, namely CK + and JAFFE, and achieved significant results of 98.98% and 93.10% accuracy, respectively. The results were analyzed using established state-of-the-art techniques. Additionally, eXplainable AI (XAI) methods like Grad-CAM and image-LIME were employed to interpret the performance and prediction outcomes of the DSCSA model.