Artificial intelligent system for automatic depression level analysis through visual and vocal expressions

Abstract

A human being’s cognitive system can be simulated by artificial intelligent systems. Machines and robots equipped with cognitive capability can automatically recognize a humans mental state through their gestures and facial expressions. In this paper, an artificial intelligent system is proposed to monitor depression. It can predict the scales of Beck Depression Inventory (BDI-II) from vocal and visual expressions. Firstly, different visual features are extracted from facial expression images. Deep Learning method is utilized to extract key visual features from the facial expression frames. Secondly, Spectral Low-level Descriptors (LLDs) and Mel-frequency cepstral coefficients (MFCCs) features are extracted from short audio segments to capture the vocal expressions. Thirdly, Feature Dynamic History Histogram (FDHH) is proposed to capture the temporal movement on the feature space. Finally these FDHH and Audio features are fused using regression techniques for the prediction of the BDI-II scales. The proposed method has been tested on the public AVEC2014 dataset as it is tuned to be more focused on the study of depression. The results outperform all the other existing methods on the same dataset.