Speech Emotion Recognition with Facial Image Validation

Abstract

Emotion Recognition using speech and facial expression is vital in communication. It is a challenging task of identifying emotion with accuracy as different people have heterogenous ways of expression. This is a nascent research area that requires attention. Proposed system recognizes the person emotional state from audio signals and validate the user through facial image. It aims to have achieve accurate, efficient and improved interaction between human and computers. To extract most important features from the speech signal, multiscale transformation techniques are used which decomposes the signals into multi-frequency bands. Both the amplitude and phase are considered as the required features and are processed. The proposed approach adopts Support Vector Machine algorithm as a supervised learning algorithm and Principal Component Analysis used for facial images. An Accuracy of 86.66% is achieved. The system is trained and tested with speech audios of 15 actors taken from the RAVDESS database. Recall rate of 73.33% is detected.