Multilingual Speech Emotion Recognition using AI

Abstract

Huge datasets and strong processing power have propelled notable progress in Speech Emotion Recognition (SER) systems.The design and assessment of a multilingual SER system, with an emphasis on English and Urdu, is the subject of this thesis. The project expands the reach of the Urdu dataset by working with a variety of speakers and making use of pre-existing English datasets like RAVDESS, TESS, SAVEE, and CREMA-D. The process consists of preparing the data, extracting features, training the model with machine learning techniques, and integrating the model into an intuitive mobile application. Comprehensive experiments are carried out to assess the performance of Long Short-Term Memory (LSTM) and Convolutional Neural Network (CNN) models on datasets in English and Urdu.The outcomes show increased accuracy rates in particular, the CNN model performs remarkably well on the English dataset and MLP Classifier on the Urdu dataset. Nonetheless, issues with the Urdu dataset’s constrained surface area call for additional development and expansion. The thesis ends with recommendations for improving the system’s possible future uses in a variety of fields, like emotional intelligence testing and lie detection.