Abstract:
The growing interest in monitoring cognitive workload has sped up the research in intelligent systems that utilize physiological signals. Mental workload (MWL), if unmanaged, can degrade human performance in safety-critical applications. This thesis introduces a deep learning (DL) approach that utilizes both EEG and ECG signals in MWL classification of the publicly available N-back dataset. Methodology begins with a multimodal-based cognitive workload classification system with four DL models and one machine learning (ML) baseline approach. GEL-RF is a hybrid combination designed to provide a lightweight and efficient baseline. The core models are an LSTM with self-attention to learn temporal dependencies and improve interpretability; an LSTM-HASTF model without self-attention to reduce computational complexity but maintain spatial-temporal learning; a proposed hybrid LSTM model with both self-attention and HASTF to jointly learn temporal and spatial interactions; and a final model that employs VAE on fusion features to reduce dimensionality and latency. All models are trained using subject-independent validation on EEG–ECG features extracted from the N-back dataset for both binary and multiclass cognitive workload classification tasks. The results show that the GEL-RF approach provided the best performance with binary accuracy of 97.0% and multiclass accuracy of 96.3%, thus showing its efficiency and generalizability. Deep model performance was best with LSTM+HASTF+SA (Model 3), achieving binary and multiclass accuracies of 94.46% and 90.63%, respectively. VAE outperformed with binary and multiclass accuracies of 94.58% and 90.65%. Model 2, LSTM with HASTF without self-attention, achieved binary and multiclass accuracies of 90.11% and 90.34%, thus showing a balance between efficiency and performance. Model 1, which combined LSTM with self-attention, showed good performance with binary and multiclass accuracies of 89.85% and 90.10%. Though DL models showed improved performance in temporal learning, the GEL-RF approach was more appropriate for real-time or restricted scenarios. This work makes the novel contribution of a scalable and interpretable model for MWL assessment in safety-critical settings like aviation, education, and healthcare.
