Characterization of negative emotions from handwriting and drawing based tasks

Abstract

Emotion detection is very important for maintaining human health especially when we are discussing in context of negative emotions like anxiety, stress, and depression. Clinical practitioners perform lengthy sessions with potential patients to assess their emotional state. To perform such sessions they use different devices that record brain signals. They ask questions from subjects and in response, they observe the brain signals. It is a lengthy process and may still not be correct because the questions and answer session makes most individuals uncomfortable and they tend to respond differently. A considerable amount of work has been done by the pattern recognition society on the detection of emotions by using facial expressions and speech signals. However, the potential of graphomotor tasks like handwriting and drawing as biomarkers of negative emotions is yet to be explored. Theories in graphology and other behavioural sciences have shown that graphomotor skills of an individual is affected by his emotional state. When individuals suffer from negative emotions, it creates a cognitive burden. Since handwriting and drawing is a productive of several cognitive processes, therefore, these tasks can be used for the detection of such emotions. Based on this postulate, we have evaluated the potential of handwriting and drawing tasks as potential screening tools for three negative emotions namely Anxiety, Stress, and Depression. Our prime focus is to exploit the dynamic handwriting signals since they contain extra information regarding the underlying processes. By extracting useful features from raw pen-based signals captured by a digitizer tablet, we have attempted to classify samples of individuals suffering from Anxiety, Stress, and Depression. We have further conducted experiments to predict the severity level of the emotional states as well. In addition to classification, we have also employed regression to predict ground truth scores computed by the domain experts using DASS scale. The results of our experiments show that our proposed model comprising of one-dimensional convolution followed by Recurrent Neural Networks can predict the emotional state of an individual with much success. Our study can serve as baseline for future researchers interested in this domain.