An Autoencoder based Emotional Stress State Detection Approach by using Electroencephalography Signals
Research Areas : Signal Processing
Keywords: EEG Signals, emotion analysis, stress analysis, autoencoder, machine learning, deep learning.,
Abstract :
Identifying hazards from human error is critical for industrial safety since dangerous and reckless industrial worker actions, as well as a lack of measures, are directly accountable for human-caused problems. Lack of sleep, poor nutrition, physical deformities, and weariness are some of the key factors that contribute to these risky and reckless behaviors that might put a person in a perilous scenario. This scenario causes discomfort, worry, despair, cardiovascular disease, a rapid heart rate, and a slew of other undesirable outcomes. As a result, it would be advantageous to recognize people's mental states in the future in order to provide better care for them. Researchers have been studying electroencephalogram (EEG) signals to determine a person's stress level at work in recent years. A full feature analysis from domains is necessary to develop a successful machine learning model using electroencephalogram (EEG) inputs. By analyzing EEG data, a time-frequency based hybrid bag of features is designed in this research to determine human stress dependent on their sex. This collection of characteristics includes features from two types of assessments: time-domain statistical analysis and frequency-domain wavelet-based feature assessment. The suggested two layered autoencoder based neural networks (AENN) are then used to identify the stress level using a hybrid bag of features. The experiment uses the DEAP dataset, which is freely available. The proposed method has a male accuracy of 77.09% and a female accuracy of 80.93%.
[1] R. E. Bender, and L. B. Alloy, “Life stress and kindling in bipolar disorder: review of the evidence and integration with emerging biopsychosocial theories”, Clinical psychology review, Vol. 31, No. 3, 2011, pp. 383–398.
[2] T. G. Pickering, “Mental stress as a causal factor in the development of hypertension and cardiovascular disease”, Current hypertension reports, Vol. 3, No. 3, 2001, pp. 249–254.
[3] T. C. Major, and J. M. Conrad, “A survey of brain computer interfaces and their applications”, in IEEE SOUTHEASTCON 2014, 2014, pp. 1–8.
[4] N. Elsayed, Z. S. Zaghloul, and M. Bayoumi, “Brain computer interface: EEG signal preprocessing issues and solutions”, Int. J. Comput. Appl, Vol. 169, No. 3, 2017, pp. 12–16.
[5] R. E. Wheeler, R. J. Davidson, and A. J. Tomarken, “Frontal brain asymmetry and emotional reactivity: A biological substrate of affective style”, Psychophysiology, Vol. 30, No. 1, 1993, pp. 82–89.
[6] A. C. Atencio et al., “Computing stress-related emotional state via frontal cortex asymmetry to be applied in passive-ssBCI”, in 5th ISSNIP-IEEE Biosignals and Biorobotics Conference: Biosignals and Robotics for Better and Safer Living (BRC), 2014, pp. 1–6.
[7] F.P. George et al. “Recognition of emotional states using EEG signals based on time-frequency analysis and SVM classifier”, International Journal of Electrical and Computer Engineering, Vol. 9, No. 2, 2019, pp. 1012-1020.
[8] T. F. Bastos-filho, A. Ferreira, A.C. Atencio, S. Arjunan, and D. Kumar, “Evaluation of Feature Extraction Techniques in Emotional State Recognition”, in 4th International conference on intelligent human computer interaction (IHCI), 2012, pp. 1-6.
[9] N. Jatupaiboon, S. Pan-ngum, and P. Israsena, “Real-time EEG-based happiness detection system”, The Scientific World Journal, 2013, pp. 1-12.
[10] M. J. Hasan, and J. M. Kim, “A hybrid feature pool-based emotional stress state detection algorithm using EEG signals”, Brain sciences, 2019, Vol. 9, No. 12, pp. 1-15.
[11] D. Shon, K. Im, J. Park, D. Lim, B. Jang, and J. Kim, “Emotional Stress State Detection Using Genetic Algorithm-Based Feature Selection on EEG Signals”, Int. J. of environmental research and public health, 2018, Vol. 15, No. 11, pp. 1-11.
[12] S. Koelstra, C. Muhl, M. Soleymani, J.S. Lee, A. Yazdani, T. Ebrahimi, T. Pun, A. Nijholt, and I. Patras, “Deap: A database for emotion analysis; using physiological signals”, IEEE transactions on affective computing, 2011, Vol. 3, No. 1, pp.18-31.
[13] Y. P. Lin, C. H. Wang, T. P. Jung, T. L. Wu, S. K. Jeng, J. R. Duann, and J. H. Chen, “EEG-based emotion recognition in music listening”, IEEE Transactions on Biomedical Engineering, 2010, Vol 57, No. 7, pp. 1798-1806.
[14] R. Jenke, A. Peer, and M. Buss, “Feature extraction and selection for emotion recognition from EEG”, IEEE Transactions on Affective Computing, 2014, Vol. 5, No. 3, pp. 327-39.
[15] M. Sohaib, C.-H. Kim, and J.-M. Kim, “A Hybrid Feature Model and Deep-Learning-Based Bearing Fault Diagnosis”, Sensors, 2017, Vol. 17, No. 12, pp. 1-16.
[16] J. A. Russell, “A circumplex model of affect”, Journal of Personality and Social Psychology, 1980, Vol. 39, No. 6, pp. 1161–1178.
[17] S.H. Oh, Y.R. Lee, and H.N. Kim, “A novel EEG feature extraction method using Hjorth parameter”, Int. Journal of Electronics and Electrical Engineering, 2014, Vol. 2, No. 2, pp. 106-110.
[18] P.C. Petrantonakis, and L.J. Hadjileontiadis, “Emotion Recognition from EEG Using Higher Order Crossings”, IEEE Transactions on Information Technology in Biomedicine, 2010, Vol. 14, No. 2, pp. 186–197.
[19] H. Shao, H. Jiang, H. Zhao, and F. Wang, “A novel deep autoencoder feature learning method for rotating machinery fault diagnosis”, Mechanical Systems and Signal Processing, 2017, Vol. 95, pp. 187-204.
http://jist.acecr.org ISSN 2322-1437 / EISSN:2345-2773 |
Journal of Information Systems and Telecommunication
|
An Autoencoder based Emotional Stress State Detection Approach Using Electroencephalography Signals |
Jia Uddin1*
|
1. AI and Big Data Department, Endicott College, Woosong University, Daejeon, South Korea |
Received: 17 Nov 2021/ Revised: 04 Mar 2022/ Accepted: 27 Apr 2022 |
|
Abstract
Identifying hazards from human error is critical for industrial safety since dangerous and reckless industrial worker actions, as well as a lack of measures, are directly accountable for human-caused problems. Lack of sleep, poor nutrition, physical deformities, and weariness are some of the key factors that contribute to these risky and reckless behaviors that might put a person in a perilous scenario. This scenario causes discomfort, worry, despair, cardiovascular disease, a rapid heart rate, and a slew of other undesirable outcomes. As a result, it would be advantageous to recognize people's mental states in the future in order to provide better care for them. Researchers have been studying electroencephalogram (EEG) signals to determine a person's stress level at work in recent years. A full feature analysis from domains is necessary to develop a successful machine learning model using electroencephalogram (EEG) inputs. By analyzing EEG data, a time-frequency based hybrid bag of features is designed in this research to determine human stress dependent on their sex. This collection of characteristics includes features from two types of assessments: time-domain statistical analysis and frequency-domain wavelet-based feature assessment. The suggested two layered autoencoder based neural networks (AENN) are then used to identify the stress level using a hybrid bag of features. The experiment uses the DEAP dataset, which is freely available. The proposed method has a male accuracy of 77.09% and a female accuracy of 80.93%.
Keywords: EEG Signals; Emotion Analysis; Stress Analysis; Autoencoder; Machine Learning.
1- Introduction
For engineering wellbeing, detecting consequences from human error is essential because unsafe and irresponsible manners of employees involved in manufacturing are clearly accountable for human-caused troubles. Among several key factors of these dangerous and irresponsible activities, lack of proper sleep leads a person to an extreme stressful situation. Stress initiates irritation, fear, sadness, vascular illness and numerous additional injurious effects [1], [2]. Numerous forms of brain signals, i.e., functional magnetic resonance imaging (fMRI), near-infrared spectroscopy (NIRS), Electrocorticography (ECoG), and electroencephalogram (EEG), are utilized for evaluating emotional conditions of individual [3]. Among all of these forms of data, EEG can be assessed non-intrusively [4]. The principal objective of this research is to categorize the emotional situation of an individual based on the sex by evaluating pre-processed freely accessible EEG signals.
Several surveys have exhibited relationships between EEG signals and several emotional situations [5–10]. In [5], an EEG-based assessment on the frontal channel with support vector machine (SVM) is designed. In [9], an in-depth analysis of power spectral density (PSD) is proposed to classify the emotional state by SVM. Among these researches, the common attribute is to consider all the features for classifier.
In this research, an EEG signal-driven emotional state classification method is established to evaluate whether a person is experiencing stress. By evaluating the signal, a hybrid feature bag is designed to create a dynamic and robust feature list. This process is divided into two parts: (1) statistical analysis from the time domain, and (2) wavelet-based feature assessment from the frequency domain. In the EEG signals, for the presence of the artifacts [11], it is hard to find the absolute feature information. This study examined pre-processed signals from the Database for Emotion Analysis Using Physiological Signals (DEAP) dataset [12]. The time domain features are defined in detail in Section 3. As the EEG signal has five indistinguishable bandwidths [13], [14], wavelet decomposition is studied to determine the frequency domain features, which are depicted in Section III. From these designed bag of hybrid features, instead of providing all of them to the classifier, some built-in feature reduction mechanism embedded classifier is used to utilize only the most significant features for final classification. Deep networks can obtain extremely characteristic features via their multi-layered model architectures. Moreover, they keep only the most representative information in each layer to reduce the dimensionality and also to improve the classification performance by selecting only the most intrinsic feature information [15]. In this research, a three layered autoencoder based neural network (AENN) is proposed for different emotional state classification. To prove the strength of the suggested technique, few comparisons are made with the approaches discussed into [5], and [9]. This paper's primary contributions can be summarized as follows:
(1) Statistical analysis in the time domain and wavelet-based feature assessment in the frequency domain combine to create a hybrid bag of features.
(2) A two layered AENN is proposed to learn and utilize only the important features through the embedded feed-forward feature selection architecture to improve the final classification accuracy.
The remainder of this paper will be organized as follows. The DEAP dataset is readily available, and the recommended approach is explained in Section 2. Section 3 contains the data agreement, evaluation of the experimental findings, and discussion, and Section 4 concludes the paper.
2- Proposed Method
In Fig. 1, a block diagram proposed approach is provided. The proposed approach is divided into four sections: (1) pre-processed data gathering [12], (2) data arrangement, (3) creation of hybrid bag of features, and (4) AENN-based classification.
The data is first down sampled to 128 Hz, and then the artifacts are eliminated from the data, as seen in this diagram. The current study's annotation was completed after filtering the data using bandpass frequency and common segmentation. The statistical features from the time domain and wavelet-based frequency domain are then examined and retrieved from each class sample. Finally, an Autoencoder-based Neural Network is presented for classification.