EEG-based emotion detection using Long Short-Term Memory Network and reinforcement learning for enhanced feature selection

Brain–computer interface systems can recognize users’ emotions through electroencephalography (EEG). EEG-based human emotion recognition is an emerging field that is gaining significant traction within the realm of brain–computer interfaces. However, due to the complexity and diversity inherent in EEG signals, emotion recognition remains a challenge in pattern recognition. The critical task of selecting salient features from EEG and achieving high recognition accuracy warrants further exploration. In this paper, a hybrid emotion detection system is proposed by incorporating the reinforcement learning mechanism into a deep learning framework. Reinforcement learning is used to recursively select informative features, while a Long Short-Term Memory Network (LSTM) and a deep neural network are employed for enhanced feature selection and emotion recognition. Specifically, the LSTM, based on input features, determines and generates the current state, thereby aiding the policy model in making action decisions. This process successively retains or removes features to improve emotion recognition in the next state. The neural net-based policy model generates the policy actions based on the current state and the corresponding reward signal from the classification result, to control the feature selections for the subsequent states. A public EEG emotion dataset of SEED is used in the experiments. Results show that the proposed network model is effective in feature selections and emotion classifications, which reduces feature dimensions by 11.3% on average, and achieves a higher recognition accuracy of 92.65% compared to other approaches. The proposed system can use the current state info for prediction and adaptive feature selection, which can accommodate the data pattern differences of individual participants and leverage the model for a good performance.