Skip navigation
User training | Reference and search service

Library catalog

Content aggregators
Please use this identifier to cite or link to this item:

Title: Multimodal approach for emotion recognition based on simulated flight experiments
Authors: Roza, V.
Postolache, O.
Keywords: Emotion recognition
Physiological sensing
Multimodal sensing
Deep learning
Flight simulation
Issue Date: 2019
Publisher: Multidisciplinary Digital Publishing Institute
Abstract: The present work tries to fill part of the gap regarding the pilots' emotions and their bio-reactions during some flight procedures such as, takeoff, climbing, cruising, descent, initial approach, final approach and landing. A sensing architecture and a set of experiments were developed, associating it to several simulated flights ( N f l i g h t s = 13 ) using the Microsoft Flight Simulator Steam Edition (FSX-SE). The approach was carried out with eight beginner users on the flight simulator ( N p i l o t s = 8 ). It is shown that it is possible to recognize emotions from different pilots in flight, combining their present and previous emotions. The cardiac system based on Heart Rate (HR), Galvanic Skin Response (GSR) and Electroencephalography (EEG), were used to extract emotions, as well as the intensities of emotions detected from the pilot face. We also considered five main emotions: happy, sad, angry, surprise and scared. The emotion recognition is based on Artificial Neural Networks and Deep Learning techniques. The Root Mean Squared Error (RMSE) and Mean Absolute Error (MAE) were the main methods used to measure the quality of the regression output models. The tests of the produced output models showed that the lowest recognition errors were reached when all data were considered or when the GSR datasets were omitted from the model training. It also showed that the emotion surprised was the easiest to recognize, having a mean RMSE of 0.13 and mean MAE of 0.01; while the emotion sad was the hardest to recognize, having a mean RMSE of 0.82 and mean MAE of 0.08. When we considered only the higher emotion intensities by time, the most matches accuracies were between 55% and 100%.
Peer reviewed: yes
DOI: 10.3390/s19245516
ISSN: 1424-8220
Accession number: WOS:000517961400179
Appears in Collections:IT-RI - Artigo em revista internacional com arbitragem científica

Files in This Item:
File Description SizeFormat 
sensors-19-05516 (2).pdfVersão Editora5.85 MBAdobe PDFView/Open

FacebookTwitterDeliciousLinkedInDiggGoogle BookmarksMySpace
Formato BibTex MendeleyEndnote Currículo DeGóis 

Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.