Are Instructed Emotional States Suitable for Classification? Demonstration of How They Can Significantly Influence the Classification Result in An Automated Recognition System
DOI:
https://doi.org/10.9781/ijimai.2018.03.002Keywords:
Face Detection, Facial Expression, Emotion, Instructed Stimuli, Real-life StimuliAbstract
At the present time, various freely available or commercial solutions are used to classify the subject's emotional state. Classification of the emotional state helps us to understand how the subject feels and what he is experiencing in a particular situation. Classification of the emotional state can thus be used in various areas of our life from neuromarketing, through the automotive industry (determining how emotions affect driving), to implementing such a system into the learning process. The learning process, which is the (mutual) interaction between the teacher and the learner, is an interesting area in which individual emotional states can be explored. In this pedagogical-psychological area several research studies were realized. These studies in some cases demonstrated the important impact of the emotional state on the results of the students. However, for comparison and unambiguous classification of the emotional state most of these studies used the instructed (even constructed) stereotypical facial expressions of the most well-known test databases (Jaffe is a typical example). Such facial expressions are highly standardized, and the software can recognize them with a fairly big percentage, but this does not necessarily point to the actual success rate of the subject's emotional classification in such a test because the similarity to real emotional expression remains unknown. Therefore, we examined facial expressions in real situations. We have subsequently compared these examined facial expressions with the instructed expressions of the same emotions (the Jaffe database). The overall average classification score in real facial expressions was 94.58%.Downloads
References
L. Abramson, I. Marom, R. Petranker & H. Aviezer, “Is fear in your head? A comparison of instructed and real-life expressions of emotion in the face and body,” in Emotion, 17(3), 2017, 557-565. doi:10.1037/emo0000252
W.N.W. Ahmad, N.M. Ali, “A Study on Persuasive Technologies: The Relationship between User Emotions, Trust and Persuasion”, in International Journal of Interactive Multimedia and Artificial Intelligence, Volume in Press, Issue in Press, 2018, 1-5. doi: 10.9781/ijimai.2018.02.010
P. Arnau-González, M. Arevalillo-Herráez & N. Ramzan, “Fusing highly dimensional energy and connectivity features to identify affective states from EEG signals,” in Neurocomputing, 244, 2017, 81-89. doi:10.1016/j.neucom.2017.03.027
K. Bahreini, R. Nadolski & W. Westera, “Towards multimodal emotion recognition in e-learning environments,” in Interactive Learning Environments, 24(3), 2016, 590-605. doi:10.1080/10494820.2014.908927
Y. Balasubramanian, K. Sivasankaran & S.P. Krishraj, “Forensic video solution using facial featurebased synoptic video footage record,” in IET Computer Vision, 10(4), 2016, 315-320. doi:10.1049/iet-cvi.2015.0238
V. Bettadapura, G. Schindler, T. Ploetz & I. Essa, “Augmenting bag-of-words: Data-driven discovery of temporal and structural information for activity recognition,” Paper presented at the Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2013, 2619-2626. doi:10.1109/CVPR.2013.338
A. Bhattacharya, D. Choudhury & D. Dey, “Edge-enhanced bidimensional empirical mode decomposition-based emotion recognition using fusion of feature set,” in Soft Computing, 2016, 1-15. doi:10.1007/s00500-016-2395-4
I. Cohen, N. Sebe, F.G. Cozman, M.C. Cirelo & T.S. Huang, “Learning bayesian network classifiers for facial expression recognition using both labeled and unlabeled data,” Paper presented at the Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 1, 2003, I/595-I/601.
D.T. Cordaro, D. Keltner, S. Tshering, D. Wangchuk & L.M. Flynn, “The voice conveys emotion in ten globalized cultures and one remote village in bhutan,” in Emotion, 16(1), 2016, 117-128. doi:10.1037/emo0000100
C. Darwin & F. Darwin, “The expression of the emotions in man and animals. The expression of the emotions in man and animals, 2009, 1-401. doi:10.1017/CBO9780511694110
M.A. DiGirolamo & J.A. Russell, “The emotion seen in a face can be a methodological artifact: The process of elimination hypothesis,” in Emotion, 17(3), 2017, 538-546. doi:10.1037/emo000024
P. Ekman, “An argument for basic emotions,” in Cognition and Emotion, 6(3-4), 1992, 169-200. doi:10.1080/02699939208411068
P. Ekman, “What scientists who study emotion agree about,” in Perspectives on Psychological Science, 11(1), 2016, 31-34. doi:10.1177/1745691615596992
P. Ekman & D. Cordaro, “What is meant by calling emotions basic,” in Emotion Review, 3(4), 2011, 364-370. doi:10.1177/1754073911410740
P. Ekman & W.V. Friesen, “Felt, false, and miserable smiles,” in Journal of
P. Ekman & W.V. Friesen, Unmasking the face: A guide to recognizing emotions from facial clues. Prentice-Hall, Englewood Cliffs, NJ (1975)
N.L. Etcoff & J.J. Magee, “Categorical perception of facial expressions,” in Cognition, 44(3), 1992, 227-240. doi:10.1016/0010-0277(92)90002-Y
B. Fasel & J. Luettin, “Automatic facial expression analysis: A survey,” in Pattern Recognition, 36(1), 2003, 259-275.
S. Fleck & W. Straßer, “Smart camera based monitoring system and its application to assisted living,” Paper presented at the Proceedings of the IEEE, 96(10), 2008, 1698-1714. doi:10.1109/JPROC.2008.928765
D. Goren & H.R. Wilson, “Quantifying facial expression recognition across viewing conditions,” in Vision Research, 46(8-9), 2006, 1253-1262. doi:10.1016/j.visres.2005.10.028
U. Hess & P. Thibault, “Darwin and emotion expression,” in American Psychologist, 64(2), 2009, 120-128. doi:10.1037/a0013386
M.S. Hossain & G. Muhammad, “Audio-visual emotion recognition using multi-directional regression and ridgelet transform,” in Journal on Multimodal User Interfaces, 10(4), 2016, 325-333. doi:10.1007/s12193-015-0207-2
M.A. Jarwar & I. Chong, “Exploiting IoT services by integrating emotion recognition in web of objects,” Paper presented at the International Conference on Information Networking, 2017, 54-56. doi:10.1109/ICOIN.2017.7899474
D. Jayatilake, T. Isezaki, Y. Teramoto, K. Eguchi & K. Suzuki, “Robot assisted physiotherapy to support rehabilitation of facial paralysis,” Paper presented at the IEEE Transactions on Neural Systems and Rehabilitation Engineering, 22(3), 2014, 644-653. doi:10.1109/TNSRE.2013.2279169
S.M. Lajevardi & Z.M. Hussain, “Feature selection for facial expression recognition based on optimization algorithm,” Paper presented at the Proceedings of 2009 2nd International Workshop on Nonlinear Dynamics and Synchronization, INDS 2009, 2009, 182-185.
R.W. Levenson, “Basic emotion questions,” in Emotion Review, 3(4), 2011, 379-386. doi:10.1177/1754073911410743
L. Likforman-Sulem, A. Esposito, M. Faundez-Zanuy, S. Clemencon & G. Cordasco, “EMOTHAW: A novel database for emotional state recognition from handwriting and drawing,” Paper presented at the IEEE Transactions on Human-Machine Systems, 47(2), 2017, 273-284. doi:10.1109/THMS.2016.2635441
K. Matsuno, C. Lee, S. Kimura & S. Tsuji, “Automatic recognition of human facial expressions,” Paper presented at the IEEE International Conference on Computer Vision, 1995, 352-359.
J.D. Mayer, P. Salovey, D.R. Caruso & G. Sitarenios, “Emotional intelligence as a standard intelligence,” in Emotion, 1(3), 2001, 232-242. doi:10.1037/1528-3542.1.3.232
D. McDuff, R. El Kaliouby & R.W. Picard, “Crowdsourcing facial responses to online videos,” Paper presented at the IEEE Transactions on Affective Computing, 3(4), 2012, 456-468. doi:10.1109/T-AFFC.2012.19
A. Mehrabian, “Pleasure-arousal-dominance: A general framework for describing and measuring individual differences in temperament,” in Current Psychology, 14(4), 1996, 261-292.
N. Mehta & S. Jadhav, “Facial emotion recognition using log gabor filter and PCA,” Paper presented at the Proceedings - 2nd International Conference on Computing, Communication, Control and Automation, ICCUBEA 2016, 2017, doi:10.1109/ICCUBEA.2016.7860054
G. Muhammad & M.F. Alhamid, “User emotion recognition from a larger pool of social network data using active learning,” in Multimedia Tools and Applications, 76(8), 2017, 10881-10892. doi:10.1007/s11042-016-3912-2
J.C. Mundt, A.P. Vogel, D.E. Feltner & W.R. Lenderking, “Vocal acoustic biomarkers of depression severity and treatment response,” in Biological Psychiatry, 72(7), 2012, 580-587. doi:10.1016/j.biopsych.2012.03.015
R. Plutchik, (2001). The Nature of Emotions. Am. Scientist, vol. 89, 344
N. Rathee, A. Vaish & S. Gupta, “Adaptive system to learn and recognize emotional state of mind,” Paper presented at the Proceeding - IEEE International Conference on Computing, Communication and Automation, ICCCA 2016, 2017, 32-36. doi:10.1109/CCAA.2016.7813726
L.D. Riek & R.N.M. Watson, “The age of avatar realism: When seeing shouldn’t be believing,” Paper presented at the IEEE Robotics and Automation Magazine, 17(4), 2010, 37-42. doi:10.1109/MRA.2010.938841
J.A. Russell, “Affective space is bipolar,” in Journal of Personality and Social Psychology, 37(3), 1979, 345-356. doi:10.1037//0022-3514.37.3.345
C. Torres-Valencia, M. Álvarez-López & Á. Orozco-Gutiérrez, “SVM-based feature selection methods for emotion recognition from multimodal data,” in Journal on Multimodal User Interfaces, 11(1), 2017, 9-23. doi:10.1007/s12193-016-0222-y
P. Visutsak, “Template-based affine registration of autistic brain images,” Paper presented at the IJCCI 2015 - Proceedings of the 7th International Joint Conference on Computational Intelligence, 2, 2015, 188-192.
Y. Yang, C. Fairbairn & J.F. Cohn, “Detecting depression severity from vocal prosody,” Paper presented at the IEEE Transactions on Affective Computing, 4(2), 2013, 142-150. doi:10.1109/T-AFFC.2012.38
A.W. Young, D. Rowland, A.J. Calder, N.L. Etcoff, A. Seth & D.I. Perrett, “Facial expression megamix: Tests of dimensional and category accounts of emotion recognition,” in Facial expression recognition: Selected works of Andy Young, 2016, 68-110. doi:10.4324/9781315715933
M. Zhang, D. Lee, A. Desai, K.D. Lillywhite & B.J. Tippetts, “Automatic facial expression recognition using evolution-constructed feature,” Paper presented at the 10th International Symposium on Visual Computing, ISVC 2014, 2014, 282-291
M. Magdin, F. Prikler. Real Time Facial Expression Recognition Using Webcam and SDK Affectiva, International Journal of Interactive Multimedia and Artificial Intelligence, (2017), http://dx.doi.org/10.9781/ijimai.2017.11.002
Downloads
Published
-
Abstract76
-
PDF46
Submit Article
Information
