Evaluating the Emotional State of a User Using a Webcam
DOI:
https://doi.org/10.9781/ijimai.2016.4112Keywords:
Classification, Emotion recognition, Facial recognitionAbstract
In online learning is more difficult for teachers identify to see how individual students behave. Student’s emotions like self-esteem, motivation, commitment, and others that are believed to be determinant in student’s performance can not be ignored, as they are known (affective states and also learning styles) to greatly influence student’s learning. The ability of the computer to evaluate the emotional state of the user is getting bigger attention. By evaluating the emotional state, there is an attempt to overcome the barrier between man and non-emotional machine. Recognition of a real time emotion in e-learning by using webcams is research area in the last decade. Improving learning through webcams and microphones offers relevant feedback based upon learner’s facial expressions and verbalizations. The majority of current software does not work in real time – scans face and progressively evaluates its features. The designed software works by the use neural networks in real time which enable to apply the software into various fields of our lives and thus actively influence its quality. Validation of face emotion recognition software was annotated by using various experts. These expert findings were contrasted with the software results. An overall accuracy of our software based on the requested emotions and the recognized emotions is 78%. Online evaluation of emotions is an appropriate technology for enhancing the quality and efficacy of e-learning by including the learner´s emotional states.Downloads
References
[1] K. Bahreini, R. Nadolski, & W. Westera, Towards multimodal emotion recognition in e-learning environments. Interactive Learning Environments, no. Ahead-of-print, 2014, p. 1-16.
[2] J. N. Bailenson, E. D. Pontikakis, I. B. Mauss, J. J. Gross, M. E. Jabon, C. A. C. Hutcherson, . . . O. John, Real-time classification of evoked emotions using facial feature tracking and physiological responses. International Journal of Human Computer Studies, 66(5), 2008. 303-317.
[3] U. Bakshi, & R. Singhal, A Survey of face detection methods and feature extraction techniques of face recognition. International Journal of Emerging Trends & Technology in Computer Science (IJETTCS), 3(3), 2014, 233-237.
[4] S. Baron-Cohen, Reading the mind in the face: A cross-cultural and developmental study. Visual Cognition, 3(1), 1996, 39-60.
[5] S. Baron-Cohen, O. Golan, S. Wheelwright, & J. Hill, Mind reading: the interactive guide to emotions. 2004.
[6] V. Bettadapura, Face Expression Recognition and Analysis: The State of the Art. arXiv: Tech Report, (4), 2012, 1-27.
[7] I. Birzniece, P. Rudzajs, D. Kalibatiene, O. Vasilecas, & E. Rencis, Application of interactive classification system in university study course comparison. Informatics in Education, 14(1), 2015, 15-36.
[8] P. Brusilovsky, I.H. Hsiao, & Y, Folajimi, QuizMap: open social student modeling and adaptive navigation support with TreeMaps. Proceedings of the 6th European conference on Technology enhanced learning: towards ubiquitous learning (EC-TEL’11), Carlos Delgado Kloos, Denis Gillet, Raquel M. Crespo García, Fridolin Wild, and Martin Wolpers (Eds.). Springer-Verlag, Berlin, Heidelberg, 2011, 71-82.
[9] U. Canzlerm & T. Dziurzyk, T. Extraction of Non Manual Features for Video based Sign Language Recognition. Proceedings of IAPR Workshop, 2002, 318-321.
[10] M. Corzilus, & M. Smids, A Real-time Emotion Mirror, 2006, Available: http://www.inf.unideb.hu/~sajolevente/papers2/emotion/emotion_mirror.pdf
[11] F. Dadgostar, & A. Sarrafzadeh, An adaptive real-time skin detector based on hue thresholding: A comparison on two motion tracking methods. Pattern Recognition Letters, 27(12), 2006, 1342-1352.
[12] K.E. DeLeeuw, R.E. Mayer, & B. Giesbrecht, How does text affect the processing of diagrams in multimedia learning. Proceedings of the 6th international conference on Diagrammatic representation and inference (Diagrams’10), Ashok K. Goel, Mateja Jamnik, and N. Hari Narayanan (Eds.). Springer-Verlag, Berlin, Heidelberg, 2010, 304-306.
[13] M. Drlík, & J. Skalka, Virtual Faculty Development Using Top-down Implementation Strategy and Adapted EES Model. Proceedings of the World Conference on Educational Technology Research. East Univ, Nicosia, Cyprus, Paper presented at the Procedia - Social and Behavioral Sciences, 28, 2011, 616-621.
[14] P. Ekman & E. L. Rosenberg, What the face reveals: basic and applied studies of spontaneous expression using the facial action coding system (FACS), Illustrated Edition, Oxford University Press, 1996, 3-559.
[15] P. Ekman, & W. V. Friesen, Constants across cultures in the face and emotion. Journal of Personality and Social Psychology, 17(2), 1971, 124- 129.
[16] M. Feidakis, T. Daradoumis, & S. Caballe, Emotion Measurement in Intelligent Tutoring Systems: What, When and How to Measure. Third International Conference on Intelligent Networking and Collaborative Systems, 2011, 807-812.
[17] J. M. Harley, F. Bouchet, M. S. Hussain, R. Azevedo, & R. Calvo, A multi-componential analysis of emotions during complex learning with an intelligent multi-agent system. Computers in Human Behavior, 48, 2015, 615-625.
[18] I. -. Hsiao, P. Brusilovsky, M. Yudelson & A.Ortigosa, The value of adaptive link annotation in e-learning: A study of a portal-based approach. Paper presented at the HT’10 - Proceedings of the 21st ACM Conference on Hypertext and Hypermedia, 2010, 223-227.
[19] I. Huhnel, M. Fölster, K. Werheid & U. Hess, Empathic reactions of younger and older adults: No age related decline in affective responding. Journal of Experimental Social Psychology, 50, 2014, 136-143.
[20] Y. Chen, R. Hwang, & C. Wang, Development and evaluation of a web 2.0 annotation system as a learning tool in an e-learning environment, Computers and Education, 58(4), 2011, p. 1094-1105.
[21] CC. Chibelushi, & F. Bourel, Facial expression recognition: a brief tutorial overview, CVonline: On-Line Compendium of Computer Vision, Proceedings of the 13th Annual ACM International Conference on Multimedia, 5, 2002, 669-676.
[22] K. Jakobs, (2009). ICT standardisation in china, the EU, and the US. Paper presented at the International Telecommunication Union - Proceedings of the 2009 ITU-T Kaleidoscope Academic Conference: Innovations for Digital Inclusion, K-IDI 2009, 2009, 177-182.
[23] Š. Koprda, P. Brečka, & M. Maroš, Project and realization of WIFI net. Proceedings of the 2009 conference Trends in education: Information Technologies and technical education, Olomouc, Votobia, 2009, 301-304.
[24] Š. Koprda, M. Munk, D. Klocoková, & J. Záhorec, Základy práce s PC a v prostredí operačného systému MS Windows. In Elearning 2006. Hradec Králové: Gaudeamus, 2006, 239-249.
[25] K. Kostolányová, E-learning Form of Adaptive Instruction. Proceedings of the 9th International Scientific Conference on Distance Learning in Applied Informatics (DIVAI 2012). Nitra: Faculty of Natural Sciences Constantine the Philosopher University in Nitra, 2012, 193-201.
[26] K. Kostolányová, O. Takács, & J. Šarmanová, Automated Assessment of Students’ Knowledge in Adaptive Instruction. Information and Communication Technology in Education (ICTE 2012). Univ Ostrave, Pedag Fac, Roznov pod Radhostem, Czech Republic, 2012, 119-130.
[27] K. Kostolányová, O. Takács, & J. Šarmanová, Adaptive Education Process Modeling. Proceedings of the 10th International Conference on Efficiency and Responsibility in Education (ERIE 2013), Prague 6th - 7th June 2013. Prague : Czech University of Life sciences, 2013, 300-308.
[28] E. Krahmer, & M., Swerts, Audiovisual Expression of Emotions in Communication. Philips Research Book Series. Springer Netherlands, 12, 2011, 85-106.
[29] H.-T. Lin, Ch.-H. Wang,, CH.-F. Lin, & S.-M. Yuan, Annotating Learning Materials on Moodle LMS. Proceedings of the 2009 International Conference on Computer Technology and Development, Volume 02 (ICCTD ‘09), IEEE Computer Society, Washington, DC, USA, 2009, 455- 459.
[30] M. Magdin, M. Cápay, New Module for Statistical Evaluation in LMS Moodle. Proceedings of the 11th European Conference on e-Learning, ECEL 2012 : 26-27 October, Groningen, The Netherlands. - Groningen : Academic Publishing International Limited, 2012. 304-310.
[31] R. E. Mayer, Applying the science of learning to medical education. Medical Education, 44(6), 2010, 543-549.
[32] A. Mehrabian, Communication without words. Psychology Today, 2(4), 1968, 53–56.
[33] P. Miller, Web 2.0: Building the New Library. Ariadne, Issue 45, 2005, Available: http://www.ariadne.ac.uk/issue45/miller/
[34] J. L. Moreira, A.Braun, & S. R. Musse, Eyes and eyebrows detection for performance driven animation. Paper presented at the Proceedings - 23rd SIBGRAPI Conference on Graphics, Patterns and Images, SIBGRAPI, 2010, 17-24.
[35] G. R. S. Murthy, & R. S. Jadon, Effectiveness of Eigenspaces for facial expression recognition. International Journal of Computer Theory and Engineering, 1 (5), 2009, 638-642.
[36] M. Nixon, & A. S. Aguado, Feature Extraction and Image Processing, 2nd ed. Academic Press, 2007.
[37] A. Ollo-López, & M. E. Aramendía-Muneta, ICT impact on competitiveness, innovation and environment. Telematics and Informatics, 29 (2), 2012, 204-210.
[38] Omron. OKAO Vision, 2009, Available: http://www.omron.com/r_d/coretech/vision/okao.html.
[39] M. Pantic, N. Sebe, J. F. Cohn, & T. Huang, .Affective Multimodal Human computer interaction, 2005
[40] U. Saeed, & J. C. Dugelay, Combining edge detection and region segmentation for lip contour extraction. Proceedings of the 6th international conference on Articulated motion and deformable objects (AMDO’10), Francisco J. Perales and Robert B. Fisher (Eds.). SpringerVerlag, Berlin, Heidelberg, 2010, 11-20.
[41] A. Sarrafzadeh, S. Alexander, F. Dadgostar, C. Fan, & A. Bigdeli, How do you know that I don’t understand? A look at the future of intelligent tutoring systems. Computers in Human Behavior, 24(4), 2008, 1342-1363.
[42] S. Srivastava, Real Time Facial Expression Recognition Using A Novel Method. International Journal of Multimedia & Its Applications (IJMA), 4(2), 2012, 49-57.
[43] M. Suk, & B. Prabhakaran, Real-time facial expression recognition on smartphones. Paper presented at the Proceedings - 2015 IEEE Winter Conference on Applications of Computer Vision, WACV 2015, 2015, 1054-1059.
[44] C.P. Sumathi, T. Santhanam, & M. Mahadevi, Automatic Facial Expression Analysis A survey. International Journal of Computer Science & Engineering Survey (IJCSES), 3(6), 2012, 47-59.
[45] J. Sun, Y. M. Chai, Guo, & H. F. Li, On-line adaptive principal component extraction algorithms using iteration approach. Signal Processing, 92(4), 2012, 1044-1068.
[46] The FERET Database [online]. Available: http://www.itl.nist.gov/iad/humanid/feret/
[47] V. T. Tinio, ICT in Education. Presented By UNDP for the Benefit of Participants To The World Summit On The Information Society. UNDP’s Regional Project, the Asia-Pacific Development Information Program (APDIP), In Association with the Secretariat of ASEAN, 2009, Available: http://www.apdip.net/publications/iespprimers/eprimer-edu.pdf
[48] M. Trojahn, F. Arndt, M. Weinmann, & F. Ortmeier, Emotion recognition through keystroke dynamics on touchscreen keyboards. Paper presented at the ICEIS 2013 - Proceedings of the 15th International Conference on Enterprise Information Systems, 3, 2013, 31-37.
[49] M. Turk, & A. Pentland, Eigenfaces for recognition. Journal of Cognitive Neuroscience, 3(1), 1991, 71–86.
[50] P. Viola, & M. J. Jones, Robust real-time object detection. International Journal of Computer Vision, 57(2), 2004, 137–154.
[51] J. Whitehill, G. Littlewort, I. Fasel, M. Bartlett, & J. Movellan, Toward Practical Smile Detection. Pattern Analysis and Machine Intelligence, 31(11), 2009.
[52] M. Yang, D. J. Kriegman, & N. Ahuja, Detecting faces in images: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(1), 2002, 34-58.
[53] Z. Zhang, Feature-Based Facial Expression Recognition: Sensitivity Analysis and Experiment with a Multi-Layer Perceptron. International Journal of Pattern Recognition Artificial Intelligence, 13 (6), 1999, 893- 911.
Downloads
Published
-
Abstract69
-
PDF92
Submit Article
Information
