A Depth Video-based Human Detection and Activity Recognition using Multi-features and Embedded Hidden Markov Models for Health Care Monitoring Systems
DOI:
https://doi.org/10.9781/ijimai.2017.447Keywords:
Feature Extraction, Hidden Markov Models, Monitoring, Human Detection ActivityAbstract
Increase in number of elderly people who are living independently needs especial care in the form of healthcare monitoring systems. Recent advancements in depth video technologies have made human activity recognition (HAR) realizable for elderly healthcare applications. In this paper, a depth video-based novel method for HAR is presented using robust multi-features and embedded Hidden Markov Models (HMMs) to recognize daily life activities of elderly people living alone in indoor environment such as smart homes. In the proposed HAR framework, initially, depth maps are analyzed by temporal motion identification method to segment human silhouettes from noisy background and compute depth silhouette area for each activity to track human movements in a scene. Several representative features, including invariant, multi-view differentiation and spatiotemporal body joints features were fused together to explore gradient orientation change, intensity differentiation, temporal variation and local motion of specific body parts. Then, these features are processed by the dynamics of their respective class and learned, modeled, trained and recognized with specific embedded HMM having active feature values. Furthermore, we construct a new online human activity dataset by a depth sensor to evaluate the proposed features. Our experiments on three depth datasets demonstrated that the proposed multi-features are efficient and robust over the state of the art features for human action and activity recognition.Downloads
References
Petersen, P., Kandelman, D., Arpin, S., & Ogawa, H. (2010). Global oral health of older people—Call for public health action. Community Dental Health, 27(4), 257–268.
Osmani, V., Balasubramaniam, S., & Botvich, D. (2008). Human activity recognition in pervasive health-care: Supporting efficient remote collaboration. Journal of Network and Computer Applications, 31(4), 628–655.
Dunn, J., Rudberg, M., Furner, S., & Cassel, C. (1992). Mortality, disability, and falls in older persons: The role of underlying disease and disability. American Journal of Public Health, 82(3), 395–400.
Reinhardt, U. (2003). Does the aging of the population really drive the demand for health care? Health Affairs, 22(6), 27–39.
Mynatt, E., & Rogers, W. (2001). Developing technology to support the functional independence of older adults. Ageing International, 27(1), 24–41.
Jalal, A., Kim, Y., Kim, Y., Kamal, S., & Kim, D. (2017). Robust human activity recognition from depth video using spatiotemporal multi-fused features. Pattern Recognition, 61, 295–308.
Xu, W., Zhang, M., Sawchuk, A., & Sarrafzadeh, M. (2012). Co-recognition of human activity and sensor location via compressed sensing in wearable body sensor networks. In Proceedings of the Ninth International Conference on Wearable and Implantable Body Sensor Networks (pp. 124–129). London.
Kreil, M., Sick, B., & Lukowicz, P. (2014). Dealing with human variability in motion based, wearable activity recognition. In Proceedings of the IEEE International Conference on Pervasive Computing and Communications Workshops (pp. 36–40). Budapest.
Kamal, S., & Jalal, A. (2016). A hybrid feature extraction approach for human detection, tracking and activity recognition using depth sensors. Arabian Journal for Science and Engineering, 41(3), 1043–1051.
Kamal, S., Jalal, A., & Kim, D. (2016). Depth images-based human detection, tracking and activity recognition using spatiotemporal features and modified HMM. Journal of Electrical Engineering and Technology, 11(6), 1857–1862.
Chen, D., Bharucha, A., & Wactlar, H. (2007). Intelligent video monitoring to improve safety of older persons. In Proceedings of the IEEE Conference on Engineering in Medicine and Biology Society (pp. 3814–3817). Lyon.
Jalal, A., Sharif, N., Kim, J., & Kim, T. (2013). Human activity recognition via recognized body parts of human depth silhouettes for residents monitoring services at smart homes. Indoor and Built Environment, 22, 271–279.
Jalal, A., Kamal, S., & Kim, D. (2014). Depth map-based human activity tracking and recognition using body joints features and self-organized map. In Proceedings of the IEEE Conference on Computing, Communication and Networking Technologies (pp. 1–6). China.
Sanchez, D., Tentori, M., & Favela, J. (2008). Activity recognition for the smart hospital. IEEE Intelligent Systems, 23(2), 50–57.
Farooq, A., Jalal, A., & Kamal, S. (2015). Dense RGB-D map-based human tracking and activity recognition using skin joints features and self-organizing map. KSII Transactions on Internet and Information Systems, 9(5), 1856–1869.
Jalal, A., Kamal, S., & Kim, D. (2014). A depth video sensor-based life-logging human activity recognition system for elderly care in smart indoor environments. Sensors, 14(7), 11735–11759.
Raptis, M., & Sigal, L. (2013). Poselet key-framing: A model for human activity recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 2650–2657). Oregon.
Cuntoon, N., & Chellappa, R. (2007). Epitomic representation of human activities. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 1–8). Minneapolis.
Bobick, A., & Davis, J. (2001). The recognition of human movement using temporal templates. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(3), 257–267.
Li, W., Zhang, Z., & Liu, Z. (2010). Action recognition based on a bag of 3D points. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (pp. 9–14). San Francisco.
Oreifej, O., & Liu, Z. (2013). HON4D: Histogram of oriented 4D normals for activity recognition from depth sequences. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 716–723). Oregon.
Wang, J., Liu, Z., Chorowski, J., Chen, Z., & Wu, Y. (2012). Robust 3D action recognition with random occupancy patterns. In Proceedings of the European Conference on Computer Vision (pp. 872–885). Florence.
Sung, J., Ponce, C., Selman, B., & Saxena, A. (2012). Unstructured human activity detection from RGB-D images. In Proceedings of the IEEE Conference on Robotics and Automation (pp. 842–849). Saint Paul.
Yang, X., & Tian, Y. (2012). Eigenjoints-based action recognition using naive-bayes-nearest-neighbor. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (pp. 14–19). Providence.
Wang, J., Liu, Z., Wu, Y., & Yuan, J. (2012). Mining actionlet ensemble for action recognition with depth cameras. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 1290–1297). Providence.
Jalal, A., & Kim, Y. (2014). Dense depth maps-based human pose tracking and recognition in dynamic scenes using ridge data. In Proceedings of the IEEE Conference on Advanced Video and Signal Based Surveillance (pp. 119–124). Seoul.
Jalal, A., Kamal, S., & Kim, D. (2016). Human depth sensors-based activity recognition using spatiotemporal features and hidden Markov model for smart environments. Journal of Computer Networks and Communications, 2016, 1–11.
Deepu, V., Madhvanath, S., & Ramakrishnan, A. (2004). Principal component analysis for online handwritten character recognition. In Proceedings of the IEEE Conference on Pattern Recognition (pp. 327–330). Cambridge.
Jalal, A., Kim, Y., Kamal, S., Farooq, A., & Kim, D. (2015). Human daily activity recognition with joints plus body features representation using Kinect sensor. In Proceedings of the IEEE Conference on Informatics, Electronics and Vision (pp. 1–6). Japan.
Wang, Y., Huang, K., & Tan, T. (2007). Abnormal activity recognition in office based on R transform. In Proceedings of the IEEE Conference on Image Processing (pp. 341–344). San Antonio.
Jalal, A. (2015). IM-DailyDepthActivity dataset. http://imlab.postech.ac.kr/databases.htm
Jalal, A., Kamal, S., & Kim, D. (2015). Individual detection-tracking-recognition using depth activity images. In Proceedings of the IEEE Conference on Ubiquitous Robots and Ambient Intelligence (pp. 450–455). Korea.
Muller, M., & Roder, T. (2006). Motion templates for automatic classification and retrieval of motion capture data. In Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation (pp. 137–146). Austria.
Seidenari, L., Varano, V., Berretti, S., Bimbo, A., & Pala, P. (2013). Recognizing actions from depth cameras as weakly aligned multi-part bag-of-poses. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (pp. 479–485). Portland.
Xia, L., Chen, C., & Aggarwal, J. (2012). View invariant human action recognition using histograms of 3D joints. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (pp. 20–27). Providence.
Keceli, A., & Can, A. (2014). A multimodal approach for recognizing human actions using depth information. In Proceedings of the IEEE Conference on Pattern Recognition Workshops (pp. 421–426). Stockholm.
Liu, L., & Shao, L. (2013). Learning discriminative representations from RGB-D video data. In Proceedings of the International Conference on Artificial Intelligence (pp. 1493–1500). Beijing.
Zanfir, M., Leordeanu, M., & Sminchisescu, C. (2013). The moving pose: an efficient 3D kinematics descriptor for low-latency action recognition and detection. In Proceedings of the IEEE Conference on Computer Vision (pp. 2752–2759). Sydney.
Li, Q., Zhou, Y., & Ming, A. (2014). Integrating joint and surface for human action recognition in indoor environments. In Proceedings of the IEEE Conference on Security, Pattern Analysis and Cybernetics (pp. 100–104). China.
Jalal, A., & Kamal, S. (2014). Real-time life logging via a depth silhouette-based human activity recognition system for smart home services. In Proceedings of the IEEE International Conference on Advanced Video and Signal-based Surveillance (pp. 74–80). Korea.
Yang, X., & Tian, Y. (2014). Super normal vector for activity recognition using depth sequences. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 804–811). Columbus.
Xia, L., & Aggarwal, J. (2013). Spatio-temporal depth cuboid similarity feature for activity recognition using depth camera. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 2834–2841). Portland.
Jalal, A., Kamal, S., & Kim, D. (2015). Shape and motion features approach for activity tracking and recognition from Kinect video camera. In Proceedings of the IEEE Conference on Advanced Information Networking and Applications Workshops (pp. 445–450). Korea.
Mabrouk, M., Ghanem, N., & Ismail, M. (2015). Semi-supervised learning for human activity recognition using depth cameras. In Proceedings of the IEEE Conference on Machine Learning and Applications (pp. 681–686). USA.
Slama, R., Wannous, H., & Daoudi, M. (2014). Grassmannian representation of motion depth for 3D human gesture and action recognition. In Proceedings of the International Conference on Pattern Recognition (pp. 3499–3504). Sweden.
Wang, C., Wang, Y., & Yuille, A. (2013). An approach to pose based action recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 915–922). Portland.
Gowayyed, M. A., Torki, M., Hussein, M. E., & El-Saban, M. (2013). Histogram of oriented displacements (HOD): describing trajectories of human joints for action recognition. In Proceedings of the International Conference on Artificial Intelligence (pp. 1351–1357). China.
Vemulapalli, R., Arrate, F., & Chellappa, R. (2014). Human action recognition by representing 3D skeletons as points in a Lie group. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.
Downloads
Published
-
Abstract49
-
PDF19
Submit Article
Information
