NrPPG-NNET: an End to End Deep Learning Approach to Remote Heart Rate Estimation Through Spatial-Temporal Representation

Abstract

This research describes the development of a deep learning approach for remote non-contact heart rate estimation.
Research question: Does video footage of a human convey enough graphic information to accurately estimate clinical parameters?
In the proposed work, we try to answer this question by building an end-to-end system that receives images (or frames) as input and produces the predicted HR estimation.
The field of non-contact HR detection is vast, therefore all previously proposed methods are investigated and analyzed. The proposed deep learning approach is called NrPPG-NNET and was developed by applying the knowledge of a previously trained Convolutional Neural Network for image recognition and retrained to resolve this new task. NrPPG-NNET T achieves competitive results in terms of bpm Mean Absolute Error and is significantly lighter than most of the previously proposed methods, moreover, it can be run in real-time on mid-range laptops (even without GPU). Thanks to its speed and ease of use, NrPPG-NNET could potentially find application in the field of human-computer interaction and clinical monitoring. In addition, during development clues that are important for future work and provide a solid foundation for further research have emerged.

Objectives

Development of a model for heart rate prediction based on artificial intelligence, taking a video as input from which frames are extracted and later pre-processed in order to obtain a proper dataset for the neural network responsible for the estimate of the heartbeat.

Research methodology

Video preprocessing for space-time mapping. Development and training of a neural network and subsequent model validation.

Conclusions

A heartbeat prediction model has been realized. The model achieves good results and is now in line with the state of the art.

Future developments

Testing different datasets and improvement of the current model.