Design and development of a Telerehabilitation platform on Human Pose Estimation by a single RGB camera

Abstract

This work mainly focuses on the development and design of a Telerehabilitation platform using of a single RGB camera. The use of such a camera allows as the delivery of as a low-cost application, as it does not require the use of additional devices. The sys-tem can be used by both the patient and the physician, indeed, a patient-side and a phy-sician-side graphical interface was designed. Through the former, the patient by log-ging in can perform the exercises of interest, and the choice falls on five different exercises. Through the second inter-face, the physician, by typing in the SSN of the patient of interest, can automatically view PDFs related to that patient. The PDFs present the patient’s biographical data, in-formation about the exercise performed, statistical data, and different graphs, thus al-lowing the physician to be able to verify any motor recovery of the subject under examination.
Artificial Intelligence techniques was used estimate the human pose. Human Pose Estimation (HPE) is an artificial intelligence technology that uses computer vision to identify and track key features of the human body from simple videos. In this work, in order to identify such key-points, the MediaPipe library was used, which enabled the identification of 33 key-points on the human skeleton. After logging in and choosing the exercise to be performed, video capture will be started by turning on the webcam. In addition to the patient and skeleton, superimposed on the patient’s body and highlighted by key points, targets will appear on the scene for the subject to try to hit. Incorporating an element of entertainment through gamification of these activities can stimulate the patient to perform the activities while playing. One of the goals set in the creation and design of the platform was to be able to create an application that could be used by individuals with different pathologies.
Therefore, it was intended to create a system that would not target a specific pathology but could, instead, cover a much wider range of pathologies. For this reason, different rehabilitation exercises were analyzed that could later be incorporated within the platform. This allowed for the definition of five different exercises, and in particular, each individual exercise is targeted to a specific part of the body and refers to different dysfunctions. During a rehabilitation session, skeleton tracking with respective key points is displayed on the patient’s body so that the system can predict an automatic comparison of patients‘ angular movements on different days. In fact, eight angles defining the amplitude of different parts of the body, achieved during exercise, and calculated for each frame, are calculated. Each of these values is automatically imported into the MySQL Database, this is because the data in MySQL are later exported for different statistical calculations to be made. In particular, the Range of Motion (ROM) is defined, which refers to the extent of movement that a joint or a combination of joints can achieve. Each calculated statistical parameter is placed within the pdf for the physician to analyze for possible motor recovery of the subject. Different users voluntarily joined the designed exercises. This allowed for a statistical analysis and a performance evaluation through which the ROM values that each user obtained in the sessions performed were determined. The evaluation of different ROM parameters, both those referring to the current session and those referring to the relationship between sessions, allowed us to analyze the skeletal movements of the users and their motor performance during the sessions. The 17 users, before doing the exercises, filled out an Excel table in which they reported the required data. This made it possible to identify five different categories on which a statistical analysis was carried out. A statistically significant difference, by p-value assessment, was shown for the category of sports and non-sports subjects who performed the various exercises. Particularly, this difference was evidenced for those exercises with the use of targets in the scene. As a result, it was possible to state that sports players have a longer reaction time and thus greater attention when hitting the targets than non-sports players. This result is important because one of the goals of adding targets is to make the subjects increase their concentration and attention; this is made possible by the presence of the targets in the scene.

Objective

Design and development of a Telerehabilitation platform on Human Pose Estimation by a single RGB camera. Generation of a platform that allows automatic comparison of angular movements performed on different days.

Research Methodology

Bibliographic and experimental research

Conclusions

This study demonstrated how the use of a Telerehabilitation application is effective in being able to determine and identify the motor progress of various subjects during rehabilitation sessions. So, it allowed us to demonstrate, how patients who could not constantly travel to specialized facilities can safely conduct such sessions from home and at the same time can be followed up by the treating physician. At the same time, such work allowed us to demonstrate how a performant system can be realized even through the use of a single camera and this by means of applied Artificial Intelligence techniques. In addition, such a platform enabled an automated comparison of the angular movements performed by subjects on different days and enable the ability to perform a visualization of results and automated statistical calculations to assist the process of analyzing and evaluating patient performance.

Future developments

Design and development of dynamic calibration and targets in the scene. Description for correct execution of exercises and quality estimation. The use, instead of 2D, of 3D pose estimation. A larger number of volunteers and a larger number of sessions performed.