Multi-Kinect skeleton fusion for physical rehabilitation monitoring

Kinect has been increasingly applied in rehabilitation as a motion capture device. However, the inherent limitations significantly hinder its further development in this important area. Although a number of Kinect fusion approaches have been proposed, only a few of them was actually considered for rehabilitation. In this paper, we propose to fuse information from multiple Kinects to achieve this. Given the specific scenario of users suffering from limited range of movements, we propose to calibrate depth cameras in multiple Kinects with 3D positions of joints on a human body rather than in a checkerboard pattern, so that patients are able to calibrate Kinects without extra support. Kalman filter is applied for skeleton-wise Kinect fusion since skeleton data (3D positions of joints) and its derivatives are preferred by physiotherapists to evaluate the exercise performance of patients. Various preliminary experiments were conducted to illustrate the accuracy of proposed calibration and fusion approach by comparing with a commercial Vicon system®, confirming the practical use of the system in rehabilitation exercise monitoring.