We wish to complement the existing physical rehabilitation process by developing interactive serious games. For this we use Microsoft Kinect, the motion sensing input device through which we gamify motor rehabilitation exercise routines. The cognitive and motor activity required in these 3D-full body motion sensing games will make the entire process engaging. Combining this with performance and result based feedback will eliminate the need of routine monitoring or training. This brings the rehabilitation process out of the confines of hospitals and a patient may use it in 'whenever-wherever' way. The exercises implemented in the application have been recommended by specialists.

Motion-based and virtual reality game systems have been used for rehabilitation earlier and are still in use. However they require lot of training on the patient's part. Such devices can be used only under the supervision of a specialist and have to be handled with care. Most devices, when being used, require sensors to be attached to the patient and this hinders movement. This is a huge drawback in cases where the patient has limited limb movement and as a result of the wires and sensors attached to his body, feels bound. All this rules out the possibility of the patient doing the exercises at home with his/her family and thus the patient has to half-heartedly go to the hospital on a routine basis.
This calls for a system which tracks impaired limb movement without attaching any sensors which would inhibit movement or require setup by the patient. In addition to this, the system should be easy to use and engage its user so that he can monitor his progress and thus be motivated towards recovery.
Motion sensing games built with using Microsoft Kinect offer this very possibility where the 'user is the controller'. With the advent of Microsoft Kinect, the concept of motion sensing spurred the development of various applications including those with NUIs. This opened a plethora of opportunities to create full body tracking games without the use of any external device attached to the player. Games are being created that any not only for the sole purpose of entertainment but also have a humanitarian approach. Kinect motion controlled games are being used for medical rehabilitation at some places but they lack essential components of conventional therapy.
This idea inspired us to create serious games which are engaging and motivating with added therapeutic value.

Depiction of a physically challenged person performing the exercise routine

Depiction of a Paraplegic patient playing the games
The software has a series of mini-games and each falls in the genre of Serious Games. Current rehabilitation games lack entertainment and engaging qualities whereas popular games lack essential components of rehabilitation. Mixing useful elements of both, serious games can be made, which create meaningful play. Similarly, our mini-games are gamified versions of conventional rehabilitation exercises that physiotherapists and occupational therapist make their patients do. The mini-games target specific limb movement problems that are common to a large group of people. Our software animates the in-game avatar of a person using real time motion capture data. Full skeleton of the user is tracked and the functions he performs are simulated in the virtual environment. Such an interactive environment encourages the person to become more involved, motivated and immersed in rehabilitation. These types of virtual environments not only have the components which are not found in conventional therapy but also provide easy, less dangerous, more fun and customizable therapy routines which can be tailored to the patients' interests and physical abilities.
Another important aspect of our software is that it provides feedback to the patient on his performance on the run as well as at the successful completion of the exercise routine. As already mentioned above, feedback enables a player to measure his progress in achieving goals, or progression of his skills over a period of time and this gives him a sense of accomplishment.
As the patient performs an exercise routine, various aspects about his performance are logged. This data helps in comparing his current performance against earlier attempts and in turn motivates him to perform better in the subsequent sessions. This data gets periodically updated on a website and the supervising doctor can view any particular patient's statistics remotely. This allows for efficient progress tracking of the patient over time which in turn helps to take informed clinical decision.

A schematic of the architecture of KinectoTherapy

The software has been created using Unity3d Game Engine.
The 3d animation and models are created using 3ds Max.
We are using the Microsoft's Kinect SDK to interact with the Kinect Sensor.

The software makes extensive uses of following NUI features:
  • Full body skeleton tracking: Movement of the entire body is detected and used in animating the avatar in gameplay.
  • Top down user map: Location of the user in the X-Y plane is tracked.
  • Hand Tracking: Movement of the wrist alone is tracked so that patients who are unable to stand or are paraplegic can also be benefited.
  • Gesture recognition: The user can perform simple gestures like 'push-in/push-out' to make selections.
We intend to create a simple and easy mechanism through which the doctor can supervise his patient's performance remotely.
Just follow these simple steps to configure KinectoTherapy:

  1. Install the Official Kinect Drivers.
  2. Plugin your Kinect Sensor, go to the games page and start playing!
Yash is alumni of DAIICT who completed his B-tech in Information and Communication Technology.
His contribution to this project is in:
  • Setting up the Kinect-Unity framework
  • Development in C# and Javascript
  • 2D and 3D graphics modelling and animation
  • Usability testing
  • Video editing and documentation
  • Creating this website

Sonali Dubey

Sonali is alumni of DAIICT who completed his B-tech in Information and Communication Technology.
Her contribution to this project is in:
  • Development in C# and Javascript
  • Usability testing
  • Interviewing Physiotherapists and other Specialtists.
  • Video editing and documentation
Dr. Anil Roy is faculty at DAIICT. His current research interest involves Applications of Image Processing, Fiber Optics and Optical Communication, High speed Semiconductor Devices, Nanoscience and Nanotechnology, Quantum Optics, Technologies for Humanitarian Challenges.
He is acting as a Mentor for this project and his guidance and constant motivation helps us going further.