Are Commercially Available Hand Tracking Devices Inclusive?
In this project, I set to assess how well a commercially available device such as LEAP is able to perform hand tracking of individuals with upper-body disabilities.
It is no secret that any AI model is only as good as the data it has been trained on. So if the data you train on is only representative of a small subset of the population, the model will be biased towards that subset.
Everyday technologies that rely on human input are oftentimes designed with an unimpaired user in mind. Such design parameters often yield to devices and tech being inaccessible by individuals with disabilities although they could have a potential to improve the standard of living for such users.
LEAP for Virtual Hand Therapy
Over the last decade that LEAP has existed on the market as an affordable hand-tracking device, a variety of research groups explored its applicability in the rehabilitation field - specifically, as a tool for virtual hand therapy. With an affordable price (between $100-250) and a simple plug-and-play setup, it showed the potential to be added to rehabilitation routine and make therapy engaging and entertaining for the user.
However, to be used for rehabilitation purposes, it needs to show high accuracy in performing hand tracking. Up to this day, the quality of this feature has not been assessed with individuals with disabilities.
Ground truth: we employed a motion capture system (Optitrack) with 14 cameras.
Assessment: Ultraleap Stereo IR170 system was used as a representation of a commercially available hand tracking device
Movements: participants performed a variety of finger and wrist movements as shown below.
Participants: individuals with a disability that affects their hand function (e.g., spinal cord injury, Parkinson disease, muscular atrophy, etc.) as well as unimpaired individuals.
Functional assessment: Jebsen Taylor Hand Function Test was performed to quantify their hand functionality.
We are currently in the process of analyzing the collected data.
Performing this assessment will help us understand the strengths and downfalls of the LEAP hand-tracking technology as a potential tool for rehabilitation employment.
Next, we will work on employing innovations in image detection AI and the infrared image data that we collected during the experiment to improve hand tracking of individuals with various upper-limb disabilities.