You are here

FAU Collections » FAU Research Repository » FAU College Collections

Feeling the beat: a smart hand exoskeleton for learning to play musical instruments

Download pdf | Full Screen View

Date Issued:
2023
Summary:
Individuals who have suffered neurotrauma like a stroke or brachial plexus injury often experience reduced limb functionality. Soft robotic exoskeletons have been successful in assisting rehabilitative treatment and improving activities of daily life but restoring dexterity for tasks such as playing musical instruments has proven challenging. This research presents a soft robotic hand exoskeleton coupled with machine learning algorithms to aid in relearning how to play the piano by ‘feeling’ the difference between correct and incorrect versions of the same song. The exoskeleton features piezoresistive sensor arrays with 16 taxels integrated into each fingertip. The hand exoskeleton was created as a single unit, with polyvinyl acid (PVA) used as a stent and later dissolved to construct the internal pressure chambers for the five individually actuated digits. Ten variations of a song were produced, one that was correct and nine containing rhythmic errors. To classify these song variations, Random Forest (RF), K-Nearest Neighbor (KNN), and Artificial Neural Network (ANN) algorithms were trained with data from the 80 taxels combined from the tactile sensors in the fingertips. Feeling the differences between correct and incorrect versions of the song was done with the exoskeleton independently and while the exoskeleton was worn by a person. Results demonstrated that the ANN algorithm had the highest classification accuracy of 97.13% ± 2.00% with the human subject and 94.60% ± 1.26% without. These findings highlight the potential of the smart exoskeleton to aid disabled individuals in relearning dexterous tasks like playing musical instruments.
Title: Feeling the beat: a smart hand exoskeleton for learning to play musical instruments.
33 views
15 downloads
Name(s): Maohua Lin
Rudy Paul
Moaed Abd
James Jones
Darryl Dieujuste
Harvey Chim
Erik D. Engeberg
Type of Resource: text
Genre: Article
Date Issued: 2023
Publisher: Frontiers Media
Physical Form: pdf
Extent: 11 p.
Language(s): English
Summary: Individuals who have suffered neurotrauma like a stroke or brachial plexus injury often experience reduced limb functionality. Soft robotic exoskeletons have been successful in assisting rehabilitative treatment and improving activities of daily life but restoring dexterity for tasks such as playing musical instruments has proven challenging. This research presents a soft robotic hand exoskeleton coupled with machine learning algorithms to aid in relearning how to play the piano by ‘feeling’ the difference between correct and incorrect versions of the same song. The exoskeleton features piezoresistive sensor arrays with 16 taxels integrated into each fingertip. The hand exoskeleton was created as a single unit, with polyvinyl acid (PVA) used as a stent and later dissolved to construct the internal pressure chambers for the five individually actuated digits. Ten variations of a song were produced, one that was correct and nine containing rhythmic errors. To classify these song variations, Random Forest (RF), K-Nearest Neighbor (KNN), and Artificial Neural Network (ANN) algorithms were trained with data from the 80 taxels combined from the tactile sensors in the fingertips. Feeling the differences between correct and incorrect versions of the song was done with the exoskeleton independently and while the exoskeleton was worn by a person. Results demonstrated that the ANN algorithm had the highest classification accuracy of 97.13% ± 2.00% with the human subject and 94.60% ± 1.26% without. These findings highlight the potential of the smart exoskeleton to aid disabled individuals in relearning dexterous tasks like playing musical instruments.
Identifier: FAUIR000534 (IID)
Persistent Link to This Record: http://purl.flvc.org/fau/fd/FAUIR000534
Host Institution: FAU

In Collections