This paper reviews the featured technologies developed over the recent years, focusing on music-based games using wearable devices for rehabilitation. The studies included in this review demonstrate differences in terms of the age of participants, target population, sample size, and sensor technologies employed. Given the lack of studies that complemented wearable devices with music-based games for motor rehabilitation therapies, such diversity in participant characteristics was anticipated.

Home-based rehabilitation and wearable devices

The literature review reveals that a substantial portion of these investigations have been executed with the aim of outpatient rehabilitation settings. Their explicit focus on developing and assessing rehabilitation devices to cater to home-based usage sets many of these studies apart. This shift towards a home-based context augments user engagement and commitment to the therapeutic process and transcends the spatial limitations imposed by traditional clinic-centric rehabilitation methodologies [66], thereby enhancing long-term adherence and fostering a sense of autonomy in the rehabilitation journey.

Furthermore, it is noteworthy that the rehabilitation device predominantly featured in these studies is the MusicGlove. This inclination towards MusicGlove finds resonance in its precedent evaluation within controlled laboratory settings, as documented in works such as [50, 53]. Notably, these lab-based evaluations also encompassed comprehensive assessment reports, a pivotal element that contributes to user acceptance and guides the development of the device. This meticulous approach to evaluation has carried over to the home-based studies, as exemplified by references such as [48, 49, 51], wherein no adverse effects have been reported.

Rehabilitation approaches performed by wearable devices

Sensor-based devices used in this review are produced to be compact and light, enabling users to wear them in various settings without being intrusive and allowing them to interact with the video game appropriately. As previously noted, these devices employ a passive rehabilitation approach that does not include actuators but relies solely on sensing technologies. This category includes data gloves [48,49,50,51, 53, 56, 61] and data straps [52], which don’t offer movement assistance. In other words, these devices are designed to assess user performance through sensor technologies and gauge the impact of the associated video game’s cues, feedback, and motivation. A noteworthy study of sensor-based devices is the hand-shaped keyboard developed by Mawase et al. [63]. This keyboard offers improved ergonomics by supporting the wrist and forearm, allowing users to channel force exertion more efficiently into their fingers.

The only studies that solely use a rigid robotic device are those by [57,58,59], which utilize the FINGER device. This active device aids the user by providing resistance against the flexion of non-targeted fingers. This device’s dual rehabilitation approach combines passive and active elements. It begins with the user independently performing finger flexion, a passive aspect of the process. Once the user reaches a specific flexion angle threshold, the active component is introduced, as the device provides performance-based assistance from the robot.

Soft robotic devices don’t employ traditional rigid actuators like hydraulic and pneumatic. Instead, they integrate lightweight actuators sensing technologies for movement assistance and sensor technology for movement assessment. The exoskeleton developed by [55] serves a crucial support function, specifically aiding users in executing wrist movements while maintaining stability. The study by Thielbar et al. [60] uses the Pneuglove, a pneumatic exoskeleton, and the CyberGlove, which was solely utilized for outcome measurement and not as a video game interface. The Pneuglove is an active device that prevents the flexion of a specific finger and initiates the user’s finger flexion movement independently. These two modes correspond to the video game’s two gameplay modes: the first mode prevents finger flexion for the keyboard game, while the second mode, based on Guitar Hero, assists the user in flexing their finger to strike musical notes. These functionalities enable the device to support a broader range of movements, aiding users across different game modes.

Two studies [54, 62] used a combination of sensor-based and robotic devices, offering an active rehabilitation approach by applying pressure on the targeted finger during key movements. Though they utilized both devices, they always worked together, unlike the FINGER device, which combines both active and passive approaches. The force exerted by the actuators can be finely adjusted, even down to a minimal level. In such cases, the assistance can be considered primarily passive, aligning with an advanced stage of rehabilitation where the user requires less aid.

Most of the devices focus on finger rehabilitation, with only one dedicated to wrist rehabilitation [55], and were tested exclusively on PwoND. Moreover, no device addressed the rehabilitation needs of both the fingers and the wrist. This is notable because various conditions, like Carpal Tunnel Syndrome-a prevalent neurological disorder-often involve simultaneous issues in both the fingers and the wrist. In cases like Carpal Tunnel Syndrome, pressure or constriction at the wrist affects the median nerve, extending from the forearm to the palm [67].

Motion tracking technologies for enhanced rehabilitation monitoring

In various studies, sensor technologies’ primary focus is precisely detecting in real-time user performance monitoring during movements. However, a case in point is the MusicGlove system, which, due to its reliance on contact sensing pads, imposes restrictions on the required movements for completion. This constraint results in a fixed set of ”acceptable movements,” limiting the system’s adaptability. MusicGlove provides feedback solely on movement completion, lacking real-time position data.

Various electronic sensors and systems have been employed in these studies, including accelerometers, gyroscopes, magnetometers, IMUs, FSRs, electrical leads, and potentiometers. Motion tracking sensors such as accelerometers, gyroscopes, magnetometers, and IMUs are essential in virtual reality video game studies. These sensors enable the development of 3D hand animation models and support hand motions across multiple axes, which is necessary for virtual piano applications. This approach helps create a sense of immersion and realism for the user, as recommended by [54].

Musical themes and adaptive video games

Many gamification approaches designed for a playful rehabilitation experience center around a musical theme, often involving playing the piano or catching musical notes. Most studies presented video games similar to Guitar Hero®, one of the most famous music video games. The primary benefits of employing commercial games lie in their widespread acceptance and budget-friendly costs. However, these games may need more comprehensive guidance or measurement features for monitoring the precise movement and positioning of the arms, hands, and fingers, limiting their effectiveness for therapeutic purposes. Therefore, the studies that employed similar versions of commercial video games realized modifications to the game, enabling it to adapt to user needs. In the case of selecting the piano in certain studies, it is suggested that this choice was made due to its familiarity among the general public.

It is crucial to allow the modification of video game parameters to achieve specific rehabilitation objectives. For example, Thielbar et al. [60] noted that the challenge of a task can be adjusted to match the user’s abilities in several ways, such as changing the level of assistance provided by the PneuGlove, altering the speed of key presses, and selecting specific key combinations for practice. This flexibility is essential when designing video games for long-term rehabilitation and self-administered care in home settings, especially in telemedicine or telerehabilitation scenarios, as mentioned by Merians et al. [62]. It is essential to balance providing an optimal challenge level and avoiding overly complex or too easy tasks, as suggested by Mihaly Csikszentmihalyi [68]. Merians et al. [62] quantified each patient’s thumb and finger range of motion, speed, fractionation, and strength before initiating exercises to establish an initial difficulty level. Since that study in 2002, more automated approaches have emerged, especially with the rapid growth of artificial intelligence applications in healthcare. Some studies have incorporated algorithms to automate the selection of challenge levels [54, 57,58,59].

The reviewed video games are designed to motivate users and guide the creation of either passive devices, where users independently initiate and complete movements without assistance, or active devices, which provide resistance to help guide the movement of specific body segments. The absence of devices that assist the user in initiating movement may limit the range of potential applications, as targeting neuroplasticity early in the progression of a neurological condition is crucial. Patients often require more assistance in such cases, and active devices that initiate movement can provide significant benefits.

Music therapy insights

The consistent use of TIMP across the reviewed studies indicates a preference for a method familiar to therapists and patients, as playing musical instruments is an everyday leisure activity for many. However, TIMP differs from traditional leisure activities in that it involves playing instruments in unconventional ways. Studies focusing solely on TIMP incorporated virtual piano or keyboard elements [52, 54, 56, 62, 63]. TIMP has been linked to improvements in grip strength, finger strength, and gross and fine hand motor skills [69], as demonstrated by clinical outcomes such as the Fugl-Meyer test and Wolf Motor Function test. These functional gains underscore the efficacy of TIMP in rehabilitative settings.

Additionally, some studies incorporated RAS, which uses auditory rhythmic cues such as repetitive pulses or metrically accentuated music [70]. In video games similar to Guitar Hero, metrically accentuated music leverages specific songs and increases the tempo to heighten the challenge, encouraging users to adapt to the quicker pace. Sun et al. [61] is the only study that combined RAS and TIMP. In this study, a metronome (repetitive pulses) served as a cue to guide users in performing specific gestures, aiding in the timing of piano key pressing.

However, it’s noteworthy that only a few studies evaluate the impact of music therapy with and without incorporating video games. This evaluation extends to understanding the effects of both visual and auditory cues on users. Considering the potential impact of music therapy, it becomes crucial for a more comprehensive assessment in future studies. It would be particularly interesting for devices that offer movement assistance, such as rigid robotics, to incorporate evaluations of the impact of music therapy. Quantifying the influence of visual and auditory cues in conjunction with movement assistance could provide valuable insights into optimizing rehabilitation outcomes. The study by English et al. strongly emphasized the impact of auditory cues, examining user performance with both visual and auditory cues compared to using visual cues alone. The study concluded that combining audiovisual cues enables participants to learn timing tasks more efficiently than relying solely on visual cues. The authors also highlight the importance of choosing the appropriate cues based on the specific skills taught and the performance metrics used to measure these skills.

PSE sessions are designed to engage patients dynamically and interactively. In a PSE scenario, patients might be prompted to perform specific hand and arm movements in response to a rhythmic auditory cue, such as a steady drumbeat or a melodic sequence. This auditory guidance is a precise and structured framework, enhancing the synchronization and coordination of the patient’s movements. The rhythmic precision of the auditory cues not only aids in motor skill development but also introduces an element of enjoyment and engagement, transforming the rehabilitation process into a more immersive and stimulating experience [35].

Despite its potential benefits, the adoption of PSE in rehabilitation processes is currently limited, primarily due to the intricate nature of the approach. The practical implementation of PSE requires the expertise of a skilled Music Therapist [34] who can tailor the exercises to each patient’s individual needs and capabilities. The scarcity of PSE in current rehabilitation practices underscores the importance of specialized practitioners in unlocking the full potential of this approach. As the field progresses, further research and exploration into applying PSE in diverse rehabilitation contexts could reveal novel treatment strategies for enhancing the effectiveness of hand therapeutic interventions.

Limitations

The search strategy was meticulously designed to encompass specific terms such as game, interface, simulation, and virtual environment. Rather than opting for the more general term ”video game,” we chose a more refined approach to capture exclusively articles that extensively discuss video games, filtering out other types of gamification methods. Although the narrative approach aligns well with the diverse descriptions of game design in the articles, it leans towards interpretation rather than strictly adhering to systematic or scoping review methodologies. The assessment of each wearable device’s effectiveness in tandem with music-based gamification approaches required nuanced interpretation, as only overarching findings were provided. Therefore, readers should look over the review’s findings with an awareness of its interpretation. Nevertheless, the narrative approach offers flexibility in contextualizing the unique features of wearable devices and the analysis of music therapy.