Individually scaled evaluation method

In this subsection, we present candidate reaching models for describing the normal RM and explain how we derived the normal reaching model for post-stroke patients. Subsequently, based on the model, we defined the individually scaled performance index and visually mapped it for the evaluation method (Fig. 1).

Fig. 1

Development process of individually scaled evaluation method. The blocks of the development procedures are denoted with marked circles in the following sections in Methods. Once an appropriate model for a normal RM is established, A.2 and A.3 are the general steps for utilizing the proposed evaluation method

Model formulation for normal reaching movement

Reaching models quantitatively describe an individual’s unique performance, and such models use kinematic variables as the inputs and yield the movement time as the outputs. To establish an appropriate normal reaching model, we first studied the characteristics of two existing models: Fitts’ law and a reaching-related model of CP [19,20,21]. We then deduced a normal reaching model using these models.

Fitts’ law, the best-known reaching model [24], is described as follows:

$$_=a+b\times }_\left(\frac\right)$$

(1)

where TM denotes the movement time, A and W are the reaching target distance and width, respectively, and a and b are the intercept and slope of the linear model, respectively. Fitts’ law was not originally proposed to accurately capture the characteristics of RM, but instead focused on the speed–accuracy tradeoff in RM with various target sizes and lengths [24]. Hence, (1) contains only target-related environmental variables and does not include any behavioral variables (those related to an individual’s behavior) that could also affect TM. For instance, there is no variable in (1) that reflects the fluctuation of the movement speed with the same target (A and W). Notably, the target size is usually fixed in a robotic reaching training task for rehabilitation because the fixed target size is recommended by Fitts’ law [28], and the speed–accuracy tradeoff is not the main issue [4, 20, 28, 29]. This implies that (1) is insufficient to accurately model RM characteristics, and this was supported by an attempt to apply (1) to describe RM in stroke patients; the result showed poor modeling accuracy [27]. It should be noted that there is another form of Fitts’ law, including the error rate [30]. However, we did not use the form with the error rate to consider their impaired movement [20, 29, 31] because the robotic reaching tasks for post-stroke patients have been a type of errorless form [30] of reaching, such as disc or pin transfer tasks [24, 32].

To improve and maximize the reaching modeling accuracy, one study reported a sophisticated reaching model that incorporated both environmental and behavioral variables, even when considering erroneous human behaviors [26]. We refer to this model as the Almanji model, and it accounts for a reaching-related (mouse pointing) movement in CP and is expressed as follows [26]:

$$_=^\times ^\times ^\times ^^\times (NSO+1)}^\times ^$$

(2)

where AS denotes the average speed of movement; EC the erroneous clicks; NS the number of submovements [26]; NSO the number of slip-offs; CI the curvature index [26], and \(\alpha , \beta\), c, d, e and \(k\) are the model parameters. Model (2) is more appropriate than (1) in terms of the modeling accuracy. However, it still has limitations in the target movement and objective of the model. Regarding the former, the target movement of (2) was a mouse-pointing task; thus, (2) contained variables related to mouse clicks, including EC and NSO. However, our target reaching task does not require a mouse click to discriminate the success of the task, hence EC and NSO were not of interest. Hence, model (2) must be simplified as follows:

$$_=^\times ^\times ^\times ^^$$

(3)

Despite its simplicity, model (3) remains unsuitable because of the model objective of its original model (2); it does not aim to describe normal reaching, but rather RM with erroneous human behaviors. It is well known that the normal RM (when one reaches the target without any erroneous behaviors) is ideally a straight movement with minimal submovement [26], and CI and NS in (2) can be considered constants (1 for CI and 3 for NS [33]). Therefore, the simplified model (3) was further modified to derive the normal reaching model as follows:

$$_=}^\times ^\times ^$$

(4)

Equation (4) implies that TM of the normal reaching is determined by reaching the target distance (A) and average moving speed (AS). In contrast to Fitts’ law (1), the proposed model (4) includes AS. The effect of the moving speed variation during RM is considered to improve the modeling accuracy of normal RM. Notably, (4) does not include the target width (W) because of the fixed target size in the robotic reaching training task.

Evaluation method based on normal reaching model

Using the model above, we developed a novel individually scaled evaluation method for post-stroke patients; the method involves the following steps: (1) establishing an individual’s normal reaching model based on (4) of one’s less-affected arm (Fig. 2), (2) obtaining the estimated ideal movement time for normal reaching (TM,e) using the model, and (3) evaluating the reaching performance of the affected arm by comparing TM,e with the actual movement time (TM,a). When TM,a significantly deviates from TM,e, the discrepancy implies that RM is abnormal because of the presence of erroneous behavior such as anomalous CI or NS, violating the definition of (4).

Fig. 2

Example linear regression fit for a reaching model. Reaching parameters such as the movement time (\(_)\), target distance (A) and average speed (AS) were logarithmically transformed. The colored plane represents the plane of the model parameters that fit the acquired data (gray dots)

To identify an individual-specific normal reaching model using (4), we used several RM datasets of the less-affected arm in various directions and distances and conducted a multiple linear regression analysis. Here, we defined the arm ipsilateral to the lesion as the less-affected arm and the contralateral arm as the affected arm in hemiplegia. Although the less-affected arms of stroke patients are not completely free from impairment, they are expected to behave closer to the condition prior to stroke onset than the affected arm. One study reported that the movement time distributions of healthy participants’ arms and post-stroke participants’ less-affected arms were similar [21], and another study reported no significant differences in the movement time between healthy and post-stroke less-affected arms [22, 23]. Hence, the RM data of the less-affected arm could be a reliable source for predicting an individual’s normal reaching unless one suffers from diplegic symptoms.

Using an established normal reaching model, we set an individually scaled standard for normal RM by estimating the ideal movement time (TM,e) for each condition. Subsequently, the actual movement time (TM,a) of the affected arm at the corresponding movement condition was compared to TM,e by defining a performance index normalized for error as follows:

$$_(i)= \frac_(i) -_(i)}_(i)}$$

(5)

where i denotes the index of the target. As the denominator term TM,e normalizes the discrepancy, the normalized error \(_\) (5) provides an individually scaled quantity that reflects the abnormality level. As the stroke reaching procedure is affected by sensorimotor noise [34, 35], the TM,a of the stroke would increase and exhibit more discrepancies from TM,e.

Visualization of evaluation results

We constructed an evaluation map (Fig. 3) based on the acquired index \(_\) (5) throughout the workspace. A high deviation of the reaching trajectory, which is represented by a black arrow line, results in a high \(_\), which is represented in the red-colored region according to the spectrum range.

Fig. 3

Example assessment profile mapping. The color gradient represents the normalized error level defined by the proposed evaluation method. The blue and black arrows are the example trajectories of the normal and affected reaching, respectively, whereas the red and blue shades in the contour represent the affected and normal reaching performance on the reaching targets respectively

The proposed visualization method allows the evaluation of an individual’s motor characteristics and prescription of performance-based training [4, 19, 36]; thus, we may implement adaptive scheduling to prioritize reaching training and prevent overtraining of a satisfactory reaching condition, where the “labor in vain” problem could arise with random scheduling [36]. This visualization is particularly beneficial in multi-distance-directional reaching training environments. Solely mapping the reaching performance with the movement time can be problematic because a short movement time is generally preferred. Because further targets are likely to have long movement times, the training could be biased toward outer-most targets regardless of the actual reaching performance in the workspace. In contrast, the proposed visualization using the normalized index (5) can objectively portray the reaching performance globally in a spatial sense.

ExperimentsExperimental design

We conducted two experiments to validate the proposed normal reaching model and verify the feasibility of the developed evaluation method. For the former, we compared candidate reaching models including the Fitts (1), Almanji (3), and proposed (4) models by examining the fit degree of the RM data of the healthy subjects. This is because their reaching tasks could be considered normal. It should be noted that we used simplified (3) instead of (2), because our reaching task required fewer conditions than the original mouse-pointing task. For the latter, post-stroke subjects performed the reaching task with the less-affected and affected arms. We obtained each patient’s normal reaching model (4) based on the RM data of the less-affected arm and evaluated the performance of the affected arm using the evaluation map with normalized error (5). Additionally, as a pilot study, we conducted the same procedures as the latter experiment with different post-stroke participants in an actual clinical setting to further justify the proposed method in an actual clinical setting with a condensed procedure. Here, the participants performed fewer number of RM trials, which was sufficient to form reaching models and evaluate the affected reaching.

Figure 4 shows the experimental setup used for the validation experiments. All subjects sat on a trunk-constraining chair and performed a visually guided reaching task with HapticMaster (MOOG, Netherlands), a three-degrees-of-freedom (DOF) end-effector–type robot (Fig. 4a). The robot recorded the movement time, position, and velocity of its end-effector for each RM trial at a 75-Hz sampling rate. Because the subjects rested their forearm on the gimbal of the robot, supporting their arm against gravity (Fig. 4c), the reaching task was constrained to motion with 2DOF. The robot was located 30 cm between the gimbal and subject (Fig. 4a). The chair had a strap to constrain the trunk movement and prevent compensatory movements (Fig. 4d). For the reaching task, a monitor was set 3 m away from the seat (Fig. 4a), and the subjects performed start-to-target reaching according to custom-made visual guidance software developed in Visual Studio (Microsoft, USA) (Fig. 4b).

Fig. 4

Experimental setup for validation experiments. a Visually guided reaching system with an end-effector–type robot. b Sample reaching targets in eight directions. c Gimbal with gravity-compensated forearm rest. d Trunk-constraining chair

Figure 5 shows the experimental setup of the pilot study. All subjects sat on a regular chair and performed a visually guided reaching task with rebless planar® (H-Robotics, South Korea), a 2-DOF end-effector–type upper limb rehabilitation robot (Fig. 5a). The robot recorded the movement time, position, and velocity of its end-effector for each RM trial at a 30-Hz sampling rate. The subjects rested their forearm on the robot end-effector to support their arm against gravity (Fig. 5c) and performed start-to-target reaching according to the custom-made visual guidance software developed in Unity (Unity Technologies, USA) (Fig. 5b).

Fig. 5

Experimental setup for the pilot study. a End-effector type upper-limb rehabilitation robot. b Sample reaching targets in eight directions. c Forearm rest of the rehabilitation robot

Participants

For the validation experiments, twelve healthy (26.2 ± 2.7 years; two females, 10 males) and seven stroke survivors (58.4 ± 8.0 years; two females, five males) participated in this study (Table 1). All survivors had an affected arm on their right side. The inclusion criteria for stroke subjects were as follows: (1) hemiplegia; (2) unilateral stroke; (3) no signs of visual, spatial, or sensory deficits; and (4) mini-mental status examination scores greater than 24, indicating that the subjects could understand the instructions for the experiments. The exclusion criteria were as follows: (1) habitual shoulder dislocation; (2) musculoskeletal disorders; and (3) Parkinson’s disease, aphasia, apraxia, diabetes. This study was approved by the local institutional review board (DGIST-150408-HR-008- 02).

Table 1 Participants details of affected arm in validation experiments

Twelve stroke survivors (67.5 ± 6.8 years; four females, eight males) participated in this pilot study (Table 2). Six stroke survivors had an affected arm on their right side. The inclusion criteria for the stroke subjects were as follows: (1) hemiplegia, (2) upper limb modified Ashworth scale score < 3, and and (3) post-stroke age > 19 years. The exclusion criteria were as follows: (1) cognitive deficits or aphasia, (2) internal medical conditions, (3) neurological or musculoskeletal disorders, and (4) other conditions that inhibit upper-limb rehabilitation exercises. This study was approved by the institutional review board of Asan Medical Center (IRB No. 2022-0981).

Table 2 Participant details of affected arm in pilot study

It is noteworthy that the stroke survivor group was older than the healthy group while there were no inclusion/exclusion criteria for this age difference. Older adults generally have longer movement times [37, 38], more variable velocities [39] and more corrective movements [40] than young subjects.

Protocols

In the validation experiments, the subjects sat on a trunk-constraining chair to which a strap was attached (Fig. 4d). The subject’s arm was secured using a gimbal (Fig. 4c). On the monitor, a cursor representing the position of the robot was displayed at the center, which was the initial position for the center-out reaching task (Fig. 4b). For each visually guided reaching task, the subjects were asked to move the cursor on the monitor to reach the target at their preferred speed (Fig. 4b). The reaching task consisted of three blocks of 40 different movement conditions (eight directions, five target distances), resulting in 120 trials (Table 3). For each block, the target randomly appeared once at every possible location. Once the cursor remained in the target position for 1 s, the trial was considered successful, and the robot returned the subject’s arm to the initial position. The reaching completion time was not constrained, and pauses or corrective movements were allowed. However, certain reaching trials were skipped if the subjects could not reach the target. The subjects were instructed to restrain their trunk movements during the reaching task and were given a rest period between blocks. The healthy subjects performed the reaching task with their dominant arm, whereas the stroke subjects performed the task with both the less-affected and affected arms. For the pilot study, the protocols were different in the chair setting and number of reaching trials. The subjects sat on a regular chair (Fig. 5a); instead, they were asked to constrain their trunk movements as much as possible. When conducting the pilot study, we recognized that the model formulation did not require all 120 trials of the RM data. Thus, in the pilot study, the reaching task consisted of blocks of 24 different movement conditions (eight directions, three target distances), resulting in 24 trials for the less-affected arm and 48 trials for the affected arm (Table 3).

Table 3 Summary of experiment configurationData analysis

During the reaching task, we collected the movement time, position, and velocity and post-processed the collected data to obtain other kinematic variables, such AS, CI, and NS. It should be noted that we calculated NS using the optimal submovement decomposition method to avoid exaggeration [33].

In the normal reaching modeling process, we rejected the RM data with outlier CI beyond two interquartile ranges (Q1–Q3). Notably, the number of reaching trials used and the number of the outliers were 115.0 ± 3.1, 5.0 ± 3.1 for the healthy participants, 107.8 ± 13.2, 7.3 ± 3.5 for the post-stroke participants in the validation experiment, and 23.4 ± 0.8, 0.6 ± 0.8 for the pilot study, respectively. It is well-known that CI is a dominant contributor to human effects on goal-directed movements [26]. We considered that such an extremely high CI represented reaching by mistake, which would not be normal reaching.

To evaluate RM using (5) based on the discrepancy between normal and erroneous RM, an appropriate normal reaching model must have the model characteristics: the model should identify normal RM, but it should not accurately identify reaching with erroneous behaviors. Hence, we analyzed the RM data to validate the proposed model (4) using the following steps. First, we evaluated the candidate reaching models using healthy RM data to determine whether the models could predict the healthy movement time, which is identical to the movement time of normal reaching. For this, we adopted the Akaike information criterion (AIC) and coefficient of determination (R2) for each model. Along with R2, which provides a general idea of how well a model fits the data (larger indicates a better fit), AIC, which measures the predictive accuracy of the model (smaller indicates higher accuracy) [41, 42], has been used for model selection studies involving human movement [26, 43]. Using the same indices (AIC and R2), we validated the candidate models using the RM data of the less-affected and affected arms of the stroke patients to determine whether the patients’ data could be used to formulate a normal reaching model.

Moreover, using the affected RM data, we further inspected model fitting to determine whether the candidate models identified erroneous reaching because we posited that an appropriate normal reaching model would not predict erroneous reaching movement times. We first constructed reaching models with the affected RM data to predict the movement time of the affected reaching (TM,aff) and observed the relationship between the residuals (TM,a − TM,aff) and error-related parameters (NS and CI). It should be noted that varying NS and larger CI values were associated with erroneous reaching. We then calculated the mean magnitude of the residuals for each model to compare the model fitting in the erroneous reaching, where a low magnitude indicates an accurate model fitting in the affected reaching, which is not preferred for the normal reaching model.

The model parameters were estimated using linear regression with logarithmic transformation, in which the movement time was the independent variable. All regression analyses, including R2 and statistical tests for AIC (non-parametric tests: Friedman test and Wilcoxon signed-rank test), were conducted using IBM SPSS (IBM Corporation, USA). AIC was calculated using the fitlm function in MATLAB R2020b (MathWorks, USA).

After validating the proposed normal reaching model, we visualized the reaching evaluation results for stroke based on \(_\) (5) as well as \(_\) (movement time) using MATLAB in both the validation experiments and pilot study. For the former visualization with \(_\), the spectrum range of the map (blue to red) was set to the minimum \(_\) and 95th percentile value was set as the maximum. This clipped maximum prevents an extremely high \(_\) from an overly inflated axis range, which does not capture the overall reaching characteristic. We used the mean of \(_\left(i\right)\) obtained by repeated reaching tasks to ensure better reliability. Additionally, we visualized the contours based on the \(_\) with the same data used for the corresponding \(_\) maps to show the distinctiveness of the proposed index \(_\).