Study 1: observations

Field notes from the Fly-on-the-wall and Shadowing activities, together with learnings from the related literature [5, 62], resulted in a Journey map, which is a graphic representation of an individual’s process to accomplish a goal [63]. Our map (Fig. 4) illustrates the recovery process of an inpatient stroke survivor at a rehabilitation center. The different phases describe the environment and the interactions among therapists, family members, and patients that allow for accomplishing specific goals (e.g., collecting patient data) and the provision of tailored treatment (e.g., modifying the physical environment to meet the patient’s capabilities). Once a phase is completed, patients move to the next phase. We identified five treatment phases (Fig. 4):

Fig. 4

The inpatient Journey map, including the five treatment phases: (1) Screening, (2) Planning, (3) Training, (4) Reflecting, and (5) Discharging. Illustrations used in this infographic, including 'people-characters,' 'furniture,' and 'gym equipment,' were downloaded from Freepik.com [91]. The first author designed the frames for the five steps and combined and edited the illustrations accordingly to show the many patient-therapist interactions [92]

Screening: During this phase, patient data (e.g., demographics, social, and clinical data) are collected in preparation for further clinical visits. To do so, neurorehabilitation experts use specific assessment tools, such as the Fugl-Meyer Assessment [64] for assessing patients’ motor function, and the Montreal Cognitive Assessment (MoCA) for assessing their cognitive function [65].

Planning: This phase involves collaboration among neurorehabilitation experts, patients, and families to define short- and long-term goals related to daily activities. Screening and planning phases are repeated after each team meeting or multidisciplinary consultation [66]. At these consultations, a team of experts who can help patients meet their specific needs during rehabilitation is involved. Team composition (e.g., physical, occupational, speech therapists, psychologists, family members) mainly depends on the patient’s clinical needs. For instance, patients suffering from visuospatial inattention (i.e., neglect) and those suffering from language impairments (i.e., aphasia) will need the support of two different specialists, either an occupational or a speech therapist, to train their lost functions. Teams meet regularly to evaluate patients’ progress and adjust therapy accordingly. Each therapist reports their evaluations and opinions on a patient’s progress to the team and proposes adjustments to the treatment plan. In consultation with physiatrists and neuropsychologists, the therapy is adjusted, and therapists help patients set more realistic recovery goals.

Training: This phase focuses on individual or group sessions where patients train their paretic limbs and cognitive functions under therapists’ guidance. Activities such as preparing a meal or dressing are deconstructed into small tasks, allowing patients to reconstruct a lost ability. During this phase, the environment, tasks, activities, and therapist’s attitudes are adjusted based on the decisions of the neurorehabilitation team. Assessment tools, and training tools—such as hand exercise balls, mirror therapy boxes [67], and rehabilitation technology [68]—are used to track and improve therapy outcomes. Moreover, therapists emotionally support patients to ensure they maintain high motivation.

Reflecting: This phase involves team members sitting with patients to reflect on their performances and set new goals (re-Planning). This allows patients to become aware of their motor-cognitive capabilities and accept their limitations.

Discharging: This final phase marks the end of inpatient care and usually the beginning of outpatient rehabilitation, with a complete treatment process following. Outpatient care focuses on continued recovery and aims to maximize independence over time [69]. This therapy does not require a hospital stay but is often provided through a clinic, day hospital, or at a patient’s home [5]. The same team of experts who planned the inpatient treatment determines if an individual can begin with outpatient rehabilitation: They must establish if a patient has improved their physical ability and can transfer the skills learned during inpatient care to their home environment.

Overall, our observations allowed us to better empathize with patients and neurorehabilitation experts and understand how treatment is tailored to each individual patient’s specific needs, goals, and capabilities.

Study 2: semi-structured interviews

The thematic analysis [56] focused on the research question: What strategies do therapists adopt to create suitable training environments for patients? After transcribing and familiarizing ourselves with the semi-structured interviews, we generated 151 codes representing the meanings and patterns discovered in the data. We then organized these codes into six groups based on shared concepts. Each group describes the quality of the training environments: Specific, Meaningful, Versatile, Educational, Safe, and Supportive. In the following paragraphs, we describe the six groups with examples of supporting participant statements. We value using inclusive language. Yet, the use of the pronouns “he” and “she” was not modified from the participants’ statements, as we also value reporting verbatim transcripts.

Specific

The theme Specific refers to the patient’s characteristics (e.g., clinical needs, personal goals, social circumstances, personality) that therapists consider when preparing the training environment. Therapists need to know as much as possible about their patients to create training conditions that help them function properly in their environment. This requires seeing patients as a whole to get a clear understanding of how their acquired brain injury influences (physically, behaviorally, cognitively, and emotionally) their functioning and behavior in daily life, and organizing the training environment so that it can meet the set requirements:

“There are a lot of things I want to know about the patient. I want to know what someone did before the stroke.. what was his way of life before?.. what kind of job did you do? How does her social life was?.. I have to make clear how someone is doing right now like: can you be independent in movements?.. What’s the level of performance right now?” (P05)

“.. every patient is unique.. everyone has different problems, different goals; every person has different needs or different questions.. every problem, every disability is different; there are a variety of ways to do exercises.” (P09)

Additionally, the qualities of a specific training environment can vary even between patients who want to practice similar daily activities, e.g., depending on their social background and social support:

“.. patients that come in here that have less socio-economic status or are not as intelligent, they complete less complex tasks in the daily living.. [instead] if you’re high-functioning executive, you [might] need to be able to organize a team.. if you.. only need to hand out newspapers in the morning, it uses a whole other base of functions..” (P01)

“.. if you have family support or not.. if you have children and at what age.. they can affect you in your daily life.. maybe one patient.. has more risk factors to develop some kinds of cognitive or physical situations.. because they are more likely to develop a fear of [to perform] the movement.. or.. depression..” (P08)

Meaningful

Meaningful refers to real-life conditions. Training environments should not be decontextualized, as this could demotivate patients; instead, they should feel naturalistic and realistic to patients. Training environments should be functional and meaningful, imposing challenges coherent with what patients may encounter once discharged.

“.. [if] I'm not living here in the city center, so I'm just from a small village, so my life is in a small village, meaning that it is less crowded, so why do I need all the information about this city?..” (P03)

“.. if.. his daily job was cleaning, and that's his main job; then, I know that if we're going to do something like finance, he may not do that very well because he didn't have to do it before..” (P05)

“.. the functionality of the activity depends on the relevance of the activity for you.. maybe the activity is the same.. [e.g.] I want to [learn how to] reach my mobile phone because I’m an influencer, but you are a writer–so, you need to reach the pen.. the movement in your body is the same.. but the environment.. is different!” (P08)

Yet, some patients may struggle with transferring the skills learned in training environments to their own home environments. These patients may not be able to imagine how to execute a task in an environment different from the training one:

“.. some patients can only make coffee at home. If they are over here [rehabilitation center] or maybe in another kitchen, they can’t find the thing.. they can’t imagine the solutions to different environments..” (P05)

“.. we are not always having the possibility to go home and practice.. you want to give them more views of what can happen and how they can react.. (but) they can’t imagine..” (P06)

Versatile

Versatile relates to the different stimuli that can be manipulated in the training environment to facilitate functional task execution (e.g., secondary tasks, sounds, group or individual sessions, and the distance of people and objects from the patient). The manipulation of the characteristics of a training environment may influence patients’ attention and the risks of making mistakes:

“.. I let her cook in her own kitchen.. and I let someone call.. that [became] a very mistake.. she was just talking about everything, and it became black in the kitchen–she forgot about there was something on the fire..” (P06)

Yet, environments should offer patients the possibility to train under different conditions that can be manipulated, such as practicing alone instead of in a group, filtering out distractions, including secondary tasks, and auditory and/or visual distractions:

“..[here] it’s silence, and there is no distraction, so, I can focus and I can read your paper.. when.. there are other people walking outside there, I get a phone call, or there’s someone who’s knocking on the door, and my attention is like everywhere, so there is so much more interaction in daily living than here in this room..” (P03)

“.. when you wear sunglasses, they get less overstimulation and less distraction because it filters away impulses [stimuli] from outside.. some patients also wear headphones when they want to lower the noises around them, when they go to the store and so they can better cook and do their stuff. Then, when in a crowded room, they shut down..” (P11)

While too many stimuli can become overwhelming, sometimes additional stimuli can further engage the patient and facilitate their adaptation to the training environment:

Educational

Educational relates to patients being educated and made aware of their capabilities. In training environments, patients should be allowed to explore the environment and make mistakes to learn from them. Thus, all elements in a training environment should be accessible for interaction. Training environments should promote understanding to help patients gain insight into their functioning.

“Just let them make a big mistake and try to let them see what happens and why this can be dangerous in the future in their own home when there’s no one around!” (P06)

“.. if you don’t know about something, you are going to imagine the worst situation.. when they have information about what happens to them, why they have these symptoms now, and what’s the natural prognostic to these kinds of symptoms—they understand it and then their mindset changes a lot..” (P08)

“.. if you write down in words what you are saying.. they can hear and read the information that they need to understand.. if they want to tell something.. they can point at it..” (P09)

Importantly, training environments should encourage patients to self-reflect on their actions and express themselves, thus promoting awareness and acceptance of their new condition:

“.. to help him in the rehabilitation and learning process.. we try to help the patient to get insight into its functioning to recover better..” (P04)

“.. there are also a lot of people who.. think they can do everything, and they don’t need help.. you have to let them see the change they have [endured]..” (P06)

“.. you have to teach them that it’s ok to have another life.. they have to learn that the life that they have now is also ok..” (P08)

Yet, to promote learning, training environments should also promote repeating the same task and following a given order.

“.. you have to make sure that all the steps are done in the same order.. and make sure they are doing it in automatic pilot and for as long as possible.. If they don’t know.. I do all the steps myself.. and I show them, and then you can slowly let them do it..themselves.” (P10)

“.. you have to make sure that all the steps are done in the same order.. and make sure they are doing it in automatic pilot and for as long as possible.. If they don’t know.. I do all the steps myself.. and I show them, and then you can slowly let them do it..themselves.” (P10)

Safe

Safe refers to training in an environment where patients can make mistakes without the risk of injury or experiencing unnecessary stress:

“.. we have to make sure that someone can try and provide a safe space to learn.” (P04)

“.. [if] I'm going to do a task in the kitchen that isn't that complicated.. and if I see that it's very complicated for the person, I know that I have to make it easier..” (P05)

To protect patients from physical harm and psychological discomfort, training environments should help balance activity levels to ensure rest periods, thus preventing overexertion or fatigue:

“.. if we notice that someone gets tired.. they are closing their eyes.. blink a lot.. start to shut themselves off.. we make sure that these tests are done at a different time..” (P01)

“.. you have to make sure that he takes a rest at the right time, or [if] he has some problems with focusing.. he has to be somewhere in a quiet space..” (P05)

Safe also refers to a training environment characterized by trust and empathy, facilitating clear and effective communication between therapists and patients who feel comfortable openly sharing their feelings:

“.. let people trust you to share the information and focus on how it was and where you are now.. we have to give them perspective, and you won’t give them.. false hope..” (P03)

“.. it is not only doing exercise but also just to talk about how they feel, how they are doing and if they are ok.. you create a bond—a connection with the patient..” (P09)

Supportive

Supportive relates to motivation and the physical and verbal support therapists give to patients.

“.. we’re looking for the motivation which is in the patient to get it out and work with.. from the motivation, go further and further to stimulate the patient even when he’s sad or anxious.” (P04)

“.. the key.. is to keep them motivated.. you have to know very good your patient, to know what motivates them.. because sometimes they want to give up..” (P08)

To promote motivation, training environments should act as intersections where patients are engaged in meaningful social interactions that encourage collaboration, peer support, and training of the desired skills:

“.. if they want to go home and there are some skills they have to learn to go home, then they are extra motivated..” (P05)

“.. if the patients really would like to go to the markets, you want to give him a possibility to exercise in conversation on the markets..” (P09)

Additionally, how support is provided to patients is important. This includes the provision of positive feedback, the ways of reporting a message (e.g., directly or indirectly), the use of alternative means to provide instructions (e.g., verbal, written instructions, or physical interactions to assist movements), and the distance of people or things from the patient (e.g., sitting next to patients, standing far from them).

“.. you have to find out what their expectations are and how they interact with people, I connect to that and that is also changing my language.. if you have a higher education, these people usually are more problem-solving kind of types: they need information, they understand.. other people get confused by the information.. the communication aspects in the environment do predict or do provoke behavioral outcomes..” (P02)

“.. when they are having aphasia, you make sure you talk slower, make shorter sentences, and ask closed questions where they can answer yes or no..” (P10)

“.. the positive feedback of the team, it helps them a lot.. to function, to regain.. recovery.. and working with the plan and with the timetables..” (P11)

“.. try to stand next to the other, and just do it together and find out together what's the best way to go..” (P06)

We used the quotations in each group to formulate twenty-two statements that would answer our research question. These statements, which report the strategies therapists seem to adopt in conventional neurorehabilitation to create suitable training environments, were subsequently incorporated into an online questionnaire in Study 3.

Study 3: online questionnaire

Table 1 summarizes the mean ratings for the 22 questionnaire items and their corresponding themes. For each statement, participants could comment on how they use it in their daily activities; below are some transcripts of participants’ comments left online. Overall, the neurorehabilitation experts strongly agreed that the training environment should be adjusted to the cognitive abilities and social environment of the patient (Items 1 & 4 in Table 1).

Table 1 Therapists’ agreement with statements from 1 (strongly agree) to 5 (strongly disagree)

Additionally, choosing between error-free learning or learning from mistakes (Item 5) and modulating the realism or complexity/distractions of the task (e.g., Items 2, 12 & 14) were generally agreed-upon approaches. However, in some cases, the manipulation of the level of complexity of the environment should be enforced due to patients’ disorders, as commented on by some respondents:

“.. it is certainly important that you make the conditions as pleasant as possible for the patient.. lights off, blinds closed, removing ambient noise.. as soon as the patient can tolerate this better, slowly increase/expand the stimuli” (Item 2—P02);

“If the patient experiences problems with communication in background noise, exercises are offered in which the patient learns to deal with these stimuli, i.e., participation in a group or radio on during an exercise, ask colleague to disturb during a treatment” (Item 12—P11)

Motivational techniques (Item 16), co-deciding with the patient, and involving the family (Items 3 & 19) were also regarded as relevant:

“Shared decision-making with patients: very important to find out what their end goals look like and how you can imitate them—or build them up in steps” (Item 3—P24)

“It is important to involve family members in the rehabilitation so that the patient can put what they have learned into practice at home” (Item 19—P09)

Relatively low ratings, but still on the ‘agree’ side, were provided for more specific educational techniques, including the use of mirrors (Item 6), video (Item 7), and the constraining of movements (Items 15 & 22). According to participants, their use is not ordinary but depends on the therapy goals, patients’ motor abilities, and behavior.

“Depending on whether this is helpful…Can be like that in certain situations, this doesn't seem like a 'must' to me.” (Item 7—P17)

“If the patient is inclined to let less affected arm things take over, immobilize them by, e.g., slinging” (Item 22—P05)

Study 4: participatory design workshop

While the previous sections focused on conventional non-VR-based sensorimotor neurorehabilitation, the present section reports findings from the focus group and the ideation session that describe neurorehabilitation experts' opinions and ideas on IVR environments.

Study 4—activity 1: focus group

The paper sheets reporting participants’ ideas can be found in Additional file 3.

Q1. What do you think about and what are the advantages and disadvantages of IVR?

Participants indicated that IVR would require fewer therapists to train the same number of patients. They also noted the versatility of IVR, including having many exercises built-in to train for different impairments, use at home, and creating targeted challenges adjusted to meet the patient’s specific clinical needs and responses.

“.. you can also cover different disciplines, different areas. You only need one tool..” (Group B)

“.. you do traffic training with a patient.. you want to see if they pay attention to the left side.. you think: maybe I should have had a few cars and cyclists coming from the left side, you can simply build them in the VR.. targeted challenges for practice..” (Group A)

“.. you don’t have to use it here, but maybe at home” (Group B)

When discussing the potential disadvantages of IVR, participants remarked that this is a costly technology and not everyone might benefit from it. Some patients might be unable to handle it cognitively or understand how it works. They also expressed their concerns about safety and excessive use. Some patients might feel unwell while using IVR (i.e., cybersickness), while others, due to excessive use, might have difficulties with transferring the skills learned in the virtual world to the real one.

Q2. How would you use IVR in rehabilitation?

Participants recommended that therapy in IVR should be targeted to the patient’s clinical impairments. They also discussed the importance of providing training within diverse real-world inspired scenarios, particularly those outside rehabilitation centers.

Their suggestion of taking pictures of patients’ homes further underlined the idea of creating virtual experiences that closely mimic real-world environments. Additionally, participants expressed interest in using IVR to simulate home visits, allowing therapists to virtually enter patients’ homes to provide remote assistance and guidance.

“.. the ideal thing would be to put on the device, walk into someone’s home, and then help them” (Group B)

Finally, they valued IVR for its potential as a tool for research, exploring related possibilities, such as eye tracking—even though eye-tracking is not directly linked to IVR and is only available in some HMDs—, and to facilitate learning by exploring, e.g., visual feedback.

“.. what does someone do with their peripheral vision, you can measure all of that in VR with eye tracking.. you can potentially use that in your visual research..” (Group A)

“.. you could apply it to different cognitive domains.. if you do something wrong, a signal comes up.. if you put your hands in the wrong position, you get a.. buzzer alarm.. or you get a red cross or.. just won’t move forward.” (Group A)

Q3. What should researchers consider when developing IVR experiences for brain-injured patients?

Overall, participants recommended that developers create engaging experiences that can be adjusted to the patient’s physical and cognitive capabilities, and age.

“Fun interactive exercises.. enjoyable and challenging.. exercises that change every time” (Group A)

“.. one person might have a lot of cognitive problems in the beginning, but you can also create a certain progression or level of difficulty.” (Group A)

“.. you should be able to adjust it so that you can put it on and operate it with one hand.” (Group B)

“Suitable for each person.. so that you don’t only have super modern things that young people have an affinity with, but also for seventy-year-olds.” (Group A)

Concerns were also raised around safety, highlighting the need to develop experiences that limit the risks of getting physically injured or fatigued during use.

“.. somehow measure whether it becomes too much for the patient.. monitoring fatigue or something, or attention going down.” (Group A)

“.. if you have real physical problems, and you’re in such a world and you think: oh! I’ll stand up! There should be.. limitations because it’s about what someone can’t do..” (Group B)

Further discussion was on the modalities of how to guide or instruct patients during use:

“Demonstrations, show example video, instructional video, little text, simple language, short sentences.” (Group A)

“Demonstrations, show example video, instructional video, little text, simple language, short sentences.” (Group A)

Study 4—activity 2: ideation session

The paper sheets reporting participants’ ideas can be found in Additional file 3. Groups A and B produced similar results regardless of the persona assigned, probably because they were tasked with designing both low and high-cognitively demanding virtual environments.

Low-cognitively demanding virtual environments

In low-cognitively demanding virtual environments, the priority is ensuring the user performs the given task. For patients with reduced cognitive abilities, it would be beneficial to train in static and empty environments (Group A) and simplified environments with none or minimal distractions (Group B), as depicted in the generated IVR environment by Group B—Fig. 5A.

“.. the first stage could be where there’s nothing happening around you as you move..” (Group A)

“.. you want to get all the distractions away. You don’t want to place him in front of a window because he can see all the movements outside. You don’t want a lot of people to walk by, that’s why you want to face him to the wall or the screen.” (Group B)

Fig. 5

Screenshot from a video recording of one of the monitors used to display to the participants what the user saw in the HMD. These are examples of: A. a low-cognitively demanding environment with no distractions; and B. a high-cognitively demanding training environment , co-created by Group B in collaboration with the VR developer

Both groups also raised concerns about the effects of bright colors and excessive noise on patients, as too much noise or light can affect attention (see an example of different lighting generated by the participants in Fig. 6). For instance, while Group A suggested using calming colors and sounds from nature and classical music, Group B recommended neutral colors and music familiar to the patient.

“Loud music sounds, voices, changes in sounds.. the low demanding side is nature sounds and classical music.” (Group A)

“.. sometimes we try to add music or things that are important to people so they are more focused on the task.. music that they prefer, to make them more alert.. the music to help patients concentrate.” (Group B)

Fig. 6

Screenshots from a video recording of one of the monitors used to display to the participants what the user saw in the HMD. These are two examples of the same virtual environment where light intensity and temperature were manipulated to: A. decrease or B. increase the intensity of sensory input, as suggested by Group A to the VR developer

This discussion also led to the benefits that some patients could experience with increasing and decreasing the number of stimuli. In particular, Group A suggested that for some patients, such as those suffering from spatial neglect, showing only one side of the virtual environment could help train the affected side, as the other half of the environment would be hidden. This is presented in the generated IVR environment shown in Fig. 7.

“.. a wall can be very useful because very often people with hemianopsia or neglect drive towards something and then there’s a wall. They can’t get past it. And if you can create more stimuli there, or trigger them that when there’s a wall and I notice I can't go further, they are challenged to go the other way.” (Group A)

“.. you want all the stimuli to come from one side so that people have to pay attention to that side, so they have to look in a certain direction” (Group A)

Fig. 7

Screenshots from a video recording of one of the monitors used to display to the participants what the user saw in the HMD. The figures illustrate the concept of a movable virtual wall within the virtual environment, designed to manipulate the amount and direction of stimuli. Participants suggested that this strategy could be beneficial for patients with neglect, as it would facilitate training of the affected side by forcing attention to the only presented objects. This concept was co-created by Group A in collaboration with the VR developer

Finally, low-cognitively demanding virtual environments should provide an introduction to the training using a video showing how to execute a task. Video may enhance comprehension of instructions, especially among patients with speech problems (i.e., aphasia).

High-cognitively demanding virtual environments

For high-cognitively demanding virtual environments, both groups described them as being dynamic, presenting increased sensory input and distractions, such as bright lights and loud sounds, moving objects (e.g., windows, doors, blinds, people), and challenging patients with secondary tasks. Figures 5B and 6B show examples of high-cognitively demanding virtual environments created by the participants.

“.. stimuli are all around you.. from multiple sides.. from left and right, increasing the one, reducing the other.. something can come from behind on the left, while you have to focus on the right, but there's a lot happening behind you.. imagine being in a space where people are actively exercising or running, and you're right in the middle of it..” (Group A)

“.. let people walk by.. in and out of the room.. people start conversations with him to see how he manages the task he’s working on.. what I would do is a person in the kitchen.. I would add a lot of music, add a lot of bells, people getting in and out and grab something out of the kitchen” (Group B)

Both groups emphasized the importance of indoor and outdoor environments for training. They also suggested environments resembling real-life ones (e.g., kitchens, bathrooms, public spaces), as they could facilitate the training of daily life activities.

“.. someone has a young baby at home, you can add, for example, baby noises..” (Group B)

“With a lot of distractions, and you prefer to keep it as close to what they are used to. So their kitchen, their supermarket, their devices, the things they are using.” (Group B);

“.. public transportation.. public locations.. bathroom, kitchen.. supermarket, traffic.. you’re in a car..” (Group A)

“.. a concert because a lot of people think.. they can handle it.. or a party from family, social environment, birthday!” (Group B)

Group A further discussed the topic of instruction and suggested adding simultaneously verbal instructions on top of other modalities to challenge the patient further.

In summary, both groups discussed the same types of stimuli—visual, auditory, and cognitive—and how their manipulation would ensure the creation of different IVR experiences. Only Group A discussed the need to provide instructions by using different means, from video to verbal and written instructions. Besides, groups shared similar ideas about low-cognitively demanding environments being static, empty, and silent, and high-cognitively demanding environments resembling real-life environments (e.g., kitchens, supermarkets, metro and train stations, shopping streets), moving elements (e.g., objects, people), bright lights, and loud sounds.