Open Collections

UBC Graduate Research

Kinecting the moves: the kinematic potential of rehabilitation-specific gaming to inform treatment for… Glegg, Stephanie M. N.; Hung, Chai-Ting; Valdés, Bulmaro A.; Kim, Brandon D. G.; Van der Loos, H. F. Machiel 2014

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata

Download

Media
42591-Glegg_S_et_al_BMEG_699_2014.pdf [ 910.38kB ]
42591-Glegg_S_et_al_BMEG_699_2014_Preprint.pdf [ 673.63kB ]
Metadata
JSON: 42591-1.0075871.json
JSON-LD: 42591-1.0075871-ld.json
RDF/XML (Pretty): 42591-1.0075871-rdf.xml
RDF/JSON: 42591-1.0075871-rdf.json
Turtle: 42591-1.0075871-turtle.txt
N-Triples: 42591-1.0075871-rdf-ntriples.txt
Original Record: 42591-1.0075871-source.json
Full Text
42591-1.0075871-fulltext.txt
Citation
42591-1.0075871.ris

Full Text

Proc. 10th Intl Conf. Disability, Virtual Reality & Associated Technologies  Gothenburg, Sweden, 2–4 Sept. 2014 ©2014 ICDVRAT; ISBN 978-0-7049-1546-6 1 Kinecting the Moves: The kinematic potential of rehabilitation-specific gaming to inform treatment for hemiplegia S M N Glegg1, C T Hung2, B A Valdés3, B D G Kim4, H F M Van der Loos5 1Therapy Department, Sunny Hill Health Centre for Children,  3644 Slocan Street, Vancouver, B.C., CANADA 2,3,4,5Department of Mechanical Engineering, University of British Columbia, 6250 Applied Science Lane, Vancouver, B.C. V6T 1Z4 CANADA 1sglegg@cw.bc.ca, 2tina.ct.hung@alumni.ubc.ca, 3bulmaro.valdes@alumni.ubc.ca, 4brandonkim.bk@alumni.ubc.ca, 5vdl@mech.ubc.ca 1,2,3,4,5http://caris.mech.ubc.ca/feathers/ ABSTRACT Two therapy applications for hemiplegic arm rehabilitation were developed and tested, along with a motion tracking application that used two interfaces (PlayStation® Move and Microsoft® Kinect™) for videogame play through a social media application developed on Facebook©. To promote affected arm use, users are required to employ bimanual symmetrical hand motions. Preliminary kinematic data analysis of two subjects obtained during user testing is presented. Clinically relevant information, such as range of motion, trunk compensation, and total distance of hand movement was extracted from kinematic data. Results showed the system is capable of accommodating users with large variation in arm function.  1.  INTRODUCTION The use of commercial gaming systems is gaining momentum in the field of rehabilitation (Galvin and Levac, 2011). These systems have been applied to target physical rehabilitation goals including upper extremity function (Luna-Oliva et al, 2013). Challenges exist, however, in the application of these systems to meet the therapeutic needs and physical capacity of different patient populations. Therapeutic gaming may be one treatment tool selected by therapists for individuals with hemiplegia as a means of providing opportunities for repetitive motor practice that targets specific movement patterns and encourages the use of the impaired limb (Orihuela-Espina et al, 2013). Accordingly, the development of novel game applications and user interfaces for these commercial systems is expanding the potential for the technology to be integrated in this way.  The purposes of this paper are to describe the development of two commercial interfaces (PlayStation Move and Microsoft Kinect) that were adapted to promote bilateral arm use during social media-based game play, and to share preliminary kinematic data of two subjects with hemiplegia using the systems. The analysis of kinematic data offered by the systems allows for the extraction of clinically relevant information that can be shared with patients’ therapists for further interpretation. Both PlayStation and Kinect systems are capable of determining the total distance moved in a session, range of motion (ROM) of the user, and hand offsets for different directional movements. Moreover, the Kinect system is capable of determining excessive trunk movements.  2.  METHOD 2.1  System Description In order to use the two motion capture interfaces, a computer application (FEATHERS Motion) was developed for the upper limb rehabilitation of hemiparetic users. Another application, FEATHERS Play on Facebook, enabled users to connect with their therapists and other participants, to receive recommendations about games, and to review their game scores. An alternate version of the application was developed for therapists to monitor users’ game scores and facilitate communication with their patients. Both applications were refined based on the results of previous usability testing conducted with rehabilitation professionals (see Valdés et al., 2014 for more details).  The FEATHERS Motion application relies on the use of bimanual motions in the frontal plane to control the  Proc. 10th Intl Conf. Disability, Virtual Reality & Associated Technologies  Gothenburg, Sweden, 2–4 Sept. 2014 ©2014 ICDVRAT; ISBN 978-0-7049-1546-6 2 mouse cursor on a Windows® 7 personal computer. Two motion modes (Visual Symmetry and Point Mirror Symmetry) are available for mapping the hand with the least movement into cursor motion. In the Visual Symmetry mode, users are required to move both hands at the same time in the same direction. In the Point Mirror Symmetry Mode, users must move both hands around the circumference of a circle, similar to steering a wheel.  2.2  Participants Participants were two male adolescents recruited through therapists at a local rehabilitation centre.  Subject 1 (19 years old) was right-hand dominant and presented with left hemiparesis with increased finger flexor tone post-traumatic brain injury and brachial plexus injury two years prior. Some decreases in both active and passive ROM for shoulder flexion, extension and external rotation persist. He was also observed to compensate with his flexors during shoulder abductions. Subject 2 (13 years old) was left-hand dominant prior to incurring a stroke 14 months ago. He presented with right hemiparesis, with weakness of the external rotators of the shoulder, no active supination of the forearm and decreased wrist flexor and extensor strength. A healthy right-handed male control (28 years old) participated as a comparison.  2.3  Procedure Each user test session included a moderator, note taker, caregiver/guardian and therapist. All sessions were audio and video recorded. University of British Columbia Ethics Board approval was obtained, along with informed consent from participants and a parent/guardian. Each user participated in a 90-minute session during which a set of tasks was completed to evaluate ease of use of the system. Users were introduced to the FEATHERS applications and the interfaces, and played “Lucky Pirate” (OUAT Entertainment) in both motion modes after receiving instructions on the movement and task requirements. Kinematic data was recorded for both interfaces, i.e., the 3D position of the PlayStation Move controllers and of all upper limb joints using Microsoft Kinect.  3.  RESULTS & DISCUSSIONS 3.1  Performance Data for One Session Joint position data were analyzed for 2.5-3 minutes per subject using six joints (wrists, shoulders, shoulder centre, and hip centre) during the Visual Symmetry play mode. Recommended filter values provided by Kinect for Windows SDK were applied to minimize jittering and to stabilize joint positions over time.  3.1.1 Total Distance Travelled in 2D. The total distance travelled by the wrists (Table 1) was calculated by subtracting the wrists’ horizontal (x-axis) and vertical (y-axis) positions from consecutive camera frames and summing the absolute values of the differences through the whole duration of the interaction. Because most of the wrists’ movements occurred in the frontal plane, only the horizontal and vertical positions were used for this calculation. In the next study phase where users are required to perform movements with larger variation in depth (z-axis), 3D data of the wrists will be used. Subject 2, who had the greatest level of impairment, appeared to cover more distance than the other two subjects. Video and kinematic data analyses suggest that this might be related to his frequent need to rest his hands in his lap between movements. This effect can be observed in the large values for both vertical distances. Subject 2 was observed to employ compensatory movements of the trunk to accommodate for his limited upper limb motor control. Overall, the values for Subject 1 were closer to the healthy control’s results. This finding may relate to the shorter and more direct trajectories between targets compared to those of Subject 2. This observation may be explained by Subject 1’s greater motor ability and the fact that he kept his arms at chest level for most of the interaction.  Table 1. Total Distance Travelled (* denotes hemiparetic side) Horizontal (m) Vertical (m)  Left Hand Right Hand Left Hand Right Hand Control 4.94 4.11 6.99 7.35 Subject 1 7.04* 6.07 8.22* 6.69 Subject 2 8.32 9.11* 13.85 7.64*  Therapists may find information about the total distance travelled useful, in conjunction with the straightness of the hands’ trajectories, in order to assess if the users’ movements are progressing towards healthy movement patterns. Distance travelled may have potential as an indicator of the recovery progress of participants. Proc. 10th Intl Conf. Disability, Virtual Reality & Associated Technologies  Gothenburg, Sweden, 2–4 Sept. 2014 ©2014 ICDVRAT; ISBN 978-0-7049-1546-6 3 3.1.2 Range of motion. Table 2 shows the ROM of each hand, computed based on the wrist movements of each subject (Figures 1-3). All figures were centred with respect to the median values of the hip centre. In the vertical direction for both hemiparetic subjects, and in the horizontal direction for Subject 1, larger ROM of the non-paretic versus the paretic arm was recorded. These findings are consistent with clinical presentation during functional tasks. Dissimilar findings for Subject 2 in the horizontal plane may be explained clinically by his limited control of the paretic side and his tendency to use compensatory trunk movements during play. The magnitudes of difference should be interpreted with caution owing to indeterminate tracking error of the system.  Table 2. Range of Motion (* denotes hemiparetic side) Horizontal (m) Vertical (m)  Left Hand  Right Hand  Left Hand  Right Hand Control 0.34 0.28 0.41 0.49 Subject 1 0.40* 0.58 0.40* 0.56 Subject 2 0.45 0.49* 0.60 0.50*     Figure 1. Healthy control wrist range of motion.       Figure 2. Subject 1 wrist range of motion.   Figure 3. Subject 2 wrist range of motion. 3.2  Data Analysis on Directional Movement In order to extract kinematic information related to the subjects’ intended direction of motion, all movements in a game session were categorized into horizontal and vertical segments. This section presents data on the upward movement that shows subjects’ trunk compensation and vertical hand offsets. 3.2.1 Trunk Compensation. Figures 4 and 5 show one upward movement trajectory for each subject. The paretic side of both subjects moved in a longer trajectory that was less straight than the unaffected side. In addition, subjects tried to synchronize both of their arms to perform a bimanual movement, evidenced by both hands stopping close to the same height. Wrist and shoulder trajectory for Subject 2 showed clear evidence of excessive trunk movement on the right side. Further 3D kinematic analysis indicated that his left shoulder moved downwards to the left and backwards, while his right shoulder moved upwards to the left and forward. No excessive trunk movement was observed in Subject 1’s trajectory. 3.2.2 Vertical offsets of both hands in an upward movement. Three upward movement data sets on vertical offsets for each subject were plotted in Figure 6. Values were calculated with respect to the paretic side (i.e. a positive value means the non-paretic side was at a higher vertical position). The vertical wrist offset of Subject 1 ranged from a value close to 0 m to 0.28 m, while for Subject 2, it ranged from -0.04 m to 0.09m. Moreover, all  Proc. 10th Intl Conf. Disability, Virtual Reality & Associated Technologies  Gothenburg, Sweden, 2–4 Sept. 2014 ©2014 ICDVRAT; ISBN 978-0-7049-1546-6 4                  Figure 4. Subject 1 wrist and shoulder trajectory.         Figure 5. Subject 2 wrist and shoulder trajectory. offsets shared a similar decreasing trend with respect to motion time. This result is consistent with the discussion in Section 3.2.1, which suggests that the subjects were trying to reach the same vertical position at the end of the motion.  Figure 6. Vertical wrist offsets for upward movements. 4. CONCLUSIONS AND FUTURE WORK Kinematic analysis of data provided by commercially available motion tracking technology could serve as an additional rehabilitation tool for therapists. While system limitations exist relative to the accuracy of gold standard motion tracking technology, this trend data can be used in tandem with clinical observations to identify variations in subjects’ gross motor movements compared to healthy controls. This study demonstrates the type of data that could be provided to therapists about the quality and amount of movement during therapeutic gaming. These results will inform the next design iteration of this project to evaluate the effectiveness of a 6-month home-based treatment using the system.  Acknowledgements: We would like to acknowledge the FEATHERS team, the staff from Sunny Hill Health Centre for Children, Reality Controls Inc., Brendan Sexton, and the participants and their families. 5. REFERENCES Galvin, J, and Levac, D, (2011), Facilitating clinical decision-making about the use of virtual reality within paediatric motor rehabilitation: Describing and classifying virtual reality systems, Dev Neurorehabil, 14, 2, pp. 112-122.  Luna-Oliva, L, Ortiz-Guitierrez, R, Martiniz, PR, Alquacil-Diego, I, Sanchez-Camarero, C, and del Carmen, M, (2013), Kinect Xbox 360 as a therapeutic modality for children with cerebral palsy in a school environment: A preliminary study, NeuroRehabil, 33, 4, pp. 513-521.  Orihuela-Espina, F, Fernandez del Castillo, I, Palafox, L, Pasaye, E, Sanchez-Villavicencio, I, and Leder, R, (2013), Neural reorganization accompanying upper limb motor rehabilitation from stroke with virtual reality-based gesture therapy, Topics in Stroke Rehabil, 20, 3, pp. 197.  Valdés, BA, Hilderman, CGE, Hung, CT, Shirzad, N, Van Der Loos, HFM, (2014), Usability testing of gaming  and social media applications for stroke and cerebral palsy upper limb rehabilitation. Proc. IEEE Eng. Med. Biol. Soc., Chicago, In Press.   Proc. 10th Intl Conf. Disability, Virtual Reality & Associated Technologies  Gothenburg, Sweden, 2–4 Sept. 2014 ©2014 ICDVRAT; ISBN 978-0-7049-1546-6 1 Kinecting the Moves: The kinematic potential of rehabilitation-specific gaming to inform treatment for hemiplegia S M N Glegg1, C T Hung2, B A Valdés3, B D G Kim4, H F M Van der Loos5 1Therapy Department, Sunny Hill Health Centre for Children,  3644 Slocan Street, Vancouver, B.C., CANADA 2,3,4,5Department of Mechanical Engineering, University of British Columbia, 6250 Applied Science Lane, Vancouver, B.C. V6T 1Z4 CANADA 1sglegg@cw.bc.ca, 2tina.ct.hung@alumni.ubc.ca, 3bulmaro.valdes@alumni.ubc.ca, 4brandonkim.bk@alumni.ubc.ca, 5vdl@mech.ubc.ca 1,2,3,4,5http://caris.mech.ubc.ca/feathers/ ABSTRACT Two therapy applications for hemiplegic arm rehabilitation were developed and tested, along with a motion tracking application that used two interfaces (PlayStation® Move and Microsoft® Kinect™) for videogame play through a social media application developed on Facebook©. To promote affected arm use, users are required to employ bimanual symmetrical hand motions. Preliminary kinematic data analysis of two subjects obtained during user testing is presented. Clinically relevant information, such as range of motion, trunk compensation, and total distance of hand movement was extracted from kinematic data. Results showed the system is capable of accommodating users with large variation in arm function.  1.  INTRODUCTION The use of commercial gaming systems is gaining momentum in the field of rehabilitation (Galvin and Levac, 2011). These systems have been applied to target physical rehabilitation goals including upper extremity function (Luna-Oliva et al, 2013). Challenges exist, however, in the application of these systems to meet the therapeutic needs and physical capacity of different patient populations. Therapeutic gaming may be one treatment tool selected by therapists for individuals with hemiplegia as a means of providing opportunities for repetitive motor practice that targets specific movement patterns and encourages the use of the impaired limb (Orihuela-Espina et al, 2013). Accordingly, the development of novel game applications and user interfaces for these commercial systems is expanding the potential for the technology to be integrated in this way.  The purposes of this paper are to describe the development of two commercial interfaces (PlayStation Move and Microsoft Kinect) that were adapted to promote bilateral arm use during social media-based game play, and to share preliminary kinematic data of two subjects with hemiplegia using the systems. The analysis of kinematic data offered by the systems allows for the extraction of clinically relevant information that can be shared with patients’ therapists for further interpretation. Both PlayStation and Kinect systems are capable of determining the total distance moved in a session, range of motion (ROM) of the user, and hand offsets for different directional movements. Moreover, the Kinect system is capable of determining excessive trunk movements.  2.  METHOD 2.1  System Description In order to use the two motion capture interfaces, a computer application (FEATHERS Motion) was developed for the upper limb rehabilitation of hemiparetic users. Another application, FEATHERS Play on Facebook, enabled users to connect with their therapists and other participants, to receive recommendations about games, and to review their game scores. An alternate version of the application was developed for therapists to monitor users’ game scores and facilitate communication with their patients. Both applications were refined based on the results of previous usability testing conducted with rehabilitation professionals (see Valdés et al., 2014 for more details).  The FEATHERS Motion application relies on the use of bimanual motions in the frontal plane to control the  Proc. 10th Intl Conf. Disability, Virtual Reality & Associated Technologies  Gothenburg, Sweden, 2–4 Sept. 2014 ©2014 ICDVRAT; ISBN 978-0-7049-1546-6 2 mouse cursor on a Windows® 7 personal computer. Two motion modes (Visual Symmetry and Point Mirror Symmetry) are available for mapping the hand with the least movement into cursor motion. In the Visual Symmetry mode, users are required to move both hands at the same time in the same direction. In the Point Mirror Symmetry Mode, users must move both hands around the circumference of a circle, similar to steering a wheel.  2.2  Participants Participants were two male adolescents recruited through therapists at a local rehabilitation centre.  Subject 1 (19 years old) was right-hand dominant and presented with left hemiparesis with increased finger flexor tone post-traumatic brain injury and brachial plexus injury two years prior. Some decreases in both active and passive ROM for shoulder flexion, extension and external rotation persist. He was also observed to compensate with his flexors during shoulder abductions. Subject 2 (13 years old) was left-hand dominant prior to incurring a stroke 14 months ago. He presented with right hemiparesis, with weakness of the external rotators of the shoulder, no active supination of the forearm and decreased wrist flexor and extensor strength. A healthy right-handed male control (28 years old) participated as a comparison.  2.3  Procedure Each user test session included a moderator, note taker, caregiver/guardian and therapist. All sessions were audio and video recorded. University of British Columbia Ethics Board approval was obtained, along with informed consent from participants and a parent/guardian. Each user participated in a 90-minute session during which a set of tasks was completed to evaluate ease of use of the system. Users were introduced to the FEATHERS applications and the interfaces, and played “Lucky Pirate” (OUAT Entertainment) in both motion modes after receiving instructions on the movement and task requirements. Kinematic data was recorded for both interfaces, i.e., the 3D position of the PlayStation Move controllers and of all upper limb joints using Microsoft Kinect.  3.  RESULTS & DISCUSSIONS 3.1  Performance Data for One Session Joint position data were analyzed for 2.5-3 minutes per subject using six joints (wrists, shoulders, shoulder centre, and hip centre) during the Visual Symmetry play mode. Recommended filter values provided by Kinect for Windows SDK were applied to minimize jittering and to stabilize joint positions over time.  3.1.1 Total Distance Travelled in 2D. The total distance travelled by the wrists (Table 1) was calculated by subtracting the wrists’ horizontal (x-axis) and vertical (y-axis) positions from consecutive camera frames and summing the absolute values of the differences through the whole duration of the interaction. Because most of the wrists’ movements occurred in the frontal plane, only the horizontal and vertical positions were used for this calculation. In the next study phase where users are required to perform movements with larger variation in depth (z-axis), 3D data of the wrists will be used. Subject 2, who had the greatest level of impairment, appeared to cover more distance than the other two subjects. Video and kinematic data analyses suggest that this might be related to his frequent need to rest his hands in his lap between movements. This effect can be observed in the large values for both vertical distances. Subject 2 was observed to employ compensatory movements of the trunk to accommodate for his limited upper limb motor control. Overall, the values for Subject 1 were closer to the healthy control’s results. This finding may relate to the shorter and more direct trajectories between targets compared to those of Subject 2. This observation may be explained by Subject 1’s greater motor ability and the fact that he kept his arms at chest level for most of the interaction.  Table 1. Total Distance Travelled (* denotes hemiparetic side) Horizontal (m) Vertical (m)  Left Hand Right Hand Left Hand Right Hand Control 4.94 4.11 6.99 7.35 Subject 1 7.04* 6.07 8.22* 6.69 Subject 2 8.32 9.11* 13.85 7.64*  Therapists may find information about the total distance travelled useful, in conjunction with the straightness of the hands’ trajectories, in order to assess if the users’ movements are progressing towards healthy movement patterns. Distance travelled may have potential as an indicator of the recovery progress of participants. Proc. 10th Intl Conf. Disability, Virtual Reality & Associated Technologies  Gothenburg, Sweden, 2–4 Sept. 2014 ©2014 ICDVRAT; ISBN 978-0-7049-1546-6 3 3.1.2 Range of motion. Table 2 shows the ROM of each hand, computed based on the wrist movements of each subject (Figures 1-3). All figures were centred with respect to the median values of the hip centre. In the vertical direction for both hemiparetic subjects, and in the horizontal direction for Subject 1, larger ROM of the non-paretic versus the paretic arm was recorded. These findings are consistent with clinical presentation during functional tasks. Dissimilar findings for Subject 2 in the horizontal plane may be explained clinically by his limited control of the paretic side and his tendency to use compensatory trunk movements during play. The magnitudes of difference should be interpreted with caution owing to indeterminate tracking error of the system.  Table 2. Range of Motion (* denotes hemiparetic side) Horizontal (m) Vertical (m)  Left Hand  Right Hand  Left Hand  Right Hand Control 0.34 0.28 0.41 0.49 Subject 1 0.40* 0.58 0.40* 0.56 Subject 2 0.45 0.49* 0.60 0.50*     Figure 1. Healthy control wrist range of motion.       Figure 2. Subject 1 wrist range of motion.   Figure 3. Subject 2 wrist range of motion. 3.2  Data Analysis on Directional Movement In order to extract kinematic information related to the subjects’ intended direction of motion, all movements in a game session were categorized into horizontal and vertical segments. This section presents data on the upward movement that shows subjects’ trunk compensation and vertical hand offsets. 3.2.1 Trunk Compensation. Figures 4 and 5 show one upward movement trajectory for each subject. The paretic side of both subjects moved in a longer trajectory that was less straight than the unaffected side. In addition, subjects tried to synchronize both of their arms to perform a bimanual movement, evidenced by both hands stopping close to the same height. Wrist and shoulder trajectory for Subject 2 showed clear evidence of excessive trunk movement on the right side. Further 3D kinematic analysis indicated that his left shoulder moved downwards to the left and backwards, while his right shoulder moved upwards to the left and forward. No excessive trunk movement was observed in Subject 1’s trajectory. 3.2.2 Vertical offsets of both hands in an upward movement. Three upward movement data sets on vertical offsets for each subject were plotted in Figure 6. Values were calculated with respect to the paretic side (i.e. a positive value means the non-paretic side was at a higher vertical position). The vertical wrist offset of Subject 1 ranged from a value close to 0 m to 0.28 m, while for Subject 2, it ranged from -0.04 m to 0.09m. Moreover, all  Proc. 10th Intl Conf. Disability, Virtual Reality & Associated Technologies  Gothenburg, Sweden, 2–4 Sept. 2014 ©2014 ICDVRAT; ISBN 978-0-7049-1546-6 4                  Figure 4. Subject 1 wrist and shoulder trajectory.         Figure 5. Subject 2 wrist and shoulder trajectory. offsets shared a similar decreasing trend with respect to motion time. This result is consistent with the discussion in Section 3.2.1, which suggests that the subjects were trying to reach the same vertical position at the end of the motion.  Figure 6. Vertical wrist offsets for upward movements. 4. CONCLUSIONS AND FUTURE WORK Kinematic analysis of data provided by commercially available motion tracking technology could serve as an additional rehabilitation tool for therapists. While system limitations exist relative to the accuracy of gold standard motion tracking technology, this trend data can be used in tandem with clinical observations to identify variations in subjects’ gross motor movements compared to healthy controls. This study demonstrates the type of data that could be provided to therapists about the quality and amount of movement during therapeutic gaming. These results will inform the next design iteration of this project to evaluate the effectiveness of a 6-month home-based treatment using the system.  Acknowledgements: We would like to acknowledge the FEATHERS team, the staff from Sunny Hill Health Centre for Children, Reality Controls Inc., Brendan Sexton, and the participants and their families. 5. REFERENCES Galvin, J, and Levac, D, (2011), Facilitating clinical decision-making about the use of virtual reality within paediatric motor rehabilitation: Describing and classifying virtual reality systems, Dev Neurorehabil, 14, 2, pp. 112-122.  Luna-Oliva, L, Ortiz-Guitierrez, R, Martiniz, PR, Alquacil-Diego, I, Sanchez-Camarero, C, and del Carmen, M, (2013), Kinect Xbox 360 as a therapeutic modality for children with cerebral palsy in a school environment: A preliminary study, NeuroRehabil, 33, 4, pp. 513-521.  Orihuela-Espina, F, Fernandez del Castillo, I, Palafox, L, Pasaye, E, Sanchez-Villavicencio, I, and Leder, R, (2013), Neural reorganization accompanying upper limb motor rehabilitation from stroke with virtual reality-based gesture therapy, Topics in Stroke Rehabil, 20, 3, pp. 197.  Valdés, BA, Hilderman, CGE, Hung, CT, Shirzad, N, Van Der Loos, HFM, (2014), Usability testing of gaming  and social media applications for stroke and cerebral palsy upper limb rehabilitation. Proc. IEEE Eng. Med. Biol. Soc., Chicago, In Press.   Running title: Kinecting the Moves  1     Kinecting the Moves: The kinematic potential of rehabilitation-specific gaming to inform treatment for hemiparesis    Stephanie Miranda Nadine Glegg1, B.Sc. (Kin), B.Sc. (OT), M.Sc. (Rehab Sci) sglegg@cw.bc.ca  Chai-Ting Hung, B.Eng. 2 tina.ct.hung@alumni.ubc.ca  Bulmaro Adolfo Valdés Benavides3 B.Eng. (Mechatronics), MPE (Biomedical Eng.) bulmaro.valdes@alumni.ubc.ca  Brandon D. G. Kim4 brandonkim.bk@alumni.ubc.ca  H. F. Machiel Van der Loos, Ph.D., P.Eng.5 vdl@mech.ubc.ca  1Therapy Department, Sunny Hill Health Centre for Children, 3644 Slocan Street, Vancouver, B.C. V5M 3E8 CANADA 2-5Department of Mechanical Engineering, University of British Columbia, 6250 Applied Science Lane, Vancouver, B.C., V6T 1Z4 CANADA 1-5 http://rreach.mech.ubc.ca/research/projects/feathers/ Running title: Kinecting the Moves  2 Abstract: Adapted commercial gaming systems are gaining momentum as cost-effective rehabilitation tools. However, a paucity of accessible rehabilitation-specific systems exist that support bimanual training for individuals with hemiparesis. The objectives of this paper are to describe the development of such a system, and to present preliminary kinematic data analysis obtained from the system to explore its clinical relevance. Two therapy applications for hemiparetic arm rehabilitation were developed and tested, along with a motion tracking application that used two interfaces (PlayStation® Move and Microsoft® Kinect™) for videogame play through a social media application developed for Facebook©. To promote hemiparetic arm use, participants were required to employ bimanual symmetrical hand motions during game play. Data were obtained from two adolescent participants with acquired brain injury and one healthy control, all of whom were part of the usability testing phase of the FEATHERS (Functional Engagement in Assisted Therapy through Exercise Robotics) project. Data from distinct movement trajectories collected during one game session were filtered using described protocols. Total distance moved, range of motion, trunk compensation, and vertical wrist offsets were extracted from kinematic data for therapist interpretation. Clinical observations, video analysis and comparison to a healthy control supported the interpretation of the results. Results showed the system is capable of accommodating participants with large variation in arm function. The kinematic data and analysis algorithms presented may be useful to inform therapists about their patients’ performance and progress during the remote monitoring of home-based therapy programs using the system.   Keywords: Serious games, virtual rehabilitation, kinematics, Kinect, hemiparesis, brain injury, rehabilitation, bimanual therapy, motion tracking, social media   Correspondence: Stephanie MN Glegg, MSc, OTR, Therapy Department, Sunny Hill Health Centre for Children, 3644 Slocan Street, Vancouver, BC V5M 3E8, CANADA. Email: sglegg@cw.bc.caRunning title: Kinecting the Moves  3 Introduction The use of commercial gaming systems is gaining momentum in the field of rehabilitation (1). Virtual reality and active video games can increase user engagement and enjoyment in rehabilitation, increasing the potential for enhancing patient outcomes (2-3). Therapeutic gaming may be one treatment tool selected by therapists for individuals with hemiparesis as a means of providing motivating opportunities for repetitive motor practice that encourages specific movement patterns and use of the impaired limb (4-5). Bimanual therapy is an effective approach to reduce impairment and to improve functional ability of the paretic arm (6). However, commercial video games are not designed to consistently optimize the use of both arms simultaneously. Challenges exist, therefore, in the application of these systems to meet the therapeutic needs and physical capacities of different patient populations (7). Accordingly, the development of novel game applications and user interfaces for commercial gaming systems is expanding the potential for the technology to be adapted and integrated for these purposes, both in clinics and in the home.  Motion tracking technology is being used increasingly both as a movement interface in these gaming systems, and as a means of capturing data about participants’ kinematic movements during rehabilitation (8-9). The Vicon system (Oxford, UK) (10), the FASTRAK system (League City, TX) (11), and the Microsoft Kinect™ (Redmond, WA) (12) are three examples of this technology. The marker-based Vicon and FASTRAK systems provide higher accuracy compared to the Kinect, however at a higher cost and required expertise, making them unsuitable for most rehabilitation applications. Conversely, the Kinect has demonstrated centimetre-level accuracy, yet high correlations of tracked data with those obtained by marker-based systems (13). These findings, along with its commercial availability, make the Kinect a potential candidate for use in a clinical or home setting for rehabilitation. Furthermore, at this time, no documented guidance exists to translate the Kinect’s kinematic data into clinically relevant information that is useful to therapists prescribing gaming interventions. The FEATHERS (Functional Engagement in Assisted Therapy through Exercise Robotics) project focuses on the development, testing and implementation of rehabilitation-specific interfaces for bimanual therapy, and the extraction of kinematic data to inform therapists monitoring patients’ performance and progress. By applying custom algorithms to data collected from the Sony PlayStation Move (Tokyo, Japan) and the Kinect systems, it is possible to determine the total distance moved by the participant, their range of motion (ROM), and the vertical hand offsets for different directional movements. Moreover, the Kinect system is capable of providing data about excessive trunk movements.  The purposes of this paper are therefore: 1) to describe the adaptation of two commercial interfaces (PlayStation Move and Microsoft Kinect) to promote bilateral arm use during social media-based game play; and 2) to share preliminary kinematic data from two participants with hemiparesis using the systems. The analysis of the kinematic data offered by the systems allows for the extraction of clinically relevant information that can be shared with therapists to inform their treatment decisions.   Methods System Description In order to use the two motion capture interfaces for the upper limb rehabilitation of individuals with hemiparesis, a computer application called “FEATHERS Motion” was developed. The FEATHERS Motion application relies on the use of bimanual motions in the frontal plane to Running title: Kinecting the Moves  4 control the mouse cursor on a Windows® 7 personal computer. Two motion modes (Visual Symmetry and Point Mirror Symmetry) are available for mapping the hand with the least movement into cursor motion. In Visual Symmetry mode, users are required to move both hands at the same time in the same direction, while in Point Mirror Symmetry Mode, users must move both hands around the circumference of a circle, similar to turning a steering wheel.  A second application, “FEATHERS Play”, enables users to connect with their therapists and other participants on Facebook to receive recommendations about games from their therapists and to review their game scores. An alternate version of the application was developed for therapists to monitor participants’ game scores and to facilitate communication with their patients.  Both applications have been refined based on the results of previous usability testing conducted with rehabilitation professionals (see (14) for details).   Participants Participants were two male adolescents recruited through therapists at a local rehabilitation centre. Subject 1 was 19 years old and right-hand dominant. He presented with left hemiparesis and increased finger flexor tone following a traumatic brain injury and brachial plexus injury two years prior. Some decreases in both active and passive ROM for shoulder flexion, extension and external rotation persisted at the time of the study. Subject 2 was 13 years old, and left-hand dominant prior to incurring a stroke 14 months before the study. He presented with right hemiparesis, with weakness of the external rotators of the shoulder, no active supination of the forearm and decreased wrist flexor and extensor strength. In addition, a healthy 28-year old right-handed male participated as a control comparison.   Study procedure Testing was carried out at a local children’s rehabilitation centre. Each user test session involved a moderator who provided information about the technology and task requirements, a note taker who recorded observations about interactions with the technology, a caregiver/guardian and a therapist who assisted with instructions, monitored and supported functional interactions based on participants’ needs, and recorded clinical observations. All sessions were audio and video-recorded. Research Ethics Board approval was obtained from the University of British Columbia, along with informed consent/assent from participants and a parent/guardian as applicable.  Each participant took part in a 90-minute session during which they were asked to complete a set of tasks to evaluate the ease of use of the system. Participants were introduced to the FEATHERS applications and both motion tracking interfaces, and played “Lucky Pirate” (OUAT Entertainment) in both motion modes using each interface after receiving instructions on the movement and task requirements.  Kinematic data were recorded for both interfaces, i.e., the 3-dimensional (3D) position of the PlayStation Move controllers, and all of the upper body joints using the Microsoft Kinect. Only kinematic data from the Kinect-based system are presented here because of the system’s capacity to allow for the analysis of trunk compensation.   Data analysis The kinematic data for each participant were captured during a game session of approximately three minutes. Joint position data were analyzed for each participant using six joints (wrists, Running title: Kinecting the Moves  5 shoulders, shoulder centre, and hip centre). Recommended filter values provided by the Kinect Windows software development kit (SDK) were applied to minimize jitter and to stabilize joint positions over time. To obtain the single session performance results, the data were filtered to remove outliers related to joint occlusions and noise. Video footage was cross-referenced to remove portions of the data in which the participant was not interacting with the system (e.g., receiving instructions, practicing, resting, etc.).  For the data analysis, four performance metrics were chosen: the total distance travelled by each hand, wrist range of motion (ROM), trunk compensation, and hand offsets during vertical movement. For the first two metrics as well as for hand offsets, joint position data at the wrist and hip centre were analyzed; for the analysis of compensation, shoulder and shoulder centre joints were added. These metrics were selected because of their potential for indicating participants’ functional abilities and movement quantity and quality. Even though these metrics can be applied to both motion modes (Visual Symmetry and Point Mirror Symmetry), only the Visual Symmetry mode is presented here to serve as an example of the system’s capability.  The total distance travelled by the wrists was calculated by measuring all the wrists’ horizontal (x-axis) and vertical (y-axis) displacements between consecutive camera frames through the entire duration of the interaction. As most of the wrist movements occurred in the frontal plane, only the X and Y values were used for this calculation. The filtered wrist data were calculated and plotted to obtain the wrist ROM. In order to extract kinematic information related to the participants’ intended direction of motion, all movements in the game session were first categorized into horizontal and vertical segments based on cursor movements. Each segment was then grouped into one of four categories: upward, downward, right and left, based on the direction of the wrist trajectories. Upward trajectories of the shoulders and wrists were plotted to identify any trunk compensation. Finally, three upward movement data sets for each participant were analyzed for vertical hand offsets. During this analysis, the positions of both wrists were compared in the vertical direction to obtain a scalar that indicated the level of vertical asymmetry in the participants’ movements. Values were calculated with respect to the paretic side (i.e., a positive value means the non-paretic side was at a higher vertical position).   Results Total distance travelled in two dimensions Table 1 summarizes the total distance travelled by the wrists for each participant. Although only movements in the frontal plane were analyzed, 3D data of the wrists will be used in the next study phase in which participants will be required to perform movements with larger variation in depth (z-axis).  [Insert Table 1 about here]  Range of motion Table 2 shows the ROM of each hand, computed based on the wrist movements of each participant. Figures 1-3 illustrate how the horizontal and vertical ROMs are different for each of the participants, and how each of the participants’ hands moved in different areas in front of their bodies. All figures were centred with respect to the median values of the hip centre.  [Insert Table 2 and Figures 1-3 about here]  Running title: Kinecting the Moves  6 Trunk compensation Figures 4 and 5 show one upward movement trajectory for each hemiparetic participant. Shoulder and wrist data are plotted and centred with respect to the median values of the hip centre.  [Insert Figures 4 and 5 about here]  Vertical offsets of both hands in an upward movement The three upward movement data sets on vertical hand offsets for all three participants are plotted in Figure 6. The vertical wrist offset of Subject 1 ranged from a value close to 0 m to 0.28 m, while for Subject 2, it ranged from -0.04 m to 0.09 m. All offsets shared a similar decreasing trend with respect to motion time.  [Insert Figure 6 about here]  Discussion This paper presents a novel interface that targets a gap in the commercial gaming field for bimanual training for individuals with hemiparesis, using existing games through a social media platform. With the integration of the Kinect sensor, motion tracking data can be harvested to inform treatment program development and progression by therapists.  With respect to the total distance travelled by the wrists, overall, the values for Subject 1 were closer to the healthy control’s results than those of Subject 2. This finding may relate to the shorter and more direct trajectories achieved between targets for Subject 1. This observation may be explained by Subject 1’s greater motor ability and the fact that he kept his arms at chest level for most of the interaction. Subject 2, who had the greatest level of impairment, appeared to cover more distance than the other two participants. Video and kinematic data analyses suggest that this might be related to his frequent need to rest his hands in his lap between movements. This effect can be observed in the large values for both left and right vertical distances.  Therapists may find information about the total distance travelled useful, in conjunction with the straightness of the hands’ trajectories, in order to assess if the participants’ movements are progressing toward more efficient movement patterns. Distance travelled may hold potential as an indicator of the recovery progress of participants in this respect. In terms of wrist ROM, in the vertical direction for both hemiparetic participants and in the horizontal direction for Subject 1, larger ROM of the non-paretic arm versus the paretic arm was recorded, consistent with their clinical presentation during functional tasks. Dissimilar findings for Subject 2 in the horizontal plane may be explained clinically by his limited control of the paretic side and his tendency to use compensatory trunk movements during play. Given the symmetrical nature of the movement requirements during game play, these data may be valuable to therapists as an indicator of quality of movement and improvement of functional reaching capacity over time, as well as a means of monitoring fatigue of the paretic arm over the course of a treatment session.  Wrist and shoulder trajectories for Subject 2 in Figure 5 showed clear evidence of excessive trunk movement on the right side (19cm and 14cm for right and left shoulder, respectively). This result was verified by video footage, where Subject 2 was observed to employ compensatory movements of the trunk during reaching. Further analysis of the video and visual inspection of the 3D trajectories related to data shown in Figure 5 indicated that Subject 2’s left shoulder moved downwards to the left and backwards, while his right shoulder moved upwards to the left and forward. These compensatory movements of the trunk were thought to Running title: Kinecting the Moves  7 accommodate for his limited upper limb motor control. No excessive trunk movement was observed in Subject 1’s trajectory. Return to pre-injury movement patterns are desired in order to promote recovery and to prevent long-term consequences of compensatory movements, such as decreased range of motion, pain, or learned non-use (15). As a result, data about compensatory movements are important for therapists, particularly in situations, such as remote home-based monitoring, in which clinical observations about these movements may not be possible.  The paretic side for both participants moved in a longer trajectory that was less straight than the unaffected side. The decreasing trend in all vertical offsets with respect to motion time was consistent with the shoulder and wrist trajectory plots, suggesting that the participants were trying to synchronize both of their arms to reach the same vertical position at the end of the motion, as evidenced by both hands stopping close to the same height. Offsets between two corresponding joints during directional movements can provide useful clinical information on motion symmetry – the smaller the offset, the more symmetric the motion. For instance, data about over- or undershooting a target with the paretic arm can provide insights into quality and control of movement; fatigue may be deduced remotely by observing trends in offsets over time. In our example, the calculated vertical offset indicated how symmetric the hands were vertically when the participant was performing a bimanual upward motion; in the future this analysis could be repeated in other directions.   Limitations Two main limitations existed with respect to data analysis. Specifically, total distance was calculated using data from two dimensions despite depth excursion data being available. This decision was based on the nature of the movements required for the gaming task used during testing; in subsequent phases of the project, 3D data analysis will be become meaningful. In addition, trunk compensation data were analyzed for single movement trajectories; in reality, users perform multiple reaching trajectories during a game session. For these data to be valuable for therapists tracking patients’ progress, custom algorithms must include a means of comparing trajectories to known locations over time. A third limitation relates to the accuracy of the Kinect, given that this is an inexpensive, commercially available gaming sensor that is not designed for sub-millimeter accuracy. However, in this study, the Kinect was shown to be a suitable solution for providing therapists with useful information about participants’ gross upper limb movements. With the advances of the next generation Kinect sensor, the potential for improved accuracy may further enhance its utility for progress tracking during rehabilitation.   Conclusion Kinematic analysis of data provided by commercially available motion tracking technology could serve as an additional tool for therapists engaged in rehabilitation, particularly in the context of home-based interventions for which regular feedback about clients’ movements are otherwise not available. While system limitations exist relative to the accuracy of gold standard motion tracking technology, this trend data can be used in tandem with clinical observations to identify variations in participants’ gross motor movements compared to healthy controls. This paper highlights the type of data that could be provided to therapists about the quality and amount of movement carried out during therapeutic gaming. These results will inform the next design iteration of the interfaces and data processing algorithms of this project. These data analysis principles will be employed in the next phase of the project to provide data to therapists Running title: Kinecting the Moves  8 monitoring a 6-month home-based treatment using the developed systems with individuals with hemiparesis.   Running title: Kinecting the Moves  9 Acknowledgements We would like to acknowledge the members of the FEATHERS team, the staff from Sunny Hill Health Centre for Children, Reality Controls Inc., Brendan Sexton, and the participants and their families. This study was funded by the Peter Wall Solutions Initiative.   Running title: Kinecting the Moves  10 References 1. Thomson K, Pollock A, Bugge C, Brady M. Commercial gaming devices for stroke upper limb rehabilitation: A systematic review. Int J Stroke 2014;9:479-88.  2. Levac D, Rivard L, Missiuna C. Defining the active ingredients of interactive computer play interventions for children with neuromotor impairments: A scoping review. Res Dev Disabil 2012 Jan-Feb;33:214-23.  3. Glegg SMN, Tatla SK, Holsti L. The GestureTek virtual reality system in rehabilitation: A scoping review. Disabil Rehabil Assist Technol 2013;9:89-111. 4. Orihuela-Espina F, Fernandez del Castillo I, Palafox L, Pasaye E, Sanchez-Villavicencio I, Leder, R. Neural reorganization accompanying upper limb motor rehabilitation from stroke with virtual reality-based gesture therapy, Topic Stroke Rehabil 2013;20:197.  5. Luna-Oliva L, Ortiz-Guitierrez R, Martiniz PR, Alquacil-Diego I, Sanchez-Camarero C, del Carmen M. Kinect Xbox 360 as a therapeutic modality for children with cerebral palsy in a school environment: A preliminary study, NeuroRehabil 2013;33:513-21.  6. Wolf A, Scheiderer R, Napolitan N, Belden C, Shaub L, Whitford M. Efficacy and task structure of bimanual training post stroke: a systematic review. Top Stroke Rehabil 2014 May Jun;21:181-96.  7. Galvin J, Levac D. Facilitating clinical decision-making about the use of virtual reality within paediatric motor rehabilitation: Describing and classifying virtual reality systems, Dev Neurorehabil 2011;14:112-22.  8. Taylor MJD, McCormick D, Shawis T, Impson R, Griffin M. Activity-promoting gaming systems in exercise and rehabilitation. J Rehabil Res Dev 2011;48:1171-86.  9. Rammer JR, Krzak JJ, Riedel SA, Harris GF. Evaluation of upper extremity movement characteristics during standardized pediatric functional assessment with a Kinect®-based markerless motion analysis system. Proc IEEE Eng Med Biol Soc 2014 Chicago:2525-8.  10. Kapur P, Jensen M, Buxbaum LJ, Jax SA, Kuchenbecker KJ. Spatially distributed tactile feedback for kinesthetic motion guidance. IEEE Haptics Symp 2010 Mar;519-26. Accessed 2014 Jun 30. URL: http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=5444606 11. Kawashima N, Popovic MR, Zivanovic V. Effect of intensive functional electrical stimulation therapy on upper-limb motor recovery after stroke: Case study of a patient with chronic stroke. Physiother Can 2013 Jan 1;65:20–8.  12. Rotella MF, Guerin K, Okamura AM. HAPI Bands: A haptic augmented posture interface. IEEE Haptics Symp 2012 Mar;163–70. Accessed 2014 Jun 30. URL: http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=6183785 13. Webster D, Celik O. Systematic review of Kinect applications in elderly care and stroke rehabilitation. J Neuroeng Rehabil 2014;11:108-31. 14. Valdés BA, Hilderman CGE, Hung CT, Shirzad N, Van Der Loos HFM. Usability testing of gaming and social media applications for stroke and cerebral palsy upper limb rehabilitation. Proc IEEE Eng Med Biol Soc 2014 Chicago:3602-5.  15. Levin MF, Kleim JA, Wolf SL. What do motor ''recovery'' and ''compensation'' mean in patients following stroke? Neurorehabil Neural Repair 2009;23:313.  Running title: Kinecting the Moves  11 Tables and Figures  Table 1. Total distance travelled by the wrists during a single game session Participant Left Hand Horizontal (m) Left Hand Vertical (m) Right Hand Horizontal (m) Right Hand Vertical  (m) Control 4.94 6.99 4.11 7.35 Subject 1 *7.04 *8.22 6.07 6.69 Subject 2 8.32 13.85 *9.11 *7.64 Note: *=hemiparetic side  Table 2. Range of motion at the wrists during a single game session Participant Left Hand Horizontal (m) Left Hand Vertical (m) Right Hand Horizontal (m) Right Hand Vertical  (m) Control 0.34 0.41 0.28 0.49 Subject 1 *0.40 *0.40 0.58 0.56 Subject 2 0.45 0.60 *0.49 *0.50 Note: *=hemiparetic side     Figure 1. Healthy Control range of motion at the wrists    Figure 2. Subject 1 range of motion at the wrists (left hemiparesis)  Running title: Kinecting the Moves  12  Figure 3. Subject 2 range of motion at the wrists (right hemiparesis)                   Figure 4. Wrist and shoulder trajectories during a single upward movement by Subject 1 (left hemiparesis)      Figure 5. Wrist and shoulder trajectory during a single upward movement by Subject 2 (right hemiparesis)  Running title: Kinecting the Moves  13  Figure 6. Vertical wrist offsets of three upward movements for each participant 

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
http://iiif.library.ubc.ca/presentation/dsp.42591.1-0075871/manifest

Comment

Related Items