UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

3D biomechanical simulation and control of the human hand Sachdeva, Prashant 2019

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata


24-ubc_2019_november_sachdeva_prashant.pdf [ 5.63MB ]
JSON: 24-1.0381004.json
JSON-LD: 24-1.0381004-ld.json
RDF/XML (Pretty): 24-1.0381004-rdf.xml
RDF/JSON: 24-1.0381004-rdf.json
Turtle: 24-1.0381004-turtle.txt
N-Triples: 24-1.0381004-rdf-ntriples.txt
Original Record: 24-1.0381004-source.json
Full Text

Full Text

3D Biomechanical Simulation and Controlof the Human HandbyPrashant SachdevaB.Tech., Indian Institute of Technology Bombay, 2012M.Tech., Indian Institute of Technology Bombay, 2012A THESIS SUBMITTED IN PARTIAL FULFILLMENTOF THE REQUIREMENTS FOR THE DEGREE OFDoctor of PhilosophyinTHE FACULTY OF GRADUATE AND POSTDOCTORALSTUDIES(Computer Science)The University of British Columbia(Vancouver)September 2019© Prashant Sachdeva, 2019The following individuals certify that they have read, and recommend to the Facultyof Graduate and Postdoctoral Studies for acceptance, the thesis entitled:3D Biomechanical Simulation and Controlof the Human Handsubmitted by Prashant Sachdeva in partial fulfillment of the requirements for thedegree of Doctor of Philosophy in Computer Science.Examining Committee:Dinesh K. Pai, Computer Science, UBC VancouverSupervisorShinjiro Sueda, Computer Science, Texas A&M UniversitySupervisorThomas Oxland, Mechanical Engineering, UBC VancouverUniversity ExaminerWilliam Evans, Computer Science, UBC VancouverUniversity ExaminerMichiel van de Panne, Computer Science, UBC VancouverSupervisory Committee MemberAdditional Supervisory Committee Members:Uri Ascher, Computer Science, UBC VancouverSupervisory Committee MemberiiAbstractThe goal of this thesis is to develop novel computational tools and software fordetailed modelling of dynamics of biomechanical systems such as the human hand,with potential applications in prosthetics, surgery, robotics, and virtual reality. Westudy the effect of the finger extensor mechanism, and musculotendon control onthe kinematic and dynamic function of the hand.Hand tendons form a complex network of sheaths, pulleys, and branches. A threedimensional model capturing its detailed anatomy would help simulate the coordi-nation and internal dynamics of the musculoskeletal system. Previous approachesinclude resource-intensive cadaver studies and mathematical force-transmissionmodels, which cannot compute hand motion under muscle action.We developed a modelling and control framework for hand musculotendon dy-namics to overcome these limitations. This approach uses Eulerian-on-Lagrangiandiscretization of tendons with a selective quasistatic assumption, eliminating un-necessary degrees of freedom and the need for generic collision detection. Unlikeprevious approaches, our approach efficiently and accurately handles constrainedmusculotendon dynamics. Using this framework, two control approaches weredeveloped for precise fingertip trajectory tracking.To apply these techniques, software tools were developed with goals of interac-tive design, experimentation, and control of hand biomechanics. They overcomelimitations of other available biomechanics software, enabling modelling of com-plex tendon arrangements, such as the finger extensor assembly. These tools cansimulate all musculoskeletal elements of the hand, and allow closed-loop simulationcontrol.With these software tools, we built a detailed anatomical model of the lumbricaliiimuscle of the finger and simulated its role in reshaping finger flexion. The lumbricalplays an important role in determining the flexion order for the interphalangeal andmetacarpophalageal joints. Prior cadaver studies have recorded this role, providingan opportunity for model validation. The in vitro experiments were reproducedsuccessfully, establishing its role in increasing the grasp reach of the hand. Wealso modelled the in vivo function of the activated lumbrical, overcoming thelimitations of cadaver experiments. Finally, a preliminary model of the full handwas constructed with the thumb and the wrist, and simulations of tenodesis graspand simple thumb motions are presented.ivLay SummaryThe structure of the muscles and tendons of the hand contribute to its immensedexterity and strength. Modelling these tendons and their interaction with the bonesis instrumental to studying and understanding the dynamics of the human hand. Wehave designed and implemented a framework for modelling these tendon structuresand to control the fingertip trajectory with muscle activation. This controlled fingersimulation is capable of numerous dexterous tasks, such as tracing words on aplane with the fingertip. We have built software and design tools to enable detailedmodelling of the finger bones, joints, and tendons, and compared the simulatedfunction of muscles with experimental data. This software can be used to simulateany musculotendon system. We demonstrate this by simulating thumb and wristfunction. Furthermore, we provide a scripting and graphical user interface forbuilding new control methods for any biomechanical simulation.vPrefaceSome portions of this dissertation are from papers that have been published, orplanned to be submitted in the near future in collaboration with other people.Chapter 4 is from Biomechanical simulation and control of hands and tendinoussystems, ACM Transactions on Graphics (TOG), by Prashant Sachdeva, ShinjiroSueda, Susanne Bradley, Mikhail Fain, and Dinesh K Pai [Sachdeva et al., 2015].The text is from the paper, which was written in collaboration by the authors. Thepresent author was a shared first author with Dr. Sueda, who together wrote thesoftware for the forward simulation. The present author led the development of, anditeration over, finger models and muscle model design. The controlled simulationand the corresponding demos were written in collaboration with Susanne Bradley[Bradley, 2015] and Mikhail Fain [Fain, 2013], with the present author leading the“whitebox” controller, and helping the development of the “blackbox” controller.The demos were made possible by early software contributions by Crawford Doran,CT images made available by Dr. Mitsunori Tada, and segmentation help providedby Dr. Benjamin Gilles.The software described in Chapter 5 is written by the present author, with high-level guidance from Dr. Sueda, Dr. Tada, and Dr. Pai. Chapters 6, 7, and 8 describeresults produced with this software.The results in Chapter 7 are from a prepared manuscript under final revisions.The manuscript is written by the present author and edited by Drs. Sueda, Tada, Pai.The development of critical numerical techniques, biomechanical design ideas, andthe development of the finger model were all led by the present author under theguidance of Drs. Sueda and Pai. Underlying cadaver data was provided by Dr. Tada.This manuscript is in the final stages for submission.viTable of ContentsAbstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iiiLay Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vPreface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viTable of Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viiList of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiList of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiiGlossary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xivAcknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xviDedication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xviii1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.1 Modelling the Human Hand . . . . . . . . . . . . . . . . . . . . 11.2 Extensor Apparatus . . . . . . . . . . . . . . . . . . . . . . . . . 31.3 Research Objectives . . . . . . . . . . . . . . . . . . . . . . . . . 41.4 Tendon Dynamics . . . . . . . . . . . . . . . . . . . . . . . . . . 51.5 Tendon-driven Control . . . . . . . . . . . . . . . . . . . . . . . 71.6 Anatomical Simulation Software . . . . . . . . . . . . . . . . . . 81.7 Validated Hand Model Construction . . . . . . . . . . . . . . . . 10vii2 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122.1 Human Body Simulations . . . . . . . . . . . . . . . . . . . . . . 122.2 Hand Motion Capture . . . . . . . . . . . . . . . . . . . . . . . . 132.3 Dynamics of the Human Hand . . . . . . . . . . . . . . . . . . . 142.3.1 Biomechanics Studies . . . . . . . . . . . . . . . . . . . 142.3.2 Simulation Models . . . . . . . . . . . . . . . . . . . . . 152.3.3 Tendon-driven Robotic Hands . . . . . . . . . . . . . . . 152.4 Musculoskeletal Control . . . . . . . . . . . . . . . . . . . . . . 163 Musculoskeletal Dynamics . . . . . . . . . . . . . . . . . . . . . . . 183.1 Rigid Bodies and Joints . . . . . . . . . . . . . . . . . . . . . . . 183.2 Discretization of Tendons . . . . . . . . . . . . . . . . . . . . . . 203.2.1 Eulerian-on-Lagrangian Strands . . . . . . . . . . . . . . 213.2.2 Selective Quasistatics . . . . . . . . . . . . . . . . . . . . 223.3 Reduced Dynamics . . . . . . . . . . . . . . . . . . . . . . . . . 263.3.1 Full Space . . . . . . . . . . . . . . . . . . . . . . . . . 273.3.2 RL Space . . . . . . . . . . . . . . . . . . . . . . . . . . 283.3.3 R Space . . . . . . . . . . . . . . . . . . . . . . . . . . . 293.3.4 Example: Circle Q1 Node . . . . . . . . . . . . . . . . . 303.3.5 Final Reduced Form . . . . . . . . . . . . . . . . . . . . 314 Modelling and Control Framework . . . . . . . . . . . . . . . . . . 334.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 334.2 Modelling the Finger . . . . . . . . . . . . . . . . . . . . . . . . 354.2.1 Finger Anatomy . . . . . . . . . . . . . . . . . . . . . . 354.2.2 Muscle Model . . . . . . . . . . . . . . . . . . . . . . . 374.3 Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 394.3.1 Controller Algorithm . . . . . . . . . . . . . . . . . . . . 404.3.2 Estimating Black Box Controller Parameters . . . . . . . 414.3.3 Extracting White Box Controller Parameters . . . . . . . 434.3.4 Comparison of White Box and Black Box Methods . . . . 444.4 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 454.4.1 Joint Coupling . . . . . . . . . . . . . . . . . . . . . . . 46viii4.4.2 Deformities . . . . . . . . . . . . . . . . . . . . . . . . . 464.4.3 Trajectory Tracking . . . . . . . . . . . . . . . . . . . . . 474.4.4 Writing . . . . . . . . . . . . . . . . . . . . . . . . . . . 504.5 Conclusions and Limitations . . . . . . . . . . . . . . . . . . . . 515 Software Design and Implementation . . . . . . . . . . . . . . . . . 525.1 Software Design . . . . . . . . . . . . . . . . . . . . . . . . . . . 535.2 Workflow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 545.2.1 Design Workflow . . . . . . . . . . . . . . . . . . . . . . 545.2.2 Experiment Workflow . . . . . . . . . . . . . . . . . . . 575.2.3 Control Workflow . . . . . . . . . . . . . . . . . . . . . 595.3 Strands Simulation Library . . . . . . . . . . . . . . . . . . . . . 615.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 646 Anatomical Modelling of the Index Finger . . . . . . . . . . . . . . . 656.1 Branching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 666.2 Tendon Inextensibility . . . . . . . . . . . . . . . . . . . . . . . 676.3 Strand Remeshing at Joints . . . . . . . . . . . . . . . . . . . . . 697 Lumbrical Muscle and Finger Flexion . . . . . . . . . . . . . . . . . 757.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 767.2 Related Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . 787.3 Cadaver Results and Interpretation . . . . . . . . . . . . . . . . . 807.4 Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 817.5 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 827.5.1 Index Finger Model . . . . . . . . . . . . . . . . . . . . . 827.5.2 Simulation Experiments . . . . . . . . . . . . . . . . . . 857.6 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 867.7 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 888 Modelling the Hand . . . . . . . . . . . . . . . . . . . . . . . . . . . 918.1 Thumb . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 918.1.1 Example Thumb Movements . . . . . . . . . . . . . . . . 938.2 Wrist and Tenodesis Grasp . . . . . . . . . . . . . . . . . . . . . 93ix8.2.1 Tenodesis Results . . . . . . . . . . . . . . . . . . . . . . 979 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 999.1 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 999.1.1 Tendon Dynamics . . . . . . . . . . . . . . . . . . . . . 999.1.2 Tendon-Driven Control . . . . . . . . . . . . . . . . . . . 1009.1.3 Anatomical Simulation Software . . . . . . . . . . . . . . 1019.1.4 Validated Hand Model Construction . . . . . . . . . . . . 1029.2 Limitations and Future Work . . . . . . . . . . . . . . . . . . . . 103Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106Appendix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123A Anatomy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123A.1 Human Hand Anatomy . . . . . . . . . . . . . . . . . . . 123A.2 Index Finger Anatomy . . . . . . . . . . . . . . . . . . . 124A.3 Digit Anatomy Differences . . . . . . . . . . . . . . . . . 127xList of TablesTable 3.1 Notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27Table 4.1 Trajectory tracking errors. . . . . . . . . . . . . . . . . . . . . 47xiList of FiguresFigure 1 Index finger tendon structure . . . . . . . . . . . . . . . . . . xivFigure 1.1 Index finger tendon structure . . . . . . . . . . . . . . . . . . 3Figure 1.2 Anatomy of the finger . . . . . . . . . . . . . . . . . . . . . 5Figure 1.3 Control simulation results . . . . . . . . . . . . . . . . . . . 9Figure 3.1 Eulerian-on-Lagrangian tendons. . . . . . . . . . . . . . . . . 21Figure 3.2 Timestep improvements with selective quasistatics. . . . . . . 24Figure 3.3 Eulerian and Lagrangian coordinates of a strand. . . . . . . . 26Figure 3.4 Q1 node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30Figure 4.1 Components of the anatomy of the finger. . . . . . . . . . . . 35Figure 4.2 Skeletal muscle FL/FV . . . . . . . . . . . . . . . . . . . . . 38Figure 4.3 Muscle model. . . . . . . . . . . . . . . . . . . . . . . . . . 39Figure 4.4 Joint coupling - OL . . . . . . . . . . . . . . . . . . . . . . . 46Figure 4.5 Clinical conditions. . . . . . . . . . . . . . . . . . . . . . . . 47Figure 4.6 Circle tracking results . . . . . . . . . . . . . . . . . . . . . 48Figure 4.7 Circle tracking muscle activations . . . . . . . . . . . . . . . 49Figure 4.8 3D circle tracking . . . . . . . . . . . . . . . . . . . . . . . . 50Figure 4.9 Finger trajectory tracking . . . . . . . . . . . . . . . . . . . . 51Figure 5.1 Software components . . . . . . . . . . . . . . . . . . . . . . 53Figure 5.2 StrandsUI interface . . . . . . . . . . . . . . . . . . . . . . . 55Figure 5.3 Scene graph . . . . . . . . . . . . . . . . . . . . . . . . . . . 56Figure 5.4 Gathering control data . . . . . . . . . . . . . . . . . . . . . 59xiiFigure 5.5 Simple kNN controller for finger model. . . . . . . . . . . . . 61Figure 5.6 Types of strands . . . . . . . . . . . . . . . . . . . . . . . . . 63Figure 6.1 Reduced model of the extensor hood . . . . . . . . . . . . . . 66Figure 6.2 Transient nodes . . . . . . . . . . . . . . . . . . . . . . . . . 69Figure 6.3 Transient nodes . . . . . . . . . . . . . . . . . . . . . . . . . 71Figure 6.4 Obstacle set method . . . . . . . . . . . . . . . . . . . . . . 72Figure 6.5 Strain vs. joint angle . . . . . . . . . . . . . . . . . . . . . . 74Figure 7.1 Index finger tendon structure . . . . . . . . . . . . . . . . . . 76Figure 7.2 lumbrical - trajectory and angles . . . . . . . . . . . . . . . . 80Figure 7.3 Index finger model . . . . . . . . . . . . . . . . . . . . . . . 83Figure 7.4 MCP stiffness . . . . . . . . . . . . . . . . . . . . . . . . . . 83Figure 7.5 Weighted lumbrical muscle results. . . . . . . . . . . . . . . 87Figure 7.6 MCP damping impact . . . . . . . . . . . . . . . . . . . . . 88Figure 7.7 Activated lumbrical muscle . . . . . . . . . . . . . . . . . . 89Figure 8.1 Thumb tendon structure . . . . . . . . . . . . . . . . . . . . 92Figure 8.2 Thumb extension with and without adduction . . . . . . . . . 94Figure 8.3 Thumb flexion and adduction . . . . . . . . . . . . . . . . . . 95Figure 8.4 Tenodesis experiment . . . . . . . . . . . . . . . . . . . . . . 97Figure 8.5 Tenodesis simulation visualization . . . . . . . . . . . . . . . 98Figure A.1 Bones of the hand . . . . . . . . . . . . . . . . . . . . . . . . 123Figure A.2 Flexor pulleys . . . . . . . . . . . . . . . . . . . . . . . . . . 125Figure A.3 Extensor hood model . . . . . . . . . . . . . . . . . . . . . . 126xiiiGlossary(a) Sagittal view(b) Dorsal viewFigure 1: Schematic representation of the index finger musculotendon struc-ture.CS central slip, DA fibers inserting at the base of the middle phalanx.DA dorsal aponeurosis, the anatomical name for the extensor apparatus.DIP distal interphalangeal, joint between the middle phalanx and distal phalanx.DI dorsal interosseous, an intrinsic muscle with the primary function ofabduction.xivEDC extensor digitorum communis, the common extensor tendon for all fourfingers.EI extensor indicis, the extra extensor tendon of the index finger.EL extensor lateral, a set of tendon fibers connecting the EDC to the lateral band.FDP flexor digitorum profundus, the primary flexor muscle.FDS flexor digitorum superficialis, the secondary flexor muscle.IM interosseous medial, the branched fibers of the interosseous musclesconnecting medially for insertion at the CS.IP interphalangeal, PIP and DIP.LB lateral band of the DA.LUM lumbrical, intrinsic muscle originating from the FDP and combining intodorsal aponeurosis.MCP metacarpophalageal, joint between the metacarpal bone and proximalphalanx.OL oblique retinacular ligament, ligament originating on the palmar face of theproximal phalanx inserting into the dorsal base of the distal phalanx. CouplesPIP and DIP motion.PIP proximal interphalangeal, joint between the proximal phalanx and middlephalanx.PI palmar interosseous, an intrinsic muscle with the primary function ofadduction.TS terminal slip, DA fibers inserting at the base of the distal phalanx.xvAcknowledgmentsI would like to thank my research supervisors, Dr. Dinesh K. Pai and Dr. ShinjiroSueda, for their guidance, useful critique, and generosity with their time. I wouldalso like to thank the members of my supervisory committee, Dr. Michiel van dePanne and Dr. Uri Ascher, for their helpful comments about my research directionand this dissertation. Thanks to my examination committee members, Dr. ThomasOxland, Dr. William Evans, and Dr. James Wakeling, for their time and feedback tohelp improve this dissertation. I would like to thank AIST, CFI, ICICS, MITACS,NSERC, and the Canada Research Chairs Program for providing funding that mademy research possible.I am grateful to Dr. Mitsunori Tada, Dr. Naomichi Ogihara, and my colleagues atDigital Human Research Group at AIST for the opportunity to work with them andcollaborate with hand surgeons from Keio University Hospital, and for Dr. Tada’sinputs on my research. I am grateful to my collaborators, Susanne Bradley, MikhailFain, and Farzad Abdolhosseini, for all their help in discussion and implementationof our work. Debanga, Darcy, Heon, Jolande, Ye, and all other lab mates I have hadover the years have provided me with opportunities to discuss research ideas, andwith many wonderful memories. For that, I am very grateful.I’d be remiss not to mention by name the closest of my friends who haveshared this journey with me. My friends from high school days—Nimesh, Kartik,Aniruddha—have been a constant connection to my younger self. The closest offriends from my undergraduate days—Devendra, Ravi, Kuldeep—have always beenthere to celebrate the happy times with me, and to help me get through the not sopleasant ones. My backpacking buddies, Pramod and Anuja, have taken me oncountless unforgettable adventures in the Pacific Northwest. With my grad life cribxvibuddy, Anasuya, I shared the highs and lows of our respective grad school lives. Ialso made a number of great friends in graduate school, and have warm memorieswith each of them. I will remember the board games and snooker matches withMichael, the potluck dinners and kayaking trips with Deepak, Shreyas, Chantaland Jake, and the late night impromptu conversations with Sampoorna. I willalso remember all the merry days spent dancing, hiking, cooking, and eating withAndrew and Lynn in Tokyo. And there are also the many friends I have made in thedancing communities in Vancouver and Tokyo. All of these friends have been anintegral part of my life, and have a special place in my heart.Moving to a new country can be full of uncertainties. I have been lucky to havenot just my brother, but my mamaji-mamiji, Shiv Raj and Lalita Kathpalia, and mycousins Tanuj and Yatin Katpalia in close proximity. Visiting them made me forgethow far away I was from home.Most of all, I would like to thank my family, who have supported me all throughmy graduate studies as they have throughout my life. My maate-baapu, Indra andSurinder Pal Sachdeva, did not just support my decision to pursue a doctorate butcheered me on in the good times and reassured me through the tough ones. Withouttheir encouragement of my curiosity and their support, I could not have daredto dream of grad school. My bhaiya, Sushant Sachdeva, and bhabhi, PushkariniAgharkar, have provided me with inspiration and guidance in my academic pursuits,and advice and support whenever I needed it. My chachaji-chachiji, Suresh andSunita Sachdeva, and my adorable behena, Agrima Sachdeva, have also been in mycorner throughout grad school. They have all had enormous contributions towardsmy growth as a person, and have helped me face challenges I encountered on thisjourney, and I will forever be in their debt.xviiTo my family.Chapter 1IntroductionHuman hands are essential to mechanical interactions between humans and theirenvironment. With the complex mechanical interplay of tendons, ligaments, musclesand bones, the human hand has at least 20 degrees of freedom (DoFs) [Sancho-Bruet al., 2011]. The tremendous strength and dexterity displayed by the hand, rangingfrom very precise movements to generation of large forces, is unmatched despitethe various attempts in robotics to mimic their structure. This renders study of thehuman hand important to various disciplines.Robots that are meant to interact with humans, or to work with tools meantfor humans, or to aid humans in everyday environments would need to exhibitsuch dexterity. In computer graphics, realistic modelling of the hands is becomingmore and more important, especially with extensive virtual reality applications. Incomputer vision, such a model would aid monocular hand gesture recognition. Inthe study of hand prosthetics and pathomechanics, understanding the function ofthe various structures of the hand is key to improved rehabilitation. These areasof application form our motivation for the study of the function and control of thehuman hand via experimentally verified numerical simulation.1.1 Modelling the Human HandStudying the function of the various parts of the human body has been part of thedomains of medicine and anatomical studies, and the hand is no exception to this.1The early seminal works to understand the human hand were therefore made withexperimentation on cadaver hands [Landsmeer, 1949; Kaplan, 1945; Sunderland,1945]. In vitro experiments on cadaver hands provide opportunities to see themovement of various components of the hand and draw inferences about theirfunction. These inferences are often limited to discrete observations. For example,Landsmeer [1949, Pg 38] noted that flexion of the third DIP joint leads to slackeningof the central slip. Specialized devices have since been used by researchers to gathernumerical data on the material properties and functioning of the various componentsof the human hand assembly [Chao et al., 1989; Valero-Cuevas et al., 1998; Kamataet al., 2016]. This experimental approach is viable for some experiments, suchas measuring the varying moment arm of the extensor assembly on various joints.For some other aspects of the anatomy, however, such as muscle activation drivencontrol, cadaver hand experiments are not viable.There has been enormous interest in building anthropomorphic robotic handsover the years, with applications in prosthetics and robotic interactions with human-centric environments. It is, therefore, not surprising that roboticists have attempted tobuild anatomically detailed human hand models to study the functioning of the hand.Examples include the ACT hand [Weghe et al., 2004], the Utah/MIT hand [Jacobsenet al., 1986], and the Shadow Robot Hand [Kochan, 2005]. Building robotic modelsof the human hand puts our understanding of the anatomy of the hand to the test.Robotic models provide a platform for testing interaction of the physical hand modelwith the physical world. However, a robotics approach to biomechanics is expensiveand time consuming. When testing a different material for the tendon segments,switching joint models, testing pathomechanics, or modifying the actuation model,robotic hand models need to go through tedious and expensive hardware changes,making them short of ideal for studying the hand.Biomechanical simulation, in contrast to robotics, provides an inexpensiveway to test such inferences and validate our understanding of the human hand.Two-dimensional biomechanical models of the hand were first used to explain thefunction of particular components of the anatomy (e.g., [Spoor and Landsmeer,1976; Leijnse and Kalker, 1995]). These models are, by definition, limited toplanar studies, model only one finger at a time, and can only probe the functionof musculotendons in flexion-extension motions about various joints, among other2limitations. Hence, this treatment of finger mechanics leaves much to be desired.Three dimensional models have also been used for biomechanics of the hand [Chaoet al., 1976; Garcia-Elias et al., 1991a; Valero-Cuevas et al., 2007], but with essentialsimplifications reducing the usefulness of the results.To summarize, previous attempts at hand modelling are either too narrow in thescope of the phenomena they model, or are focused on dynamics with assumptionssuch as the moment arm of tendons across joints, or bow-stringing of tendons. Withthe use of better computational techniques for biomechanics, we can create robusthand models with significantly improved anatomical detail without forcing suchassumptions.1.2 Extensor Apparatus(a) Sagittal view(b) Dorsal viewFigure 1.1: Schematic representation of the index finger musculotendon struc-ture.The difficulty of understanding and modelling the functioning of mechanisms3of the finger lies in the complex structure and motion of the extensor apparatus.*The extensor apparatus provides routing and insertions for four different musclesof the human hand—three intrinsic (smaller muscle in the hand), and one extrinsic(originating in the forearm)†. It forms a complicated branched structure and exhibitssignificantly variable moment at the joints based on the configuration of the hand.A dorsal view of the extensor hood design is shown in Fig. 1.1. As noted bySunderland [1945], it is misleading to speak of isolated, individual and specificactions of the extensor digitorum, lumbrical and interossei muscles. The extensorhood appears to function as a well integrated unit in every movement of the digits.The simulation models developed so far for human hand dynamics simulations,including our first attempt described in Chapters 3 and 4, have significant short-comings in modelling the extensor apparatus of the human hand (see Section 4.2.1).Due to its complex structure, tendon force transmission within the extensor hoodand its contribution to the finger kinetics has not yet been computationally modelledin a manner which would allow deriving the motion of the finger joints from dynam-ics. Fok and Chou [2010], for example, do not model force transmission and useboundary constraints instead and hence do not simulate true dynamics. Leijnse andSpoor [2012] reverse engineer the extensor apparatus using only a 2 dimensionalmodel instead of a 3D model. And Valero-Cuevas et al. [2007] model an incompleteextensor hood without a LUM muscle or the OL.1.3 Research ObjectivesWe set out to build a computational model of the human hand which can be useful forsimulating anatomical observations of human hand movement, with applications in(a) improved hand pathology diagnosis, rehabilitation, and surgical repair, (b) betterhand simulation and tracking for computer graphics and VR applications, and(c) building better mechanical structure and control for anthropomorphic robotichands. For these applications, we break down our larger vision of a computationalmodel of the human hand into the following goals:*For description of the extensor apparatus, please see Section A.2.†The index finger has one extra muscle, namely the extensor indicis proprius, see the Glossary,and refer to Section A.34Figure 1.2: Anatomy of the finger [Fok and Chou, 2010]• Tendon Dynamics: Developing techniques for simulating tendon dynamicsin a rigid body simulation framework.• Control Strategies: Designing control strategies for a hand simulation modelfor fingertip trajectory control.• Anatomical Simulation Software: Building software for construction andsimulation of biomechanical models which can model the complexities of thehuman hand.• Validated Hand Model Construction: Constructing a hand model whichcan be validated with in vivo and in vitro experimental data.We introduce each of these objectives, the previous work in the area, and how weaddress the limitations of previous work.1.4 Tendon DynamicsWith increasing availability of computational power, the bottleneck to a compre-hensive biomechanical simulation framework for the human hand has been thelack of numerical techniques to handle the complete dynamics of a musculotendonsimulation. The human hand exhibits highly constrained configurations for tendons:5for example, the pulley structures constraining the flexors [Doyle, 1988], the originof the lumbrical muscle from the FDP (see Fig. 1.2), and the wrist [Kapandji, 2007,Pg 192]. Moreover, the tendon paths are not constant across varying configurationsof the joints [Deshpande et al., 2008]. Not only do these anatomical structuresexhibit axial movement along the length of the tendon or ligament, but often lateralmovement with respect to the bones, as is the case with the oblique retinacularligament and the extensor apparatus (Section A.2).Line-of-force (LoF) models have often been used for musculoskeletal simulationof the human hand [Albrecht et al., 2003; Tsang et al., 2005; Johnson et al., 2009].Various simulation libraries such as OpenSim [Delp et al., 2007] and Anybody[Damsgaard et al., 2006] allow building such models in a generic fashion. Valero-Cuevas et al. [2007] used an LoF model of the extensor hood, modelled to validatethe observed switching behaviour. Normative models for the extensor hood havealso been developed on these lines by Synek and Pahr [2016] and Dogadov et al.[2017] based on the works of An et al. [1983], Chao et al. [1989], and Valero-Cuevas et al. [1998]. These models simulate the muscles and tendons as kinematicpaths with the inertia of muscles and tendons lumped onto the bones in canonicalconfiguration. In some cases, there are assumptions made regarding the momentarms of tendons at the joints they cross. With these assumptions, the models fail totake into account the shape and routing of tendons accurately.AimOur aim is to develop a simulation for a musculotendon system with the followingfeatures:1. The model must incorporate the elastodynamics of stiff tendons and ligaments.2. It should simulate routing of tendons across bones through tendon sheathswith coupled dynamics.3. The resulting model should integrate well with a rigid body dynamics frame-work.Cubic B-spline curve based strands have been used by Sueda et al. [2008] to6model a tendon-driven human hand. While Sueda et al. [2008] developed a handdynamics model, the tendons in the model lose degrees of freedom in unintuitiveand unexpected ways due to implicit parameterization [Sueda, 2010]. Therefore,replacing the spline strands with another model is necessary. Sueda et al. [2011]provide another method for discretization of tendon-like strands using Eulerian-on-Lagrangian strands. However, since they model the elastodynamics along the tendonmaterial, we are forced to use very small timesteps (∼ 0.1 ms) when modellingstrands as stiff as tendons.With our tendon dynamics model, we address the limitations of previous modelsand simulate hand motion from tendon dynamics. To overcome the limitations of[Sueda et al., 2011], we define a selective quasistatics approach for modelling stiffligaments, discussed in Chapter 3. Moreover, unlike the elastic strands used by[Sueda et al., 2008] for tendons, we implement inextensible strands. By using ourapproach, we take into account the coupling between tendons and bones along thecomplete route of the tendon, and do so efficiently.1.5 Tendon-driven ControlOne could describe the hand as a system of articulated rigid bodies driven by tendonswith muscle actuation controlled by the central nervous system. Hand control may,therefore, be modelled with a hierarchy of control mechanisms [Valero-Cuevas,2005]. At the lowest level, one can describe the kinematics and dynamics of eachfinger, while on a higher level, one needs to consider hand postures and grasps.The computer graphics and haptics communities have been interested in devel-oping models for grasping and interaction with the environment. A large number ofsuch works rely on motion capture data [Bohg et al., 2014; Sahbani et al., 2012].Data driven methods of control for grasping, such as [Zhao et al., 2013], and othersuch hand animation methods are plagued by the lack of a detailed human handmodel and hence do not provide much insight into the internal dynamics of thehuman hand. The lack of a musculotendon model and any associated data fromhand tracking also prevents any applications from building a dynamics controllerfor an anatomically designed hand model. In robotics, the control has similarly beenfocused on robot hand grasps, particularly on optimum grasp determination. These7methods examine the placement of contacts on the surface of the grasped object, andconstruct the torques for the joints. To analyze the various grasping solutions forany object given a hand structure, numerous tools have been developed includingGraspit! [Miller and Allen, 2004]. Other such tools like HandGrasp [Walha et al.,2011] have been applied to the human hand too, e.g., [Miller et al., 2005].Underlying the higher level grasp style control often cited in robotics, lowerlevel control of the human hand requires neuromuscular control of the finger posture.The production of dynamic fingertip forces is an under-constrained problem becauseof musculotendon redundancy. Multiple sets of muscular activations could lead tothe same dynamic fingertip forces. Resolving this redundancy requires developmentof new tendon-driven control strategies.AimWe aim to develop techniques to perform musculoskeletal control. These techniquesshould be able to precisely control fingertip trajectory with muscle activations todrive tendons and bones.In pursuit of this objective, we developed two different control approaches forprecise muscle-driven fingertip control. In the process, we developed a numericallystable piece-wise linear muscle model based on physiological evidence reportedby Robinson et al. [1969].1.6 Anatomical Simulation SoftwareThe structural details of the human hand are not easy to incorporate in a compu-tational dynamics framework. Anatomically viable modelling of the human bodyrequires tools not easily available within Computer Aided Design (CAD) or anima-tion frameworks. Various toolkits have been developed for building biomechanicalmodels of various parts of the human body, e.g. Anybody by Damsgaard et al.[2006] and OpenSim by Delp et al. [2007]. One can model the human hand in theseframeworks but the tendons will only be treated in a kinematic manner, i.e., at eachsimulation step, the mechanical properties of the tendon such as the strain will becalculated to compute the resulting forces and torques on the bones and joints at8(a) Controlling the fingertip to trace a circle. (b) Reproducing fingertip trajectory usingour simulation framework.Figure 1.3: Controlling the finger tip.only the origin and insertion of the tendons. Due to the nature of tendon routing viapulleys (refer to Fig. 3.1 and Section A.2), tendons have significant impact on thedynamics of the bones they route across between the origin and insertion and cannotbe ignored. While the tendons do not exhibit high strain [Goldstein et al., 1987],the movement of the various sections of the extensor apparatus has been shown toaffect the moment exerted across joints by the actuators [Garcia-Elias et al., 1991a].AimWe aim to build a biomechanical simulation software framework to aid modelling,simulation, and tendon-driven control of the hand. Specifically, the software should1. provide a graphical interface for modelling musculotendons on bones congru-ent with our techniques created to fulfill the previous mentioned aims,2. allow fast iteration over biomechanical models, i.e., let the user quicklychange the model and observe the effects, and3. aid easy development of control strategies.By building our own custom biomechanical simulation software, we are not limitedby the numerical methods and elements of biomechanical simulation available in9these frameworks. We extended Autodesk Maya® with custom plugins to create thebiomechanical simulation visually, and amended the model based on observed sim-ulation behaviour. In addition, we developed a framework for creating experimentsand designing fingertip control strategies on the biomechanical simulation. In partic-ular, this would allow biomechanics researchers to create and share experiments andcontrol strategies, something that was lacking from related previous works [Suedaet al., 2008; Sueda, 2010; Synek and Pahr, 2016; Dogadov et al., 2017].1.7 Validated Hand Model ConstructionOne of the primary motivations to build, simulate, and control an anatomicallydetailed model for the human hand is the reduced cost of testing of hypotheses,tendon-based prosthetic models, and surgical repair methods. Currently, workingwith cadaver hands is the principal way to generate hypotheses regarding the func-tioning of the human hand, e.g., [Hauck, 1923; Landsmeer, 1949; Young and Rayan,2000; Arnet et al., 2013]. With a working and validated biomechanical model, onecould run tests with the computational model and then proceed with testing thehypothesis on a cadaver hand or live subjects. The tools created in the process couldbe applied to creating models of tendinous prosthetic or robotic models, and testingthe efficacy of surgical repair strategies for pathological conditions of the humanhand [Rettig, 2004].Models of the finger presented by Valero-Cuevas et al. [2007] to explain extensorapparatus dynamics, and by Leijnse and Spoor [2012] for simulation of IP jointcoupling represent some of the attempts at a model of parts of the hand to explainparticular clinically observed phenomena. However, neither of these simulate thetrajectory of the finger based on a biomechanical model for finger tendon dynamics.AimOur goal is to build a model of the hand with anatomical accuracy which can explainclinically observed phenomena as validation of the techniques for tendon dynamics.Specifically, we choose to model the function of the lumbrical muscle in modulatingfinger flexion as observed by Arnet et al. [2013] and Kamata et al. [2016]. The LUM10muscle plays a key role in increasing the reach of the finger during flexion, andconventional models are unable to simulate its effects.In the chapters that follow, we first describe the previous work related to thisthesis in Chapter 2. In Chapter 3 we develop the musculoskeletal dynamics tech-niques to model tendinous biomechanical systems. In Chapter 4 we design controltechniques for biomechanical simulations created with these techniques. Followingthis, our software framework for anatomical simulation is presented in Chapter 5.This software framework and the underlying techniques are validated with a modelof the lumbrical muscle in Chapter 7. While the lumbrical muscle was previ-ously simulated by Sueda [2010], our model uses inextensible tendons with tendonremeshing (discussed in Chapter 7), and adds the oblique retinacular ligament toimprove fidelity of the results.Lastly, in Chapter 8 we simulate a model of the human hand to show that thesetechniques apply not only to the finger, but also to the wrist and thumb, which aresignificant components of a comprehensive hand model.11Chapter 2Related Work2.1 Human Body SimulationsVarious biomechanical simulators have been made for many parts of the human body.Traditionally, computer graphics has utilized the simpler approach to modelling thehuman body using joint torques instead of musculotendon actuators. However, morerecently, sophisticated biomechanical models have been used to produce physicallyaccurate results.A technique to determine facial muscle activations given by Sifakis et al. [2005]has been utilized to simulate speech by Sifakis et al. [2006] using motion capturedspeech examples. Lee and Terzopoulos [2006] utilize a neural networks basedhierarchical controller on top of an anatomically consistent neck model to allowsimple neck movements. The whole upper body was modelled and controlled byLee et al. [2009] to generate a sequence of target poses. Full body musculoskeletalmodelling with the Anybody modelling system was presented by Andersen et al.[2013]. Furthermore this type of modelling was extended with two-way flesh-bonecoupling to develop a swimming controller by Si et al. [2014].For locomotion, early attempts were made to model the lower body by Donget al. [2002] and Komura et al. [2000]. More recently, Hill-type muscle models,described in Section 4.2.2, along with co-variance matrix adaptation (CMA) forobjective optimization to build locomotion controllers for humanoids and otherbiped characters [Lee et al., 2014; Geijtenbeek et al., 2013; Wang et al., 2012]. By12minimizing the square of the activation levels of muscles among other objectives,these models are able to produce human-like locomotion. Full body muscle-drivengait simulation has also been done using OpenSim by Rajagopal et al. [2016].Moreover, physics-based body animation has been personalized with data-driventechniques on captured 3D scans by Kadlecˇek et al. [2016].One significant difference between the parts of the human body modelled in thepapers mentioned above and the human hand is the complexity of tendon routing.It renders the straightforward application of these techniques to the human handinsufficient.2.2 Hand Motion CaptureIn both computer graphics and virtual reality, the modelling of human hands isimportant to animation. As a result, kinematic modelling of human hands hasbeen explored heavily. Most of this work has focused on geometry acquisition andretargeting of the whole hand [Kurihara and Miyata, 2004; Li et al., 2007; Huanget al., 2011]. With Kinect [Ren et al., 2011] and Leap Motion [Weichert et al., 2013]controllers, hand gesture recognition has been made easily accessible, but they arestill not sufficient for acquiring accurate hand configuration information. Usingsignificant pre-processing using multiple cameras, Han et al. [2018a] significantlyimprove realtime capture of the hand configuration with markers using a singlecamera. Beyond capturing hand gestures, Pham et al. [2018] estimate hand objectcontact forces from markerless visual tracking using recurrent neural networks(RNNs).More sophisticated retargeting techniques have been developed using physics-based simulation for grasp synthesis [Pollard and Zordan, 2005; Kry and Pai, 2006;Ye and Liu, 2012; Zhao et al., 2013]. In particular Pollard and Zordan [2005] extractjoint limits, and active and passive parameters of the hand model using motion datafor grasping motions, and Zhao et al. [2013] extend the work by data-driven motionsynthesis to complement physics based motion.132.3 Dynamics of the Human Hand2.3.1 Biomechanics StudiesSunderland [1945], Landsmeer [1949] and many others laid the groundwork tostudy the function of the various components of the hand. Smith et al. [1964] tookinspiration from these and proposed a planar mathematical model for the studyof the forces involved in a finger pinch. Many models utilize a 2D mathematicalrepresentation of the finger to analyze the function of the extensors, flexors, intrinsicor, lumbrical muscles in isolation or in combination with other musculotendons[Spoor and Landsmeer, 1976; Spoor, 1983; Leijnse and Kalker, 1995]. Hollisteret al. [1992] and Nanno et al. [2006] have studied the structure of the thumb. Garcia-Elias et al. [1991a] and Lee [2008] studied the extensor mechanism of the fingerwhich was also comprehensively reviewed by Landsmeer [1949]. The extensormechanism was also studied by Lee et al. [2008] to compute the effective staticmoment arms (ESMAs) for each of the tendons of the extensor apparatus. Anothercommon area of investigation is the coupling of IP joints, for example, the sagittalplane model presented by Buchner et al. [1988]. Leijnse and Spoor [2012] builta model of the extensor hood based on the previously reported IP coupling data[Leijnse et al., 2010]. Data from such experiments has also been used to constructnormative models [An et al., 1983; Chao et al., 1989].As one would expect, three dimensional biomechanical models have been usedto simulate multiple aspects of the hand. Chao et al. [1976] pursued the investigationinto hand function with a 3D model using free body analysis to produce a system ofequations to describe the dynamics. Sancho-Bru et al. [2001] and Wu et al. [2009b]present biomechanical models of the finger, while Valero-Cuevas et al. [2003] andWu et al. [2009a] built a 3D model for the thumb. Valero-Cuevas [2005] describedan integrative approach for neuromuscular control of a 3D finger model. The use ofsuch models is made for tendon transfer simulation, investigation of moment armsof tendons and the study of neuromuscular control to name a few areas. However,just like the 2D models discussed previously, these models have many evidentlimitations. Most of these works use kinematic simulation for tendons and almostall of them use simplified joint models.14The biomechanics community has had a strong interest in numerically validatingmodels (see Section 2.3.2) of the human hand. Valero-Cuevas et al. [2007] collecteddata on force transmission from the input of the extrinsic extensor tendon to theoutputs at terminal and central slip. Interphalangeal joint coupling data was gatheredby Leijnse et al. [2010], and the lumbrical muscle function in flexion of the handwas quantified by Arnet et al. [2013]; Kamata et al. [2016].2.3.2 Simulation ModelsSimulation models based on solid mechanics such as FEM models have often beenused for human body simulations, for example the modelling of complex musclegeometry using finite element models by Blemker and Delp [2005]. Lee et al.[2012] provide a detailed survey of such models. While these simulation modelshave been successfully applied for simulation of musculotendons of the knee, face,and skeletal muscles [Li et al., 1999; Sifakis et al., 2005; Kaufman et al., 2010],three dimensional models are not appropriate for simulation of the tendons of thehuman hand. The thin and anisotropic nature of the tendons would require manydisproportionately small elements if modelled in a volumetric fashion. Eulerian-on-Lagrangian (EoL) simulations can be utilized to mitigate this. Fan et al. [2013]simulate highly deformable objects using EoL methods. Li et al. [2013] use 2D EoLmethods for skin simulation, and more recently, EoL cloth simulation was proposedby Piddington et al. [2015]. Fan et al. [2014] applied it to simulation of activevolumetric muscles. We use the one dimensional formulation of EoL simulationto model strands as presented by Sueda et al. [2011] and apply it to simulate thehuman hand.2.3.3 Tendon-driven Robotic HandsSimulation techniques developed for human hands may also be applied to modellingtendon driven robotics. Two such projects are discussed below.Utah/MIT Dextrous Hand. The Utah/MIT Dextrous Hand [Jacobsen et al., 1986]was designed to be an anatomically-inspired robotic hand with the aim of creating adexterous hand as a research tool. With a 3 finger and one thumb design, this was15one of the first tendon driven hands. Control of the design was also attempted alongwith tactile sensing for the fingers [McCammon and Jacobsen, 1990].ACT hand. The ACT hand [Weghe et al., 2004], a physical robotic hand developedat the University of Washington, is designed keeping anatomical correctness in mind.The structure comes complete with an extensor hood model [Wilkinson et al., 2003],as discussed in Section A. A kinematic thumb model [Chang and Matsuoka, 2006]was also created for the ACT hand. Control of the ACT hand was implementedwith reduced dimensionality control by Malhotra et al. [2012]. This control methodinspired our strategy for controlling the human hand simulation [Sachdeva et al.,2015] described in Chapter 4.Shadow Dexterous Hand. One of the leading efforts to build an anthropomorphicmuscular hand is the Shadow Hand* [Kochan, 2005]. The Shadow Hand is designedto closely match the human hand with a total of 24-DoFs and human-like thumbmovements [Scharfe et al., 2012]. The UW hand [Xu et al., 2013] is a well-describeddrop-in replacement for the proprietary design of the Shadow Hand with a simulatorbuilt in MuJoCo [Todorov et al., 2012]. Details on the simulator are presented byErez et al. [2015].2.4 Musculoskeletal ControlThe control problem for the human hand can be defined at various levels. At thelowest level, one could define a reaching control problem, which is only concernedwith the end position of the fingertips. Covariance Matrix Adaptation (CMA)[Geijtenbeek et al., 2013; Wang et al., 2012], and Policy Improvement with PathIntegrals (PI2) [Rombokas et al., 2012] are some examples of reaching control.Higher-level grasping control has been successfully achieved by Röthling [2007],and Andrychowicz et al. [2018] provided a method for reinforcement learning (RL)based in-hand dexterous manipulation.A second formulation would be trajectory control. Controlling the trajectory oflinear systems is well studied, but these strategies do not apply to general non-linear*https://www.shadowrobot.com/products/dexterous-hand/16systems very well. Hence, the use of hierarchical control is common. In robotics,the Utah/MIT Dextrous Hand uses a hierarchical control strategy [Speeter, 1990]with joint based low level control, followed by higher level control specificationlanguage. More recently, Kumar et al. [2016] combined time-varying linear modelswith trajectory optimization for adaptive optimal control. For developing our controlmethods, the reduced dimensionality control strategy implemented by Malhotra et al.[2012] for the ACT hand combined with the linearized tendon control by Suedaet al. [2008] serve as inspiration for our hierarchical control strategy in Chapter 4.17Chapter 3Musculoskeletal DynamicsSince the simulator described here builds on a dynamics simulator described bySueda et al. [2011], the rigid body simulation framework used here is the same.We summarize some of the important mathematical concepts. We first describethe representation for rigid bodies, which are used to model bones, and the corre-sponding joint formulation. To augment the rigid body dynamics with the dynamicsof tendons, we address the discretization of tendons while handling constraintsimposed on the tendons by connective tissues. The dynamics of rigid bodies anddiscretized tendons are combined into a single combined quadratic optimizationproblem with linear constraints. In the latter half of this chapter, we review theconstruction of this optimization problem and the linear spaces involved.3.1 Rigid Bodies and JointsThe configuration of rigid body i is represented by the SE(3) transformation matrix,used with homogeneous coordinates. This is denoted by0iE =(0i Θ 0i p0 1)(3.1)where the subscript and superscript represent the different coordinate frames, i for arigid body, and 0 for the world. 0iE refers to the SE(3) transformation matrix fromthe frame of the rigid body i to the world frame. 0i Θ is the rotation matrix and 0i p18is the origin of the i frame expressed in world coordinates. In this way, the worldposition 0x may be computed from the local position ix in the rigid body frame usingthe following equation.0x = 0iEix (3.2)All rigid body velocities are specified in their own local frame of reference. Thecorresponding local spatial velocity vector for the rigid body i in its own frame ofreference is given by iφi ∈ R6 vector specified in twist coordinates.iφi =(iwiivi)(3.3)The mathematical basis for using an R6 representation for the velocity of an SE(3)transformation matrix is explained in detail by Murray et al. [1994].This frame velocity vector can be used to compute the velocity at a point x inframe i to world coordinates using0x˙ = 0i Θ(−[ix] I)iφi (3.4)= Γ (0iE,ix) iφiwhere the 3x6 matrix Γ (0iE, ix) = 0i Θ(−[ix] I)is called the material Jacobianmatrix. This Jacobian is instrumental in using reduced degrees of freedom forrepresenting points (such as strand nodes discussed in Section 3.2.1) constrained torigid bodies.The transform for the R6 twist vector iφi from frame i to world frame twistvector 0φi, corresponding to the rigid transform 0iE from Eq. (3.1), is given by theadjoint 0iAd of the coordinate transform.0φi = 0iAdiφi (3.5)0iAd =(0i Θ 0[0i p]0i Θ 0i Θ)(3.6)With all rigid velocities expressed in their own local coordinate frames, we19can drop the leading superscript and write the discretization of the Newton-Eulerequation for computing the velocity at the next timestep (φ 1)Mφ 1 = Mφ 0+hf0 (3.7)where M is the mass matrix, φ 0 is the velocity at the current timestep, f0 correspondsto the sum of Coriolis force and external forces on the body at the current timestepand h is the timestep size.Joint constraints between rigid bodies i and k at joint j are implemented using theadjoint formulation, from which we can derive various types of joints by purging therows corresponding to the non-fixed degrees of freedom. For example, for a hingejoint about the z-axis (axis number 2), the f ixed rows {0,1,3,4,5} correspondingto x and y rotational axes, and all translational axes would be selected. We canset the relative velocities in the joint frame, corresponding to the fixed joint axesδφ j[ f ixed,1:6 ] to 0. δφ j is computed asδφ j = jiAd φi− jkAd φk (3.8)= ( jiAd− jiAd ikAd)(φiφk)When the joint is at its lower limit, an inequality constraint δφ j > 0 can beapplied, which makes sure that the angular velocity of the joint is positive. Similarly,inequality constraint δφ j < 0 is applied for a joint at its upper limit.For a detailed description of the mathematics involved, please refer to [Sueda,2010].3.2 Discretization of TendonsOne of the main challenges in building a musculotendon simulator is the handlingof constraints on tendons. A tendon may move freely in the axial direction butis constrained throughout its length by a series of sheaths and pulleys. To handlethese complex constraints in the tendon network of the finger, we use the Eulerian-on-Lagrangian strands approach to strand discretization described by Sueda et al.[2011].20(a)A1 A2 A3 A4 A5 FDP (b)Figure 3.1: (a) Eulerian-on-Lagrangian strand illustration. Filled nodes areLagrangian nodes and empty nodes are Eulerian. (b) Flexor pulleysholding the tendon close to the bones.3.2.1 Eulerian-on-Lagrangian StrandsA strand is represented as a series of nodes, each of which contains both Lagrangianand Eulerian coordinates: (xi,si). The Lagrangian coordinates of a node, xi, are theworld position of the node, and the Eulerian coordinate, si, is the material coordinateof the strand at that node. See Fig. 3.1(a) for an illustration. The Lagrangiancoordinates encode the geometric path of the strand, and the Eulerian coordinateencodes the actual strand material at the nodes.The portion of the strand between any two consecutive nodes is called a segment.As an illustration, it may be helpful to visualize a textured elastic band passingthrough a node, with its two ends stretched apart and fixed (Fig. 3.1(a)). TheEulerian coordinate corresponds to the elastic band’s 1D texture coordinate, whichis tied to the material point along the elastic band. In a purely Lagrangian simulator,the strand material, visualized via the texture, is fixed to the nodes, as in Fig. 3.1(a)-top. If we move the middle node to the left (Fig. 3.1(a)-middle), then the material iscompressed in the left segment and stretched in the right segment with respect tothe previous configuration. If we relax the assumption that the material is fixed tothe nodes, as in Fig. 3.1(a)-bottom, using an Eulerian node, then the material willstart to flow from left to right. If the nodal positions (Lagrangian coordinates) arefixed, then we have a purely Eulerian simulation, a technique commonly used influid dynamics. In the constrained strands framework, both Lagrangian and Eulerian21coordinates are allowed to change.This separation of the geometric path and the strand material is critical for amusculotendon simulation because it decouples the routing constraints from thedynamics. Consider, for example, the pulleys holding the flexor tendons in place (A-1 through A-5 in Fig. 3.1(b)). With a purely Lagrangian approach, constraining thetendon to go through these pulleys would require the tendon to be discretized veryfinely near the start and end of each of the pulleys. With the Eulerian-on-Lagrangianapproach, we require just the right number of nodes, since we can place the nodesexactly at the start and end of the pulleys.We take advantage of the fact that tendons are always in close contact withother anatomical structures, and we know apriori where the routing constraints onthe tendons are going to be. We therefore do not need a general formulation ofcontact and its associated cost in simulation and modelling time. Collisions needto be handled only by the Lagrangian part of the discretization, which determinesthe kinematic path of the strand; the Eulerian part is oblivious to collision detectionand resolution. This is a major advantage for tendon simulation, since most of themovement is in the axial direction, which means that the Lagrangian part does notmove in space too much, and is often completely stationary with respect to theskeleton.3.2.2 Selective QuasistaticsTendons are very stiff, making them challenging to simulate numerically with areasonable timestep. One approach for incorporating such stiff forces is to approx-imate them by hard constraints, instead of elastic forces. It is necessary that weuse unilateral constraints and not bilateral constraints on the length of the strandfor inextensible strands. There are two problems with modelling a stiff tendon as abilaterally constrained strand. First, bilateral constraints prevent strand compression,which means that they can push as well as pull. Second, and more importantly,bilateral constraints can cause locking in some situations that are prevalent in handsimulation because of the complexity of the tendon network of the finger, whichcontains several branching and merging tendons. Modelling tendons as bilaterallyconstrained strands can cause this tendon network to turn into a reduced degrees-of-22freedom (DoF) system. With our formulation, we can easily model these loops, asshown in Fig. 4.4.Our solution is to use unilateral constraints for tendon modelling, and limit theuse of high stiffness strands to muscles where modelling the stiffness is necessaryfor correctness of simulation dynamics. When high stiffness strands are used, weuse a novel selective quasistatic discretization of the strand. The basic assumptionof the selective quasistatic discretization is that the strain propagates instantaneouslyalong the strand. This is similar to the idea proposed for discrete elastic rodsby Bergou et al. [2008], where the twist of the strand was assumed to propagateinstantaneously. Similarly, we propose to enforce all neighboring segments to havethe same strain, eliminating the Eulerian coordinates from the equations of motion.Note the selective nature of the elimination—even if we eliminate the Eulerian DoFsfrom the equations of motion, the strand is still dynamic since its Lagrangian DoFsare still present.Static condensation techniques, e.g., [DiMaio and Salcudean, 2002], removeall internal degrees of freedom using a quasistatic assumption. With our selectivequasistatic formulation, on the other hand, we gain the ability to insert LagrangianDoFs without increasing the number of Eulerian DoFs—a flexibility that is usefulfor routing tendons around bones.Another advantage of using selective quasistatics is improved conditioning,allowing larger time steps. As an illustration, imagine two consecutive nodescoming very close to each other at some finger posture (e.g., the small spacingbetween the A2 and A3 pulleys in Fig. 3.1(b)). Intuitively, without the quasistaticassumption, this would cause the Eulerian DoF of a node to affect only a smallamount of the mass of the strand located around these nodes. With selectivequasistatics, the inertia of the Eulerian DoFs of these small segments are distributedto their surrounding Lagrangian DoFs, improving the condition number of thesystem matrix, thus allowing us to take larger time steps. The comparison of thelargest time steps allowed as a function of the number of nodes is shown in Fig. 3.2.With quasistatic nodes, the maximum time step remains constant, whereas withoutthe quasistatic assumption, the time step decreases rapidly with the number ofnodes.To derive the Jacobian for the Eulerian DoFs, consider a strand consisting23Number of Nodes100 101Normalized Time Step10-1100EulerianQuasistaticFigure 3.2: An elastic strand stretching and sliding on a plank (from [Sachdevaet al., 2015]). The left end of the strand is fixed, and the right end isattached to the hanging box. The left-most node is a standard Lagrangiannode; the remaining are Eulerian nodes that allow the strand material toslide through them. The Lagrangian coordinates of all the nodes are fixedto the plank. The plot shows a log-log comparison of maximum timesteps (∆t) vs. the number of Eulerian nodes with and without selectivequasistatics. With selective quasistatics (red), the maximum ∆t remainsconstant (normalized to 1.0). Without selective quasistatics (blue), themaximum ∆t decreases rapidly.of n+ 1 segments as shown in Fig. 3.3. Let node 0 denote the first node, andn+ 1 denote the last node. Using the quasistatic strain assumption, we wantto eliminate s1 through sn by expressing them as a function of the LagrangianDoFs (x0, · · · ,xn+1) and the first and last Eulerian DoFs (s0,sn+1). The derivationstartswith the computation of the strain of segment i between nodes i and i+1:εi =li∆si−1 (3.9)where ∆si = si+1−si, ∆xi = xi+1−xi, and li = ‖∆xi‖. These quantities are computedfrom the coordinates of the two nodes of the ith segment. The quasistatic strainconstraint implies that the local strain values of all the segments are equal; i.e.,ε0 = · · ·= εi = εi+1 = · · ·= εn (3.10)Using the expression for strain given above and rearranging, we can rewrite εi = εi+124as−li+1si+(li+ li+1)si+1− lisi+2 = 0. (3.11)This holds for all neighboring segments i = 0, · · · ,n−1. Assembling these into alinear system, we obtain a tridiagonal matrix equation Ls = b, whereL =l0+ l1 −l0−l2 l1+ l2 −l1. . .−ln−1 ln−2+ ln−1 −ln−2−ln ln−1+ lns =s1s2...sn−1sn, b =l1s00...0ln−1sn+1.(3.12)L is (n×n), s is (n×1), and b is (n×1). Solving this equation gives the EulerianDoFs (s1, . . . ,sn) that make the strain the same throughout the strand given theLagrangian DoFs (x0, · · · ,xn+1) and the first and last Eulerian DoFs (s0,sn+1). Inorder to eliminate these DoFs, we also need the Jacobian matrix, which maps thevelocities of the remaining DoFs to the velocities of the eliminated DoFs. Taking thetime derivative of s = L−1b and using the identity for the derivative of the inverse,we arrive at s˙1...s˙n=−L−1∆S∆X︸ ︷︷ ︸Jx˙0...x˙n+1 . (3.13)(We assumed here that s0 and sn+1 are fixed. It is also possible to relax this as-sumption.) The resulting (n×3(n+2)) matrix, J, is the Jacobian for the eliminated25Figure 3.3: If we assume that the strain is the same throughout the strand, wecan eliminate the Eulerian coordinates s1 through sn given x0, . . . ,xn+1,s0, and sn+1.Eulerian coordinates. The blocks of J are composed of:∆S =−∆s1 ∆s0. . .−∆sn ∆sn−1 ,∆X =−∆x¯T0 ∆x¯T0. . .−∆x¯Tn ∆x¯Tn ,(3.14)where ∆x¯ = ∆x/‖∆x‖. ∆S is (n× (n+1)), and ∆X is ((n+1)×3(n+2)). J isof size (n×3(n+2)) and is the Jacobian for the eliminated Eulerian coordinates.Although J is dense, it is relatively small, and since all of the matrices involved arebanded, it is cheap to form.This Jacobian matrix, henceforth labelled JQ, can then be used to eliminatethe internal Eulerian DoFs from equations of motion (i.e., reduced coordinateapproach), or it can be used to define the equality constraint matrix between theinternal Eulerian DoFs and the remaining DoFs (i.e., maximal coordinate approach).In our implementation, we take the reduced approach to obtain a smaller but densersystem matrix.3.3 Reduced DynamicsThe previous section defined the treatment of Eulerian-on-Lagragian strands formodelling tendon dynamics within a rigid dynamics framework. We can now26Symbol DescriptionR RigidL LagrangianE EulerianSymbol DescriptionRL Rigid-LagrangianR ReducedN NodeTable 3.1: Notationdiscuss how all of this would come together into a comprehensive optimizationproblem to simulate the dynamics of the system. This section serves as a referencefor the mathematics of reduced spaces for the simulation. The final form of theequation leads to a denser, but much smaller matrix. The three spaces here are theF space, RL space and R space. Each of them and their relationships are describedin the following sections.3.3.1 Full SpaceFirst let us assume that all nodes are quasistatic nodes with 3 positional DoFs (calledQ3 nodes). Then the system velocity vector is given as a concatenation of threevelocities given as followsv =vRvLvE (3.15)where vR is the concatenation of rigid velocities, vL is the concatenation of La-grangian velocities of the EoL nodes, and vE is the vector for Eulerian velocities ofthe EoL nodes.vR =(. . .φi . . .)T(3.16)vL =(. . . x˙ j,k . . .)T(3.17)vE =(. . . s˙ j,k . . .)T(3.18)where φi is the 6 degrees-of-freedom (DoF) velocity vector for the rigid body i,x˙ j,k is the Lagrangian velocity for node k of strand j, ands˙ j,k is the Eulerian velocity for node k of strand j27Using the semi-implicit formulation of Baraff and Witkin [1998] for time, theequation to solve in full space is given by:M˜v(t+h) = Mv(t)+h f , (3.19)where M˜ is given byM˜ = (M+hD+h2K). (3.20)Here, h is the timestep, D is the damping matrix, K is the stiffness matrix, and f isthe force vector. The size of v is ((6nR+4nN)×1) where nR is the number of rigidbodies and nN is the total number of nodes.3.3.2 RL SpaceUsing the quasistatic assumption, vE can be completely determined by vL, givingEq. (3.21). This generates a velocity vector of size ((6nR+3nN)×1). This velocityvector, named vRL, contains the velocities of the rigid bodies and Lagrangian DoFsfor the nodes.vE = JQvL (3.21)vRL =(vRvL)(3.22)The full velocity v can be computed as a function of vRL using equationEq. (3.23)v = JRL vRL, (3.23a)JRL =IR 00 IL0 JQ . (3.23b)JRL is of size ((6nR+4nN)× (6nR+3nN)) and IR is of size (6nR×6nR). IL isof size (3nN×3nN).28JQ is a block diagonal matrix of the formJQ =. . . 0JQ j0. . . (3.24)For strand j with n j nodes, JQ j is the Jacobian from x˙ j to s˙ j of size (n j×3n j), i.e.,s˙ j = JQ j x˙ j. We can use the Jacobian JRL to “reduce” Eq. (3.19) to RL space, asfollows:M˜JRLvRL(t+h) = (Mv(t)+h f ). (3.25)3.3.3 R SpaceTo change Q3 nodes to reduced nodes, which may be Q0, Q1, Q2 or Q3, we canapply a similar transformation, to get the reduced velocity. The reduced velocity forx˙, is given by ˙˜x,. All reduced node velocities combined form vr. Appending vr tothe rigid velocities vR gives us the reduced space velocity vR.vR =(vRvr)(3.26)where vR is of size 6nR +∑ j∑k(n j,k) where n j,k is the size of ˙˜x j,k, which is thereduced velocity of the node k of strand j. n j,k is 0 for Q0 nodes, 1 for Q1, 2 for Q2and 3 for Q3 nodes, and similarly for L nodes.vRL = JR vR (3.27)=(I 0Γ Ω)vR (3.28)Γ is formed using the material Jacobian (see Eq. (3.4)) of the frame of the rigid bodyreference b j,k, and Ω is constructed by concatenating Ω j,k. Together they transformthe velocity φ(b j,k) for b j,k and reduced node velocity ˙˜x j,k for node k of strand j tox˙ j,k.29Γ=. . .. . . Γ j,k. . .. . . (3.29)Ω=. . .Ω j,k. . . (3.30)x˙ j,k =Ω j,k ˙˜x j,k +Γ j,kφ(b j,k) (3.31)where Ω j,k is the function that maps corresponding to the curve or the surface.3.3.4 Example: Circle Q1 NodeTo illustrate the R space treatment of Q-nodes with different DoFs, we presentthe formulation for a Q1 node on a circle. Q1 nodes have one effective degree offreedom and are used prominently to allow motion along the surface of the bone,constrained to a curve. We provide the important results for Q1 nodes as an examplefor how nodes of different degrees of freedom may be implemented.bxCθnFigure 3.4: Q1 node at x fixed to a circle C on a rigid body b. Note thatthe circle’s axis is in a generic direction, and not co-incident with thelongitudinal axis of the cylinder.In this section, we use the case illustrated in Fig. 3.4. We consider the case of aQ1 node n constrained to a circle around a rigid body b. The position of the node isgiven by a single parameter θn along with the local position of the node with respect30to the frame for b. The velocity of this node can be given by 2 steps.1. Find the velocity of the centre vc of the circle C which is given byvc = Γbφb (3.32)Γb =Θb([ln]T I)(3.33)2. Find the velocity of the node x˙ on the circle given vcx˙ = vc+ω × r (3.34)= vc+ θ˙n((lˆ× rx cosθn)+(lˆ× ry sinθn)) (3.35)Ωn = (lˆn× rx cosθn)+(lˆn× ry sinθn) (3.36)Defining Ωn as in equation Eq. (3.36), we can compute velocity of the node asfollows.x˙ = Γbφb+Ωn θ˙n (3.37)3.3.5 Final Reduced FormThe final reduced dynamics equation takes a form similar to equation Eq. (3.19) andis given byJTRLM˜JRLJRvR(t+h) = JTRL(Mv(t)+h f ) (3.38)=⇒ M˜RvR(t+h) = JTRL(Mv(t)+h f ) (3.39)Eq. (3.38) represents the reduced dynamics in the absence of constraints. Note thatthe complete derivation of this equation would involve the quadratic velocity vectorsince the derivative of the Jacobian J˙ is non-zero. We choose to ignore these effects.Refer to [Shabana, 2005] for further details. With joints, joint limits and tendoninextensibility, we need to incorporate equality and inequality constraints into thedynamics solver.31Using the velocity formulation of Gauss’s principle of least constraint [Redonet al., 2002, eq. 6], we can optimize the velocity for the next timestep q˙1 given thecurrent velocity q˙0, force applied f , timestep h, and the equality, and inequalityconstraint Jacobians (Geq, Gi respectively) as follows.q˙1 = argminq˙||q˙− (q˙0+hM˜−1R f )||M˜Rs.t. Giq˙ ≤ giGeqq˙ = geq(3.40)The computed velocity q˙1 is then integrated over the timestep h to get the updatedgeneralized position vector q1 using a forward Euler step.q1 = q0+hq˙1 (3.41)Since the constraints are implemented in velocity space, the accumulation ofnumerical integration errors may cause the position constraints to drift. Let thesubscript ia denote the active set of the inequality constraints. We can denote theactive constraint matrix with Ga = [ GTia GTeq ]T and the corresponding positionaldrift ∆ga = [ ∆gTia ∆gTeq ]T . A post-stabilization step [Ascher et al., 1994; Ascher,1997; Cline and Pai, 2003] for this active constraint set is performed after integratingthe velocity. (M GTaGa 0)(s∆qλ)=(0−∆ga)(3.42)The position drift is now corrected with s∆q: sq1 = q1 + s∆q, where sq1 is thestabilized position for the next timestep.32Chapter 4Modelling and ControlFrameworkThe tendons of the hand and other biomechanical systems form a complex networkof sheaths, pulleys, and branches. By modelling these anatomical structures, we canobtain realistic simulations of coordination and dynamics that were previously notpossible. In this chapter, we build on the Eulerian-on-Lagrangian tendon dynamicswith the selective quasistatic formulation described in Chapter 3 towards this goal.Moreover, we introduce two control methods for biomechanical systems: first, ageneral-purpose learning-based approach requiring no previous system knowledge,and a second approach using data extracted from the simulator. We use variousexamples to compare the performance of these controllers.4.1 IntroductionAlthough simulations of hands and grasp have consistently received attention inthe graphics community, modelling and simulating the dynamics of the complextendon network of the hand has remained relatively unexplored. Much of theprevious simulation techniques for the hand have been based on rigid links witheither joint torques [Pollard and Zordan, 2005; Kry and Pai, 2006; Liu, 2009] orline-of-force muscles [Albrecht et al., 2003; Tsang et al., 2005; Johnson et al., 2009].In the real hand, however, the motion of the digital phalanges are driven primarily33by muscles originating in the forearm acting through tendons passing through acomplex network of sheaths and pulleys [Valero-Cuevas et al., 2007]. This designhas an important effect on the compliance and coupling of the fingers.In our approach, we model each tendon as a physical primitive rather than usingjoint torques or moment arms, as described in Chapter 3. This is advantageousbecause it allows us to properly model the complexity of the tendon network, whichwe believe is important for obtaining realistic motions. As an added bonus, thebiomechanical modelling of tendons and ligaments can give us proper joint couplingfor free. As a simple example, let us consider the coordinated motion of the joints,shown in Fig. 4.4. The two distal joints of the finger (PIP and DIP) flex andextend in a coordinated fashion not because of the synchronized activation signalscomputed by the brain, but because of the arrangement of tendons and ligaments inthe finger. In our simulator, pulling on a single tendon (flexor digitorum profundus)achieves this result, whereas with a torque-based approach, we would need tomanually coordinate the torques at these two joints. Another example is in thecoupling between the extensor tendons in the back of the hand. Although thislack of complete independence between the fingers is partly due to the neuralcontrol, mechanical coupling has been hypothesized as having a significant role[Lang and Schieber, 2004]. We are also able to simulate hand deformities, whichhave applications not only in surgical planning and medicine, but also in computergraphics (Figs. Fig. 4.5(a) and Fig. 4.5(b)). Some virtual character designs arebased on deformities, injuries, or just the reduced functionality due to aging, andthe ability to procedurally produce anatomically-based abnormal characters mayprove useful, since obtaining real-world data of such characters can be a challenge.Contributions Our contributions to the simulation and control of the internal dy-namics of the human hand build on the methods discussed in Chapter 3. We addressthree main issues that are especially important for biomechanical simulation. First,by applying the Eulerian-on-Lagrangian discretization, discussed in Section 3.2.1,of the tendon strand, we greatly simplify the contact handling between tendonsand bones. Second, we develop a new formulation to deal with highly stiff strands,by assuming that strain and stress propagate instantaneously through the strand,allowing us to use large time steps even for stiff tendons (see Section 3.2.2). Third,34we introduce and assess two methods for control of these systems (Section 4.3.2,Section 4.3.3), which allow precise fingertip trajectory control.4.2 Modelling the FingerThe tendon topology of the finger is prohibitively complex for controlled dynamicssimulations. To mitigate this, we design a simplified anatomy of the finger. We thenaddress the muscle model, followed by two control methods for musculotendoncontrol of the finger.DIP PIP MCP Distal Middle Proximal Metacarpal FDP FDS EDC DI/PI LUM OL ExtLat IntMed Sagittal View Dorsal View EDC DI LUM OL IntMed ExtLat PI Figure 4.1: Components of the anatomy of the finger.4.2.1 Finger AnatomyWhile the detailed anatomy of the finger has been discussed in Section A, webriefly describe here the simplified model used for the index finger of the right handas a representative model. Each finger consists of the 4 bones: metacarpal, theproximal phalanx, the middle phalanx, and the distal phalanx. While the MCP jointis modelled as a universal joint, with 2 DoFs, the PIP and DIP have been modelled asrevolute joints with 1 DoF.The three long tendons are extensor digitorum communis (EDC), flexor digitorumsuperficialis (FDS), and flexor digitorum profundus (FDP). These tendons originate35from outside the hand, at the forearm, and are hence called “extrinsic.” The three“intrinsic” musculotendons are the lumbrical (LUM), dorsal interosseous (DI), andpalmar interosseous (PI). Each finger has a slightly different insertion arrangementof these three intrinsic musculotendons. The tendons may also be divided into twogroups based on their functionality and branching. FDP and FDS form the majorflexors of the finger, while EDC, LUM, PI, and DI combine to form the extensorapparatus or dorsal aponeurosis.*To deal with the complexity of finger tendons and build effective control, somesimplifications were applied to the tendon topology. In case of the flexors, whilethe FDS functions independently, the LUM tendon is supposed to originate at FDPbefore merging into the extensor apparatus. We decouple the LUM tendon and FDPfor the purposes of this model, leading to an independent FDP tendon.The extensor apparatus of the finger has been divided into 4 different functionalstrands. The primary extensor tendon, or the EDC, forms the thick strand in themiddle providing the major force for finger extension. The muscle for the EDCoriginates behind the wrist (on the forearm, which is considered fixed here). TheEDC tendon insertion lies on the proximal end on the dorsal side of the distalphalanx. Supplementing the EDC in extension at the MCP are the intrinsic tendonsDI and PI. In our model, the DI and PI insert at the proximal phalanx, instead ofbeing part of the collective extensor apparatus (see Section 4.2.1). To add to this is alumbrical muscle on the lateral side of the finger. It winds laterally from the palmarside of the metacarpal bone, where it originates, to insert into the dorsal side of thedistal phalanx with the EDC. Note that the lumbrical muscle is modelled withoutcoupling to the FDP. The oblique retinacular ligament (OL), which spans the PIPand DIP joints, helps synchronize these two joints when the FDP is pulled. Thisensures interphalangeal coupling, as described in Appendix A.2. The two crossingtendons, extensor lateral (ExtLat) and interosseous medial (IntMed) transfer tensionfrom extrinsic extensors and intrinsic extensors to the PIP and DIP. Note that ExtLatand IntMed were only used for forward simulations in some examples, and not in thecontrol demos.We can also simplify the construction of the complete hand model by using*This is a simplification for the purposes of providing a concise explanation. LUM muscle, forexample, is not strictly speaking an extensor, or flexor.36the arrangement in the index finger for all fingers and omitting the symmetriccomponents on the ulnar side.4.2.2 Muscle ModelMuscles are elastic non-linear visco-elastic solids capable of producing active force.Physically plausible actuation of tendons would require the use of a well-establishedmuscle model. The functioning and modelling of muscular force generation, how-ever, is an active area of research and a decisive model has not been convergedupon.The most commonly accepted muscle model is the Hill-Zajac muscle model[Zajac, 1989]. The force exerted by a muscle is modelled as the sum of passiveand active forces: f = fP+ fA, usually depicted as the muscle’s passive and activeforce-length (FL) curves. The passive FL curve is a monotonically increasingfunction modelled as a hyperelastic material model, and the active FL curve is aconcave function obtained from physiological experiments, starting with the classicexperiments by Hill [1938]. When the muscle is activated, the active FL curve isscaled by the activation level of the muscle, and the active force is computed withEq. (4.1) as a function of the activation level of the muscle (a), the current strain (ε)and the current strain rate (ε˙) .fA = a · f0 ·FL(ε) ·FV (ε˙) (4.1)The Force-Length (FL) and Force-Velocity (FV ) curve for the muscle aremeasured empirically. For the FL curve, this is done by stretching the muscleisometrically to different lengths, and then measuring the tension for the passiveforce. For the total active force, the tension is measured again after activation at thegiven length. The typical FL and FV curve can be found in Fig. 4.2. The resultingtotal force, shown in Fig. 4.3(left), contains a region of negative stiffness, whichmeans that as an active muscle is stretched, its force output decreases. Numerically,the negative slope manifests itself as instabilities in simulations, since any pertur-bation away from equilibrium is magnified by the negative force. In the presentcontext, we propose a simpler lumped muscle model, which relates the force atthe origin of the tendon to the tendon excursion (i.e., displacement of the tendon37(a) (b)Figure 4.2: The typical force-length and force-velocity relationship of theskeletal muscleorigin). Such lumped models are widely used in biomechanics, but our model differssignificantly from the standard Hill-Zajac model [Zajac, 1989], and may be moreuseful in applications. We use a simple piecewise linear model that does not sufferfrom these difficulties; more detailed, non-linear constitutive models may also beused, depending on the application. We model the total muscle force as a piecewiselinear function that is shifted to the left when activated. Fig. 4.3(middle & right)shows our muscle model. The passive FL curve is simply the unshifted curve. Whenthe muscle is activated, it pulls on the tendon even at zero excursion (i.e., in theisometric condition). Finally, because the function is always positive, the musclenever pushes back on the tendon.Even though this model is simple and robust, it is also potentially more realisticthan the standard model as a constitutive model of controlled muscles. The problemwith the direct application of Hill’s widely misunderstood FL curve is that it is nota constitutive model of active muscle; rather, it is a depiction of maximum forcewhen isometrically activated at different lengths. It is now well known that muscleactivation and stretch do not commute. Unlike the predictions of the standard model,it has been shown experimentally that if a muscle is activated first and then stretched,the resulting force does not decrease [Epstein and Herzog, 1998]. Moreover, thebehaviour of entire muscles during stretch is mediated by a complex neural signalbased on feedback from muscle proprioceptors and changes in fiber recruitment.38a	  FL	  0	   𝜀	  FL	  0	   𝜀	  0	  FL	  𝜀	  a	  Figure 4.3: (left) A typical FL curve of an activated muscle, depicting forceas a function of tendon excursion. (middle) Piece-wise linear musclemodel. The muscle exerts no force if the excursion is non-positive. Theforce increases linearly otherwise. (right) When the muscle is activated,the functions are shifted to the left.There are very few studies of how muscles behave in vivo, especially in humans. Astriking exception is the classic paper of Robinson et al. [1969] which measuredFL curves of active human eye muscles in vivo. Fig. 2 in that paper is closer toours than to the standard model. Our model needs further investigation, but there isphysiological support for using it.4.3 ControlIn this section, we present our controller framework and discuss two methods usedto learn the necessary control parameters. Section 4.3.2 describes a general learningmethod, to which we will refer henceforth as “black box control,” that uses noprior system knowledge and is suitable for control of a “black box” simulator.We then present in Section 4.3.3 a method—denoted as “white box control”—forextracting the relevant parameters for control of a biomechanical system directlyfrom the simulator. We compare these methods to explore how much we benefitfrom exploiting the inner workings of the system we are attempting to control, andto assess how a general-purpose learning method performs compared to a methodwith detailed prior system knowledge.394.3.1 Controller AlgorithmSince our plant is non-linear and input-constrained, we employed a hierarchicalcontrol approach, modelling the plant as a simple unconstrained system on a higher(kinematic) level, and an input-constrained locally-linear system at the lower (muscleactivation) level. For concreteness, we consider the index finger, but the method ismore general.At the high level, we model the fingertip motion as simple controlled dynamicalsystem:q˙t+1 = q˙t + vt (4.2)where qt is a (3×1) task descriptor variable (in this case, the fingertip location) attime t, and vt are the kinematic controls.The kinematic controls vt are defined by:vt := vp+ vb. (4.3)Here vp = vp(qt , t) is the (3×1) passive dynamics term, the computation of whichis described in Section 4.3.2; for a complete derivation of the equations of motion,we refer the reader to our supplementary material. vp captures the significantnonlinearities in dynamics, and allows the high level controller to assume thesimpler form of Eq. (4.2). vb = vb(qt , t) is the (3×1) feedback control term fortracking the reference trajectory. It is computed by:vb = KP(qr−qt)+KD(q˙r− q˙t). (4.4)KP and KD are scalar gains, and qr is the target configuration at a given timestep.In turn, the low-level activation controller transforms kinematic controls vtcomputed by the high-level controller to activations ut at each timestep t using theformulation of [Sueda et al., 2008]:ut = argminuα||R(qt)u− vt ||2+β ||u||2+ γ||u−ut−1||2s.t. 0≤ u≤ 1(4.5)where α,β , and γ are blending weights, and R(qt) is the activation-velocity matrix40(AVM) described below. The first term implements the kinematic controls, thesecond term penalizes large activations, and the third penalizes large changes inactivations between timesteps. We scale the parameters such that α+β + γ = 1.Setting α = 1 and β = γ = 0 corresponds to no smoothing. Thus, the parameters ofthe controller are the gains KP and KD, R(qt), the passive dynamics term vp(qt , t),and the smoothing terms α,β , and γ .This model is similar to those used by Matsuoka and co-workers; it extends themodel of Malhotra et al. [2012] by including passive dynamics compensation andby making the matrix R configuration-dependent. We used the fingertip location asa choice of the configuration variable as by Sueda et al. [2008]. Unlike joint angles[Deshpande et al., 2013] or tendon excursions [Malhotra et al., 2012], fingertipposition does not fully describe the finger configuration. There is, however, somephysiological basis for this choice of configuration variable as evidence exists thathumans do use fingertip position directly for reaching movements [Bédard andSanes, 2009].4.3.2 Estimating Black Box Controller ParametersPD gains and smoothing. The PD gains were empirically selected to be KP =0.23 2∆t , KD = 0.95. The factor2∆t is used for bringing the position and velocity termsin Eq. (4.4) to a common scale, and is derived from the observation that the absolutevalue of the velocity is ∆t2 times larger than the absolute value of the change inposition if the body is constantly accelerated for time ∆t. The smoothing parametersare also chosen empirically, though we find that the unsmoothed controllers worknearly as well as the best smoothed controllers (see Section 4.4). Both the gains andsmoothing parameters are chosen from a user-specified set of possible parametervalues via automatic selection of the parameters which give the best performanceon a trial task.Activation-Velocity Matrix. The matrix R(qt) is estimated at N = 60 configurationsq and approximated at a new point by a distance-weighted average. Because ofthis choice of approximation method, our controller can be seen as a linearlyblended composition of controllers with different matrices R, which are valid in the41neighborhood of the corresponding configurations q [Burridge et al., 1999].The data were collected with an extension of the self-identification method[Malhotra et al., 2012]. At each of the N stable configurations q (chosen by applyinga constant, random activation for long enough to let the system stabilize), the passivedynamics are computed by applying zero activations for one timestep, and recordingthe resulting velocity q˙0. The ith column of R(q) is estimated by fully activating thecorresponding muscle ui for one timestep at the configuration q, and subtracting q˙0from the resulting velocity q˙i.Passive dynamics term. Our passive dynamics compensation term is related toequilibrium point control [Shadmehr, 1998]. We define an equilibrium point as apair of state and control {qeq,ueq} such that qeq = f (qeq,ueq). That is, the systemwith dynamics qt+1 = f (qt ,u(qt , t)) does not move. Unlike traditional equilibriumpoint control we use an equilibrium point to record the configuration-dependentpart of the passive dynamics term; we ignore the velocity-dependent terms in theblack box controller since these are harder to estimate.We gathered training data by sampling n≈ 39000 points on a uniform grid inthe activation space. Each sampled point became an activation that we applied to theplant until it reached a stable configuration. The resulting configuration was then anequilibrium point q(i) corresponding to the applied activation u(i), where the notationu(i),q(i) represents the ith training data point. To estimate the equilibrium activationueq corresponding to some target position qr, we performed an approximate m-nearest-neighbour search of the points q in the training data. Then, at a state qtwhen we are attempting to reach a target qr, we set the passive dynamics term tovp = R(qt)ueq.To improve the efficiency of the m-nearest-neighbour search, we used a locality-sensitive hashing (LSH) algorithm [Indyk and Motwani, 1998]. The advantage ofthis approach is that its space and query time are both linear in the number of pointsto search, compared to other current nearest-neighbour algorithms which have eitherspace or query time that is exponential in the dimension of the data [Andoni andIndyk, 2008]. These algorithms are based on the existence of locality-sensitivehashing functions, which possess the properties that, for any two points p,r ∈ Rd :1. If ||p− r|| ≤ D then Pr[h(r) = h(p)]≤ P1422. If ||p− r|| ≥ cD then Pr[h(r) = h(p)]≥ P2where P1 > P2 and c≥ 1.We can concatenate several LSH functions to amplify the gap between P1and P2. Specifically, for parameters k and L, we choose L functions g j(r) =(h1, j(r), ...,hk, j(r)) where the hash functions hl, j(1 ≤ l ≤ k,1 ≤ j ≤ L) are cho-sen at random from a family of LSH functions. We then construct a hash tablestructure by placing each point qi from the input set into the bucket g j(p), j= 1, ...,L.To process a query point q, we scan through the buckets g1(q), ...,gL(q) and retrievethe points stored in them. For each retrieved point, we compute the distance betweenit and q and report the m closest points.For our purposes, the hash functions used consist of projecting a point onto arandom 1-dimensional line in Rd , which is then partitioned into uniform segments.The bucket into which a particular point is hashed corresponds to the index ofthe segment containing it. For Euclidean distances, these functions are locality-sensitive; for proof of this, we refer the reader to [Andoni and Indyk, 2008]. Weset the algorithm parameters to k = 20,L = 30 and m = 1, as we found that theseparameters were most effective in predicting activations leading to a given target.In particular, experiments using local polynomial regression models with differentvalues of m did not yield improved accuracy over m = 1. We speculate that this isdue to sparsity in the training data, and the non-linearity of the mapping betweenactivation and position space.4.3.3 Extracting White Box Controller ParametersThe Activation Velocity Matrix, R(qt), can be extracted directly from the simulator,similar to the strategy used by Sueda et al. [2008]. The forward simulator solves thevelocity-level Karush-Kuhn-Tucker (KKT) system:(M GTG 0)(q˙t+1λ)=(f (q˙t ,qt)+ fA(qt)0), (4.6)where λ is the vector of Lagrange multipliers and fA(qt) is the active impulse.Note that qt in Eq. (4.6) need not be the same as in Eq. (4.2), but for simplicity ofexposition we will use the same notation in this section. The impulse on the right43hand side is separated into two parts: impulse due to muscle activations (2nd term)and impulse due to all other forces (1st term). We can rewrite Eq. (4.6) asM˜ ˜˙qt+1 = f˜ (q˙t ,qt)+ f˜A(qt), (4.7)where each quantity represents the corresponding block matrix/vector. In partic-ular, ˜˙qt+1 is the concatenation of the next velocities and the Lagrange multipliers(constraint force magnitudes).f˜A(qt) can be further decomposed into a matrix vector product, f˜A(qt) = A˜(qt)ut ,where ut is the vector of muscle activations, and A˜(qt) is the matrix that maps muscleactivations to impulses. Here, we assumed that the impulses are linear in muscleactivations, a common assumption for many muscle models [Zajac, 1989]. Thisallows us to write˜˙qt+1 = M˜−1 f˜ (q˙t ,qt)+ M˜−1A˜(qt)ut . (4.8)This gives us an affine map from muscle activations to resulting system velocities.For most tasks, we are interested in just some of the velocities rather than thewhole system velocity. We therefore apply an extractor matrix Γ: q˙t+1 = Γ ˜˙qt+1.Applying this matrix to both sides and adding and subtracting q˙t from the right side,we obtain an equation in the form of Eq. (4.2):q˙t+1 = q˙t +(vp+R(qt)ut), (4.9)where vp = ΓM˜−1 f˜ (q˙t ,qt)− q˙t is the passive part of the kinematic controls andR(qt) = ΓM˜−1A˜(qt) is the AVM.4.3.4 Comparison of White Box and Black Box MethodsBecause the black box control method requires no prior knowledge of the systemwe want to control, it is well-suited for problems in which we lack access to thesystem’s internal workings. When the system’s inner workings are known, as isthe case with the index finger simulator, we can generally achieve better resultswith some variant of the white box control method, as this will yield a much betterapproximation of the system dynamics. The disadvantage of white box methods,44apart from their inapplicability in the absence of significant prior knowledge, is indevelopment time: as there is no ‘one-size-fits-all’ method, this approach must betailor-made for the system we wish to control. This can require significantly morework than general-purpose black box methods.In this paper, we compare the white box and black box methods for control of theindex finger. The black box method suffers from the sparsity of the training data; itestimates the AVM based on values observed at discrete points in the configurationspace, in contrast to the white box method, which extracts the AVM from thesimulator at each timestep. The principal disadvantage of our white box method isthe failure of the extracted AVMs to take joint limits into account. Our simulatordeals with joint limit constraints separately from the AVM calculations; thus, theAVM values extracted near joint limits tend to imply that applying certain activationswill lead to unreachable configurations. This can lead to some unexpected behaviournear the joint limits. The black box controller does not suffer from this particulardifficulty, as all AVM estimates are obtained empirically and thus take positionalconstraints into account.4.4 ResultsWe implemented our system in C++. Simulations were run on a commodity PCwith an Intel Core i5 3570 processor and 16GB of memory. The bone geometrywas obtained from a CT scan and reconstructed using custom software. The jointaxes were obtained from motion-capture data, and the tendon paths were createdmanually with a 3D modelling software, based on standard textbook models in theliterature [Kapandji, 2007]. We did not render the skin, but it should be relativelyeasy to augment existing work on skinning to add both cutaneous and subcutaneousmotion [Sueda et al., 2008; McAdams et al., 2011; Li et al., 2013] for rendering.For the index finger plant, the computation time for a simulation with a 3 mstime step is approximately 10 seconds per second of controlled simulation. Thisbreaks down into 8 seconds for simulating the plant and 2 seconds for computingthe controls.45(a) (b)(c) (d)Figure 4.4: Synchronized joint motion with the oblique retinacular ligament.(a-b) Without the ligament, pulling on the tendon flexes the DIP fully,before the PIP. (c-d) With the ligament, the same tendon flexes the fingerin a much more natural, synchronized fashion.4.4.1 Joint CouplingWith a tendon-based simulator, we obtain natural joint coupling for free. Fig. 4.4demonstrates how pulling on a single tendon produces synchronized flexion in twojoints. A single tendon (FDP), with no other tendons and ligaments present, pulls onthe distal phalanx of the index finger. The joints flex by “rolling”: first, the distaljoint DIP fully flexes, and then the proximal joint (PIP) starts to flex. With the OLadded, which spans both the DIP and the PIP, pulling on the FDP causes both jointsto flex in a natural, synchronized fashion.4.4.2 DeformitiesBy changing some of the tendon parameters, we can simulate clinical deformitiesof the hand. We simulated two common deformities: boutonnière and swan-neck[Zancolli, 1979].In the boutonnière deformity, the DIP hyper-extends, and the PIP remains lockedin a flexed position, as shown in Fig. 4.5(b). We simulate this injury by cutting the46(a) Swan-neck deformity. (b) boutonnière deformityFigure 4.5: Clinical conditions.Tracking ErrorUnsmoothed SmoothedAverage Max Average MaxBlack Box 0.021 0.12 0.017 0.10White Box 0.0045 0.087 0.0045 0.085Table 4.1: Table of trajectory tracking errors (in cm) for the circle trackingexperiment. We report both the average distance between the targetfingertip position and the actual fingertip position over the task, as well asthe maximum distance between the target and actual positions.insertion of the EDC into the middle phalanx. Once the lateral bands (intrinsic andextrinsic lateral) fall below the rotation axis of the PIP, no muscle can extend thejoint, causing the joint to remain flexed unless fixed externally.The swan-neck deformity has the opposite joint configuration: hyper-extensionof the PIP and flexion of the DIP, as shown in Fig. 4.5(a). There are many causes ofthis deformity, and among them is the elongation of the OL. In the simulator, wefirst lengthen the OL and then pull on the EDC to extend the PIP. Once the OL risesabove the axis of rotation of the PIP, pulling on the FDP causes the DIP to flex andPIP to hyper-extend.4.4.3 Trajectory TrackingTo test the control algorithms described in Section 4, we experimented with tracing acircle of radius 0.6 cm. In this and the next experiment, we constrain the fingertip to47−0.5 0 0.5−0.6−0.4− 1(a)−0.5 0 0.5−0.6−0.4− 1(b)−0.5 0 0.5−0.6−0.4− 1(c)−0.5 0 0.5−0.6−0.4− 1(d)Figure 4.6: Results of tracking a circle trajectory with the fingertip (targettrajectory is shown by a dotted line, actual trajectory by a solid line).(a-b) Black box controller, without smoothing (left) and with smoothingparameters α = 0.971,β = 0.0145,γ = 0.0145 (right). (c-d) White boxcontroller, without smoothing (left) and with smoothing parametersα = 0.999,β = 0.0,γ = 0.001 (right)lie on a plane to simulate drawing on a touchscreen. Fig. 4.6 shows the trajectoriestracked for both the black and white box controllers, with and without smoothing,with the corresponding activation patterns shown in Fig. 4.7. Table 4.1 shows theaverage per-timestep tracking error for each run, measured as the average Euclideandistance between the target fingertip position qr and actual fingertip position qt ateach timestep, as well as the maximum positional error at any timestep.As expected, the white box controller outperforms the black box controller inboth smooth and unsmoothed control. Both the activations and the movementsgenerated by the white box controller are smoother than for the black box, likely asa result of the relative sparsity of the black box’s training data. For the smoothedexamples, we used a script to automatically select the best smoothing parametersfrom a set of 1500 different values. The black box controller’s performance isvisibly improved by the use of appropriate smoothing parameters, while we find48DI EDC FDPFDS LUM PI(a)DI EDC FDPFDS LUM PI(b)DI EDC FDPFDS LUM PI(c)DI EDC FDPFDS LUM PI(d)Figure 4.7: Graph of muscle activations (ranging from 0 to 1) generatedover the duration of the circle tracking task (time range from 0 to1 second). Each figure shows the activation of the muscles (clock-wise from left): dorsal interosseous (DI), extensor digitorum commu-nis (EDC), flexor digitorum profundus (FDP), flexor digitorum super-ficialis (FDS), lumbrical (LUM), and palmar interosseous (PI). (a-b)Black box controller, without smoothing (left) and with smoothing pa-rameters α = 0.971,β = 0.0145,γ = 0.0145 (right). (c-d) White boxcontroller, without smoothing (left) and with smoothing parametersα = 0.999,β = 0.0,γ = 0.001 (right)that it makes almost no difference for the white box controller.Lastly, we have included a 3-dimensional tracking example in Fig. 4.8. Perform-ing the circle tracing task in 3 dimensions is, understandably, more difficult thantracing a circle constrained to a plane, but in this case we achieve good results usingthe unsmoothed white box controller. The average error for this run was 0.012 cm49(a)DI EDC FDPFDS LUM PI(b)Figure 4.8: Results of tracking a circle trajectory in 3 dimensions, using thewhite box controller without smoothing. (a) Screenshot of the task and(b) the resulting muscle activations (ranging from 0 to 1) generated overthe duration of the task (0 to 1 second).and the maximum error was 0.19 cm.4.4.4 WritingHere we performed a variation on our trajectory tracking task in which the controllerimitated a human writing on a tablet screen. We obtained the trajectory by having ahuman subject write the word “SIGG” on a touchscreen. The subject’s writing wascaptured and converted to a sequence of numerical coordinates suitable for use by thecontroller; this can be done on any touchscreen platform. As before, we constrainedthe fingertip location to lie on a plane. As our model only allows for control of theindex finger and does not incorporate wrist movement, we manually translated thewrist after each letter is written to allow the controller to write the entire word. Weperformed this task using the white box controller without smoothing.Fig. 4.9 shows side-by-side shots of the controller and a human subject attempt-ing this task. See our accompanying video for a more detailed comparison. Theaverage tracking error for this task—measured in the same way as for the circletracking task—was 0.013 cm, while the maximum error observed was 0.18 cm.50(a) Capturing human finger motion. (b) Reproducing the trajectory.Figure 4.9: Reproducing fingertip trajectory using our simulation framework.4.5 Conclusions and LimitationsWe presented a framework for biomechanical simulation with tendons that canhandle the complex routing constraints of the hand, such as pulleys and sheaths. Byextending the constrained strands framework of Sueda et al. [2011], we were able toefficiently handle contact between bones and tendons, and to take large time stepsdespite the stiffness of the tendons. We showed that modelling the tendon networkgives us coupled motion of the digits, as well as energy storage in tendons. Wealso simulated deformities of the hand by changing some of the tendon parameters.Finally, we developed two methods for control of a human index finger model,which we were able to use successfully in trajectory tracking.Although we were able to simulate a highly complex system of tendons andligaments, there are still many more approximations that remain. For example,the system is sensitive to values of the biomechanical parameters; it would beuseful to learn these from measurements of human hands. We still approximatebiomechanical joints as simple mechanical joints; an interesting avenue of futurework would be to extend this framework for handling joint limits using ligaments.51Chapter 5Software Design andImplementationIn Chapter 4, our first attempt towards the simulation of biomechanics of the humanhand was described. The experience of building this simulation software motivatedthe design of simulation tools conducive to the different types of useful workflowsfor such simulations. To model the biomechanics of the human hand, we need toolsto simplify the design process, and to evaluate the results of the simulation. Suchtools could be applied to the design of cadaver studies, surgical experimentation,and medical education. However, they would be of value to the scientific communityonly if we can make them easily accessible to biomechanics researchers, computerscientists, and surgeons. To achieve this goal, we present not just a simulationlibrary, but design tools for visual modelling and interaction with the simulation aswell.A few biomechanics simulation tools like OpenSim by Delp et al. [2007] (in-spired by another proprietary toolkit SIMM) and Anybody by Damsgaard et al.[2006] have been developed. OpenSim is popular in the biomechanics community,but has not gained acceptance in the hand modelling and surgery community, mainlybecause of the complexity of the human hand. The human hand is comprised ofintricately interconnected musculotendons, which provide it the required dexterity.This interconnectivity of musculotendons and their close contact with bones, partic-ularly the branching involved in the extensor hood structure, and the force exerted by52pulleys holding the tendons close to the bones of the finger, is difficult to capture incommonly used biomechanics software like OpenSim and Anybody. Moreover, thedynamics of inextensible tendons cannot be modelled either. With stiff extensibletendons, we would get a much smaller timestep size, and the accuracy of the tendonforces would be limited by accurate tendon path computation (see Section 6.2). Dueto these limitations, even recent experiments on extensor mechanism modelling, e.g.,[Synek and Pahr, 2016; Dogadov et al., 2017], utilize custom simulation software.5.1 Software DesignThe design, modelling, and simulation tools can be split into modular componentsthat interface with each other. The most important part of this set of tools is thesimulation library built based on the dynamics methods described in Chapter 3. Thenumerical simulation of the dynamics of the scene are confined to a C++ librarynamed Strands. All the peripheral software tools must communicate with thislibrary to provide inputs and read back results. To this end, we define interfaces forinformation exchange to and from the library. This library is described in detail inSection 5.3.C++Figure 5.1: Software componentsFor intuitive simulation design, the scene, specifying the various rigid bodies,tendons, and their interconnectivity, must be designed in a graphical modellinginterface. Writing a graphical modelling interface akin to the GUI accompanying53OpenSim* would be a lot of unnecessary work, so we chose to do the graphicalmodelling within an already available 3D computer graphics application, AutodeskMaya®. In order to use Autodesk Maya® to model biomechanical elements andinteractively set properties of the various components, we needed to add threedifferent plugins - a data storage “Node”† (SceneNode), a plugin to interface withStrands (StrandsCmd), and a UI layer (StrandsUI) to create models and simulatetheir dynamics.The Maya® interface is great for design and simulation, but experimentationand control place different requirements on the system. For experimentation andcontrol, we would like to be able to generate the results with minimal intervention.This involves running the simulation with varied parameters and accruing therequired simulation output to generate results. For this, the last element of the set ofsoftware tools is a Python interface provided for experimentation and control. Therelationship between these different components is presented in Fig. 5.1. We firstdescribe the design tools in context of the workflow supported by them and thendiscuss the architecture and usage of the Strands library.5.2 WorkflowThe design of software tools for simulating the hand is guided by requirementsfrom different type of workflows. We discuss three different workflows which wefind important for effective modelling of and experimentation using human handsimulation: Design, Experiment, and Control workflows. Each workflow caters toparticular needs for an effective and useful simulator for the human hand.5.2.1 Design WorkflowAnatomists have made various claims on the effect of different components of thehand musculotendon topology on its dynamics. In some cases, these claims couldbe based on anatomical investigation [Landsmeer, 1949], in other cases on morerigorous experiments with changed tendon topology in cadavers [Hauck, 1923], andbased on mathematical model based in other works [Leijnse and Kalker, 1995].*https://simtk-confluence.stanford.edu:8443/display/OpenSim†Terminology from Maya® https://help.autodesk.com/view/MAYAUL/2017/ENU/54(a) Strand visualization: Filled red nodes are L-nodes,and hollow green lined nodes are E-nodes. Transient nodes(see Section 6.3) are also visualized as E-nodes.(b)Figure 5.2: StrandsUI interfaceEach of these modalities of reasoning about human anatomy have their limita-tions. Anatomical investigations are difficult to replicate, and could misattributefunction to different tendons due to complex musculotendon and ligament topology.Mathematical model based reasoning without simulating motion of the joints fromforce inputs requires assumptions regarding parameters such as tendon slack vari-ables, and moment arms of tendons over joints in various configurations. Testingmodifications of tendon topology on cadavers requires significant surgical expertise,is resource-intensive (cadavers) and slow to execute.This forms the motivation to support a design workflow to test various hypothe-ses about the function of tendons. We developed a graphical design interface whichallows interactive modifications to tendon topology, axes orientation, and other suchvisually relevant parameters. Once these changes have been made, the simulationcan be run and the output visualized in the same interface. This design workflowallows faster iterations over the model by allowing the user to focus on the effectsof model changes.To achieve this, we leveraged graphical design tools available in AutodeskMaya®. We augmented Maya® with some plugins:551. SceneNode - A Dependency Node in Maya is a node in the graphical scenedescription. We created SceneNode as a new Dependency Node which corre-sponds to the root of the scene graph for the Strands library. This providesscene properties such as gravity, timestep etc. It also connects to all the otherMaya dependency nodes which correspond to elements in the scene (strands,joints, rigid bodies etc.). We used Message Attributes from Maya API toconnect the different elements of the scene and form a scene graph like theone presented by Sueda et al. [2011]. We present a simple scene graph toillustrate the concept in Fig. 5.3(c).(a) Rigid bodies(b) Joint added(c) Strand added (d) SimulationFigure 5.3: Simple scene and the corresponding scene graph.2. StrandsCmd - To manipulate the scene graph captured by SceneNode, wedeveloped a command plugin named StrandsCmd made available to Maya®via Python or Mel (Maya® Embedded Language). The operations on the scenegraph covered by this tool include adding rigid bodies to get a graph like theone shown in Fig. 5.3(a). We can also add joints using StrandsCmd—herewe add a joint between the rigid bodies to get Fig. 5.3(b); and lastly wecan add strands and edit their nodes to attach them to rigid bodies—here weadd a strand with 4 nodes, 2 for each rigid body in Fig. 5.3(c). Moreover,StrandsCmd also provides an interface to the Strands simulation library, tointerpret the scene graph constructed at SceneNode like the examples inFig. 5.3, and run the simulator. It then fetches the simulator output to be56visualized in Maya®. The visualization for the scene graph in Fig. 5.3(c) isreproduced in Fig. 5.3(b).3. StrandsUI - To make the editing of the scene graph seamlessly visual, weprovide a UI wrapper on StrandsCmd. When any given scene elements areselected in Maya®, this UI wrapper filters the available valid operations onthe scene graph concerning the element. For example, a strand node may bevalid input for splitting a strand at that node using StrandsCmd, but is nota valid input for a hinge joint creation operation (requiring 2 rigid bodies)using StrandsCmd.All the software tooling described above is intended to simplify the workflow ofmodelling and simulation of the human hand. You can start with creating a modelin Maya® using the StrandsUI (using StrandsCmd under the hood) to edit the scenegraph stored in SceneNode. Once the SceneNode instance has all the requiredelements of the model, they are made available to an instance of Scene class ofthe Strands library. The Strands library can now be simulated and probed at anysimulation time point. This output is made available back to Maya® plugins, whereStrandsUI can now be used to select an instance of the simulation run.5.2.2 Experiment WorkflowOnce the topology of the model is finalized, and the visual construction of the modelcompleted, the user may like to run experiments to test the simulation responseto changes in parameters. Visual inspection of simulation results one by one fordifferent parameters hardly forms a reasonable method for such experimentation.Therefore, a different type of workflow is also necessary. This workflow constructsplots of output variables, such as finger joint angles, over a series of simulations runwith changes to parameters such as joint stiffness and damping.To achieve this, we provide an exporter for use with Maya® to output an XMLformat description of the scene described by the SceneNode, which can be importedinto the Strands library and simulated. This is done via Python scripting whichlets us use Python’s XML editing and plotting tools to change parameters and plotoutput variables of interest. A C interface for Strands is exposed to Python usingthe ctypes Python package. This interface provides all the necessary functions for57importing a scene, stepping through it, and recording the results. Note that thePython interface is completely independent of Maya®, once the XML description ofthe scene is available. A sample script for running an experiment is found in List. 1.1 import strands_mod as sm2 from ctypes import *3 ## Other import commands4 ## Initialization code5 strands_lib = sm.load_strands_mod()6 nSteps = 18007 scene_p = strands_lib.load_scene(scene_filename)8 ## Call any editing commands on scene9 state_attrs = strands_lib.get_state_attrs(scene_p)10 ## Step the scene and check/record state11 for iStep in range(0, nSteps):12 strands_lib.step(scene_p)13 ## Get output state14 state = strands_lib.get_state(scene_p)15 ## Process/store state16 ## Plotting/processingListing 1: Outline of a typical experiment script.This workflow has been utilized to study the role of the lumbrical muscle playsin finger flexion. We will review the details of this study in Chapter 7, but we candescribe the use of this workflow for the study. The tendon topology of the indexfinger required for this workflow is first created using the design workflow describedin Section 5.2.1. This scene description is exported to an XML file using the Maya®plugins. Using the Python interface, we import this scene description into a Sceneclass data-structure of the Strands library. We script the running of the simulator inPython, and at each step get the state of the scene elements. Of these state outputs,we plot the joint angles. For each parameter value used for the input variable in theexperiment, in this case the weight of the mass attached to the lumbrical tendon, thescene XML is edited in Python, and the scene simulation, as described, is repeated.Once the output variables, the angles of the finger joints, have been plotted for theruns of each simulation, we can compare the these plots to the observations from58the cadaver experiments conducted by Kamata et al. [2016].5.2.3 Control WorkflowThe two workflows described above have been designed to be open loop, i.e., inputsmust be specified in advance and cannot be set based on observed outputs. A naturalextension to this is to allow closed loop control of the simulation. For this, thesimulation needs to allow the current state to be observed, and inputs to be setto modify the simulation results. The Strands simulation library allows inputs tothe simulator in the form of muscle activation values (combined together as anactivation vector). These inputs can be scripted via the Python interface. A typicaloutline for a closed-loop controller using this interface is provided in List. 2. Like atypical closed-loop controlled simulation, when a target is provided, the controllercomputes the input (activation vector for our simulation) to reach the target. We canthen step the simulation and refresh the activations until the target is reached.One way to create a controller for the human hand is to collect a large trainingdataset to train a controller using machine learning. One example of a controllerbased on a large training dataset is the blackbox controller used to control the handmodel in Chapter 4. The controller, based on the equilibrium point hypothesis[Shadmehr, 1998], used input data pairs in the form of muscle activation inputs andcorresponding equilibrium output state to drive precise fingertip control.Figure 5.4: Using forks of simulation to gather control data.Another way to generate useful data for training a controller is to be able toobserve the response of the simulator to a particular activation input in a given state.To enable this, the simulator can be initialized to any valid internal state. Using thisfunctionality, the simulator can be forked at any point (illustrated in Fig. 5.4), and591 import strands_mod as sm2 import controller as ctrl3 ## other import commands4 ## Initialization code5 strands_lib = sm.load_strands_mod()6 scene_p = strands_lib.load_scene(scene_name)7 ## Call any editing commands on scene8 state_attrs = strands_lib.get_state_attrs(scene_p)9 ## Step the scene and check/record state10 while true:11 ## Get target - e.g. trajectory input12 target, running = get_target()13 if not running: exit14 reached = false15 while not reached:16 ## Get output state17 state = strands_lib.get_state(scene_p)18 ## Process state and compute next input activations19 activations = ctrl.compute(state, target)20 ## Set activation vector21 strands_lib.set_act_vec(activations)22 strands_lib.step(scene_p)23 ## Evaluate termination condition24 reached = ctrl.reached_target(target)Listing 2: Outline of a typical closed-loop controller.each copy of the simulator may be provided with a different activation vector. Thisway, the response of the simulator to various state-action pairs may be collected.A simple example of this class of controllers is a k-Nearest Neighbor (kNN)controller. We implement a kNN controller by collecting final fingertip locationfor different levels of flexor and extensor activation for a simple two-tendon fingermodel with a flexor and extensor. This model is similar to the finger model laterimplemented for the full hand model in Chapter 8. For any given target position,the controller looks up the K nearest neighbors (K=3 in this example). The acti-vations for these K points is averaged and the activation for both the tendons is60(a) Fully flexed finger (b) Partially flexed fingerFigure 5.5: Simple kNN controller for finger model.set accordingly. Two output states generated using this controller are illustrated inFig. Strands Simulation LibraryA custom simulation software was deemed necessary because a large number ofcustom elements required for our model are missing from OpenSim, describedbriefly above. By writing our own simulator, we avoid any limitations placed by thealready established OpenSim software architecture.The Strands simulation library is written in C++14 and provides a Scene class toconstruct scenes for simulation. This library stands by itself, and can be integratedinto any GUI software with C++ bindings. The library deals purely with thenumerical simulation elements of the software. All aspects of dynamics simulationare dealt with in this part of the code. Eigen library for linear algebra [Guennebaud,Jacob, et al., 2010] was used as the base. Linear solvers available to Eigen were used,while quadratic programming is performed with the C API for Mosek optimizationtoolkit [ApS, 2017].This library is written using object oriented programming concepts of abstrac-tion, encapsulation, inheritance, and polymorphism. All physical objects, classes61such as Rigid and Point, hold information in the F linear space defined in Sec-tion 3.3.1. Underlying degrees of freedom, defined as children of DoF class, dealwith the R space, and each of the physical objects owns a corresponding DoFinstance. The abstraction of underlying reduced degrees of freedom in R space iskey to allowing uniform treatment of nodes of different types. Prominently usednodes are the 0-DoF nodes fixed to rigid bodies, and 1-DoF nodes constrained tolie on a curve (a circle or line for example). The treatment of strands using Strandclass, due to its special nature, holds its own degrees of freedom, and handles matrixinformation for both RL and R space at once. We briefly describe each of the majorclasses that form the Strands library.DoFAn abstract class, DoF, is used to represent the conversion of degrees of freedomfrom R space to RL space. Each instance owns a position vector x, velocity vectorv, Jacobian block vectors (Γ and Ω, described in Chapter 3), a source DynObj, anda target DynObj. Subclasses for each degree of freedom are: DoFFixed (0-DoF),DoFCurve (1-DoF), DoFSurface (2-DoF), DoFFull (3-DoF), DoFFrame (6-DoF).Each implementation of the abstract class supports two key functions:1. computeJacobian - This function computes the Jacobian block vectors,Γ and Ω, which transform reduced velocity vector (in R space) of the DoFalong with the velocity of the source into the velocity (in F space) of thetarget.2. computeTargetCoordinates - This function implements the coordi-nate computation for the target, using the reduced position vector x, andthe position vector of the source.These functions define the interpretation of the DoF in F space.DynObjAll physical objects in the scene, e.g. points, rigids, and strands, derive from theabstract class DynObj. A DynObj instance owns a unique DoF instance do f , a62velocity vector v, a position vector x, a force vector Force, and a mass matrix M.Rigid, Strand, Point are implementations of the abstract DynObj class.RigidRigid bodies are represented with Rigid class instances, each with a 6-DoF DoF-Frame, (6×6) mass matrix, and the corresponding (6×1) velocity and positionvectors.PointUnderlying every node of a strand is a point representing the physical properties.While a node belongs to only one strand, point ownership is not unique. Manynodes (at most one end node per strand) may be represented with one underlyingpoint. This point corresponds to a physical object and hence derives from DynObj.Each Point has an underlying DoF instance do f , (3×3) mass matrix, and thecorresponding (3×1) velocity and position vectors. do f may represent 0-3 degreesof freedom depending on how the motion of the point is constrained.Strand(a) Junction to junction. (b) Rigid to rigid. (c) Rigid to junction.Figure 5.6: Types of strandsEach strand of the scene is described by 0 or more Q-nodes flanked by 2 L-nodes. When two or more strands are connected at a junction, it is assumed thattheir material degrees of freedom are defined by the same Point at the junction, andtherefore, every junction is treated as an L-node. In other words, a strand can befrom (a) junction to junction, i.e. connecting two or more strands, or (b) rigid to63rigid, or, (c) rigid to junction. Each node of a strand has an underlying DoF, whichdescribes its motion.5.4 DiscussionThe software tools presented here are effective tools for simulating the humanhand. As we will see in Chapters 7 and 8, these software tools can be successfullyused for reproducing lumbrical muscle function and tenodesis grasp with a wristmodel respectively, using anatomical models. The faster iteration and visualizationcycle made possible with the plugins to Maya® allowed better understanding ofassumptions and choices made in the modelling of the hand. The design workflowtools led to insights into the missing techniques and various modelling choices inthe simulation model, which are discussed in Chapter 6.As with any software, there are some improvements which would markedlychange the simulation experience with this software. Most important of these isperformance. Simulating one second of time takes about ∼ 1.2s of simulation timefor a finger model, and scales poorly with model complexity. Profiling the perfor-mance of the simulator and optimizing it where useful would improve interactivemodelling and speed up data collection for any control algorithms built on top of it.64Chapter 6Anatomical Modelling of theIndex FingerMany human finger models have been created for various studies, with a number ofthese focused on simulating the extensor apparatus or dorsal aponeurosis, since itis a rather complex substructure of the finger tendon network. Winslow’s rhombus[Winslow, 1732] is used as an approximation to the extensor apparatus by Valero-Cuevas et al. [2007] to explain the change in the ratio of tendon transmissionto the central slip and the terminal slip to the distal joints of the finger. Leijnseand Spoor [2012] reverse engineered the parameters for the extensor apparatusmorphology using a 2D kinematic model. Synek and Pahr [2016] evaluated theimpact of the extensor mechanism on the maximum isometric fingertip forces, whileDogadov et al. [2017] proposed a parameterized model of the extensor mechanism.However, the focus of these studies was on specific results regarding the kinematicsor dynamics of the finger. The studies did not establish a comprehensive threedimensional simulation of finger dynamics.Some three dimensional models have been proposed to attempt to explain thekinematics and/or the dynamics of the finger (see Section 2.3.2), but none of theseattempts have successfully created a comprehensive model that can• be verified numerically against kinematic trajectory data,• simulate plausible dynamics of the finger, and65• produce fine tuned fingertip motions via muscular activation control.We attempted this task with a model based on tendon strands mathematicallymodelled as defined by Sueda et al. [2011] and outlined in Chapters 3 and 4. Byexhibiting fine-grained muscular activation based control of the fingertip, we providea proof of concept for control of tendon-models based on inextensible strands. Toestablish higher confidence in the simulator, the simulation techniques are extendedto allow reproduction of kinematics and dynamics observed in the finger in vivoand in vitro. The salient simulation techniques necessary for effective anatomicalsimulation of the finger include effective branching, tendon inextensibility, andtendon-joint interaction.6.1 BranchingFigure 6.1: Reduced model of the extensor hood [Garcia-Elias et al., 1991a].At the branching points, represented by letters A to H in Fig. 6.1, two or moresections of the extensor apparatus merge. Since there is no movement of materialbetween sections merging at these points, these points can be represented withL-nodes.*Further we need to decide on how many degrees of freedom to provide for thesenodes. Providing 3 DoFs to these nodes would allow them to move in all directionsand hence, penetrate through the bones, requiring collision detection. There are twooptions worth considering for these nodes:• 1-DoF L-nodes: 1-DoF nodes constrained to lie on a curve on a finger bone.*If there was movement of material between sections, no type of node would be correct.66• 2-DoF L-nodes: 2-DoF nodes constrained to lie on the surface of a fingerbone.While 2-DoF nodes sound ideal, there are numerical concerns pertaining to them.If two nodes of the same strand get very close, at least one of which is an L-node, themass of the strand segment between them can become very small, since the Q-nodesallow mass to flow between segments. The low mass of this segment will ruin theconditioning of the mass matrix, in spite of the quasistatic formulation. One coulduse L-nodes everywhere to avoid forming low mass segments along the strand, butthen we cannot route the tendon over joints and allow the natural motion of tendons.1-DoF L-nodes allow us to prevent such numerical instability by constraining thenodes to lie on their own curve along the bone, hence creating a minimum distancebetween them. For a detailed treatment of 1-DoF nodes, see Section Tendon InextensibilityTendons are made of extensible but stiff material. Some studies [Qian et al., 2014;Garcia-Elias et al., 1991b] have tried to quantify the stiffness of tendons acrossvarious segments of the extensor apparatus. The usefulness of simulating tendonsas elastic elements, however, is unclear. On the other hand, simulating tendons asstiff unilateral springs makes the dynamics optimization quite stiff courtesy of thehigh stiffness of tendon segments.Treating tendons as inextensible is not without precedent. Leijnse and Kalker[1995], for example, evaluate the role of the lumbrical muscle in a geometric fashion,only taking tendon tautness into consideration, completely ignoring tendon stiffnessand its related stretching of tendon elements.To opt for a better timestep, we limit ourselves to studying the finger withinextensible tendons. Note that a solver for quadratic programming is alreadyrequired for the simulator to address joint limits, so the numerical solver remainsunchanged with additional unilateral constraints added. The formulation of strandinextensibility in velocity space, as given by Sueda et al. [2011], uses the strain67expressionε =l∆s−1 (6.1)where l = ‖∆x‖. This is rearranged to get the positional form g = l2−∆s2. Differ-entiating this expression gives us the velocity constraint:g˙ = Gq˙ = 2(l δ lδ x˜0 lδ lδ x˜1 ∆s −∆s)˙˜x0˙˜x1s˙0s˙1 (6.2)This inequality constraint, however, is implemented in the velocity constraint,and is therefore only active once the positional constraint has been violated, i.e.,the tendon is already stretched beyond its original length. When the timestepis sufficiently large, the tendon strain may increase significantly, disrupting theresults of the simulation and deviating significantly from the tendon inextensibilityrequirement. To correct this, we can add a weaker formulation of the inextensibilityconstraint, which is active when −ε0 < ε < 0 for a chosen threshold ε0 > 0. Thisweaker formulation ensures that the segment does not stretch beyond its materiallength.∆(‖∆x0(t+h)‖−‖∆s0(t+h)‖)< ‖∆s0(t)‖−‖∆x0(t)‖ (6.3)=⇒ h∗ δδ t(‖∆x0(t+h)‖−‖∆s0(t+h)‖)< ‖∆s0(t)‖−‖∆x0(t)‖ (6.4)=⇒(δ lδ x˜0δ lδ x˜1 1 −1)˙˜x0˙˜x1s˙0s˙1< 1h(‖∆s0(t)‖−‖∆x0(t)‖) (6.5)This formulation is weaker because the discretization on the left hand sideof Eq. (6.3) is performed along the direction of the segment, and ignores the velocityin the plane perpendicular to the direction of the segment. However, it remediesthe over-extension of the strand in the absence of the stronger velocity constraint68described in Eq. (6.2). This allows us to increase the timestep of the simulationwithout worrying about over-extension of inextensible strands.6.3 Strand Remeshing at JointsModelling tendons with straight line segments through via-points, as is done inour simulation, is prone to significant errors in tendon path calculations. As aconsequence, the dynamics of the simulation are also affected. We demonstratethis misbehaviour with an example of flexion with a weighted lumbrical musclesimulated with and without transient nodes. This is illustrated in Fig. 6.2.(a) Misbehaviour without transient nodes. (b) PIP transient nodes in action.(c) Expected behaviour under loaded LUM when using transient nodes.Figure 6.2: Necessity of transient nodes for experiment covered in Chapter 7.One way to deal with this would be to add nodes over the joint. However, given69the nature of interaction of strands over joints, this can lead to nodes on a strandcoming too close to each other, resulting in a poorly conditioned mass matrix. Evenin cases where the matrix isn’t poorly conditioned, weird geometries of the strandcan result (knots for example).When the strand geometry is poorly computed across a joint, tendon excursionvalues can be affected (see Fig. 6.5), which leads to changes in the computed muscleforce. Moreover, if the geometry of strands over joints is ignored, forces that shouldbe computed at the joint will rather end up being computed elsewhere on the rigidbodies. The incorrect geometry computation is a type of bow-stringing [An et al.,1979], that other hand models have assumed in the past [Synek and Pahr, 2016]. Wewould like to avoid these artifacts, particularly at joints.In light of these concerns, we decided to utilize the obstacle set method, devisedby Garner and Pandy [2000], to generate transient nodes for strand segments passingover finger joints.Obstacle Set MethodThe problem of computational kinematic modelling of tendon paths across jointshas been addressed by Garner and Pandy [2000]. Similar to our assumption, theforce transmission by the tendons is assumed to be along the locus of the transversecross-sectional centroids of the musculotendons. The tendons are idealized asfrictionless elastic bands moving freely over neighbouring anatomical constraints.The kinematic interaction of tendons with joints is handled by reducing joints toregular shaped rigid bodies such as spheres and cylinders. The method is called theobstacle set method.Each joint, for the purposes of computing the kinematic interaction, is replacedby a regular rigid shaped object, referred to as an obstacle. The reference frame foran obstacle is naturally the same as the reference frame for the joint, which is definedin relation to the two rigid bodies involved. Obstacle via points are defined in thereference frame of the obstacle. The segment of the musculotendon falling withinthe boundary obstacle via-points is referred to as the obstacle set. Any dynamicinteraction between the joint and the tendon happens across the obstacle set. Tocompute the obstacle set, four different methods have been proposed by Garner70and Pandy [2000]—single sphere, single cylinder, double cylinder, sphere-cappedcylinder. These can be used to model different joints of the human body, such as thesphere capped cylinder for the shoulder joint.For the IP joints and the MCP joint, the single cylinder model works well. Thisis illustrated in Fig. 6.3.Figure 6.3: Illustration of transient nodes for a joint with single cylinder model.Nodes S0, S1, S2, and nodes P0, P1 are defined nodes of the strand pivotedto rigid bodies A and B. Nodes Q and T are transient nodes that aregenerated assuming a cylindrical joint shape.Transient NodesBased on the obstacle set method, we modify the implementation of strands fortendons with transient nodes. These are 0-DoF Eulerian nodes defined in the frameof reference of the rigid bodies. They help resolve the strand-joint interaction, byapplying the obstacle set method when relevant. When the segment of the strand ispassing through the joint, transient nodes modify the simulation path for the strandas well as the dynamics of the strand-joint interaction.The following configurations exist in relevance to the transient nodes of strands71Figure 6.4: Obstacle set calculation at different joint angles. Strand interactingwith the joint is shown in red, while the dashed yellow line is the strand“flying off” the joint.passing over joints:• Strand interacts with the joint.r.r0 < 0 or‖r‖<∥∥rj∥∥ (6.6)Here r0 is the shortest vector from the joint centre to the strand in the initialjoint configuration, r is the same vector in the current joint configuration,and rj is an arbitrary joint radius vector. The first condition corresponds tothe new point being in the other direction. The second condition checks forwhether the joint is being intersected. An example of this is drawn in Fig. 6.4.Since the strand interacts with the joint, the transient nodes are active andsimulated.• Complementary condition. The strand, in this case, does not interact withthe joint, and can effectively be allowed to “fly-off” from the joint to some72extent (e.g., FDP at PIP).† The transient nodes are ignored in the mass matrixby setting their corresponding sub-matrix to identity, and ignoring them instrand-related matrix (mass and Jacobian) calculations.Eq. (6.6) is evaluated before every timestep, so the transient nodes can be intro-duced to avoid strand-joint intersection. Two transient nodes are placed tangentialto the assumed cylindrical shape of the joint. The number of these transient nodesmay be increased to improve the accuracy of the force transmission from the tendonto the bones due to the interaction at the joint.ExampleTo show the impact of transient nodes on strand strain, we show a simple exampleof a strand with a single segment, i.e., made of 2 L-nodes, crossing over a hingejoint attaching two rigid bodies. This setup is depicted in Fig. 6.5(a). Figure 6.5(d)shows the strand intersecting the joint, while in Fig. 6.5(c), transient nodes split thestrand into three segments, preventing that intersection. The lack of transient nodesleads to an underestimation of the strain by as much as 50% at a joint angle of only1.0 radian, which is not uncommon for joints of the index finger.†The strand can only fly-off to a limited extent since the constraints on the nodes of the strand stillkeep it close. Nonetheless, the strand will not interact at the joint.73(a)0.0 0.2 0.4 0.6 0.8 1.0Joint Angle (radians) StrainStrand Strain vs Joint Anglewith transient nodeswithout transient nodes(b)(c) (d)Figure 6.5: A plot of strand strain vs joint angle for a hinge joint between tworigid bodies.74Chapter 7Lumbrical Muscle and FingerFlexionAll the techniques presented in previous chapters are geared towards simulating ananatomically verifiable model of the hand. In this chapter, we put the simulationtechniques to the test by constructing a three-dimensional dynamics simulationto assess the role of the lumbrical (LUM) muscle in flexion of the index fingerdriven by the flexor digitorum profundus (FDP) tendon. Recall that the LUM muscleoriginates from the FDP tendon, and attaches into the extensor hood. Due to itscomplex routing, the muscle modifies finger kinematics in a unique fashion, makingit difficult to understand its function. The model is first used to reproduce cadaverexperiments of the LUM tendon detached from the FDP with a constant tensionapplied to it. This simulation model is subsequently modified with the LUM musclemodelled to emulate its in vivo structure originating at the FDP.This is the first three-dimensional simulation to show the increase in fingertipreach with increased LUM muscle activation. We successfully reproduce in vivoand in vitro experiments investigating LUM muscle function and report the effect ofjoint dynamics parameters. These two simulation experiments serve as experimentalcomparisons to establish the utility of our model.75(a) Sagittal view(b) Dorsal viewFigure 7.1: Schematic representation of the index finger musculotendon struc-ture. For more detail, please refer to the glossary.7.1 IntroductionThe lumbrical muscle is a relatively small intrinsic muscle of the finger, that gets itsname from its worm-like appearance. In spite of its small size [Chao et al., 1989;Jacobson et al., 1992], it plays a major role in finger movement. It stabilizes themotion of the interphalangeal joints of the finger during MCP flexion [Arnet et al.,2013; Kamata et al., 2016], leading to a larger arc for the fingertip. It also actsas a radial deviator of the MCP joint [Rayan et al., 1997; Kamata et al., 2016]. Inreconstructive surgery for tetraplegic hands, McCarthy et al. [1997] also show theimportance of balancing intrinsic muscle function for strong grip strength.The LUM muscle can play these different roles because of the interconnectedanatomy of the index finger muscles. The index finger of the human hand has 4degrees of freedom–one for each of the IP joints, and two for the MCP joint–andseven muscles act on the bones. Of these muscles, the function of the extrinsicmuscles can be clearly assessed as extensors or flexors. The extrinsic flexors—FDP& FDS—insert directly into bones via respective tendons. On the other hand, theextrinsic extensors and intrinsic muscles feed into a fascial sheet known as the dorsal76aponeurosis (DA) [Landsmeer, 1949] or the extensor mechanism [Garcia-Elias et al.,1991a]. A schematic representation of these structures can be found in Fig. 7.1.The extensor mechanism of the index finger consists of 5 input tendons (EDC,EI, LUM, DI, PI) and two insertions – a proximal insertion at the base of the middlephalanx via the central slip (CS), and a distal insertion at the base of the distalphalanx via the terminal slip (TS). Of these 5 tendons, EI and EDC are known to beextrinsic extensors. However, the function of the intrinsic muscles, LUM, DI, and PI,is dependent on the posture of the finger [Synek and Pahr, 2016], and is the subjectof active study [Dogadov et al., 2017; Al-Sukaini et al., 2016]. Of these, the LUMmuscle has a unique topology. It originates from the FDP tendon on the palmar sideof the metacarpal bone, and merges into the extensor hood next to the MCP joint[Landsmeer, 1949; Zancolli, 1979; Eladoumikdachi et al., 2002]. In this way, theFDP tendon is coupled with the extensor mechanism, while the other flexor, FDS,is topologically independent. The closed loop nature of the LUM musculotendonallows it to influence both flexion and extension of the finger.The examination of fingertip forces [Valero-Cuevas et al., 1998, 2007; Synekand Pahr, 2016] and the various parameters of the extensor mechanism structure[Qian et al., 2014; Dogadov et al., 2017] provide ways to reason about the role of thedifferent tendon components of the apparatus. However, a three dimensional modelof the dynamics and kinematics of the finger is yet to be designed. Particularly, thefunction of the LUM muscle and its effect on the index finger joints during flexionhas not been simulated by any dynamics model to our knowledge. Such a modelwould help shed light on how the the LUM muscle affects hand function, and helpguide surgical interventions in pathological cases.We propose a three dimensional dynamics model to advance our understandingof the role of the LUM muscle in flexion of the joints of the finger. To parameterizeand evaluate such a model, we need recorded observations from real-world experi-ments. We use both in vitro and in vivo observations. For in vitro observations, weuse cadaver studies of LUM muscle function on flexion by Arnet et al. [2013] andKamata et al. [2016]. Arnet et al. [2013] reported that intrinsic muscle force canextend the fingertip reach—the distance between the fingertip and the palm—bychanging the order of flexion of the PIP and MCP joints. In addition, Kamata et al.[2016] probed the minimum tension required at the LUM tendon to change the77order of flexion of finger joints. The latter use a computerized setup to controlthe excursion of the tendons of the index finger of a cadaver hand. Starting in anextended initial state, the FDP tendon is pulled at constant rate, while applying astatic weight on the LUM tendon. This weight is routed along the FDP tendon tomimic the force direction of the LUM muscle. This experiment forms the basis ofthe first simulation experiment presented here.Sueda [2010] attempted to simulate the in vitro experiment by Kamata et al.[2016]. However, our implementation in this chapter is significantly differentfrom the implementation by Sueda [2010]. Our model incorporates inextensibletendons with exponential joint stiffness used only for the MCP joint, performstendon remeshing across joints, and reproduces results for the whole range ofweights in the experiment, not just extreme weights. With our model, we alsoperform experiments with and without the OL, and test the effects of joint dampingparameters. Unlike experiments by Sueda [2010], our experiments are performed ina reproducible fashion using the software framework described in Chapter 5. Thesoftware framework makes it easy to change the model as needed and study theeffects of different parameters.For in vivo observations, we refer to data collected by Al-Sukaini et al. [2016]on the patterns of extrinsic and intrinsic muscle dominance in flexion of MCP andPIP joints. To compare our simulation to this data, a second simulation experimentis designed with the LUM muscle originating from the FDP tendon, as is the case invivo [Zancolli, 1979].7.2 Related WorksThe moment arms of tendons of the finger across the different joints were measuredby An et al. [1983]. This enabled the construction of various three dimensionalbiomechanical models of the human index finger based on these observations.Valero-Cuevas et al. [1998] provide a model to compute the forces at the indexfinger-tip produced by excitation of the muscles with an extensor mechanism basedon Winslow’s tendinous rhombus model for the extensor mechanism Zancolli [1979].Using this statics model, the impact of the extensor mechanism on the maximumisometric fingertip force production was further studied by Synek and Pahr [2016]78with parameters measured by An et al. [1979]. A similar model was used to computethe impact of the input force distribution between intrinsic and extrinsic tendons onthe output force distribution at the middle and terminal insertions of the extensormechanism Valero-Cuevas et al. [2007]. The tendons were represented as elasticstrings in the latter study. Dogadov et al. [2017] also use the elastic string tendonmodel to compute length parameters for the EL and IM bands. These studies utilizea dynamics model for force computations, but do not provide a way to use theforce output to compute the kinematics of the finger. Moreover, the use of elastictendons necessitates accurate computation of tendon trajectory to measure the lengthof the tendon segments, which would require very slow timesteps, and a precisetopological model. Relaxing this requirement allows us to use larger timesteps inour simulation allowing faster iterations over model topology.The use of inextensible strands to model the finger has previously been exploredby Leijnse and Kalker [1995]. To reason about the motion of the finger undervarious tendon excursions, Leijnse and Kalker [1995] provide a two dimensionalmathematical model constructed with variable moment arms and generalized slackvariables. The impact of a taut lumbrical tendon is investigated with this model[Leijnse and Kalker, 1995, Figure 5a-c]. This mathematical model is able to predictthe increase in fingertip reach observed in cadaver studies Arnet et al. [2013];Kamata et al. [2016]. However, even though the model provides a foundation forworking with inextensible strands for computing hand kinematics, the reasoning isqualitative, and no kinematics or dynamics can be computationally predicted fromthis system.Finger dynamics cannot be accurately predicted without a model for passivestiffness of joints. The joint stiffness of the MCP joint has been investigated in anumber of studies Kamper et al. [2002]; Qin et al. [2010]; Kuo and Deshpande[2012]. Kuo and Deshpande [2012] compare passive joint forces to the contributionof muscle-tendon units and show that passive stiffness at the MCP provides thedominant contribution to joint stiffness throughout the flexion-extension range.We use the measurements provided by Kuo and Deshpande [2012] to set stiffnessparameters for our model.79(a) Trajectory (b) AnglesFigure 7.2: Finger trajectory and joint angle observations by Kamata et al.[2016]7.3 Cadaver Results and InterpretationAs predicted by the 2D model given by Leijnse and Kalker [1995], both the studiesobserve a significant change in flexion motion at the application of extreme weightson the intrinsic muscles. In observations by Arnet et al. [2013], this change wasobserved around 3.675 N, while they were observed by Kamata et al. [2016] at1.470 N. Note that the actual load at which the behaviour changes may be differentbecause of the differences in the experiment methods. Among these differences, thefinger configuration affecting the direction of gravitational force application, andthe EDC conditions during the experiment appear to be prominent.Order of flexion of joints. The order of flexion of the PIP and MCP joints is alteredwith the change in load on the intrinsic muscles. The MCP joint flexes earlier andfaster with heavier loads, compared to a more gradual and delayed flexion of theMCP when a light load is applied on the LUM tendon. This can be observed in thetrajectory and angle graphs reproduced from [Kamata et al., 2016] in Fig. 7.2(a).80The observed joint angles are reproduced in Fig. 7.2(b). When the LUM tendon isnot loaded, the PIP flexes faster (∼ 45◦ change in 10 s) compared to the MCP (lessthan 40◦ change in 10 s). When the LUM tendon is loaded with 1.960 N, however,the MCP reaches full flexion in about 3 seconds, while the PIP does not begin flexingtill much later (at ∼ 10 s).Fingertip to palm distance. When the MCP flexes before the PIP, the fingertip topalm distance also increases during flexion [Kamata et al., 2016; Arnet et al., 2013].This effectively increases the reach of the grasp. Hence it is hypothesized by Arnetet al. [2013] that the intrinsic muscles play a role in making the hand grasp moreamenable to bigger objects. The pre-grasp posture of the hand is also observed,in vivo, to involve significant extension of the joints for pre-shaping before objectcontact [Bain et al., 2015].MCP stabilization. The load on the LUM tendon modifies the lateral angle ob-served at the MCP. Kamata et al. [2016] observe that the angle of inclination of thefinger in the ulnar direction reduces with increasing load. This is interpreted as thelumbrical playing the role of a stabilizer against gravity for the MCP joint.7.4 ObjectivesInspired by the observations of Arnet et al. [2013] and Kamata et al. [2016], wedecided to build a model of the function of the lumbrical muscle. This three-dimensional biomechanical model should be capable of simulating the motion ofthe finger under different LUM loads. Along the lines of the cadaver experiments,the flexor (FDP) should be pulled at constant velocity to flex the finger with differentloads on the LUM muscle. Such a simulation would help establish the importantanatomical features of the human hand that aid the role of the LUM muscle inextending the reach of the hand grasp. Moreover, with the capabilities of oursimulator, we can also model this function of the LUM muscle with the muscleoriginating at the FDP tendon. With this modified simulation, we can observe thefunction of the lumbrical as would be expected in vivo.817.5 Methods7.5.1 Index Finger ModelWe designed a simulation model of the human index finger, capable of computingthe force transmission between tendon segments and resulting motion across thetendon structure. The mathematical treatment of the dynamics of the differentstructures—bones, joints, and tendons—for the index finger is described by Suedaet al. [2011] and covered in Chapter 3. The bones are modeled as rigid objects,joints as constraints between them, and tendons as inextensible strands with routingconstraints. We deal with the branching of LUM tendon and the lateral and centralbands as discussed in Section 6.1. On top of this model, we adapt the obstacleset method [Garner and Pandy, 2000] to improve tendon routing as described inSection 6.3, along with a joint stiffness model described below.Index finger structureThe geometric model of the index finger was constructed from anonymized CT scansof the finger bones of one of the subjects of the study by Kamata et al. [2016]. Thedesign of the extensor mechanism is based on the reduced model given by Garcia-Elias et al. [1991a] (see Fig. 6.1). The sheet-like extensor apparatus is reducedto a network of thin strands. Note that the experiment described by Kamata et al.[2016] leaves most tendons slack. Therefore, we model the LUM and FDP tendonsin isolation, with LB, CS, and TS – the relevant parts of the extensor mechanism. Inaddition, the OL is added to the model to provide coupling of the IP joints [Leijnseet al., 2010]. See Fig. 7.3 for a schematic of the index finger model. The weightedLUM tendon in Fig. 7.3(a) is replaced with a LUM muscle originating at the FDPtendon for the second experiment in Fig. 7.3(b).Joint parametersIn the absence of availability of joint stiffness and damping parameters for thesubjects studied by Kamata et al. [2016], a joint stiffness model was constructedbased on previously reported results [Kuo and Deshpande, 2012; Kamper et al.,2002], and joint damping parameters measured by Hajian and Howe [1997].82(a) Weighted lumbrical experiment schematic(b) in vivo lumbrical experiment schematicFigure 7.3: Schematics of the index finger model used for experiments.−20 0 20 40 60 80MCP Flexion Angle (degree)−200−150−100−50050100Passivemoment(Nmm)Best fit [Kuo, Deshpande 2012]ScaledNeutral Flexion AngleRange [Kuo, Deshpande 2012]Figure 7.4: Scaled MCP stiffness model in comparison to reference parametersreported by Kuo and Deshpande [2012]. The neutral flexion angle ishighlighted with a dot.83For joint stiffness data, the large variability in the observations by Kuo andDeshpande [2012] suggests that a specific model for a given hand model mightbe necessary. To create an approximate model for MCP stiffness for the cadaverbehaviour observed by Kamata et al. [2016], we used the MCP stiffness curvereported by Kuo and Deshpande [2012, Fig. 3]. The linear coefficients were scaledto satisfy the following relationship at θu, the MCP flexion upper limit, betweenτmcp, the MCP joint passive stiffness torque, and τ200g, the torque generated at theMCP joint by the 200 g load attached to the LUM.τmcp(θu)< τ200g(θu) (7.1)Equation (7.1) ascertains that the stable flexion angle attained by the MCP, i.e., theangle at which the passive flexion of the MCP can counter the torque generatedat the joint by the load applied to the lumbrical, under a 200 g load is the sameas the flexion upper limit. This is a necessary requirement to match the observeddata. Without any passive joint stiffness at the MCP, its response to any weight onthe LUM tendon is binary, i.e., it will either move to full flexion, or full extensionwithout any FDP excursion, depending on whether the torque from the lumbrical, asmodelled, favours flexion or extension of the MCP. By contrast, the data observedby Kamata et al. [2016] shows that below 100 g lumbrical load, the MCP does notapproach full flexion under the torque applied by the LUM load without significantFDP excursion. Therefore, a passive joint stiffness model is deemed necessary for asatisfactory model of LUM muscle function.For the joint damping of the MCP joint, we start with the in vivo parameter valuereported by Hajian and Howe [1997] and tune it manually to fit our model withthe reported cadaver behaviour [Kamata et al., 2016]. It is worth noting that thechoice of joint damping parameters significantly impacts the behaviour of the fingersimulation. This impact is reported with different damping values in Section 7.6.The joint dynamics of the IP joints are poorly understood. For the purposes ofthis simulation, we assume the joint stiffness at IP joints to be negligible. This isbound to lead to some artifacts, since the joints are not truly stiffness-free, but weavoid introducing unnecessary uncertainty of poorly understood parameters.84Muscle and tendon materialIn order to model the LUM muscle function via the extensor mechanism, the tendonsare modelled as inextensible strands. Many models that describe and/or simulate theextensor apparatus [Valero-Cuevas et al., 1998, 2007; Qian et al., 2014; Synek andPahr, 2016; Dogadov et al., 2017] treat the extensor mechanism as an arrangementof extensible elements such as elastic strings (only active when taut). However, thisrequires the exact geometry of the extensor mechanism to be reproduced for precisefunction. By using an inextensible material model for tendons, the forces acrossthe tendon sections can be determined independent of the precise configuration ofthe extensor apparatus. This choice also improves numerical performance, allowingfaster iterations over the model. The second simulation experiment describedin Section 7.5.2 requires a muscle model for the LUM muscle. We used the piece-wise linear muscle model described in Chapter Simulation ExperimentsWeighted lumbricalFor the first simulation experiment, the finger is set to the starting configurationdescribed by Kamata et al. [2016]. The finger is set to a zero-extension posture—flexion angle for both IP joints and the MCP joint set to zero—where applicable. Insome cases, the initial posture for the finger chosen by Kamata et al. [2016] happensto deviate from the zero extension posture. In these cases, an extra wrench is appliedacross the corresponding joints for an interval of one second to achieve the initialobserved posture.The lumbrical muscle is kept under a constant weight w. At the start of thesimulation, the FDP tendon is pulled in the proximal direction at 2 mm/s, and thejoint flexion angles—DIP, PIP, and MCP—are recorded. The experiment ends whenthe FDP tendon cannot be pulled any longer when the joints are at their respectivelimits of flexion. This experiment is repeated for w ranging from 0.098 N−1.960 Ncorresponding to 10 g-200 g weight used by Kamata et al. [2016]. This simulation isperformed without the OL first, and then with the OL to assess the role of the OL inmaintaining IP joint coupling. The results are compared with observations reported85by Kamata et al. [2016].Emulating anatomical structureFor the second experiment, this same simulation is repeated, but with the originof the LUM muscle at the FDP tendon at the palmar side along the metacarpalbone [Eladoumikdachi et al., 2002] (see Fig. 7.3(b)). This mimics the anatomicalstructure of the LUM muscle more closely. At any given MCP flexion, the flexionangle of the PIP determines the reach and shaping of the grasp, along with howthe grip force is distributed across the palmar surface [Al-Sukaini et al., 2016]. Tomeasure the effect of the LUM muscle on the fingertip reach, the PIP flexion angleis measured against the MCP flexion angle for the various activation levels of thelumbrical muscle. The difference between the first and second experiment lies in theco-excursion of LUM and FDP tendons in the latter simulation. in vivo PIP flexion vs.MCP flexion observations for the intrinsic-dominant and extrinsic-dominant flexionwere reported by Al-Sukaini et al. [2016]. We use these as comparison for ourresults.7.6 ResultsUpon simulation, the weighted lumbrical model described in Section 7.5.2 faithfullyreproduces the effect of constant tension applied to the lumbrical on the change inorder of flexion of MCP and PIP joints. In Fig. 7.5, the halfway flexion point forthe different joints have been marked. As the load on the LUM tendon increases(without OL as in Fig. 7.5(a), as well as with OL as in Fig. 7.5(b)), the rate of flexionof MCP keeps rising, resulting in the halfway flexion point moving to the left ofthe curve. On the other hand, the PIP flexion starts later with increasing load onthe LUM tendon, leading to a halfway point further right along the curve. Withincreasing load, the halfway points of the MCP and PIP move away from each other.The motion of the DIP joint in the absence of the OL does not appear to bedirectly affected by the PIP joint. PIP joint flexion does not change significantly uponintroduction of the OL. However, when the OL is present, the halfway flexion pointsfor the DIP joint are aligned with those of the PIP joint (see Fig. 7.5(b)). Therefore,our model successfully demonstrates the IP joint coupling principle [Hahn et al.,86Figure 7.5: Flexion of index finger joints under various loads applied to thelumbrical tendon. The OL forces DIP flexion to follow PIP flexion.1995; Leijnse et al., 2010].In addition, we would like to emphasize the importance of joint dynamicsfor explaining the data observed by Kamata et al. [2016]. For this purpose, themodel is simulated for varying MCP damping values to capture the importanceof considering joint dynamics in the simulation model. As shown in Fig. 7.6,insufficient joint damping causes non-smooth MCP joint movements. Increasing thedamping parameter makes the IP joints respond earlier to flexion, and reduces theangular velocity of the MCP joint for any given load on the LUM tendon.The results for the anatomical emulation model, described in Section 7.5.2,show similar lumbrical muscle function in extending the reach of the hand grasp bydelaying the flexion of the PIP joint in comparison to the MCP joint. The results areproduced in Fig. 7.7 with a similar color scheme as Fig. 7.5 representing increasingactivation. For this particular model, we also provide the PIP vs MCP flexion anglesfor different activation levels of the lumbrical muscle. The change in the curve withincreasing activation is comparable to in vivo results reported by Al-Sukaini et al.[2016].87Figure 7.6: Effect of reduced MCP damping on LUM muscle function duringflexion.7.7 DiscussionOur simulation method is capable of emulating anatomical structure of the LUMmuscle originating from the FDP tendon, and attached into the extensor hood. Whenthe LUM tendon is pulled with greater force, the IP joints flex later, while the MCPjoint flexes earlier in response to a constant pull on the primary flexor, FDP. This isin accordance with observations described in previous studies [Arnet et al., 2013;Kamata et al., 2016]. This function of the lumbrical can also be interpreted as anincrease in the reach of the fingertip for a partially flexed MCP joint. Unlike theaforementioned cadaver studies, we also simulate the coupling of the LUM muscle88Figure 7.7: Flexion of index finger joints under various activations of theLUM muscle modeled to emulate the anatomical structure. The PIP jointflexion is delayed by lumbrical muscle activation. Note that each curvecorresponds to a level of LUM muscle activation.with the FDP tendon to emulate the in vivo functional anatomy of the LUM muscle.The anatomical emulation model can be used to change the strength of the LUMmuscle or activation incrementally, and observe the change in flexion trajectory. Thiscan serve as a tool for evaluating relative LUM muscle strength based on measuredtrajectory and joint stiffness. Moreover, this model reinforces the importance ofconnective tissue other than tendons in the flexion of the human hand. The impactof the joint stiffness provided by ligaments at the MCP joint, the damping inherentto the MCP joint, and the IP joint coupling due to OL, are clearly demonstrated withthis simulation.The various assumptions made in the process of building this simulation canguide future experiments to help parameterize a model of this complexity. Inparticular, experiments to study the motion of individual joints of a cadaver fingerin isolation in response to flexion would aid in constructing joint stiffness functions89for the described finger tendon model. In the absence of recorded joint stiffnessmeasurements by Kamata et al. [2016], we chose to use the best fit parametersmeasured in vivo by Kuo and Deshpande [2012] and scale them as necessary forthe MCP joint. The scaled parameters do not fall in the measured range of jointstiffness parameters, but since the parameter measurements were done in vivo, therecould be compounding factors due to connective tissues not taken into account inthis simulation model. Isolated joint stiffness measurements for the IP joints werealso unavailable, and were chosen to be zero. Despite these shortcomings, thissimulation model successfully reproduced LUM muscle function.In summary, we have described the first simulation method for the role of thelumbrical in hand grasp motion that constructs the motion of the finger based onforce inputs. Our simulation experiments show that the lumbrical modifies therelative motion of MCP and PIP joints as a primary function during flexion, asconjectured by Leijnse and Kalker [1995]. We overcome the limitations of previousstudies with a 3D model capable of computing the coupled impact of musculotendonand joint dynamics on the motion of the finger. With this model, we conjecture thatthe coupling of the IP joints by the OL, and the passive stiffness and damping ofthe MCP joint play an important role in LUM muscle function. Additional cadaverexperiments can help better parameterize a model of this complexity.90Chapter 8Modelling the HandIn Chapter 7 we discussed the modelling of the index finger to simulate the lumbricalmuscle for anatomical validation. The techniques discussed so far, and the softwaredescribed in Chapter 5 can further be used to model the complete hand. In thischapter we explore the modelling of the thumb and the wrist towards simulating theentire hand.8.1 ThumbThe thumb is responsible for the dexterity of the human hand. If the thumb is lostin an accident, the hand is almost useless, and complex procedures are carried outto replace it with a finger in a procedure known as the “pollicization of a finger”[Buck-Gramcko, 1971; Taghinia and Upton, 2011]. The most important function ofthe thumb of the human hand is opposition—the ability to bring into contact thepulp of the thumb and any other fingertip. Opposition of the thumb with each of thefingers makes the thumb indispensable to human hand function.Opposition of the thumb is made possible because of a sum of elementarymovements [Kapandji, 2007] about all three axes of rotation for the first metacarpalbone about the trapezometacarpal or TM joint, and secondary movements of theother bones about the MCP and IP joints of the thumb (see Fig. 8.1). The majorityof the range of motion of the thumb is provided by two axes of rotation of the TMjoint [Kapandji, 2007; Hollister et al., 1992]. However, neither of the TM joint axes91(a) Sagittal view(b) Dorsal viewFigure 8.1: Schematic representation of the thumb musculotendon structurewith the relevant modelled muscles. Note the axes directions and loca-tions for the joints of the thumb.lie in the anatomical plane of the joint. The joint does not have a single centreof rotation. One axis, primarily for flexion/extension* motion with some axialrotation for the metacarpal, passes through the trapezium. The second axis passesthrough the metacarpal. The two axes do not intersect. For further details on thesecompound motions, please see [Hollister et al., 1992; Chao et al., 1989].Since our simulation software supports any axis of rotation as a constraint,opposition may be simulated in our work. However, setting the parameters perfectlyfor modelling the thumb opposition with all the fingers of the hand is beyond thescope of this thesis. To complete the hand model, however, we developed a thumbmodel with the required component movements for opposition using some of theextrinsic muscles as a proof of concept. For this thumb model, we choose a TM joint*sometimes referred to as antepulsion/retropulsion92design similar to the MCP joints of the other digits with a single centre of rotationbetween the two bones, while the MCP joint of the thumb is modelled with a hingejoint. We modelled three of the nine motor muscles of the thumb, the extensorpollicis longus (EPL), flexor pollicis longus (FPL), and the adductor pollicis (AP).These muscles were chosen to effectively model some simple thumb movements.8.1.1 Example Thumb MovementsWith the three muscles in place, we built the following representative examples topresent the typical motions of the thumb.Extension - with and without adduction. Extension of the thumb by activatingthe EPL muscle, as the name suggests, extends the IP, MCP, and TM joints . In theabsence of adduction using the AP muscle, it also abducts the TM joint (Fig. 8.2(c)-(d)). However, when the AP is activated, the TM joint is abducted, but still extended(Fig. 8.2(a)-(b)).Adduction. Activating only the adductor muscle AP allows a partially flexed thumbto be brought closer to the digits as in Fig. 8.3(a)-(b).Flexion. The thumb may be flexed to bring it over the palm as in Fig. 8.3(c)-(d).To achieve this, only the FPL muscle is activated. As with the extension example,adduction would bring the thumb closer to the palm.8.2 Wrist and Tenodesis GraspThe wrist is made up of 8 different bones and the relative motion of these bonesis challenging to model. We simplified this structure with the wrist modelled asone rigid body, with no relative motion between the carpal bones. However, thissimplification does not prevent simulation of useful wrist motions. Tenodesis graspis an example of such a motion involving the wrist and the finger joints.The concept of “dynamic synergistic wrist motion” was established in a series ofin vitro experiments by Cooney et al. [1989]; Horii et al. [1992]. This motion refersto the passive digital motion induced while performing synergistic wrist motions.93(a) Side view - Adducted extension(b) Top view - Adducted extension(c) Side view - Extension(d) Top view - ExtensionFigure 8.2: Thumb motion examples: Side and top views of adducted exten-sion, and extension respectively. Note the adduction change between (b)and (d).When the wrist is extended via one of the three carpal extensors†, the extensortendons of the wrist are rendered lax, and the flexor tendons are rendered taut(referred to as the tenodesis effect). The taut flexor tendons lead to passive flexionof the digital joints, referred to as the “tenodesis grasp.” Alternatively, during wristflexion, the extensor tendons are rendered taut, leading to extended digital joints.Understanding this synergistic wrist motion could lead to better splint design†extensor carpi radialis longus, extensor carpi radialis brevis, extensor carpi ulnaris94(a) Bottom view - adduction (b) Top view - adduction(c) Front view - flexion (d) Side view - flexionFigure 8.3: Thumb motion examples: Adduction and flexion examples of thethumb.95which respects the natural kinematics of the hand. Towards this goal, Su et al.[2005] investigated the passive digital motion induced with synergistic wrist motionin vivo using a three dimensional motion analysis system. Their results, reproducedin Fig. 8.4, show the inverse correlation between wrist angle and the IP joint angles,capturing the essential effect of tenodesis. PIP joint was found to have the highestcorrelation, followed by the MCP joint and then the DIP joint.To reproduce the effect of tenodesis on the finger joints, we model the EDCand FDP tendons. For the EDC, only the central band is modelled with the middleinsertion at the base of the middle phalanx. For the wrist, an extensor tendon ismodelled, but not the flexor, since gravity produces flexion of the wrist. With thistendon network, we need to set the right tendon lengths and passive muscle stiffnessparameters. Both EDC and FDP muscle lengths need to be in a range determined bythe following criteria:• If the EDC musculotendon is too long, the tendon may be slack when thewrist is in full flexion, or it may not apply enough force to have an observabletendonesis-based extension of the digital joints.• If the EDC is too short, it may not allow full flexion of the finger even in afully extended wrist configuration. In flexion of the wrist under gravity, itmay be supporting a large portion of the weight of the wrist, because of theexcessive passive tension at the EDC muscle.• Similarly, if the FDP musculotendon is too short, it may hinder the naturalextension range of the finger.• If the FDP musculotendon is too long, the tendon may be slack when the wristis in full extension, or it may not apply enough force, to have any observabletendonesis-based flexion of the digital joints.With these criteria, we design a model of the hand with the wrist and simulatethe motion of the fingers throughout the range of wrist flexion/extension. Theparameters are decided with these criteria by trial and error for this preliminaryexperiment and warrant further refinement.968.2.1 Tenodesis Results(a) PIP (b) MCP(c)Figure 8.4: PIP and MCP joint angles across the range of wrist flexion. (a)-(b)in vivo results by Su et al. [2005]. (c) Our tenodesis simulation results.We observe the PIP and MCP joint angles through the range of wrist flexion inour simulation model (Fig. 8.4(c)). For comparison, PIP and MCP angles observedby Su et al. [2005] are reproduced in Fig. 8.4(a)-(b). The effect of tenodesis isvisible as the inverse correlation of both the PIP and MCP angles with respect to thewrist flexion angle. The corresponding simulation visualization for the extendedand flexed wrist posture is reproduced in Fig. 8.5. The results for DIP joint are not97listed here because the DIP joint angles are dependent on the weak correlation withthe PIP joint enforced by the OL (see Section A.2 and Section 4.4.1).(a) Extended wrist posture (b) Flexed wrist postureFigure 8.5: Tenodesis simulation visualization: Effect of extended and flexedwrist postures on the PIP and MCP joints.AnalysisThe results of this experiment may be dependent on the parameters of the model,and this will require further study. However, we can give a preliminary analysis ofthese simulation results:• The inverse correlation of the PIP and MCP flexion angle vs the wrist flexionangle is reproduced by our model as expected.• The PIP covers a wider range of flexion angles for the range of flexion of thewrist joint, with a higher negative slope than the MCP, similar to the in vivoobservations [Su et al., 2005].• We could not reproduce the decreasing slope of the correlation of the MCP inFig. 8.4(b). This would require further investigation.98Chapter 9ConclusionIn this thesis, we had set out to create a validated human hand model for itsvarious applications. This thesis paves the way for such a model to be constructedin numerical methods as well as software, with some future work to be doneto achieve the complete objectives. By means of these projects, this researchprovides a platform for biomechanics researchers and roboticists to perform virtualexperiments on the human hand in a cost effective manner via simulation, toenhance our understanding of the functioning of the human hand. Building acomprehensive simulation of the human hand with robust neuromuscular controlis a large undertaking. We broke it down into tractable pieces in this thesis andtook significant steps towards building a framework for human hand modelling,simulation, validation, and control. We review the work done towards each objectiveof this thesis in order, and evaluate the successes and limitations.9.1 Contributions9.1.1 Tendon DynamicsWe first tackled the modelling techniques required to simulate the dynamics ofthe human hand. This involves the simulation of hand bones with joints, tendons,and muscles. The work on modelling tendon-like strands by Sueda et al. [2011]using Eulerian-on-Lagrangian discretization paved the way to incorporate these99tendon motion constraints without explicit collision handling. The technique ismodified to suit tendon simulation needs and used to simulate a hand model with asimplified anthropomorphic tendon structure [Sachdeva et al., 2015]. We describethe dynamics in Chapter 3, and the resulting finger simulation in Chapter 4. Thisanthropomorphic simulation model can also be modified to simulate deformities ofthe hand, such as the boutonnière deformity.Our model incorporates the elastodynamics of stiff tendons. By specifyingthe routing constraints of tendons using Eulerian-on-Lagrangian discretizationof tendons with selective-quasistatic, we are able to efficiently simulate tendondynamics in a rigid body framework. These techniques accomplish the first objectiveof this thesis.9.1.2 Tendon-Driven ControlUsing this simulation framework, we successfully implemented precise fingertip tra-jectory control on the simulated hand model using two control methods—“whitebox”control using internal dynamics information, and data-driven “blackbox” control.We learned that the “blackbox” controller negotiates joint limits much better, whichis a result of the dynamics information available in the collected data for the con-troller, as opposed to “whitebox” controller, which does not make use of unilateralconstraints like joint limits when computing dynamics. On the other hand, in con-tinuous motion where joint limits do not come into play, the “whitebox” controllerperforms much better. In Chapter 4, we describe these control methods and theirapplication to a biomechanical hand simulation model.Our goal for this objective was to develop techniques to perform musculoskeletalcontrol for precise fingertip trajectory tracking. The two techniques achieve thisgoal with some limitations. While our simulation model can track the fingertip withsome degree of accuracy, the results attained with these techniques are a little errorprone, with unwanted oscillations of the fingertip, as noted in Chapter 4. Anotherlimitation of the control results is the occasional over-activation of muscles, whichis uncharacteristic of smooth hand control.1009.1.3 Anatomical Simulation SoftwareEven with the improved simulator, it was difficult to iterate over the design of thesimulation model. Since there was no easy way to interactively edit the modeldetails and observe the effect on the simulation, each iteration over the simulationmodel was time consuming, and the effect of model changes were difficult toassess. Therefore, we identified the need for new simulation software, which wouldbuild on the lessons learned. A new biomechanics simulation software system wasdeveloped, which simplified the design process and the evaluation of the results ofthe simulation.Beyond just developing the required numerical software and designing the re-quired numerical techniques, our goal was to provide design tools to allow visualinteraction with the framework for modelling and simulation. Text based represen-tation of the design or parameters without visual feedback make design iterationscumbersome and unproductive. Easy to use design tools with an integrated sim-ulation library can allow biomechanics researchers to not have to deal with theunderlying library or formats used for simulation data, while getting faster feedbackon the design and functioning of their model. A new biomechanics simulationsoftware system was developed with these goals in mind, which simplified thedesign process and the evaluation of the results of the simulation.A simulation library written in C++, named Strands, forms the centrepiecearound which a number of tools are created. These tools were designed for easeof modelling, experimentation, and control of biomechanics of the hand. Forthe interactive design of the simulation model, a host of plugins for AutodeskMaya® were created. Maya® provides the required graphical modelling tools,and the plugins augment its functionality to allow biomechanical modelling andvisualization of the results generated from the simulation library Strands. Moreover,Python extensions were provided for the library for experimentation and controlworkflows. With this Python plugin, controllers for the simulator can be built usingany of the myriad of machine learning tools available. We described the softwaretools and their corresponding workflow in detail in Chapter 5, and also presentsimple examples for each workflow.Our biomechanics simulation framework fulfills our objectives of providing101a visual modelling interface allowing fast iterations over biomechanical models,as well as our objective of aiding the development of control strategies. With thissoftware, researchers can effectively share their simulation and control experimentsin a reproducible fashion, something lacking from previous related works [Suedaet al., 2008; Sueda, 2010; Synek and Pahr, 2016; Dogadov et al., 2017].9.1.4 Validated Hand Model ConstructionFor these software tools to be useful to the biomechanics community, the simulationsgenerated should be able to generate motion sequences which can be validated within vivo and in vitro experimental data. We put these software tools to the test bysimulating an anatomically realistic model of the extensor hood to validate functionof the lumbrical in modulating flexion of the finger, observed in in vitro experimentsby Arnet et al. [2013] and Kamata et al. [2016], and in vivo observations by Al-Sukaini et al. [2016]. These experiments are run using the Python extensions forthe Strands library. This led to significant additions in the dynamics modelling forthe finger, which are discussed in Chapter 6, while the experiments are covered inChapter 7.To our knowledge, this is the first three-dimensional dynamics model of lumbricalmuscle function that is able to simulate the flexion of the joints of the index fingerduring finger flexion motion with the FDP under different levels of activation ofthe LUM muscle. With this model, we illustrated that the LUM muscle modifiesthe relative motion of MCP and PIP joints as a primary function during flexion, asconjectured by Leijnse and Kalker [1995]. Previous computational models of thelumbrical were limited to 2D and could not compute the motion of the finger fromits dynamics. Our model overcomes these limitations with a 3D model capable ofcomputing the coupled impact of musculotendon and joint dynamics—stiffness anddamping parameters—on the motion of the finger. The MCP joint was modelledas a hinge joint to focus on the flexion/extension of the joints of the finger. Wefurther modified the model to emulate the anatomical structure of the lumbrical[Zancolli, 1979; Leijnse and Kalker, 1995], and compared the flexion of the MCPjoint to the flexion of the PIP joint, at different activation levels of the LUM muscle,as observed by Al-Sukaini et al. [2016]. Our simulation supports the hypothesis102that the IP joint coupling due to the OL and the MCP joint stiffness and damping areimportant elements of lumbrical function.Hand ModellingLastly, to extend this work to detailed anatomical modelling of the complete hand,we present simulations of the thumb and the wrist in Chapter 8. Modelling thethumb is a necessary step towards creating any form of tendon driven hand graspingcontroller, and by simulating example thumb motions, we show that this can bedone in our framework. With the wrist model, we present the flexion of the fingersresulting from the extension of the wrist, called the tenodesis grasp. This is com-pared to in vivo tenodesis grasp observations by Su et al. [2005]. Some observationswere successfully reproduced, but the MCP behaviour in our simulation does notmatch the observations of Su et al.. This discrepancy may be due to the stiffnesscurve of the MCP joint (see Section 7.5.1) and requires further investigation.Our final objective in this thesis was towards the construction of a validatedhand model. With the results we present about our lumbrical model in Chap-ter 7, we provide preliminary validation of our simulation techniques, modelling aclinically-observed phenomenon not covered by previous biomechanical models. Acomprehensive hand simulation is incomplete without modelling the wrist and thethumb, so we provide a model for both of them with some minimal anatomical ele-ments as a proof-of-concept that these structures can be modelled with our system.In the future, the modelling of the complete anatomy for the wrist and the thumb,and simulations of more such clinically-observed phenomena should be compiledto increase the confidence of biomechanics researchers in the usefulness of thesetechniques.9.2 Limitations and Future WorkDespite the successes of the presented methods, there are some gaps to fill in orderto complete the objectives of the thesis.The routing of tendons using Eulerian-on-Lagrangian discretization is based onsome assumptions about the movements of the tendon in particular directions. Char-acterizing the motion of the tendon segments, particularly for the dorsal aponeurosis,103is one way to identify the validity of these assumptions. Tracking tendon movementaccurately in relation to the bone is a non-trivial task. One way to do this wasimplemented by Sueda [2010]—stainless steel markers injected into the tendons.The motion of these markers can be observed in CT scans of finger motion generatedwith the device utilized by Kamata et al. [2016].Another major limitation on the applicability of this work is the difficulty inextracting the parameters important to modelling the dynamics of the human hand.The measurement of isolated parameters such as tendon stiffness, joint damping, andjoint stiffness are difficult to measure in vivo. With the right in vitro measurementsetup, it is feasible to measure these parameters. One could measure these usingthe setup deployed by Kamata et al. [2016]. This is a resource intensive exercise,and it is unclear whether measuring the parameters precisely for a set of cadaverhands would aid fitting the parameters to a new hand specimen. The problemof parameterization could alternatively be tackled by fitting parameters to matchmeasured hand motion data. In Chapter 7, for example, we fit joint parameters bytrial and error to demonstrate the viability of using our hand simulation to reproducelumbrical muscle function. For more accurate parameter fitting, the first step wouldbe to identify a usable dataset to match in simulation.With the right set of parameters, and underlying motion model, an accuratedynamics model can be built using the software framework presented in this thesis.Control of such a model of the human hand requires a two-level approach - a lowerlevel control for fingertip trajectory, and a higher level control for overall handmotion. We developed techniques for lower level control of fingertip trajectory, buthigher level control of the hand is not addressed in this thesis. With the right tech-niques, the fingertip trajectory controller presented in this thesis can be integratedwith hand level control for gestures, grasping, and other hand motions.There are multiple potential applications for this research, each with its owndirection of projects. This work could be applied in surgical simulation and research.This is probably the most well-defined future project stemming from this thesis. Anumber of deformities of the human hand can be modelled in our hand simulation(see Section 4.4.2). This provides the opportunity to test new procedures for differenthand deformities in our simulation framework. Restoring finger function in thecases of structural deformities like the boutonnière deformity require specialized104procedures such as the Matev procedure [Matev, 1964]. These procedures arebuilt on an intuitive understanding of hand function, and studying their effect onthe hand with cadaver experiments. They often involve an attempt at restoringpart of the function by mimicking the original finger structure by making somesurgical alterations. With a simulation model, surgeons can iterate faster over theseprocedures and analyze any shortcomings to optimize them.Another application lies in monocular tracking of human hands for virtualand augmented reality. With an improved hand model, a better understandingof the domain of possible hand postures could be mapped, leading to improvedtracking results. This may require parameterizing the simulation for the user’s handmotion. The topology of the tendons of the hand may vary from person to person[Perkins and Hast, 1993]. Once the topology has been decided, we can work onparameterizing the hand model.Domains of AR/VR [Han et al., 2018b], robotic hands for prosthetic/industrialapplications [Blana et al., 2016; Andrychowicz et al., 2018], and hand surgery[Duzgun et al., 2017] are seeing significant advances year after year. The availabilityof a comprehensive hand simulation framework to aid research would significantlyadvance the state-of-the-art in these areas.105BibliographyA. Al-Sukaini, H. P. Singh, and J. J. Dias. Extrinsic versus intrinsic hand muscledominance in finger flexion. Journal of Hand Surgery (European Volume), 41(4):392–399, Jan 2016. ISSN 2043-6289. doi:10.1177/1753193415619774. →pages 77, 78, 86, 87, 102Irene Albrecht, Jörg Haber, and Hans-Peter Seidel. Construction and animation ofanatomically based human hand models. In Proceedings of the 2003 ACMSIGGRAPH/Eurographics symposium on Computer animation, pages 98–109.Eurographics Association, 2003. → pages 6, 33K.N. An, E.Y. Chao, W.P. Cooney, and R.L. Linscheid. Normative model of humanhand for biomechanical analysis. Journal of Biomechanics, 12(10):775–788, Jan1979. ISSN 0021-9290. doi:10.1016/0021-9290(79)90163-5. → pages 70, 79K.N. An, Y. Ueba, E.Y. Chao, W.P. Cooney, and R.L. Linscheid. Tendon excursionand moment arm of index finger muscles. Journal of Biomechanics, 16(6):419–425, 1983. doi:10.1016/0021-9290(83)90074-x. → pages 6, 14, 78Michael Skipper Andersen, Jian Yang, Mark de Zee, Lelai Zhou, Shaoping Bai, andJohn Rasmussen. Full-body musculoskeletal modeling using dual microsoftkinect sensors and the anybody modelling system. In 14th InternationalSymposium on Computer Simulation in Biomechanics, pages 23–24, 2013. →page 12Alexandr Andoni and Piotr Indyk. Near-optimal hashing algorithms forapproximate nearest neighbor in high dimensions. Communications of the ACM,51:117–122, Jan 2008. doi:10.1145/1327452.1327494. → pages 42, 43Marcin Andrychowicz, Bowen Baker, Maciek Chociej, Rafal Jozefowicz, BobMcGrew, Jakub Pachocki, Arthur Petron, Matthias Plappert, Glenn Powell, AlexRay, et al. Learning dexterous in-hand manipulation. arXiv preprintarXiv:1808.00177, 2018. → pages 16, 105106MOSEK ApS. MOSEK Optimizer API for C. Version 8.1., 2017. → page 61Ursina Arnet, David A. Muzykewicz, Jan Fridén, and Richard L. Lieber. Intrinsichand muscle function, part 1: Creating a functional grasp. The Journal of HandSurgery, 38(11):2093–2099, Nov 2013. ISSN 0363-5023.doi:10.1016/j.jhsa.2013.08.099. → pages 10, 15, 76, 77, 79, 80, 81, 88, 102Uri M Ascher. Stabilization of invariants of discretized differential systems.Numerical Algorithms, 14(1-3):1–24, 1997. → page 32Uri M. Ascher, Hongsheng Chin, and Sebastian Reich. Stabilization of daes andinvariant manifolds. Numerische Mathematik, 67(2):131–149, Mar 1994. ISSN0945-3245. doi:10.1007/s002110050020. → page 32G. I. Bain, N. Polites, B. G. Higgs, R. J. Heptinstall, and A. M. McGrath. Thefunctional range of motion of the finger joints. Journal of Hand Surgery(European Volume), 40(4):406–411, May 2015. ISSN 2043-6289.doi:10.1177/1753193414533754. → page 81David Baraff and Andrew Witkin. Large steps in cloth simulation. In Proceedingsof the 25th annual conference on Computer graphics and interactive techniques -SIGGRAPH ’98, 1998. doi:10.1145/280814.280821. → page 28Patrick Bédard and Jerome Sanes. Gaze and hand position effects onfinger-movement-related human brain activation. J. Neurophysiol., 101(2):834–842, Feb 2009. doi:10.1152/jn.90683.2008. → page 41Miklós Bergou, Max Wardetzky, Stephen Robinson, Basile Audoly, and EitanGrinspun. Discrete elastic rods. ACM Trans. Graph., 27(3):63:1–63:12, Aug2008. → page 23Dimitra Blana, Edward K. Chadwick, Antonie J. van den Bogert, and Wendy M.Murray. Real-time simulation of hand motion for prosthesis control. ComputerMethods in Biomechanics and Biomedical Engineering, pages 1–10, Nov 2016.ISSN 1476-8259. doi:10.1080/10255842.2016.1255943. → page 105Silvia S. Blemker and Scott L. Delp. Three-dimensional representation of complexmuscle architectures and geometries. Annals of Biomedical Engineering, 33(5):661–673, 2005. doi:10.1007/s10439-005-1433-7. → page 15Jeannette Bohg, Antonio Morales, Tamim Asfour, and Danica Kragic. Data-drivengrasp synthesis—a survey. IEEE Transaction on Robotics, 30(2):289–309, 2014.doi:10.1109/tro.2013.2289018. → page 7107Susanne Bradley. Applications of machine learning in sensorimotor control.Master’s thesis, University of British Columbia, 2015. → page viHelmut J. Buchner, Margaret J. Hines, and Hooshang Hemami. A dynamic modelfor finger interphalangeal coordination. Journal of Biomechanics, 21(6):459–468, 1988. doi:10.1016/0021-9290(88)90238-2. → page 14Dieter Buck-Gramcko. Pollicization of the index finger. The Journal of Bone &Joint Surgery, 53(8):1605–1617, Dec 1971. ISSN 0021-9355.doi:10.2106/00004623-197153080-00015. → page 91Robert R. Burridge, Alfred A. Rizzi, and Daniel E. Koditschek. SequentialComposition of Dynamically Dexterous Robot Behaviors. Int J Robot Res, 18(6):534–555, June 1999. ISSN 0278-3649. doi:10.1177/02783649922066385. URLhttp://ijr.sagepub.com/cgi/doi/10.1177/02783649922066385. → page 42L.Y. Chang and Y. Matsuoka. A kinematic thumb model for the act hand. InProceedings 2006 IEEE International Conference on Robotics and Automation,2006. ICRA 2006., 2006. doi:10.1109/robot.2006.1641840. → page 16E Y S Chao, K-N An, W P Cooney, and R L Linscheid. Biomechanics of the Hand.World Scientific Pub Co Pte Lt, 1989. doi:10.1142/0321. → pages2, 6, 14, 76, 92E.Y. Chao, J.D. Opgrande, and F.E. Axmear. Three-dimensional force analysis offinger joints in selected isometric hand functions. Journal of Biomechanics, 9(6):387–IN2, 1976. doi:10.1016/0021-9290(76)90116-0. → pages 3, 14M.B. Cline and D.K. Pai. Post-stabilization for rigid body simulation with contactand constraints. IEEE International Conference on Robotics and Automation,2003. doi:10.1109/robot.2003.1242171. → page 32W.P. Cooney, G.T. Lin, and Kai-Nan An. Improved tendon excursion followingflexor tendon repair. Journal of Hand Therapy, 2(2):102–106, Apr 1989. ISSN0894-1130. doi:10.1016/s0894-1130(89)80047-x. → page 93Michael Damsgaard, John Rasmussen, Søren Tørholm Christensen, EgidijusSurma, and Mark de Zee. Analysis of musculoskeletal systems in the anybodymodeling system. Simulation Modelling Practice and Theory, 14(8):1100–1111,2006. doi:10.1016/j.simpat.2006.09.001. → pages 6, 8, 52Scott L. Delp, Frank C. Anderson, Allison S. Arnold, Peter Loan, Ayman Habib,Chand T. John, Eran Guendelman, and Darryl G. Thelen. Opensim: Open-source108software to create and analyze dynamic simulations of movement. IEEETransactions on Biomedical Engineering, 54(11):1940–1950, 2007.doi:10.1109/tbme.2007.901024. → pages 6, 8, 52A. D. Deshpande, N. Gialias, and Y. Matsuoka. Contributions of intrinsicvisco-elastic torques during planar index finger and wrist movements. IEEETransactions on Biomedical Engineering, 59(2):586–594, 2012.doi:10.1109/tbme.2011.2178240. → page 123Ashish D. Deshpande, Ravi Balasubramanian, Ralph Lin, Brian Dellon, and YokyMatsuoka. Understanding variable moment arms for the index finger mcp jointsthrough the act hand. In 2008 2nd IEEE RAS & EMBS International Conferenceon Biomedical Robotics and Biomechatronics, 10 2008.doi:10.1109/biorob.2008.4762911. → page 6Ashish D Deshpande, Jonathan Ko, Dieter Fox, and Yoky Matsuoka. Controlstrategies for the index finger of a tendon-driven hand. Int J Robot Res, 32(1):115–128, January 2013. ISSN 0278-3649. doi:10.1177/0278364912466925. →page 41S.P. DiMaio and S.E. Salcudean. Needle insertion modelling and simulation. InICRA, volume 2, pages 2098 – 2105 vol.2, 2002. → page 23Anton Dogadov, Mazen Alamir, Christine Serviere, and Franck Quaine. Thebiomechanical model of the long finger extensor mechanism and its parametricidentification. Journal of Biomechanics, 58:232–236, Jun 2017. ISSN 0021-9290.doi:10.1016/j.jbiomech.2017.04.030. → pages 6, 10, 53, 65, 77, 79, 85, 102Feng Dong, G.J. Clapworthy, M.A. Krokos, and Jialiang Yao. An anatomy-basedapproach to human muscle modeling and deformation. IEEE Trans. Visual.Comput. Graphics, 8(2):154–170, 2002. doi:10.1109/2945.998668. → page 12James R. Doyle. Anatomy of the finger flexor tendon sheath and pulley system.The Journal of Hand Surgery, 13(4):473–484, 1988.doi:10.1016/s0363-5023(88)80082-0. → pages 6, 125Guillaume-Benjamin Duchenne. Physiologie der Bewegung. Verlag von TheodorFischer, 1885. → page 126Serdar Duzgun, Alpay Duran, Ekrem Keskin, Ahmet K Yigit, and HasanBuyukdogan. Chronic boutonniere deformity: cross-lateral band technique usingpalmaris longus autograft. The Journal of hand surgery, 42(8):661–e1, 2017. →page 105109Firas Eladoumikdachi, Paula Lee Valkov, John Thomas, and David T. Netscher.Anatomy of the intrinsic hand muscles revisited: Part ii. lumbricals. Plastic andReconstructive Surgery, 110(5):1225–1231, Oct 2002. ISSN 0032-1052.doi:10.1097/01.prs.0000024443.27141.1d. → pages 77, 86M. Epstein and W. Herzog. Theoretical Models of Skeletal Muscle. John Wiley andSibs, 1998. → page 38Tom Erez, Yuval Tassa, and Emanuel Todorov. Simulation tools for model-basedrobotics: Comparison of bullet, havok, mujoco, ode and physx. In 2015 IEEEinternational conference on robotics and automation (ICRA), pages 4397–4404.IEEE, 2015. → page 16Mikhail Fain. Control of complex biomechanical systems. Master’s thesis,University of British Columbia, 2013. → page viYe Fan, Joshua Litven, David I. W. Levin, and Dinesh K. Pai.Eulerian-on-lagrangian simulation. ACM Transactions on Graphics (TOG), 32(3):1–9, 2013. doi:10.1145/2487228.2487230. → page 15Ye Fan, Joshua Litven, and Dinesh K. Pai. Active volumetric musculoskeletalsystems. ACM Transactions on Graphics (TOG), 33(4):1–9, 2014.doi:10.1145/2601097.2601215. → page 15Kim Seng Fok and Siaw Meng Chou. Development of a finger biomechanicalmodel and its considerations. Journal of Biomechanics, 43(4):701–713, 2010.doi:10.1016/j.jbiomech.2009.10.020. → pages 4, 5Marc Garcia-Elias, Kai-Nan An, Lawrence Berglund, Ronald L. Linscheid,William P. Cooney, and Edmund Y.S. Chao. Extensor mechanism of the fingers. i.a quantitative geometric study. The Journal of Hand Surgery, 16(6):1130–1136,1991a. doi:10.1016/s0363-5023(10)80079-6. → pages 3, 9, 14, 66, 77, 82, 126Marc Garcia-Elias, Kai-Nan An, Lawrence J. Berglund, Ronald L. Linscheid,William P. Cooney, and Edmund Y.S. Chao. Extensor mechanism of the fingers.ii. tensile properties of components. The Journal of Hand Surgery, 16(6):1136–1140, Nov 1991b. ISSN 0363-5023. doi:10.1016/s0363-5023(10)80080-2.→ page 67Brian A. Garner and Marcus G. Pandy. The obstacle-set method for representingmuscle paths in musculoskeletal models. Computer Methods in Biomechanicsand Biomedical Engineering, 3(1):1–30, Jan 2000. ISSN 1476-8259.doi:10.1080/10255840008915251. → pages 70, 71, 82110Thomas Geijtenbeek, Michiel van de Panne, and A. Frank van der Stappen. Flexiblemuscle-based locomotion for bipedal creatures. ACM Transactions on Graphics(TOG), 32(6):1–11, 2013. doi:10.1145/2508363.2508399. → pages 12, 16Steven A. Goldstein, Thomas J. Armstrong, Don B. Chaffin, and Larry S. Matthews.Analysis of cumulative strain in tendons and tendon sheaths. Journal ofBiomechanics, 20(1):1–6, Jan 1987. ISSN 0021-9290.doi:10.1016/0021-9290(87)90261-2. → page 9Gaël Guennebaud, Benoît Jacob, et al. Eigen v3. http://eigen.tuxfamily.org, 2010.→ page 61P. Hahn, H. Krimmer, A. Hradetzky, and U. Lanz. Quantitative analysis of thelinkage between the interphalangeal joints of the index finger. Journal of HandSurgery, 20(5):696–699, Oct 1995. ISSN 0266-7681.doi:10.1016/s0266-7681(05)80139-1. → page 87A. Z. Hajian and R. D. Howe. Identification of the mechanical impedance at thehuman finger tip. Journal of Biomechanical Engineering, 119(1):109, 1997.ISSN 0148-0731. doi:10.1115/1.2796052. → pages 82, 84Shangchen Han, Beibei Liu, Robert Wang, Yuting Ye, Christopher D Twigg, andKenrick Kin. Online optical marker-based hand tracking with deep labels. ACMTransactions on Graphics (TOG), 37(4):166, 2018a. → page 13Shangchen Han, Beibei Liu, Robert Wang, Yuting Ye, Christopher D. Twigg, andKenrick Kin. Online optical marker-based hand tracking with deep labels. ACMTransactions on Graphics, 37(4):1–10, Jul 2018b. ISSN 0730-0301.doi:10.1145/3197517.3201399. → page 105Gustav Hauck. Die ruptur der dorsalaponeurose am ersten interphalangealgelenk,zugleich ein beitrag zur anatomie und physiologie der dorsalaponeurose. Arch.klin. chir, 123:197–232, 1923. → pages 10, 54Archibald Vivian Hill. The heat of shortening and the dynamic constants of muscle.Proceedings of the Royal Society of London. Series B - Biological Sciences, 126(843):136–195, Oct 1938. ISSN 2053-9193. doi:10.1098/rspb.1938.0050. →page 37Anne Hollister, William L. Buford, Loyd M. Myers, David J. Giurintano, andAndrew Novick. The axes of rotation of the thumb carpometacarpal joint.Journal of Orthopaedic Research, 10(3):454–460, 1992.doi:10.1002/jor.1100100319. → pages 14, 91, 92111E. Horii, G.T. Lin, W.P. Cooney, R.L. Linscheid, and K.N. An. Comparative flexortendon excursion after passive mobilization: An in vitro study. The Journal ofHand Surgery, 17(3):559–566, May 1992. ISSN 0363-5023.doi:10.1016/0363-5023(92)90371-u. → page 93Haoda Huang, Ling Zhao, KangKang Yin, Yue Qi, Yizhou Yu, and Xin Tong.Controllable hand deformation from sparse examples with rich details. In ACMSIGGRAPH/Eurographics symp. comput. anim., pages 73–82, 2011. → page 13P. Indyk and R. Motwani. Approximate nearest neighbor: Towards removing thecurse of dimensionality. In Proc. STOC, pages 604–613, 1998. → page 42S. Jacobsen, E. Iversen, D. Knutti, R. Johnson, and K. Biggers. Design of theutah/m.i.t. dextrous hand. In Proceedings. 1986 IEEE International Conferenceon Robotics and Automation, 1986. doi:10.1109/robot.1986.1087395. → pages2, 15Mark D. Jacobson, Rajnik Raab, Babak M. Fazeli, Reid A. Abrams, Michael J.Botte, and Richard L. Lieber. Architectural design of the human intrinsic handmuscles. The Journal of Hand Surgery, 17(5):804–809, Sep 1992. ISSN0363-5023. doi:10.1016/0363-5023(92)90446-v. → page 76Elliot R. Johnson, Karen Morris, and Todd D. Murphey. A Variational Approach toStrand-Based Modeling of the Human Hand, pages 151–166. Springer Tracts inAdvanced Robotics. Springer Science + Business Media, 2009.doi:10.1007/978-3-642-00312-7_10. → pages 6, 33Petr Kadlecˇek, Alexandru-Eugen Ichim, Tiantian Liu, Jaroslav Krˇivánek, andLadislav Kavan. Reconstructing personalized anatomical models forphysics-based body animation. ACM Transactions on Graphics (TOG), 35(6):213, 2016. → page 13Y. Kamata, T. Nakamura, M. Tada, S. Sueda, D. K. Pai, and Y. Toyama. How thelumbrical muscle contributes to placing the fingertip in space: a threedimensional cadaveric study to assess fingertip trajectory andmetacarpophalangeal joint balancing. Journal of Hand Surgery (EuropeanVolume), 41(4):386–391, Jul 2016. ISSN 2043-6289.doi:10.1177/1753193415597113. → pages2, 10, 15, 59, 76, 77, 78, 79, 80, 81, 82, 84, 85, 86, 87, 88, 90, 102, 104, 127Derek G Kamper, T George Hornby, and William Z Rymer. Extrinsic flexormuscles generate concurrent flexion of all three finger joints. Journal of112Biomechanics, 35(12):1581–1589, Dec 2002. ISSN 0021-9290.doi:10.1016/s0021-9290(02)00229-4. → pages 79, 82Ibranim A. Kapandji. The Physiology of the Joints, Volume 1: Upper Limb,volume 1. Churchill Livingstone, 2007. → pages 6, 45, 91, 123, 126, 127Emanuel B Kaplan. Functional significance of the insertions of the extensorcommunis digitorum in man. The Anatomical Record, 92(3):293–303, 1945. →page 2Kenton R. Kaufman, Duane A. Morrow, Gregory M. Odegard, Tammy L. HautDonahue, Patrick J. Cottler, Samuel Ward, and Richard Lieber. 3d model ofskeletal muscle to predict intramuscular pressure. In ASB Annual Conference,2010. → page 15Anna Kochan. Shadow delivers first hand. Industrial robot: an internationaljournal, 32(1):15–16, 2005. → pages 2, 16Taku Komura, Yoshihisa Shinagawa, and Tosiyasu L. Kunii. Creating andretargetting motion by the musculoskeletal human body model. The VisualComputer, 16(5):254–270, 2000. doi:10.1007/s003719900065. → page 12Paul G. Kry and Dinesh K. Pai. Interaction capture and synthesis. ACM Trans.Graph., 25(3):872–880, Jul 2006. ISSN 0730-0301.doi:http://doi.acm.org/10.1145/1141911.1141969. → pages 13, 33Vikash Kumar, Emanuel Todorov, and Sergey Levine. Optimal control with learnedlocal models: Application to dexterous manipulation. In 2016 IEEEInternational Conference on Robotics and Automation (ICRA), pages 378–383.IEEE, 2016. → page 17Pei-Hsin Kuo and Ashish D. Deshpande. Muscle-tendon units provide limitedcontributions to the passive stiffness of the index finger metacarpophalangealjoint. Journal of Biomechanics, 45(15):2531–2538, Oct 2012. ISSN 0021-9290.doi:10.1016/j.jbiomech.2012.07.034. → pages 79, 82, 83, 84, 90Tsuneya Kurihara and Natsuki Miyata. Modeling deformable human hands frommedical images. In ACM SIGGRAPH/Eurographics symp. comput. anim., pages355–363, 2004. → page 13J. M. F. Landsmeer. The anatomy of the dorsal aponeurosis of the human finger andits functional significance. Anat. Rec., 104(1):31–44, 1949.doi:10.1002/ar.1091040105. → pages 2, 10, 14, 54, 77, 126113C. E. Lang and M. H. Schieber. Human finger independence: limitations due topassive mechanical coupling versus active neuromuscular control. J.Neurophysiol., 92(5):2802–2810, November 2004. → page 34Dongwoon Lee, Michael Glueck, Azam Khan, Eugene Fiume, and Ken Jackson.Modeling and simulation of skeletal muscle for computer graphics: A survey.Now, 2012. → page 15Sang Wook Lee. Effect of finger posture on the tendon force distribution within thefinger extensor mechanism. Journal of Biomechanical Engineering, 130(5):051014, 2008. doi:10.1115/1.2978983. → page 14Sang Wook Lee, Hua Chen, Joseph D. Towles, and Derek G. Kamper. Estimationof the effective static moment arms of the tendons in the index finger extensormechanism. Journal of Biomechanics, 41(7):1567–1573, 2008.doi:10.1016/j.jbiomech.2008.02.008. → page 14Sung-Hee Lee and Demetri Terzopoulos. Heads up! ACM Transactions onGraphics (TOG), 25(3):1188, 2006. doi:10.1145/1141911.1142013. → page 12Sung-Hee Lee, Eftychios Sifakis, and Demetri Terzopoulos. Comprehensivebiomechanical modeling and simulation of the upper body. ACM Transactions onGraphics (TOG), 28(4):1–17, 2009. doi:10.1145/1559755.1559756. → page 12Yoonsang Lee, Moon Seok Park, Taesoo Kwon, and Jehee Lee. Locomotioncontrol for many-muscle humanoids. ACM Transactions on Graphics (TOG), 33(6):1–11, Nov 2014. doi:10.1145/2661229.2661233. → page 12J.N.A.L. Leijnse and J.J. Kalker. A two-dimensional kinematic model of thelumbrical in the human finger. Journal of Biomechanics, 28(3):237–249, 1995.doi:10.1016/0021-9290(94)00070-k. → pages 2, 14, 54, 67, 79, 80, 90, 102J.N.A.L. Leijnse and C.W. Spoor. Reverse engineering finger extensor apparatusmorphology from measured coupled interphalangeal joint angle trajectories - ageneric 2d kinematic model. Journal of Biomechanics, 45(3):569–578, 2012.doi:10.1016/j.jbiomech.2011.11.002. → pages 4, 10, 14, 65J.N.A.L. Leijnse, P.M. Quesada, and C.W. Spoor. Kinematic evaluation of thefinger’s interphalangeal joints coupling mechanism-variability, flexion-extensiondifferences, triggers, locking swanneck deformities, anthropometric correlations.Journal of Biomechanics, 43(12):2381–2393, 2010.doi:10.1016/j.jbiomech.2010.04.021. → pages 14, 15, 82, 87114Duo Li, Shinjiro Sueda, Debanga R. Neog, and Dinesh K. Pai. Thin skinelastodynamics. ACM Transactions on Graphics (TOG), 32(4):1, 2013.doi:10.1145/2461912.2462008. → pages 15, 45G. Li, J. Gil, A. Kanamori, and S. L.-Y. Woo. A validated three-dimensionalcomputational model of a human knee joint. Journal of BiomechanicalEngineering, 121(6):657, 1999. doi:10.1115/1.2800871. → page 15Ying Li, Jiaxin L. Fu, and Nancy S. Pollard. Data-driven grasp synthesis usingshape matching and task-based pruning. IEEE Trans. Vis. Comput. Graphics, 13:732–747, July 2007. → page 13C. Karen Liu. Dextrous manipulation from a grasping pose. ACM Trans. Graph.,28:59:1–59:6, Jul 2009. → page 33Mark Malhotra, Eric Rombokas, Evangelos Theodorou, Emanuel Todorov, andYoky Matsuoka. Reduced dimensionality control for the act hand. In 2012 IEEEInternational Conference on Robotics and Automation, 5 2012.doi:10.1109/icra.2012.6224651. → pages 16, 17, 41, 42Ivan Matev. Transposition of the lateral slips of the aponeurosis in treatment oflong-standing "boutonnière deformity" of the fingers. British Journal of PlasticSurgery, 17:281–286, 1964. ISSN 0007-1226.doi:10.1016/s0007-1226(64)80044-8. → page 105Aleka McAdams, Yongning Zhu, Andrew Selle, Mark Empey, Rasmus Tamstorf,Joseph Teran, and Eftychios Sifakis. Efficient elasticity for character skinningwith contact and collisions. ACM Trans. Graph., 30(4):37:1–37:12, Jul 2011. →page 45Ian D. McCammon and Steve C. Jacobsen. Tactile Sensing and Control for theUtah/MIT Hand, pages 239–266. Dextrous Robot Hands. Springer Science +Business Media, 1990. doi:10.1007/978-1-4613-8974-3_11. → page 16Clare K. McCarthy, James H. House, Ann Van Heest, Jacalyn A. Kawiecki, AnnDahl, and Dan Hanson. Intrinsic balancing in reconstruction of the tetraplegichand. The Journal of Hand Surgery, 22(4):596–604, Jul 1997. ISSN 0363-5023.doi:10.1016/s0363-5023(97)80115-3. → page 76A. Miller, P. Allen, V. Santos, and F. Valero-Cuevas. From robotic hands to humanhands: a visualization and simulation engine for grasping research. IndustrialRobot, 32(1):55–63, 2005. doi:10.1108/01439910510573309. → page 8115Andrew T Miller and Peter K Allen. Graspit! a versatile simulator for roboticgrasping. Robotics & Automation Magazine, IEEE, 11(4):110–122, 2004. →page 8Richard M Murray, Zexiang Li, and S Shankar Sastry. A mathematical introductionto robotic manipulation. CRC press, 1994. → page 19Mitsuhiko Nanno, William L. Buford, Rita M. Patterson, Clark R. Andersen, andSteven F. Viegas. Three-dimensional analysis of the ligamentous attachments ofthe first carpometacarpal joint. The Journal of Hand Surgery, 31(7):1160–1170,2006. doi:10.1016/j.jhsa.2006.05.007. → page 14Randolph E Perkins and Malcolm H Hast. Common variations in muscles andtendons of the human hand. Clinical Anatomy: The Official Journal of theAmerican Association of Clinical Anatomists and the British Association ofClinical Anatomists, 6(4):226–231, 1993. → page 105Tu-Hoa Pham, Nikolaos Kyriazis, Antonis A Argyros, and Abderrahmane Kheddar.Hand-object contact force estimation from markerless visual tracking. IEEEtransactions on pattern analysis and machine intelligence, 40(12):2883–2896,2018. → page 13Kyle Piddington, David I. W. Levin, Dinesh K. Pai, and Shinjiro Sueda.Eulerian-on-lagrangian cloth. In Proceedings of the 14th ACM SIGGRAPH /Eurographics Symposium on Computer Animation - SCA ’15, August 2015.doi:10.1145/2786784.2795138. → page 15Nancy S. Pollard and Victor B. Zordan. Physically based grasping control fromexample. In Proceedings of the 2005 ACM SIGGRAPH/Eurographicssymposium on Computer animation - SCA ’05, 2005.doi:10.1145/1073368.1073413. → pages 13, 33Kai Qian, Kay Traylor, Sang Wook Lee, Benjamin Ellis, Jeffrey Weiss, and DerekKamper. Mechanical properties vary for different regions of the finger extensorapparatus. Journal of Biomechanics, 47(12):3094–3099, Sep 2014. ISSN0021-9290. doi:10.1016/j.jbiomech.2014.06.035. → pages 67, 77, 85Jin Qin, David Lee, Zhizhong Li, Hua Chen, and Jack T. Dennerlein. Estimating invivo passive forces of the index finger muscles: Exploring model parameters.Journal of Biomechanics, 43(7):1358–1363, May 2010. ISSN 0021-9290.doi:10.1016/j.jbiomech.2010.01.014. → page 79116Apoorva Rajagopal, Christopher L Dembia, Matthew S DeMers, Denny D Delp,Jennifer L Hicks, and Scott L Delp. Full-body musculoskeletal model formuscle-driven simulation of human gait. IEEE Transactions on BiomedicalEngineering, 63(10):2068–2079, 2016. → page 13G. M. Rayan, D. Murray, K. W. Chung, and M. Rohrer. The extensor retinacularsystem at the metacarpophalangeal joint. Journal of Hand Surgery, 22(5):585–590, Oct 1997. ISSN 0266-7681. doi:10.1016/s0266-7681(97)80351-8. →page 76Stephane Redon, Abderrahmane Kheddar, and Sabine Coquillart. Gauss’ leastconstraints principle and rigid body simulations. In Robotics and Automation,2002. Proceedings. ICRA’02. IEEE International Conference on, volume 1,pages 517–522. IEEE, 2002. doi:10.1109/robot.2002.1013411. → page 32Zhou Ren, Jingjing Meng, Junsong Yuan, and Zhengyou Zhang. Robust handgesture recognition with kinect sensor. In Proceedings of the 19th ACMinternational conference on Multimedia - MM ’11, 2011.doi:10.1145/2072298.2072443. → page 13Arthur C Rettig. Athletic injuries of the wrist and hand: part ii: overuse injuries ofthe wrist and traumatic injuries to the hand. The American journal of sportsmedicine, 32(1):262–273, 2004. → page 10DA Robinson, DM O’meara, AB Scott, and CC Collins. Mechanical componentsof human eye movements. Journal of Applied Physiology, 26(5):548–553, 1969.→ pages 8, 39Eric Rombokas, Mark Malhotra, Evangelos Theodorou, Yoky Matsuoka, and EmoTodorov. Tendon-driven variable impedance control using reinforcementlearning. In Robotics: Science and Systems VIII, 7 2012.doi:10.15607/rss.2012.viii.047. → page 16Frank Röthling. Real robot hand grasping using simulation-based optimisation ofportable strategies. PhD thesis, Faculty of Technology, Bielefeld University,2007. → page 16Prashant Sachdeva, Shinjiro Sueda, Susanne Bradley, Mikhail Fain, and Dinesh KPai. Biomechanical simulation and control of hands and tendinous systems.ACM Transactions on Graphics (TOG), 34(4):42, 2015. → pages vi, 16, 24, 100Anis Sahbani, Sahar El-Khoury, and Philippe Bidaud. An overview of 3d objectgrasp synthesis algorithms. Robotics and Autonomous Systems, 60(3):326–336,2012. → page 7117J.L Sancho-Bru, A Pérez-González, M Vergara-Monedero, and D Giurintano. A3-d dynamic model of human finger for studying free movements. Journal ofBiomechanics, 34(11):1491–1500, 2001. doi:10.1016/s0021-9290(01)00106-3.→ page 14Joaquín L Sancho-Bru, Antonio Morales, Antonio Pérez-González, Beatriz E León,José L Iserte, Margarita Vergara, Marta C Mora, and Pablo JRodríguez-Cervantes. Towards a Realistic and Self-Contained BiomechanicalModel of the Hand, pages 211–240. Theoretical Biomechanics. InTech, 2011.doi:10.5772/19977. → page 1Hanno Scharfe, Norman Hendrich, and Jianwei Zhang. Hybrid physics simulationof multi-fingered hands for dexterous in-hand manipulation. In 2012 IEEEInternational Conference on Robotics and Automation, pages 3777–3783. IEEE,2012. → page 16Ahmed A. Shabana. Dynamics of Multibody Systems. Cambridge University Press,2005. ISBN 9780511610523. doi:10.1017/cbo9780511610523. → page 31Reza Shadmehr. Equilibrium point hypothesis. In The handbook of brain theoryand neural networks, pages 370–372. MIT Press, 1998. → pages 42, 59Weiguang Si, Sung-Hee Lee, Eftychios Sifakis, and Demetri Terzopoulos. Realisticbiomechanical simulation and control of human swimming. ACM Transactionson Graphics (TOG), 34(1):1–15, 2014. doi:10.1145/2626346. → page 12Eftychios Sifakis, Igor Neverov, and Ronald Fedkiw. Automatic determination offacial muscle activations from sparse motion capture marker data. ACMTransactions on Graphics (TOG), 24(3):417, 2005.doi:10.1145/1073204.1073208. → pages 12, 15Eftychios Sifakis, Andrew Selle, Avram Robinson-Mosher, and Ronald Fedkiw.Simulating speech with a physics-based facial muscle model. In Proceedings ofthe 2006 ACM SIGGRAPH/Eurographics Symposium on Computer Animation,SCA ’06, pages 261–270, Aire-la-Ville, Switzerland, Switzerland, 2006.Eurographics Association. ISBN 3-905673-34-7. → page 12Edwin M. Smith, Robert C. Juvinall, Leonard F. Bender, and J. Raymond Pearson.Role of the finger flexors in rheumatoid deformities of the metacarpophalangealjoints. Arthritis & Rheumatism, 7(5):467–480, 1964.doi:10.1002/art.1780070503. → page 14118Thomas H. Speeter. Control of the utah/mit dextrous hand: Hardware and softwarehierarchy. J. Robotic Syst., 7(5):759–790, 1990. doi:10.1002/rob.4620070507.→ page 17C.W. Spoor. Balancing a force on the fingertip of a two-dimensional finger modelwithout intrinsic muscles. Journal of Biomechanics, 16(7):497–504, 1983.doi:10.1016/0021-9290(83)90064-7. → page 14C.W. Spoor and J.M.F. Landsmeer. Analysis of the zigzag movement of the humanfinger under influence of the extensor digitorum tendon and the deep flexortendon. Journal of Biomechanics, 9(9):561–566, 1976.doi:10.1016/0021-9290(76)90096-8. → pages 2, 14F.-C. Su, Y.L. Chou, C.S. Yang, G.T. Lin, and K.N. An. Movement of finger jointsinduced by synergistic wrist motion. Clinical Biomechanics, 20(5):491–497, Jun2005. ISSN 0268-0033. doi:10.1016/j.clinbiomech.2005.01.002. → pages96, 97, 98, 103Shinjiro Sueda. Strand-based musculotendon simulation of the hand. PhD thesis,University of British Columbia, 2010. → pages 7, 10, 11, 20, 78, 102, 104Shinjiro Sueda, Andrew Kaufman, and Dinesh K. Pai. Musculotendon simulationfor hand animation. ACM Trans. Graph., 27(3):83:1–83:8, Aug 2008. ISSN0730-0301. doi:10.1145/1360612.1360682. → pages6, 7, 10, 17, 40, 41, 43, 45, 102Shinjiro Sueda, Garrett L. Jones, David I. W. Levin, and Dinesh K. Pai. Large-scaledynamic simulation of highly constrained strands. ACM Transactions onGraphics (TOG), 30(4):1, 2011. doi:10.1145/2010324.1964934. → pages7, 15, 18, 20, 51, 56, 66, 67, 82, 99Sydney Sunderland. The actions of the extensor digitorum communis, interosseousand lumbrical muscles. American Journal of Anatomy, 77(2):189–217, 1945.doi:10.1002/aja.1000770203. → pages 2, 4, 14A. Synek and D.H. Pahr. The effect of the extensor mechanism on maximumisometric fingertip forces: A numerical study on the index finger. Journal ofBiomechanics, 49(14):3423–3429, Oct 2016. ISSN 0021-9290.doi:10.1016/j.jbiomech.2016.09.004. → pages 6, 10, 53, 65, 70, 77, 78, 85, 102Amir H. Taghinia and Joseph Upton. Index finger pollicization. The Journal ofHand Surgery, 36(2):333–339, Feb 2011. ISSN 0363-5023.doi:10.1016/j.jhsa.2010.11.022. → page 91119Emanuel Todorov, Tom Erez, and Yuval Tassa. Mujoco: A physics engine formodel-based control. In 2012 IEEE/RSJ International Conference on IntelligentRobots and Systems, pages 5026–5033. IEEE, 2012. → page 16Winnie Tsang, Karan Singh, and Eugene Fiume. Helping hand. In Proceedings ofthe 2005 ACM SIGGRAPH/Eurographics symposium on Computer animation -SCA ’05, 2005. doi:10.1145/1073368.1073414. → pages 6, 33F.J. Valero-Cuevas, Jae-Woong Yi, D. Brown, R.V. McNamara, C. Paul, andH. Lipson. The tendon network of the fingers performs anatomical computationat a macroscopic scale. IEEE Transactions on Biomedical Engineering, 54(6):1161–1166, 2007. doi:10.1109/tbme.2006.889200. → pages3, 4, 6, 10, 15, 34, 65, 77, 79, 85Francisco J. Valero-Cuevas. An integrative approach to the biomechanical functionand neuromuscular control of the fingers. Journal of Biomechanics, 38(4):673–684, 2005. doi:10.1016/j.jbiomech.2004.04.006. → pages 7, 14Francisco J Valero-Cuevas, Felix E Zajac, and Charles G Burgar. Largeindex-fingertip forces are produced by subject-independent patterns of muscleexcitation. Journal of Biomechanics, 31(8):693–703, Aug 1998. ISSN0021-9290. doi:10.1016/s0021-9290(98)00082-7. → pages 2, 6, 77, 78, 85Francisco J. Valero-Cuevas, M.Elise Johanson, and Joseph D. Towles. Towards arealistic biomechanical model of the thumb: the choice of kinematic descriptionmay be more critical than the solution method or the variability/uncertainty ofmusculoskeletal parameters. Journal of Biomechanics, 36(7):1019–1030, 2003.doi:10.1016/s0021-9290(03)00061-7. → page 14Chiraz Walha, Hala Bezine, Nacer K M’Sirdi, Aziz Naamane, and Adel M Alimi.Handgrasp: a new simulator for human grasping. hand, 245:0–001, 2011. →page 8Jack M. Wang, Samuel R. Hamner, Scott L. Delp, and Vladlen Koltun. Optimizinglocomotion controllers using biologically-based actuators and objectives. ACMTransactions on Graphics (TOG), 31(4):1–11, 2012.doi:10.1145/2185520.2185521. → pages 12, 16M. Vande Weghe, M. Rogers, M. Weissert, and Y. Matsuoka. The act hand: designof the skeletal structure. In IEEE International Conference on Robotics andAutomation, 2004. Proceedings. ICRA ’04. 2004, 2004.doi:10.1109/robot.2004.1308775. → pages 2, 16120Frank Weichert, Daniel Bachmann, Bartholomäus Rudak, and Denis Fisseler.Analysis of the accuracy and robustness of the leap motion controller. Sensors,13(5):6380–6393, 2013. doi:10.3390/s130506380. → page 13D.D. Wilkinson, M.V. Weghe, and Y. Matsuoka. An extensor mechanism for ananatomical robotic hand. In 2003 IEEE International Conference on Roboticsand Automation (Cat. No.03CH37422), 2003. doi:10.1109/robot.2003.1241602.→ page 16Jacques Benigne Winslow. Exposition anatomique de la structure du corps humain,volume 4. chez Guillaume Desprez... et Jean Desessartz, 1732. → page 65John Z. Wu, Kai-Nan An, Robert G. Cutlip, Michael E. Andrew, and Ren G. Dong.Modeling of the muscle/tendon excursions and moment arms in the thumb usingthe commercial software anybody. Journal of Biomechanics, 42(3):383–388,2009a. doi:10.1016/j.jbiomech.2008.11.008. → page 14John Z Wu, Kai-Nan An, Robert G Cutlip, and Ren G Dong. A practicalbiomechanical model of the index finger simulating the kinematics of themuscle/tendon excursions. Bio-medical materials and engineering, 20(2):89–97,2009b. doi:10.3233/BME-2010-0618. → page 14Zhe Xu, Vikash Kumar, and Emanuel Todorov. The uw hand: A low-cost, 20-doftendon-driven hand with fast and compliant actuation. The International Journalof Robotics Research, 2013. → page 16Yuting Ye and C. Karen Liu. Synthesis of detailed hand manipulations usingcontact sampling. ACM Transactions on Graphics (TOG), 31(4):1–10, 2012.doi:10.1145/2185520.2185537. → page 13Christopher M Young and Ghazi M Rayan. The sagittal band: anatomic andbiomechanical study. The Journal of hand surgery, 25(6):1107–1113, 2000. →page 10FE Zajac. Muscle and tendon: properties, models, scaling, and application tobiomechanics and motor control. Critical reviews in biomedical engineering, 17(4):359–411, 1989. ISSN 0278-940X. → pages 37, 38, 44E. Zancolli. Structural and dynamic bases of hand surgery. Lippincott Williams &Wilkins, 1979. → pages 46, 77, 78, 102Wenping Zhao, Jianjie Zhang, Jianyuan Min, and Jinxiang Chai. Robust realtimephysics-based motion control for human grasping. ACM Trans. Graph., 32(6):121207:1–207:12, November 2013. ISSN 0730-0301.doi:10.1145/2508363.2508412. → pages 7, 13122AppendixA AnatomyWe provide a short introduction of the anatomy of the human hand. This is byno measure a complete introduction and has been based summarized mostly fromobservations of [Kapandji, 2007].A.1 Human Hand AnatomyFigure A.1: Bones of the handThe human hand can be considered asa linkage system of bones spanned byligaments, tendons and muscles. Thebones are held together at the joints,flanked by ligaments for stability andfilled with synovial fluid. The physio-logical properties of joints and musculo-tendons leads to passive visco-elasticitywhich damps the motion of the handand stabilizes the joints [Deshpandeet al., 2012]. The muscles (except thelumbrical) originate at bones: the ex-trinsic muscles begin at the forearm, while the intrinsic muscles originate at bonesof the hand. At the insertions, they connect to the bones by long tendons that canslide over these bones producing variation in moment arms. The muscles controllingthe biomechanical structure are multi-articulate, i.e., they control multiple joints atthe same time. These properties make muscular control particularly difficult.123The various parts of the hand can largely be divided into digits, palm and wrist.The digits are made up of long thin bones called phalanges (plural for phalanx).Each digit is attached to the wrist via a metacarpal bone. These metacarpal bonestogether form the palm of the hand. The wrist, or the carpus, lies at the base of thehand and is made up 8 different bones held together by various ligaments. The wristforms the basis for movement of the hand with respect to the forearm.A.2 Index Finger AnatomyWe discuss the anatomy of the index finger as a representative for modelling thefingers of the hand. While there are some differences between fingers, the extensorapparatus and most other elements remain similar. We discuss the differences andvariations in Section A.3.Bones and JointsEach finger is made up of 4 bones including the corresponding wrist bone. Startingat the palm, these bones are the metacarpal bone, proximal phalanx, intermediateor middle phalanx, and the distal phalanx, respectively. The joints between thesebones, named accordingly, in order are the metacarpophalageal (MCP), proximalinterphalangeal (PIP), and distal interphalangeal (DIP) joints. The interphalangealjoints (PIP and DIP) exhibit one degree of freedom (flexion-extension) with noactive (and little passive) abduction-adduction or pronation-supination motions. TheMCP joint can be modelled with two degrees of freedom for flexion-extension andabduction-adduction. Note that the joint between the wrist and the metacarpalbones is known as the carpometacarpal joint (CMC joint) and is assumed to havezero degrees of freedom. If we were to model the wrist, the CMC joints for the fourfingers would move together to form common axes of motion about the wrist.Muscles and TendonsMotion of the joints is controlled by 7 different muscles: four extrinsic musclesof the forearm and 3 intrinsic muscles contained within the hand. The extrinsicmuscles are composed of 2 extensors and 2 flexors, while the intrinsic musclesinclude the dorsal and palmar interossei, abbreviated to dorsal interosseous (DI) and124palmar interosseous (PI), along with the lumbrical. Note that while the DI, PI andLUM insert into the extensor hood apparatus, they are not characterized as extensorsor flexors.Figure A.2: An artist depiction of the flexor pulleys. ’A’ stands for ’AnnularLigament’, ’C’ for ’Cruciate Ligament’ [Doyle, 1988]Flexors. The primary flexors of the fingers are flexor digitorum profundus (FDP)and flexor digitorum superficialis (FDS). These 2 tendons travel through a seriesof anatomical pulleys (Fig. A.2) that shape the tendon path. These pulleys ensureconsistent dynamic coupling of bones and tendons, which is important to the use ofthe strands framework for modelling the tendons.* The FDP inserts into the distalphalanx, while the FDS inserts into the middle phalanx.Dorsal Aponeurosis. The dorsal aponeurosis is a branched sheet that sits on thedorsal side of the finger and forms the extensor apparatus. The dorsal aponeurosis isformed by the tendons of the extensor communis, interossei and lumbrical muscles,providing a mechanism for the coordination of the movements of the second and*Without dynamic coupling, we would need to implement full collision detection for strands flyingoff the surface.125Figure A.3: Extensor hood model [Garcia-Elias et al., 1991a]. Here the letterscorrespond to junctions of the tendon network where 2 or more tendonsmeet.third phalanges [Duchenne, 1885; Landsmeer, 1949]. The extensor apparatus, asshown in Fig. A.3, is a heavily branched structure, and acts as the insertion forthe intrinsic tendons DI (at H), LUM (at G), and PI (mirroring DI). The primaryextensors are the extensor indicis (EI) and the extensor digitorum communis (EDC)and form the main tendons of the extensor apparatus feeding at A. Transverse oroblique structures (segment GC is known as interosseous medial (IM), and segmentAF is called extensor lateral (EL)) connect the tendons together, and also to thepalmar plate near the MCP joint. In addition, the transverse and oblique retinacularligaments connect the structure to the palmar plate at the PIP joint and the pulleys ofthe flexor tendons. For more information on extensor apparatus components, pleaserefer to the Glossary.Oblique Retinacular Ligament . The OL plays a significant role in the extensormechanism. It ensures coupled movement of the DIP joints with flexion and/orextension of the PIP joint. The PIP joint stretches the fibers of the OL and causesa 40° automatic extension of the DIP joint [Kapandji, 2007, Pg 244]. Conversely,the automatic flexion of the DIP joint is also observed on flexion of the PIP. Abnor-malities of the retinacular ligament are a common cause of pathology in the humanhand.126Interossei and Lumbricals. The interossei insert into the extensor hood and func-tion as adductor and abductor muscles of the fingers. The lumbrical muscle isrelatively weak as compared to the interossei with the physiological cross-sectionalarea, or PCSA, approximately (1/10)th that of the interossei. It is claimed to act asa flexor at the MCP joint and as an extensor for the IP joints. The role of lumbricalmuscles as MCP stabilizers has also been investigated [Kamata et al., 2016].A.3 Digit Anatomy DifferencesWhile the fingers of the hand are anatomically similar to each other, much more sothan the thumb, there are still certain differences that are notable:• The first and second dorsal interossei insert radially into the index and middlefingers, while the third and fourth DI insert on the ulnar side of the middleand ring fingers.• The abductor digiti minimi inserts a the ulnar side of the little finger, equiva-lent to the DI mentioned above.• Similarly, for adduction, the palmar interossei insert into the thumb, indexfinger, ring finger and little finger, respectively on the medial side distal tothe MCP joint.• Aiding the individual extension of the index finger and little finger are themusculotendons extensor indicis and extensor ditigi minimi, respectively.The anatomy of the thumb is rather unlike the fingers, and is driven by ninedifferent motor muscles. Some of these are covered in Chapter 8, and for furtherdetail, the reader is referred to [Kapandji, 2007].127


Citation Scheme:


Citations by CSL (citeproc-js)

Usage Statistics



Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            async >
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:


Related Items