Open Collections

UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Byte your tongue : a computational model of human mandibular-lingual biomechanics for biomedical applications Stavness, Ian Kent 2010

You don't seem to have a PDF reader installed, try download the pdf

Item Metadata

Download

Media
ubc_2011_spring_stavness_ian.pdf [ 26.33MB ]
[if-you-see-this-DO-NOT-CLICK]
Metadata
JSON: 1.0071518.json
JSON-LD: 1.0071518+ld.json
RDF/XML (Pretty): 1.0071518.xml
RDF/JSON: 1.0071518+rdf.json
Turtle: 1.0071518+rdf-turtle.txt
N-Triples: 1.0071518+rdf-ntriples.txt
Original Record: 1.0071518 +original-record.json
Full Text
1.0071518.txt
Citation
1.0071518.ris

Full Text

Byte Your TongueA Computational Model of Human Mandibular-LingualBiomechanics for Biomedical ApplicationsbyIan Kent StavnessB.Eng., University of Saskatchewan, 2004B.Sc., University of Saskatchewan, 2004M.A.Sc., University of British Columbia, 2006A DISSERTATION SUBMITTED IN PARTIAL FULFILLMENT OFTHE REQUIREMENTS FOR THE DEGREE OFDoctor of PhilosophyinTHE FACULTY OF GRADUATE STUDIES(Electrical and Computer Engineering)THE UNIVERSITY OF BRITISH COLUMBIA(Vancouver)December 2010c Ian Kent Stavness, 2010AbstractBiomechanical models provide a means to analyze movement and forces inhighly complex anatomical systems. Models can be used to explain causeand e ect in normal body function as well as in abnormal cases where under-lying causes of dysfunction can be clari ed. In addition, computer modelscan be used to simulate surgical changes to bone and muscle structure al-lowing for prediction of functional and aesthetic outcomes.This dissertation proposes a state-of-the-art model of coupled jaw-tongue-hyoid biomechanics for simulating combined jaw and tongue motor tasks,such as chewing, swallowing, and speaking. Simulation results demonstratethat mechanical coupling of tongue muscles acting on the jaw and jaw mus-cles acting on the tongue are signi cant and should be considered in orofacialmodeling studies. Towards validation of the model, simulated tongue veloc-ity and tongue-palate pressure are consistent with published measurements.Inverse simulation methods are also discussed along with the implemen-tation of a technique to automatically compute muscle activations for track-ing a target kinematic trajectory for coupled skeletal and soft-tissue models.Additional target parameters, such as dynamic constraint forces and sti -ness, are included in the inverse formulation to control muscle activationpredictions in redundant models. Simulation results for moving and de-forming muscular-hydrostat models are consistent with published theoreti-cal proposals. Also, muscle activations predicted for lateral jaw movementare consistent with published literature on jaw physiology.As an illustrative case study, models of segmental jaw surgery with andwithout reconstruction are developed. The models are used to simulate clin-ically observed functional de cits in movement and bite force production.The inverse simulation tools are used to predict muscle forces that couldtheoretically be used by a patient to compensate for functional de cits fol-lowing jaw surgery. The modeling tools developed and demonstrated in thisdissertation provide a foundation for future studies of orofacial function andbiomedical applications in oral and maxillofacial surgery and treatment.iiPrefaceParts of this dissertation have been published elsewhere. Chapter 3 andAppendix C have been published in Stavness, Lloyd, Payan, and Fels [186]( c Wiley (2010), reproduced with permission). For Chapter 3, I wrote thesource code for the model, performed the simulations and analysis, and wrotethe text. Dr. Payan assisted with analysis and writing. Dr. Fels providededitorial feedback on the manuscript. Dr. Lloyd wrote the material that Ihave reproduced in Appendix C as background on the simulation techniquesused in this dissertation.Versions of Section 4.1, Section 5.1, Section 5.3 have been publishedin Stavness, Hannam, Lloyd, and Fels [185] ( c Taylor & Francis (2010),reproduced with permission). I formulated and implemented the inversesolver in consultation with Dr. Lloyd. I developed the jaw surgery modelswith Dr. Hannam. I performed the inverse simulations and analysis, andwrote the manuscript. Dr. Hannam and Dr. Fels provided feedback on themanuscript.Versions of Section 5.1 and Section 5.2 have been published in Hannam,Stavness, Lloyd, Fels, Miller, and Curtis [72] ( c Elsevier (2010), reproducedwith permission). Dr. Hannam performed the de cit simulations and anal-ysis with my assistance. Dr. Hannam wrote the text for the manuscriptthat I have adapted and included in Section 5.2. Dr. Fels, Dr. Curtis, andDr. Miller provided editorial feedback on the manuscript.Section 3.4 describes ongoing collaborative work. I am developing theface model jointly with Dr. Payan and the soft-palate model jointly withDr. Hui Chen. The integration of hyolaryngeal data and model has beenpublished online in Stavness, Ludlow, Chung, and Fels [184]. Data wascollected by Dr. Ludlow and Dr. Chung. I developed the model, formulatedthe data-model integration plan, and wrote the text for the manuscript underthe guidance of Dr. Fels and Dr. Ludlow.iiiTable of ContentsAbstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iiPreface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iiiTable of Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . ivList of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viiiList of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ixGlossary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiAcknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . xii1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . 81.3 Dissertation Outline . . . . . . . . . . . . . . . . . . . . . . . 102 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112.1 Functional Data Recording . . . . . . . . . . . . . . . . . . . 132.1.1 Movement . . . . . . . . . . . . . . . . . . . . . . . . 132.1.2 Forces . . . . . . . . . . . . . . . . . . . . . . . . . . . 182.1.3 Muscle activity . . . . . . . . . . . . . . . . . . . . . 192.2 Structural Data Measurement . . . . . . . . . . . . . . . . . 202.2.1 Measuring anatomical structure . . . . . . . . . . . . 202.2.2 Imaging anatomical structure . . . . . . . . . . . . . 212.2.3 Measuring mechanical tissue properties . . . . . . . . 242.3 Biomechanical Modeling . . . . . . . . . . . . . . . . . . . . 252.3.1 Rigid bone modeling . . . . . . . . . . . . . . . . . . 252.3.2 Deformable tissue modeling . . . . . . . . . . . . . . 272.3.3 Muscle modeling . . . . . . . . . . . . . . . . . . . . . 292.3.4 Isolated orofacial models . . . . . . . . . . . . . . . . 33ivTable of Contents2.3.5 Coupled orofacial models . . . . . . . . . . . . . . . . 342.4 Inverse Methods . . . . . . . . . . . . . . . . . . . . . . . . . 352.5 Biomedical Applications . . . . . . . . . . . . . . . . . . . . . 392.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 413 Jaw-Tongue-Hyoid Model . . . . . . . . . . . . . . . . . . . . 433.1 Model Creation . . . . . . . . . . . . . . . . . . . . . . . . . 443.1.1 Jaw-hyoid model . . . . . . . . . . . . . . . . . . . . . 453.1.2 Tongue model . . . . . . . . . . . . . . . . . . . . . . 463.1.3 Registration . . . . . . . . . . . . . . . . . . . . . . . 483.1.4 Attachment . . . . . . . . . . . . . . . . . . . . . . . 483.2 Simulation Descriptions . . . . . . . . . . . . . . . . . . . . . 503.2.1 Jaw activated tasks . . . . . . . . . . . . . . . . . . . 513.2.2 Tongue activated tasks . . . . . . . . . . . . . . . . . 533.3 Simulation Results . . . . . . . . . . . . . . . . . . . . . . . . 533.3.1 Jaw-tongue-hyoid coupling . . . . . . . . . . . . . . . 533.3.2 Comparison with published data . . . . . . . . . . . . 573.3.3 Integration Error . . . . . . . . . . . . . . . . . . . . 603.4 Additional Orofacial Sub-Models . . . . . . . . . . . . . . . . 613.4.1 Face model . . . . . . . . . . . . . . . . . . . . . . . . 613.4.2 Soft-palate model . . . . . . . . . . . . . . . . . . . . 633.4.3 Hyoid-larynx model . . . . . . . . . . . . . . . . . . . 633.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 643.6 Directions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 673.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 674 Inverse Simulation Methods . . . . . . . . . . . . . . . . . . . 694.1 Inverse Solver Formulation . . . . . . . . . . . . . . . . . . . 714.2 Analysis with Canonical Models . . . . . . . . . . . . . . . . 774.2.1 Point inverse . . . . . . . . . . . . . . . . . . . . . . . 774.2.2 Rigid-body inverse . . . . . . . . . . . . . . . . . . . . 814.2.3 Deformable-body inverse . . . . . . . . . . . . . . . . 814.3 Analysis with Anatomical Models . . . . . . . . . . . . . . . 844.3.1 Jaw inverse . . . . . . . . . . . . . . . . . . . . . . . . 844.3.2 Tongue inverse . . . . . . . . . . . . . . . . . . . . . . 884.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 904.5 Directions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 924.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93vTable of Contents5 Segmental Jaw Surgery Models . . . . . . . . . . . . . . . . . 955.1 Model Creation . . . . . . . . . . . . . . . . . . . . . . . . . 965.2 Forward Dynamics: De cit Simulations . . . . . . . . . . . . 995.2.1 Simulation descriptions . . . . . . . . . . . . . . . . . 1015.2.2 Simulation results . . . . . . . . . . . . . . . . . . . . 1015.2.3 Discussion . . . . . . . . . . . . . . . . . . . . . . . . 1045.3 Inverse Dynamics: Compensatory Simulations . . . . . . . . 1075.3.1 Simulation descriptions . . . . . . . . . . . . . . . . . 1075.3.2 Simulation results . . . . . . . . . . . . . . . . . . . . 1085.3.3 Discussion . . . . . . . . . . . . . . . . . . . . . . . . 1125.4 Directions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1165.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1176 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1186.1 Dissertation Contributions . . . . . . . . . . . . . . . . . . . 1186.2 Directions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1216.3 Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . 124Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125AppendicesA List of Publications . . . . . . . . . . . . . . . . . . . . . . . . . 148A.1 Journal Publications . . . . . . . . . . . . . . . . . . . . . . . 148A.2 Conference Publications . . . . . . . . . . . . . . . . . . . . . 149A.3 Research Visits . . . . . . . . . . . . . . . . . . . . . . . . . . 149A.4 Research Talks . . . . . . . . . . . . . . . . . . . . . . . . . . 150A.5 Master’s Publications . . . . . . . . . . . . . . . . . . . . . . 150A.6 Additional Publications . . . . . . . . . . . . . . . . . . . . . 151B Head and Neck Anatomy . . . . . . . . . . . . . . . . . . . . . 152B.1 Bone Structures . . . . . . . . . . . . . . . . . . . . . . . . . 152B.1.1 Cranium . . . . . . . . . . . . . . . . . . . . . . . . . 152B.1.2 Mandible . . . . . . . . . . . . . . . . . . . . . . . . . 154B.1.3 Dentition . . . . . . . . . . . . . . . . . . . . . . . . . 155B.1.4 Hyoid bone . . . . . . . . . . . . . . . . . . . . . . . . 156B.1.5 Vertebrae . . . . . . . . . . . . . . . . . . . . . . . . . 156B.2 Soft-Tissues and Muscles . . . . . . . . . . . . . . . . . . . . 156B.2.1 Face . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157viTable of ContentsB.2.2 Jaw muscles . . . . . . . . . . . . . . . . . . . . . . . 158B.2.3 Tongue . . . . . . . . . . . . . . . . . . . . . . . . . . 162B.2.4 Soft-palate . . . . . . . . . . . . . . . . . . . . . . . . 164B.2.5 Pharynx . . . . . . . . . . . . . . . . . . . . . . . . . 165B.2.6 Larynx . . . . . . . . . . . . . . . . . . . . . . . . . . 166B.2.7 Epiglottis . . . . . . . . . . . . . . . . . . . . . . . . . 169B.2.8 Temporomandibular joint . . . . . . . . . . . . . . . . 169C ArtiSynth Simulation Software . . . . . . . . . . . . . . . . . 172C.1 Physical Simulation Framework . . . . . . . . . . . . . . . . 173C.1.1 Friction, damping, and stabilization . . . . . . . . . . 175C.1.2 System solution and complexity . . . . . . . . . . . . 177C.1.3 Attachments between bodies . . . . . . . . . . . . . . 179C.1.4 Contact handling . . . . . . . . . . . . . . . . . . . . 181C.1.5 Simulation engine summary . . . . . . . . . . . . . . 184C.1.6 Validation using ANSYS . . . . . . . . . . . . . . . . 185C.2 Graphical Toolset . . . . . . . . . . . . . . . . . . . . . . . . 187C.2.1 Model component hierarchy . . . . . . . . . . . . . . 188C.2.2 Viewing, selection, and editing . . . . . . . . . . . . . 188C.2.3 Properties, control panels, and probes . . . . . . . . . 189viiList of Tables3.1 Physiological cross-sectional area of jaw and tongue muscles . 463.2 Muscle activation amplitudes for jaw-tongue-hyoid simulations 505.1 REST to OPEN to CLOSE inverse simulation results . . . . . 1125.2 Stable unilateral clenching inverse simulation results . . . . . 114viiiList of Figures1.1 Upper airway anatomy, CT image and illustration . . . . . . 21.2 Jaw-tongue-hyoid model in ArtiSynth . . . . . . . . . . . . . 42.1 Early photographic recordings of horse movement by Muybridge 132.2 Computed tomography images of the head . . . . . . . . . . . 222.3 Magnetic resonance images of the head . . . . . . . . . . . . . 232.4 Force characteristics of the Hill-type muscle model . . . . . . 323.1 Jaw-hyoid model with labeled muscles . . . . . . . . . . . . . 453.2 Tongue model with labeled muscles . . . . . . . . . . . . . . . 473.3 Jaw-tongue-hyoid model . . . . . . . . . . . . . . . . . . . . . 493.4 Muscle activation patterns for jaw-tongue-hyoid simulations . 513.5 Jaw opening simulation results . . . . . . . . . . . . . . . . . 543.6 Jaw activated tasks simulation results . . . . . . . . . . . . . 553.7 Tongue retraction simulation results . . . . . . . . . . . . . . 563.8 Tongue-palate contact simulation results . . . . . . . . . . . . 573.9 Tongue velocity during simulated and recorded retraction . . 583.10 Assessment of integration error . . . . . . . . . . . . . . . . . 603.11 Face model integrated with jaw-tongue-hyoid model . . . . . 613.12 Soft-palate model registered to jaw-tongue-hyoid model . . . 623.13 Hyoid-larynx model and video  uoroscopy landmarks . . . . . 644.1 Point-model inverse simulation: kinematic target . . . . . . . 784.2 Point-model inverse simulation: output muscle activations . . 784.3 Point-model inverse simulation: output muscle activations . . 794.4 Rigid-body model inverse simulation . . . . . . . . . . . . . . 804.5 Deformable beam inverse simulation: protrusion . . . . . . . 824.6 Deformable beam inverse simulation: output muscle activations 824.7 Deformable beam inverse simulation: complex movement . . . 834.8 Inverse jaw simulation: lateral incisor movement . . . . . . . 854.9 Inverse jaw laterotrusion simulation: output muscle activations 864.10 Tongue inverse simulation: tip elevation . . . . . . . . . . . . 89ixList of Figures4.11 Tongue tip elevation inverse simulation: output muscle acti-vations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 905.1 Segmental jaw resection and reconstruction models . . . . . . 975.2 Segmental jaw resection model with labeled muscles. . . . . . 985.3 Segmental jaw resection model conventions and restraints . . 1005.4 Incisor displacements during FORCE and OPEN simulations 1035.5 NOCON model during individual closer muscle activations . . 1055.6 Target jaw postures to move from REST to OPEN to CLOSE 1095.7 REST to OPEN inverse simulation results . . . . . . . . . . . 1105.8 OPEN to CLOSE inverse simulation results . . . . . . . . . . 1115.9 Target jaw postures for unilateral clenching tasks . . . . . . . 113B.1 Upper airway anatomy, sagittal cross-section view . . . . . . 153B.2 Skull, lateral view . . . . . . . . . . . . . . . . . . . . . . . . 153B.3 Mandible, medial and lateral views . . . . . . . . . . . . . . . 154B.4 Tooth and alveolar process, lateral view . . . . . . . . . . . . 155B.5 Lower and upper dentition . . . . . . . . . . . . . . . . . . . . 155B.6 Facial muscles, lateral view . . . . . . . . . . . . . . . . . . . 157B.7 Jaw closing muscles . . . . . . . . . . . . . . . . . . . . . . . . 158B.8 Jaw opening muscles . . . . . . . . . . . . . . . . . . . . . . . 159B.9 Tongue muscles, lateral view . . . . . . . . . . . . . . . . . . 161B.10 Tongue muscles, posterior view . . . . . . . . . . . . . . . . . 161B.11 Soft-palate muscles, posterior view . . . . . . . . . . . . . . . 164B.12 Larynx and pharynx . . . . . . . . . . . . . . . . . . . . . . . 165B.13 Oral cavity during swallowing . . . . . . . . . . . . . . . . . . 166B.14 Neck muscles, frontal view . . . . . . . . . . . . . . . . . . . . 167B.15 Temporomandibular joint . . . . . . . . . . . . . . . . . . . . 170B.16 Temporomandibular joint ligaments . . . . . . . . . . . . . . 170C.1 Errors due to decoupling constraints from the velocity solve . 177C.2 Factor times for A as a function of system size . . . . . . . . 179C.3 Contact handling between two deformable models . . . . . . . 182C.4 ArtiSynth and ANSYS static comparisons . . . . . . . . . . . 185C.5 ArtiSynth and ANSYS time integration comparisons . . . . . 187C.6 ArtiSynth model viewer and timeline controller . . . . . . . . 188C.7 ArtiSynth model hierarchy view and control panel . . . . . . 189xGlossary1D One Dimensional2D Two Dimensional3D Three DimensionalDOF Degrees-of-FreedomEMG ElectromyographyEPG ElectropalatographyEM ElectromagneticEMA Electromagnetic ArticulometryVF Video- uoroscopyUS UltrasoundCT Computed TomographyCBCT Cone-Beam Computed TomographyMRI Magnetic Resonance ImagingVHD Visible Human DatasetEPH Equilibrium Point HypothesisFEM Finite-Element MethodOSA Obstructive Sleep ApneaPCA Principal Components AnalysisCSA Cross-Sectional AreaTMJ Temporomandibular JointxiAcknowledgementsMy rich and positive experience in graduate studies is due in large part tomy supervisor Sid Fels. Many thanks for his guidance, mentorship, andfriendship over the past six years. I have had the pleasure of working closelywith a number of  ne researchers during this time as well. Thanks to JohnLloyd for many fruitful sessions at the white board and at the pub. Thanksto Alan Hannam for many enjoyable discussions and for providing me with aclear model of excellent academic scholarship. Also, thanks to Yohan Payanfor a productive collaboration over the past year.Thanks must go to Christy Ludlow for hosting my visit to her lab at theNational Institutes of Health as well as to Chris Peck and Alex Wirianskifor hosting my visit to their research group at the University of Sydney.Acknowledgement is due to researchers at Gipsa-Lab, Grenoble for kindlyproviding data to support this project, including Pierre Badin for providingthe CT image data as well as Pascal Perrier, Stephanie Buchaillard, andMohammad Ali Nazari for providing the tongue and face model geometry.Finally, a heart felt thank-you to my family and friends for their loveand support during this phase of my life. Special thanks to my parents forproviding me the strong foundation upon which my accomplishments aregrounded. My deepest thanks go to my dear, Sara, thank you for your dailyencouragement, much laughter, and endless love.xiiChapter 1IntroductionComputer simulation is becoming an integral aspect of biomedical researchand practice, in applications ranging from basic research in physiology toclinical applications in treatment planning and surgical training. Medicalimaging technology has revolutionized our ability to observe internal struc-tures of the body and is one of the most signi cant biomedical advances ofthe 20th century. Computer simulation aims to build upon medical imag-ing to further our understanding of how the body functions. Examples ofbiological simulation include visualizing dynamic movements from static im-ages, predicting unobservable variables such as stresses within tissue, andilluminating mechanisms of sensorimotor control. An important example ofbiomedical computer simulation is the dynamic modeling of muscle-drivenanatomical structures as a means to better understand their function innormal and pathological cases. Modeling such structures is non-trivial, andoften involves the combined simulation of hard structures (such as bones),interconnected by constraints (such as joints), attached to various soft tis-sues (such as skin, fat, mucosa, muscle and tendon), and in contact witheach other and the environment. This dissertation proposes computationalmethods to analyze human orofacial biomechanics in order to better under-stand structure-function relationships and motor control strategies in oralmovement, including mastication and speech production, towards develop-ing new tools for computer-assisted oral and maxillofacial surgery.1.1 MotivationThe research described in this dissertation is motivated by a desire to createcomputational tools for analyzing oral and facial biomechanics in the context11.1. MotivationTongueLipsMandibleHyoid boneEpiglottisTracheaThyroid cartilageCricoid cartilageVertebraeSoft palateHard palateVocal foldsTongueLipsLarynxFigure 1.1: Mid-sagittal CT image and illustration of the upper airwayanatomy. CT data courtesy of Dr. Pierre Badin, Gipsa-Lab, Grenoble.Illustration c Elsevier (2010), Drake et al. [47], adapted with permission.of biomedical applications, such as improving our understanding of upper-airway dysfunction and related medical and surgical treatments. Humanupper-airway anatomy is structurally complex and critical to a number oflife-sustaining functions, making it a good candidate for analysis throughbiomechanics simulation.Biomedical applicationsComputational biomechanics has a wide range of viable applications inmedicine. Diagnosis of motor system disorders can be improved with biome-chanics simulation to augment and enhance medical images and other ob-servational data. Further, biomechanics simulation can help to determinecause-and-e ect in order to establish a deeper understanding of motor sys-tem dysfunction. Biomechanics simulation can also aid in planning of treat-ment and surgical interventions. Computer models can be used to evaluatealternative treatment paths or tailor a particular treatment to a speci cpatient. Also, enhancing post-operative evaluation with biomechanics sim-21.1. Motivationulation can help to guide patient rehabilitation. In Chapter 5, we investi-gate segmental jaw resection and reconstruction as a case study of applyingbiomechanics analysis to post-operative analysis and prediction.Diagnosis Comprehensive dynamic models of the orofacial region will en-hance our understanding of both its normal physiological function and itsdysfunctions, such as Obstructive Sleep Apnea (OSA) [86], swallowing dis-orders [110], and speech pathologies. Simulations can predict immeasurablebiomechanical quantities that may correlate with dysfunction and there-fore could be used to enhance diagnosis protocols. For example, forces inthe Temporomandibular Joint (TMJ) are di cult to measure, but can bedetermined in a biomechanical analysis of jaw movement and used in thediagnosis of TMJ disorder.Treatment planning Simulating a variety of potential treatments, suchas di erent surgical procedures or prostheses, could inform a treatment planand complement other factors such as clinician experience and intuition.For example, in surgery planning simulation can be used to predict theconsequences and de cits associated with a particular surgical alterationto a musculoskeletal system. Given the mechanical complexity of ThreeDimensional (3D) musculoskeletal structures, the small sample size of pa-tients, and the signi cant risks involved, surgical innovation is necessarilyconservative. Computer simulation allows for iterative re nement of surgicalprocedures with little cost and risk to patients. Section 5.2 reports simula-tions comparing theoretical post-operative de cits for jaw surgery with andwithout reconstruction. This type of analysis could be used on a patient-speci c bases to determine, given the planned extent of tissue resection,whether or not a jaw reconstruction would be bene cial.Post-operative rehabilitation Biomechanics simulation of surgically re-constructed anatomy can also provide post-operative bene t by guiding re-habilitation. Given a model of a speci c patient’s reconstruction, simulationof di erent muscle activation patterns may illuminate new motor strategies31.1. MotivationFigure 1.2: Screen shot of our ArtiSynth modeling toolkit showing the jaw-tongue-hyoid model along with a timeline for controlling input/output dataand a control panel for adjusting model properties.to compensate for the altered musculoskeletal structure. Novel motor strate-gies could potentially guide post-operative therapy decisions, for example,which muscles are important to strengthen, or conversely which musclesforces need to be reduced via an intervention such as botulinum toxin in-jection. Section 5.3 reports inverse simulations predicting jaw muscle forcesrequired to compensate for simulated de cits consequent to jaw resection.Importance of orofacial function and dysfunctionThe human orofacial anatomy, pictured in Figure 1.1, is involved in a num-ber of life sustaining functions including chewing, swallowing, and breathing.Therefore, modeling orofacial biomechanics for understanding and treatingorofacial dysfunction has a large potential bene t. Dysphagia, or swallow-ing disorders, is a serious concern for stroke patients as aspiration-relatedpneumonia leads to 40,000 deaths per year in North America [110]. OSA isa serious respiratory condition involving tissue collapse in the upper airway41.1. Motivationduring sleep that a icts 12 million people in United States. In addition toeating and breathing, the orofacial anatomy is central to speech production.The analysis of speech production mechanisms has the potential to bene tthe treatment of speech pathology.Head and neck cancer or trauma is commonly treated with surgical re-section and reconstruction of orofacial tissues, including the jaw and tongue.Head and neck cancer is the sixth most common cancer worldwide and ac-counts for 500,000 new cases per year [24]. Post-surgical de cits can includedysphagia, chewing disorders, OSA [141], speech de cits, as well as signif-icant alteration of facial aesthetics. These surgical procedures are highlycomplex, highly invasive, and can severely impact patient quality of life.Traditional approaches to observing and recording orofacial function arereviewed in Section 2.1 and we will argue that the inaccessibility of manyfunctionally signi cant parameters, as well as the structural complexity ofthe orofacial anatomy, present signi cant barriers for traditional data collec-tion methods to provide su cient insight into normal and abnormal orofacialfunction. Advanced biomechanical toolsets, such as the jaw-tongue-hyoidmodel developed in Section 3.1 and pictured in Figure 1.2, are required tomake new progress in this area.Complexity of orofacial anatomyThe high complexity of orofacial anatomy, as shown in Figure 1.1, is an-other motivation for developing new computational techniques to investigateorofacial function; anatomical complexity complicates empirically derivedfunctional hypotheses and prevents complete characterization with empiricaltechniques. The orofacial anatomy is composed of rigid structures includ-ing the cranium, jaw, and hyoid bone, highly deformable muscle activatedtissues such as the tongue, soft palate, and pharynx, and larynx, an intri-cate arrangement of many muscles some of which are capable of exertingvery large forces (up to 200 N during tooth clenching), and various con-tact and constraint situations including bite contact and the TMJ. Giventhe complexity of the upper airway, it is not surprising that its function51.1. Motivationand motor control strategies are not completely understood, especially forcomplex motor actions such as speech production that involve the coordi-nation of multiple structures in very fast movements up to 20 cm/s [144].Appendix B provides background information on the orofacial anatomy andmuscles. Section 2.2 reviews techniques employed to measure and imageanatomical structures and to measure and estimate the mechanical proper-ties of biological tissues.Need for mechanicsWhile movement can be directly recorded with a variety of techniques, otherindirect biomechanical variables, such as forces, are also important and use-ful. For example, force predictions are important for understanding toothloading when designing dental prosthesis to ensure that the prosthesis is suf- ciently durable, given the expected loading during chewing and clenching.Tooth forces are also important to ensure that reshaping the occlusal sur-faces do not generate abnormal or unbalanced TMJ forces that could leadto articular disc dysfunction.Analyzing force information given kinematics and reaction force record-ings requires information about the mechanics of the system under analysis.As mentioned above, the complexity of human body mechanics is signi cant,even as compared with modern robotic systems, and in particular biome-chanical systems exhibit unique features: compliant, non-linear actuators aswell as kinematic and motor redundancy. Additionally, human movementsinvolve the dynamics of the body and therefore dynamics are important asopposed to simply forces at a quasi-static equilibrium. Section 2.3 reviewspreviously proposed mechanical models of biological systems, focusing pri-marily on previous models of the human orofacial subsystems, including thejaw, tongue, larynx, and face.Need for dynamicsDynamics are important in speech production, which involves very fastmovement of the vocal tract articulators (up to 80 cm/s [149]). The dy-61.1. Motivationnamic trajectory of the vocal tract articulations, not just static postures,are important to the acoustics of speech. Also, inertial e ects are non-negligible for fast orofacial movements, especially for the tongue. Therefore,we have focused on dynamic simulation techniques, as opposed to quasi-static techniques, both for creating forward dynamic simulations of orofacialmovements and in the development of inverse techniques to calculate muscleactivations to drive fully dynamic models. Appendix C provides backgroundmaterial on the forward dynamics formulation in ArtiSynth, which was usedto create models of the normal jaw-tongue-hyoid (Section 3.1) and abnormaljaws (Section 5.1).Need for holistic modelsThe anatomical structures of the upper airway and face are interconnected.Therefore isolating a subsystem with boundary conditions in a model maybe insu cient to fully reproduce orofacial function. This is especially truefor the jaw-tongue-hyoid system, where the position of the tongue withinthe oral cavity is largely determined by the positioning of the jaw and hy-oid, on which the tongue is said to \ride"[78]. Jaw-tongue coupling e ectsmay also play a role in co-articulation e ects in speech production [128].Chapter 5 describes the development and analysis of the  rst-of-its-kind dy-namically coupled jaw-tongue-hyoid model using dynamic Finite-ElementMethod (FEM) combined with rigid-body dynamics. Further, an inves-tigation of incorporating the face, soft-palate, and larynx are detailed inSection 3.4.Need for inverse methodsWhile the capability to create representations of musculoskeletal systemsand compute the dynamics of the resulting coupled mechanical system hasadvanced markedly, generating useful, plausible, and accurate motion sim-ulations is still a signi cant challenge. Generating motion simulations forbiomechanical models involves computing an input signal to \drive" a modelto perform a desired motor task. Tasks are typically described in terms of71.2. Contributionskinematics (motion) while the drive is thought of as time-varying excitationof motor units or muscles. Controlling dynamics simulations of multi-muscleanatomical systems is challenging due a high-dimensional redundant controlspace. Manual trial-and-error tuning of muscle activation inputs to a forwarddynamics simulation is intractable and di cult to evaluate with respect torecorded subject data. Inverse methods will increase the utility of biome-chanics modeling by systematically predicting activations to achieve targetoutputs.Further, understanding the motor control strategies underlying humanfeeding and speech production is an important and active area of research.In speech production, the traditional approach uses statistical analysis ofjaw and tongue kinematics; however, this approach cannot determine whichaspects of the observed movements arise from central motor commands andwhich are due to the mechanics of the peripheral musculoskeletal system.Chapter 4 describes the inverse-dynamics techniques developed within Ar-tiSynth for automatic muscle activations prediction for trajectory trackingwith the dynamic models.1.2 ContributionsThe contributions of this dissertation include creating state-of-the-art mod-els for orofacial biomechanics (Chapter 3), developing new tools for inverse-dynamics simulation (Chapter 4), and applying the models and tools tothe analysis of segmental jaw surgery as an example biomedical application(Chapter 5). Publications resulting from this work are listed in Appendix Aand the main contributions are summarized here.Modeling of coupled jaw-tongue-hyoid biomechanicsi. Created a novel model of the jaw-tongue-hyoid system. Wedeveloped a state-of-the-art model of coupled jaw-tongue-hyoid biome-chanics in order to analyze combined jaw and tongue motor tasks, suchas chewing, swallowing, and speaking.81.2. Contributionsii. Demonstrated the signi cance of coupling. We demonstratedthat the mechanical coupling of tongue muscles acting on the jaw, andvice versa, are signi cant and are an important factor for considerationin future orofacial modeling studies.iii. Compared simulations with recorded tongue velocity and pres-sure. We found that simulations of tongue velocity in speech and max-imum voluntary tongue-palate pressure compared well with publishedmeasurements.iv. Used as a test case for the ArtiSynth platform. We used thejaw-tongue-hyoid model as a test-case of our simulation platform, Ar-tiSynth, to ensure su cient capabilities and  delity for modeling allupper airway structures.Inverse techniques for hard/soft muscle-tissue modelsi. Formulated trajectory-tracking for muscle-activated dynamicFEM models with constraints. We implemented a technique toautomatically compute muscle activations to track a target kinematictrajectory for coupled skeletal and soft-tissue (dynamic FEM) struc-tures.ii. Formulated novel target parameters: constraint forces andsti ness. We extended the inverse formulation to include additionaltarget parameters, including dynamic constraint forces and sti ness, tocontrol muscle activation predictions in redundant models.iii. Predicted beam and tongue muscle activations consistent withmuscular-hydrostat theory. We predicted muscle activations neededto move and deform muscular-hydrostat models that are consistent withpublished theoretical proposals.iv. Predicted plausible muscle activations for lateral jaw move-ment. We predicted muscle activations for lateral jaw movement thatare consistent with published literature on jaw physiology.91.3. Dissertation OutlineApplication to the analysis of segmental jaw surgeryi. Created models of segmental jaw surgery with/without re-construction. We developed models of segmental jaw resection andreconstruction through structural alterations to a model of the normaljaw system.ii. Compared mechanical basis of functional de cits between mod-els. We simulated functional de cits in movement and bite force pro-duction that are observed clinically consequent to jaw resection.iii. Applied inverse toolset to predict muscle forces to compen-sate for de cits. We predicted muscle forces that could be used tocompensate for functional de cits in a jaw surgery patient.1.3 Dissertation OutlineThis dissertation is structured around the three main research contributions.Chapter 2 reviews previous approaches to characterizing orofacial structureand function, including measurement techniques and modeling approaches.Chapter 3 details the jaw-tongue-hyoid model, simulation results, and evalu-ation. Chapter 4 describes the inverse simulation methods developed withinthe ArtiSynth framework and simulation results for hard and soft tissuemodels. Chapter 5 details the segmental jaw surgery models used to an-alyze post-operative functional de cits and predict compensatory musclepatterns. Chapter 6 summarizes the dissertation contributions, describes di-rections for future work, and provides concluding remarks. The appendicesprovide additional background material. Appendix A lists the publicationsand research talks associated with the dissertation, Appendix B provides aoverview of orofacial anatomy, and Appendix C describes the mathematicalframework for physics simulation in ArtiSynth.10Chapter 2Related WorkThis chapter reviews the tools and techniques used to analyze human biome-chanics focusing on studies applied to orofacial structure and function. Thetraditional approach to biomechanical analysis involves recording observa-tions of human movement and applying statistics to describe relationshipswithin the observations. Such observational studies have generated a wealthof data regarding human motor physiology. However, functional recordingtechniques are limited by low spatial and temporal resolution, the di cultyof simultaneous recording in multiple modalities, the inability to directlytransduce salient variables in humans, and the complexity of the humanmusculoskeletal and motor systems. These limitations reduce the e ective-ness of observational studies for providing a deep understanding of the rela-tionships between peripheral biomechanics and central motor control. Thehuman orofacial region is particularly challenging for observational study be-cause of the 3D nature of soft-tissue deformations and because vocal tractarticulators are located within mouth, making them hard to view and ac-cess for movement and muscle recordings. Section 2.1 reviews techniquesfor functional data recording and discusses related limitations and issues.Recent advances in computation simulation of physical phenomena, suchas solid continuum mechanics, have opened new avenues to investigate hu-man biomechanics by developing mathematical representations of the anatom-ical structure. Modeling approaches rely on structural information extractedfrom medical imaging modalities, such as Computed Tomography (CT) andMagnetic Resonance Imaging (MRI) data, and on mechanical properties oftissues that have been measured in vivo in humans or animals, ex vivo on atest bench, or on cadaver specimens. While based on a range of data sourcesand founded on a number of assumptions, models can provide explanatory11Chapter 2. Related Workpower and can  ll gaps in functional recordings. Section 2.2 reviews meth-ods for analyzing and quantifying anatomical structure and tissue propertiesfor the purpose of building computational models.Early biomechanical models used greatly simpli ed representations, suchas stick- gure limb models. Increases in computational power and more e -cient numerical methods have led to models with much richer representationsof the 3D structure of bones, muscles, ligaments, joints, and soft-tissues.These advanced modeling techniques are required for complex anatomicalstructures such as the orofacial region. As models become more accurate,they can be used to analyze casual relationships between anatomical struc-tures and observed functions and ultimately to make predictions extrapo-lating from functional recordings. Section 2.3 describes previously reportedorofacial biomechanical models, including di erent modeling methods forskeletal, soft-tissue, and muscle structures.Biomechanical models require muscle forces as input and simulate re-sulting movements and reaction forces. Integrating models with functionaldata is challenging because muscle forces are hard to record experimentally.Therefore, inverse simulation methods are important for predicting muscleforces required to match simulated movements with recorded movements.A number of inverse techniques have been proposed and are reviewed inSection 2.4, though little work has been reported for models of the jaw ortongue.As a tool to analyze hypothetical biomechanical situations, models areuniquely suited for biomedical applications, such as analyzing dysfunctionalsystems and evaluating potential avenues of treatment. In particular, computer-assisted surgery involves integrating computer technologies, including anatom-ical modeling and biomechanical simulation, into surgical planning and ex-ecution. Section 2.5 reviews work on computer-assisted dentistry and oraland maxillofacial surgery.122.1. Functional Data RecordingFigure 2.1: Early photographic recordings of horse movement by Muybridge,The Horse in Motion. Subtitle reads, \Sallie Gardner," owned by LelandStanford; running at a 1:40 gait over the Palo Alto track, 19th June 1878.2.1 Functional Data RecordingA number of functional data recording techniques have been developed inorder to quantitatively observe human movement, forces, and muscle acti-vations. In this section we review how these techniques have been appliedto observe jaw, tongue, face, and larynx function.2.1.1 MovementJaw and tongue movement has been previously reviewed by Miller [117] andHiiemae and Palmer [78] respectively. In this section, we organize our reviewof orofacial movement analysis by recording technique.Optical photographic techniquesQuantitative analysis of human movement was enabled by the invention ofoptical photographic techniques. Muybridge pioneered photographic record-ings of animal and human movement [131]. His studies included horse loco-motion, as pictured in Figure 2.1, illustrating that a horse’s hooves are not132.1. Functional Data Recordingin contact with the ground during galloping. Early photographic techniqueswere modi ed and re ned speci cally for the purpose of recording move-ment as described in Chapter 1 of Bernstein [18]. Such techniques includedmultiple exposures on a single  lm of a moving subject augmented withhigh-contrast re ective markers (chronophotography) or incandescent lightbulbs (cyclography). Cyclical movements of a stationary subject, such as ling, were recorded on a continuously exposed moving  lm (kymocyclogra-phy). Mirrors were also used to capture multiple perspectives of a movementof the same exposure in order to reconstruct spatial motion. Quantitativeanalysis was performed by manual measurements made on the exposed  lm.Modern video tracking systems use image processing techniques to au-tomatically locate landmarks in each frame of video and multiple camerasto automatically determine the 3D position of landmarks in space. Auto-matic landmark detection is typically aided with active light-emitting diodemarkers (e.g. NDI Optotrak [134]) or passive re ective markers (e.g. ViconMotion Systems [207]). Such tracking systems can measure 3D marker po-sitions with sub-millimeter accuracy at sampling rates of 200 Hz or higher.Optical tracking systems have been used to measure jaw movement insix Degrees-of-Freedom (DOF) by rigidly a xing multiple markers to theupper and lower teeth [60, 206]. Our work on the integration of optical jawtracking data and jaw models is discussed in Section 4.3.1. Facial movementduring speech has been recorded with markers distributed over the surface ofthe face [102, 179]. Section 3.4.1 describes our preliminary work developinga model of face dynamics that could be used to predict muscle forces fromoptical tracking data of face movements.The main limitation of optical techniques is that line-of-sight is requiredbetween markers and cameras. This is problematic for measuring movementof the tongue and other vocal tract articulators, which are internal to theoral cavity. A study by Abd-El-Malek [1] circumvented the problem by us-ing subjects with missing teeth, which provided a window to view insidethe mouth during mastication and allowed for visual inspection and illus-tration of tongue shapes. Also, small cameras or optical lenses can be  tthrough external openings or incisions in the body to view internal body142.1. Functional Data Recordingstructures. Endoscopy uses a  exible light-transmitting tube that can beinserted through the oral or nasal cavity for a top-down view of the oro-pharyngeal cavity and larynx. Endoscopy is limited to a single top-downview, making 3D laryngeal movements di cult to interpret. However, Selbieet al. [168] used endoscopy to verify a model of 3D cricoarytenoid movement.Other approaches to measuring internal body structures include medicalimaging and electromagnetic techniques that do not require line-of-sight.Cineradiography and video uoroscopyThe earliest quantitative recordings of human tongue movement used x-ray cineradiography to analyze kinematics in feeding [13] and speech [149].Higher resolution, higher frame rate, and lower radiation x-ray recordingis achieved with modern Video- uoroscopy (VF) techniques. VF projects3D movement to a Two Dimensional (2D) plane and therefore multiple pro-jections, e.g. anteroposterior projection in addition to mediolateral projec-tion, are required to accurately characterize 3D movement [79]. Radiopaquemarkers have been used to more easily identify landmarks in a similar way aslight emitting/re ecting markers are used in optical tracking systems. Theposition of standardized landmarks for the jaw, tongue, soft-palate, and hy-oid bone, have been digitized from lateral VF recordings used to analyze thekinematic correlation between articulators in both eating and speech move-ments [79, 115]. VF has also been used to observe hyoid position relative tothe jaw and tongue during wide jaw opening [130] and vowel postures [23].A comparative study observed an anterior shift in hyoid position duringspeech as compared to chewing which was attributed to a need for increasedhypopharynx width during speech [79].VF is also widely used in swallowing analysis and the diagnosis of swal-lowing disorders. The \modi ed barium swallow" [107] is a standard pro-tocol for assessing risk of aspiration. In the procedure, patients are imagedwith lateral projection VF while swallowing a radiopaque liquid (barium),in order to observe whether or not liquid  ows into the airway. Section 3.4.3discusses our preliminary model of hyolaryngeal biomechanics, which we152.1. Functional Data Recordinghave integrated with a dataset of lateral VF recording of the tongue, hyoid,and larynx movement induced by electrical intramuscular stimulation [31].Magnetic resonance imagingMRI techniques use the magnetic properties of hydrogen atoms to createimages of soft-tissue structures in the body [158]. Static MRI techniqueshave been used to capture 3D vocal tract shape during held vowel pos-tures [15, 52]. The tissue to air boundary provides high contrast changesin the image, making the airway highly visible. Maintaining a particulartongue posture limits the scan time, which reduces the spatial resolution ofthe scans. Cricothyroid articulation [191] and tongue muscle structure [190]have also been analyzed during static vowel postures with MRI. DynamicMRI is limited by long acquisition durations and high speed temporal sam-pling comes at the cost of spatial resolution. Cine-MRI involves acquisitionof a time-series of single slice MRI images and requires multiple repetitionsof the same speech utterance in order to reconstruct a dynamic MRI image.Tagged cine-MRI has been used to track the position of internal points inthe tongue tissue through the cine sequence and provides a measure of localtongue deformation [142]. MRI data of 3D tongue surface shape and internaldeformation is a promising modality for integration with 3D biomechanicaltongue models, as discussed in Section 4.3.2.UltrasoundUltrasound (US) imaging uses echoes from pulsed US waves emitted into tis-sue to determine tissue impedance, whereby rapid changes in tissue impedance,such as boundaries between di erent tissue types or between tissue and air,cause re ections that can be transduced. US has been used to visualizethe mid-sagittal contour [182] and the 3D shape [187] of the tongue surfacein speech, which is visible due to US re ection at the tongue surface to airboundary. US recordings are fast, but spatial registration of the tongue con-tour to the mandible or palate can be challenging. US has also recently beenused to characterize tongue movement in glossectomy patients and reported162.1. Functional Data Recordingobservations of higher tongue velocity during certain speech movements ascompared to normal subjects [159].Electromagnetic trackingElectromagnetic (EM) tracking systems measure the position of markerswithin a known EM  eld by sensing the current induced in small coils withinthe markers. EM systems do not require line-of-sight and modern systemsare self-calibrating (e.g. Polhemus Fastrak [155], Carstens AG500 [34], NDIWave [135]), however their accuracy degrades if the EM  eld is distortedby metallic objects in the tracking workspace. EM tracking systems havebeen proposed speci cally for measuring jaw movement, including the ki-nesiograph and the sirognathograph, though optical jaw tracking systemsprovide higher accuracy and faster sampling rates, as described above.Using EM tracking systems for measuring vocal tract articulations iscalled Electromagnetic Articulometry (EMA) and has been widely used toanalyze tongue and jaw movement in speech [52, 148]. In such studies,markers are glued to the surface of the tongue, lip, and teeth. EMA systemsare limited in the number of markers that can be tracked simultaneously,typically up to eight. Therefore, speech studies usually arrange the mark-ers in the mid-sagittal plane since speech movements are mostly bilaterallysymmetrical. A sparse sampling of points on the surface of the tongue doesnot provide detailed spatial information as to the tongue’s 3D shape, but isit e ective in recording the kinematics of a few points of interest, e.g. themovement of the tongue tip. The integration of EM tongue tracking dataand dynamic tongue models is discussed in Section 4.3.2.Measuring mastication or swallowing kinematics with EMA can be prob-lematic because markers glued to the tongue and teeth tend to fall o duringchewing movements. Also, markers placed too far posterior on the tonguemay initiate a pharyngeal re ex (gag re ex). Despite these limitations, arecent study has reported EMA recordings for liquid swallowing that showquantitative coordination between the jaw and tongue.172.1. Functional Data RecordingElectropalatographyElectropalatography (EPG) devices embed an array of contact sensors withina prosthetic palate in order to recorded time-varying patterns of contact be-tween the tongue and palate during tongue movement. In separate studies,EPG was combined with US [188] and MRI [52] to assess tongue movementin speech.2.1.2 ForcesMovement in a biomechanical system is created by muscle forces, whichare considered internal forces in the system. External forces arise when abiomechanical system contacts with the environment. Typically only exter-nal forces are available for measurement. Here we discuss mechanisms torecord external forces in the orofacial system.Tongue-palate force transducersPressure between the tongue and palate can be measured with  uid- lledbulbs placed between the tongue and hard-palate. A study of 853 nor-mal subjects reported a maximum tongue-palate pressure of 40.4 9.8 kPa(mean standard deviation) for adult males aged 40-49 and showed an age-related decrease in maximum pressure in males, which is a factor in swallow-ing disorders [201]. Recent advances in electronic force sensing technologyhas led to a new device, which is similar to EPG but capable of measuringforce at a number of discrete point on the hard palate [83]. This device hasbeen used to record spatial and temporal patterns of tongue-palate pres-sure force during chewing [82] and swallowing [137]. A preliminary studymeasured tongue-tip pressure against the anterior hard palate during speech(repeated /ta/ sequences) with a single point force sensor embedded in a fullupper denture [91]. Section 3.3 discusses our comparison of maximum vol-untary tongue-palate pressure recordings as a means to validate the tonguemuscle force levels in our biomechanical model.182.1. Functional Data RecordingBite force transducersA number of di erent devices have been proposed for measuring force gen-erated between the upper and lower teeth during jaw function [57]. Biteforce during maximal voluntary clench has been measured at di erent lo-cations around the dental arch, illustrating that tooth forces are largest atthe posterior molars and diminish in magnitude at the anterior incisors [76].Maximal  rst molar bite force for an intact mandible is within the range of216-740N [210].ManometryManometry uses a  exible tube with embedded pressure sensors to measurepharyngeal pressures [94]. The manometer tube is inserted through thenasal cavity, along the posterior pharyngeal wall and the lowest pressuresensor is placed at the level of the esophageal sphincter in order to calibrateits spatial position. The dataset used to develop our hyolaryngeal model,discussed in Section 3.4.3, includes manometric recordings of pharyngealpressures during intramuscular stimulation of tongue and laryngeal musclesas well as during liquid swallowing.2.1.3 Muscle activityElectromyography (EMG) involves transducing electrical signals associatedwith muscle activation; however EMG can be di cult to record for small,deep muscles in the head and neck, and the relationship between EMGand muscle force is complex for dynamic movements [173]. As discussedin Chapter 4, movement and contact forces are easier to measure directlythan muscle forces, making model-based estimation of muscle forces duringmovement an attractive option in future clinical experiments.EMG has been used in combination with movement recordings in anattempt to better characterize muscle activity. A number of EMG studieshave been reported for the jaw, including Moller’s pioneering work on jawmuscle activity during chewing [124] with twenty-six subjects. Other stud-ies have investigated activity of the medial [70] and lateral [214] pterygoid192.2. Structural Data Measurementmuscles, which are deep muscles and challenging to access and susceptibleto crosstalk from adjacent muscle groups. Fine-wire EMG recordings ofregional activation in the upper and lower heads of the lateral pterygoidmuscle have also been performed [85, 129] with wire locations within themuscle veri ed through CT imaging of the subject post-recording before thewires were removed [139]. These studies found di erential recruitment indi erent regions of the muscle and support the hypothesis that the upperand lower heads of the lateral pterygoid muscle functionally co-contract andare both \jaw opener" muscles.In the tongue, EMG recording has proven to be a signi cant challengedue to the interdigitation of di erent muscle groups and wire movementduring large tongue deformation; however, a few studies have reported EMGrecordings for vowel tongue postures [49, 121]. In a recent EMG study,the relative contributions of genioglossus and intrinsic tongue muscles werecompared in protrusion tasks and found that both contributed to protrusivetongue movement, but that the intrinsics alone were recruited for generatingprotrusive force against an external resistance [154].2.2 Structural Data MeasurementThe physical arrangement of bones, soft-tissue, and muscles is important forinterpreting the functional observations discussed in the previous section.Orofacial anatomy is reviewed in Appendix B and in this section we discussthree fundamental methods used in anatomical investigation: measurementof cadaver specimens, imaging of living humans, and bench testing of excisedtissue samples.2.2.1 Measuring anatomical structureTraditional methods of study in anatomical science center on the dissectionand measurement of cadaver specimens. Skeletal structure is well preservedin cadaver specimens and anatomical features of bone surfaces can be usedto determine muscle attachment sites. For example, the mylohyoid line, a202.2. Structural Data Measurementprominent ridge along the inner surface of the mandible, is the attachmentsite of the mylohyoid muscle, which forms the  oor of the mouth. Jawmuscle size and  ber properties have been reported through cadaver mea-surements [203{205]. These measurements form the basis for the muscleproperties used in our jaw model described in Section 3.1.1. Tongue muscu-lature has also been examined through cadaveric examination resulting indetailed morphological descriptions of the 3D muscle shapes and  ber struc-ture [120, 192]. These studies form the basis for the spatial and functionalmuscle de nitions used in tongue models, including our model as describedin Section 3.1.2.Histology is another analysis technique that involves microscopy of thinlysliced tissue specimens. Staining techniques are used to highlight di erenttissue structures in the specimen. Histology has been used to determine thepercentage of di erent muscle  ber types (fast-fatigue, fast, or slow type bers) in tissue samples from di erent muscle groups. The Visible HumanDataset (VHD) is a macroscale histology of the entire body [132] consistingof photographed slices of frozen cadaver specimens in 1 mm slices for a malespecimen and 0.3 mm for a female specimen. The VHD was used to helpde ne the muscle geometry in previous tongue models [29].The principle limitation of cadaver studies is the fact that the soft-tissueshape and structure degrades shortly after death, reducing the accuracy ofmuscle size and shape estimates. This is particularly problematic for thetongue, due to its complex arrangement of muscle  bers. Also, anatomi-cal measurements, drawings, and photographs of cadaver dissection do notprovide rich information about the 3D nature of the structures under inves-tigation, which is critical for 3D modeling.2.2.2 Imaging anatomical structureMedical imaging technology has revolutionized our ability to observe internalstructures of the body. Modern techniques provide digital, volumetric 3Ddatasets revealing 3D anatomical structures. Importantly, medical imagingprovides access to living subjects, though x-ray imaging modality usage is212.2. Structural Data MeasurementCT Image Cone-Beam CT Image(a) (b)Figure 2.2: A comparison of medical computed tomography (a) and cone-beam computed tomography (b) images of di erent subjects. Medical CTcaptures a wider range of tissue density at a higher radiation dosage thancone-beam CT. CT image courtesy of Dr. Pierre Badin, Gipsa-Lab.limited due to radiation exposure. Medical image technology is an activearea of research and new imaging methods and image processing techniquesare improving in terms of spatial and temporal resolution and contrast [158].Computed tomographyCT uses multiple x-ray projections from di erent angles to compute 3Dvolumetric data of dense tissue. CT imaging provides high contrast andhigh spatial resolution (0.5 mm3 voxel) images and is well-suited for boneimaging since dense tissues absorb x-rays. Multi-slice systems use multiplex-ray detectors simultaneously to rapidly image and reconstruct CT data.The principle drawback of CT imaging is radiation exposure.Cone-Beam Computed Tomography (CBCT) reduces the radiation ex-pose to the patient, but images a narrower range of tissue density thanmedical CT, limiting its use to bone tissues alone. A comparison of CTand CBCT images are shown in Figure 2.2. Our original jaw-hyoid model,described in Section 3.1.1, used skeletal structure derived from CBCT data,while the new jaw-tongue-hyoid model morphology was registered to CT222.2. Structural Data MeasurementT1-Weighted MRI T2-Weighted MRI(a) (b)Figure 2.3: A comparison of T1 weighted (a) and T2 weighted (b) magneticresonance images of the same subject.data (Section 3.1.3).Magnetic resonance imagingMRI uses magnetic resonance in hydrogen atoms to create high-resolution(1 mm3 voxel) images of soft-tissue structures. MRI parameters can be ad-justed to highlight di erent tissue types. For example, a comparison of T1-weighted versus T2-weighted images is shown in Figure 2.3. High-resolutionMRI has been used to study the spatial extent, shape, and path of extrinsictongue muscles [190] as well as the shape of the laryngeal cartilages [169].Di usion tensor MRI can be used to isolate  ber directions within muscletissue and has been applied to the tongue [64].Image segmentationImage segmentation is the process of specifying regions within medical im-ages that correspond with tissue boundaries. An example is segmenting theboundary of the mandible from adjacent soft-tissues. The di culty of imagesegmentation is related to the degree of intensity variation across the tis-sue boundary. Segmentation of well-de ned tissue boundaries, such as the232.2. Structural Data Measurementboundary between bone and soft-tissue or soft-tissue and air in CT data (seeFigure 2.2), can be achieved with simple thresholding methods. Segmenta-tion of soft-tissue to soft-tissue boundaries, such as between adjacent musclegroups in MRI data, is more challenging and typically requires semi-manualor manual speci cation of regions by a trained anatomist. Automatic imagesegmentation is a active area research, especially with respect to brain andheart imaging domains. Image segmentation in the upper airway domain iscomplicated by soft-tissue movement artifacts between slices or scans mak-ing automatic segmentation techniques less e ective.2.2.3 Measuring mechanical tissue propertiesMechanical properties of human and animal tissues have been examinedthrough ex vivo mechanical testing (see [48] for review). There are onlya few reported studies on orofacial tissue properties. Human tongue andcheek tissue properties, which are incorporated in our tongue model (Sec-tion 3.1.2), have been experimentally examined though indentation testingon cadaver specimens [63]. Also, a suction-based device has been devel-oped for in vivo mechanical testing [164]. Soft-palate tissue properties havebeen measured ex vivo with cadaver specimens [21]. Laryngeal tissue andmuscle properties have been experimentally tested on canine tissues [5, 89].Given the limited literature, experimental measurement of orofacial and up-per airway tissue mechanics is an open research area. In particular, studiesexamining the anisotropic properties of tongue tissue are needed.New techniques for in vivo measurement are needed to assess living tissuemechanics and mechanical changes during muscle activation. Elastography,with US and more recently MRI, is making progress on the analysis of invivo tissue mechanics. The majority of elastography studies have focusedon assessing tissue elasticity for tumor detection, though recent studies haveattempted to estimate elasticity against ground truth data (see Mariappanet al. [112] for a recent review). There are a number of challenges in adaptingthe techniques to the orofacial region, including the required mechanicalvibration of the tissue during the imaging protocol.242.3. Biomechanical Modeling2.3 Biomechanical ModelingThe previous two sections have discussed common methods used by oral bi-ologists and anatomists to gather information about the function and struc-ture of orofacial anatomy. In a general sense, modeling is the process ofabstracting general information about an anatomical system from a spe-ci c set of physical measurements. Models can take on a variety of forms.Some are created as a means of abstracting higher-level forms of informationfrom speci c datasets. For example, Principal Components Analysis (PCA)models reduce a high-dimensional dataset to a small number of principle di-mensions, which can provide insight into those aspects or characteristics ofthe data that are most important. Other models are created by combiningand synthesizing multiple datasets and/or data modalities into a commonframework. For example, a geometric model of 3D anatomy may includebone shapes extracted from CT data together with muscle shapes foundin MRI data. Synthesis models have the added complication that multi-ple datasets must be transformed into a common format, or in the case ofanatomical data, they must be co-registered such that they are spatially con-gruent. The models developed in this dissertation are biomechanical mod-els that combine structural anatomical information with mechanical tissueproperties and functional recordings of movement and force. This sectiondescribes the process of biomechanical model creation, focusing on skele-tal, soft-tissue, and muscle modeling approaches. In addition, previouslyproposed biomechanical models of the orofacial structures are reviewed.2.3.1 Rigid bone modelingBones are often approximated as rigid bodies in biomechanical models ofgross body movement. Multibody techniques [172] can e ciently simulatethe dynamics of numerous rigid bodies along with constraints and contactbetween bodies. Commercial multibody simulation packages include Solid-Works [43] and ADAMS [126]. Several open simulation systems and archi-tectures have been presented to the biomedical community in recent years.OpenSim [46] is a multibody simulator designed for musculoskeletal analysis.252.3. Biomechanical ModelingOur ArtiSynth modeling platform is capable of articulated rigid-body sim-ulation with contact and constraints as well as deformable body simulationas discussed in Section 2.3.2.GeometrySurface meshes are used to de ne bone boundaries in 3D space and arecomposed of a set of 2D triangles or other 2D polygons. They are usedfor visualizing 3D surfaces, as well as to de ne muscle insertion and originlocations, and to detect collisions between structures in dynamic simulations.Surface mesh generation Generic meshes for bones have been createdby anatomists and artists. For example, average-valued dimensions havebeen reported for the mandible [217] and the laryngeal cartilages [50]. Genericmeshes can be co-registered to a subject; however for subject-speci c mod-els it is usually more desirable to generate bone mesh surfaces directly frommedical imaging data of the subject, if such data are available. Surfacemesh generation follows directly from the image segmentation techniquesdescribed in Section 2.2.2. Once region boundaries are de ned as a closedset of voxels in a 3D image dataset, a surface mesh can be constructed alongthe boundary with techniques such as marching cubes. Mesh decimation iscommonly used to reduce the number of triangles in a mesh while attempt-ing to maintain the same surface shape. However, the decimation processcan cause mesh artifacts, such as holes or poorly-conditioned (skinny) tri-angles. A number of open source packages for surface mesh processing andediting are available, including Blender [65] and MeshLab [208].Mesh registration Models with multiple meshes from di erent sourcesor datasets require those sub-meshes to be co-registered into the same spa-tial reference frame. Registering bone meshes from the same subject can bedone by  nding a rigid transformation (translation and rotation) to mini-mize the sum squared distance between a set of corresponding landmarksde ned on each mesh [84]. Registering bone meshes from di erent subjects,given the wide individual variation in orofacial structure size and shape,262.3. Biomechanical Modelingrequires a ne or non-linear transformation. A ne transformations can becomputed from corresponding landmarks in a similar fashion as rigid trans-formations, but include non-uniform scaling and shearing deformations. Ana ne transformation will capture the gross di erences in size and shape;however,  nely detailed shape di erences require a non-linear transforma-tion or morphing. Bucki et al. [30] describes a non-elastic mesh-based reg-istration method that we used to adapt generic jaw and skull meshes to CTdata of a speci c-subject for our jaw-tongue-hyoid model, as discussed inSection 3.1.3.DynamicsSolving for rigid body dynamics requires mass and inertia information foreach body. A common approximation computes an inertia based on the sur-face mesh geometry assuming a constant density. Also, the mass of muscletissue and soft tissue structure is commonly lumped into the mass and in-ertia of the skeletal structures. Experimentally measured masses have beenreported for the mandible [217] and the laryngeal cartilages [50].2.3.2 Deformable tissue modelingDeformable biological tissues include muscles, connective tissue, fat, and mu-cosa. Cartilage and bone structures are also deformable under su cientlylarge loads, though they are often approximated as rigid in models moreconcerned with their gross movements than their internal deformations. De-formable structures are commonly modeled using FEM approaches [17, 22]that compute approximate solutions to the partial di erential equation gov-erning continuum mechanics by spatial and temporal discretization.Commercial software packages for FEM modeling include ANSYS [12]and SIMULIA [42] and are primarily designed for the structural mechan-ics analysis of synthetic materials as opposed to biological tissues. Giventhe computational complexity of FEM techniques, solution times can bevery slow: a one-second simulation of the FEM tongue model describedin [29] can require many hours of computing time. A number of open source272.3. Biomechanical Modelingresearch-oriented toolkits have also been developed, including FEBio [211],a  nite element toolkit with special support for tissue modeling and somesupport for rigid bodies, contact and constraints. Systems geared towardsurgical training include Gipsi [35] and Spring [125]. Sofa [6] provides a gen-eral software architecture in which models can be partitioned into di erentsubmodels for simulating appearance, behavior, and/or haptic response. Tothe best of our knowledge none of these open simulation systems yet providean interactive environment with fully coupled FEM/multibody capabilities.Our ArtiSynth platform combines both FEM and multibody capabilitieswithin an interactive graphical environment and is described in Appendix C.GeometryVolumetric meshes are used to de ne a volume in 3D space and are composedof a set of 3D tetrahedrons, hexahedrons, or other volumetric elements.The volumetric mesh de nes the spatial discretization of the continuummechanics equations in FEM. For accurate FEM solutions, volumetric meshelements have quality requirements concerning their size and shape, such asminimum aspect ratios (see Shewchuk [174] for detailed discussion of FEMmesh quality requirements).Volumetric mesh generation Volumetric mesh generation presents agreater challenge than surface meshes as it involves creating a 3D meshwith 3D elements that  ll a volume without holes or unconnected nodes.Henshaw [75] provides a current review of the state-of-art in automatic vol-umetric mesh generation. Tetgen [178] is a freely available software packagefor generating tetrahedral volumetric meshes from surface meshes and com-mercial FEM software packages include similar tools. Hexahedral meshes aremore challenging to generate than tetrahedral meshes, but are more desirablefor FEM analysis. This is because hexahedral meshes are less susceptible tolocking (erroneous increase in sti ness) when simulating incompressible ornearly-incompressible materials. Manual mesh creation is also common, asis the case for our reference tongue model mesh (Section 3.1.2).282.3. Biomechanical ModelingMesh registration Deformable mesh registration is also a more challeng-ing problem than rigid or a ne registration of bone meshes. Deformable tis-sues can require non-linear transformations, or morphing, to provide a good t between subjects. Bucki et al. [30] report an energy-based mesh mor-phing and registration method that includes constraints on element qualityand has been used to adapt face meshes to a speci c subject. An initiala ne transformation is useful as a good initial guess for more complex non-linear morphing algorithms, which are commonly formulated as optimizationproblems and therefore can be sensitive to initial conditions.DynamicsSolving for deformable body dynamics requires parameters relating to theirmass, sti ness and damping. These parameters depend on the materialused to represent the deformable tissue. For example, a linear material usesYoung’s Modulus (associated with sti ness) and Poisson’s ratio (associatedwith compressibility). Non-linear materials require additional parametersas stress varies non-linearly with strain. Elasticity parameters derived fromtissue measurement, as described above in Section 2.2.3, are dependent onthe chosen material and are  t to the data in order to recreate observeddeformations over a range of prescribed loads in the model.2.3.3 Muscle modelingIn addition to passive skeletal and soft-tissue structure, biomechanical mod-els require muscle force in order to simulate active movements. Musclemodels include a de nition of a muscles spatial extent, its attachment tosurrounding structures (origin and insertion locations), and its force gener-ating capabilities and dynamics.GeometryMuscle geometry de nes how muscle forces are transmitted to surroundingstructures by de ning muscle attachments at origin and insertion locations.292.3. Biomechanical ModelingMuscle geometry that accounts for a muscle’s volumetric extent can alsode ne contact forces between the muscle tissue and adjacent structures.Line-based The most common geometric model for muscle forces is oneor more connected line segments. Muscle forces are transmitted betweenthe two end-points (insertion and origin sites) and intermediate points areusually treated as ideal pulleys that transmit force without energy dissipa-tion. Line-based muscle models are simple and e ective for skeletal musclesthat have small origin/insertion areas and direct paths, which is the case formany jaw and laryngeal muscles. Therefore, we use piecewise-linear musclegeometries in our jaw model (Section 3.1.1). Broad or  at muscles, with largeorigin/insertion areas, can be approximated by a number of muscle \lines"in parallel. Origin/insertion points are typically chosen as the centroid ofattachment regions, however a muscle’s e ective line of action can be modi- ed by its  ber architecture. Muscles with complex paths, wrapping aroundbones and/or through joints, are approximated by de ning \waypoints" tocreate a piecewise linear path wrapping around adjacent structures [46].The \waypoint" formulation can lead to inaccurate transmission of forcesto adjacent structures, especially in complex muscle paths, such as with thewrist, shoulder, or knee.Spline-based Spline-based muscle paths are an extension of line-basedmodels to provide additional path DOF and more accurate wrapping ofmuscles around bones and through joints. This approach has been used tomodel the complex structure of the forearm and hand, demonstrating forcetransmission and sliding between musculotendons and bones surfaces [189].Volumetric A more sophisticated approach for modeling 3D muscle struc-tures is volumetric muscle models that allow for the spatial size of the mus-cle and attachment region to be included in the model. Volumetric musclemodels use deformable tissue modeling techniques, as described above, tosimulate passive mechanical properties of muscle tissue [193]. Active forcesare incorporated in the FEM mesh along the muscle  ber lines-of-action with302.3. Biomechanical Modelingeither discrete line segments, as is used in our reference tongue model (seeSection 3.1.2 and [29]) or transversely-isotropic FEM materials [212]. Vol-umetric methods are required for muscle-activated soft-tissues such as theface, tongue, and soft-palate, and have also been used to model the spatialextent of the jaw muscles [160].Pennate Pennate muscles have muscle  bers oriented obliquely to themuscles’ principle line-of-action. The pennate structure is thought to bea mechanism to trade-o a muscle’s capacity to shorten with its capacityto generate force. The jaw closing muscles, which are primarily intendedto generate bite forces during mastication, have a pennate structure (seeSection B.2.2). For example, the masseter muscle is noted to have multiplemuscle sheets of pennate  bers oriented at di erent angles, which has beensuggested as a mechanism for maintaining bite force throughout a range ofjaw closing rotation [68]. Pennate-muscle structure is commonly neglectedin biomechanics models, making it a interesting area for future investigation.DynamicsA number of parameters are required to characterize the dynamics of mus-cle tissue, including both its passive properties (similar to deformable tissueparameters) and also its force generation under activation. These parame-ters are typically non-linear and are dependent on the chosen muscle modelformulation. Sophisticated models of muscle dynamics can also include acti-vation dynamics and fatigue. A common approximation lumps muscle massinto the mass of adjacent bone structure. This approximation does nottake into account spatial changes in muscle mass due to muscle lengthen-ing/shortening during movement and can cause signi cant errors in dynamicsimulation [140]. The total mass of jaw muscles [205] is roughly equivalentto the mass of the mandible [217], which would suggest that jaw musclemass should be explicitly accounted for in dynamic models. However, jawmovements are slow during chewing and small during speech and thereforethe inertial e ects are less signi cant. Volumetric muscle models, such as312.3. Biomechanical ModelingTensionLengthTotal  TensionPassive StretchContractileRest  LengthMuscleTendonCE SEPECE - Contractile ElementSE - Series Elastic ElementPE - Parallel Elastic Element(a) (b)Figure 2.4: The standard Hill-type muscle model representation [80] illus-trated with mechanical schematic (a) and force-length characteristics (b).our tongue model, incorporate muscle tissue mass into their dynamics.Hill-type model The Hill-type muscle model [216] is widely used in dy-namic modeling of musculoskeletal systems, including previous jaw models.It includes a non-linear force-length relationship in a three-element model,as shown in Figure 2.4.Muscle properties Another muscle model derived from measurementson feline hindlimb muscles reports detailed measurements of activation dy-namics [27]. Also, canine intrinsic laryngeal muscle properties have beenmeasured with ex vivo experiments [5]. For the jaw muscles, Hill-type modelparameters have been derived from measurements of muscle Cross-SectionalArea (CSA) and  ber lengths on cadavers [203{205]. Also, a model-basedstudy reported that low passive tensions in the jaw closer muscles were re-quired to achieve maximum wide gape [146]. We use the Hill-type muscleformulation for the jaw muscles in our model (Section 3.1.1) with CSA pa-rameters given in Table 3.1.322.3. Biomechanical Modeling2.3.4 Isolated orofacial modelsBiomechanical models of individual structures of the human upper airwayhave been developed and used since the 1970s. Model complexity has in-creased due to both the acquisition of new knowledge about anatomical,neurophysiological, and physical characteristics of the articulators, and thevast growth in computational capacity. Early models were based on a 2Dmid-sagittal representation of the airway [144, 150, 151, 162], whereas recentmodels attend to the 3D structure of the articulators.Jaw models3D models of jaw biomechanics commonly use line-based muscle models torepresent the jaw muscles as well as 3D representations of the TMJ andteeth [101, 146]. Recent models have been used to examine TMJ loadingduring chewing [71], open-close movements [199], and after jaw growth [45].A few models have used FEM to model volumetric muscles [160] or thearticular disc [100].Tongue modelsEarly tongue models focused on planar 2D representations because speechproduction is bilaterally symmetric and thought to be primarily a mid-sagittal phenomena. The move to 3D was motivated by the fact that thetongue’s mid-sagittal shape is controlled in part to the mediolateral expan-sion and contraction. Dang and Honda [41] investigated speech productionmovement with a 2D spring-mass model of the tongue using a mid-sagittalmesh with mediolateral thickness. Recent tongue models employ 3D FEMmeshes with hyperelastic constitutive models to better represent non-lineartissue properties [16, 29, 62].Other oral and facial modelsA number of 3D biomechanical face models have been proposed mainly forsynthesizing visual speech. Early models used a spring-mass system ap-332.3. Biomechanical Modelingproach [194], whereas recent models have employed FEM methods. Sifakiset al. [179] proposed a face model with detail volumetric muscle structureand quasi-static FEM coupled with motion-capture data of face movementduring speech utterances. Nazari et al. [133] used line-based muscle modelsalong with a dynamic, hyperelastic and nearly incompressible FEM in or-der to simulate speech movements, including lip rounding and protrusion.Larynx models have primarily focused on intrinsic laryngeal muscles andvocal fold mechanics [90, 122] as opposed to the movement of the hyola-ryngeal complex within the neck. The mechanics of the intrinsic larynx isprimarily applicable to speech production, but is also applicable to airwayprotection during swallowing. Very few biomechanical models of the humansoft-palate and pharynx have been reported. Previous soft-plate modelsused highly simpli ed mid-sagittal plane 2D FEM meshes and simpli edmuscles [19, 145]. A more complex pharynx model that used a sagittal 2DFEM model with mediolateral thickness to represent detailed oropharyngealstructures was used to analyze the mechanics of OSA [86].2.3.5 Coupled orofacial modelsWhile many models of individual orofacial structures have been developed,few models combining and integrating adjacent structures have been re-ported. The focus on individual structures in previous work is likely due tothe signi cant technical challenges with properly modeling the dynamicalcoupling between soft bodies (tongue, lips, soft palate) and hard structures(jaw, hyoid bone, hard palate) in 3D.Sanguineti et al. [162] have reported a highly simpli ed 2D jaw-tongue-hyoid model with a 2D FEM to represent the tongue. Fang et al. [54] havereported a 3D model of the jaw-tongue-hyoid using a discrete mass-springnetwork to represent the tongue tissue. Discrete mass-spring models areknown to be less numerically stable and less able to accurately characterizenon-linear, incompressible biological tissues as compared to FEM. So, to ourknowledge, our approach is unique for providing a 3D modeling frameworksimulating full dynamical interaction between soft and rigid articulators us-342.4. Inverse Methodsing FEM/rigid-body techniques, as discussed in Chapter 3. Such dynamiccoupling is important, especially in speech production where it has beenclearly shown that consideration of the dynamic interaction of vocal tractbony and soft structures is needed to correctly account for orofacial dynam-ics [162].2.4 Inverse MethodsModel-based techniques for estimating muscle forces during movement havebeen proposed for a variety of musculoskeletal systems. Such inverse meth-ods are useful because muscle force is di cult to measure in vivo. Thesetechniques have been recently reviewed by Erdemir et al. [53] and are brie ydescribed in this section. Some of the techniques are concerned with mod-eling the human motor system and incorporate theories of the human mo-tor control and learning, whereas others have a direct correspondence torecorded data.Humans exhibit tremendous motor skill and dexterity. Therefore, it fol-lows that theories of human motor control may inform inverse methods forbiomechanical models. In a complementary manner, computational modelsprovide a means to evaluate theories of human motor control that are basedon experimental observation. Two prominent computational models of mo-tor control include the Equilibrium Point Hypothesis (EPH) and supervisedlearning. Sti ness is also an important mechanism in motor system models.Inverse-dynamics techniques use numerical methods to provide estimates ofmuscle force with direct relationship to recorded data. Inverse-dynamicstechniques, therefore, can be considered \data-driven" approaches as theyare formulated with a close association to recorded kinematic and other datadescribing the motor task under investigation.Equilibrium-point hypothesisThe EPH approach controls a model’s individual degrees of freedom bychanging the rest-length of spring-like muscle models to drive the model352.4. Inverse Methodsbetween successive quasi-static positions [55]. EPH has been used to studyjaw motion in speech, including neuron-motor control [162] and the e ectsof jaw sti ness [175]. It has also been used with a biomechanical tonguemodel [29] to predict muscle forces need to achieve static vowel postures.EPH models incorporate sophisticated muscle dynamics and re exes. Thequasi-static target position assumption implicit in the EPH is controversialand has been disputed [66] and defended [56] by a number of researchers.Supervised LearningMachine learning techniques have been used in attempts to recreate prop-erties of the human motor system in computational models. Supervisedlearning is typically implemented in an arti cial neural network structureand uses labeled training data to adjust the parameters in the network inorder to reduce prediction error. Learning an inverse model of the motorsystem requires a motor error training signal, whereas sensory error (mo-tion error) is more readily available to the motor system. The distal-errortechnique proposed by Jordan and Rumelhart [93] uses a forward model toconvert sensory error into motor error for supervised learning of a feedfor-ward controller. Alternatively, the feedback-error learning model proposedby Kawato et al. [95] uses a  xed feedback controller to map sensory-error tomotor-error for learning an inverse internal model. Porrill et al. [156] haveproposed a recurrent control model that involves supervised learning of aforward model using a sensory-error training signal. Supervised learningtechniques for complex inverse mappings typically su er from slow learningrates and require large amounts of labeled data, which limits their applica-tion to biomechanics simulation. The methods also rely on an approximateinverse solution as a basis (\reference structure" in feedback-error learningmodel [95] and the \brainstem" in Porrill’s recurrent system [156]).Sti ness ModelingSti ness is a functionally important mechanism in the biological motor sys-tem [26] that is frequently neglected in computational models. In optimiza-362.4. Inverse Methodstion approaches, the use of a minimum muscle energy constraint is equiv-alently a minimum sti ness solution. Sti ness in a biomechanical systemis increased with an increase in muscle activation, through co-activation ofantagonist muscle pairs, and by short-latency stretch re ex gains.The control of arm sti ness through co-activation of antagonist muscleshas been illustrated in a simple model of a single joint [81]. The joint torquesinduced by  exor and extensor muscles are given by  F = (T Ko q) aFand  E = (T Ko q) aE, where  q is an angular displacement from rest,T is the maximum isometric muscle-induced torque at the joint, and Ko isthe intrinsic sti ness of the joint. The net isometric torque at a given jointangle is computed by the sum of the  exor-induced and extensor-inducedjoint torques ( n =  F + E), which leads to: n = T(aF aE) Ko(aF +aE) q (2.1)and illustrates that torque and sti ness about a joint can be independentlycontrolled by the di erence and sum of muscle activations respectively [81].A generalized formulation of sti ness control through muscle co-activationis described in Section 4.1.Static optimizationStatic-optimization based inverse dynamics involves estimating net forces ina biomechanical system from recorded kinematics. For example, in simplelimb models joint angle trajectories q(t) are measured, numerical di er-entiation is used to estimate joint velocity _q and acceleration  q, and netjoint torques are estimated from the kinematic variables using knowledgeabout the inertia of the limb segments. The net forces in a biomechan-ical system, however, are insu cient to characterize muscle force due tomuscle redundancy. Therefore, static optimization is performed at each in-stant of movement to decompose net forces (e.g. joint torques) into muscleforces [39, 215]. The optimization relies on an instantaneous cost function,such as minimum excitation, to resolve muscle redundancy. One drawbackof traditional inverse dynamics is that errors in recorded position data are372.4. Inverse Methodsmagni ed in di erentiation for velocity and acceleration [180]. A linear pro-gramming formulation of the static-optimization approach was proposed forpoint-to-point movements of the jaw [98] and is the only previously reportedinverse method for jaw movement modeling.Dynamic optimizationDynamic optimization techniques have also been proposed to compute mus-cle forces over the duration of a movement task. These techniques are usedto predict muscle activity in motor tasks de ned more broadly than speci ckinematic trajectories. For example, while trajectory-tracking methods willspecify a time-series of positions targets to describe the movement, dynamicoptimization methods may only de ne the start and end positions. Dy-namic optimization allows for more biologically plausible optimality criteriato be introduced into the solution, such as minimizing metabolic energyexpenditure per unit distance over the duration of the movement task [8].Full dynamic optimization, however, remains computationally expensive andcurrent methods are intractable for complex biomechanical models. A com-parison of dynamic and static optimization found little di erence in theresulting computed muscle activations, assuming a perfect inverse dynam-ics estimation of joint torques [9], which suggests that static optimizationtechniques can be su cient at least for some motor tasks.Optimal controlAnother approach to dynamic optimization uses optimal control methods tocompute a optimal trajectory of muscle activity for a given motor task. Aparticular formulation, called optimal feedback-control, theorizes that onlytask-related parameters are optimally controlled by the motor system, andhas reported promising results in predicting muscle forces in limb move-ments [197]. The optimal feedback-control method has also been extendedto incorporate a hierarchal structure [198], which relates to observations bymotor physiologists regarding hierarchical organization of the human motorsystem [106].382.5. Biomedical ApplicationsForward-dynamics assisted trackingTo mitigate the inaccuracies of static optimization inverse dynamics, re-cent techniques employ a hybrid approach and compute an inverse solutionof muscle activations that accurately drive a forward dynamics simulationthrough the target kinematics. This approach was termed \forward dynam-ics assisted data tracking" by Erdemir et al. [53] and can improve accuracyas tracking errors in the forward dynamics simulation are used in a feedbackcontrol law to adjust the inverse solution. Computationally e cient algo-rithms have been proposed for gait analysis using either static per-timestepoptimization [196] or optimal control [171] to decompose joint torques tomuscle forces. General formulations of the trajectory-tracking approach havebeen proposed for quasi-static FEM models [179] and dynamic multi-bodysimulation with musculotendon elements [189]. Combining and extendingthese two methods, in Chapter 4 we develop a trajectory-tracking methodfor dynamic FEM models and combined dynamic rigid-deformable models.We also formulate new target parameters, in addition to movement, includ-ing constraint forces and sti ness. The constraint force targets are used topredict muscle activations needed to generate desired reaction forces in thesystem, such as a target bite force during jaw clenching simulations. Thesti ness target can be used to control co-activation of antagonist muscles.2.5 Biomedical ApplicationsBiomechanical models can be applied to biomedical situations, such as an-alyzing dysfunctional cases and evaluating potential avenues of treatment.Modeling studies in orofacial biomedicine have been recently reviewed by Han-nam [69]. Here we point to a few studies that are most closely related toour e orts in modeling the functional consequences of jaw surgery describedin Chapter 5.Jaw surgery In reconstructive jaw surgery, physical 3D models have beenapplied to the fabrication of custom- tting mandible prosthesis with rapid-392.5. Biomedical Applicationsprototyping techniques [20]. Modeling techniques have also been used todesign mechanical templates for reshaping bone grafts for use in jaw recon-struction. The template approach has been shown to reduce the duration ofjaw reconstruction procedures from two-years to six weeks [195]. A recentbiomechanical jaw simulation of distraction osteogenesis (lengthening oneside of the mandible) was used to analyze TMJ forces in a single patientbefore and after treatment [45].Dental implants Numerous studies have applied biomechanical modelingto the analysis of tooth forces and dental implants (see [209] for a review).Most studies use FEM in order to assess the force distribution of implantedsynthetic teeth in order to optimize implant design and location within themandible. Force distribution in tooth restorations has also been analyzedthrough 3D FEM tooth models [92].Maxillofacial surgery Biomechanical face models with static skull struc-tures have been used to predict aesthetic outcomes of maxillofacial surg-eries [36, 123]. These studies kinematically modify skull structure to mimica surgical procedure, e.g. lengthening of the mandible, and use passive FEMmodels of the facial tissue to predict the resulting impact on the face sur-face. To date, such models have not been used to predict post-operativefacial motion or function.Glossectomy Biomechanical tongue models have been used to simulatethe e ect of glossectomy [28, 59]. In a similar methodology as our jawsurgery model, these studies modi ed the structure of a tongue model tomimic tongue resection and reconstruction with free- ap soft-tissue grafts.The reported simulations deal primarily with the e ect of sti ening a sub-region of the tongue, representing a legion or reconstruction, on tonguemovements.Scar tissue modeling Modeling scar tissue mechanics is relevant to manybiomedical applications as tissue scarring arises from burns, radiation ther-402.6. Summaryapy, tissue grafts, and surgical wounds [118]. Few physiological models ofscar tissue have been reported in the literature. Along with a better charac-terization of scar tissue mechanics, computational models of scar tissue areneeded for use in biomedical modeling applications.2.6 SummaryTo summarize, a wide range of techniques have been developed in order toobserve and analyze human movement. Earliest observations used photo-graphic techniques to capture body movement and modern tracking systemsallow for accurate, fast, and automatic quanti cation of movement. The oro-facial system presents a challenge as functionally important structures arelocated within the oral cavity and are therefore less accessible for obser-vation and measurement. Medical imaging techniques are used to captureboth anatomical structure of bones, soft-tissues, and muscles and dynamicmovements of the face, jaw, tongue, and vocal tract. Modeling is a natu-ral extension of observational analysis, whereby anatomical structure andmechanics are combined within a mathematical representation and used tosynthesize hypothetical function. Many models have been proposed in theliterature for the jaw, tongue, face, and other orofacial sub-components.However, few models have integrated these to create holistic models of thecoupled orofacial system. Also, the utility of biomechanical models is lim-ited by a lack of motor control algorithms to automatically simulate pre-scribed motor tasks. As our capabilities to accurately model biomechanicalsystems and simulate dynamic movements increase, numerous biomedicalapplications become feasible, including analyzing dysfunction and planningtreatment for patients.This chapter presented a review of biomechanics modeling approachesand demonstrated the utility of modeling approaches. We have identi edthree areas of investigation for orofacial modeling:  rstly, isolated modelsof individual orofacial structures do not adequately represent the couplednature of the orofacial system; secondly, forward dynamics models alone areinsu cient for analyzing orofacial movements and inverse techniques are412.6. Summaryrequired; and thirdly, models of normal orofacial mechanics can be alteredto re ect dysfunctional systems and applied to analyze potential treatments.In the following three chapters we describe our contributions to these openresearch problems.42Chapter 3Jaw-Tongue-Hyoid ModelIn the previous chapter, we outlined approaches taken by previous researchersto measure, analyze, and model human orofacial anatomy and function.Many models of the face, jaw, tongue, and larynx have been proposed with avariety of modeling techniques and at a variety of modeling  delities. Thesemodels have almost exclusively focused on individual structures in isolation.The degree to which passive mechanical linkages cause functionally impor-tant coupling between structures has not been analyzed. One reason forthe lack of integrated models of orofacial components is the complexity andcomputational cost of FEM simulations. The ArtiSynth simulation plat-form includes state-of-the-art simulation techniques for simulating coupledrigid-body/FEM systems with e cient computational methods and is de-tailed in Appendix C. The coupled mixed-body approach is well-suited tothe analysis of gross movements where the motion of bony structures canbe approximated as rigid while also attending to deformation for soft-tissuestructures.The structural anatomy of the head and neck includes a complex ar-rangement of bones, muscles, ligaments, and soft-tissues, as discussed inSection 2.2 and reviewed in Appendix B. The jaw muscles connect themandible above to the cranium and below to the hyoid bone. The tongue ts within the oral cavity and attaches to the jaw (genioglossus, geniohy-oid, mylohyoid), hyoid (hyoglossus), soft-palate (palatoglossus), and cra-nium (styloglossus). Muscles exert di erent force magnitudes in relationto their size and neural activation. Muscle and soft-tissue interconnectionsbetween sub-structures transmit forces throughout the system. A compu-tational model of the interconnected muscles and structures permits theanalysis of internal forces that are not readily measurable.433.1. Model CreationThe structural and force couplings in the orofacial anatomy is thoughtto be an important aspect of orofacial function. Chewing is considered tobe primarily a jaw muscle action, though coordination of the tongue bodyand buccinator muscle in the cheek is necessary to form and stabilize thefood bolus during the chewing cycle. It is likely that passive stretch offacial tissue and the tongue tissue between the jaw-hyoid has a mechanicale ect, but its extent is not clear without analysis of passive tissue sti nessesand muscle forces. In speech, the coupling of jaw-tongue-hyoid structuresis readily apparent. It is known that both the jaw and tongue articulatorswork together to form the vocal tract shape. Also, the level of muscle forceis small as compared with chewing and therefore passive coupling is likelymore signi cant. Co-articulation in speech production, the observation thatthe relative contributions of jaw and tongue movement to a particular vocaltract articulation are dependent on phonetic context [128], may also be duein part to biomechanical coupling.In this chapter, we develop a new dynamically coupled model of the jaw-tongue-hyoid complex in order to analyze the nature and signi cance of theinterconnected structures. Section 3.1 describes the jaw-tongue-hyoid modelimplementation. Section 3.2 and Section 3.3 present simulations of simplelingual and mandibular motor tasks, focusing on the dynamic interactionbetween the tongue soft structure and the jaw rigid body. Section 3.4 re-ports preliminary results on integrating the face, soft-palate, and larynx inorder to create a complete model of orofacial and upper airway biomechan-ics. Section 3.5 and Section 3.6 discuss the implications of the model anddirections for future re nement.3.1 Model CreationBuilding on our experience with the 3D jaw-hyoid [71] and the 3D tongue [29]models, we have developed the  rst 3D jaw-tongue-hyoid dynamical modeltaking into account full coupling between the FEM tongue model and thejaw-hyoid bony structures. As described in Appendix C, we depend on thevarious components of ArtiSynth to provide dynamic simulations of inter-443.1. Model CreationMTPTATSLPILPDMSMMPDIPDFront Oblique Sagittal Cut-AwayFigure 3.1: Front, oblique, and sagittal cut-away views of the jaw-hyoidmodel. Muscle groups include the anterior, middle, and posterior temporalis(AT, MT, PT), deep and super cial masseter (DM, SM), medial pterygoid(MP), superior and inferior heads of the lateral pterygoid (SLP, ILP), andanterior and posterior bellies of the digastric muscle (DI, PD).actions due to muscle forces of the jaw-tongue-hyoid complex and contactphenomena such as tongue-palate collisions.3.1.1 Jaw-hyoid modelFor the jaw-hyoid model, we started from a previously published model de-veloped in ArtiSynth and used to simulate free jaw movements [183] andchewing [71]. The model included rigid-bodies for the skull, jaw, and hy-oid bone, point-to-point Hill-type actuators for the jaw muscles, constraintsurfaces for the TMJs, and planar unilateral constraints for teeth contact.The same Hill-type muscle dynamics were used from the original jaw-hyoidmodel with force capacity proportional to maximum CSA based on previousstudies of jaw [104, 146] and tongue [29] muscles and listed in Table 3.1. Theinstantaneous force generating capacity of the jaw muscles vary non-linearlywith length and linearly with shortening velocity consistent with previousjaw models [104]. The skull is  xed in space. The adapted jaw-hyoid modelis shown in Figure 3.1.453.1. Model CreationJaw Closer MusclesName AT MT PT SM DM MPMaxForce (N) 158.0 95.6 75.6 190.4 81.6 174.8CSA (mm2) 395 239 189 476 204 437Jaw Opener MusclesName SP IP DIMaxForce (N) 28.7 66.9 40.0CSA (mm2) 72 167 100Tongue Intrinsic MusclesName GGA GGM GGP VERT TRANS IL SLMaxForce (N) 32.8 22.0 67.2 36.4 90.8 16.4 34.4CSA (mm2) 82 55 168 91 227 41 86Tongue Extrinsic MusclesName STY HG MY GHMaxForce (N) 43.6 118 35.4 32.0CSA (mm2) 109 295 88 80Table 3.1: Physiological Cross-Sectional Area (CSA) and maximum forcegenerating capability of jaw and tongue muscles.3.1.2 Tongue modelFor the dynamic tongue model, we implemented the model by Buchaillardet al. [29] in ArtiSynth, as pictured in Figure 3.2. The original tongue modelwas based on the anatomy of a single subject using CT data and developedin the ANSYS environment [12] representing the tongue with hexahedral nite elements and hyperelastic properties. Thanks to a collaboration withBuchaillard and colleagues, we were able to obtain data for the 3D tonguemesh and description of the lingual muscular  bers. The mesh and musclegeometry were imported into the ArtiSynth environment, using a large defor-mation FEM framework, hexahedral elements with a density of 1040 kg/m3,and a  fth order incompressible Mooney-Rivlin material with c10 = 1037,c20 = 486, and c01 = c11 = c02 = 0 Pascals. The deviatoric potential energy^ of this material is hence^ = c10( IC 3) +c20( IC 3)2 (3.1)463.1. Model CreationSTYILMHGGAGHGGPGGMSLSTYMHFront Back Sagittal Cut-AwayFigure 3.2: Front, back, and sagittal cut-away views of tongue model. At-tachment nodes are also shown for the jaw (front view, red spheres) andthe hyoid bone (back view, blue spheres). Muscle groups include the ge-nioglossus (GGA, blue; GGM, green; GGP, red), styloglossus (STY, cyan),geniohyoid (GH, magenta), mylohyoid (MH, orange), hyoglossus, (HG, red),vertical (VERT, green), transverse (TRANS, blue), inferior longitudinal (IL,cyan), and superior longitudinal (SL, magenta) muscles.where  IC is the  rst invariant of the deviatoric component on the rightCauchy-Green tensor [22]. The Mooney-Rivlin material was chosen for con-sistency with the reference tongue model and because previous studies haveexperimentally measured the material parameters from indentation tests oncadaveric tongue and cheek tissue [63]. Incompressibility was implementedusing a constraint based approach to maintain the volume of each hexahedralelement.Muscles are represented by sets of elements and implemented with node-to-node  ber forces distributed throughout the muscle elements along theprinciple direction of action. We chose to use a straightforward model formuscle activation with  ber forces directly scaled by input activation, asopposed to the Equilibrium Point Hypothesis (EPH) ( -model) used in theoriginal tongue model. Our aim was to quantify the coupling between tongueand jaw and not to work on the  -model motor control assumptions providedby the equilibrium point hypothesis [55]. Tissue sti ening due to muscleactivation was also modeled in the same way as Buchaillard and colleagues,i.e. a linear increase of c10 and c20 values ranging between (1037 Pa, 486 Pa)at no activation and (10370 Pa, 4860 Pa) at full muscle activation. As in the473.1. Model Creationoriginal model, each muscle’s force capacity was a function of its CSA (seeTable 1 in [29]), with force capacity distributed across  bers weighed by thevolume of their surrounding elements. We are currently investigating a moresophisticated muscle modeling approach, as discussed in Section 3.5, using atransversely anisotropic material to more accurately represent muscle tissuemechanics with FEM.3.1.3 RegistrationTo conform the di erent morphology of the two models, we adapted skeletaland muscle geometry of jaw-hyoid model to  t CT data (shown in Fig-ure 1.1a) for the subject upon which the Buchaillard tongue model wasbased. The 3D jaw, skull, and hyoid bone surface meshes were morphedwith an energy-based non-linear mesh registration algorithm [30] to a 3Dskull surface segmented from CT data. Symmetry was attained by mirroringthe left-side of the registered meshes. The inertia of the jaw and hyoid bonewere computed from new mesh shapes, assuming uniform density of 3600kg/m3 and 2000 kg/m3 for the jaw and hyoid bone respectively. Jaw muscleorigin and insertions points were adapted with the same non-elastic trans-formation as was applied to the surface meshes and were manually veri edto correspond to plausible anatomical landmarks. We removed the point-to-point geniohyoid and mylohyoid muscles from the jaw-hyoid model as thesewere included in the tongue model. The anterior and posterior digastricmuscles were connected to the hyoid bone with the digastric sling modeledas a pulley.3.1.4 AttachmentHyoid suspension in the reference tongue model was done using eight ver-tical springs with 220 N/m sti ness to connect the hyoid bone to a  xedlarynx [29]. Vertically-oriented One Dimensional (1D) springs do not accu-rately represent the 3D sti ness of connective tissue, ligaments, and mus-cles connecting the hyoid bone to the pharynx and larynx within the neck.Therefore, in our coupled jaw-tongue-hyoid model, instead of vertical 1D483.1. Model CreationFront Oblique Sagittal Cut-AwayFigure 3.3: Front, oblique and sagittal cut-away views of the dynamicallycoupled jaw-tongue-hyoid model.springs, we use a linear 6-DOF translational/rotational spring to connectthe hyoid bone to a  xed larynx. Lacking su cient physiological data onthe sti ness of the hyoid within the neck, we set the spring sti ness to beconsistent with the reference tongue model (8 220 N/m). We are currentlydeveloping a dynamic larynx model, which will allow us to more accuratelyrepresent the extrinsic laryngeal connective tissue and muscles in order tosimulate hyolaryngeal movement within the neck (see Section 3.4.3). Giventhe limitation of only including passive hyoid sti ness in the model our cur-rent simulation results do not focus in the accuracy of hyoid movement.We couple the dynamics of the jaw, tongue, and hyoid models by de ningattachment constraints between the FEM nodes of the tongue and the jawand hyoid rigid bodies, as described in Section C.1.3. Point to rigid bodyattachments can be made at arbitrary locations and are not required tobe coincident with the rigid-body surface mesh. The attachment pointsin the model are shown in Figure 3.2. Tongue-jaw attachments includethe insertion of genioglossus and geniohyoid onto mandibular geniotubercleand the insertion of mylohyoid along mandibular mylohyoid ridge. Tongue-hyoid attachments include the entire region around the anterior-superiorsurface of the hyoid bone, including insertions of geniohyoid, mylohyoid, andhyoglossus muscles. The posterior medial surface of tongue is not attachedallowing the base of the tongue to move relative to the hyoid bone. The493.2. Simulation DescriptionsJaw TasksCLRy DI ILP SLPrest | | | |clench 10 | | |open | 15 15 15hinge-open | 15 | |protrude | | 15 15right-lateral | | 15? 15?Tongue TasksCLRy SL TRANS GGP GGM STYretract | | | | | 25palate 0.5 30 30 60 30 |max-palate 1 100 100 80 30 10Table 3.2: Percentage muscle activation used in jaw-tongue-hyoid tasks.yJaw closing muscles (AT, MT, PT, MP, SM, and DM). ?Only left-sidedmuscles activated.resulting combined model is pictured in Figure 3.3. The soft palate andpalatoglossus muscle are not included in the current model as we are notcurrently interested in soft-palate movement. However, an extended modelthat includes additional upper-airway structures is under development asdescribed in Section 3.4.3.2 Simulation DescriptionsWe chose to simulate a set of tasks similar to those reported for the jaw andtongue models in isolation, including free jaw movements [183], unilateralchewing [71], and tongue movements in speech [29]. All of the tasks, with theexception of unilateral chewing, involve simple piece-wise linear input mus-cle activations so that the passive dynamic coupling e ects can be analyzed.Our objective was to analyze the e ect of dynamic coupling by using muscleactivation to drive the coupled jaw-tongue-hyoid model and observe di er-ences in the movement of the \active" body as well as movement inducedon the \passive" body.503.2. Simulation Descriptions0 1002003004005006007008000Time (ms)Activation (%)0 200 4000Time (ms)Activation (%)Figure 3.4: Input muscle activation pattern for jaw tasks (left) and tonguetasks (right). The activation amplitude for each task is given in Table 3.2.3.2.1 Jaw activated tasksJaw movement tasks used jaw muscle activation as input and were per-formed both with the jaw-hyoid model alone and with the jaw-tongue-hyoidmodel. All jaw tasks involved a simple pattern of muscle input (rest, ramp-up, hold, ramp-down, rest) as illustrated in Figure 3.4. The duration ofthese standardized jaw movements (600 ms) are consistent with an averagechewing cycle. The muscle sets and activation amplitudes were chosen to bewithin physiologically-plausible ranges for jaw movement (10%{15%) andare summarized in Table 3.2.A nominal jaw movement task is rest posture: the equilibrium position ofthe model with no muscle activation and under downward gravity. In relaxedhumans, the jaw typically rests with a 4-6 mm incisal separation. We expectslightly wider gape in jaw-tongue-hyoid model than in the jaw-hyoid modelalone, though a majority of the tongue body rests on the hyoid bone, andtherefore should result in minimal jaw lowering. Static jaw clenching wassimulated with bilateral activation of jaw closing muscles. With no tonguemuscle activation we expect the tongue to remain stationary within themouth during tooth clenching.Opening is simulated by bilateral activation of lateral pterygoids alongwith the anterior belly of digastric. We chose to simulate a moderate openinggape, with 15% activation in each muscle. Maximum jaw opening in humansis 50 mm on average though both backward head rotation and hyoid posi-tioning become important at wide gape [130] and would complicate the task.We also simulated hinge-like jaw opening with activation of digastric alone.513.2. Simulation DescriptionsWe expect reduced opening with the jaw-tongue-hyoid model due to passivecompression at the  oor-of-mouth. The tongue is actively retracted duringvowel production, such as /a/, however our opening simulation is with thetongue at rest and therefore we expect the tongue to passively protrudefrom the mouth during jaw opening [108]. For this reason, the simulationwill require contact handling (see Section C.1.4) between the tongue tip andthe lower teeth.Protrusion is simulated by bilateral activation of lateral pterygoid mus-cles. The e ect of jaw protrusion on the tongue has important implicationsfor OSA as a common therapeutic device is a dental appliance used to ad-vance the jaw and tongue in order to open airway [167]. We expect reducedjaw protrusion in jaw-tongue-hyoid case as compared to the jaw-hyoid modelbecause the lingual elastic connection between the jaw and hyoid should pro-vide some resistance to jaw movement. We also expect forward translationof the base of the tongue. Right laterotrusion is simulated by activation ofthe left-side lateral pterygoid muscles. We also expect reduced lateral devi-ation in the laterotrusion task with jaw-tongue-hyoid model as compared tothe jaw-hyoid model alone.Unilateral chewing involves a complex pattern of jaw muscle activity [124].We simulated right-sided chewing movement using the same muscle ac-tivation patterns and food bolus that were reported for our original jawmodel [71]. An elastic, spherical food bolus (10 mm in diameter) was posi-tioned between the right  rst molars, which provided resistance during theclosing phase of the chewing stroke and collapsed when the applied forceexceeded 35 N. Since our adapted jaw-hyoid model has a di erent bone andmuscle geometry, we expect that its chewing movement will be altered, butstill plausible, as compared to the original jaw-hyoid model. We also expectthat the chewing movement for the jaw-tongue-hyoid model will be signif-icantly altered as the muscle patterns were previously tuned by Hannamet al. [71] to a model without a tongue.523.3. Simulation Results3.2.2 Tongue activated tasksTongue movement tasks used tongue muscle forces to move and deform thetongue within the mouth. Tongue tasks involved a ramp-up, hold, andramp-down pattern of muscle input similar to the jaw tasks, but with fastertransitions (50 ms, see Figure 3.4) for consistency with the speed of speechmovements. Tongue retraction was simulated by activating styloglossus us-ing the activation trajectory shown in Figure 3.4. We expect that, with thejaw at rest, a retracted tongue posture should induce backward movementof jaw.Tongue-palate contact is an important movement for speech. Tongue tipcontact with the anterior hard palate was simulated by activation of superiorlongitudinal, posterior genioglossus, and transverse muscles as illustratedin Figure 3.8. We stabilized the jaw with low-level (0.5%) activation ofjaw closing muscles to maintain a nearly closed jaw posture. We expectthat tongue-palate contact will induce a downward movement on the jaw,causing the jaw to open wider. We also performed a maximum tongue-palatepressure simulation by ramping the superior longitudinal and transversemuscles to maximum activation. The ability to generate tongue-to-palatepressure is an important component of healthy swallowing function and weexpect that the model’s maximum tongue-palate pressure will be comparablewith recorded maximum tongue pressure measurements. The tongue-palatecontact simulation required contact handling between the tongue tip andthe hard palate surface mesh (see Section C.1.4).3.3 Simulation Results3.3.1 Jaw-tongue-hyoid couplingWe observed a number of interesting in uences of dynamic coupling on thesimulated jaw-tongue-hyoid movements. Jaw movements were altered bythe presence of the tongue and tongue movements were observed to inducejaw movement. We found a resting jaw posture with 5.6 mm and 6.6 mmincisal gape for the jaw-hyoid and jaw-tongue-hyoid models respectively.533.3. Simulation Results0 5 10−20−15−10−50Anterior − Posterior (mm)Inferior − Superior (mm)JTHJHJTHJHFigure 3.5: The jaw-tongue-hyoid model pictured during rest posture(Rest) and at peak jaw opening (Open) with grid spacing of 10 mm. Jawopening induced passive tongue protrusion such that the tongue tip restedon the lower teeth. Right-most plot shows a lateral view of incisor pointmovement for opening and hinge-opening with the jaw-hyoid model alone(JH; blue) and the jaw-tongue-hyoid model (JTH; red) with point spacingof 10 ms.The larger incisal gape at rest in the jaw-tongue-hyoid case is due to theadded mass of the tongue, but change is small. Also, resting jaw postureis thought to be maintained by low level tonic jaw muscle activity [71] thatwas not included in our rest posture simulations. Static clenching withthe jaw-tongue-hyoid model simulated correctly with the tongue remainingstationary in the mouth.The results of the jaw opening and hinge-opening simulations are shownin Figure 3.5. In both cases the amplitude of jaw opening is reduced inthe jaw-tongue-hyoid model due to compression of the lower portion of thetongue between the jaw and hyoid. Figure 3.5 also illustrates the 3D modelat the wide gape position showing that jaw opening does indeed cause passiveforward protrusion of the tongue such that the tongue tip is resting on thelower teeth.Protrusion and laterotrusion movements are shown in Figure 3.6. Theamplitude of protrusion was reduced in the jaw-tongue-hyoid model, likelydue to stretching of tongue tissue between the jaw and hyoid, but the am-plitude of lateral deviation was comparable. Interestingly, the tongue also543.3. Simulation ResultsProtrusion−15 −10 −5 0−10−50Anterior − Posterior (mm)Inferior − Superior (mm)JTHJHRight Lateral Movement−10 −5 0−10−50Anterior − Posterior (mm)Inferior − Superior (mm)051015−10−50Right − Left (mm)Inferior − Superior (mm)JTHJHJTHJHRight Unilateral Chewing−5 0 5 10−25−20−15−10−50Anterior − Posterior (mm)Inferior − Superior (mm)−50510−25−20−15−10−50Right − Left (mm)Inferior − Superior (mm)JTHJHJTHJHFigure 3.6: Lateral and frontal views of incisor point movement during jaw-muscle activated simulations for the jaw-hyoid model (JH; blue) and thejaw-tongue-hyoid model (JTH; red) with point spacing of 10 ms.553.3. Simulation Results0 0.51 1.5 2−1.5−1−0.500.5Anterior − Posterior (mm)Inferior − Superior (mm)Figure 3.7: The jaw-tongue-hyoid model pictured during rest posture(Rest) and at tongue retraction with styloglossus activation (Retracted)with grid spacing of 10 mm. The right-most panel plots a lateral view ofincisor displacement with point spacing of 10 ms.induced signi cant downward movement of the jaw during protrusion andlaterotrusion. The tongue model is pulled forward during jaw protrusiontransferring more of its weight from the hyoid bone to the jaw.Right-side chewing movement produced by applying muscle patternstuned for a di erent jaw geometry provided a plausible tear-drop shapedincisor movement in the current jaw-hyoid model as shown in Figure 3.6.The movement compares well with one produced by the original jaw-hyoidmodel (see Figure 2 in [71]). The amplitude of the chewing envelope is re-duced with the jaw-tongue-hyoid model, which is consistent with the jawopening and laterotrusion simulations.Simulation of tongue retraction in the mouth is pictured in Figure 3.7(left panels), along with a plot of incisor displacement (right-most panel).Styloglossus activation initially causes an upward and backward movementof the incisor as the condyles move up the articular slope, followed by a back-ward and downward displacement as the tongue retracts farther. The tongueretraction simulation also demonstrates the large range of tongue movementcapable with the model and motivates the need for a large-deformation FEMapproach.Figure 3.8 shows the mid-sagittal position of jaw, tongue, and hyoidfor the tongue-palate contact simulation. Tongue lifting and palate contact563.3. Simulation Results−4−3−2−1 0345678Anterior − Posterior (mm)Inferior − Superior (mm)Figure 3.8: The jaw-tongue-hyoid model pictured during rest posture(Rest) and with tongue lifted into contact with palate (Palate-Contact)with grid spacing of 10 mm. The right-most panel plots a lateral view ofincisor point displacement, which starts before tongue-palate contact ( de-notes the beginning of tongue-palate contact) with point spacing of 10 ms.causes a downward jaw movement as expected. The right-most panel plotsthe incisor point displacement, which starts before tongue-palate contact(as denoted by the  on the plot). The initial downward jaw movementis caused by tongue muscle activation and it increases as force is appliedbetween the tongue and palate.3.3.2 Comparison with published dataThe jaw-tongue-hyoid model has been assembled from previously reportedreference jaw [71] and tongue models [29]. Qualitative evaluation of the cou-pled model shows similar levels of force and range of movement as exhibitedby each individual model. Therefore, the new implementation of these mod-els in the ArtiSynth framework compares well with the previously publishedversions. As a step toward validation of the jaw-tongue-hyoid model we havemade preliminary comparisons to published data on tongue kinematics andforces.We used the tongue retraction simulation as a means to compare tonguevelocity generated in the model with measured tongue movement duringspeech. Payan and Perrier [144] recorded the movement of surface tonguepoints with EMA during tongue retraction in a [y-o] speech utterance for573.3. Simulation Results0 0.10.20.30.40.50.60.70.80.9050100150200Time (s)Velocity (mm/s)0 50 100150200050100150200Time (ms)Backward Velocity (mm/s)Data ModelFigure 3.9: Backward velocity of a point on the upper tongue surfacerecorded with an electromagnetic tracking system an during [y-o] speechutterance (Data; reproduced from [144] page 15,  gure 9a) and simulatedwith the tongue model (Model; point shown as cyan sphere in Figure 3.7).the speaker upon which the tongue model was based. Figure 3.9 showsthe recorded velocity pro le reproduced from Payan and Perrier [144] aswell as the simulated anterior-posterior velocity of one node on the tongue’supper surface. Simulating tongue retraction with a physiological level ofstyloglossus activation (25%) generates a peak velocity that compares wellwith the recorded velocity. Both velocity pro les are bell shaped and thepeak velocity values are 210 mm/s and 215 mm/s for the recorded and sim-ulated movements respectively. The velocity pro le in the simulated caseis narrower and also slightly asymmetric (the right side of the bell-shapedcurve tapers o more slowly). The asymmetry may be due to the fact thatincreasing velocity is created by styloglossus activation while decreasing ve-locity is caused only by passive forces slowing down the tongue. In recordedmovement antagonist muscles may be recruited to slow down the tonguemore quickly. Also, our simulated tongue retraction was generated with alinearly ramped styloglossus activation, whereas the real vowel movementmay be caused by a exponential increase in styloglossus activation, whichwould widen the velocity pro le. The damping parameters in the tonguemodel will a ect the tongue velocity pro le and could also contribute to thediscrepancy in velocity pro le shape. We plan to investigate these aspects583.3. Simulation Resultsof speech-like tongue movement simulations further in future studies.We used the maximum tongue-palate pressure as a metric to evaluatewhether or not the parameters for tongue muscle forces in the model arewithin a plausible range. The maximum force-generating capability of themodel’s tongue muscles are proportional to their CSAs (see Table 3.1 and[29, 205]). We also use a CSA-to-maximum-force constant of 40 N/cm2 [146].Direct measurement of muscle force in vivo is not possible; therefore werely on external force measurements as an indirect means to evaluate theresultant force generation capability with the jaw-tongue-hyoid model. Inparticular, Utanohara et al. [201] used a balloon-type disposable oral probeto measure tongue pressure by having subjects compress it onto the palatewith maximum voluntary e ort. The authors recorded pressures for a largesubject pool (850 subjects) and report 40.4  9.8 kPa (mean  standarddeviation) maximum tongue pressures for subjects between forty and forty-nine years of age. Pressure between the tongue-palate contact in the modelwas calculated by dividing the magnitude of the contact constraints by thearea of the contact contours (see Section C.1.4 for discussion of collisiondetection and handling in ArtiSynth). Maximum tongue-palate pressuresimulated with the model was 38.2 kPa, which compares well with the meanvalue of 40.4 kPa reported by Utanohara et al. [201]. The range of measuredpressures ( 9.8 kPa) could also be used to assess the sensitivity of thetongue muscle parameters. For example, applying plausible ranges of tonguemuscle CSAs or CSA-to-maximum-force constant in the model could be usedto determine if they achieve the measured range of tongue-palate pressures.We plan to move to a more sophisticated model of tongue muscle ac-tuation by incorporating muscle  ber forces into the FEM material witha transversely isotropic constitutive law as discussed below in Section 3.6.This will require additional parameters to describe how the force generatingcapacity of tongue muscles vary with muscle length and shortening veloc-ity (e.g. a Hill-type muscle formulation, see Section 2.3.3). The maximumvoluntary tongue-palate pressure validation will then need to be revisitedfor the new muscle formulation and additional experimental data may beneeded given the additional model parameters.593.3. Simulation Results0 0.2 0.4 0.6 0.8 1051015 Displacement Norm (mm)Lower Incisor0 0.2 0.4 0.6 0.8 102468 Displacement Norm (mm)Lower Incisor0 0.2 0.4 0.6 0.8 1051015 Displacement Norm (mm)Tongue TipTime (s)0 0.2 0.4 0.6 0.8 102468101214 Displacement Norm (mm)Tongue TipTime (s)Jaw Opening Tongue-Palate ContactFigure 3.10: Comparison of results for 1 ms (solid lines) and 5 ms (dottedlines) integration steps for the tongue tip (top) and lower incisor (bottom)during jaw opening (left) and tongue-palate contact (right) simulations.3.3.3 Integration ErrorThe simulations reported above were computed using a second-order New-mark integrator (see Section C.1) with a 5 ms integration step size. In orderto assess the numerical error we compared with results computed with a1 ms integration step. Di erences between 5 ms and 1 ms integration stepswere found to be small. Figure 3.10 plots lower incisor point and tonguetip displacements for the jaw opening and tongue-palate contact simula-tions computed with 5 ms and 1 ms step sizes. Error was computed as thedi erence between the displacement trajectories relative to the maximumdisplacement. The jaw opening simulation showed a very small di erencebetween the 5 ms and 1 ms integration step conditions: the incisor dis-placement error had average and maximum values of 0.5% and 1.4% andthe tongue tip displacement error had average and maximum values of 0.7%and 1.6%. Larger discrepancies were found in the tongue-palate contact task,which involves signi cant contact situations: the incisor displacement error603.4. Additional Orofacial Sub-ModelsFront Sagittal CutawayFigure 3.11: Frontal and sagittal cut-away views of an integrated FEM facemodel integrated with the jaw-tongue-hyoid model showing tongue protru-sion through the lips.had average and maximum values of 1.1% and 3.8% and the tongue tip dis-placement error had average and maximum values of 2.0% and 10.0%. Theintegration error in the contact simulation is likely due to the discontinuousnature of contact as well as the spatial discretization of the palate/tonguesurface meshes on which collisions are detected and responses are generated.Even in the worst case, though, the integration error remains small.3.4 Additional Orofacial Sub-ModelsWe are also expanding our jaw-tongue-hyoid model to include other anatom-ical structures toward a complete model of orofacial and upper airway biome-chanics. Our preliminary models include the face, soft-palate, and larynx.3.4.1 Face modelWe have registered and integrated a muscle-activated FEM model of theface [133], as shown in Figure 3.11. The face model has been adapted tothe same subject using CT data in a similar fashion as the skull meshesfor the jaw model [30]. The model includes three layers of tissue and line-613.4. Additional Orofacial Sub-ModelsFront Side BackFigure 3.12: Oblique front, side and oblique back views of the soft-palatemodel registered to the jaw-tongue-hyoid model. The mandible is cut-awayin the side and oblique back views for clarity.based muscle models representing the primary facial muscle groups. We haveperformed preliminary simulations with the face-jaw-tongue-hyoid modelactivating the jaw and tongue muscles with the face at rest in order toinvestigate the e ect of passive facial soft-tissue forces on jaw movement.This model has a number of potential high-impact applications. Face andlip motion are important to audio-visual communication. A large amountof face deformation and lip opening is caused by jaw motion and previousface models have not included muscle-activated jaw structures. Also, facialtissues are important in mastication because the cheek (buccinator muscle),together with the tongue, is used to form and manipulate the food bolusduring chewing. The integrated model will allow us to simulate chewingwith a more realistic, free- oating food bolus. Finally, face modeling has anumber of biomedical applications, most notably predicting aesthetic out-comes of alterations to underlying bone structure, such as in orthognathicsurgery [36]. Our integrated orofacial model allows for prediction of facialappearance during dynamic jaw and face movements, as opposed to previousmodels that were limited to static facial aesthetics.623.4. Additional Orofacial Sub-Models3.4.2 Soft-palate modelWe have also developed a preliminary FEM model of the soft-palate, aspictured in Figure 3.12. The model geometry was build by manual seg-mentation of MRI data and includes line-based muscle  bers. The modelhas been approximately registered to the jaw-tongue-hyoid model. On-goingwork includes adapting and registering the geometry to  t the subject uponwhich the jaw-tongue-hyoid model was built, which will be informed byexisting datasets on that subject’s soft-palate shape [170]. Also, the soft-palate muscles need to be connected to surrounding structures, including thepalate elevator muscles to the skull (which may require more detailed bonestructure at the base of the skull to determine muscle paths and insertionsites, see Appendix B), the palatoglossus muscle (anterior palatal arch) tothe tongue, and the palatopharyngeus muscle (posterior palatal arch) to thepharynx (which has not yet been modeled).The soft-palate is similar in structure to the tongue, but on a smallerscale. It is functionally important in swallowing, as it seals o the nasophar-ynx from the oral cavity, and in respiration, as it is a common site of airwayobstruction in OSA. Therefore, the integrated tongue-palate-pharynx modelhas a number of important potential biomedical applications.3.4.3 Hyoid-larynx modelWe have developed a model of the larynx, including the cricoid and thyroidcartilages and extrinsic laryngeal muscles, as pictured in Figure 3.13a. Weare interested in the gross movement of the larynx within the neck as itanchors the hyoid bone and tongue through passive connective tissue andthe extrinsic laryngeal muscles. Elevation of the hyoid and larynx duringswallowing has been associated with airway protection and down-folding ofthe epiglottis [202].We are building the hyolarynx model based on a dataset of hyoid andlarynx movements, induced by intramuscular stimulation of tongue and la-ryngeal muscles, recorded with VF [31]. One frame of VF data is shown inFigure 3.13b with manually selected landmarks. The mechanical properties633.5. Discussion(a) (b)Figure 3.13: The dynamic hyoid-larynx model (a) and one frame video  uo-roscopic data showing manually selected landmarks (b). Video  uoroscopyis used to record movement of the hyoid bone and larynx induced by intra-muscular stimulation of extrinsic laryngeal and tongue muscles.of the extrinsic laryngeal muscles are largely unknown. Therefore, we areusing the muscle stimulation data to estimate the e ect of individual muscleforces on hyolaryngeal movement in order to select plausible ranges of forceparameters for the model. Clearly, the tongue plays a signi cant role in hy-oid and larynx movements and therefore the integrated tongue-hyoid-larynxmodel is required to accurately assess muscle function. We plan to applythe hyolaryngeal model to analyze the mechanics of hyoid elevation and air-way protection. Also, electrical muscle stimulation is being used clinicallyin dysphagia therapy [109] and biomechanical analysis could help informstrategies for e ective muscle stimulation during swallowing.3.5 DiscussionThe simulations reported here are a proof of concept demonstrating the ef-fectiveness of our hard-soft tissue simulation and motivating the need toinclude dynamic coupling in simulations of jaw-tongue-hyoid movements.The reported simulations demonstrate that a wide range of movement, largeforces, and large tissue deformations are possible within the current simula-tion framework. We have provided preliminary qualitative comparisons of643.5. Discussiontongue velocity and pressure to illustrate that the model behaves within aplausible range of human movement and force production. Model valida-tion is important and highly dependent on a model’s intended use. Recentreviews have provided high-level guidelines for biomechanical model valida-tion [7], with a focus on modeling domains where detailed experimental dataare available and direct validation metrics are applicable. As discussed inSection 2.1 and stated by Hannam [69], acquiring detailed experimental datafor upper-airway function with su cient quality for direct model validationis a signi cant challenge. Indeed, one of the main motivations for develop-ing upper airway models is the di culty of experimentally assessing upper-airway biomechanics. We believe that indirect validation metrics, such asthe incisor movement, tongue velocity and pressure comparisons made inSection 3.3.2, are the only feasible approaches to validating biomechanicalmodels of the upper airway. Further indirect validation of the jaw-tongue-hyoid model is planned, as additional quantitative data were recorded fromthe subject upon which the model is based [14].The passive suspension of the hyoid bone to a  xed larynx in our modelis a limitation. The amplitude of hyoid displacement in our simulationsis smaller than reported in experimental recordings of eating and speakingmovements [79]. This may be attributed to the sti ness of the 6-DOF hy-olaryngeal spring in the model, but is also likely due to the  xed larynxand lack of hyoid depressor muscles. The restricted jaw gape and forwardtongue protrusion during jaw opening may be due in part to the reducedhyoid movement, though backward rotation of the head has also been shownto be important to wide jaw opening [99, 130]. We are currently developinga dynamic larynx model, as discussed in Section 3.4.3, including extrinsic la-ryngeal connective tissue and muscles, which will allow us to better analyzehyoid movement in our coupled model.The results of the chewing simulation show that the muscle patternsof [71] are applicable to a di erent skull morphology, as they produced avery similar chewing pattern, suggesting that they are not overly sensitiveto skeletal or muscle geometries. The results also show that the addition ofpassive tongue tissue has a signi cant e ect on free jaw movements and the653.5. Discussionchewing movement, due to both the passive elastic connection between thejaw and hyoid made by the tongue, especially in compression, e.g. reducingjaw opening, as well as the additional mass of the tongue body, particularlyin jaw protrusion.Biomechanical simulations of the type reported here are challenging tocreate. However, the interactivity a orded by computational performancein ArtiSynth allows for reasonably fast re nement and exploration of themodel’s capability. The jaw-tongue-hyoid model described above has twofree rigid bodies and 946 FEM nodes for a total of 2505 degrees of freedom.With respect to (Equation C.6), the addition of point-based tongue attach-ments, incompressibility, and jaw joints result in an ^M that is 2505 2505and a G that is typically 2505 740 (varying somewhat depending on thenumber of FEM contacts). In addition, a few unilateral constraints areused to implement bite contact. Solution times for (Equation C.6) using themethod described in Section C.1.2 vary from around 130 ms to 200 ms (de-pending on whether unilateral constraints are in play) on a 2.6 GHz Core2 Duo processor. Overall solution time (including collision detection andall the steps of Section C.1.5) for a 600 ms jaw opening task with a timestep size of 5 ms is around 40 seconds and a 400 ms tongue retraction taskwith a time step of 10 ms is around 20 seconds. Much of this involves Javacode that could be signi cantly optimized. This improves on the computa-tion time reported in [29], where a 100 ms task for the same FEM tonguemodel (with jaw/palate contact) required 40 minutes of computing time ona similar computer using ANSYS.Complex movements require precise coordination among a large numberof muscle input degrees-of-freedom making trial-and-error tuning of muscleinputs to generate simulations tedious and likely over- tted to a particularmodel’s geometry. We believe that optimization-based inverse dynamicsapproaches are a promising direction in this regard, which we discuss furtherin the following chapter.663.6. Directions3.6 DirectionsOur preliminary results also point to a few promising directions. We planto investigate what changes in muscle patterns are required to improve thechewing stroke in the jaw-tongue-hyoid model, e.g. increasing jaw openermuscle activation in order to attain a larger jaw opening during the simulatedchewing cycle. It is noteworthy that the original muscle patterns reportedby Hannam et al. were reported as being low in amplitude, therefore muscleactivation amplitudes could be increased and remain plausible. We also planto investigate activating the tongue muscles in concert with the jaw musclesto simulate tongue movements [78] and palate contact pressure patterns [138]during the chewing cycle, which would add a dynamically changing inertiaas opposed to the current passive tongue mass.The tongue tissue is currently modeled as isotropic with point-to-pointactuators embedded within the material to model anisotropic muscle  berforces. In the current model we increase the sti ness of elements associatedwith muscle activation as an approximation of skeletal muscle sti ening.However, we are currently incorporating a transverse-isotropic material [212]into ArtiSynth, which will provide a more realistic representation of skele-tal muscle mechanics. The tongue muscles are particularly challenging tomodel because multiple muscle groups, with di erent principal  ber direc-tions, converge and interdigitate within the tongue body. For this reasonwe are investigating a formulation allowing for the superposition of multipletransverse-isotropic materials with di erent principal directions.3.7 SummaryTo summarize, we have created a 3D model of coupled jaw-tongue-hyoidbiomechanics to advance the state-of-art in orofacial modeling. Previouslyproposed models focused on isolated subcomponents, neglecting the couplingforces from surrounding and adjacent anatomical structures. Our contribu-tion, by creating an integrated model, is the ability to assess the assumptionin previously proposed isolated jaw and tongue models that coupling e ects673.7. Summaryare small.The model was built by adapting and dynamically coupling a referencemulti-body jaw/hyoid model and a reference FEM tongue model so thatthe muscle forces of the tongue imparted forces on the jaw/hyoid and vice-versa. With our coupled model we were able to evaluate the signi canceof mechanical coupling. Simulations of isolated muscle activations showedthat the presence of passive tongue tissue reduced jaw movement and activetongue muscle forces induced jaw movement. Simulations also demonstratedthat the model behaved within a plausible range of motion for chewing andspeech production. As a step toward validation of the model, we comparedsimulated tongue velocity and tongue-palate pressure to recorded humanmeasurements and found consistent values for each. Our jaw-tongue-hyoidmodeling e orts also served to verify that our underlying simulation plat-form, ArtiSynth, is su ciently accurate and robust for the challenges oforofacial modeling, namely, large tissue deformations, large muscle forces,and hard/soft tissue coupling and contact. We are currently expandingthe jaw-tongue-hyoid model to include deformable face, soft-palate, and lar-ynx models and have created preliminary simulations of integrated face-jaw-tongue-hyoid movements.In order to simulate complex movements we need to specify coordinated,time-varying muscle activations as input to the forward dynamics model. Inthe next chapter, we describe methods for automatically generating muscleactivations to realize speci c target outputs.68Chapter 4Inverse Simulation MethodsIn the previous chapter, we described a model of jaw-tongue-hyoid dynamicsand its evaluation with motor tasks generated by simple piece-wise linearmuscle activation inputs. A signi cant challenge in the application of biome-chanical models to the analysis of motor tasks is the speci cation of muscleinputs. In this chapter, we describe inverse methods for automatically pre-dicting complex patterns of muscle activation input required to drive a modelthrough a prescribed trajectory of movement and/or forces.Predicting muscle activity from kinematic information is an inverse prob-lem because anatomical systems have more muscles than kinematic DOF andare therefore redundant. For example, the human mandibular system hasthirty muscle groups (some of which have sub-regions that are independentlyactivated) and only six kinematic DOF. Motor redundancy also exists in thetongue, though it is less obvious than in articulated skeletal systems suchas the jaw or limbs. The tongue is a deformable soft-tissue structure andtherefore has many kinematic DOF, however kinematic analysis has shownthat the tongue changes shape within a low dimensional motion space. Forexample, in speech movement 3D tongue shape can be characterized by asmall number of DOF. The tongue has ten distinct muscle groups, some ofwhich are likely di erentially activated, and therefore can be considered re-dundant within the reduced-space of kinematics observed in physiologicallyrelevant tasks.Another challenge with specifying muscle activity as input to forwarddynamics models is that muscle activity is di cult to measure. EMG in-volves transducing electrical signals associated with muscle activation; how-ever EMG can be di cult to record for small, deep muscles in the head andneck, and the relationship between EMG and muscle force is complex for dy-69Chapter 4. Inverse Simulation Methodsnamic movements [173]. One approach for validating biomechanical modelsis to apply recorded EMG patterns as input and compare simulated kine-matics and forces with measurements. However, due to the challenges withEMG recording and the inability to record from all muscle simultaneously,it is desirable to do the opposite: use kinematic and force measurements asinput to the model and compare predicted muscle forces to recorded EMG.As described in Section 2.4, recently reported trajectory-tracking tech-niques for inverse-dynamics simulation have shown promising results withlimb models [196], complex muscle and tendon models [189], and quasi-staticFEM models [179]. These techniques use per-timestep static optimization,as opposed to optimizing over the full time-varying trajectory, but they doincorporate model mechanics and are computationally e cient.In this chapter, we extend the previously proposed trajectory-trackinginverse approach for use with dynamic FEM models, such as our tonguemodel, as well as with coupled rigid-deformable models, such as our jaw-tongue-hyoid model described in the previous chapter. We also formulatenew target parameters, in addition to kinematic targets, that can be used asinput to the inverse simulation in order to select desirable muscle activationpatterns to overcome motor redundancy. The new target parameters includeconstraint force magnitudes and sti ness. Constraint force targets can beused to generate desired reaction forces in the system, such as a target biteforce during jaw clenching simulations. The sti ness target can be used tocontrol co-activation of antagonist muscles. These new target parametersallow for multiple modalities of target data, such as both motion and contactforce recordings, and permit systematic analysis of potential muscle activa-tion patterns. Section 4.1 describes the mathematical formulation of theinverse solver implemented in ArtiSynth. Section 4.2 demonstrates prop-erties of the inverse toolset using simpli ed canonical models. Section 4.3reports results of the toolset to more complex jaw and tongue models. Fi-nally, Section 4.4 and Section 4.5 discuss the implications of inverse modelingtechniques and directions for further investigation.704.1. Inverse Solver Formulation4.1 Inverse Solver FormulationThe inverse-dynamics tracking algorithm was implemented within the Ar-tiSynth biomechanics simulation toolkit, which is described in more detailin Appendix C. ArtiSynth is designed to simulate the dynamics of hard andsoft tissue structures using coupled rigid-body and FEM models. WithinArtiSynth, a mechanical system consists of an assembly of rigid bodies andparticles (which include FEM nodes). Let q, u, and f denote the compositeposition, velocity and force vectors for these components. Following fromprevious work [179, 189] we let f be partitioned into f = fp+fa, where fp arethe passive forces arising from muscle stretch, ligaments, and scar tissue,and fa are the active forces arising from muscle activation. The system’sdynamic behavior is then determined by Newton’s second law,M _u = f(q;u;t) = fp(q;u) + fa(q;u;a(t)) (4.1)where M is a block diagonal mass matrix.We use Hill-type muscle models that are linear in activation and non-linearly dependent on the length and shortening velocity, so thatfa =  (q;u)a (4.2)where a is a vector of activation levels bounded between 0 100% for eachmuscle. The matrix  relates muscle activations to system forces and canbe determined either analytically or numerically; we currently use an ana-lytic formulation. We neglect the calcium-dependent activation dynamics ofmuscle tissue, which is typically modeled as a  rst-order low-pass  lter [216].The consequence of this assumption is that predicted muscle forces couldchange faster than physiologically possible, however this was not found tooccur for target movements with physiologically plausible velocities. The hu-man tracking measurements place constraints on velocities that get re ectedback into physiologically realistic muscle activation values. Situations couldarise where muscle activations rapidly switch between agonist muscles, asmentioned by [189] who found that a damping regularizer was needed to714.1. Inverse Solver Formulationreduce such oscillations. The inclusion of muscle activation dynamics in themodel would also limit such oscillations in predicted muscle activations.The mechanical system may also contain bilateral and unilateral con-straints; the former include articulating joints between rigid bodies andFEM incompressibility, while the latter include contact conditions and jointlimits. Unilateral constraints are not considered here, but we do utilize bi-lateral constraints, which take the form of linear equality constraints on thevelocity:G(q)u = 0: (4.3)Di erentiating this leads also to acceleration constraintsG(q) _u = g; g  _Gu: (4.4)For example, to constrain one point of a rigid-body to a planar surface aconstraint is formed to prevent translation of the point normal to the plane,i.e. G = (nx;ny;nz;0;0;0), where (nx;ny;nz) is the normal vector of theplane.Constraints are enforced by forces applied to GT, so that Equation 4.1becomesM _u = fp(q;u) +  (q;u)a + GT(q) (4.5)where  are Lagrange multipliers giving the magnitudes of the constraintreaction forces.Solving the system dynamics involves integrating Equation 4.5 forwardin time. At present, this is done using a  rst order integrator that is semi-implicit with respect to the passive forces fp (which are often sti ). Lettingh equal the time step, and using a superscript to denote values at step i,this leads toMuk+1 = Muk +hfk+1p (q;u) +h k(q;u)a + GkT(q) (4.6)724.1. Inverse Solver Formulationwhere  now denotes constraint impulses. fk+1p is approximated usingfk+1p  fkp + @fp@q  q + @fp@u  u = fkp + @fp@qhuk+1 + @fp@u (uk+1 uk):Combining this with Equation 4.6 and Equation 4.4 leads to the system ^M  GTG 0! uk+1 != Muk +h^fp +h ag!(4.7)where^M  M h@fp@u  h2@fp@q ; ^fp fkp  @fp@u uk (4.8)are the mass matrix and force term augmented with Jacobian terms requiredfor the implicit solve. Unlike M, ^M is neither block diagonal nor symmetricpositive de nite, but it is sparse and symmetric.Solving for  in Equation 4.7 we  nd: = (G ^M 1GT) 1g Q (k +h a) (4.9)where Q  (G ^M 1GT) 1G ^M 1 and k  Muk + h^fp. Back substitutingfor  and solving for uk+1 in Equation 4.7 yieldsuk+1 = QTg + ^M 1Pf (k +h a) (4.10)where Pf  (I GTQ) is a matrix that projects forces into the rangecompatible with the constraints G, and QT = ^M 1GT(G ^M 1GT) 1 (bythe symmetry of ^M). Equations Equation 4.9 and Equation 4.10 relatemuscle activations to future constraint forces and velocities and can be usedto formulate an optimization over muscle activations with a cost functionthat includes desired movement and constraint force goals.Movement goalMovement is the traditional goal for inverse dynamics simulation and mostrecently has been used in rigid-body [189] and quasi-static FEM [179] mod-734.1. Inverse Solver Formulationels. Our contribution includes extending this formulation to dynamic FEMmodels. The movement goal of the algorithm is given as a target veloc-ity trajectory u , and we desire to  nd muscle activations that minimize12ku  uk+1k2. Substituting for uk+1 from Equation 4.10 the optimizationterm for the movement target can be expressed as a quadratic form in a: m(a) = 12k u Hmak2 (4.11)where  u  u  QTg ^M 1Pfk and Hm  h ^M 1Pf . For a rigid bodythe target movement can be speci ed as either: the full 6D position andorientation of the body, the 3D position of a single point on the body (inwhich case the body’s motion is partly unconstrained), or the 3D positionof multiple points on the body (in which case a best least-squares  t tothe points is used), as discussed below in Section 4.2.2. Our contributionextends the formulation to work with FEM models, as discussed below inSection 4.2.3. Target velocities can be computed from the target positiontrajectory at each timestep providing online correction to position errors.Constraint force goalOur contribution also includes extending the inverse dynamics formulationwith new goals: constraint forces and sti nesses. The constraint force goal isgiven as target values for Lagrange multipliers,  , which are the magnitudesof the constraint reaction impulses. Given target constraint forces  , thecorresponding impulses are h , and so we desire to  nd muscle activationsthat minimize 12kh   k2, which leads to a second term in the optimizationcost function which is also a quadratic form in a: c(a) = 12k   Hcak2 (4.12)where    h  (G ^M 1GT) 1g + Qk and Hc   hQ . We can selec-tively include a subset of the constraints into the optimization term. Theconstraint-force goal is used in Section 5.3 to simulate clenching with thejaw model.744.1. Inverse Solver FormulationSti ness goalThe sti ness goal is given as a desired sti ness in each DOF. It is formulatedby extending a model of single joint sti ness control proposed by Hogan [81](see Section 2.4 for more details), which shows that torque and sti nessabout a joint can be independently controlled by the di erence and sum ofmuscle activations. System forces arising from muscle activation is given byfa =  (q;u)a (Equation 4.2), whereas sti ness due to co-activation is givenby:Ka = Hka (4.13)where Hk abs( (q;u)), the element-wise absolute value of the  matrix.Given the target sti ness values, K , we desire to  nd muscle activationsthat minimize 12kK  Kak2, which leads to a third term in the optimizationcost function: K(a) = 12kK  Hkak2 (4.14)The control of muscle co-activation using the sti ness goal is demonstratedbelow in Section 4.2.1.It is important to note that intrinsic muscle sti ness increases with mus-cle activation and may be more dominant than co-activation induced sti -ness in certain biological systems or motor situations. The force-positionjacobian, @f=@q, which we calculate for implicit-integration, is a localizedsystem sti ness matrix and could be exploited to control sti ness arisingboth from individual muscle activation and co-activation of multiple mus-cles. Also, stretch re exes are currently not included in the biomechanicalmodels we have considered, and re ex feedback gains are used in biologicalsystems to increase e ective sti ness. Including intrinsic muscle sti ness andre ex feedback gains in our formulation would result in less co-activation toachieve the same sti ness goal.754.1. Inverse Solver FormulationPer-timestep static optimizationGiven the above target terms (motion, constraint forces, and/or sti ness)the inverse-dynamics algorithm solves a static optimization problem at eachintegration timestep of the forward dynamics simulation in order to trackthe target trajectory. Static optimization allows for more e cient compu-tation because the system dynamics are linearized at each timestep. Asdiscussed in Section 2.4, dynamic optimization or optimal control formula-tions provide an optimization over the full time-varying trajectory. Whilethe per-timestep static optimization may lead to sub-optimal muscle ac-tivations as compared to an optimal control formulation, the technique issigni cantly more computationally e cient. This is particularly importantfor coupled rigid-deformable FEM models that have non-trivial forward dy-namics solutions (see Section C.1.2 for a discussion of expected complexity).The per-timestep optimization problem is underdetermined for a biome-chanical system with redundant muscle activations. We include a weighted‘2-norm regularization term, 12aTW 1a, where W is a diagonal matrix ofmuscle CSA, in order to select the most e cient set of activations [4]. Otherregularization terms may be used within our formulation, such as an ‘1-normterm, kak1, to select the smallest set of non-zero activations as a sparsityconstraint or a damping term, 12ka ak 1k2, to enforce smooth activations.Combining the movement and constraint force goals, regularization, andmuscle activations bounds, we arrive at the complete optimization problem,which takes the form of a quadratic program:mina wm m(a) +wc c(a) +wk K(a) + wa2 aTW 1asubject to 0 a 1 (4.15)where wm, wc, wk, and wa are weights are used to trade-o between costterms. For all simulations reported in this chapter the weights were setto wm = 1, wc = 1, and wa = 0:1 so that minimizing tracking error waspreferred over small activations.The inverse dynamics optimization is solved at each timestep to pro-764.2. Analysis with Canonical Modelsvide muscle activations to the forward dynamics simulation. Biomechanicsmodels, especially those that include FEM models, can have many dynamiccomponents resulting in a large, sparse KKT system in Equation 4.7 (seeSection C.1 for more details). We use the KKT system solver in ArtiSynthto compute  u and   as well as Hm and Hc, which are formed by solvingthe system for each column of  . The resulting quadratic program is densebut tends to be small since its dimension is the size of a, i.e. the number ofactivations being solved for. The quadratic program is also convex, whichmeans it can be solved as a linear complementarity problem, which is doneusing an implementation of the Cottle-Dantzig algorithm ([38]) containedin ArtiSynth.4.2 Analysis with Canonical ModelsHere we evaluate and analyze the inverse-dynamics trajectory tracking algo-rithm proposed in the previous section with a number of canonical models,including a point, rigid-body, and deformable-body models. These simula-tions verify the correctness of the implementation and are used to illustrateproperties of the algorithm, which are more readily apparent in these sim-pli ed cases. The canonical models serve as a basis for the application of theinverse toolset to more complex anatomical models described in Section 4.3.4.2.1 Point inverseThe canonical point model has 2-DOF and includes 16 muscle actuators,arranged radially in the x-y plane, as pictured in Figure 4.1. The cardinaldirection muscles (N, S, E, W) have two times larger CSA than the ordinaldirection muscles (NE, SE, SW, NW). The kinematic target was speci edas a smooth displacement in the north-east direction from the center.Muscle redundancy The point model serves as an illustrative exampleof motor redundancy and is used to demonstrate the e ect of di erent regu-larization terms and sti ness goals on the output muscle activations. Muscle774.2. Analysis with Canonical Modelst = 0s t = 0:25s t = 0:5sFigure 4.1: The canonical 2-DOF point model with 16 muscles shown as redlines. Line thickness denotes muscle CSA. Cardinal direction muscles (N,S, E, W) have two times larger CSA than ordinal direction muscles (NE,SE, SW, NW). The panels picture the inverse simulation of a north-eastdisplacement with the target point shown as a cyan sphere.‘2-Norm CSA-‘2-Norm ‘1-Norm01N01NNE01NEActivation (%)01ENE101ETime (s)01N01NNE01NEActivation (%)01ENE101ETime (s)01N01NNE01NEActivation (%)01ENE101ETime (s)Figure 4.2: Muscle activations predicted for the point model simulation withdi erent regularization terms, including an unweighted ‘2-norm (12aTa), amuscle CSA-weighted ‘2-norm (12aTW 1a), and an ‘1-norm (kak1), wherea the muscle activations vector and W is a diagonal matrix of muscle CSA.784.2. Analysis with Canonical ModelsLow Stiffness High Stiffness01N01NE01E01SE01SActivation (%)01SW01W101NWTime (s)01N01NE01E01SE01SActivation (%)01SW01W101NWTime (s)0 5 1000.10.20.30.40.5Force Disturbance MagnitudeAverage Tracking ErrorK*=0K*=5K*=10K*=20(a) (b) (c)Figure 4.3: Muscle activations predicted for the point model simulation withdi erent sti ness goals, including minimum sti ness (a) and high sti ness(b). Increasing desired sti ness reduces tracking error for a random forcedisturbance applied to the point (c).activations predicted to drive the point model through the north-east dis-placement trajectory are plotted in Figure 4.2 for di erent conditions. In allcases the kinematic trajectory was exactly reproduced. The cardinal direc-tion muscles were activated preferentially with CSA-weighted regularization(Figure 4.2b) as compared to unweighted regularization (Figure 4.2a).We also evaluated di erent sti ness goals. A minimum sti ness goalresulted in zero co-activation (Figure 4.3a), whereas a high sti ness goalproduced co-activations that change dynamically throughout the task (Fig-ure 4.3b). Figure 4.3c illustrates that sti ness can be used to regulate forcedisturbances. We performed a series of simulations with a random force dis-turbance applied to the point with increasing magnitude. The disturbancesimulations were performed for di erent sti ness goals and we found that in-creased sti ness reduced the tracking error induced by the force disturbance,as shown in Figure 4.3c.794.2. Analysis with Canonical ModelsUnconstrainedPoint-to-Plane Constraintt = 0s t = 0:25s t = 0:5sFigure 4.4: The canonical 6-DOF rigid block model with 36 muscles shown asred lines. The panels picture inverse simulation of an oblique displacementtarget (shown as the cyan wire-frame block) for an unconstrained model(upper panels) and a model with the lower back corner (magenta point)constrained to a planar surface (lower panels).804.2. Analysis with Canonical Models4.2.2 Rigid-body inverseThe canonical rigid body model is a block with 6-DOF and includes threesets of orthogonally arranged muscles (36 muscles total) for full controllabil-ity of position and orientation. The rigid body model is used to demonstratetrajectory tracking under dynamic constraints.Constrained dynamics The inverse algorithm is formulated to track akinematic trajectory in the presence of constraints on the model’s dynamics.Figure 4.4 illustrates an inverse simulation of translation with the canonicalrigid-body model. The upper panels show the block following a 6-DOFtranslation trajectory with no rotation, as shown by the cyan wireframemesh. The lower panels show the same simulation, but for a model with apoint-to-plane constraint on the back-bottom corner of the block. In thiscase, the simulation computes muscle activations to make the block followthe trajectory as best as possible while maintaining the dynamic constraint,and the block slides along the plane and rotates to minimize the distanceto the target position. We can also simulate a desired constraint force thatcauses increased muscle activation in order to apply force against the point-to-plane constraint during the movement trajectory.4.2.3 Deformable-body inverseThe canonical deformable body model is a FEM beam with orthogonallyarranged muscle  bers throughout the FEM mesh. The mesh has 3x3x7hexahedral elements, 128 nodes (384 DOF), and 276 muscle  bers. Nodeson the back end of the beam are  xed in space. The deformable body modelis used to illustrate the e ect of muscle grouping and muscular-hydrostattype motor recruitment patterns as proposed by [96].Muscle groupingMuscle  bers within a model can be grouped based a priori knowledge aboutthe spatial extent of muscle  bers for particular muscle groups. In the beammodel we compare the e ect of di erent muscle grouping in an elongation814.2. Analysis with Canonical ModelsTwo Anteroposterior DivisionsAPSeven Anteroposterior DivisionsAPMMPMMAAMPt = 0s t = 0:25s t = 0:5sFigure 4.5: The canonical 384-DOF deformable body model and inversesimulation of protrusion for a model with two (upper panels) and seven(lower panels) anterior-posterior muscle group divisions.01A101PActivation (%)Time (s)01A01AM01MA01MActivation (%)01MP01PM101PTime (s)(a) (b)Figure 4.6: Muscle activations predicted to protrude the beam’s tip for amodel with two (a) and seven (b) anterior-posterior muscle group divisions.Transverse muscles are activated to compress the posterior section.824.2. Analysis with Canonical ModelsFigure 4.7: Inverse simulation with the deformable beam model trackinga complex movement trajectory involving combined bending, lengthening,and shortening.motor task. The kinematic target is speci ed as protrusion of the nodes atthe beam’s tip, which is equivalent to elongating the beam while also notcompressing the tip laterally or vertically. Figure 4.5 pictures the results ofelongation simulation for a model with two anterior-posterior muscle groups(upper panels) and a model with seven anterior-posterior muscle groups(lower panels). The model with more muscle groups deforms more smoothlyduring elongation.Muscular-hydrostat motor recruitmentThe muscular-hydrostat theory proposes that movement in non-skeletal mus-cle tissue systems, such tentacles, trunks, and tongues, is generated throughthe combined e ect of tissue incompressibility and the spatial arrangementof muscle  bers [96]. For example, elongation is generated by activation oftransversely or helically arranged muscle  bers, while bending is generatedby unilateral activation of longitudinal muscle  bers. The theory is sup-834.3. Analysis with Anatomical Modelsported by morphological evidence, such as the fact that longitudinal  bersin many tentacles are located along the outer surface of the tentacle in orderto provide better leverage for bending [96].Our simulations of beam elongation support the muscular-hydrostat the-ory as the inverse solution recruits transversely-oriented horizontal and ver-tical muscle groups. Further, we are able to generate combined bending,shortening, and lengthening deformations by specifying a more complexkinematic trajectory for the nodes at the beam’s tip, such as is shown in Fig-ure 4.7. Bending movements were generated by the recruitment longitudinalmuscle  bers along the side of the beam toward which bending occurred.4.3 Analysis with Anatomical ModelsThe canonical models serve to demonstrate speci c aspects of the inversesolver implementation within a simpli ed modeling context. In this sec-tion we discuss the application of inverse modeling tools to more complexanatomical models.4.3.1 Jaw inverseInverse simulations with the jaw model predict muscle forces needed to movethe jaw in the presence of dynamic constraints at the TMJ. Figure 4.8 pic-tures the results of the inverse simulation of lateral incisor movement. Lat-eral incisor movement is achieved primarily through jaw rotation as straightlateral jaw translation is limited both by ligaments and the shape of theTMJ. We compared two alternative kinematic targets, as shown in Fig-ure 4.8: a full 6-DOF kinematic trajectory (upper panels) and a 3D positiontrajectory for the lower incisor point (lower panels). The 3D incisor pointtarget provides an under-constrained kinematic target for the jaw, and re-quired an additional lateral TMJ constraint, representing the lateral walland capsular ligament of the TMJ, in order to induce realistic jaw rota-tion (as opposed to implausible straight lateral translation of the jaw body).Both simulations were found to track the target kinematics with small error.844.3. Analysis with Anatomical Models6-DOF Jaw Target3-DOF Incisor Point Targett = 0s t = 0:25s t = 0:5sFigure 4.8: The jaw model during inverse simulation of a right lateral move-ment with a 6-DOF target shown as a cyan wireframe mesh (upper panels)and a 3D incisor point target shown as a cyan point (lower panels). The3D point target simulation required an additional lateral constraint on theright TMJ to attain rotation from the under-constrained target kinematics.The predicted muscle activation for each condition are plotted in Figure 4.9and are consistent with the jaw physiology literature that the left-side lateralpterygoid muscle is used in right lateral jaw movement [117]. Interestingly,the 6-DOF-target simulation exhibited larger muscle forces, suggesting in-creased jaw sti ness was required to maintain the speci ed jaw orientationduring the movement, whereas the 3D-point-target simulation required lessmuscle activity to follow the incisor point trajectory.Application to jaw model validationAs stated in the introduction of this chapter, one application of inversemodeling tools is model validation, whereby predicted model muscle acti-vation patterns are compared to recorded EMG for a particular kinematictrajectory. To overcome motor redundancy in the model, the inverse toolset854.3. Analysis with Anatomical Models6-DOF Jaw 3-DOF IncisorTarget Target08LIP08LPMH08RDMActivation (%)08LAMH08RSM108RATTime (s)02LIP02LPMH02RDMActivation (%)02LAMH02RSM102RATTime (s)Figure 4.9: Muscle activations predicted for lateral jaw movement with 6-DOF kinematic target and 3D incisor point kinematic target. Recruitedmuscle groups included the inferior head of the left lateral pterygoid (LIP),the left mylohyoid (LPMH, LAMH), right masseter (RDM, RSM), and rightanterior temporalis (RAT).864.3. Analysis with Anatomical Modelsprovides a number of tunable parameters that choose one particular setof \optimal" muscle activations for a particular movement. However, the\optimality" condition employed by the central nervous system to generatemuscle activity is unknown, making comparisons of model predictions torecorded data problematic. Here we propose the application of biomechan-ics modeling to inform the motor recruitment strategies in the context oflateral jaw movement.Lateral jaw movement has been widely analyzed and is known to beprimarily driven by contra-lateral (side opposite of the movement) lateralpterygoid muscles [117]. The upper and lower heads of the lateral pterygoidmuscle have straight muscle  bers with broad insertion sites on the skull (seeAppendix B for detailed description) and therefore have di erent  ber anglesand lengths that change over the course of a lateral jaw movement. Therelative activation of di erent regions of the muscle during lateral movementhas been characterized in studies using  ne wire EMG [85, 129] with wirelocations within the muscle veri ed through CT imaging of the subject post-recording before the wires were removed [139]. One hypothesis is that thedi erent timing of muscle activation in di erent regions of the muscle isrelated to changing biomechanical properties of the muscle  bers throughoutthe movement, such as the mechanical advantage. Mechanical advantagehas been proposed as a means to explain muscle recruitment in both theintercostal muscles [61] and the dorsal interosseous muscles [87]. The inversesimulation tools along with the dynamic jaw model provide a means toperform this analysis.Simulated lateral jaw movement with a biomechanical model that in-cludes detailed muscle  ber information for the lateral pterygoid musclescan make predictions of biomechanical correlates, such as mechanical ad-vantage, during the movement. Mechanical advantage could be formulatedwith respect to a muscle  ber’s spatial location, e.g. its moment arm forgenerated the desired torque, or with respect to instantaneous length andshortening velocity which a ect its instantaneous force generation capacity(see Section 2.3.3). In a similar way as we have de ned a desired sti -ness goal in the inverse algorithm, one could formulate a optimality term874.3. Analysis with Anatomical Modelsto maximize mechanical advantage. If the hypothesis of maximizing me-chanical advantage is valid, then inverse simulation of lateral jaw movementshould predict similar di erential muscle activations in regions of the lateralpterygoid muscle as was observed in the EMG recordings.In addition to the redundancy problem, relating predicted muscle activa-tions to EMG recordings for free jaw movements are challenging because themagnitude of muscle activation in free jaw movements is low, thus providinga low signal-to-noise ratio in recorded EMG signals. We have proposed touse jaw movements under externally applied resistive force in order to elicitlarger EMG signal amplitude. Given simultaneously recorded jaw move-ment and external force magnitude and direction, we can use the inversesimulation to predict muscle activity in the model and compare with EMGrecordings. We have performed pilot jaw movement and EMG studies witha 1D force sensor to record the subject-applied force resisting the movementdirection. We found that the 1D force sensor is insu cient to accuratelydetermine directed applied force and we are currently working to re ne theprotocol with a 3D force sensor. A further extension would be to apply acontrolled force to the jaw from a robotic arm connected to the lower jawthrough a custom- t dental appliance, in a manner similar to that whichhas been done to measure jaw sti ness experimentally [176], but at higherforce magnitudes.4.3.2 Tongue inverseSimilar to the deformable beam model reported in Section 4.2.3, the tonguemodel includes a larger number of muscle  bers distributed throughout anFEM mesh (see Section 3.1.2 for a detailed description). Controlling themovement and shape deformation of the tongue requires complex patternsof muscle activation and is thus a good candidate for the application ofinverse simulation. As an example, we simulated upward and backwardmovement on the tongue tip, which is an important tongue posture usedin the production of an English /r/ sound. We simulated tongue elevationby specifying an arcing upward and backward target trajectory for a set884.3. Analysis with Anatomical Modelst = 0s t = 0:25s t = 0:5s t = 0:75s t = 1:0sFigure 4.10: The tongue model during inverse simulation for elevating thetongue tip. Target nodes are shown with semi-transparent cyan points.of forty nodes at the tongue tip. The tongue tip elevation movement isshown in Figure 4.10 and the predicted muscle activations are plotted inFigure 4.11. The superior longitudinal muscle is most active in generating abackward tongue bending, which is consistent with the muscular-hydrostattheory of motor recruitment. In addition, the transverse, mylohyoid, andgeniohyoid muscles are activated to prevent lateral and downward expansionof the tongue body in order to maintaining tongue tip protrusion during thetongue tip elevation motion.Application to recorded tongue movementAs discussed in Section 2.1.1, Electromagnetic Articulometry (EMA) hasbeen used to record the 3D movement of a small number of points on thetongue surface. This type of data is a natural  t with the inverse trajectorytracking solver that uses the 3D position of FEM nodes as a target kine-matics trajectory. EMA data is limited in spatial resolution as only a smallnumber of markers can be tracked at one time. In speech recording studies,EMA markers are typically located along the mid-sagittal plane in order tobest characterize the mid-sagittal tongue contour since speech productionis predominantly a bilaterally symmetric motor task. The 3D tongue shape(and consequently the 3D vocal tract shape) is important to the acoustics ofvowel sounds and a small number of EMA marker point positions does notadequately describe the tongue’s volumetric extent. In the context of theinverse solver the kinematic information of a small number of surface nodesonly provides local movement information for the tongue. Global positioninformation requires the 3D shape of the tongue surface.894.4. Discussion010MSL010PSL010ASLActivation (%)010TRANS010MH1010GHTime (s)Figure 4.11: Muscle activations predicted for elevating the tongue tip. Re-cruited muscle groups include the middle, posterior, and anterior  bers ofthe super cial longitudinal muscle (MSL, PSL, ASL), transverse (TRANS),mylohyoid (MH), and geniohyoid (GH) muscles.We are investigating the use of tongue shape information as a means toprovide global kinematic targets into the inverse solver. 3D tongue shapeshave been segmented from MRI data of static vowel postures [14] for thesame subject on whom the tongue model’s morphology is based. Further,the internal deformation of the tongue may be important for determiningthe relative contribution of antagonist muscle groups. Tagged cine-MRIdata can be processed to track the 3D position of tagged points within thetongue tissue [142] and is a promising direction for attaining the necessarykinematic information for inverse tongue model simulations.4.4 DiscussionThe inverse-dynamics solver proposed in this chapter is a tool for sys-tematically exploring a biomechanical model’s behavior and for integratingrecorded kinematic and other data into biomechanics simulations. Accu-rately measuring all of the kinematic, force, and muscle variables simulta-904.4. Discussionneously for a particular motor task with a single subject is a di cult, ifnot impossible, task. The inverse-tracking toolset presented in this chapterallows data recorded for a subset of variables to be integrated with a biome-chanical model in order to predict other unmeasured variables. Importantly,inverse solutions predict those variables (muscle forces) that are challengingto record from kinematic variables that are much easier to record.While the inverse tracking method provides a natural  t for integrat-ing recorded data with biomechanical models, it is limited to the analysisof speci c motor tasks that are characterized by a full kinematic trajectory.The algorithm requires smooth kinematic trajectories that are di erentiatedfor the velocity trajectory used in Equation 4.11. Kinematic errors arisingfrom numerical di erentiation and misalignment of joint centers-of-rotationin limb models have been discussed in the literature [180]. Our method isless sensitive to these errors as the output of the forward-dynamics model isused in a feedback loop, though excessively noisy and/or discontinuous po-sition trajectory are still unacceptable for trajectory-tracking. Interpolationbetween sparse position measurements will also in uence the predicted mus-cle activations. For static position target inputs, a non-linear root- ndingoptimization, such as the method proposed by Sifakis et al. [179], may bemore appropriate and e cient.In addition, low-level trajectory speci cation is inappropriate for senso-rimotor physiology investigations that are concerned with  nding the emer-gent properties of the motor system for high-level motor tasks without pre-supposing the low-level coding (e.g. task-space kinematics versus joint-spacekinematics). Such investigations would require higher level sensorimotorcomponents, such as re exes, the spinal-cord, brainstem, and motor cortex,built \on top of" biomechanics models in order to specify high-level motortasks as input to inverse simulation. For example, one could specify a start-ing and ending point for a desired movement in a model without specifyingthe precise trajectory in between.We also currently neglect the calcium-dependent activation dynamics ofmuscle tissue, which is typically modeled as a  rst-order low-pass  lter [216].The consequence of this assumption is that predicted muscle forces could914.5. Directionschange faster than physiologically possible, however this was not found tooccur for target movements with physiologically plausible velocities. Alsopredicted muscle activations would likely lag recorded EMG signals due tothe activation dynamics of real muscle tissue. Thelen and Anderson [196]consider muscle activation dynamics within their Computed Muscle Controlframework and incorporating activation dynamics into our model one of ourplanned directions.Our current trajectory tracking algorithm does not include the unilateralconstraints in the forward dynamics that arise from joint limits and contact.The presence of inequality constraints in Equation 4.7 would result in a nonquadratic optimization problem for Equation 4.15, which would require morecomplex non linear programming methods. We have found that contactconstraints for deformable bodies can be treated as bilateral for the durationof an integration timestep (see Section C.1.4). This could lead to oscillatingor sticking behavior; however, this has not been found to be an issue inour simulations as the constraints arising from deformable contact tend tobe reasonably decoupled. Handling contacts in this fashion allows them tobe included in our inverse solutions, but the scheme is less appropriate forrigid-body contacts that result in more tightly coupled constraint situations.4.5 DirectionsOur short term directions for modeling applications of the inverse toolsetfocus on integrating recorded human data with models for the purposesof validation. The inverse simulation of lateral jaw movement presented inSection 4.3.1 lays the groundwork for a more comprehensive study comparingthe results to EMG recordings. This will require a more sophisticated modelof the lateral pterygoid muscle that includes muscle vectors for the sub-region  bers in both the upper and lower heads of the muscle. Mechanicaladvantage as a goal for motor recruitment would be examined in the contextof lateral pterygoid recruitment.We are also interested in investigating motor control strategies in oro-facial motor tasks, including speech production. A long-standing debate in924.6. Summarythe speech motor control literature concerns the validity of the EPH, whichposits that speech gestures are controlled as static target-to-target posi-tions [66]. EPH has been used in the reference tongue model [29]; however,the technical capability to test alternative control strategies that incorpo-rate dynamics information has been lacking. The inverse simulation toolsetdescribed in this chapter  lls this technological gap and can be applied toevaluate the signi cance of controlling dynamics in speech movements. Weplan to compare predicted muscle activations from inverse-dynamics versusEPH control for two data-sets from the same speaker: mid-sagittal EMArecordings of speech utterances and 3D tongue shapes segmented from mag-netic resonance images MRI. We expect that dynamics may be importantin vowel-vowel tongue transitions that have large movement amplitudes.In future studies, we plan to use the inverse modeling tools with thefull coupled jaw-tongue-hyoid model as described in the previous chapter.To date, the inverse methods have been applied to the jaw and tonguemodels in isolation in order to assess the results within a simpler context.In the previous chapter, we illustrated that coupling e ects can be signi -cant in combined jaw-tongue-hyoid movements, such as speech production,and therefore we expect that accurate prediction of jaw and tongue mus-cle activity using the inverse toolset will require the full model of coupledjaw-tongue-hyoid biomechanics. We expect inverse simulations with the jaw-tongue-hyoid will provide insight into the biomechanical underpinnings ofco-articulation e ects in speech production.4.6 SummaryComputational models of biomechanical systems have greatly increased in delity and complexity. A limiting factor in the application of such mod-els to particular applications in biomedicine and biological science remainsthe ability to generate movement simulations without trial-and-error speci- cation of input muscle activity. In this chapter, we developed an inverse-dynamics algorithm for automatically predicting muscle activity to drivebiomechanics models with combined skeletal and muscle-tissue structures.934.6. SummaryWe have extended previously reported trajectory tracking-based inversedynamics approaches to work within the ArtiSynth framework for muscle-activated rigid and deformable body models with dynamic constraints. Wehave also included additional target parameters that can be used to bettercontrol the selection of muscle activation given motor redundancy. Con-straint force magnitude targets can be controlled such as to generate biteforces at the teeth for the jaw model. Co-activation of antagonist muscles canalso be modi ed to control sti ness. In sum, these contributions help to cre-ate a more comprehensive inverse-dynamics toolset than has been previouslyproposed. Using the inverse toolset, we simulated motion and deformation ofmuscular-hydrostat models, including a beam model and a physiologically-based tongue model, illustrating muscle activations that are consistent withtheoretical proposals. Also, we simulated lateral jaw movement and foundmuscle activations consistent with published jaw physiology.In the next chapter, we provide a case study analyzing segmental jawresection and reconstruction using the forward dynamic jaw model describedin Chapter 3 and the inverse modeling toolset described in this chapter.94Chapter 5Segmental Jaw SurgeryModelsIn the previous two chapters we have described a new biomechanical modelof the jaw-tongue-hyoid complex and new simulation tools for making auto-matic predictions of muscle force patterns required to elicit speci c move-ment and force trajectories from the model. Our primary motivation forbuilding these modeling tools is in their application to orofacial biomedicine.In this chapter, we investigate one example biomedical application in detail,segmental jaw surgery, in order to demonstrate the utility of the modelingtoolset.Treatment of oral cancer commonly involves surgical resection of cancer-ous tissue, in addition to radiation therapy and chemotherapy. Dependingon the size and location of the lesion, tissue resection can involve the por-tions of the mandible (hemimandibulectomy), tongue (glossectomy),  oorof mouth, and associated muscles. Vascularized osteocutaneous, osteomy-ocutaneous and alloplastic grafts are commonly used to restore mandibularcontinuity after hemimandibulectomy [74, 77, 116, 153, 165]. Grafts providea functional joint on the a ected side [114, 152], though articular com-plications can include erosion of the temporal fossa, dental malocclusion,infection, and graft migration [33, 114, 143]. With or without jaw recon-struction, hemimandibulectomy signi cantly alters jaw biomechanics andde ciencies in mastication, speech and other orofacial functions are oftenobserved [2, 40, 73, 77]. Typically, the mandible deviates to the defectiveside on opening, and chewing is performed on the normal side [161, 166].Altered sensation, salivary  ow and biomechanics likely a ect biting, ma-955.1. Model Creationnipulating and comminuting food [40, 113, 166, 200] and are concerns inoral rehabilitation.Functional jaw recordings, including movement, bite force, and EMG,have been used to analyze the intact masticatory system (see Section 2.1and [117]), but are rarely performed after hemimandibulectomy. Anecdotalinformation and clinical observations suggest that jaw-opening is not undulyrestricted, and that biting is frequently accompanied by frontal jaw rota-tion. Contributing factors include an asymmetrical musculature, a unilat-eral articulation in mandibles without continuity, and post-operative tissue-scarring.In this chapter, we analyze hemimandibulectomy models with and with-out mandibular continuity. Section 5.1 describes the models’ creation. Sec-tion 5.2 describes forward dynamic simulations performed with the goalof replicating clinically observed post-operative de cits, including atypicalresting postures and abnormal movements during opening and clenching.Section 5.3 reports the results of inverse simulations used to determinewhat is physically possible for the ungrafted hemimandibulectomy modeland whether particular patterns of muscle use could be employed to com-pensate for given functional de ciencies.5.1 Model CreationThe hemimandibulectomy models, depicted in Figure 5.1, are based on theintact jaw model described in Section 3.1.1. Model NOCON (no condyle)simulated a left-side, composite jaw resection from the condyle to the leftcanine, without restoration of mandibular continuity. Model CON (graft-related condyle) simulated a left-side resection with continuity restored bymeans of an alloplastic graft similar to that described by Marx et al. [114].The inertial properties of the mandible fragment and graft were computedbased on its geometric shape resulting in a new mass of 126 g from 200 gfor the normal jaw and a new centre-of-mass shifted from midline in thenormal jaw to inside the body of the jaw fragment inferior to the secondmolar. The hyoid was  xed in both the models, which would be achieved965.1. Model CreationFigure 5.1: The two models used in the study. Model NOCON shows jawresection without mandibular continuity. Model CON shows continuity re-stored by alloplastic grafting. Both models have left side muscle loss.by activation of infrahyoid muscles to sti en the hyoid; hyoid movement inhemimandibulectomy patients has not been reported, but likely di ers fromnormal subjects due to scar tissue on the a ected side.In both models, left-side muscles were removed as part of the resection.The intact muscles are shown in Figure 5.2 and included the right ante-rior, middle, and posterior temporalis (RAT, RMT, RPT), right deep andsuper cial masseter (RDM RSM), right medial pterygoid (RMP), right su-perior and inferior lateral pterygoid (RSP, RIP), right mylohyoid (RMH)right and left geniohyoid (RGH, LGH), and right and left digastric (RDI,LDI) muscles. Hill-type muscle models simulated individual muscle CSAand length-tension properties and produced passive forces proportional tomuscle stretch and active forces proportional to muscle activation. MuscleCSA are listed in Table 3.1.When the jaw was in the maximal intercuspal position, the right condylesin both models, and the alloplastic condyle in CON, were centered in theirarticular fossae. The joints were modeled as a bilateral constraint surface,curvilinear in the anteroposterior direction to represent the articular fossa975.1. Model CreationFigure 5.2: Model NOCON showing intact muscle groups including the rightanterior, middle and posterior temporalis (RAT, RMT, RPT), right deepand super cial masseter (RDM RSM), right medial pterygoid (RMP), rightsuperior and inferior lateral pterygoid (RSP, RIP), right mylohyoid (RMH),right and left geniohyoid (RGH, LGH), and right and left digastric (RDI,LDI) muscles. The e ect of post-operative scarring is represented by theSPRANT spring element.985.2. Forward Dynamics: De cit Simulationsand eminence, with no mediolateral constraint. A posterior constraint wasincluded to represent the posterior aspect of the articular fossa. In all sim-ulations the joint worked in compression, i.e. the forces in the system neverworked to pull the joint apart, therefore it was modeled with bilateral con-straints. For clenching simulations we added an additional planar constraintat the right  rst molar to simulate tooth contact.Passive soft-tissue forces representing tissue scarring were modeled withlinear, damped springs. These had sti nesses of 200 N/m and damping-constants of 10 Ns/m to restrict jaw motion without eliminating it. Thesprings permitted incisal separations of at least 15 mm, and counteracted thetendency of the jaw to position itself to the right as a result of passive muscleforces on that side. In NOCON, a spring attached to the anterior portion ofthe jaw fragment (SPRANT) drew this end of the native mandible laterally,posteriorly and inferiorly from its initial position. In CON, a posteriorspring attached to the gonial region of the graft (SPRPOST) initially drewthe gonial angle of the graft inferiorly and posteriorly, and thereafter createdtensile forces proportional to gonial displacement in any direction. The scarsprings and spatial coordinate frame are shown in Figure 5.3. The x-y planewas oriented to the Frankfort horizontal plane. Each origin was in the bodyof the hyoid, with positive x-,y- and z-axes indicating left lateral, posteriorand superior respectively.5.2 Forward Dynamics: De cit Simulations 1Forward dynamic simulations were created with ArtiSynth (see Appendix C).The goal of the forward dynamic simulations was to evaluate if the modi- ed jaw model could replicate commonly observed de cits in jaw movementduring rest, muscle-driven opening, applied force opening, and unilateralclenching. Additionally, we aimed to compare di erences between the NO-CON and CON models.1A version of the section has been published in Hannam et al. [72]. Dr. Hannam wrotethe text for the manuscript, which I have adapted and included in this section.995.2. Forward Dynamics: De cit SimulationsFigure 5.3: Model conventions and restraints. Model illustrated is NOCONwith soft-tissue spring SPRANT. In CON, spring SPRPOST was attachedto graft’s gonial angle (not shown). For clarity, only left side articular con-straining surfaces are illustrated. Black spheres indicate incisor point, leftcondylar-center, and molar contact point locations in intercuspal position.1005.2. Forward Dynamics: De cit Simulations5.2.1 Simulation descriptionsSimulated external forces on the jaw, and muscle activation, either singlyor in groups, were chosen as a means to provide insight into the movementpatterns and jaw instabilities seen clinically. The relaxed rest position ofthe mandible without postural muscle activity (RRP) was assessed withsoft-tissue restraint 1 second after gravitational acceleration from the in-tercuspal position. External force applied downwards to the mandibularincisors perpendicular to the occlusal plane (FORCE) simulated manualjaw-opening from RRP with soft-tissue restraint. This force increased at10 N/second, and the simulation was terminated when the incisor-pointreached approximately 30 mm of inferior displacement. Jaw opening dueto muscle activation (OPEN) simulated voluntary jaw opening from RRPwith soft-tissue restraint. Here, RSP, RIP, RDI and LDI were driven simul-taneously, reaching 10% of maximum contraction in 0.5 seconds. Unilateralmolar contact on the una ected side on jaw closure from RRP (UNIMOL)was simulated by activating individual jaw-closing muscles in the presenceof soft tissue restraint. RAT, RMT, RPT, RDM, RSM and RMP were eachdriven independently to 10% of maximum contraction in 0.5 seconds, similarto the protocol used by Koolstra and Van Eijden [97].5.2.2 Simulation resultsRelaxed resting jaw postureThe displacement of the incisor-point at RRP is plotted in Figure 5.4 forboth models. In NOCON, SPRANT caused the jaw to rotate to the defectside, the incisor point displacing markedly left, posteriorly and inferiorly,and there was minimal displacement of the right condyle. In CON, thee ect with SPRPOST was less, the jaw remaining closer to the midline ata reduced incisal separation.1015.2. Forward Dynamics: De cit SimulationsForce-induced jaw-openingTrajectories of incisor-point displacement during FORCE are compared inFigure 5.4. In both models, inferior displacement of the incisor point closelyapproximated the opening target of 30 mm with forces less than 5 N (3.29N and 3.38 N for NOCON and CON). In NOCON, the incisor point ap-proached the midline as the jaw opened, ending 8.38 mm posterior to itsmaximal intercuspal position (not shown). The path in CON paralleled thatin NOCON, but began and ended on the right (una ected) side, indicatingfrontal clockwise jaw rotation.Muscle-induced jaw-openingIncisor point displacements during OPEN are compared in Figure 5.4. Inboth models, the incisor point moved to the left (a ected) side. In NOCON,the marked lateral movement of the incisor point to the a ected side partlyresulted from its initial position in RRP, where it was already displaced tothe left. Greater lateral deviation occurred in CON, where the incisor pointmoved initially to the right, then markedly to the left.Muscle-induced jaw-closingThe e ects of muscle activation in the UNIMOL simulation were pronouncedin the NOCON model and resulted in signi cant postural instability (largefrontal-plane rotations of the jaw fragment), whereas the e ects in the CONmodel were less striking and generally resulted in more stable behavior.Graphic examples of jaw rotation in NOCON are illustrated in Figure 5.5.In NOCON, the actions of RAT, RMT RPT and RDM were similar. Themost common e ect was movement of the incisor-point laterally to the leftand posteriorly, especially for RPT and RDM. This was associated withmarked lateral displacement of the right condyle. RPT caused signi cantsuperior movement of the incisor point, and excessive movement of theright condyle posteriorly and inferiorly along its posterior planar constraint.These marked translations and rotations are shown collectively in Figure 5.5.RSM activation caused excessive lateral movement of the incisor point to1025.2. Forward Dynamics: De cit Simulations-10 -5 0 5-30-25-20-15-10-50Lateral  Displacement  (mm)Vertical  Displacement  (mm)FORCE-10 -5 0 5 10-30-25-20-15-10-50Lateral  Displacement  (mm)Vertical  Displacement  (mm)OPEN 10RRPOPENNOCONIPCONLEFTRIGHTLEFTRIGHTFigure 5.4: Frontal view of mandibular incisor-point displacements dur-ing FORCE and OPEN. Movements began from jaw’s relaxed rest position(RRP) and are referenced to maximal intercuspal position IP (large crosses).1035.2. Forward Dynamics: De cit Simulationsthe left (de cient) side, as well as posteriorly and inferiorly. Here, therewas minimal displacement of the right condyle, indicating predominant 3-dimensional jaw rotation. RMP activation moved the incisor point exces-sively left, posteriorly and superiorly. Although there was no displacementof the condyle, RMP had a strong rotational action on the jaw, di erent tothat due to RSM activation (Figure 5.4).5.2.3 DiscussionDynamic modeling can be used to study jaw biomechanics by simulating thee ects of mandibular surgery and reconstruction. The approach is physics-based, and suitable for solving complex dynamic interactions amongst mul-tiple components. Anatomical structures can be readily modi ed, and tis-sue properties can be assigned provided their parameters are known. Pre-dictably, the RRPs occurred at a wider incisal separation than clinical postu-ral rest because low-grade postural muscle activity was not simulated [213].Soft tissue forces on the defect (operated) side, especially those due to scar-ring, would normally have a restraining e ect on this motion, so the RRPsobtained with SPRANT and SPRPOST seem in line with clinical impres-sions. Also, less incisal separation would be anticipated clinically due topostural muscle activity. Jaw deviation in RRP was less for CON than forNOCON, suggesting that scarring in a jaw with a bilateral articulation couldresult in a relatively normal RRP.The motions caused by FORCE re ected the di erent components inthe two models. Both easily reached their opening targets of 30 mm witha low applied forces of 3-4 N. In NOCON, SPRANT functioned as a simpletether, freely allowing incisal separation. In CON however, SPRPOST actedcloser to the jaw’s transverse axis of rotation and limited movement in anydirection. High sti ness values assigned here would be expected to restrictjaw motion, and it is signi cant that the sti ness of wounded porcine skin ishigher than the 200 N/m used in the present study [37]. A clinical protocolsimilar to the FORCE simulations (applying known forces to the mandibularincisor of a patient and tracking jaw motion) might be useful for estimating1045.2. Forward Dynamics: De cit SimulationsFigure 5.5: NOCON jaw postures caused by individual closing-muscle acti-vation in right molar contact. Simulations halted when jaw motion becameunrealistic at 1.36, 1.45, 1.43 and 1.19 s for RMT, RDM, RMP and RSMrespectively. Muscle forces (Fm) and torques (Tm) expressed at jaw centersof mass indicate directions only (scaled for clarity). + denotes incisor-pointlocation at intercuspal position. Grid spacing is 10 mm.1055.2. Forward Dynamics: De cit Simulationsscar sti ness in clinical situations.The di erence between FORCE and OPEN trajectories can be explainedby the primarily inferiorly-directed force in the former, and the primarilyoblique muscle forces in the latter. In OPEN, the bilateral articulation inCON reduced this lateral deviation, but did not eliminate it. Deviated jaw-opening is a common clinical observation associated with muscle loss, andhas functional implications with respect to mastication, since jaw-closingmust begin from the defect side, and the maximal intercuspal position ap-proached mediolaterally. In the present study, wider incisal separationscould have been reached with more muscle drive, and using additional mus-cles might have increased it further. More drive in the digastric and ge-niohyoid muscles, however, would have resulted in less lateral jaw deviationbecause these muscles have poor angles of attack, and their e ectivenessdiminishes as the jaw opens [99].Analysis of the biomechanical role of single muscles is a unique featureof computational modeling since living subjects are unable to activate jawmuscles individually. Clinically unrealistic movements resulting from single-muscle activation in both models were therefore not surprising, but werehelpful in revealing the actions of muscles likely contributing to mandibu-lar instability. The marked rotation caused by RSM after molar contactin both models, and by RMT in NOCON, substantially explained clinicalobservations of frontal plane rotation. The tendency of RAT, RMT, RPTand RDM to translate the jaw laterally in a mandible without continuitywould normally be resisted by the temporomandibular ligament [97]. Con-tinuous, or perhaps exclusive, use of such muscles may explain mandibularlateral displacement during occlusal contact sometimes observed clinicallyin mandibular resection patients without reconstruction.The jaw instabilities demonstrated in UNIMOL partly explained thechallenges for patients having to  nd new strategies of muscle contraction.For further investigation in the following section, we intend to determinethe extent to which combined muscle use might provide stability by usinginverse dynamics simulation.1065.3. Inverse Dynamics: Compensatory Simulations5.3 Inverse Dynamics: CompensatorySimulationsHaving simulated post-operative de cits in a hemimandibulectomy model,we proposed to use the model to predict possible compensatory muscle pat-terns that hypothetically could be employed by patients and reinforced bypost-operative rehabilitation. Without experimental data upon which tobase muscle input, however, forward-dynamic simulations require trial-and-error approaches to determine whether a particular jaw posture or movementcan be attained by activating the remaining jaw muscles. Here we describethe use of the inverse-dynamics techniques described in Chapter 4 to revealthe muscle forces needed to reduce de cits in the NOCON model.5.3.1 Simulation descriptionsThe compensatory simulations were chosen to counteract the two mainde cits observed in the forward dynamic simulations:  rstly, lateral devia-tion during jaw opening and, secondly, instability during unilateral clench-ing. The ungrafted case, model NOCON, exhibited more pronounced devia-tions and instabilities and was therefore used in compensatory simulations.Hinge movement simulationsWe were interested in determining the muscle forces needed for midline jawmovements with the model. Therefore we simulated movement betweenthree jaw postures that are illustrated in Figure 5.6: REST a relaxed rest posture deviated toward the a ected side androtated clockwise in the frontal-plane OPEN a midline opening posture with 20 mm inter-incisal separationand no frontal-plane rotation, a typically maximal opening gape duringchewing CLOSE a midline posture with the jaw closed to just before  rst toothcontact1075.3. Inverse Dynamics: Compensatory SimulationsTwo movement simulations were performed: from REST to OPEN mov-ing the jaw fragment from the deviated rest posture to the midline, andfrom OPEN to CLOSE moving the jaw fragment in centric relation relativeto the maxilla, for a \hinge-like" movement with no frontal-plane rotation.Smooth target position trajectories were generated with quintic splines fromthe start posture to the end posture over a 0.5 s duration. The target veloc-ity trajectories were computed online with  nite-di erencing which providedon-line correction of position errors as the simulation progressed. We spec-i ed the full six degree-of-freedom position trajectory of the jaw in orderto control its orientation. The target trajectories referenced to the 3D in-cisor point movement are shown for REST-to-OPEN in Figure 5.7a and forOPEN-to-CLOSE in Figure 5.8a.Unilateral clench simulationsWe were interested in determining if a stable unilateral clench could beachieved through the recruitment of an ensemble of closing muscle groupswith appropriately-balanced activation. We performed clenching simulationswith di erent jaw postures to investigate the e ect of jaw position on theability of the model to generate bite force.5.3.2 Simulation resultsDeviated rest to midline retrusive openThe muscle patterns predicted by the inverse simulation for the REST-to-OPEN movement are shown in Figure 5.7b and Table 5.1. Incisor pointmovement to midline along with counter-clockwise rotation of the jaw frag-ment to a neutral orientation was accomplished by co-activation of the right-sided digastric and posterior temporalis muscles. The average position errorof the incisor point during the movement was 0.36 mm, 0.22 mm, and 0.39mm in the left, anterior, and inferior directions respectively. Posterior tem-poralis inserts into the coronoid process and has a force vector best angledto apply the required torque to rotate the fragment to a symmetric midline1085.3. Inverse Dynamics: Compensatory SimulationsFigure 5.6: Medial and frontal views of the jaw postures for relaxed rest(REST), 20 mm retrusive midline open (OPEN), and retrusive midline closeto before tooth contact (CLOSE). + denotes incisor-point location at inter-cuspal position. Grid spacing is 10 mm.1095.3. Inverse Dynamics: Compensatory Simulations0 0.1 0.2 0.3 0.4 0.5−20−15−10−50510Position (mm)0 0.1 0.2 0.3 0.4 0.5−25−20−15−10−505Velocity (mm/s)Time (s)  0 0.1 0.2 0.3 0.4 0.50510152025Muscle Activations (%)0 0.1 0.2 0.3 0.4 0.502468Muscle Forces (N)Time (s)  RDIRMHRIPRSPRPTRSMTARGET TRAJECTORIES MUSCLE PREDICTIONSFigure 5.7: Movement simulation from REST to OPEN. Target positionand velocity trajectories for mandibular incisor point in lateral (solid lines),vertical (dotted lines), and anteroposterior directions (dashed lines) shownin left-side panels. Muscle activations and forces of the primary musclesused to track target trajectory shown in right-side panels.posture. Digastric is activated to open the jaw to 20 mm as is the case inretrusive opening with an intact mandible.Hinge closing from midline open postureThe muscle patterns predicted to move the jaw fragment along a midlinehinge closing trajectory in the OPEN-to-CLOSE movement are shown inFigure 5.8b and Table 5.1 The average position error of the incisor pointduring the movement was 0.08 mm, 0.02 mm, and 0.03 mm in the left, pos-terior, and inferior directions respectively. Posterior temporalis was againrecruited in order to keep the fragment at midline and its activity increased1105.3. Inverse Dynamics: Compensatory Simulations0 0.1 0.2 0.3 0.4 0.5010203040506070Muscle Activations (%)0 0.1 0.2 0.3 0.4 0.501020304050Muscle Forces (N)Time (s)  RDIRIPRSPRPTRDM0 0.1 0.2 0.3 0.4 0.5−20−15−10−50510Position (mm)0 0.1 0.2 0.3 0.4 0.5−40−200204060Velocity (mm/s)Time (s)  TARGET TRAJECTORIES MUSCLE PREDICTIONSFigure 5.8: Movement simulation from OPEN to CLOSE. Target positionand velocity trajectories for mandibular mid-incisor point in lateral (solidlines), vertical (dotted lines), and anteroposterior directions (dashed lines)shown in left-side panels. Muscle activations and forces of the primarymuscles used to track target trajectory shown in right-side panels.during the closing movement. Lateral pterygoids were also co-contracted,presumably to balance the lateral temporalis force and to maintain a midlinemovement.Unilateral clenchThe three simulated clenching postures are illustrated in Figure 5.9 andpredicted muscle activation and force magnitudes are provided in Table 5.2.For comparison, maximal  rst molar bite force for an intact mandible iswithin the range of 216-740N [210]. Clenching with the jaw fragment de-viated toward the a ected side was the most stable act. Moving toward a1115.3. Inverse Dynamics: Compensatory Simulationsrest posture peak open posture peak close posturerest!open open!close%, [ N ] %, [ N ] %, [ N ] %, [ N ] %, [ N ]RPT 2.3 [ 1.8 ] 11.2 [ 7.9 ] 0.2 [ 0.3 ] 64.9 [ 49.0 ] 40.2 [ 30.4 ]RSM 0.0 [ 0.6 ] 0.0 [ 1.3 ] 0.0 [ 1.3 ] 0.0 [ 1.3 ] 0.0 [ 0.0 ]RDM 0.0 [ 0.2 ] 0.0 [ 0.3 ] 0.0 [ 0.3 ] 7.4 [ 5.8 ] 2.7 [ 2.2 ]RMP 0.1 [ 0.9 ] 1.1 [ 1.9 ] 0.1 [ 1.0 ] 1.1 [ 2.0 ] 0.0 [ 0.0 ]RSP 0.7 [ 0.2 ] 8.6 [ 2.5 ] 4.6 [ 1.3 ] 14.8 [ 4.3 ] 8.7 [ 2.5 ]RIP 1.2 [ 0.8 ] 6.1 [ 4.1 ] 0.0 [ 0.0 ] 38.1 [ 25.5 ] 23.9 [ 16.0 ]RDI 1.5 [ 0.3 ] 20.2 [ 2.5 ] 11.7 [ 1.0 ] 23.1 [ 5.5 ] 6.7 [ 2.7 ]RMH 0.4 [ 0.1 ] 4.2 [ 0.7 ] 3.4 [ 0.5 ] 3.4 [ 0.5 ] 0.0 [ 0.0 ]Table 5.1: Muscle activations (%) and forces (N) for movements betweenREST, OPEN, and CLOSE postures. Muscle force includes both passiveforce due to muscle stretch and active force proportional to activation.midline position required muscle recruitment to maintain the posture andreduced bite force magnitudes. At 15 mm lateral and backward deviationof the mandibular mid-incisor point, the simulation was able to generate124 N of force at the bite constraint; however positioning the jaw mediallyto a 10 mm deviation reduced the bite force to 111 N and greatly increasedthe co-activation of opener muscles. In the midline posture, the inverse sim-ulation with a high target bite force was unable to prevent rotation of thejaw fragment. However, a stable midline posture with no rotation of the jawfragment was possible with a diminished bite force of 25 N. The main closermuscles recruited for clenching were anterior temporalis, medial pterygoid,and deep masseter.5.3.3 DiscussionThe simulated recruitment of antagonist muscles for both free jaw move-ments and unilateral clenching suggests that functional de cit caused byunilateral muscle and articular loss may be at least partly overcome bysti ening the system with opposing muscles. The current simulations thusserve as a proof-of-concept of an inverse modeling approach to determine thebiomechanical plausibility of hypothesized movements and associated mus-1125.3. Inverse Dynamics: Compensatory SimulationsFigure 5.9: Superior and frontal views of jaw postures used in clenchingtasks. The model was able to generate a 124 N bite force at 15 mm incisordeviation (15DEV); a 111 N bite force at 10 mm incisor deviation (10DEV);and a 25 N bite force at midline intercuspal position (IP). + denotes incisor-point location at intercuspal position. Grid spacing is 10 mm.1135.3. Inverse Dynamics: Compensatory Simulations124 N clench at 111 N clench at 25 N clench at15DEV 10DEV IP%, [ N ] %, [ N ] %, [ N ]RAT 89.9 [ 140.9 ] 90.1 [ 141.7 ] 16.3 [ 25.8 ]RMT 0.0 [ 0.1 ] 0.0 [ 0.0 ] 4.7 [ 4.5 ]RPT 0.0 [ 0.1 ] 0.0 [ 0.1 ] 2.8 [ 2.1 ]RSM 16.0 [ 29.2 ] 19.3 [ 36.5 ] 6.8 [ 13.0 ]RDM 41.9 [ 34.1 ] 86.8 [ 70.2 ] 1.2 [ 1.0 ]RMP 88.8 [ 154.5 ] 89.0 [ 155.5 ] 11.5 [ 20.1 ]RSP 4.5 [ 0.8 ] 15.6 [ 2.7 ] 0.3 [ 0.0 ]RIP 5.0 [ 2.5 ] 78.0 [ 39.0 ] 5.1 [ 2.5 ]LDI 27.2 [ 4.8 ] 14.9 [ 5.1 ] 0.0 [ 0.0 ]RDI 44.2 [ 20.0 ] 77.5 [ 37.2 ] 0.0 [ 0.0 ]RMH 4.7 [ 0.5 ] 2.3 [ 0.3 ] 0.0 [ 0.0 ]LGH 5.3 [ 0.6 ] 8.3 [ 1.3 ] 0.0 [ 0.0 ]RGH 5.7 [ 0.7 ] 8.5 [ 1.4 ] 0.0 [ 0.0 ]Table 5.2: Muscle activations (%) and forces (N) for unilateral clenchingsimulations. Muscle force includes both passive force due to muscle stretchand active force proportional to activation.1145.3. Inverse Dynamics: Compensatory Simulationscle patterns in an altered jaw system. Comparison of the model predictionswith patient data will require signi cant quantitative clinical measurementson patients.In the reported movement simulations, posterior temporalis was used toprovide a torque to move the incisor point to the midline in a right lateralmovement to compensate for the left lateral deviation caused by passive scartissue forces. Such movement would normally be accomplished by the con-tralateral lateral pterygoids which are missing in the hemimandibulectomycase. Ipsilateral temporalis contributes to normal lateral movement [117].Therefore, it is plausible that right-side posterior temporalis is recruited tocompensate for the missing lateral pterygoid muscles in a left-sided de cit.Ipsilateral lateral pterygoids co-activate with posterior temporalis, which isatypical, but necessary in the model to generate medial forces at the condyleto balance the lateral forces generated by posterior temporalis. We expectthat the inclusion of lateral constraint at the joint, such as the lateral aspectof the articular fossa or the temporomandibular ligament, would reduce theneed for lateral pterygoid co-activation.Predicted muscle activations are a ected by the choice of regularizationterm in the optimization (see Section 4.1 for discussion). Di erent optimalityconditions have been proposed for a variety of physiological and numericalreasons (see Erdemir et al. [53] and Ait-Haddou et al. [4]). We use an ‘2-norm weighted by the inverse of each muscles CSA. The main di erenceobserved with CSA-weighted regularization was in the relative contributionof synergistic muscles. In the lateral pterygoids, for example, the inferior-head activity was increased and superior-head activity decreased when usingthe CSA-weighted regularizer as compared to an unweighted ‘2-norm, be-cause inferior-head is a signi cantly larger muscle. The relative scaling ofantagonist groups was una ected by regularizer weighting as it is prescribedby the requirements of tracking the target trajectory.Our current tracking algorithm is formulated only for bilateral con-straints. Contact modeling requires unilateral constraints that add inequal-ity conditions on the dynamics equation (see Section C.1). This limitationreduces the number of tasks we can currently model: in hinge jaw opening1155.4. Directionsthe condyle does not move forward o the posterior joint constraint and inclenching with the teeth always in contact. As discussed in Section 4.4, itmay be possible to treat contacts as bilateral constraints for the duration ofa timestep and break contact if the constraint is found to be pulling apart.The bite constraint, modeled here as a planar contact surface alignedto the occlusion plane, simulates a patient with  at teeth. Here, we foundit impossible to hold a midline intercuspal position and generate a bite-force more than 25 N. Moreover, all bite forces were considerably less thanthose normally generated in the intact jaw during tooth-clenching, i.e. 216-740N [210]. A reduced capacity to generate bite-force in the present casecan be expected given the loss of half the closing muscles. Notwithstandingthe additional, atypical contributions of antagonistic muscles made availablein our simulations, the postural e ect on bite-force generation re ects theextreme biomechanical conditions needed to create any signi cant unilateralbite force in the hemimandibulectomy patient.5.4 DirectionsOur current models were limited by the absence of a tongue, a  xed hyoidapparatus, and arbitrary soft-tissue scar forces. Incorporating a 3D FEMmodel of tongue tissue is a straight-forward direction, given the coupled jaw-tongue-hyoid model presented in Chapter 3, and would also allow for theinvestigation of glossectomy cases. Incorporating a physiologically-based3D FEM model of scar tissue is signi cantly more challenging. Detailedinformation on the extent and location of scarring in hemimandibulectomypatients is currently unavailable. The physical properties of soft-tissuesaltered by wound-healing and radiation therapy also remain unde ned. Analternative approach to improve the realism of scarring in the model wouldinvolve characterizing the functional e ect of scarring on patients. Oneapproach to measure scar-induced jaw sti ness would be to systematicallyapply external forces to a patient’s lower jaw, while simultaneously trackingjaw motion. Such techniques have been used to characterize jaw sti nessand muscle viscoelasticity in normal subjects [147, 176].1165.5. SummaryOur current models use a planar bite constraint, representing contactbetween  at teeth. It is possible that a modi ed dental interface, such asinclined tooth surfaces to guide the jaw fragment toward the midline duringclenching, has a signi cant e ect on inverse predictions of muscle activityduring clenching. In future work, actual occlusal surface shapes used indental reconstruction could be incorporated in our model, making it possibleto study the e ects of occlusal con gurations on system biomechanics andpredicted muscle patterning.5.5 SummaryComputer models of anatomical systems can be used to analyze structurallyaltered and functionally compromised cases. In this chapter, we have de-scribed a study analyzing jaw movements in compromised mandibles withand without continuity, as would occur in patient after segmental jaw surgeryconsequent to oral cancer. Through forward dynamic simulations, we illus-trated the functional consequences of missing muscle and bone structure andrecreated clinically observed de cits in the model. We also compared themechanics of de cits in cases with and without jaw grafting. Further, withnew inverse modeling techniques, were we able to predict what muscle forceswould theoretically be needed to compensate for post-operative de cits.These compensatory muscle predictions could be used to inform patient-speci c rehabilitative therapy. Through the case study of jaw surgery, havedemonstrated that computer modeling presents a promising approach to un-derstanding the biomechanics of surgically altered musculoskeletal systems.In the  nal chapter, we summarize the contributions of the dissertationand propose a few broader directions for future investigation.117Chapter 6ConclusionsTo conclude, this chapter reviews and summarizes the contributions of thisdissertation. The research publications and presentations associated withthis work are also listed. Finally, longer-term directions drawing on thisresearch are presented along with concluding comments.6.1 Dissertation ContributionsThe contributions of this dissertation include an advanced jaw-tongue-hyoidmodel, inverse simulation toolset, and model-based analysis of jaw surgery.While these contributions are made within the context of orofacial anatomyand function, the framework and methodology is widely applicable to mus-cle activated skeletal and soft-tissue systems and biomedical applicationsinvolving biomechanical analysis.Modeling of coupled jaw-tongue-hyoid biomechanicsi. Created a novel model of the jaw-tongue-hyoid system. Byadapting previously reported reference models to a single-subject withinthe same simulation platform we have created the most complete phys-iologically based computer model of the oral region to date. The modelprovides a foundation on which to analyze the biomechanics and neu-romotor control of oral motor tasks, such as mastication, deglutition,and speech production, which involve the coordination of multiple oralarticulators.ii. Demonstrated the signi cance of coupling. Through forward dy-namic simulation with isolated muscle activations we were able to deter-1186.1. Dissertation Contributionsmine that the mechanical coupling of tongue-muscles acting on the jaw,and vice versa, were signi cant. In particular, the elastic connection oftongue tissue between the jaw and hyoid were observed to restrict jawmovement. Also, the movement of the tongue within the oral cavity in-duced non-negligible jaw movements with the jaw at rest. Jaw-tonguecoupling e ects are therefore an important factor for consideration infuture orofacial modeling studies.iii. Compared results with recorded tongue velocity and pres-sure. Through forward dynamic simulation of tongue muscle activationwe evaluated biomechanical variables in comparison to reported hu-man tongue measurements. The velocity pro le of simulated backwardtongue movement compared well with recorded EMA data. Tongue topalate pressure simulated with maximum muscle activation in tongue-palate contact agreed with average valued human measurements. Thesetwo qualitative comparisons provide a framework for future validationmeasurements in order to further verify that the model behaves withinplausible ranges for physiological and maximum-voluntary tasks.iv. Used as a test case for the ArtiSynth platform. The modelwas developed concurrently with the underlying simulation platform,ArtiSynth. It provided a su ciently challenging test case to advancethe modeling  delity of ArtiSynth to be appropriate for the oral, pha-ryngeal, and laryngeal systems; in particular, with respect to hard-softtissue coupling and contact.Inverse techniques for hard/soft muscle-tissue modelsi. Formulated trajectory-tracking for muscle-activated dynamicFEM with constraints. We extended inverse trajectory-trackingtechniques, previously proposed for articulated rigid-body models oflimbs, for a generalized forward dynamics of rigid-body and FEM mod-els within the ArtiSynth platform. The inverse solver is appropriate forcomplex muscular-hydrostat structures, modeled with dynamic FEM,1196.1. Dissertation Contributionsas well as coupled skeletal/muscle-tissue structures. The inverse tech-niques provide a signi cant added-value to the simulation toolset allow-ing for systematic investigation of muscle recruitment, given a partic-ular desired kinematic trajectory, as opposed to trial-and-error basedmuscle tuning.ii. Formulated novel target parameters: constraint forces andsti ness. We have extended the inverse formulation to include tar-get parameters in additional to kinematics, providing a comprehensivetoolset for exploring motor redundancy in motor tasks involving dy-namic constraints or sti ness. Constraint force magnitude targets areused to select muscle activations to increase forces within constrainedDOF, such as to generate bite forces in the jaw model. Sti ness tar-gets are used to co-activate antagonist muscle groups to increase systemsti ness which is important in many motor tasks.iii. Predicted beam and tongue muscle activations consistent withmuscular-hydrostat theory. We used the inverse simulation toolsetto automatically compute muscle activations needed to move and de-form muscular-hydrostat models, including a beam model and a phys-iologically based tongue model. The simulation results are consistentwith theoretical proposals and provide a signi cant contribution overprevious work by systematically evaluating muscular-hydrostat motorrecruitment without a priori selection of muscle patterns.iv. Predicted plausible muscle activations for lateral jaw move-ment. We applied the inverse simulation toolset to automatically com-pute jaw muscle activations in lateral jaw movement. The simulationresults are consistent with published jaw physiology. We also proposeda framework to analyze lateral pterygoid recruitment with respect tomechanical advantage in comparison to published EMG studies.1206.2. DirectionsApplication to the analysis of segmental jaw surgeryi. Created models of segmental jaw surgery with/without re-construction. We developed models of segmental jaw resection andreconstruction through structural alterations to a model of the normaljaw system. The procedure illustrates the application of a biomechan-ical model to biomedical treatment and rehabilitation planning.ii. Compared mechanical basis of functional de cits between mod-els. We simulated movement and bite force de cits that are observedclinically consequent to jaw resection. By comparing a model withand without mandibular grafting we illustrate the biomechanical un-derpinnings of lateral deviation and unstable jaw fragment movements.The simulations provide a foundation for future studies on the e ect ofscarring and the design of mandible and dental prostheses.iii. Applied inverse toolset to predict muscle forces to compensatefor de cits. We applied the inverse simulation toolset to automati-cally predict muscle forces in the model to compensate for functionalde cits. Results show that atypical co-activation of antagonist mus-cles was capable of aligning and stabilizing a unilaterally resected jawmodel. The technique provides a methodology for analyzing alteredand compromised anatomical systems and could inform rehabilitativemuscle strengthening or motor retraining.6.2 DirectionsShort term directions for the research components of this dissertation arediscussed at the end of each research chapter. Here we discuss broader direc-tions for the future potential of integrating computational tools, includingdigital models and biomechanics simulation, in medical research and prac-tice.Protocols for clinical data collection The modeling work presentedin this dissertation has focused on mechanisms to permit the integration1216.2. Directionsof experimental data with biomechanics models. In particular, the inversetrajectory-tracking methods described in Chapter 4 suggest that simultane-ous recordings of detailed kinematic data and muscle activity are useful toevaluate model-based muscle force predictions. Also, Section 3.3.2 illustratesthat maximum voluntary e ort experiments, such as maximum tongue-palate pressure, provide a means to calibrate a model’s maximum muscleforce parameters. In general, measurements of external contact forces dur-ing motor tasks provide a means to assess simulated internal muscle forcesin a model, which are impossible to directly measure in humans. Finally, oure orts to model the mechanical consequences of jaw surgery revealed thatthe properties of scar tissue play a signi cant functional role and are notwell described in the literature. In addition to detailed measurement of scarvolume and mechanical properties, we suggest that clinical assessment of thee ect of scar sti ness on jaw mobility would be helpful. Applying knownexternal forces to the patient’s lower jaw in a number of directions and mag-nitudes, while simultaneously measuring jaw movement, would provide dataon the e ective jaw sti ness induced due to post-operative scarring. Thesespeci c experimental measurements are examples of what we see as a promis-ing future promising direction for iterative re nement of both modeling andexperimental recording techniques as each process informs the other.Clinical work ow management Clinical and surgical work ows for mul-tistage medical procedures, such as oral and maxillofacial reconstructivesurgery, are complex and involve a large volume of patient data from multi-ple sources. Current clinical work ows are organized in ways that can leadto ine cient  ow of information and reduced team communication has beenassociated with higher rates of surgical morbidity and mortality [44]. Digitalmodels present a promising and tangible mechanism to synthesize informa-tion for a single patient across di erent data sources and imaging modalitiesas well as to integrate information and statistics across patient populations.Digital modeling tools also provide highly visual artifacts for maintainingcommunication between treatment stakeholders, especially the patient. Aninformed patient is more likely to be a satis ed patient and digital mod-1226.2. Directionsels have the potential to help keep a patient well-informed throughout thetreatment process.Treatment alternatives Biomechanical models have the potential to im-pact treatment planning through quantitative analysis of trade-o s betweentreatment alternatives. For example, radiation therapy in head and neckcancer can cause damage and scarring in healthy tissue, which could po-tentially cause more adverse biomechanical alterations than tissue resectionand reconstruction. With accurate, validated, and patient-speci c models,these alternative treatment pathways could be evaluated quantitatively interms of expected biomechanical outcomes.Tissue regeneration Reconstructive medicine can bene t from compu-tational biomechanics because grafting and transplanting tissue has bothaesthetic and mechanical motivations. Regenerative medicine, includingsynthetic tissues and tissue regeneration, has the potential to vastly in-crease our capacity to alter, reshape, and restore anatomical structures andthe need for computer tools to plan and guide such activities will increaseaccordingly [32].Biological modeling science Advances in biomedical technology, includ-ing models and simulation techniques, are driving an emerging scienti cdiscipline that integrates biological and medical knowledge with engineer-ing expertise. This discipline, the science of biological modeling, requiresa broad knowledge base in biological science, empirical techniques, com-puter methods, and mathematics, as demonstrated by the breadth of relatedwork presented in this dissertation. Students undertaking such study willlikely be engineers with biological/medical interests, and biologists, doctors,and dentists with a strong technical background. Realizing the potentialof this emerging  eld requires developing both formal curricula and dedi-cated research communities. Paramount to its success will be an open andstandards-based research environment that will allow researchers to com-pare and synthesize new models, share common data-sets for both normal1236.3. Concluding Remarkssubjects and patients, and recreate published modeling and data-processingtechniques in order to verify and validate results.6.3 Concluding RemarksTo conclude, this dissertation has presented new models and simulation tech-niques to analyze jaw and tongue movements, muscle forces, and biomechan-ics. We have applied these modeling techniques in the biomedical domainand are working toward the development of new tools for dentists, doctors,and surgeons to better diagnose and treat orofacial and upper airway dys-function. This work constitutes a central component of a larger projectdeveloping models of the entire upper airway. The research has been carriedout within an interdisciplinary team and has lead to a number of ongoingcollaborations and projects. The models and simulation platform are openlyavailable on the web for use by other researchers.124Bibliography[1] S. Abd-El-Malek. The part played by the tongue in mastication anddeglutition. Journal of Anatomy, 89(2):250{254, 1955. ! pages 14[2] R. Adell, B. Svensson, and T. B agenholm. Dental rehabilitationin 101 primarily reconstructed jaws after segmental resections-possibilities and problems. An 18-year study. Journal of Cranio-Maxillofacial Surgery, 36(7):395{402, 2008. ! pages 95[3] M. J. Aftosmis, M. J. Berger, and J. E. Melton. Robust and e cientcartesian mesh generation for component-based geometry. AIAAJournal, 36(6):952{960, 1998. ! pages 181[4] R. Ait-Haddou, A. Jinha, W. Herzog, and P. Binding. Analysis ofthe force-sharing problem using an optimization model. Mathemati-cal Biosciences, 191(2):111{122, 2004. ! pages 76, 115[5] F. Alipour, I. R. Titze, E. Hunter, and N. Tayama. Active and pas-sive properties of canine abduction/adduction laryngeal muscles.Journal of Voice, 19(3):350{359, 2005. ! pages 24, 32[6] J. Allard, S. Cotin, F. Faure, P.-J. Bensoussan, F. Poyer, C. Duriez,H. Delingette, and L. Grisoni. SOFA { an open source framework formedical simulation. In Medicine Meets Virtual Reality (MMVR ’07),pages 13{18, 2007. ! pages 28[7] A. E. Anderson, B. J. Ellis, and J. A. Weiss. Veri cation, validationand sensitivity studies in computational biomechanics. ComputerMethods in Biomechanics and Biomedical Engineering, 10(3):171{184, 2007. ! pages 65[8] F. C. Anderson and M. G. Pandy. Dynamic optimization of humanwalking. Journal of Biomechanical Engineering, 123(5):381{390,2001. ! pages 38125Bibliography[9] F. C. Anderson and M. G. Pandy. Static and dynamic optimizationsolutions for gait are practically equivalent. Journal of Biomechan-ics, 34(2):153{161, 2001. ! pages 38[10] M. Anitescu and G. D. Hart. A constraint-stabilized time-steppingapproach for rigid multibody dynamics with joints, contact and fric-tion. International Journal for Numerical Methods in Engineering,60(14):2335{2371, 2004. ! pages 176[11] M. Anitescu and F. A. Potra. A time-stepping method for sti multibody dynamics with contact and friction. International Jour-nal for Numerical Methods in Engineering, 55(7):753{784, 2002. !pages 175[12] ANSYS, Canonsburg, PA. ANSYS FEM simulation software. http://www.ansys.com, 2010. ! pages 27, 46[13] G. Ardran and F. Kemp. The protection of the laryngeal airwayduring swallowing. The British Journal of Radiology, 25(296):406{416, 1952. ! pages 15[14] P. Badin and A. Serrurier. Three dimensional modelling of speechorgans: articulatory data and models. IEICE Technical Report, 106(177):29{34, 2006. ! pages 65, 90[15] P. Badin, G. Bailly, L. Reveret, M. Baciu, C. Segebarth, andC. Savariaux. Three-dimensional linear articulatory modeling oftongue, lips and face, based on MRI and video images. Journal ofPhonetics, 30(3):533{553, 2002. ! pages 16[16] A. Baker. A biomechanical model of the human tongue for under-standing speech production and other lingual behaviors. PhD thesis,Department of Linguistics, University of Arizona, 2008. ! pages 33[17] T. Belytschko, W. K. Liu, and B. Moran. Nonlinear  nite elementanalysis for continua and structures. Wiley, New York, 2000. !pages 27[18] N. A. Bernstein. The co-ordination and regulation movements. Perg-amon Press, Oxford, 1967. ! pages 14[19] D. A. Berry, J. B. Moon, and D. P. Kuehn. A  nite element modelof the soft palate. The Cleft Palate-Craniofacial Journal, 36(3):217{223, 1999. ! pages 34126Bibliography[20] R. Bibb, D. Eggbeer, P. Evans, A. Bocca, and A. Sugar. Rapid man-ufacture of custom- tting surgical guides. Rapid Prototyping Jour-nal, 15(5):346{354, 2009. ! pages 40[21] M. Birch and P. Srodon. Biomechanical properties of the human softpalate. The Cleft Palate-Craniofacial Journal, 46(3):268{274, 2009.! pages 24[22] J. Bonet and R. D. Wood. Nonlinear Continuum Mechanics forFinite Element Analysis. Cambridge University Press, Cambridge,UK, 2008. ! pages 27, 47[23] A. Bothorel. Cin eradiographie des voyelles et consonnes du franccais: recueil de documents synchronis es pour quatre sujets: vues lat eralesdu conduit vocal, vues frontales de l’ori ce labial, donn ees acous-tiques. Institut de phon etique de Strasbourg, Strasbourg, France,1986. ! pages 15[24] P. Boyle and B. Levin. World cancer report 2008. IARC Press,online, 2008. ! pages 5[25] B. Brogliato and V. Acary. Numerical methods for nonsmooth dy-namical systems. Springer, Berlin, 2008. ! pages 185[26] I. E. Brown and G. E. Loeb. A reductionist approach to creatingand using neuromusculoskeletal models. In J. M. Winters and P. E.Crago, editors, Biomechanics and Neural Control of Posture andMovement, pages 148{163. Springer-Verlag, New York, 2000. !pages 36[27] I. E. Brown, E. J. Cheng, and G. E. Loeb. Measured and modeledproperties of mammalian skeletal muscle. II. The e ects of stimulusfrequency on force-length and force-velocity relationships. Journal ofMuscle Research and Cell Motility, 20(7):627{643, 1999. ! pages 32[28] S. Buchaillard, M. Brix, P. Perrier, and Y. Payan. Simulations of theconsequences of tongue surgery on tongue mobility: Implications forspeech production in post-surgery conditions. International Journalof Medical Robotics and Computer Assisted Surgery, 3(3):252{261,2007. ! pages 40[29] S. Buchaillard, P. Perrier, and Y. Payan. A biomechanical modelof cardinal vowel production: Muscle activations and the impact of127Bibliographygravity on tongue positioning. Journal of the Acoustical Society ofAmerica, 126(4):2033{2051, 2009. ! pages 21, 27, 31, 33, 36, 44, 45,46, 48, 50, 57, 59, 66, 93[30] M. Bucki, M. Nazari, and Y. Payan. Finite element speaker-speci cface model generation for the study of speech production. ComputerMethods in Biomechanics and Biomedical Engineering., 13(4):459{467, 2010. ! pages 27, 29, 48, 61[31] T. A. Burnett, E. A. Mann, S. A. Cornell, and C. L. Ludlow. Laryn-geal elevation achieved by neuromuscular stimulation at rest. Jour-nal of Applied Physiology, 94(1):128{134, 2003. ! pages 16, 63[32] D. L. Butler, S. A. Goldstein, R. E. Guldberg, X. E. Guo,R. Kamm, C. T. Laurencin, L. V. McIntire, V. C. Mow, R. M.Nerem, R. L. Sah, L. J. Soslowsky, R. L. Spilker, and R. T. Tran-quillo. The impact of biomechanics in tissue engineering and regen-erative medicine. Tissue Engineering Part B: Reviews, 15(4):477,2009. ! pages 123[33] E. R. Carlson and R. E. Marx. Mandibular reconstruction usingcancellous cellular bone grafts. Journal of Oral and MaxillofacialSurgery, 54(7):889{897, 1996. ! pages 95[34] Cartsens, Munich, Germany. AG500 electromagnetic articulograph.http://www.articulograph.de, 2010. ! pages 17[35] M. C. Cavusoglu, T. Goktekin, and F. Tendick. GiPSi: a frame-work for open source/open architecture software development fororgan-level surgical simulation. IEEE Transactions on InformationTechnology in Biomedicine, 10(2):312{322, 2006. ! pages 28[36] M. Chabanas, V. Luboz, and Y. Payan. Patient speci c  nite ele-ment model of the face soft tissues for computer-assisted maxillofa-cial surgery. Medical Image Analysis, 7(2):131{151, 2003. ! pages40, 62[37] D. T. Corr, C. L. Gallant-Behm, N. G. Shrive, and D. A. Hart.Biomechanical behavior of scar tissue and uninjured skin in aporcine model. Wound Repair and Regeneration, 17(2):250{259,2009. ! pages 104128Bibliography[38] R. W. Cottle, J.-S. Pang, and R. E. Stone. The Linear Complemen-tarity Problem. Academic Press, Boston, MA, 1992. ! pages 77,177[39] R. D. Crowninshield and R. A. Brand. A physiologically based crite-rion of muscle force prediction in locomotion. Journal of Biomechan-ics, 14(11):793{801, 1981. ! pages 37[40] D. A. Curtis, O. Plesh, A. J. Miller, T. A. Curtis, A. Sharma,R. Schweitzer, R. L. Hilsinger, L. Schour, and M. Singer. A com-parison of masticatory function in patients with or without recon-struction of the mandible. Head & Neck, 19(4):287{296, 1997. !pages 95, 96[41] J. Dang and K. Honda. Construction and control of a physiologicalarticulatory model. Journal of the Acoustical Society of America,115(2):853{870, 2004. ! pages 33[42] Dassault Syst emes, V elizy-Villacoublay, France. Simulia. http://www.simulia.com, 2010. ! pages 27[43] Dassault Syst emes, V elizy-Villacoublay, France. Solidworks. http://www.solidworks.com/, 2010. ! pages 25[44] D. L. Davenport, W. G. Henderson, C. L. Mosca, S. F. Khuri, andR. M. Mentzer Jr. Risk-adjusted morbidity in teaching hospitalscorrelates with reported levels of communication and collaborationon surgical teams but not with scale measures of teamwork climate,safety climate, or working conditions. Journal of the American Col-lege of Surgeons, 205(6):778{784, 2007. ! pages 122[45] M. de Zee, P. M. Cattaneo, P. Svensson, T. K. Pedersen,B. Melsene, J. Rasmussen, and M. Dalstra. Prediction of the ar-ticular eminence shape in a patient with unilateral hypoplasia of theright mandibular ramus before and after distraction osteogenesis { asimulation study. Journal of Biomechanics, 42(8):1049{1053, 2009.! pages 33, 40[46] S. L. Delp, F. C. Anderson, A. S. Arnold, P. Loan, A. Habib, C. T.John, E. Guendelman, and D. G. Thelen. OpenSim: open-sourcesoftware to create and analyze dynamic simulations of movement.IEEE Transactions on Biomedical Engineering, 54(11):1940{1950,2007. ! pages 25, 30129Bibliography[47] R. L. Drake, W. Vogl, A. W. M. Mitchell, R. Tibbitts, andP. Richardson. Gray’s anatomy for students. Churchill Livingstone,Elsevier Inc., Philadelphia, PA, 2nd edition, 2010. ! pages 2, 152,153, 154, 155, 157, 158, 159, 161, 164, 165, 167, 170[48] F. A. Duck. Physical properties of tissue: a comprehensive referencebook. Academic Press, London, UK, 1990. ! pages 24[49] P. R. Eastwood, G. T. Allison, K. L. Shepherd, I. Szollosi, and D. R.Hillman. Heterogeneous activity of the human genioglossus muscleassessed by multiple bipolar  ne-wire electrodes. Journal of AppliedPhysiology, 94(5):1849{1858, 2003. ! pages 20[50] H. Eckel, C. Sittel, P. Zorowka, and A. Jerke. Dimensions of thelaryngeal framework in adults. Surgical and Radiologic Anatomy, 16(1):31{36, 1994. ! pages 26, 27, 168[51] H. Edelsbrunner and E. P. M ucke. Simulation of simplicity: a tech-nique to cope with degenerate cases in geometric algorithms. ACMTransactions on Graphics, 9(1):66{104, 1990. ! pages 181[52] O. Engwall. Combining MRI, EMA and EPG measurements in athree-dimensional tongue model. Speech Communication, 41(2-3):303{329, 2003. ! pages 16, 17, 18[53] A. Erdemir, S. McLean, W. Herzog, and A. J. van den Bogert.Model-based estimation of muscle forces exerted during movements.Clinical Biomechanics, 22(2):131{154, 2007. ! pages 35, 39, 115[54] Q. Fang, S. Fujita, X. Lu, and J. Dang. A model-based investigationof activations of the tongue muscles in vowel production. AcousticalScience and Technology, 30(4):277{287, 2009. ISSN 1346-3969. !pages 34[55] A. Feldman. Once more on the equilibrium-point hypothesis(lambda model) for motor control. Journal of Motor Behavior, 18(1):17, 1986. ! pages 36, 47[56] A. Feldman and M. Latash. Testing hypotheses and the advance-ment of science: recent attempts to falsify the equilibrium point hy-pothesis. Experimental Brain Research, 161(1):91{103, 2005. !pages 36130Bibliography[57] C. P. Fernandes, P. O. J. Glantz, S. A. Svensson, and A. Bergmark.A novel sensor for bite force determinations. Dental Materials, 19(2):118{126, 2003. ! pages 19[58] B. R. Fink, R. W. Martin, and C. A. Rohrmann. Biomechanics ofthe Human Epiglottis. Acta Oto-Laryngologica, 87(3):554{559, 1979.! pages 169[59] S. Fujita, J. Dang, N. Suzuki, and K. Honda. A computationaltongue model and its clinical application. Oral Science International,4(2):97{109, 2007. ! pages 40[60] L. M. Gallo, K. Fushima, and S. Palla. Mandibular helical axis path-ways during mastication. Journal of Dental Research, 79:1566{1572,2000. ! pages 14[61] S. C. Gandevia, A. L. Hudson, R. B. Gorman, J. E. Butler, andA. De Troyer. Spatial distribution of inspiratory drive to theparasternal intercostal muscles in humans. The Journal of Physi-ology, 573(1):263, 2006. ! pages 87[62] J. M. Gerard, R. Wilhelms-Tricarico, P. Perrier, and Y. Payan. A3D dynamical biomechanical tongue model to study speech motorcontrol. Recent Research Developments in Biomechanics, 1:49{64,2003. ! pages 33[63] J. M. Gerard, J. Ohayon, V. Luboz, P. Perrier, and Y. Payan. Nonlinear elastic properties of the lingual and facial tissues assessed byindentation technique: application to the biomechanics of speechproduction. Medical Engineering & Physics, 27(10):884{892, 2005.! pages 24, 47[64] R. J. Gilbert and V. J. Napadow. Three-dimensional muscular ar-chitecture of the human tongue determined in vivo with di usiontensor magnetic resonance imaging. Dysphagia, 20(1):1{7, 2005. !pages 23[65] GNU General Public License. Blender. http://www.blender.org/,2010. ! pages 26[66] H. Gomi and M. Kawato. Human arm sti ness and equilibrium-point trajectory during multi-joint movement. Biological Cybernet-ics, 76(3):163{171, 1997. ! pages 36, 93131Bibliography[67] E. Guendelman, R. Bridson, and R. Fedkiw. Nonconvex rigid bodieswith stacking. In ACM SIGGRAPH Papers, pages 871{878, 2003. !pages 184[68] A. Hannam and A. McMillan. Internal organization in the humanjaw muscles. Critical Reviews in Oral Biology & Medicine, 5(1):55,1994. ! pages 31, 158, 160[69] A. G. Hannam. Current computational modelling trends in cran-iomandibular biomechanics and their clinical implications. Journalof Oral Rehabilitation, in press, 2010. ! pages 39, 65[70] A. G. Hannam and W. W. Wood. Medial pterygoid muscle activityduring the closing and compressive phases of human mastication.American Journal of Physical Anthropology, 55(3):359{367, 1981. !pages 19[71] A. G. Hannam, I. Stavness, J. E. Lloyd, and S. Fels. A dynamicmodel of jaw and hyoid biomechanics during chewing. Journal ofBiomechanics, 41(5):1069{1076, 2008. ! pages 33, 44, 45, 50, 52,54, 56, 57, 65, 172[72] A. G. Hannam, I. Stavness, J. E. Lloyd, S. Fels, A. J. Miller, andD. A. Curtis. A comparison of simulated jaw dynamics in modelsof segmental mandibular resection versus resection with alloplasticreconstruction. Journal of Prosthetic Dentistry, 104(3):191{198,2010. ! pages iii, 99[73] M. Haraguchi, H. Mukohyama, D. J. Reisberg, and H. Taniguchi.Electromyographic activity of masticatory muscles and mandibularmovement during function in marginal mandibulectomy patients.Journal of Medical and Dental Sciences, 50(4):257{264, 2003. !pages 95[74] C. Head, D. Alam, J. A. Sercarz, J. T. Lee, J. D. Rawnsley, G. S.Berke, and K. E. Blackwell. Microvascular  ap reconstruction ofthe mandible: a comparison of bone grafts and bridging plates forrestoration of mandibular continuity. Otolaryngology-Head and NeckSurgery, 129(1):48{54, 2003. ! pages 95[75] W. D. Henshaw. Automatic grid generation. Acta numerica, 5:121{148, 2008. ! pages 28132Bibliography[76] O. Hidaka, M. Iwasaki, M. Saito, and T. Morimoto. In uence ofclenching intensity on bite force balance, occlusal contact area, andaverage bite pressure. Journal of Dental Research, 78(7):1336, 1999.! pages 19[77] D. A. Hidalgo and A. L. Pusic. Free- ap mandibular reconstruction:a 10-year follow-up study. Plastic and Reconstructive Surgery, 110(2):438{449, 2002. ! pages 95[78] K. M. Hiiemae and J. B. Palmer. Tongue movements in feedingand speech. Critical Reviews in Oral Biology & Medicine, 14(6):413, 2003. ! pages 7, 13, 67[79] K. M. Hiiemae, J. B. Palmer, S. W. Medicis, J. Hegener,B. Scott Jackson, and D. E. Lieberman. Hyoid and tongue surfacemovements in speaking and eating. Archives of Oral Biology, 47(1):11{27, 2002. ! pages 15, 65[80] A. V. Hill. The heat of shortening and the dynamic constants ofmuscle. Philosophical Transactions of the Royal Society B: BiologicalSciences, 126(843):136{195, 1938. ! pages 32[81] N. Hogan. Adaptive control of mechanical impedance by coactiva-tion of antagonist muscles. IEEE Transactions on Automatic Con-trol, 29(8):681{690, 1984. ! pages 37, 75[82] K. Hori, T. Ono, and T. Nokubi. Coordination of tongue pressureand jaw movement in mastication. Journal of Dental Research, 85(2):187{191, 2006. ! pages 18[83] K. Hori, T. Ono, K. Tamine, J. Kondo, S. Hamanaka, Y. Maeda,J. Dong, and M. Hatsuda. Newly developed sensor sheet for mea-suring tongue pressure during swallowing. Journal of ProsthodonticResearch, 53(1):28{32, 2009. ! pages 18[84] B. K. P. Horn, H. M. Hilden, and S. Negahdaripour. Closed-formsolution of absolute orientation using orthonormal matrices. Journalof the Optical Society of America A, 5(7):1127{1135, 1988. ! pages26[85] B. Y. Huang, T. Whittle, and G. M. Murray. Activity of inferiorhead of human lateral pterygoid muscle during standardized lateraljaw movements. Archives of Oral Biology, 50(1):49{64, 2005. !pages 20, 87133Bibliography[86] Y. Huang, D. P. White, and A. Malhotra. Use of computationalmodeling to predict responses to upper airway surgery in obstructivesleep apnea. The Laryngoscope, 117(4):648{653, 2007. ! pages 3, 34[87] A. L. Hudson, J. L. Taylor, S. C. Gandevia, and J. E. Butler. Cou-pling between mechanical and neural behaviour in the human  rstdorsal interosseous muscle. Experimental Physiology, 587(4):917{925,2009. ! pages 87[88] T. J. R. Hughes. The  nite element method: linear static and dy-namic  nite element analysis. Dover Publications, New York, 2000.! pages 173, 185, 186[89] E. J. Hunter and I. R. Titze. Re nements in modeling the passiveproperties of laryngeal soft tissue. Journal of Applied Physiology,103(1):206, 2007. ! pages 24[90] E. J. Hunter, I. R. Titze, and F. Alipour. A three-dimensional modelof vocal fold abduction/adduction. Journal of the Acoustical Societyof America, 115:1747, 2004. ! pages 34, 169[91] C. Jeannin, P. Perrier, Y. Payan, A. Dittmar, and B. Grosgogeat.Tongue pressure recordings during speech using complete denture.Materials Science and Engineering: C, 28(5-6):835{841, 2008. !pages 18[92] W. Jiang, H. Bo, G. YongChun, and N. LongXing. Stress distri-bution in molars restored with inlays or onlays with or without en-dodontic treatment: A three-dimensional  nite element analysis.Journal of Prosthetic Dentistry, 103(1):6{12, 2010. ! pages 40[93] M. I. Jordan and D. E. Rumelhart. Forward models: Supervisedlearning with a distal teacher. Cognitive Science, 16(3):307{354,1992. ! pages 36[94] P. Kahrilas, J. Logemann, S. Lin, and G. Ergun. Pharyngeal clear-ance during swallowing: a combined manometric and video uoro-scopic study. Gastroenterology, 103(1):128, 1992. ! pages 19[95] M. Kawato, K. Furukawa, and R. Suzuki. A hierarchical neural-network model for control and learning of voluntary movement. Bio-logical Cybernetics, 57(3):169{185, 1987. ! pages 36134Bibliography[96] W. M. Kier and K. K. Smith. Tongues, tentacles and trunks: thebiomechanics of movement in muscular-hydrostats. Zoological Jour-nal of the Linnean Society, 83(4):307{324, 1985. ! pages 81, 83,84[97] J. Koolstra and T. Van Eijden. Three-dimensional dynamical capa-bilities of the human masticatory muscles. Journal of Biomechanics,32(2):145{152, 1999. ! pages 101, 106[98] J. Koolstra and T. Van Eijden. A method to predict muscle controlin the kinematically and mechanically indeterminate human masti-catory system. Journal of Biomechanics, 34(9):1179{1188, 2001. !pages 38[99] J. Koolstra and T. Van Eijden. Functional signi cance of the cou-pling between head and jaw movements. Journal of Biomechanics,37(9):1387{1392, 2004. ! pages 65, 106[100] J. Koolstra and T. Van Eijden. Combined  nite-element and rigid-body analysis of human jaw joint dynamics. Journal of Biomechan-ics, 38(12):2431{2439, 2005. ! pages 33[101] J. H. Koolstra and T. Van Eijden. The jaw open-close movementspredicted by biomechanical modelling. Journal of Biomechanics, 30(9):943{950, 1997. ! pages 33[102] T. Kuratate, H. Yehia, and E. Vatikiotis-Bateson. Kinematics-basedsynthesis of realistic talking faces. In Auditory-Visual Speech Pro-cessing (AVSP ’98), page online, 1998. URL http://marcs.uws.edu.au/links/avisa/avsp98/papers/0028.pdf. ! pages 14[103] C. Lacoursi ere. Ghosts and machines: regularized variational meth-ods for interactive simulations of multibodies with dry frictional con-tacts. PhD thesis, Computer Science Deptartment, Umea University,Sweden, 2007. ! pages 175, 178[104] G. E. Langenbach and A. G. Hannam. The role of passive muscletensions in a three-dimensional dynamic model of the human jaw.Archives of Oral Biology, 44(7):557{73, 1999. ! pages 45[105] R. J. Last. Anatomy Regional and Applied. J. & A. Churchill Ltd.,London, UK, 4th edition, 1966. ! pages 152135Bibliography[106] G. E. Loeb, I. E. Brown, and E. J. Cheng. A hierarchical foundationfor models of sensorimotor control. Experimental Brain Research,126(1):1{18, 1999. ! pages 38[107] J. A. Logemann. Role of the modi ed barium swallow in manage-ment of patients with dysphagia. Otolaryngology-Head and NeckSurgery, 116(3):335{338, 1997. ! pages 15[108] A. A. Lowe and W. D. Johnston. Tongue and jaw muscle activity inresponse to mandibular rotations in a sample of normal and anterioropen-bite subjects. American Journal of Orthodontics, 76(5):565{576, 1979. ! pages 52[109] C. L. Ludlow. Electrical neuromuscular stimulation in dysphagia:current status. Current Opinion in Otolaryngology & Head and NeckSurgery, 18(3):159|164, 2010. ! pages 64[110] D. S. Lundy, C. Smith, L. Colangelo, P. A. Sullivan, J. A. Loge-mann, C. L. Lazarus, L. A. Newman, T. Murry, L. Lombard, andJ. Gaziano. Aspiration: cause and implications. Otolaryngology-Headand Neck Surgery, 120(4):474{478, 1999. ! pages 3, 4[111] C. Lunk and B. Simeon. Solving constrained mechanical systems bythe family of newmark and  -methods. Journal of Applied Math-ematics and Mechanics (ZAMM), 86(10):772{784, 2006. ! pages174[112] Y. K. Mariappan, K. J. Glaser, and R. L. Ehman. Magnetic res-onance elastography: A review. Clinical Anatomy, 23(5):497{511,2010. ! pages 24[113] M. Marunick, B. Mathes, and B. Klein. Masticatory function inhemimandibulectomy patients. Journal of Oral Rehabilitation, 19(3):289{295, 1992. ! pages 96[114] R. E. Marx, J. E. Cillo Jr, V. Broumand, and J. J. Ulloa. Outcomeanalysis of mandibular condylar replacements in tumor and traumareconstruction: A prospective analysis of 131 cases with long-termfollow-up. Journal of Oral and Maxillofacial Surgery, 66(12):2515{2523, 2008. ! pages 95, 96[115] K. Matsuo and J. B. Palmer. Kinematic linkage of the tongue, jaw,and hyoid during eating and speech. Archives of Oral Biology, 55(4):325{331, 2010. ! pages 15136Bibliography[116] B. Mehta and S. Schaal. Forward models in visuomotor control.Journal of Neurophysiology, 88(2):942{953, 2002. ! pages 95[117] A. J. Miller. Craniomandibular muscles: their role in function andform. CRC-Press, Boca Raton, FL, 1991. ! pages 13, 85, 87, 96,115[118] A. J. Miller and D. J. Van Daele. The pharynx and scar tissue. InOPAL Workshop, 2009. URL http://www.magic.ubc.ca/artisynth/uploads/OPAL/Arthur Miller.pdf. ! pages 41[119] K. Miller, G. Joldes, D. Lance, and A. Wittek. Total Lagrangianexplicit dynamics  nite element algorithm for computing soft tissuedeformation. Communications in Numerical Methods in Engineering,23(2):121{134, 2007. ! pages 174[120] K. Miyawaki. A study on the musculature of the human tongue.Annual Bulletin RILP, University of Tokyo, 8:23{49, 1974. URLhttp://www.umin.ac.jp/memorial/rilp-tokyo/R08/R08 023.pdf. !pages 21[121] K. Miyawaki, H. Hirose, T. Ushijima, and M. Sawashima. A prelimi-nary report on the electromyographic study of the activity of lingualmuscles. Annual Bulletin RILP, University of Tokyo, 9:91{106, 1975.URL http://www.umin.ac.jp/memorial/rilp-tokyo/R09/R09 091.pdf.! pages 20[122] S. Moisik. A Three-Dimensional Model of the Larynx and the La-ryngeal Constrictor Mechanism: Visually Synthesizing Pharyngealand Epiglottal Articulations Observed in Laryngoscopy. PhD thesis,Department of Linguistics, University of Victoria, 2008. ! pages 34[123] W. Mollemans, F. Schutyser, N. Nadjmi, F. Maes, and P. Suetens.Predicting soft tissue deformations for a maxillofacial surgery plan-ning system: From computational strategies to a complete clinicalvalidation. Medical Image Analysis, 11(3):282{301, 2007. ! pages40[124] E. Moller. The chewing apparatus: an electromyographic study ofthe action of the muscles of mastication and its correlation to facialmorphology. Acta Physiologica Scandinavica, Supplementum, 280:1{229, 1966. ! pages 19, 52137Bibliography[125] K. Montgomery, C. Bruyns, J. Brown, S. Sorkin, F. Mazzella,G. Thonier, A. Tellier, B. Lerman, and A. Menon. Spring: A gen-eral framework for collaborative, real-time surgical simulation. InMedicine Meets Virtual Reality (MMVR ’02), pages 23{26, 2002. !pages 28[126] MSC Software Corporation, Santa Ana, California. ADAMS Multi-body Dynamics. http://www.mscsoftware.com, 2010. ! pages 25[127] M. M uller and M. Gross. Interactive virtual materials. Graphicsinterface (GI ’04), pages 239{246, 2004. ! pages 173[128] K. G. Munhall, M. Kawato, and E. Vatikiotis-Bateson. Coarticula-tion and physical models of speech production. In M. B. Broe andJ. B. Pierrehumbert, editors, Papers in Laboratory Phonology V: Ac-quistion and the Lexicon, chapter 1, pages 9{28. Cambridge Univer-sity Press, Cambridge, UK, 2000. ! pages 7, 44[129] G. M. Murray, T. Orfanos, J. Y. Chan, K. Wanigaratne, and I. J.Klineberg. Electromyographic activity of the human lateral ptery-goid muscle during contralateral and protrusive jaw movements.Archives of Oral Biology, 44(3):269{85, 1999. ! pages 20, 87[130] T. Muto and M. Kanazawa. Positional change of the hyoid bone atmaximal mouth opening. Oral Surgery, Oral Medicine, Oral Pathol-ogy, 77(5):451{455, 1994. ! pages 15, 51, 65[131] E. Muybridge. Muybridge’s Complete human and animal locomo-tion: all 781 plates from the 1887 Animal locomotion. Courier DoverPublications, 1979. ! pages 13[132] National Library of Medicine. Visible human dataset. http://www.nlm.nih.gov/research/visible/visible human.html, 2010. ! pages 21[133] M. Nazari, P. Perrier, M. Chabanas, and Y. Payan. Simulation ofdynamic orofacial movements using a constitutive law varying withmuscle activation. Computer Methods in Biomechanics and Biomedi-cal Engineering, 13(4):469{482, 2010. ! pages 34, 61[134] Northern Digital Inc., Waterloo, ON. Optotrak optical positiontracking system. http://www.ndigital.com, 2010. ! pages 14[135] Northern Digital Inc., Waterloo, ON. Wave electromagnetric posi-tion tracking system. http://www.ndigital.com, 2010. ! pages 17138Bibliography[136] J. P. Okeson. Management of Temporomandibular Disorder andOcclusion. Mosby, Saint Louis, MO, 5th edition, 2003. ! pages 152,155[137] T. Ono, K. Hori, and T. Nokubi. Pattern of tongue pressure on hardpalate during swallowing. Dysphagia, 19(4):259{264, 2004. ! pages18[138] T. Ono, K. Hori, K. Tamine, and Y. Maeda. Evaluation of tonguemotor biomechanics during swallowing { from oral feeding models toquantitative sensing methods. Japanese Dental Science Review, 45(2):65{74, 2009. ! pages 67[139] T. Orfanos, L. Sarinnaphakorn, G. M. Murray, and I. J. Klineberg.Placement and veri cation of recording electrodes in the superiorhead of the human lateral pterygoid muscle. Archives of Oral Biol-ogy, 41(5):493{503, 1996. ! pages 20, 87[140] D. K. Pai. Muscle mass in musculoskeletal models. Journal ofBiomechanics, 43(11):2093{2098, 2010. ! pages 31[141] W. R. Panje and D. K. Holmes. Mandibulectomy without recon-struction can cause sleep apnea. The Laryngoscope, 94(12):1591{1594, 2009. ! pages 5[142] V. Parthasarathy, J. L. Prince, M. Stone, E. Z. Murano, andM. NessAiver. Measuring tongue motion from tagged cine-MRI us-ing harmonic phase (HARP) processing. Journal of the AcousticalSociety of America, 121:491, 2007. ! pages 16, 90[143] A. Patel and R. Maisel. Condylar prostheses in head and neckcancer reconstruction. Archives of Otolaryngology-Head and NeckSurgery, 127(7):842, 2001. ! pages 95[144] Y. Payan and P. Perrier. Synthesis of VV sequences with a 2Dbiomechanical tongue model controlled by the Equilibrium PointHypothesis. Speech Communication, 22(2-3):185{205, 1997. ! pages6, 33, 57, 58[145] Y. Payan, P. Perrier, C. Vilain, and X. Pelorson. Finite elementmodels of the tongue and velum for a physical understanding ofsleep apnea syndrome. In Proceedings of Computer Methods inBiomechanics and Biomedical Engineering (BBE’01), page online,2001. ! pages 34139Bibliography[146] C. C. Peck, G. E. J. Langenbach, and A. G. Hannam. Dynamic sim-ulation of muscle and articular properties during human wide jawopening. Archives of Oral Biology, 45(11):963{982, 2000. ! pages32, 33, 45, 59[147] C. C. Peck, A. S. Sooch, and A. G. Hannam. Forces resisting jawdisplacement in relaxed humans: a predominantly viscous phe-nomenon. Journal of Oral Rehabilitation, 29(2):151{160, 2002. !pages 116[148] J. Perkell, M. Cohen, M. Svirsky, M. Matthies, I. Garabieta, andM. Jackson. Electromagnetic midsagittal articulometer systemsfor transducing speech articulatory movements. The Journal of theAcoustical Society of America, 92:3078, 1992. ! pages 17[149] J. S. Perkell. Physiology of speech production: Results and implica-tions of a quantitative cineradiographic study. Research MonographNo. 53. MIT Press, Cambridge, MA, 1969. ! pages 6, 15[150] J. S. Perkell. A physiologically-oriented model of tongue activity inspeech production. PhD thesis, Speech Communication Group, Mas-sachusetts Institute of Technology, 1974. ! pages 33[151] P. Perrier, Y. Payan, M. Zandipour, and J. Perkell. In uences oftongue biomechanics on speech movements during the production ofvelar stop consonants: A modeling study. Journal of the AcousticalSociety of America, 114(3):1582{1599, 2003. ! pages 33[152] D. H. Perrott, H. Umeda, and L. B. Kaban. Costochondral graftconstruction/reconstruction of the ramus/condyle unit: long-termfollow-up. International Journal of Oral and Maxillofacial Surgery,23(6):321{328, 1994. ! pages 95[153] G. J. Petruzzelli, K. Cunningham, and D. Vandevender. Impact ofmandibular condyle preservation on patterns of failure in head andneck cancer. Otolaryngology-Head and Neck Surgery, 137(5):717{721,2007. ! pages 95[154] L. J. Pittman and E. F. Bailey. Genioglossus and intrinsic elec-tromyographic activities in impeded and unimpeded protrusiontasks. Journal of Neurophysiology, 101(1):276, 2009. ! pages 20140Bibliography[155] Polhemus, Colchester, VT. Fastrak magnetic position tracking sys-tem. http://www.polhemus.com, 2010. URL http://www.polhemus.com. ! pages 17[156] J. Porrill, P. Dean, and J. V. Stone. Recurrent cerebellar architec-ture solves the motor-error problem. Proceedings of the Royal Soci-ety B: Biological Sciences, 271(1541):789{796, 2004. ! pages 36[157] F. A. Potra, M. Anitescu, B. Gavrea, and J. Trinkle. A linearly im-plicit trapezoidal method for integrating sti multibody dynamicswith contact, joints, and friction. International Journal for Numeri-cal Methods in Engineering, 66(7), 2006. ! pages 175, 184[158] J. L. Prince and J. M. Links. Medical Imaging Signals and Systems.Pearson Prentice Hall, Upper Saddle River, NJ, 2006. ! pages 16,22[159] O. Rastadmehr, T. Bressmann, R. Smyth, and J. C. Irish. Increasedmidsagittal tongue velocity as indication of articulatory compensa-tion in patients with lateral partial glossectomies. Head & Neck, 30(6):718{726, 2008. ! pages 17[160] O. R ohrle and A. J. Pullan. Three-dimensional  nite element mod-elling of muscle forces during mastication. Journal of Biomechanics,40(15):3363{3372, 2007. ! pages 31, 33[161] E. D. Roumanas, N. Garrett, K. E. Blackwell, E. Freymiller, E. Abe-mayor, W. K. Wong, J. Beumer, K. Fueki, W. Fueki, and K. K. Ka-pur. Masticatory and swallowing threshold performances with con-ventional and implant-supported prostheses after mandibular  bulafree- ap reconstruction. Journal of Prosthetic Dentistry, 96(4):289{297, 2006. ! pages 95[162] V. Sanguineti, R. Laboissi ere, and D. J. Ostry. A dynamic biome-chanical model for neural control of speech production. Journal ofthe Acoustical Society of America, 103(3):1615{1627, 1998. ! pages33, 34, 35, 36[163] O. Schenk and K. G artner. Solving unsymmetric sparse systemsof linear equations with PARDISO. Future Generation ComputerSystems, 20(3):475{487, 2004. ! pages 179141Bibliography[164] P. Schiavone, E. Promayon, and Y. Payan. LASTIC: A Light Aspi-ration Device for in vivo Soft TIssue Characterization. In BiomedicalSimulation (ISBMS ’10), volume Springer LNCS 5958, pages 1{10.Springer, 2010. ! pages 24[165] H. Schliephake, R. Schmelzeisen, R. Schnweiler, T. Schneller, andC. Altenbernd. Speech, deglutition and life quality after intraoraltumour resection: a prospective study. International Journal of Oraland Maxillofacial Surgery, 27(2):99 { 105, 1998. ! pages 95[166] R. Schmelzeisen, F. W. Neukam, T. Shirota, B. Specht, andM. Wichmann. Postoperative function after implant insertion invascularized bone grafts in maxilla and mandible. Plastic and Re-constructive Surgery, 97(4):719, 1996. ! pages 95, 96[167] W. Schmidt-Nowara, A. Lowe, L. Wiegand, R. Cartwright, F. Perez-Guerra, S. Menn, et al. Oral appliances for the treatment of snoringand obstructive sleep apnea: a review. Sleep, 18(6):501{510, 1995.! pages 52[168] W. S. Selbie, L. Zhang, W. S. Levine, and C. L. Ludlow. Using jointgeometry to determine the motion of the cricoarytenoid joint. Jour-nal of the Acoustical Society of America, 103:1115{1127, 1998. !pages 15[169] W. S. Selbie, S. L. Gewalt, and C. L. Ludlow. Developing ananatomical model of the human laryngeal cartilages from magneticresonance imaging. Journal of the Acoustical Society of America,112:1077, 2002. ! pages 23, 168[170] A. Serrurier and P. Badin. A three-dimensional articulatory modelof the velum and nasopharyngeal wall based on MRI and CT data.Journal of the Acoustical Society of America, 123:2335{2355, 2008.! pages 63[171] A. Seth and M. G. Pandy. A neuromusculoskeletal tracking methodfor estimating individual muscle forces in human movement. Journalof Biomechanics, 40(2):356{366, 2007. ! pages 39[172] A. A. Shabana. Dynamics of multibody systems. Cambridge Univer-sity Press, Cambridge, UK, 2005. ! pages 25142Bibliography[173] M. Sherif, R. Gregor, L. Liu, R. Roy, and C. Hager. Correlation ofmyoelectric activity and muscle force during selected cat treadmilllocomotion. Journal of Biomechanics, 16(9):691, 1983. ! pages 19,70[174] J. R. Shewchuk. What is a good linear element? interpolation, con-ditioning, and quality measures. In Eleventh International MeshingRoundtable, pages 115{126, online, 2002. Sandia National Laborato-ries. ! pages 28[175] D. M. Shiller, R. Laboissiere, and D. J. Ostry. Relationship betweenjaw sti ness and kinematic variability in speech. Journal of Neuro-physiology, 88(5):2329{2340, 2002. ! pages 36[176] D. M. Shiller, G. Houle, and D. J. Ostry. Voluntary control of hu-man jaw sti ness. Journal of Neurophysiology, 94(3):2207{2217,2005. ! pages 88, 116[177] T. Shinar, C. Schroeder, and R. Fedkiw. Two-way coupling of rigidand deformable bodies. In ACM SIGGRAPH/Eurographics Sympo-sium on Computer Animation (SCA ’08), 2008. ! pages 176, 184[178] H. Si. Tetgen: A quality tetrahedral mesh generator and a 3d delau-nay triangulator. http://tetgen.berlios.de/, 2009. ! pages 28[179] E. Sifakis, I. Neverov, and R. Fedkiw. Automatic determination offacial muscle activations from sparse motion capture marker data.ACM Transactions on Graphics, 24(3):417{425, July 2005. ! pages14, 34, 39, 70, 71, 73, 91[180] M. P. T. Silva and J. A. C. Ambrosio. Sensitivity of the resultsproduced by the inverse dynamic analysis of a human stride to per-turbed input data. Gait and Posture, 19(1):35{49, 2004. ! pages38, 91[181] S. I. Silverman. Oral Physiology. Mosby, Saint Louis, MO, 1961. !pages 166[182] B. C. Sonies, T. H. Shawker, T. E. Hall, L. H. Gerber, and S. B.Leighton. Ultrasonic visualization of tongue motion during speech.Journal of the Acoustical Society of America, 70:683, 1981. ! pages16143Bibliography[183] I. Stavness, A. G. Hannam, J. E. Lloyd, and S. Fels. An integrated,dynamic jaw and laryngeal model constructed from ct data. InBiomedical Simulation (ISBMS ’06), Springer LNCS, volume 4072,pages 169{177, 2006. ! pages 45, 50[184] I. Stavness, C. L. Ludlow, B. Chung, and S. Fels. Hyolaryngealbiomechanics modeling with intramuscular stimulation data. InOPAL Workshop, pages 73{78, 2009. ! pages iii[185] I. Stavness, A. G. Hannam, J. Lloyd, and S. Fels. Predicting musclepatterns for hemimandibulectomy models. Computer Methods inBiomechanics and Biomedical Engineering, 13(4):483{491, 2010. !pages iii[186] I. Stavness, J. Lloyd, Y. Payan, and S. Fels. Coupled hard-soft tissuesimulation with contact and constraints applied to jaw-tongue-hyoiddynamics. International Journal of Numerical Methods in Biomedi-cal Engineering, in press, 2010. ! pages iii, 172[187] M. Stone and A. Lundberg. Three-dimensional tongue surfaceshapes of English consonants and vowels. Journal of the AcousticalSociety of America, 99(6):3728{3737, 1996. ! pages 16[188] M. Stone and E. Vatikiotis-Bateson. Trade-o s in tongue, jaw, andpalatecontributions to speech production. Journal of Phonetics, 23(1-2):81{100, 1995. ! pages 18[189] S. Sueda, A. Kaufman, and D. K. Pai. Musculotendon simulation forhand animation. In ACM SIGGRAPH Papers, pages 1{8, 2008. !pages 30, 39, 70, 71, 73[190] S. Takano and K. Honda. An MRI analysis of the extrinsic tonguemuscles during vowel production. Speech Communication, 49(1):49{58, 2007. ! pages 16, 23, 162, 163[191] S. Takano, K. Honda, and K. Kinoshita. Observation of cricothyroidjoint motion using 3d high-resolution mri. In Voice Physiology andBiomechanics (ICVPB ’04), page online, 2004. ! pages 16[192] H. Takemoto. Morphological analyses of the human tongue muscu-lature for three-dimensional modeling. Journal of Speech, Language,and Hearing Research, 44(1):95, 2001. ! pages 21, 162144Bibliography[193] J. Teran, E. Sifakis, S. Blemker, V. Ng-Thow-Hing, C. Lau, andR. Fedkiw. Creating and simulating skeletal muscle from the visiblehuman data set. IEEE Transactions on Visualization and ComputerGraphics, 11(3):317{28, 2005. ! pages 30[194] D. Terzopoulos and K. Waters. Analysis and synthesis of facial im-age sequences using physical andanatomical models. IEEE Transac-tions on Pattern Analysis and Machine Intelligence, 15(6):569{579,1993. ! pages 34[195] E. Teubner, D. Rohner, A. Deak, A. Lorenzon, and C. Marinello.Navigated implant placement using a bone supported CT-guidedsurgical template: a case report. Schweizer Monatsschrift fur Zahn-medizin, 119(12):1211{1234, 2009. ! pages 40[196] D. G. Thelen and F. C. Anderson. Using computed muscle controlto generate forward dynamic simulations of human walking fromexperimental data. Journal of Biomechanics, 39(6):1107{1115, 2006.! pages 39, 70, 92[197] E. Todorov and M. I. Jordan. Optimal feedback control as a theoryof motor coordination. Nature Neuroscience, 5(11):1226{1235, 2002.! pages 38[198] E. Todorov, W. Li, and X. Pan. From task parameters to motorsynergies: A hierarchical framework for approximately optimal con-trol of redundant manipulators. Journal of Robotic Systems, 22(11):691{710, 2005. ! pages 38[199] M. Tuijta, J. H. Koolstraa, F. Lobbezoob, and M. Naeijeb. Di er-ences in loading of the temporomandibular joint during opening andclosing of the jaw. Journal of Biomechanics, 43(6):1048{1054, 2010.! pages 33[200] M. L. Urken, D. Buchbinder, H. Weinberg, C. Vickery, A. Sheiner,R. Parker, J. Schaefer, P. Som, A. Shapiro, W. Lawson, et al. Func-tional evaluation following microvascular oromandibular reconstruc-tion of the oral cancer patient: a comparative study of reconstructedand nonreconstructed patients. The Laryngoscope, 101(9), 1991. !pages 96[201] Y. Utanohara, R. Hayashi, M. Yoshikawa, M. Yoshida, K. Tsuga,and Y. Akagawa. Standard values of maximum tongue pressure145Bibliographytaken using newly developed disposable tongue pressure measure-ment device. Dysphagia, 23(3):286{290, 2008. ! pages 18, 59[202] D. Van Daele, A. Perlman, and M. Cassell. Intrinsic  bre architec-ture and attachments of the human epiglottis and their contribu-tions to the mechanism of deglutition. Journal of Anatomy, 186(Pt1):1, 1995. ! pages 63, 169[203] T. Van Eijden, J. Koolstra, and P. Brugman. Architecture of thehuman pterygoid muscles. Journal of Dental Research, 74(8):1489,1995. ! pages 21, 32[204] T. Van Eijden, J. H. Koolstra, and P. Brugman. Three-dimensionalstructure of the human temporalis muscle. The Anatomical Record,246(4):565{572, 1996. ! pages[205] T. Van Eijden, J. Korfage, and P. Brugman. Architecture of the hu-man jaw-closing and jaw-opening muscles. The Anatomical Record,248(3):464{474, 1997. ! pages 21, 31, 32, 59, 160[206] E. Vatikiotis-Bateson and D. J. Ostry. An analysis of the dimen-sionality of jaw motion in speech. Journal of Phonetics, 23:101{117,1995. ! pages 14[207] Vicon Motion Systems, Los Angeles, CA. Vicon MX. http://www.vicon.com, 2010. URL http://www.vicon.com. ! pages 14[208] Visual Computing Lab of the Italian National Research Council.Meshlab. http://meshlab.sourceforge.net/, 2010. ! pages 26[209] N. Wakabayashi, M. Ona, T. Suzuki, and Y. Igarashi. Nonlinear nite element analyses: Advances and challenges in dental applica-tions. Journal of dentistry, 36(7):463{471, 2008. ! pages 40[210] A. Waltimo and M. Kononen. A novel bite force recorder and max-imal isometric bite force values for healthy young adults. EuropeanJournal of Oral Sciences, 101(3):171{175, 1993. ! pages 19, 111,116[211] J. Weiss and S. Maas. FEBio - a  nite element tool for biomechan-ics, 2007. URL http://imechanica.org/node/1897. ! pages 28[212] J. A. Weiss, B. N. Maker, and S. Govindjee. Finite element imple-mentation of incompressible, transversely isotropic hyperelasticity.146Computer Methods in Applied Mechanics and Engineering, 135(1-2):107{128, 1996. ! pages 31, 67[213] A. Woda, P. Pionchon, and S. Palla. Regulation of mandibular pos-tures: mechanisms and clinical implications. Critical Reviews in OralBiology & Medicine, 12(2):166, 2001. ! pages 104[214] W. W. Wood, A. G. Hannam, and K. Takada. The electromyo-graphic activity of the inferior part of the human lateral pterygoidmuscle during clenching and chewing. Archives of Oral Biology, 31(4):245{253, 1986. ! pages 19[215] G. T. Yamaguchi, D. W. Moran, and J. Si. A computationally ef- cient method for solving the redundant problem in biomechanics.Journal of Biomechanics, 28(8):999{1005, 1995. ! pages 37[216] F. Zajac. Muscle and tendon: properties, models, scaling, and ap-plication to biomechanics and motor control. Critical Reviews inBiomedical Engineering, 17(4):359, 1989. ! pages 32, 71, 91[217] F. Zhang, C. C. Peck, and A. G. Hannam. Mass properties of thehuman mandible. Journal of Biomechanics, 35(7):975{978, 2002. !pages 26, 27, 31147Appendix AList of PublicationsMany of the contributions and ideas described in this dissertation have beenpreviously presented in publications and oral presentations, which are listedhere.A.1 Journal PublicationsIan Stavness, John Lloyd, Yohan Payan, and Sidney Fels. Coupled hard-soft tissue simulation with contact and constraints applied to jaw-tongue-hyoid dynamics. International Journal of Numerical Methodsin Biomedical Engineering, in press, 2010.Ian Stavness, Alan Hannam, John Lloyd, and Sidney Fels. Predictingmuscle patterns for hemimandibulectomy models. Computer Methodsin Biomechanics and Biomedical Engineering, 13(4):483{491, 2010.Alan G. Hannam, Ian Stavness, John E. Lloyd, Sidney Fels, Don Curtis,and Art Miller. A comparison of simulated jaw dynamics in mod-els of segmental mandibular resection versus resection with alloplasticreconstruction. Journal of Prosthetic Dentistry, 104(3):191{198, 2010.Sandro Palla, W.L. Hylander, A.S. McMillan, E.W.N. Lam, M. Watanabe,G.E.J. Langenbach, I. Stavness, C.C. Peck, and A.G. Hannam. Frommovement to models: a tribute to Professor Alan Hannam. Journalof Orofacial Pain, 22(4):307{316, 2008.148A.2. Conference PublicationsA.2 Conference PublicationsIan Stavness, John Lloyd, Yohan Payan, and Sidney Fels. Dynamichard-soft tissue models for orofacial biomechanics. In ACM SiggraphTalks, 2010.Ian Stavness, Christy Ludlow, Bethany Chung, and Sidney Fels. Hyola-ryngeal biomechanics modeling with intramuscular stimulation data.In OPAL Workshop, pages 73{78, 2009.Sidney Fels, Ian Stavness, and others. Advanced Tools for Biomechan-ical Modeling of the Oral, Pharyngeal, and Laryngeal Complex. InProc of 4th International Symposium on Biomechanics, Healthcare andInformation Science (ISBHIS ’09), 2009.Ian Stavness, Alan G. Hannam, John E. Lloyd, and Sidney Fels. Towardspredicting biomechanical consequences of jaw reconstruction. In Procof 30th IEEE Engineering in Medicine and Biology Conference (EMBC’08), pages 4567{4570, 2008.Ian Stavness, Alan G. Hannam, John E. Lloyd, Eric Vatikiotis-Bateson,and Sidney Fels. Tools for predicting biomechanical consequences ofalterations to orofacial anatomy. In Proc of 3rd International Sympo-sium on Biomechanics, Healthcare and Information Science (ISBHIS’08), 2008.Sidney S. Fels, John E. Lloyd, Ian Stavness, Alan Hannam, and EricVatikiotis-Bateson. ArtiSynth: A 3d biomechanical simulation toolkitfor modeling anatomical structures. In Journal of the Society forSimulation in Healthcare, volume 2, page 148, 2007.A.3 Research VisitsOctober-November 2009 with Dr. Chris Peck and Dr. Greg MurrayJaw Function Research Unit, Westmead Hospital, University of Syd-ney, Australia149A.4. Research TalksOctober 2008 with Dr. Christy LudlowLaryngeal and Speech Section, The National Institutes of HealthBethesda, MDA.4 Research TalksOctober 2009 { Upper Airway Biomechanics Modeling Sleep Research Meeting, Woolcock Institute, Sydney, Australia Prince of Wales Medical Research Institute, Sydney, Australia MARCS Auditory Lab, University of Western Sydney, AustraliaJuly 2008 { Dynamic Jaw and Tongue Modeling with ArtiSynth Orofacial Pain and Motor Control: The History of Three Giants inOrofacial Neurosciences Symposium, TorontoJune 2008 { Biomechanics and Motor Control Simulation in ArtiSynth Department of Biomedical Sciences, University of Maryland Physical Medicine and Rehabilitation, John Hopkins Hospital Laryngeal and Speech Section, The National Institutes of HealthMay 2007 { Biomechanical Modeling of Jaw-Hyoid Anatomy with ArtiSynth Language, Mind and Brain Centre, McGill University, MontrealA.5 Master’s PublicationsAlan G. Hannam, Ian Stavness, John E. Lloyd, and Sidney Fels. A dy-namic model of jaw and hyoid biomechanics during chewing. Journalof Biomechanics, 41(5):1069{1076, 2008.Ian Stavness, Alan G. Hannam, John E. Lloyd, and Sidney Fels. An inte-grated, dynamic jaw and laryngeal model constructed from ct data. InProceedings of the International Symposium on Biomedical Simulation(ISBMS 06), Springer LNCS, 4072:169177, 2006.Sidney Fels, John Lloyd, Kees van den Doel, Ian Stavness, and EricVatikiotis-Bateson. Developing physically-based, dynamic vocal tract150A.6. Additional Publicationsmodels using ArtiSynth. In Proceedings of the 7th International Sem-inar on Speech Production (ISSP 06), pages 419426, 2006.A.6 Additional PublicationsIan Stavness, Billy Lam, and Sidney Fels. pCubee: A perspective-corrected handheld cubic display. In ACM CHI ’10: Proceedings ofthe SIGCHI conference on human factors in computing systems, pages1381{1390, 2010.Billy Lam, Ian Stavness, Ryan Barr, and Sidney Fels. Interacting with apersonal cubic 3d display. In ACM Multimedia Technical Demonstra-tions, pages 959{960, 2009.Ian Stavness and Sidney Fels. Cubee: thinking inside the box. In ACMSiggraph Emerging Technologies, page 5, 2006.Ian Stavness and Sidney Fels. Cubee: a cubic 3D display for physics-based interaction. In ACM Siggraph Sketches, page 165, 2006.Ian Stavness, Jen Gluck, Leah Vilhan, and Sidney Fels. The MU-SICtable: a map-based ubiquitous system for social interaction with adigital music collection. In Proceedings of International Conference inEntertainment Computing (ICEC 05), Springer LNCS, (3711):291302,2005.151Appendix BHead and Neck AnatomyThis appendix provides an overview of head and neck anatomy focusing ondescriptions of jaw, tongue, and laryngeal muscles. More detailed anatomicaldescriptions can be found elsewhere, including Drake et al. [47], Okeson[136], and Last [105]. Figure B.1 provides a mid-sagittal cut-away view of theupper airway. In this appendix, we describe the bony structures of the head,including the cranium, mandible, teeth, hyoid bone, and vertebrae, as wellas soft-tissue structures, including the face, tongue, soft-palate, pharynx,larynx (thyroid, cricoid, arytenoid cartilages), and epiglottis. We reviewmuscles of the jaw, hyoid, tongue, and upper airway that are relevant to themodeling work described in this dissertation.B.1 Bone StructuresThe head and neck skeleton includes the skull (cranium and mandible),hyoid bone, and vertebrae.B.1.1 CraniumThe cranium is composed of a large number of fused bones. The frontal (fa-cial) areas of the cranium includes the frontal, nasal, zygomatic bones, andmaxilla. The top and back areas of the skull are formed by the parietal andoccipital bones respectively. The temporal bones form the lower side regionsof the skull and include the mastoid, styloid, and zygomatic processes.Maxilla The maxilla forms the upper jaw and is fused with the surround-ing bony structure of the cranium. The maxilla extends from the  oor of the152B.1. Bone StructuresTongueLipsMandibleHyoid boneEpiglottisTracheaThyroid cartilageCricoid cartilageVertebraeSoft palateHard palateVocal foldsFigure B.1: Sagittal cross-section of the upper airway showing the skull,lips, tongue, palate, pharynx, and larynx. c Elsevier (2010), Drake et al.[47], adapted with permission.ParietalTemporalMandibleMaxillaFrontalOccipitalMastoid ProcessStyloid ProcessZygomatic ProcessZygomaticNasalFigure B.2: Lateral view of the of the skull showing the mandible and thefused bones of the cranium. c Elsevier (2010), Drake et al. [47], adaptedwith permission.153B.1. Bone StructuresMylohyoid lineBodyRamusCondylar headCondylar neckCondylar processCoronoid processFigure B.3: Medial and lateral views of the mandible showing and the as-cending ramus forming the coronoid and condylar processes. c Elsevier(2010), Drake et al. [47], adapted with permission.nasal cavity and lower border of the orbit to the hard palate and alveolarprocess of the upper dental arch (see Figure B.2).Temporal bone The temporal bone serves as the articular surface for themandibular condyle. The posterior area of the temporal bone is concave andforms the articular fossa (colloquially referred to as the \jaw joint socket"due to its concave shape). Anterior to the fossa is the articular eminence, aconvex bony process that is the pathway for the condyle during jaw openingand protrusion (see Figure B.15). The articular eminence is designed tosustain large joint loads as it consists of thick, dense bone.B.1.2 MandibleThe mandible, pictured in Figure B.3, is a u-shaped bone connected to theskull through muscles, ligaments, and contact at the TMJ. The frontal arch-shaped portion, or body of the mandible, forms the alveolar process for thelower teeth. The posterior portion of the mandible extends upward into theascending ramus that forms two processes. The anterior coronoid process attens mediolaterally and serves as the insertion site for the temporalis154B.1. Bone StructuresTooth crownAlveolarprocessTooth rootGingivaPeriodontalligamentFigure B.4: A tooth seated within the bony alveolar process and connectedby the periodontal ligament. c Elsevier (2003), Okeson [136] adapted withpermission.CaninesPremolarsMolarsCaninesPremolarsMolarsIncisorsIncisorsFigure B.5: The lower and upper dentition showing the molor, premolar,canine, and incisal teeth. c Elsevier (2010), Drake et al. [47], adapted withpermission.muscle, extending the mechanical advantage of this muscle for generatingclosing torque on the jaw. The posterior process of the ascending ramus,which has a convex shape, is termed the \condyle" and articulates with thecranium at the TMJ.B.1.3 DentitionThe human dentition consists of an upper and lower arch each containingsixteen teeth, as shown in Figure B.5. The tooth body is divided intoan exposed crown and an internal root that anchors it to the bone. The155B.2. Soft-Tissues and Musclestooth root sits within bony sockets called alveolar processes in the mandibleand maxilla and are attached by the periodontal ligament, as shown inFigure B.4. The periodontal support  bers provide force distribution andshock absorption for the teeth, as well as sensitive nerve endings to detecttooth loads. The upper dental arch is slightly wider and longer than its lowercounterpart in order for tooth crowns to  t together during intercuspation.B.1.4 Hyoid boneThe hyoid bone is a  oating u-shaped bone located in the neck betweenthe jaw and larynx. It is connected above to the mandible and skull by thesubmandibular muscles and below to the thyroid cartilage by the hyothyroidmembrane along its length, as shown in Figure B.12. The hyoid serves asan anchor for the posterior muscle  bers of the tongue.B.1.5 VertebraeThe vertebrae are the ringed bones that enclose and protect the spinal cord.They extending in a chain from the base of the cranium to the pelvis. Theupper section, named the cervical vertebrae (C1 thru C7), form the skele-ton of the neck and provide support to skull. Functionally, each vertebraarticulates in sequence to enable movement of the head. The vertebrae alsoprovide convenient anatomical landmarks as their dense structure is easilyvisible in x-ray and CT images (see Figure 2.2).B.2 Soft-Tissues and MusclesSoft-tissues in the upper airway surround the skeletal structures and gener-ate forces that articulate jaw and vocal tract. The upper airway soft-tissuesare shown in Figure B.1 and include cartilage, muscles, ligaments, tendons,fascia (connective tissue), fat, mucous membrane that lines the inner surfaceof upper airway, and skin that covers the outer surface of the face.156B.2. Soft-Tissues and MusclesFigure B.6: Lateral view of the head showing the facial muscles. c Elsevier(2010), Drake et al. [47], adapted with permission.B.2.1 FaceThe super cial muscles of the face are shown in Figure B.6. The cheeksand lips are important for shaping the upper airway. During masticationand swallowing the lips seal o the oral cavity and the buccinator musclein the cheek works with the tongue to form and position the bolus duringmastication. Lip shape and movement is also critical to the acoustics andvisual appearance of speech production. The face includes a number of thinmuscles located within the super cial fascia layer of facial tissue. Facialmuscles originate from the skull or fascia and insert into the skin in orderto deform the surface of the face and control facial expressions. All facialmuscles are innervated by the facial nerve [cranial nerve VII].157B.2. Soft-Tissues and MusclesTemporalisDeep MasseterSuperficial MasseterMedial PterygoidFigure B.7: Jaw closing muscles include the masseter, temporalis, and me-dial pterygoid muscles. c Elsevier (2010), Drake et al. [47], adapted withpermission.B.2.2 Jaw musclesThe jaw muscles produce forces that move the jaw and generate tooth forcesduring chewing and clenching. The submandibular muscles are capable ofopening the jaw and lifting the laryngeal complex during swallowing. Thejaw muscles also stabilizes the mandible and prevents extreme jaw displace-ments through passive tension generated by muscle stretch.Jaw closing muscles Jaw closing muscles include the masseter, tempo-ralis, and medial pterygoid muscles and are pictured in Figure B.7. Eachmuscle has a large attachment area and can be further subdivided based onmuscle  ber direction [68]. The masseter muscle is typically divided into su-per cial and deep parts; however, the muscle is further compartmentalizedinto layers of muscle sheets with di erent  ber angles. The masseter muscleoriginates along the length of the zygomatic process. The super cial  bersinsert into the lateral lower portion of the ramus with a forward angle, whilethe deep  bers run mostly vertical, inserting into the lateral upper two-158B.2. Soft-Tissues and MusclesAnterior DigastricPosteriorDigastricUpper Head of Lateral PterygoidLower Head of Lateral PterygoidStylohyoidMedial PterygoidMylohyoidPosterior DigastricAnteriorDigastricFigure B.8: Jaw opening muscles include the upper and lower heads of thelateral pterygoid muscles and the anterior and posterior belly of the digastricmuscle. c Elsevier (2010), Drake et al. [47], adapted with permission.styloidthirds of the ramus. The fan-shaped temporalis muscle originates from alarge area on the lateral side of the skull and inserts in the coronoid processand the medial side of the ramus. The temporalis  bers run between thezygomatic process and the temporal bone. The muscle is typically groupedinto anterior  bers that are directed vertically and posterior  bers that areangled backward. The medial pterygoid muscle is a thick, heavily pennatedmuscle group originating from the pterygoid fossa and inserting into themedial lower part of the ramus. The jaw closing muscles are innervated bythe mandibular branch of the trigeminal nerve [V3].Jaw opening muscles The jaw opening muscles include the lateral ptery-goid and digastric muscles and are pictured in Figure B.8. The lateral ptery-goid muscle is divided into a superior and inferior heads. The inferior headoriginates from the outer surface of the lateral pterygoid plate while theupper head originates from the infratemporal surface of the sphenoid bone.Both heads insert onto the anterior neck of the condyle and the capsule ofthe TMJ (see Figure B.15. The lateral pterygoid is activated during jaw159B.2. Soft-Tissues and Musclesopening to cause forward protrusion. The digastric muscle is the primaryjaw opener causing the jaw to hinge downward when contracted. The an-terior belly of the digastric originates from the digastric fossa on the lowerborder of the mandible. The posterior belly originates from the mastoidnotch on the temporal bone. These muscle  bers connect and form an in-termediate tendon that is connected to the hyoid bone through a  brousloop, creating pulley-like mechanism. The  brous loop is also the insertionsite of the stylohyoid muscle, which originates from the styloid process ofthe temporal bone. The digastric and stylohyoid muscles are also shownin Figure B.14. The hyoid bone is also connected to the mandible by themylohyoid and geniohyoid muscles. The mylohyoid, lateral pterygoid, andanterior belly of the digastric muscles are innervated by the mandibularbranch of the trigeminal nerve [V3]. The posterior belly of the digastric andstylohyoid muscles are innervated by the facial nerve [VII].Jaw muscle architecture The micro-architecture of the jaw muscles hasbeen described by Van Eijden et al. [205] in detailed dissection studies. Thestudy found the jaw closing muscles to have a number of common archi-tectural characteristics, including short muscle  bers, large percentage oftendon tissue, large pennation angles, large cross sectional sizes, and rela-tively large mass, which are all indicative of physiological design for largeforce generation. In comparison to the closers, the jaw opening muscles werefound to have smaller cross sectional sizes, smaller percentage tendon tis-sue, smaller pennation angles, and longer  ber lengths. These physiologicalfeatures suggest the openers are designed for larger excursion and highershortening velocities. The macro-architecture of the jaw muscles has alsobeen described by Hannam and McMillan [68]. The study found the jawclosing muscles to have multiple muscle sheets of pennate  bers oriented atdi erent angles, which was suggested as a mechanism for maintaining biteforce throughout a range of jaw closing rotation.160B.2. Soft-Tissues and MusclesMandibleHyoglossusGeniohyoidHyoid bonePalatoglossusStyloglossusGenioglossusSuperior LongitudinalFigure B.9: Lateral and sagittal cross-section views of the tongue muscles.c Elsevier (2010), Drake et al. [47], adapted with permission.GenioglossusHyoglossusSuperiorLongitudinalMylohyoidStyloglossusInferior LongitudinalVertical &TransverseInferiorLongitudinalSeptumHyoid boneFigure B.10: Posterior views of tongue muscles with horizontal and verticalcut-away. c Elsevier (2010), Drake et al. [47], adapted with permission.161B.2. Soft-Tissues and MusclesB.2.3 TongueThe tongue is a large deformable organ and is the main articulator for chang-ing the shape of the oral cavity. It is composed of a number of intrinsicmuscles and is connected to surrounding orofacial structures through a setof extrinsic muscles. Tongue model descriptions have been reported fromcadaver studies [192] and recently from MRI analysis [190]. The tongueplays a role in mastication by breaking up food as well as forming and po-sitioning the food bolus between chewing strokes. During swallowing, thetongue presses upward against the palate and contracts in a wave-fashion topush the bolus backward to the oropharynx, as illustrated in Figure B.13.Extrinsic tongue muscles The genioglossus muscle accounts for the bulkof the tongue tissue (see Figure B.9). Its  bers originate from the mentalspine on the midline medial surface of the mandible, radiate widely in thetongue body, and insert into the mid-sagittal septum of the tongue from tipto base. The genioglossus causes tongue protrusion as the  bers pull thetongue body forward and downward.The mylohyoid is a broad, thin muscle that forms the \ oor of themouth," as shown in Figure B.10. It originates from the mylohyoid linealong the medial surface of the mandible and muscle  bers run mediolater-ally and insert on a midline raphe. Posterior mylohyoid  bers insert on theanterior surface of the hyoid bone.The geniohyoid muscle originates from the inferior mental spine on themidline medial surface of the mandible and runs backward, above the my-lohyoid, inserting on the anterior portion of the hyoid bone, as shown inFigure B.10. Geniohyoid activation pulls the hyoid forward and slightlyupward depending on the position of the jaw relative to the hyoid.The hyoglossus muscle originates along the length of each side of thehyoid and insert into the lateral body of the tongue. The hyoglossus  bersrun predominantly vertically with the posterior  bers angled slightly for-ward from the back of the hyoid toward the tongue. The  bers interdigitatewith the styloglossus muscle at the lateral extent of the tongue body. Hyo-162B.2. Soft-Tissues and Musclesglossus activation pulls the tongue downward and slightly backward as wellas causing the tongue body to  atten vertically. Also, if the tongue is heldforward in the oral cavity, hyoglossus activation will raise the hyoid withinthe neck.The tongue connects above to the soft-palate with the palatoglossus mus-cle and to the skull with the styloglossus muscle as shown in Figure B.9. Thepalatoglossus muscle is a thin muscle and primarily used to lower the soft-palate, however it does contribute to tongue retraction. The styloglossusmuscle is the main tongue retractor, originating from the styloid process,running downward and forward along the lateral extent of the tongue body,and inserting into the tongue tip. The styloglossus  bers are angled antero-posteriorly and their path changes with tongue deformation [190].Intrinsic tongue muscles The intrinsic tongue muscles originate andinsert within the tongue body and are arranged into muscle groups withroughly orthogonal directions, as shown in Figure B.10. The vertical andtransverse muscles include inferior-superior and mediolateral  bers respec-tively and are interdigitated throughout the tongue body. They cause verti-cal and transverse  attening of the tongue and are activated during tongueprotrusion in order to prevent lateral expansion of the tongue body, allow-ing the tip to protrude forward with genioglossus activation. The inferiorlongitudinal muscle has anteroposteriorly angled  bers that run along theinferior side of the tongue tip, through the tongue body, and interdigitatewith the styloglossus muscle. The superior longitudinal muscle also has an-teroposterior angled  bers, but is located only along the superior surfaceof the tongue body. The superior longitudinal muscle’s super cial locationprovides leverage for backward bending of tongue body and lifting of thetongue tip.Innervation The majority of tongue muscle are innervated by the hy-poglossal nerve [XII]. Exceptions include the palatoglossus muscle (vagusnerve [X], mylohyoid muscle (trigeminal nerve [V3]), and geniohyoid muscle(cervical nerve C1).163B.2. Soft-Tissues and MusclesPalatoglossusPalatopharyngeusSuperiorConstrictorTensor veli palatiniLevator veli palatiniPalatine TonsilUvulaFigure B.11: Posterior view of the soft-palate showing superior pharyngealconstrictor (green) and the soft-palate muscles (pink). c Elsevier (2010),Drake et al. [47], adapted with permission.B.2.4 Soft-palateThe soft-palate is a structure of muscle tissue forming the upper back por-tion of the throat between the oropharynx and nasopharyx. It has numerousextrinsic muscle connections to adjacent structures, as shown in Figure B.11,in order to permit its movement and deformation. It is structurally similarto the tongue, but smaller and with a less complex structure and range of de-formation. The soft-plate is connected above to the skull by the levator velipalatini muscle. The soft-palate is connected below to the tongue and phar-ynx by the palatoglossus and palatopharyngeous muscles. These musclesare visible looking into the mouth, as the palatoglossal and palatopharygealarches in front of and behind the palatine tonsils (see Figure B.11). Sti -ening of the soft-palate is achieved by mediolateral forces generated by thetensor veli palatini muscle, which originates above the soft-palate in theskull, runs downward, and bends at a right-angle to insert laterally intothe soft-palate body. The uvula is a midline portion of the soft-palate thathangs down in the back of the throat and it is sti ened and elevated by164B.2. Soft-Tissues and MusclesSuperior ConstrictorMiddle ConstrictorInferior ConstrictorSCMCICSCMCICHyoid boneCricoid cartilageThyroid cartilageHyoglossusGenioglossusEpiglottisFigure B.12: Oblique frontal and lateral view of the pharynx and larynx.Pharyngeal constrictors are shown: superior constrictor (SC), middle con-strictor (MC) and inferior constrictor (IC). The laryngeal cartilages andepiglottis are visible, along with a cut-away of the tongue root revealing thehyoglossus and genioglossus muscles. c Elsevier (2010), Drake et al. [47],adapted with permission.the small musculus uvulae muscle. The soft-palate muscles are innervatedby the vagus nerve [X], except for the tensor veli palatini muscle with ininnervated by the mandibular division of the trigeminal nerve [V3].B.2.5 PharynxThe pharynx is the muscle tissue that forms the back of the throat. Thepharyngeal constrictors are illustrated in Figure B.12 and consist of three at, overlapping cylindrical bands of muscle tissue originating from a mid-line pharyngeal raphe and wrapping laterally around the airway to insertinto anterior structures. The superior constrictor inserts into a pterygo-mandibluar raphe (contiguous with the buccinator muscle) and forms the165B.2. Soft-Tissues and MusclesFoodBolusTongueVertebrae Hard palateSoft palateHyoid boneLipsMandibleVocal foldsEpiglottisFigure B.13: Mid-sagittal cross-section of the oral cavity during swallowingshowing closed lips, upward displaced tongue, clenched jaw, and elevatinghyoid. c Elsevier (1961), Silverman [181], adapted with permission.lateral walls of the orophayrnx. The middle constrictor inserts along thehyoid bone and stylohyoid ligament. The inferior constrictor inserts alongthe oblique line of the thyroid cartilage. During swallow, the pharyngealconstrictors activate in sequence from top to bottom to transport the foodbolus from the oropharynx to the upper esophageal sphincter and into theesophagus. The pharyngeal constrictor muscles are innervated by the vagusnerve [X].Three longitudinal muscles originate above the phayrnx and insert intothe pharyngeal wall and are active in elevation of the pharynx and larynx.They include the stylopharyngeus, palatopharyngeus, and salpingopharyn-geus muscles and originate from the styloid process, soft-palate, and pharyn-gotympanic tubes respectively. The stylopharyngeus muscle is innervatedby the glossopharyngeal nerve [IX] while the palatopharyngeus and salpin-gopharyngeus muscles are innervated by the vagus nerve [X].B.2.6 LarynxThe larynx is formed by the thyroid, cricoid and arytenoid cartilages asshown in Figure B.12. The 3D shape of the laryngeal cartilages have been an-166B.2. Soft-Tissues and MusclesHyoid boneMylohyoidAnterior DigastricThyrohyoidSternothyroidOmohyoidSternohyoidThyroid cartilageCricoid cartilageStylohyoidPosterior DigastricFigure B.14: Frontal view of the submandibular and neck muscles. c Else-vier (2010), Drake et al. [47], adapted with permission.167B.2. Soft-Tissues and Musclesalyzed by measurements on cadaver specimens [50] and MRI analysis [169].The cricoid cartilage is the posterior border of the larynx and is locateddirectly superior to the trachea. The arytenoid cartilages are small pairedpyramidal structures that articulate along the superior posterior surface ofthe cricoid. Their pyramidal shape allow for the attachment of multiplemuscles and the vocal folds. The thyroid cartilage forms the anterior andsuperior border of the larynx and is open toward the posterior. The two sidesof the thyroid cartilage form an anterior prominence, known colloquially asthe \Adam’s apple" in men. The vocal and vestibule folds insert into theposterior surface of the thyroid cartilage. During swallowing the arytenoidcartilages approximate with the thyroid cartilage, and along with closure ofthe vocal and vestibular folds, seal o the airway from the oropharynx.Extrinsic laryngeal muscles Muscles connect the larynx above to thehyoid bone (thyrohyoid muscle) and below to the sternum (sternohyoid,omohyoid, and sternothyroid muscles), as shown in Figure B.14. Thesemuscles function to lower and stabilize the laryngeal complex. The thyro-hyoid muscle tends to approximate the hyoid bone and larynx; it will eitherraise the larynx or lower the hyoid bone depending on the relative activationof the muscle above the hyoid or below the larynx. The thyrohyoid muscle isinnervated by the anterior ramus of cervical nerve C1, and the sternohyoid,omohyoid, and sternothyroid muscles are innervated by the anterior rami ofcervical nerves C1 through C3.Intrinsic laryngeal muscles Internal to the larynx, an intricate arrange-ment of small muscles articulate the laryngeal cartilages and manipulate thevocal folds, which are attached between the arytenoid cartilages and the in-ner surface of the thyroid cartilage. The cricothyroid muscles are used toarticulate the thyroid cartilage relative to the cricoid cartilage, which ismostly rotation about the cricothyroid joint with a small amount of trans-lation. The cricoarytenoid, transverse, and oblique arytenoid muscles artic-ulate the arytenoid cartilages relative to the cricoid cartilage in a complexmotion with combined mediolateral translation and oblique rotation. The168B.2. Soft-Tissues and Musclesintrinsic laryngeal muscles are responsible for positioning and sti ening thevocal folds and controlling glottal opening and vocal fold vibration [90].The intrinsic laryngeal muscles are innervated by the vagus nerve [X]: thecricothyroid muscle from the external branch of superior laryngeal nerve andall others from the recurrent laryngeal branch.B.2.7 EpiglottisThe epiglottis is a leaf-shaped cartilage structure located posterior to thethyroid cartilage and hyoid bone and is pictured in Figure B.12. Its broadportion extends upward posterior to the base of the tongue and is easilydistinguishable on lateral medical images of the neck (see Figure 1.1). Theepiglottis functions in airway protection during swallowing by folding back-ward and downward over the arytenoid cartilages of the larynx. The timingand function of the epiglottis have been examined in VF studies and its in-ternal mechanics has also been investigated [202]; however the mechanismscausing downfolding are not completely understood. There are no explicitmuscle attached to the upper portion of the epiglottis that have su cientleverage to pull down the epiglottis and therefore downfolding has been sug-gested to be a passive mechanism [58]B.2.8 Temporomandibular jointThe TMJ is a compound joint composed of the mandibular condyle, tem-poral bone, and a deformable articular disc allowing for combined rotationand translation of the jaw. TMJ is pictured in Figure B.15 with the jaw atrest and during jaw protrusion. The TMJ is formed by the articular fossaand eminence of the temporal bone and the condyle of the mandible boneseparated by the articular disc. The disc is composed of  brous connectivetissue and is most dense anteriorly and medially, where the largest jointforces are transmitted. The jaw closing muscles keep the TMJ in compres-sion in normal jaw function and the articular disc is held in place primarilyby the concave shape of the articular fossa. A  brous membrane around thedisc and the capsular ligament also help to keep the disc in place. A number169B.2. Soft-Tissues and MusclesArticular discArticular eminenceArticular fossaLateral Pterygoid muscleJaw at rest Jaw protrusionFigure B.15: Lateral cut-away view of the temporomandibular joint at restand during jaw protrusion. c Elsevier (2010), Drake et al. [47], adaptedwith permission.SphenomandibularligamentStylomandibular ligamentLateral ligamentCapsuleFigure B.16: Lateral view of the TMJ capsule and mandibular ligaments.c Elsevier (2010), Drake et al. [47], adapted with permission.170B.2. Soft-Tissues and Musclesof other mandibular ligaments help to prevent distraction of the TMJ, asshown in Figure B.16. Some muscle  bers of the upper head of the lateralpterygoid muscle insert into the  brous membrane capsule of the articulardisc and pull the disc forward during jaw protrusion.The surfaces of the condyle, articular eminence, and disc are smooth andthe joint cavities lining the surfaces produce synovial  uid, which reducefriction during joint motion. The disc deforms to  t the irregular bonycontact surfaces in order to distribute forces evenly. Destructive forces orstructural joint changes can irreversibly change disc morphology and lead toTMJ disorder. Also, large unbalanced TMJ loading can cause dislocation ofthe articular disc in some cases.171Appendix CArtiSynth SimulationSoftware 2Our physical simulation system is embedded within the ArtiSynth platform(http://www.artisynth.org), an open-source Java-based biomechanical mod-eling toolkit developed at the University of British Columbia under thedirection of Dr. Sidney Fels. Originally designed for speech applications,ArtiSynth has evolved into a tool for physiological research (particularlyneuromotor control) and clinical treatment planning. I joined the ArtiSynthteam in January 2005 during my Master’s thesis during which time I devel-oped the reference jaw-hyoid-larynx model working with Dr. Alan Hannamunder the supervision of Dr. Sidney Fels. During my PhD, I have madea technical contribution to the ArtiSynth system by developing the inversesimulation tools described in Chapter 4. The modeling contributions de-scribed in Chapter 3 and Chapter 5 were performed using the ArtiSynthtoolkit and have served as a means to develop requirements for ArtiSynth’sfeatures and to evaluate ArtiSynth’s performance. Dr. John Lloyd is theprinciple designer and developer of the ArtiSynth platform.ArtiSynth models are generally created in Java code, using the packagesand classes of the ArtiSynth API. Graphical editing and model creation isalso supported. Applications to date have focused on the jaw and oral re-gion [71], but it is broadly applicable to biomechanical modeling in general.Key system features include (1) an architecture that supports extensive in-teractivity, including graphically based model editing and simulation control2A version of this section has been previously published in Stavness et al. [186]. Dr.Lloyd wrote the material for this section of the manuscript, which I have included in thisappendix as background on the relevant simulation techniques used in this dissertation.172C.1. Physical Simulation Framework(Figure 1.2), and (2) a physics engine that combines FEM and multibodycapabilities, with constraints and contact handling, as described below.C.1 Physical Simulation FrameworkArtiSynth models consist of a hierarchy of components, which include dy-namic components such as particles, FEM nodes, or rigid bodies, force e ec-tors such as point-to-point muscles (including Hill and other types), linearor nonlinear  nite elements, and constraints such as joints or collision spec-i ers. FEM capabilities include support for tetrahedral, hexahedral, andsome higher-order elements, along with linear, corotated linear [127], andsome hyperelastic materials including a 5-parameter Mooney-Rivlin mate-rial. We now describe the mathematical framework for the dynamic simu-lation of these models.Let q and u be the generalized positions and velocities of all the dynam-ical components, with _q related to u by _q =  u ( generally equals theidentity, except for components such as rigid bodies, where it maps angularvelocity onto the derivative of a unit quaternion). Let f(q;u;t) be the forceproduced by all the force e ector components (including the  nite elements),and let M be the (block-diagonal) composite mass matrix. By representingrigid body velocity and acceleration in body coordinates we can ensure thatM is constant. Newton’s second law then givesM _u = f(q;u;t): (C.1)In addition, bilateral and unilateral constraints give rise to locally linearconstraints on u of the formG(q)u = 0; N(q)u 0: (C.2)Bilateral constraints include rigid body joints, FEM incompressibility asso-ciated with the mixed u-P formulation [88], and point-surface constraints,while unilateral constraints include contact and joint limits. Constraints173C.1. Physical Simulation Frameworkgive rise to constraint forces (in the directions G(q)T and N(q)T) whichsupplement the forces of (Equation C.1) in order to enforce the constraintconditions. In addition, for unilateral constraints, we have a complemen-tarity condition in which Nu > 0 implies no constraint force, and a con-straint force implies Nu = 0. Any given constraint usually involves onlya few dynamic components and so G and N are generally sparse. Solvingthe equations of motion requires integrating (Equation C.1) together with(Equation C.2).The presence of deformable bodies generally makes this system sti ,implying the need for an implicit integrator to obtain e cient performance3.For the work described in this paper, we use a semi-implicit second-orderNewmark integrator [111], with  = 1=2 and  = 1=4 (also known as thetrapezoidal rule). Letting k index values at a particular time step, and hdenote the time step size, this leads to the update rulesuk+1 = uk + h2 ( _uk + _uk+1); qk+1 = qk + h2 ( kuk +  k+1uk+1); (C.3)subject toGk+1uk+1 = 0; Nk+1uk+1 0: (C.4)Since G and N tend to vary slowly between time steps we can approximate(Equation C.4) usingGkuk+1 = gk; Nkuk+1 nk; (C.5)where gk  h _Gkuk and nk  h _Nkuk. Likewise, we use the approxima-tion  k+1  k +h_ k. For _uk+1, recalling that M is constant, an estimateof the (unconstrained) value of _uk+1 can be obtained from _uk+1 M 1fk+1,with fk+1 approximated by the  rst-order Taylor seriesfk+1 fk + @fk@u  u +@fk@q  q:3With very soft tissue, it may sometimes be possible to use explicit methods [119],particularly if sti ness-proportional damping is excluded.174C.1. Physical Simulation FrameworkPlacing this into the expression for uk+1 in (Equation C.3), multiplying byM, noting that  q = h=2( kuk +  k+1uk+1) and  u = uk+1 uk, andincorporating the constraints (Equation C.5), we obtain the system0B@^Mk  GkT  NkTGk 0 0Nk 0 01CA0B@uk+1 z1CA+0B@ Muk h^fk gk nk1CA =0B@00w1CA;0 z?w 0: (C.6)where w is a slack variable,  and z give the average constraint impulsesover the time step, and^Mk M h2@fk@u  h24@fk@q  k+1 and ^fk fk 12@fk@u uk + h4@fk@q  kuk:The complementarity condition for unilateral constraints is enforced by 0 z ? w  0. A more detailed explanation of this formulation can be foundin [157].System (Equation C.6) is a mixed linear complementarity problem, asingle solve of which is required to determine uk+1 for each semi-implicitintegration step. Other types of integrators give rise to similar systems.A fully implicit integrator (not currently implemented in ArtiSynth) wouldrequire (Equation C.6) to be applied iteratively at each time step.For  nite element models, the localized sti ness and damping matricesare embedded within @fk=@q and @fk=@u, which means that for modelsdominated by FEM components ^M will have an FEM sparsity structure.ArtiSynth FEM models also use a lumped mass model, which ensures thatM is block diagonal and makes it easier to interconnect FEM models withmass-spring and rigid body components.C.1.1 Friction, damping, and stabilizationCoulomb (dry) friction can be included by extending (Equation C.6) toinclude either a linearized friction cone [11, 157] or a (more approximatebut easier to solve) box friction [103]. ArtiSynth currently implements box175C.1. Physical Simulation Frameworkfriction, and since the friction in our system tends to be quite small, weapply this as a post-hoc correction to uk+1 (in the manner of [177]), usinga simpli ed version of (Equation C.6), with M instead of ^M and extraconstraints added in the tangential directions at contact points.Di erent forms of viscous damping are available, including translationaland rotary damping applied directly to particles and rigid bodies, and damp-ing terms embedded in point-to-point springs and muscle actuators. ForFEM models, Rayleigh damping is available, which takes the formDF =  MF + KFwhere MF is the portion of the (lumped) mass matrix associated with theFEM nodes and KF is the (instantaneous) FEM sti ness matrix. DF isthen embedded within the overall system matrix @f=@u.In addition to solving for velocities, it is also necessary to correct posi-tions to account for drift from the constraints, including interpenetrationsarising from contact. This can be done at each time step using a modi edform of (Equation C.6) which computes an impulse  q that corrects thepositions while honoring the constraints:0B@^Mk  GkT  NkTGk 0 0Nk 0 01CA0B@ q z1CA+0B@0 g n1CA =0B@00w1CA;0 z?w 0; (C.7)where  g and  n are the constraint displacements that must be corrected.If the corrections are su ciently small, it is often permissible to use M inplace of ^Mk, which improves solution e ciency since M is constant andblock-diagonal.While such stabilization can sometimes be incorporated directly into(Equation C.6) [10], we prefer to perform the position correction separatelyas this (a) allows for the possibility of an iterative correction in the case oflarger errors, and (b) explicitly separates the computed velocities from the176C.1. Physical Simulation Framework(a) (b) (c) (d)Figure C.1: Problems with decoupling constraints from the velocity solve:In (a), a uniform 3x6x3 FEM grid of linear material with a Poisson ratioof 0 is about to be compressed by a block. The decoupled solve causes thetop contacting nodes to bunch up on the surface (b), completely squashingthe top two element layers, while the lower nodes hardly move at all; thecoupled solve (c) causes the correct uniform displacement for all nodes. In(d), a decoupled solve causes a tongue model attached to a jaw to exhibitlarge vertical errors when the jaw clenches upwards against the bite plane.impulses used to correct errors.C.1.2 System solution and complexityFor notational convenience, in this section we will drop the k superscriptsfrom ^M, G, N, g, n, and ^f in (Equation C.6) and assume that these quan-tities are all evaluated at time step k.System (Equation C.6) is a large, sparse mixed linear complementar-ity problem [38] that is not particularly easy to solve, given the unilateralconstraints and the fact that ^M is not block diagonal. If ^M is symmet-ric positive de nite (SPD), it is equivalent to a convex quadratic program.If there are no unilateral constraints (N = ;), then it reduces to a linearKarush-Kuhn-Tucker (KKT) system.Generally, ^M is symmetric (unsymmetric terms sometimes arise from ro-tational e ects but these are usually small enough to ignore) and hence willalso be SPD for small enough h(since M is SPD). However, the resulting sys-tem is still harder to solve than non-sti multibody systems where ^M = M.This is because ^M, while still sparse, is not block-diagonal. Multi-body177C.1. Physical Simulation Frameworksystems are often solved using the projected Gauss-Seidel method [103].However, this involves a sequence of iterations, each requiring the computa-tion of Gi ^M 1GTi or Ni ^M 1NTi , which is easy to do for a block-diagonalM but much more costly for ^M.It is tempting to follow the approach we use for friction and decouplethe velocity and constraint solves, by  rst computing u = ^M 1(Muk +h^f)and then applying constraints to u in a post-hoc fashion, using a version of(Equation C.6) in which ^M is replaced with M. This can be done by variousmethods, including Gauss-Seidel iteration, and is equivalent to projectingu onto the space of legal velocities. Unfortunately, this does not propagateconstraint e ects properly throughout the system, and can result in verylarge errors when the constraint forces are large, as illustrated in FigureFigure C.1.At present, ArtiSynth solves (Equation C.6) by using a Schur comple-ment to turn it into a dense regular linear complementarity problem NA 1  NTz +  NA 1b n = w0 z?w 0 (C.8)whereA  ^M  GTG 0!;  N  N 0 ; b  Muk +h^fg!which is solved using Keller’s algorithm [103]. uk+1 and  can then beobtained using back-substitution: uk+1 != A 1 b +  NTz : (C.9)Keller’s algorithm is a pivoting method with an expected complexity ofO(m3), wheremis the number of unilateral constraints. In addition, forming(Equation C.8) and the back solve of (Equation C.9) requires m + 1 solvesof a system involving A. This is done using the Pardiso sparse direct solver178C.1. Physical Simulation Framework102 103 104100102104number of nodessolve time (ms)Figure C.2: Log-log plot showing factor times for A as a function of thenumber of nodes (which is proportional to the size of A) for a series of 3DFEM problems with a uniformly increasing node density. The slope of theline indicates a complexity of O(n1:7).[163], and entails a once-per-step factoring of A, plus m+1 solve operations.Experimentally, we have determined that the complexity of factoring A(using Pardiso) for 3D FEM type problems is roughly O(n1:7), where n isthe size of A (Figure Figure C.2). Similarly, we have also determined thatthe complexity of solving a factored A is roughly O(n1:3). Hence we canexpect the overall complexity for solving (Equation C.6) to beO(m3) +mO(n1:3) +O(n1:7):This works well provided that the number of unilateral constraints m issmall. To help achieve this, we can sometimes treat the unilateral constraintsarising from contact as bilateral constraints (i.e., entries in G) on a per-stepbasis, as described further in Section C.1.4.C.1.3 Attachments between bodiesIn creating comprehensive anatomical models, it is often necessary to attachvarious bodies together. Most typically, this is done by connecting points ofone body to speci c locations on another body. For example, FEM nodes179C.1. Physical Simulation Frameworkmay be attached to particular spots on a rigid body, or to other nodes of adi erent FEM model.To facilitate this, ArtiSynth provides the ability to attach a dynamiccomponent to one or more master components. Let the set of attachedcomponents be denoted by  , and the remaining set of unattached activecomponents be denoted by  . In general, the velocity uj of an attachedcomponent is related to the velocities u of the active components by alocally linear velocity constraint of the formuj + Gj u = 0:Gj will be sparse except for entries corresponding to the master compo-nents to which j is attached. Letting G  denote the composite matrixformed from Gj for all attached components, we haveIu + G  u = 0for the constraints that enforce all attachments.We could simply add these constraints to (Equation C.6) and solve theresulting system, but this would increase both the system size and solutiontime. Instead, we use the attachments to actually reduce the size of (Equa-tion C.6). Consider  rst the subsystem involving only bilateral constraints.As in Section C.1.2, we drop the k superscripts from ^M, G, N, g, n, and^f in (Equation C.6) and assume that these quantities are all evaluated attime step k. Letting b Muk +h^f and partitioning the system into activeand attached components yields0BBBB@^M  ^M  GT  GT  ^M  ^M  GT  IG  G  0 0G  I 0 01CCCCA0BBBB@uk+1 uk+1     1CCCCA=0BBBB@b b g 01CCCCA:180C.1. Physical Simulation FrameworkThe identity submatrices make it easy to solve for uk+1 and   :uk+1 = G  uk+1 ;   = b  ^M  uk+1 + ^M  G  uk+1  GT    and hence reduce the system to ^M0 G0TG0 0! uk+1   != b0g !(C.10)where^M0 P ^MPT; G0 GPT; b0 Pb; with P  I  GT   :Similarly, unilateral constraints can be reduced via N0 = NPT. Thereduction operation can be performed in O(n) time and results in a systemthat is less sparse but generally faster to solve than the original.Most attachments in ArtiSynth are point-based, with the most commonkind being the attachment of an FEM node to a rigid body. It is alsopossible to attach FEM nodes to the faces and edges of an FEM element,allowing us to handle the so-called \T-junction" problem and create FEMmodels with non-conforming element faces. This is quite useful for creatinglocalized subdivisions of particular elements, particularly hexahedrons.C.1.4 Contact handlingCollision detection can be enabled between any combination of rigid or de-formable bodies. It is assumed that the bodies in question contain a triangu-lar surface mesh. A bounding-box hierarchy is used to determine if any twosurfaces meshes intersect. If they do, then a tracing algorithm (similar to[3]) is used to compute all the intersection contours between the two meshesas shown in Figure Figure C.3. Such contour tracing can be done relativelyquickly but does require the use of robust geometry predicates similar tothose in [51]; this is particularly true because collision conditions tend todrive the contacting surfaces into degenerate mesh con gurations.Determining the intersection contour allows us to easily create a set of181C.1. Physical Simulation FrameworkFigure C.3: Contact handling between two deformable models (with thetopmost rendered as a wireframe), showing the intersection contour (blue)and the contact normals (black lines) of interpenetrating vertices from theupper mesh.constraints for correcting the interpenetration and preventing interpenetrat-ing velocities. It also provides a good estimate of the contact area, whichcan be used for determining contact pressure.For collisions involving a deformable body, we locate all mesh verticeswhich are interior to the contour. Each such vertex corresponds to surfaceFEM node which is interpenetrating the other body. For each interpene-trating node, we then  nd the nearest point and face on the opposite mesh,and use the face’s normal n as a contact normal. A linear one-dimensionalconstraint is then created which prohibits relative motion in the negativenormal direction between the node and nearest point. If the opposite faceis located on a deformable body, this results in a constraint between the ve-locity of the node vn and the velocities v0, v1, v2 of three nodes associatedwith the nearest face:nTvn w0nTv0 w1nTv1 w2nTv2 0where w0, w1, and w2 are the barycentric coordinates of the nearest pointwith respect to the face. If the opposite face is located on a rigid body,then the constraint is between vn and the body’s translational and angular182C.1. Physical Simulation Frameworkvelocity, vb and !b (expressed in body coordinates):nTvn nTRvb + nTR(p !b) 0where R is the rotational transformation from body to world coordinatesand p is the location of the nearest point in body coordinates. Each of theseconstraints can be expressed in the general form Niu  0, where Ni is arow of N and u is the vector of all velocities.These constraints serve both to prevent interpenetrating velocities inthe contact direction, and to remove interpenetrations during the positioncorrection step (Equation C.7), with the correction distance taken to be thedistance d between the interpenetrating node and its nearest point on theother body.Intersection contours are also used to determine contact constraints forrigid-body/rigid-body contact, although we omit the details here for brevity.As mentioned in Section C.1.2, the solution time of (Equation C.6) canbe greatly improved if some contact constraints can be temporarily treatedas bilateral constraints within a particular time step. By default, ArtiSynthdoes this for contact involving deformable bodies, since such bodies havemany degrees of freedom and their contact constraints tend to be somewhatdecoupled. To prevent sticking, each contact’s vertex-face pair is storedbetween time steps, and if it reappears in the next step, it is used as a contactconstraint only if its corresponding  value computed in (Equation C.6) is 0, implying that there is no force trying to make it separate. This is ine ect an active set method, with the active set used to solve (Equation C.6)being updated between steps.It should be noted that we do not claim that the collision handlingscheme described here is optimal for all applications. In particular, we donot currently implement edge-edge type contacts, and so there can be someinterpenetration which depends on the coarseness of the surface meshes.However, the collision handling is properly isolated from the rest of thesimulation, and other collision handling schemes can be easily used as longas they provide a set of constraints for enforcing the contact and resolving183C.1. Physical Simulation Frameworkinterpenetrations.C.1.5 Simulation engine summaryThe complete ArtiSynth simulation engine is summarized below. It usesthe concept of [67, 177] whereby velocities are computed in advance of po-sitions, subject to constraints, to help prevent constraint violations duringthe subsequent position computation.1. Compute contacts (as per Section C.1.4) and the bilateral and unilat-eral constraint matrices Gk and Nk.2. Correct positions qk to remove interpenetration and drift errors, using(Equation C.7).3. If necessary, adjust Gk and Nk to re ect changes in q.4. Solve for uk+1 using (Equation C.6).5. Adjust velocities uk+1 for dry friction, as described in Section C.1.1.6. Compute new positions: qk+1 = qk +h=2( k+1uk+1 +  kuk).This algorithm is generally applicable to any rigid-deformable body dy-namics. In the absence of constraints, the above system turns into a trape-zoidal rule solution of a regular ordinary di erential equation, for whichglobal errors can be expected to be proportional to O(h2). The inclusion ofconstraints, particularly non-smooth unilateral ones, makes formal conver-gence and error analysis more di cult. However, the main velocity update(Equation C.6) is the same as that described in [157], which is shown to bestable and have second-order convergence under certain assumptions.Generally speaking, our method is a time-stepping scheme which uses xed (or adaptively varying) time steps, as opposed to an event-drivenscheme in which the integration time intervals are precisely aligned withcontact events but which becomes impractical in the presence of large num-bers of contacts. To the extent to which results exist, time-stepping schemes184C.1. Physical Simulation Framework(a) (b) (c)Figure C.4: Examples used for validation, shown in their  nal positionswith a rainbow plot of the resulting Von Mises stresses and the locations oftheir respective reference nodes. (a) Beam example, ArtiSynth. (b) Cubeexample, ArtiSynth (maximum stress 2787 Pa). (c) Cube example, ANSYS(maximum stress 2661 Pa).are typically shown to have less accuracy but better convergence propertiesthat event-driven ones [25].C.1.6 Validation using ANSYSTo help assess the performance of our integration scheme, we compared itagainst the commercial  nite element package ANSYS for two test examples:a beam,  xed at one end and allowed to fall under gravity, and a cube,resting on a  at surface and subjected to a downward load applied to severaltop nodes. It should be noted that ArtiSynth uses several simpli cationscompared to ANSYS, notably the use of semi-implicit integration and alumped mass model.The beam example (Figure C.4 (a)) consisted of a beam with dimensions0.1 x 0.02 x 0.02 m divided uniformly into 8 x 4 x 4 hexahedral elements,with a density of 1040 kg/m3, Rayleigh damping coe cients of  = 20 s 1and  = 0:015 s, and a  ve-parameter Mooney Rivlin material with c10 =10370, c20 = 486 and c01 = c11 = c02 = 0 (Pascals). Incompressibilityin both system was enforced using a mixed u-P formulation [88], and timeintegration was performed for .4 seconds using a one millisecond time step.To assess dynamic performance, the resulting z displacement and velocityof a reference node located in the middle of the free end was comparedover time between ArtiSynth and ANSYS (Figure C.5 (left)). The dynamic185C.1. Physical Simulation Frameworkbehavior was essentially identical: the resulting displacement error (relativeto the maximum displacement) had maximum and average values of 0.3%and 0.08%. Likewise, the resulting velocity error (relative to the maximumvelocity) had maximum and average values of 1.3% and 0.4%. We alsodetermined the errors in total displacement and Von Mises stress for all thenodes in the  nal position: the maximum and average displacement errors(relative to the maximum displacement) were 0.06% and 0.04%, while themaximum and average Von Mises stress errors (relative to the maximumstress value) were 0.9% and 0.13%.The cube example (Figure C.4 (b)) used a cube with a width of 0.1 m inall directions and divided uniformly into 6 x 6 x 6 hexahedral elements, witha density of 1040 kg/m3, Rayleigh damping coe cients of  = 20s 1 and = 0:015s, and a  ve-parameter Mooney Rivlin material with c10 = 1037,c20 = 486 and c01 = c11 = c02 = 0 Pascals (identical to the material used forour tongue model described below). In addition to gravity, an immediateexternal load of -0.8 Newtons was applied in the vertical direction to the ninenodes located in the middle of the top surface, resulting in the deformationshown in Figure C.4. Incompressibility in both system was enforced usingthe B-bar method [88], and the example was integrated for 0.2 seconds witha one millisecond time step. To assess dynamic performance, a referencenode was selected in the middle of the top surface and its z displacementsand velocities were compared (Figure C.5 (right)). Displacement errors hadmaximum and average values of 2.7% and 1.5%, while the velocity errorshad maximum and average values of 22% and 1.4%. The large value forthe velocity error occurred at the beginning, where ANSYS computed anunexpected initial upward velocity for the node. Compared with ANSYS,the ArtiSynth behavior was slightly more damped. For all nodes in the nal position, the maximum and average displacement errors were 3.5% and0.5%, while the maximum and average Von Mises stress errors were 5.6%and 0.5%. Much of this error was due to di erences in the way ArtiSynthand ANSYS compute pressure for the B-bar method, resulting in di erentdilational displacements: in ArtiSynth the model compressed slightly, whilein ANSYS it in ated slightly.186C.2. Graphical Toolset0 100200300400−0.05−0.04−0.03−0.02−0.010z displacement0 4080120160200−0.02−0.010z displacement0 100200300400−0.4−0.3−0.2−0.1−00.1time (ms)z velocity0 4080120160200−0.6−0.4−0.20time (ms)z velocityBeam example Cube exampleFigure C.5: Time integration comparisons between ANSYS (solid lines) andArtiSynth (dotted lines), showing the z displacement (top) and velocity(bottom) of a single reference point in the beam and cube examples.These results help demonstrate that our simulation approach is com-petitive with commercially available codes, in addition to be considerablymore e cient: ArtiSynth was 20 and 10 times faster for the beam and cubeexamples, respectively.C.2 Graphical ToolsetArtiSynth is implemented in Java, both for portability and to take advan-tage of Java’s extensive graphical user interface (GUI) support. Models aregenerally created in Java code, using the packages and classes of the Ar-tiSynth API. Interactive editing is also possible, as described below. Modelscan be saved and loaded using a text  le format which can also be used formodel creation.187C.2. Graphical ToolsetFigure C.6: ArtiSynth application showing two model views (including amedical image plane), and the timeline for arranging probes.C.2.1 Model component hierarchyAn ArtiSynth model is arranged as a hierarchy of components, which includedynamic components such as particles or rigid bodies, force e ectors such aspoint-to-point muscles (including Hill and other types), linear or nonlinear nite elements, and constraints such as joints or collision speci ers. Othermodels can be included in the hierarchy as submodels. At the top of thehierarchy is the RootModel, which can contain special components such asprobes and control panels used for interactive simulation control.C.2.2 Viewing, selection, and editingArtiSynth provides a graphical interface for model viewing, editing, andsimulation control (Figure C.6). Rendering is done using OpenGL, anda Jython console permits scripting and interactive command interaction.Multiple viewers can be created, with aids such as grids, clipping and slicingplanes, and image planes overlaid with medical imaging data (Figure C.6,188C.2. Graphical ToolsetFigure C.7: Navigation panel (left) and muscle control panel (right) for arat hind limb model (bone meshes courtesy of Dr. Dinesh Pai).right).The component hierarchy can be viewed and components can be selectedusing a navigation panel (Figure C.7). Components can also be selected inthe viewers.Models can be edited using a context-based system in which the currentset of selected components determines the available editing functions. Vari-ous components can be added, duplicated, and deleted. Portions of a modelcan also be written to a  le for later use. Dragger  xtures provide transla-tion, rotation and scaling. While it is often easier to create complex modelsin code, interactive editing provides a way to adjust and modify existingmodels, and perform certain kinds of specialized tasks, such as insertingmuscle  bers into an FEM.C.2.3 Properties, control panels, and probesModel components can export various properties that describe attributessuch as mass, damping, force or position. These are implemented using theJava re ection package and can thus be declared easily using one or two linesof code. A collection of properties can be gathered into a composite property;examples of this include render properties that control rendering attributessuch as color, shading, and visibility, and material properties, which control189C.2. Graphical Toolsetthe parameters of FEM constitutive laws.Properties can be declared as inheritable, so that their values can beeither explicitly set or inherited from an identical property located in acomponent further up the hierarchy. This provides an inheritance function-ality similar to that found in scene graphs. For example, you can set adefault material sti ness value within an FEM model component, and thenoverride this value as necessary within individual FEM elements.Properties provide the main connection handles for various interactivesimulation tools. These include control panels, which contain widgets for ad-justing property values. A widget can be automatically created and addedto a control panel by simply selecting a speci c property of a particularcomponent (or group of components). In a typical application, a user mightcreate a panel containing slider widgets connected to the excitation proper-ties for a set of muscles (Figure C.7). These can then be adjusted duringsimulation to see the e ect of activating speci c muscles.A property whose value is a numeric scalar or vector can also be at-tached to a channel of input or output data (known as a probe), which maybe used to control (or record) the property’s value over time. Probes canbe temporally arranged in a graphical timeline (Figure C.6) which providescontrol over their start time and duration. In neuromotor control work,input probes are commonly used to drive muscle excitation levels, whileoutput probes are used to record the resulting forces, velocities, and posi-tions. The numeric data within a probe can be displayed graphically, andfor input probes can be graphically edited by adding or deleting knot points,changing interpolation, etc.190

Cite

Citation Scheme:

    

Usage Statistics

Country Views Downloads
United States 39 0
Canada 36 1
Germany 31 5
Unknown 19 0
Netherlands 9 8
Taiwan 8 0
Russia 5 0
Ukraine 5 1
Romania 5 0
India 4 1
United Kingdom 3 0
Pakistan 3 0
Poland 2 0
City Views Downloads
Unknown 55 2
Braunschweig 22 5
Saskatoon 16 1
Mountain View 15 0
Vancouver 14 0
Amsterdam 8 8
Taipei 7 0
Ashburn 6 0
Washington 5 0
Kharagpur 2 0
Hefei 2 1
Elk Grove Village 2 0
Ottawa 2 0

{[{ mDataHeader[type] }]} {[{ month[type] }]} {[{ tData[type] }]}
Download Stats

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
http://iiif.library.ubc.ca/presentation/dsp.24.1-0071518/manifest

Comment

Related Items

Admin Tools

To re-ingest this item use button below, on average re-ingesting will take 5 minutes per item.

Reingest

To clear this item from the cache, please use the button below;

Clear Item cache