Open Collections

UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Measuring and comparing human walking motions for computer animation Harrison, Jason 2001

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata

Download

Media
831-ubc_2002-731758.pdf [ 40.47MB ]
Metadata
JSON: 831-1.0051507.json
JSON-LD: 831-1.0051507-ld.json
RDF/XML (Pretty): 831-1.0051507-rdf.xml
RDF/JSON: 831-1.0051507-rdf.json
Turtle: 831-1.0051507-turtle.txt
N-Triples: 831-1.0051507-rdf-ntriples.txt
Original Record: 831-1.0051507-source.json
Full Text
831-1.0051507-fulltext.txt
Citation
831-1.0051507.ris

Full Text

Measuring and Comparing Human Walking Motions Computer Animation by Jason Harrison B . E . , State University of New York at Stony Brook, 1992 M . S c , University of British Columbia, 1994 A THESIS S U B M I T T E D IN P A R T I A L F U L F I L L M E N T OF T H E R E Q U I R E M E N T S F O R T H E D E G R E E O F Doctor of Philosophy in T H E F A C U L T Y O F G R A D U A T E STUDIES (Department of Computer Science) We accept this thesis as conforming tp the required standard. The '"University of British Columbia December 2001 © Jason Harrison, 2001 In presenting this thesis in partial fulfilment of the requirements for an advanced degree at the University of British Columbia, I agree that the Library shall make it freely available for reference and study. I further agree that permission for extensive copying of this thesis for scholarly purposes may be granted by the head of my department or by his or her representatives. It is understood that copying or publication of this thesis for financial gain shall not be allowed without my written permission. Department of Computer Science The University of British Columbia 2366 Main Mall Vancouver, B C Canada V6T 1Z4 h t t p : / / w w w . c s . u b c . c a Date Abstract Computers process and store human movement in a different manner from how humans perceive and observe human movement. The leading paradigm used by computer animation tools uses three tightly coupled models. These models, used in movies such as Toy Story, Final Fantasy: The Spirits Within, and Monsters Inc., are (1) a set of time signals, Q{t), that specify the kinematics of the movement, (2) a mapping, A, between Q(t)md the position, orientation, and posture of the human figure, and (3) a "costume" or "visual appearance" that specifies the outer appearance of the human body. In sharp contrast to the exactness of computers, it is not well understood how we visually perceive human movements. It is believed that we utilize the motor control centers of our brains to recognize and interpret the movements of others. However, we do not know how observed movements are encoded or how they are translated into descriptions. Neither do we understand the process we use to translate descriptions or "mental images" of movements into physical movements. In order to build higher-level computer animation tools for selecting, specifying, or modifying movements represented by computer models we need to know how the parameters of a movement, V(Q), affect our perceptions and judgements. We present a participant-based experimental and analytical methodology for gathering information on the relationships between three motion spaces: the first motion space is the "me-chanical motion space," a vector space of motion signals, Q{t); the second motion space is the "psycholog-ical motion space" in which humans encode and organize motions according to their features; and the third motion space is the "linguistic motion space" that humans use to describe movements using words. We demonstrate our experimental and analytical methodology with two participant experiments that uti-lize computer animation displays of human walking figures to determine the effect of the parameters on judgements and descriptions of the movements. The first is a broad initial experiment to demonstrate the collection of judgements of the similarity of movements and descriptions of the movements from human observers using a wide range of human walking movements. The second is an in depth experiment to de-termine the properties of the psychological motion space by using a narrower range of walking movements that includes movements created by interpolating motion parameters. We conclude from the first experiment that the relationships between the motion spaces, for a small group of participants with similar backgrounds in social dance, are sufficiently explained by linear functions. We conclude from the second experiment that the psychological motion space does not have the metric properties necessary to treat similarity judgements as approximations of the distances between gaits in a metric space. We also conclude that the psychological motion space formed by similarity judgements is similar across a wide range of participant backgrounds (dancers, runners, and neither), genders (males and females) and the presentation of direction of walking across the screen (left to right or right to left). Finally we suggest opportunities for future research and applications of our work in computer animation, human-computer interaction and psychophysics. Contents Abstract ii Contents iv List of Tables xi List of Figures xiv Acknowledgements xxii Dedication xxiii 1 Introduction 1 1.1 Computer Representation of Human Movement 4 1.2 Why this Thesis Topic Now? 7 1.3 Outline of Thesis 9 2 Related Work 13 2.1 Representing Motion Signals 14 2.1.1 Human Movements as Motion Signals . 16 2.1.2 Alternative Representations of Human Movements 17 2.2 Recording Physical Movements as Motion Signals 20 2.2.1 Historical Development of Motion Capture 20 2.2.2 Who Records Motion and What do They Record 21 2.2.2.1 Visual Effects Companies 21 2.2.2.2 Digital Puppeteers 21 2.2.2.3 Kinesiologists and Clinical Gait Analysts 21 2.2.3 Cost and Size of a Motion Capture Facility 23 2.2.4 Markerless Video Based Motion Capture 25 2.2.5 Motion Capture and Analysis Software 25 2.3 Manipulating the Quality, Style, and Path of Movements 28 2.3.1 Manipulating Movement Quality and Style 28 2.3.2 Manipulating Movement Path 30 2.4 Motion Signal Databases 31 2.4.1 Motion Generators 32 2.4.2 Motion Galleries 35 2.5 Human Judgements of Motion 37 2.5.1 Separating Movement Pattern from Visual Form: Point Light Displays 37 2.5.2 Apparent Motion 39 2.5.3 Interaction of Movement and Shape 40 2.5.4 Phase Variability of Oscillating Motion 41 2.5.5 Comparison and Judgement of Computer Generated Human Movements 41 3 Experiment Design 45 3.1 Methodology 47 3.LI Stimuli Selection 48 3.1.2 W a l k e r 49 3.1.3 Flow of Data from Stimuli Generation through Analysis 52 3.2 Experimental Procedure 53 3.2.1 Participants 54 3.2.2 Stimuli Presentation 54 3.2.3 Pre-Experiment 55 3.2.4 Instruction of Participants 55 3.2.5 First Session: Motion Comparison Trials 56 3.2.6 Second Session: Motion Rating Trials 60 3.2.7 Post Experiment 63 3.3 Equipment 63 3.3.1 Hardware 63 3.3.2 Software 64 3.4 Data Collection and Notation 65 3.5 Data Analysis Techniques 67 3.5.1 Principal Components Analysis 69 3.5.2 Multidimensional Scaling (MDS) 73 3.6 Formal Tests of Hypotheses 74 3.6.1 Relationship Between Mechanical and Psychological Motion Spaces 75 3.6.1.1 Differences of Gait Parameters Correlate with Dissimilarity Judgements . 76 3.6.1.2 Z-Scores of Gait Parameters Correlate with Dissimilarity Judgements . . . 76 3.6.1.3 Distances Between Z-Scores of Gait Parameters Correlate with Dissimi-larity Judgements 77 3.6.2 Relationship Between Mechanical and Linguistic Motion Spaces 77 3.6.2.1 Gait Parameters Correlate with Ratings Along Descriptive Scales 78 3.6.2.2 Z-Scores of Gait Parameters Correlate with Ratings Along Descriptive Scales 78 3.6.3 Relationship Between Psychological and Linguistic Motion Spaces 79 3.6.3.1 Dissimilarity of Gaits Correlate with Differences Between Ratings Along Scales 79 3.6.3.2 Dissimilarity of Gaits Correlate with Differences of Z-Scores of Ratings . 79 3.6.3.3 Dissimilarity of Gaits Correlate with Minkowski Distances Between Z-Scores of Ratings 80 3.6.4 Significance of Correlations 80 3.7 Summary 81 4 Experiment One: Comparing and Describing Motions 82 4.1 Overview 82 4.2 Mechanical Motion Space 83 4.3 Lingusitic Motion Space 86 4.3.1 Common Patterns Among the PC's of the Participants Ratings 91 4.3.2 Interpreting Individual PC's of the Participants Ratings 94 4.3.3 Summary of Principal Components of Participant's Ratings 97 4.4 Additional Analysis of Motion Comparison Trials 99 4.5 Analysis of Relationships Between Motion Spaces 100 4.6 Relationship Between the Mechanical and Psychological Motion Spaces 102 4.6.1 Correlation Between Dissimilarity Judgements and Differences of Walker ' s Pa-rameters 102 4.6.2 Correlation Between Dissimilarity Judgements and Differences of Z-Scores of Walker ' s Parameters 105 4.6.3 Correlation Between Dissimilarity Judgements and Distances Between Z-Scores of Walker ' s Parameters 107 4.6.4 Summary 109 4.7 Relationship Between the Mechanical and Linguistic Motion Spaces 109 4.7.1 Regression Between Parameters and Ratings 110 4.7.2 Regression Between Z-Scores of Parameters and Ratings 113 4.7.3 Summary 114 4.8 Relationship Between the Psychological and Linguistic Motion Spaces 115 4.8.1 Dissimilarities versus Differences of Ratings Along Scales 115 4.8.2 Dissimilarities versus Differences of Z-Scores of Ratings 118 4.8.3 Dissimilarities versus Distances Between Z-Scores of Ratings 120 4.8.4 Summary 121 4.9 Conclusions 122 5 Experiment Two: Metric Properties of Motion Dissimilarity Judgements 124 5.1 Overview 124 5.2 Differences Between Experiments One and Two 125 5.2.1 Participants 126 5.2.2 Stimuli 126 5.2.3 "Triangular" Gaits 127 5.2.4 "Linear" Gaits 128 5.2.5 Skipping Trials 128 5.2.6 Motion Comparison Trials 129 5.2.7 Motion Rating Trials 130 5.2.8 Reversed Direction ofWalking 131 5.2.9 Summary of Differences Between Experiments One and Two 131 5.3 Analysis of Participant Responses and Hypothesis 132 ,5.4 Average Variance of Dissimilarity Judgements 134 5.5 Metric Properties of Dissimilarity Judgements 136 5.5.1 Testing the Non-Degeneracy Metric Property 137 5.5.1.1 Judgement of Self-Similarity of Gaits 139 5.5.1.2 Judgement of Non-Self-Similarity of Gaits 143 5.5.2 Testing the Symmetry Metric Property 148 5.5.3 Testing the Triangle Inequality Metric Property 150 5.5.3.1 Comparison of the Groups of Participants 151 5.6 Multidimensional Scaling of Dissimilarity Judgements 159 5.6.1 MDS — Triangular Stimuli 160 5.6.2 MDS — Linear Stimuli 171 5.6.3 Comparisons of the Groups of Participants 185 5.7 Conclusions 193 6 Conclusions and Future Research Directions 195 6.1 Future Interdisciplinary Research Directions 197 6.1.1 Reliability of Movement Observers 198 6.1.2 Formation of Movement Dissimilarity Judgements 198 6.1.3 Style of Movement 198 6.1.4 Experiment Design and Analysis Technique Extensions 198 6.2 Future Research in Computer Animation and Human-Computer Interaction 199 6.2.1 Maximizing Constrained Interfaces 199 6.2.2 Maximizing Use of Computational Resources 200 6.2.3 Searching for Movements 201 6.2.4 Skilled Motor Activity Instruction .201 6.3 Summary 202 Bibliography 203 A Additional Analysis of Experiment One 214 A. l Motion Rating Trials 214 A.1.1 Regression Between Walke r ' s Parameters and Ratings 214 A.1.2 Regression Between PCA of Walke r ' s Parameters and Ratings 221 A. 1.3 Correlations Between Rating Scales 227 A. 1.3.1 Are the Correlations Linear Correlations? 229 A. 1.4 Common Patterns Among The Participant's PCs 239 A. 1.5 Interpreting Individual Participant's Ratings PCs .245 A.l.5.1 PCA of Participant #l's Ratings 245 A. 1.5.2 PCA of Participant #2's Ratings 248 A. 1.5.3 PCA of Participant #5's Ratings 251 A.l.5.4 PCA of Participant #6's Ratings 254 A. 1.5.5 PCA of Participant #7's Ratings 258 A. 1.5.6 PCA of Participant #8's Ratings 261 B Stimuli Used In Experiments 265 B. l Stimuli Used Experiment One 265 B. 2 Stimuli Used in Experiment Two 302 B. 2.1 "Triangular" Gaits 302 B.2.2 "Linear" Gaits 302 C Notes on Computational Algorithms 315 C. l Correlation 315 C.2 Principal Components Analysis 317 C.2.1 Example Multivariate Data Set 318 C.2.2 Resulting Principal Components 320 C.2.3 Analysis and Interpretation of the PCs 322 C.2.4 Matlab's p r i n c o m p Function 323 D Participant Questionnaires and Forms 324 D . l Ethics Review Forms 325 D.2 Amendment to Ethics Review 333 D.3 Participant Consent Form 334 D.4 Questionnaire A 336 D.5 Questionnaire B 341 D. 6 Participant Receipt 347 E Experiment Instructions to Participants 348 E. l Motion Comparison Trial Instructions 348 E. 2 Motion Rating Trial Instructions 350 F Participant Demographic Information 352 F. l Participants from Experiment One 352 F.2 Participants from Experiment Two 353 F.3 Advertisement for Participants 354 F.4 Directions to the Lab 359 G The Mechanical Motion Space 361 List of Tables 1.1 Elements of Q(t) from the computer animation of a human walking movement 10 4.1 The variance (Afc {Vw)), proportion of variance and sum of proportion of variance for the first ten principal components of the mechanical motion space 86 4.2 Plot of the strength of the principal components of the ratings 89 4.3 Probabiligy of asymmetry of dissimilarity judgements as determined using ANOVA 101 4.4 For each participant the motion parameters that correlate stronger than rp — 0.4 with their dissimilarity judgements 104 4.5 Summary of which parameters correlate strongly with the dissimilarity judgements 105 4.6 Correlations between dissimilarity judgements and Z-scores of motion parameters 106 4.7 Strength of correlation between dissimilarity judgements and distances in mechanical mo-tion space: best distance norm and number of dimensions 108 4.8 For each rating scale, the three parameters with the largest proportion of rating variance explained, R 2 , for participant #6's ratings 112 4.9 Significant regression coefficients between #6's ratings, pRs(i), and principal components of the parameters, Zk(Vw) 114 4.10 Strongest correlations between dissimilarity judgements and differences of ratings for each participant 116 4.11 The three strongest positive correlations between participant's dissimilarity judgements and differences of Z-scores of ratings 119 4.12 Comparison of the strongest correlation between dissimilarity judgements and differences of ratings versus strongest correlation between dissimilarity judgements and distance between ratings using Z-scores of ratings 121 4.13 For each participant the distance metric and number of components that produce the strongest correlations between the dissimilarity ratings and distances in the linguistic motion space. . . 121 5.1 ANOVA summary table comparing the groups of participants according to mean average trial variances 135 5.2 Summary of trials in which participants judged two different gaits as being similar within one mark of the Similar end of the scale 144 5.3 Summary of results of ANOVAs comparing dissimilarity judgements for pairs of trials in which the order of the gaits is reversed .149 5.4 For each participant the number of triangles formed from the dissimilarity judgements of the triangular and linear gaits which violate the triangular inequality 152 5.5 ANOVA summary table comparing the groups of participants according to goodness of fit to the triangular inequality (CA) 154 5.6 Table of average dissimilarity judgements by participant #1 for the triangular gaits. 1 = Lower Crouch, 2 = Upper Body, 3 = Upper Tipping, 4 = Average 161 5.7 Results of the regressions between the parameters of the triangular gaits and the A L S C A L configuration produced from the dissimilarity judgements of the triangular gaits 163 5.8 Results of the regressions between the parameters of the linear gaits the A L S C A L configu-ration produced from the dissimilarity judgements of the linear gaits 180 5.9 ANOVA summary table comparing the groups of participants according to weirdness index for the A L S C A L scalings of the triangular stimuli (two dimensional) and linear stimuli (two and three dimensional) 186 B . l Parameter values for gait 1: Bent Over Notwist 276 B.2 Parameter values for gait 2: Bent Over Overstride 277 B.3 Parameter values for gait 3: Bent Over Twisting 278 B.4 Parameter values for gait 4: Big Steps 279 B.5 Parameter values for gait 5: Bounce 280 B.6 Parameter values for gait 6: Crouched Twisting 281 B.7 Parameter values for gait 7: Crouching Overstride 282 B.8 Parameter values for gait 8: Hip Notoe Strike 283 B.9 Parameter values for gait 9: Hip Toe Strike 284 B.10 Parameter values for gait 10: Leaning Back Swinging Arms 285 B . l l Parameter values for gait 11: Marching 286 B.12 Parameter values for gait 12: Marching Bent Arms 287 B.13 Parameter values for gait 13: Marching Straight Arms 288 B.14 Parameter values for gait 14: Medium Bounce 289 1 B.15 Parameter values for gait 15: Normal 290 B.16 Parameter values for gait 16: Sensory Ataxia 291 B.17 Parameter values for gait 17: Shoulder Pelvis Rotation 292 B.18 Parameter values for gait 18: Small Bounce 293 B.19 Parameter values for gait 19: Small Locked Steps 294 B.20 Parameter values for gait 20: Small Steps 295 B.21 Parameter values for gait 21: Swing Arms 296 B.22 Parameter values for gait 22: Swing Arms Lift Legs 297 B.23 Parameter values for gait 23: Swing Upper 298 B.24 Parameter values for gait 24: Tip Toes Sneaky 299 B.25 Parameter values for gait 25: Tiptoe 300 B.26 Parameter values for gait 26: Twist Upper 301 B.27 Parameter values for gait 1: Lower Crouch 305 B.28 Parameter values for gait 2: Upper Body 307 B.29 Parameter values for gait 3: Upper Tipping 308 B.30 Parameter values for gait 4: Average 309 B.31 Parameter values for gait 5: Stiff Upright 310 B.32 Parameter values for gait 6: 75% Stiff Upright/25% Super Crouch Twisting 311 B.33 Parameter values for gait 7: 50% Stiff Upright/50% Super Crouch Twisting .312 B.34 Parameter values for gait 8: 25% Stiff Upright/75% Super Crouch Twisting 313 B.35 Parameter values for gait 9: Super Crouch Twisting 314 X l l l List of Figures 1.1 The psychological, linguistic and mechanical motion spaces 3 1.2 Sample frames from a computer animation display of a human walking movement rendered using the "Akira" visual form 5 1.3 An articulation approximating the skeletal structure of the human body used to animated the "Akira" visual form. Open circles indicate hinge joints with only one rotational axis. Shaded circles indicate joints with two rotational axes. Dark circles indicate joints with three rotational axes 6 1.4 Visual forms used to visualize the motion of the articulation in Figure 1.3 6 2.1 Human body articulations for representing movement 15 2.2 Simple human-like visual forms 16 2.3 Joint angle plots of walks generated by Bruderlin's (1995) Walker 17 2.4 Screen shot of K A Software being used to analyse a roundhouse kick 27 2.5 Naming convention for video game motion files 33 2.6 Bruderlin's (1995) walk generator , 34 2.7 An actuated double pendulum rooted and some possible movements 36 2.8 An indexing scheme for jumps of a Luxo Lamp 37 2.9 Synthetic point light display created by using the "Akira" visual form with an all black surface material and attachement of white spheres at the joints. The walking motion is the same as in Figure 1.2 39 2.10 Velocity patterns of the end-effector of a robotic arm 43 2.11 Frames from animations used in a motion comparison experiment 44 3.1 The psychological, linguistic and mechanical motion spaces 46 3.2 Bruderlin's (1995) walk generator 50 3.3 A walking motion (and example frames) as presented to a participant 51 3.4 Flow of data from the production of the gaits through the analysis of the participant's re-sponses in the two experiments 52 3.5a Presentation of the first walking motion in a motion comparison trial from Experiment One . 57 3.5b Presentation of the second walking motion in a motion comparison trial from Experiment One 58 3.5c The Similar-Dissimilar scale 59 3.6a Presentation of the first walking motion in a motion rating trial from Experiment One . . . . 61 3.6b Recording the description 62 4.1 Scatter plots of Z(VW) for the first five components 85 4.2 Box plot of the variance explained by each principal component for participants #1, #2, #5, #6, #7 and #8 90 4.3 Plots of the magnitude and sign of the coefficients of the first principal component of each participant's ratings 92 4.4 Plots of the magnitude and sign of the coefficients of the second principal component of participants #1, #2, #5, #6, and #7, and the third principal component of participant #8. . . . 93 4.5 Plot of the strength of the principal components of #l's ratings and plots of the coefficients of the components 95 4.6 Strength of correlation between rating differences and dissimilarity judgements for all par-ticipants and rating scales 117 5.1 Two configurations of comparisons for primary gaits and interpolated gaits used to define the stimuli and motion comparison trials 127 5.2 Screen shots from Experiment Two illustrating changes made to the experiment control soft-ware that would be apparent to a participant 133 5.3 Plots of the number of mis-judgements of self-similarity 140 5.4 Plot of mis-judgements of self-similarity 141 5.5 Plot of mis-judgements of non-self-similarity 143 5.6 The number of times participants mis-judged a non-self-similar trial 147 5.7 A star network of stimuli for a future experiment to test hypothesis of order effect of dissim-ilarity judgements 147 5.8a Box plot of the c,\ for triangular gaits based on experience. Differences are not significant. . 155 5.8b Box plot of the CA for triangular gaits based on gender. Differences are not significant. . . . 156 5.8c Box plots of the CA for triangular gaits based on experimentally controlled direction of walking of the figure 156 5.8d Box plots of the CA for linear gaits based on experience. The difference is not significant. . . 157 5.8e Box plots of the CA for linear gaits based on gender. The difference is not significant 157 5.8f Box plots of the C/i for linear gaits based on experience. The difference is not significant. . . 158 5.9 A L S C A L scaling of the dissimilarity judgements for the triangular stimuli for all participants. 161 5.10 Participant weights for the two dimensional A L S C A L solution for the triangular stimuli. . . 165 5.11 The squared correlation, or RSQ, is the proportion of the variance of a participant's dissim-ilarity judgements accounted for by the distances between the stimuli in the overall con-figuration. RSQ is computed by summing the squared weights for each participant (see Figure 5.10 for the weights) 167 5.12 Kruskal's SSTRESS(l) values indicate the "goodness of fit" or the disagreement between a participant's dissimilarity judgements and the overall configuration. High values indicate a poor agreement, low values a good agreement 168 5.13 SSTRESS(l) versus RSQ: A strong correlation between SSTRESS(l) and RSQ indicates that participants with "poorer fits" (on the right) also had different patterns of variation in their dissimilarity judgements than the rest of the participants 169 5.14 Plot of weirdness indices for the participants from the triangular stimuli 170 5.15 Three-dimensional A L S C A L scaling of the dissimilarity judgements for the linear stimuli for all participants 171 5.16 Participant weights for the three dimensional A L S C A L solution for the linear gaits 172 5.17 Participant weights for the three dimensional A L S C A L solution for the linear gaits 173 5.18 Participant weights for the three dimensional A L S C A L solution for the linear gaits 174 5.19 Participant weights for the three dimensional A L S C A L solution for the linear gaits 175 5.20 RSQ of the three dimensional configuration: 176 5.21 SSTRESS(l)of the three dimensional configuration 177 5.22 Weirdness index of the three dimensional configuration 177 5.23 Two-dimensional A L S C A L scaling of the dissimilarity judgements for the linear stimuli for all participants : 179 5.24aParticipant weights for the two dimensional A L S C A L solution for the linear gaits 181 5.24bParticipant weights for the two dimensional A L S C A L solution for the linear gaits. (Rescaled) 182 5.25 RSQ of the two dimensional configuration 183 5.26 SSTRESS(l) of the two dimensional configuration 183 5.27 Weirdness Index of the two dimensional configuration 184 5.28 Comparison of the SSTRESS(l) between the two and three dimensional configurations. . . . 184 5.29 Box plots for ANOVA comparisons according to weirdness index for two dimensional A L S C A L solution for triangular gaits and three dimensional A L S C A L solution for the linear gaits 187 5.30 Box plots for ANOVA comparisons according to weirdness index for two dimensional A L S C A L solution for triangular gaits and three dimensional A L S C A L solution for the linear gaits 188 5.31 Box plots for ANOVA comparisons according to weirdness index for two dimensional A L S C A L solution for triangular gaits and three dimensional A L S C A L solution for the linear gaits 189 5.32 Box plots for the ANOVA comparison according to weirdness index for two dimensional A L S C A L solution for the linear gaits. . 190 5.33 Box plot of the weirdness indices by direction of walk (left to right, and right to left) for the two dimensional A L S C A L configuration of the linear stimuli illustrating the only significant result between the groups 191 5.34 Plots of the dimensional weightings by direction of walk (left to right, and right to left) for the two dimensional A L S C A L configuration of the linear stimuli illustrating the only significant result between the groups 192 A. 1 Plots of the number of rating scale pairs correlated stronger than the indicated amount for each participant. For the eight rating scales, there are twenty-eight unique pairs (ignoring self pairings). Each participant's ratings of the gaits is used to compute the correlation of each of these pairs. Pairs with correlations stronger than the thresholds indicated are then counted and plotted above. Larger values of \rp\ indicate stronger correlations 228 A.2 Scatter plots of participant #l's ratings for each pair of ratings scales. There do not appear to be any non-linear correlations between the rating scales 230 A.3 Scatter plots of participant #2's ratings for each pair of ratings scales. There do not appear to be any non-linear correlations between the rating scales 231 A.4 Scatter plots of participant #3's ratings for each pair of ratings scales. There do not appear to be any non-linear correlations between the rating scales 232 A.5 Scatter plots of participant #4's ratings for each pair of ratings scales. #4 used only the most "significant" rating scale to describe each motion — and her use of the rating scales tended to be binary rather than continuous 233 A.6 Scatter plots of participant #5's ratings for each pair of ratings scales. There do not appear to be any non-linear correlations between the rating scales 234 A.7 Scatter plots of participant #6's ratings for each pair of ratings scales. There do not appear to be any non-linear correlations between the rating scales 235 A.8 Scatter plots of participant #7's ratings for each pair of ratings scales. There do not appear to be any non-linear correlations between the rating scales 236 A.9 Scatter plots of participant #8's ratings for each pair of ratings scales. There do not appear to be any non-linear correlations between the rating scales 237 A. 10 Scatter plots of participant #9's ratings for each pair of ratings scales. M O tended to use only a few rating scales to describe each motion — and his use of the rating scales tended to be binary rather than continuous 238 A . l 1 Plots of the magnitude and sign of the coefficients of the first four principal components of participants #1 ratings. The horizontal axes are the rating scales, while the vertical axes are the coefficient values 239 A. 12 Plots of the magnitude and sign of the coefficients of the first four principal components of participants #2 ratings. The horizontal axes are the rating scales, while the vertical axes are the coefficient values 240 A. 13 Plots of the magnitude and sign of the coefficients of the first four principal components of participants #5 ratings. The horizontal axes are the rating scales, while the vertical axes are the coefficient values 241 A. 14 Plots of the magnitude and sign of the coefficients of the first four principal components of participants #6 ratings. The horizontal axes are the rating scales, while the vertical axes are the coefficient values 242 A.15 Plots of the magnitude and sign of the coefficients of the first four principal components of participants #7 ratings. The horizontal axes are the rating scales, while the vertical axes are the coefficient values 243 A. 16 Plots of the magnitude and sign of the coefficients of the first four principal components of participants #8 ratings. The horizontal axes are the rating scales, while the vertical axes are the coefficient values 244 A. 17 On the top: Plot of the variance explained by each principal component of participant #l's ratings. The height of each bar indicates the number of normalized rating scales the compo-nent is equivalent to. The line shows the cumulative variance explained by using the first n principal components. The dotted horizontal line indicates the threshold of 80% explained variance. On the bottom: plots of the coefficients of #l's first four PCs. Horizontal axis is are the rating scales, vertical axis is magnitude and sign of coefficients 246 A.18 On the top: Plot of the variance explained by each principal component of participant #2's ratings. The height of each bar indicates the number of normalized rating scales the compo-nent is equivalent to. The line shows the cumulative variance explained by using the first n principal components. The dotted horizontal line indicates the threshold of 80% explained variance. On the bottom: plots of the coefficients of #2's first four PCs. Horizontal axis is are the rating scales, vertical axis is magnitude and sign of coefficients 249 A . 19 On the top: Plot of the variance explained by each principal component of participant #5's ratings. The height of each bar indicates the number of normalized rating scales the compo-nent is equivalent to. The line shows the cumulative variance explained by using the first n principal components. The dotted horizontal line indicates the threshold of 80% explained variance. On the bottom: plots of the coefficients of #5's first four PCs. Horizontal axis is are the rating scales, vertical axis is magnitude and sign of coefficients 252 A.20 On the top: Plot of the variance explained by each principal component of participant #6's ratings. The height of each bar indicates the number of normalized rating scales the compo-nent is equivalent to. The line shows the cumulative variance explained by using the first n principal components. The dotted horizontal line indicates the threshold of 80% explained variance. On the bottom: plots of the coefficients of #6's first four PCs. Horizontal axis is are the rating scales, vertical axis is magnitude and sign of coefficients 255 A.21 On the top: Plot of the variance explained by each principalcomponent of participant #7's ratings. The height of each bar indicates the number of normalized rating scales the compo-nent is equivalent to. The line shows the cumulative variance explained by using the first n principal components. The dotted horizontal line indicates the threshold of 80% explained variance. On the bottom: plots of the coefficients of #7's first four PCs. Horizontal axis is are the rating scales, vertical axis is magnitude and sign of coefficients 259 A.22 On the top: Plot of the variance explained by each principal component of participant #8's ratings. The height of each bar indicates the number of normalized rating scales the compo-nent is equivalent to. The line shows the cumulative variance explained by using the first n principal components. The dotted horizontal line indicates the threshold of 80% explained variance. On the bottom: plots of the coefficients of #8's first four PCs. Horizontal axis is are the rating scales, vertical axis is magnitude and sign of coefficients 262 B . l Frames showing half a stride from gaits one through three 267 B.2 Frames showing half a stride from gaits four through six 268 B.3 Frames showing half a stride from gaits seven through nine 269 B.4 Frames showing half a stride from gaits ten through twelve . 270 B.5 Frames showing half a stride from gaits thirteen through fifteen 271 B.6 Frames showing half a stride from gaits sixteen through eighteen 272 B.7 Frames showing half a stride from gaits nineteen through twenty 273 B.8 Frames showing half a stride from gaits twenty-one through twenty-four. 274 B.9 Frames showing half a stride from gaits twenty-five and twenty-six 275 B.10 Frames showing half a stride from gaits one through three 303 B . l l Frames showing half a stride from gaits four through six 304 B.12 Frames showing half a stride from gaits seven through nine 306 Acknowledgements I will start by acknowledging the Department of Computer Science for being a island of rationality in the chaos which often characterizes the University of British Columbia. The support provided by its members have helped me through the tough times and I hope that I have reciprocated when possible. Specifically, the Imager computer graphics laboratory has been a great place work. The current faculty, Kellogg Booth and Brian Fisher, have been most helpful with their advice and guidance on this research. Past faculty, Alain Fournier and David Forsey, posed many questions which helped to focus my interest in computer animation and human-computer interaction. Roger Tarn, Barry Po, Jon Salter, and Mary Georgilas — as well as my committee — read early drafts of this thesis and provided helpful comments. I gratefully acknowledge the support of the National Sciences and Engineering Research Council of Canada "Interactive Computer Graphics" research grant to Kellogg S. Booth. Dedicated to Mary Georgilas, M D , PHT Chapter 1 Introduction Computers process and store human movement in a different manner from how humans perceive and observe human movement. The leading paradigm used by computer animation tools employs three tightly coupled models. These models, used in movies such as Toy Story, Final Fantasy: The Spirits Within, and Monsters Inc., are (1) a set of time signals, Q(t), that specify the kinematics of the movement, (2) a mapping, A, between Q(t) and the position, orientation, and posture of the human figure, and (3) a "costume" or "visual appearance" that specifies the outer appearance of the human body. The separation of kinematics from the mapping and from the visual appearance allows a computer animator to manipulate the movement of a human figure, Q(t), without having to adjust the mapping or visual model simultaneously. Typically, A is an articulation approximating the skeletal structure of the human body and Q(t) specifies the position, orientation and joint angles of the articulation for every time t. In sharp contrast to the exactness of computers, it is not well understood how we visually perceive human movements. It is believed that we utilize the motor control centers of our brains to recognize and interpret the movements of others. This recognition is robust under a variety of viewing conditions including displays created using moving points of light with camouflaging masks rather than solid figures. However, our ability to recognize the motions of animals presented using moving point displays is poor and does not improve even when we have extensive experience observing the motion of the animals in day to day contact. Therefore it is hypothesized that we use a "motor code" — specific to the structure of the human body — to represent motion and unify the visual elements we perceive. With such wide differences between how computers and humans represent human movement, computer ani-mation relies heavily on the skills of computer animators to translate imagined movements of characters into computer based representations. To assist animators, many computer animation researchers focus on techni-cal areas including increasing the realism of the visual models, refining techniques to record the movement of human actors, and building algorithms to assist in the editing of recorded movements. In other words, research has tended to focus on the limitations of computer based representations of human movement rather than higher-level techniques for adjusting motions. For example, if you wanted to adjust the gross path of a movement while not affecting its style of movement, almost none of the prior work in computer animation is applicable. In order to build higher-level computer animation tools for selecting, specifying, or modifying movements represented by computer models we need to know how the parameters of a movement, V(Q), affect our perceptions and judgements. This requires knowledge of computer animation, human-computer interaction and visual psychophysics. Thus, it is useful to introduce three different types of motion spaces to assist in our discussion of the relationship between the parameters of a movement and the perceptions and judgements formed by a human observer. The first motion space is the "mechanical motion space," a vector space of motion signals, Q(t). Computer animation tools operate in this space. Q(t) is summarized with a set of parameters, V(Q) which describes Q(t) in a compact form. For example, if Q(t) are the motion signals for a walking motion, V(Q) would include walking speed, step frequency, stride length, etc. The second motion space is a conceptual space in which humans encode and organize motions. We shall call this space the "psychological motion space" in all of our discussions. Although we know little about the structure or properties of this space, we can hypothesize that judging the similarity of two movements requires the computation of a "distance" between them. When we categorize a movement as belonging to the class of "running," "walking," or "throwing," we probably use the shortest distance between the movement and the exemplars of these different classes. The third motion space is also a conceptual space that humans use to describe movements using words. This "linguistic motion space" contains attributes in which concepts of "slower" and "bouncy" are defined. This is also the space used to interpret the labels on the user interfaces of computer animation tools, although there is little reason to believe that there is a solid basis for choosing the labels that are used. Figure 1.1 illustrates the relationships between the spaces. Mechanical Motion Space Figure 1.1: The three motion spaces used to discuss the relationship between the parameters of a movement and the perceptions and judgements formed by a human observer. The mechanical motion space is the space computer animation tools operate in. The psychological motion space is the space we use to perceive and code the features of movements. The linguistic motion space is the space we use to describe movements using words. The main contribution of this thesis is a methodology for exploring the relationships between these spaces by collecting similarity judgements and descriptions of movements made by human observers. The similarity judgements are used to determine which parameters, V(Q), are used to judge the similarity of movements. The descriptions are used to determine which parameters, V(Q), correlate with descriptive opposites such as fast-slow, bouncy-smooth, and young-old. Rather than presenting a unified model of human motion perception, motion judgement, or motion descrip-tion, we will present the results of two experiments. The first is a broad initial experiment to demonstrate the collection of similarity judgments and descriptions of the movements from human observers using a wide range of human walking movements. The second is an in depth experiment to determine the properties of the psychological motion space by using a narrower range of walking movements that includes movements created by interpolating motion parameters. 1.1 Computer Representation of Human Movement This thesis concerns the movement of humans as represented by a computer. In a computer animation system, motion signals are transformed using the mapping A and the visual appearance of a human figure into a visual presentation termed a computer animation display. Computer animation displays of human figure animations rapidly create and present images ("frames") that show successive poses, positions and orientations of a human movement. Computer animation displays allow changes in viewpoint and arbitrary playback rates. Figure 1.2 shows eight frames from a computer animation display of a human walking movement. Computer animation programs represent human movement using three models: time signals specifying the movement of the figure (Q(t)), a mapping between Q(£)and position, orientation and posture of the human figure (A), and the visual appearance of the human figure. In a computer program, the mapping A typically involves the specification of the skeletal structure of a human. This structure is usually represented by a non-cyclic hierarchical articulation with the root at the hips (an approximate center of mass and movement). In computer animation applications, the human body requires from seventeen to over one hundred joints as illustrated in Figure 1.3. Some joints can be modeled Frame 39 Frame 41 Frame 43 Frame 45 Frame 47 Frame 49 Frame 51 Frame 53 Figure 1.2: Sample frames from a computer animation display of a human walking movement rendered using the "Akira" visual form. as simple hinges with one rotational axis while other joints are modeled as "ball and socket" connections with three rotational axes.1 The appearance of the human figure is defined by a geometric model of skin and clothing, which is built on top of the skeletal model. Figure 1.4 shows six visual representations of the human body that range from a stick figure to a somewhat realistic fully shaded rendering. Super-realistic models of the human figure as used in Final Fantasy: The Spirits Within require the specifi-cation of both a geometric model and the physics-like characteristics of the skin, clothing, and hair.2 For this thesis, only Q{t), the mapping A, and the geometric model are used to create computer animation displays of human movements. 3 The movement of the human figure — that is the change in posture, position, and orientation of the artic-ulation — is specified by the elements of Q(t), one function for each rotational or translational joint axis, 'Many of the joints are in the hands. Excluded from our definition of A is the ability to include facial expressions in Q(<)since these are not necessary for gross human body motions. 2 For example, animators can specify the motion of the hair using a combination of explicit and implicit methods. The explicit methods require the specification of the shape and movement of each strand of hair. The implicit methods use physics-like models to move the hair in reaction to the motion of the body. The physics-like models require definitions of the length, mass, thickness, stiffness, natural curl, air movement, gravity, etc. 'We also have to specify a set of rendering parameters: viewpoint, camera parameters, spatial and temporal sampling, lighting, material properties, etc. But we'll ignore these and assume that this set is the same for all presentations and that many of these details are either defined by the visual appearance of the articulation or by the space the form is presented in. head Figure 1.3: An articulation approximating the skeletal structure of the human body used to animated the "Akira" visual form. Open circles indicate hinge joints with only one rotational axis. Shaded circles indicate joints with two rotational axes. Dark circles indicate joints with three rotational axes. Stick Flat Contour Figure 1.4: Visual forms used to visualize the motion of the articulation in Figure 1.3. and six functions specifying the position and orientation of the articulation as a whole. Table 1.1 on page 10 lists values of Q(t) for each frame of the walking movement presented in Figure 1.2. There are three figure translational components, three figure rotational components, and seventy-eight joint orientations specified for each frame of the animation. 1.2 Why this Thesis Topic Now? It is now possible to automatically translate video recordings of human movements into motion signals. While these algorithms require visual markers and usually human guidance, it is increasingly becoming possible for computers to translate video into motion signals. The signals are then edited using computer animation tools and presented using computer animation displays. As it becomes easier to record human movements using motion signals, limitations of our computer anima-tion tools will become painfully apparent. For example, the tools currently available cannot modify the style of a movement — such as happy, awkward, drunk, graceful, or fast — except by destroying it. To modify the gross action of a movement while preserving its movement style requires us to define movement style in a manner compatible with computer algorithms. Perhaps the best method to define movement style is to compare and contrast movements believed to have a common style. We also believe that large databases of recorded movements will be created as it becomes easier to record hu-man movements. Applications for these databases include "clip art" movements for use in video games, mul-timedia documents, or PowerPoint™presentations. Medical records could be extended by adding recordings of movements demonstrating the progress of neuromuscular diseases and responses to treatment. We currently do not have proven methods to search databases of human movements other than by hand-coded categories. Even if it was possible to categorize all movements, we believe that for many database queries the question will likely be "find a movement like this one" or "with these characteristics." To build databases that are searchable with similarity queries, we need to understand the relationships between motions signals and the judgements and descriptions formed by human observers. Using computer animation displays we can study the relationships between the motion spaces by answering the following questions: • Which features of a movement do humans pay most attention to? What is the relative importance of these features? • How are the features combined to make decisions about the similarity or difference of two move-ments? • Which techniques can we use to extract perceptually salient features from motion signals? • How do we encode models of human motion perception, observation and judgement as computational algorithms? What is the trade-off between computational complexity and correctness of comparison? • How should movements be presented using computer animation displays so that humans can make judgements and comparisons accurately and reliably? The answers to these questions will help us construct higher-level tools for selecting, specifying, or modi-fying motion descriptions used in computer animation. Thus, we will focus on questions which allow us to gather information for future construction of higher-level computer animation tools: • How should movements be presented using computer animation displays so that humans can make judgements and comparisons accurately and reliably? • Which relationships exist between the parameters of motion signals, judgements of similarity, and descriptions of motions? Answering these questions is not an easy task. Potentially there are an almost infinite number of human movements. The structure of psychological and linguistic motion spaces is most probably non-linear. And the relationships between the spaces are bidirectional, requiring separate treatment of six relationships. To gather information on the relationship between the mechanical and psychological motion spaces we had participants judge the similarity of pairs of human movements — presented using computer animation displays. We used judgements of similarity to determine the salience of the various parameters of the motion signals. We also used multidimensional scaling to form an approximate picture of the participant's "psychological motion space." To gather information on the relationship between the mechanical and linguistic motion spaces we had participants describe motions using descriptive scales. These descriptions allowed us to determine how variations in the motion parameters map to variations along each of several descriptive scales. We also examined the relationship between the psychological and linguistic motion spaces by testing the hypothesis that similar motions have similar descriptions and dissimilar motions have dissimilar descrip-tions. Principal components analysis of both the descriptions and the motion signal parameters allowed us to compare the Euclidean approximations of linguistic and mechanical motion spaces to each other and to the psychological motion space. Since each participant had different biases, experiences, and perceptual filters, their responses were not av-eraged. This treatment places the contributions of this thesis into a sub-area of human-computer interaction research in which individual differences outweigh the impact of system design choices and in which it is assumed that populations of users can not be represented adequately by their average behavior and abilities. 1.3 Outline of Thesis The remainder of this thesis is presented as follows. First, in Chapter 2 we review how motion signals are recorded and represented in computers, how they are manipulated, what collections of motions are available, and methods for comparing and categorizing motions. A review of the relevant research in the area of visual psychophysics of motion perception is included for thoroughness. Chapter 3 discusses the design of two experiments to collect particular sets of motion similarity judgements and motion descriptions from participants. The recruitment of participants who were expected to have specific experience and skill observing, comparing and describing motions is also discussed. Chapter 4 presents the results from the first experiment, which used a wide range of human walking move-ments to explore the relationships between the three motion spaces. Chapter 5 presents the results from the second experiment, which was designed to determine the properties of the psychological motion space by using a narrower range of walking movements that include movements created by interpolating motion parameters. Chapter 6 presents our conclusions, future research directions and applications. Table 1.1: Elements of Q(t) from the computer animation of a human walking movement illustrated in Figure 1.2 specifying the translation (_tx, _ty, _tz) and rotation (_rx, _ry, _rz) of the articulation from Figure 1.3 using the visual form "Akira" from Figure 1.4. Elements of Q(t) from the computer animation of a human walking movement Element of Q{t) Frame 39 Frame 41 Frame 43 Frame 45 Frame 47 Frame 49 Frame 51 Frame 53 f i g u r e . t x -0.001 -0.003 -0.005 -0.007 -0.007 -0.008 -0.010 -0.008 f igure_ty +0.892 +0.894 +0.916 +0.939 +0.950 +0.946 +0.933 +0.916 f i gu re _tz + 1.830 + 1.926 +2.019 +2.113 +2.207 +2.301 +2.395 +2.489 f igure_rx +0.000 +0.000 +0.000 +0.000 +0.000 +0.000 +0.000 +0.000 f igure_ry +8.889 +6.667 +4.444 +2.222 +0.000 -2.500 -5 .000 -7.500 f igure_rz -1.500 -4.500 -5.538 -4.615 -3.692 -2.769 -1.846 -0.923 r _hip_rx -34.610 -33.157 -25.151 -15.405 -5.334 +2.308 +11.418 + 10.567 r_hip_ry -8.889 -6.667 -4 .444 - 2 . 2 2 2 +0.000 +2.500 +5.000 +7.500 r J i i p_ rz -2.512 -5.670 -6.843 -6.016 -5.124 -4.282 -3.456 - 2 . 4 3 7 l_hip_rx +3.161 -1.755 - 8 . 0 0 0 -21.697 -33.124 -35.000 -32.488 -29.183 l_hip_ry -8.889 -6.667 -4 .444 -2 .222 +0.000 +2.500 +5.000 +7.500 l_hip_rz -0.536 -3.635 -4.754 -3.806 -2.858 -1.910 -0.962 -0.015 r_knee_rx +24.432 +30.750 +26.871 +20.694 +13.347 +10.482 +4.704 +18.078 l_knee_rx +46.607 +58.999 +68.769 +77.000 +74.107 +60.930 +39.733 +17.640 r_ankle_rx -99.059 -94.005 -91.720 -95.290 -98.012 -102.790 -106.122 -112.785 r_ankle_ry - 5 . 0 0 0 - 5 . 0 0 0 - 5 . 0 0 0 - 5 . 0 0 0 - 5 . 0 0 0 - 5 . 0 0 0 -5 .000 - 5 . 0 0 0 r_ankle_rz +1.012 +1.170 +1.305 + 1.401 + 1.432 +1.513 + 1.609 +1.514 l_ankle_rx -102.790 -78.144 -68.129 -64.993 -83.885 -102.778 -100.013 -97.248 l . ankle_ry +5.000 +5.000 +5.000 +5.000 +5.000 +5.000 +5.000 +5.000 l_ankle_rz -0.964 -0.865 +0.785 +0.809 +0.834 +0.859 +0.884 +0.908 r_metatarsal J : X +0.000 +0.000 +0.000 +0.000 +0.000 +0.000 +0.000 -5.860 1 .meta ta rsa l J T X -36.976 -60.000 -51.429 -34.286 -17.143 +0.000 +0.000 +0.000 L l _ r x -0.919 -1.156 -1.193 -1.146 -1.068 -0.977 -0.890 -0.825 L2_rx -0.919 -1.156 -1.193 -1.146 -1.068 -0.977 -0.890 -0.825 L3_rx -0.919 -1.156 -1.193 -1.146 -1.068 -0.977 -0.890 -0.825 L4_rx -0.919 -1.156 -1.193 -1.146 -1.068 -0.977 -0.890 -0.825 L5_rx -0.919 -1.156 -1.193 -1.146 -1.068 -0.977 -0.890 -0.825 continued from previous page Elements of Q(t) from the computer animation of a human walking movement Element of Q(t) Frame 39 Frame 41 Frame 43 Frame 45 Frame 47 Frame 49 Frame 51 Frame 53 T l _ r x +0.000 +0.000 +0.000 +0.000 +0.000 +0.000 +0.000 +0.000 T2_rx +0.000 +0.000 +0.000 +0.000 +0.000 +0.000 +0.000 +0.000 T3_rx +0.000 +0.000 +0.000 +0.000 +0.000 +0.000 +0.000 +0.000 T4_rx +0.000 +0.000 +0.000 +0.000 +0.000 +0.000 +0.000 +0.000 T5_rx +0.000 +0.000 +0.000 +0.000 +0.000 +0.000 +0.000 +0.000 T6_rx +0.000 +0.000 +0.000 +0.000 +0.000 +0.000 +0.000 +0.000 T7_rx +0.000 +0.000 +0.000 +0.000 +0.000 +0.000 +0.000 +0.000 T8_rx -0.459 -0.578 -0.597 -0.573 -0.534 -0.488 -0.445 -0.413 T9_rx -0.459 -0.578 -0.597 -0.573 -0.534 -0.488 -0.445 -0.413 T10_rx -0.459 -0.578 -0.597 -0.573 -0.534 -0.488 -0.445 -0.413 T l l _ r x -0.459 -0.578 -0.597 -0.573 -0.534 -0.488 -0.445 -0.413 T12_rx -0.459 -0.578 -0.597 -0.573 -0.534 -0.488 -0.445 -0.413 C l _ r x +0.000 +0.000 +0.000 +0.000 +0.000 +0.000 +0.000 +0.000 C2 . rx +0.000 +0.000 +0.000 +0.000 +0.000 +0.000 +0.000 +0.000 C3_rx +0.000 +0.000 +0.000 +0.000 +0.000 +0.000 +0.000 +0.000 C4_rx +0.000 +0.000 +0.000 +0.000 +0.000 +0.000 +0.000 +0.000 C5_rx +0.000 +0.000 +0.000 +0.000 +0.000 +0.000 +0.000 +0.000 C6_rx +0.000 +0.000 +0.000 +0.000 +0.000 +0.000 +0.000 +0.000 C7_rx +0.000 +0.000 +0.000 +0.000 +0.000 +0.000 +0.000 +0.000 L l _ r y +0.222 +0.167 +0.111 +0.056 +0.000 -0.063 -0.125 -0.188 L2_ry +0.222 +0.167 +0.111 +0.056 +0.000 -0.063 -0.125 -0.188 L3_ry +0.222 +0.167 +0.111 +0.056 +0.000 -0.063 -0.125 -0.188 L4_ry +0.222 +0.167 +0.111 +0.056 +0.000 -0.063 -0.125 -0.188 L5_ry +0.222 +0.167 +0.111 +0.056 +0.000 -0.063 -0.125 -0.188 T l _ r y +0.444 +0.333 +0.222 +0.111 +0.000 -0.125 -0.250 -0.375 T2_ry +0.444 +0.333 +0.222 +0.111 +0.000 -0.125 -0.250 -0.375 T3_ry +0.444 +0.333 +0.222 +0.111 +0.000 -0.125 -0.250 -0.375 T4_ry +0.444 +0.333 +0.222 +0.111 +0.000 -0.125 -0.250 -0.375 T5_ry +0.444 +0.333 +0.222 +0.111 +0.000 -0.125 -0.250 -0.375 T6_ry +0.444 +0.333 +0.222 +0.111 +0.000 -0.125 -0.250 -0.375 T7_ry +0.000 +0.000 +0.000 +0.000 +0.000 +0.000 +0.000 +0.000 continued from previous page Elements of Q(t) from the computer animation of a human walking movement Element of Q(t) Frame 39 Frame 41 Frame 43 Frame 45 Frame 47 Frame 49 Frame 51 Frame 53 T8_ry + 1.556 + 1.167 +0 778 +0.389 +0.000 -0.438 -0.875 -1.313 T9_ry + 1.556 + 1.167 +0 778 +0.389 +0.000 -0.438 -0.875 -1.313 TlO.ry +1.556 + 1.167 +0 778 +0.389 +0.000 -0.438 -0.875 -1.313 T l l - r y + 1.556 + 1.167 +0 778 +0.389 +0.000 -0.438 . -0.875 -1.313 T12_ry + 1.556 + 1.167 +0 778 +0.389 +0.000 -0.438 -0.875 -1.313 Cl_ry -0.381 -0.286 - 0 190 -0.095 +0.000 +0.107 +0.214 +0.321 C2_ry -0.381 -0.286 - 0 190 -0.095 +0.000 +0.107 +0.214 +0.321 C3_ry -0.381 -0.286 - 0 190 -0.095 +0.000 +0.107 +0.214 +0.321 C4_ry -0.381 -0.286 - 0 190 -0.095 +0.000 +0.107 +0.214 +0.321 C5_ry -0.381 -0.286 - 0 190 -0.095 +0.000 +0.107 +0.214 +0.321 C6_ry -0.381 -0.286 - 0 190 -0.095 +0.000 +0.107 +0.214 +0.321 C7_ry -0.381 -0.286 - 0 190 -0.095 +0.000 +0.107 +0.214 +0.321 L l _ r z +0.300 +0.900 + 1 108 +0.923 +0.738 +0.554 +0.369 +0.185 L2.rz +0.300 +0.900 + 1 108 +0.923 +0.738 +0.554 +0.369 +0.185 L3_rz +0.300 +0.900 + 1 108 +0.923 +0.738 +0.554 +0.369 +0.185 L4_rz +0.300 +0.900 + 1 108 +0.923 +0.738 +0.554 +0.369 +0.185 L5_rz +0.300 +0.900 + 1 108 +0.923 +0.738 +0.554 +0.369 +0.185 r_shoulder_rx -14.332 -1.776 +9 783 +17.357 +20.714 +21.988 +21.740 +20.530 r_shoulder_ry +2.667 +2.000 +1 333 +0.667 +0.000 -0.750 -1.500 -2.250 r_shoulder_rz +0.000 +0.000 +0 000 +0.000 +0.000 +0.000 +0.000 +0.000 l_shoulder_rx +16.737 +12.407 +8 044 +3.555 -1.491 -7.025 -12.397 -16.956 l_shoulder_ry +2.667 +2.000 +1 333 +0.667 +0.000 -0.750 -1.500 -2.250 l_shoulder_rz +0.000 +0.000 +0 000 +0.000 +0.000 +0.000 +0.000 +0.000 r_elbow_rx +6.109 +8.210 +10 267 + 12.413 + 14.780 +17.500 +22.048 +27.413 l_elbow_rx +28.153 +24.339 +20 478 + 16.710 + 13.171 +10.000 +7.396 +5.643 Chapter 2 Related Work This thesis focuses on the perception and similarity judgement of human motions that have been recorded or originated in a digital (computer) format. Prior to the development of computer representations of human movements researchers used video tape, photography, and notation systems to record and analyse human motions. Historically, we could divide the various techniques for recording human movement into those techniques that operate in real-time as the movement is performed, such as video recording, and those techniques that require large multiples of "extra" time to record the movement, such as manual notation systems or rotoscoping. We could also divide the techniques into those that represent human movement in a format immediately presentable to a human observer, such as video recording, and those techniques which record the movement using representations that require mechanical or human interpretation such as joint angle functions, symbolic notations, or impressions of the quality of the movement. However, advances in computer vision and computer graphics have erased many of the distinctions that used to be made between methods for recording human movement. Video recordings can be transformed into motion signals specifying the motion of limbs and joints. Notation of movements can be translated into motion signals and thus computer animations displays of the movements. Video from multiple cameras can be combined to form new viewpoints of movements (Ollis and Williamson 2001). Moving light displays are a special case of computer animation displays of human movements. In this chapter we present some of the methods for recording human movement and their applications. We will start by discussing some of the desired characteristics of representations of motion signals. 2.1 Representing Motion Signals Ideally, a motion signal would record the entire motion of a human being. This includes the gross and minor movements, the voluntary and involuntary, the skeletal and facial (and eyes). Ideally, we would be able to record physical movements without worrying about occlusions, recording volumes, or workspace limitations. Sometimes we would also like to record contact forces too. Given that today's technology is limited, we will restrict ourselves to motion signal representations that record only gross skeletal motions and allow only modest changes in the visual form, such as changing the external appearance while maintaining the distances between the skeletal joints. Given these constraints one such possible motion signal representation is a collection of time signals, one for each rotational angle of each skeletal joint. Burtnyk and Wein (1976) were the first to suggest that computer animations be specified by manipulating a two dimensional skeleton attached to an image of an animation character. The animator would specify the posture of the skeleton at "key frames" of the animation and the computer would interpolate the poses of the skeleton and reshape a two-dimensional image of the character to match the pose. Zeltzer (1982) presented a method of defining the skeleons of humans and animals by using a compact notation. A skeleton is a hierarchy of rotational joints linked by body segments. Motion signals are used to representing the position, pose, and motion of human-like characters. The position of the root of the skeleton specifies the position of the character's body, while the joint angles specify the posture. The translation of root as the character "runs" moves the whole body forward. The rotation of the shoulder joint moves the upper arm, lower arm, hand, and fingers automatically. Figure 2.1 illustrates two possible skeletons for specifying the pose of human-like characters. The major benefit of using a hierarchy of joints is that the minimal amount of information is used to specify the pose of character. If, instead the pose of the character is specified as the position and orientation of the limbs, then the connections between the limbs would also have to be maintained each time the pose was manipulated — requiring the specification of more information (Zeltzer 1982). Instead, by using a hierarchy of joints, the pose of the character can be specified as a hierarchy of transformations from the root to the feet, hands and head. Most skeletal representations simplify the joint structure of the human body by, for example, removing most of the joints between the spinal vertebrae, changing the shoulder joint from a compound joint to a simple ball and socket joint, and treating all joints as ideal pin or ball and socket joints and ignore the sliding rotational contacts that occur in human joints such as the wrist, elbow, and fingers. Figure 2.1: The bodies of many types of computer animated characters can be represented by chains of limb/body segments arranged in a tree like hierarchy with a single root. Of the two articulations, (a) is a high-fidelity example suitable for feature movies while (b) is more likely to be used in video games. Note how in (a) the motion of the clavicle, scapula and shoulder are modeled individually as well as the flexibility of the spine. Each circle indicates a rotational joint of one, two, or three degrees of freedom. The joints connect the body segments (bones) which are modeled as fixed-length offsets between the joints. The joint angles are used to specify the pose of the character. The joints of the hands are not drawn, but require an additional three joints per digit. An additional benefit of this representation is that torques about the joints by simulated muscles can be inte-grated over time to dynamically simulate and control the movement of a character. In the reverse direction, • head (a) Boulic et al.'s (1996) high-fidelity artic-ulation proposed for the MPEG-4 Standard. (b) BioVision's (1999) articulation used for demonstrating the capability of their motion capture studio. The motion of the hands and feet were not included in this representation but could be added. clinical gait analysts compute the torques required to create the recorded motions. Animators attach geometrical "body parts" to the joints of the hierarchy to costume and "skin" their charac-ters. Chadwick, Haumann and Parent (1989) is the classic reference on constructing a character using layers from skeleton to muscles, fat, skin and clothing. For this thesis, we used a simple visual form of a human, the "Akira" model provided by Bruderlin (1995), illustrated in Figure 2.2. One potential problem with such extremely simple body models such as the cylin-drical body model used by Leventon and Freeman (1998) (also in Figure 2.2) is that human observers may not expect them to move in human-like ways, and thus may perceive their motion differently from the mo-tion of a more realistic model. Evidence of the importance of the realism of the visual form is presented by two studies, one by Hodgins, O'Brien and Tumblin (1998, 1997) on discriminating motion and the other by Kourtzi and Shiffrar (1999) on apparent motion. We will speak more about these studies in Section 2.5. (a) Akira (b) Cylinder (c) Solid (d) Stick Figure 2.2: Simple human-like visual forms, used by (a) Bruderlin (1995), (b) Leventon and Freeman (1998), and (c) Hodgins, O'Brien and Tumblin (1998, 1997). Costumes (a) and (b) are defined by simple polygonal surfaces, the human-like costume (c) is defined by spline surfaces and the stick-figure (d) by polygons. 2.1.1 Human Movements as Motion Signals Above we quickly noted that the rotation of the joints of the skeleton would be represented using a set of motion signals. Motion signals specify the continuously changing position and pose of an articulation. A simple way of thinking about these changes is to treat each joint angle independently as a time signal, or motion curve. For example Figure 2.3 (on page 17) illustrates the joints angles over time of two walking motions. Most computer animation packages support the editing of these "[joint] function curves" and the specification of a curve by a set of key points (value, time) and an interpolation function through the points. If we interpret a motion as a collection of time signals we can apply many signal processing algorithms from frequency analysis to filtering techniques. However, because there are many additional constraints on the motion that are unspecified by the pose data, it is usually not sufficient to only use joint angle information. For example, foot contacts with the ground should not shift while the character's weight is on the foot — otherwise the character seems to have sliding feet. Also, if the size of the character is changed, then contacts with other objects need to be adjusted by changing the pose or position — for example, the position of a shorter character may need to be adjusted to keep the feet in contact with the floor. Figure 2.3: Plots of the joint angles of two walks generated by Bruderlin's (1995) Walker. The stride length, stride frequency, and stride velocity are the same for the two walks. It is time consuming for a human to determine their similarity and differences because this requires viewing a video of the motion. An eventual application of our research is to have a computer determine the similarity and differences of motions such as these. 2.1.2 Alternative Representations of Human Movements Rather than using motion signals to record human movements, we have several other options. The first is to use a human-interpreted notation system to record the posture and position of the human body as it moves. Benesh Movement Notation, Labanotation and Eshkol-Wachman Movement Notation are the three most popular systems in use today. Hutchinson Guest (1984, 1989) presents the historical development of these and other movement notation systems — going back to Beauchamp-Feuillet dance notation developed in the 1600's, Benesh Movement Notation was developed by Rudolf Benesh about the time of World War Two mostly for the recording of ballet steps, however it is a general purpose notation system. It uses anterior views of the pose of the dancer specified using a shorthand notation that is drawn from left to right as "frames" of the movement. The Benesh Institute acts as the custodian for Benesh Movement Notation and distributes The Benesh Notation Editor which runs under Microsoft Windows 95™or later. Singh, Beatty, Booth and Ryman (1983) developed a computer system for recording Benesh Movement and Dransch (1985) extended it. The primary goal of the research was an investigation into human-computer interaction issues rather than the production of computer animation displays of the transcribed movements. Labanotation was developed by Rudolf von Laban in the 1930's as a method for both recording dance, and later, for analysing movements for their qualities such as effort, duration, timing, and spatial extent (Hutchinson 1970). Basic Laban Notation indicates the pose and movement of the dancer using symbols drawn to either side of a vertical staff which indicates the progression of time from bottom to top. The movement of the hips and legs is indicated closest to the staff with the left side of the body indicated to the left of the staff, and the right side of the body on the right side of the staff. As symbols are placed further outward they indicate the pose of the shoulders, arms and legs. Movement through space is indicated with diagrams drawn specifying the relative position and orientation of the dancers on the dance floor. Since 1968, the Dance Notation Bureau has recorded ballet and modern dances using Laban Notation. Calvert and Chapman (1978), Badler and Smoliar (1979) built a computer based system to translate from Laban Notation to a computer driven movie of moving figures. Experience from Calvert's system was later used to develop Life Forms (Jetha, Bruderlin, Calvert and Mah 1993) which is now marketed by Credo Interactive (2001). Experience from Badler et al.'s system was later used to develop Jack: The Human Modeling and Simulation System (2001). Laban's extension to notating only the temporal-spatial path of a motion was his concept of "effort." Laban defined four effort elements' which attempt to capture the interaction between path and temporal elements of a movement which often involve coordinated movements of the whole body. (Chi 1999) writes: 'The elements are indirect versus direct, light versus strong, sustained versus quick, and free versus bound. Initially, we tried to deduce movement characteristics from motion capture data [motion sig-nals]. We collected 3D motion capture data of a Certified Movement Analyst (CMA) trained in L M A [Laban Movement Analysis] performing numerous examples of combinations of Ef-fort Elements. Analysis of the motion capture data led only to the most obvious conclusions; i.e.: Sudden is short in duration, Sustained is longer in duration, and Strong tends to have large accelerations. The inability to deduce the more subtle characteristic qualities of Effort arose from several factors . . . Effort is embodied in the whole person and manifested in all body parts . . . movements such as visual attention, changes in muscular tension, facial expressions, and breath patterns are not adequately captured by current motion capture technology. Chi et al. (1999, 1999) built and evaluated a keyframe interpolation system utilizing Laban's concepts of movement effort elements. By adjusting the "effort parameters" used to interpolated key poses of a figure, Chi was somewhat successful at creating movements which were recognized by Laban Movement Analysts (LMAs) as having the specified effort elements. Eshkol-Wachman movement notation was also developed for dance, but unlike Laban Notation, which spec-ifies all movement relative to the pelvis and is specific to the human body, Eshkol-Wachman can be used to notate non-human and human movements rooted at a foot or any other limb. Movements of body parts are recorded by noting in the appropriate body part row, the starting posture, direction of movement, and ending posture. The passage of time occurs left to right, but is not precisely specified. Jacobs et al. (1988) sug-gested using Eshkol-Wachman movement notation as a formal descriptive system to record the movements of animals and people observed by anthropologists and ethonobiolgists rather than the typical practice of recording "behaviors." The major limitation of "pen and paper" notation systems is that they were developed for use by humans as both the recorders and the interpreters. Thus to "observe" a motion recorded in Laban, or Benesh, or Eshkol-Wachman notation you would have have to either understand the notation, or have someone else — possibly a computer program — translate a movement from its notated form to a physical or visual movement. The benefit of these notation systems is that they require very little equipment, perhaps only a video camera and video tape recorder to aide in the accurate notation of complex movements. Many social dancers are fond of using Laban or Benesh-like notation systems to record dance steps in small notebooks during the course of a class and there are frequent enquiries for information about "the best" system to use on the newsgroup rec . a r t s . dance. 2.2 Recording Physical Movements as Motion Signals Motion capture is the task of measuring the translation and rotation of an object in physical space and then recording that information in a computer readable format as motion signals. To capture the motion of a human body, it is necessary to measure the position and orientation of the limbs and body segments by tracking small markers or sensors placed on the body at joints and locations which shift as little as possible relative to the skeleton as movements are performed. Software is then used to combine the marker spatial positions with the known position of the markers on the body and the measured limb and body segment lengths into joint angles. Bodenheimer et al. (1997) and Menache (1999) describe the process of motion capture. Kines (1998b, 1998a) specifically discusses the process of planning and directing a motion capture session. Mulder (1994) reviewed the technologies used in and the commercial suppliers of human movement tracking systems. 2.2.1 Historical Development of Motion Capture Prior to computer based motion capture systems, and even before film recording, scientists attempted to record motion using mechanical devices and multiple cameras. The earliest of these appears to be Etienne-Jules Marey, 1830-1906, who used and developed many devices to record and visualize motion and dynamic phenomena: walking, running, jumping, falling, of humans, horses, cats, heart rate, pulse rate, breathing, etc. An exhibition in January, 2000 at the Fondation Electricité de France, Espace Electra (6 rue Recamier, Paris 7) was devoted to his works. The exposition web site is www.expo-marey.com/ANGLAIS/ home. html. At approximately the same time Eadweard Muybridge, 1830-1904, was investigating the use of stop-action photography to record human and animal movements. Muybridge used arrays of twelve to twenty-four cam-eras equipped with fast shutters that were synchronized to take pictures in quick succession. Two volumes of his photographs were reprinted recently by Dover Publications: Human Figure in Motion (1955), and Animals in Motion (1957). 2.2.2 Who Records Motion and What do They Record Motions are recorded for commercial, research, and medical applications. However, due to the cost of motion capture equipment and facilities, recording motion is a very rare activity compared to video recording or live visual motion observation. For situations where the motion does not need to be viewed from a new angle, digitally costumed, or analysed as part of a medical procedure - video recording is often done instead. 2.2.2.1 Visual Effects Companies Visual effects companies capture motion for a variety of reasons. The first is the hope that the motion data will capture all of the nuances of the performer. For example, Digital Domain captured the motion of Michael Jackson for his video "Ghosts" (Jackson 1997) so that he could be digitally costumed as a skeleton. This type of motion capture is often termed performance capture or digital costuming. 2.2.2.2 Digital Puppeteers Digital puppeting or performance animation is created by rendering an imaginary creature whose movement is controlled by tracking the movement of a human (Sturman 1998). The accuracy of the motion tracking is often very low as a trade off between speed and accuracy. "Speed" in this context is the lag, or transport delay, between the movement of the puppeteer, and the computer generated display of the movement. By minimizing the lag, the puppet can react to other actors and events in a lifelike fashion in real-time. 2.2.2.3 Kinesiologists and Clinical Gait Analysts Kinesiologists study the biomechanical properties of human bodies. Included in this is the biomechanical study of how people move and perform tasks. The goal of kinesiologists is to understand how movement can be improved and to determine how various factors influence the performance - such as modifications to training or equipment. Thus it is often necessary to make precise measurements of how the person moves. Perhaps the most exhaustive example of the relationship between movement and judged performance was performed by Takei (1989). He and his colleagues have presented a statistical deterministic model analysing the relationship between the mechanical performance of elite male gymnasts performing a hand-spring vault and the resulting judge's marks at the 1987 Pan American Games. They have also examined the hand-spring with full turn (1998), female gymnasts performing the hand-spring vault (1990), the hand-spring and salto forward tucked vault (1990), the compulsory vault (1991a, 1991b, 1992, 1996) and dismount from the horizontal bar (1992). Clinical gait analysis is a union of kinesiology and surgical medicine. Clinical gait analysis is the sys-tematic study of human walking and is used to diagnose medical conditions that affect how people walk (Whittle 1996). The goal of clinical gait analysis is to decide which courses of medical or surgical treatment should be employed to improved the gait of a patient. For a patient with cerebral palsy, this may include or-thotics to brace joints or stretch muscles, botox2 injections to relax muscles, or surgery to lengthen tendons (Gage 1991). Other applications of gait analysis include McClay's (1995) studies of running injuries and treatments. These applications often involve the collection of additional measurements such as forces ex-erted on the ground and electromyography (EMG) to measure the electrical activity of contracting muscles. Family doctors can perform many tests in their examination rooms to help them diagnose neurological disor-ders, such as hemiplegia (most often caused by a stroke), Parkinson's disease, cerebellar ataxia, foot drop, or sensory ataxia. Many of these disorders produce "striking and characteristic gaits" (Swartz 1989). However, as medical researchers propose treatment regimens to slow or reverse the effect of neurological disorders, making an early diagnosis of a movement disorder becomes more important (Blanchet 1999). Recently re-searchers have been investigating methods of detecting early progression of movement disorders — which often have genetic markers — allowing pharmaceutical treatment to begin before clinical symptoms appear. For example, Huntington's disease is a hereditary disorder of the central nervous system, with symptoms appearing between the ages of 30 to 50 years (Family Caregiver Alliance 1996). Characterized by chronic progressive chorea (quick, jerking, uncontrollable movements of the limbs, trunk and face) and dementia 2Botulinum Toxin: The toxin responsible for botumlism. This toxin is used in many other situations when continuous muscle tension result in pain — such as migraines or neck pain — injections typically need to be repeated every six to twelve months. (poor short-term memory and judgment, and depression, irritability and apathy) without remissions, phar-maceutical treatment can help control the involuntary movements and emotional disorders.3 For further reading in gait analysis we suggest Perry (1992), the collection by Craik and Oatis (1995), or for quicker reading Whittle's (1996) introduction to gait analysis covering normal and pathological gait as well as visual and measurement techniques for analysing gait. For further reading on clinical motion analysis we suggest Cappozzo and Paul's (1997) chapter on the history of instrumented human motion analysis, and the collections edited by Allard, Stokes and Blanchi (1995) and Allard, Cappozzo, Lundberg and Vaughan (1997). 2.2.3 Cost and Size of a Motion Capture Facility Motion capture facilities cost on the order of $60,000-$300,000. For example, Vicon Motion Systems Mcam system with eight cameras, each with 1000 by 1000 pixels and capable of 120 frames per second capable of tracking 9 mm spherical markers was quoted at US$200,000 in December 2000. Even renting a motion capture facility is expensive. Biovision charges US$4000 to capture and process "20 simple motions" which include motions of less than five seconds duration, no props, no close contact with objects such as the floor, and only one actor. So, if we had hired BioVision to capture the thirty-five walking movements used in our experiments, we could have expected to have paid about US$7000 (BioVision 1999). We do not know of any Vancouver facilities that are available for rental though Electronic Arts and Children's hospital (Sunnyhill) have motion capture facilities. In addition to recording equipment, a motion capture studio requires a large space to allow full body motion through the space as well as enough room for all sensor and analysis equipment. To capture the performance of an actor, a twenty foot on a side square area may be necessary. A discussion of "how large a clinical gait lab should be" is available at the Clinical Gait Analysis web site (Various 2001). The following lab sizes are drawn from that discussion. For camera based Vicon 8 system 3Medical Anecdote: My wife Mary is a medical doctor and had a difficult elderly patient who was suffering emotional swings and was possibly physically abusing his wife. Referrals to a neurologist did not result in a medical diagnosis and there was not enough evidence of abuse to take action. No one knew really what was happening or what action to take. A local nurse who is responsible for home visits saw the gentleman walking down the street and suggested, based on her observations, that he might be suffering from Huntington's. None of the doctors involved had probably ever seen him walk for long periods. The diagnosis was later confirmed. from Oxford Metrics,4 a minimum lab size of 6 by 11 meters is suggested. The Laboratoire de Cinésiologie at the Hôpital Cantonal Universitaire in Geneva, Switzerland has a walkway that is 39 meters long, 2.8 meters wide and 2.8 meters tall. While the entire length is used to gather E M G data, only 7 by 0.9 by 1.7 meters of the space is inside the recording volume of their five camera VICON system. Subjects walk for 4.7 meters before and 4.7 meters after the visual recording volume and electromyography (EMG) data is collected over the entire walk. The "extra" distance before and after the recording volume is required by The Accreditation Board of Clin-ical Movement Analysis Laboratories in Europe ( A B C M A L E ) . A B C M A L E requires that subjects should be able "to take at least three steps in a straight line before and after traversing the measurement vol-ume. .. Continuous walking investigations may be facilitated by incorporation into the measurement volume of a circular or 'figure of eight' format."5 The total space required, the cost of the equipment and the salaries of the technical staff necessary for motion capture may seem excessive for what seems to amount to "digital video recording." In any case, the visual effects industry is convinced that there are effects, such as digitally costuming a well known dancer, that cannot be created in any other way. However, for clinical gait analysts the effectiveness of motion capture facilities is defined by the ability of the systems to detect deviations in gaits beyond the abilities of skilled observers. Over the last couple of decades, clinical gait analysis has evolved from visual observation to motion capture based analysis and in turn has increased the effectiveness of surgical operations on children with cerebral palsy (Gage 1991). Part of the increase in effectiveness of clinical gait analysis is felt to be due to use of motion capture and computer models of human movement. As to the use of motion capture, Saleh and Murdoch (1985) tested the ability of skilled observers to record gait deviations of five amputees whose prostheses had been adjusted in a systematic way to effect gait deviations. The observers were able to find only 22.2% of the predicted gait deviations and were unable to comment on 15.6% of all the required observations. The motion capture system combined with analysis detected 3.4 times as many deviations as did visual observation. 4This is a optical system using infrared lights to illuminate retro-reflective markers observed by eight infrared cameras. 5 h t t p : / / a b c m a l e . e e . u n i a n . i t / U R D B V e r / U R D B - 9 9 - F r a m / F r a m e l . a s p 2.2.4 Markerless Video Based Motion Capture In comparison to marker based motion capture — which uses either a combination of markers and video cameras, or radio receivers and transmitters to track the movement of the human body — markerless video based motion capture systems attempt to record the movement of the human body without the need for markers or special clothing. For a survey of the work on computing articulated motion from video we suggest either Cédras and Shah (1995) or Aggarwal and Cai (1997). As an example of the latest work in this area, Deutscher, Blake and Reid (2000) presented a method for recovering the full articulated body motions without special preparation of subjects or restrictive assumptions. 2.2.5 Motion Capture and Analysis Software Motion capture equipment is usually bundled with analysis software, although there is sometimes software available from independent vendors. However, we have formed the impression that many clinical gait anal-ysis labs use a combination of turn-key software and customized analysis tools. This is especially so in research labs. While comparisons of the technical capabilities of commercially available systems have been made (Richards 1998), it is difficult to find independent reviews of the software systems. One example of this type of software is GaitLab, an inexpensive clinical gait analysis software package. Vaughan's (1999) GaitCD package contains a collection of captured gaits from normal subjects and those with cerebral palsy, and software to process the files. Input are measured marker positions and ground forces and computed are joint angles, velocities, accelerations, and torques. The motion files included with the GaitCD package were captured using hardware and software by Peak Performance Technologies of Englewood, Colorado (2001). Whereas GaitLab requires a motion capture facility, an alternative is to manually track the movement of the human across video frames. This process was introduced in 1917 by Max Fleischer6 as a method of copying previously filmed live action footage using a rotoscope, which projected each frame of the film onto an animator's drawing board (Fleischer 1917). For human movement analysis Schleihauf's (1999) K A Software allows a researcher to record a motion 'Animator of Popeye, Superman, and Betty Boop. using two orthogonal video cameras and then manually identify joint positions in each video frame. K A Software can be used to display stick figures and motion paths along with graphs of joint angles, velocities and accelerations, and the original video frames. The package is aimed at undergraduate biomechanics students in custom-designed class projects. Figure 2.4 is a screen shot of the software being used to analyse a round house kick. K A Software is an educational freeware software suite for kinematics analysis designed at San Francisco State University, Department of Kinesiology. l:;'.K3 Multiview Plot RSKICK File Data View Help ME1 R e s e a r c h e r : B o b S c h l e i h a u f I n s t i t u t i o n : S a n F r a n c i s c o S t a t e U n i v e r s i t y P a r t i c i p a n t : R o b _ S c h l e i h a u f C o n d i t i o n : s k i l l e d r o u n d h o u s e k i c k S m o o t h i n g : 2 L a s t D i g i t i z e d : 2 / 2 6 / 9 8 1 1 : 1 3 : 5 2 P M D E F F i l e n a m e : R s k i c k v v K \ T 0 3 4 2 0 4 2 3 0 . 5 2 9 0 . 6 6 2 0 . 8 1 7 0 . 9 9 3 1 . 1 8 2 1 3 6 3 1 5 0 5 1 . 5 6 0 1 5 2 4 1 . 4 7 3 0 0 0 7 0 . 0 0 1 0 0 0 0 . 0 2 2 0 . 0 8 2 0 . 1 8 9 0 . 3 4 8 0 . 5 4 6 0 . 7 5 0 0 . 9 1 2 1 . 0 0 3 1 . 0 1 4 0 7 4 9 0 7 6 2 0 . 7 6 9 0 . 7 5 9 0 . 7 3 8 0 . 7 2 4 0 . 7 3 0 0 . 7 6 3 0 . 8 2 6 0 . 9 1 1 0 . 9 7 9 0 . 9 8 6 4 . 9 1 5 . 7 5 7 . 1 3 9 . 0 1 1 1 . 1 9 1 3 5 6 1 5 . 5 4 1 5 . 7 0 1 3 . 1 2 8 . 8 2 4 . 4 6 2 . 5 1 5 6 . 0 9 8 4 0 2 1 2 2 . 3 8 1 5 7 . 6 5 1 8 3 . 7 5 1 8 7 . 6 9 1 5 8 . 4 6 1 9 4 . 0 0 3 0 1 . 9 0 3 3 9 . 8 7 3 1 6 . 3 6 2 7 0 . 6 7 1 5 . 5 4 V - R T o E V 10 1 5 t i m e (1 ISO s e c ) 3 0 0 : 2 5 0 2 0 0 1 5 0 I n o 5 0 -1 0 1 5 t i m e ( 1 fèO s e c ) Step | f7 Point j  Segment |  Joint | IRTÔË p Q w k S t l ! |7 G w k P i c Exit | Figure 2.4: Screen shot of K A Software being used to analyse a roundhouse kick. The left column contains images from recorded video. The centre column shows three orthogonal views of a stick figure performing the kick the path of the left toe of the subject. The right column contains at the top, a spreadsheet of measured and computed data, and below, velocity and acceleration graphs of the right toe. 2.3 Manipulating the Quality, Style, and Path of Movements One of the motivations of our research is the desire to build higher-level computer animation tools that operate at a level above motion signals. For example, we would like to manipulate the quality or style of a movement without affecting its general path, or manipulate the path or action of a movement without affecting its quality or style. 2.3.1 Manipulating Movement Quality and Style One of the first papers on editing the quality of a motion while preserving the path is by Loomis, Poizner, Bellugi and Hollerbach (1983). Loomis et al. were studying point-light recordings and renderings of Amer-ican Sign Language. By rotoscoping recorded movements of native signers, Loomis et al. were able to create synthetic signs and manipulate their movement quality independently of spatial path. The application of this work was the creation of complex stimuli for psycholinguistic experiments. Much later, Unuma, Anjyo and Takeuchi (1995) and also Unuma and Takeuchi (1991) attempted to use Fourier analysis to represent cyclical motions such as running and walking. By interpolating and extrapo-lating between the Fourier coefficients they were able to create motions containing "emotive" content such as happy or sad walks, or normal and brisk walks. Their goal was to be able to extract the "briskness" factor from a walk and add it to a run to create a "brisk run." Amaya, Bruderlin and Calvert (1996) applied signal processing techniques to create an "emotional trans-form" capable of determining the difference between a neutral and emotional movement as contained in the timing and amplitude differences of the motions, Rose, Cohen and Bodenheimer (1998) presented a system that parameterizes cyclical motion captured "verbs", such as a walk, with "adverbs" that alter the motion. For example, they can create a continu-ous walking motion up and down hills, and turning left and right. They can also add emotive variations such as happiness and sadness. The target application for this work is the real-time control of a digital puppet or video games character, with smooth transitions between motion verbs and adverbs. Perlin (1995) presented a method to generate motion with "personality" by using expressions containing pseudo-random noise functions to generated the joint angle motions. Also aimed at real-time motion gen-eration, Perlin's system allows the operator to control a character while specifying particular moods and attitudes to be conveyed. Kinematic constraints prevent "impossible" transitions. The above approaches use kinematics, as represented by the motion signals, rather than dynamics to model the motions. In contrast, Phillips and Badler (1991) presented a method of maintaining dynamic constraints such as balance and stability, which they consider "characteristics of human-like movement," while the user specifies goal-oriented motions. The weakness of these techniques is that they are difficult to evaluate and that there have been few attempts to validate the effect of the proposed manipulation methods. One possible evaluation technique is to employ Laban Movement Analysts to evaluate the Effort components of the resulting movements. The only motion creation technique that has been evaluated by Laban Movement Analysts is Chi's (1999) PhD Thesis on creating motion specified through a combination of keyframing and Laban's Effort descrip-tors.7 Chi created several video sequences of a character moving its arms, and then presented these se-quences with and without effort components. Four Laban Movement Analysts (LMAs) viewed the video sequences twice, the first time to familiarize themselves with the character and its movement, and a second time to judge which of the four Effort elements8 were present in each segment to which extent (positive, not present, negative). The L M A s judged at least 53% of the Effort elements correctly — that is in agreement with the input settings of Chi's system, with the in-house consultant judging 76.6% correctly. The major-ity of the incorrect judgements were "not present" judgements rather than opposite to the intended effort element. The other technique, is to have participants in an experimental setting categorize the resulting movements according to their emotional content or attempt to determine which of two movements was manipulated instead of simply recorded. The first technique has been used by Paterson and Pollick (2000, 2001, 2001) to determine the role of velocity in affect discrimination. Using both movements created by actors attempting to express different affects and neutral movements manipulated with signal processing similar to Amaya et al.'s (1996) techniques, Paterson et al. have participants name the resulting affects associated with each 7 Also Chi, Badler and Pforisch ( 1999). 8 Indirect versus Direct, Light versus Strong, Sustained versus Quick, Free versus Bound. movement. Their results indicate a strong effect of velocity on the modulation of affect. 2.3.2 Manipulating Movement Path The term movement path can be used to refer to either the gross spatial path of the body of the figure or to the action of the figure. Movement paths are most often manipulated for two purposes. The first is to reduce the number of key frames used to define the motion. The second is to reuse a motion in a new situation such as altering a step of a walk cycle to adjust for uneven terrain, to retarget an existing motion to drive a new character articulation, or to adjust the start or end of the motion to transition from one motion segment to another. Because motion capture records the value of every joint angle of an articulation at every sample time there is a huge redundancy in the specification of the motion signals. This redundancy is not present in the movements created by character animators who specify a motion with only the necessary keyframes. An animator working with the redundant information recorded by motion capture would need to adjust many more keyframes than they would have to if they were working with the typically sparse keyframes specified by another animator. Thus there is a need to reduce the keyframes recorded by motion capture to be able to work efficiently with motion capture data. Balaguer and Gobbetti (1995) presented a method to reduce the number of points used to define a three dimensional motion path (with timing). However we are unaware of any algorithms specifically targeted at the task of data reduction for an articulation. This is a difficult task because the output from such an algorithm should aim to capture the important poses, not just tradeoff error tolerances against data reduction. Various techniques have been suggested for manipulating a motion captured movement path without first reducing the keyframes. Usually this means employing signal processing techniques to make changes to the motion while minimizing changes to the motion quality. Bruderlin and Williams (1995) suggested the use of multiresolution motion filtering, multitarget motion interpolation with dynamic time warping, waveshaping and motion displacement mapping to adjust move-ments without having to adjust individual keyframes. These techniques allow large time scale adjustments such as exaggeration while preserving fine details such as those thought to define the style of the movement. Similarly, Witkin and Popovic (1995) presented a motion warping technique to combine motion clips by minimizing the energy necessary to transition between the movements. Guenter, Rose, Bodenheimer and Cohen (1996) also worked on the problem of creating transitions between motion clips using space-time constraints and inverse kinematics to generate "seamless and dynamically plausible transitions." Their technique attempted to address the limitations of purely motion signal based approaches: motion signals do not describe a deep understanding of the motion's structure. Thus the preser-vation of "dynamic plausibility," perhaps the strongest reason to use motion capture to record the movements in the first place, is not insured. Along these same lines, Popovic and Witkin (1999, 1999) presented a physi-cally based space-time optimization approach to transform movements while maintaining essential physical properties, such as momentum. Inherent in their technique is the need to specific the physical characteristics of the figure such as segment mass distributions. Finally, a combination of spacetime constraints and signal processing approaches has been suggested by Gleicher et al. Gleicher and Litwinowicz (1996) presented a constraint based approach to editing motion. The constraints, specified as points and times (which naturally extends to motion paths) allow motions to be modified while maintaining the constraints. Gleicher extended this work to interactive editing of motions and to retargetting a motion to a new articulation while maintaining constraints — even in the event the articulation changed in scale during the course of the animation (1997, 1998). Gleicher's (2001) latest work allows the spatial path of movements to be adjusted, for example a straight line walk can be curved, and circular dances can be straightened. Again, there have been very few evaluations of these techniques beyond their publication successes. 2.4 Motion Signal Databases One of the motivations of this thesis is the need to be able to retrieve motion signals stored in large databases based on the style, action, or similarity of the movements. The simplest motion signal database is just a collection of motion files, perhaps organized into hierarchical folders or directories. Commercial motion libraries are organized by type of action. For example, the Viewpoint Digital Motion Library (2001) contains at this writing over 400 motions categorized as Dancing, Fighting, Miscellaneous, Jogging/Running, Sports and Walking motions. Credo Interactive's MeGa MoCap (Credo Interactive 2001) library is organized into the categories: Sitting and Talking & Drinking, Walking and Running, Walking with Objects, Walking and Running 2, Negotiating Obstacles, Carrying the Weight, Motion Challenged, Perky Posturing, Dirty Work, Ambient Moves, At Home & Office, and Hard Movin' . Motek's Stock Moves (Motek 2001) are categorized as Walk, Run, Fall, Squat, Shot, Wait, Jump, Saltos, Slips and Slides, and Freestyle. Similar to a hierarchical classification is the scheme suggested by Kines (1998a, 1998b) for a video game with multiple characters and multiple motions per character. Kines's scheme entails creating the file name for the motion by concatenating codes for the character and action, this is illustrated in Figure 2.5. We have been unable to find any motion signal databases that are organized or searchable other than by hierarchical categories. One reason may be that the current motion signal databases contain less than a thou-sand motions. For databases of video recordings the search techniques are a bit more refined. For example, Cheung and Zakhor (2000) presented a method of determining the number of copies of digital video files found in the web. By selecting seed images from each video file they are able to produce signatures for each file that allows the video files to be clustered according to "similarity." 2.4.1 Motion Generators As an alternative to the capture and storage of typical human movements such as walking, a computer program can be written to automatically generate time signals that move a human articulation in a simulation of the motion. Such a program, which we call a motion generator, provides the user with a collection of parameters that "capture" the possible variations of the motions. As an example, Bruderlin (1995) created Walker, a program to simulate straight-line walking, that creates an animation of a walking figure. As illustrated in Figure 2.6, the program uses twenty-nine inputs to vary the style of the walk by computing the motion of thirty-six joints defining eighty-eight rotational axes. The inputs are: walk velocity, step length, step frequency, shoulder rotation, degrees of arm swing, angle arms are raised from the sides, elbow rotation (min, max), torso tilt (forward/backward), torso sway (side to side), SG S A 001A Character: SG SuperGuy EV Evil Villainess HB Henchman B Activity: Activity modifier: S Standing A Attack W Walking T Transition C Crouching L Loop Direction 001 Forward 002 Right 003 Left 004 Low 005 High Version 014 Shoot Figure 2.5: Naming convention for video game motion files with multiple characters as proposed by Kines (1998b) for animations of multiple characters in a video game. The naming convention is a shorthand that does not specify all the details of the motion. For example: SGSA001A is SuperGuy punching forward from a standing position. Left out of this description is the size of the capture space (small), the duration of the movement (15 frames), whether the motion can be looped continuously (no), and the need for any props torso rotation (left to right), lateral displacement, rotation of the pelvis about the vertical axis and list from side to side, bounciness on foot strike as the supporting knee bends, amount of over stride, hip flexion as leg swings forward, knee angle during swing, midstride and impact, stride width (distance left to right of foot placements), angle of foot with respect to straight forward ("pigeon or duck toes"), and whether the heel or toe strikes the ground first. A motion generator trades off analysis and programming time against a huge decrease in the storage required to hold all the possible motions the generator is capable of creating. The amount of time it takes to generate a new walk is very small in the case of Walker, new walks are generated in real time with adjustments to the parameters affecting the walking motion immediately. However these are not the only tradeoffs when deciding between the use of recorded motion versus a motion generator. Motion generators, by their fixed implementation as computer codes, imply a fixed definition of the types of motions they are capable of generating. Although the parameters to the program allow a large amount of flexibility, there are always constraints and assumptions hidden in the implementation of the program that cannot be excised if the program is to automatically generate motion. If the program is so completely flexible as to have no constraints, then we require the use of general purpose keyframe animation (no). Figure 2.6: Screen shot of kinematic walk simulator output and controls created by Bruderlin (1995). The program uses twenty-nine inputs to vary the style of the walk by computing the motion of thirty-six joints totaling eighty-three degrees of freedom. The inputs are: walk velocity, step length, step frequency, shoulder rotation, degrees of arm swing, angle arms are raised from the sides, elbow rotation (min, max), torso tilt (forward/backward), torso sway (side to side), torso rotation (left to right), lateral displacement, rotation of the pelvis about the vertical axis and list from side to side, bounciness on foot strike as the supporting knee bends, amount of over stride, hip flexion as leg swings forward, knee angle during swing, midstride and impact, stride width (distance left to right of foot placements), angle of foot with respect to straight forward ("pigeon or duck toes"), and whether the heel or toe strikes the ground first. system, where an animator specifies the motion, not a computer algorithm. 2A.2 Motion Galleries When à motion generator has a non-linear, or non-obvious, mapping from the input parameters to the output motions it is difficult for a user to create a specific type of motion by picking values for the input parameters. Marks et al. (1997)9 proposed the use of Design Galleries as an automatic way to explore the .possible outputs of a simulation system. By picking a suitable set of metrics on the generated (output) motions a dispersal algorithm searches for a set of input-parameter vectors to optimally disperse the generated motions. Two options were proposed for presenting the generated motions: an organization algorithm could be used to build a hierarchy for "easy and intuitive browsing by the user," or a principal components analysis could be used to present a two or three dimensional layout of principal dimensions of motion variation. For example, Marks et al. (1997) built a Design Gallery for an actuated double pendulum, see Figure 2.7. The simulation of the pendulum has fourteen input parameters: the rod lengths, the bob masses, the initial angular positions and velocities of the rods, and the amplitude, frequency, and phase of both sinusoidal torques. For each input, 20 seconds of motion are simulated and twelve metrics are computed on the output motion: the difference in rod lengths and masses, the average Cartesian coordinates of each bob, and loga-rithms of the average angular velocity, the number of velocity reversals, and the number of revolutions for each rod. • . We have recreated the system described by Marks et al. (1997) and have found its motion features to be extremely lacking. A dynamical system such as a two link pendulum is a chaotic system which undergoes transitions between many unstable states. Attempting to describe the behaviour of such a system with statistics gathered over its entire motion simulation does not lead to an intuitive description of its behaviour. A simpler example of using motion features was presented by Lamouret and van de Panne (1996). They used a motion signal database to store various dynamically simulated hopping motions of a Luxo Lamp moving across rough terrain10. To generate new hopping sequences they automatically pick the next hop from the 'Marks, Andalman, Beardsley, Freeman, Gibson, Hodgins, Kang, Mirtich, Pfisler, Ruml, Ryall, Seims and Shieber (1997) l 0 The Luxo Lamp was first animated by John Lasserter of Pixar the short film "Luxo Jr" (USA 1986), see h t t p : / /www. p i x a r . c o m / s h o r t s / l j r / . Figure 2.7: An actuated double pendulum rooted at h and a sample of the possible traces of the position of m2 of the pendulum. Marks et al. (1997) presented the algorithm used to create the samples and we coded our own copy of their algorithm. Although many traces seem to indicate that several of these systems have the same behaviour, the behaviour of all of these systems changes over time from "pattern" to "chaotic" due to the influence of gravity. Unfortunately, a static trace of the movement of the end of the pendulum fails to capture the various patterns the pendulum exhibits over time. For example, if we compare the motion of the system responsible for the two "square" traces (marked with a *), we find that they only have a similar "square" trajectory near the same time — the remainder of the time, the two trajectories are hugely different because the ratio of the rod lengths are reversed — 83:17 versus 14:86. database based on a simple rule such as "short jumps" or "long jumps" and then blended the transition between jumps. Each jump was indexed by the length and height of the jump, as illustrated in Figure 2.8. Figure 2.8: Jumps in Lamouret and van de Panne's motion signal database are indexed by jump distance (a) and hop height (b). Adapted from Lamouret and van de Panne (1996, Figure 2) 2.5 Human Judgements of Motion The most popular theory for how human perceive and judge the movements of other humans is that the visual centres of our brains have evolved to recognize the articulated movement of human bodies separate from the recognition of the visual form of human bodies. 2.5.1 Separating Movement Pattern from Visual Form: Point Light Displays The use of point light displays to study the "motion pattern" of humans and animals rather than the visual presentation of their moving bodies was proposed by Johansson (1973, 1975). Point light displays" (PLDs) are created by attaching small lights or retroreflective patches to the joints of an actor and filming their movement so that the bright points are recorded moving in an otherwise dark environment. Point light displays of humans moving (walking, running, dancing) are easily, quickly, and accurately recog-nized by humans. The sparsity of these displays has led many people to wonder if humans use a structured "Sometimes termed "Moving Light Displays." or unstructured model for interpreting motion — that is do they try to fit an articulated model to the moving points, or do they use some other feature of the movement to recognize it. Interestingly, Cohen et al. (2000) have shown that humans are biased towards an ability to recognize the movements of humans and our ability to recognize the movements of animals using PLDs when masked with extra randomly moving points is only slightly better than chance. This implies that we use knowledge of our own movements to encode and recognize the movements of other humans, and that our ability of encode and recognize the movements of animals relies on additional features. Cutting and Kozlowski (1977) studied the ability of viewers to recognize themselves and their friends from PLDs. They later extended this work to recognizing the sex of human walkers (Kozlowski and Cutting 1977) and produced a program to generate synthetic walkers as PLDs (Cutting 1978) which is still used by researchers today. Dittrich, Troscianko, Lea and Morgan (1996) studied the problem of judging emotional state as commu-nicated by two trained dancers (one male, one female) viewed as either fully lit video scenes12 or PLDs. Recognition of emotionality was higher for video, but the results for PLDs were also significantly above chance. Paterson et al. (2000, 2001) also found this difference which suggests that point light displays impose a burden for some recognition and categorization tasks. Accuracy of Recognition PLD Video Chance Paterson and Pollick (2001) 59% 71% 20% Dittrich, Troscianko, Lea and Morgan (1996) 63% 88% 20% Other research with PLDs has demonstrated that cats can be trained to discriminate between a PLD of a walking cat and a distractor (Blake 1993), that choreographed walks can be reproduced from observation (Ille and Cadopi 1995), that the motion of the extremities is important (Mather, Radford and West 1992) which was verified with an experiment to test the conspicuousness of pedestrians at night (Owens, Antonoff and Francis 1994), and that the movement of animals can be recognized (Mather and West 1993). It is important to note that point light displays are one extreme of computer animation displays — the other being super-realistic renderings of the human form. Point light displays are easily created using computer l 2 The actors were hooded so that facial expressions were not recorded. animation displays by using a black visual form with white spheres attached to the body near the joints. By using a black visual form and a black rendering environment, the movement of figure self-occludes the white spheres in a optically correct manner. We present in Figure 2.9 frames from a computer animation display of the "Akira" visual form where Akira's usual appearance has been modified to be all black with small white spheres attached at the joints. Frame 39 Frame 41 Frame 43 Frame 45 Frame 47 Frame 49 Frame 51 Frame 53 Figure 2.9: Synthetic point light display created by using the "Akira" visual form with an all black surface material and attachement of white spheres at the joints. The walking motion is the same as in Figure 1.2. Cédras and Shah (1994) survey the attempts to analyse point light displays with computer vision algorithms. Little and Boyd (1995, 1998) presented a method to describe the motion of humans walking by analysing video recordings of walks. They then extended their analysis to images of point light displays placed at the joints of the humans. Their conclusion was that point light displays sample the optical flow at the best points — the joints — and preserve many statistical features of the motion (Boyd and Little 1997). 2.5.2 Apparent Motion If two images are briefly flashed in rapid succession, observers will sometime report seeing motion between the two images. For example, if both images contain a dot, the first on the left and the second on the right then the perception reported is that a single dot is moving back and forth. The amount of time between the presentation of the two images affects the perception of movement: too small, and two dot flashing nearly simultaneously are perceived, too long and two dots alternately flashing are perceived. While elementary motion detectors such as Rienchardt-detectors are sufficient to explain the perception of two flashing dots the images can be made of two poses of the human body. In this case the viewer will perceive movement between the two poses. Shiffrar and Freyd (1990, 1993) studied the motion paths perceived by subjects when they were presented with apparent motion displays of the human body. If the inter-stimulus times were too small then the subjects perceived the "shortest" motion path possible, which may have violated biomechanical limitations. Thus a longer motion path between two poses requires a longer inter-stimulus time. This implies that the perceptual system responsible for coding human movements has built-in biases regarding the amount of time a movement requires. Kourtzi and Shiffrar (1999) later examined the possibility that human observers take into account movement limitations of the human body when perceiving apparent motion. They found that subjects perceived the apparent motion of a realistic geometric model of the human body differently than a body made of cylinders. The use of realistic model was more likely to produce motion paths that did not violate biomechanical constraints. 2.5.3 Interaction of Movement and Shape Mak and Vera (1999) studied the effects of motion and shape on categorization. Subjects were presented with either two geometric objects or two animal silhouettes and told the properties of the objects. They were then shown the objects moving in specific ways such a jumping or linear movements. The movement of third novel object or animal was then displayed. The subjects were then asked to categorize the new object or animal as belonging to one of the first two objects. Four-year-olds categorized primarily on motion cues, while seven-year-olds tended to use motion cues for animals but not for geometric shapes. This appreciation of the uniqueness of motion to animals was confirmed by testing adults on the same tasks — adults categorize animals significantly more often on motion. 2.5.4 Phase Variability of Oscillating Motion Bingham, Schmidt and Zaal (1999) studied the ability of viewers to judge the variability of phase of two oscillating objects. Their interest is derived from observations that coordinated rhythmic movements of two joints in different limbs have only two stable phases, at 0° and 180°. However, the relative phase of 180° has more variability than at 0°, and as the frequency of oscillation increases, a transition to 0° occurs. This leads to the question, can we perceive variability of phase? Bingham et al. (1999) found that we can, but only at 0° and 180° — otherwise we tend to judge relative phase. 2.5.5 Comparison and Judgement of Computer Generated Human Movements As the computer representation of human movements already separates "motion pattern" from "visual form" the typical problem in computer animation is creating the "correct" or "natural" motion for a particular form (body). There has been much work on the problem of creating "natural looking" motions with the least amount of work either by using motion generators, motion capture, or by creating computer animation tools which allow computer animators to specify sparse keyframes which are interpolated to create the final motion. Bartels and Hardtke (1987, 1989) created a method to control the parameterization along a motion path as a function of time. By adjusting the mapping between time and parameterization variations in the apparent kinetics can be specified. Other methods of adjusting the kinematics of a motion by adjusting the param-eters used to interpolate keyframes include Kochanek and Bartels's (1984) Tension, Continuity, and Bias splines and Steketee and Badler's (1985) interpolation system which supports second-derivative continuity (continuity of acceleration), kinetic control (as with Bartels and Hardtke), and phrasing control for joining successive movements. Brotman and Netravali (1988) proposed a method of interpolating key poses of a figure specified at keyframes by modeling the motion as an evolution of a physical system with masses and velocities. By minimizing the energy necessary to perform the movement Brotman and Netravali claimed to produce "smooth and natural motion that is subjectively better than that produced by other interpolation methods" which only interpolate the keyframes of the motion signals without regard to kinematic or dynamic constraints. Witkin and Kass (1988) presented a related method called "Spacetime Constraints" which allowed for very sparse keyframes such as those specifying only the end goal to be achieved and an initial state. Their solver then refines the initial trajectory until the goal is achieved using minimum local energy consumption. Again, smooth, physically-plausible motions are generated with very little movement character or style. Later work has focused on slightly higher level specifications than motion signal specification or trajectory optimization, van de Panne (1997) introduced a method of specifying the walking motion of a figure by specifying the placement and timing of foot placements. Walking, turning, leaping, and running move-ments can be generated by a spacetime constraints simulator which maximizes the physical plausibility and perceived comfort of the movements. In computer animation cyclical motions are often animated by looping a short motion clip, or by using a simple motion generator. The result is a motion that appears unnatural because of its repetitiveness. Bodenheimer, Shleyfman and Hodgins (1999) proposed to reintroduce "natural-looking variability" into a motion by adding noise. Applying principles of biomechanics to their dynamic simulation of a human runner (Hodgins, Wooten, Brogan and O'Brien 1995), they generated ten videos of a male runner with increasing amounts of noise added. Observers were asked which video appeared most "natural". More interestingly, the noise addition technique used on the male runner did not translate directly to the female runner. Some subjects preferred a female running movement with small amount of noise while others preferred a large amount. One hypothesis the authors presented is the addition of noise does not address the fundamental problem with their simulation of a female runner's motion: that it may be inherently less natural-looking and thus adding a little or a lot of noise makes it look better. Hodgins, O'Brien and Tumblin (1998, 1997) have reported on the problem of discriminating motions and the effect of rendering models on the human motion judgements. Comparisons between solid rendered models of a runner were more accurate than were stick figure models. These results may be explained by the lack of occlusion cues with the stick figure, or the participants' sensitivity to the simulated runner's motion. However, Hodgins, O'Brien and Tumblin as expert viewers of the motions reported 100% accuracy at this task. Shibata and Inooka (1998) simulated human arm motions using both video and an industrial robot in order to determine which factors are important to give a perception of "human-likeness". Movement between two points along a straight line was performed under various velocity patterns and peak velocity as illustrated in Figure 2.10. Significant "human-likeness" was perceived based on velocity peak position (relative to normalized movement time) and maximum velocity of the movement. velocity Figure 2.10: Shibata and Inooka (1998) used velocity patterns such as these to control a robotic arm. Partic-ipants observed the movement as presented using both a computer animation display and a physical robotic arm. Participants reported the "human-likeness" and "friendly-ness" of the movement on a number of scales. Figure 2.11: Two sample frames from animations used by Hodgins, O'Brien and Tumblin (1998, 1997). to test human judgements of motion as affected by rendering model. Each frame is taken from a four second (120 frame) animation of a single rendering model. Comparisons were made by first viewing one animation and then another rendered with the same rendering model. Side by side comparisons were not made - do not be confused by this illustration. Chapter 3 Experiment Design In order to build higher-level computer animation tools for selecting, specifying, or modifying movements represented by computer models we need to know how the parameters of a movement, V(Q), affect our perceptions and judgements. We will treat the relationship between motion parameters, perceptions, and judgments as mappings between three different types of motion spaces. The first motion space is the "me-chanical motion space," a vector space of motion signals, Q(t), in which Computer animation tools operate. Q(t) is summarized with a set of parameters, V(Q) which describes Q(t) in a compact form. For example, if Q{t) are the motion signals for a walking motion, V(Q) would include walking speed, step frequency, stride length, etc. The second motion space is a conceptual space in which humans organize motions ac-cording to their features. We shall call this space the "psychological motion space" in all of our discussions. Although we know little about the structure or properties of this space, we can hypothesize that judging the similarity of two movements requires the computation of a "distance" between them. Thus, the structure of this space is defined by a "proximity measure" of pairs of movements. The third motion space is also a conceptual space that humans use to describe movements using words. This "linguistic motion space" contains attributes in which concepts of "slower" and "bouncy" are defined. The structure of this space is defined by multivariate "descriptions" of movements. Figure 3.1 illustrates the relationships between the spaces. Given these definitions, we have completed experiments to collect from participants their judgements of the similarity ("proximity") of pairs of movements, and their descriptions of the movements using several descriptive rating scales. To determine the relationship between the mechanical and psychological motion spaces we analyse the relationship between the proximities and differences of motion parameters. To deter-mine the relationship between the mechanical and linguistic motion spaces we analyse the effect of varying motion parameters on the ratings. To determine the relationship between the psychological and linguistic motion space we analyse the relationship between proximities and differences of ratings. Mechanical Motion Space Figure 3.1: The three motion spaces used to represent human movements. The mechanical motion space is the space computer animation tools operate in, its structure is defined by the parameters of motions. The psychological motion space is the space we use to perceive and code the features of movements, its structure is defined by similarity judgements of pairs of motions. The linguistic motion space is the space we use to describe movements using words, its structure is defined by multivariate "descriptions" or ratings of motions. The relationships between these spaces describe how the parameters of a movement (mechanical) affect the judgements (psychological) and descriptions (linguistic) formed by a human observer. 3.1 Methodology Participants reported their judgements and descriptions of human movements using two experimental tasks. The first task was direct judgement of the similarity of pairs of movements. The second task was description of movements using a set of descriptive scales labeled with pairs of words with opposite meanings that can be used to describe the movements. Two experiments were conducted. The first, "Experiment One: Comparing and Describing Motions," was a broad initial experiment performed to demonstrate the collection of similarity judgments and descriptions of the movements from human observers using a wide range of human walking movements. The second, "Experiment Two: Metric Properties of Motion Dissimilarity Judgements," is an in depth experiment to determine the properties of the psychological motion space by using a narrower range of walking movements that includes movements created by interpolating motion parameters. 3.1.1 Stimuli Selection In our experiments we used human walking movements ("gaits") created using Bruderlin's (1995) procedu-ral gait generator W a l k e r . We selected whole body walking movements rather than other movements such as arm movements because human walking movements have a number of useful properties. First, walking movements are easily described by a small set of motion parameters such as joint angle ranges which we believe are approximations of perceptually salient features. This makes the task of determining the relation-ships between the mechanical motion space and the psychological and linguistic motion spaces easier. It is also easier to "interpolate" or "blend" walking movements than it is for goal-oriented movements such as opening doors, sitting down or picking up objects. Second, the duration of presentation of walking movements can be shortened or extended over a wide range without changing any of the motion parameters. This is not true for non-rhythmic movements which cannot be arbitrarily scaled temporally. Third, there are several viewpoints that can be used for presenting walking movements while goal oriented movements often have optimal viewing directions. Fourth, the velocity of the arms and legs is also much slower in walking than in jogging or running which allows participants to form reliable judgements. Finally, and most importantly, human walking movements are natural and familiar human movements that are perceived and observed everyday; it is not necessary to train participants in their observation or recruit participants with special backgrounds. As we discussed in the prior chapter, walking movements have successfully been used in other experiments on human movement perception. 3.1.2 Walker Walker generates the walking movement of a human figure along a straight path and creates a computer an-imation display of the motion using a variety of visual forms. The gaits generated by Walker are rhythmic, symmetric, and are specified by twenty-two parameters, such as the speed of walking, stride length, step frequency, and joint angle limits of the arm swings, etc. Walker outputs to a computer file the movement of thirty-six joints as rotations around eighty-three joint axes.1 The user interface to Walker is illustrated in Figure 3.2. An additional consideration for using Walker and the "Akira" visual form, was that the presentation of the walking movements using a computer animation display is possible using computer graphics workstations. This allowed complete control over the presentation of the trials, and automated data and response logging which would not have been available if we had used pre-rendered displays such as video tape recordings of computer animation displays and paper response forms. Our experiment control software presents the gaits, generated by Walker using a profile view of the articulation and visual form specified by "Akira," a male humanoid figure wearing a green shirt, blue pants and black shoes against a dark gray background with a light gray grid as illustrated in Figure 3.3. The specific details and descriptions of the gaits used in the experiments are included in Appendix B. 'Twenty four joints are used to represent the joints between the spinal vertebrae. Thus fifty-three of the rotations are the rotational axes of the joints in the spine: the five lumbar joints have three rotational axes each, the twelve thoracic and seven cervical joints have two rotational axes each. Figure 3.2: Screen shot of Bruderlin's (1995) W a l k e r showing the output window and controls. The program uses twenty-nine inputs to vary the style of the walk by computing the motion of thirty-six joints totaling eighty-three degrees of freedom. The inputs are: walk velocity, step length, step frequency, shoulder rotation, degrees of arm swing, angle arms are raised from the sides, elbow rotation (min, max), torso tilt (forward/backward), torso sway (side to side), torso rotation (left to right), lateral displacement, rotation of the pelvis about the vertical axis and list from side to side, bounciness on foot strike as the supporting knee bends, amount of over stride, hip flexion as leg swings forward, knee angle during swing, midstride and impact, stride width (distance left to right of foot placements), angle of foot with respect to straight forward ("pigeon or duck toes"), and whether the heel or toe strikes the ground first. ] i 79 84 89 94 Figure 3.3: On the top: One frame of a walking motion as it was presented to a participant. The area the figure appears in is 4.25 inches wide by 3 inches tall (this should match the printed size). On the bottom: One walking step (half a stride) from a video clip of a walking motion — frame numbers are indicated below each frame. 3.1.3 Flow of Data from Stimuli Generation through Analysis Figure 3.4 illustrates how the data was produced and analysed in our experiments. Twenty-two parameters of Walker were varied to produce the gaits. These parameters are used to define the mechanical motion Participants judged the similarity of pairs of gaits in "motion comparison trials." Similarity judgements are recorded using a "continuous"2 scale with the value zero indicating "identity" and the value one indicat-ing "completely different." Thus participant's responses are a measure of the proximity of gaits in their personal psychological motion space and we use the term "dissimilarity judgement" rather than "similarity judgement" in our analysis. Figure 3.4: Flow of data from the production of the gaits through the analysis of the participant's responses in the two experiments. Walker ' s input parameters were varied to produce the gaits, some of which were interpolated from other gaits. Participants compared pairs of gaits in motion comparison trials, and described each gait in motion rating trials. Correlation analysis between the motion parameters and the dissimilarity judgements and ratings are performed to determine the influence of the parameters. Correlations between the dissimilarities and ratings were computed to test the relationship between dissimilarities of pairs of gaits, and distances between descriptions of gaits. Ratings and parameters are analysed using principal components analysis. In Experiment Two, some gaits are created by interpolation of the parameters of other gaits and multidimensional scaling is used to analyse the dissimilarity judgements. space. Stimuli Trials Responses and Analysis Participants could record their judgement anywhere along the scale, but its limited responses to any of 995 screen pixels. After the participants completed all of motion comparison trials they described each gait in "motion rating trials." Ratings of each gait were recorded using eight "continuous" scales labeled with pairs of words with opposite meanings that can be used to describe the gaits. The left hand label had a value of zero and the right hand label a value of one. These multivariate ratings define the structure of the linguistic motion space. Having collected the dissimilarity judgements and ratings of the gaits we proceeded to the testing of the relationships between the participant's psychological and linguistic motion spaces and Walker ' s mechan-ical motion space. Because dissimilarity judgements are a measure of the distance between pairs of gaits, when we analysed the relationship between the psychological and mechanical motion spaces we computed differences between the parameters of pairs of gaits. Likewise, when we analyse the relationship between the psychological and linguistic motion spaces we compute differences between the ratings of pairs of gaits. Specifically, we tested for the following linear relationships between the motion spaces: • each participant's dissimilarity judgements correlates with differences of Walker ' s parameters. • Walker ' s parameters correlates with each participant's ratings, correlations between dissimilarity judgements and differences of ratings. As will be discussed in Section 3.5.1, we also performed separate principal component analyses of the ratings and the parameters to determine the dimensionality and principal dimensions of the linguistic and mechanical motion spaces. The reduced dimensionality ratings and parameters are also compared to the dissimilarity judgements. In Experiment Two, some gaits were created by interpolation of the parameters of other gaits. As will be discussed in Section 3.5.2 we also used multidimensional scaling to transform the dissimilarity judgements into configurations of the gaits in Euclidean spaces. 3.2 Experimental Procedure An experiment was composed of two sessions, both completed on the same day with a short break between them. A session was composed of blocks, and blocks of trials. Breaks were allowed between blocks, but not between trials. The first session was composed solely of motion comparison trials, and the second session solely of motion rating trials. A l l blocks in a session contained the same trials but presented in a different order. A l l trials were self-paced, with breaks allowed between blocks. Response times were recorded but were not analysed because participants were told that the experiment was not timed. 3.2.1 Participants Because the task of judging and describing motions is novel, we felt that it was important to recruit par-ticipants who have experience observing, learning, imitating, and comparing human movements. For Ex-periment One, social dancers3 with various amounts of dance experience were recruited. Although it was felt that the task required only basic motion perceptual skills, it was also believed that participants with previous experience observing human motion might be capable of relaxing and forming judgements with more reliability or speed. Also, dancers are trained through their dance classes to describe movements and communicate to others how to modify their movements to approach a common standard. In Experiment Two we focused more on the properties of the psychological motion space and felt that the properties of this space would be much more consistent across the general population. In Experiment Two we were attempting to determine if the general results of Experiment One generalized to non-dancers and by focusing on basic motion perception we felt we should recruit a wider range of participants better reflecting the general population. Participants were classified as dancers, runners, and non-dancer non-runners {i.e., "normals") — as well as males and females. A l l participants were naive to the hypotheses and were paid cash for their involvement. 3.2.2 Stimuli Presentation In each trial, the figure walked left to right across a small area of the screen, approximately 4.25 inches wide by 3 inches tall. 4 Behind the figure is drawn a dark gray background with a light gray grid. The remainder of the screen is black. The figure appeared by walking from beyond the left edge of the gray square and continued to walk until beyond the right edge. Figure 3.3 illustrates a single frame from a video clip of a walking motion as it was presented to a participant, and a sequence of frames from a single gait. 3Social dancers are dancers who dance with a partner: Ballroom, Latin, Swing, etc. 4 The drawable portion of the screen is 13.5 inches wide by 11 inches tall, 18 inches diagonal. 3.2.3 Pre-Experiment Prior to beginning the experiment, participants signed a consent form as required by the university ethics review board. The form gave the participant some basic information about the experiment, along with outlining the payment and withdrawal conditions. Participants then completed Questionnaire A to collect demographic information and their experience with dance, running, aerobics, sports, movement therapies, and creating animation. Copies of the consent form and Questionnaire A are included in Appendix D.3 and Appendix D.4. 3.2.4 Instruction of Participants The experiment was self-paced and included instructions presented on the computer monitor. After making sure the participant was seated comfortably, the we observed the participant interact with the experiment system and answered any questions. To emphasize that the current screen is part of a tutorial, the word "Tutorial" appears at the top of the computer screen in a bright yellow square until the end of the instructions. The instructions included details about the experiment, the motions, the presentation of some example mo-tions, and a short test block of trials. Copies of the instructions are included in Appendix E. Participants were instructed to click on the "skip" button in the lower left hand corner of the screen only if they felt that their attention drifted during the presentation of the stimulus and that they failed to observe the gait. This excluded the trial from analysis and was thought to prevent the participants from guessing if they failed to observe one of the gaits. We will discuss our conclusions about the participants possible strategic use of the "skip" button in our analysis of Experiment Two in which participants were allowed to skip the current trial "until later." Before the participant began any of the trials they were shown all of the walking motions and were allowed to replay all of the motions if they wished. We again asked the participant if they had any questions. After answering any questions, the we informed the participant that their progress would be monitored from the other room and that we would return at the end of the first block. We then left the room. Participants were instructed to use the first block to learn the experimental task and were told that their responses in the first block are not be examined too closely. Generally, the participants commented that after about ten trials that they felt comfortable comparing or describing motions. After the first block, we returned and asked the participant if they had any questions. Between each block participants were reminded by the experiment software and ourselves to take a break. To move forward through the experiment, the participant used the computer mouse to click on the button on the bottom right hand corner of the screen. This button finishes trials and starts blocks: Begin Block 1 Finish Trial 1 of 26 Finish Trial 2 of 26 Finish Trial 26 of 26 End Block 1 Begin Block 2 3.2.5 First Session: Motion Comparison Trials The first session of the experiment was composed solely of motion comparison trials. The purpose of these trials was to collect judgements of the similarity of movements. These judgements defined the structure of the psychological motion space and allowed us to determine the perceptually salient parameters of the gaits as well as the relationships between the psychological motion space and the linguistic and mechanical motion spaces. In a motion comparison trial, two walking motions were presented sequentially in time and the participant judged the dissimilarity of the two motions. Their judgement was indicated by using a continuous scale labeled "Similar" on the left end and "Dissimilar" on the right end. The scale was manipulated using a computer mouse. Participants were not allowed to replay the motions or make a judgement about the dissimilarity of the motions while the motion is being presented. This was ensured by hiding the scale and labels during the presentation of the motions. Figures 3.5a-3.5b illustrates a motion comparison trial. Each trial took about ten seconds: seven seconds for the presentation of the gaits and two to three seconds for the participant to make a judgement and click on the "Finish Trial" button. The motion comparison trials were combined into four blocks of trials. Each block consisted of the same trials in a different order, and each trial consists of the presentation of a pair of gaits and the recording of a dissimilarity judgement. Short breaks were offered between blocks and a longer break before beginning the second session of the experiment. In Experiment Two the first half of Questionnaire B (see Appendix D.5) was completed by the participant to collect opinions about the task of comparing walking movements. Figure 3.5a: Presentation of the first walking motion in a motion comparison trial from Experiment One 1 • Finish Trial 1 of 3 Figure 3.5c: After the presentation of the two walking motions the Similar-Dissimilar scale is displayed with the marker at the Similar of the scale. After indicating their judgement of dissimilarity, the participant clicks on the "Finish Trial 1 of 3" button in the lower right hand corner of the screen. Participants were instructed to click on the "skip" button in the lower left hand corner of the screen only if they felt their attention drifted during the presentation of the stimulus and they failed to observe the walking motion. The "Tutorial" label in the upper left hand corner of the screen appears only during the training portion of the experiment. 3.2.6 Second Session: Motion Rating Trials The second session was composed solely of motion rating trials. The purpose of these trials was to collect descriptions of the movements. These descriptions define the structure of the linguistic motion space and allowed us to determine the relationships between the linguistic motion space and the psychological and mechanical motion spaces. In a motion rating trial, a single walking motion was presented and then the eight descriptive rating scales were displayed so the participant could record their description of the gait. Each scale was labeled with a pair of descriptive words with opposite meanings. The participant recorded the description of motion on each scale according to its labels and then pressed a button in the lower right corner of the screen labeled "Finish Trial." Participants were not allowed to replay the motion or enter a rating on the scales while the motion is being presented so that they were not to performing two tasks (observation and description) at once. This was ensured by hiding the scales and labels during the presentation of the motion. Figures 3.6a-3.6b illustrates a motion rating trial. Each trial taok about fifteen seconds: four seconds for the presentation of the gait and about ten seconds for the participant to record ratings on each of the eight descriptive scales and click on the "Finish Trial" button. We will discuss how the labels were selected in Section 5.2.7. The motion rating trials were combined into four blocks of trials. Each block consisted of the same trials in a different order, and each trial consisted of the presentation of a single gait and the recording of the ratings on all eight descriptive scales. Short breaks were offered between blocks and a longer break before beginning the second session of the experiment. In Experiment Two the second half of Questionnaire B (see Appendix D.5) was completed by the participant to collect opinions about the task of rating walking movements. normal strange Figure 3.6b: After the presentation of the gait the eight rating scales are displayed. After indicating their description of the motion, the participant clicks on the "Finish Trial 1 of 3" button in the lower right hand corner of the screen. Participants are instructed to click on the "skip" button in the lower left hand corner of the screen only if they felt their attention drifted during the presentation of the stimulus and they failed to observe the walking motion. The "Tutorial" label in the upper left hand corner appears only during the training portion of the experiment. 3.2.7 Post Experiment At the end of the experiment, any questions the participant had about the purpose and applications of the ex-periment were answered. Participants in Experiment One were paid $25, and the duration of the experiment was 2-2.5 hours. The duration of Experiment Two was 1.5-2 hours and remuneration was decreased to $20. A copy of the receipt signed by the participants is included in Appendix D.6. 3.3 Equipment The participant sessions primarily took place in FSC2430, the "Landscape Immersion Laboratory." A few sessions were held in FSC2330 the "Imager Laboratory." Experiments were held on weekdays and weekends and the availability of FSC2430 was extremely helpful in completing the experiment with very little impact on the other activities of the Imager lab. 3.3.1 Hardware Two computers are available to display the gaits and collect the data: "redgreen" Silicon Graphics IMPACT10000. 5 Normally located in the Imager Lab. "onyx" a Silicon Graphics Onyx. 6 Available in either the Imager Lab or the Landscape Immersion Lab. These are our fastest Silicon Graphics workstations not dedicated to other tasks. During the experiments, processor load and user count is monitored to ensure that no other users log in and start any processes. The advantage of using the Onyx is that its video display can be mirrored in another room, allowing monitoring of the progress of the experiment. The progress of experiments run on Redgreen require the monitoring of 5 C P U : MIPS R10000 Processor Chip Revision: 2.5; 1 195 M H Z IP28 Processor; FPU: MIPS R10010 Floating Point Chip Revision: 0.0; ; Main memory size: 192 Mbytes; Secondary unified instruction/data cache size: 1 Mbyte; Instruction cache size: 32 Kbytes; Data cache size: 32 Kbytes; Graphics board: High Impact. 6 C P U : MIPS R10000 Processor Chip Revision: 3.4; 2 250 M H Z IP27 Processors; FPU: MIPS R10010 Floating Point Chip Revision: 0.0; Main memory size: 1024 Mbytes; Instruction cache size: 32 Kbytes; Data cache size: 32 Kbytes; Secondary unified instruction/data cache size: 4 Mbytes; Graphics board: InfiniteReality2E. the experiment system data log files.7 Participants are seated at a comfortable height and about three feet from the monitor so that the gaits subtend about one degree of their visual field. Specifically, the participants are instructed to raise or lower their chair seat height until the gait presentation area is at eye level, and move their chair back and forth until the area is the width of their hand at arms length. Lighting in the room is adjusted to "dim" so there are minimal reflections or glare on the monitor. The refresh rate of the experiment computer is set to 72 Hz, and due to processing requirements, the update rate of the walking motions is on average 70 Hz on "redgreen" and 72 Hz on "onyx." The gait display routines are coded to synchronize the real world clock with the frame rate so that frames of the gait are dropped if necessary. The gaits were created with a simulation time step of 30 Hz and because playback rate and stability are not experimental variables, an update rate of 30 Hz is expected.8 To assist in accurate playback, each motion is re-coded to allow variable frame display times ranging from 6 Hz to 72 Hz. Statistics — mean, mode and standard deviation — are gathered on the actual update rate variation of each gait during a trial for later analysis and confirmation that the gaits are presented correctly. 3.3.2 Software The experiment control software was written in C by the author. Gait display routines were provided by the Credo Interactive "Life Forms API." This API is compatible with the motions created by W a l k e r . Interface widgets for the interaction tasks were provided by the Forms Library written by Mark Overmars (1992). The experiment is specified by a script file that describes: • the visual form to be used to display the gaits • the gaits to be used • the instructions to be presented to the participant • the start and stop of each block • when to remind the participant to take a break • the trials 7 The experiment control system flushes all output to the data log on a regular basis so that monitoring can be performed using the UNIX command t a i l - f on another workstation. 8 The update rate of a visual stimulus is the highest frequency of change. The refresh rate is how often the visual stimulus is redrawn. The refresh rate must be at least as high as the "flicker fusion" rate while the update rate can be lower. The gaits were generated using W a l k e r as * . w l k files. These files were converted to the Life Forms API format, * . seq, using l o c 2 s e q supplied with the Life Forms API. 3.4 Data Collection and Notation When discussing participant responses, participants are referred to by number, e.g., #1, with separate num-bering for Experiment One and Experiment Two. In equations, participant are indicated by the prefix sub-script letter P. Gaits, are indicated by number also and in equations by the symbols i, j, and k. Motion comparison trials are indicated by a pair of gaits, e.g., T{i,j) would be gait i presented first followed by gait j. Participant dissimilarity judgements are pô{i,j)-For each motion comparison trial, the following information is recorded by the experiment system: • block (t) and trial specified according to gaits presented (i and j) • frame jitter while presenting gait (not analysed) • manipulation of similar-dissimilar scale: value and time (not analysed) • final dissimilarity judgement (<5) and time to complete judgement (time not analysed) • whether the trial is skipped The dissimilarity judgement of gaits i and j by participant P in block t is written p(t)8{i, j). For each motion rating trial, the following information is recorded by the experiment system: • block (t) and trial specified according to gait presented (i) • frame jitter while presenting gait (not analysed) • manipulation of each rating scale: value and time (not analysed) • final rating values (R) on each scale (s) and time to complete rating (time not analysed) • whether the trial is skipped The rating of gait % on scale s by participant P in block t is written p(t)Rs{i)-Each participant compared and rated the motions in four blocks numbered 0, 1, 2, and 3. Other than some simple checks we will not be examining the trials from block 0. When we do examine participant judgements or descriptions we have attempted to be as clear as possible whether we are using the raw responses from each block: p(t)ô(h J) Dissimilarity judgement of gaits i and j by participant P in block t. p(t)Rs (0 Descriptive rating of gait i on scale s by participant P in block t. Or whether we are using the averaged responses for each trial across blocks, indicated using a line drawn above the symbol: pS(i, j) Averaged dissimilarity rating by participant P of gaits i and j where i was presented first. pRs{i) Averaged rating of gaits i on scale s by participant P. To summarize, we will use the following notation to refer to participant responses, gaits and trials. i, j, k Gait i, j, and k. When we refer to a specific gait we will do so by name or number as listed in Appendix B. Occasionally we use these symbols for indices such as parameter or dimension number. T(i,j) Comparison trial T(i, j) in which gait i was presented first followed by gait j. P Participant P. P(t)S(h j) Dissimilarity judgement of gaits i and j by participant P in block t, where i was presented first followed by j. Without the subscripts i and j, pô(-, •) is a vector of dissimilarities for all trial pairs across experiment blocks. pô(i,j) Averaged dissimilarity rating by participant P of gaits i and j where i was presented first. Without the subscripts i and j p6(.,.) is a vector of averaged dissimilarities for all trial pairs (averaged across experiment blocks). s Descriptive rating scale s. p(t)Rs{i) Descriptive rating of gait i on scale s by participant P in block t. pRs(i) Averaged rating of gait i on scale s by participant P. When we test the relationships between the mechanical and linguistic motion spaces we use the parameters used by W a l k e r to define the gaits: Vwt The vector of parameters used by W a l k e r to create gait i. T-Wj(i) ' s t n e first parameter of gait i, 7-^(2) t r i e second parameter, etc. There are twenty-two parameters used in these experiments. Vw is the matrix of parameters for all of the gaits, Vw ^) IS a vector of the values of the k parameter for all gaits. When we test the relationships between the mechanical and psychological motion spaces we need to com-pute differences between the parameters of the gaits since the dissimilarity judgements are the "distance" between gaits. We will test for a linear relationship between dissimilarity judgements, pô(i,j), and differ-ences of the parameters, \Vw.(k) - "Pwj-(fc) I-When we analyse the relationships between the psychological motion space and the linguistic motion space we need to compute differences between ratings. We will test for a linear relationship between dissimilarity judgements, pô(i,j), and differences of the descriptions along rating scales, | pRs{i) - pRs(j)\. p A s ( i , j ) Absolute difference between averaged ratings of gaits i and j on scale s by participant P: pAs(i,j) = j pRs{i) — pRs(j)\. Without the subscript s then pA,(i,j) is the vector of absolute differences between pRs(i) and pRs(j) across all scales. Without the subscripts i and j pAs(.,.) is the vector of absolute differences on scale s for all trial pairs (excluding self-similar trials, T(i, i)). 3.5 Data Analysis Techniques We analysed each participant's responses independently as we cannot assume that their psychological and linguistic motion spaces are the same because each will have personal biases, experiences, and perceptual filters. When possible we will present patterns common to many participants. We tested the relationships between the motion spaces by using correlation and regression. Our hypothesis of the relationship between the mechanical and psychological motion space is that the larger the difference in the parameters of two gaits, the larger their judged dissimilarity. Our hypothesis of the relationship between the mechanical and linguistic motion space is that changes in the parameters correlate with changes in the ratings along the scales. Finally, we hypothesize that gaits judged to be dissimilar will have different ratings, and gaits judged to be similar will have similar ratings. Specifically, we will use Pearson's Product Moment Correlation Coefficient to compute the correlation be-tween proximities in the psychological motion space and the differences in parameters (mechanical) or rat-ings (linguistic). The correlation coefficient of two variables is a dimensionless measure of the relationship between the two variables: r p ( A ' y ) - a(X)a(Y)(n - I) where, X is the mean of X, and a(X) is its standard deviation. For example, we will analyse the relationship between the mechanical and psychological motion spaces by testing if the average dissimilarity of two gaits, pô(i,j), correlates with difference in the parameters used to define the gaits, \VVj(k) - ^(/c)!* w e w ' " t e s t t t i e t w o hypotheses: Hi : Vfc,r p( P5(i,j),\VWi{k) - Pv.{k)\) = 0 Hi : 3k,rp( P6(i,j),\V^i[k) - VWj{k)\) > 0 Where HQ is the null hypothesis (no relationship for any parameter VW(k)) a r , d H\ is the alternative hypothe-sis that there exists at least one parameter such that there is a positive correlation between dissimilarity and difference in the value of parameter k. We will use similar hypotheses for testing the relationship be-tween the psychological and linguistic motion spaces. When analysing the relationship between the linguistic and mechanical motion spaces we use regression between the parameters of the gaits (independent variable) and the ratings (dependent variable): y = bXiix + bXio Regression between x (independent) and y (dependent). In the general multi-linear regres-sion model the regression coefficients can be described by a matrix, Bx = [bxp bxv-\ • • • bXio], and a; is a matrix of p column vectors of independent variables and a column vector of l's. The values of the b's are solved to minimize the sum of squares difference (error) between the left and right hand sides: yi yi = _ yn _ 2-1,p X2,V %n,p 1 bx,p %:;p-\ Jx.O pRs{.) = bpn{k):iVW(k) + bVw p Rs(-) w.(fc) (*)> w.(k)> (fc). Least squares regression between parameter k of the gaits, Vv and averaged ratings of the gaits on scale s by participant P. Vector of averaged ratings of gaits on scale s by participant P. The vector of values for parameter k used by W a l k e r to create all the gaits. Magnitude and direction (slope) of relationship between Walke r ' s parameter VM ^  and participant P's average ratings on descriptive scale s. The y-axis intercept. By normalizing P w ^  and pRs{.) to have zero mean and a standard deviation of one we can test the significance of the regression coefficients by observing that bK,(k),i = rpCPw.(k), PRs{-)) which allows us to use the same tests for significance as we will use for correlation.9 Additionally, b-pv ( J t ) ) i tells us which parameters P w influence which rating scales s and the direction of influence. 3.5.1 Principal Components Analysis Principal components analysis (PCA) is a tool for finding a set of orthogonal basis vectors (e.g., the principal components, pes) that best describe the variation in a set of multivariate data. The motion parameters used to define the mechanical motion space and the ratings used to define the linguistic motion space have elements which are correlated (within the spaces). For example, when we created the gaits, we tended to adjust the shoulder and elbow swings in a similar fashion so that the motion of the arms looked "natural", also it would not be too surprising to find that ratings on "young-old" and "fast-slow" ratings scales are correlated. We can use principal components analysis to remove these correlations and construct a set of orthogonal basis vectors, PC, which explain variation in the motion parameters and ratings: P C ( X ) The principal components (pes) of X. Also known as the eigenvectors of X. Each row of X is an observation, each column a variable. P C i ( X ) is the first pc of X, PC2ÇX) is the second pc of X, and so on. The P C ( X ) form an orthonormal basis and are sorted by magnitude such that their corresponding eigenvalues are decreasing. Each column of PC{ pRs{i)) is a basis vector. \d{X) The eigenvalues of X, also the amount of variance of each PCd(X) . Sorted by magnitude such that Xi(X) > X2(X) >•••> Xn(X). Z(X) The Z-scores of X, also known as X transformed into the space defined by P C ( X ) : Z(X) = XPC{X) 9 We will discuss this test later. where X - X l ,2 X 2 j 2 -X"n-2,1 - X n - 2 , 2 • • Xn-2,d ^ 7 1 — 1 , 1 - ^ n - 1 , 2 • -^71 ,1 Xnfi and P C ( X ) P C i ( X ) ! PC2(X)1 P d ( X ) 2 P C 2 ( X ) 2 P d ( X ) d P C 2 ( X ) d P C r f ( X ) ! P Q ( X ) 2 P C d ( X ) d and X has zero column means: n [lln where [ l ] n is a column vector of n l's, and ri > d. are the values along the first pc, Z2(X) are the values along the second pc, and so on. {X) are first d dimensions of Z(X). Z(VWi) The parameters of gait i transformed into the orthonormal space such that Z i ( P W j ) corre-sponds to P C i (PW f ). Z( pRs{i)) The ratings of participant P transformed into the orthonormal space such that Z\( pRs{i)) coiTesponds to PC\( pRs{i)). Jackson (1991) thoroughly discusses PCA, while Manly (1994) gives a quicker introduction to the technique. An example of using M A T L A B to perform PCA is included in Appendix C.2. In addition to forming orthogonal representations of the mechanical and linguistic motion spaces, we use principal components analysis to reduce the dimensionality the spaces. For example, to reduce the dimen-sionality of the motion parameters we select the first k dimensions of the Z-scores of the motion parameters so as to explain 95% of the variation in the original parameters. Because the principal components are or-dered according to decreasing variance, we take the first k parameters, Zi_fc(Pw)> which sum to 95% of the original variance: k 5 > z ( P w ) / ^ V a r ( P w ) >0.95 This dimensionality reduction is easier to perform if we first normalize each of the parameters to have zero mean and a standard deviation of one. This normalization insures that none of Walker ' s parameters dominates the principal components simply because they are expressed in different units. For example, while many of the parameters have a range from zero to one, the parameter d e s i r e d _ s t e p _ f r e q u e n c y is expressed in steps per minute which has a range from 72 to 144. Because the normalized parameters have variances of one, we can simply accept the first k principal components which have A(7->w) summing to 95% of the total variance: k ^2*i{Vw) > 0.95 x 8 i Or we can use only the principal components which have A(T-w) > 1-0. because any component with a variance less than 1.0 explains less variance than any of the original parameters. For the gaits used in Experiment One, 95% of the variation is explained by ten of the principal components, while only three components are necessary for the gaits in Experiment Two. We will discuss these results in more detail in the two chapters to follow. Having transformed the motion parameters into an orthogonal representation we can analyse the relationship between the mechanical and psychological motion spaces by testing if the average dissimilarity of the two gaits, pS(i,j), correlates with differences in the Z-scores of the parameters, \Zk(Vwt) — (7^. ) |, we will test the two hypotheses: Hi : Vfc,r p( pS(i, j),\Zk(VVi) - Zk(VWj)\) = 0 Hi : 3k,rp(Pô(i,j),\Zk{PVi) - Zk{VVj)\) > 0 Where HQ is the null hypothesis (no relationship for any principal component PCk(Vw)) and H\ is the alternative hypothesis that there exists at least one Z-score k with a is a positive correlation with the dissim-ilarity judgements. We cannot assume that every participant will have used the same features of the gaits to form their dissimilarity judgements, thus each participant will probably have a different set of "most salient" principal components of the parameters. An additional pair of hypotheses is that distance — rather than difference — in the orthogonal space of motion parameters correlates with dissimilarity judgements. Because the principal components of the pa-rameters, PC(VW), are orthogonal and are ordered according to their variance, i.e., Xk(Vw) > Xd+iCPw), we can add dimensions in a structured manner to the computation of the distance between the gait according to the Z-scores. To compute distances we use | | X , - Xj\\p The distance between vectors X , and Xj using the general Minkowski distance function \\Xi — Xj\\ = y2_^ \Xik — Xjk\ The Euclidean distance metric, L 2 , is \\X{ - X j | | 2 = \Jj2(Xik -Xjk)2. The city-block metric, L 1 , is | |X j - X j H 1 = ]T f c \Xik - Xjk\. And the dominance metric, L°°, is | | X i - XjW00 = maxfc \Xik - Xjk\. We will use the Euclidean norm (L 2 ) , dominance norm and the city-block norm ( i 1 ) . v Z{l_k){pRs{i)) - Z{1_k)(pRs{j)) Distance between averaged Z-scores of gait i and j computed using distance metric LP and the first k dimensions. This leads to an additional hypothesis based on the number of dimensions, k, used to compute the distance between two gaits: Hi: V f c , r p ( p J ( z , i ) , | | Z 1 _ f c ( P W i ) - Z 1 _ f e ( W j ) i n > 0 Hi : 3 f c , r p ( p 5 ( M ) , | | ^ i - f c ( P W t ) - Z1_k(Wj)\\P) > 0 Where HQ is the null hypothesis (no relationship for any ranges of components, 1 — k) and H% is the alternative hypothesis that there exists a k such that the distances along components {1 ,2 , . . . , fc} correlate positively with the dissimilarity judgements. We limit ourselves to ranges of the principal components from the first to k simply to reduce the number of correlations we compute. It is possible that each participant will have a different optimal range, k and distance metric, L p that maximizes H%. We will use similar hypotheses for testing the relationship between the psychological and linguistic motion spaces by computing the principal components of the ratings. 3.5.2 Multidimensional Scaling (MDS) In the analysis of Experiment Two we will use multidimensional scaling (MDS) to transform the participants dissimilarity judgements into Euclidean space configurations of the gaits. We use MDS for two reasons, the first is to form a "picture" of the psychological space for formation of future hypotheses and experiments — this is the traditional use of MDS. The second is to compare the participants by examining how well their dissimilarity judgements fit the overall configuration formed by using all participants. Some of the properties of the configuration produced by MDS are: • distances between points (the stimuli) in the common space approximate dissimilarities, • all of the participants' dissimilarities are taken into account to form a common space, • individual participant differences in perception or strategy are represented in a separate space. The general problem that MDS addresses is the conversion of a set of distances between points into a spatial arrangement of the points best reflecting the distances. The dimensions of the configuration are usually treated as perceptually salient dimensions but other techniques must be used to confirm such a conclusion. Since the dissimilarity judgements can include errors the task of MDS algorithms is to construct the spatial arrangement that minimizes the difference between the given distances and the distances between the points in the spatial arrangement. We used SPSS's implementation of A L S C A L to perform multidimensional scalings of the dissimilarity judgements. In addition to reporting the spatial configuration of the stimuli in an Euclidean space, A L S C A L reports for each participant included in the scaling the following statistics: Participant Weights Linear scaling factors which are used to scale the overall configuration to better fit the participant's dissimilarity judgements. SSTRESS(l) a measure of how well distances in the overall configuration (after scaling) fit the partici-pant's dissimilarity judgements. A relative measure of "goodness of fit." RSQ Proportion of variance of the participant's dissimilarity judgements accounted for by dis-tances in the (scaled) overall configuration - also the sum of the square of the weights. Varies between zero — no correlation between distances and dissimilarities — to one — perfect correlation between distances and dissimilarities. Weirdness A measure of the heterogeneity of the weights. If the weights are all about the same value, then the participant's configuration is a uniform scaling of the perceptual dimensions of the overall configuration. While SSTRESS(l) is used to terminate the iterative fitting process, it can also be used to decide the number of dimensions necessary for the configuration. For example, if the number of dimensions is decreased by one and SSTRESS(l) does not change very much then the lower dimensional configuration is usually considered to be a better solution. As SSTRESS(l) increases, RSQ decreases. Weirdness can also be used to determine the optimal number of dimensions. If many participants have large weirdnesses and there is a common agreed upon set of "favorite largest weights" then dimensions repre-sented by the lesser weights can probably be discarded and a lower dimensional fit attempted. Weirdness can also be used to compare populations of participants. For more information about MDS, we suggest Davison's (1983) textbook as a fairly neutral introduction to the MDS models and algorithms. Young (1987) reviews the historical development of MDS, and also includes many examples of applications from different fields, but we tended to feel that he did not point out the pitfalls and assumptions of MDS as well as Davison. The SPSS Professional Statistics Manual 6.1 discusses the SPSS implementation and use of A L S C A L (Norusis 1994) (but note later editions). Jackson (1991, Chapters 10-11) discusses the relationship and similarities between MDS and PCA: they can both be used for dimensionality reduction, but PCA works with the positions of the objects while MDS starts with the distances between the objects. 3.6 Formal Tests of Hypotheses Relationship Between Psychological and Mechanical Motion Spaces We hypothesize a linear relation-ship between the proximities in the psychological motion space and distances in the mechanical motion space. Specifically: • Dissimilarity of gaits, pô(i,j), correlates with differences of Walke r ' s parameters for gaits, \VWi^)~ • Dissimilarity of gaits, pS(i,j), correlates with differences of Z-scores of Walker ' s parameters for gaits, \Zk{VVi) - Zk(VVj)\. • Dissimilarity of gaits, pS(i,j), correlates with distance between Z-scores of Walke r ' s parameters for gaits,\\Z^ k(Vw,) - Z{_k(Wj)\\p. Positive correlations would indicate which motion parameters are perceptually salient parameters used to form dissimilarity judgements. The positive correlations would then allow us to focus on specific parameters when testing more complex models of dissimilarity judgements. Relationship Between Linguistic and Mechanical Motion Spaces We hypothesize a linear relationship between the motion parameters of the gaits and the descriptions made using the rating scales. Specifically: • Rating of gaits along scale s, pRs(i), correlates with parameters of gaits, Vw • Rating of gaits along scale s, pRs(i), correlates with Z-scores of parameters of gaits, ^(Pw*)-Positive correlations would indicate which motion parameters correspond to descriptive concepts of the motions. The positive correlations would then allow us to focus on specific parameters when testing more complex models of description formation. Relationship Between Psychological and Linguistic Motion Spaces We hypothesize a linear relation-ship between proximities in the psychological motion space and distances in the linguistic motion space. Specifically: • Dissimilarity of gaits, pô(i, j), correlates with differences of ratings of gaits along scale s, \ pRs{i) — PRS{J)\-• Dissimilarity of gaits, pô(i, j), correlates with differences of Z-score k of ratings of gaits, \Zk( pRs(i)) Zk{pRs(J))\-• Dissimilarity of gaits, pS(i,j), correlates with distance between Z-scores {1 — A;} of ratings of gaits, i i ^ i - j b d - p ^ o ' ) ) i r -Positive correlations would indicate the nature of the relationship between the psychological and linguistic motion spaces. If dissimilarities correlate directly with a rating scale s then that would indicate that scale s is a dimension of the psychological motion space. 3.6.1 Relationship Between Mechanical and Psychological Motion Spaces Informally our hypotheses can be stated as: The larger the difference between the parameters of pairs of gaits, the larger the dissimilarity judgement. 3.6.1.1 Differences of Gait Parameters Correlate with Dissimilarity Judgements We test that judgement of dissimilarity of pairs of gaits correlates with differences of the parameter values used to create the gaits. Specifically, the averaged dissimilarity judgements of gaits i and j by participant P, pô(i,j), will be tested for a significant correlation with differences of a subset of the parameters used to create the gaits, \VWi(k) — ^Wj(fc)l- This leads to the two hypotheses: HlQ : Vk,rp(p6(i,j), \Vw.(k) - Pw,-(*)D = 0 Hi : 3k,rp{pS(i,j),\VWi(k) - VWj(k)\) > 0 If we do not reject HQ then variation in the parameters, VW, does not effect changes in judgements of the similarity of the gaits, pô(i,j). If we find that variation in the k parameter of the gaits positively correlates with the dissimilarity judgements then we conclude that the parameter is a perceptually salient parameters. We do not assume that all participants' dissimilarity judgements will correlate with variation in the same set of parameters. 3.6.1.2 Z-Scores of Gait Parameters Correlate with Dissimilarity Judgements Because there are correlations within Walker ' s parameters principal components analysis will be used to transform the correlated gait parameters into uncorrelated components and to reduce the number of dimen-sions needed to describe the parameters of the motion signals. We test that dissimilarity judgements of pairs of gaits correlates with differences of the Z-scores of the parameters used to create the gaits, along the principal components of the gait parameters. Specifically, differences of the Z-scores of pairs of gaits will be tested for a significant correlation with the averaged dissimilarity judgement of the pairs of gaits, pô(i,j). Our hypotheses are: Hi : Vk,rp( PS(i,j),\Zk(VVi) - Zk{VVj)\) = 0 Hi : 3k, rp( P5(i, j),\Zk(VWi) - Zk(VVj)|) > 0 If we do not reject H2 then the principal directions of variation of the parameters, PC(Vw), do not effect changes in judgements of the similarity, pô(i,j), of the gaits. If we find that variation along the k principal component of the parameters of the gaits positively correlates with the dissimilarity judgements then we con-clude that the principal component parameter is a perceptually salient dimension. We do not assume that all participants' dissimilarity judgements will correlate with variation in the same set of principal components. 3.6.1.3 Distances Between Z-Scores of Gait Parameters Correlate with Dissimilarity Judgements Because the principal components of the parameters, PC(V^), are uncorrected and are ordered according to their variance, i.e., Xk{Vw) > Xd+i(Pw), we test for a correlation between the dissimilarity judgements of pairs of gaits and the Minkowski distance the Z-scores of the gait parameters. Our hypotheses are: Hi : \/k,Vp,rp(P5(i,j), WZi-kiP^) - Z^k{Wj)\\P) > 0 Hi : 3 A ; , 3 p , r p ( p ^ , j ) , | | ^ _ , ( P W l ) - Z ^ W > 0 If we do not reject Hi then distances in the orthogonalized mechanical motion space do not correlate with dissimilarity judgements. If we find that distances computed using the Minkowski norm LP in the first ^-dimensions positively correlate with the dissimilarity judgements then it is possible that dissimilarity judgements are formed using the FP norm and k dimensions. We do not assume that all participants' dissimilarity judgements will correlate with the same norm LP and first ^-dimensions. To compute distances we will use the Euclidean norm (L2), dominance norm and the city-block norm (L1). 3.6.2 Relationship Between Mechanical and Linguistic Motion Spaces Informally our hypotheses can be stated as: Changes in parameters of gaits affect proportional changes in descriptions of gaits. 3.6.2.1 Gait Parameters Correlate with Ratings Along Descriptive Scales We will test that descriptions of the gaits, pRs{i), to positively correlate with the gait parameters, VWi(ky For each rating scale, s, we test all gait parameters, k, for positive correlations with the ratings. Our hy-potheses are: HQ : Vfc,r p ( P ^ a (*) ,"P w . ( f c ) ) = 0 H \ : 3k,rp(pRs(i),Vw{k))>0 If we do not reject HQ then variation in the parameters, Vw, does not effect changes in the descriptions of the gaits, pRs(i). If we find that the k parameter of the gaits positively correlates with the descriptions along scale s then we conclude that the parameter corresponds with the linguistic scale. We do not assume that all participants' ratings will correlate with the the same sets of parameters. 3.6.2.2 Z-Scores of Gait Parameters Correlate with Ratings Along Descriptive Scales We will test that the descriptions of the gaits, pRs(i), correlate with the principal components of the gait parameters, Zk{Wi). For each rating scale, s, we test each principal component, PCk(Vw), for positive correlations with the ratings. Vk,rp(PRa(i),Zk(PWi)) = 0 3k,rp{pRs{i),Zk{VVi))>0 If we do not reject HQ then the principal directions of variation of the parameters, PC(PW), do not corre-spond to the descriptive scales. If we find that the k principal component positively correlates with rating scale s then we reject HQ, if this correlation is strong then we conclude that PC(VW) corresponds to scale s. We do not assume that all participants' ratings will correlate with the same set of principal components. Hi: Hi: 3.6.3 Relationship Between Psychological and Linguistic Motion Spaces Informally our hypotheses can be stated as: The larger the difference of description between the parameters of two gaits, the larger the dissimilarity judgement. 3.6.3.1 Dissimilarity of Gaits Correlate with Differences Between Ratings Along Scales We will test if the dissimilarity judgements of pairs of motions correlate with variation in along the rating scales. That is, we will test if pô(i,j) correlate with | pRs(i) - pRs(j)\ along the scales s: H60: \/s,rp(P5(i,j),\pRs(i)- PRs(j)\) = 0 Ht : 3s,rp(pô(i,j), \ PRs(i) - PRSU)\) > 0 If we do not reject HQ then dissimilarity judgements do not correspond with variation along any of the rating scales. If we find that variation in ratings along scale s positively correlates with dissimilarity judgements then we reject HQ and conclude that rating scale s describes a perceptually salient feature of the gaits. We do not assume that all participants' ratings and dissimilarity judgements will correlate along the same scales. 3.6.3.2 Dissimilarity of Gaits Correlate with Differences of Z-Scores of Ratings We will test if variation along the the principal components of the ratings correlates with the dissimilarity judgements. Specifically, differences of the Z-scores of the descriptions of two gaits will be tested for correlations with the averaged dissimilarity judgement of the two gaits, p5(i,j). Our hypotheses are: Hi : Vk:rp(PS(i,j),\Zk(pRs(i)) - Zk( PR,{j))\) = 0 H7a : 3k,rp(P6(t,j),\Zk(pRs(i)) - Zk(PRs(j))\) > 0 If we do not reject Hi then dissimilarity judgements do not correspond with variation along the principal components of the ratings. If we find that variation in ratings along component k positive correlates with the dissimilarity judgements then we reject HQ and conclude that component k describes a perceptually salient feature of the gaits. We do not assume that all participants' ratings and dissimilarity judgements will correlate along the same components. 3.6.3.3 Dissimilarity of Gaits Correlate with Minkowski Distances Between Z-Scores of Ratings Because the principal components of the parameters, PC( pRs(i)), are uncorrected (orthogonal) and are or-dered according to the amount of variance they account for in the parameters of the gaits, i.e., Xk( pRs{i)) > X(i+i( pRs(i)), we test for a correlation between the dissimilarity judgements of pairs of gaits and the Minkowski distance between the Z-scores of the ratings. Our hypotheses are: tf08 : VKrpipSiitfiAfr-kipRstt)) - Z1.k(PRa(J))\\P) = 0 Hi : 3k,rv{P'5{i,j),\\ZX-k{PR8{i)) - Zx.k{ PRs{j))\\P) > 0 If we do not reject HQ then distances in the orthogonalized linguistic motion space of the participant do not correlate with dissimilarity judgements. If we find that distances computed using the Minkowski norm LP in the first ^-dimensions positively correlate with the dissimilarity judgements then it is possible that dissimi-larity judgements are formed using the LP norm and ^dimensions. We do not assume that all participants' dissimilarity judgements will correlate with the same norm IP and first ^ -dimensions. To compute distances we will use the Euclidean norm (L2), dominance norm (L°°), and the city-block norm 3.6.4 Significance of Correlations We would like to see positive correlations stronger than 0.5 for the alternative hypotheses. This level of correlation between the gait parameters and the responses would be described as "moderate correlation, substantial relationship." Specifically, we will test the null hypothesis for each correlation of rp = 0 or "no relationship." The alternative hypothesis is that \rp\ > 0. For Experiment One, the probability of no relationship with an observed correlation of \rp\ = 0.5, would be p = 0.00016 or Student's £(50) = 4.083. That is, an observed correlation of \rp\ = 0.5 would only occur sixteen times out of one hundred thousand replications if there was truly no correlation. For Experiment Two, the probability of no relationship with an observed correlation of \rv\ = 0.5, would be p = 0.0036 (t(32)=3.162). That is, an observed correlation of \rp\ = 0.5 would only occur thirty-six times out of ten thousand if there was truly no correlation. These tests are two-sided, in many cases we will want to test for a positive correlation different from zero, rp > 0, in this case the probabilities of error are halved, e.g., rp = 0.5, t(50)=4.083, p = 0.00008. We have included a discussion of statistical tests for Pearson's product moment correlations in Appendix C . l for those not be familiar with the formulation of these tests. 3.7 Summary In the next two chapters we analyse the responses of the participants in the two experiments. The purpose of Experiment One is to demonstrate the collection of similarity judgments and descriptions of the movements made by dancers using a wide range of human walking movements. Using our hypotheses we will test for linear relationships between the mechanical, psychological, and linguistic motion spaces. The purpose of Experiment Two is an in depth experiment to determine the properties of the psychological motion space by using a narrower range of walking movements and a larger group of participants drawn from the general population. Chapter 4 Experiment One: Comparing and Describing Motions Experiment One was a participant experiment to demonstrate the collection of similarity judgments and descriptions of a wide range of human walking motions. The main goal of this experiment was to verify that our technique for collecting the judgements and descriptions produces repeatable results across a small group of participants. We analysed the participants' judgements and descriptions to determine how they are affected by the pa-rameters of the walking motions. Specifically, the first goal for this experiment was to determine how many dimensions are necessary to describe the walking motions. Our second goal was to determine which param-eters of the walking motions affect the participants' similarity judgements. Our third goal was to determine how the parameters of the walking motions affect the participants' descriptions. Our final goal was to deter-mine the relationship between the similarity judgements and the descriptions of the walking motions. 4.1 Overview In this experiment participants first compared the motion of walking figures ("the stimuli") using a contin-uous similar-dissimilar scale. Then the participants rated each motion along eight linguistic scales defined by adverbs and adjectives that could be used to describe the motions. Ten paid volunteer participants were recruited for this experiment. A l l participants were naive to the hypotheses and were paid $25 for their in-volvement. The participants included six females, and four males. A l l but one of the participants described themselves as dancers. Further details about the participants can be found in Appendix F. We started by analysing the mechanical and linguistic motion spaces defined respectively by the principal components of the motion parameters of the gaits and the principal components of the participant's rating of the gaits. We then looked at the dissimilarity judgements of the motions which define the psychological motion space to verify that the participants were able to compare the motions reliably. Having defined the motion spaces we analysed the relationships between the psychological, linguistic, and mechanical motion spaces. 4.2 Mechanical Motion Space The mechanical motion space of Experiment One was defined by the motion parameters of twenty-six gaits created using Bruderlin's gait generator W a l k e r . 1 Although the participants may have used features of the gaits that correspond with the motion parameters, it is likely that linear combinations of motion parameters also strongly correspond with their perceptions. For example, when we created the gaits, we tended to adjust the shoulder and elbow swings in a similar fashion so that the motion of the arms looked "natural." Also, some parameters were adjusted only for very extreme gaits and thus their strong correspondence with participant response may not reflect their perceptual salience but rather correlation with the settings of other parameters. Because we expected the motion parameters to be correlated, we used principal components analysis to reduce the dimensionality of the mechanical motion space and produce a new space that better describes the variation of the parameters. The first component, or dimension, points along the direction of maximum variation in the original gait parameters. The second dimension is orthogonal to the first and points along the direction of maximum remaining variation, and so on. Before computing the principal components (PCs) of the gait parameters we normalized each of the param-'The names, descriptions, sample animation frames, and parameters of the gaits can be found in Appendix B. I. eters to have zero mean and a standard deviation of one. This normalization insured that none of the param-eters dominates the principal components simply because they are expressed in different units. For example, while many of the parameters have a range from zero to one, the parameter d e s i r e d _ s t e p _ f r e q u e n c y is expressed in steps per minute which has a range from 72 to 144. We computed: PC(VW) The principal components (PCs) of the gait parameters. PC\(VW) is the first PC of the gait parameters, -PC^T-w) is the second PC of Pw> and so on. The PC(PW) form an orthonormal basis and are sorted by magnitude such that their corresponding eigenvalues are decreasing. Xk{Pw) The eigenvalues of the gait parameters, also the amount of variance of each PCk(Vw). Sorted by magnitude such that Ai (T^ w) > \2{Pw) > • • • > ^n(Pw)-Z(VW) The Z-scores of the gait parameters, also known as Vw transformed into the space defined by PC(VV). Z{VV) = PC(Vv). Zx (Vw) are the values along the first PC, Z2{VW) are the values along the second PC, and so on. Z( 1_ f c)(7 ? w) are first k dimensions of Z(Vw). As the PC(Vw) lie along the maximum variation in the gait parameters, and since each successive com-ponent captures less and less variation, we reduced the dimensionality of the mechanical motion space by approximating the space with the PCs that capture, say 90%, of the variation in Vw or by accepting only the PCs that have a variance greater than one. The 90% cut-off requires the first ten PCs, which are listed in Table 4.1, while the variance of one cut-off requires only the first seven. Note how arra.out and f oo t_ang le — which affect motion perpendicular to the viewpoint used in the experiment — are the first two dominant parameters of the PC\(V\j). In other words, when we adjusted arm_out and f oo t_ang le , which was rare, we also tended to change many other parameters — often to their most extreme values. Thus arm_out and f oo t_ang le are "sentinel parameters" that indicate "this motion is very different from normal." The twenty-six gaits we created are not easily described by a small number of "mechanical space" dimen-sions. However, it remains a task for future experiments to determine how to best create a sampling of motions in the psychological motion space. In Figure 4.1 are scatter plots of the gait "positions" using Z\-$(V\rt). As you can see, there are outlying gaits, and large unexplored regions of the parameter space. Perhaps the approach of Marks et al. (1997) could be used to better sample the mechanical space after the perceptually salient features have been identified. When we analysed the relationships between the mechanical motion space and the linguistic motion space 0 CM C CD C o Q . , 1 - 2 O -10 -5 0 Component 1 -5 0 Component 1 - 5 0 Component 1 -5 0 Component 1 Figure 4.1: Scatter plots of Z(Vw) for the first five components. These scatter plots show that while the variation in the gaits is dominated by a few of the gaits, there is generally a good spread of the gaits given that twenty-two parameters were adjusted to create only twenty-six gaits. Component 1 is used as the horizontal position in each plot, and the four next strongest components are used for the vertical axes. These five components represent 68.4% of the normalized variance of the gait parameters — equivalent to the variance of fifteen parameters. k Afe(7>„) % Variance % Variance First Two Dominant Parameters 1 4.2047 19.11 19.11 arm_out(#6), foot_angle(#21) 2 3.6384 16.54 35.65 hip_swing3(#18), knee_midss(#16) 3 2.8567 12.98 48.64 desired.velocity(#l), heel_strike_flag(#29) 4 2.5427 11.56 60.19 desired_step.length(#2), desired_step_frequency(#3) 5 1.8074 8.22 68.41 arm_swing_factor(#5), knee_impact(#17) 6 1.3511 6.14 74.55 torsoJilt(#9), kneeJmpact(#17) 7 1.0832 4.92 79.47 percent-Shoulder_rot(#4), knee_midss(# 16) 8 0.8792 4.00 83.47 bounciness(#14), torso_tilt(#9) 9 0.7943 3.61 87.08 torso_sway_max(#10), bounciness(#14) 10 0.6694 3.04 90.12 pelvis_rot_max(#12), lateral_disp_factor(#l 1) Table 4.1: The variance (Xk(Vw)), proportion of variance and sum of proportion of variance for the first ten principal components of the mechanical motion space. In the last column are listed the two parameters with the largest coefficients for each component. The horizontal line between the seventh and eighth components indicates the cut-off point if we use A > 1 as our criterion rather than X > 0.9 we used both the original parameters, 7-w, and the Z-scores, Z(VW). When analysing the relationships between the mechanical motion space and the psychological motion space we used the differences of the original parameters, \Vv^k) - ^ (it) I. t n e differences of the Z-scores, \Zk(V^) - Zk(VV])\, and Minkowski distance between the gaits using the Z-Scores, \\Z1-k(VWi)-Z1-k(Wj)\\p. 4.3 Lingusitic Motion Space In the second part of the experiment the participants rated each walking motion using eight continuous scales: fast—slow flexible—stiff smooth—bouncy straight—crouching energetic—tired still—swinging light—heavy upright—tipping normal—strange The labels for the rating scales were selected by constructing a list of descriptive words that could be used to describe human walking motions. Candidate words were collected from books and texts on movement observation, dance instruction and nonverbal communication.2 Additionally, visitors to the Imager labora-tory during an "open house" were invited to describe the walking motions proposed for use in Experiment One by checking off words on the initial list and suggesting additional words. The initial list consisted of: angular, bouncy, determined, energetic, fast, flexible, flying, heavy, in-a-hurry, light, lumbering, normal, old, rough, running, slow, slumped, smooth, snappy, sneaky, stiff, stomping, swinging, twisty, waddling, and young. The final set of labels was formed by pairing the words according to opposite meanings and eliminating all but one of synonyms, for example fast, flying, in-a-hurry, and running were reduced to fast. The primary reason for having participants rate gaits was to gather descriptions of the gaits to define the participants' linguistic motion spaces. We performed a correlational analysis of the descriptions of gaits and the motion parameters to determine which parameters affect ratings along the each of the descriptive scales. As with the motion parameters, we expected the rating scales to be correlated. For example, we expected the fast-slow and young-old rating scales to be strongly correlated. By computing the principal components of the each participant's ratings, pRs{i), we approximated their linguistic motion space with the set of com-ponents that describe as much of the variance as possible, up to a limit, say 80-90% of the original variance. We used these components when we examined the relationships between the participants' psychological and linguistic motion spaces. Computing interprétable PCs of the participant's pRs{i), required two things: linear correlations between the scales and strong correlations between the ratings scales. We examinated scatter plots of the ratings scales to determine that the scales do appear to be linearly correlated — rather than non-linearly. Copies of the scatter plots can be found in Appendix A. 1.3.1. The second requirement of strong correlations presents the expectation that the variance among the ratings will be distributed in various directions. If none of a participant's rating scales are (highly) correlated then performing a principal components analysis on their ratings will not find a meaningful set of new dimensions. For each of the participants, the correlation between each of the rating scales was computed to determine the strength of agreement between the rating scales. This forms the correlation coefficient matrix: 2 Miller and Johnson-Laird (1976) "Language and Perception." Moore and Yamamoto (1988) "Beyond Words: Movement Observation and Analysis." and Wall and Murray (1990) "Children and Movement: Physical Education in the Elementary School" C(P) = rp{pRi, pRi) rp{PRi, PR2) rP{pR2, PRI) rp(pR2, pR2) '•p(pRs, pRi) rp(PRs,pR2) rp(pRi, pR8) rp{pR2, pRs) rp{ pR8, pR8) Where, PRS Descriptive rating of gaits on scale s by participant P. rp(pRs, pRt) Pearson's product moment correlation coefficient between rating scales s and t. When we examined the non-diagonal elements of the correlation coefficient matrices we found that partic-ipants #2, and #3 tended to have only weak correlations: \rp\ < 0.4, while the rest of the participants had many rating scales correlated stronger than \rp\ > 0.5. For #3 this is one reason why she was excluded from further analysis. Though #2's correlations are weak, we decided to include her since her principal components appeared to be very similar to the other participants. You can find a summary of rating scale correlations in Appendix A.1.3. Having confirmed that the remaining six participants have strong correlations between rating scales, and that the rating scales have linear correlations, we computed the principal components of the descriptions. Using the M A T L A B Statistics Toolbox, we computed the principal components for all of the participants independently. The M A T L A B function p r i n c o m p was given the ratings of each participant, pRs{i), and it returned four matrices (per participant): PC( pRs{i)) The Principal Components (PCs) of pRs{i). Also known as the eigenvectors of pRs{i). Each row of pRs(i) is an observation, each column a variable. PC\{ pRs(i)) is the first PC of pRs(i), PC2{ pRs{i)) is the second PC of pRs(i), and so on. The PC(PRs(i)) form an orthonormal basis and are sorted by magnitude such that their corresponding ei-genvalues are decreasing. Each column of PC{ pRs{i)) is a basis vector. \k{pRs(i)) The eigenvalues of pRs{i) and also the amount of variance of each PCk{ pRs{i))- Sorted by magnitude such that Xi{pRs(i)) > A 2 ( pRs{i)) > • • • > K{ pRs(i))-Z( pRs{i)) The ratings of participant P transformed into the orthonormal space such that Z\{ pRs{i)) corresponds to P C i ( R i s A ) . T 2 Hotelling's T 2 statistic: a measure of the multivariate distance of each rating, pRs(i) from the centre of the data set. Al l ratings scales were normalized, per participant, to have zero mean and variances of one. This normal-ization was performed because the scales might have different dynamic ranges for each participant. The main effect of normalization is that the principal components are computed using a correlation coefficient matrix rather than a covariance matrix. Also, normalizing the ratings means that the A( pRs{i)) returned by p r i n c o m p can be interpreted as the number of ratings scales each component is equivalent to. Also, the direction of the principal components was 'standardized.' Since the algorithm used to compute eigenvectors can produce the same principal components but with reversed direction depending on the algo-rithm's implementation and the specific correlation coefficient matrix, two participants may have principal components that point in similar directions but the coefficients will have opposite signs. To negate this con-fusion, the first coefficient of each principal component for each participant was examined, if it was negative, then the signs of the principal component's coefficients, and corresponding Z-scores were reversed. This standardization can easily be implemented in M A T L A B . We have summarized in Table 4.2 the average, across participants, proportion of variance explained of the components and the cumulative explanatory power of the components. We have excluded participants #3, #4, and #9 when we computed the values presented in Table 4.2. We will only summarize their components: #3's first PC has a variance of 1.6, and #4's variance of 1.1, and #9's a variance 2.2. This is a result of their poor correlations between ratings scales. Without strong correlations between the rating scales, principal components analysis cannot find principal directions of variation that combine rating scales. Average % Average Yl % Average Component Variance Variance Variance 1 3.6619 45.77 45.77 2 1.5361 19.20 64.98 3 0.9947 12.43 77.41 4 0.7088 8.86 86.27 5 0.4655 5.82 92.09 6 0.2890 3.61 95.70 7 0.2118 2.65 98.35 8 0.1321 1.65 100.00 Table 4.2: Plot of the strength of the principal components of the ratings. Principal components for each participants ratings were computed separately and the variance and proportion of variance of the PCs of participants #1, #2, #5, #6, #7 and #8 were averaged to create these values. Shown in Figure 4.2 is a box plot of the variance of each component across the participants. As you can see, the first principal component is fairly strong, explaining on average the variance of three and a half rating scales. The second component explains on average one and a half rating scales, the third another scale, and the remaining components less and less variance. 4 5 Principal Component Figure 4.2: Box plot of the variance explained by each principal component for participants #1, #2, #5, #6, #7 and #8. The principal components of each participant's ratings were computed and the strength of the components is presented in this plot. Ratings were normalized to have mean of zero and variance of one, thus the strength of a component (its eigenvalue) indicates the number of normalized rating scales the component is equivalent to. 4.3.1 Common Patterns Among the PC's of the Participants Ratings One reason for using PCA to analyse the ratings is to reduce the dimensionality of the linguistic motion space. We were conservative and set a target of approximating 80% of the variance of the ratings. This threshold tends to also be near the transition from where the variances of the PCs explain more than one rating scale (A > 1) to where the PCs explain less than one rating scale (A < 1). Using this threshold, each of the participant's ratings can be approximated by the first three to four PCs. The last three PCs account for only 6-12% of the variance. Many of the participants have common PCs — or at least PCs that point in very similar directions relative to the other participant's PCs. For example, in Figure 4.3 we have plotted the coefficients of each participant's first PC. As you can see, participants #2, #5, #6, #7, and #8 have a very similar first PC — the maximum angle between any of the first PCs is 61.5°. If we leave out #1, the maximum angle is 35.2°. Across the participants, this dimension has coefficients with about the same magnitude — thus this dimension is a weighted average of the rating scales. The second PCs of #1, #5, and #6 and the third PC of #8 are also similar. As we can see in Figure 4.4, the common pattern is one of coefficients with about the same magnitude — thus we have another weighted average of the rating scales. However, three rating scales are reversed relative to the other five. The third and fourth PCs tend to be dominated by one or two large coefficients. It is with these dimensions that we see true individual differences. Though each participant interpreted the rating scales differently, rating scales 1 (fast-slow), 2 (flexible-smooth), 3 (smooth-bouncy) and 6 (light-heavy) very often dominate the third and fourth PCs. Plots of these coefficients are included in Appendix A. 1.4. Participant #1 Participant #2 Rating Scale Index Rating Scale Index Participant #5 Participant #6 Rating Scale Index Rating Scale Index Participant #7 Participant #8 Rating Scale Index Rating Scale Index Figure 4.3: Plots of the magnitude and sign of the coefficients of the first principal component of each participant's ratings. Note the commonality between #2, #5, #6, #7 and #8. The horizontal axes are the rating scales, while the vertical axes are the coefficient values. Participant #1 Participant #2 • ' 1 -0 ) o Î î : • Rating Scata Index Rating Seals Index Participant #5 Participant #6 Rating Scale Index Rating Scale Index Participant #7 Participant #8 (third PC) Rating Scale Index Raling Scale Index Figure 4.4: Plots of the magnitude and sign of the coefficients of the second principal component of partic-ipants #1, #2, #5, #6, and #7, and the third principal component of participant #8. The horizontal axes are the rating scales, while the vertical axes are the coefficient values. 4.3.2 Interpreting Individual PC's of the Participants Ratings To interpret the individual participant's rating scale PCs, PC( pRs{i)), we looked at both the strength of the components, \{pRs{i)), and the coefficients assigned to each rating scale, Z(pRs{i)). This process determined how many dimensions are required to approximate the participant's linguistic motion space, and what labels we assigned to the components. We will demonstrate the interpretation process with #l's PC( pRs(i)). Complete interpretations of each of the participant's rating scale PCs can be found in Ap-pendix A . 1.5. We have plotted in Figure 4.5 the strength of the PCs of #l's ratings and the coefficients of the first four PCs. The first two PCs are very strong, accounting for 4.1 and 1.9 of the variance of the original rating scales. The remaining six PCs have variances below 0.58. Together, the first two PCs account for 75.1% of the original variance in #l's ratings. Since we are mainly interested in dimensionality reduction, we could either use only the PCs with A > 1 or use as many PCs as necessary to account for, say 80% of the original variation. In #l's case, 82.4% of the variation is achieved by using the first three rating scales. To interpret the PCs, we use both the magnitude and the sign of the coefficients of the PCs, for #l's first PC, PCi(#l), we have the following coefficients: Rating Scale PCi (# l ) fast—slow 0.3610 flexible—stiff -0.2485 smooth—bouncy -0.4381 young—old 0.2601 energetic—tired 0.2897 light—heavy -0.3574 graceful—spastic -0.4349 normal—strange —0.3841 Since larger coefficients have more influence than smaller coefficients, we can reorder the rating scales by decreasing coefficient magnitude. For PCi (#1), the coefficients are all close to the same value so the reordering has a minor effect. The signs of the coefficients tell us which rating scales should be "reversed," by flipping the labels left to right — it does not matter whether we negate negative or positive coefficients, as long as we are consistent and flip the associated labels. After ordering the scales by coefficient magnitude, and flipping the scales according to sign of coefficient, 1 2 3 4 5 6 7 8 P r i n c i p a l C o m p o n e n t Coefficients of P C ^ * ! ) 1 2 3 4 5 6 7 8 Coefficients of P C (#1) 1 2 3 4 5 6 7 Coefficients of PC 3(#1) 1 2 3 4 5 6 7 8 Coefficients of P C (#1) 1 2 3 4 5 6 7 Figure 4.5: Top graph: Plot of the variance explained by each principal component of participant #l's ratings. The height of each bar indicates the number of normalized rating scales the component is equivalent to. The line shows the cumulative variance explained by using the first n principal components. The dotted horizontal line indicates the threshold of 80% explained variance. Middle and bottom graphs: plots of the coefficients of #l's first four PCs. Horizontal axes are the rating scales, the vertical axes are magnitude and sign of coefficients. we have: Rating Scale PCi(#l) bouncy—smooth 0.4381 spastic—graceful 0.4349 strange—normal 0.3841 fast—slow 0.3610 heavy—light 0.3574 energetic—tired 0.2897 young—old 0.2601 stiff—flexible 0.2485 Because the coefficients all have approximately the same magnitude, this dimension is a weighted average of the ratings scales. By reading down the left-hand and right-hand scale labels we find that in one direction along this dimension we have motions which are: "bouncy, spastic, strange, fast, heavy, energetic, young, stiff" In the other direction we have motions which are: "smooth, graceful, normal, slow, light, tired, old, flexible" We can interpret PC2(#1) in a similar fashion, ordering and flipping the scales according the coefficients, results in: Rating Scale PC2(#1) young—old 0.5632 energetic—tired 0.5271 flexible—stiff 0.4966 light—heavy 0.2685 fast—slow 0.1956 graceful—spastic 0.1676 normal—strange 0.1232 smooth—bouncy 0.0690 This dimension is dominated by the young—old, energetic—tired, and flexible—stiff rating scales, in one direction along PC2(#1) we have motions that are: "young, energetic, and flexible" and in the other direction we have motions which are: "old, tired, and stiff" Finally, to reach 82.4% of the variation in the ratings, we add PC.3(#1): Rating Scale PC 3(#1) normal—strange fast—slow graceful—spastic tired—energetic young—old smooth—bouncy heavy—light stiff—flexible -0.7047 -0.4891 -0.4397 -0.1657 -0.1372 -0.1076 -0.0818 -0.0792 This dimension is dominated by the normal—strange rating scale, with much smaller contributions from the fast—slow and heavy—light scales. So, in one direction along PC3(#1) we have motions that are: " N O R M A L , fast, and heavy" and in the other direction we have motions which are: "STRANGE, slow, and light" To summarize, the principal components of participant #l's ratings are: 4.3.3 Summary of Principal Components of Participant's Ratings The participant's ratings tend to have moderately strong correlations (rp > 0.5) between rating scales which allowed us to use principal components to reduce the dimensionality of the linguistic motion space to 3-4 "interesting" dimensions. As noted, several of the participants have components which point in similar directions, for example, • #2, #5, #6, #7, and #8 have a very similar first PC: "bouncy, spastic, strange, fast, heavy, energetic, young, stiff" versus "smooth, graceful, normal, slow, light, tired, old, flexible" "bouncy, spastic, strange, fast, heavy, energetic, young, stiff' versus "smooth, graceful, normal, slow, light, tired, old, flexible" "young, energetic, and flexible" versus "old, tired, and stiff" ' N O R M A L , fast, and heavy" versus "STRANGE, slow, and light' • Second PCs of #1, #5, and #6 and third PC of #8: "young, energetic, and flexible" versus "old, tired, and stiff" • Third and fourth PCs tend to be dominated by one or two ratings scales (large coefficients for these scales). Principal Components Analysis can only analyse one set of data at a time, in our case, the ratings of one participant at a time. While we looked for components with similar coefficients, we could not simply pool ratings data to analyse the commonalities between the participants' ratings. Instead, to analyse fc-groups of data we would have to use Common Principal Components. CPC allows us to compare the covariance matrices of each of the A;-groups and compare the resulting principal components for commonalities. A statistical requirement of CPC is we must not normalize the rating scales and must use covariance matrices rather than matrices of correlation coefficients. In this experiment, we normalized the rating scales to even out the individual participant's interpretations of the rating scales. Perhaps after future experiments verify the rating scales as a repeatable measure of the perceptual properties of motions we will be able to use covariance. Until that time, we felt that mixing correlation and covariance in our analyses would only confuse things at this early stage when we are still verifying the experiment methodology. Readers interested in CPC, will find in Jackson (1991) a short introduction to the methods of comparing covariance matrices in Section 16.6, Keramidas, Devlin and Gnanadesikan (1987) discusses a graphical technique of comparing covariance matrices, and Flury (1988) is a complete discussion of CPC. In an upcoming section, we will present our use of the principal components of the participants' ratings in our analysis of the relationships between the psychological and linguistic motion spaces. Since the compo-nents are independent we tested for correlations between differences of Z-scores along individual compo-nents and the dissimilarity judgements, as well as distances of Z-scores along multiple components and the dissimilarity judgements. 4.4 Additional Analysis of Motion Comparison Trials In addition to the relationships between the motion spaces we were interested in confirming that the partici-pants were capable of performing the experimental task in a reliable fashion. In pursuit of this goal several analyses of the participant's responses were performed to confirm that participants were not "guessing" throughout the experiment. We computed the average variance of dissimilarity judgements per participant, and examined histograms of the participants' dissimilarity judgements. These analyses revealed that participants #4 and #9 judged almost every single pair of gaits to be completely dissimilar. We were unable to instruct #4 and #9 to report judgements of "relative similarity." As expressed by #9 in her post-experiment interview, "The first motion is the instructor, and the second motion is the student. If I perceive the two motions to be different then they are different." For this reason #4's and #9's responses have been ignored in many of the analyses and additional participants were recruited to replace them. We also computed the correlation of each participant's dissimilarity judgements between experiment blocks to determine if participants' dissimilarity judgements were stable throughout the experiment. The weakest block-pair-wise correlation for any participant was rp > 0.45 (t(50)=3.56, p=0.0008). We also did not find significantly lower correlations between blocks 0-3 than between blocks 0-1, 1-2, or 2-3. This indicates that variance in dissimilarity judgements is most likely due to participant inaccuracy or order effects between trials rather than changing strategies or learning effects. An additional question asked is whether the order of presentation affect the dissimilarity judgements? That is does, per (i, j) = pô(j, i)1 If the order of presentation does matter, then future experiments would require symmetrically balanced trials, which could potentially mean n2 trials for n motions. As an artifact of the random pairing of the gaits, there were three gait pairs that were compared in both orders, that is "symmetrically": T(5, 9) and T(9, 5) bounce versus hip toe strike T ( l l , 19) and T(19,11) marching versus small locked steps T(13,18) and T(18,13) marching straight arms versus small bounce Our null hypothesis is that the average dissimilarity judgement of two gaits is the same regardless of order of presentation. The alternative hypothesis is that the average dissimilarity judgement of two gaits is affected by order of presentation, making the two judgements different: We used ANOVA to compare the average dissimilarity judgements of the above trial pairs within each partic-ipant's judgements. ANOVA reports the probability, p, that the difference between the average dissimilarity judgements is due simply to variation in the participant's responses. In other words, if we feel that order of presentation affects dissimilarity judgements, and we reject the null hypothesis, we would be making an error with probability p — however this ignores the cumulative nature of making an error we make many comparisons. Table 4.3 presents the average of the dissimilarity judgements for each of the trials in questions and the p-values computed by ANOVA. If there were many small p-values (p < 0.1) in Table 4.3 then this would indicate that judgements of dissimilarity are strongly affected by order of presentation. While we expect some hysteresis, and while there are four small p-values that could cause us to reject the null hypothesis, it does appear that the order of presentation does not strongly affect dissimilarity judgements. This implies that future experiments could use half the number of trials (n 2 /2 rather than n2). While this may not seems like a very large decrease it does help to decrease the length of the experimental session which in turn leads to an avoidance of participant mental fatigue. We examined the symmetry of dissimilarity judgements more closely in Experiment Two. 4.5 Analysis of Relationships Between Motion Spaces Having computed the principal components of the mechanical and linguistic motion spaces, and confirmed that the dissimilarity judgements have some consistency, we then analysed the relationships between the motion spaces. Ho • pô{i,j) = pô{j,i) H, a. pS{i,j) ^ pS{j,i) Average of Dissimilarity Judgements p# p<J(5,9) PÔ(9,5) P<$(11,19) P<J(19,11) P<J(13,18) PCJ(18,13) #1 0.7033 0.5500 0.6533 0.8467 0.4267 0.3200 #2 0.4767 0.6500 0.5233 0.4633 0.2367 0.4733 #5 0.2633 0.3267 0.4067 0.3433 0.3633 0.4333 #6 0.6500 0.7200 0.6700 0.7300 0.4100 0.5000 #7 0.5650 0.6000 0.2500 0.5367 0.1900 0.1000 #8 0.5100 0.5600 0.7833 0.7567 0.4033 0.5667 Probability of Average of Dissimilarity Judgements are Different T(5,9) vsT(9,5) T ( l l , 1 9 ) vsT (19 , l l ) T(13,18) vsT(18,13) P# p-value p-value p-value 1 0.1625 0.0316* 0.4027 2 0.1283 0.7590 0.0574* 5 0.3709 0.6137 0.6525 6 0.3974 0.5989 0.4190 7 0.8178 0.0877* 0.2046 8 0.6130 0.7223 0.0707* Table 4.3: Probabiligy of asymmetry of dissimilarity judgements as determined using ANOVA. Average of dissimilarity judgements and probability of differences of means are not significant, p-values computed using ANOVA within participants (df = 1,4), starred are p < 0.1. As noted in the previous chapter, we hypothesized linear correlation between proximities in the psychologi-cal motion spaces and distances in the other two spaces. We also hypothesized linear relationships between the motion parameters and the ratings. We used correlation to test these relationships. For this experiment there are fifty-two gait pairs, and thus all correlations were tested for significance using n = 52 (df = n — 2 = 50). Probabilities of observing a correlation \rv\ > 0 when the null hypothesis is 7- = 0 are presented in Appendix C . l . 4.6 Relationship Between the Mechanical and Psychological Motion Spaces We test the relationship between the mechanical and psychological motion spaces by comparing dissimilar-ity judgements and • differences of Walker ' s parameters • differences of Z-scores of Walker ' s parameters • distance of Z-scores of Walker ' s parameters 4.6.1 Correlation Between Dissimilarity Judgements and Differences of Walker's Param-eters We started by testing for a positive correlation between the dissimilarity judgements and differences between the parameters used by Walker to create the gaits. We used the following quantities: pô(i,j) Averaged dissimilarity rating by participant P between gaits i and j where i was presented first. Without the subscripts i and j pô(.,.) is a vector of averaged dissimilarities for all trial pairs (averaged across experiment blocks). •pWi The vector of parameters used by Walker to create gaits i. \PV]i - PV]j | is the absolute difference in the parameters between gaits i and gaits j. rp{pS{.,.),\VVi(k) - ^ w f c ( | ) ) Pearson's product moment correlation coefficient between the vector of dissimilarity judge-ments and the vector of differences between the parameter k used to define the gaits. The pairing of gaits in the computation of |"Pw,-(fc) —^w f c(|) matches the order of the trials T(i,j) used to define p6(.,.). To summarize, 76.5% of the 132 correlations can be considered weak {rp < 0.3). However, of the remain-ing correlations, each participant has at least one correlation greater than 0.4, and three of the participants (AS, JA, and JDH) have a correlation stronger than 0.5. The probability of observing \rp\ = 0.5 when we assume rp = 0 is 0.00016. (A table of probabilities can be found in the Appendix C.l .) This indicates that variation in at least one parameter strongly influences judgements of dissimilarity. The parameters with the strongest correlation are listed in Table 4.4 While most of the parameters that correlate strongly with dissimilarity judgements would be visible in a profile view of a gait, two are not. arm.out , which controls how high the arms are raised from the side of the torso, or f oo t_ang le , which controls the angle the feet make with the direction of walking (i.e., duck toed versus pigeon toed), are included in the above list as an artifact of how the gaits were created. As noted in Section 4.2 these parameters are associated with the first principal component of the parameters and thus indicate "this motion is very different from normal." Thus correlations between these parameters and the dissimilarities, will more often correlate with these parameters than with any of the other parameters that tend to vary more "randomly." Since the participant's dissimilarity judgements correlate strongly with differences in these parameters we have found a linear relationship between the psychological and mechanical motion spaces. When we created the gaits, we attempted to select settings of Walke r ' s parameters to create a wide range of gaits while minimizing the total number of gaits.4 Also, the gaits were paired randomly in an attempt to sample the possible pairings and prevent the participants from using a strategy based on the order of the trials to form their dissimilarity judgements — however hysteresis is still possible which is why the trial order across blocks was varied. Because the gaits vary widely from the "normal gait," the formation of a dissimilarity judgement should require the integration of many parameters. In the next section, we will attempt to determine what these parameters are by comparing the dissimilarity judgements to the Z-scores of the parameters. Because the parameters of the gaits vary along several principal directions, in the next section we report on our analysis of the relationship between the dissimilarity judgements and the parameters but use the •'Six participants times twenty-two parameters. d W a l k e r provides twenty-two parameters to define the style of a gait. Creating a small, but complete, sample of gaits was not an easy task as we had to trade off "ranges of parameters" for point samples: If we had given each parameter two possible settings then 2 2 2 = 4,194, 304 gaits would have resulted. p# rP Parameter Name and Number 1 0.405 arm_out(#6) 2 0.510 bounciness(#14) 5 0.656 arm_out(#6) 5 0.453 t o r s o _ t i l t ( # 9 ) 5 0.433 f oot_angle(#21) 6 0.456 arm_out(#6) 6 0.428 e 1 b OWJ: O t _max(#8 ) 6 0.407 t o r s o _ t i l t ( # 9 ) 7 0.545 arm_out(#6) 7 0.560 bounciness(#14) 7 0.429 f oot.angle(#21) 8 0.435 arm_out(#6) 8 0.439 e l b o w j r o t _max(#8) 8 0.430 t o r s o _ t i l t ( # 9 ) 8 0.454 knee_midss(#16) 8 0.538 hip_swing3(#18) Table 4.4: For each participant, correlations between their dissimilarity judgments of pairs of walking mo-tions and the differences in parameters of the walking motions that are stronger than rp = 0.4. principal components of the parameters instead of the parameters themselves. 4.6.2 Correlation Between Dissimilarity Judgements and Differences of Z-Scores of Walker's Parameters Although in some sense, the principal components of the gait parameters — which we computed in Sec-tion 4.2 — are "arbitrary" and possibly have no relationship to the perceptually salient parameters, the motion parameters that correlate strongly with the dissimilarity judgements (last section) also tend to be the dominant parameters of the principal components. Table 4.5 presents the strongest three correlations be-tween the dissimilarities and differences of the parameters for each participant and indicates which principal component the parameter dominates. Strongest Correlations, rp from §4.6.1 Parameter # (Component k) p# First Second Third First Second Third 1 0.4054 0.3832 0.3618 6(1) 17(5) 9(6) 2 0.5105 0.3759 0.3147 14(8) 6(1) 8(7) 5 0.6560 0.4529 0.4327 6(1) 9(6) 20(1) 6 0.4560 0.4283 0.4068 6(1) 8(7) 9(6) 7 0.5598 0.5453 0.4293 14(8) 6(1) 20(1) 8 0.5378 0.4538 0.4389 17(5) 15(7) 8(7) Table 4.5: Summary of which parameters correlate strongly with the dissimilarity judgements and which principal component are dominated by the parameters. For each participant, the strongest three correlations between parameters and their dissimilarity judgements are list first, then the parameter number and in paren-thesis the number of the component dominated by the parameter. This table combines data from Tables 4.4 and 4.1. We also computed the correlations between the dissimilarity judgements and differences of Z-scores of the parameters along each principal component. We computed: pô(i, j) Averaged dissimilarity rating by participant P between gaits i and j where i was presented first. Without.the subscripts i and j pô(.,.) is a vector of averaged dissimilarities for all trial pairs (averaged across experiment blocks). Z(Pvi) The Z-scores of the gait parameters, also known as Vw transformed into the space defined by PC(VW). Z{VM) = PC{PW). Z\{Vu) are the values along the first PC, Z2{VV) are the values along the second PC, and so on. Z^_k)(Vw) are first k dimensions of Z(Vw). rp(p5(.,.l\Zk(VVi) - Zk(PVj)\) Pearson's product moment correlation coefficient between the vector of dissimilarity judge-ments and the vector of differences between the kth Z-score of the parameters used to de-fine the gaits. The pairing of gaits in the computation of \Zk(PWi) — Zk(VWj)\ matches the order of the trials T(i,j) used to define p<5(.,.). As with the correlations between the dissimilarities and the parameters (Section 4.6.1) most of the correla-tions are very weak. 94.7% of the correlations are rp < 0.3.5 However, when we use the absolute value of the correlations ( | 7 p | < 0.3) this decreases to 85.6%. There are two interesting patterns in these correlations. First, every participant's dissimilarity judgments correlates stronger than rv > 0.3 with only the first two principal components: PCi(Vw) and P C ^ P w ) , with PC\ {Vw) forming the participant's strongest positive correlation. Second, the next strongest correlation for the participants has an interesting pattern of favoring PC^iV^), PC^(V^), and PCs{Vvi) — indicating that these components capture perceptually salient variation in the motions. As we can see in Table 4.6 two participants each favor one of these components with their second best correlation. Also listed in Table 4.6 are the strongest negative correlations. Except for#5's negative correlation with PCz{V^) of-0.3187, all of the strong negative correlations are with principal components with very small variances. Thus these strong correlations are along components representing very little variation of the motion parameters. Positive Correlations Negative Correlations First Second Second First p# PC f cePw) rp PCk(Vw) rp PCk(V„) rp PCk{Vv) rp #1 î 0.4472 2 0.2171 17 -0.3185 15 -0.3464 #2 î 0.3255 8 0.2866 12 -0.3212 20 -0.3795 #5 î 0.7247 4 0.1900 16 -0.3331 3 -0.3817 #6 î 0.5238 4 0.2127 12 -0.3367 17 -0.3402 #7 î 0.5473 8 0.2349 15 -0.2449 16 -0.2583 #8 î 0.4404 2 0.3422 17 -0.2212 20 -0.3103 Table 4.6: Correlations between participant's dissimilarity judgements and differences of Z-scores of mo-tion parameters. The two strongest positive correlations and strongest two negative correlations are listed. The positive correlations indicate that components PCit2,4,s{Pw) correspond to dimensions of the psycho-logical motion space. While the negative correlations, which are along components with variances less than 0.3993, suggest that taking only the first 8-10 principal components of the parameters sufficiently captures the perceptually salient variation of the gaits. From the results of this and the last section, we conclude that the participants judged the dissimilarity of the gaits based on differences defined by Walke r ' s parameters. While the correlations between the dis-similarities and the differences of the Z-scores of the parameters are small, the strong positive correlations occur along the first eight principal components — that is the components with variances greater than one. The strong negative correlations, except for participant #5, are along the weakest of the components — that 3125 correlations out of 132 — six participants times twenty-two components. is the components with variances less than 0.3993. The pattern of correlations between #5's dissimilarity judgements and the differences of the Z-scores of the parameters is very interesting. The correlation with PCi(V\fi) is very strong, perhaps as strong a correlation we'd expect between the psychological and me-chanical motion spaces which implies that for #5 either the first principal component captures almost all of the perceptually salient variation of the gaits, or that he formed dissimilarity judgements by focusing on only the gait variation lying along PCi(Vw). Whereas we do not know how the participants combined the differences in the parameters to make their judgements, some of these correlations could be considered strong enough to hypothesize a model utiliz-ing linear combinations of the parameters. Let us see if using the distance between multiple PCs of the parameters improves on these correlations. Because the principal components of the parameters are uncorrelated, we can use them to define an Eu-clidean space and test if distances in this space correlate with dissimilarity judgements. Our analyses fol-lowed the same lines as in this section, with the difference that we used ranges of principal components, PC ii 2 fc}(^w)> rather than individual components and the Minkowski distance norms, Ll, L2, and L°°. 4.6.3 Correlation Between Dissimilarity Judgements and Distances Between Z-Scores of Walker's Parameters To compute distances in the "principal mechanical motion space" we needed to decide how many dimensions we're going to use and which distance metric we use to compute distances. Because we knew that the linear distance model for dissimilarities is unlikely to be the true psychological model, we tried a few distance metrics and observed the effect of varying the the number of principal components (PCs) — up to the ten components presented before. We used variations on the Minkowski distance metrics: City-block metric, Ll k Euclidean distance metric, L2 Dominance metric, L CO We compute: rp(PS(.,.),\\Zk(PWi)-Zk(VWj)\\p) forp = {1, 2, oo}, and d = { 1 , 2 , . . . , 10}. To summarize, when using the L 1 metric the strongest correla-tions occur with the first or the first two PCs. The L 2 and L°° metrics have strongest correlation when using the first, two, or all ten PCs. Four participant's dissimilarity judgements are best correlated when using the L°° metric, and one participant each for the other two norms. Table 4.7 summarizes the strongest correlation found, the metric used, and the number of PCs used. For comparison purposes, the right most column lists the strongest correlation between the participants' dissim-ilarity judgements and the parameters is also provided — these are the values computed in §4.6.1 — as we can see, five of the participant have stronger correlations using distance between Z-scores of the components than along any single parameter. Table 4.7: Using the indicated distance metric, number of components (1 - k), the correlation between distances in the mechanical motion space and dissimilarity judgements is listed in the fourth column. The fifth column lists the strongest correlation computed between each participant's dissimilar judgements and differences in the parameters. A l l of the correlations are significant atp < 0.0034 (rp > 0.4, t(50)=3.09). Not surprisingly, each participant has a different "ideal subset" of the principal components of the gaits when forming a judgement of dissimilarity. #6 and #7 have the most agreements in the parameters they use than any other participants. #2 seems to rely more strongly on a single parameter (bounc ines s ) than linear combinations of parameters. The first dimension of the mechanical motion space, PC\(Pw), seems to be a perceptual dimension of #5. #1 and #8 appears to utilize two dimensions of their psychological spaces. P# LP k §4.6.1 rp 1 L°° 2 2 L°° 10 5 IP 1 6 L°° 10 7 L°° 10 8 L2 2 0.572 0.405 0.725 0.611 0.656 0.636 0.405 0.510 0.656 0.456 0.560 0.538 On the other hand the different distance metrics proposed by these correlations are somewhat suprising. While all but #8 could be said to use the dominance norm, her strong correlation indicates that the Euclidean norm definitely fits her dissimilarity judgements the best. Also, the selection of subsets of components is quite disjoint, either one, two or "all" components having the strongest correlations. However, what this table does not show is that maximum correlations between the dissimilarities and the distance between Z-scores does not dramatically rise to these maximums. Instead they tend to vary at a value slightly below those listed in the fifth column of Table 4.7. 4.6.4 Summary There is evidence of linear relationships between the psychological and mechanical motion spaces. We found strong significant correlations between the dissimilarity judgements and the "distances" between the motion parameters of the gaits. These correlations indicate that given the motion parameters of two gaits we would be able to approximately compute the dissimilarity of the gaits using simple distance norms. 4.7 Relationship Between the Mechanical and Linguistic Motion Spaces We tested the relationship between the mechanical and linguistic motion spaces by comparing ratings of the motions and • Walker ' s parameters • Z-scores of Walke r ' s parameters Instead of correlation, we used regression so that we determine not only the strength of the relationship between the rating scales and the parameters but also the magnitude of direction of the relationship — that is the slope between the parameters (independent) and the ratings (dependent). 4.7.1 Regression Between Parameters and Ratings We performed a regression of the gait parameters against each participants' ratings of the gaits to determine the relationship between the rating scales and the parameters. This is potentially a huge analysis: for each participant, parameter, and scale we regressed between the gait parameters and the ratings made on the descriptive scales using the average of the trials of the three blocks. This required 1,056 least squares linear regressions (six participants times twenty-two parameters times eight rating scales). We then interpreted the strength of each correlation and it's direction. We could not use multiple linear regression for this analysis — each participant made three ratings of the twenty-six gaits, resulting in only twenty-six (averaged) observations on each of the rating scale. We have twenty-two parameters, thus we have almost as many independent variables as observations. This artificially inflates the proportion of variance explained, R 2 . Also, as we saw in §4.2, most of the variation in the gait parameters can be represented by ten principal components. In performing the regression, we normalized both the ratings (per scale, per participant) and the parameters. While the normalization has no effect on the significance of any of the regression results, it did simplify our interpretation of the slopes of the regressions, 6pw ( f c ) ] i , by standardizing the "units" of the parameters and the rating scales. We then computed a least squares regression between each parameter and each rating scale: P-Rs( = )&7Vw,l7V(fc) + &7Y(t),0 Where a "." indicates that all of the stimuli are include in the vector and we used the following values: pRs() Vector of averaged ratings of gaits on scale s by participant P. Normalized for regression to have zero mean and unit standard deviation along each rating scale s. Vw The vector of values for parameter k used by W a l k e r to create all the gaits. Normalized for regression to have zero mean and unit standard deviation along each parameter k. b-pv w > i Magnitude and direction (slope) of relationship between Walke r ' s parameter 7>W ^) and participant F 's average ratings on descriptive scale s. Since we normalize pRs(i) and Vw.(k) brw{k),\ = rp{ P i î s ( î ) , P w b-p ... o The y-axis intercept. W. (K) 1 As an example of how the resulting regressions are interpreted, we use participant #6. The remaining participant's regressions between pRs(i) and Vw ^) a r e presented in Appendix A. 1.1. For participant #6, each rating scale is significantly correlated with a number of gait parameters. Significance was tested by checking that the 95% confidence intervals for the value of the regression coefficients, bpw (k)ii, do not include zero. In other words, we accept the 5% probability, per regression, that any coefficient, 6-pw i, that does not appear to be zero could equal zero. We limited ourselves to the three significant parameters, per rating scale, with the largest proportion of variance explained, R 2 . The resulting parameters are listed in Table 4.8 according to the rating scale they are correlated with. By interpreting the direction and strength of the slopes of the regressions, we found that #6's perception of "speed" of a gait was affected by the maximum angle, and thus speed of rotation, of elbow, knee, and hip rotations. His perception of "flexibility" was affected by shoulder and back rotation, as well as elbow, and arm swinging. "Bounciness" of a gait was controlled by the legs, especially the flexion of the knees at heel strike and as the leg takes the body weight — not tilting and swaying of the torso. "Young" gaits had lots of arm, elbow and shoulder swinging. "Energetic" gaits required elbow, arm and hip swinging. "Light" gaits had very little torso tilting, the knees swing up a bit more. "Graceful" gaits had very little torso sway or bounce on heel strike. "Strange" gaits had lots of torso sway, turned out feet, and bounce. Also, joint angle velocities, not mean joint angles, affected the descriptions. These interpretations gave us a good idea of how #6 interpreted the rating scales and perceived the variation in the walking motions due to the gait parameters. In future experiments we could investigate the relationship between joint angle velocities and joint angle ranges. For example, Paterson, Pollick and Sanford (2000) have investigated the role of movement speed and emotional affect and found that the major effect of an observed movement's speed is the modulation of intensity of judgements of emotional affect. Picking the parameter with the highest R 2 , results in picking the parameter that minimizes the root mean square error (between pRs(i) and bpw {k),iPvs (&))• We could either accept the parameter with the largest R 2 for each scale, or we could have attempted to add additional parameters to the regression model using the next highest R 2 and so on. Unfortunately, we could not use any simple rules to add additional parameters as the significant parameters could have been as correlated with each other as they are with the rating scale. Adding the parameter with the next highest R 2 may or may not add a correlated parameter — thus artificially increasing R 2 . One alternative was use to stepwise multiple regression taking into account the Scale Parameter R 2 and bv A Parameter R 2 a " d b-Pw W , l Parameter R 2 3 n d W , l fast—slow elbow.ro t-inax 0.486 -0.697 knee.swing2 0.306 -0.553 hip_swing3 0.264 -0.514 flexible—stiff percent _shoulder.ro t 0.430 -0.656 e 1 bow.r o t _max 0.331 -0.575 arm.swing.f actor 0.322 -0.568 smooth—bouncy young—old hip.swing3 0.233 0.482 elbow.rot_max 0.255 -0.505 bounciness 0.222 0.471 arm.swing. factor 0.246 -0.496 knee-Impact 0.218 0.467 percent -shoulder.ro t 0.236 -0.486 energetic—tired elbow_rot_max 0.448 -0.670 arm_swing_f actor 0.423 -0.650 hip_swing3 0.333 -0.577 light—heavy graceful—spastic normal—strange t o r s o . t i l t 0.443 0.665 torso.sway_max 0.223 0.473 torso .sway .max 0.240 0.490 knee_swing2 0.151 -0.389 f oot.angle 0.183 0.427 foot.angle 0.200 0.448 foot-angle 0.127 0.357 bounciness 0.165 0.406 bounciness 0.194 0.440 Table 4.8: For each rating scale, the three parameters with the largest proportion of rating variance ex-plained, R 2 , for participant #6's ratings. Also listed is the slope of regression, 6-pw ( J f e ) i i . Values of frpw i > 0 indicate that increasing the parameter moves #6's rating towards the right hand label on the rating scale, b-pw < 0 indicate that increasing the parameter moves #6's rating towards the left hand label. correlations between the parameters. Unfortunately M A T L A B only has a manually controlled stepwise multiple regression solver so we instead used the Z-scores of the parameters as they are pairwise orthogonal. 4.7.2 Regression Between Z-Scores of Parameters and Ratings By regressing the Z-scores of the parameters, Z(VW), against the ratings, pRs(i), we used uncorrected dimensions of the gait parameters and thus, as we wished, used multiple dimensions to "explain" the effect of the PC(VV) on the rating scales. Before computing the regressions we normalized the Z-scores, Zk{V^), and the average ratings, pRs(i), to have zero mean and standard deviation of one. Again, the normalization does not change the significance of any of the regression results, but it helped to interpret the slopes of the regressions, bz^Vw)tl. Because PCA removed the correlations between the gait parameters, we expected many fewer significant regression coefficients between the parameters, as the independent variable, and the ratings, as the dependent variable. In fact that is what we find: only one or two of the components of the "mechanical motion space" have a significant correlation against each rating scale. Again, significance of a regression coefficient is tested using the 95% confidence intervals, and the R2 values indicate the proportion of shared variance between the Z-scores, Zk(Vw), and the ratings, PRs(i). Each bzk(j>v),\ is the slope of the regression line, which helped to interpret the direction and strength of the interaction between a change along a component and the resulting change in the rating scale. For participant #6 we have the significant regressions listed in Table 4.9. For #6 the second component is the fast-slow component, and the first component the young-old dimension. The fifth component is #6's dimension of normal/graceful-strange/spastic, while the fourth component with its negative slope is the bouncy-smooth dimension. Component seven is the heavy-light dimension, again a negative slope. Since the principal components are uncorrected we could have added components to attempt to increase the correlation between the PCs of the gait parameters and #6's ratings, but we did not. Tables summarizing the relationship between the participants' ratings and the the Z-scores can be found in Appendix A. 1.2. Each participant has a different set of correlations reflecting their individual biases, interpretations of the rating scale labels, and observations of the gaits. Regardless, we concluded that participants were able to describe the gaits using their perceptions of the gait parameters because the correlations are moderately strong which is unlikely if they were guessing or unable to form consistent descriptions which correlated with changes in the parameters. For each scale: Component k of Zk (7-w) Rating Scale R2 b Z k ( V v ) i l fast—slow k = 2 0.451 0.672 flexible—stiff 1 5 0.470 0.686 0.187 0.433 smooth—bouncy 4 0.152 -0.390 young—old 1 0.481 0.693 energetic—tired 2 1 0.316 0.562 0.208 0.456 light—heavy 7 0.163 -0.404 graceful—spastic 5 0.205 0.453 normal—strange 5 0.172 0.414 Table 4.9: Significant regression coefficients between #6's ratings, pRs(i), and principal components of the parameters, Zk(Pw). For each rating scale one or two principal components has significant regression coefficients 4.7.3 Summary There is evidence of linear relationships between the mechanical and linguistic motion spaces. We found strong significant correlations between the ratings of the gaits and their motion parameters. The regression coefficients would allow us to compute a description of a gait given its motion parameters. 4.8 Relationship Between the Psychological and Linguistic Motion Spaces We have already demonstrated a causal relationship between the gait parameters and the dissimilarity judge-ments, and a causal relationship between the gait parameters and the descriptive ratings of the motions. What we have not yet examined are the relationships between the dissimilarities and the descriptions. While the first two relationships allow us to work towards building computer applications that compute dissimilarities or descriptions of motions from their parameters, the last relationship allows us to "complete the triangle" and build applications that find motions with similar descriptions. We tested the relationship between the psychological and linguistic motion spaces by comparing the dissim-ilarity judgements of pairs of gaits and • differences of ratings along each rating scale • differences of Z-scores of ratings • distance of Z-scores of ratings 4.8.1 Dissimilarities versus Differences of Ratings Along Scales We needed to computer the following values to compare the dissimilarities against the differences of the ratings: 'pô(i, j) Averaged dissimilarity rating by participant P between gait i and j where i was presented first. Without the subscripts % and j pô(.,.) is a vector of averaged dissimilarities for all trial pairs (averaged across experiment blocks). pRs{i) Averaged rating of gait % on scale s by participant P. pAs(i, j) Absolute difference between averaged ratings of gaits i and j on scale s by participant P: pAs(i,j) = I pRs{i) - pRs(j)\. Without the subscript s then PA,(i,j) is the vector of absolute differences between pRs{i) and pRs(j) across all scales. Without the subscripts i and j pAs(.,.) is the vector of absolute differences on scale s for all trial pairs T(i,j). rp{P5{.,.), P  S ( , . ) ) Correlation between averaged dissimilarities and absolute differences on rating scale s for participant P. As expected, each participant has a different pattern of correlations between their dissimilarity judgements, p5(i,j), and differences along the rating scales, rp(PS(i,j), p A s ( i , j ) ) . Except for participant'#8, the participants have correlations greater than 0.55 along at least one rating scale. #8's highest correlation is 0.42. This supports our hypotheses that the rating scales can be used to distinguish between the gaits. To summarize, there are strong correlations between dissimilarities and differences between ratings on the smooth-bouncy; young-old, and energetic-tired scales. The two strongest correlations for each participant are listed in Table 4.10 In Figure 4.6 we present the pattern of rp( P6(.,.), pA s ( . , . ) ) from all participants and all rating scales using a combination of box plots and a gray-level plot. Two Strongest Correlations Scale (rp(p5{i,j), PAs{i,j))) Scale (rp( PS(i,j), PAs(i,j))) 1 fast—slow (0.55) smooth—bouncy (0.58) 2 young—old (0.55) smooth—bouncy (0.44) 5 young—old (0.61) flexible—stiff (0.52) 6 young—old (0.61) energetic—tired (0.55) 7 young—old (0.68) energetic—tired (0.68) 8 energetic—tired (0.42) light—heavy (0.34) Table 4.10: Strongest correlations between dissimilarity judgements and differences of ratings for each participant. I o O 1 2 3 4 5 6 7 Rating Scale Number Figure 4.6: On the top right: Box plot of the correlations between participant's judgements of dissimilarity and the participants' ratings of each gait in the trial pairs according to rating scale. Vertical axis is cor-relation, horizontal axis is rating scale number. On the bottom left: Box plot of the correlations between the participants' judgements of dissimilarity and the participants' ratings of each gait in the trial pairs ac-cording to participant. Horizontal axis is correlation, vertical axis is participant number. On the bottom right: Gray-level plot of the correlations between the participants' judgements of dissimilarity and the par-ticipants' ratings of each gait in the trial pairs. Vertical axis is participant number, horizontal axis is rating scale number. Light values indicate weak correlations and dark values strong correlations. 4.8.2 Dissimilarities versus Differences of Z-Scores of Ratings The correlations between the dissimilarity judgements and differences of the Z-scores of the ratings tells us which of the principal components of the ratings correspond with perceptually salient features of the gaits used to form dissimilarity judgements. When we computed the principal components of the participant's ratings in Section 4.3 we found that most participants ratings could be represented with the first three to four principal components. The variance of the later components dropped off very quickly implying that they likely represented noise in the ratings. By computing the correlation between the dissimilarity judgements and the differences of the Z-scores of the ratings we can confirm these conclusions. We compute: Z{ pRs) The ratings of participant P transformed into the orthonormal linguistic motion space such that Z\(pRs) corresponds to PC\( pRs)-Z( pRs(i)) Average of Z-scores for gait i across the rating trial blocks. \Zk(PRs(i)) - Zk(PRs{j))\ Difference between averaged kth Z-score of gaits i and j. rP{ pS(i,j), \Zk(PRs(i)) - Zk(PRs{j))\) Correlation between averaged dissimilarities and difference of kth Z-score of gaits. If the correlations along the first PC( pRs) — those with large variances — were strong while the corre-lations along the later PC( pRs) were weak then this would indicate that the linguistic motion space, at least in the sense that it can be used to describe similarities between motions, is very accurately described by using only the significant principal components of the ratings. In Table 4.11 we have listed the three strongest positive correlations between the dissimilarities and the Z-scores of the ratings for each participant. As you can see, the pattern tends to be a very strong corre-lation along the first two components, PC\( pRs) and PCïi pRs), with most participants having quickly decreasing correlations with later components. This indicates that the first principal components of the rat-ings capture variation of the descriptions of the gaits that corresponds with the perceptually salient features of the gaits used to form similarity judgements. When we computed the principal components of the ratings, we used as our stopping rule 80% of the variance of normalized ratings. This stopping rule tends to also only accept PC(pRs) with variances First Second Third P# PCk(pRs) PCk(pRs) PCk(pRs) #1 1 #2 1 #5 1 #6 1 #7 1 #8 2 0.6341 0.6162 0.4653 0.6144 0.7381 0.3157 2 2 4 3 2 6 0.2495 0.0055 0.4604 0.3627 0.2466 0.3132 8 6 5 6 3 3 0.1275 0.0210 0.2958 0.2132 0.1604 0.1686 Table 4.11: The three strongest positive correlations between participant's dissimilarity judgements and dif-ferences of Z-scores of ratings. The positive correlations indicate that components with the strong variances (lowest k) correspond to dimensions of the psychological motion space. greater than 1.0. As we can see from Table 4.11 these rules tend to produce components that can be used to describe the gaits corresponding with the features of the gaits used to form similarity judgements. We can also use the correlations presented Table 4.11 to determine if the variance of the ratings captured by a principal component is due to noise, hysteresis, or another process that weakens the relationship between the linguistic and psychological motion spaces. For example, if a rating scale was inconsistently used by a participant, either because they could not decide what its labels meant or because they kept changing their mind as to which features of the gaits correspond to the scale, then the ratings on that scale would tend vary independently of the other scales. A principal component would lay along this independent variation and would have a variance near one. Based on its variance, this "noise PC" would then be among the PCs selected to describe the linguistic motion space of the participant. These principal components are those with variances near one and coefficients dominated by a single rating scale. However, to identify components that are due to correlated "noise" along two or more rating scales we would have to identify components with that have very low correlations with the dissimilarity judgments, yet high enough variances to be selected as dominant PCs. Thus we are looking for principal components of the ratings that are strong in their variance and but have a weak correlation with the dissimilarity judgements. When we use the arbitrary thresholds of rp( pS(i,j), \ Zk{ pRs 0.3 and A f c( PRS) > 0.2 * 8 (i.e., A f e( PRS) > 1.6), we find that #8's first PC and the second PCs of #1, #2, #7 are such principal components. Thus, these components represent dimensions of their linguistic motion spaces that do not seem to correspond to their psychological motion spaces. Of course, if the rela-tionship between the psychological and linguistic motion spaces are non-linear then this simple test will not correctly identify PCs due to correlated noise. As only the PCs of a few of the participants pass this test, a non-linear relationship between the linguistic and psychological motion spaces should be considered in future experiments. 4.8.3 Dissimilarities versus Distances Between Z-Scores of Ratings As we saw in the last section the variation along the principal components of the rating scales correlates with dissimilarity judgements. What are the correlations between the dissimilarity judgements and the distances between the Z-scores of the ratings? We computed: Z{ pRs{i)) The ratings of participant P transformed into the orthonormal linguistic motion space such that Z\(pRs(i)) corresponds to PC\( pRs(i)). Z( pRs(i)) Average of Z-scores for gait i across the rating trial blocks. v Z{l_k)(pRs(i)) - Z{l_k)(pRs(3)) Distance between averaged Z-scores of gait i and j computed using distance metric LP and the first k dimensions. rp{p5{i,j), Z^_k)(pRs{i)) - Z{l_k)(pRs{j)) Correlation between averaged dissimilarities and distances between ratings in linguistic motion space. We then computed the correlations for each of the participants, for each of the k = {1, 2 , . . . , 8} compo-nents, and each distance norms (p = {l,2,oo}). Table 4.12 are compared the strongest correlation from the last section — rp(p8(i,j), pA.(i,j)), listed in Table 4.10 — and the strongest correlation between p5(i, j) and 1  Z\ — Z\ | | p . For most of the participants, the correlations between p5(i, j) and the principal components are stronger. However, #5 and #8 have lower correlations. This indicates that, for #5 and #8, the original ratings scales are better at describing differences between the motions than linear combinations of the scales. Because #5 and #8 have correlations between their rating scales on the same order of magnitude as the other participants, this again reflects individual differences. As we varied the subset of components — from one to eight — used to compute the distances between descriptions of the motions in the linguistic motion space, and try different distance metrics, we again found that participant's dissimilarities either correlate strongest with the first two components, or with all of the rp from §4.8.1, Table 4.10 P# max(rp(p6(i,j), F A , ( i , j ) ) ) rp( P5{i, j), \\ZX - Zx\f) 1 0.58 0.63 2 0.55 0.62 5 0.61 0.47 6 0.61 0.61 7 0.68 0.74 8 0.42 0.32 Table 4.12: Comparison of the strongest correlation between dissimilarity judgements and differences of ratings for each participant against the strongest correlation between dissimilarity judgements and distance between ratings using Z-scores of ratings (typically along the first PC as listed in Table 4.11). components. Table 4.13 lists the distance metric and number of components that produces the strongest correlation between dissimilarity judgements and distances in the linguistic motion space. Although, for k = 1 all of the distance norms compute the same distances, it does appear that L 1 is the best norm for combining components of the linguistic space into distances that approximate dissimilarities. P# LP k rp 1 J? 13 0.6717 2 L p 1 0.6162 5 L 1 6 0.5867 6 W 1 0.6144 7 L 1 2 0.7873 8 L l 8 0.3311 Table 4.13: For each participant the distance metric and number of components that produce the strongest correlations between the dissimilarity ratings and distances in the linguistic motion space. 4.8.4 Summary There is evidence of linear relationships between the psychological and linguistic motion spaces. We found moderately strong significant correlations between the dissimilarity judgements and the "distances" between ratings. These correlations indicate that given the ability to compute "descriptions" of two gaits we would be able to approximately compute the dissimilarity of the gaits using simple distance norms. 4.9 Conclusions The goal of this experiment was to test the experiment design of having participants judge the dissimilarity of pairs of walking motions and describe the walking motions. We have verified that participants are capable of reporting consistent dissimilarity judgements and consistent descriptions. The strong positive correlations between the gait parameters and the participants' dissimilarity judgements demonstrate that participants were able to compare motions using the experiment design. These correlations also demonstrate the existence of linear relationships between the psychological and mechanical motion spaces. Assuming that dissimilarity judgements approximate distances between motions in the participant's psychological motion space, then distances in the psychological motion space correlate with the various dimensions (and principal components) of the mechanical motion space. We also found strong positive correlations between the gait parameters and the participants' descriptions of the gaits demonstrate that participants were able to describe motions using the experiment design. Some participants' descriptions correlate more strongly with gait parameters than linear combinations of the gait parameters formed using PCA. Using the correlations between the participant's ratings and the principal components of the parameters we could create personalized interfaces to W a l k e r . For example, #6's interface would have a "fast-slow" control that maps to 0.672PCL2(7;w)> and a "smooth-bouncy" control that maps to —0.390PC4(7J'W). (These values are taken from Table 4.9.) The linguistic motion spaces were approximated by four orthogonal dimensions. By comparing the mag-nitudes and signs of the principal components of their ratings we found two dimensions common to most of the participants: "young fast bouncy" versus "old slow and smooth" "graceful normal flexible" versus "spastic strange stiff." In contrast to the linguistic motion space, the mechanical motion space requires ten principal components. The difference in the number of dimensions between these two space leads to several interesting questions about how we form descriptions of movements. Do we we "compress" variation in human movements into a small number of descriptive dimensions? Does the small number of linguistic dimensions explain why we have difficulty describing movements using words? Is this difficulty cause not by a lack of vocabulary, but that we lack the ability to construct and maintain descriptions of movements using more than a small number of dimensions? The relationship between the psychological and linguistic motion space was investigated using correlations between dissimilarity judgements and difference in the ratings. While each of the participants have a differ-ent set of correlations, most participants agree that the smooth—bouncy, young—old and energetic—tired scales can be used to distinguish between the walking movements. We could use this knowledge to construct a new user interface to Walker that used controls with these labels to specify the general parameters of a gait. We map settings of these controls to Walker ' s parameters using the regression coefficients between the rating scales and Walker ' s parameters. While the conclusions drawn from this experiment are tentative, they allow us to use the experimental methodology to further investigate the relationships between the motion spaces. For example, how easy is it to "train" participants to use particular features of motions to form judge dissimilarity judgements? How much training is required and are these judgements stable over long periods? Although some of the participants could be described as expert "movement judges" — at least in the area of social dancing — we did not attempt to compare the responses of experts and novices. Do structural differences exist between the psychological motion spaces of novices and experts? Through our analysis of Experiment One we have tested for correlations between the three motion spaces. Because we have found strong positive correlations between the spaces it makes sense to create higher-level character animation tools that use models of human similarity judgement and formation of descriptions of motion. In the next chapter we discuss Experiment Two which focuses on the issue of treating dissimilarity judgements as distances in a metric space. If we find that dissimilarity judgements can be treated as distances in a metric space, then the creation of higher-level animation tools will be a relatively easy process. If we find that dissimilarity judgements can not be treated as distances in a metric space, then we will need to use stronger techniques to transform differences between the parameters of motions into dissimilarity judgements. Chapter 5 Experiment Two: Metric Properties of Motion Dissimilarity Judgements Experiment Two was a participant experiment to investigate the properties of the similarity judgments. If the judged dissimilarity of two motions can be treated as a distance in a metric space — such as our physical Euclidean space — then we will be able to use very simple models to transform parameters and descriptions of motions into distances that approximate human dissimilarity judgements. If not, we will have to use stronger, more complex models than simple distances between the parameters of the motions. 5.1 Overview In the last chapter we concluded that, for a small group of participants, made up mostly of social dancers, particular parameters of human gaits affect their similarity judgements and descriptions of the gaits. We concluded that participants descriptions of the gaits tended to require about four principal components to capture most of the variation in the ratings and that there are common directions of variation in the par-ticipants descriptions. We also found correlations between the descriptions of the gaits and the similarity judgements. In this chapter we present our analysis of a second experiment which focused on the properties of the simi-larity judgements — specifically their metric properties which would allow us to treat motion dissimilarity judgements as distances between the motions in a metric space. As we focused on the psychological motion space, which we expected to be much more uniform across humans, we expanded our pool of participants in both quantity and range of backgrounds. 5.2 Differences Between Experiments One and Two The stimuli and trials of Experiment One were selected to explore the relationships between the motion spaces while Experiment Two was more focused on the structure of the psychological motion space. Differ-ences between the experiments included recruitment of participants, number and creation of stimuli, number and structure of trials, and slight changes in the experiment control software. Experiment One utilized a wide range of gaits and a small number of participants to determine if the rela-tionships between the psychological, linguistic, and mechanical motion spaces are linear. In our analysis of Experiment One we also demonstrated the validity of the experiment design for collecting motion dissimi-larity judgements and descriptions of motions. In contrast, Experiment Two focused on a much smaller motion parameter space to determine if it will be possible to build computational measures of the dissimilarity of motions. We also recruited a larger number and range of participants in an attempt to sample from the general population of individuals. The primary focus of Experiment Two was to investigate the metric properties of motion dissimilarity judge-ments: HQ™ Non-degeneracy: only the self-distance is zero, and distances are never negative: pô(i,j)> pô(i,i) — 0. HQ™ Symmetry: distances between points are symmetric: pS(i,j) = pô(j,i). HQ171 Triangular Inequality: sum of lengths of two sides of a triangle is never less than the length of the third side: pS(i,j) + pS(j,k) > p5(i,k). If dissimilarity judgements have these properties then we can treat dissimilarity judgements more as actual distances between motions and less as "judgements" — that is we can treat dissimilarities as approximating distances. If dissimilarity judgements do not have these properties then we will be forced to model the psychological motion space as a non-vector space. And reconsider its usefulness in disambiguating the mapping between the linguistic and mechanical motion spaces. 5.2.1 Participants As Experiment Two focused on the structure of the psychological motion space the recruitment of partici-pants was expanded to include members of the general population, i.e., non-dancers. Specifically, a total of thirty participants were recruited and classified based on experience level as runners, dancers and non-dancer non-runners. Participants were also classified based on gender (male, female). Participant demographics for the Experiment Two are listed in Appendix F. 5.2.2 Stimuli To test the metric properties of the motion space it was necessary in Experiment Two to have trials that combined all possible stimuli pairs. As the number of unique motions, n, increases, the number of unique pairs increases quadratically (n 2). For Experiment Two, two configurations of gaits were used to interpolate gaits and pair them into trials. Figure 5.1 illustrates the two configurations, the first configuration is a triangle defined by three "primary" gaits, indicated with the filled circles. The gait parameters of the primary gaits were interpolated to create the interpolated gait in the center of the triangle. The second configuration is defined by two primary gaits on the ends of a line and the gait parameters of the primary gaits were interpolated to create three interpolated gaits along the line. The use of interpolation of gait parameters also allowed us to test the effect of interpolation in the mechanical motion space on proximities in the psychological motion space. These two small configurations of stimuli were selected to minimize the experiment session duration. If 9 cr o o o o o Figure 5.1: Two configurations of comparisons for primary gaits (filled circles) and interpolated gaits (un-filled circles). On the left, a triangle of three primary gaits with interpolated gait in the center requires sixteen comparisons (arcs) to test the metric properties. On the right, two primary gaits are interpolated to create three new gaits, requiring ten comparisons to test metric properties and parameterization. nine stimuli had been arranged in single configuration we would have eighty-one possible trial pairs. The arrangement of the four triangular stimuli into sixteen trial pairs and five linear stimuli into twenty-five trial pairs results in a total of only forty-one trial pairs, which allowed us to keep the participant portion of the experiment duration under two hours. 5.2.3 "Triangular" Gaits The four gaits assembled into the triangular configuration will be referred to as the "triangular gaits." These gaits were paired to define sixteen trial pairs. The primary triangular gaits are: 1: Lower Crouch knees bent slightly, arms straight down but swinging up on each step, upright torso, very little body bounce up and down. 2: Upper Body normal walk with arms swinging up and elbow flexing on each step, torso rotating with arms towards rear foot on each step. 3: Upper Tipping normal walk with torso dipping forward violently on each heel strike The gait parameters of the above gaits were interpolated to create the centre gait: 4: Average slight bend in knees, torso dipping forward on each heel strike, arms swinging and elbows flexing. 5.2.4 "Linear" Gaits The five gaits assembled into the linear configuration will be referred to as the "linear gaits." These gaits were paired to define twenty-five trial pairs. The linear gaits are: 5: Stiff Upright stiff legs and arms; torso stiff, straight and leaned backwards. 6: 75% Stiff Upright/25% Super Crouch Twisting "normal" walk. 7: 50% Stiff Upright/50% Super Crouch Twisting much like the normal walk except this motion is more fluid and more bouncy. 8: 25% Stiff Upright/75% Super Crouch Twisting knees bent, and torso tipped over, arms swinging and elbows flexing, torso rotating to face rear foot on each step. 9: Super Crouch Twisting more bent over than 25% Stiff Upright/75% Super Crouch Twisting, bouncing and twisting is almost violent. Animation frames of the gaits can be found in Appendix B.2. 5.2.5 Skipping Trials In Experiment One skipping a trial by clicking on the "skip" button excluded it from analysis. In Experiment Two, skipping a trial caused it to be presented again at the end of the block. In Experiment Two, after all of the trials were presented once, participants were told how many trials they skipped and then the skipped trials were presented again in random order. In general, upon a second attempt of the skipped trials, participants did not skip the trials again.1 We felt that this change struck a reasonable balance between keeping participants from guessing because they could not form a judgement — because they were distracted or bored — and strategic use of the "skip" button in trials where the participant was not sure if they had seen the same motion twice. Analysis of the skipped motion comparison trials revealed that overall, only 3.5% of the trials were skipped, however per participant this ranges from none to a high of thirty skips (out of 164 trials), with participants 'Specifically, only #6 and #30 skipped a single trial twice. skipping on average less than six trials. On average each trial was skipped four times across all participants, with the linear stimuli slightly more likely to be skipped. The motion comparison trials that were skipped the most often were: T((,3),4) Nine times T ( (, 4), 1 ) Fourteen times T((,7),6) Ten times T((,7),8) Nine times As we will see in Section 5.5.1.2 these trials in general were difficult to judge implying that the "skip" button may be used a bit too strategically by some participants. A l l other motion comparison trials were skipped six or fewer times across all participants. Less than 2% of the motion rating trials were skipped. The majority of participants skipped no or only one trial. 5.2.6 Motion Comparison Trials In Experiment One, the marker indicating the current judgement on the Similar-Dissimilar scale appeared at the left "Similar" end of the scale after presentation of the second motion — which may have biased participant's judgements. This behaviour was modified in Experiment Two by having the marker remain invisible until the participant clicked on the Similar-Dissimilar scale. Additionally vertical "notches" or "marks" were added to the scale to aid the participants in placing the marker at a consistent position. The addition of these marks did not constrain the placement of the marker - they only visually guided the participants in their placement of the marker. In Experiment One, the motion comparison trials were combined into four macro-blocks of fifty-two trials. Each macro-block is composed of two blocks of twenty-six trials to allow participants to take a break every ten to fifteen minutes. The order of trials within a block was randomized across blocks but was the same order for all participants. In Experiment Two, the motion comparison trials are combined into four blocks of forty-one trials. The order of trials within a block was randomized across blocks and was a different order for each participant. 5.2.7 Motion Rating Trials In Experiment One, the markers indicating the current description of the gait on the descriptive scales, appeared at the left hand edge after the presentation of the motion for each trial. This behaviour was modified in Experiment Two by having the markers remain invisible until the participant clicked on a the rating scale. In Experiment One, a participant could leave the markers at the left-hand edge and the system would record that rating. In Experiment Two, the "Finish Trial" button would not work unless all of the scales had been used and "notches" were added to the scales to aid in the recording of consistent ratings. In Experiment One, the motion rating trials were combined into four blocks of twenty-six trials. Each trial consisted of a single motion, with all motions presented in random order across the blocks. However, the trial orders were the same for all participants. In Experiment Two, the motion rating trials were combined into four blocks of nine trials and trials were presented in random order in each of four blocks, with a different random order for each participant. In Experiment One, the rating scales were labeled with the following pairs of adverbs and adjectives: fast-slow, flexible-stiff, smooth-bouncy, young-old, energetic-tired, light-heavy, graceful-spastic, and normal-strange. Analysis of the ratings from Experiment One revealed that several of the rating scales were either under-used: they had either the smallest variance and smallest max-min difference by the largest number of subjects; or redundant: highly correlated with other scales, for example young-old and energetic-tired, and graceful-spastic and normal-strange. Based on this analysis, the labels from Experiment one that were not reused were: fast-slow, young-old, and graceful-spastic. To offer a balance between stylistic scales and specific body part motions, in Experiment Two, the scales were labeled with the following pairs of adverbs and adjectives: flexible-stiff, smooth-bouncy, straight-crouching, energetic-tired, still-swinging, light-heavy, upright-tipping, and normal-strange. Reasons for these changes are discussed in Chapter 5. 5.2.8 Reversed Direction of Walking In Experiment Two, for eleven of the thirty participants, the direction of the walking figure was reversed so that it walked across the screen right to left rather than left to right. This reversal was performed to test if the common western language reading direction (left to right) affected the judgements.2 If the perception and judgement of walking motions is a simple and basic task that all people are capable of, then there should be no differences in the judgements between "left-to-right" and the "right-to-left groups."3 5.2.9 Summary of Differences Between Experiments One and Two To summarize, the following changes were made to the experiment protocol and system for Experiment Two: • gaits created using interpolation (rather than manually selected gait parameters) • gaits paired in motion comparison trials according to network (rather than random pairings) • motion comparison trials involving comparison of the same gait to itself • randomization of trials within a block so that all participants perform the trials in a different order (rather than same order for all participants) • re-presentation of skipped trials at the end of a block (rather than exclusion from experiment) • automatic reminder to take a break between blocks • "notches" added to scales • scale markers not appearing until first mouse button click on the scale (rather than setting it to left end) • ensuring that responses are made on all scales used in rating trials (rather than recording a rating at the left end) • prompting for participant initials and password before starting experiment sessions • different labels on ratings scales • reversal of direction of walking across the screen for some participants • Questionnaire B added Changes that would apparent to a participants are illustrated in Figure 5.2. (Figures 3.5a-3.5c on pages 57-59- and Figures 3.6a-3.6b on pageg 61-62 show screen shots from Experiment One.) 2 It is becoming more difficult to find readers experienced in reading right to left from an early age. Chinese writing recently switched to left to right, which leaves only Arabic and Hebrew. 3Participants in the "right-to-left" group are: #3, #4, #6, #7, #8, #9, #12, #17, #19, #20, and #21. 5.3 Analysis of Participant Responses and Hypothesis In Experiment One we were interested in the relationships between the psychological, linguistic and me-chanical motion spaces. For Experiment Two, we are less interested in the relationships between the spaces: dissimilarity judgements are linearly related to the difference between gait parameters, • dissimilarity judgements are linearly related to the difference between gait ratings, and • gait ratings are linearly related to the gait parameters than the structure of the psychological motion space and it's relationship to the mechanical motion space. However, we did test the strength of the correlations between the spaces. To summarize, many participants had correlations stronger than 0.8, and all stronger than 0.4. We hypothesize that the smaller number of gaits, relatively larger number of comparisons, and use of interpolation of gait parameters allowed the participants responses to correlate more strongly. Also analysed were the average variance of dissimilarity judgements per participant and the distribution of dissimilarity judgements per participant to confirm that participants tended to reproduce the same judgement for each identical trial. Some participants were of course much more reliable than others and we shall discuss some of our conclusions later. The correlation of each participant's dissimilarity judgements between experiment blocks were computed to determine if participants dissimilarity judgements were somewhat stable throughout the experiment. The weakest case block-pair-wise correlation for any participant was rp = 0.45 (t(39)=3.15, p=0.0014). How-ever, twenty-three of the participants consistently correlated with their own judgements at rp > 0.7 across all blocks. This indicates that variance in dissimilarity judgements is most likely due to participant inaccuracy or hysteresis rather than changing strategies or learning effects. In order to reduce the length of this chapter we will not be presenting our analysis of the motion ratings or the principal components analysis of the gait parameters. Both the motion descriptions and the gait parameters require two or three principal components to approximate 95% of the variation. Many of the motion rating scale pairs are strongly correlated — reflecting the interpolated gait parameters. (a) Motion Comparison Trial Tutorial ill: f l e x i b l e s t i f f s m o o t h b o u n c y s t r a i g h t c r o u c h i n g e n e r g e t i c t i r e d WÊÊBÊÊËÊÊÊÊÊÊÊÊÊÊÊÊÊÈÊÊÊÊÊÊÊÊÊÊ s t i l l s w i n g i n g l i g h t h e a v y ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ H 1 1 1 1 1 1 i 1 u p r i g h t t i p p i n g m i i i i i i i 1 n o r m a l s t r a n g e jaw—il 1 F i n i s h T r i a l 1 o f 3 H (b) Motion Rating Trial Figure 5.2: Screen shots from Experiment Two illustrating changes made to the experiment control soft-ware that would be apparent to a participant, (a) motion comparison trial: "notches" added to the Similar-Dissimilar scale, (b) a motion rating trial, "notches" added to the descriptive rating scales and changes in the labels on the descriptive rating scales. In this experiment we are investigating if the perceptual motion space has metric properties. If the dissimilar-ity judgements do not have metric properties then it may be necessary to treat dissimilarities not as distances in some metric space, but to first transform them into distances with metric properties. Additionally, we may be forced to search for non-linear models of the relationship between the motion parameters and the dissimilarity judgements as the simple computation of the distance between the parameters of two motions. And specifically, if the symmetric property fails us, we will have to use n2 comparison trials for n stimuli, rather than a lower value such as (n 2 — n)/2. While this factor of two may seem insignificant, it allows us to halve the duration of the experiments which helps to avoid participant fatigue. While we will dedicate the bulk of our presentation to the metric properties of dissimilarity judgements, we also used multidimensional scaling to transform the dissimilarity judgements into configurations of gaits in two and three dimensional Euclidean spaces. The configurations served not to verify any properties of the dissimilarity judgements but to generate a picture of the psychological motion space that may lead to further hypotheses. 5.4 Average Variance of Dissimilarity Judgements One way of determining how reliable a participant was at judging the dissimilarity of the gaits is to compute the average variance of the participants' dissimilarity judgements. Each participants judged the dissimilarity, pS(i,j), of gait i and j three times. We use the notation Var( pô{i,j)) to indicate variance of the judgement of dissimilarity of gaits i and j by participant P. We computed the average dissimilarity judgement variance using the formula Most of the participants have an average dissimilarity judgement variance less than 0.025, which corre-sponds to a standard deviation of 0.16 which is approximately the width of three intervals demarcated by the notches along the Similar-Dissimilar scale. However, there are five participants (#12, #3, #20, #28, and #27) whose average variance jumps above 0.025. To summarize, dissimilarity judgements are consistent. Var( pS) = EijVar(pafoj)) 41 We used the average dissimilarity judgement variance to compare the groups of participants. The reliability of the participants of each group may have been affected by their experience observing human motion (dancers, runners and "normals"), their gender (male, female) or the experimental variable of direction the walking figure walked {i.e., across the screen to the left or right). We use between-subjects ANOVA to test our null hypotheses that the groups mean average trial variances are equal: H0 : Var(p{D a n c e r s}<5) = Var( P{R u n n e r s}<5) = Var( p{Normais}<5) H0 : Var( P{M a l e s}(5) = Var( p{Femaies}<5) Ho : Var( pjLeftto Right}<5) — V A R ( P{Right to Left}<5) We summarize in Table 5.1 the results of the ANOVAs. The p-values indicate there are no significant differences between the participants based on experience, gender, or direction of walking — including interactions. The lack of significant difference indicates that participant groups have similar perceptions and similar decision forming strategies. Source SS df MS F P Experience 1.4103 x 10-" 2 7.0515 x 10" -5 0.9498 0.3993 Error 0.0020 27 7.4238 x 10" -5 Gender 1.4919 x l O " 6 1 1.4919 x 10" -6 0.0195 0.8900 Error 0.0021 28 7.6570 x 10" -5 Direction 1.5501 x 1 0 - 4 1 1.5501 x 10--4 2.1806 0.1509 Error 0.0020 28 7.1087 x 10" -5 Table 5.1: ANOVA summary table comparing the groups of participants according to mean average trial variances. The resulting p-values indicate there are no significant differences between the participants based on experience, gender, or direction of walking. The lack of significant difference indicates that participant groups have similar perceptions and similar decision forming strategies. 5.5 Metric Properties of Dissimilarity Judgements In Experiment One we assumed that judgements of the dissimilarity of two gaits approximate the "distance" between the two gaits in a psychological motion space. This is an assumption that is generally assumed to be true when analysing dissimilarity judgements (Davison 1983) since we know how to treat distances mathematically. However, for dissimilarity judgements to truly approximate distances in a metric space they must have several properties. These properties are:4 H0irn Non-degeneracy: only the self-distance is zero, and distances are never negative: pS(i,j) > pS(i,i) = 0. H2rn Symmetry: distances between points are symmetric: pô(i,j) = pô(j,i). jj3m Triangular Inequality: sum of lengths of two sides of a triangle is never less than the length of the third side: pô(i,j) + pô(j,k) > pô(i,k). The literature surrounding dissimilarity judgements and their metric properties appears to be divided into two camps. On one side are those who model deviations of dissimilarity judgements from the metric properties as "noise." On the other side are those who study non-linear or non-metric space psychological models of dissimilarity judgements and thus see the "noise" as evidence of non-linear — possibly chaotic — processes. Carroll and Arabie (1998) briefly presents the historical development of both groups and their models. The literature on dissimilarity judgement predicts we should not expect motion dissimilarity judgements to have the metric distance properties (Davison 1983). If the dissimilarity judgements fail to form a plausible metric space then we may be forced to treat the "non-metricity" as "experimental error" and transform the dissimilarities into metric distances. For example, in future experiments we could use multidimensional scaling — introduced later — to transform the dissimilarities into a configuration of the gaits in an Euclidean space and use the newly computed distances rather than the dissimilarities. 4 For a discussion of these properties see Young (1987, Chapter 4) for a discussion of metric and specialized spaces, especially as they pertain to multidimensional scaling, a technique used later in this thesis. If we treated the non-metricity of dissimilarity judgements as "noise" we would not be taking the position of the first group and reject the second. To some extent, we are not extremely concerned with the arguments of either group. For one reason, we are using stimuli that are not easily distinguishable — even when they are presented at the same time. And for another reason, the primary goal of this thesis is not to propose a psychological model of motion similarity judgements, but to examine the relationship between gait parameters and individual participant responses. 5.5.1 Testing the Non-Degeneracy Metric Property Starting with the first metric property, non-degeneracy, we tested pô(i,j) > pô(i,i) = 0 for all pairs of gaits. To test this property we used ANOVA to test if the the average of the dissimilarity judgements for each gait for each participant upholds the non-degeneracy property. Since ANOVA can only test one part of this property at a time, we tested p5{i,j) > pô(i,i) separately from pô(i,i) > 0. For each participant we tested the two hypotheses: Specifically, using M A T L A B , one-way analyses of variance (ANOVA) can be used to test within participants the hypotheses: H^a : P5(i,i) = PÔ(J,J) V i , j € {1,2,3,4,5,6,7,8,9} Hlb : P6(i,i) = P0(j,k) J,k e {1, 2, 3,4, 5, 6, 7, 8, 9}, j + k HlQa : pô(l,l) = P6(2,2) = P<J(3,3) = P<J(4,4) = Pr5(5 ;5) = P<J(6,6) = P<J(7,7) = pr5(8,8) = Pr5(9,9) and Hlb: P8{l,l) = pS(j,k) Vj,k G {1,2,3,4} or {5,6,7,< P6{2,2) = pS(3,k) Vj,k G {1,2,3,4} or {5,6,7, ( l,9},j^k pS(3,3) = pS(j,k) Vj,fc G {1,2,3,4} or {5,6,7,< \9},j^k P6(4A) = pS(j, k) Vj,k G {1,2,3,4} or {5,6,7,< pS(5,5) = pS(j,k) Vj,k G {1,2,3,4} or {5,6,7, p5(6,6) = pS{j, k) Vj,k G {1,2,3,4} or {5,6,7, 3,9},J ï k P5(7,7) = pS(j, k) Vj,fc G {1,2,3,4} or {5,6,7, 3, 9},j ^ k pS(8,8) = pS(3,k) Vj,fc G {1,2,3,4} or {5,6,7, i,9},j^k pS(9,9) = pS(3,k) Vj,k G {1,2,3,4} or {5,6,7, l,9},j^k We expect that H0la holds (is not rejected), and that HQ* is rejected in favor of pô(i, i) < pô(j, k) for all trial pairs for all participants. HQ0, requires one test per participant, while Hlb requires 288 tests per participant.5 The sheer number of tests, just for the first metric property, hints that ANOVA is not the correct tool. If we used a = 0.01 for each of these tests, we quickly exceed a type-I error rate of 100%, if we reduce a then our type-II error rate quickly climbs. Thus we can either be wrong or have non-significant results. There is another technique we used instead the above tests to compare average dissimilarity judgements. We instead examined the dissimilarity judgements for the trials involving judgement of self-similarity, T(i, i), for evidence that participants did not always report a judgement of "exactly the same", pS{i,j) < e. We also examined the dissimilarity judgements for the trials involving judgement of "non-self-similarity" - that is trials where i ^  j (T(i,j)) — for evidence that participants were unable to always report a judgement of "not exactly the same", p5{i, j) > e. We used a value of e equal to the distance between two notches on the Similar-Dissimilar scale. 5288 = 9 x (12 + 20), Hlb tests each of the 9 possibilities of p5(i, i) against each of the 12 combinations of pS(j, k) V j , k € {1,2,3,4}, j ^ k. and against the 20 combinations of pS(j,k) V j , k 6 {5, 6, 7, 8, 9}, j / k. 5.5.1.1 Judgement of Self-Similarity of Gaits Nine of the forty-one motion comparison trials involved the judgement of the dissimilarity of a single gait to itself. Participants were specifically instructed: If you think the two motions are exactly the same motion, you should indicate this by clicking at the very left [similar] end of the scale. On the Similar-Dissimilar rating scale there were seventeen vertical marks ("notches") to visually guide the recording of judgements. The space between any two vertical marks was 5.6% of the width of the scale. The marker used to indicate a judgement on the scale had a width of 1% of the scale which is about the narrowest it could be without appearing too thin. Any judgement to the right of the left-most mark would be more than 5.6% "dissimilar."6 As any judgement between the left end of the scale and this mark could be recorded just as easily as anywhere else on the scale, we hypothesized that any participant who rated a gait as more dissimilar than this amount was unsure if they had seen the same gait twice. Figure 5.3 presents the number of times each participant judged one of the nine gaits in a trial as being "not exactly the same" as itself, pS{i, i) > e, and the number of times they made this judgement every time they repeated the trial across the blocks. This graph lead to the realization that most of the participants failed to reliably to recognize and judge the self-similarity of the gaits. On average the participants made six mis-judgements out of twenty-seven trials. Note, that we've already established that the average trial variances were low indicating that judgements are consistent, what we are enumerating here are judgements greater than a threshold — the other judgements of the same trial pairs were nearby, they just were not beyond the threshold. However, we found two populations of participants in this data. The majority made relatively few mis-judgements of self-similarity (one third of trials or less), while a minority of six participants made a mis-judgement from half to almost all the time! As we can see in Figure 5.3 participants #5, #13, #15, #24, #27, #28, and #29 were extremely unreliable at this task. 61/18 = 5.6%. i—i—i—i—r i — i — i — i — i — i — i — i — i — i — i — i — i — i — i — i — r 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 Participant N u m b e r 8 -Si •=. 7 I J I I L J 1_ 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 Participant N u m b e r Figure 5.3: On the top: For each participant, the number of times when presented with the same motion twice in a motion comparison trial that they recorded a dissimilarity judgement more than one mark from the similar end of the Similar-Dissimilar scale, pô{i, i) > e. On the bottom: The number of gaits judged as "not exactly similar", pô(i, i) > e, each and every time the gait was presented twice in a motion comparison trial. There are twenty-seven trials involving a single motion compared to itself: nine gaits compared in each of three blocks. One possible explanation for the difficulty of judging the self-similarity of the gaits is perhaps that two or more gaits are too similar and participants were unsure if they were comparing gait P to itself or to gait P'. Figure 5.4 presents the number of times each gait was judged as not self-similar by all of the participants and by all participants except #5, #13, #15, #24, #27, #28, and #29. Most of the gaits have a similar number of mis-judgements, but gait Stiff Upright (#5) was most likely to be correctly compared to itself. 4 5 6 Gait Number Figure 5.4: Bar graph comparing the number of times each gait was judged to be not "the same motion" using tolerance p5(i,i) > e with and without "unreliable" participants. From the left, first four gaits are from the triangular stimuli, then the five linear gaits. Bar height indicates the total number of trials judged as more than one mark from the similar end of the Similar-Dissimilar scale. White bars: all participants; black bars: all participants except for #5, #13, #15, #24, #27, #28, and #29. Each gait was compared to itself by thirty participants in three blocks. The worst case would have the white bars ninety units tall, and the black bars sixty-nine units tall. We concluded that motion dissimilarity judgements are degenerate in that pô(i, i) > e. In the next section we présente our tests of the other half of the non-degeneracy property, pS(i,j) > 0, using pô(i,j) < e. 5.5.1.2 Judgement of Non-Self-Similarity of Gaits When presented with two different gaits, participants tended to judge them as dissimilar. If we count the number of times each participant reported a dissimilarity judgement of two different gaits to the left of the left-most notch on the Similar-Dissimilar scale, pô{hj) < e, we have the totals presented in Figure 5.5. The participants who did poorly at judgements of self-similarity (#5, #13, #15, #24, #27, #28, and #29) tended to do better at the task of discrimination — which may indicate that they tended to bias their judgements towards the Dissimilar end of the scale. 2 0 1 — i — i — i — i — n — i — i — i — i — i — i — i — i — i — i — i — i — i — i — r — i — i — i — i — i — i — i — i — i — i — I 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 2324252627 2829 30 Participant Number Figure 5.5: For each participant, the number of times they reported a dissimilarity judgement to the left of the left-most notch on the Similar-Dissimilar scale when presented with two different gaits. There are a total of ninety-six non-self-similarity trials judged by each participant: thirty-two gait pairs compared in each of three blocks. Trial First Gait Second Gait E pS(i,j) < 1 /18 ,*^ (1,4) Lower Crouch Average 2 (4,1) Average Lower Crouch 4 (2,1) Upper Body Lower Crouch 1 (2,4) Upper Body Average 7 (4,1) Average Upper Body 5 (3,4) Upper Tipping Average 11 (4,3) Average Upper Tipping 1 (5,6) SU 75% SU/25% SCT 1 (6,5) 75% SU/25% SCT SU 2 (6,7) 75% SU/25% SCT 50% SU/50% SCT 1 (7,6) 50% SU/50% SCT 75% SU/25% SCT 2 (7,8) 50% SU/50% SCT 25% SU/75% SCT 6 (8,7) 25% SU/75% SCT 50% SU/50% SCT 6 (8,9) 25% SU/75% SCT SCT 16 (9,8) SCT 25% SU/75% SCT 12 Table 5.2: Summary of trials in which participants judged two different gaits as being similar within one mark of the Similar end of the scale. The horizontal line in the middle of the table separates the trials of the triangular stimuli from the linear stimuli. This data is graphed in Figure 5.6. One possible explanation for the difficulty of discriminating between two different gaits is perhaps that two or more gaits are too similar and participants were unsure if they were comparing gait P to itself or to gait P'. Of the thirty-two trials involving the comparison of two different gaits, when we took all of the participants judgements into account, seventeen trials have no mis-judgements within one mark of the Similar end of the scale. We list in Table 5.2 the other fifteen trials. What is most interesting is the pattern of asymmetry in these mis-judgements. For the triangular gaits, the patterns of mis-judgements are asymmetric with one order of presentation tending to result in more mis-judgements than the reverse order of presentation. The upper body versus lower crouch trial has only one mis-judgement out of ninety trials (thirty participants times three presentations each). For the linear gaits, the mis-judgements are much more balanced. One possibility that explains this pattern of asymmetry is the strategy participants used to compare gaits. Figure 5.6 summarizes the data in the Table 5.2. As we saw in Section 5.2.5 trials T((,3),4), T((,4), l) , T((,7),6), and T((,7),8) were skipped the most often across all participants providing evidence that the "skip" button was used strategically to minimize uncertainty. One possible explanation for this pattern is participants attempted to remember a small number of features of the first presented gait. These features could be binary features such as "arms swinging" or they could be continuous features such as "arms swinging, a little bit." Extra mental storage would be given to features that were abnormal. Given this model, if a participant "picked" a poor set of features to memorize the first gait, the second gait would be poorly compared. Also, extreme gaits would consume more "feature descriptors" or bias the selection of features memorized to the extreme features. With the triangular gaits we should expect to see more asymmetric failures of such a strategy. As the trian-gular gaits involved three primary "extreme" gaits — with a couple of unique extreme features per primary gait — and an averaged gait, we would expect that participants using the above strategy would remember the extreme features of each primary when it was presented first and then have difficulty comparing the gait to the averaged gait due to a lack of overall gait description. That is, if the extreme features you remember do not match well, and if you do not have any more information, are you sure that you are not seeing the same gait again? To trade off uncertainty of perception with the need to record a dissimilarity judgement many participants probably thought it was best to record a judgement near, but not at, the similar end of the scale. In the reverse presentation order, the average gait would be given a much more uniform description since "nothing stands out" and then the second primary gait would have at least one feature that would distinguish the two gaits. With the linear gaits we begin to see a lower bound on the resolution of the features used to remember the gaits. A l l of the mis-judgements of the linear gaits occurred between gaits next to each other along the line. It is interesting that as we move along these gaits from Stiff Upright to Super Crouch Twisting the number of mis-judgements increases and with the last two gaits an asymmetry appears. Unfortunately since we did not predict this behaviour when we were designing the experiment it is difficult to test for statistical significance at this point for many reasons. First, we've picked only fourteen of the trials out of forty-one for further analysis, second we recorded continuous judgements and would have to bin the judgements to perform an analysis on the data as presented. We could argue that participants used the marks on the similar-dissimilar scale to guide their judgements and may have "binned" their judgements as they made them — but our statistical power would be very weak. So we will leave this for future experiments. As the asymmetrical mis-judgement effect is very interesting and should be examined further we will just suggest a few changes to the experiment methodology for such experiments. A method of reporting "my confidence in this judgement" would create a parallel set of data that may expose participant awareness of "uncertainty" of their judgements. Reaction time should be recorded also, providing the ability to look at priming effects caused by presentation order. However, these modifications are really attempts to compensate for participants who cannot separate their judgements from their uncertainties — which reflects the general population. Finally, a different network of trials could be used to expose the asymmetry effect. Since we only have evidence for the effect when the gaits were similar, a star-graph of trials could be used: n unique "extreme" motions would be compared to their average (requires only 2n unique trials). An additional n + 1 self-similarity trials would also allow us to check the reliability of self-similarity judgements and changes in strategies across blocks. Figure 5.7 illustrates such a network. We must conclude that motion dissimilarity judgements are degenerate in that dissimilarity judgements of two different stimuli are occasionally near zero, pô{hj) < £• 20 I 1 1 1 1 1 1 1 I F 1 1 1 p 1 r 5 18-53 (1,4) (4,1) (2,4) (4,2) (3,4) (4,3) (5,6) (6,5) (6,7) (7,6) (7,8) (8,7) (8,9) (9,8) Trial Pair Trial Pair Figure 5.6: Summation over all participants of the number of times participants reported a dissimilarity judgement to the left of the left-most notch on the Similar-Dissimilar scale when presented with two different gaits, p5{i,j) < e. Trial numbers are indicated along the horizontal axis, the left graph is for triangular trials and the right graph for linear trials. There are a total of ninety-six non-self-similarity trials judged by each participant: thirty-two gait pairs compared in each of three blocks. 6 Figure 5.7: A star network of stimuli with n unique "extreme" gaits and a centre average gait requires 2n trials (per block) to test the asymmetric mis-judgement of non-self-similarity and another n + 1 trials (per block) to test reliability of judgement of self-similarity (per block). 5.5.2 Testing the Symmetry Metric Property The second metric property, symmetry, tests p5(i, j) = pS(j, i) for all pairs of gaits. As we saw in Experi-ment One, there is weak evidence that dissimilarity judgements are not symmetric, and in the last section we saw that dissimilarity judgement of two different gaits are sometimes degenerate in an asymmetric fashion: participants were more likely to report pô(i,j) < 0 than the reverse trial pô(j,i)-For each participant we tested the two null hypotheses: H2QA : p6(i,j) = PS(j,i) € {1,2,3,4},; + j : PS(i,j) = PS{j,i) V i , j G {5, 6, 7, 8, 9}, z + j We expect that HQ0, and FtQb hold (are not rejected). Hça requires six tests per participant, and HQB ten tests per participant. This is statistically tractable however our statistical power is very low. These tests lead to a more useful conclusion than the non-degeneracy property: If the symmetry property holds for a set of gaits (or other motions), then we do not need to include symmetric trials in future experiments that use the same gaits. This reduces the number of trials from n 2 to n 2 / 2 for n stimuli. When we performed the ANOVAs we found that participants tended to report symmetric judgements of gaits. There are very few pairs of motions with asymmetric dissimilarity judgements. We list in Table 5.3 the participants and gaits for which pS{i,j) ^ pS(j, i) using p < 0.05 (df = 1, 4). As in Experiment One, we have a found that a fraction, 7.9% to be precise, of the pairs of dissimilarity judgements are asymmetric. Of the sixteen possible symmetric pairs of dissimilarity judgements, six par-ticipants have symmetric judgements (p < 0.05), fifteen participants have only one asymmetric judgement, five participants have two asymmetric judgements, three participants have three asymmetric judgements and one participant has four. Out of thirty participants, and sixteen trial pairs we have failed to find systematic asymmetry of dissimi-larity judgements. For those participants who are listed as having three or more asymmetric judgements in the above table, we have evidence that they were in general unreliable at judging the dissimilarity of motions, which may explain the asymmetry in their judgements. Participants #13, #15, #25 and #28 were p# i 3 p-value P# i 3 p-value 1 5 8 0.0199 16 2 3 0.0261 2 3 4 0.0148 16 5 7 0.0420 2 6 8 0.0281 17 1 4 0.0006 3 5 7 0.0040 19 3 4 0.0005 5 2 4 0.0018 20 7 9 0.0175 5 7 8 0.0217 21 3 4 0.0023 6 6 8 0.0361 23 3 4 0.0456 7 7 9 0.0434 24 6 9 0.0021 10 8 9 0.0390 25 1 4 0.0112 11 7 8 0.0327 25 5 6 0.0035 12 3 4 0.0292 25 5 8 0.0023 13 2 4 0.0255 26 5 6 0.0060 13 5 6 0.0457 26 5 7 0.0018 13 5 9 0.0090 28 1 4 0.0006 13 7 9 0.0469 28 3 4 0.0390 14 6 8 0.0043 28 5 9 0.0371 15 3 4 0.0056 29 6 7 0.0228 15 5 7 0.0003 29 8 9 0.0126 15 7 9 0.0355 30 5 7 0.0083 Table 5.3: Summary of results of ANOVAs comparing dissimilarity judgements for pairs of trials in which the order of the gaits is reversed. We list for each participants, the gaits for which pô(i,j) ^ pS(j, i) using p < 0.05 (df = 1,4). already noted in Section 5.5.1.1 as having many responses that indicate they were unreliable at reporting a dissimilarity judgement of "exactly the same motion" when presented with the same motion twice. So, as in Experiment One we conclude that motion dissimilarity judgements are symmetric. 5.5.3 Testing the Triangle Inequality Metric Property The last metric property, the triangle inequality, requires pô(i,j)+ pô{j,k) > pô(i, k) for all gait triples. This property is statistically the most difficult of the metric properties to test as it involves addition of dissimilarity judgements. A naive approach would be to test: Hla : p5(i,j) + pô(j, k) > p5(i,k) Vi,j,k G {1,2,3,4} Hlb : p6{i,j) + P0(j,k) > P5(i,k) Vi,j,k G {5,6,7,8,9} where the left hand sides, p6(i,j) + pS(j, k), indicate that we add dissimilarities per participant. However, to compute the values on the left hand side of these hypotheses we will need to estimate its variance, Var( pô(i,j) + pô(j,k)). For two independent random variables, X\ and X%, we know that Var(Xi + X2) = Var(-Xi) + Var (X 2 ) , 7 which implies p5(i,j) + pS(j, k) has variance Var( pS{i,j)) + Var( P8(j,k)). But we should not expect dissimilarity judgements pô{i,j) and pS{j,k) behave as inde-pendent random variables. So we need another option. Later we will be using multidimensional scaling (MDS) to produce a config-uration of the gaits in an Euclidean space from their pairwise dissimilarities. One of the first steps of the MDS procedure is to compute an additive constant for each participant, cp, to add to their dissimilarity judgements: pô(i,j) = pô(i,j) + cP such that for all gait triples the triangular inequality holds: * * * p5(i,j) + p5(j,k) > P5(i,k) and positivity holds pS(i,j) > 0, where * pS(i, j) is the adjusted average dissimilarity judgement between gait i and j by participant P 7See Larsen and Marx (1981, p. 115) or Neter and Wasserman (1974, p. 5). * pô(j, k) is the adjusted average dissimilarity judgement between gait j and k by participant P * pS(i, k) is the adjusted average dissimilarity judgement between gait i and k by participant P Estimating the value of cp is known as the the additive constant problem in M D S . 8 Instead of estimating cp, we counted the number of times cp > 0 for all triangular combinations of dissimilarity judgements. If the count is above 0, then the triangle inequality does not hold for participant P. This technique nicely avoids having to compute the variance of dissimilarity judgements pr5(i, j) + pô{j, k). When we computed cp for every triangle9 we found almost every participant violates the triangular inequal-ity at least once and summarize these violations in Table 5.4. Though the triangular inequality does not hold, the number of trials needed in future experiments is not greatly affected. If all of the metric properties held, and we knew that they would always hold, then we could use the absolute minimum number of comparisons necessary, (2n — k — 2)(k + l ) /2 , where k is the number of Euclidean dimensions we hypothesize are necessary to represent the psychological motion space. This lower bound is probably very rarely used in psychological experiments. Instead, if we accept that the symmetry property holds and that we are not particularly interested in knowing p5(i, i) so we require n(n - l ) /2 trials. 5.5.3.1 Comparison of the Groups of Participants We would like to be able to determine if any participants in any one group of participants has a better adherence to the metric properties than members in another group. Unfortunately, we did not develop a simple measure of "metricity" of a participant's dissimilarity judgements. As an option, we estimated cP by using: cp = max (p5{i,k) - Pô(i,j) - p6{j,k)) , i,j,k 8Davison (1983) discusses this transformation on p. 5 and 77, Young and Lewyckyj (1996) discusses it on p. 59, and SPSS Inc. (1997) discusses it on the page numbered 8. 9 For the triangular gaits, there are 24 possible triangles, for the linear gaits, there are 60 possible triangles. We have been conservative and assumed that symmetry matters, thus pS(i,j) ^ pS(j, Ï) and p5(j, k) ^ p5(k,j). #cP > 0 p# Triangular Linear Total 1 2 16 21 2 1 11 14 3 0 8 10 4 5 8 15 5 0 1 1 6 5 11 19 7 1 17 21 8 3 8 13 9 6 6 14 10 2 14 19 11 5 17 26 12 3 17 24 13 3 11 17 14 3 6 11 15 3 7 12 16 4 14 21 17 5 15 24 18 4 19 27 19 3 9 14 20 3 5 10 21 1 5 7 22 1 12 15 23 3 10 15 24 4 9 15 25 3 20 27 26 4 14 21 27 0 3 4 28 3 1 5 29 0 0 0 30 3 7 12 Table 5.4: For each participant the number of triangles formed from the dissimilarity judgements of the triangular and linear gaits which violate the triangular inequality. and then compared groups of participants by using their cp's for the triangular and linear stimuli as a measure of the conformance of the participant's dissimilarity judgements to the triangular inequality: values greater than zero are bad. While this was not a very good test for comparing the non-degeneracy or symmetry properties between groups, it was a pretty nice test for comparing the triangular inequality. In the previous section, when we tested the triangular-inequality, we were not given any feedback on how "untriangular" the dissimilarity judgements are. When we computed cp we discovered that of the twenty-four triangular combinations of the triangular gaits and sixty combinations of the linear gaits, almost all participants have at least one combination of dissimilarity judgements that does not conform to the triangular inequality. For each combination of dissimilarity judgements we computed an error of the fit of the judgements to the triangular inequality: To determine if any of the groups of participants were any better or worse at conforming to the triangular inequality we used ANOVA to compare the groups using the participant's cp's. The higher a participant's cp is, the worse their dissimilarity judgements maintain the triangular inequality.10 Using ANOVA, the cp's for the groups of participants, were each compared by experience, gender, and the direction of walking. We summarize the results in Table 5.5, the resulting p-values indicate there is only one significant difference between the groups of participants. The only significant result is for the direction of walking for the triangular gaits. Participants who saw the figure walk right to left have higher average values of cp than those who saw the figure walk left to right. However this difference only exists among the judgements of the triangular gaits. For the linear and all gaits combined," there is no significant difference. Box plots of these comparisons are presented in Figures 5.8a-5.8f. We believe that this significant difference between the groups does not contradict our earlier conclusion that there were no differences based on average trial variance. Since we do not have a single statistic that we 1 0Note that our definition of cp picks the "worst case" deviation from the triangle inequality, however the significant differences occur even if we use cp = e(hj> where 77. is the number of triangles. "We combine the cp's for the triangular and linear gaits by taking the maximum value of the two cp's for each participant. = {pS{i,k) - PS(i,j) - p6(j,k)) and computed cp = max e( i,j,k). max(cp) for Triangular Gaits Source SS df MS F P Experience 0.0576 2 0.0288 0.9970 0.3822 Error 0.7797 27 0.0289 Gender 2.1599 x 10 -4 , 2.1599 x 10~4 0.0072 0.9329 Error 0.8371 28 0.0299 Direction 0.1201 1 0.1201 4.6896 0.0390 Error 0.7172 28 0.0256 max (cp) for Linear Gaits Source SS df MS F P Experience 0.0157 2 0.0079 0.393.1 0.6788 Error 0.5408 27 0.0200 Gender 0.0182 1 0.0182 0.9451 0.3393 Error 0.5384 28 0.0192 Direction 0.0057 1 0.0057 0.2915 0.5935 Error 0.5509 28 0.0197 max(cp) for All Gaits Source SS df MS F P Experience 0.0070 2 0.0035 0.1590 0.8538 Error 0.5953 27 0.0220 Gender 0.0235 1 0.0235 1.1344 0.2959 Error 0.5789 28 0.0207 Direction 0.0193 1 0.0193 0.9285 0.3435 Error 0.5830 28 0.0208 Table 5.5: ANOVA summary table comparing the groups of participants according to goodness of fit to the triangular inequality (c^). The only significant result is for direction of walking for the triangular gaits (p = 0.039). Participants who say the figure walk right to left have higher average values of CA than those who saw the figure walk left to right. However this difference only exists among the judgements of the triangular gaits. can use to compare the groups we are attempting to compare them using a variety of statistics. Some of these statistics are much more reliable in that they combine many responses, cp on the other hand only indicates the worst possible triple of dissimilarity judgements rather than a more continuous measure such as the "average misfit" of dissimilarity judgements. Comparison of CA by Experience for Triangular Gaits Runner Dancer Normal Figure 5.8a: Box plot of the CA for triangular gaits based on experience. Differences are not significant. Comparison of CA by Gender for Triangular Gaits Female Male Figure 5.8b: Box plot of the CA for triangular gaits based on gender. Differences are not significant. Comparison of CA by Direction of Walk for Triangular Gaits Left to Right Right to Left Figure 5.8c: Box plots of the CA for triangular gaits based on experimentally controlled direction of walking of the figure. The difference is not significant. Comparison of c.4 by Experience for Linear Gaits Runner Dancer Normal Figure 5.8d: Box plots of the CA for linear gaits based on experience. The difference is not significant. Comparison of CA by Gender for Linear Gaits Female Male Figure 5.8e: Box plots of the c.4 for linear gaits based on gender. The difference is not significant. Comparison of CA by Experience for Direction of Walk Gaits 0.8 r 1 1 0.7 h 0.6 r 0.5 0.4 0.3 0.2 r 0.1 1 Left to Right Right to Left Figure 5.8f: Box plots of the CA for linear gaits based on experience. The difference is not significant. 5.6 Multidimensional Scaling of Dissimilarity Judgements To this point, we have been working with a single participant's dissimilarity judgements at a time. We have found strong positive correlations between the three motion spaces (Section 5.3 and that dissimilarity judgements do not have all of the metric properties (Section 5.5). We could continue to treat each participant's dissimilarity judgements individually, or we could attempt to apply an analysis model that takes into account the individual differences of the participants yet allows us to examine all of the participants' dissimilarities at once. For example, we would like to to visualize the psychological motion space of the participants. In this space, stimuli pairs that have large dissimilarity judgements would be placed far from each other, and pairs with small dissimilarity judgements would be placed near to each other. We would also like to compare the "perceptual motion spaces" of multiple participants to see if there is a general agreement of the dimensions, and arrangement and relative distances of stimuli. Some of the desired properties of this visualization are: • that it produce an arrangement of points (the stimuli) in an Euclidean space, • that all of the participants' dissimilarities be taken into account to form a common space, • that distances between points (the stimuli) in the common space approximate dissimilarities, • that individual participant differences in perception or strategy are represented in a separate space. Multidimensional Scaling (MDS) is one technique for transforming the dissimilarities between stimuli (points) into a spatial arrangement. The general problem that MDS addresses is the conversion of a set of distances between points into a spatial arrangement of the points best reflecting the distances. Since the distances can include errors the task of MDS algorithms is to construct the spatial arrangement that minimizes the difference between the given distances and the distances between the points in the spatial arrangement. The SPSS procedure A L S C A L was used to perform an asymmetric multidimensional scaling of the par-ticipants' dissimilarity judgements using the Generalized (weighted) Euclidean Model. The output of this procedure includes: • a common stimuli configuration in n-dimensions, • the weighting of each dimension by each participant, • an overall weighting of the dimensions by all participants, • the stress and squared correlation of the configuration with respect to each participant's averaged dissimilarity judgements, • and a measure of the heterogeneity of each participant (their weirdness), A L S C A L does not output information about systematic asymmetry in the judgements. Nor does it "average" the participants' dissimilarity judgements to compute the common stimuli configuration.12 For more information about the SPSS implementation of A L S C A L see Norusis (1994). For information about the PC version of A L S C A L , available for free download, see Young and Lewyckyj (1996). 5.6.1 MDS — Triangular Stimuli To review, in the first part of the experiment, participants first judged the dissimilarity of two gaits at a time. Each of these comparisons was performed with each gait appearing first, and each trial was attempted three times. For each trial (pair of gaits) these judgements are averaged across three blocks. Thus for each participant, we have a matrix of dissimilarities. For example, #l's matrix for the triangular stimuli is presented in Table 5.6. A L S C A L was used to scale the dissimilarity judgements for the triangular stimuli for all participants. Due to constraints the implementation of the algorithm imposes on the output dimensions, only a two-dimensional solution was possible. The two-dimensional stimuli configuration in Figure 5.9 demonstrates that the partic-ipants dissimilarity judgements form a spatial arrangement of the gaits that approximates the configuration used to define the gaits. l 2 For those readers who are interested, Ashby et al. (1994) discusses the dangers of simply averaging across participants when using MDS. Second Gait Presented 1 2 3 4 1 0.0167 0.2233 0.3200 0.1433 First Gait Presented 2 0.3433 0.0233 0.3633 0.0933 3 0.2867 0.3500 0.0133 0.0667 4 0.2600 0.0933 0.2367 0.0167 Table 5.6: Table of average dissimilarity judgements by participant #1 for the triangular gaits. 1 = Lower Crouch, 2 = Upper Body, 3 = Upper Tipping, 4 = Average 1.5 1 0.5 <M 0 o '(/> c CD E Q -0.5 -1 -1.5 -2 -1 -0.5 0 0.5 1 1.5 2 Dimension 1 Figure 5.9: A L S C A L scaling of the dissimilarity judgements for the triangular stimuli for all participants. The first dimension varies from the left mechanical/stiff/linear motions of the legs and arms, these dampen down as we move to the right, and arrive at "flexible" motion of the back and upper body. The second dimension reflects the part of the body that is moving — as a function of "height" from the floor. Stimuli are: (1) Lower Crouch, (2) Upper Body, (3) Upper Tipping, (4) Average. Interpreting the dimensions was somewhat difficult given the small number of points. The second dimension was a bit easier to interpret. At the lower end, "extreme" knee and hip motion dominates, giving way to lower back and finally "extreme" elbow motion. Thus the second dimension reflects the part of the body that is moving — as a function of "height" from the floor. The first dimension was a bit more difficult to describe — the left hand side seems to contain mechani-cal/stiff/linear motions of the legs and arms, these dampen down as we move to the right, and arrive at "flexible" motion of the back and upper body. We used regression to determine which of Walker ' s input parameters correspond to the dimensions. Ten parameters were varied to create the triangular stimuli — each of these parameters sets the maximum or minimum angle a joint of the walking figure will attain: Shoulder Rotation ( p e r c e n t _ s h o u l d e r j r o t ) Controls the amplitude of shoulder rotation oscillation about the vertical axis (in the trans-verse plane). Arm Swing (arm_swing_f a c t o r ) Set the amplitude of arm swing oscillation forward and backward at the shoulder joint. The end points of the arm swing are adjusted so that the arm tends to swing forward much more than it swings backward. Elbow Maximum (elbow_rot_max) Controls the maximum elbow flexion oscillation angle. When set to its lowest value there is the joint is locked at the e lbow_rot_min value. The parameter e lbow_rot_min controls the minimum angle of elbow extension. Torso Tilt ( to r so_ t i 11) Controls the angle the torso tips forward at the hips. Torso Sway ( torso-swayjnax) Controls the amplitude of the torso tilt oscillation. Hip Flexion Swing (h ip_swing3) Controls the amount of extra flexion of the hip to raise the leg up as before it descends for heel strike. Bounciness ( bounc ines s ) Controls the amount the body lowers just after heel-strike by adjusting the flexion of the knees. Knee Mid Stride (knee_midss) Controls the flexion of the knees throughout the stride. Knee Bend Impact (knee_impact) Controls the amount of extra flexion of the knee at heel strike. Dimension 1 Dimension 2 Parameter R 2 p-value R 2 p-value p e r c e n t _ s h o u l d e r _ r o t 0.2828 0.4682 0.6231 0.2107 arm_swing_f ac t o r 0.2828 0.4682 0.6231 0.2107 elbow_rot_max 0.2828 0.4682 0.6231 0.2107 t o r s o _ t i . l t 0.9971 0.0014 0.0179 0.8660 torso_sway_max 0.9971 0.0014 0.0179 0.8660 h ip_swing3 0.2178 0.5333 0.8525 0.0767 b o u n c i n e s s 0.2178 0.5333 0.8525 0.0767 knee_midss 0.2178 0.5333 0.8525 0.0767 knee_impact 0.2178 0.5333 0.8525 0.0767 knee_swing2 0.2178 0.5333 0.8525 0.0767 Table 5.7: Results of the regressions between the parameters of the triangular gaits and the A L S C A L con-figuration produced from the dissimilarity judgements of the triangular gaits. Large R2 values (near 1) and low p-values (p < 0.1) indicate strong relationships between variation in the parameter and dimension produced by A L S C A L . Knee Bend Swing (knee_swing2) Controls the amount of extra flexion the knee bends as the foot leaves the ground — allows the foot to "kick up" before the leg swings forward. For each of these parameters, a simple regression was performed using M A T L A B ' s function r e g r e s s between their values for each of the stimuli and the positions of the stimuli along each of the two dimensions in the MDS solution. That is, for the positions of the i stimuli along Dimension 1 (xu) and j parameters values {yj) the regression equation: yj = b0 + bxxu was solved for bo and b\ using least-squares. We're not especially interested in the values of &o and b\, but rather the proportion of explained variability, R 2 and the regression coefficient p-value. We present these values in Table 5.7 Note that the repetition of the R 2 and p-values in the Table 5.7 are an artifact of the use of interpolation to create the gaits and configuration of the gaits computed by MDS. Because gaits 3 and 4 are aligned horizon-tally they tend to cancel out the variation in parameters corresponding to Dimension 2 {e.g., they increase the squared errors). There is a similar effect along Dimension 1 where only the parameters specifically varied to create gait 3 have strong correlations. The large R values indicate strong correlations between the parameters and the dimensions. Dimension 1 is primarily the upper body tilt and sway, with a bit of the other parameters thrown in, while Dimension 2 is the variation in the flexion amounts of the shoulder, elbow, hips, and knees without any of the torso sway. Granted, this is using only four stimuli, and thus the p-values are not extremely significant, but it does give a fairly reasonable interpretation of the dimensions for the number of stimuli. Each of the participants has a different weighting of the two stimuli configuration dimensions13. Each participant's weights scale the configuration to better match their dissimilarities. Participants that tended to weigh Dimension 1 more than Dimension 2 would tend to have wider and flatter configurations. We have plotted the participants' weights in Figure 5.10 to help in the interpretation of the weights — it is important to interpret the weights as a vector from the origin rather than as a "spatial distribution." The closer the points are to the unit arc — that is the longer the vector — the better the overall configuration accounts for the dissimilarity judgements of a specific participants. However it is not the length of the vector from the origin that equals the proportion of variation of the dissimilarity judgements accounted for it is the sum of squared weights. Thus the dashed line with radius of 0.9 reflects a proportion of variance accounted for of 0.81, and the dotted line with radius 0.8 is a proportion of variance of 0.64. Since weights are vectors from the origin, participants who have vectors pointing in the same direction are more in agreement than participants whose weights merely place them near each other. For example, in Figure 5.10, participants #14 and #16 have very similar weights while participants #21 and #26 have different weights. An "overall importance" of each dimension can be computed by taking the mean of the squared weights, this results in a weight of 0.4802 for Dimension 1 and 0.3597 for Dimension 2. Thus, overall the participants judged the gaits as being slightly more dissimilar left to right, than top to bottom. The squared correlation, or RSQ, is the proportion of the variance of a participant's dissimilarity judgements accounted for by the distances between the stimuli in the overall configuration scaled by the participant's weights. We compute RSQ by summing the squared weights for each participant. As we can see in the top-left graph of Figure 5.11, the overall configuration approximates between 68% and 93.6% of the participants' 1 3Note that SPSS orders the dimensions so that Dimension 1 has the highest weights, Dimension 2 the next highest weights and so on. 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Dimension 1 Figure 5.10: Participant weights for the two dimensional A L S C A L solution for the triangular stimuli. Each pair of weights can be used to scale the overall configuration — participants near the bottom-right of the graph treated dissimilarities between the gaits such that Dimension 1 is more important than Dimension 2, while participants near the top middle of the graph did the reverse. The unit arc marks the boundary of the possible weight values, and a 100% proportion of variance of the dissimilarity judgements explained by the overall configuration. The proportion of variance is the sum of squared weights — thus the dashed arc is 81% explained, and the dotted arc is 64% explained. "Overall importance" of the dimensions is computed by taking the mean of the squared weights, the • marks this value, and the dotted line from the origin extends through the centroid of the weights. dissimilarity judgements. In the top-right graph of Figure 5.12 are plotted Kruskal's SSTRESS(l) values for each of the participants. The SSTRESS(l) values are relative: a "low" SSTRESS(l) value indicates a "good fit" of the dissimilarity judgements to the distances in the configuration, while a "high" SSTRESS(l) value indicates a poor fit. In A L S C A L , SSTRESS(l) is used to terminate the iterative algorithm — when SSTRESS(l) no longer decreases, the algorithm stops. We can also use SSTRESS(l) to help us decide what the optimal number of dimensions. SSTRESS(l) tends to decrease as we add dimensions but also tends to "bottom out" after the optimal number of dimensions has been reached. There are several reasons why a participant's SSTRESS(l) value would be high relative to the SSTRESS(l) values of the other participants — the first is simply that the distances in the overall configuration do not closely approximate the dissimilarity judgements of a participant. This may indicate that the participant's judgements would be better approximated by a different configuration or a larger number of dimensions. However, as we can see in Figure 5.13 there is a definite trend towards lower RSQ as the SSTRESS(l) increases. Finally, the '"weirdness index" is a measurement of how balanced a participant's weights are. A participant's weirdness is near zero, if their weights are almost equal. However, if a participant has one large weight and one small weight1 4 then their weirdness will be nearer to one as their weights will not point towards the overall weights. The weirdnesses of the participants are plotted in Figure 5.14. To summarize, the A L S C A L scaling of the dissimilarity judgements for the triangular stimuli produced a two-dimensional configuration similar to the configuration used to define the stimuli. Most of the partici-pants' dissimilarity judgements were fitted well with a nice cluster of participants agreeing on the weightings of the two dimensions with fairly tight spreads to either side of the overall weights. The proportion of vari-ances of the dissimilarity judgements accounted for by the overall configuration is between 68% and 93.6%. The correlation between RSQ and SSTRESS(l) indicates that participants had a range of patterns of vari-ation in the dissimilarity rather than forming two or more groups. The correlation between the weirdness index and the angle of the weights along the unit arc confirm this. Or one large weight and many small weights in the case of higher dimensions. Squared Correlation Between Participant Dissimilarities and Distances in Overall MDS Solution (RSQ) o £0.9 S 0.8 Q 0.7 0.6 0.5 0.4 n 0.3 3-0.1 12 28 14 24 13 16 25 2 5 15 27 6 19 1 4 26 23 10 21 20 17 11 3 8 30 18 7 22 9 29 Figure 5.11: The squared correlation, or RSQ, is the proportion of the variance of a participant's dissimi-larity judgements accounted for by the distances between the stimuli in the overall configuration. RSQ is computed by summing the squared weights for each participant (see Figure 5.10 for the weights). SSTRESS(l) of Participant Dissimilarities versus Distances in Overall MDS Solution 0.4 j i — i — i — i — i — i — i — i — i — i — i — ; — i — i — i — i — i — i — i — i — i — i — i — i — i — i — i — i — i — r — 0.35 -0.3 -0.25 -29 22 7 9 3 8 30 18 10 21 17 23 11 27 20 19 1 26 5 15 2 4 25 6 14 13 16 24 28 12 Participant Number Figure 5.12: Kruskal's SSTRESS(l) values indicate the "goodness of fit" or the disagreement between a participant's dissimilarity judgements and the overall configuration. High values indicate a poor agreement, low values a good agreement. SSTRESS(l) versus RSQ 0.2 0.25 0.3 SSTRESS(I) 0.4 Figure 5.13: SSTRESS(l) versus RSQ: A strong correlation between SSTRESS(l) and RSQ indicates that participants with "poorer fits" (on the right) also had different patterns of variation in their dissimilarity judgements than the rest of the participants. 1 i—i 1 1 1 — i — i 1 — i 1 — i 1 1 1—i 1 — i 1 1 1 — i 1 — i 1 1 1 1 1 1 1 r 0.9 -0.8 -0.7 -g 0.6 -20 18 12 29 22 10 7 15 27 8 1 30 5 19 11 6 2 23 25 17 3 9 4 24 28 16 14 26 21 13 Participant Number Figure 5.14: Plot of weirdness indices for the participants from the triangular stimuli. A participant with weights proportional to the overall weights has a weirdness of zero, the minimum value. A participant with one large weight and many low weights has a weirdness near one. A participant with exactly one positive weight has a weirdness of one, the maximum value for non-negative weights. 5.6.2 MDS — Linear Stimuli A L S C A L was also used to scale the dissimilarity judgements of the linear stimuli. Three-dimensional and two-dimensional configurations were computed. The relative weightings of the dimensions indicate that a one-dimensional solution best explains the dissimilarity judgements. Figure 5.15 presents the three-dimensional configuration. Our nice line of interpolated gaits is no where to be found ! Dimension 1 Figure 5.15: Three-dimensional A L S C A L scaling of the dissimilarity judgements for the linear stimuli for all participants. Orthogonal plots of the configuration in two dimensions are plotted as dashed lines. When we look at the participants' weights of each dimension in Figures 5.16-5.19, we see that Dimension 1 has disproportionately high weights compared to Dimensions 2 and 3. Since A L S C A L orders the dimensions according to the strength of the weights we should always expect that Dimension 1 has the largest weights, Dimension 2 the next highest and so on. We have almost a factor of 10 difference in the overall importance of Dimension 1 compared to Dimension 2: Dimension Overall Weight Ï 0.7988 2 0.0820 3 0.0204 Thus, Dimensions 2 and 3 are probably spurious, due to "noise" and "error" amplified by the large number of participants used to compute the stimuli configuration. Dimension 2 0 0 Dimension 1 Figure 5.16: Participant weights for the three dimensional A L S C A L solution for the linear gaits. Relative values of the weights indicate that Dimension 1 is the most important with Dimensions 2 and 3 probably unnecessary. 0 t 1 1 ' — . '• • • • • i 0 0.2 0.4 0.6 0.8 1 Dimens ion 1 Figure 5.17: Participant weights for the three dimensional A L S C A L solution for the linear gaits. Relative values of the weights indicate that Dimension 1 is the most important with Dimensions 2 and 3 probably unnecessary. 0.8h 0.6h 0.2 r-~ L _ 1 - . 1 . 1 0 0.2 0.4 0.6 0.8 1 Dimension 2 Figure 5.18: Participant weights for the three dimensional A L S C A L solution for the linear gaits. Relative values of the weights indicate that Dimension 1 is the most important with Dimensions 2 and 3 probably unnecessary. 0.8h 0.6 h E b 0.4 -0.2 h 0 l 1 1 — . — L _ _ J • • •• • • » 0 0.2 0.4 0.6 0.8 1 Dimension 1 Figure 5.19: Participant weights for the three dimensional A L S C A L solution for the linear gaits. Relative values of the weights indicate that Dimension 1 is the most important with Dimensions 2 and 3 probably unnecessary. In Figures 5.20-5.22 we see that the SSTRESS(l) values for the three-dimensional configuration are in general low and the RSQ's are high, however the weirdness index is rather high for most of the participants — reflecting the fact that the dimensional weights are dominated by Dimension 1. The extra dimensions are allowing a better fit to all of the dissimilarity judgements, but most participants do not need three dimensions — their judgements can be approximated using only one dimension. Squared Correlation Between Participant Dissimilarities and Distances in Overall MDS Solution (RSQ) 1 1 — i — i — i — i — i — i — i — o £ o . 9 2 0.8 Q ! 0.7 0.6 0.5 0.4 ; 0.3 3-0.1 ~i 1 1 r~ 25 1 28 13 20 6 19 7 3 17 15 9 26 18 30 8 5 11 23 14 16 12 24 4 2 27 29 22 10 21 Participant Number Figure 5.20: RSQ of the three dimensional configuration: The two-dimensional configuration for the line stimuli dissimilarities is not simply a two dimensional pro-jection of the three dimensional solution but a completely independent solution computed using only two dimensions. In Figure 5.23 we have the stimuli arranged in a ' U ' or a 'Horseshoe' — a configuration gen-erally considered 'good luck' 1 5 and an indication that only the first dimension is necessary. The overall importances of each dimension also indicate that a one dimensional solution is appropriate: Dimension 1 has an overall importance of 0.8104 and while the overall importance of Dimension 2 is only one tenth (0.0814). We also have a very nice symmetrical and balanced spacing of gaits along Dimension 1. Interestingly, the two-dimensional configuration provides evidence for a non-linear relationship between l 5See 'Do Horseshoes Bring Good Luck?' Jackson (1991, Section 11.5). SSTRESS(l) of Participant Dissimilarities versus Distances in Overall MDS Solution 0.4 j — i — i — i — i — i— i— i — i— i— i — i — i — i— i — i— i— i — i— i— i — i — i — i— i— i — i— i— i — r ~ z — 0.35 -0.3 -0.25 -21 29 22 27 10 5 4 2 24 16 12 23 30 14 9 8 11 3 28 19 15 26 20 18 13 6 17 7 1 25 Participant Number Figure 5.21: SSTRESS(l) of the three dimensional configuration. Walke r ' s input parameters and dissimilarity judgements. As we can see in Figure 5.23, the spacing be-tween the gaits along the first dimension is not uniform. Instead two gaits on each end are "bunched up" leaving a larger gap on either side of gait 7. Thus a uniform linear interpolation of Walke r ' s input parame-ters does not result in dissimilarities between gaits that are uniformly spaced. However, the deviation is not too excessive. We can use regression to determine which of the W a l k e r input parameters correspond to the dimensions. Nineteen parameters were interpolated to create the linear stimuli. 1 6 In addition to the parameters changed for the triangular stimuli, eight additional parameters were used to define the end points of the linear stimuli: Arm Out (arm_out) Controls how high the arms are raised from the sides of the torso (no oscillation) Elbow Minimum (elbow_rot_min) Controls the minimum elbow flexion oscillation angle. Pelvis Lateral Displacement ( l a t e r a l _ d i s p _ f a c t o r ) Controls the amplitude of lateral displacement of the pelvis as the weight is shifted between feet. Pelvis Rotation ( p e l v i s _r o t _max) Controls the amplitude of pelvis rotation about the vertical axis (in the transverse plane). Pelvis List ( p e l v i s _ l i s t _ m a x ) Controls the amplitude of pelvis rotation clockwise and counter-clockwise (in the coronal plane). Stride Width ( s t r i d e _ w i d t h _ f a c t o r ) Controls the distance laterally between the feet on each step. Foot Angle (f oo t_angle) Controls the angle the feet make with the direction of walking (i.e., duck toed versus pigeon toed). Leg Overstride ( o v e r s t r i d e ) Controls the distance between the forward heel-striking foot and the center of mass — small values move the figure over the foot, large values move the foot forward as if "brak-ing" or attempting to slow the walk down. As before, for each of these parameters, a simple regression was performed using M A T L A B ' s function r e g r e s s between their values for each of the stimuli and the positions of the stimuli along each of the two dimensions in the MDS solution. We present the R2 and p-values of the regressions in Table 5.8. 1 6There were actually twenty-four interpolated parameters, but five of these are internal parameters not accessible through the W a l k e r GUI. -1.3277 -0.8608 -0.0077 0.8926 1.3036 1.5 CM 0.5 g c CD -0.5 -1.5 -1.5 » i f 5 ( t 7 i K -0.5 0 Dimension 1 0.5 1.5 (a) A L S C A L scaling of dissimilarity judgements for linear simuli. 0.5 CM C o CO c CD -0.5 -1.5 -1.076 -0.6976 -0.0062 0.7234 1.0564 I I I I 1 1 1 ! 1 . l ^ f . i i i — nr i I -1 -0.5 0 0.5 1.5 Dimension 1 (b) A L S C A L scaling weighted by overal weights. Figure 5.23: Two-dimensional A L S C A L scaling of the dissimilarity judgements for the linear stimuli for all participants, (a) the unweighted configuration (equal weights for both dimensions), (b) the configuration weighted by the overall weights (0.8104, 0.0814). Gait 5 is Stiff Upright, 6 is 75% Stiff Upright/25% Super Crouch Twisting, 7 is 50% Stiff Upright/50% Super Crouch Twisting, 8 is 25% Stiff Upright/75% Super Crouch Twisting, and 9 is Super Crouch Twisting. Dimension 1 Dimension 2 Parameter R 2 P -value R2 P -value p e r c e n t _ s h o u l d e r _ r o t 0 9845 0 0925 0 0008 0 6188 arm_swing_f ac t o r 0 9845 0 0925 0 0008 0 6188 arm_out 0 9845 0 0925 0 0008 0 6188 e lbow_rot_min 0 9845 0 0925 0 0008 0 6188 e 1 bow_r o t _max 0 9845 0 0925 0 0008 0 6188 t o r s o _ t i . l t 0 9845 0 0925 0 0008 0 6188 t o r s o _s way _max 0 9845 0 0925 0 0008 0 6188 l a t e r a l _ d i s p _ f a c t o r 0 9845 0 0925 0 0008 0 6188 p e l v i s _ r o t _ m a x 0 9845 0 0925 0 0008 0 6188 p e l v i s _ l i s t _ m a x 0 9845 0 0925 0 0008 0 6188 b o u n c i n e s s 0 9845 0 0925 0 0008 0 6188 knee_midss 0 9845 0 0925 0 0008 0 6188 knee_impact 0 9845 0 0925 0 0008 0 6188 h ip_swing3 0 9845 0 0925 0 0008 0 6188 knee_swing2 0 9845 0 0925 0 0008 0 6188 s t r i d e _ w i d t h _ f a c t o r 0 9845 0 0925 0 0008 0 6188 f oo t_ang le 0 9845 0 0925 0 0008 0 6188 o v e r s t r i d e 0 9845 0 0925 0 0008 0 6188 Table 5.8: Results of the regressions between the parameters of the linear gaits the A L S C A L configuration produced from the dissimilarity judgements of the linear gaits. Large R2 values (near 1) and low p-values (p < 0.1) indicate strong relationships between variation in the parameter and dimension produced by A L S C A L . As expected, the parameters all correlate extremely strongly with Dimension 1 and have almost no linear correlation with Dimension 2. Figure 5.24a- 5.24b presents the participant's weights for the two dimensional configuration we see that a large number of the participants preferred Dimension 1 over Dimension 2. There are fourteen participants with a Dimension 2 weight less than 0.1 1 7 and five participants with a Dimension 2 weight of 0. 1 8 As we can see in Figures 5.25-5.28, though the SSTRESS(l) and weirdness indices indicate that the judge-ments of some participants are not fitted well by the two dimensional configuration — the high RSQ's, most higher than 80%, indicate that the two dimensional configuration does explain a very large proportion of the dissimilarity judgements. 17#1, #2, #3, #10. #11, #12. #13, #15, #16, #18, #22, #24, #26, #27. 18#1, #12, #16, #18. #22. 0.8 0.7 0.6 C\J c o 2 0.5 CO E Q 0.4 0.3 0.2 0.1 — _ l i i i "•" T i i • r •»-» \ . N . \ N 14- \ X \ \ \ \ \ - \ - 2 0 \ 2 8 - ^ \ \ - 5 \ \ \ - \ \ v - 6 \ \ \ - \ 8 - . 2 \ \ \ x 2 1 - \ \ \ 2 5 - \ - 1 7 \ \ - 2 3 -\ 1 9 - \ \ . \ \ . \ 'overall importance' n V " ' | " , , , , , , i- * 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Dimens ion 1 Figure 5.24a: Participant weights for the two dimensional A L S C A L solution for the linear gaits. Relative values of the weights indicate that Dimension 1 is the most important with Dimension 2 having only one tenth the importance of Dimension 1. Each pair of weights can be used to scale the overall configuration — participants near the bottom-right of the graph treated dissimilarities between the gaits such that Dimen-sion 1 is more important than Dimension 2, while participants near the middle of the graph balanced both dimensions. The unit arc marks the boundary of the possible weight values, and a 100% proportion of vari-ance of the dissimilarity judgements explained by the overall configuration. The proportion of variance is the sum of squared weights — thus the dashed arc is 81% explained, and the dotted arc is 64% explained. "Overall importance" of the dimensions is computed by taking the mean of the squared weights, the • marks this value, and the dotted line from the origin extends through the centroid of the weights. 0.2 E 0.1 E Q 'overall importance' •30 •27 •15 •13 24- " 2 •11 •10 •26 18 16 12 22 0.8 0.9 Dimension 1 Figure 5.24b: A close up of the participant weights for the two dimensional A L S C A L solution for the linear gaits. Squared Correlation Between Participant Dissimilarities and Distances in Overall MDS Solution (RSQ) 1 1 — i — i — i — i — i — i — i — i — i — i — i — i — i — i — \ — o GO or. 0.9 S 0.8 O 1 0.7 ; o.6 i 0.5 0.4 5 CO 0.3 3-0.1 h— i— i— i— i — i — r ~ 28 25 1 13 20 3 6 17 7 9 15 5 19 18 14 8 26 30 23 11 27 24 4 29 16 12 2 21 22 10 Participant Number Figure 5.25: RSQ of the two dimensional configuration. Figure 5.26: SSTRESS(l) of the two dimensional configuration. Weirdness Index 1 [ — i — i — i — i — i — i — i — i — i — i — i — i — i — i — i — i — i — i — i — i ! r 0.9 -0.7 -0.6 -" a "w S 0.5 -19 23 9 17 25 4 21 30 7 29 8 27 6 5 3 20 15 2 28 14 24 11 10 26 13 1 12 16 18 22 Participant Number Figure 5.27: Weirdness Index of the two dimensional configuration. Difference in SSTRESS(l) Between 3D and 2D MDS Solutions Increase in SSTRESS(1) from Three to Two Dimensional Configuration for each Participant Figure 5.28: Comparison of the SSTRESS(I) between the two and three dimensional configurations. 5.6.3 Comparisons of the Groups of Participants Are there any differences between the groups of participants based on the multidimensional scalings of their dissimilarity judgements? One method of testing for differences is to compare their mean weirdness indices. Since the weirdness is a measure of how far a participant's dimensional weights are from the ideal weights (i.e., equal weights for each dimension). Using ANOVA, the weirdness index of each group of participants were compared. Our null hypothesis is the mean weirdnesses of each group of participants is equal. We summarize the results of the ANOVAs in Table 5.9. The only significant result is the mean weirdness index for the linear two dimensional configura-tion is lower for participants who saw the figure walk across the screen to the left than participants who saw the figure across the screen to the right (p — 0.0120). Recall that the weirdness index is a measure of the imbalance of a participants dimensional weights with a weirdness near one indicating that one dimension is more favored than the others. Box plots of these comparisons are presented in Figures 5.29-5.34. A preponderance of low weirdness for the participants who saw the figure walk across the screen to the left would indicate that they have dimensional weights that do not favor Dimension 1 over Dimension 2 as most of the participants do. While more balanced weights would produce a more "two dimensional U " configuration as illustrated in Figure 5.23, the participants tend to have weights with strongly favor Dimension 1 over Dimension 2, which produces a very flatted shallow " U " configuration. As we can see in Figures 5.33-5.34 the participants who saw the figure walk right to left tend to have weights which are only slightly more in balance than the participants who saw the figure left to right. To summarize, the psychological space that participants compared the linear stimuli in is basically one dimensional — the configuration of stimuli along this dimension is nicely balanced. The spacing of the stimuli along the line has a non-uniform spacing of "close, far, far, close." Neither the experience level of participants (dancers, runners, or normals) or gender (male, female) has significantly different means according to the weirdness index for any of the configurations of stimuli. However there is a difference between the participants based on which direction the figure walked across the screen. Weirdness for Two Dimensional Configuration of Triangular Stimuli Source SS df MS F P Experience 0.0642 2 0.0321 0.8545 0.4367 Error 1.0147 27 0.0376 Gender 0.0560 1 0.0560 1.5323 0.2260 Error 1.0229 28 0.0365 Direction 7.7336 x 10" 4 1 7.7336 x 10~ 4 0.0201 0.8883 Error 1.0781 28 0.0385 Weirdness for Three Dimensional Configuration of Linear Stimuli Source SS df MS F P Experience 0.1187 2 0.0594 1.6289 0.2148 Error 0.9836 27 0.0364 Gender 0.0190 1 0.0190 0.4918 0.4889 Error 1.0834 28 0.0387 Direction 0.0469 1 0.0469 1.2445 0.2741 Error 1.0555 28 0.0377 Weirdness for Two Dimensional Configuration of Linear Stimuli Source SS df MS F P Experience 0.2095 2 0.1048 1.1843 0.3214 Error 2.3888 27 0.0885 Gender 0.0233 1 0.0233 0.2538 0.6183 Error 2.5750 28 0.0920 Direction 0.5325 1 0.5325 7.2168 0.0120 Error 2.0659 28 0.0738 Table 5.9: ANOVA summary table comparing the groups of participants according to weirdness index for the A L S C A L scalings of the triangular stimuli (two dimensional) and linear stimuli (two and three dimen-sional). G r o u p weirdness for Tr iangular Stimuli: T w o Dimens iona l Configurat ion R u n n e r D a n c e r Normal G r o u p weirdness for Linear Stimuli: T h r e e Dimens iona l Configurat ion R u n n e r D a n c e r Normal Figure 5.29: Box plots of the weirdness indices by experience (dancer, runner, normal) female for the two dimensional A L S C A L configuration of the triangular stimuli and the three dimensional A L S C A L configu-ration of the linear stimuli. No differences are significant. G e n d e r we irdness for Tr iangular Stimuli: T w o Dimens iona l Conf igurat ion F e m a l e M a l e G e n d e r weirdness for L inear Stimuli: T h r e e Dimens iona l Configurat ion 0.8 0.6 CD c " O '05 3 0.4 0.2 0 I . 1 F e m a l e M a l e Figure 5.30: Box plots of the weirdness indices by gender (male, female) for the two dimensional A L S C A L configuration of the triangular stimuli and the three dimensional A L S C A L configuration of the linear stimuli. No differences are significant. Direction of Walk weirdness for Triangular Stimuli: T w o Dimens iona l Configuratior Left to Right Right to Left Direction of W a l k we irdness for Linear Stimuli: T h r e e D imens iona l Configuratior Left to Right Right to Left Figure 5.31: Box plots of the weirdness indices by direction of walk (left to right, and right to left) for the two dimensional A L S C A L configuration of the triangular stimuli and the three dimensional A L S C A L configuration of the linear stimuli. No differences are significant. G r o u p we irdness for Linear Stimuli: T w o Dimens iona l Configurat ion R u n n e r D a n c e r Nor m al G e n d e r we irdness M e a n s for Linear Stimuli: T w o Dimens iona l Configuration F e m a l e M a l e Figure 5.32: Box plots of the weirdness indices by experience (dancer, runner, normal; male, female; di-rection of walk: (left to right, and right to left) for the two dimensional A L S C A L configuration of the linear stimuli. No differences are significant. Direction we irdness for Linear Stimuli: T w o Dimens iona l Configuration 0.8 -I 0.6-3 4 6 7 8 9 12 17 19 20 21 1 2 5 10 11 13 14 15 16 18 222324252627282930 Right to Left Left to Right Figure 5.33: Box plot of the weirdness indices by direction of walk (left to right, and right to left) for the two dimensional A L S C A L configuration of the linear stimuli illustrating the only significant result between the groups. The mean weirdness index for the linear two dimensional configuration is lower for participants who saw the figure walk right to left than the participants who saw the figure walk left to right atp = 0.0120. 0.9 0.8 0.7 0.6 c g 2 0.5 Q> I b 0.4 0.3 0.2 0.1 0 L 0 0 I \ » \ ( x 9 \ » \ 1 \ \ \ t \ 1 *4 \ t 1 1 -30 1 \ \ \ x7 'overall importance' .. 1 1 1 1 1 1 *3 1 -15 I 24- "2 | I -11 I ' 1 0 I -13 |1 18 16..12 22 0.8 0.9 1 Dimension 1 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Dimension 1 Figure 5.34: Plots of the participant's weights for the two dimensional configuration of the linear stimuli. Participants who saw the figure walk right to left — who on average have lower weirdness indices — are marked with ' x's. These participants tend to have more balanced weights but still favor Dimension 1 over Dimension 2. 5.7 Conclusions The goal of this experiment was to ascertain the metric properties of motion comparisons by having partici-pants judge the dissimilarity of human gaits and to describe each gait using adjective-adverb scales. Thirty participants were recruited with backgrounds in social dance, running, and neither. Additionally, the direction the figure walked across the computer screen was controlled. The response of the groups of participants were compared to determine if there were differences based on average variance of dissimilarity judgements, metric properties of dissimilarity judgements, and strength of correlation between dissimilarity judgements and differences of gait description. While a few significant differences were found, we failed to find systematic differences between participants based on gender, background or the direction of walking. It was determined that dissimilarity judgements are degenerate. When participants are presented with the same motion twice they occasionally respond that the motions are different. When presented with two different motions the occasionally respond that the motions are identical. Possible changes to the experiment design include marking a region of the Similar-Dissimilar scale as "identical", changing the labels to be "Identical" on the Similar end, allowing participants to use a second scale to indicate the "uncertainty" of their judgement. There are interesting asymmetries in the degeneracy of dissimilarity judgements that should be investigated in future experiments. We are willing to accept that dissimilarity judgements are symmetric based on order of presentation. This allows us to use only n(n — l ) /2 trials rather than n2 — n. The triangle inequality does not hold. Thus dissimilarity judgements are not metric. If we wish to treat dissimilarity judgements as approximations of the distance between motions in a Euclidean psychological motion space then we will need to use a technique such as multidimensional scaling to form an approximate configuration of the stimuli in a Euclidean space. There is weak evidence that walking direction across the screen affects motion dissimilarity judgements, but no evidence for differences based on gender or background. Multidimensional scaling was used to reproduce the stimuli configurations from the dissimilarity judge-ments. The triangular stimuli require a two dimensional space and the linear stimuli a one dimensional space. Linear interpolation of Walke r ' s gait parameters does not result in linear interpolation of dissimi-larity judgements (see Figure 5.23). Chapter 6 Conclusions and Future Research Directions We require an understanding of the relationships between the psychological, linguistic, and mechanical motion spaces before we can build high-level tools for adjusting computer representations of human move-ments. Our ultimate goal is to build tools capable of adjusting the gross action of a movement without affecting the its style or extract a movement style from a collection of movements for the creation of new movements. The purpose of this research was to demonstrate and validate our experimental and analytic methodology for the collection of judgements and descriptions of human movements and determination of the relation-ships between the motion spaces. We have presented the results of two participant experiments in which participants compared and described the motion of computer animation displays of human walking figures. Analysis of the participants' responses found linear correlations between the psychological, linguistic, and mechanical motion spaces. The goals of Experiment One was to test the experiment design of having participants judge the dissimi-larity of pairs of walking motions and describe the walking motions. Our goal was to demonstrate strong correlations between the motion spaces and find common patterns within a small somewhat homogeneous group of participants. In Experiment One, moderately strong linear relationships between the three motion spaces were found. Additionally, we discovered principal components common to several participants' descriptive ratings as well as components that reflect individual differences between the participants. The existence of common components indicates that participants interpreted the descriptive rating scales in similar fashions with re-spect to the gaits used as stimuli. These common components also suggest that human-computer interfaces to animation systems would not need to be custom tailored for each user, rather they could be "initialized" with a set of standard controls and then refined by adjusting the standard controls or by choosing some from a set of controls reflecting the user's perceptual weights. We conclude that the participants' descriptions and similarity judgements of the gaits are generally consistent within our model while individual differences reflect personal psychological differences. We concluded that the goals of Experiment One were achieved. Based on our observations of the agreements among the participants' linguistic motion spaces we decided to focus on the psychological motion space for Experiment Two. The goals of Experiment Two was to determine if motion dissimilarity judgements have the properties nec-essary to be treated as distances in a metric space. We also set out to compare the psychological motion spaces of the participants to determine if participants with different backgrounds or genders form different spaces. We focused on the psychological motion space by broadening our recruitment of participants and narrowing the range of walking movements. In Experiment Two, we demonstrated that dissimilarity judgements of movements do not have all of the metric properties necessary to be treated as distances. However, they seem to have the symmetry property which means we can greatly reduce the number of trial pairs in future experiments. Using multidimensional scaling we transformed the participants' dissimilarity judgments into configura-tions approximating the geometry of the networks used to create the stimuli. For a motions with only a few principal dimensions of variation we found a strongly correlated coupling between the mechanical motion space and the psychological motion spaces. We also discovered that interpolation of dissimilarity judge-ments does not reflect interpolation of Walke r ' s motion parameters. However the relationship between the dissimilarities and motion parameters is monotonie and smooth. Comparisons of the responses of the groups of participants did not reveal that social dance or running experience greatly influences dissimilarity judgements. Participant gender and effect of walking direction across the screen also failed to produce strong significant effect across our measures. In the introduction we posed two questions the answers to which will guide us towards the construction of higher-level computer animation tools: • How should human movements be presented using computer animation displays so that humans can make judgements and comparisons accurately and reliably? • For computer animation representations of human movements, which relationships exist between the parameters of motion signals, judgements of similarity, and descriptions of motions? To the first question we have provided an experiment design which our analysis confirms will allow re-searchers to collect dissimilarity judgements and descriptive ratings of motions. This sets the stage for principled examination of the perception of human movement and the design of interfaces for higher-level computer animation tools. To the second question we have demonstrated several analysis techniques — principal components analysis, regression, and multidimensional scaling — for determining the parameters of linear relationships between the motion spaces. These techniques were used to confirm our experimen-tal methodology and can be used to analyse human judgements in order to build better human-computer interfaces. We will leave to future research the investigation of how features are combined to form dissimi-larity judgements, the testing of non-linear models of the relationships between the motion spaces, and the selection of optimal motion parameters and descriptive scales for particular tasks. 6.1 Future Interdisciplinary Research Directions In the next section we describe a few applications requiring an understanding of the relationships between the three motion spaces. First we suggest research opportunities outside of the field of computer animation. 6.1.1 Reliability of Movement Observers The medical interpretation of gait analysis data relies heavily on the experience and interpretations of physi-cians. Skaggs et al. (2001) found that physicians at the same institution tend to agree on the interpretation of gait analysis data, but physicians at different institutions differ. Is it possible to build algorithms to deter-mine the "correct" interpretation of a set of gait data? Can we use our techniques to determine the principal dimensions of gait abnormalities which can then be used to train physicians to make more specific and consistent interpretations? 6.1.2 Formation of Movement Dissimilarity Judgements Which motion parameters correspond to perceptually salient features of movements? How are features of movements combined to form dissimilarity judgements? Are different features used by novice and expert observers of movements? How does experience performing the observed movements affect dissimilarity judgements? How are movement observational skills acquired and developed? 6.1.3 Style of Movement How do we define style of movement? Which motion parameters affect style? Is there a parameter which determines if a motion looks like "biological motion" versus "mechanical motion" versus "cartoonish mo-tion"? Why do some people consider mechanical motions un-life-like, motion captured human movement "dead" and cartoonish motion "alive"? What computer vision applications can be created with this knowl-edge? 6.1.4 Experiment Design and Analysis Technique Extensions The walking movements used in our experiments represent only a sample of possible human movements and styles drawn from a small well formed space. Movements that would be more difficult to use with our ex-perimental methodology include non-repetitive movements and movements requiring individual viewpoints. As well, motion parameters would need to be defined — although joint angle ranges may be useful for a variety of movements. However, many possible theories, models, and parameters can be derived and tested with the mechanical motion space we used and then extrapolated to larger spaces. There are a variety of analysis techniques which we have not used to produce models of the relationships be-tween the motion spaces. Because we were specifically interested in validating the experiment methodology we designed our experiments to explore the widest range walking movements possible rather than answer specific questions about the relationship between specific motion parameters and participant responses. As our experimental methodology supports additional analytic techniques, new insights into the perception of human movement could be generated in future experiments that: • use multi-step regression between the motion parameters and the ratings, • compute common principal components analysis of the ratings, • perform factor analysis of the relationship between the motion parameters and the coordinates of the stimuli output by multidimensional scaling, • non-linear non-metric models of movement similarity judgements, • classify the participants based on their dissimilarity judgements, perhaps using cluster analysis or outlier detection. 6.2 Future Research in Computer Animation and Human-Computer Inter-action These are a few of our "blue sky" ideas of what we could do if we understood the relationships between the three motion spaces and could easily translate motion signals into dissimilarity judgements and descriptions. 6.2.1 Maximizing Constrained Interfaces What are the "just noticeable differences" of human movement? How many different human movements are necessary to specify all movements? Given an interface constrained by a physically small device, such as a cell phone, or personal digital assistant, how can we succinctly specify and manipulate human movements? We could start to answer these questions by finding a small number of very wide perceptual dimensions of motion. Along the same lines, Long et al. (2000) presented an experiment methodology for designers of pen-stroke based interfaces to determine if their interaction gestures are easily distinguishable.1 Because Long et al. used static images of completed strokes an obvious extension of their work would be to use judgements of the similarity of gestures made after attempting to reproduce the gestures. 6.2.2 Maximizing Use of Computational Resources Although video game hardware continues to increase in speed and capacity, there continues to be a trade-off between the computational resources dedicated to the "game intelligence" and the computational resources used to create the computer animation display presented to the player. As with level-of-detail rendering which reduces rendering requirements by removing detail from distant objects, similar rules may be appli-cable to the computation of movement. Oesker et al. (2000) reported that the realism of character movements positively affects the judged skill of the characters in a simulated soccer game. However, what their study failed to show was if consistent realism was necessary throughout the team, across the playing field, or for all movements. What trade-offs are possible between the resources dedicated to game intelligence and movement production? It is necessary to realistically animate only the characters the player is most likely attending to and use the saved computational resources to compute more complex reactions to the human player's actions? Can the player be tricked into thinking that the game intelligence is actually more skillful than it really is by increasing the realism of the movements? Answers to these questions will become more necessary as video games move from the display of pre-recorded human movements to the computation of movements based on dynamic and kinematic principles. 'Also Long, Landay and Rowe (1999), Long (2001). 6.2.3 Searching for Movements As it becomes easier to translate video recordings of human movements into motion signals we will have the problem of specifying the movement we would like to retrieve from a database of movements. Rather than depend on descriptions and categorizations of movements, we should be able to specify and refine our searches by using example movements — performed in front of camera or selected from small corpus — to indicate what we are and are not interested in. The problem of sampling a mechanical motion space to produce small demonstrative sets of motions is an in-teresting problem that extends outside of human figure animations. For example, we might want to produce a set distinguishable haptic "icons" for a particular haptic device, or determine the perceptual dimensions of a haptic interface. 6.2.4 Skilled Motor Activity Instruction Many sports and physical activities require extensive investments in time and resources towards receiving instruction, practicing movements, and receiving correction. If computers were given the ability to capture human movements and compare them to a database of known good and bad movements, then computer movement tutors could be created. For example, social dance instruction currently relies on the expertise of the instructor to present dance steps, create choreography (sequences of steps), and provide feedback at a pace that students can understand and retain. The expense of the lesson, travel time, and other commitments usually mean that students take one lesson a week of usually an hour to an hour and a half in length, and engage in no additional practice until the next week's lesson. Imagine that these students have at home a computer, with a video camera, and a database of dance steps which the computer can use to compare the movements the student performs. Given only a small amount of knowledge — say the name of dance step or the name of the dance — the computer can determine if the student is performing the motion in a similar fashion to the stored movements. The computer can evaluate the student's movement and provide feedback, exercises, or computer animations to watch. Because the computer based tutoring lacks the social dimension of dance class and the opportunity to prac-tice leads and follows with another person, we do not propose it as a complete replacement for dance classes but as a tool much more advanced than scribbled dance notes, commercial dance videos, and books of static images of dance steps. This example is extendable to physical training, physical rehabilitation, and many other sports. 6.3 Summary When we set out to begin this research we thought we would immediately develop higher-level character animation tools which would extract a human movement style from one set of motion signals and apply it to another. We quickly backed away from this goal when we realized the lack of research on the relationships between the parameters of movements and the perceptions and judgements that we make about them. We believe that our contribution of an experimental design, analytical methodology, and conclusions from two participant experiments set the ground work for future research. Bibliography Aggarwal, J.K. and Q. Cai (1997). Human Motion Analysis: A Review. In Proceedings of the IEEE Nonrigid and Articulated Motion Workshop, pp. 90-102. June 15-16, San Juan, Puerto Rico. (p25) Allard, Paul, Aurelio Cappozzo, Arne Lundberg, and Christopher L . Vaughan (Editors) (1997). Three-dimensional Analysis of Human Locomotion. John Wiley & Sons. (p23) Allard, Paul, Ian A.F. Stokes, and Jean-Pierre Blanchi (1995). Three-Dimensional Analysis of Human Move-ment. Human Kinetics. Selected papers from an invited symposium on three-dimensional analysis held in Montreal in July, 1991. (p23) Amaya, Kenji, Armin Bruderlin, and Tom Calvert (1996). Emotion from Motion. In Wayne A. Davis and Richard Bartels (Editors), Proceedings of Graphics Interface '96, pp. 222-229. Canadian Information Processing Society, Canadian Human-Computer Communications Society. ISBN 0-9695338-5-3. ( P P28,29) Anderson, T. W. and Stanley L. Sclove (1978). An Introduction to the Statistical Analysis of Data. Houghton Miffline Company, Boston. (p315) Ashby, F. G., W. T. Maddox, and W. W. Lee (1994). On the dangers of averaging across subjects when using multidimensional scaling or the similarity-choice model. Psychological Science, volume 5(3), pp. 144-151. (pl60) Badler, Norman I. and Stephen W. Smoliar (1979). Digital Representations of Human Movement. ACM Computing Surveys, volume 11(1), pp. 19-38. (p 18) Balaguer, Jean-Francis and Enrico Gobbetti (1995). Sketching 3D Animations. Computer Graphics Forum (Proceeding of Eurographics), volume 14(3), pp. C/241-C/258. (p30) h t t p : / /www. c r s 4 . i t / w r / b i b / p a p e r s / e g 9 5 . ps . gz Bartels, Richard H. and Ines Hardtke (1987). Kinetics for Key-Frame Interpolation. Technical Report CS-87-07, Computer Science Department, University of Waterloo. (p41) Bartels, Richard H. and Ines Hardtke (1989). Speed Adjustement for Key-Frame Interpolation. In Proceed-ings of Graphics Interface '89, pp. 14-19. (p41) Bingham, GP, RC Schmidt, and FTJM Zaal (1999). Visual perception of the relative phasing of human limb movements. Perception & Psychophysics, volume 61(2), pp. 246-258. (p41) BioVision (1999). Web Page: h t t p : / / w w w . b i o v i s i o n . c o m / s e r v i c e s / p r i c e . h t m . (p23) BioVision (1999). BioVision Sample Data Files. Available from their website http://www. b i o v i s i o n .com/ samples/data . htm. (p 15) Blake, Randolph (1993). Cats Perceive Biological Motion. Psychological Science, volume 4(1), pp. 54—57. (p38) Blanchet, Pierre J. (1999). Movement Disorders: Features and Diagnosis. The Canadian Journal of CME, volume 11(7), pp. 119-130. (p22) Bodenheimer, Bobby, Charles Rose, Seth Rosenthal, and John Pella (1997). The Process of Motion Capture: Dealing with the Data. In Daniel Thalmann and Michiel van de Panne (Editors), Computer Animation and Simulation '97. Proceedings of the Eurographics Workshop., pp. 3-17. Budapest, Hungary, 1-2 September. (p20) http://www.research.microsoft.com/research/graphics/hfap/cas97 .ps. gz Bodenheimer, Bobby, Anna V Shleyfman, and Jessica K. Hodgins (1999). The Effects of Noise on the Perception of Animated Human Running. In N . Magnenat-Thalmann and D. Thalmann (Editors), Computer Animation and Simulation '99, pp. 53-63. Springer-Verlag, Wien. Eurographics Animation Workshop. (p42) http://www.ce.gatech.edu/~bobbyb/pubs/variability99.pdf Boulic, Ronan, Tom Molet, Tolga Capin, Igor Pandzic, Nadia Magnenat Thalmann, and Daniel Thal-mann (1996). EPFL/University of Geneva: Body Representation Proposal. Technical Report MPEG96/1211, International Organisation for Standardisation, ISO/IEC JTT1/SC29/WG11, Cod-ing of Moving Pictures and Audio, (p 15) http://ligwww.epf1.ch/~boulic/mpeg4_snhc_body/body_prop.html Boyd, Jeffery E. and James Little (1997). Global versus Structured Interpretation of Motion: Moving Light Displays. In Proceedings of IEEE Nonrigid and Articulated Motion Workshop (CVPR-97). San Juan, Puerto Rico, 15-16 June 1997. (p39) http://vision.ucsd.edu/~j effboyd/papers/cvpr97.ps.gz Brotman, L . and A . Netravali (1988). Motion Interpolation by Optimal Control. In Computer Graphics, volume 22(4), pp. 309-315. (P41) Bruderlin, Armin (1995). Procedural Motion Control Techniques for Interactive Animation of Human Fig-ures. Ph.D. thesis, Simon Fraser University, School of Computing Science, (ppxiv, xv, 16, 17, 32, 34, 48,50, 83,361) f t p ://fas.sfu.ca/pub/cs/theses/199 5/ArminBruderlinPhD.ps.Z Bruderlin, Armin and Lance Williams (1995). Motion Signal Processing. In Robert Cook (Editor), Com-puter Graphics (ACM SIGGRAPH 1995 Proceedings), Annual Conference Series, pp. 97-104. A C M SIGGRAPH, Addison Wesley, Los Angeles, California, 6-11 August 1995. (p30) Burtnyk, N . and M . Wein (1976). Interactive Skeleton Techniques for Enhancing Motion Dynamics in Key Frame Animation. Communications of the ACM, volume 19(10), pp. 564-569. (p 14) Calvert, T. W. and J. Chapman (1978). Notation of Movement with Computer Assistance. In ACM 78, Proc. of the Annual Conf, pp. 731-736. (p 18) Cappozzo, Aurelio and John P. Paul (1997). Instrumental Observation of Human Movement: Historical Development. In Paul Allard, Aurelio Cappozzo, Arne Lundberg, and Christopher L. Vaughan (Ed-itors), Three-dimensional Analysis of Human Locomotion, chapter 1, pp. 1-25. John Wiley & Sons, Ltd. (p23) Carroll, J. Douglas and Phipps Arabie (1998). Multidimensional Scaling. In Measurement, Judgement, and Decision Making. Academic Press, (p 136) Cédras, Claudette and Mubarak Shah (1995). Motion-based recognition: a survey. Image and Vision Com-puting, volume 13(2), pp. 129-155. (p25) Cédras, Claudette and Mubark Shah (1994). A Survey of Motion Analysis from Moving Light Displays. In CVPR94, pp. 214-221. (P39) Chadwick, John E., David R. Haumann, and Richard E. Parent (1989). Layered Construction for Deformable Animated Characters. In Computer Graphics (ACM SIGGRAPH 1989 Proceedings), pp. 243-252. Published as V23(4) of Computer Graphics, (p 16) Cheung, Sen-ching S. and Avideh Zakhor (2000). Efficient Video Similarity Measurement and Search. In Proceedings 7th IEEE International Conference on Image Processing, volume 1, pp. 85-89. Vancouver, B.C. Canada. Demonstration at h t t p : / / w w w - v i d e o . e e c s . b e r k e l e y . e d u / ~ c h e u n g s c / c l u s t e r / s e a r c h . h t m l More information at h t t p : / / w w w - v i d e o . e e c s . b e r k e l e y . e d u / ~ c h e u n g s c / w o r k / d o c s / i l p 0 1 . h t m l . (p32) h t t p : / / w w w - v i d e o . e e c s . b e r k e l e y . e d u / " c h e u n g s c / w o r k / d o c s / i c i p O O . p d f Chi, Diane (1999). A Motion Control Scheme for Animating Expressive Arm Movement. Ph.D. thesis, University of Pennsylvania, Department of Computer & Information Science. Also published at IRCS-99-06. (pp 18, 19,29) f t p : / / f t p . c i s . u p e n n . e d u / p u b / i r e s / t r / 9 9 - 0 6/ Chi, Diane M . , Norman I. Badler, and Janis Pforisch (1999). Animating Expressivity Through Effort Ele-ments. In Siggraph'99 Conference Abstracts and Applications, p. 231. (pp 19, 29) Cohen, Leslie, Thomas F. Shipley, Eve Marshark, Kathy Taht, and Denise Aster (2000). Detecting Animals in Point-Light Displays. In Proceedings of the Twenty Second Annual Meeting of the Cognitive Science Society. Philadelphia, Pennsylvania, USA, August 13-15. Poster 1019. (p38) h t t p : / / w w w . c i s . u p e n n . e d u / ~ i r c s / c o g s c i 2 0 00/PRCDNGS/SPRCDNGS/ a b s t r c t s / c o h s m t a . p d f Craik, Rebecca L . and Carol A. Oatis (Editors) (1995). Gait Analysis: Theory and Applications. Mosby. (p23) Credo Interactive (2001). Web Page: h t t p : / / w w w . c r e d o - i n t e r a c t i v e . c o m / . (pp 18, 32) Cutting, J (1978). A program to generate synthetic walkers as dynamic point-like displays. Behavior Research Methods & Instrumentation, volume 10, pp. 91-94. Fortran code plus brief description. (p38) Cutting, James E. and Lynn T. Kozlowski (1977). Recognizing friends by their walk: Gait perception without familiarity cues. Bulletin of the Psychonomic Society, volume 9(5), pp. 353-356. (p38) Dance Notation Bureau (2001). Phone: 212-564-0985, Fax: 212-904-1426. (p 18) h t t p : / / w w w . d a n c e n o t a t i o n . o r g / Davison, Mark L. (1983). Multidimesional Scaling. John Wiley & Sons. (pp74, 136, 151) Deutscher, Jonathan, Andrew Blake, and Ian Reid (2000). Articulated Body Motion Capture by Annealed Particle Filtering. In Proceedings CVPR'2000. (p25) http://www.robots.ox.ac.uk/~vdg/abstracts/cvpr2000-j deutsch.html Dittrich, Winand H. , Tom Troscianko, Stepen E. G. Lea, and Dawn Morgan (1996). Perception of emotion from dynamic point-light displays represented in dance. Perception, volume 25(6), pp. 727-738. (p38) Dransch, Detlef O. K. (1985). ChoreoScribe: A Graphics Editor to Describe Body Position and Movement Using Benesh Movement Notation. Master's thesis, Department of Computer Science, University of Waterloo, (p 18) Family Caregiver Alliance (1996). Fact Sheet: Huntington's Disease. Web Page: http://www. c a r e g i v e r . org/ f actsheets/huntingtons . html. (p22) Fleischer, Max (1917). Method of Producting Moving Picture Cartoons. US Patent #1,242,674. (p25) Flury, Bernard (1988). Common Principal Components and Related Multivariate Models. John Wiley & Sons, New York. (P98) Foley, James D., Andries van Dam, Steven K. Feiner, and John F. Hughes (1990). Computer Graphics: Principles and Practice, Second Edition. Addison-Wesley. (pp362, 363) Gage, James R. (1991). Gait Analysis in Cerebral Palsy (Clinics in Developmental Medicine No. 121). Mac Keith Press. (pp22, 24) Gleicher, Michael (1997). Motion Editing with Spacetime Constraints. In Proceedings of the 1997 Sympo-sium on Interactive 3D Graphics, pp. 139-148. (p 31) http://www.gleicher.com/mike/Papers/SpacetimeEditing.ps Gleicher, Michael (1998). Retargetting Motion to New Characters. In Computer Graphics (ACM SIG-GRAPH 1998 Proceedings), (p 31) Gleicher, Michael and Peter Litwinowicz (1996). Constraint-Based Motion Adaptation. Technical Report Apple TR 96-153, Apple Computer, Advance Technology Group. Also Journal of Visualization and Computer Animation, 9, 65-94 (1998). (p31) Gleicher, Michael L . (2001). Motion Path Editing. In Proceedings of the 2001 ACM Symposium on Inter-active 3D Graphics, (p 31) http://www.cs.wise.edu/graphics/Papers/Gleicher/Mocap/path.pdf Guenter, Brian, Charles F. Rose, Bobby Bodenheimer, and Michael F. Cohen (1996). Efficient Generation of Motion Transitions using Spacetime Constraints. In Holly Rushmeier (Editor), Computer Graphics (ACM SIGGRAPH 1996 Proceedings), Annual Conference Series, pp. 147-154. A C M SIGGRAPH, Addison Wesley, New Orleans, Louisiana, (p 31) Hodgins, Jessica K., James F. O'Brien, and Jack Tumblin (1997). Do Geometric Models Affect Judgments of Human Motion? In Proceedings of Graphics Interface '97, pp. 17-25. Kelowna, B.C., Canada. (ppl6,42,44) http://www.cc.gatech.edu/gvu/animâtion/papers/rencmp.ps.Z Hodgins, Jessica K., James F. O'Brien, and Jack Tumblin (1998). Perception of Human Motion with Dif-ferent Geometric Models. IEEE Transactions on Vision and Computer Graphics, volume 4(4), pp. 101-113. (pp16,42,44) h t t p : / / w w w . g v u . g a t e c h . e d u / a n i m â t i o n / P a p e r s / h o d g i n s : 199 8 : J H M . p d f Hodgins, Jessica K., Wayne L . Wooten, David C. Brogan, and James F. O'Brien (1995). Animating Hu-man Athletics. In Robert Cook (Editor), Computer Graphics (ACM SIGGRAPH 1995 Proceedings), Annual Conference Series, pp. 71-78. A C M SIGGRAPH, Addison Wesley, Los Angeles, California, 06-11 August 1995. (p42) Hutchinson, Ann (1970). Labanotation. 2nd edition. Theatre Arts Books, (p 18) Hutchinson Guest, Ann (1984). Dance Notation: The Process of Recording Movement on Paper. Dance Books, London, (p 17) Hutchinson Guest, Ann (1989). Choreo-Graphics: A Comparison of Dance Notation Systems from the Fifteenth Century to the Present. Gordon and Breach, New York, (p 17) Ille, Anne and Marielle Cadopi (1995). Reproducing choreographic walks from the observation of a kine-matic point-light display: Effect of skill level. Journal of Human Movement Studies, volume 29(3), pp. 101-114. (p38) Jack (2001). Jack: The Human Modeling and Simulation System. Web Page: h t t p : / / w w w . c i s . u p e n n . e d u / \ ~ {}hms/ j a c k . h t m l . Center for Human Modeling and Simulation, Department of Computer and Information Science, University of Pennsylvania, (p 18) Jackson, J. Edward (1991). A User's Guide to Principal Components. John Wiley & Sons, Inc. (pp70, 74, 98, 176) Jackson, Michael (1997). Blood on the Dance Floor: HIStory in the Mix. Sony/Columbia. UPC: 98707800726. (P21) Jacobs, W. J[ake], James R. Blackburn, M . Buttrick, Timorthy J. Harpur, D. Kennedy, Michael J. Mana, M . A MacDonald, Leslie M . McPherson, Dennis Paul, and James G. Pfaus (1988). Observations. Psychobiology, volume 16(1), pp. 3-19. (p 19) Jetha, Zeenat, Armin Bruderlin, Tom W. Calvert, and Sang Mah (1993). On the Edge of the Creative Process: An Analysis of Human Figure Animation as a Complex Synthesis Task. In Proceedings of ACM INTERCHI'93 Conference on Human Factors in Computing Systems - Adjunct Proceedings, Short Papers (Posters): Designing for Extra-ordinary Users and Uses, pp. 19-20. (p 18) Johansson, Gunnar (1973). Visual Perception of Biological Motion and a Model for its Analysis. Perception and Psychophysics, volume 14(2), pp. 201-2111. (p 37) Johansson, Gunnar (1975). Visual Motion Perception. Scientific American, volume 232(6), pp. 76-88. (P37) Keramidas, E. M . , S. J. Devlin, and R[amanathan] Gnanadesikan (1987). A Graphical Procedure for Com-paring the Principal Components of Several Covariance Matrices. Communications in Statistics, Part B - Simulation and Computation, volume 16(1), pp. 161-191. (p98) Kines, Melianthe (1998a). Directing a Motion Capture Shoot. Game Developer. (pp20, 32) Kines, Melianthe (1998b). Planning a Motion Capture Shoot. Game Developer. (pp20, 32, 33) Kochanek, Doris H. U . and Richard H . Battels (1984). Interpolating Splines with Local Tension, Conti-nuity, and Bias Control. In H . Christiansen (Editor), Computer Graphics (ACM SIGGRAPH 1984 Proceedings), pp. 33^13. A C M , Minneapolis, M N . (p41) Kourtzi, Zoe and Maggie Shiffrar (1999). Dynamic representations of human body movement. Perception, volume 28(2), pp. 49-62. (pp 16,40) Kozlowski, Lynn T. and James E. Cutting (1977). Recognizing the sex of a walker from a dynamic point-light display. Perception & Psychophysics, volume 21(6), pp. 575-580. (p38) Lamouret, Alexis and Michiel van de Panne (1996). Motion Synthesis by Example. In Ronan Boulic and G. Hegron (Editors), 7th Eurographics Workshop on Animation and Simulation. (pp35, 37) h t t p : / / w 3 i m a g i s . i m a g . f r / ~ A l e x i s . L a m o u r e t /PS / m s b e . p s . g z Larsen, Richard J. and Morris L . Marx (1981). An Introduction to Mathematical Statistics and Its Applica-tions. Prentice-Hall, Inc., Englewood Clifs, N.J. (pl50) Leventon, Michael E. and William T. Freeman (1998). Bayesian estimation of 3-d human motion from an image sequence. Technical Report 98-06, M E R L - A Mitsubishi Electric Research Laboratory, (p 16) h t t p ://www .me r1 . com/ repo r t s/TR98-06 / i ndex .h tml Little, Jim and Jeffery E. Boyd (1995). Describing Motion For Recognition. In Proceedings of the Interna-tional Symposium and Computer Vision, pp. 235-240. Coral Gables, Florida, 21-23 November 1995. (p39) h t t p : / / w w w . c s . u b c . c a / s p i d e r / l i t t l e / l i n k s / p a p e r s / s c v 9 5 . p s Little, Jim and Jeffery E. Boyd (1998). Recognizing People by Their Gait: the Shape of Motion. Videre, volume 1(2), p. 32. (p39) h t t p : / / m i t p r e s s . m i t . e d u / e - j o u r n a l s/VIDE/0 01 / a r t i c l e s / v l n 2001.pdf Long, Allan Christan, Jr. (2001). Quill: a Gesture Design Tool for Pen-based User Interfaces. Ph.D. thesis, University of California, Berkeley, Computer Science. (p200) h t t p ://www. c s . b e r k e l e y . e d u / ~ a l l a n l/2 0 00 / w o r k / p u b s / d i s s e r t a t i o n.pd%f Long, Allan Christian, Jr., James A . Landay, and Lawrence A. Rowe (1999). Implications for a gesture design tool. In Proceeding of Human Factors in Computer Systems (SIGCHI). (p200) h t t p ://www . c s . b e r k e l e y . e d u / %7E a l l a n l/2000 / w o r k / p u b s / p a p e r s / c h i 9 9 / c h i 9 9 . p d f Long, Jr., A[llan] Chris[tian], James A . Landay, Lawrence A. Rowe, and Joseph Michiels (2000). Visual Similarity of Pen Gestures. In Human Factors in Computing Systems conference (CHI'2000). April 1-6, The Hague, Netherlands. (P200) h t t p ://www . c s . b e r k e l e y . e d u / " a l l a n l / r e s e a r c h / p a p e r s / c h i O O / Loomis, Jeffery, Howard Poizner, Ursula Bellugi, and John Hollerbach (1983). Computer Graphic Modeling of American Sign Language. In Computer Graphics (ACM SIGGRAPH 1983 Proceedings), pp. 105-114. Published as V17(3) of Computer Graphics. (p28) Mak, Benise S. K. and Alonso H . Vera (1999). The Role of Motion in Children's Categorization of Objects. Cognition, volume 71(1), pp. B11-21. (p40) h t t p : / / p r o d . l i b r a r y . u t o r o n t o . c a : 9 5 0 1 / c g i - b i n / m e d i a . c g i / d a t a b a s e = journals/doc_id=/00100277/v0071i01/99000190/type=application/pdf/ h a n d l e = 0 0 0 0 0 9 2 8 . p d f / 9 9 0 0 0 1 9 0 . p d f • Manly, Bryan F.J. (1994). Multivariate Statistical Methods: A Primer, Second Edition. Chapman & Hall. (pp70, 318, 320, 321) Marks, J., B. Andalman, P.A. Beardsley, W. Freeman, S. Gibson, J. Hodgins, T. Kang, B. Mirtich, H . Pfister, W. Ruml, K. Ryall, J. Seims, and S. Shieber (1997). Design Galleries: A General Approach to Setting Parameters for Computer Graphics and Animation. In Turner Whitted (Editor), Computer Graphics (ACM SIGGRAPH 1997 Proceedings), pp. 389^100. A C M SIGGRAPH. (pp35, 36, 84) h t t p : / / w w w . g v u . g a t e c h . e d u / g v u / a n i m â t i o n / p a p e r s / d e s i g n _ g a l 1 e r i e s . p s . gz Martin, Paul and Patrick Bateson (1993). Measuring Behaviour: An introductory guide. Second edition. Cambridge University Press, (p 315) Math Works (2001). Statistics Toolbox: User's Guide Version 3. The Math Works. For use with Matlab version 6 (build 12). (p323) h t t p : / / w w w . m a t h w o r k s . c o m / a c c e s s / h e l p d e s k / h e l p / p d f _ d o c / s t a t s / s t a t s _ t b . p d f Mather, George, Kirstyn Radford, and Sophie West (1992). Low-Level Visual Processing of Biological Motion. Proceedings of the Royal Society of London Series B-Biological Sciences, volume 249, pp. 149-155. (P38) Mather, George and Sophie West (1993). Recognition of animal locomotion from dynamic point-light displays. Perception, volume 22(7), pp. 759-766. (p38) McClay, Irene S. (1995). The Use of Gait Analysis to Enhance the Understanding of Running Injuries. In Rebecca L. Craik and Carol A . Oatis (Editors), Gait Analysis: Theory and Applications, chapter 29, pp. 395^111. Mosby-Year Book, St Louis, Missouri. (p22) Menache, Alberto (1999). Understanding Motion Capture of Computer Animation and Video Games. Mor-gan Kaufmann Publishers. (p20) Miller, George A. and Philip N . Johnson-Laird (1976). Language and Perception. Belknap Press of Harvard University Press (Cambridge, Massachusetts), (p 87) Moore, Carol-Lynne and Kaoru Yamamoto (1988). Beyond Words: Movement Observation and Analysis. Gordon and Breach. (p87) Motek (2001). Motek. Web Page: h t t p : / /www. e -mo tek . com. (p 32) Mulder, Axel (1994). Human Movement Tracking Technology. Hand Centered Studies of Human Movement Project 94-1, Simon Fraser University, School of Kinesiology. (p20) h t t p : / / w w w . c s . s f u . c a / ~ a m u l d e r / p e r s o n a l / v m i / H M T T . p u b . h t m l Muybridge, Eadweard (1955). The Human Figure in Motion. Dover Publications. (p21) Muybridge, Eadweard (1957). Animals in Motion. An electro photographic investigation of consecutive phases of muscular actions. Dover Publications, (p 21) Neter, John and William Wasserman (1974). Applied Linear Statistical Models: Regression, Analysis of Variance, and Experimental Designs. Richard D. Irwin, Homewood, Illinois, (pp 150, 316) Norusis, Marija J. (Editor) (1994). SPSS Professional Statistics 6.1. SPSS Inc., Chicago, Illinois. The algorithms in the SPSS Professional Statistics 6.1 software package are now included with the SPSS Base 10.1 software. Unfortunately, details of the algorithms are not discussed in the SPSS Base 10.1 manuals. (pp74, 160) Oesker, Markus, Heiko Hecht, and Bernhard Jung (2000). Psychological Evidence for Unconscious Processing of Detail in Real-time Animation of Multiple Characters. Journal of Visualization and Computer Animation, volume 11(2), pp. 105—112. (p200) h t t p : / / w w w 3 . i n t e r s c i e n c e . w i l e y . c o m / c g i - b i n / f u l l t e x t ? I D = 7 2 504117&PLACEBO=IE.pdf Ollis, Mark and Todd Williamson (2001). The Future of 3D Video. IEEE Computer, volume 34(6), pp. 97-99. h t t p : / /www. z a x e l . com. (p 13) http://zaxel . c o m / p r e s s r o o m/Zaxel-Future_of _ 3 D_Video.pdf Overmars, Mark H. (1992). Forms Library: A Graphical User Interface Toolkit for Silicon Graphics Work-stations. Originally distributed via FTP from ftp://ftp.cs.ruu.nl/pub/SGI/FORMS/. Version 2.1. Superseded by XForms (http: / / w o r l d , std . c o m / ~ x f o r m s / ) . (p64) Owens, D.A., R.J. Antonoff, and E.L. Francis (1994). Biological Motion and Nighttime Pedestrian Con-spicuousness. HUMAN FACTORS, volume 36(4), pp. 718-732. (p38) Paterson, Helena M . and Frank E. Pollick (2001). Form and Animacy in the Perception of Affect from Biological Motion. Poster at Vision Sciences Society First Annual Meeting. Available from the author: h t t p : / / s t a f f . p s y . g l a . ac . uk/ ~helena/. (pp29, 38) Paterson, Helena M . , Frank E. Pollick, and Anthony J. Sanford (2000). The Role of Velocity in Affect Discrimination. In Proceedings of the Twenty-third Annual Conference of the Coginitive Science Society, (pp29, 38, 111) Peak Performance Technologies (2001). Peak Performance Technologies, Inc. Web Page: h t t p : / /www. p e a k p e r f o r m . c o m / . (p25) Perlin, Ken (1995). Real Time Responsive Animation with Personality. IEEE Transactions on Visualization and Computer Graphics, volume 1(1), pp. 5-15. (pp28, 29) h t t p : / / m r l . n y u . e d u / i m p r o v / p e r l i n e . i e e e . t a r . g z Perry, Jacquelin (1992). Gait Analysis: Normal and Pathological Function. S L A C K Incorporated. (p23) Phillips, Cary B. and Norman I. Badler (1991). Interactive Behaviors for Bipedal Articulated Figures. In Thomas W. Sederberg (Editor), Computer Graphics, pp. 359-362. ISSN 0097-8930. Published as V25(4) of Computer Graphics, (p 29) Pollick, Frank E., Helena M . Paterson, Armin Bruderlin, and Anthony J. Sanford (2001). Perceiving affect from arm movement. Cognition, volume 82(2), pp. B51-61. (p29) Popovic, Zoran (1999). Motion Transformation by Physically Based Spacetime Optimization. Ph.D. thesis, Carnegie Mellon University, School of Computer Science, (p 31) Popovic, Zoran and Andrew Witkin (1999). Physically Based Motion Transformation. In Alyn Rockwood (Editor), Computer Graphics (ACM SIGGRAPH 1999 Proceedings), pp. 11-20. A C M SIGGRAPH, Los Angelas, California, August 8-13. (p 31) Poynton, Charles (2000). Frequently Asked Questions about Color. Web Page: h t t p : / / w w w . i n f oramp . n e t / \ ~ { J p o y n t o n / C o l o r F A Q . h t m l . (p362) Richards, James G. (1998). The Measurement of Human Motion: A Comparison of Commercially Avail-able Systems. In Proceedings of the Fifth International Symposium on the 3-D Analysis of Human Movement. (p25) > h t t p : / / w w w . a r i e l n e t . c o m / t o p i c s / c o m p a r i s o n Rose, Charles, Michael F. Cohen, and Bobby Bodenheimer (1998). Verbs and Adverbs: Multidimensional Motion Interpolation. IEEE Computer Graphics and Applications, volume 18(5), pp. 32-40. (p28) Saleh, Michael and George Murdoch