UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Incorporating affect into the design of 1-D rotary physical controls Swindells, Colin Edward 2007

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata

Download

Media
831-ubc_2007-268063.pdf [ 27.19MB ]
Metadata
JSON: 831-1.0052050.json
JSON-LD: 831-1.0052050-ld.json
RDF/XML (Pretty): 831-1.0052050-rdf.xml
RDF/JSON: 831-1.0052050-rdf.json
Turtle: 831-1.0052050-turtle.txt
N-Triples: 831-1.0052050-rdf-ntriples.txt
Original Record: 831-1.0052050-source.json
Full Text
831-1.0052050-fulltext.txt
Citation
831-1.0052050.ris

Full Text

INCORPORATING AFFECT INTO THE DESIGN OF 1-D ROTARY PHYSICAL CONTROLS by COLIN EDWARD SWINDELLS B.A.Sc. (Hons.), Simon Fraser University, 2000 M.Sc, Simon Fraser University, 2002  A THESIS SUBMITTED IN PARTIAL FULFILMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY  m THE FACULTY OF GRADUATE STUDIES (Computer Science)  THE UNIVERSITY OF BRITISH COLUMBIA 7 M A R C H 2007 © Colin Edward Swindells, 2007  Abstract  The visceral emotional reactions that users have to technologies is increasingly understood to be important in terms of safety, performance, and pleasure in its own right. This thesis systematically explores users's emotional (affect) reactions to everyday physical manual controls, in order to inform a design process that considers appropriate affective response as well as performance relationships. Design of both mechanical and emerging mechatronic physical controls are addressed. This novel design process includes parameterizing second order (inertial) dynamics using a system identification technique, and rendering models on a custom force-feedback knob. Next, this thesis explores biometric and self-reported measures of the affective responses elicited by these dynamics, and an iterative prototyping tool for rapid refinement of the "feel" of physical controls. This research impacts use of the passive physical interfaces such as mechanical knobs and sliders that are already ubiquitous in our everyday environments, as well as the active physical controls that are emerging in embedded computing environments such as cars, games, and medical devices.  ii  Table of C o n t e n t s  Abstract  ii  Table of Contents  iii  List of Tables  vii  List of Figures  viii  Symbols  xii  Abbreviations  xiv  Acknowledgements  xv  0 Introduction  1  0.1 0.2 0.3 0.4 0.5  Motivation for Affective Design of Physical Controls Motivation for Improving Physical Control Design Current Design of Physical Control Dynamics Approach Presented in this Thesis Examples of Application Contexts 0.5.1 Game Character Control 0.5.2 Manual Control of Complex Systems 0.5.3 Media Manipulation 0.6 Publication Listing  2 2 4 5 •. 7 7 9 9 10  1 Haptic Camera 1.1 1.2  1.3 1.4 1.5 1.6  Introduction Sensors 1.2.1 Position Sensing 1.2.2 Velocity Sensing 1.2.3 Acceleration Sensing and Gravity Compensation 1.2.4 Torque Sensing Actuator Gripper Mounting Interfacing to a Computer  12  ,  12 14 14 14 15 17 18 18 20 21  iii  1.7 1.8  PID controller Haptic Camera Summary  r.  23 26  2 Characterizing Haptic Environments 2.1  2.2  2.3  2.4  2.5  Knob Model 29 2.1.1 Selected Previous Characterization Work. 33 2.1.2 Validating the Characterization using Simulated Data 34 2.1.2.1 Step 1: Generating Test Spatial Data 35 2.1.2.2 Step 2: Choosing Model Parameters 36 2.1.2.3 Step 3: Generating Test Torque Data 37 2.1.2.4 Step 4: Fitting to the Test Data 38 2.1.2.5 Step 5: Comparing Commanded and Fitted D a t a . . . . . . 39 Characterizing Mechanical Test Knobs 44 2.2.1 Sensor Verification and Calibration 44 2.2.1.1 Spatial Test Results 45 2.2.1.2 Torque Test Results 46 2.2.1.3 Test Results 47 2.2.2 Capture of Five Knobs 48 2.2.2.1 Data Collection 49 2.2.2.2 Data Preparation 49 2.2.2.3 Data Results 49 2.2.2.4 Summary 67 Comments on the Characterization Process 70 2.3.1 Human versus Machine Characterization Strengths 71 2.3.2 Perceptibility of Specific Phenomena 71 2.3.3 Fusing Multiple Characterization Techniques 72 2.3.4 Dynamic Range of Sensors 74 2.3.5 Dividing a Complex Problem into Simpler Sub-Problems... 74 2.3.6 Choosing Appropriate Model Complexity 74 2.3.7 Dealing with Practical Data Capture Issues 75 Noise Source Summary 76 2.4.1 Mechanical Improvements .76 2.4.2 Electrical Improvements 77 2.4.3 Algorithmic Improvements 77 Characterization Summary 78  3 Rendering and Validation 3.1 3.2 3.3  28  —  80  Introduction 80 Balancing Machine and Human Capabilities 81 Rendering Model 84 3.3.1 Psychophysical Appropriateness of the Rendered M o d e l . . . 86 3.3.2 Maximum Rendering Capabilities of the Haptic K n o b . . . . . . 87 3.4 Haptic Matching Experiments 88 3.4.1 Method 89 3.4.1.1 Participants: Novices and Experts 89  3.4.1.2 Apparatus 90 3.4.1.3 Procedure ' 93 3.4.2 Results 95 3.4.3 Discussion 103 3.4.3.1 Human vs. Machine Performance 103 3.4.3.2 Sticky Notes from Novice Participants 106 3.4.3.3 Discussion of Field Notes from Expert Participants... 107 3.4.4 Summary of Human vs. Machine Performance Ill 3.5 Future Rendering Work Ill 3.5.1 Technical Enhancements Ill 3.5.2 User Study Enhancements 112  4 Measuring Affect 4.1  116  Introduction 4.1.1 Motivation 4.1.2 The Impact of Affective Response to Haptic Stimuli Related Affect Research 4.2.1 Measuring the Feel of Haptic Knobs 4.2.2 Dimensions of Affect 4.2.3 Performance Trade-offs and Design Implications 4.2.4 Practical Issues in Collecting Affect Measurements 4.2.5 Measuring Affect with Self Reports 4.2.6 Measuring Affect with Biometric Sensors Overview of Haptic Affect Study Series Study 1: Foundation Tests of Self Reports and Biometrics 4.4.1 Method 4.4.1.1 Participants 4.4.1.2 Apparatus 4.4.1.3 Procedure . . . : 4.4.2 Results 4.4.3 Discussion 4.4.4 Study 1 Conclusions Study 2: Rendered Interaction using List Selection 4.5.1 Method 4.5.1.1 Participants 4.5.1.2 Apparatus 4.5.1.3 Procedure 4.5.2 Results 4.5.3 Discussion 4.5.4 Study 2 Conclusions Study 3: Rendered Interaction using Rotary Pointing 4.6.1 Method 4.6.1.1 Participants 4.6.1.2 Apparatus 4.6.1.3 Procedure t  4.2  4.3 4.4  4.5  4.6  116 118 119 120 120 121 123 123 124 126 128 131 131 131 132 133 135 139 140 141 ,142 142 142 144 .147 151 152 153 .153 154 154 159  v  4.6.2 Results 4.6.3 Discussion 4.6.4 Study 3 Conclusions 4.7 Conclusions and Future Work for Affect User Studies  5 Haptic Icon Prototyper 5.1 5.2  5.3  5.4 5.5  5.6  Introduction and Related Work Example Prototyping Tool and Rendering Setup 5.2.1 Haptic Icon Prototyper Summary 5.2.2 Mathematical Representations 5.2.3 Rendering the Prototypes using a Force Feedback K n o b . . . Task Example: Prototyping a Fan Knob 5.3.1 Primitive Waveform Editing 5.3.1.1 Position Tiles. 5.3.1.2 Velocity Tiles 5.3.1.3 Acceleration Tiles 5.3.2 Organizing Haptic Behavior Primitives . . Supporting Iterative Design General Design Principles 5.5.1 Necessary Attributes of Tools to Support Haptic Behavior Design 5.5.1.1 Scope and General Capabilities 5.5.1.2 Usability 5.5.1.3 Representations 5.5.2 Accommodating Different Types of Users Conclusions and Future Work for Rapid Prototyping  6 Conclusion 6.1 6.2  Primary contributions Future Work  161 169 173 174  177 178 180 181 182 183 184 185 185 188 189 189 191 195 196 196 196 197 197 198  200 201 204  References  207  Appendix A Ethics Forms  217  Certificate of Approval Amendment Form for an Approved Project Consent Form  Appendix B Defense Programme  218 219 221  224  vi  List of T a b l e s  Table 1. Haptic camera sensor resolutions 14 Table 2. Modeled Results for noisy simulated data ......39 Table 3. Swept sine constants for sensor tests using Equation 8 45 Table 4. Means and standard deviations of spatial and torque differences between commanded and recorded signals 47 Table 5. Intuitive descriptions for five mechanical test knobs 48 Table 6. Low pass filter stopband edge frequencies 49 Table 7. Two independent sets of fitted parameters for all knobs 51 Table 8. Periodicity check for test knobs with detents 73 Table 9. Maximum continuous torque responses of the haptic knob 87 Table 10. Axial and radial play of the haptic knob at the handle 87 Table 11. Slider value ranges 90 Table 12. Physical units that map to standardized slider values of 0 -1 (for use in reading Figure 41) 97 Table 13. Comparison of Haptic Camera and Human Expert Dynamic Property Estimates (units are in mNm, rad, and s for torque, angle, and time, respectively) 100 Table 14. Comparison of detent estimates for knobs with detents 101 Table 15. Participant tags for dynamic knob properties 102 Table 16. Tactile Stimuli for Study 1.... 132 Table 17. Average values across all 12 tactile stimuli.. 138 Table 18. Force-Feedback Models Used in Study 2 (and Study 3) 143 Table 19. Seven Dynamic Knob Stimuli for Study 2 146 Table 20. Seven Dynamic Knob Stimuli for Study 3 157 Table 21. Non-parametric correlations grouped by time 162 Table 22. Pairwise comparisons of time for selected knob renderings 163 Table 23. Non-parametric correlations grouped by rating 165 Table 24. Non-parametric correlations grouped by SC 165 Table 25. Non-parametric correlations grouped by EMG 165 Table 26. Pairwise comparisons of rating for selected knob renderings 166 Table 27. Variance of Movement Times for each Index of Difficulty for All Knobs : ....169  vii  List of F i g u r e s  Figure 1. Process for affective physical control design Figure 2. Application scenarios - Different knob dynamics could elicit appropriate user responses for changing (a) character actions in a video game, (b) boiler settings in a power plant, and (c) slider bar settings in an on-line search tool. ; Figure 3. Rotary Haptic Camera Components — Exploded View Figure 4. Rotary Haptic Camera Components — Operational View Figure 5. Accelerometer housing (dimensions in mm) Figure 6. Example gravity compensation for accelerometer Figure 7. Custom amplifier for torque sensor Figure 8. Example knob grippers Figure 9. a) zoom of rotary Haptic Camera during capture of a 34 mm diameter stereo volume knob, and b) overview of rotary Haptic Camera during capture of a 51 mm automobile fan knob Figure 10. Simplified block diagram of the Haptic Camera's closed loop feedback system.... Figure 11. Open loop and closed loop root loci Figure 12. Open loop and closed loop step responses Figure 13. Karnopp Friction Model •. Figure 14. Detent Position Model Figure 15. Simulated position, velocity, and acceleration curves Figure 16. Simulated torques without (top) and with (bottom) noise Figure 17. Fitted torque vs. position plots with (top) and without (bottom) velocity and acceleration components of simulated data with noise added Figure 18. Fitted torque vs. velocity plots with (top) and without (bottom) position and acceleration components of simulated data with noise added Figure 19. Zooms of fitted torque vs. position (top) and torque vs. velocity plots (bottom) illustrating the very small 95% confidence intervals Figure 20. Commanded (thick gray) and measured (thin colored) position (top), velocity (middle), and acceleration (bottom) values from a x single commanded swept sine signal Figure 21. Commanded (thick gray) and measured (thin colored) torque from a single commanded swept sine signal Figure 22. Torque vs. velocity plots for K n o b friction f ° ^ torques (top) and with position and acceleration components removed  5  .8 13 13 16 17 18 19 22 24 25 26 31 32 36 38 41 42 43 46 47  r a  high  viii  (bottom) Figure 23. Torque vs. velocity plots for K n o b for all torques (top) and with position and acceleration components removed (bottom) Figure 24. Zooms of torque vs. velocity plots for Knobhigh friction ( P) * Knob (bottom) Figure 25. Torque vs. position plots for K n o b i detents f ° " torques (top) and with velocity and acceleration components removed (bottom) .... Figure 26. Torque vs. velocity plots for K n o b detents f ° ^ torques (top) and with position and acceleration components removed (bottom) Figure 27. Zooms of torque vs. position (top) and torque vs. velocity plots (bottom) for K n o b t Figure 28. Torque vs. position plots for K n o b detents f ° ^ torques (top) and with velocity and acceleration components removed ' (bottom) Figure 29. Torque vs. velocity plots for K n o b detents f ° * torques (top) and with position and acceleration components removed (bottom) Figure 30. Zooms of torque vs. position (top) and torque vs. velocity plots (bottom) for K n o b  54  high i n e r t i a  t0  high i n e r t i a  55  a n c  56  rau  subt  e  57  ra  subtle  subtle d e t e n  s  58 59  ra  m o d e r a t e  60  r au  m o d e r a t e  61  m o d e r a t e  detents  v z  -  Figure 31. Torque vs. position plots for K n o b . detents f ° ^ torques (top) and with velocity and acceleration components removed (bottom) - 1 test 63 Figure 32. Torque vs. position plots for Knob _ detents f ° ^ torques (top) and with velocity and acceleration components removed (bottom) - 2 test 64 Figure 33. Torque vs. velocity plots for K n o b . detents for all torques (top) and with position and acceleration components removed (bottom) 65 Figure 34. Zooms of torque, vs. position (top) and torque vs. velocity plots (bottom) for K n o b . 66 Figure 35. Example static friction parameter estimation on the torque vs. velocity data from K n o b Figure 36. Photo of K n o b . t s revealing several hints for an appropriate detent model (e.g., 12 "clicks" per revolution and non-sinusoid grooves) 73 Figure 37. Rendered haptic knob apparatus 81 Figure 38. Haptic knob rendering model 85 Figure 39. Apparatus for human system identification 91 Figure 40. Novice ratings of satisfaction for how closely each rendered knob matched itsoftarget mechanical test knob Camera parameteriza-96 Figure 41. Comparisons expert, novice, and Haptic r a  n o n  s i n e  st  ra  non  sine  n d  n o n  n o n  s i n e  s i n e d e t e n t s  subtle d e t e n t s  n o n  7 0  s i n e d e t e n  tions for all five test knobs. Human parameterizations were performed in clusters, setting only the parameters present in  ix  those knobs 99 Figure 42. The affect grid. After exposure to a stimulus, participants place an "x" in a box to self-evaluate their level of valence and arousal (based on Russell et al. [79]) ....126 Figure 43. EMG electrode placement on the forehead, and SC electrode placement on the index and middle fingers of the non-dominant (left) hand. .' .128 Figure 44. Method for estimating a single biometric affect value for one stimulus for either a single EMG or SC voltage curve. 134 Figure 45. Mean self-reported arousal and valence ratings for 12 tactile surfaces listed in Table 16. 135 Figure 46. EMG (lower) and SC (upper) data for a participant touching a sheet of double-sided sticky tape, and a participant's hand being touched by the experimenter. High-frequency components of the raw EMG data were smoothed using a third order low-pass Butterworth filter with 0.1 Hz cutoff frequency; the presented SC data are the raw values 136 Figure 47. Mean biometric arousal (SC) and valence (EMG) ratings for 12 tactile surfaces listed in Table 16 137 Figure 48. Statistical significance response difference of arousal and valence self-report ratings for the 12 tactile surfaces listed in Table 16 138 Figure 49. Experimental Apparatus for Study 2 142 Figure 50. Screen capture of the graphic display 144 Figure 51. Mean self-reported valence ratings for the freeform exploration and target finding tasks 147 Figure 52. Mean times of target acquisition times for each stimulus in the target finding, task 148 Figure 53. Statistically significant self-reported valence ratings of the freeform exploration and the target finding tasks. Shaded circles represent significant differences between two tasks, where a heavy line means valence was reported to be most different between those two renderings [p < .05]. Labels for the seven knobs are listed in Table 19 150 Figure 54. Statistical significance results for performance (acquisition times) within the target finding task for the seven knobs listed in Table 19 151 Figure 55. Experimental Apparatus for Study 3 155 Figure 56. Pointing to a graphical target 156 Figure 57. Graphic target amplitudes and widths 158 Figure 58. Pointing target acquisition task. Participants start at a small target circle (step 1), then perform three rapid target acquisitions to a target disks (steps 2-4). After each target acquisition, the old disk disappears and the new disk is displayed. No disks are visible after the final target acquisition 160 Figure 59. Knob, width, and amplitude vs. target acquisition time 164  x  Figure 60. Knob, width, and amplitude vs. rating (valence) , 166 Figure 61. Movement Time vs. Index of Difficulty for All Knobs 168 Figure 62. Example Movement Time vs. Index of Difficulty for a Particular Knob ( K n o b nts) • 168 Figure 63. Screen capture of a Haptic Icon Prototyper showing the 3 main. interaction regions: 1) waveform editor, 2) tile palette, and 3) tile panel 182 Figure 64. Example haptic knob used with our icon prototyper 184 Figure 65. Example fan knob for illustrating icon prototyper interactions....184 Figure 66. Screen captures showing user interaction steps to create a haptic tile of a torque ramp 186 Figure 67. Screen captures showing user interaction steps to create one version of the basic detent module that will be used for the Off, Lo, Med, and Hi fan settings 187 Figure 68. Example waveform for a constant damping effect 188 Figure 69. Example waveform for a Stribeck friction effect 188 Figure 70. Screen captures showing steps with the tile pane to organize haptic tiles to build the static and dynamic haptic behaviors for an example fan knob 190 Figure 71. Example amplitude adjustment using a "mirror" line along the position axis in the waveform editor 193 Figure 72. Example frequency adjustment by dragging the right boundary of a haptic tile leftward in the tile pane 194 Figure 73. Example superimposition of two haptic tiles to create a single haptic tile 195 F e wD  e t e  xi  Symbols  A  Sinusoid amplitude constant; magnitude of detents  Bj ve  Viscous damping constant  B ^_ Viscous damping boundary for negative velocities B l+  Viscous damping boundary for positive velocities  Ci  Accelerometer calibration multiplier for converting from  ye  ca  measured volts to rads/s Cj ve  2  Negative value of dynamic friction  C l+ Positive value of dynamic friction ve  D i_ Stick / slip transition boundary for negative velocities V£  D l+ Stick / slip transition boundary for positive velocities ye  M  Moment of inertia constant (rotational mass)  P  Sinusoid period; number of "clicks" per revolution on a knob with detents  S  Sinusoid phase shift (curve fitting)  Av  "Stuck" state velocities in a stick-slip friction model (curve fitting)  0  Angular position  9  Angular velocity  9  Angular acceleration  xii  T x  Torque (angular force) Mean  2  a  Standard deviation (also SD) 2  rj  Partial eta squared (measure of statistical effect size)  xiii  Abbreviations  ABS Acrylonitrile Butadiene Styrene (plastic) CPR  Counts Per Revolution  Hz  Hertz  Nm  Newton-meters  P W M Pulse Width Modulation rad  Radians  s  Seconds  SD  Standard Deviation  V  Volts  xiv  Acknowledgements  Many graduate students, faculty, and staff in the Departments of Computer Science, Electrical and Computer Engineering, and Forest Resource Management at The University of British Columbia have helped this research. In particular, George Pava, Michael Shaver, and Lewis Johnson helped with realtime middleware. Tian Lim, Lu Yu, and Mike Xue helped conduct user studies and process data. Dr. Melanie Tory, Dr. Jason Harrison, Dr. Barry Po, Dr. Ryan Gandy, and Mark Hancock all provided valuable user study advice. Dr. Heather McLaren and Dr. Regan Mandryk suggested biometric analyses and other emotion-based research. Dr. Wolfgang Heidrich, Abhijeet Ghosh, Shruthi Achutha provided access to a 3D printer and helped print plastic parts. Doug Poison and Donald Dawson conducted design and precision machining of custom hardware. Dr. Theodore Milner provided cables, and Honeywell Sensotec engineers helped make a custom torque sensor. Evgeny Maksakov and Vic Chung performed much Java and C++ graphical user interface development. Dr. Chen Greif and Iman Brouwer provided non-linear curve fitting and practical control theory expertise. Dr. Christopher Hasser helped with position sensing. Craig Swindells and Mazda Canada provided physical controls from automobile cockpits. Laszlo Hollander, Dave Brent, Ciaran-Llachlan Leavitt, Ron Fussell, Bruce Dow, Lavana Lea, Juliet O'Keefe, and Valerie McRae frequently aided with technical support and administration issues. Mario Enriquez, Steve Yohanan, Jocelyn Smith, Andrew Chan, and others in The Sensory Perception and Interaction Research Group provided technical guidance and a rewarding laboratory environment. Also, folks from the Human Communication Technologies Laboratory and the Interaction Design Reading Group at the University of British Columbia, and the Center for Intelligent Machines at McGill provided constructive feedback throughout  XV  the progression of this research. Thank-you to my committee and advisors: Dr. Karon MacLean, Dr. Kellogg Booth, Dr. Michael Meitner, Dr. Joanna McGrenere, and Alan Boykiw. Also, my examiners Dr. Anthony Hodgson, Dr. Dinesh Pai, and Dr. Edward Colgate, and chairs Dr. Ronald Rensink and Dr. Robert Rohling. This thesis research was financially supported by a Precarn fellowship, Natural Sciences and Engineering Research Council grants, and an industrial grant from Immersion Corporation.  xvi  Chapter 0 Introduction Physical controls, such as knobs, sliders, and buttons, are common within residential, commercial, and industrial settings. For example, knobs in home stereos change radio stations, scroll wheels in computer mice aid scrolling in word processors, and sliders in manufacturing plants adjust conveyor belt speeds. When users do not think about the inertia of a stereo volume knob, the "clicks" on a scroll wheel, or the friction on a conveyor belt control, i  the designers of these controls have done their jobs well. Understanding what makes a good design is still a major challenge for the design community. This thesis presents a novel, rapid design cycle for developing physical control dynamics, and user studies that examine emotional and performance effects of these physical dynamics. Two types of physical controls are explored: mechanical physical control dynamics (e.g., the feel of a kitchen stove knob) that are ubiquitious in our everyday environments, and emerging mechatronic controls (e.g., the feel of a force-feedback iDrive knob in an automobile cockpit [39]) that are rapidly increasing in prevalence because of the trend towards embedding small, specialized computers into our surroundings (ubiquitous computing). A primary theme of this thesis is that physical control design should not be based solely on mathematical or physical properties of knobs (more generally, on any manual control), but also on human-centred properties that are more subjective, such as emotion. In addition to its technical innovations, this thesis describes both quantitative and qualitative user studies to understand these elements that can not be deduced by mechanical means alone. We examine knobs in this research, but our results are applicable to other physical controls such as sliders and buttons.  1  0.1 Motivation for Affective Design of Physical Controls Emotional reactions always accompany thoughts [106]. For example, people rarely see just a "house". They instead see a handsome house, a pretentious house, or an ugly house [110]. As with any object, every interaction with a knob, slider, or button, elicits conscious and unconscious emotional responses. The, intimacy created by sustained physical contact with such interfaces makes emotion-based research an important design dimension. Affective responses are the type of emotional responses that are visceral and unconscious — our gut reactions [70]. This thesis focuses on affective responses because they are the least variable, and therefore most generalizable, across people with different personal backgrounds, cultures, ages, and genders. Affect can be defined almost exclusively using two dimensions: valence and arousal [79,106]. Valence measures whether an affective response is good or bad, and arousal measures the magnitude of the affective response. People typically interact with physical controls for a few seconds per use. However, these short interactions occur frequently each day, and during many days of the year. The prevalence of physical controls in our environment means that relatively small improvements in the appropriateness of each affective and performance design could have a significant cumulative longterm impact on users. The relative value of affective and performance design is difficult to judge without comprehensive and appropriate metrics. Performance metrics for physical control dynamics have been studied and used extensively; metrics for the affective design of dynamics for physical controls have been less explored. This thesis introduces metrics to aid judgements of the value of affective design for physical controls, and development of physical dynamics that elicit more appropriate affective responses from target users.  0.2 Motivation for Improving Physical Control Design Mark Weiser noted that "the most profound technologies are those that disappear. They weave themselves into the fabric of everyday life until they  2  are indistinguishable from it"  [102].  This transition towards more transparent  computing infrastructure involves progression away from general-purpose computers to ubiquitous special-purpose embedded computation, keyboards and mice are being replaced by dedicated physical controls such, as knobs, sliders, and switches. Traditional design has always concerned itself with "feel". For example, Knowles and Sheridan's [49] early work on friction and inertia was applied directly to make the dials of rotary telephones "feel right". Their work focused on passive mechanical controls — purely mechanical interfaces. A new generation of these familiar manual controls are becoming active: the way they 'feel' is programmed to reflect measured user actions and situational context. Such controls are often termed "mechatronic" because they contain both mechanical and electronic components. For example, the BMW iDrive, a haptic (force-feedback) knob embedded in an automobile cockpit, provides a different "feel" in different situations such as tuning the radio or adjusting the fan speed. This haptic knob is designed to help the driver focus more visual attention and cognitive effort on driving instead of on interacting with typical 'comfort' features such as climate control and music selection [39]. Because of their pervasiveness in the developed world, it is worth examining our interactions with passive manual control interactions in detail, with the goal of gaining insight for the design of future active controls. Although active mechatronic knobs currently cost more than passive, mechanical knobs, active knobs are more versatile. Manufacturers can replace a collection of special purpose mechanical knobs with one or perhaps a few mechatronic knobs that can be easily programmed for multiple purposes. Updates can then be done in firmware instead of needing to change a collection of mechanical parts. A n economy of scale is therefore perhaps possible with active knobs that cannot be achieved with simple mechanical knobs.  3  0.3 Current Design of Physical Control Dynamics Mechanical physical controls, such as door knobs, have been designed and refined for centuries. Mechatronic physical controls, such as force-feedback door knobs that enable a person to "feel" the human activity behind a closed door [59], have been developed since the end of the 20 century. Like th  any mature product, knobs must not only function well, they must function in a way that induces an appropriate emotional response. For example, a door knob for a building entrance that feels heavy, solid, and does not rattle will induce a sense of security and quality for the environment that lies beyond the door. These emotional (affective) responses typically influence a person's initial decision about adoption, and their decisions of maintained usage. Consequently, a product that performs well but elicits inappropriate affective feedback to its user(s) will often be rejected. Despite this, affective parameters are typically discerned relatively late in the design cycle via focus groups and written surveys of users. Development and refinement of mechanical assemblies typically involves iterative construction of physical controls - a time-consuming and costly process. Psychophysical experiments to estimate the appropriate dynamics typically involve complex physical mass and cable assemblies for each dynamic property of interest [49]. With such a setup, certain dynamic combinations are difficult, or impossible, to produce. All of this makes it difficult to optimize the "feel" of physical controls except through cumbersome trial-and-error iteration. Despite the advantages that mechatronic controls may seem to enjoy, the current situation is actually not much'better when it comes to optimizing affective parameters. Current mechatronic physical control development typically involves extensive coding of physical mechanics using complex real-time systems. Although easier to refine than a mechanical setup, relating the code to the "feel" of the physical control dynamics is a difficult process requiring technical sophistication and design savvy. For these reasons, the refinement of  4  Deployed Knob  Haptic Icon Prototyper — S t e p  5  Measuring Affect Step 4  Figure 1: Process for affective physical control design dynamics and evaluation of the affective responses elicited by these dynamics are rarely conducted in a systematic, tightly coupled design process.  0.4 Approach Presented in this Thesis This thesis presents a holistic process for iteratively designing physical control dynamics that elicit appropriate affective responses in their target user groups. Although this process is applicable to any physical control, knobs were chosen as the test physical control. Figure 1 illustrates the major steps in the design process that we developed. The figure also illustrates the high-level structure of this thesis. Each step is described in a separate chapter, which we  5  briefly summarize here while noting the main contributions made by our research. Step/Chapter 1: Haptic camera — We capture representative dynamics from the plethora of existing knobs by measuring the dynamic position-, velocity-, and acceleration-dependent responses to sinusoid torque stimuli. This step's contribution is the development of a novel apparatus, a Haptic Camera, for capturing rotary position-, velocity-, and accelerationbased dynamics at resolutions that meet or exceed the best current systems. This includes development of novel gripping techniques and enhancements to a state-of-the-art torque sensor for active measurement. Step/Chapter 2: Characterization — We fit the captured data to nonlinear mathematical models that are practical to accurately render, and intuitive for designers to manipulate. This step's contribution is the refinement of non-linear curve fitting techniques for position-, velocity-, and acceleration-based mechanical models. Step/Chapter 3: Rendering and validation — We render the modelled dynamics, based on physical knobs and/or manipulated simulations, on a force-feedback knob. This step's contribution is the development of a haptic knob that updates at 10 kHz, providing up to 180 mNm continuous torque, while maintaining 9.8 x 10" rad positional resolution. Typical force-feedback knobs 6  update at < 1 kHz, provide ~18 mNm of continuous torque, and maintain 3.1 x 10 rad positional resolution [39]. Step/Chapter 4: Measuring affect — We explore the visceral emotional responses to the modelled dynamics using self-report and biometric measures. This step's contribution is the exploration of affect relationships between knob dynamics through user studies involving self-report rating scales and biometric electromyographic and skin conductance techniques to  6  measure relative and absolute affective responses, respectively, along the two primary affect dimensions of valence and arousal. We also observed that moving a physical control towards different distances or towards different target sizes has less of an effect on performance compared to affective characteristics such as knob rendering type. Step/Chapter 5: Haptic icon prototyper — We describe a Haptic Icon Prototyper which allows signal design that combines dynamic properties of existing knobs and non-mechanically-based properties for rendering on a force-feedback knob. We address both the functionality and user interface issues for modifying and tuning haptic representations. Final haptic representations can then be deployed into physical controls within products. This step's contribution is the development of a novel prototyping testbed, the Haptic Icon Prototyper and its design principles, for iteratively designing and editing position-, velocity-, and acceleration-based dynamic properties. Related work and conclusions for each step are contained in their respective chapters. Chapter 6 describes overall conclusions and future work.  0.5 Examples of Application Contexts There are many everyday uses of physical knobs that may eventually be met by mechatronic knobs. Figure 2 provides three diverse application contexts in which affective design of knob dynamics could aid user interaction. Each of these applications is explained in more detail in this section.  0.5.1 Game Character Control Computer and video games have become a major sector in the entertainment industry. Figure 2(a) shows a scene from a video game called The Act by Cecropia (clips at: http://www.cecropia.com/theAct). Instead of a keyboard and mouse or a.complex game pad, character interaction in The Act is controlled by a single knob. Rotating the knob can change, for example, the protagonist's level of courage. The central theme of the game is to observe the  7  Search Results:1 .10  shopping i.  j  < 8 S ) Tree C l a s s i c s - The W o r l d ' s F i n e s t A r t i f i c i a l C h r i s t m a s T r e e s ' * Designer and traditional pre-til artificial Christmas treesfromTree Classics, delivered throughout the US. i-  2-  » .:< : f-M-  •1,  igy  wwwtreeclasstcs com  —  169) N a t i o n a l C h r i s t m a s Tr>» A s s o c i a t i o n Home P a c t e " * Organization representing the growers of Christmas trees and greens. Features a guide lo Christmas lieo (arms and retail tots throughout the United States wchnstmastree.org  Qfdvnng RetulU I - 100 of elMM 324000000 bi  Seardi Results: 1 10 shopping  —  >  —  —  i  H U B a f chin q  Q\ Tr«« - Wikipedia, the tree encyclopedia*' For other senses of this word, see tree {disambiguation) The coniferous Coast Redwood, the tallest nee species or. earth An oak tree in Denmark 1  it . —  en w«apedia KgMfcVTree  $8) Lite on Earths Life on Earth Life on Earth Lawson. F S , R L Chariebois. and J -A R Dillon 1996 Phylogenetic analysis of carbarnoytphosphate synthetase genes complex evolutionary history includes an internal duplication within a gene which can root i 1,--1 i phytogeny anzona edu/tiee'iife mini 1  Figure 2: Application scenarios - Different knob dynamics could elicit appropriate user responses for changing (a) character actions in a video game, (b) boiler settings in a power plant, and (c) slider bar settings in an on-line search tool. 8  effects of making characters "charming, tough, sexy, aggressive, sweet, goofy, ..." [3]. The knob used in the current prototype of The Act is passive (non-actuated). If it were active, its haptic feel could subtly meld with the current emotional context of the video game to enhance a player's gaming experience: the knob could feel 'harsher' as a scene's mood becomes tenser. Determining knob dynamics such as friction, inertia, or detents ('clicks') for a particular emotional context would require an understanding of how these influence a player's emotional response, as well as how transitions between different knob dynamics should integrate with the animated scene, and which dimensions of emotion are most important at a particular segment in the game.  0.5.2 Manual Control of Complex Systems Many settings for time and safety critical environments, such as the power plant interface illustrated in Figure 2(b), require operators to manually interact with the system either on a routine basis or during emergency override situations. Special cover plates often act as a barrier preventing accidental use of a sensitive physical control. Adding active feelings to the movements of such physical controls could reinforce a recognition of safe and unsafe settings for the operator. For example, a knob for controlling atomizing steam pressure or air flow in the power plant could feel 'unpleasant' at risky settings, but feel 'pleasant' at conservative settings. Such psychological reinforcement could be particularly beneficial during emergency situations because the increased reliance on haptic feedback would leave more of the operator's cognitive resources to focus on the emergency situation.  0.5.3 Media Manipulation Yahoo's Mindset is a search engine prototype where users can adjust a slider widget to adjust the content of their query results. With the current user interface (try it: http://mindset.research.yahoo.com), users move their computer mouse to adjust the slider widget towards either "shopping" or "research". Figure 2(c) illustrates results for a search on "tree" - a word with  9  many different context-dependent meanings. Query results from a "shopping" setting focus on Christmas trees or garden stores, whereas query results from a "research" setting focus on tree biology or computational data structures. If an active haptic control were part of a typical desktop computer setup, an active haptic slider, knob, or scroll wheel could reinforce the current content or subtly provide a wider range of selections. For example, if the search results logically "chunk" into several clusters, the respective number of physical detents could be rendered on the physical control. Further, friction and inertia renderings by an expert system could subtly suggest previously viewed slider positions or settings believed to be of greater interest to the user.  0.6 Publication Listing Some of the contributions made in this thesis have been published, or are currently under review for publication. The automated haptic camera construction and characterizations described in Chapters 1 and 2 is published in the joint proceedings of the IEEE Eurohaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (World Haptics) [89]. The initial two affect user studies described in Chapter 4 that measure the effectiveness of self-report ratings and biometric measurement when touching tactile surfaces, and conducting a list selection performance task are published in the proceedings of Graphics Interface [90]. The third affect study described in Chapter 4 that compares affect and timing values for pointing  10  tasks of varied difficulty is published in the proceedings of the A C M Conference on Human Factors and Computing Systems (CHI) [91]. A n exploratory study of how to represent haptic behaviors, including dynamics of physical controls, was presented at a workshop at the A C M Conference on Human Factors in Computing Systems (CHI) [93]. The novel Haptic Icon Prototyper tool and design guidelines for future tools was presented at the IEEE Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (HAPTICS) [92].  11  Chapter 1 Haptic C a m e r a A Haptic Camera does for touch what a typical photographic camera does for vision. A visual camera captures the appearance of an environment in order to build an image consisting of a 2D grid of colored pixels, which then becomes a representation of the environment. Likewise, a Haptic Camera captures the feel of an environment and fits these measurements to a haptic model. The measured feeling can then be simulated by rendering the model on an appropriate haptic device. The Haptic Camera described in this thesis i  focuses on measurement of one kind of touch — kinaesthetic rotation about one axis (knobs). Physics-based dynamic models of friction and inertia "store" the feel of an environment, and guide the rendering of feelings on a force-feedback knob. This chapter summarizes the, physical development of the rotary Haptic Camera. Usage of the Haptic Camera, including development and validation of the haptic models, are then described in subsequent chapters.  1.1 Introduction The new Haptic Camera [89] is an extension of similar mechanical property characterization devices such, as those developed by MacLean [56], Colton and Hollerbach [20], and Richard [77]. The primary mechatronic extensions of this Haptic Camera are: • Rotary form factor: previous work focussed on characterizations of linear surfaces or switches. • Gripping: a stiff physical coupling between the mechanical knob and Haptic Camera, effective for bidirectional motion, was achieved using custom designed plastic molds. • Gravity compensation: gravity influences rotary acceleration measure-  12  merits whenever the Haptic Camera's axis of rotation is not perfectly orthogonal to the Earth's surface. Therefore, we implemented a gravity calibration process that would work when characterizing at any orientation to the Earth's surface.  -  • Resolution: theoretical torque granularity as fine as 0.2 mNm is possible with the current setup.  Torque  T e s t  Knob  400 mm  1  Figure 3: Rotary Haptic Camera Components — Exploded View  400 mm Figure 4: Rotary Haptic Camera Components — Operational View  13  1.2 Sensors Sensors were needed to measure angular position, velocity, acceleration, and torques. Table 1 lists resolutions for each of these quantities. Summaries of the different sensors are provided below. The position resolution is based on the manufacturer's calibrated value of an interpolated encoder tick. The velocity resolution is the triangulation of the position resolution and the controlling computer's update rate. The acceleration resolution is based on the accelerometer's micromachined beam sensitivity, the instrumentation amplifier's voltage resolution and update rate, and the controlling computer's update rate. The torque resolution is based on the manufacturer's calibrated value of four-point testing of the sensor's internal strain gauge. Table 1: Haptic camera sensor resolutions Position 9.8 xlO' rad 6  Velocity 2.0 xlO" rad/s 4  Acceleration 2.8 rad/s  Torque 2  1.8 xlO" Nm 4  1.2.1 Position Sensing Rotary position was measured using a custom mounted M2000-M05256-4-R1910-HA MicroE brand position sensor. This sensor features a 19.05 mm diameter rotary glass grating with 2500 counts per revolution (CPR) at 256 levels of interpolation to give a total of 640 000 CPR (9.8 xlO" rad). Posi6  tion was measured at a 900 kHz update rate. Because position resolutions as high as 4.2 million CPR are possible using this technology, a Haptic Camera with more accurate spatial measurement capabilities could be designed. However, the mounting and gripping components of such a haptic camera would require a significantly more complex mechanical design with very fine tolerances.  1.2.2 Velocity Sensing Velocity was measured by differentiating the position measurement at each timestep using Equation 1.  14  High frequency noise was then smoothed from these velocity values using a 10 order Butterworth IIR low-pass filter with a passband ripple of 3 th  dB and stopband attenuation of 50 dB. Refer to Brouwer [11] for a more detailed description of real-time Butterworth filter implementation. This 10  th  order filter design was chosen to achieve good smoothing performance while still maintaining the generally accepted haptic update rate of 1000 Hz.  1.2.3 Acceleration Sensing and Gravity Compensation Taking the 2  derivative of the position with respect to time results in  an unacceptably noisy acceleration signal. Thus, acceleration was measured using a micromachined accelerometer (ADXL202) fixed to a custom built ABS plastic housing, shown in Figure 5. The sensor power supply line was also conditioned with a 0.1 uF and a ferromagnetic bead. This accelerometer was  15  used for both determining a gravity compensation function and sensing acceleration of a test knob during characterizations.  Calibration of the accelerometer for gravity effects was performed by measuring accelerometer values over two extremely slow rotations of the Haptic Camera — one clockwise and one counterclockwise. Each rotation took approximately 16 seconds and revolved at the same constant angular velocity. In theory, gravity effects could be ignored if the axis of rotation was perfectly in line with gravity. Figure 6 shows sample measured accelerometer values that fill a range of positions (0° - 360°) for the rotary Haptic Camera under such zero acceleration conditions. This data can then be fitted to a sinusoid using four parameters, A, B, C, and D, shown in Equation 2. During a dynamic test involving acceleration, the effects of gravity can be removed by subtracting T  from the measured accelerometer values.  16  (2)  4000  facc(x)  = -305.3787 sin ( x / 0.9998 + 2.0098 ) + 2449.9817  3 pos (rad)  4  Figure 6: Example gravity compensation for accelerometer  1.2.4 Torque Sensing Torque was measured with a Honeywell-Sensotec QWFK-8M rotary torque sensor. The torque sensor was factory calibrated to 175.5 mNm / 5.0 V using standard strain-gauge calibration techniques — known torques were applied to the torque sensor and the resulting voltages were measured and fitted to a straight line. The torque sensor provided a -2.011 mV / V output voltage that was too small to be read directly into most A / D converters. A custom AD524CD instrumentation amplifier circuit shown in Figure 7 was built to boost torque sensor output voltages from a range of ±10 mV to a range of ±1000 mV. These  17  - 18V  Figure 7: Custom amplifier for torque sensor resulting signals could then be read into a standard ±1.25 V I/O board A / D channel such as the Measurement Computing DAS1602. The AD524CD was chosen for its superior dynamic and noise performance characteristics. It provided a 25 MHz gain product with settling time < 15 us to < 0.01% of a 20 V step with noise < 0.3 uV peak-to-peak.  1.3 Actuator Actuation was performed with a +12 V Maxon RE40 DC motor supply-, ing 181 mNm / 12 V. Commanded voltages from the controlling PC were amplified using a Copley 2122 PWM amplifier capable of supplying 10 A continuous current and peak current of 20 A to the motor. The motor shaft was directly applied to the knob because gears and cables would have degraded torque transmission quality.  1.4 Gripper A gripper provides a stiff physical coupling between the surface of a test knob and the Haptic Camera's sensors. An optimal gripper needs to have: • low mass • high stiffness • high adhesion (coupling rigidity) between the knob cap and Haptic  18  Camera motor's shaft for bidirectional rotation. • conformance to a wide variety of knob diameters, depths, and surface properties such as knurling and extrusions. • facilitates alignment of actuator, sensor, and test knob axes. Rapid prototyping a unique gripper for each knob is an effective way to achieve all the above listed criteria. A single reconfigurable gripper would theoretically be more desirable than a unique gripper for each knob, but reconfigurable grippers are generally bulkier and/or less stiff. Figure 8 shows example knob grippers spanning a wide range of shapes. The following procedure was used to grip each knob: Adhesive  Figure 8: Example knob grippers 1. Measure knob cap: Calipers were used to measure the dimensions of mechanical test knobs, and a 3D gripper model (a negative mold of the knob) was manually created using a 3D modeling applica-  19  tion (SolidWorks). A more generalizable and systematic approach is to scan the knob cap surface using a 3D scanner that automatically generates a 3D gripper model. For example, we created one model by scanning a knob cap with a Cyber ware Model 3030 3D scanner [23], but the caliper-and-SolidWorks approach was easier for the simple geometries of our test knobs. Specifically, it was faster to create a simple geometry from scratch than to develop denoising and segmentation algorithms for a scanned model. 2.  Create gripper: The gripper was printed as an ABS (Acrylonitrile Butadiene Styrene) plastic part using a Stratasys FDM Vantage i 3D printer. Other lightweight, stiff materials, such as polycarbonate, would yield a good gripper too.  3.  Attach gripper: Permanent industrial grade double-sided tapes, such as 3M VHB Multi-purpose Acrylic tape, form very strong bonds between the gripper and most metal or plastic knob caps. Tape adhered extremely well to all of our test knobs. Harsh solvents, such as lighter fluid or goof-off™, enabled clean removal of the grippers from their respective metal and plastic test knobs without any surface damage. Thus, measurement of the knob dynamics was possible without causing any permanent damage to the test knobs.  1.5 M o u n t i n g  The mounting provides an adjustable physical support for the cases of test knobs. The Haptic Camera needs to be mounted to provide: • ability for fine adjustment along both Cartesian axes perpendicular to the axis of rotation.  20  • high stiffness throughout the assembly. • a way to easily interchange test knobs and grippers. Developing a custom assembly from a typical lathe bed and chuck, as shown in Figure 9, effectively accomplished the above goals. Test knobs were mounted in a chuck to enable fine positioning adjustments perpendicular to the axis of rotation. The chuck was chosen because it can firmly grip a wide assortment of test knob housings. A lathe bed has the added advantages of high-stiffness throughout the complete assembly, and aptly aligns the axes of the Haptic Camera motor and test knob to be parallel to the lathe bed. To align a test knob, the chuck was iteratively adjusted until the gripper (and sensing assembly) could smoothly glide along the lathe bed axis to encapsulate the test knob (see the top of Figure 8). Adhesive was then applied to the knob'cap, the gripper (and sensing assembly) was slid along the lathe bed until it affixed to the test knob, and finally the sensor assembly was locked to the lathe bed by turning a set screw at its base. Alignments within .25 mm were readily achieved. The setup just described is designed for testing knobs in a lab environment. Instead of removing a test knob from its environment and mounting it into the Haptic Camera's chuck, the Haptic Camera's sensor and actuator assembly could be directly attached to knobs in their typical environments (e.g., automobile cockpits, home electronics, etc.). For example, field measurements could be obtained by mounting the custom Haptic Camera assembly (see Figure 3) on a stiff tripod or custom clamp mechanism. In this scenario, no lathe bed or chuck mounting is needed for the test knob because the knob is mounted in its natural environment.  1.6 Interfacing to a Computer Like most haptic devices, the Haptic Camera requires relatively demanding computational resources: • 1000 Hz update rate (10 000+ Hz desired).  21  Figure 9: a) zoom of rotary Haptic Camera during capture of a 34 mm diameter stereo volume knob, and b) overview of rotary Haptic Camera during capture of a 51 mm automobile fan knob  22  • < 20 [is variation between updates. • < 20 ms lag. A 3.0 GHz PC with 2 GB of R A M running a Timesys 4.0 Linux kernel was interfaced to the Haptic Camera via a Measurement Computing DASI602 I/O board. Software development was written in C++ using custom A C E / T A O based middleware — real-time platform middleware (RTPM). See Pava and MacLean [75] for details. During experimental trials, scheduling the Timesys processes to update within 10 u,s for more than 99% of the updates was often difficult to achieve, although good update rates of 5 000 - 10 000 Hz update rate were readily attained. Future versions could explore the use of other realtime operating systems such as VxWorks or QNX to improve performance. A custom embedded solution for the haptics using microcontrollers and/or FPGAs would probably be an even better solution.  1.7 PID controller The DC motor was controlled with a PID controller (Proportional-Integral-Derivative) after estimating proportional, integral, and derivative constants — Kp, Ki, and Kd, respectively — with the root locus technique demonstrated by Messner and Tilbury [63]. Figure 10 shows the simplified block diagram that was used to model the Haptic Camera system. We used a typical plant model, G(s), for a DC motor and PID controller, D(s), described by Equations 3 and 4, respectively. Franklin et al. [32] describe foundational theory and applications of feedback control for dynamic systems. Consequently, only the salient aspects of our PID controller are described below.  23  Readers can refer to Franklin et al. [32] for more detailed descriptions of block diagrams, dynamic modeling, Laplace transforms, and root locus design.  Input U(s) (Volts)  +  Controller  Motor  M •  D(8)  G(s)  Output Y(s) (Radians)  Figure 10: Simplified block diagram of the Haptic Camera's closed loop feedback system Equation 3 shows a transfer function with appropriate parameters for the Maxon RE40 motor (plant) used for the Haptic Camera and haptic knob. K s((Js + b)(Ls + R) + K ) 2  (3)  where G(s) is the motor (plant) transfer function. 9 is the motor position in radians. V is the input voltage in volts. K is the electromotive force constant (0.0438 Nm / A). J is the rotor's inertia (1.0 x 10" kg m ). 6  2  b is the damping ratio (6.87197 x 10" Nm s / rad). 8  L is the electric inductance (8.3 x 10" H). 4  R is the electric resistance (7.73 Q). s is the frequency-domain Laplace transform parameter. Equation 4 shows a transfer function with appropriate parameters for the PID controller D(s) =  s  (4).  24  where D(s) is the controller (PID) transfer function. Kd is the derivative control constant. Kp is the proportional control constant. Ki is the integral control constant. s is the frequency-domain Laplace transform parameter.  Closed Loop - Root L o c u s  O p e n Loop - Root Locus  •am -2000 Real Axis  O p e n Loop - Root L o c u s Zoom  2000  *<m a) Closed Loop - Root L o c u s Zoom  Figure 11: Open loop and closed loop root loci The root locus technique is a graphical tool for estimating the stability of a system. Poles and zeros respectively define adjustable and fixed attributes of the dynamic system. The basic idea with root locus design is to iteratively modify the controller variables to move the poles in a complex s-plane until a stable system is observed. Stability is achieved using carefully placed poles to pull the root locus graphs into the left-hand side of the complex s-plane (the negative real axis). The closed-loop poles are the solutions to Equation 5.  25  0 = 1 +D(s)G(s)  (5)  Figure 11 shows two zoom levels of open loop and closed loop root loci. A pole at -3720 (see Figure 11a) is safely far into the negative real axis, but the two other poles (see Figure l i b ) at -229 and 0 are likely to influence instability. Choosing Kp = 200, Ki = 5000, and Kd = 1.0 results in closed loop root loci shown in Figures llc-d. Zeros at -171 and -29.3 help pull the root loci curves completely into the negative real domain, suggesting a stable controller. Step responses for the open loop and closed loop systems are shown in Figures 12a-b, respectively. This relatively conservative controller design results in.an acceptable step response that peaks in less than 50 ms, as shown in Figure 12b.  O p e n loop step response  , in'  :  .  Reiponse l o S l o p Disturbance  Figure 12: Open loop and closed loop step responses Values of Kp, Ki, and Kd were manually adjusted from their starting values of Kp = 200, Ki = 5000, and Kd = 1.0 during system identifications to compensate for the unknown dynamics introduced by each test knob during captures. 1.8 H a p t i c C a m e r a S u m m a r y  This chapter described the development of a device, a rotary Haptic Camera, for measuring the torque, space, and time responses of rotary manual controls (knobs). Chapter 2 uses such responses to parameterize inertia, fric-  26  tion, and detent properties for a set of mechanical knobs. Equivalent or modified renderings of these dynamic properties can then be displayed on a forcefeedback knob, such as the one described in Chapter 3. Thus, future affect and performance design of both rendered and mechanical physical controls can utilize a Haptic Camera apparatus to leverage the information contained within the vast collection of existing mechanical knobs that are ubiquitous within our work and play environments. Novel attributes of this Haptic Camera are its rotary form factor, customized knob grippers, gravity compensation method, and sensing resolution.  27  Chapter 2 Characterizing Haptic Environments We can better understand and enhance the feel of knobs, including detents, friction, and inertia, by fitting mathematical models to the measured behavior of real physical knobs. Both the physical characteristics and subjective human responses of these knob models can then be studied and improved. This chapter introduces a dynamic model and a non-linear least squares characterization procedure for the detents, friction, and inertia of physical controls. Our characterization process involves stimulating a physical knob with a swept sine trajectory using our Haptic Camera apparatus. We then fitted measured position, velocity, acceleration, and torque responses to a second order dynamic model. We first validate our characterization procedure by using it to model simulated data with known dynamic properties, with and without additive noise. Once validated, we characterize a set of five real mechanical test knobs using the rotary Haptic Camera apparatus described in Chapter 1. General characterization conclusions such as differences between human and machine sensitivities to detents are then formulated by observing the simulated and test knob characterizations. This Haptic Camera apparatus and characterization of 5 mechanical knobs is published in Swindells & MacLean[89]. Readers wishing to skip the mathematical characterization details may wish to jump to Section 2.3, "Comments on the Characterization Process" on page 70. In subsequent chapters, we examine both the physical characteristics and the subjective human experiences of the detent, friction, and inertia properties captured by the procedure described here.  28  2.1 Knob Model We modeled torque responses to a knob's acceleration, velocity, and position using non-linear least-squares fitting. Different parameters of the knob model (Equation 6) were evaluated separately to improve fitting quality and speed. We based our characterization approach on a Lur'e system of linear dynamic and non-linear static parts [30]. Although the Karnopp friction model has non-linear dynamic stick-slip boundary parameters (C i_ and ve  C i+), ve  these parameters can be solved using a linear approach because they  are constants multiplied by a non-linear function (reside outside the sgn() function). If parameters resided inside the sgn() function, we would.need to solve for such parameters using a non-linear approach. Matlab's l s q c u r v e f i t command was used to fit the static parameters P^ and Sp and Matlab's os  0S  \ [backslash] command was used to fit the dynamic parameters M ,  C i_,  acc  C i+, B i_, ve  ve  and B i+, and the static parameter A ve  p o s  (because A  pos  ve  resides out-  side the sin() function) [62]. For function minimization we used the Levenberg-Marquardt method instead of the more traditional Gauss-Newton method, because the Levenberg-Marquardt method has been shown to find a better fit for medium-scale problems like the ones in this thesis [34]. Equation 6 illustrates the system model used for both system identification and rendering of haptic knobs (Chapter 3, "Rendering and Validation" on page 80 describes our haptic renderings). Torque, position, velocity, and acceleration values are captured using the Haptic Camera, and then the remaining parameters were fit to the model in Equation 6. Figures 13 and 14 illustrate the velocity and position functions introduced in Equation 6. acc. part ( T  T=  C . sgn 0. + B vel  veU  6_ + C  vel+  sgn 0 + B +  ve{+  f l C C  -)  0 + vel. part ( r ) +  ve/  () 6  pos. p a r t ( ^ p 0  pos  29  where T is the torque rendered to the force-feedback knob. pos' vel'  T  r  a  n  dx  acc  a  r  P  e  o s m o n  ' velocity, and acceleration torque  parts that sum together to form t. 9,9,  and 9 are the rotational position, velocity, and acceleration. 9_  and 9 are the negative and positive parts of the velocities, and have the +  same vector length as 6 . For example, 6_ contains the same negative velocities as 9 , but has zeros to replace the positive velocities of 9 . is an acceleration constant, intuitively similar to inertia.  M  acc  • C _ and C i vel  ve  B _ and B vel  A  are the negative and positive values of dynamic friction.  +  y e l +  are the negative and positive values of viscous friction.  , and S  , P  pos  are a set of possible position parameters to render  detents. Examples of other possible position functions include one or more ramps, K B , or polynomials P3 0  3  nnc  ' pos '  r  A  pQS  pos  J  , and S  , P  pQS  r  + P 2 & + PI 9+ PO . pos pos. pos n  change the amplitude, period, and phase shifts.  Equation 7 and Figure 13 illustrate the version of Karnopp's friction model used for characterization and rendering [44]. The Karnopp model in Equation 7 is estimated by combining parameters fit to Equation 6 (C vel+ vel-  C  B  a  n  vel+)  dB  w i t h a d d i t i o n a l  parameters (Av, D  yd  and  ,  D ) vgl+  which are determined by visual inspection of torque versus velocity plots such as Figure 35 on page 70. More sophisticated models such as the Stribeck effect could be used, but the practical ability to both capture and render subtleties in torque beyond the Karnopp model are beyond the scope of this thesis [87].  max(D , ve/  ^friction  min(D , , v£  C  ve(+  +  W  W  ) )  s g n 9 *B ,J ve  -Av < » < 0 te te  (7)  0 < 0 < Av « > Av  30  where other  x  r e  P  the non-frictional torque currently applied to the knob.  r e s e n t s  Av is the values below which we assume the velocity to be zero. l_  D ve  a n  d  i+  D v e  a  r  e  the negative and positive values of static friction.  The other constants are the same as above.  D C  vel+ vel+  Av 6  Figure 13: Karnopp Friction Model Figure 14 illustrates a general detent model. A  represents the  amplitude of a detent. Each crossing of the sinusoid across the x-axis represents a "groove" or "valley" of one detent. Consequently, P frequency of the detent. S  changes the  will shift the position of the detents along the  knob. A variety of position-related terms would be appropriate for the position function  T  .  pQS  For example, one could add a harmonic term to better deal  with torques that are not purely sinusoidal. However, each added term may help or hinder fitting of data to the overall model. Most curve fitting algorithms (including l s q c u r v e f i t or \ in Matlab) have a more difficulty fitting models containing more terms. Consequently, a challenging trade-off exists. A model with more terms is more generalizable and may better represent the  31  statics and dynamics of a test knob. However, a model with more terms must be significantly better than a model with fewer terms in order to compensate for the resulting lowered ability of the curve-fitting algorithm to fit these additional terms. If better curve fitting algorithms become available, then a position function with more terms could be employed to create more general model that better fits a wider variety of physical control statics. Similarly, more sophisticated velocity (t j) and acceleration ( ? ye  a c c  ) functions could  theoretically contribute to better fits of physical control dynamics.  T  4  1  * pos  /"~  A  \ S pos  p pos  J  '\  /  6  _  1  Figure 14: Detent Position Model For the characterizations performed in this thesis, the three parameters for position ( A  p o s  , P  pos  , and S  ) were the minimum number of parame-  ters to fit detents with even a modest level of quality. When rendering, one can often ignore phase shift, S  ; but, manually setting phase shift to zero  before a Haptic Camera capture is difficult and error-prone. S  is conse-  quently needed during data fitting. Friction parameters were chosen as previously done by Richard [77]. C , , C , B , and B , represent the two +  +  32  primary states of the stick-slip Karnopp friction model. C ^ and C ye  the forward and reverse "stuck" state, and  v e / +  for  and # / for the forward ve  +  and reverse "slip" state. Av, £> _, and D ^ can be effectively estimated by vg/  +  visual data inspection as in shown later in this chapter (Figure 35 on page 70). Consequently, these three parameters were not fitted using our automated curve fitting procedure (every additional fitted parameter lowers overall fitting performance).  2.1.1 Selected Previous Characterization Work This characterization method is an extension and combination of several previous efforts: • Richard [77] characterized the friction properties of three surfaces by linearly sliding across each surface with a load cell. The velocity and acceleration parameters shown in Equation 6 were fitted using a least-squares algorithm. • MacLean [56] used a haptic interface to measure the non-linear stiffness of a momentary switch using a piecewise linear model. Different regions of the switch were individually characterized using nonlinear force versus position curves. • Miller and Colgate [66] also characterized force versus displacement data. They used a wavelet network to work in the spatial frequency domain, thus avoiding the need to manually segment out different linear and non-linear regions. • Weir et al. [101] took a very pragmatic approach by visualizing mechanical properties of switches using colored plots of position, velocity, or torques. Differences between switches were compared by looking at the same 2D or 3D "haptic profiles". However, these "haptic profiles" were not modeled (parameterized). Modeling and 33  parameterization provides us with a mechanism to analyze, modify and flexibly render the captured dynamics. . • Feo [30] took the approach of modeling with a single chaotic dynamic system. Instead of testing physical mechanical systems, Feo tested periodic signals such as time series from spoken vowels and electrocardiograms. The linear dynamic parts and non-linear static parts identified by Feo closely resemble the dynamic and static components in Equation 6. While promising, Feo's [30] work has not been applied to haptic problems.  2.1.2 Validating the Characterization using Simulated Data Effectiveness of the characterization procedure can be tested using simulated perfect and noisy data that would typically be captured by the Haptic Camera. One example characterization of simulated data is illustrated below. The steps taken to test the characterization were: 1.  Generate noiseless and noisy sets of physically realizable position, velocity, and acceleration vectors (6,0,  and 6 ) based on a swept  sine trajectory (velocity as the derivative of position, and acceleration as the derivative of velocity). 2. For each model being simulated, choose a set of scalar values for the model parameters (e.g., A  ,£  y e / +  , and M , Qcc  respectively)  in Equation 6. 3. Generate a torque vector (r) by computing Equation 6 using these model parameters and the kinematic vectors from Step 1. Simulated noise can then be applied to the torque vector. 4. Use the computed torque, position, velocity, and acceleration vectors to fit the data using a non-linear least-squares curve algorithm (Matlab's l s q c u r v e f i t and \ commands) to the parameters in  34  Equation 6, thus generating an estimate of the commanded model parameters. 5.  Compare the values of the commanded and fitted values of the parameters in Equation 6.  Details of the above steps are described below.  2.1.2.1 Step 1: Generating Test Spatial Data Physically valid swept sine trajectories for position, velocity, and acceleration were generated using Equation 8. First and second derivatives were calculated from the position function, and were used for the velocity and acceleration values, respectively, to ensure a physically valid dataset. . \ n V({b-a)t, V - csin<r- — + a\ \b-a\_\ a J 1  2 -a  1  (8)  e e  2  471 cf(b-a) t• ^ —r^— + a  2-  1  2nc(b - a)  cosi  (b-a)  71 (b-a)  Equation 9 shows the parameters used for this simulation, and Figure 15 illustrates the three signals. t. --  »  a =  n -, n = 1, 30000 5000'  (9)  0, b = 1.5, c = 1.0, d = 3.0  35  0> >  _1fJ I 0  1  1  _ %2  L-  2  I  3 Time (s)  I  I  4  5  6  Time (s) Figure 15: Simulated position, velocity, and acceleration curves  2.1.2.2 Step 2: Choosing Model Parameters The model parameters chosen for this simulation are shown below in Equation 10. Equation 10 is the same as Equation 6, but the symbolic scalars have been replaced by numeric values. In addition to the parameter set shown, we simulated several other model parameterizations in an effort to minimize bias. Equation 10's parameters were chosen such that neither T T i, or r  ,  dominated the resulting torque r over the typical operating posi-  tions, velocities, and accelerations of the Haptic Camera. For simulation of the other test models, we obtained similar quality results to those reported for this set of parameters.  36  0.40 +  acc. part ( T y  r=  K  J  acc'  1.5sgnf? +1.00 +2.Osgn0 + 1.56> +  vel. part(T )  1.0sin(^+ 0.9)  pos.  +  +  (10)  vg/  part(T  p 0 J  )  2.1.2.3 Step 3: Generating Test Torque Data Simulated torques were generated with Equation 10. A noisy torque signal was then generated using Equation 11. After reviewing the position, acceleration, and torque sensor technical specification sheets [7, 36, 64], we estimate the torque sensor to produce less than 0.1% randomly distributed peak-to-peak torque variation over 10 Hz -100 kHz, the position sensor to produce less than 0.1% variation over 1 Hz -10 kHz, and the acceleration sensor to produce less than 4% variation over 1 Hz - 2.5 kHz. Further, in normal processing, we sampled data from these sensors at rates between 5 kHz -10 kHz, then low-pass filtered (in software) these data at 500 Hz cutoffs for the torque and acceleration data, and 1000 Hz cutoffs for the position and velocity (calculated as the derivative of the position) data. Given these sensor and process specifications, we conclude that the real sampled data input to our characterization algorithm would not exhibit noise more debilitating than broad spectrum (flat across a frequency range that includes 1 -1000 Hz) with peakto-peak torque variations up to 4.5%. Equation 11 below was designed as a worst-case noise test: it provides broad-spectrum, peak-to-peak torque noise beyond an average 4.5% magnitude. To do this, it uses Matlab's rand function, which produces a trajectory that can be spectrally classified as noise between 0 - 2500 Hz that has random magnitude with consistent average values throughout this frequency range. In  37  other words, the noise is evenly distributed throughout the 0 - 2500 Hz frei  quency range. \oisy  =  r+4.0(rand(length(z))-0.5)  (11)  where lengthQ = the number of torque values in T. rand() = a uniformly distributed random number between 0 and 1. The resulting torque signals with and without noise are shown in Figure 16.  Time (s)  Figure 16: Simulated torques without (top) and with (bottom) noise  2.1.2.4 Step 4: Fitting to the Test Data The spatial data shown in Figure 15 and the torque data shown in Figure 16 were then modeled using the previously described separated nonlinear least-squares curve procedure. Using the torque signal with no noise, a  38  perfect fit of the parameters was found after 7 iterations and 43 function counts with the Levenberg-Marquardt algorithm. With the noisy data, after 7 iterations and 46 function evaluations, we obtained parameter fits as shown in Table 2.  Table 2: Modeled Results for noisy simulated data Param  Target Fitted  M  vel-  ^vel-  0.4  1.5  l.O  2.0  1.5  1.0  0.398  1.512  0.999  1.992  1.500  1.008  C  acc  vel+  C  vel+  B  pos  A  P  pos  ^pos  0.2  0.9  0.199  0.884  2.1.2.5 Step 5: Comparing Commanded and Fitted Data The modeled values shown in Table 2 closely match the target values. Figures 17,18, and 19 illustrate these results. The top plot of Figure 17 shows the overall fitted torque vs. position and 95% confidence interval data superimposed on top of the commanded noisy torque data. The 95% confidence interval suggests that repeated calculations of torque would have a mean value within this confidence interval for 95% of cases. The bottom of Figure 17 shows the torque data (T  ) after the modeled acceleration (r  ) and veloc-  ity ( r | ) components have been segmented (subtracted) from the overall k  torque (r) at the top of Figure 17 (refer to Equation 10). The captured value of M  was used when calculating r ULL-  C4 O  . Similarly, the captured position- and O  velocity- related parameters were used to calculate "*  pos  and r  vel  , respec-  tively. Figure 18 shows the torque vs. velocity equivalents of Figure 17. In all of the plots in Figure 17 and Figure 18, the 95% confidence interval data lies almost on top of the fitted curve and thus is difficult to see. The quality of the solid fitted line compared to the 95% CI dashed lines can be better seen by  39  zooming into regions of the adjusted torque vs. position, and torque vs. velocity plots, as shown in Figure 19. Even in the presence of noisy data, the quality of these model fits are high; the residual error is small relative to a human's perception. These good fitted results for noisy simulated data suggest that our characterization procedure should work reasonably well for data obtained from the Haptic Camera.  40  0) ID  -10 h  Position (rad)  Position (rad)  Figure 17: Fitted torque vs. position plots with (top) and without (bottom) velocity and acceleration components of simulated data with noise added  41  Velocity (rad/s)  Figure 18: Fitted torque vs. velocity plots with (top) and without (bottom) position and acceleration components of simulated data with noise added 42  R a w x_ pos • Fitx pos  95% CI  0.133 0.1335 0.134 0.1345 0.135 0.1355 0.136 0.1365 Position (rad)  2.025 2.02  A  2.015 3> 2.01  ''A  o  :—  2.005  1.995 0.006 0.008 0.01 0.012 0.014 0.016 0.018 0.02 0.022 0.024 Velocity (rad/s)  Figure 19: Zooms of fitted torque vs. position (top) and torque vs. velocity plots (bottom) illustrating the very small 95% confidence intervals  43  2.2 Characterizing Mechanical Test Knobs We chose five mechanical knobs spanning a wide range of position, velocity, and acceleration dependent mechanical properties. The current Haptic Camera is designed for knobs that provide reaction torques between 0 - 200 mNm. Most consumer electronics, such as automobile consoles and household appliances, contain knobs with torque profiles within this range. Calibration of the Haptic Camera, and characterization of five mechanical knobs are described in the following section.  2.2.1 Sensor Verification and Calibration Spatial and torque tests were performed to test and calibrate the position, velocity, acceleration, and torque measurement capabilities. The basic approach with these tests was to send specific voltages to the DC motor (command  the position of the motor) while sensing the motor's responses. For these  spatial and torque tests, we created a swept-sine position trajectory which resulted in simultaneously controlled variation of position, velocity, and acceleration. We also created a separate swept-sine torque trajectory. We then compared these commanded values to the actual recorded results from the sensors. Rotary position was commanded using a closed-loop PID controller for both tests using a swept sine, as previously shown in Equation 8. Tests were run for a 5 second interval while operating at a 5000 Hz update rate (over a set of 25 000 contiguous data points). Three calibration repetitions were performed for each of position and torque using the swept sine (Equation 8) constants listed in Table 3, and the results were averaged into a single response trajectory for each position and torque. The large value of c (1.0) was used to account for the motor shaft displacement in the spatial test; the motor shaft was anchored for the torque test. Spatial test values of a, b, c, and d were chosen to achieve velocities and accelerations up to the stability limits of our PID controller. Torque test values of a, b, c, and d were chosen to achieve torques up to operating limits of our torque sensor.  44  Table 3: Swept sine constants for sensor tests using Equation 8  Test  b  a  c  d  Spatial  0.0  1.5  1.0  3.0  Torque  0.0  1.5  0.02  3.0  2.2.1.1 Spatial Test Results Figure 20 shows one of the three calibration repetitions, with the commanded position trajectory, and its two derivatives (velocity and acceleration) shown as wide gray lines. The Haptic Camera was unloaded (the motor could spin freely to the commanded position). Measured results (thin overlaid lines) closely followed the commanded position and velocity values. Acceleration values were also acceptable after fitting a scaling constant, C , using a least-squares algorithm ( \ [backslash] in Matlab) with Equacal  tion 12. Three independent accelerometer calibrations were performed to estimate an averaged C  as (x=1148, a =117). We made these calculations  £al  using the same position-based swept sine trajectories described above. None of the other sensors required such calibration. t  §  = calK C  ( ) 12  where & = calibrated acceleration in rad/s . t  C j = 1148 (rad/s ) / V conversion constant. 2  6 = raw acceleration from the Haptic Camera in volts. r  45  Swept S i n e Input  Figure 20: Commanded (thick gray) and measured (thin colored) position (top), velocity (middle), and acceleration (bottom) values from a single commanded swept sine signal  2.2.1.2 Torque Test Results To test torque measurement ability, the Haptic Camera gripper was anchored to prevent any movement, and a small positional swept sine was commanded (refer to Equation 8 on page 35 and Table 3 on page 45). Figure 21 illustrates one of the three calibration executions of a commanded (wide gray line) and measured torque (thin overlaid line) signal.  46  Time (s)  Figure 21: Commanded (thick gray) and measured (thin colored) torque from a single commanded swept sine signal  2.2.1.3 Test Results Means and standard deviations of the difference between the commanded and recorded values are listed in Table 4. These results include a slight phase lag of < 1% after 25 000 updates that can be seen in Figures 20 and 21. Table 4: Means and standard deviations of spatial and torque differences between commanded and recorded signals 9 (rad)  9 (rad/s)  0 (rad/s ) 2  r (Nm)  Mean  0.0012  0.0016  0.664  0.0010  SD  0.0032  0.0244  0.415  0.0018  The test results were generally of good quality, and felt to be adequate to proceed with the capture task. Primary areas for improvement are the phase shift between the commanded and measured trajectories, noise in the acceleration measurements, and limited torque range. The phase shifts could be reduced by using a real-time operating system or embedded solution with better timing than Timesys Linux provides. Acceleration measurements could be improved using a sensor fusion approach combining optical encoder and accelerometer measurements (refer to Abidi & Gonzalez [2]). A better torque dynamic range would require fundamental technological advancements to current state-of-the-art torque sensors and actuators.  47  2.2.2 Capture of Five Knobs Two captures for each of five mechanical knobs were performed. The first two knob environments were uniform in all parameters across their entire position range, whereas the last three knobs had detents while exhibiting uniform friction and inertia over their entire position range. Thus, Knob^igh  friction  and Knobhigh inertia were primarily chosen to explore linear velocity and acceleration effects; whereas, K n o b  Stents'  K n o b  m o d e r a t e detents'  a  n  d  K n o b  non-  were primarily chosen to explore non-linear positional effects.  sine detents  Knob _ j non  subtle  s  ne  detents  w a s  chosen as a "worst case" because it has a large amount  of backlash and very noticeable velocity non-linearities along different regions of the detents (it violates our Lur'e system assumption). Because of the many ways in which it does not match the structure of our model, we did not predict good fits; however, we used it as a test of methodology's robustness to this situation. Knob labels, their intuitive descriptions, and actual knob sources are listed in Table 5. Table 5: Intuitive descriptions for five mechanical test knobs  Knob label Knob  friction  high  Description Original Source Uniform position; moderate friction; low inertia High-end 1960's AM/FM  Knob  inertia  hign  Uniform position; low friction; high inertia High-end 1960's AM/FM  K  n o D  s u b t l e detents  radio volume knob  radio tuning knob  Very subtle, consistent detents of 30 "clicks" / 360°; low friction; low inertia Volume knob from a 2004 Mazda  K n ° D m o d e r a t e detents  626automobile  Moderate, consistent detents of 12 "clicks" / 360°; moderate friction; low inertia High-end 1970's audio amplifier source selection knob  Knob  non  .j s  ne  detents  Wide non-sinusoidal detents of 12 "clicks" / 360° and backlash; moderate friction; low inertia Fan settings knob from a 2004 Mazda 626 automobile  48  2.2.2.1 Data Collection Data was collected using a spatial swept sine trajectory as in Equation 8 using the values defined in the first row of Table 3. This process resulted in measured vectors of equal length for position, velocity, acceleration, and torque. The following sections describe how we prepared and fitted each of these measured vectors into Equation 6 to estimate the acceleration-, velocity-, and position-based model parameters for each of our five test knobs.  2.2.2.2 Data Preparation Before fitting, the data was sorted by position and passed through a low pass filter to remove high-frequency noise. Pre-sorting by position is not detailed here because it is the same as previous characterization research such as MacLean [56], Richard et al. [77], and Colton and Hollerbach [20]. Third order Chebyshev type IIIIR low pass filters were applied to position, velocity, acceleration, and torque vectors. These filters all had a stop band ripple of 20 dB and each had an edge frequency as listed in Table 6. Values for the edge frequencies were chosen to roughly match the commonly accepted minimum haptic update rate of 1000 Hz. Phase shifts in the filtered data were avoided through acausal filtering— one stage using forward filtering and a second stage using backward filtering. Table 6: Low pass filter stopband edge frequencies e 1000 Hz  6  1000 Hz  e 500 Hz  T  500 Hz  2.2.2.3 Data Results Table 7 lists the fitted position, velocity, and acceleration parameters generated in two independent characterizations of each test knob. The last row in the table lists the overall 95% confidence intervals for the torque responses of each test knob, computed using Matlab's n l p r e d c i command in the statistics toolbox. Smaller 95% CI values indicate better quality parame-  49  terizations. Figures containing point clouds of data will have point clouds of 95% CI values — an intermediate 95% CI value for each data point (see Matlab's n l p r e d c i documentation for details  Q  Curve fitting quality decreases with each additional parameter to be solved. Consequently, wereduced the number of parameters to be solved by the curve fitting procedure when possible. Knob ig n  n  f i ti n r  C  0  a  n  d  Knobhi  gri  i  n  e  r  n  a  detent values A„„„ , Ppos ', and Spos„ are blank in Table 7 because they were pos ' J not fitted. This approach enabled the curve fitting algorithm to be focussed on characterizing the friction and inertia attributes of these knobs because we knew a priori that these knobs did not contain detents. Only one estimate for each of D  ,D , vel+  and Av is given in Table 7 because these estimates were  not parameterized using our curve fitting algorithm. They were instead calculated visually by an expert (as done by previous researchers such as Richard et al. [77]).  50  M  acc  (mNm/rad/s ) 2  vel-  C  l 1  (mNm/rad/s) vel-  B  (mNm/rad/s) vel+  C  (mNm/rad/s) vel+  B  (mNm/rad/s) ^vel-  !  (mNm) vel+  D  (mNm)  pos  A  i  1  i  (mNm) pos  0)  s pos  (rad)  non-sine detents  moderate detents  subtle detents  1  0.069  0.28  0.034  0.049  -0.0048  2  0.091  0.27  0.035  0.048  0.0085  1  -50  -3.8  -2.2  -1.1  -35  2  -42  -13  -2.3  -0.12  -3.5  1  -1.8  -0.56  -0.16  -0.48  -6.4  2  -3.5  -0.37  -0.14  -0.62  -2.9  1  50  3.8  . 2.2  1.1  35  2  42  13  2.3  0.12  3.5  1  7.5  0.47  0.25  0.56  6.7  2  1.5  0.61  0.23  -0.0062  1.4  1  -150  -17  -10  -20  -200  150  17  10  20  200  0.040  0.015  0.010  0.010  0.015  1  1.2  -11  -202  2  0.97  -11  -61  1  0.076  0.034  0.16  2  0.076  0.035  0.041  1  0.00046  ^0.19  -0.16  2  0.17  -0.22  -1.9  2 1 2 1  (mNm)  high inertia  Parameter set #  high friction  Table 7: Two independent sets of fitted parameters for all knobs  2  95% CI  1  0.26  0.18  ,0.098  0.073  3.3  (mNm)  2  0.35  0.20  0.084  0.072  5.1  51  Figures 22 - 34 illustrate captured torque vs. position and torque vs. velocity data that highlight the detent and friction components of the five knobs, respectively. Torque vs. acceleration plots are not shown because only one acceleration variable, M  , was fitted with the capture model. Because  the resulting plots would simply be constant lines, they would be of negligible interpretive value. Plot legends in Figures 22 - 34 show five different types of data according to the five bullets listed below. These legends all relate to the x, r T ?, and r  p Q S  ,  torques as defined in Equation 6 on page 29.  • Raw t: Raw torque values from the torque sensor after being lowpass filtered. (See "Data Preparation" on page 49.) • Fitted v. torque result from Equation 6 using measured position, velocity, and acceleration values with fitted values of parameters such as  , B,, , and pos  and  • Raw Tp  QS  (raw r v  pos  '  vel+' T  V G  ^:  „. acc  raw torque values after subtracting non-position  = raw r - fitted r  acc  - fitted r ,) or non-velocity modvel  -  1  }  eled components (raw tvel, = raw r - fitted racc - fitted rpos') from r  v  the measured torque vs. position or torque vs. velocity plots, respectively. For example, "clean" the modeled position-based detents and acceleration-based inertia from the measured torque vs. velocity plot to leave behind the (empirical) velocity-based friction components. • Fitted  T  pos  and  T : yel  torque results after subtracting modeled non-  position (fitted T = fitted r - fitted racc - fitted rvel',) or nonpos r  v  velocity components (fitted r  v g /  = fitted r- fitted %  acc  - fitted T  )  from the torque vs. position or torque vs. velocity plots, respectively. • 95% CI: 95% confidence interval (e.g., n l p r e d c i command in Matlab) for the data presented in each plot.  52  As we can see in Figures 22 - 34, the gray colored raw torque data appear as point clouds around the fitted black colored torque data. If the torque, spatial, and temporal data have been well captured by the Haptic Camera, and the curve fitting algorithm effectively segmented and fitted the physical knob statics and dynamics, then the data points will be packed tightly together according to a model such as Equation 6. However, perfect capture is not possible because of mechanical noise, electrical noise, physical limitations, and algorithmic imperfections which are discussed later in this chapter (Section 2.3, "Comments on the Characterization Process" on page 70.and Section 2.4, "Noise Source Summary" on page 76). For example, one can see oscillations in the raw and fitted torques at the top of Figure 22 (bottom left of the plot), and one can see some poorly fitted and segemented raw torque data at the bottom of Figure 23 (bottom left of the plot).  53  300 Rawx Fitx  -250 -15 1  1  -10  1  -5  1  1  0  1  5  10  '  15  Velocity (rad/s)  Figure 22: Torque vs. velocity plots for Knob g for all torques (top) and with position and acceleration components removed (bottom) ni  nf r i c t i o n  54  300  Rawx Fitx  Figure 23: Torque vs. velocity plots for Knob gh i } for all torques (top) and with position and acceleration components removed (bottom) ni  nert  a  55  Rawt . vel  Fit  T  .  vel  9 5 % CI ^ O b h i g h friction  -2.6  -2.5  -2.4  -2.3  -2.2  -2.1  Velocity (rad/s)  Knobhigh inertia  -2.6  -2.5  -2.4  -2.3  -2.2  -2.1  Velocity (rad/s)  Figure 24: Zooms of torque vs. velocity plots for Knobhigh friction ( P) Knobhigh inertia (bottom) to  a n c  *  56  30 r Rawx Fitx 20  E z ;  10  E  ^^W *^ 4  -0.1  0 0.1 Position (rad)  0.2  0.3  0.4  -0.1  0 0.1 Position (rad)  02  0.3  0.4  -20  -30 -0.4  -0.3  -0.2  Figure 25: Torque vs. position plots for K n o b detents ^ ^ torques (top) and with velocity and acceleration components removed (bottom) o r a  subt]e  57  300 Rawx Fitx  200  100 E Z E  mm  cr — o  -100  -200  -300 -5  0 Velocity (rad/s)  300 r  200  100  a,  0  -100  -200  -300  0 Velocity (rad/s)  5  Figure 26: Torque vs. velocity plots for K n o b i Stents ^ ^ torques ( P) and with position and acceleration components removed (bottom) o ra  subt  to  e  58  A  3.5 3 «=• 2.5 oT  2 1.5 1 0.5 h 0 -0.1  -0.05  0.05 Position (rad)  -2 -3 -4 -5 E  •If Ii "'I'll »l|l»itlHppiW^»  E  <x>  -6  -9h -10 -2.6  -2.5  -2.4 -2.3 Velocity (rad/s)  -2.2  -2.1  Figure 27: Zooms of torque vs. position (top) and torque vs. velocity plots (bottom) for K n o b subtle  d e t e n t s  59  -0.1  0 0.1 Position (rad)  Figure 28: Torque vs. position plots for K n o b detents f ° torques (top) and with velocity and acceleration components removed (bottom) ra  u  m o d e r a t e  60  -200  -300  h  0 Velocity (rad/s)  Figure 29: Torque vs. velocity plots for K n o b detents f ° ^ torques (top) and with position and acceleration components removed (bottom) r  a  m o d e r a t e  61  12  -6 -  -7 -  _ I -2.6 8  1  1  1  1  -2.5  -2.4  -2.3  -2.2  Velocity (rad/s)  1  -2.1  Figure 30: Zooms of torque vs. position (top) and torque vs. velocity plots (bottom) for K n o b m o d e r a t e  detents  62  600 Rawx Fitx  400  E E. Z3 CT  o  -200  -400  -600  0.3  0.4  0.5  0.8  Position (rad)  600  400  200 E E  o>  0  ZD  -200  -400  -600  0.1  0.2  0.3  0.4  0.5  0.6  0.7  0.8  Position (rad)  Figure 31: Torque vs. position plots for K n o b . detents f ° torques (top) and with velocity and acceleration components removed (bottom) — 1 test ra  non  u  sine  st  63  600  r  -400 h  0  0.1  0.2  0.3  0.4  0.5  0.6  0.7  0.8  Position (rad)  600  -600  r  1  0  1  0.1  ' 0.2  1  0.3  1  1  0.4  Position (rad)  1  0.5  1  0.6  1  0.7  0.8  Figure 32: Torque vs. position plots for K n o b . detents f ° torques (top) and with velocity and acceleration components removed (bottom) — 2 test ra u  non  sine  64  300  200  100 E  E  <D cr  0  -100  -200 h  -300  0 Velocity (rad/s)  Figure 33: Torque vs. velocity plots for K n o b . detents f ° ^ torques (top) and with position and acceleration components removed (bottom) ra  non  sine  65  -210  -215 0.1 1  0.15  0.25  0.2  0.3  0.35  0.4  Position (rad)  -194 Rawx .  vel  Fitx ,  -196  vel 95% CI  - u ^ v ^ : ^  «'>V-  +  -198 v ~ w * * r * : : : .  •;• * ^  *-  :  •z. -200 E  ZJ  % -202  **  -204  •*... *•  -206  •* •s  •••••••••  #  -208 -2.6  -2.5  -2.4  -2.3  -2.2  -2.1  Velocity (rad/s)  Figure 34: Zooms of torque vs. position (top) and torque vs. velocity plots (bottom) for K n o b . non  sine  d e t e n t s  66  2.2.2.4 Summary Analyses of the captured data are summarized for each knob. Knob^ghfriction'The  top  a n  d bottom plots of Figure 22 have a clearly  defined form similar to the Karnopp friction model. Capturing dynamics were expected to be easier for this knob than the other knobs because manual exploration of Knob igh friction revealed no obvious detents or high inertia compon  nents, and suggested a uniform, moderate friction. The loop of Raw t ^ data at the top right of Figure 22 (bottom) and oscillations such as those at the bottom left of Figure 22 (top) suggest a small amount of variance from the fitted model (Equation 6). Figure 24 shows 95% CI torque values < 1 mNm for torques with magnitudes greater than 140 mNm, indicating a successful fit. Knob  high  inertia'  The higher inertia and subtle friction of Knobhigh inertia  made fitting more difficult than Knobhigh friction- Consequently, a much greater difference can be seen between the Raw r (top) and Raw r j (bottom) plots in Figure 23 compared to Figure 22. Nevertheless, Figure 23 highlights very good separation of velocity and acceleration effects as demonstrated by the subtle Karnopp friction form that is apparent in Figure 23 (bottom) — the resulting plot after removing acceleration (inertia) and position components from Figure 23 (top). Acceleration torque effects of magnitudes ~200 mNm completely dominate the velocity effects in the top of Figure 23. But, the bottom of Figure 23 shows relatively clean separation of small ~20 mNm velocity effects from the dominating acceleration effects. A small amount of improperly fitted data can be seen at the bottom left of Figure 23. This region of Raw t i ve  torque data was probably caused by saturation of the torque sensor dur-  ing the captures because the torque sensor and data acquisition hardware were only rated to ±180 mNm. Some of this noise could also be due to the less accurate closed-loop control for such a heavy, high inertia knob. For example,  67 v  a small amount of oscillation can be seen at the bottom left of Figure 22. Like Knob  high  friction'  Knob ij i su  t e  Figure 24 the 95% CI suggest a successful fit. detents'-  The double "ghosted" Raw r and Fitted r data at the  top of Figure 25 were due to velocity and acceleration effects. The expected sinusoid for a knob with detents is clearly visible at the top of Figure 25, and well segmented into the Raw x  pQS  and Fitted  T  pQS  values at the bottom of  Figure 25. The bottom plots of Figures 25 and 26 suggest that the very subtle -10 mNm peak detents and friction were successfully fit. Additionally, the 95% CI plots of Figure 27 suggest a dominance of appropriate signal data over noisy data such as the "salt and pepper" noise sprinkled throughout these plots. Knob  detents  1  mocierate  Figure 28 clearly shows the larger amplitude and  lower frequency detents of K n o b detents-  detents  a s  compared to Knob h i su  t  e  Comparing Figure 28 to Figure 29, we can see that the detents domi-  nate the feel of K n o b detents  m o d e r a t e  moc  j . er  ate  detents-  This domination contrasts with Knob h i su  t  e  where the detents and friction had similar contributions to the knob's  feel. The bottom of Figure 29 is a considerable improvement over the top of Figure 29, but the ghosting present in this plot, and the bottom of Figure 28, suggest increased difficulty fitting these data compared to equivalent data for Knob derate detentsmo  Nevertheless, the fit was relatively successful, as sug-  gested by the good 95% CI plots of Figure 30. -sine detents'  Figures 31 and 32 show the difficulty attempting to  fit data to the Equation 6 model. For example, before performing a capture, manually turning K n o b  non  .  sine  detents  revealed significant backlash and non-  sinusoid detents. For example, physical slips due to the considerable backlash can be seen between 0.1- 0.2 rad in Figures 31 and.32. These torque vs. position fits were also quite sensitive to subtle changes in the fitting procedure such as when initial conditions were needed by the l s q c u r v e f i t algorithm for the non-linear model parameters. Nevertheless, the fitting procedure could 68  "see beyond" the backlash and fitted a sine wave of appropriate frequency to the captured K n o b  non  .  sine  detents  data. As one would hope, these <positional •  effects were successfully isolated from the velocity and acceleration effects. The torque vs. velocity plots in Figure 33 shows a good fit to the friction model parameters even though some aliasing can be seen in both the Raw r and Raw T [ plots. Such aliasing was not present in comparable plots for the other ve  knobs. As a further indication of difficulty fitting, the magnitudes of the 95% CIs from the fitted torques in Figure 34 were more than 10 times greater than the other knobs. Overall, the modest success fitting Knob  n o n  . i s  n e  detents  s  u  §~  gests a promising robustness to the fitting procedure since the feeling of Knobnon-sine detents  s o  clearly deviated from the model structure of Equation 6.  Figure 35 gives a detailed zoom of the torque vs. velocity data plot to show how the static friction parameters were estimated for each of the five test knobs. The density of Raw T ^ data points abruptly drop beyond ±0.01 rad to v e  indicate the appropriate Av boundary. Similarly, the densities for torque drop off beyond ±10 mNm to indicate the appropriate D ^ and D  vel+  boundaries;  although, in this example, the density drop-offs for the D j stick-slip boundaries were less clear than for Av. Some noise is visible near ±10 mNm in Figure 35, but is virtually non-existent elsewhere in the plot. Overall, a clear match to the Karnopp friction model is evident for this example knob containing very subtle amounts of friction.  69  40 30 20  E  1 0  vel+  3 CT  -10  vel+  -20 -30 -40  -0.06  -0.04  -0.02 0 0.02 Velocity (rad/s)  0.04  0.06  Figure 35: Example static friction parameter estimation on the torque vs. velocity data from K n o b subtle  d e t e n t s  2.3 Comments on the Characterization Process The most important characterization principle explored during the above dynamics capture procedure was the need to balance the strengths of human experience with the precision and consistency of a computing system. For example, most physical controls have considerable non-linearities, but appropriate non-linear curve fitting techniques require currently unsolved mathematical problems. Until, and probably after, such math problems are solved and refined for practical numerical analysis, human intervention in the capture process will greatly enhance the practical fitting of dynamic model parameters. For example, Matlab's system identification toolbox helps designers iteratively create and refine linear dynamic models from measured inputoutput data [55], although creation of custom Matlab scripts and user interfaces were found to be more useful than Matlab's system identification user  70  interface for the data fitting described in this thesis. Better tools for involving humans in data fitting processes are clearly needed. The following sub-sections summarize characterization issues for physical controls according to several broad categories.  2.3.1 Human versus Machine Characterization Strengths The key characterization strategy is to let the humans do what they do best (provide context), and the apparatus do what it does best (precisely execute commands). For example, as we'll see in subsequent sections, the detents in Knob bti su  e  detents  were very subtle and easily confused by novice users to be  a kind of friction. The fitting procedure, however, had no difficulty identifying and segmenting the detent and friction components of the knob. Conversely, humans could easily identify severe backlash in K n o b  non  .  sine  detents  m  a  t  was  not included in the fitting model. Consequently, initial fits using the same procedure used for with K n o b  surjt  i  e  detents  yielded poor results. A n experienced  person could intervene in a computational process to improve practical capturing results: • Prior to exploration:  Before capturing data, manually feeling the  physical control to identify the dominant physical components such as detents, friction, inertia, backlash, and logical regions. • During iterative  refinement:  Iteratively refine curve fitting settings  such as maximum number of function counts, iterations, initial conditions, and spatial regions of interest.  2.3.2 Perceptibility of Specific Phenomena Curve fitting generally becomes more complex and less reliable with the addition of every parameter to the model. For example, can the subtleties of a Stribeck friction model be practically felt by most users? If not, attempting to capture and render such subtleties might unnecessarily jeopardize fitting of parameters that do have a significant impact to the user. Which parameters  71  need linear modeling, and which ones need non-linear characterization algorithms? Linear fits are generally faster to converge, more reliable, and do not need initial conditions compared to non-linear fits. Are there detents? Are they regularly spaced? Are some detents of greater amplitude than others? Do they feel more like a sine wave or a triangle wave?  2.3.3 Fusing Multiple Characterization Techniques Can some parameters be "seeded" or augmented using parameterizations or model structure insights derived from°another characterization technique? For example, visually examining at the back of K n o b  non  .  sine  detents  (  s e e  Figure 36) can provide several hints for choosing a good model. The frequency of detents can be determined by counting the number of grooves on the white ring. Such a frequency calculation was used to determine which of the two competing torque vs. position fits were correct: Figure 31 or Figure 32. Thus, ^pos  a  n  d  ^pos  c  o  m  d  have been determined before attempting a data fit to  focus fitting efforts on fitting the other model parameters that were more difficult or impossible to estimate using other methods. Alternatively, parameters such as Ppos and Spos could be left as variables before fitting,' and then used a  an independent "reality check" on the success of the overall fit. As an additional exercise to check the validity of the fitting procedure such a reality check is illustrated below. Each of two measured values of P  pos  for the knobs with detents were averaged then multiplied by 2% to  obtain measured period values. "Gold standard" numbers of detents for Knob  subtle  detents' K^tJmoderate detents'  a  n  d  ^  n o D  n o n - s i n e detents  then calcu-  w e r e  lated to be 30,12, and 12 per revolution, respectively. Dividing by 2K yields the periods as shown in Table 8. Values for K n o b detents  subt  matched quite well, and the match for K n o b  considering K n o b  non  .  sine  detents'  s  i  e  non  detents  .  sine  a n d  K  detents *  s  n o b  moderate  acceptable  previously mentioned non-conformities to 72  the fitting model. It is also important to note that the two averaged values for Knob btie detents  a R  su  d Knob  m o d e r a t e  two measured estimates for K n o b  detents non  .i s  ne  were almost the same, whereas the detents  over- and under-shot the  "gold standard" value by about 50%.  Table 8: Periodicity check for test knobs with detents  Period  Subtle Detents  Moderate Detents  Non-sine Detents  Measured (rad)  0.216  0.476  0.640  "Gold Standard" (rad)  0.209  0.524  0.524  ate detent model (e.g., 12 "clicks" per revolution and non-sinusoid grooves)  73  2.3.4 Dynamic Range of Sensors Our Haptic Camera's torque sensor is an example where dynamic range significantly limits characterization quality. The best available torque and force sensors can only reliably provide 1000 data points between their minimum and maximum calibration settings. The torque sensor used for the capture of the above five knobs was very carefully chosen to span an optimal range of torques. Here, we used a coarse torque sensor to isolate the maximum torques required for each knob before purchasing and using a more sensitive second torque sensor.  2.3.5 Dividing a Complex Problem into Simpler Sub-Problems Can the physical control be logically broken into different spatial regions? If so, a collection of simpler models could be used instead of one more complex model. For example, a more sensitive torque sensor could be used to explore low torque regions of a physical control, or higher order effects such as frequency and inertia could be initially estimated by first analyzing flat spatial regions on a physical control containing detents. Colton and Hollerbach [20] suggested weighting techniques that would ideally guide a fitting algorithm towards a solution by favouring regions with less noise.  2.3.6 Choosing Appropriate Model Complexity Is friction a dominant component? Inertia? Detents? Because each additional parameter into a model will reduce the quality of fits to that model, the best practical characterization will often be to a simple model where the most dominant parameters should be included in a model — as opposed to a complex more physically accurate model that is extremely difficult to properly characterize. A Stribeck friction model could be more successfully captured without the presence of strong inertia or detents. However, in a physical con-  74  trol with significant inertia, detents, and backlash, maybe a simple damping constant would be more appropriate. How many input and output degrees of freedom? For this research, a 1 DOF DC motor and 3 input sensors (position, acceleration, and torque) were used. Consequently/more focussed identification techniques could be successfully applied compared to arrays of multi-DOF systems, (e.g., linear algebra to deal with relative positions of multi-link assemblies and singularities were not needed).  2.3.7 Dealing with Practical Data Capture Issues What imperfect real world problems arose during data capture? For r  example, did a transducer saturate during one phase of data capture? Over which spatial or temporal regions of the physical control being captured was closed-feedback control most stable?  1  What were the resolutions of the data and desired model(s)? For this research, high-quality transducers were used. Consequently, explorations of subtle torques could be successfully explored. Many commercial applications would use cheaper, lower quality transducers requiring different modeling parameters. For example, smoother bearings in higher quality DC motors can be more difficult to control compared to stiffer bearings in lower quality DC motors. Such stiffness acts as a mechanical filter, reducing undesirable highfrequency vibrations. Also, if a target actuator is incapable of effectively producing the output commanded by a model, or the user will not be able to perceive the model's nuances, a simpler model could produce a better feeling haptic system. Smoother renderings could be produced using a Karnopp friction model instead of the more mechanically accurate Stribeck friction model if the system lacks the position and torque resolutions needed to stably produce and control the Stribeck model — as is the case with most current systems.  75  2.4 Noise Source Summary Future haptic camera apparatus could reduce noise and improve capture quality by addressing mechanical, electrical, and algorithmic uncertainties such as:  2.4.1 Mechanical Improvements, • Use a stronger (stiffer) bond between the knob gripper and test knob. For example, applying glues or physical screws into the test knob might permanently alter or destroy its cap; but, one would achieve a stiffer coupling that is more resistant to oscillations and hysteresis when the haptic camera is in motion. • Improve the alignment mechanism between the haptic camera sensors and the test knob. For example, one could use a 6 axis torque sensor to monitor and compensate for axis misalignment. Compensation could include finely adjusting a physical chuck grounded to the test knob or algorithmic post-capture processing of the collected data. • • Develop and implement higher-performance sensor and actuator .  materials. For example, the torque sensor used for the current haptic camera has a mass of 100 g. A lower mass torque sensor, such as a custom designed strain gauge, would enable the DC motor to command higher bandwidth excitation signals. Improved strain gauge materials engineering would yield more accurate torque sensing because hysteresis reduces resolution of current strain gauge technologies. Similarly, improved micromachining processes would yield micromachined accelerometer beams capable of sensing higher resolutions.  76  2.4.2 Electrical Improvements • Improve electrical shielding of sensor cables and actuator supply cables. For example, the DC motor produces electrical noise that can influence the sensitive uV to mV level signals from the torque and acceleration sensors. Capacitors on the motor supply lines, medicalgrade sensor cable shielding, and analog filtering effectively reduced electrical noise in the current haptic camera apparatus; however, further refinements could improve the resolution of captured data. • Reduce the number of electronic components and cable lengths. The current haptic camera apparatus covers about 1 m of desk space. A 2  smaller package could be obtained by re-arranging existing components and removing unnecessary components, such as printed circuit board routing and sockets for unused channels of the I/O board. The resulting package would typically pick up less noise.  2.4.3 Algorithmic Improvements • Repeatedly conduct more independent samples of the same test knob until desired measures, such as mean and 95% confidence interval, have stabilized within a level of error deemed appropriate by the experiementer. For example, some parameter estimates for K n o b detent  m  sine  .  Table 7 on page 51 vary by an order of magnitude. Addi-  tional samples would improve confidence in the final estimated mean value. • Perform non-linear least-squares fits (e.g., l s q c u r v e f i t in Matlab) with different models of varying complexity and different initial conditions. For example, one could attempt to use a different prototype position function than a sinusoid when fitting the detents of  77  Knob j _ j s  ne  c  et:ent  (refer to Equation 6 on page 29).  • Implement software with better real-time performance. Although absolute computer clock frequencies and storage access times con, tinue to improve, microsecond (or smaller) update inconsistencies can lead to perceptually noticeable haptic rendering errors. • Devise and use better curve fitting algorithms and procedures. For example, for a knob dominated by inertia, one could explicitly measure the inertia's admittance as a function of frequently. This would enable one to better estimate the frequency range over which the Haptic Camera made good measurements.  2.5 Characterization Summary We fitted inertia, friction, and detent models to two simulated and five real mechanical test knobs. Characterization of a noiseless simulated knob was nearly perfect. Characterization of a simulated knob, with greater average noise than the noise generated by our Haptic Camera apparatus, contained small errors listed in Table 4 on page 47. The worst observed error for this noisy simulated knob was a modest < 1% phase lag. Overall, the characterization procedure was deemed successful from a mathematical viewpoint, and appropriate for fitting real mechanical knobs. Five mechanical test knobs were then fitted to a physical model with 8 parameters (1 acceleration, 4 velocity, and 3 position). The fits for Knob ig n  tiorv  K n o b  h i g h inertia' ^ o b s u b t l e detents'  a  n  d  K ^ b m o d e r a t e detents  w  e  r  e  n  f i . r  c  a 1 1  deemed to be of good quality because the 95% confidence intervals of their resultant torques were all less than 0.5 mNm. Even K n o b  non  .j s  ne  detents' which  was known a priori to have much backlash, non-sinusoidal detents, and other deviations from our fitting model, was fitted with reasonable accuracy. Fitted torques for K n o b _ non  sine  detents had  a 95% confidence interval of less than 5  mNm. We used detent period as an independently measurable check of our  78  fits for the five mechanical test knobs. The number of "clicks" per revolution were counted for K n o b detents'  m  e  n  subtle d e t e n t s  , Knob  m o d e r a t e d e t e n t s  , and K n o b  non  .  sine  compared to the fitting results. These independent estimates of  detent period had 3.0%, 9.5%, and 23.5% relative accuracies, respectively. At this point, we do not know whether the fitting qualities for these five mechanical test knobs are acceptable to users. User studies are needed to judge the quality of these characterizations in terms of subjective experiences of humans. Chapters 3 and 4 conduct and analyze such user studies.  79  Chapter 3 Rendering and Validation A key assumption underlying the work reported in this thesis is that active haptic rendering can mimic not only the physical characteristics of a mechanical knob but also elicit a comparable subjective experience from a human who uses the knob. Chapter 1 described the Haptic Camera that we developed to measure the engineering parameters of a knob. Chapter 2 described how we process those measurements to produce a real-time characterization of a knob. This chapter describes a particular rendering system (hardware and algorithm) that uses these models, and tests the hypothesis that the rendering system accurately conveys to the user the "feel" of a knob given its model. This is accomplished through a user study in which users matched an active haptic rendering to a passive mechanical test knob by adjusting the parameters of the model for the active knob. Both quantitative and qualitative results from the experiment support the hypothesis that the renderings evoke a subjective experience similar to the mechanical knob. We first describe the rendering system, then we describe the user study and interpret the results of the study in terms of the stated hypothesis. We close with a discussion of how the user study might be improved, and how the choice of parameters in the rendering model might have influenced the results of the study. These studies help us better understand the ability of people to tune parameters in haptics, and provide insights into what people feel and how they know they've felt it.  3.1 Introduction Figure 37 illustrates the haptic knob apparatus. It uses the same components as the Haptic Camera (description started on page 12), except the torque sensor and accelerometer attached to the motor's shaft were replaced with a  80  custom made 60 mm diameter x 15 mm deep polycarbonate knob. The torque sensor was not required for closed-loop feedback because calculated model torques could be used with good accuracy, as verified in the torque test results beginning on page 46. Although an accelerometer may have helped provide more accurate rotational acceleration readings, the accelerometer was not used because its cables restricted the knob's rotation. This choice favored the user's ability to freely rotate the knob over slightly improving acceleration sensing.  Figure 37: Rendered haptic knob apparatus  3.2 Balancing Machine and Human Capabilities In recent short-term memory research, it has been observed that people have a single central memory capacity limit averaging about 4 chunks of information [22]. In contrast, our Haptic Camera's Karnopp friction model and detent model (refer to Equation 6 on page 29 and Figures 13-14 beginning on page 31) respectively contain 7 and 3 parameters. Clearly, a compromise between realistic human capacities and multi-parameter mathematical models is needed for user study participants to adequately understand and manipu-  81  late the physical models captured by our Haptic Camera. We therefore sought ways to simplify the capture model to make it more accessible to human subjects, while retaining control over the most perceptually important components: • Inertia can effectively be modeled, from both machine and human perspectives, with a single parameter virtual mass [19,78]; so,  M  Qcc  should clearly be retained as one of our free parameters. • Display of friction, modeled using a Karnopp model, represented an opportunity for parameter reduction, because it contained 7 parameters in our capture model. It is commonly believed that the stick-slip boundaries (C /+ and C ^ ) and damping components (B j and ve  ye  +  B _) of the Karnopp friction model are the most perceptually releve!  vant. These friction components were the focus of previous characterization research such as Richard et al. [78]. Thus, for the present purposes we dropped three of the model components [D i ye  +  > ^ /. / ve  and Av), combined the two stick-slip boundaries ( C / and' C _ ) ye  +  yg/  into C i , and combined the two damping components ( ^ / and ve  ve  B _) vel  into B  vel  +  , that were present in the capture model.  • We were also able to simplify the detent model, given the specific knobs that we modeled. Our three test knobs containing detents all had relatively high detent frequencies of at least 12 detents per revolution. Consequently, we safely removed the phase shift parameter, S  n  , from the detent model that we presented to user study partic-  82  ipants. Together, these simplifications left us with 5 parameters (1 inertia, 2 friction, and 2 detent) — Cowan's [22] recommended upper bound for the number of percepts one can handle simultaneously. To verify and possibly extend the most aggressive of these friction model reductions, we then conducted an informal pilot study with 4 participants to test the relative contributions of the Karnopp stick-slip boundaries and damping slip states. All participants were graduate students in the Computer Science Department at the University of British Columbia. Participants were instructed to adjust 5 physical sliders that modified the feeling of a forcefeedback knob (refer to Figure 39 on page 91). The sliders respectively changed the 5 parameters introduced in the previous chapter along unit ranges similar to those obtained by the Haptic Camera characterizations of the 5 test knobs described in Chapter 1. The participants all had extreme difficulty perceptually discerning the different stick-slip friction boundaries. However, they were easily able to discern and modify the damping slip state of a Karnopp friction model rendered on the haptic knob. Therefore, we further simplified our friction model to consist of only the damping element of the original Karnopp friction model. That is, we modelled friction as pure damping because users could not distinguish pure damping from damping with stick-slip boundaries. The stick-slip states were not displayed even as fixed parameters (not present or adjustable by the users), on the grounds that any non-perceptually justified model complexity undermines our ability to stably render the whole second order model. This simplification resulted in four perceptually influential model parameters (1 inertia, 1 friction, and 2 detent) for our user studies in this chapter and Chapter 4. These parameters balanced both machine and human capabilities. Chase and Simon [16] observed that expert chess players more readily chunked information than novices. In line with such research, this chapter describes both novice and expert studies using our simplified 4-parameter  83  model. The expert study was designed to probe more deeply into the underlying physics of the haptic knob, whereas the novice study was designed to probe typical responses of average end-users of physical controls. Our rendering model and user studies are described in further detail below. 3.3 R e n d e r i n g M o d e l  Figure 38 shows the rendering model used to generate torques on the haptic knob shown in Figure 37 (compare with Equation 6 on page 29). The model is visually organized into three layers to illustrate the three intuitive components of the model — detents, friction, and inertia terms — that closely correspond to the model used during the capture of mechanical knob dynamics (refer to Equation 6, page 29). Key attributes of the model include: • Sine equation for creating position-dependent detents of varied amplitude and period. • Damping constant corresponding to the "slip" state of the Karnopp model used during captures. • Inertia simulation using a virtual mass as suggested by Colgate & Schenkel [19] and previously used by Brouwer [11] for a mass simulation of a haptic laparoscopic tool.  84  Detent Layer  r  sin(G/P ) pos  -pos  G  Low Passi  de/dt  \  )  Friction Layer  e J  *  ++  'vm  'vm  vm  Figure 38: Haptic knob rendering model The model shown in Figure 38 has several "simplifying" attributes relative to the captured model parameters. These "simplifying" attributes reduced the number of model terms towards a number (4) that was more manageable for participants to quickly comprehend and manipulate. The simplifications also slightly improved the stability of the renderings. Changes compared to the captured model parameters include: • Phase shift variable S  was removed from the detent model com-  pos  ponent. • Damping variables from the Karnopp friction model, B  yel  B l+ ve  and  / were averaged to create an overall damping factor of B  ,  vel  and the "stick" state of the Karnopp model was ignored. • The accelerometer was removed from the shaft in favor of a virtual mass simulation [19] where acceleration is calculated by double differentiating the encoder position. This model had the main rendering advantage that position-, velocity-, and acceleration-sensing could all be determined with a single high-quality optical encoder.  85  • Conservative virtual mass constants K B  = 12 000 Nm / rad and  = 88 Nm / rad/s were chosen and fixed for user trials.  vm  • "Low Pass 1" was the same IIR Butterworth filter described in Section 1.2.2, "Velocity Sensing" on page 14, but a second filter "Low Pass 2" was added to the virtual mass rendering. "Low Pass 2" was simply the mean value of the last three r  acc  updates (box filtering).  3.3.1 Psychophysical Appropriateness of the Rendered Model The models we are using for haptic camera capture and consequent rendering are similar to psychophysical models used by other researchers to describe kinematic movement of a person's hand during a rotation task. Our model describes the dynamics of a physical control, not of the user's hand. However, we argue that if our knob model effectively encompasses state-ofthe-art hand motion models, then it has the potential to feel perceptually complete. Equation 13 is a non-linear mass-spring model of movement used by Novak et al. [71] to describe rapid hand movement experimentation with a passive rotary control. Novak et al.'s model represents typical human wrist motion, and is applicable for describing complicated finger and wrist turning motions associated with knob turning tasks. In the models used in this thesis, we omitted the exponential damping term of 0.2 because its exact value is disputed (as described in Novak et al.'s discussion of related work [71]).  +  acc  M  0=  e  a c c  *>'°- +  vel. part ( r )  2  ' pos AB-6 ) eq'  ve/  K  y  n  Bn  -Part(W  r  < ) 13  pos. part ( pos' T ) r  where the parameters of this biomechanic human hand model are pos' vel' *  T  T  anc  acc  T  a  r  et  n  e  position, velocity, and acceleration torque  86  parts that sum together to form r. 6,6,  and 6 are the rotational position, velocity, and acceleration.  M  is an acceleration constant, intuitively similar to inertia.  Bi  is a velocity damping constant, intuitively similar to friction.  acc  ve  K  is a positional spring constant, intuitively similar to stiffness.  3.3.2 M a x i m u m R e n d e r i n g C a p a b i l i t i e s o f t h e H a p t i c K n o b  Table 9 lists the maximum dynamic response constants that could be continuously rendered at a 5000 Hz update rate using the haptic knob. Excluding commanded renderings, the haptic knob mechanism, including the motor, had negligible friction and a low inertia of 0.013 mNm / rad/s . The knob had 2  small axial and radial plays as listed in Table 10. Subjectively, the detents and damping had low amounts of perceived instability [17] (there was an absence of unmodelled jitter and activeness; the detents, of course, comprised modeled bumpiness). At high inertia settings, the spring constant for the virtual mass, K~ , could be felt when making vm  rapid motions with the knob. Also, the inertia rendering introduced a "perceived instability" that felt like a rough texture of a few mNm in magnitude; however, this texture felt similar for all slider settings. Table 9: Maximum continuous torque responses of the haptic knob Torque constant  M a x i m u m continuous torque  Position  180 mNm  Velocity  45 mNm / rad  Acceleration  1.5 mNm / rad/s  2  Table 10: Axial and radial play of the haptic knob at the handle Knob  shaft play  Axial (motor shaft)  M a x i m u m values .05 - .15 mm  87  Table 10: Axial and radial play of the haptic knob at the handle  Knob shaft play Radial (ball bearing)  Maximum values .025 mm  3.4 Haptic Matching Experiments User studies were performed to better understand the relationships between automated and human-centered parameterizations of physical control dynamics. Certain components of any mathematical model and physical control apparatus will match particular psychophysical models of novice and expert human participants more appropriately than others. The following user studies are steps toward understanding how to design physical control dynamics that better take into account human psychophysics. The primary research questions for these studies are: What human responses to dynamic properties can and can not be accurately matched by an automated system identification process (Haptic Camera)? What aspects of dynamic properties do people rely on when forming their percepts ? What are the relative strengths and weaknesses of the automated and human system identification processes?  Both novice and expert participants were recruited to explore the typical and maximum human limits of dynamics estimation, respectively. These parameterization user studies closely parallel previous research in graphics. For example, Bartels et al. [10] performed parameterization studies using graphical shape matching tasks to test human-centered interaction styles for a collection of mathematical representations of graphical curves. The approach taken with these matching user studies was to have participants adjust physical parameters of inertia, friction, and detent models of a rendered knob. Participants iteratively adjusted and felt the rotation of the rendered knob until it felt as similar as possible to a static mechanical test knob. Participants actively adjusted physical parameters for the rendered knob. Our study design serves as an excellent structured task for qualitative data collection and a good task for quantitative data collection. This approach  88  does however have a few limitations. Most notably, the short term memory of a typical participant can only process a small number of physical parameters (< 4), and each comparison is time consuming (up to 2-5+ minutes). Requiring participants to handle high cognitive loads can lead to greater uncertainty in the user studies results. Refer to Section 3.5.2, "User Study Enhancements"on page 112 for a discussion of additional possible user study designs.  3.4.1 Method The participants, apparatus, and procedure for the user validation study are described below.  3.4.1.1 Participants: Novices and Experts We recruited both novice and expert participants. Novices were chosen because they represent a typical user's sensitivity and vernacular understanding of how detents, friction, and inertia feel. Experts (individuals with training in mechanical systems and models) were chosen to explore the bounds of human perception of mechanical control dynamics. Experts had a heightened awareness of how underlying physics and mathematics change the feeling of detents, friction, and inertia, as well as language to verbalize these percepts. For example, experts understand the differences between Karnopp and Stribeck friction models, understand that detents can be modeled with torque versus position sinusoids, and that inertia is a predominantly accelerationdependent effect. Novices relied solely on their daily experiences with physical controls such as knobs, whereas experts also relied on their thorough understanding of mechanics. Even if novices and experts were both able to create equally good mental.models of a haptic behavior, experts would typically be able to more clearly articulate and describe their mental models. The procedures for the novice and expert user studies differed slightly in an effort to better utilize their respective skill sets. These procedures are described in Section 3.4.1.3, "Procedure" starting on page 93. Generally, experts were given more freedom and more background information than the  89  novices in an effort to focus on more subtle and refined dynamic attributes of the physical controls. Right-handed, paid participants were individually tested in the two parts (novice and expert) study. Both parts took approximately one hour to complete. Fifteen novices (10 female & 5 male) with ages ranging from 20-29 years (M= 24.7, SD= 2.8), and five experts (3 male & 2 female) with ages ranging from 23-31 years (M= 27.2, SD= 3.2) participated. Novices were students or staff at the University of British Columbia. Experts were graduate students or post-doctoral students from Mechanical Engineering laboratories at the University of British Columbia employed in haptic-related research projects. None of the experts were directly affiliated with, or knowledgeable of, the author's research.  1  3.4.1.2 Apparatus Participants interacted with the apparatus shown in Figure 39. By adjusting four physical sliders, participants changed the dynamics of a haptic knob to match the dynamics of five mechanical test knobs (one at a time) to the best of their ability. These five test knobs were labeled sequentially with letters " A " - "E". The four physical sliders controlled magnitudes of the four rendering parameters M ,  B ,  QCC  YE[  A  PQS  , and P  PQS  in Equation 14 (see page 91), as  applied to the current virtual knob rendering. Table 11 lists the minimum (bottom) and maximum (top) slider settings. Table 11: Slider value ranges  Slider  Minimum -> Maximum Values 0 -> .75 mNm / rad/s  M acc  2  vel  0 -> 30 mNm / rad/s  pos  0 -> 15 mNm  B  A  P  -> 5 (0 —> 32 detents / revolution) oo  pos  90  Figure 39: Apparatus for human system identification  M  acc  6 +  acc. part (r r  v  ) acc'  vel. part ( r ) yg/  (14)  pos  where the parameters of this haptic knob rendering model are t is the torque rendered to the force-feedback knob. pos' vel'  T  r  a  n  d  acc  T  P  a r e  o s m o n  ' velocity, and acceleration torque  parts that sum together to form T. 6,6, M  and 6 are the rotational position, velocity, and acceleration. is an acceleration constant, intuitively similar to inertia.  acc  B i is a damping constant, intuitively similar to friction. pos  A  a  n  d  ^pos  a r e a m  P u d e and period parameters for rendering u t  detents. Results from the previously described pilots with novices and experts (see page 83) suggested participants experienced too high of a cognitive load when altering any more than four or five knob variables at the same time. Consequently, allowing users to modify all the position, velocity, and acceler-  91  ation dependent variables in the captured model would have been overwhelming to the participants. As illustrated in Figure 39, the sliders were visually chunked into a pair of physical sliders for modifying M  acc  and  B , vel  separated by an empty slot, and a pair of physical sliders for modifying A and P  . The sliders for M  QCC  and B  yel  independently adjusted the respec-  tive feelings of inertia and friction, whereas A  and P  worked together  to adjust the detents (refer to Equation 14). The mechanical test knobs were organized according to this "division by detents". Knob jgh friction (labeled n  "A") and K n o b Knob  subtle  nign  i  n e r  ti  a  (labeled "B") did not have detents, whereas knobs  detents' Knobmoderate detents'  a n  (labeled "C", "D", and "E"). Sliders A needed to model K n o b  high f r i c t i o n  d Knob  non  .  sine  and P  and K n o b  i n e r t i a  bad detents  were therefore not  pQs  high  detents  . <  For the qualitative aspect of the study, participants were also given sticky notes and a pen, then asked to label the sliders with descriptive keywords. The five test knobs (refer to Table 5 on page 48) captured by the previously described Haptic Camera apparatus were each provided with matching smooth, white ABS plastic caps. The test knobs were organized along a row beneath the haptic knob, which was also provided with a similar plastic cap. Each plastic cap measured a 70 mm diameter, 16.5 mm depth, and 3 mm filleted edge. These caps ensured participants compared only the dynamic properties of the knobs, not textural surface properties on the handle. Exposing participants to the surface textures of the test knobs would have introduced additional haptic noise, and visual multimodal effects, into the comparisons. Disguising the identity of the active haptic knob from the participants would eliminate chances of participants being influenced by their preconceived biases towards either an active knob or a passive test knob. A randomized layout designed to disguise the identity of the haptic knob was not used  92  because the rendered knob would quickly become apparent to the participant with any such layout. Participants would readily determine a controlled active knob within a set of test knobs because an active knob would change its dynamics as the participants adjusted physical slider settings. But, the test knobs would not change.  3.4.1.3 Procedure The experimenter manually reset the physical sliders to their off (bottom) positions at the beginning of each session and individual trial. In a familiarization phase, participants were instructed to explore the effects of each slider on the haptic knob. They were first instructed to alter M , Qcc  then  B . yel  Next they were instructed to move Apos and Ppos near the middle of each J  slider's range, and observe the effects of each position-based slider. Participants explored the effects of each slider on the haptic knob until they felt comfortable and confident using the apparatus. They then wrote down keywords on sticky notes to describe each slider's effect on the haptic knob. Each of these sticky notes were affixed beneath the appropriate physical slider. During this apparatus exploration phase, the experimenter aurally described the underlying physics (mass, damping, detent amplitude, and detent frequency) modified by each slider to the expert participants, but not to the novices. The experts would likely be able to determine the underlying physics themselves; so, explicitly telling the experts these underlying physics allowed us to more quickly progress towards studying more interesting, subtle knob attributes. In an effort to minimize participant bias, none of the participants were told whether the five test knobs were mechanical or mechatronic (force-feedback) knobs; nor were specific inertia, friction, or detent properties of the five test knobs discussed with any of the participants. For the novice participants, the knobs were tested in two groups: (i) without detents, and then  (ii) with detents. The order of knobs was randomized  within each group and only the relevant sliders were made accessible for each  93  group (M  adc  P  and B  ye[  for knobs without detents, and M ,  B  QCC  v e l  and  , A  pQs  for knobs with detents). This ordering of studying knobs without  detents before knobs with detents was justified due to the benefits in learning accrued from gradually increasing the task's cognitive load (compensating for cognitive load differences was deemed more important than the possible introduction of small memory biases). Novices were instructed to take as long as they desired (typically about two minutes) to adjust the. A/  and B ^ v  sliders to-match each knob a total of three times. First, participants were required to match either K n o b i n  gri  friction  then Knobhigh inertia'  o r  vice-versa  (knobs without detents). After performing three repetitions with each of the two knobs without detents (Knobhigh friction and Knobhigh inertia)' they were then instructed to adjust all four sliders to match three repetitions with each of the knobs with detents ( K n o b  subtle  det  ents'  K n o b  m o d e r a t e detents'  a  n  d  Knob . non  sine detents) •  For each repetition, a randomized ordering of the knobs was presented to the participant. A trial consisted of using physical sliders to match the "feel" of the active knob to match the "feel" of a test knob as closely as possible, then rate how similar these two knobs felt. For all trials, participants were instructed to rotate the knobs with their right (dominant) hand, and adjust the sliders with their left (non-dominant) hand. This protocol prevented additional noise in the collected data caused by perceptual and/or cognitive differences related to right and left hand usage. After each trial, participants were asked to rate how satisfied they were with the match between the rendered haptic knob and the mechanical test knob. Participants gave a rating between 1 for strongly agree and 9 for strongly disagree to the question, "I am satisfied with the match between the rendered and mechanical knob." Experts followed the same procedure as the novices, except the experts were instructed to (i) adjust all four sliders when matching all five test knobs, and (ii) perform one very careful block consisting of a randomized ordering of 94  the five test knob trials, in lieu of three rapid, repeated trials. Adjusting all four sliders for all five test knobs was not believed to be a burden because the experts were trained in mechanical systems and models, and they had more time to explore these sliders compared to novices. We felt it was appropriate to ask experts to only perform one more careful block (taking more time per matched knob) because experts, being experts, were less likely to learn about the knob models during the trials. Furthermore, experts were asked to verbalize their current thoughts and strategies during the experiment (think aloud protocol [29]). These expert comments were transcribed by the experimenter for future qualitative analysis.  3.4.2 Results Figure 40 illustrates how satisfied novice participants were with how closely each final knob rendering matched each of the five test knobs. Participants gave favorable satisfaction ratings for all test knob matchings [M= 2.5, SD= 1.0] for a range of 1-9 with 1 being most satisfied, but significant rating differences between the knobs were not observed. Pairwise comparisons between the satisfaction ratings for the knobs were tested using a typical non-parametric test, a Wilcoxon Signed-Ranks test with Bonferroni correction. Significant differences were found between: • Knob  high  f  riction  - Knob  non  .  detents i  Z  sine  =  • Knobhigh friction " Knobmoderate detents i K  • Knobhigh inertia "  Knob  non  _  sine  detents [  -  = 2  = -  Z  2  moderate detents [ . Z  '  5 8  -'  Z  • Knobhigh inertia " n o b  2  =  i7  4 3  V  '1 01  <  V  <  -] 03  ' V<  - ]  V  - 1  I- ' 61  <  015  10  95  These significant pairs are marked in Figure 40.  | H Ui  CO  I o  .52  TO D) c  TO 4H  or g  3H  0)  2-i  <  J>  No Detents O-O O—O •—•  Cr  Detents  S i g n i f i c a n t at . 1 0 level (2-tailed) S i g n i f i c a n t at . 0 5 level (2-tailed) S i g n i f i c a n t at .01 level (2-tailed)  Figure 40: Novice ratings of satisfaction for how closely each rendered knob matched its target mechanical test knob  96  Figure 41 illustrates the relationships between expert participants, novice participants, and Haptic Camera parameterizations of the five test knobs. To better compare participant slider settings, the dependent axis of Figure 41 is scaled to the minimum and maximum stable operating levels for the haptic knob. Table 12 lists the equivalent physical units that map to unit slider values of 0 —> 1 for the four sliders (for use in reading Figure 41). Error bars (a = .05) are shown for expert and novice participants. Such error bars were not calculated for the automated system identification (haptic camera) values because only two independent captures were obtained. Nevertheless, parameter value differences and overall 95% confidence intervals for the dependent variable (torque) are shown in Table 7 on page 51. Error bars are not displayed for the independently obtained 'gold standard' values because they are, by definition, zero ('perfect' data). Table 12: Physical units that map to standardized slider values of 0 —> 1 (for use in reading Figure 41) Slider (from left to right)  Parameter  Inertia  Physical m a p p i n g 0 -» .75 mNm / rad/s  acc  M  Damping  vel  0 - 4 30 mNm / rad/s  pos  0->15 mNm  B  Detent Amplitude  A  Detent Frequency  P  2  0 —> 31.4 clicks / revolution  -  1  pos  The two leftmost shaded columns of Figure 41 display slider settings for knobs without detents — Knob igh n  friction  a  n  d  K^Dhigh i  most) column displays inertia parameter settings, M  n e r t  i a  The first (left-  , for these two knobs,  and the second shaded column displays friction parameter settings, these two knobs. A.  and P  pos  , for  are not displayed because they are only  used exclusively for knobs with detents. For example, looking at the leftmost  97  shaded column, circles, squares, and stars respectively represent the novice, expert, and automated (Haptic Camera) parameter estimates for inertia, of the two knobs K n o b  high f r i c t i o n  and K n o b  M  acc  high i n e r t i a  The four rightmost shaded columns of Figure 41 display slider settings for knobs  with  detents — K n o b  subt  i  e  detents' K n o b m o d e r a t e detents'  Knob  non  -sine  98  4) Automated system identification •fa Expert human identification U  Novice human identification  d^p Independently obtained value (when available)  .8 --  I " 'I -67  a  C .3H ID .2.1 - .0 jar  6® 6«* if vs- A * vf>  tr ef  7/ vV Damping (Friction) (Bvel)  Inertia (/Wacc)  <*•  &  tf  •cr  77  Damping (Friction) {Bvel)  Inertia (Mace)  vs*  JOT  Detent Amplitude (Apos)  vS"  W Detent Frequency  (Ppos)  Sliders  Figure 41: Comparisons of expert, novice, and Haptic Camera parameterizations for all five test knobs. Human parameterizations were performed in clusters, setting only the parameters present in those knobs. detents-  M  acc  From left to right, these shaded columns display parameter settings for,  , B  vel  A  ,and P pos  pos '  Table 13 compares the Haptic Camera and expert participant parameterizations, along with their 95% confidence intervals for the overall torque data (r in Equation 6 on page 29). This table presents a subset of the same data as in Figure 41. As with previous tables, units are in mNm, rad, and s for torque, angle, and time, respectively.  99  M  acc  95% CI , Haptic Camera  B^  Expert  Bi  95% CI .Haptic  Camera  4'  Expert  A  pos  .033  .024  .02  .022  .024  .015  .022  8-8  •-2A  24  5.1  12  5.1  14  2.6  1.9  4.5  2.8  6.6  .01  .07  .011  • ,;130 1.3  pQs  Expert P r  P  pos  .  •.-^y'*0;  95% CI •\ Haptic Camera  non-sine detents ' .002  .035  ve  ve  moderate detents .049  .28  Haptic Camera M > acc Expert  subtle detents  high inertia  Knob  high friction  Table 13: Comparison of Haptic Camera and Human Expert Dynamic Property Estimates (units are in mNm, rad, and s for torque, angle, and time, respectively)  .41  0  pos  95% CI  •'.  6.7  8.3  .91  1.5  -22 ' - :^:/:48; '.'  .41  .20  .43  .50  .00024  .057  .040  ;  :  For the special case of detent frequency, independently obtained "gold standard" values can be easily calculated for the period by counting the number of "clicks" while manually turning the knobs with detents about one complete revolution (refer to Figure 36 on page 73). The number of "clicks" was also validated using visual inspection for K n o b detents-  m o d e r a t e  detents  Visual confirmation was not performed for K n o b  subt  i  e  a n d  Knob  detents  non  .  sine  because  the confirmation could not be performed without permanently disassembling the mechanical knob sub-components. Table 14 lists the perfect "gold stan-  100  dard" values for these knobs beside the values obtained by the Haptic Camera and expert participants. "Gold standard" values for inertia were difficult to obtain because of the need to physically model a complicated inertia for the test knobs. Friction "gold standards" are even more difficult to obtain because surface material and geometrical properties between all moving parts need to be obtained. Calculating stick-slip frictional effects also impedes estimation of "gold standard" detent amplitude values. One would need to first calculate the geometries and material properties of the detents, then estimate the reaction torques generated as a user rotates through the detent. These other alternative estimation methods are tedious and error-prone, to be used as "gold standards".  Table 14: Comparison of detent estimates for knobs with detents Knob  subtle detents  moderate detents  non-sine detents  Haptic Camera Measured (detents/ rev)  29.1  13.2  15.2  Expert Measured (detents / rev)  25.1  15.4  12.9  30  12  12  Independently Obtained (detents/rev)'  Table 15 lists the terms which each of the 15 novices recorded on their slider sticky notes. Data from experts is not described because slider settings were explained to the expert participants; so results of their sticky notes would be biased.  101  Table 15: Participant tags for dynamic knob properties  #  M  vel  B  acc  A  P  pos  pos  1  weight, "whoosh"  friction, heavy  bigness of detents  how many detents  2  spinny  stiffer, like moving through mud  really bumpy  small bumps, big bumps for fine tuning  3  spin faster  spin slower & stops  feeling bumps  spacing bumps  4  rotational force control  friction force control for a control cycle of rotation  smoothness of rotation control  5  weight  friction  bump height  # of bumps  6  increase resistance; no brake  increase resistance; + break  increase stage effect  decrease stage width  7  smooth, but heavy  light & smooth; buttery  turning a smooth knob in definite steps  turning a knob with shorter steps in between  8  spin automatically  less resistant  clicks  faster  9  more friction  more friction; feels better than  more cranky  cranky  1 [Mace]  10  momentum  pudding  bump size  bump frequency  11  resistant spin  smooth spin  wobbly  knobbly  12  slingy  hard to turn  big clicky  clicky  13  resistance  sensitivity  smoothness  ditto (smoothness)  14  . easy; little bumpy  touch, sticky, but smooth  stiff, large bumps  less stiff, smaller bumps  velocity control  amplitude of detents  frequency of detents  15  inertia  102  3.4.3 Discussion  •  Our discussion starts by comparing the quantitative haptic matching results between the Haptic Camera, experts, and novices. Next, qualitative data analyses are performed to provide a deeper understanding of the quantitative haptic matching results. The ability of novice participants to identify dynamic knob properties is addressed by examining these participants' sticky note memory aids. And, more complex perceptual attributes of knob dynamics are distilled from the experimenter's field notes collected from the expert participants' "think aloud" comments.  3.4.3.1 Human vs. Machine Performance Many interesting relationships emerged between the parameterizations by human participants and the Haptic Camera. Our quantitative analysis focusses on relationships between the Haptic Camera and these experts because parameterizations by experts were more closely aligned with the Haptic Camera estimates and only experts performed a "think aloud" protocol. Precise relationships between Haptic Camera and participant estimates were performed for detent frequency because independent "gold standards" could be calculated. Because the Haptic Camera's characterization employed the same algorithm for fitting detent period as for the other physical parameters, it is likely that the Haptic Camera reliably fitted the other dynamic properties to their respective models as well. However, we can only conjecture that these underlying mathematical models appropriately represent human-centered psychophysical responses to these knob dynamic properties. Both Haptic Camera and participant estimates must therefore be collectively considered if we are to understand how well the "feel" of the mechanical test knobs were captured and rendered. The following paragraphs use the user study results to compare these machine- and human-derived parameterizations. Detent & damping confusion:  Looking at the Knob j-,ti su  e  detents  values  for B i and P„ in Figure 41, one can see relatively large differences between ve  os  103  the human and Haptic Camera estimates. For this low-amplitude detent (see the Ap values for K n o b 0S  subt  i detents e  m  Figure 41), participants most likely had  difficulty discerning whether the knob feeling was a frictional effect or a series of low frequency detents (the true knob mechanics). Referring to the Knob b_ su  tie detents column in Table 14,  one can see that the Haptic Camera was able to (i)  observe the detents as detents — not friction, and (ii) correctly estimate the detent frequency within a 3% relative error as compared with the average expert participant relative error of 18%. These errors could be calculated because independent "gold standard" values were available for detents. Robustness to non-modeled detent properties: Examining frequency  values for K n o b  .  non  sine  detents  m  Table 14, one can see that human experts were  better able to segment out (mentally set aside) backlash and non-linearities of m  K non-sine detents nob  a  n  was the Haptic Camera. In an effort to test a worst-  case scenario, the latter used a model known a priori to poorly match K n o b . non  sine detents (refer  to Equation 6 on page 29). Meanwhile, the relative low fre-  quency and high magnitude of the detents (see the A^ values for Knob _ j QS  detents  m  confused B  vel  m  s  ne  Figure 41) likely aided the expert parameterization.  Inertia  inertia  non  & damping confusion:  and M  acc  The novice and expert participants often  parameters (refer to Knob igh friction n  a  n  d  Kn°bhigh  Figure 41). This confusion could possibly be explained by the fact that  both properties have an initial resistance component as one begins to turn a knob. Conversely, the Haptic Camera algorithm treats position-, velocity-, and acceleration-dependent parameters as equally difficult mathematical parameters to solve. Participants may also have been confused by the ability to change mass with a slider. Prior research does suggest that people can become good at perceiving and dealing with changes in mass. Although dynamically changing mass is a foreign concept for most physical controls, people do experience change of mass in other everyday experiences. Learnt behaviors from these  104  experiences may transfer well to active physical control use. For example, Turvey has studied one's ability to balance a half-full glass of water in one's hand, and explored center-of-mass versus perceived length discontinuities for people holding different baseball bats [97]. Further research based on Swindells et al.'s [94] user studies using an experimental apparatus that could dynamically change center of mass could also be insightful. Actual versus perceived satisfaction:  The self-reported satisfaction rat-  ings of Figure 40 illustrate interesting relationships between actual versus perceived performance. For example, participants were significantly more satisfied with their parameterizations of Knobhigh friction compared to K n o b ate detents bad  m o d e r a t e d e t e n t s  and K n o b  non  .  sine d e t e n t s  a n  d Knobhigh inertia  . Because K n o b  m o d e r  .  subtle detents that were often confused with frictional texture,  mean satisfaction ratings that fall between those of knobs without detents (Knobhigh friction and Knobhigh inertia) and knobs with detents ( K n o b detents  a n  d Knob _ i non  s  ne  de  tents)  a r e  m o d e r a t e  consistent with other observations. The  lower satisfaction ratings for K n o b  m o d e r a t e d e t e n t s  and K n o b  non  .  sine d e t e n t s  could  be due to increased cognitive load dealing with detents in addition to inertia and damping parameters. Additionally, the feel of K n o b  non  _  sme  d e t e  nts  w a s  impossible to match using the sliders because the underlying rendered physical model was known to differ from the actual physical model. Even though participants were able to deal with these model differences very well (e.g., see Knob  n o n  .  s i n e  de  tents  frequency estimates in Table 14), participants may have felt  more cognitive strain in the process. This result supports the need for appropriate affective design because it is an example where participants were dissatisfied even though they performed relatively well. Absolute versus relative estimation: Comparing  expert participants  and Haptic Camera values for each of the knobs and parameters in Figure 41, one can clearly see agreement between the relative Haptic Camera/human relationships for individual parameters even when the absolute values found by the Haptic Camera and humans do not agree. That is, for a given parameter  105  such as M  acc  or B i, ve  the ratio of [experts' value for Knob ] / [experts' value for n  Knob ] was similar to [Haptic Camera value for Knob ] / [Haptic Camera m  n  value for Knob ]. For example, looking at the damping scores, expert particim  pants as a group did a good job estimating the relative damping levels between the different test knobs. Looking closely at the K  n o b  h i g h friction  a n  d Knobhigh inertia'  o  n  e c a n s e e m  a  t  Bi ve  parameters for  the Haptic Camera values  for both knobs are similar ratios to the corresponding values provided by the expert participants. This dominance of relative processing for human participants over absolute processing for automated capture is consistent with visual psychology research, such as Snowden [83], and is generally consistent with current psychophy'sics theory such as Stevens' assertion that participants make judgements on a ratio scale [85].  3.4.3.2 Sticky Notes from Novice Participants Analysis of novice sticky notes provides insight into the novice participants' ability to determine and understand fundamental detent, friction, and inertia renderings.  /  The labels summarized in Table 15 on page 102 provide a strong indication that most participants were able to quickly and correctly identify the four sliders into appropriate categories — inertia, damping friction, detent amplitude, and detent frequency, respectively. For example, participant 10 used the terms "momentum" and "pudding", and participant 1 used the terms "weight 'whoosh'" and "friction, heavy" as labels for "inertia" and "damping". Although "whoosh" and "pudding" are not technical terms for inertia and damping, they are excellent non-technical, vernacular descriptions. Similar terminology could greatly enhance accessibility and understanding to nontechnical users of ubiquitous computing devices containing active dynamics. Although less universal and specific, participant l l ' s terms "wobbly" and "knobbly" terms for amplitude and frequency of detents, respectively,  106  indicate that this individual clearly understood the concept of detents. Only the labels from participant 9 induce serious concern that the participant did not adequately understand the effects of each slider. Participant 9 used the same label "friction" for both the friction parameter ( ) parameter (Af  and the inertia  ). Participant 9 also used the same vague term "cranky" for  both the detent amplitude (A  ) and period (P  ). Participant 13's labels  also seem questionable since detent amplitude and frequency are both labeled "smoothness". Nevertheless, two or fewer participants out of fifteen having experienced confusion during the initial training phase of the user study is promising. More important is the suggestion that the previously mentioned confusion between inertia and damping (e.g., see Figure 41) is likely due to the complexity of the particular task, rather than the participants' lack of intuitive understanding of fundamental properties of physics. 3.4.3.3 D i s c u s s i o n o f F i e l d N o t e s f r o m E x p e r t P a r t i c i p a n t s  Discussion of the field notes collected from sessions with the expert participants are organized according to several broad themes (field notes were not collected from novices). Strategies:  All the experts used a variety of grasping techniques on the  haptic and test knobs to explore various dynamic properties. Initial coarse categorizations were typically performed with a whole-hand grasp of a knob, then finer single finger motions (usually with the index or middle finger) were used for more.sensitive, refined judgements. When comparing damping and inertia, experts typically rotated the knob slowly at first to feel some velocitybased feedback, then they progressively made faster, more "jerky" motions to explore inertia. Another common technique for inertia estimation was spinning the knob as fast as possible, then timing how long the knob slid past one or more fingers lightly touching the edge of the knob. Experts typically first categorized a test knob as "with detents" or "without detents". Next, experts tended to refine the rendered knob's detents  107  (if present), then friction, and inertia were explored. In other words, the experimenter observed experts using an exploration strategy of position-, then velocity-, then acceleration-based parameters. Experts would then iterate towards their final solution by tweaking whatever parameters seemed least correct. Experts iteratively refined their slider settings with frequent exploration back and forth between the test knob, rendered knob, and the physical sliders. Experts also attempted .to use visual cues from the spinning knob, but this strategy was (intentionally) quite difficult because all the knobs were fitted with uniform, white plastic caps. Parameter  interactions:  When increasing the inertia, two experts stated  that this made detents feel less noticeable. One expert elaborated by saying the physical interaction between inertia and detent amplitude "felt right". In other words, based on physics, one would expect detents to be less noticeable on knobs with higher amounts of inertia. These statements suggest that the interactions between different position-, velocity-, and acceleration-based effects occurred as expected based on fundamental laws of physics, but these physical properties were occasionally difficult for even experts to mentally segment. One expert was frustrated because damping and inertia affected each other — even though this is the kind of interaction he should have felt. These comments by experts suggest that segmentation of properties away from realistic physics could improve tool usability for designers of rendered or mechanical knobs for "real world" applications. For example, designers might more easily create a physically realizable model if they could manipulate a single parameter re-mapped to a combination of system model parameters. In other words, such an approach would not separate the user and system models when appropriate. Physically non-realizable models may also be interesting in their own right. For example, a momentum-like parameter that does not interact with detents or friction could theoretically be rendered on a haptic knob even  108  though such knob dynamics would be difficult, if not impossible, to create on a "real" mechanical knob. Confidence: A n area that did not improve participant confidence in  their characterizations was the relation of the slider movement to the resulting effect on knob feel. An expert felt that the sliders did not seem to act in a linear manner, but the slider action was in fact linear. This statement suggests a conflict where a linear relationship in an engineering space may not be linear in a perceptual space. Non-linear slider mappings may therefore be more intuitive  for parameter estimation, as suggested by Stevens' power law [85]. Also, two experts were unsatisfied with the "jittery" feeling on the rendered haptic knob when all sliders were set to their maxima. High inertia, high amplitude detents are technically challenging to render [67], and appropriately dealing with practical control issues is a significant barrier to haptic control adoption. There were, however, unexpected confidence boosters too. One expert did not initially recognize the detents on Knob t, i su  t  e  detents'  but this expert was  quickly able to identify subtle detents by rotating the knob at different velocities. The expert then adjusted the detent amplitude, A^ , and damping, B i, os  ve  parameters to create an appropriate rendering of K n o b b i su  t  e  detents  o  n  the hap-  tic knob. If this expert was not confident in perceptually relating the appropriate physics-based properties using the damping and detent sliders, he would not have been able to make an appropriate rendering. Another expert also made the comforting comment that the angular differences between the haptic knob mounting and the test knob, mountings did not interfere with parameter estimation. This result suggests a reasonable robustness of the data to the physical layout of the knobs. Experts typically spent between two and six minutes adjusting the four sliders to match a single test knob. Experts would often switch between the rendered knob and test knob over a dozen times for each trial. This large amount of time and iteration per trial suggests the task was moderately difficult, and suggests that even the experts required significant effort to distin-  109  guish dynamic parameters, despite their eventual proficiency in accomplishing the task, Validity:  In addition to the previously described difficulties with stiff-  ness and closed-loop feedback control, a primary validity-related problem was eradicating the sounds of clicks from the mechanical test and rendered knobs when experts were exploring detents (multimodal effects). Because the novice participants were not asked to think-aloud during their experiment trials, they were able to wear noise canceling headphones that reduced or eliminated this problem. One expert experienced difficulty getting the amplitude setting of Knob  subt  i  e  detents  high enough to be felt, but not too high. Conversely, two  experts mentioned that the haptic knob did not feel stiff enough. These are common difficulties with almost all force-feedback technologies. These dynamic range issues are gradually being addressed within the haptics community through a combination of better mechatronics and better control algorithms. For example, greater stiffness could be obtained using haptic controllers with built-in braking mechanisms [40], or carefully timed bursts of force [50]. Finally, in terms of assessing how "real" the rendered knobs felt — as apposed to "simulated" — to these experts accustomed to haptic rendering, perhaps the most promising comments came from two experts who asked if the test knobs were a combination of mechanical and rendered knobs. Specifically, the feel of test K n o b  subt  i  e  detents  w a s  described as "complex, sophisti-  cated... like a haptic knob." Interestingly, two curious novice participants asked similar questions when informally chatting with the experimenter after completing their studies. This confusion between mechanical and rendered knobs is a strong indication that the quality, and therefore the validity, of rendered dynamic properties was reasonably good for at least some of the renderings. The expert's comment also suggest that haptic controls could potentially provide a richer, more "full-bodied" dynamic feel than what is possible with most mechanical controls.  110  3.4.4 Summary of Human vs. Machine Performance User studies comparing haptic knob renderings for five mechanical test knobs were conducted by asking novice and expert participants to adjust four parameters of a rendered knob to match the feel of a test knob. Similar relative detent, friction, and inertia parameterizations were observed by human expert and Haptic Camera estimation methods. Independent "gold standard" checks of detent frequencies for K n o b sine detents  subtle  det  ents/  K n o b  m o d e r a t e detents/  and  K  n  ° nonb  with the Haptic Camera averaged 3.0%, 9.5%, and 23.5% relative  accuracies, respectively, whereas human experts averaged 17.7%, 24.8%, and 7.2%. These data, combined with data for damping estimation, suggest that human ability to make accurate and confident parameterizations were more robust to irregularities such as unmodelled non-linearities and backlash compared to an automated test procedure. Human participants probably mentally parameterized the knob dynamics to a more general model compared to the model used by the Haptic Camera (see Equation 6 on page 29). Conversely, the Haptic Camera significantly outperformed human experts (and human novices) when an appropriate physical model was used. For most knobs, such models are relatively easy to choose, and can be tested for accuracy using techniques such as confidence interval calculations on final curve fitting results (e.g., see Table 7 on page 51). These studies help demonstrate that the Haptic Camera apparatus can effectively capture knob dynamics as perceived by a human.  3.5 Future Rendering Work  '  Future work needs to be explored in several key areas including technical enhancements and additional user studies.  3.5.1 Technical Enhancements Both software and hardware technical components could be enhanced. Improved acceleration sensing and better inertia rendering could be achieved  111  by developing a custom wireless rotational accelerometer and embedding it into the knob cap. Electromechanical noise from the nearby motor and amplifier could introduce significant communications challenges. More intuitive manipulation of conceptual physical properties could be achieved by designing algorithms that better enable users to segment and recombine physics-based sub-components such as inertia, friction, and detents. For example, even experts had difficulty teasing apart friction and inertia effects. Some more complicated mathematical arrangement of physical properties may lead towards more natural, independent-feeling rendered subcomponents from the designer's and user's perspectives. Virtual mass oscillations could be reduced, and detent amplitudes could be more faithfully rendered, by developing and testing haptic technologies that are stiffer and more stable. These could include physical controls with braking mechanisms, and practical improvements to real-time closed loop control.  3.5.2 User Study Enhancements Several enhancements to the study apparatus and procedure could yield additional interesting results. Differences between experts and novices could be compared with a larger 2x2 experimental design where half the experts got the novice treatment and half the novices got the expert treatment. Because several participants questioned whether the five passive knobs were indeed all passive, the identity of the active knob could be conceivably hidden from the participants in matching studies where the rendered knob parameters did not change during a trial (dynamics of passive knobs can not possibly change, so attempting to mask the identity of active knobs containing variable dynamic properties would be futile). More direct comparisons between the Haptic Camera parameterizations and participant parameterizations using pairs of haptic knobs would be interesting too. When Haptic Camera and participant parameterizations fully agree, the Haptic Camera can be said to have made a best-fit parameterization  112  of the mathematical and human-centered properties. Otherwise, only the mathematical properties have been adequately parameterized — assuming the Haptic Camera functioned properly and the underlying model is perceptually complete. To this end, we summarize a possible study design for comparing these Haptic Camera and human-centered properties. The goal of such a study would be see if participants believed that either their user-designed knob parameters or the automated Haptic Camera parameters better matched a "real" test knob. Participants would first perform a haptic matching study similar to that described in this chapter. Next, participants would perform a second phase of trials using a pair of active knobs. One active knob would render dynamic parameters from the Haptic Camera, and the other active knob would render the participant's choices from the first phase. Participants would successively be asked to choose which active knob feels most like one of the five passive knobs. Selection of the active knob perceived to be most appropriate could be performed by pressing one of two physical buttons, or graphical buttons On a touch screen (similar to the touch screen used in Study 3 of Chapter 4 starting on page 153). A more detailed exploration of individual dynamic properties, such as inertia or detent amplitude, could be performed using the same procedure. Instead of rendering complete dynamic models on the active knobs, only certain dynamic properties would be rendered. \An additional user study variant could iteratively present a participant with two rendered knob behaviors, and ask the participant to Choose the one that is closest to a target knob. Next, the previously chosen rendering and a new rendering would be presented to the participant. Instead of requesting the participant or experimenter to choose successive renderings to feel, a computer program would iterate towards increasingly similar tuples of rendered knobs until a predetermined similarity threshold was met. Such a process would reduce the cognitive load needed by the participants during experimental trials compared to the matching experiments used in this thesis. However, this process would be less suitable as a structured task for qualitative  113  data gathering. Complex relationships between inertia, friction, and detent behaviors may also lead to conflicting final parameter sets chosen by participants. For example, during the validation studies performed for this thesis, participants confused some subtle detent.and subtle friction renderings. Such confusion may be more difficult to deduce in this proposed new study design because its participants would not be given the opportunity to build cognitive models of the rendered knobs. In contrast to the previous two alternative study designs, one could design studies that increase focus on qualitative data collection. Participants' exploration strategies and difficulty parameterizing knob dynamics could be recorded using additional participant monitoring, such as video capture and coding. The same procedure as the matching studies described in this chapter would still be appropriate. For example, one could record how often the participants switched between the active and passive knobs during a comparison trial. Calculating the total time for each trial would also give an indication of difficulty for the different knob types. Individual dynamic properties could also be calculated and analyzed by observing relative time differences spent adjusting each physical slider. The slider increments used for our studies may not have been optimal. For example, certain linear or non-linear mappings of model parameters may be more intuitive for users. Future user studies could leverage procedural design parallels from studies that have already been performed for graphics. For example, user-selected gamma correction for the pixels on a monitor have been found to work best with a linear mapping of a non-linear intensity function [107]. Such non-linear mapping studies for physical controls should also leverage existing psychophysics work. Future studies could also leverage biomechanics work such as Houk et al. [37]'s exploration of fractional power damping model parameters for rotary hand movement. The oscillations and rough textures due to minor instability for the high magnitude inertia renderings could be reduced if the renderings were based on appropriate psychophysical study results. Such psychophysical user stud-  114  ies should compare natural frequency and mechanical vibration effects on both task performance and affective responses. These studies could help identify rendering regions where participants are less or more perceptually sensitive to particular rendering instabilities. Common everyday tasks could be used as bases for further studies mapping one's ability to discern rendered dynamics of physical controls. For example, when filling a glass of water, a person must deal with dynamically changing mass. Even though rendered dynamics are a relatively new technology, most users will have already explored similar dynamic effects during their typical daily routines. Results from such studies should suggest appropriate parallels between existing psychophysical experience and the feel of emerging physical control technologies.  115  Chapter 4 M e a s u r i n g Affect In this chapter, we focus on the methodology (designs and analyses) which allows us to evaluate the affective result of a given haptic interface. Chapters 1 - 3 describe how to capture and render dynamics of a physical control, and Chapter 5 describes how to modify and tune these physical control dynamics. However, a physical control that functions technically may be a poor overall design if it does not emotionally engage its target user(s). Thus, our approach is to elucidate appropriateness by combining perspectives of affect and performance.  4.1 Introduction Designers of human-computer interfaces often overlook issues of affect. An example illustrating the importance of affective design is the frustration many people feel when working with a poorly designed computing device. Productivity and safety would improve if such computing interfaces were redesigned to induce more pleasant user emotional responses. Despite its apparent importance, there is a dearth of mechanisms for actually measuring and utilizing affect in the context of designing interfaces. To address this, we built a testbed to explore the relationships between affect and performance. For the purposes of this thesis, we were most interested in affective response to variations in haptic aspects of the interface. First, we conducted a foundational study on real surface textures with two explicit goals: (i) to judge the usefulness of two types of affect measurement tools for the subtle dynamics typically felt on physical controls, and (ii) to establish a set of "baseline" affective responses to a relatively broad set of naturally occurring haptic stimuli, for later comparison to the potentially more subtle variations in response to rendered environments. Next, we describe a pair of studies based  116  on a rotary manual control (a force-feedback knob). With them, we prototyped an evaluation methodology which is applicable to other single degree-of-freedom physical controls such as sliders, switches, and buttons. Study 1 compares self-report and biometric responses to surface textures as a test of the affect measurement apparatus. These textures were chosen to span a wide variety of affective responses. For example, we conjectured that stroking a soft piece of silk with one's hand would feel more pleasant than stroking coarse sandpaper. Studies 2 and 3, on the other hand, compare affective responses to rendered controls being used to carry our specific classes of tasks. Unlike the texture study, this allows us to consider the impact of task on affective responses. Experimental procedures using physical controls were performed to measure response time and affective response as users manipulate knobs in a list selection task (Study 2) and a Fitts-like [31] rotational task (Study 3) while being forced to use a variety of kinematic trajectories. Studies 2 and 3 leverage the grounding knowledge gained by measuring the affective responses to real materials in Study 1 to better understand subtle relationships of rendered dynamics. The tasks represented by Studies 2 and 3 capture the physical interactions inherent in many 'real-world' applications involving physical controls. Relationships were found between affective response, task performance, and the parameters of the haptic rendering for the active control. Participants generally preferred physical control renderings that improved task performance, although counter-examples were also found. For example, two radio tuning knobs with different levels of friction and inertia may enable a user to tune the same radio station with equal proficiency. However, one knob may produce a much more favorable affective response. This chapter results in a set of insights for how the dynamics of physical controls can be used to influence affect both in its own right, and in relationship to performance tasks. Affective responses to knob renderings were  117  also observed to agree with previous research involving their physical counterparts. The first and second studies have already been published in Swindells et al. [90]. The third study has been published in Swindells et al. [91]. Here, we present together these two studies and a subsequent third study, following motivation and background.  4.1.1 Motivation Examining the red-green-blue (RGB) tri-chromatic color space can help motivate the need for the following haptic user studies (e.g., see [81,107]). Most color televisions and computer monitors blend combinations of red, green, and blue to produce a seemingly full spectrum of colors for a human observer. For example, colors such as yellow and magenta can be effectively conveyed by blending combinations of primary RGB colors. Color displays would be prohibitively complex and expensive if they needed to instead display individual colors using an array of individual wavelengths for individual colors. Thus, user studies exploring the appropriate color models, such as trichromatic color spaces, for human observers were crucial initial research steps for successful, widespread adoption of color displays within our work and leisure environments. Contemporary haptics research is at a stage of appropriate model building that parallels early vision work in RGB color spaces. We need to figure out the most perceptually important haptic attributes to develop more useful active haptic renderings. What are haptic efficiency equivalents of graphic RGB color spaces? The three presented studies contribute to the goal of finding appropriate feeling, easy to develop haptic models and renderings that will lead to practical, widely adopted mechanisms for designing haptic controls.  118  4.1.2 The Impact of Affective Response to Haptic Stimuli Affective computing refers to computing devices that relate to, arise from, or deliberately influence one's emotions [76], Furthermore, our affective responses always accompany thought [106]. For example, we rarely see a "house"; instead, we see a handsome house, an ugly house, a pretentious house [110]. In terms of computer systems, we see a cool, sleek new computer, hear an upbeat cell phone ring tone, or feel a comfortable stylus. Such affective judgments are believed to be independent of, and temporarily precede most higher-level perceptual and cognitive operations [5]. In other words, affective responses are a 'first level' response to our environment. These 'gut' affective responses then influence higher-level emotional judgements, which are more cognitive. Consequently, higher-level operations vary between individuals depending on personal background, age, gender, affiliated culture, etc. This chapter focusses on affect as a potentially potent design dimension for manual controls, because of the intimacy enforced by the need for sustained physical contact and the overall simplicity of these interfaces which highlights what is there. Affective design aspects are already recognized as important in some contexts: as a product line is iteratively refined, its level of adoption by users and its commercial success becomes more dependent on non-technical attributes such as appropriately induced emotional responses [42, 70]. Well-known examples of this are the visceral impact of "heavy" but expensive-feeling stereo volume control knob, and the careful design of the sound and feel of a high-end car door closing or the trademarked throaty roar of a Harley Davidson motorcycle's engine revving. More currently, customized cell phone cover plates and ring tones, which offer few performance benefits, but typically induce strong^emotional responses from users and influence sales.  119  4.2 Related Affect Research This section describes previous research measuring the "feel" of the passive counterparts of the active physical controls used in Studies 2 and 3. Next, affective research is summarized, followed by self-report and biometric methods for measuring properties of affect.  4.2.1 Measuring the Feel of Haptic Knobs Inspired by the need to design dials for rotary phones that 'felt right', Knowles and Sheridan [49] performed early human factors work comparing several friction and inertia parameters for knobs using physical mass and cable pulley mechanisms to study performance (as was common in that era, e.g. [38, 69,105]) and subjective responses (which was not as common). For example, they found that participants had difficulty detecting less than 1520% changes in friction and inertia, that subjects preferred low friction levels, and that subjects preferred at least a small amount of inertia. Such human factors research is again relevant as embedded mechatronic interfaces, including force-feedback physical controls, become feasible and cost effective. Our three studies take inspiration from this visionary early work. We have replaced the purely mechanical setup of Knowles and Sheridan with an actively controlled, and thus more versatile, display which allows us to rapidly change a variety kinematic properties; and, we utilize and compare new measures of affect. Meanwhile, current state-of-the-art force-feedback controls can feel almost as good as traditional mechanical controls. Force-feedback technology is rapidly progressing, and force-feedback controls are much more flexible — both from design and usage perspectives. We are already seeing' active force-feedback physical controls that do not need to feel like a "real" mechanical control. How should these emerging force-feedback renderings feel? This is a critical period where we need to leverage extensive early human factors research and contemporary haptic research to guide new state-of-the-  v  120  art physical controls. Comparative studies, such as those described here, are aimed at this problem. There have been explicit attempts to design controls to display affect parameters. For example, MacLean [59] demonstrated an active door knob with dynamics and temperature that changed depending on the activity behind the door. Thus, a person could use the door knob handle to 'feel' various current and recent activity that took place behind the door, including their emotional content. Recently, there has been attention to rendering active force feedback for one-degree-of-freedom (1 DOF) displays. Novak et al. [71] studied the kinematic properties of rapid hand movements in a knob turning task, and fitted rotary hand trajectories to a non-linear mass-spring model of movement similar to the underlying model used within our haptic knob. Hasser and Cutkosky [35] modeled a human hand grasping a haptic knob by fitting to a linear, second-order translational model at the fingertip with single constants for rotational acceleration, velocity, and position. We chose haptic rendering models that are complementary to these human hand models, so that our apparatus would effectively render convincing acceleration-, velocity-, and position-dependent haptic feedback (refer to Section 3.3, "Rendering Model," on page 84).  4.2.2 Dimensions of Affect Affective and cognitive processes can occur in less than 10 ms. People are often unaware of the presence of such processes [96]. Zajonc [110] states that affective responses are believed to be inescapable, irrevocable, implicate the self, difficult to verbalize, and often separable from content. Many terms exist to classify emotion. Norman [70] uses the terms • Visceral: primary, automatic, unconscious responses (e.g., the graphic display is bright, the cell phone ring tone is loud, the stylus is smooth)  121  • Behavioral:  slightly less automatic unconscious responses (e.g., the  bright graphic display causes surprise, the loud cell phone ring tone is annoying, the smooth stylus is comforting). • Reflective:  responses involving conscious thought and reflection  (e.g., I like how clicking this button with the stylus causes the display brightness to increase). Generally, reflective responses are most influenced by social and cultural attributes, whereas visceral responses vary less from person to person. For example, a bright graphic display will be equally bright for an office worker or a tribal native who has never seen a graphic display before. Of course, there is no hard, exact boundary between these levels of emotion. Visceral responses will vary the least between different people or groups such as office workers, teenagers, or Lithuanians; whereas, reflective responses will vary the most. Spence [84] suggests that the sense of touch is well suited to perception of differences in emotion. Unlike the senses of sight or sound that only receive stimuli from the environment, people both receive and react to the environment when using the sense of touch. This tight coupling between a person's sensory and motor systems people creates a sense of intimacy that influences one's affective state. Haptic feedback can consequently play a significant role in influencing affective responses. This is an additional motivation for our choice of haptic examples. For over 100 years, psychology researchers have consistently reported almost all affect variability to be described by three dimensions [73,106]. Other researchers have since validated and refined these dimensions. For example, Lang's self-assessment mannequin (SAM) [51] uses the terms: • Valence (e.g., pleasantness) • Arousal (e.g., excitement) • Dominance (e.g., control, status, or prestige)  122  4.2.3 Performance Trade-offs and Design Implications Jordan [42] describes how affective responses become increasingly important as a type of product matures. For example, one expects a relatively high-level of writing performance from any pen — a mature product. Ergonomic and aesthetic properties become defining characteristics distinguishing widely accepted and widely shunned pens. Sometimes, a person may use an ergonomically or aesthetically favorable pen even it does not perform as well as a different pen. Norman [70] presents extreme examples of widely adopted products that are poorly designed from a performance perspective, but elicit superior emotional reactions. Thus, designers need to carefully consider performance and emotional trade-offs when designing products. Sometimes improving a product's emotion-inducing properties will conflict with performance. Other times, emotion and performance improvements will complement and support each other. Like any product, the knob behaviors described in this thesis require careful affective design decisions to ensure appropriate performance properties occur in the final product. The results of the following affect-based user studies are not intended to be hard rules to optimize affective responses. Instead, they are intended to act as tools in a designer's palette. For example, a highly arousing, negative valence haptic behavior may be a appropriate for a power plant physical control setting that is likely to result in a power failure. Conversely, the same haptic behavior may be inappropriate for an automobile physical control for adjusting the cabin ventilation settings. Thus, the guidelines generated from these user studies are intended to be used in a similar way as a designer would use color. For example, Kobayashi [48] describes how some colors elicit more arousing reactions than others, and how some colors elicit a sense of 'complexity', 'elegance', 'intelligence', 'charm', etc.  4.2.4 Practical Issues in Collecting Affect Measurements Self-report measures and biometric recordings are the primary methods of obtaining objective affective responses [25]. Generally, self-report mea-  123  sures are preferred for analyzing smaller, relative differences between stimuli, and biometric measurements are better for absolute measurements. We make this claim based on two theories. First, relative measurements are easier to obtain than absolute measurements because relative judgements require detection  whereas absolute judgements require measurement [68]. Second, when  making a relative rating, participants can filter out many effects that are of little relevance to the study [15]. Biometric measurement can not be focused like self-reports. For example, with careful, specific instructions, participants can be more easily guided to focus on details of a design. Previous researchers, Barriera-Viruet et al. [9], have studied trade-offs between self-report and direct measurement (e.g., biometric) procedures in detail. Participants can interpret instructions differently than desired by the experimenter. Such misinterpretations are the most influential sources of noise in self-reported measures. Although biometric measures are less influenced by such misinterpretations, they are more sensitive to the environment [13]. Biometric tests are difficult to use in uncontrolled environments such as field studies. For example, the biometric stimuli of interest are often overwhelmed by other environmental stimuli in the field such as personal conversations, traffic, and weather. Learnt and biological differences will also introduce noise and influence biometric measurement validity.  4.2.5 Measuring Affect with Self Reports Self-reports involve participants rating their agreement or disagreement to a statement according to a numeric scale. Self reports are often used to rate valence, arousal, and dominance dimensions. Thus, a participant will typically be exposed to a stimulus for 5-8 seconds, and then be asked to rate valence, arousal, and/or dominance on a scale (e.g., 1-9). Exposure times of 58 seconds have been estimated to give participants enough time to experience the stimulus, without giving them time for much conscious thought (a "gut" reaction is desired) [52]. Approximately half of one's affective judgment variability is along the valence dimension, and slightly less than half of the vari-  124  . ability is along the arousal dimension. The small, possibly negligible, remaining affect variability is. mostly along the dominance dimension. Hundreds of studies, predominantly vision-based psychology studies, have used these scales. Because the dimensions of valence and arousal are believed to account for almost all affective variability, Russell et al. [79] proposed and used these as the basis for a two-dimensional affect grid. Studies measuring more subtle affective states within the main dimensions of valence, arousal, and dominance have had some, but more limited, success. For example, Lang et al. [52] mapped particular affective attributes to defined subregions of a 2-D valence and arousal (affect) grid. Figure 42 illustrates placement of some particular affective attributes, including joyful, fearful, ennui, sad, and enraged, on an affect grid. Others, such as Killgore [47], have further refined our understanding and verified the validity of the affect grid, and Desmet [25] has independently broken down the "general" valence dimension into seven distinctly positive and seven distinctly negative emotional sub-dimensional subtleties [25]. Several caveats arise when using any rating scale. Daniel gives an excellent summary of standardizing rating scales for measuring scenic beauty of forest photos [24]. Valence, arousal, and dominance rating scales can be standardized in a similar manner. The two most common problems are: • Some participants.give overly positive or negative responses (the average responses by a participant are positively or negatively biased). • Participants may view the differences between scale values differently (relative differences will vary across participants). To help reduce this effect, experimenters often ask participants to use the full range of the scale for their responses.  125  ;ited  (JC  yful  \  —^  1  )  )  .1  1IUI  J  }  L—  CO  o  hate  A  L  4  sa  O3laxe o epres;sed^  <;  snnui  ) 6  Valence Figure 42: The affect grid. After exposure to a stimulus, participants place an "x" in a box to self-evaluate their level of valence and arousal (based on Russell et al. [79]).  4.2.6 Measuring Affect with Biometric Sensors Affective responses correlate with a variety of biological responses including changes in muscle tension, skin conduction, heart rate, blood pressure, and breathing rate. Analyses of facial responses have been used by researchers for over 100 years (e.g., Duchenne de Boulogne [26]). More recently, Ekman and Friesen developed the Facial Action Coding System (FACS) where six affective attributes — joy, sadness, disgust, anger, surprise, and fear — can be manually coded from still images or video [27]. However, direct measurement with sensors can be more accurate, and increasingly, is technically feasible. For example, functional magnetic resonance imaging  126  (fMRI) and electroencephalogram (EEG) sensors have been used to monitor brain activity variations in the prefrontal cortex for different affective responses [5,45]. Although these accurately record affective responses, fMRI machines are very expensive and their magnetic fields can interfere with many force-feedback interface technologies. Electromyographic (EMG) measurement of facial muscles is often more practical than full-head EEG or fMRI (for reasons of cost, ethics, and complexity). For example, Surakka and Hietanen studied EMG responses in facial expression research [88]. Picard gives a detailed, comprehensive discussion of biometric use [76]. Winton et al. [104] determined that skin conductance (SC) measured on muscles in a participant's index and middle fingers (digitus secundus and digitus medius) varied linearly with arousal ratings such as those used by Lang et al. [52]. This SC test is often referred to as a 'lie detector' when used by police because the sensors pick up increased sweat that occurs from elevated arousal levels typically accompanied by lying. Parallel work by Schwartz et al. [80] was performed with electromyography (EMG) electrodes applied to a participant's facial muscles (corrugator supercilii and zygomaticus major). Corrugator supercilii muscle tension was found to measure valence slightly more effectively than the zygomaticus major measurement. This EMG test is a simple way of examining a person's facial expressions such as frowning. For example, as a person smiles, certain electrical voltage levels fluctuate in facial muscles as they tense and relax. Based on the success of these techniques elsewhere, we respectively used EMG and SC tests to measure absolute valence and arousal levels for the studies in this thesis. Valence and arousal were measured using the same 9point rating scale developed by Lang [51] for the Self-Assessment Manikin. Others have successfully used similar biometric EMG and SC tests to monitor  127  valence and arousal, respectively (e.g., see Conati el al. [21] and Mandryk and Inkpen [60]). Figure 43 shows EMG and SC electrode placement.  Figure 43: EMG electrode placement on the forehead, and SC electrode placement on the index and middle fingers of the non-dominant (left) hand.  4.3 Overview of Haptic Affect Study Series Prior affect research helped guide the user study procedures. We conjectured that self-reports would be more suitable than biometrics for the user studies in this thesis because (i) self-reports should be better able to focus on the most dominant emotional attributes for each user study task [15], and (ii) affect perception and biometric measurement is not well understood [54]. Biometric tools are potentially more powerful than self-reports because they are an absolute measurement instrument (not a relative detection instrument) [68]. However, difficulties using biometric instruments for our studies were not unexpected because affect differences between individual surface textures and knob dynamics were conjectured to be relatively small compared to other life experiences. The user studies focus on the two dimensions of valence and arousal because they are the first and second, respectively, most influential dimensions of affect [79]. We performed three studies as a starting point for future systematic evaluation and design of rendered physical controls:  128  1.  Study 1: Foundation Tests of Self Reports and Biometrics Objective:  We estimated and compared the effectiveness of self-  reports and biometrics for measuring valence and arousal reactions. The study utilized a set of haptic stimuli conjectured to span a wide affect domain typically encountered in one's daily work and play environments. These results formed a foundation and reference frame for further study of more subtle and novel haptic effects, such as those elicited by force-feedback knobs, sliders, and buttons. Approach:  Accepted self-report and biometric procedures from  visual psychology research were applied to haptic stimuli hypothesized to span a moderately wide range of affective responses. Specifically, participants responded to touching 12 different tactile textures, such as silk, putty, and acrylic, using rating scales, electromyography (EMG), and skin conductance (SC) measures. 2.  Study 2 : Rendered Interaction using List Selection  Objective: We estimated and compared valence and arousal responses elicited from renderings on a force-feedback physical control (a haptic knob). The experimental procedure leveraged insights gained during Study 1 with respect to the efficacy of the self-report and biometric measures, and in terms of an intuitive reference frame (the tactile surfaces of Study 1 are better understood than the more novel haptic renderings of Studies 2 and 3). Results were studied both in the context of a typical performance task, and in a context-independent way. Approach:  Self-report and biometric measures from Study 1 were  applied to use rendered physical controls in a task context. Specifi-  129  cally, participants rotated 7 different force-feedback knob renderings to select a target number in a graphical list. 3.  Study 3: Rendered Interaction Objective:  using Rotary  Pointing  Building on the Study 2 results, we further explored the  effects of task context on valence and arousal responses to renderings on a force-feedback control (a haptic knob). Like Study 2, we collected affective responses within the context of a target acquisition task that closely matched typical "real world" knob usage. However, unlike Study 2, a spectrum of coarse-to-precise task levels were studied, so as to further exercise the concept of "control appropriateness" and its impact on affective response. Approach:  Self-report and biometric measures from Study 1 were  again applied to rendered physical controls. Specifically, participants rotated the same 7 knob renderings as in Study 2, but performed a rotary pointing task consisting of 2 target widths and 2 target amplitudes that represented a spectrum of motor control precision.  130  4.4 Study 1: Foundation Tests of Self Reports and Biometrics Study 1 is a foundation for Studies 2 and 3 comparing the effectiveness of self-report and biometric responses to twelve tactile surfaces. The affective results of Study 1 serve as an intuitive reference to which we could relate the more subtle knob dynamics used in subsequent studies. The tactile results are also interesting in their own right. The primary research questions for Study 1 are: . How effective are self-report, EMG and SC estimates of valence and arousal for measuring different tactile surface textures? And, how might these affect results from tactile surfaces guide user study design of more subtle haptic attributes such as inertia,friction,and detent differences between physical controls?  4.4.1 Method The participants, apparatus, and procedure for Study 1 are described below.  4.4.1.1 Participants A total of nine paid students (5 male and 4 female) were recruited from a range of disciplines at the University of British Columbia. Participants were right-handed and ranged in age from 24 to 33 years (M = 26.2, SD = 2.82). Each participant took approximately 20 minutes to complete the study.  131  4.4.1.2 Apparatus Participants sat at an empty desk in a quiet room with dimmed illumination. Twelve physical tactile textures were chosen to span a broad range of tactile feedback (see Table 16).  Table 16: Tactile Stimuli for Study 1  #  Label  Description  1  FUR  Fox fur  2  GEL  Moist water-based gel  •3  PTY  Silly Putty™ surface  4  SND  80 grit sandpaper  5  ACR  Acrylic sheet  6  GLS  Glass sheet  7  BSH  Brush with fine plastic tines  8  WD  Maple wooden board  •9  OIL  Glass sheet covered in olive oil  10  STK  Double-sided sticky tape on an acrylic sheet  11 12  . HND SLK  Hand touched by experimenter Silk  Self-report (valence and arousal), EMG (valence), and SC (arousal) measures were obtained for this study. Self-reports of valence and arousal  were measured at the end of each  trial. Participants used a pen to mark an "x" in an empty affect grid cell. Each affect grid filled the majority of a single-sided 81/2" x 11" paper printout (see Figure 42). EMG was measured at a 32 Hz update rate by placing two AgCl ProComp+ triodes: one centered on the participant's forehead, and one directly above the right eye. The sensors were oriented perpendicular to each other to measure activity of the corrugator supercilii and depressor supercilii muscles,  132  respectively. Raw EMG voltage traces were low-pass filtered using a 3rd order Butterworth filter with a pass band ripple of 10 dB and stop band attenuation of40dB. SC was measured at a 32 Hz update rate by placing two AgCLProComp+ electrodes on the index and middle fingers (digitus secundus and digitus medius) of the participant's left hand. The SC technology combines finger muscle voltage measurements into a single low frequency waveform, so no data filtering was needed.  4.4.1.3 Procedure The experimenter familiarized each participant with the apparatus — excluding the stimuli. Two EMG electrodes were then placed on the participant's forehead as shown in Figure 43, and skin conductance sensors were placed on the index and middle fingers of the participant's left hand. For a trial, the participant stroked a randomly selected texture with their right hand for approximately 3-5 seconds (longer times would have facilitated participants to make reflective thoughts that were not the target of these experiments). After each trial, participants were instructed to mark arousal and valence on two scales of 1 to 9 by marking a single "x" on a paper-based affect grid. Participants were asked to try to use the full range of the scales. After marking the affect grid, the participant placed the marked affect grid face-down onto a stack of completed sheets. The experimental design used a within-subject factor (tactile stimulus) with 12 levels and one repetition. One complete repetition of all twelve levels of tactile texture were executed to familiarize the participants with the experiment. Participants were blindfolded while the tactile textures were in front of them, and they were not blindfolded when marking self-reports. The blindfold was a loosely fixed scarf that did not interfere with the biometric sensors, and enabled the participants to easily raise above their eyes when marking the affect grid. Single EMG (valence) and SC (arousal) scores for each target acquisition were determined by a neurophysiology expert using an assessment pro-  133  cedure. To determine a valence score for a trial, the expert observed muscle activity collected from each participant's forehead. For each trial, a single biometric valence curve was calculated by subtracting the depressor supercilii EMG voltage trace from the corrugator supercilii EMG voltage trace. The expert then manually identified the trial's peak voltage on this smoothed difference trace and subtracted it from the baseline (the flat region preceding a user's knob manipulation) to determine a signed valence value. Mean voltages were calculated between these start and end boundaries as shown in Figure 44. A positive peak-minus-baseline value indicated positive valence, and a negative peak-minus-baseline value indicated a negative valence. To determine an arousal score, the expert manually identified the peak SC voltage and subtracted it from the baseline to determine an unsigned arousal value where higher values represent higher arousal. SC values should be positive because participants are assumed to start a trial from a nonaroused, unstimulated state. Accurately and repeatedly determining a baseline voltage is a difficult task that makes biometric measurement inherently uncertain.  sensor voltage  •  .j—peak+  affect value ^—base-* T  stimulus start  t stimulus end time (  Figure 44: Method for estimating a single biometric affect value for one stimulus for either a single EMG or SC voltage curve.  134  4.4.2 Results The data were checked for fit to a normal distribution using a Q-Q plot. Based on this, normality was assumed. Figure 45 shows means for the arousal and valence ratings (from the affect grid) for the 12 stimuli listed in Table 16. J=  9-f  FUR GEL PTY SND ACRGLS BSH WD OIL STK HND SLK Stimulus  Figure 45: Mean self-reported arousal and valence ratings for 12 tactile surfaces listed in Table 16. A one way A N O V A was run for the self-report ratings, significant main effects for the affect grid ratings were found between stimulus and arousal (F(ll, 88) = 10.8, p < -001,77 = .574), and between stimulus and valence 2  (F(7.14, 57.2) = 10.6, p < .001, 77 = .571). A Huynh-Feldt correction for spheric2  ity (uncorrelated data) was used because Mauchly's test for sphericity (uncorrected data) yielded £ = .649 for valence. No significant main effects were  135  observed for the biometric data. An example is shown in Figure 46. Mean EMG and SC voltages for the 12 stimuli are shown in Figure 47.  15  Touching Sticky T a p e  >: CD CJ) CO %  Stroked Hand  10  5  >  S C (Arousal! O — 385 1  390  395  400  405  410  Time (s) 20  10 h  > -3-  ro o >  Touching Sticky T a p e  :  0F EMG C ' - - EMG D  -10  -2i  — E M G D - C (Valence) 85  390  395  400  405  410  Time (s)  Figure 46: EMG (lower) and SC (upper) data for a participant touching a sheet of double-sided sticky tape, and a participant's hand being touched by the experimenter. High-frequency components of the raw EMG data were smoothed using a third order low-pass Butterworth filter with 0.1 Hz cutoff frequency; the presented SC data are the raw values.  136  ™i  1  1  1  FUR G E L P T Y S N D A G R G L S B S H W D  1  1  1  r  OIL S T K H N D S L K  Stimulus Figure 47: Mean biometric arousal (SC) and valence (EMG) ratings for 12 tactile surfaces listed in Table 16  Figure 48 shows statistically significant pairwise comparisons with Bonferroni correction for the 12 tactile surfaces lists in Table 16. Each line in Figure 48 represents a significant effect to either p < .01, p < .05, or p < .10 level.  137  Stimuli separated by heavy dark lines were perceived as the most different from one another in terms of arousal or valence. SLK  SLK  Figure 48: Statistical significance response difference of arousal and valence self-report ratings for the 12 tactile surfaces listed in Table 16 . Table 17 lists the average mean and standard deviations across all 12 tactile stimuli for the biometrics and self-reports. Means or standard deviations in Table 17 are not directly comparable between different instruments because the instruments use different units. Means and standard deviations . between EMG and SC measures are also not comparable because they measure different physical properties — muscle activation voltages and moisture levels, respectively. However, individual mean and standard deviation values for a particular instrument and measure are comparable Table 17: Average values across all 12 tactile stimuli  Instrument Biometrics (uV) Measure  Mean  Self-reports (units)  SD  SD  Mean  Arousal  3.8  1.6  5.3  1.5  Valence  -0.4  8.4  4.6  1.3  138  4.4.3 Discussion We observed significant valence and arousal self-report differences between affective responses to the 12 tactile surfaces, however no significant biometric differences were observed (not entirely unexpected). When rating stimuli, participants were asked to use as much of the affect grid as possible. This rating approach yields a relative scaling for affect in the context of the tactile stimuli observed. Conversely, biometric measurements represent affect differences in the context of all the participants' life experiences and their evolutionary affective predispositions '(affective judgements have both learnt and biological components [41]). Compared to this, the 12 tactile stimuli presented span a relatively small affective range:  the difference between feeling glass  or acrylic pales in comparison to skydiving versus relaxing on the beach. Nevertheless, in the context of a user interface environment, small effects add up. Very high effect sizes were observed in the rating data for arousal, r\ = .574, and for valence, rj = .571. Cohen recommends classifying low, medium, and 2  high effect sizes to be t] = .01, 77 = .059, and 77 = .138, respectively [18]. 2  2  2  These high effect sizes from our data suggest that different haptic stimuli can elicit practically important valence of arousal responses, and further user studies are worth conducting. Figure 45 shows arousal and valence rating differences between the 12 different stimuli. The acrylic sheet (stimulus ACR) was rated the least arousing stimulus (M = 2.7). Glass (stimulus GLS), wood (stimulus WD), and silk (stimulus SLK) were also ranked as low arousal textures — M = 3.7, M = 4.4, M = 4.6, respectively. Touching a sheet covered in oil (stimulus OIL) received a high arousal rating (M = 7.3), and the lowest valence rating (M = 1.1). It is interesting to note that there was strong agreement (low variance) among participants that touching oil was not pleasant (low valence); participants varied more in their rating of how strongly they disliked the stimulus. Ratings of valence for the experimenter touching the participant's hand had a wide variance.  139  The upper section of Figure 46 shows example biometric measurements of arousal, and the lower section of Figure 46 shows example biometric measurements of valence. At about 403 seconds into the trial, an increase in skin conductance (SC) occurs when a participant's hand is touched by the experimenter (starting at 400 seconds into the trial), suggesting an increased level of participant arousal. Skin conductance measurements typically have a 2-3 second lag, and this is exactly what we observe in Figure 46. A less pronounced increase in SC was also observed when the participant touched double-sided sticky tape. The EMG D - C curve is slightly positive for hand stroking, indicating a slightly positive valence (preference) for the hand being touched. Conversely, a very strong preference reaction was observed when the participant touched double-sided sticky tape. The EMG D - C curve dips sharply indicating a strong negative valence (dislike) for the sticky tape. The biometric results illustrate the point that with biometric data, the calculation of arousal and valence scores is quite subjective, perhaps as much so as participant subjectivity in stimulus rating. For example, there is approximately a 2 second lag between the time a participant becomes aroused, and their skin conductance becomes elevated. Where to start and stop recording voltages values for a particular stimuli, and normalization of stimuli are nontrivial problems. The error bars of the biometric voltages in Figure 47 are very large. Consequently, we can not draw meaningful relationships between these bar charts and the self-reports in Figure 45. These biometric voltages are given primarily for completeness. They may also suggest the statistical power (e.g., number of subjects) needed for more meaningful future studies involving such biometric measurements.  4.4.4 Study 1 Conclusions Self-report scales effectively measured valence and arousal levels between different tactile surface textures. Individual biometric responses to stimuli, such as the graphs in Figure 46, suggest that EMG and SC biometric  140  measures could usefully estimate respective valence and arousal levels within studies that have greater statistical power (e.g., by using more participants, by identifying and removing of both procedural and technological noise sources before the study, and by developing and applying more sophisticated filtering algorithms to collected raw biometric data). These results support our conjecture that the self-report rating instrument would yield more statistically significant results than equivalent biometric instruments for the subtle haptic stimuli of our experiments. The high effect sizes from our data suggest that comparisons of more subtle haptic stimuli (such as those in Studies 2 and 3), could elicit practically strong and important valence and arousal responses. Future studies are worth conducting. They should initially rely on self-report measures, instead of biometrics, because self-report scales require fewer resources to observe potentially interesting trends. Follow-up studies could then compare particularly interesting factors and levels. These studies would rely more heavily on biometric results and require larger numbers of participants to obtain statistically significant and reliable data.  4.5 Study 2: Rendered Interaction using List Selection With Study 2, we tested whether task context made a difference in affective response to a given rendered haptic environment. To this end, users felt rendered friction, inertia, and detent knob environments with and without the context of a graphical scrolling task. Differences in emotional responses between the haptic renderings used for this interaction study were conjectured to be subtle and difficult to record compared to the textures used in the tactile study. We expected to observe different emotional responses to knob environments regardless of task context, and also for these responses to vary further within the context of the graphical scrolling task. For example, knob environments that helped a participant's task completion time were expected to elicit more positive emotional responses.  141  The primary research questions for Study 2 are: What affective responses do participants have for inertia, friction, and detent physical control attributes regardless of task? How do these affective responses to different renderings compare within the context of a list selection performance task?  4.5.1 Method The participants, apparatus, and procedure for Study 2 are described below.  4.5.1.1 Participants A total of fifteen paid, right-handed students (9 male and 6 female) were recruited from a range of disciplines at the University of British Columbia; ages ranged from 24 to 27 years (M = 24.7, SD = 1.18). Each participant took approximately one hour to complete the study.  4.5.1.2 Apparatus Figure 49 illustrates the experimental setup. Participants sat at a desk approximately 50 cm away from a graphic display measuring 36 cm wide by 29 cm high, and used their right hand to interact with a force-feedback haptic knob anchored to the desk. Noise canceling headphones were worn to block sounds from the force-feedback device. Visual distractions were reduced by seating the participants at a desk facing a corner of the room. •50 c m  cn  £  m  "CD  ©  Ico -go  O % CD CO W CD O X  ForceFeedback Knob  Figure 49: Experimental Apparatus for Study 2.  142  Participants interacted with the custom built force-feedback knob illustrated in Figure 49. Torques were supplied using a Maxon RE40 DC motor, and position was measured at 320 000 counts / revolution. A 5000 Hz haptic update loop was coded in C++ using the Real-Time Platform Middleware (RTPM) [75]. OpenGL was used to code the graphic display. The graphic client obtained the knob position from the haptics server to maintain a 60 Hz graphic update rate. We custom built this setup because haptic knob systems capable of rendering such dynamic effects are not commercially available. Three haptic models, with parameter values a,, b, and m, were used as illustrated in Table 18. To improve stability; velocities were low-pass filtered using a tenth order Butterworth filter with a 400 Hz cutoff frequency. Inertia was modeled using a spring and damper virtual coupling to a simulated mass [19].  Table 18: Force-Feedback Models Used in Study 2 (and Study 3)  Torque  Model Detents Viscous Damping Inertia  T = a j sin (a2 0)  (15)  be  (16)  r = m0  (17)  T  =  Figure 50 illustrates the graphic display. An almost black background was used (the background had a touch of blue to reduce participant eye strain). A red target value was shown to the left or right of a larger, cyan counter value located in the centre of the screen. Rotating the knob counterclockwise / clockwise would respectively decrement / increment the counter value by 1 unit. The target value appeared to the left / right of the counter if the target was respectively less / more than the counter, respectively. If the counter equaled the target, the target would appear on both sides of the counter. The target could therefore change sides during a trial.  143  •*  -36 cm  •  Figure 50: Screen capture of the graphic display Time, self-report (valence), EMG (valence), and SC (arousal) measures were obtained for this study. data was recorded by the haptic server every 100 ms dur-  Timestamped  ing each target acquisition. Self-reports  of valence  were collected using the Self Assessment Man-  nequin (SAM) 1-D rating scale for valence [51]. The SAM is a 9-point scale of adjoining boxes. Participants rate valence by placing an "x" in one of the boxes. A value of 1 corresponded to low valence and 9 corresponded to high valence. This scaling the reverse of the original SAM [51], but consistent with the affect grid given to participants in Study 1. EMG and SC were measured as in Study 1.  4.5.1.3 Procedure The experimental design used two within-subject factors (context and knob stimulus) with two complete repetitions. The context factor had 2 levels: • Freeform exploration:  approximately 5 s freeform exploration of dif-  ferent knob models followed by 10 s to record their valence rating  144  followed by a 4 s rest before the next condition. The display was turned off for this factor. • Target finding:  a timed target task taking approximately 5 s (see  Figure 50) where participants rotated the knob until they matched the counter value to the target value. The same 10 s rating time and 4 s rest time allowances as the freeform exploration task were then given. Task completion times were measured to enable performance comparisons. Because the target finding task might have influenced the way participants performed the freeform exploration task, all participants performed the freeform exploration task first, then the target finding task. The knob stimulus factor had 7 levels of damping, inertia and detents as shown in Table 19 (refer to Equations 15,16, and 17 for coefficient meanings). The stimuli were presented to the participants in a randomized order. The experimenter described the apparatus and procedure to the participants. At the end of each trial (one particular knob rendering level), participants were instructed to mark the valence (preference) on a scale of 1 to 9. To familiarize the participants with the experiment, participants executed two tri. als for each of the freeform exploration and target finding contexts. Unlike Study 1, we only asked participants to rate valence (not arousal) because pilot studies suggested that participants had difficulty assigning different arousal ratings to these stimuli — presumably because these stimuli were more subtle and abstract to the participants compared to the tactile surfaces used in Study 1.  145  Table 19: Seven Dynamic Knob Stimuli for Study 2  #  Label  b  m  a  lf  a  Description 2  "Real World" Example  1  not rendered  0  0  0,0  No force feedback (control)  2  low friction  2.6  0  0,0  Small viscous friction Portable radio volume knob  3  high friction  5.3  0  0,0  Large viscous friction High quality sink faucet  4  low inertia  0  .06  0,0  Small inertia Wheel on a small toy car  5  high inertia  :  0  .38  0,0  Large inertia Fishing reel (free spinning)  6  few detents  0  0  14,271.  1 detent / graphic list item 3-settingfan knob  7  many detents  0  0  14, 2TX/3  High frequency detents Mouse scroll wheel  1. Units in Nm, rad, and s  146  4.5.2 Results As in the Study 1, the data were checked and confirmed for normality using a Q-Q plot. Figure 51 shows mean self-reported valence ratings of the 7 stimuli listed in Table 19.  if a u  c  o re  >  I? o5 .—  E re a> 2  of C  c  £  ro  OJ  la  o  o a  c j£ re >  X  UJ  E re «> 5  E c o  H-  0  o  Not Rendered  Low  High  Friction  Low  High  Few  Many  Detents  Inertia Knob  Figure 51: Mean self-reported valence ratings for the freeform exploration and target finding tasks A 2 x 7 A N O V A was performed to compare the context and knob stimulus factors. Significant main effects were observed for task context (freeform exploration or target finding) (F(l, 14) = 5.75, p < .031, rj = .291) and stimulus 2  (F(4.48,46.5) = 5.79, p < .001, n = .293) as well as a significant interaction 2  between task context and stimulus (F(2.92,40.8) = 4.89, p < .006,77 = .259). 2  Huynh-Feldt corrections for sphericity were used for the task main effect and the task x stimulus interaction because Mauchly's test for sphericity yielded e = .746 and e = .628, respectively. The final repetition of 7 stimuli was used to  147  Low Rendered  High  Low  F r i c t i o n  l n e r t i a  High  Few D  e  Many t  e  n  t  s  Knob Figure 52: Mean times of target acquisition times for each stimulus in the target finding task calculate all statistics except for reliability statistics where inter-repetition consistency was determined. No significant main effects were found for the biometric data, although non-significant trends were observed, as in the first experiment. Consequently, the following results focus on the self-report rating data and performance times. We tested the reliability of the self-reports using a Cronbach alpha test. High alpha values result from high inter-correlations of items. Within the context of our experiments, high alpha values suggest a stronger likelihood of observing the same effects for the valence self-report ratings in replicated experiments. Cronbach alpha tests for reliability were performed between the 1st (training), 2nd, and 3rd repetitions. For the freeform exploration task, KnoD  l o w friction'  Knob  low  inertia'  a n <  ^ Knob jgh inertia had low Cronbach alpha scores n  148  of a= .61, a= .59, a= .57, respectively. The other stimuli were all above the recommended value of a> .70 [72]. For the target finding task, Knobj  f i tion  ow  r  C  had a below-threshold Cronbach alpha score of a = .64. The other stimuli were all above the recommended value of a> .70. These reliability scores (Gronbach alpha values) suggest an acceptable chance of repeating the observed results. However, replicating the study with more participants (e.g., 50+) would be advisable before making knob interaction design decisions based on these results. This suggestion is based on comparable visual psychology studies, such as Russell [79], that tested 50+ participants. For practical reasons, favorable affect ratings are often of secondary concern to performance ratings. Consequently, we compared relationships between affect and performance for the target finding task, for which we had performance data. A significant main effect for stimulus was observed (F(4.78, -66.9) = 5.68, p < .001,77 = .288). Huynh-Feldt corrections for sphericity were 2  used for the stimulus factor because Mauchly's test for sphericity yielded 8 = .797. Figure 53 shows the significant pairwise comparisons, with Bonferroni correction, of observed valence ratings within the freeform exploration and target finding tasks. Here and in Figure 54, lines represent significant differences between two stimuli. Significant differences between freeform exploration and target finding tasks were also found for K n o b 15.68, p  <  .001, K n o b  low m e r t i a  , F(l, 14) = 5.02, p  <  n o t ren  dered' ^(1/14) =  .042, and K n o b  few  detente  0->  F  14) = 14.91, p < .002 conditions — as represented by the shaded circles in Figure 53.  149  many detents  o few detents  not rendered  high inertia Q  p low friction  low inertia  #  v  high friction  P<-01 p < 05  Valence  p < .10  Freeform Exploration many detents o  few detents  not rendered  high inertia  /  low inertia  O low friction  high friction  p < .01 p < .05  Valence  p < .10  Target Finding  Figure 53: Statistically significant self-reported valence ratings of the freeform exploration and the target finding tasks. Shaded circles represent significant differences between two tasks, where a heavy line means valence was reported to be most different between those two renderings [p < .05]. Labels for the seven knobs are listed in Table 19.  150  Figure 54 shows the significant pairwise comparisons with Bonferroni correction observed for target acquisitions times of the target finding task for the seven knobs listed in Table 19. many detents O  few detents  O  O  not rendered  high inertia ©-^  low friction  low inertia p < .01 p < .05 p<.10  high friction Time Target Finding  Figure 54: Statistical significance results for performance (acquisition times) within the target finding task for the seven knobs listed in Table 19  4.5.3 Discussion As one might expect, giving participants a context in which to evaluate the stimuli produced variation in their valence ratings. For example, comparing the freeform exploration and target finding tasks, there was a significant decrease in valence ratings for K n o b valence for K n o b  rew  detents (  decrease), and K n o b  low  i n e r t i a  o n e  n o t ren  dered'  a n c  ^ significant increase in a  detent per counter number increase /  .  Looking at the performance data in Figure 52, one can see statistically significant acquisition time differences between K n o b inertia-  nigrl  f j ion r  ct  a n c  ^ Knobi  ow  During post-experiment discussions, several participants remarked that  the larger amount of inertia in Knob igh inertia versus Knobj i rtia made finen  ow  ne  tuning more difficult at the start and end of the target finding task (e.g., they  151  felt that they were overshooting). K n o b  not r e n d e r e d  had less friction and inertia  than the other stimuli, but there was very large variance between participant times and inconclusive performance differences. This supports the notion that there is a 'sweet-spot' for both friction and inertia, when positioning performance is at stake. In other words, these findings with rendered knobs are similar to previous mechanical knob dynamics work that showed participants prefer, and perform better, with small amounts of friction and inertia, but not too much of either [49]. There is a general trend that knobs that aided a participant's target finding performance corresponded to higher valence ratings (see Figures 51 and 52). However, a more interesting result is the relatively larger valence differences (see Figure 51) between small and large friction levels (Knobi and Knobhigh friction)/ t i a  a n  ow  friction  d between small and large inertia levels (Knob[  ow i n e r  .  and Knobhigh inertia)- compared to the performance differences (see  Figure 52). In other words, cases were observed where participants tended to prefer one stimulus over another stimulus even though there were minimal performance reasons to make such a preference rating.  4.5.4 Study 2 Conclusions Significant valence ratings were observed among inertia, friction, and detent renderings in both a freeform exploration and target acquisition tasks. Participants gave similar valence ratings for moderate magnitude renderings of inertia, friction, and detents compared to a non-rendered knob. This preference for moderate feeling dynamics, regardless of task context, is consistent with similar research using purely mechanical controls [49]. Cases were also observed where inertia and detent renderings resulted in both faster target acquisition times and more positive valence ratings. Thus, positive valence ratings for rendered haptics were observed for particular renderings regardless of task context, and occasionally coincided with improved performance in a target acquisition task.  152  4.6 Study 3: Rendered Interaction using Rotary Pointing The purpose of this study was to compare affect relationships between fine and coarse motor movements as a function of knob rendering model. To do this, we developed a structured target finding task to exercise the different haptic rendering models in a controlled way. This study builds on Study 2's simpler target-finding tasks, involving only a single level of difficulty, where we found some meaningful relationships among the presence of task, knob rendering model, affective response, and performance. Exploring multiple levels of difficulty is important because a particular physical control's dynamics may only be appropriate for tasks of a small range of difficulty levels. For example, high magnitudes of inertia may elicit positive valence and be appropriate when turning a knob to a general location; however, the same knob may inappropriately contribute to overshoots during a fine positioning task. A task based on Fitts Law [31] was developed for this study because the abstract concept of acquiring targets of different sizes is'directly relevant to many physical control tasks such as changing a radio station, fan setting, or video game character's actions. Specifically, target acquisitions of graphical targets of varying amplitudes and widths enabled participants to experience different difficulty levels for a consistent and tightly controlled set of kinematic renderings. The primary research question for Study 3 was: How do participant affective responses to inertia, friction, and detent attributes of physical controls vary as a function of fine and coarse target selection tasks?  4.6.1 Method The participants, apparatus, and procedure for Study 3 are described below.  153  4.6.1.1 Participants Nineteen paid participants (9 female, 10 male) were individually tested. Each participant took approximately one hour to complete the study. All participants were right-handed students from a range of disciplines at the University of British Columbia. Their ages ranged from 19-35 years (M = 23.2, SD = 3.6).  4.6.1.2 Apparatus Participants sat at a desk and used their right hands to interact with a haptic knob embedded in a graphic display. The lights in a self-contained experiment room were dimmed to focus the participant's attention on the task. Figure 55 shows the apparatus. Figure 56 illustrates how a typicalparticipant rotated the haptic. knob towards a projected graphical disk while feeling force-feedback rendered through the knob. A third computer controlled a touch pad used to collect self-reports, and it monitored biometric sensors to obtain participant EMG and - SC readings. Participants wore noise canceling headphones that played a waterfall sound with a 'near-Gaussian' audio distribution to mask distracting audio cues from the apparatus.  )  154  A)  Haptic knob embedded into a graphic display  B)  Touch sensitive surface for self-reports  C)  Participant chair (with grounding pad)  D)  Biometric sensors (EMG and SC)  E)  Trial haptics computer and peripheral hardware (all under table)  F)  Trial graphics scheduling computer  G)  Biometric and self-report computer Figure 55: Experimental Apparatus for Study 3.  155  Figure 56: Pointing to a graphical target Kinematic renderings were computed according to the same equations as in Study 2 (refer to Equations 15,16, and 17 on page 143, respectively), to create the haptic effects summarized in Table 20. Renderings were created with the same force-feedback knob apparatus used for Study 2.  156  Table 20: Seven Dynamic Knob Stimuli for Study 3 #  Label  b  m  ay &2  Description "Real  World"  Example  1  not rendered  0  0  0,0  No force feedback (control)  2  low friction  2.6  0  0,0  Small viscous friction Portable radio volume knob  3  high friction  7.9  0  0,0  Large viscous friction High quality sink faucet  4  low inertia  0  .06  0,0  Small inertia Wheel on a small toy car  5  high inertia  0  .21  0,0  Large inertia Fishing reel (free spinning)  6  few detents  0  ' 0  7,2TX  1 detent / graphic list item 3-setting fan knob  7  many detents  0  0  19, 2TI/5  High frequency detents Mouse scroll wheel  1. Units in Nm, rad, and s Figure 57 illustrates the knob mounted in the centre of a rear-projected display with 1024 x 768 resolution and 1500 lumens of brightness. The polycarbonate cap on the knob had a diameter of 64 mm, depth of 13 mm, and a 3 mm filleted edge. The black knob needle was 100 mm in length, extending to the centre of a white graphic target disk that was displayed by the software during trials. Participants pointed to graphical targets of two amplitudes (30° and 200°) and two widths (5 mm and 40 mm). The same OpenGL software infrastructure from Study 2 was used to drive the graphic display. The graphical client obtained knob position data from the haptics server to maintain a 60 Hz graphical update rate.  157  A = 30°, 200° '  \  s \ \  / \  /  300 m m  I i i i •i i . i i  •  • •  \  _ _ _ _ _  ^  .  —  \  •  i i i i  i i . i i i i i i i •i i i i  W = 5 m m , 4 0 m m (1.4°, 11.3°) 375 m m  Figure 57: Graphic target amplitudes and widths Time, self-report (valence), EMG (valence), and SC (valence) measures were obtained for this study. Timestamped Self-reports  data was recorded as in Study 2. of valence  were measured at the end of each trial using a  touch screen controlled by an array of nine graphical buttons. We used a MERL Diamond Touch™ touch screen (an input-only device with no graphic display), although its more advanced features were not used and other touch screens would perform equally well. Similarly, Visual Basic was used to program the button behavior, but other graphics software would work equally well. The graphical buttons were not projected on the touch screen because the participant's hand and body could have produced distracting shadows on such a displayed surface. Instead, the white touch pad had a set of nine 3 cm x 3 cm boxes that were etched using a black pen onto the touch screen surface to  158  create a 9-point rating scale where 1 and 9 represented extreme high and low valences, respectively. This rating scale is the reverse of that used in Studies 1 and 2, but is the same as the original SAM [51]. To select a self-report rating scale item, participants would use their index finger to press a box on the touch screen instead of using a selection tool such as a pen or mouse. This procedure improves upon the procedures of Studies 1 and 2 because participants could rate a stimulus with greater speed and ease. EMG and SC were measured as in Study 1.  4.6.1.3 Procedure Each participant completed 4 blocks each consisting of all 28 combinations of 7 knobs x 2 amplitudes x 2 widths presented in a random order for each participant in each block. The experimenter read instructions to the participant from a script before the experiment. Participants were given a few minutes to rest between blocks. Figure 58 shows how every trial required three rapid movements of the knob back and forth, reminiscent of a classic Fitts tapping task. For each trial, the participant first aligned the knob's pointer over a small, white 5 mm diameter graphical disk. One of the 7 haptic renderings was then applied to the knob. Upon display of one of four possible graphic target disks, the participant moved the knob to acquire it. Once over the graphical target disk, the target disappeared and a second target disk appeared with the same traversal distance (Fitts-like amplitude magnitude) and the same diameter (Fitts-like width) as the first disk, but requiring an opposite traversal direction. After rotating the knob towards this second disk, it was replaced by a third graphical target disk of the same size and location as the first disk. After acquiring the third disk, the movement part of the trial ended, and the participant then rated the appropriateness of the haptic knob rendering for the particular amplitude and target width used in the trial. In other words, a participant would rate how well or poorly the current knob rendering helped them perform the current graphical target acquisition. This description of appropriate-  159  ness was a more consistent and understandable method for obtaining valence as compared to asking participants to explicitly rate 'valence' — a word that many people are not familiar with. Participants were instructed to supply their response on a scale of 1 to 9 by pressing a graphical cell on the touch pad using the index finger of the right hand.  Stepl Step 2 Step 3 Step 4 Figure 58: Pointing target acquisition task. Participants start at a small target circle (step 1), then perform three rapid target acquisitions to a target disks (steps 2-4). After each target acquisition, the old disk disappears and the new disk is displayed. No disks are visible after the final target acquisition. The three successive target acquisitions in each trial were used to give participants a sufficient amount of time to mount a visceral response to each haptic rendering. The repeated angular velocity 'ramp-ups' and 'ramp-downs' as each of the three graphical targets were acquired enabled participants to quickly experience consistent velocity and acceleration force-feedback responses. Thus the 'feeling' of each knob rendering was tightly controlled for each graphical target acquisition trial. The first block of trials was treated as a training task, although participants were not told this. The other blocks were performed to control for three types of apparatus difficulties known a priori by the author. These difficulties were (i) controlling haptic stability during rendering, (ii) maintaining good EMG and SC electrode contact to the participant's skin, and (iii) electrically grounding the response touch pad. Efforts were taken to minimize all of these. For stability, a proportional-derivative-integral haptic torque controller was designed using a root locus technique, the knob velocities were low-pass filtered with a 10th order real-time Butterworth filter containing a 500 Hz cutoff frequency, and accelerations were rendered using a 'virtual mass' [19]. To  160  maintain electrode contact, participants were asked to raise their eyebrows and then frown following application of the EMG electrodes. Electrical grounding of the touch pad was tested by asking participants to select each rating scale item several times before starting the experiment. Biometric responses and, to a lesser degree, self-reports are sensitive to the most minor of experimental disruptions. In an effort to obtain a complete set of high quality data (at the cost of larger data quantities), a block was discarded if the complete apparatus did not perform perfectly for the entire block (e.g., the knob controller had to be stable, the biometric contacts had to be maintained, and the touchpad had to function for every trial in the block). Nine participants experienced at least two blocks with absolutely no disruptions. From these data, the first two blocks containing no disruptions were gathered to form 18 complete sets of data for statistical analysis. Because the biometric data require more statistical power than other data, such as selfreports and performance times, to observe meaningful results, this reduction in amount of data for statistical analysis was conjectured to have a minimal impact on the self-report and performance results.  4.6.2 Results We first tested for data reliability, and consistency with previous affect theory. We then examined statistical results to answer our two primary research questions: (i) how do physical control dynamics influence affective responses, and (ii) how do affective responses correlate with physical performance for a given physical control dynamic? Statistics were performed for the parametric scale measures (SC, EMG, and time) and non-parametric ordinal measure (rating) to achieve two goals: (i) quantify associations between variables, and (ii) compare groups of variables. To quantify associations between parametric and non-parametric, measures, the more conservative Spearman correlation was used. To compare groups of three or more parametric groups, repeated ANOVAs were performed, then pairwise comparisons with Bonferroni correction were used to  161  compare individual levels. Similarly, to compare groups of three or more nonparametric groups, a Friedman test was performed, and Wilcoxon tests were used to compare individual levels. Data Reliability: To validate the reliability and repeatability of our  data, we conducted Cronbach alpha standardized item tests on the 18 final cases to ascertain consistency across the first two disruption-free blocks 2, 3 and 4. This yielded a= .896, which is well above the recommended minimum value of a > .7 [72]. Data for all three metrics were also checked and confirmed for normality. We concluded that our data were reliable. Pre-Statistics Filtering: The same filtering procedure used for Studies 1  and 2 was performed. Raw collected biometric data required low-pass filtering before statistics could be performed. No filtering was needed for the time and self-report measures. Performance Results: Table 21 summarizes correlations between time  and amplitude, width, knob rendering, and rating. Rows show the correlation, A  p, and the level of significance, p. Significant non-parametric correlations were found between target acquisition time and amplitude, knob, and rating.  Table 21: Non-parametric correlations grouped by time Time |  P (504)  Amplitude r  :  Knob  Rating  .^ ;P- ;t298;'  'y.l2^i .006**  V  Width ;  .778  ;  .000**  .000**  * Significant at .05 level (2-tailed) ** Significant at .01 level (2-tailed)  A repeated measures ANOVA was used to test the amplitude, width, and knob factors. Amplitude and knob main effects were observed for time [F(l, 8) = 5.3, < -05, t] = .399, and F(4.69,37.5) = 10.9, p < .001,7] = .576, 1  2  v  respectively]. A Huynh-Feldt correction of e= .782 was applied to the knob data to correct for a lack of sphericity. Table 22 shows the standard errors (SE)  162  and significance level (p) of the time differences for six knob pairs. Pairwise comparisons with Bonferroni correction were performed between (i) the nonrendered control knob, K n o b renderings, K n o b  high h i c t i o n  not ren  d ed'  , Knob  a n  e r  high i n e r t i a  d each of the three types of knob , and K n o b  m a n y de  tents'  a  n  d  (ii) the  two levels of each knob friction, inertia, and detent renderings.  Table 22: Pairwise comparisons of time for selected knob renderings Knob Pairs  high friction  high inertia  high detents  high friction  high inertia  many detents  not rendered  not rendered  not rendered  low friction  low inertia  few detents  SE  .046  -045  V  .040*  .133  .027 • .001**  ^051 •;' .931  :3d32 1. .549  .023: .029*  * Significant at .05 level (2-tailed) ** Significant at .01 level (2-tailed)  Figure 59 shows the previously described main effects as well as pairwise comparisons for knob, width, and amplitude vs. target acquisition time.  163  Not  Low  High  Rendered Friction  Low  High F e w M a n y  Inertia  Detents  Knob  1.4°  11.3  0  30°  200'  5 m m 40 m m Width  Amplitude  O - O Significant at . 0 5 level (2-tailed) • - • Significant at .01 level (2-tailed)  Figure 59: Knob, width, and amplitude vs. target acquisition time Affect Results:  Spearman correlations were calculated as shown in  Tables 23, 24, and 25. For each measure rating, SC, and EMG, rows show the non-parametric correlation, p, and the level of significance, p. Significant correlations were observed between rating and amplitude, knob, and EMG. Significant correlations were also observed between EMG and knob. A repeated-measures A N O V A conducted for EMG and SC did not show statistically significant results. Non-parametric tests for rating showed significant differences for amplitude and knob rendering factors. Specifically, Wilcoxon Signed Ranks tests between rating and amplitude were significant [Z = 3.51, p < .001], and between rating and width were marginally significant [Z = 1.68, p < .092]. A Friedman test on the knob rendering showed significant rating differences of  164  £ (6, N = 72) = 49.49, p < .001. A total of six post hoc Wilcoxon Signed Rank tests with Bonferroni correction were performed on the same six pairwise comparisons that were performed for time (refer to Table 22, "Pairwise comparisons of time for selected knob renderings," on page 163). Table 26 shows these significant rating differences observed from all six tested pairs of knobs  Table 23: Non-parametric correlations grouped by rating Rating i  P  Amp.  Width  Knob  ( )  105.  -.034  :v:.lm:  V  .018*  .443  .008**  504  SC '.  -  0  5  3  .231  EMG .135 .002**  * Significant at .05 level (2-tailed) ** Significant at .01 level (2-tailed)  Table 24: Non-parametric correlations grouped by SC SC p (504) V  Amp.  V  Width  -.054 '. • ^ v -.015; :  .228  .742  Knob  .022  ?  .619  * Significant at .05 level (2-tailed) ** Significant at .01 level (2-tailed)  Table 25: Non-parametric correlations grouped by EMG EMG  Amp.  Width  p (504)  -.005  -.009  V  .919  . .835  Knob  .049*  * Significant at .05 level (2-tailed) ** Significant at .01 level (2-tailed)  165  Table 26: Pairwise comparisons of rating for selected knob renderings high friction  high inertia  many detents  high friction  high inertia  many detents  not rendered  not rendered  not rendered  low friction  low inertia  few detents  Z  -2.30  -3.37  -3.39  -2.91  -3.32  -4.14  V  .022*  .001**  .001**  .004**  .001**  .000**  Knob Pairs  * Significant at .05 level (2-tailed) ** Significant at .01 level (2-tailed)  Figure 60 illustrates the previously described main effects and pairwise comparisons for knob, width, and amplitude vs. rating (valence).  9-1  Not  L o w High  Rendered Friction  L o w High F e w M a n y Inertia  Detents  Knob  1.4°  11.3°  30° 200°  5 mm 40 mm  Width  Amplitude  O-O Significant at .05 level (2-tailed) Significant at .01 level (2-tailed)  Figure 60: Knob, width, and amplitude vs. rating (valence)  166  Fitts Law Results: Because  this study's pointing task is essentially a  rotary version of Fitts Law [31], we checked if the target acquisition time (movement time) data followed the Welford version of Fitts Law (see Equation 18). Welford demonstrated improved performance using his slightly modified version of Fitts Law for tasks with indices of difficulty (ID) less than 1  three [103]. Index of Difficulty = l o g ( ^ ^ + O.s)  (18)  Figure 61 shows the Movement Time (MT) vs. Index of Difficulty (ID) graph for the data points from all seven types of knobs combined. Four columns of ID data represent the four different combinations of target amplitude and target width. Figure 62 shows a MT vs. ID graph for the data points of one typical knob, Kriobf  ew  detents-  Fitted lines had slightly increasing slopes for  each of the individual active rendered knob conditions, and a slightly negative slope for the control rendered knob condition K n o b  not  r e n  dered-  We- observed  R < .1 for the fitted lines of all the plots. In other words, adherence to Fitts 2  Law was not observed. Similarly, we observed R < .1 for MT vs. A / W plots. 2  Table 27 lists effective target widths for each of the four target amplitude/width conditions. The idea of effective target width is to look at the variation (variance) from the mean of the final location when a participant acquired each target.  167  1  2  . . . .  . . . .  ^  4  •  •  1 0 A  3o°  :  A  200° i;  W 08 A  1 1 - 3  °!  3  0  O  A  200° I  W  •  l  i  4  o  j j  <  Js  i*  E  A  <•  • •  F c 0.6  !  ;  Fit Line 04  0 2  i R = .006 2  0.0  00  i  t  * 05  f  1  I  0  1  5  2 0  25  Welford Index of Difficulty  Figure 61: Movement Time vs. Index of Difficulty for All Knobs 1.2  -R  10  1  5  2 0  2 5  Welford Index of Difficulty  Figure 62: Example Movement Time vs. Index of Difficulty for a Particular Knob ( K n o b ) F e w D e t e n t s  168  Table 27: Variance of Movement Times for each Index of Difficulty for All Knobs Amplitude  30°  30°  200°  200°  Width  1.4°  11.3°  1.4°  11.3° f ;U8,2-;j  Index of Difficulty (ID) Variance (a ) (degrees ) 2  2  1.02  .92  1.39  1.42  4.6.3 Discussion Analyses are organized according to several high-level questions. (1) Are the affect results what we would expect?  Finding significant Spearman correlations (see Tables 23, 24, and 25) between EMG (valence) and rating (valence), but not between SC (arousal) and rating (valence), is exactly what one would expect from previous research [52, 79], which reports the primary emotional dimensions of valence and arousal to be orthogonal. Main effects for the self-report ratings were observed to be significant even though equivalent main effects for the EMG were not observed to be significant. It is likely that involuntary biometric measurements such as these are calibrated to the full range of human experience, over an individual's lifetime and perhaps over many successive generations of human development (i.e. a full-scale response might be genetically enabled even if never experienced by an individual). Conversely, the self-report ratings for valence span only the context of these stimuli, and subjects are able to voluntarily self-calibrate. Differences between the knob renderings and graphical disks were small compared to levels of previous visual psychology studies such as Lang [52] that compared powerful images including dismembered body parts, furry seals, and nude models. Consequently, the relative significance of the EMG valence to the self-report valence indicate absolute valence whereas the individual dif-  169  ferences among the self-reports indicate relative valence. Thus, the Study 3 results suggest that although affect stimuli were weak compared to very strong stimuli in other studies, participants could (i) tell the difference between, and (ii) had consistent and measurable preferences for particular position-, velocity-, and acceleration-based knob dynamics. The remaining analyses focus on these self-report ratings, and utilize the target acquisition times to compare preference / performance relationships. (2) How did the affective responses vary in the knob rendering, target width, and target amplitude levels?  As shown in Figure 60, many significant self-reported valence differences were observed. Participants generally preferred the more subtle renderings of friction, inertia, and detents to the stronger renderings. One might argue that participants were not able to feel the difference between the subtle renderings and the K n o b  not  ren  d rede  but this is unlikely because all the render-  ings differed in magnitude > 20% from the K n o b  not  ren  dered  a s  recommended  by previous human factors studies using mechanical knobs (e.g., Knowles and Sheridan  [49]).  The similar valence scores for K n o b  not  ren  d red e  versus K n o b i  ow  friction'  Knob  low  inertia'  an<  -^ Knobf  ew  detents suggest  that the haptic renderings  can be made to feel as good as a passive mechanical control. This is important because vibrations that occur in virtually all active rendered haptic devices are anecdotally believed to feel unpleasant. The similarity in valence results between K n o b Knob  not  ren  dered  a n  d each of K n o b  low  friction'  Knob  low i n e r t i a  , and  few  detents suggest  that slight inconsistencies inherent in active haptic  controls can be reduced to insignificant levels from an affect standpoint. The valence findings that generally favor knobs with small amounts of friction (velocity-dependent) and a small amount of inertia (acceleration-dependent) are also consistent with previous findings using mechanical knobs [49]. We  170  thus have further evidence that the friction and inertia renderings used in this experiment correctly model mechanical friction and inertia dynamics. Differences in self-reported valences between the haptic levels for friction, inertia, and detent stimuli, were greater than the differences between the rotary pointing task-related parameter settings of width and amplitude (A " 30  and A oo° or W 2  5 m m  and W4o ). These results suggest that, for this task, hapmm  tic rendering had similar or greater effects on the participant valence measures than the pointing task index of difficulties (different graphical target widths and amplitudes). (3) How did response times (task performance) vary in the knob rendering, target width, and target amplitude levels?  Figure 59 shows graphic and haptic temporal performance results that one would intuitively expect. Movement times took longer a) towards greater amplitude targets, b) with higher friction knob renderings, c) with lower inertia knob renderings, and d) with detents that were not spaced in a logical relationship to the task. Comparing time with K n o b friction  not r e n d e r e  d  t o  Knob  low f r i c t i o n  and K n o b  high  (Figure 59), higher friction appears to reduce performance. Presumably,  the finer control afforded by the additional friction was more than offset (negatively) by the extra physical exertion needed to rotate the knob. Although only moderately significant (see Table 22 on page 163), finding similar times for K n o b i i tia ow  ner  a n c  * Knobhigh inertia  mately 10% faster than the control knob K n o b  not  m a  t are both approxi-  rendered' despite  a 3.5x inertia  variation between the two samples, is a helpful finding for haptic designers. Inertia is more difficult to render than friction or detents because accurate acceleration is technically more challenging to measure than velocity or position. Times for these inertia stimuli suggest that a small amount of inertia improves performance, but larger amounts of inertia provide minimal additional performance benefits for tasks of the sort we studied. Also of interest to  171  designers, times for the detents were similar to inertia renderings, and significantly less than friction and control renderings. Because detent rendering only requires position sensing, programmable as well as mechanical detents are much easier and less expensive to produce compared to inertia renderings. For example, a programmable detent rendering can be designed from a simple potentiometer and braking actuator instead of an optical encoder and servo motor. The shorter times for K n o b  man  y  detents  compared to Knobf  ew  detents  a  r  e  probably due to a combination of (i) high frequency detents more closely resembled continuous friction than low frequency detents, and (ii) 10° / click of stimulus Knobf  ew  detents  felt hTce an intuitive mapping to the 30° and 200°  amplitudes whereas the 2° / click of / K n o b  m a n y  detents  felt more like a texture.  (4) How did task performance results compare with affect results?  As a reflection of the complex interdependencies of preference and performance, valence responses sometimes agreed with, and sometimes disagreed with, time responses. An example disagreement was that participants preferred Knobf detents  even though K n o b  tions faster than Knobf  ew  man  y  detents  detents  (  s e e  ew  helped them perform the target acquisiFigures 59 and 60). This result is an exam-  ple where participants preferred the feel in its own right — regardless of its performance influences. Although both amplitude and knob main effects for time were statistically significant, the knob differences may be more practically significant than the amplitude (or width) differences. Mean times for amplitudes of 30° and 200° varied by ~5% whereas mean times for the most extreme knob renderings Knob  high  friction  and K n o b  m a n y  detents  varied by -25%. These results suggest  that designing appropriate haptic feedback for a physical control can influence  172  temporal performance more than the spatial organization of the control's settings. (5) How do the data fit to Fitts Law? Adherence to Fitts Law or a linear MT vs. A / W relationship were not observed. In Figures 61 and 62, we see four columns of data — one for each of the combinations of target amplitude and width. Movement times for each of these columns varies significantly, but the mean value is similar for each index of difficulty. This data arrangement results in extremely low R values for the 2  fitted data (R < .1). 2  The effective target widths are another way of observing the A N O V A result showing no statistical significance for target width. Even though the large target is approximately 8 times as large as the small target, the effective target sizes (variation) are quite similar. The large target has the smallest variance (for small amplitude) and the largest (for large amplitude) variance. These results suggest that participants often stopped beyond half the width of the small target (1.4° / 40  5  mm), but almost never missed the large target (11.3°  /  mm). Overall, the similar movement times for each index of difficulty in Fig-  ures 61 and 62 supports the conjecture that knob rendering is a more impor-  j  tant design consideration than target width or amplitude. Figure 59 on page 164 also suggests this conjecture. For example, the mean movement time for K n o b i  ow  Knob  detents- However,  m a n y  f i tion r  c  * rn°re than 50% greater than the mean movement time for s  the movement times for the different target widths  and amplitudes are very similar.  4.6.4 Study 3 Conclusions Study 3 builds upon the single-leveled performance task of Study 2 by comparing affective responses to a target selection task containing different levels of precision. The knob renderings generally had a greater influence on valence ratings than the task's index of difficulty. As a reflection of the com-  173  plex interdependencies of preference and performance, valence responses sometimes agreed with, and sometimes disagreed with, performance responses.  4.7 Conclusions and Future Work for Affect User Studies We provided three inter-related studies to measure valence and arousal responses to haptic stimuli. Study 1 demonstrated the effectiveness of selfreport rating scales, EMG and skin conductance for measuring valence and arousal responses to a range of real tactile surfaces. These results provided grounding reference points for study of affective responses to rendered haptics. Study 2 demonstrated affective responses to position-, velocity-, and acceleration-dependent renderings on a physical control both without context, and within the context of a list selection performance task. Study 3 built upon Study 2's observations by comparing valence and arousal responses to target acquisition times of varying precision. Two main contributions were demonstrated: a process for measuring visceral emotional responses to physical controls, and affect user study data for guiding appropriate physical control design. We demonstrated the effectiveness of a general process using self-reports and biometrics for measuring relative and absolute levels of the affect induced by physical controls. A validated mechanism to measure affect valence was used to demonstrate that physical control renderings of position-, velocity-, and acceleration-based effects can significantly influence affective responses. Rendered parameters of the physical knob model were then shown to significantly influence target acquisition times in a tightly controlled performance task; and significant relationships between affective responses and these performance results were discovered. For example, smaller magnitude knob renderings of friction and inertia were preferred to larger ones, detents that were perceived as textures were preferred to 'louder' more distinct detents, and renderings could be made to feel as good as 'real' mechanical knobs. A key take point here is that "classic Fitts analysis" did little to explain performance, whereas the affective  174  characteristics tell us much more. Even the simple intuition that moving farther should take longer did not result in a big effect in terms of relative times. Rather, the style of the knob rendering has at least as much to do with task time as does the distance moved. Future work will include additional self-reported affect studies with a greater variety of haptic interfaces and contexts. For particularly interesting small subsets of interfaces and contexts, biometric studies with more participants and repetitions will be used to explore more absolute affect ratings and individual differences. Other more expressive user study contexts, such as moving a graphical object on a computer screen, could also yield interesting results. More subtle study of weightings among, affect, performance and cost could help motivate more rapid adoption of appropriate affective interfaces into commercial products. Furthermore, comparing larger sets of mechanical and rendered mechatronic controls could yield additional interesting insights into the cost-benefit tradeoffs of various position-, velocity-, and accelerationbased dynamics. Instead of adding one dynamic effect to a base physical control, combinations of position-, velocity-, and acceleration-based dynamics could be rendered to better understand relationships between various dynamic properties. Now that self-reports for valence have been shown to accurately reflect biometric data in Study 3, a similar experiment with a 2-D affect grid could provide further insights into relationships between valence and arousal — assuming that the task is easy enough for participants to rate the different experiment levels using the affect grid. More specific affective attributes represented as sub-regions on the affect grid could then be compared to extensive vision-based studies using the affect grid. For example, haptic behaviors could be categorized with more specific emotional labels such as "joyful" or "fearful" (refer to Figure 42 on page 126). Future work should also include rotary Fitts Law studies! These user studies should test a greater selection of indices of difficulty (> 4) with a single type of knob. Such a study might more conclusively determine whether rotary target acquisition follows Fitts-like behavior, a finding that the results in our  175  study do not support. Because the long amplitude condition of 200° in our study is close to the maximum possible wrist rotation a participant can perform without clutching. This amplitude may be a special case from a human motor control perspective. User studies involving a larger number of target amplitudes and widths might confirm this, and perhaps identify other special cases in human motor control. Most important is the integration of the presented evaluative testbed into an efficient design cycle. Self-report and biometric methods of affective t  measurement, and target acquisition tasks similar to those presented in Studies 2 and 3, could be incorporated into the design cycle of passive and active physical controls. The test bed would thereby improve appropriate affect and performance attributes of physical control dynamics.  176  Chapter 5 H a p t i c Icon P r o t o t y p e r In previous chapters we have described how the "feel" of a physical control can be captured by a Haptic Camera and characterized for subsequent haptic rendering that faithfully captures both the mechanical and the emotional aspects of the physical device. We then described user studies that investigated the degree to which the affective properties of a haptic rendering could be properly measured. This chapter is in some ways the capstone of the research. It describes how we analyze, develop, and share haptic behaviors for practical use in interface design. We describe a testbed, called the Haptic Icon Prototyper (HIP) that has been developed to support rapid interactive prototyping of haptic interfaces for knobs. Key concerns in the design of the Haptic Icon Prototyper are its functionality, which builds directly on the work reported in the earlier chapters of this thesis, and on its usability. The Haptic Icon Prototyper presents a variety of representations of a haptic behavior for manipulation by a designer. These multiple representations are,a prerequisite for building useful design tools for envisioning, expressing, and iteratively manipulating haptic behaviors. These behaviors are then rendered on a haptic device. Two contributions are presented in this chapter. We introduce a custom Haptic Icon Prototyper that includes novel interaction features that were iteratively developed in response to our emerging understanding of the necessary affordances. We discuss the lessons learnt from its development, and from our experiences with developing a variety of haptic devices, to identify design choices for haptic prototyping tools such as the testbed we developed.  177  5.1 Introduction and Related Work Haptic behaviors are touch-based interactions that represent some kind of meaning to the user. For example, a particular vibrotactile pattern from a cell phone can indicate a caller's identity [74], or a unique feeling on a radio tuning knob might convey a station's genre of music. A general-purpose prototyping tool should be able to facilitate the design of many types of haptic interfaces, encompassing different types of interaction, degrees of freedom (DOF), and dynamism tool. Our development over a 2-year period of a specific Haptic Icon Prototyper allowed us to identify a collection of design issues. Further insights gained from using the Haptic Icon Prototyper allowed us to are then used as a basis to develop a set of general guidelines for haptic prototyping tools. But first, some related work helps set the context of this tool and the haptic behavior guidelines it helped us to identify. Many approaches can be used to prototype haptic behaviors [28, 39,57, 74]. Collections of tactile surfaces can aid exploration of haptic behaviors for a tactile interface. For example, one could use surfaces of silk, wood, sandpaper, and metal to directly express how a mechatronic tactile system might feel in particular system states. Collections of mechanical assemblies could similarly represent how a kinaesthetic haptic behavior might feel. A more theoretical or conceptual approach could involve sketching waveforms or, more generally, expressing mathematical representations, of a haptic behavior over space and/or time. Often ignored are visualizations of physical device limitations, or the user's psychophysical boundaries, and how these constraints might affect the rendering and perception of haptic behaviors. For example, Weir et al. [101] developed the Haptic Profile concept to visualize subtleties of switch movements (e.g., friction resistance profiles over position). Such techniques are especially important for examining differences between haptic models and physical realities, including unintended hysteresis or backlash. An example of perceptual design technique is the use of 2-D multidimensional scaling plots  178  to quantitatively show perceptual differences between several haptic icons [58]. Although perceptual sensitivity would be a useful addition to a haptic prototyping tool, there are many unknowns within the perceptual limitations of haptic behaviors. So, currently, the best approach for haptic prototype design is to perform perceptual user studies to compare several designed haptic behaviors after they have been developed. For example, Lee and Hannaford [53] explored haptic thresholds of a person's index finger when using a pen based haptic display under various force feedback conditions. We have already described the mathematical representation we have developed for haptic behaviors. These are the internal representation used within the Haptic Icon Prototyper. There are a variety of external representations that are seen by the designer who uses the Haptic Icon Prototyper. Some are very closely related to the underlying mathematical representation, but others are more abstract. Each is chosen by the designer for exploring the full range of the design space. Previous work on video and audio mixing and editing applications, and 3-D animation systems provided inspiration because they contain many user interaction concepts that are relevant to haptic prototyping tools. For example, audio and animation editors offer modifiable graphical representations of audio / video data elements and data streams such as Apple's iMovie ' [8], Adobe's Soundtrack Pro [4], and Alias's Maya [6]. Often, these are visual representations of waveforms or motion paths, sometimes referred to as "channels". A full video sound track, or animation is built up of many channels that are combined and blended in various ways to provide a complete "user experience" for the ultimate viewer or listener. In a similar manner, waveforms representing haptic behaviors can be edited. For example, the audio icon work pioneered by Gaver [33], and further documented by Buxton et al. [12], provides a starting point for haptic icon development. People have successfully used audio tools to create haptic effects. Chang and O'Sullivan [14] used audio waveform tools to create recorded haptic icons to be played through vibrotactile actuators in cell  179  phones. Nevertheless, the fundamental differences between audio and haptic modalities necessitate the creation of tools tailored for haptic development. Many movie editors and flowchart tools use palettes of components to represent data streams and elements. For example, stencils in Microsoft Visio [65] contain rectangle, circle, and triangle shapes for building flowcharts, and palettes of film clips in Apple iMovie can be combined into sequences using cuts and dissolves [8]. A haptic prototyping tool could similarly utilize graphical tiles to represent and organize collections of haptic behaviors. Additional user interaction techniques from movie editors such as fading, merging and filtering effects are also relevant to haptic behavior design, where similar operations can be used to blend primitive haptic behaviors into more complex compound behaviors. Although graphical user interface (GUI) tools are usually assumed when describing prototyping tools, custom physical interfaces (tangible media) may be more appropriate - especially in early stages of design. For example, many rich physical interactions, such as physical splicing of film, were lost when movie editors switched from editing physical rolls of film to GUI-based video editing. Snibbe and MacLean [82] describe a set of prototyping techniques for haptic manipulation of digital media that try to recapture this aspect of editing. When designing haptic behaviors, designers need to be able to directly experience the behavior quickly and easily, to promote highly iterative design. Our Haptic Icon Prototyper is tightly coupled with a haptic Tenderer that uses the mathematical representation developed in our research. The designer is thus able to see and manipulate multiple representations of the haptic behavior, as well as to actually experience it during the design process.  5.2 Example Prototyping Tool and Rendering Setup. Previous haptic icon prototyping tools, such as the Hapticon Editor [28] and Immersion Studio [39], are specialized GUI tools for editing spatial and temporal torque waveforms for a particular device. The next few sub-sections  180  describe a new custom haptic icon prototyping tool and then use it as a concrete example to support discussion of more general haptic prototyping tool design principles. This icon prototyper is primarily designed for creating fast prototypes for 1 DOF haptic actuators such as knobs, sliders, pressure actuators, or temperature actuators. Its target users are designers who are knowledgeable about haptic behaviors, but do not necessarily have a strong programming or mathematics background. Our Haptic Icon Prototyper contains three major enhancements to previous work: • The concept of "haptic tiles" to arrange and organize collections of haptic icon primitives. • Streamlined interactions for faster haptic prototyping iterations. • Support for active haptic properties and interactions. The Haptic Icon Prototyper focuses on renderings of kinaesthetic forcebased waveforms, but our interface design supports mappings to other waveforms such as temperature or moisture. Design of a representation for a 1-DOF actuator is also considered. State changes and sequencing of haptic icons are currently performed by designing separate haptic icons for each state and DOF, then referencing these icons from programmed code.  5.2.1 Haptic Icon Prototyper Summary Figure 63 illustrates a screen capture of the Haptic Icon Prototyper. It contains three main interaction regions: 1.  Waveform editor.  Represents the magnitude of a haptic signal vs.  space or time. " 2.  Tile palette.  Contains tiles representing basic haptic effects that can  be combined to create a haptic icon. 3.  Tile pane.  Enables combining basic haptic tiles into more sophisti-  cated tiles.  181  UJL!  \ Haptic Icon Prototyper v. 0.7 Fie  Edit Action  Options  Help Position  jTocHj %  0  f  1  jc\data\invKt.nsi  Torque | 100+j %  O li  ±1  Pos p  Vel  /  ©  (7  Acc  P  UMJU1J2J  r  • Hamicon Editor v 0.7 ' (butil 09 02 200:0  Figure 63: Screen capture of a Haptic Icon Prototyper showing the 3 main interaction regions: 1) waveform editor, 2) tile palette, and 3) tile panel  5.2.2 Mathematical Representations The underlying representations used to store the haptic behaviors will constrain and shape any haptic prototyping tool. For our tool, we use Equation 19 as our underlying representation for position-based icons and Equation 20 for our time-based icons. These equations represent the reactions of the actuator to the user's hand position, velocity, and acceleration over space (Equation 19) or time (Equation 20). Matrices of such equations and/or higher order effects could be used to handle multiple-DOF and higher fidelity actuators, making the approach of our one DOF prototyper generalizable to higher DOF and/or higher fidelity haptic behavior design. N,  Nj  W) = Hfje)d , pos i= 1  r  N,  +2/ ie)o + , £ ,/ acc T^. vel /=1  (19)  /=1  182  where T is the torque applied to the haptic actuator (a force-feedback knob). 6,6,  and 6 are the user's hand position, velocity, and acceleration  applied to the actuator. f  are the position dependent functions (e.g., stiffness).  fl  are the velocity dependent functions (e.g., damping).  f  are the acceleration dependent functions (e.g., mass).  pos  ve  acc  N. are the number of position dependent functions. Nj are the number of velocity dependent functions. N  k  are the number of acceleration dependent functions. N,  Nj  N,  * 0 = Z. /pos (06 +. Z. vel/ ,(06 + Z / (06. , . acc i=1  j =1  (20)  / =1  where T is the torque applied to the haptic actuator (a force-feedback knob). t is the time.  5.2.3 Rendering the Prototypes using a Force Feedback Knob Rendering the haptic behaviors designed with our haptic prototyper was accomplished using a smaller version of the more specialized setup using the work described in Chapters 2-4 that uses the custom Real-Time Platform Middleware (RTPM) infrastructure [75]. The haptic knob is shown in Figure 64. It operates with an update rate of 10 kHz, 0.001° positional accuracy, and 180 mNm maximum continuous torque when connected to a realtime Linux PC via an I/O board. These functional specifications enable a wide variety of haptic behaviors, and dynamic simulations, to be rendered effectively.  183  Figure 64: Example haptic knob used with our icon prototyper  5.3 Task Example: Prototyping a Fan Knob The specific example of designing a four-setting fan knob is used to illustrate the rapid prototyping testbed. A fan knob was chosen because it is a simple, easily understood control that suits our needs of communicating usage of the interaction design concepts (other examples might be more likely commercial targets, but are harder to explain prototyping tool concepts). The design challenge is to make the haptic behavior for the knob shown in Figure 65. When rotating the knob, the user should feel "clicks" at the Off, Lo, Med, and Hi setting angles. Part of the challenge is determining the particular damping and inertia parameters that will give the knob the right "feel".  Figure 65: Example fan knob for illustrating icon prototyper interactions  184  The following two sections illustrate how primitive waveforms are edited to define "tiles", and how primitive tiles can be organized and modified into more sophisticated compound tiles that can be used for high-level design. The steps described by walking through the creation of the haptic behavior for the simulated fan knob using our Haptic Icon Prototyper.  5.3.1 Primitive Waveform Editing To generate up the spring, damping, and inertia settings, we sketched the following torque waveforms based on the user's hand position, velocity, and acceleration, respectively. For our fan example (see Figure 65), the user will rotate the knob with their hand along a 1-D rotary path between the Off, Lo, Med, and Hi settings. The underlying representation of position-based Equation 19 is therefore more appropriate than the time-based Equation 20 for building the desired haptic behaviors.  5.3.1.1 Position Tiles Let's assume we first want to create a force ramp clockwise (CW) from the Hi position. Figure 66 shows an interaction sequence for creating a torque ramp. The user starts with a new waveform editor pane (step 1), and drags a few points up (steps 2-3) to define a ramp. The click and drag motion of a point is quite fluid because the underlying representation is a cubic spline interpolation. Typing exact numbers into a text box can specify precise angle and torque parameters associated with the current position on the waveform editor. Next the user selects the finished ramp waveform with a "click and drag" mouse motion (step 4). Then the user drags the selection into the tile palette (refer to Figure 63) and releases it to create a new ramp tile. Continuing this example as illustrated in Figure 67, we can create the detents ("clicks") for the Off, Low Med, and Hi settings. Instead of creating the points from scratch, we first open a collection of tile templates in the tile palette, and drag a tile representing a long series of detents from the tile palette to the waveform editor (step 1). We then select a section containing 4 clicks, and  185  0  30  60  VX'  90  150  190  2 in  2*J  5>—©—e—e—e—©—o o ©• o—©• » o—©•—e—©—©—©—©•—©  0  »  to  «  a  30  60  90  0  33  €0  90  k—«—©—©  o  e  e  o  d o  1»  lfc>  210  10  130  o  o  150  o  240  ^ k l  •  HE  o  Figure 66: Screen captures showing user interaction steps to create a haptic tile of a torque ramp  186  create a new tile by selecting and dragging a portion of the waveform back into the tile palette (step 2). The sine wave selection will create the feeling of 4 clicks because the waveform selection crosses the position axis 4 times on the torque vs. position plot. After creating a new tile, we could create a new container for this tile and subsequent tiles, or we could add the tile to an existing tile hierarchy that had meaning to us. The tabs in our tile palette enable users to create and organize collections of tiles according to their own mental models. Furthermore, the tiles are stored in a text file format that is easily read and written by other software packages such as Matlab [61]. Thus, more complex functionality that is not currently integrated into the icon prototyper, such as bandpass filtering, can be rapidly performed in another package on a particular tile and then reintegrated into the Haptic Icon Prototyper. 30  60  120  90  1S>  180  2111  5  MM) .cons  •s.  il iVWvW  30  60  90  120  ISO  It  210  !  VA  >*'.  VWvWJ  / \  bJ  Figure 67: Screen captures showing user interaction steps to create one version of the basic detent module that will be used for the Off, Lo, Med, and Hi fan settings  187  5.3.1.2 Velocity Tiles Figure 68 shows a standard damping effect across the knob movement. Also, increased damping clockwise (CW) from the Hi fan position, and counterclockwise (CCW) from the Off fan position is illustrated in Step 4 of Figure 70 using two instantiations of a tile with a steeper slope f ^ ve  (refer to  Equation 19). More complex velocity-dependent effects such as Stribeck friction properties (see Figure 69) could be achieved with a curvilinear f ^ ve  and  appropriate event mechanisms. The relationship between higher-order tiles (velocity and acceleration) and the knob position is described in Section 5.3.2, "Organizing Haptic Behavior Primitives," on page 189.  188  5.3.1.3 Acceleration Tiles One acceleration tile similar to Figure 68 can be used to create the effect of a constant mass for the knob. In other words, a constant line is created for face  m  Equation 19.  5.3.2 Organizing Haptic Behavior Primitives Organizing and editing collections of haptic tiles into more sophisticated haptic behaviors is important because prototyping is typically an iterative learning process. The organization of position, velocity, and acceleration tiles with our fan knob example is continued in this subsection. We have introduced haptic tiles as a way to organize primitive tiles and facilitate iterative design towards more complex tiles. Figure 70 illustrates use of the haptic tile pane. To aid portability and re-use, the haptic tiles in the tile palette do not have associated position, velocity, or acceleration properties. Instead, such properties are inherited from the tile pane region on which the tile is placed. The same waveform shape can be used for any spatial or temporal properties including position-, velocity-, acceleration-, or time-based haptic behaviors. In our implementation, we have chosen to color and label different channels for position, velocity, and acceleration regions of the haptic pane. Generally, haptic tiles can be dragged and dropped into the tile pane and then easily re-arranged or removed. Steps 1-5 of Figure 70 illustrate the tile placement for: 1.  A wall behavior for CCW rotation beyond the Opposition.  2. Detents for the Off, Lo, Med, and Hi knob positions. 3. A wall behavior for CW rotation beyond the Hi position. 4. Subtle damping behavior over the range of Off, Lo, Med, and Hi positions; and, stronger damping beyond the Off and Hi positions. 5. A constant simulated mass feeling for the knob.  189  Tiles  Position Channel  !*•!*<*  i•  I  4I»v  r  ~ j 9 /  I  /  /  I  I  t  / / / /  •  0  FT*}  Ve  Velocity I Channel  /  /  t  /' i  r  t  /  1  r  1  1  PCS !  Accel. Channel  VVI  s  j  0  ;  *cc ;  1 »1  1  Figure 70: Screen captures showing steps with the tile pane to organize haptic tiles to build the static and dynamic haptic behaviors for an example fan knob  190  During design, physically feeling and changing the component effects of particular haptic tiles, and particular collections of haptic tiles, is often more important than exploration of the final haptic behavior. To support such exploration, a haptic rendering of any channel or combination of channels in the tile pane might be felt by simply selecting the channels and pressing the play button (see Figure 63). Consequently, the feeling of the walls, detents, damping, and inertia behaviors can all be felt individually - or in any combination. This functionality greatly helps the haptic designer build up appropriate mental mappings between the haptic tile waveforms and their physically rendered behaviors.  5.4 Supporting Iterative Design Our icon prototyper was primarily designed to explore user interface widgets that enable an experienced designer to rapidly iterate between various static and dynamic behaviors to be rendered on a 1 DOF haptic actuator. The primary interest is with tool design choices that facilitate rapid exploration of a variety of haptic parameters. Designers rarely have a clear idea of the perfect, finished haptic behavior before starting the design. They need to be able to create several possible behaviors and be able to rapidly compare these behaviors. Furthermore, once one, or a small number, of promising behaviors have been developed, subtle refinements need to be made. During such a refinement process, designers perform frequent comparisons of many haptic behaviors, and often returns to previously designed behaviors after rejecting a particular design change. When prototyping, users must be able to rapidly realize their haptic behavior intentions. Reducing a 2 second operation to 1 second can be the difference between a usable and an unusable haptic prototyping tool. The following lists some of our design decisions to improve interaction fluidity in the user interface: • Single interaction window. Most interaction functionality is within the waveform editor, tile palette, and tile pane regions. Instead of  191  having multiple regions available via a separate window, they are grouped close to one another in a single window so that "click and drag" motions between all three regions can be performed easily and rapidly. Additional space is obtained with use of tabs (e.g., several tabs can be selected in the tile palette). • Focus on notification over correctness. A  message bar provides the  user with most status and error messages. Dialogs and pop-up messages are usually avoided. This interaction technique reduces distractions to the user's primary task. • Speed and fluidity over precision. The  general form of haptic behav-  iors are quickly obtained using the waveform editor and tile regions. The idea is that the user would use our haptic prototyper to explore the types of waveforms and haptic behaviors that are most appropriate for the current task. Precise fine tuning may be better done afterwards in another tool such as Matlab and/or C++. Nevertheless, the icon prototyper is still an important step in the haptic design process. It is analogous to paper prototyping or storyboarding a graphic user interface design before starting to code. Paper prototyping like our Haptic Icon Prototyper - is more flexible for brainstorming different options; but eventually, a less flexible, more time consuming method is needed to develop the final product because such tools typically offer higher fidelity or better contextualization support. Some specific examples of interaction sequences designed to speed up haptic icon exploration are: Changing amplitude. Figure 71  shows the selection of a waveform seg-  ment (step 1); and, the resulting amplitude increase after dragging any point in the selection (step 2). Various distortions are possible with such amplitude adjustments. For example, should the waveform points be evenly redistrib-  192  uted between the maximum value and the mirror line? Should points be simply translated? Or, should a more complex redistribution function (not supported in our current interface) be used? Additionally, a "mirroring" line placed along the position axis such that points above the line are stretched upwards and points below the line are stretched downwards. Our interface supports horizontal and vertical mirrors. I»  120  60  ISO  ,180  21D  2*>  •I f 30  60  90  120  150  180  210  240  0 ±1 Figure 71: Example amplitude adjustment using a "mirror" line along the position axis in the waveform editor • Changing frequency.  Figure 72 shows a way to decrease the frequency  by directly interacting with a tile. A user selects the right boundary of the tile (step 1), and then drags the boundary leftward to yield a shorter tile of higher frequency (step 2). Similar to the amplitude modification as described above, there are different desirable actions for clicking and dragging tiles. For example, should the waveform in the tile be squeezed linearly, or according to some function? Should the waveform be cut?  193  o  Pos  r Pos  p  /www  r  Figure 72: Example frequency adjustment by dragging the right boundary of a haptic tile leftward in the tile pane • Combining  tiles.  Once a collection of two or more haptic tiles have  been designed to represent a haptic behavior, combining these tiles into a single entity is often desirable. For example, suppose we wish to add a subtle, high frequency clicking feel to our knob motion. We could add a high frequency sine wave to the haptic behavior illustrated in Figure 73. Such an added haptic behavior could be constructed by simply adding another line of haptic tiles as shown by step 1 in Figure 73. We could additionally superimpose the tile for this new subtle clicking behavior to the tile(s) for the existing behavior for the Off, Lo, Med, Hi feeling. Step 2 in Figure 73 illustrates such a superimposition procedure performed in our haptic tile pane. However, for iterative exploration, the user may wish to keep the two active lines illustrated in step 1 such that incremental modifications of either channel will be superimposed into the currently tendered haptic behavior "on the fly".  194  Pos (7  \ \  p  ^ \  Pos N  \ i  \  V  j  1 \ \ 1 1  r  , ,  Pos  r  ji  Pos  r Pos P  J  Figure 73: Example superimposition of two haptic tiles to create a single haptic tile • Changing previous designs. The criteria to design suitable haptic  behavior for a given task is not known a priori. Thus, the ability to cycle between previous states is needed in a rapid prototyping tool. Figure 73 shows one example addition - a subtle clicking feel. Example features in our Haptic Icon Prototyper that support rapid exploration include the ability to select channels to play in the tile pane, drag and drop support of tiles between interface regions, additional tabs for "scratch" collections of tiles in the tile palette, and undo functionality.  5.5 General Design Principles Building on the presented example tool and application, some general design principles for a haptic behavior design tool are listed. Although the current Haptic Icon Prototyper is not able to completely meet this list of demanding criteria, the concepts employed in this example icon prototyping tool are a step towards such a goal.  195  5.5.1 Necessary Attributes of Tools to Support Haptic Behavior Design 5.5.1.1 Scope and General Capabilities The ideal haptic icon prototyping tool would be able to: • Completely represent the psychophysical capabilities of the user via a standard set of mathematical relations. We showed that we could achieve such psychophysical capabilities for a haptic knob in Chapter 3. • Link up to haptic rendering, hardware in a way that provides consistent, high-fidelity renderings of the haptic behavior. • Provide usable interaction techniques for designers to effectively create and modify haptic renderings. • Have easy-to-understand mental mappings between the underlying mathematical representations, the interaction techniques, and the final haptic renderings. • Integrate seamlessly with other haptic development tools, and development tools for other sensory modalities (vision, hearing, smell, and taste).  5.5.1.2 Usability Resulting haptic icon renderings should effectively enable the user to complete his/her desired task. Thus, the designed haptic icon must: • Function technically (be psychophysical^ perceivable, and relate to the user's preconceived mental models of the task.) • Function socially (fit into the task's social and cultural milieu.)  196  5.5.1.3 Representations The basic haptic functional requirements of a complete icon prototyping tool would be to represent and convey haptic: • Type.  Kinaesthetic, tactile, temperature, moisture, or pressure sensa-  tions - including the body site. For example, control of a pressure device on a person's back. • Interactions. Detailed 1 DOF  attributes, and higher-level interactions  between several DOF. For example, how does the force profile of a haptic knob change in response to the position of the knob, the rate which it is moved, and the current state of the system? How are a series of discrete vibrotactile stimuli excited on person's arm to create the feeling of a continuous motion (sensory saltation)? • Psychophysics.  Biological properties of the user's body site(s), and  how these properties relate to technical attributes. For example, is a particular change in pressure level perceived differently when perceived when resting or active? How will a senior citizen perceive the same surface texture compared to a teenager? • Natural Semantics.  What the icon means to the user in terms of the  target task. Will the smooth, heavy feel of a radio tuning knob induce a sense of quality and admiration for the radio? What vibrotactile behavior of a cell phone best conveys the term "call completed" or "friend calling"? What pressure should be applied to a car driver via a haptic seat to suggest s/he turn left to avoid an obstacle?  5.5.2 Accommodating Different Types of Users Different users will want to use a haptic prototyping tool in different ways. Users of haptic icon prototyping tools can be categorized and labelled into three groups:  197  • "Programmers".  Require the richest amount of flexibility and control.  Are willing to learn and use more complicated user interfaces and write programs to create sophisticated and/or novel haptic effects. Example tools include Matlab Simulink [4] and the C++ programming language. • "Designers".  Are technically savvy, but may not have the program-  ming skills - or desire to use them - compared to programmers. The desired technical components of haptic effects are usually less specialized and novel compared to programmers. They require tools to rapidly iterate and carefully tune haptic effects. They will usually focus more attention to higher-level concepts than programmers. Our example Haptic Icon Prototyper fits into this category. • "End users".  Operators of systems containing haptic components  such as cell phones, automobiles, or test controls. They wish to focus on relatively pre-defined, stock customizations that are specialized to their particular device. For example, cell phone users may wish to download and customize vibrotactile ring tones on the web.  5.6 Conclusions and Future Work for Rapid Prototyping We have presented a haptic prototyping tool, containing several novel user interaction techniques, as a contribution towards the larger goal of developing a prototyping resource. Specifically, this Haptic Icon Prototyper is primarily targeted towards rapid, iterative development of single degree-offreedom haptic icon designs. Using the previous prototyping tool examples, high-level design attributes of tools to support haptic behavior design were then summarized. A set of haptic icon prototyping tools should support design based on all haptic types (e.g., kinaesthetic and tactile), many levels-of-  198  detail, interactions between all degrees-of-freedom, psychophysical capabilities of the user(s), and mental models related to the user(s) task(s). Future enhancements to the Haptic Icon Prototyper include integration of visualizations representing the psychophysical limitations and physical device rendering limitations related to the haptic behavior being designed. For example, future HIP visualizations could display just noticeable difference (JND) thresholds based on a typical user's finger tip sensitivity. Rendering device limitations could include visualizations for stability and torque thresholds for particular actuator or sensor implementations. The current HIP does not provide visual feedback of rendering limitations. Thus, one must currently use trial and error iterations to test the stability of a rendering, and possibly change the closed-loop control parameters - which are not yet modifiable in the current HIP graphical user interface. Additional research into psychophysical limitations and mechatronic modeling would aid such visualizations. Functionality to incorporate state changes and multi-degree-of-freedom interactions would also be beneficial. Tighter integration with a more generic tool that supports advanced filtering capabilities, such as Matlab, would be helpful (e.g., have our Haptic Icon Prototyper and a Matlab session share common data structures). Thus, users could start a haptic design session with a haptic icon prototyping tool, and then make more time-consuming, detailed refinements to promising haptic behaviors using Matlab console commands and then reload the icon into the prototyping tool. More explicit shared data structures and handles for integration with other modalities, such as video and audio, would also be helpful for developing tightly coupled multimedia applications. Formal, structured qualitative user studies should be performed to explore and enhance the usability of future icon prototyping tools.  199  Chapter 6 Conclusion We have lived with physical mechanical controls in our residential, commercial, and industrial environments for centuries. As computing infrastructure becomes ubiquitous, with computation embedded seamlessly in our surroundings, mechatronic controls become increasingly useful and desirable interaction tools. Force-feedback controls will replace traditional personal computing interfaces like keyboards and mice when we learn to design them so as to realize the promise of affect and performance benefits for the often complex, specialized tasks of embedded computing. We currently take for granted the high-quality affective design of many mature mechanical controls like knobs, sliders, and buttons, where the aesthetics and appropriateness of the feel have evolved over centuries. But as functional requirements of active controls become more refined, their affect will become an increasingly important contributor to the success or failure of these products. In this thesis, we describe a novel haptic design process — combining affect and performance techniques — which we developed for the purpose of iterative modeling and rendering of dynamic physical controls. Our user studies demonstrated an evaluative methodology for measuring affect and performance responses to haptic stimuli. And, our haptic icon prototyping tool demonstrated rapid prototyping of haptic behaviors. Using a set of real mechanical knobs as a starting point, we illustrated through capture, rendering, and a sequence of user testing how position-, velocity-, and accelerationdependent dynamic properties can be refined to design physical controls that are appropriate to their tasks from both affect and performance perspectives.  200  6.1 Primary contributions Summaries of contributions for each of the major thesis sections are described below. Step 1: Haptic camera — A novel testbed was developed to capture rotary torque responses of physical knobs. Resolutions for torque, position, velocity, and acceleration were 1.8xl0" Nm, 9.8 x IO" rad, 2.QxlQ rad'/s, and 4  6  A  2.8 rad/s , respectively. Compensations for gravity effects were demonstrated using a custom-built rotary accelerometer, and fitting stationary acceleration readings to a sine wave. A novel gripping procedure was defined by making custom ABS plastic encasements for each test knob. These encasements had low mass and high stiffness, and could be attached to and removed from a wide variety of test knob surfaces. Step 2: Characterization — Two simulated data sets that obeyed basic laws of physics, and 5 mechanical test knobs were characterized to a physical model with 8 parameters (1 acceleration, 4 velocity, and 3 position). Characterizations of an ideal dataset produced results with negligible noise; and, characterizations of noisy data sets, containing random noise with magnitudes up to 20% of the signal, were parameterized with errors < 1.7% of the commanded test signal parameters. Position-, velocity-, and acceleration-dependent properties of 5 physical knobs with torque responses < 200 mNm were successfully segmented. Torque parameters for the 4 knobs with "typical" underlying dynamic structures were characterized with mean 95% confidence intervals < 0.3 mNm from the fitted torques. The final "worst case" test knob's parameters were characterized with mean 95% confidence intervals < 4.5 mNm even though the knob was known a priori to have considerable backlash and non-uniform detents that deviated from the fitted model. Step 3: Rendering and validation — Participants were able to keep track of 4 independent dynamic parameters (1 acceleration, 1 velocity, and 2 position) while altering the dynamics of a force-feedback knob. The captured  201  parameters of the 5 mechanical test knobs were found to agree with parameterizations by novice and expert participants, and when available, with "gold standard values". For example, detent period could be determined by counting the number of clicks per revolution. This "gold standard" provides insight into the quality of all the Haptic Camera and expert human parameter estimates. For the two "appropriate model" and one "inappropriate model" characterization tasks, the Haptic Camera algorithm estimated detent period to 3.0%, 9.5%, and 23.5% relative accuracies, respectively, whereas human experts averaged 17.7%, 24.8%, and 7.2% relative accuracies, respectively. The Haptic Camera algorithm treated the period parameter in a similar way to position-, velocity-, and acceleration-based parameters, but humans have increasing difficulty exploring position-, velocity-, and accelerationbased parameters. Even when a completely inappropriate physical model was used, the Haptic Camera was able to identify knob dynamics to acceptable accuracy. These results suggest that the Haptic Camera based on an appropriate physical model (i) performed good quality characterizations below the suggested thresholds suggested by previous research such as Knowles and Sheridan [49], and (ii) significantly outperformed expert human observers. Step 4: Measuring affect —  Significant correlations between self-report  ratings of valence and electromyographic (EMG) biometrics were found for rendered knob dynamics. Furthermore, a lack of correlation between selfreport ratings of valence and skin conductance (SC) biometric measurements for the same rendered knob tasks suggests conformance to the theory that valence and arousal comprise two primary, orthogonal dimensions of affect. Inertia and detent renderings were also shown to provide significant temporal performance improvements over a test knob with negligible inertia and no detents within a tightly controlled pointing task. As a reflection of the complex  202  interdependencies of preference and performance, valence responses sometimes agreed with, and sometimes disagreed with, performance responses. Affective results previously observed for mechanical knobs were also observed with equivalent rendered dynamics. For example, subtle magnitudes of rendered friction, inertia, and detents were observed to be preferred to stronger renderings - as were observed with mechanical dynamics by Knowles and Sheridan [49]. These results suggest that the rendered dynamic parameters were sufficiently close to their physical counterparts to effectively elicit appropriate affective responses. We also observed that classic Fitts Law-type analysis had less of an effect on performance compared to affective characteristics such as knob rendering type. Specifically, compared to different knob renderings, we observed little variation when participants rotated a knob along different target amplitudes or towards different target widths. Thus, the dynamics and statics of the physical control had at least as much to do with task time as the distance moved or target size. Step 5: Haptic icon prototyper — Making haptic behaviors typically  involves tedious and abstract process of coding and replaying. Instead of this approach, we developed a novel tool, called the Haptic Icon Prototyper, for refining and playing haptic behaviors. This prototyper linked to a force-feedback knob and utilized the same mathematical models as the Haptic Camera to provide rapid design and test iterations. Informal evaluation with 15 novice designers over 2 years supported claims of rapid design iteration using this non-model user interface. Encapsulating haptic behaviors into "tiles" was intuitive, efficient, and enhanced save and recall of previous states. Organization of position-, velocity-, and acceleration-based widgets provided clear mental mappings for users to develop detent, friction, and inertia renderings. The key to designing good haptic behaviors is to represent haptic primitives in ways that are easy-to-understand, modify, and combine. Our haptic icon prototyping tool contributes to this goal of better haptic behavior design.  203  6.2 Future Work More immediately, continuation of the work described in this thesis would benefit from efforts in the following areas: • Analysis of a larger collection of real physical knobs in the context of their intended purpose, and comparisons with other physical controls such as sliders, could further improve our understanding of performance- and affect-based trends. For example, do timer knobs typically have higher frequency detents than non-timer knobs? Do heavier kncibs elicit higher valence responses, regardless of function? • Development of more sophisticated non-linear curve fitting mathematics to handle models with more parameters. These could include further enhancements to Gauss-Newton and Levenberg-Marquardt iterative techniques, or new approaches such as refinements of the chaos-based approach suggested by Feo [30]. • Rendering performance could be improved using a more responsive real-time operating system, or a custom embedded solution. • Additional affect studies with the 2-D affect grid, and more participants, could facilitate classification of subtle emotional states such as excited, joyful, fearful, relaxed, ennui, and sad to particular physical control dynamics and contexts. Furthermore, magnetic resonance imaging (MRI) for asymmetrical prefrontal lobe activity of the brain could be used instead of EMG and SC biometrics to relate physical control dynamics to affective states on an absolute reference scale. • Study of more sophisticated, adaptive knob renderings. This thesis focused on rendering knob behaviors that mimicked mechanical knob behaviors, but rendered physical controls are capable of producing more sophisticated behaviors. Active knob renderings can  204  adapt and change based on the user's previous interactions, the current system state, or the surrounding environment. How would such actively changing knob behaviors influence a user's affective and performance responses?  ,  >.  • Iterative refinement of the Haptic Icon Prototyper via longitudinal usability studies would improve the effectiveness of the tool. Additionally, interface tools to visualize and edit physical control dynamics based on psychophysical principles, and device-specific control parameters would increase the utility of the tool. For example, a user could visualize the just-noticeable difference and valence differences between two or more physical control models, or tweak the spring and damper constants for a mass-spring model of inertia, respectively. • Current and future work could be collected and distilled into highlevel design guidelines for representing dynamic haptic properties. More generally, future work should improve our understanding of affect- and performance-based relationships of mechanical and mechatronic physical controls by focussing in three key areas: 1. The value of affective design in its own right, and in relationship to performance-based design, needs to be further explored. 2.  The quality, power consumption, and cost of mechatronic control technologies needs to be further refined before these controls can become as ubiquitous as mechanical physical controls.  3. Additional affect- and performance-based user studies will aid design of more generalizable analyses and intricate relationships between different types of physical controls (e.g., sliders and buttons), haptic sub-modalities (e.g., tactile, temperature, and pres-  205  sure), other sensor modalities (e.g., vision and hearing), usage contexts (e.g., games, manufacturing plants, medical devices, automobiles and household appliances), etc. Addressing these three principles will leverage deployment and adoption of physical controls that are well designed from both performance and affective perspectives.  206  References  [I]  3M, Inc. (2006). Double Coated Tapes. Available at: http://solu-  tions.3m.com. [2]  Abidi, M.A. and Gonzalez, R.C. (1992). Data Fusion in Robotics and Machine Intelligence.  [3]  San Diego: Academic Press.  Adams, E. (2005). The designer's notebook: The act - emotion control with single knob gameplay. Gamasutra, 12 April 2005.  [4]  Adobe, Inc. (2006). Soundtrack Pro. Available at: http:// www.adobe.com.  [5]  Allen, J.B., Coan, J.A., and Nazarian, M . (2004). Issues and assumptions on the road from raw signals to metrics of frontal EEG asymmetry in emotion. Biological Psychology, 183-218.  [6]  Alias Systems Corp..(2005). Maya. Available at: http:// usa.autodesk.com.  [7]  Analog Devices, Inc. (2004). ADXL103/ADXL203 Precision ±1.7g Single/ Dual Axis Accelerometer.  Available at: http://www.analog.com.  [8]  Apple, Inc. (2006). iMovie. Available at: http://www.apple.com.  [9]  Barriera-Viruet, H., Sobeih, T.M., Daraiseh, N . , and Salem, S. (2006). Questionnaires vs. observational and direct measurements: a systematic review. Theoretical Issues in Ergonomics Science, 7(3), 261-284.  [10]  Bartels, R.H., Beatty, J.C., Booth, K.S., Bosch, E.G., and Jolicoeur, P. (1993). Experimental comparison of splines using the shape matching paradigm. Transactions on Graphics, 22(3), 179-208.  [II]  Brouwer, I. (2004). Cost-performance trade-offs in haptic design. M.Sc. Thesis. Dept. of Mechanical Engineering, University of British Columbia.  [12]  Buxton, B., Gaver, B., and Bly, S. (1990). The Use of Non-Speech Audio at the Interface. In Tutorial Notes of Conference on Human Factors in Com-  207  puting Systems (CHI), A C M :  [13]  Cacioppo, J.T. and Tassinary, L.G. (1990). Inferring Psychological Significance From Physiological Signals. American Psychologist, 45(1), 1628.  [14]  Chang, A. and O'Sullivan, C. (2005). Audio-Haptic Feedback in Mobile Phones. In Extended Abstracts of Conference on Human Factors in Computing Systems (CHI), A C M .  [15]  Chapman, D.W. (1932). Relative effects of determinate and indeterminate Aufgaben. American Journal of Psychology, 44,163-174.  [16]  Chase, W.G. and Simon, H.A. (1973). Perception in chess. Cognitive Psychology, 4(1),  [17]  55-81.  Choi, S., and Tan, H.Z. (2004). Toward Realistic Haptic Rendering of Surface Textures. Computer Graphics and Applications, March/ April, 4044.  [18]  Cohen, J. (1973). Eta-squared and partial eta-squared in communication science. Human Communication Research, 28, Oxford Journals, 473-490.  [19]  Colgate, J.E., and Schenkel, G. (1994). Passivity of a Class of SampledData Systems: Application to Haptic Interfaces, In Proc. of American Control Conference.  [20]  Colton, M.B., and Hollerbach, J.M. (2005). Identification of Nonlinear Passive Devices for Haptic Simulations. In Proc. of Haptic Interfaces for Virtual Environments and Teleoperator Systems (HAPTICS),  IEEE, 363 -  368. [21]  Conati, C , Chabbal, R., and Maclaren, H. (2003). A Study on Using Biometric Sensors for Monitoring User Emotions in Educational Games. In Workshop on Assessing and Adapting to User Attitudes and Affect: Why, When, and How ? User Modeling,  [22]  Springer.  Cowan, N . (2001). The Magical Number 4 in Short-term Memory: A Reconsideration of Mental Storage Capacity. Behavioral and Brain Sciences,  [23]  24(1), 87-114.  Cyberware, Inc. (1999). Model Shop Color 3D Scanner. Available at:  208  http://www.cyberware.com. [24]  Daniel, T.C. (1990). Measuring the Quality of the Natural Environments, American Psychologist, 45, 633-637.  [25]  Desmet, P.M.A. (2003). Measuring emotion: development and application of an instrument to measure emotional responses to products. In M.A. Blythe, K. Overbeeke, A.F. Monk, and P.C. Wright (Eds.), Funology.from usability to enjoyment (Vol. 3),  [26]  Dordrecht: Kluwer, 111-123.  Duchenne de Boulogne, C.-B. (1862). The Mechanism of Human Facial Expression,  Paris: Jules Renard. (edited and translated by R. Andrew  Cuthbertson (1990). Cambridge: Cambridge Univ. Press). [27]  Ekman, P. and Friesen, W. (1978). Manual for the Facial Action Coding System (FACS),  [28]  Palo Alto: Consulting Psycologists Press.  Enriquez, M.J. (2003). The Hapticon Editor: A tool in Support of Haptic Communication Research. In Proc. of Haptic Interfaces for Virtual Environments and Teleoperator Systems (HAPTICS),  [29]  Ericsson, K. and Simon, H. (1993). Protocol Analysis: Verbal Reports as Data, 2  nd  [30]  IEEE, 356 - 362.  Edition.  Boston: MIT Press.  Feo, O.D. (2003). Self-emergence of chaos in the identification of irregular periodic behavior. Chaos: An Interdisciplinary Journal of Nonlinear Science,  [31]  13(4), 1205 -1215.  Fitts, P.M. (1954). The information capacity of the human motor system in controlling the amplitude Q { movement. Journal of Experimental Psychology, 47(6), APA Press, 381-391.  [32]  Franklin, G.F., Powell, J.D., and Emami-Naeini, A. (1993). Feedback Control of Dynamic Systems, 3rd Edition,  [33]  Menlo Park: Addison-Wesley.  Gaver, W. W. (1986). Auditory icons: Using sound in computer interfaces. Human-Computer Interaction, 2(2), 167 -177.  [34]  Gill, P.R., Murray, W., and Wright, M.H. (1981). Practical Optimization. Academic Press, 136 -137.  [35]  Hasser, C.J. and Cutkosky, (2002). M.R. System identification of the human hand grasping a haptic knob. In Symposium on Haptic Interfaces  209  for Virtual Environment and Teleoperator Systems (HAPTICS), IEEE Press.  Honeywell Sensotec, Inc. (2004). QWFK-8M Miniature Reaction Torque Transducers. Available at: http://www.sensotec.com. Houk, J.C., Barto, A G . , and Fagg, A-H., (2000). Fractional power damping model of joint motion. In M.L. Latash (Ed.), Progress in motor control, Volume 2 —Structure-Function Relationships in Voluntary Movements: Human Kinetics. Howland, D. and Noble, M.E. (1953). The effect of physical constants of a control on tracking performance. Journal of Experimental Psychology, 46, APA Press, 353-360. Immersion Corporation. (2004). TouchSense Programmable Rotary Modules. Available at: http://www.immersion.com/industrial/docs/  TSRotary_Oct04_v3_LR.pdf Immersion Corporation. (2006). PR-1000 Rotary Controller with Braking Actuator. Available at: http://www.immersion.com/industrial/  rotary/ products Ittelson, W.H. (1973). Environmental perception and contemporary perceptual theory. In W.H. Ittelson (Ed.), Environment and cognition. New York: Seminar Press. Jordan, P. (2000). Designing Pleasurable Products: An Introduction to the New Human Factors, CRC Press. Kahneman, D. and Tversky, A. (1979). Prospect theory: An analysis of decision under risk. Econometrica, 47, 263-291. Karnopp, D. (1985). Computer simulation of stick-slip friction in mechanical dynamic systems. Journal of Dynamic Systems, Measurement, and Control, 107, ASME, 100 -103.  Kemp, A.H., Gray, M.A., Eide, P., Silberstein, R.B., and Nathan, P.J. (2002). Steady-State Visually Evoked Potential Topography during Processing of Emotional Valence in Healthy Subjects. Neurolmage, 17,16841692. Khalid, H.M., and Helander, M G . (2004). A framework for affective  210  customer needs inproduct design. Theoretical Issues in Ergonomics Science, 5(1),  27-42.  Killgore, W.D. (1998). The Affect Grid: a moderately valid, nonspecific measure of pleasure and arousal. Psychological Reports, 83(2), 639-642. Kobayashi, S. (1992). Color Image Scale. Kobahashi International Press. Knowles, W.B. and Sheridan, T.B. (1966). The "Feel" of Rotary Controls: Friction and Inertia. Human Factors, 8, 209-215. Kuchenbecker, K.J., Fiene, J., and Niemeyer, G. (2006). Improving Contact Realism Through Event-Based Haptic Feedback. Transactions on Visualization and Computer Graphics, 12(2), IEEE,  219-230:  Lang, PJ., Greenwald, M.K., Bradley, M.M., and Hamm, A.O. (1993). Looking at Pictures: Affective, facial, visceral and behavioral reactions. Psychophysiology, 30(3), 261-273. Lang, P.J. (1995). The Emotion Probe. American Psychologist, 50(5), 372- . 385. Lee, G. and Hannaford, B. (2003). Anisotropics of Touch in Haptic Icon Exploration. In Proc. of International Conference on Intelligent Robots and Systems, IEEE/RSJ.  Lindgaard, G., and Whitfield, T.W.A. (2004). Integrating aesthetics within an evolutionary and psychological framework. Theoretical Issues in Ergonomics Science, 5(1),  73-90.  Ljimg, L. (2005). System Identification Toolbox User's Guide (5.02 ed.), Mathworks, Inc. MacLean, K.M. (1996). The Haptic Camera: A Technique for Characterizing and Playing Back Haptic Properties of Real Environments. In Proc. of Haptic Interfaces for Virtual Environments and Teleoperator Systems (HAPTICS),  ASME/IMECE, 459 - 467.  MacLean, K.M. (2000). Designing with Haptic Feedback. In Proc. of Robotics and Automation  (ICRA),  IEEE.  MacLean, K.M, and Enriquez, M.J. (2003). Perceptual Design of Haptic Icons. In Proc. ofEurohaptics, 351 - 363.  211  MacLean, K.E. and Roderick, J.B. (1999). Smart Tangible Displays in the Everyday World: a Haptic Door Knob, In Proc. ofHAPTlCS,  IEEE/  ASME Press. Mandryk, R.L., and Inkpen, K.M. (2004). Physiological Indicators for the Evaluation of Co-located Collaborative Play. In Proc. of Conference on Computer Supported Collaborative Work (CSCW), A C M .  Mathworks, Inc. (2004). Matlab and Simulink. Available at: http:// www.mathworks.com. Mathworks, Inc. (2005, July 19). Solution Number: 1-18E03. Available at: http://www.mathworks.com/support/solutions/data/l18E03.html?product=OP&solution=l-18E03. Messner, W.C, and Tilbury, D.M. (1998). Control Tutorials for Matlab and Simulink: A Web-Based Approach., Addison-Wesley. Available at: http://  www.engin.umich.edu / group / ctm/. MicroE Systems, Inc. (2004). Mercury 2000 Smart Encoder System: 19.05 mm Rotary Grating.  Available at: http://www.microesys.com.  Microsoft, Inc. (2006). Visio. Available at: http://www.microsoft.com. Miller, B.E. and Colgate, J.E. (1998). Using a Wavelet Network to Characterize Real Environments for Haptic Display. In Proc. of Dynamic Systems, and Control Division, DSC 58,  ASME, 257 - 264.  Miller, B.E., Colgate, J.E., and Freeman, R.A. (2000). Guaranteed Stability of Haptic Systems with Nonlinear Virtual Environments. Transactions on Robotics and Automation,  16(6), IEEE,  712-719.  Miller, G.A. (1956). The Magic Number Seven, Plus or Minus Two: Some Limits on our Capacity for Processing Information. Psychological Review, 63(2), 81-97. •'  Morgan, C.T., Cook, J.C., Chapanis, A., and Lund, M.W. (1963). Human Engineering Guide to Equipment Design.  Norman, D.A. (2004). Emotional Design: Why we love (or hate) everyday things.  New York: Basic Books.  Novak, K.E., Miller, L.E., and Houk, J.C. (2000). Kinematic properties of  212  rapid hand movements in a knob turning task. Experimental Brain Research, 232(4), Springer, 419-433.  [72]  Nunnelly, J.C. (1978). Psychometric theory, 2nd Ed., New York: McGrawHill.  [73]  Osgood, C.E., Suci, G.J., and Tannenbaum, P.H. (1957). The measurement of meaning. Chicago: University of Illinois Press.  [74]  O'Sullivan, C. and Chang, A. (2005). Dimensional Design; Explorations of the Auditory and Haptic Correlate for the Mobile Device, In Proc. of International Conference on Auditory Display, 212-217.  [75]  Pava, G. and MacLean, K.E. (2004). Real-Time Platform Middleware for Transparent Prototyping of Haptic Applications. In Proc. of Haptic Interfaces for Virtual Environments and Teleoperator Systems (HAPTICS), IEEE.  [76]  Picard, R. (1995). Affective computing, Cambridge: MIT Press.  [77]  Richard, C. (2000). On The Identification and Haptic Display of Friction. Ph.D. Thesis. Dept. of Mechanical Engineering, Stanford University.  [78]  Richard, C. (1999). Friction Identification and Haptic Display of Friction. In Proc. of Haptic Interfaces for Virtual Environments and Teleoperator Systems (HAPTICS), ASME/IMECE.  [79]  Russell, J.A., Weiss, A., and Mendelsohn, G.A. (1989). Affect Grid: A Single-Item Scale of Pleasure and Arousal. Journal of Personality and Social Psychology, 57(3), 493-502.  [80]  Schwartz, G.E., Fair, P.L., Salt, P., Mandel, M.R., and Klerman, G.L. (1976). Facial muscle patterning to affective imagery in depressed and nondepressed subjects. Science, 192,489-491.  [81]  Schwartz, M.W., Cowan, W.B., and Beatty, J.C. (1987). A n experimental comparison of RGB, YIQ, LAB, HSV, and opponent color models. Transactions on Graphics, 6(2), 123-158.  [82]  Snibbe, S.S., MacLean, K.E., Shaw, R., Roderick, J.B., Verplank, W., and Scheeff, M . (2001). Haptic Metaphors for Digital Media. In Proc. of'Symposium on User Interface Software and Technology (UIST), A C M .  [83]  Snowden, R.J. (1992). Sensitivity to relative and absolute, motion. Per-  213  ception, 21(5), 563-568.  [84]  Spence, C. (2003). Crossmodal Attention and Multisensory Integration: Implications for Multimodal Interface Design. In Proc. of International Conference on Multimodal Interfaces.  [85]  Stevens, S.S. (1957). On the psychophysical law. Psychophysical Review, 64(3), 153-181.  [86]  Stratasys, Inc. (2005). FDM Technology. Available at: http:// www.stratasys.com.  [87]  Stribeck, R. (1902). Die wesentlichen Eigenschaften der Gleitund Rollenlarger [The key qualities of sliding and roller bearings]. Z. Vereines Deutsch Ingen  [88]  46(38,39), 1342-1348,1432-1437.  Surakka, V. and Hietanen, J.K. (1998). Facial and emotional reactions to Duchenne and non-Duchenne smiles. International Journal of Psychophysiology, 29(1),  [89]  22-33.  Swindells, C. and MacLean, K.E. (2007). Capturing the Dynamics of Mechanical Knobs. In World Hap tics Conference (WHC), IEEE Press.  [90]  Swindells, C , MacLean, K.E Booth, K.S., and Meitner, M.J. (2006). V  Evaluating Affect in Interaction: A Case-Study Using Haptic Feedback. Graphics Interface.  [91]  Swindells, C , MacLean, K.E., Booth, K.S., and Meitner, M.J. (2007). Exploring Affective Design for Physical Controls. In Conference on Human Factors in Computing Systems (CHI), CHI Letters 9(1),  [92]  A C M Press.  Swindells, C , Maksakov, E., MacLean, K.E., and Chung, V. (2006). The Role of Prototyping Tools for Haptic Behavior Design. In Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (HAPTICS),  [93]  IEEE.  Swindells, C , Smith, J.D., and MacLean, K.E. (2005). A n Exploration of Representations to Aid Design of Haptic Behaviors. Hands-on Haptics: Exploring Non-Visual Visualization Using the Sense of Touch, In Conference on Human Factors in Computing Systems (CHI), A C M , 5-8.  [94]  Swindells, C , Unden, A., and Sang, T. (2003). TorqueBAR: A n  214  Ungrounded Haptic Feedback Device, In International Conference on Multimodal Interaction (ICMI), A C M .  [95]  Tellegen, A., Watson, D., and Clark, L.A. (1999). On the dimension and hierarchical structure of affect, Psychological Science, 10(4).  [96]  Tesser, A., and Martin, L. (1996). The psychology of evaluation. In E.T. Higgins and A.W. Kruglanski (Eds.), Social Psychology: Handbook of Basic Principles. New York: Guilford Press, 400-432.  [97]  Turvey, M.T. (1996). Dynamic Touch, American Psychologist, 52(11), 1134-1152.  [98]  Ulrich, R.S. (1984). View Through a Window May Influence Recovery from Surgery, Science, 224, 420-421.  [99]  Ulrich, R.S. (1986). Effects of Hospital Responses to Vegetation and Landscapes, Landscape and Urban Planning, 13, 29-44.  [100] Valspar, Inc. (2006). Goof-off solvent. Available at: http://www.valspar.com/ val/ resident/ goof-off .jsp. [101] Weir, D.W., Peshkin, M., Colgate, J.E., Buttolo, P., Rankin, J., and Johnston, M . (2004). The Haptic Profile: Capturing the Feel of Switches. In Proc. of Haptic Interfaces for Virtual Environments and Teleoperator Systems (HAPTICS), ASME/IMECE.  [102] Weiser, M . (1991). The computer for the 21st century, Scientific American, 265(3), 66-75.  [103] Welford, A. T. (1968). Fundamentals of skill. London: Methuen. [104] Winton, W., Putnam, L, and Kraus R. Facial and autonomic manifestations of the dimensional structure of emotion. (1984). Experimental Social Psychology, 20,195-216.  [105] Woodruff, B. and Helson, H . (1967). Torque Sensitivity as a Function of Knob Radius and Load, 80(4), Univ. of Illinois Press, 558-571.  [106] Wundt, W. (1907). Outlines of psychology. Leipzig: Wilhelm Engelmann. [107] Wyszecki, G., and Stiles, W. (1982). Color Science: Concepts and Methods, Quantitative Data and Formulae, 2nd Edition. New York: Wiley.  [108] Yang, D. and Lee, W. (2004). Disambiguating music emotion using soft-  215  )  ware agents. In Proc. of International Conference on Music Information Retrieval.  [109] Yang, S., Tan, H.Z., Buttolo, P., Johnston, M., and Pizlo, Z. (2003). Thresholds for Dynamic Changes in a Rotary Switch. In Proc. of EuroHap tics.  [110] Zajonc, R.B. (1980). Feeling and Thinking: Preferences Need No Inferences. American Psychologist, 35(2), 151-175.  216  Appendix A Ethics F o r m s This research was approved by the Behavioural Research Ethics Board at The University of British Columbia under certificate number B01-0470. Dr. Karon MacLean acted as the faculty advisor for the group of user studies involving human participants titled: Physical and multimodal user interfaces usability & psychophysics.  Copies of the Certificate of Approval, Behavioural Ammentment Form and Participant Consent Form are included in this Appendix.  217  1.2 Amendment Form for an Approved Project The following two pages contain a copy of the approved Behavioural Amendment Form.  UBC  Date Received  Behavioural Research Ethics Board  m  O f f i c e o f Research Services S u i l e 102, 6 1 9 0 A g r o n o m y R o a d , V a n c o u v e r , B.C. V 6 T 1Z3 P h o n e : (604) 827-5112, F a x : (604) 822-5093  Request for Annual Review or Amendment of an Approved Project Items 1, 2, 3 and 4 must be completed to identify the file being amended or renewed. 1. Principal Investigator or Faculty Advisor (as recorded on the previously approved Certificate) Surname: MacLean Given Name(s): Karon UBC Faculty / Department: Science / Computer Science Telephone number: 604-822-8169 2. Title of the Project: (as recorded on the previously approved Certificate) Physical and multimodal user interfaces - usability & psychophysics 3. Certificate of Approval number: B01-0470 Instructions:  • H O • •  4. Funding Agency or Company: PRECARN / Institute for Robotics and Intelligent Systems (IRIS)  (Check all items which apply)  Refer to the Guidance Notes for more information.  Annual Review: Since January 2000 Certificates of Approval for initial approval and annual renewals have been issued for a period of one year only. Amendments: Amendments are minor changes to an ongoing study. Approval of an amendment will not change the anniversary date for annual review. Full Board Review: If the amendment involves new procedures that do not meet the 'Minimal Risk' criteria (see Guidance Note), the amendment request and a copy of the original application must be prepared for full board review and received prior to the meeting deadline. Investigators: The name of the Principal Investigator on the Certificate of Approval can only be changed at the original Principal Investigator's request. Otherwise, a new Application for Ethical Review must be submitted. Grants: In all cases there must be an exact title and funding agency or company match between the grant application or funded project and the Certificate of Approval. Use this form to add a new title or agency to the record.  5. Changes to the Study Design: Check any areas that have changed and describe in more detail in item 13. H • • • • D Q • • D D  Investigators Sponsor Title Research method Subjects Recruitment method Procedures Study Location Time required of the subject Reimbursement Access to data  •  There have been no changes to the study design.  7. Principal Investigator or Faculty Advisor: / agree to abide by the Tri-Council Policy for Ethical Conduct for Research Involving Human Subjects.  Signature  Date  6. Changes or additions to study documents: Check any that has been revised and attach a copy with the changes highlighted in bold type or underlined. Include a new version date on all revised documents. • D • [3 • • K • •  Recruitment letter Advertisement Poster Consent form Covering letter (consent) for questionnaires Tests Interviews Questionnaires Other  •  There have been no changes to the study documents.  8. Contact Person: (This section must be updated. Correspondence & Certificates will only be sent to this person) Name: Karon MacLean Title: Assistant Professor, Computer Science Complete Address: 201-2366 Main Mall Vancouver. BC V6T 124 Telephone number: 604-822-3169 Fax Number: 604-822-5485  219  Amendments: (Complete items 9, 10, 11, or 12 only if different from items 1, 2, 3, or 4) 9. New Principal Investigator or Faculty Advisor: 10. New Co-Investigator or Student: Surname: (If more than one use item 13 or an additional page) Given Name(s): Surname: SWINDELLS Academic Rank: Given Name(s): Colin Edward UBC Faculty / Department: Academic Rank: Ph.D. Candidate UBC Faculty / Department: Science / CPSC  11. New Title:  12. New Funding Agency or Company: Grant application number (if available): 13. Describe any changes to the study: A student co-investigator has been added for these experiments (Colin Swindells). The studies will continue to use the same type of tasks and multimodal displays as proposed in the original ethics approval request. The procedure includes a brief interview at the end of the experiment; this was accepted in a previously filed amendment. The questions asked at the end of the experiment have been changed. A s before, the interview will be tape recorded (audio only). A l s o , the consent form has been updated to reflect the new student co-investigator.  Annual Renewal: 14. Number of Subjects Admitted to the Study to date: 15. Progress report: Provide a brief summary of the progress of the study. (Note the Tri Council Policy Statement indicates in Article 1.13 that, "Continuing review should consist of at least the submission of a succinct annual status report to the REB. The REB shall be promptly notified when the project concludes.")  Form Revised: January 20, 2004  This form must be typewritten  Page 2/2  Check the web page to ensure that you have the latest version of the form. (http://www.ors.ubc.ca/ethics/behavioural/b-forms.htm)  220  1.3 Consent Form The  following three pages contain a copy of the consent form signed by  user study participants.  UBC THE  U N I V E R S I T Y  OF BRITISH  C O L U M B I A  Department of Computer Science  PARTICIPANT'S C O P Y CONSENT FORM  2366 Main Mall Vancouver, B.C.  Canada V6T IZ4  tel: (604)822-9289 tax: (604) 822-5485  Project Title: Physical and Multimodal User Interfaces - Usability and Psychophysics Principal lnv«tlgator:K. M a c L e a n , tel. • Co-lnvestigator:Colin Swindells, tel. Computer Science  Ph.D.  Candidate, Department of  T h e p u r p o s e o f this s t u d y is to e x a m i n e e m o t i o n a l r e s p o n s e s to the m o v e m e n t b e h a v i o u r o f i n s t r u m e n t controls such as knobs. T h e task y o u w i l l p e r f o r m h a s b e e n p r o g r a m m e d o n ac o m p u t e r . Y o u w i l l b e a s k e d to r e s p o n d to e a c h s u c c e s s i v e task b y interacting w i t h ahaptic display. Y o u m a y b e a s k e d to w e a r h e a d p h o n e s for the delivery o f a u d i t o r y input. Y o u w i l l a l s o b e a s k e d to w e a r external (i.e., n o n - i n v a s i v e ) s e n s o r s that c o l l e c t s o m e b a s i c p h y s i o l o g i c a l i n f o r m a t i o n s u c h as the heart rate, respiration rate, s o m e m u s c l e activity, a n d p e r s p i r a t i o n . P l e a s e tell the e x p e r i m e n t e r if y o u find the a u d i t o r y s t i m u l u s level a n d / o r s e n s o r p o s i t i o n i n g u n c o m f o r t a b l e , a n d adjusts will b e m a d e . Y o u will be a s k e d to a n s w e r q u e s t i o n s in aquestionnaire a n d an interview as part o f the experiment. T h e interview will b e a u d i o taped for later analysis. Y o u w i l l r e c e i v e p r a c t i c e w i t h s p e c i f i c i n s t r u c t i o n s f o r the t a s k b e f o r e y o u b e g i n . If y o u are n o t s u r e about a n y instructions, o r w i s h to h a v e m o r e practice, d o not hesitate to ask. TIME COMMITMENT:  1 hour session  HONORARIUM:  You will receive $10 for participating in this experiment. Your identity will be confidential: you will not be identified by name in any study reports. Experimental results will be stored in a secure Computer Science account accessible only to the experimenters. The anonymous datafrom the experiment will be used in scholarly publication's).  CONFIDENTIALITY:  Y o u understand that the experimenter will A N S W E R A N Y Q U E S T I O N S y o u h a v e about the instructions o r the p r o c e d u r e s o f this study. After participating, the e x p e r i m e n t e r will a n s w e r a n y questions y o u h a v e about this study. Y o u u n d e r s t a n d t h a t y o u h a v e t h e R I G H T T O R E F U S E to p a r t i c i p a t e o r to w i t h d r a w f r o m t h e s t u d y at any time without penalty of any form. Y o u h e r e b y C O N S E N T to participate in this s t u d y a n d a c k n o w l e d g e R E C E I P T o f a c o p y o f the consent form. If y o u h a v e a n y c o n c e r n s r e g a r d i n g y o u r t r e a t m e n t as ar e s e a r c h subject y o u m a y c o n t a c t the R e s e a r c h S u b j e c t I n f o r m a t i o n L i n e in the U B C O f f i c e o f R e s e a r c h S e r v i c e s at 6 0 4 - 8 2 2 - 8 5 9 8 .  Page J of 3  revised 7/27/2004  221  UBC THE  UNIVERSITY  OF  RESEARCHER'S COPY CONSENT FORM  BRITISH  C O L U M B I A  Department of Computer Science 2366 Main Mall Vancouver, B.C. Canada V 6 T tel: (604) 8 2 2 - 9 2 8 9 tax: (604) 8 2 2 - 5 4 8 5 "  1Z4  Project Title: Physical and Multimodal User Interfaces - Usability and Psychophysics Principal InvestlgatorK. M a c L e a n , tel. Co-lnvestlgator:Colin Swindells, tel. > _ Computer Science  Ph.D. Candidate, Department of  T h e p u r p o s e o f this s t u d y is to e x a m i n e e m o t i o n a l r e s p o n s e s to the m o v e m e n t b e h a v i o u r o f controls such as knobs.  instrument  T h e task y o u will p e r f o r m has b e e n p r o g r a m m e d o n ac o m p u t e r . Y o u will b e a s k e d to r e s p o n d to e a c h s u c c e s s i v e t a s k b y i n t e r a c t i n g w i t h ah a p t i c d i s p l a y . Y o u m a y b e a s k e d to w e a r h e a d p h o n e s f o r the d e l i v e r y o f a u d i t o r y input. Y o u w i l l a l s o b e a s k e d to w e a r external (i.e., n o n - i n v a s i v e ) s e n s o r s that c o l l e c t s o m e basic p h y s i o l o g i c a l i n f o r m a t i o n s u c h as the heart rate, respiration rate, s o m e m u s c l e activity, a n d p e r s p i r a t i o n . P l e a s e t e l l t h e e x p e r i m e n t e r i f y o u find t h e a u d i t o r y s t i m u l u s l e v e l a n d / o r s e n s o r p o s i t i o n i n g u n c o m f o r t a b l e , a n d adjusts will b e m a d e . Y o u will b e a s k e d to a n s w e r questions in aquestionnaire a n d an interview as part o f the experiment. T h e interview will b e a u d i o taped for later analysis. Y o u w i l l r e c e i v e practice w i t h s p e c i f i c instructions for the task b e f o r e y o u b e g i n . If y o u are not sure about a n y instructions, o r w i s h to h a v e m o r e practice, d o not hesitate to ask. TIME COMMITMENT:  1 hour session  HONORARIUM: CONFIDENTIALITY:  You will receive $10 for participating in this experiment. Your identity will be confidential: yon will'notbe identified by name in any study reports. Experimental results ii ill he stored in a secure Computer Science account accessible only fo the experimenters. The anonymous datafrom the experiment will be used in scho/ar/y publication (sj.  Y o u understand that the experimenter will A N S W E R A N Y Q U E S T I O N S y o u have about the instructions or the p r o c e d u r e s o f this study. A f t e r participating, the experimenter will a n s w e r a n y questions y o u h a v e a b o u t this study. Y o u u n d e r s t a n d that y o u h a v e the R I G H T T O R E F U S E to participate o r to w i t h d r a w any time without penalty of any form.  from  t h e s t u d y at  Y o u h e r e b y C O N S E N T to participate in this s t u d y a n d a c k n o w l e d g e R E C E I P T o f a c o p y o f the consent form. If y o u h a v e a n y c o n c e r n s r e g a r d i n g y o u r t r e a t m e n t a s ar e s e a r c h s u b j e c t y o u m a y c o n t a c t the R e s e a r c h S u b j e c t I n f o r m a t i o n L i n e i n the U B C O f f i c e o f R e s e a r c h S e r v i c e s at 6 0 4 - 8 2 2 - 8 5 9 8 .  Page 2 of 3  revised 7/27/2004  THE  UNIVERSITY  OF BRITISH  COLUMBIA  You hereby CONSENT to participate in this study and acknowledge RECEIPT of a copy of the consent form: N A M E (please print)  SIGNATURE  D A T E  If you have any concerns regarding your treatment as a research subject you may contact the Research Subject Information Line in the UBC Office of Research Services at 604-822-8598.  Page 3 o/3  revised7/27/2004  Appendix B Defense Programme This Appendix contains a copy of the four page programme for the final oral defense that was held on 7 February 2007 at the University of British Columbia.  224  Faculty of Graduate Studies  THE  U N I V E R S I T Y OF BRITISH  COLUMBIA  PROGRAMME  The Final Oral Examination For the Degree of DOCTOR OF PHILOSOPHY (Computer Science) COLIN EDWARD SWINDELLS B.A.Sc. (Hons.), Simon Fraser University, 2000 M.Sc, Simon Fraser University, 2002 Wednesday, February 7, 2007, 9:00 am Room 200, Graduate Student Centre " I n c o r p o r a t i n g A f f e c t i n t o t h e D e s i g n o f 1-D R o t a r y Physical Controls" EXAMINING  COMMITTEE  Chair: Dr. Robert Rohling (Electrical and Computer Engineering/Mechanical Engineering) Supervisory Committee: Dr. Karon MacLean, Research Supervisor (Computer Science) Dr. Kellogg Booth, Research Supervisor (Computer Science) Dr. Joanna McGrenere (Computer Science) University Examiners: Dr. Dinesh Pai (Computer Science) Dr. Anthony Hodgson (Mechanical Engineering) External. Examiner: Dr. Edward Colgate Department of Mechanical Engineering Northwestern University Evanston, IL USA  225  ABSTRACT  The visceral emotional reactions that users have to technologies are increasingly understood to be important in terms of safety, performance, and pleasure in its own right. This thesis systematically explores users' emotional (affect) reactions to everyday physical manual controls, in order to inform a design process that considers appropriate affect response as well as performance relationships. Design of both mechanical and emerging mechatronic physical controls are addressed. This novel design process includes parameterizing second order (inertial) dynamics using a system identification technique, and rendering models on a custom force-feedback knob. Next, this thesis explores biometric and self-reported measures of the affective responses elicited by these dynamics, and an iterative prototyping tool for rapid refinement of the "feel" of physical controls. This research impacts use of the passive physical interfaces such as mechanical knobs and sliders that are already ubiquitous in our everyday environments, as well as the active physical controls that are emerging in embedded computing environments such as cars, games, and medical devices.  BIOGRAPHICAL NOTES Academic Studies:  B.A.Sc. (Hons.), S i m o n Fraser University, 2000 M . S c , S i m o n Fraser University, 2 0 0 2  GRADUATE STUDIES Field of Study:  Human-Computer  Interaction  Department of Computer  Science  Courses ( 5 0 0 l e v e l a n d a b o v e )  Instructors  C P S C 520  Dr. C. Greif  N u m e r i c a l Solution o f Differential Equations  D r . K. D e  Voider  Practices C P S C 554  P h y s i c a l Interface D e s i g n a n d  D r . K. M a c L e a n  Evaluation EECE 596  H u m a n Interface  EPSE 592  Experimental Designs a n d Educational  FRST 525C  Technologies  Dr. S. F e l s Dr. B. Z u m b o  Research  Environmental  Perception  Dr. M. Meitner  SELECTED AWARDS Precarn Scholar 2004 Precarn Scholar 2005  SELECTED PUBLICATIONS Swindells, C , M a c L e a n , K.E., Booth, K.S.,Meitner, M. (2007)  Exploring  Affective  Proceedings of Conference on Human Factors in Computing Systems (CHI), CHI Letters, 9(1), A C M P r e s s .  D e s i g n f o r P h y s i c a l C o n t r o l s . In  Swindells, C . a n d MacLean, K.E. (2007) K n o b s . In  Capturing the Dynamics of Mechanical  Proceedings of World Haptics Conference (WHC),  Swindells, C , M a c L e a n , K.E., Booth, K.S.,Meitner, M . (2006)  IEEE  A Case-Study of  A f f e c t M e a s u r e m e n t T o o l s f o r P h y s i c a l U s e r I n t e r f a c e D e s i g n . In  Graphics Interface(GI),  Press.  Proceedings of  Canadian Human-Computer Communications  Society.  S w i n d e l l s , C , M a k s a k o v , E., M a c L e a n , K . E . , a n d C h u n g , V . ( 2 0 0 6 ) T h e R o l e o f P r o t o t y p i n g T o o l s f o r H a p t i c B e h a v i o r D e s i g n . In Proceedings of the Symposium on Haptic Interfaces for Virtual Environments and Teleoperator Systems (HAPTICS), IEEE  Press.  S w i n d e l l s , C , P o , B . A . , H a j s h i r m o h a m m a d i , I., C o r r i e , B . , D i l l , J . C , F i s h e r , B . D . , a n d Booth, K.S.(2004) Comparing C A V E , Wall, a n d Desktop Displays for Navigation a n d . W a y f i n d i n g i n C o m p l e x 3 D M o d e l s . In  International (CGI),  IEEE  Press.  Proceedings of Computer Graphics  Swindells, C , Unden, A., Sang, T. (2003) TorqueBAR: An Ungrounded Haptic Feedback Device. In Proceedings of International Conference on Multimodal Interfaces (ICMI), A C M Press. Tory, M. and Swindells, C . (2003) Comparing ExoVis, Orientation Icon, and In-Place 3D Visualization Techniques. In Proceedings of Graphics Interface, Canadian Human-Computer Communications Society. Swindells, C . (2002) Use that there! Pointing to determine device identity. M.Sc. Thesis, Computing Science, Simon Fraser University. Swindells, C , Inkpen, K.M., Dill, J.C., and Tory, M. (2002) Use that there! Pointing to establish device identity. In Proceedings of User Interface Software and Technology (UIST).CHI Letters 4(2), A C M Press. Swindells, C. (2000) A Diagnostic Test Suite for the Virtual Hand Laboratory. B.A.Sc. Thesis, Engineering Science, Simon Fraser University. Swindells, C, Dill, J.C., and Booth, K.S. (2000) System Lag Tests for Augmented and Virtual Environments. In Proceedings of User Interface Software and Technology (UIST), CHI Letters 2(2), A C M Press. SELECTED PRESENTATIONS Swindells, C . (2006) Material Aspects of the eBook. In Contexts, Pragmatics, and Theory ofE-Books, Advanced Consultation, Digital Humanities Summer Institute, Department of English, University of Victoria. Swindells, C , Smith, J.D., and MacLean, K.E. (2005) A n Exploration of Representations to Aid Design of Haptic Behaviours. In Proceedings of Conference on Human Factors in Computing Systems (CHI) Workshop -- Hands-on Hapticsi Exploring Non-Visual Visualization Using the Sense of Touch, A C M Press. SUPERVISORY COMMITTEE Dr. Karon MacLean Dr. Kellogg Booth Dr. Joanna McGrenere Dr. Michael Meitner  

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
http://iiif.library.ubc.ca/presentation/dsp.831.1-0052050/manifest

Comment

Related Items