UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Interaction with large stereoscopic displays : Fitts and multiple object tracking studies for virtual… Rajendran, Vasanth Kumar 2012

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata

Download

Media
24-ubc_2013_spring_rajendran_vasanth.pdf [ 9MB ]
Metadata
JSON: 24-1.0052210.json
JSON-LD: 24-1.0052210-ld.json
RDF/XML (Pretty): 24-1.0052210-rdf.xml
RDF/JSON: 24-1.0052210-rdf.json
Turtle: 24-1.0052210-turtle.txt
N-Triples: 24-1.0052210-rdf-ntriples.txt
Original Record: 24-1.0052210-source.json
Full Text
24-1.0052210-fulltext.txt
Citation
24-1.0052210.ris

Full Text

Interaction with Large Stereoscopic Displays Fitts and Multiple Object Tracking Studies for Virtual Reality by Vasanth Rajendran B.Tech., Indian Institute of Technology Madras, 2010 A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF SCIENCE in The Faculty of Graduate Studies (Computer Science) THE UNIVERSITY OF BRITISH COLUMBIA (Vancouver) January 2013 c© Vasanth Rajendran 2012 Abstract Two studies measured human performance in virtual environments, one for mid-air pointing and one for multiple object tracking. Mid-air pointing is simi- lar to pointing in everyday life: it relies on the human sensorimotor system and it is unconstrained by contact with a surface or object. Multiple object track- ing is the perceptual and cognitive process of maintaining awareness of the individual identities of a set of objects that are changing their relative locations within a visual scene. Both tasks are often components of training or simu- lation systems in virtual environments. We investigated how performance in each is affected by the virtual environment and how that might be used to bet- ter calibrate virtual environments for optimal “real world” performance. Fitts’s Law predicts movement time as a function of movement amplitude and target width. It is used to evaluate pointing performance in computer ap- plications, including those for virtual environments. It has proven extremely robust. However, alternative two-part models, such as Welford’s, are some- times more accurate. One-part models consider only the ratio of movement amplitude and target width; two-part models treat each as independent vari- ables. We conducted a Fitts-style study of pointing performance in a virtual envi- ronment that varied physical distance to the screen and virtual distance to the targets. We found that Welford-style two-part models predicted pointing time significantly better than did Fitts-style one-part models using an F-test on the goodness of fit (R2). The separable contributions of movement amplitude and target width were captured in a parameter derived from the two-part models that varies linearly with the virtual distance to targets. We describe how to use the parameter to calibrate a VR environment for pointing tasks. We designed a second study of multiple-object tracking in VR environ- ments to determine whether tracking performance might differ in a virtual ii Abstract environment. Results from a pilot experiment were not conclusive. Tracking accuracy for 3D followed the same pattern as for 2D; so, additional studies will be required to determine if there are differences. iii Preface All research in this thesis was conducted under the supervision of Dr. Kel- logg Booth. The design of the experiment in Chapter 3 was co-supervised by Dr. James T. Enns. Ethics approval for experimentation with human subjects was granted by the Behavioural Research Ethics Board, UBC (BREB Number H11-01756). I am the primary contributor of all work in this thesis. The experimen- tal software and design for Chapter 2 is based on similar software written by Dr. Garth Shoemaker. I collaborated with Julia Freedman on conducting the pilot experiments described in Chapter 3. iv Table of Contents Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ii Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iv Table of Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . v List of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viii List of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.1 Virtual Reality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.2 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.2.1 Research Questions . . . . . . . . . . . . . . . . . . . . . 6 1.3 Summary of Research Contributions . . . . . . . . . . . . . . . . 6 1.4 Outline of the Thesis . . . . . . . . . . . . . . . . . . . . . . . . . 7 2 Models for Pointing Performance in VR . . . . . . . . . . . . . . . . 8 2.1 Background and Related Work . . . . . . . . . . . . . . . . . . . 9 2.1.1 Pointing Methods for VR . . . . . . . . . . . . . . . . . . 9 2.1.2 One-Part Models of Pointing Performance . . . . . . . . 10 2.1.3 Two-Part Models of Pointing Performance . . . . . . . . 11 v Table of Contents 2.1.4 Comparing Pointing Models Statistically . . . . . . . . . 13 2.1.5 Effective Width We . . . . . . . . . . . . . . . . . . . . . . 14 2.2 Experimental Apparatus and Software . . . . . . . . . . . . . . 15 2.2.1 Apparatus . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 2.2.2 Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 2.3 VR Pointing Experiment . . . . . . . . . . . . . . . . . . . . . . . 19 2.3.1 Task and Stimuli . . . . . . . . . . . . . . . . . . . . . . . 19 2.3.2 Participants . . . . . . . . . . . . . . . . . . . . . . . . . . 20 2.3.3 Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 2.3.4 Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 2.3.5 Measures . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 2.3.6 Hypotheses . . . . . . . . . . . . . . . . . . . . . . . . . . 23 2.4 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 2.4.1 ANOVA for Movement Time . . . . . . . . . . . . . . . . 24 2.4.2 Regression with One-Part and Two-Part Models . . . . . 25 2.4.3 The Parameter k as a Function of DV . . . . . . . . . . . 27 2.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 2.5.1 A Closer Look at the Data . . . . . . . . . . . . . . . . . . 29 2.5.2 One-Part and Two-part Models for Pointing . . . . . . . 30 2.5.3 Parameter k as Virtual Distance DV Varies . . . . . . . . 32 2.5.4 Calibrating VR Systems with Pointing Tasks . . . . . . . 34 2.5.5 Using the Calibrated Values . . . . . . . . . . . . . . . . 34 2.6 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 3 Multiple Object Tracking in VR . . . . . . . . . . . . . . . . . . . . . 36 3.1 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 3.1.1 Multiple Object Tracking . . . . . . . . . . . . . . . . . . 37 3.1.2 Allocentric Tracking and Scene Stability . . . . . . . . . 37 vi Table of Contents 3.1.3 Multiple Object Tracking in VR . . . . . . . . . . . . . . 38 3.2 Experimental Apparatus and Software . . . . . . . . . . . . . . 38 3.2.1 Apparatus . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 3.2.2 Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 3.3 Multiple Object Tracking Experiment . . . . . . . . . . . . . . . 40 3.3.1 Task and Stimuli . . . . . . . . . . . . . . . . . . . . . . . 40 3.3.2 Display Conditions . . . . . . . . . . . . . . . . . . . . . 43 3.3.3 Participants . . . . . . . . . . . . . . . . . . . . . . . . . . 43 3.3.4 Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 3.3.5 Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 3.3.6 Measures . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 3.3.7 Indications from the Pilot Study . . . . . . . . . . . . . . 45 3.4 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 4 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 4.1 Research Contributions . . . . . . . . . . . . . . . . . . . . . . . 49 4.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 Appendices A Pointing Experiment Data and Questionnaires . . . . . . . . . . . . 56 B MOT Experiment Data and Questionnaires . . . . . . . . . . . . . . 64 vii List of Tables 2.1 Significant ANOVA results for movement time (MT) . . . . . . 24 2.2 Modelling movement time (ms) using the Fitts and Welford for- mulations using actual width W. . . . . . . . . . . . . . . . . . . 26 2.3 Modelling movement time (ms) using the Fitts-Shannon and Welford- Shannon formulations using actual width W. . . . . . . . . . . . 26 2.4 Modelling movement time (ms) using the Fitts and Welford for- mulations using effective width We. . . . . . . . . . . . . . . . . 26 2.5 Modelling movement time (ms) using the Fitts-Shannon and Welford- Shannon formulations using effective width We. . . . . . . . . . 26 2.6 Linear regression of k and DV (cm) for various cases. . . . . . . 29 3.1 Significant ANOVA results for MOT tracking accuracy (pcorrect) 46 A.1 Raw performance results for the experiment described in Chap- ter 2. The movements times (MT) were obtained by averaging over all subjects for each combination of Dv, Ds, A and W. . . . 56 A.2 ANOVA results for movement time (MT) . . . . . . . . . . . . . 60 B.1 Raw performance results for the pilot study described in Chap- ter 3. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 viii List of Figures 1.1 Levels of virtualization in VR. . . . . . . . . . . . . . . . . . . . . 3 2.1 Ray-pointing techniques for very large displays (Jota, Nacenta, Jorge, Carpendale, & Greenberg, 2010) . . . . . . . . . . . . . . . 10 2.2 Method of computing effective width We (MacKenzie, 1991) . . 15 2.3 Apparatus: (a) Large screen stereoscopic display (b) Hand-held pointer (c) Head gear (d) WiiMote for click events. . . . . . . . . 16 2.4 Computing the cursor position from the head and hand-held pointer positions. . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 2.5 Placement of the targets and the participant in an experimental trial. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 2.6 An illustration of the participant performing the pointing task for DS = DV . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 2.7 An illustration of the participant performing the pointing task for DS > DV . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 2.8 Regression R2 values as they vary with virtual distance. . . . . . 27 2.9 k values calculated using the Welford formulation (a) A, W (b) A, We . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 2.10 Scatterplot of MT vs. effective ID (Note that the points are iden- tical in the right and left plots. Points are connected in two dif- ferent ways to illustrate the separability effect): (a) Lines connect points representing tasks with the same movement amplitude A, (b) Lines connect points representing tasks with the same target width W . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 ix List of Figures 2.11 Calibrating the virtual depth based on k values . . . . . . . . . . 33 3.1 Apparatus: (a) Large screen stereoscopic display (d) Head gear 39 3.2 A screenshot of the MOT experiment. . . . . . . . . . . . . . . . 40 3.3 Various stages of the MOT experiment: (1) Rest, (2) Highlight, (3) Track, and (4) Probe. . . . . . . . . . . . . . . . . . . . . . . . 42 3.4 2D (flat) and 3D (stereoscopic) conditions . . . . . . . . . . . . . 43 3.5 Controlling scene stability by projecting onto a dihedral surface. 44 3.6 Plot of accuracy values and number of tracked objects for differ- ent viewing conditions. . . . . . . . . . . . . . . . . . . . . . . . . 47 4.1 Calibrating VR systems based on task performance. . . . . . . . 48 A.1 Pre-Experiment questionnaire for the pointing experiment de- scribed in Chapter 2 . . . . . . . . . . . . . . . . . . . . . . . . . . 62 A.2 Post-Experiment questionnaire for the pointing experiment de- scribed in Chapter 2 . . . . . . . . . . . . . . . . . . . . . . . . . . 63 B.1 Pre-Experiment questionnaire for the MOT experiment described in Chapter 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66 B.2 Post-Experiment questionnaire for the MOT experiment described in Chapter 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 x Acknowledgements First, I thank my supervisor Dr. Kellogg S. Booth for his support, guidance and the untiring discussions in his office. I am always amazed at his dedication — in spite of his ever-busy schedule, he always found time guide me in my research and explore every single detail. I could not have finished this thesis on time without the last-minute revision sessions in his office. I thank Dr. James T. Enns for his time and advice as my second reader. Even on Christmas Eve, he sent in his revisions so promptly. I am very grateful to both of them. The research reported in this thesis was supported in part by the Natu- ral Science and Engineering Research Council of Canada (NSERC) through the Discovery Grant program and by the Networks of Centres of Excellence program through the Graphics, Animation and New Media NCE (GRAND). Equipment was provided under funding from the Canadian Foundation for Innovation and the British Columbia Knowledge Development Fund. All of this support is gratefully acknowledged. I thank my friends for being the best distraction ever. They are a key part of my life at Vancouver; I shall always cherish these memories. I cannot thank my parents enough for making me what I am, and for their absolute love and confidence. Finally, I bow down to my little sister for patiently enduring my daily laments and offering words of wisdom. Genetically speaking, she is undoubt- edly a vast improvement over me, as she always likes to point out. This is my formal acknowledgement of the same. xi Chapter 1 Introduction In this thesis we explore the question of whether virtual environments can be used to reliably assess human behaviour for tasks that are of interest. This question is important because virtual environments are often used for train- ing or simulation exercises where the objective is to elicit the same behaviour as in the real world for specific tasks or activities. If the behaviour induced by a virtual environment is not sufficiently similar to what would take place in a real world situation, any measurements made in the virtual environment might not be valid predictors on which to base decisions about real-world envi- ronments. The research reported in this thesis investigated two different tasks and assessed the degree to which human performance on those tasks was the same in a virtual environment as it was in a real-world environment. The goal of the research was to first measure whether behaviour was the same and then to determine if adjustments to the calibration of the virtual environment based on any differences that were observed could produce a re-calibrated virtual en- vironment that would induce similar or identical behaviour to that of a target real-world environment. The virtual environment used in our studies was a large-screen stereoscopic wall display capable of tracking a user’s body position and orientation and projecting the proper binocular images onto the display to simulate a real- world environment. We investigated two specific tasks that are commonly per- formed in virtual environments: mid-air pointing and multiple object tracking. For mid-air pointing we wanted to better understand how well a virtual en- vironment induces the same type of pointing behaviour that occurs in the real world when people point at objects with their hands. We refer to this as “mid- air pointing” because a person’s hand is moving freely and is not constrained by contact with any surface or object. This is a very common task that occurs in many training or simulation activities in virtual environments. Mid-air point- 1 1.1. Virtual Reality ing involves the human sensorimotor system: the motor actions of pointing are mediated by visual information. We conducted an experiment that used our virtual environment to simulate the distance to objects that were being pointed to and tested whether pointing performance (time required to point) was the same regardless of whether the distance to the object was measured in physical space or virtual space. We found that performance was affected when the dis- tance to objects was not the same in physical and virtual space. We were able to model this using a two-part model of pointing that is derived from earlier work by Welford (1968). We discuss how this model might be used to better calibrate virtual environments if the goal is to achieve real-world performance in a mid-air pointing task using a virtual environment. The second task, multiple object tracking, was chosen because it does not involve the sensorimotor system. It is primarily based on visual perception with no motor component, only a perceptual and cognitive component. In multiple object tracking a person “follows” a number of objects while over time the objects move about in space. The task being performed is maintaining an accurate knowledge of the individual identities of objects as they change their locations. We conducted a pilot experiment to determine if the 3D stimuli provided by a virtual environment induces the same multiple object tracking performance (accuracy) as do traditional 2D stimuli. This was a preliminary investigation, intended as a first step in exploring this question. We did not find a difference between 3D and 2D stimuli, but our pilot study was too small to be conclusive. We make recommendations for a future study that might lead to a better understanding of how virtual environments should be calibrated to best support multiple object tracking tasks. In this chapter, we give a brief introduction to virtual reality, followed by the core problem statement for our research in more detail. We then summarize our research contributions and outline the rest of the thesis. 1.1 Virtual Reality Ellis (1991) defines virtualization as . . . the process by which a human viewer interprets a patterned sen- sory impression to be an extended object in an environment other 2 1.1. Virtual Reality Figure 1.1: Levels of virtualization in VR. than that in which it physically exists. Virtual Reality (VR) environments often consist of head-coupled, virtual image, stereoscopic displays, with means of interaction and input to generate a coordinated sensory experience. Some systems include auditory and tactile information. VR environments are used in entertainment (gaming, movies, virtual ex- periences, etc.), training and simulation (military training, driving and flight simulators, etc.) and medical and scientific visualization. Depending on the application, the requirements of a VR system might be different. In training applications, it is desirable that the VR system produce responses and task performance that is nearly identical to that in the real world. In simulation and visualization applications, it is important that the perceived virtual envi- ronment be free of any distortions or disparities associated with stereoscopic viewing. Three levels of virtualization can be identified in VR systems (Figure 1.1): (1) virtual space, (2) virtual image, and (3) virtual environment. The virtual space is the abstract environment that our virtual objects reside in. This is application-specific, and is not a relevant part for our work. The virtual im- age is the interactive system that is presented to the user, mainly consisting of stereoscopic images and techniques for interaction. The virtual environment is the ‘space’ that the users perceive themselves to be in. 3 1.2. Problem Statement The mapping from a virtual image to the virtual environment lies at the core of this thesis. The accuracy of this mapping is limited by available tech- nology, by the characteristics of human perception and often by individual dif- ferences. Our thesis aims to identify a few of these limitations, and develop work-around ‘calibration’ methods that can bring a VR system closer to its de- sired performance. There has been previous work that suggests approaches for VR calibra- tion. Jones, Lee, Holliman, & Ezra (2001) describe an approach to calibrat- ing the camera space against a user space based on user-reported depth val- ues. Yuan, Pan, & Daly (2011) perform depth tuning of stereoscopic 3D con- tent based on models of human visual comfort. Iyer, Chari, & Kannan (2008) describe a method for determining parallax values for stereoscopic viewing that minimize visual artifacts in 3D television. These approaches are based on stereoscopic perception models – either derived from literature, or from user- reported values. They primarily aim to reduce the distortions associated with stereoscopic viewing. In VR applications, it is often desirable to produce task performance identical to that in real-life tasks, not simply reduce perceived distortion, but these approaches do not take task performance into account for calibration. Our research examines task performance for two tasks, mid-air pointing and multiple object tracking, in order to develop calibration criteria that will yield real-life performance in virtual environments. 1.2 Problem Statement Understanding the control characteristics for human movement, visual track- ing, and vestibular responses (Ellis, 1991) is needed to determine their limits when working in VR environments, and to design VR systems that best suit a given set of tasks. The main research problem addressed in this thesis is to study mid-air pointing and multiple object tracking in virtual reality environments. These are two different tasks, one involving the sensorimotor system, the other in- volving only perception and cognition. They are often used as components of training or simulation exercises in virtual environments. Our interest is in understanding how well these two task are supported by current virtual envi- ronment technology and whether it may be necessary to alter the parameters 4 1.2. Problem Statement of the visual components of the virtual environment to better support these tasks. Appropriate parameters would need to be determined through a cali- bration process. The experiments we report are intended to serve as the basis for future calibration procedures. Our VR system comprises a large head-coupled stereoscopic wall display, with motion tracking for pointing. Mid-air pointing is an important method of input and interaction with VR environments. It is one of the fundamental interaction techniques. Users may also need to keep track of multiple objects and interface elements within the VR environment. We are interested in deter- mining performance limits and how those might depend on the parameters of a virtual environment. We formulated two sets of research questions. We based our study of mid-air pointing on two classes of models for point- ing performance that predict movement time given the movement amplitude and the width of the target object. One class of models is “one-part models” because they only involve the ratio of movement amplitude to target width. “Two-part models” treat these as separate parameters. The most common one- part model is Fitts’s Law. Welford proposed a two-part model that extends Fitts’s Law. Both models predict increased movement time as movement am- plitude increases or as target width decreases. These are explained further in Chapter 2. Our study was designed to help us determine whether one-part or two-part models better predict pointing performance in a virtual environ- ment and if the extra degree of freedom in a two-part model might be useful in calibrating a virtual environment for mid-air pointing tasks. We based our study of multiple object tracking on previous work reported in the literature that characterizes tracking performance in terms of the number of objects being tracked, their relative speeds, and other properties of the scene in which the objects are moving. One property of the scene that has not been discussed in the literature is the use of 3D virtual environments. Our study was designed to help us determine whether a 3D virtual environment provides similar performance to the traditional 2D displays that have been used in most previous studies and if a multiple object tracking task performed in a virtual environment might be useful in calibrating the virtual environment for tasks that do not involve the sensorimotor system. 5 1.3. Summary of Research Contributions 1.2.1 Research Questions Our goal was to answer the following research questions: 1. Pointing Performance: (a) Are one-part models of pointing performance (such as Fitts’s Law), that predict movement time as a function of just the ratio of move- ment amplitude and target width, sufficient for describing pointing in VR environments? (b) Are two-part models (such as Welford’s Law), that allow for sepa- rable contributions of movement amplitude and target width to the movement time, more appropriate for describing pointing perfor- mance in VR? (c) Can we calibrate a VR setup based on pointing task performance? 2. Multiple Object Tracking: (a) Does multiple object tracking performance differ in the stereoscopic viewing conditions in VR? (b) How does changing the scene coherence affect multiple object track- ing performance? (c) Can we perform a controlled multiple object tracking experiment to calibrate a VR setup? 1.3 Summary of Research Contributions We summarize the primary contributions of the research described in this the- sis. We revisit these again in Chapter 4. 1. We studied the characteristics of mid-air pointing performance in VR environments. Through an experiment, we showed that accepted one- part models of pointing performance (such as Fitts’s Law) do not ade- quately describe pointing in VR. We showed that two-part models (such as Welford’s Law) are better suited. We experimentally derived a param- eter k that accurately describes the degree-of-separability of the move- ment amplitude A and target width W. We then derived a calibration method for VR setups that uses these k values to adjust virtual depth. 6 1.4. Outline of the Thesis 2. We designed an experiment to study multiple object tracking in VR envi- ronments. In our experiment, we controlled the display viewing condi- tions (stereoscopic 3D, head-coupling and scene coherence), and studied how these affect performance. We present results from a pilot study and outline future experiments. 1.4 Outline of the Thesis In Chapter 2 of this thesis, we present an experiment comparing one-part and two-part models of pointing for VR systems. The experiment suggests a novel calibration technique for virtual environments that uses data from this type of experiment to derive calibration parameters that ensure that pointing perfor- mance matches real-world behaviour. In Chapter 3, we describe a pilot exper- iment studying multiple object tracking in VR systems. Preliminary analysis of the data from the pilot study suggests that suitable calibration procedures might be based on multiple object tracking experiments. This requires addi- tional research. The relevant background and literature are presented within each of these chapters. In Chapter 4, we summarize our results and outline future work in this area. 7 Chapter 2 Models for Pointing Performance in VR Many interaction techniques for virtual reality (VR) systems depend on mid- air, physical pointing by the user. Fitts’s Law (Fitts, 1954), the most commonly used model of pointing, predicts the time required to rapidly move to a target area as a function of the movement amplitude (distance moved while point- ing) and target width (the size of the object being pointed to). According to Fitts’s Law, movement time increases with amplitude but decreases with tar- get size, both following a logarithmic function. The full mathematical details are provided in Section 2.1.2. Fitts’s Law has been quite successful in model- ing pointing performance in many mouse-based computer applications, and is often the model of choice for evaluating performance for new pointing de- vices and techniques, including how pointing tasks are performed in virtual environments. Performance in a pointing task is usually measured by move- ment time and error rate. Often movement time is the primary measure of performance. Other models for pointing performance have been developed. Welford’s (1968) is one that we will consider. In this chapter, we describe a Fitts-style pointing experiment in VR envi- ronments where we also vary physical distance to the screen and the virtual distance to the targets. We found that two-part models (such as Welford’s Law) described pointing performance significantly better than did Fitts-like one-part models in most cases. Accurate models of pointing performance help in devel- oping interaction techniques and informing design decisions for VR. We also believe that a better understanding of pointing performance will lead new cal- ibration techniques for virtual environments. Two-part models of pointing allow for separable contributions of move- ment amplitude A and target width W to the movement time MT. The rela- 8 2.1. Background and Related Work tive magnitude of these contributions can be captured in a parameter k that, we found, varies linearly with the virtual distance to the targets. In our ex- periment, we also found that pointing performance varies with the physical distance to the screen in addition to the expected variation with virtual dis- tance. This suggests that the virtual environment may not be providing the same sensory information as would a real-world environment. In a perceptually-ideal VR system, the perceived environment should be invariant of the physical position of the screen. In particular, when the VR sys- tem is used in simulation and training, it is desirable to get task performance characteristics that are identical to the real world. The parameter k can be used to ‘calibrate’ the VR system for pointing-based tasks. The contributions of this chapter are two-fold. First, we present results from an experiment that shows two-part models are more appropriate for evaluating pointing performance in VR systems. Second, we describe how such a pointing task can be used to calibrate a VR environment. Section 2.1 presents the background and relevant literature. In Section 2.2, we introduce the apparatus and software used in our experiments. We describe the VR pointing experiment in Section 2.3 and present the results in Section 2.4. We discuss our hypotheses in light of these results in Section 2.5. Finally, we present conclusions and directions for future research in Section 2.6. 2.1 Background and Related Work Relevant related work lies in the areas of virtual reality research and pointing performance research. We review them both in this section. 2.1.1 Pointing Methods for VR There are a variety of distant pointing techniques for very large displays. The simplest of these are ray pointing techniques – methods of projecting the pointer position and orientation to the target plane. Jota et al. (2010) compare four such techniques (illustrated in Figure 2.1). There are two types of movement control for pointing – translational and rotational. Techniques based on rotational control are not suited for the Fitts- 9 2.1. Background and Related Work style task in our experiment. In addition, we found that our precision in mea- suring pointer orientation was insufficient for accurately computing the cur- sor position on the target plane. We therefore use translational pointing tech- niques. A Comparison of Ray Pointing Techniques for Very Large Display Ricardo Jota1, Miguel A. Nacenta2, Joaquim A. Jorge1, Sheelagh Carp nd l 2, and Saul Gree berg2 1Inesc-ID, 2University of Calgary  ABSTRACT Ray-pointing techniques are often advocated as a way for people to interact with very large displays from several meters away. We are interested in two factors that can affect ray pointing: the par- ticular technique's control type, and parallax. Consequently, we tested four ray pointing variants on a wall display that covers a large part of the user's field of view. Tasks included horizontal and vertical targeting, and tracing. Our results show that (a) techniques based on 'rotational control' perform better for targeting tasks, and (b) techniques with low parallax are best for tracing tasks. We also show that a Fitts's law analysis based on angles (as opposed to linear distances) better approx- imates people's ray pointing performance. KEYWORDS: Large displays, ray pointing, distant pointing, image- plane, targeting, tracing, index of difficulty, ISO 9241, parallax. INDEX TERMS: H5.2.User Interfaces: Input devices and strategies 1 INTRODUCTION People often interact with a large digital display by distant point- ing, or directly pointing at the display with their finger, laser poin- ter, or other input device. Ray pointing is a class of techniques that uses ray casting (the intersection of a ray with a surface) to determine where a person is pointing to, i.e., the precise cursor position on the distant display. Ray pointing is advocated as a natural interaction technique with these displays [2,20] as: it al- lows people to interact from the distance (as opposed to direct- input) [21,31]; it does not require any physical surface to operate on (as opposed to mouse-based pointing) [27,28]; it is easily un- derstood by people as it builds upon everyday pointing gestures [31]; and it allows multiple users to interact on the same display without their bodies physically getting in the way [17]. Thus it is no surprise that ray pointing is increasingly used in both commer- cial and research systems, especially for large horizontal and ver- tical displays [6,17,21,30]. Even game consoles are exploiting ray pointing for interaction (e.g., Nintendo Wii). As large and very large displays (i.e., wall-sized displays) be- come widespread, ray pointing will likely become a primary way to interact with all kinds of interfaces from a distance or to access hard to reach areas of the display. This is why there is already a significant amount of literature devoted to the performance and different modes of ray pointing; for example, different variants of ray pointing are used and studied by Virtual Reality researchers [1,4,24], and there have been some efforts to characterize laser pointing (one of the possible ray pointing variants) for interaction with 2D interfaces [14,15,22,27]. However, in our own design and implementation of large-display interfaces we have found that previous work in the area does not suffice because of several rea- sons. A) Although the largest diversity of ray pointing alternatives has been studied in VR, ray pointing tasks and setups (e.g. CAVEs, stereoscopic displays) are substantially different than the more common 2D tasks that we are interested in. B) Previous empirical work on 2D tasks is focused almost exclusively on laser pointing or studies only small displays [14,16,27]. C) With a few exceptions [12,15] previous work for 2D environments does not try to provide explanations or general principles of the differences in ray pointing. We build upon this previous research by contributing, via a new experiment: a) tests of two fundamental 2D interaction tasks (tar- geting and tracing), one of which has not previously been studied in the context of ray pointing; b) comparisons of four different ray pointing variants (laser pointing, arrow pointing, image-plane pointing and fixed-origin pointing) which map to two previously unstudied factors relevant to ray pointing in 2D tasks (parallax and control type); and c) identification of specific issues related to the large size of the displays, like the effect of the location of targets with respect to users. Our experimental results show that targeting performance is best explained by the control type factor, with rotational control being generally superior to position control, whereas for tracing tasks it is the presence of parallax that better explains differences between variants. The study also contributes rigorous support for the use of an angular formulation of Fitts’s law [10,12] for large- displays, as opposed to the traditional linear formulation for both tasks. These results have important implications for designers and re- searchers alike. First, designers must now consider image-plane techniques and how the pointing device is held, especially if they want to support tracing tasks (e.g., menu selection, drawing). Second, researchers can now consider parallax and control type as important factors, and have solid evidence to start using angular adaptations of Fitts’s law for interaction in large displays. 2 RAY POINTING FUNDAMENTALS We now describe the particular ray pointing techniques we use and how we implemented them in our study. We define generic ray pointing as any cursor-movement tech- nique that determines the position of the cursor through the inter- section of a ray with a distant object or surface (see Figure 2, left). For our purposes, the distant object or surface is a large display. We consider only monocular absolute 2D pointing. 1Rua Alves Redol 9, Apart. 13069 1000-029 Lisboa, Portugal {jota.costa,jaj}@inesc-id.pt   2Comp. Sci., 2500 University Dr. NW. Calgary, AB, Canada T2N 1N4. {miguel.nacenta, saul.greenberg, sheelagh.carpendale}@ucalgary.ca    Figure 1. Four variants of ray pointing Laser Arrow Image-Plane Fixed-Origin Figure 2.1: Ray-pointing techniques for very large displays (Jota et al., 2010) Fixed-origin pointing techniques (like shadow reaching by Shoemaker, Tang, & Booth (2007)) compute the cursor position by extrapolating the line between a fixed point in space and the user’s hand position. A variant of this tech- nique is image-plane pointing (Figure 2.1(d)), where the projection origin is the user’s eye position. Image-plane pointing minimiz s isual parallax, which we found in our pilot studies o be more comfortable for pointing. This is in agree- ment with the conclusions drawn by Jota et al. (2010). Also they found that image-plane pointi g produced better p rformance than fixed-origin pointing for targeting and tracing tasks. 2.1.2 One-Part Models of Pointing Performance The best known one-part model of pointing performance is Fitts’s Law (Fitts, 1954). The log term is often called the Index of Difficulty (ID). 10 2.1. Background and Related Work MT = a+ b log2 ( A W ) (2.1: Fitts) In one-part models, the movement time (MT) depends only on the ratio of movement amplitude (A) and target width (W), but not their absolute values. Soukoreff & MacKenzie (2004) present arguments for the use of an alterna- tive one-part formulation based on an information-theoretic interpretation of Fitts’s Law. The additive constant of one within the ID term, they argue, makes for a more faithful transfer of Shannon’s information theory. When modelling real-world data, the Shannon formulation often produces better R2 values than does Fitts’s Law. MT = a+ b log2 ( A W + 1 ) (2.2: Shannon-Fitts) We shall use both Equation 2.1 and Equation 2.2 in our experiment.. 2.1.3 Two-Part Models of Pointing Performance Welford introduced a two-part model of pointing performance (Welford, 1968) to account for deviations from Fitts’s Law that he observed in an improved version of Fitts’s original experiments. In this model, the ID term is separated out into a linear combination of two log terms. MT = a+ b1 log2(A)− b2 log2(W) (2.3: Welford) Two-part formulations allow for separable contributions of A and W to movement time MT. Compared to Fitts’ Law, the Welford two-part formu- lation has been largely overlooked in the HCI literature. But there has been work that suggests that A and W need to be separable in modelling movement time (Shoemaker, Tsukitani, Kitamura, & Booth, in press; Wobbrock, Cutrell, Harada, & MacKenzie, 2008). Our work concludes with a similar finding. Shoemaker et al. introduce a variation on Welford’s two-part model which combines aspects of the Shannon-Fitts formulation. The mapping of noise and signal to movement amplitude A and target width W is the same as with Equa- tion 2.2, but the signal and noise are broken into independent terms as with the 11 2.1. Background and Related Work Welford’s formulation. MT = a+ b1 log2(A+W)− b2 log2(W) (2.4: Shannon-Welford) We shall use and compare both Equation 2.3 and Equation 2.4 in our exper- iment. The k Value for Pointing at a Distance Kopper, Bowman, Silva, & McMahan (2010) examined distal pointing on large displays with a laser pointer. Recall that laser pointing (Figure 2.1 (a)) is based on angular movements. An experimental task for laser pointing would vary the angular movement amplitude α and the angular target width ω. Kopper et al. validated a model that allows for separable effects of α and ω through an exponent k. MT = a+ b log2 ( α ωk + 1 ) (2.5) Inspired by this formulation, Shoemaker et al. present an alternative ver- sion of Welford’s Law, for linear measures A and W. MT = a+ b log2 ( A Wk ) (2.6) Note that Equation 2.6 is mathematically equivalent to Welford’s two-part formulation in Equation 2.3. The k and b values in Equation 2.6 are equal to (b2/b1) and b1 from Welford’s formulation, respectively. The exponent k encapsulates the relative magnitudes of the separable con- tributions of A and W to the movement time MT. If the experimental results determine that k = 1, then b1 = b2 and the model is Fitts’s Law. Thus, Fitts’s Law will model the data just as well as does Welford’s model. The more k deviates from unity, the worse Fitts’s Law will perform compared to Welford. Therefore, k is a good indicator of the applicability of Fitts’s Law. 12 2.1. Background and Related Work Unit Independence of b1, b2 and k in Two-Part Models In Equation 2.3, both A and W possess units of distance, and yet appear within logarithms. They are intended to be used as dimensionless values. It is de- sirable to treat log2(A) and log2(W) similarly to the index of difficulty ID = log2(A/W), which is dimensionless. As Graham & MacKenzie (1996) explain, Welford anticipated this and proposed nominal values A0 and W0 that normal- ize A and W respectively. MT = a+ b1 log2 ( A A0 ) − b2 log2 ( W W0 ) (2.7) In our analysis, we will assume that the constants A0 and W0 are both 1cm and simply use Equation 2.3 with the absolute values of A and W in centime- tres. Note that the choice for A0 and W0 only affect a; the values of b1 and b2 (and hence k) are independent of this choice. MT = a+ b1 log2 ( A A0 ) − b2 log2 ( W W0 ) = [a− b1 log2(A0) + b2 log2(W0)] + b1 log2(A)− b2 log2(W) = a′ + b1 log2(A)− b2 log2(W) Secondly, the units chosen for measuring A and W will impact the derived constants. Similar to the previous argument, we can show that only the in- tercept value a is affected by a change in units. The values of b1 and b2 (and hence k) are independent of the units used. Therefore, k is ideal in capturing the separable effects of A and W. 2.1.4 Comparing Pointing Models Statistically One of the goals of this chapter is to determine if two-part models describe our data better than do one-part models. Soukoreff & MacKenzie (2004) observed that two-part models will inevitably perform better than their one-part equiv- alents because of the additional degree of freedom. It is necessary, therefore, to determine if a two-part model performs better because it captures aspects 13 2.1. Background and Related Work of the data that a one-part model cannot, and not merely because of the extra degree of freedom. There are several tests for comparing linear models: the F-test, the Akaike Information Criterion (AIC), and the Schwarz Criterion (known as the BIC). As observed by Shoemaker et al., the AIC and BIC tests are suitable tools for model selection, but not for hypothesis testing. They will not be able to determine if a higher degree-of-freedom model is significantly better at modeling the data compared to a lower degree-of-freedom model. The F-test provides a means of testing such a hypothesis, and hence, is appropriate for our purposes. A limitation of the F-test is that it can only compare models that are nested, i.e. if the greater degree-of-freedom model can be made equivalent to the lower degree-of-freedom model by enforcing an additional constraint. For example, Equations 2.1 and 2.3 are equivalent if b1 = b2. Similarly, Equations 2.2 and 2.4 are equivalent if b1 = b2. In our analyses, we group these equations accord- ingly. The F-test we use is F(p2 − p1, n − p2) where p2 = 3 is the number of parameters in the greater degree-of-freedom model, p1 = 2 is the number of parameters in the simpler nested model and n is the number of sample points. A p-value of .05 or less is considered significant with this test. 2.1.5 Effective Width We Crossman and Welford (Welford, 1968) described a method of computing the effective width (We) of a target derived from the observed distribution of tap positions. The width of a target is adjusted to reflect what a participant actu- ally did, rather than what the participant was expected to do. This post-hoc adjustment on target width tries to maintain the information-theoretic analogy for the Fitts and similar models. These models are for rapid, aimed movements and assume the presence of a nominal and consistent error rate (traditionally 4%). When this condition is not met, an adjustment to the target width is intro- duced such that the error rate is 4%. There are two ways for arriving at the effective target width We. If the standard deviation (σ) of the taps (about the centre of the targets) is known, then We is defined as, We = 4.133σ (2.8) 14 2.2. Experimental Apparatus and Software Figure 2.2: Method of computing effective width We (MacKenzie, 1991) (Using the probability distribution of a normal distribution, we see that P[Z ≥ 2.066] = 0.02, and therefore a width of 2.066× 2 in z-units, i.e. 4.133σ around the mean, has a probability of 96%.) Alternately, the error rate may be used to approximate the adjustment in Equation 2.8, if the standard deviation for the taps is not available. We = W × 2.066 z(1− perror2 ) if perror > 0.0049% W × 0.5089 otherwise. (2.9) (perror is the error rate for a given condition, and P[Z ≤ z(x)] = x.) 2.2 Experimental Apparatus and Software The experimental apparatus forms a part of the VR laboratory at the University of British Columbia. We developed experimental software for displaying the cursor and targets, tracking the subjects and logging relevant data. We describe these in detail in this section. 15 2.2. Experimental Apparatus and Software Figure 2.3: Apparatus: (a) Large screen stereoscopic display (b) Hand-held pointer (c) Head gear (d) WiiMote for click events. 2.2.1 Apparatus We use a display made of a glass screen of 5.16m×2.85m in size and rear- projected by a 4×3 array of 800px×600px stereo projectors. The images of adjacent projectors overlap by 160px with a blending function to minimize the appearance of discontinuities. The overall resolution of the display is therefore 2720px×1480px. The entire display is frame-sequential stereoscopic at 60Hz. Users view this display through shutter glasses that are synchronized with the projectors. The display is driven by a computer with an 8-core Intel Processor, 6GB of RAM, dual NVIDIA GeForce GTX 260 graphics processors and running Windows 7. An arrangement of five Vicon cameras (a high speed infrared-based mo- tion capture system) are used to track the participant. We use head gear and a hand-held pointer each fitted with specially-designed reflective balls as shown in Figure 3.1. With these, we can track the participant’s head and the pointer position and orientation accurately in the room. We use this to compute and project a virtual cursor on the screen for mid-air pointing (as described in Sec- 16 2.2. Experimental Apparatus and Software tion 2.1.1). ‘Tapping’ on targets was performed using the thumb (A) button on a hand- held Nintendo Wii Remote. Participants held this remote with their left hand (the non-pointing hand) to minimize the disturbance that could be caused dur- ing clicking in high CD gain conditions. 2.2.2 Software The experimental software is written in C# using the Microsoft XNA Game Studio 4.1 library. The WiimoteLib library (WiimoteLib - .NET Managed Library for the Nintendo Wii Remote, 2012) is used to communicate with the remote and detect click events. The Vicon motion tracking system is managed by Vicon IQ software. From our code, we query this software every (1/60)th of a second to obtain the positions of the user’s head and the pointer. The Vicon system is an optical motion tracking system. Sometimes it loses track of either the hand-held pointer or the head gear. When this happens, the cursor either gets stuck (when both are lost) or the cursor position is incorrect (when only the head tracker is lost). This is an inherent and unavoidable lim- itation of optical tracking systems. We’ve tried to minimize such occurrences through the placement of the five cameras around the room. Our software records all click events and the timing for each trial condition. In addition, we also record the raw tracking data, 60 times a second. We use this to perform checks on the data and, if necessary, analyze the movement patterns in greater detail. When the Vicon system loses track of the pointers, the API returns NULLs on calls for the position and orientation data. When this happens, we record this in the raw position stream log. During data analysis, we identified taps for which this happened for more than three consecutive frames, and removed them from our data before computing averages. Shorter drops are more common and are usually unnoticeable. The distribution of the occurrences among different positions of the participant in the room (DS) was (80, 15, 13). The reason for the higher number of drops for DS =110cm was because the participant stood very close to the screen and all cameras had to track from the sides or behind. Another source of error we could identify was multiple-clicking on the but- ton. In the flow of the experiment, the participants sometimes ended up click- 17 2.2. Experimental Apparatus and Software ing the button multiple times. This resulted in false unsuccessful taps (on dou- ble clicks), and falsely timed successful taps (on the third tap following a dou- ble click). Using the raw position data, we filter out most of such cases too. In total, we removed 133 taps from the data, which was 1% of the total number of taps. Figure 2.4: Computing the cursor position from the head and hand-held pointer positions. Our software uses image-plane pointing (Section 2.1.1) to compute the cur- sor position on the screen. The line from the user’s eye to the hand is extrapo- lated to intersect with the virtual screen (corresponding to the virtual distance (DV) of a trial condition). This is then rendered on the physical screen for stereoscopic viewing. This is illustrated in Figure 2.4. 18 2.3. VR Pointing Experiment screen targets Ds = 110cm, Dv = 110cm screen targets Ds = 110cm, Dv = 220cm screen targets Ds = 110cm, Dv = 330cm screentargets Ds = 220cm, Dv = 110cm screen targets Ds = 220cm, Dv = 220cm screen targets Ds = 220cm, Dv = 330cm screentargets Ds = 330cm, Dv = 110cm screentargets Ds = 330cm, Dv = 220cm screen targets Ds = 330cm, Dv = 330cm Figure 2.5: Placement of the targets and the participant in an experimental trial. 2.3 VR Pointing Experiment We performed an experiment to evaluate physical pointing on large stereo- scopic displays for different target depths. The general design of our experi- ment is based on a related experiment of Shoemaker et al. (in press). In their experiment, they evaluate physical pointing on a large display for varying gain values. In our experiment, we evaluate physical pointing for different target depths in 3D and physical distances to the screen. 2.3.1 Task and Stimuli The experimental task was a serial 1-D tapping task between two target pairs, modelled closely after the tasks used by Shoemaker et al. (in press), and the original experiment by Fitts Fitts (1954). Though ISO 9241-9 (Douglas, Kirk- patrick, & MacKenzie, 1999) defines a 2D task for pointing performance, we use a 1-D task because we were concerned with the fundamental applicability of Fitts’s Law. Targets were bars of width W and fixed height, spaced apart by amplitude A. The cursor was a thin (0.5 cm) vertical line with the same height as the 19 2.3. VR Pointing Experiment targets. The cursor position was computed as described in Section 2.2.2. Each trial condition was defined by four independent variables: the move- ment amplitude A, the target width W, virtual depth of the targets DV and the physical distance DS of the participant from the screen. In cases where DV equaled DS, the targets were displayed on the plane of the physical screen (i.e. no stereopsis). In cases where DV was larger than DS, the targets were displayed behind the physical screen (i.e. with positive parallax in stereoscopic viewing), and in cases where DV was smaller than DS, the targets were dis- played in front of the physical screen (i.e. with negative parallax). These cases are illustrated in Figure 2.7. For each target pair, a participant first tapped the start target and then per- formed a sequence of eight back-and-forth taps between the two targets. The active target was highlighted blue and the non-active target was always grey. The participant was required to correctly tap the start target to initiate the trial. Times and locations of all the eight taps were recorded by the software. On a tap, the active target briefly flashed green to indicate success, or red to indicate an incorrect tap. There was no requirement to correct errors. One target was placed directly in front of the participant, and the other target was to its right, at a distance defined by the current amplitude (A) condition. This arrange- ment was chosen to avoid any impact of cross-lateral inhibition effect (CIE) (Kerphart, 1971; Schofield, 1976). This effect takes into account the observation that hand movements that cross the midline of the body are more complex than those that do not involve midline crossing. 2.3.2 Participants We recruited 21 participants through on-campus advertising. As a requirement for participation in the experiment, all were right-handed and had normal or corrected-to-normal vision. One of the participants’ data had to be discarded due to equipment malfunction. Of the remaining 20 participants, 8 were female and 12 were male. Ages ranged from 20 to 28, mean 23.3. All participants were regular computer users (9+ hours per week). They were compensated $10 for participating. 20 2.3. VR Pointing Experiment Figure 2.6: An illustration of the participant performing the pointing task for DS = DV . Figure 2.7: An illustration of the participant performing the pointing task for DS > DV . 21 2.3. VR Pointing Experiment 2.3.3 Design A within-subjects design was used. The independent variables were virtual distance of targets DV (110cm, 220cm, 330cm), distance of the screen from the participant DS (110cm, 220cm, 330cm), target width W (5cm, 10cm, 20cm), and movement amplitude A (25cm, 50cm, 75cm). All the variables were fully crossed. For a given Ds, all three DV conditions were presented consecutively, and for a given DS, DV pair, the participant performed trials for all combina- tions of A and W. We decided against completely mixing up DS to avoid the participant having to move back-and-forth between different positions in the room. The order of the conditions were partially counterbalanced across the participants. In summary, the experimental design was as follows: 20 participants × 3 screen distances (DS) × 3 virtual distances (DV) × 3 movement amplitudes (A) × 3 target widths (W) × 8 taps = 20 participants × 648 taps = 12960 total taps 2.3.4 Procedure The experiment was performed in a single session for each participant, last- ing approximately 45 minutes. Participants filled out a consent form and a pre-questionnaire (Figure A.1 in the appendices) gathering demographic in- formation. They were introduced to the apparatus and the pointing task. The participants were instructed to complete the task as quickly as possible with a goal of ∼95% accuracy. Each of them were then asked to go through a practice session consisting of at least five randomly chosen (DS, DV , A, W) combina- tions (without duplicates, and representing all DS and DV values), and were then invited to practice more until they were comfortable with the system. For every DV and DS pair, a practice A and W pair was presented at the beginning to allow the participant to get used to the new depth conditions (and consequently, the CD gain). This pair was presented in the regular flow of the experiment, and the participant was not informed that this was a practice pair. 22 2.3. VR Pointing Experiment Between each block of the experiment representing a DS value, the participants were required to take a break for at least three minutes. They were encouraged to take more time to rest, but very few did so. After all the trial blocks were completed, the participant filled out a post- questionnaire (Figure A.2 in the appendices) to gather feedback and comments on the experiment. 2.3.5 Measures Pointing performance was measured as the time taken to perform each indi- vidual tap action. The trial for each A and W pair began when the participant tapped the start target successfully, and ended with the eighth tap. For each tap in the experiment, the software recorded the movement time MT, and the position of the click as the offset from the centre of the current target, ∆tap. This is a total of 12960 observations (from the calculations presented in Section 2.3.3). 2.3.6 Hypotheses We derived our hypotheses from related work of Shoemaker et al. and the as- pects of VR and stereoscopy discussed in Chapter 1. H1 One-part formulations (i.e. Fitts and Shannon-Fitts) will not accurately model pointing performance at all distances. H2 Two-part formulations (i.e. Welford and Shannon-Welford) will accurately model pointing performance at all distances. H3a The exponent k will vary monotonically with DV . H3b The exponent k will vary linearly with DV . H4 The screen distance DS has no effect on the pointing performance. 23 2.4. Results Factor Sphericity Correction e dfe f f ect d ferror F p Partial η2 DV Sphericity Assumed 2 38 12.27 0.000 0.392 Greenhouse-Geisser 0.749 1.50 28.46 12.27 0.000 0.392 Huynh-Feldt 0.799 1.90 30.36 12.27 0.000 0.392 A Sphericity Assumed 2 38 369.68 0.000 0.951 Greenhouse-Geisser 0.796 1.59 30.23 369.68 0.000 0.951 Huynh-Feldt 0.857 1.71 32.55 369.68 0.000 0.951 W Sphericity Assumed 2 38 229.72 0.000 0.924 Greenhouse-Geisser 0.554 1.11 21.06 229.72 0.000 0.924 Huynh-Feldt 0.564 1.13 21.42 229.72 0.000 0.924 DV×DS Sphericity Assumed 4 76 36.78 0.000 0.659 Greenhouse-Geisser 0.836 3.34 63.52 36.78 0.000 0.659 Huynh-Feldt 1.000 4.00 76.00 36.78 0.000 0.659 DV×A Sphericity Assumed 4 76 11.20 0.000 0.371 Greenhouse-Geisser 0.660 2.64 50.13 11.20 0.000 0.371 Huynh-Feldt 0.776 3.10 58.95 11.20 0.000 0.371 DV×W Sphericity Assumed 4 76 7.78 0.000 0.291 Greenhouse-Geisser 0.718 2.87 54.60 7.78 0.000 0.291 Huynh-Feldt 0.860 3.44 65.36 7.78 0.000 0.291 A×W Sphericity Assumed 4 76 6.39 0.000 0.252 Greenhouse-Geisser 0.880 3.52 66.90 6.39 0.000 0.252 Huynh-Feldt 1.000 4.00 76.00 6.39 0.000 0.252 Table 2.1: Significant ANOVA results for movement time (MT) 2.4 Results We present the results of our experiment in this section. We follow up with a discussion of their support for our hypotheses in Section 2.5. 2.4.1 ANOVA for Movement Time Significant main effects of DV , DS, A and W were found. Significant interac- tions of DV × DS, DV × A, DV ×W, and A ×W were also found. The sig- nificant results are summarized in Table 2.1, and the effects for all factors are presented in Table A.2. 24 2.4. Results The fact that movement amplitude A and target width W affect pointing performance is fundamental to any discussion of pointing performance. Hence, the finding that there were significant effects of A, W and A×W is not surpris- ing. The finding of a significant effect of DV was also expected because DV af- fects the CD gain for pointing. Similar effects for gain have been found by other researchers (Casiez, Vogel, Balakrishnan, & Cockburn, 2008; Koester, LoPresti, & Simpson, 2005; MacKenzie & Riddersma, 1994; Shoemaker et al., in press). The interaction of DV × A and DV ×W were unexpected, but not surprising, as the role of DV in influencing MT is not entirely understood. The effect of DS on MT was surprising. In an ideal VR setting, the virtual scene perceived by a user should not be affected by the position of the physical screen (provided the images shown are adjusted for the different screen posi- tions appropriately, which we did in our case). Also, the pointing projection is on the virtual target plane at DV , and is independent of DS. 2.4.2 Regression with One-Part and Two-Part Models We performed a linear regression analysis using data averaged over all partici- pants (Table A.1). This is a standard practice (Fitts, 1954; Soukoreff & MacKen- zie, 2004). Using data from individual trials, or from individual participants is very rare in literature, and results in lower R2 values for the models. The regression parameters computed using the Fitts and Welford models are shown in Table 2.2, and those computed using Fitts-Shannon and Welford- Shannon models are shown in Table 2.3. In order to adjust for the accuracy, we performed a similar analysis using the effective target width We instead of W (Zhai, Kong, & Ren, 2004). We computed We using both methods outlined in Section 2.1.5, and the values obtained were similar. In the rest of the analysis, we use the We values computed using the standard deviation of tap positions. The results are presented in Tables 2.4 and 2.5. As discussed in Section 2.1.4, we perform F-tests to compare pairs of nested models. Two-part models were found to characterize the data significantly better for virtual depths 220cm and 330cm than the one-part models. When using effective width We, the one-part models’ performance drops off steeply for DV > 110cm, but the two-part models maintain a constantly good fit. A visu- 25 2.4. Results DV Fitts Welford F-test a b R2 a b1 b2 k R2 F p sig? 110 602.9 517.2 0.976 337.4 567.3 485.0 0.86 0.982 1.94 0.213 no 220 627.7 493.9 0.967 1091.5 406.3 550.1 1.35 0.987 8.78 0.025 yes 330 708.5 533.6 0.961 1358.4 410.9 612.4 1.49 0.993 29.59 0.002 yes Table 2.2: Modelling movement time (ms) using the Fitts and Welford formu- lations using actual width W. DV Shannon-Fitts Shannon-Welford F-test a b R2 a b1 b2 k R2 F p sig? 110 247.4 649.7 0.987 -98.9 722.3 627.1 0.87 0.995 8.65 0.026 yes 220 289.1 619.9 0.976 780.4 516.9 651.9 1.26 0.993 15.57 0.008 yes 330 347.4 667.1 0.962 1067.9 516.0 714.0 1.38 0.993 28.57 0.002 yes Table 2.3: Modelling movement time (ms) using the Fitts-Shannon and Welford-Shannon formulations using actual width W. DV Fitts Welford F-test a b R2 a b1 b2 k R2 F p sig? 110 474.6 634.2 0.972 473.4 634.4 634.0 1.00 0.972 0.00 0.996 no 220 624.1 615.8 0.885 1642.6 484.0 814.4 1.68 0.962 12.23 0.013 yes 330 862.3 681.8 0.741 3304.5 500.1 1293.9 2.59 0.975 56.91 .0002 yes Table 2.4: Modelling movement time (ms) using the Fitts and Welford formu- lations using effective width We. DV Shannon-Fitts Shannon-Welford F-test a b R2 a b1 b2 k R2 F p sig? 110 38.4 803.3 0.982 -3.5 810.7 798.3 0.98 0.982 0.04 0.850 no 220 138.2 821.4 0.897 1234.1 650.0 972.7 1.50 0.970 14.70 0.009 yes 330 256.6 962.6 0.745 2841.2 705.1 1491.8 2.12 0.974 52.68 .0003 yes Table 2.5: Modelling movement time (ms) using the Fitts-Shannon and Welford-Shannon formulations using effective width We. 26 2.4. Results Fitts vs Welford Shannon−Fitts vs Shannon−Welford l l l l l l l l l l l l l l l l l l l l l l 0.96 0.97 0.98 0.99 1.00 0.8 0.9 1.0 Using Actual W idth Using Effe ctive  W idth 110 220 330 110 220 330 Virtual Distance (cm) R −S qu ar ed Model  l lOne−Part (Fitts type)    Two−Part (Welford type) Figure 2.8: Regression R2 values as they vary with virtual distance. alization of R2 values as they vary with DV is shown in Figure 2.8. Additional support for two-part models is found in Section 2.4.3. Based on these results, we discuss our hypotheses and make a case for two-part models in Section 2.5. 2.4.3 The Parameter k as a Function of DV To test the hypothesis that the k values varies based on gain, we performed a linear regression analysis on the k values computed using Welford and Shannon- Welford models, using actual target width W and effective target width We. The results are presented in Table 2.6 and plotted in Figure 2.9. The k values vary linearly (R2 > 0.9) with virtual depth DV . When using effective width, the k values are modelled very accurately using a linear function (R2 > 0.99). 27 2.5. Discussion Welford Shannon−Welford  1.0 1.5 2.0 2.5 1.0 1.5 2.0 2.5 Using Actual W idth Using Effective W idth 110 220 330 110 220 330 Virtual Distance (cm) k ● ● ● ● ● ● ● ● ● ● ● ● Using Actual W idth Using Effective W idth k       R2 = 0.994 adj R2 = 0.987       R2 = 0.997 adj R2 = 0.994       R2 = 0.902 adj R2 = 0.804       R2 = 0.916 adj R2 = 0.832 Figure 2.9: k values calculated using the Welford formulation (a) A, W (b) A, We 2.5 Discussion We summarize the results according to our hypotheses, and then discuss each in more depth. H1 One-part formulations (i.e. Fitts and Shannon-Fitts) will not accurately model pointing performance at all distances. Somewhat supported. H2 Two-part formulations (i.e. Welford and Shannon-Welford) will accurately 28 2.5. Discussion Model Width Used Intercept Slope R2 Adjusted R2 Welford W 0.597 2.89× 10−3 0.902 0.804 Shannon-Welford W 0.656 2.34× 10−3 0.916 0.832 Welford We 0.169 7.22× 10−3 0.994 0.987 Shannon-Welford We 0.401 5.14× 10−3 0.997 0.994 Table 2.6: Linear regression of k and DV (cm) for various cases. model pointing performance at all distances. Supported. H3a The exponent k will vary monotonically with DV . Supported. H3b The exponent k will vary linearly with DV . Supported. H4 The screen distance DS has no effect on the pointing performance. Not supported. 2.5.1 A Closer Look at the Data A visualization of movement time data is shown in Figure 2.10. The graphs in the first column present a scatterplot for different virtual depths DV , with lines connecting points that represent tasks with the same movement amplitude A. The graphs in the second column present the same data, with lines connecting points that represent tasks with the same target width W. The visualization reveals a pattern similar to that shown by Welford (Fig- ure 5.8, page 158 in Fundamentals of Skill, 1971). The movement time increases roughly linearly with ID within either a fixed A or W value, but not across changes in both. This separable effect of A and W increases with increase in DV . A similar effect has been found by Shoemaker et al. (in press) from both reanalysis of other researchers’ data (Graham, 1996; Casiez et al., 2008; Tsuki- tani et al., 2011), as well as from conducting a pointing experiment of their own for varying gain. Both Graham and Shoemaker conclude that Welford’s two- part formulation was necessary to account for this pattern, and to accurately 29 2.5. Discussion model movement time. The applicability of a two-part formulation was fur- ther supported by Graham’s analysis of hand velocity and acceleration profiles during pointing, which revealed separable effects of A and W during different temporal segments of the target acquisition motion. Therefore, we expect one-part models to perform poorly at modelling the movement time data for DV > 110cm, while two-part models should account for the separable effect. As discussed in Section 2.1.3, the k values serve to quantify the separability of contributions of A and W to movement time. From Figure 2.10, we expect k to be close to 1 for a virtual depth DV of 110cm. For increasing DV , we expect k to monotonically increase. 2.5.2 One-Part and Two-part Models for Pointing There is no universally accepted threshold for a ‘good’ R2 value to determine if a formulation accurately models pointing performance. MacKenzie’s sugges- tion (MacKenzie, 1992) of R2 ≥ 0.9 as a guideline when evaluating Fitts’s Law results has been used in the literature, and we employ this as our threshold. One-part models (Fitts and Shannon-Fitts) were successful in characterizing movement time when using the actual target widths W. The Fitts formulation produced a fitting accuracy ranging from R2 = 0.961 to R2 = 0.976 at different virtual depths. The Shannon-Fitts formulation was slightly better at modelling the data, with accuracy ranging from R2 = 0.987 to R2 = 0.962. These R2 values beat Mackenzie’s 0.9 threshold. However, for both formulations, it is clear that the R2 values are decreasing as DV increases (Figure 2.8). When using the effective width We, the one-part models were unsuccessful. For both formulations, the R2 values decrease steeply as DV increases. Using the Fitts formulation, the accuracy ranges from R2 = 0.741 to R2 = 0.972. Us- ing MacKenzie’s threshold of 0.9, we find that the Fitts formulation fails to pro- duce an acceptable fit for virtual depths DV of 220cm and 330cm. The Shannon- Fitts formulation fares slightly better, with accuracy ranging from R2 = 0.745 to R2 = 0.982. Again, Shannon-Fitts fails to produce acceptable fits for virtual depths DV of 220cm and 330cm. The results using effective widths are more relevant since We is a more ac- curate representation of the task (Soukoreff & MacKenzie, 2004). The reason for the poorer fit using effective width We is apparent from examining the k 30 2.5. Discussion l l l l l l l l l l l l l l l l l l l l l l l l l l l 500 1000 1500 2000 2500 500 1000 1500 2000 2500 500 1000 1500 2000 2500 110 220 330 0.0 0.5 1.0 1.5 2.0 2.5 Effective ID M ov e m e n t T im e (s) A l l l25cm 50cm 75cm (Grouped by A) l l l l l l l l l l l l l l l l l l l l l l l l l l l 500 1000 1500 2000 2500 500 1000 1500 2000 2500 500 1000 1500 2000 2500 110 220 330 0.0 0.5 1.0 1.5 2.0 2.5 Effective ID M ov e m e n t T im e (s) W l l l5cm 10cm 20cm (Grouped by W) Figure 2.10: Scatterplot of MT vs. effective ID (Note that the points are iden- tical in the right and left plots. Points are connected in two different ways to illustrate the separability effect): (a) Lines connect points representing tasks with the same movement amplitude A, (b) Lines connect points representing tasks with the same target width W 31 2.5. Discussion values from the two-part models (Figure 2.9). It is clear that k deviates much more from 1 when using We than it does when using W. As we have discussed, the more k deviates from 1, the worse a one-part model will be at describing the data. We thus conclude that hypothesis H1 is somewhat supported. The results behave as expected when using either W or We; the quality of fit decreases for increasing DV . When using We, the one-part models result in unacceptable R2 values for DV greater than 110cm. Both two-part models (Welford and Shannon-Welford) produced a consis- tently good fit at every virtual depth DV , both when using actual target widths W and when using effective widths We. The Welford formulation, when us- ing actual target widths W, produced regression fits ranging from R2 = 0.982 to R2 = 0.993. When effective widths We were used instead, the regression fits ranged from R2 = 0.962 to R2 = 0.975. The Shannon-Welford formu- lation, when using actual target widths W, produced regression fits ranging from R2 = 0.993 to R2 = 0.995. When effective widths We were used instead, the regression fits ranged from R2 = 0.970 to R2 = 0.982. More importantly, the two-part models were found to describe the data significantly better than corresponding one-part models for DV > 110cm (see F-test results in Table 2.2, Table 2.3, Table 2.4 and Table 2.5). Thus, we conclude that hypothesis H2 is supported by our data. 2.5.3 Parameter k as Virtual Distance DV Varies The k values vary monotonically and linearly with DV . For analyses using actual target widths W, the k values fit a linear model with R2 = 0.902 for the Welford formulation and R2 = 0.916 for the Shannon-Welford formulation. For analyses using effective target widths We, the k values were even more accurately modelled, with R2 = 0.994 for the Welford formulation and R2 = 0.997 for the Shannon-Welford formulation. Therefore, we conclude that our hypotheses H3a and H3b are supported. 32 2.5. Discussion Figure 2.11: Calibrating the virtual depth based on k values 33 2.5. Discussion 2.5.4 Calibrating VR Systems with Pointing Tasks In a perceptually ‘perfect’ VR system, the perceived environment and the point- ing performance should be invariant of the physical screen distance DS, as long as the virtual depth DV is held constant. From our ANOVA, we find that this is not the case. An effect of DS on movement time was found with F1.85,35.08 = 6.50, p = 0.005. In trials with DV = DS, it is identical to viewing and interacting with the large display in 2D (without any stereopsis) from a distance DS. Hence, there is no distortion associated with stereoscopic viewing in this case. The k values obtained from these trials can therefore be considered to be a ‘correct’ mapping of the pointing performance to the distance DV . For a condition with DV 6= DS, we can estimate a corrected value DcV based on the k value for the condition. For the case DS = 110cm, DV = 330cm, the k-value is 1.3, which represents the pointing task performance. Our assumption is that the DV values are unre- liable, while the k-values are. We adjust the DV using the linear fit line shown in the figure. The distance value corresponding to a k-value of 1.3 on this line is 280cm. Hence, DcV for this case is 280cm. The calibration for the rest of the DV-DS combinations is illustrated in Figure 2.11. This DcV is the calibrated virtual distance that would produce pointing per- formance characteristics identical to that with a physical distance DS = DV . The sample size we have is insufficient to make this calculation with sufficient confidence. 2.5.5 Using the Calibrated Values Once we have the calibrated values DCV , we can use them to obtain real-world performance for the task on which the calibration was performed. We do this by inverting the calibration tables (or function). For example, in a virtual environment, where the user is at a known dis- tance from the screen (DS), where we want to measure real world pointing performance at an object that is at a distance l from the user, the display system would use DCV = l to determine a new value l ′ = DV , where DV is the virtual distance whose calibrated equivalent is l. This achieves exactly what we want. We know, from our calibration experiment, that when we use DV = l′, the 34 2.6. Conclusions performance corresponds to DCV = l, which is the performance we wanted to measure. We have thus overcome the shortcomings caused by the mismatch between performance in a virtual environment and performance in the real world. 2.6 Conclusions Traditionally, Fitts’s Law has been the first model of choice for evaluating point- ing performance in new displays and interaction techniques. Displays for Vir- tual Reality have relatively unexplored form factors and input methods. Our contributions in this chapter are two-fold: (1) studying models of pointing per- formance for VR, and (2) developing a method for ‘calibrating’ VR systems. We performed an experiment studying pointing performance in Virtual Re- ality setups, and compared Fitts-like one-part models and Welford-like two- part models. We found that two-part models significantly outperform one-part models in characterizing pointing performance across varying target depths. We also found that the relative magnitude of the separable contributions of A and W to movement time, captured by a parameter k, varies linearly with the target depth. We describe a method for calibrating VR systems in order to produce pointing performance identical to that in ‘real’ environments. We outline a future experiment validating this calibration process. 35 Chapter 3 Multiple Object Tracking in VR In virtual reality (VR), users often have to keep track of multiple objects in the environment. This is particularly important in applications such as stereo- scopic displays for air traffic control or military operations. Understanding the characteristics of multiple object tracking can help in designing displays and visualization techniques for VR. Our research builds on the work of Liu et al. (2005). They investigated multiple object tracking performance across varying object and scene speeds, as well as different scene coherence conditions. Scene coherence refers to how easily a given scene can be perceptually mapped to a 3D space by an observer. Scene incoherence may result from, among other things, distortion of depth and disparities between visual cues. In this chapter, we design an experiment to study multiple object tracking performance across 2D and stereoscopic 3D viewing conditions; and across various scene coherence conditions. We report results from a pilot experiment and outline future experiments. Unlike pointing, which was discussed in Chapter 2, multiple object track- ing does not involve the full human sensorimotor system. It is largely based on perception and cognition. We therefore should not assume that the real- ism of a virtual environment will be the same for pointing tasks and multiple object tracking tasks. The preliminary investigation reported in this chapter is intended to explore this further and determine the next steps required in a full research study. It is possible that different calibration criteria might exist for optimal pointing performance vs. optimal tracking performance. If that is true, the techniques described here could form the basis for task-specific cali- bration for multiple object tracking in virtual environments. 36 3.1. Related Work Section 3.1 presents the background and relevant literature. In Section 3.2.1, we introduce the apparatus and software used in our experiments. We describe the design of the MOT experiment and report results from a pilot study in Section 3.3. Finally, we present directions for future experiments in Section 3.4. 3.1 Related Work Relevant related work lies in the areas of virtual reality research and visual perception. 3.1.1 Multiple Object Tracking Multiple Object Tracking (MOT) is the ability to track multiple randomly mov- ing objects based solely on their spatiotemporal history. This was introduced to test a perception mechanism called Visual Index (Pylyshyn & Storm, 1988). Re- search studying this ability has since been used to investigate various aspects of visual perception, like perceptual organization (Yantis, 1992), attention to depth (Viswanathan & Mingolla, 2002) and the nature of visual object repre- sentations (Scholl & Pylyshyn, 1999). In literature, we find extensive research on human MOT performance and underlying mechanisms. It has been found that targets can be tracked with- out significant decrease in performance in the presence of occluders or distur- bances (Scholl & Pylyshyn, 1999; Horowitz, Birnkrant, Fencsik, Tran, & Wolfe, 2006), in presence of non-predictable motion (Keane & Pylyshyn, 2006), when the targets are indistinguishable from distractor objects (Scholl, Pylyshyn, & Franconeri, 1999; Bahrami, 2003) and if the entire scene undergoes transforma- tion (Liu et al., 2005). MOT has been shown to be non-attentive (Leonard & Pylyshyn, 2003) and allocentric (Liu et al., 2005). 3.1.2 Allocentric Tracking and Scene Stability Our work follows the work of Liu et al. which studies MOT across scene trans- formations, i.e. moving the entire scene of objects in complex ways in 3D across the viewing screen while the objects move randomly within the scene ‘box’. 37 3.2. Experimental Apparatus and Software They found that these transformations had negligible effects on the tracking ac- curacy and conclude that the tracking is performed using an allocentric (scene- based) frame of reference, not retinal frame-based. Another aspect of allocentric tracking is the stability of the perceived 3D environment. Liu et al. looked at the tracking accuracy when all the visual cues (bounding boxes, axes) were removed in the MOT experiment. They found that tracking is still allocentric, with accuracy comparable to when the visual cues were visible. The motion patterns of the objects were sufficient to allow observers to perceive a coherent 3D environment. The scene stability was directly reduced by projecting the image onto the junction of two dihedral surfaces. Though the retinal projection was identical, the tracking accuracy was significantly reduced under this condition. When the observer was off-axis compared to the projector, the accuracy was much worse. 3.1.3 Multiple Object Tracking in VR The finding that tracking is robust despite scene transformations is important for the design of shared-user environments. It has also been found that the tracking is unaffected by unpredictable motion (Pylyshyn & Storm, 1988; Yan- tis, 1992) or even small but abrupt viewpoint changes (Huff, Jahn, & Schwan, 2009), as would be the case in shared environments. An important requirement for tracking, as mentioned in Section 3.1.2, is the stability of the scene. Stereoscopic displays used in VR systems have an inher- ent depth disparity (Wann, Rushton, & Mon-Williams, 1995; Ogle, 1953). The question of whether this affects the stability of the scene and tracking accuracy has not been studied. 3.2 Experimental Apparatus and Software The experimental apparatus forms a part of the VR laboratory at the Univer- sity of British Columbia, and is similar to the setup used in Chapter 2. We developed experimental software for displaying the stereoscopic MOT scene, tracking the subjects for head-coupled VR and logging relevant data. We de- 38 3.2. Experimental Apparatus and Software scribe these in detail in this section. 3.2.1 Apparatus We use a display made of a glass screen of 5.16m×2.85m in size and rear- projected by a 4×3 array of 800px×600px stereo projectors. The images of adjacent projectors overlap by 160px with a blending function to minimize the appearance of discontinuities. The overall resolution of the display is therefore 2720px×1480px. The entire display is frame-sequential stereoscopic at 60Hz. Users view this display through shutter glasses that are synchronized with the projectors. The display is driven by a computer with an 8-core Intel Processor, 6GB of RAM, dual NVIDIA GeForce GTX 260 graphics processors and running Windows 7. An arrangement of five Vicon cameras (high speed infrared-based motion tracking system) are used to track the user’s head position. We use head gear fitted with specially-designed reflective balls as shown in Figure 3.1. With these, we can track the participant’s head position and orientation accurately in the room. We use this to change the displayed scene perspective in real-time to produce head-coupled VR. Figure 3.1: Apparatus: (a) Large screen stereoscopic display (d) Head gear 3.2.2 Software The experimental software is written in C# using the Microsoft XNA Game Studio 4.1 library. The software receives a stream of position and orientation data from the Vicon motion capture system, and uses this to compute the per- spective for the displayed scene. For each trial, the software records the exper- 39 3.3. Multiple Object Tracking Experiment Figure 3.2: A screenshot of the MOT experiment. imental conditions, and the user’s response and timing. In addition, we also log the user’s head position every (1/60)th of a second. 3.3 Multiple Object Tracking Experiment We designed an experiment to evaluate multiple object tracking on large stereo- scopic displays for different viewing conditions. The general design of our experiments are based on the original MOT experimental setup (Pylyshyn & Storm, 1988) and that of Liu et al. (2005). 3.3.1 Task and Stimuli Our experimental scene is shown in Figure 3.2. A set of n (in our pilot exper- iments, 15) identical objects are shown moving around inside a 3D cube of 450px width (a visual angle of 22◦ at the screen plane). The surface of the cube is translucent (opacity = 0.2). In future experiments, the cube could be made 40 3.3. Multiple Object Tracking Experiment fully transparent. The objects are spheres of 15px in diameter (a visual angle of 0.74◦ at the screen plane). They move in straight lines, at 90px/s, and bounce off each other and off the walls on a collision. The scene box containing the moving objects undergoes motion involving simultaneous changes in translation, rotation and scale. Each of these trans- formations, considered separately, follows a sinusoidal function so that the changes in direction are smooth. We varied the speeds in our pilot experiments to arrive at a suitable set of parameters for our experiment. // rotation sceneTransformationMatrix = Matrix.CreateFromYawPitchRoll( t * 0.134, t*0.356, time*0.78); // scale sceneTransformationMatrix = Matrix.CreateScale(Math.Sin(t) / 2.0 + 1.0) // translation sceneTransformationMatrix = Matrix.CreateTranslation( Math.Sin(t*1.1) * 100.0, Math.Cos(t*2.0) * 100.0, Math.Sin(time * 0.9 + Math.PI / 4) * 100.0); The four successive stages of each trial are as follows. These are illustrated in Figure 3.3. 1. Rest (user) At the beginning of the trial, the n objects are assigned random positions and movement directions in the cube. The objects start moving. When the participant is ready, they press a button, which starts the highlight phase. 41 3.3. Multiple Object Tracking Experiment Figure 3.3: Various stages of the MOT experiment: (1) Rest, (2) Highlight, (3) Track, and (4) Probe. 2. Highlight (4s) Of the n objects, nT (based on the condition) are highlighted for 4s. This designated the target set that the participants are required to track through the tracking phase. 3. Track (10s) The highlighted objects return back to normal and are indistinguishable from the other objects. All the objects continue moving for 10s. The par- ticipants are required to follow the original set of nT highlighted objects through this phase. 42 3.3. Multiple Object Tracking Experiment screen screen Figure 3.4: 2D (flat) and 3D (stereoscopic) conditions 4. Probe (user) One of the n objects is highlighted, and the participant is asked to re- spond whether this probe object was part of the original set of nT objects or not. The participants are instructed to respond accurately, but to guess when uncertain. 3.3.2 Display Conditions In our pilot experiments, we compared MOT performance between 2D (flat) and 3D (stereoscopic) viewing conditions. The primary aim of this compar- ison was to find out if tracking performance was affected by the additional dimension and the distortions associated with stereoscopic viewing. Figure 3.4 illustrates these two conditions. Liu et al. (2005) controlled scene coherence by projecting the image onto the junction of two dihedral surfaces. Pointing performance was compared for on- axis and off-axis conditions. We replicate these scenarios by simulating them in VR. Figure 3.5 illustrates these conditions. 3.3.3 Participants We recruited five participants for our pilot study. As a requirement for par- ticipation in the experiment, they had normal or corrected-to-normal vision. Three were male, two were female. Ages ranged from 25 to 44 (mean 31). 43 3.3. Multiple Object Tracking Experiment screen screen On-Axis O-Axis Figure 3.5: Controlling scene stability by projecting onto a dihedral surface. All participants were regular computer users (9+ hours per week). They were compensated $10 for participating. 3.3.4 Design A within-subjects design was used. The independent variables were the num- ber of objects to be tracked nT (2, 3, 4, 5) and the type of display D (non- stereoscopic (flat), stereoscopic (3D)). For each condition, there was one trial where the probe belonged to the tracked set, and another where the probe did not. The order of the conditions was randomized. The experiment was per- formed in two fully-crossed blocks. In summary, the design was as follows: 5 pilot participants × 2 blocks × 4 sets of objects to be tracked (nT) × 2 display types (D) × 2 probe types = 5 pilot participants × 32 trials = 160 trials 3.3.5 Procedure The experiment was performed in a single session for each participant, last- ing approximately 45 minutes. Participants filled out a consent form and a pre-questionnaire (Figure B.1 in the appendices) gathering demographic in- formation. They were introduced to the tracking task and the apparatus. They performed a set of practice trials followed by the experimental blocks. Between 44 3.3. Multiple Object Tracking Experiment each trial, there was a break of 15s. They were encouraged to take more time to rest, but few did so. Following the trial blocks, the participants filled out a post-questionnaire (Figure B.2 in the appendices) to gather feedback and comments on the exper- iment. 3.3.6 Measures The first measure of MOT performance we examine is the tracking accuracy. This is measured as the proportion of correct responses to the probe for each trial condition. pcorrect = Ncorrect Ntotal (3.1) Second, we use a measure of capacity (K) (Liu et al., 2005) to evaluate MOT performance. K = nT ( p+correct − p+incorrect 1− p+incorrect ) (3.2) Here, p+correct is the proportion of positive responses that are correct, and p+incorrect is the proportion of positive responses that are incorrect. For a given condition, the upper limit of K is nT . 3.3.7 Indications from the Pilot Study We performed an analysis of variance (ANOVA) on the within-subjects factors nT (2,3,4,5) and the display D (non-stereoscopic, stereoscopic). The results of the analysis are summarized in Table 3.1. A significant main effect of nT was found on the tracking accuracy pcorrect. The accuracy decreases with increasing nT , which agrees with the results ob- tained by Liu et al. (2005). A plot of the accuracy values is shown in Figure 3.6. No effect of the display type D was found. It is possible that the effect might be significant with a larger number of subjects. The overall tracking accuracy for the experiment is 0.79 for all conditions (0.78 for the flat conditions, 0.80 for the 45 3.4. Conclusions Factor Sphericity Correction e dfe f f ect d ferror F p Partial η2 nT Sphericity Assumed 3 12 4.844 0.020 0.548 Greenhouse-Geisser 0.515 1.55 6.18 4.844 0.060 0.548 Huynh-Feldt 0.777 2.33 9.32 4.844 0.032 0.548 D Sphericity Assumed 1 4 0.151 0.717 0.035 Greenhouse-Geisser 1.000 1.00 4.00 0.151 0.717 0.035 Huynh-Feldt 1.000 1.00 4.00 0.151 0.717 0.035 nT × D Sphericity Assumed 3 12 0.258 0.854 0.060 Greenhouse-Geisser 0.606 1.82 7.28 0.258 0.760 0.060 Huynh-Feldt 1.000 3.00 12.00 0.258 0.854 0.060 Table 3.1: Significant ANOVA results for MOT tracking accuracy (pcorrect) stereo conditions). We think that this configuration of experimental parameters is ideal for the research questions we’re trying to investigate. 3.4 Conclusions In this chapter, we described the design of a multiple object tracking (MOT) experiment for VR. We performed a pilot study with five subjects to test the feasibility of the experiment. Our results indicate that MOT accuracy decreases with increasing number of tracked objects in VR. The results were insufficient to show if the stereo display conditions affect MOT accuracy. We think that with a full-scale experiment we might be able to answer this conclusively. Most importantly, we wish to study how scene coherence, as measured by track- ing performance, is affected by the inherent disparities in stereoscopic viewing conditions. We describe future experiments to measure this disparity and de- velop techniques to calibrate a VR system to produce MOT performance com- parable to that in real-world environments. Once this has been done, similar to what we saw for pointing, we may be able to use the calibration data we obtain to adjust for performance differences in a virtual environment and the real world. In this case, the adjustment will be for MOT performance, not pointing performance. One of the interesting and open research questions is whether different tasks, such as pointing and MOT, will have different calibration functions. This remains to be explored. 46 3.4. Conclusions ll l l l l l l 0.4 0.6 0.8 1.0 2 3 4 5 Number of Tracked Objects Ac cu ra cy Viewing Condition l lFlat Stereo Figure 3.6: Plot of accuracy values and number of tracked objects for different viewing conditions. 47 Chapter 4 Conclusions In Virtual Reality (VR) systems, it is important that the perceived virtual envi- ronment matches with the abstract virtual space. A VR system maps a virtual space to a perceived environment through the virtual image (as defined in Sec- tion 1.1). We aim to create an environment that agrees with our virtual space, and produces task performance from users that is identical to a real-life task. Figure 4.1: Calibrating VR systems based on task performance. We do this by calibrating the virtual image in the VR system for a specific task or application. Given a VR setup, we measure the performance for a task and compare it against the expected real-life performance characteristics. If we can devise a method to modify the virtual image in such a way that the new calibrated image produces the expected performance, we have a VR system that is calibrated for the task. This is illustrated in Figure 4.1. 48 4.1. Research Contributions 4.1 Research Contributions In this thesis, we have investigated pointing and multiple object tracking per- formance in VR. We have also described how a VR system can be calibrated, and indicated directions for future research. The following is a summary of our research contributions: 1. Pointing in VR. • We showed that accepted one-part models of pointing (such as Fitts’s Law) do not adequately describe pointing performance in VR. • We showed that two-part models of pointing (such as Welford’s Law) accurately describe pointing performance in VR across all vir- tual distances. • We experimentally derived a parameter k that describes the relative magnitude of the separable contributions of movement amplitude A and target width W to the movement time. We showed that this parameter k varies linearly with the virtual distance. • We described a method that uses the k values to calibrate a VR sys- tem. 2. Multiple Object Tracking in VR. • We designed and implemented an experiment to study multiple ob- ject tracking in VR across different viewing conditions. • We conducted a pilot study to compare MOT performance between non-stereoscopic (flat) and stereoscopic viewing conditions. • Results from our pilot experiment follow the expected MOT per- formance trends. Though the data is insufficient to validate MOT- based calibration techniques, we expect future experiments to be ef- fective in doing so. 4.2 Future Work In Chapter 2, we suggested a future experiment to demonstrate the calibra- tion process based on pointing in VR. Our knowledge of how k varies with 49 4.2. Future Work DV is not yet complete. It would be worthwhile to investigate how the linear function differs with different display setups and input methods. In our work, we have used image-plane pointing as our pointing technique. Future work should investigate pointing methods based on angular movements (like laser pointing) and more complex techniques (like shadow reaching) in VR. In Chapter 3, we report a pilot study comparing MOT performance between non-stereoscopic and stereoscopic viewing conditions. In future work, exper- iments comparing performance across different viewing conditions should be performed. Once we have an understanding of how MOT performance be- haves in VR, we need to develop calibration techniques based on MOT tasks in VR. An important aspect of our work is looking at task-specific calibration for VR. Previous work on VR calibration is largely based on perceptual models of stereoscopic viewing – either derived from literature, or from user-reported depth values (Jones et al., 2001; Yuan et al., 2011). These techniques aim to reduce the distortions associated with stereoscopic viewing. Often in VR, it is desirable to produce task performance identical to that in real-life tasks. Cal- ibration techniques based on perceptual models fail to do this. Task-specific calibration is designed to ‘calibrate’ a VR system for a given task, such that the resulting performance characteristics match with the expected real-world performance. In many applications of VR, a user is required to interact with the system simultaneously through different means. For example, they could be tracking objects on the screen while performing pointing and input tasks; and there has been research suggesting that finger tapping and tracking might share a common resource (Trick, Guindon, & Vallis, 2006). It is therefore important to study how one task might affect the performance in the other. 50 References BAHRAMI, B. (2003). Object property encoding and change blindness in mul- tiple object tracking. Visual Cognition, 10(8), 949–963. CASIEZ, G., VOGEL, D., BALAKRISHNAN, R., & COCKBURN, A. (2008). The impact of control-display gain on user performance in pointing tasks. Human–Computer Interaction, 23(3), 215–250. DOUGLAS, S. A., KIRKPATRICK, A. E., & MACKENZIE, I. S. (1999). Test- ing pointing device performance and user assessment with the iso 9241, part 9 standard. In Proceedings of the sigchi conference on human factors in computing systems: the chi is the limit (pp. 215–222). New York, NY, USA: ACM. ELLIS, S. (1991, May). Nature and origins of virtual environments: A biblio- graphical essay. Computing Systems in Engineering, 2(4), 321–347. FITTS, P. M. (1954). The information capacity of the human motor system in controlling the amplitude of movement. Journal of Experimental Psychol- ogy, 47(6), 381–391. GRAHAM, E. D. (1996). Pointing on a computer display. Unpublished doctoral dissertation, Simon Fraser University. GRAHAM, E. D., & MACKENZIE, C. L. (1996). Physical versus virtual point- ing. In Proceedings of the sigchi conference on human factors in computing systems: common ground (pp. 292–299). New York, NY, USA: ACM. HOROWITZ, T., BIRNKRANT, R., FENCSIK, D., TRAN, L., & WOLFE, J. (2006). How do we track invisible objects? Psychonomic Bulletin & Review, 13(3), 51 References 516–523. HUFF, M., JAHN, G., & SCHWAN, S. (2009). Tracking multiple objects across abrupt viewpoint changes. Visual Cognition, 17(3), 297–306. IYER, K. N., CHARI, M., & KANNAN, H. (2008). A novel approach to depth image based rendering based on non-uniform scaling of depth values. In Proceedings of the 2008 second international conference on future generation communication and networking symposia - volume 03 (pp. 31–34). Washing- ton, DC, USA: IEEE Computer Society. JONES, G. R., LEE, D., HOLLIMAN, N. S., & EZRA, D. (2001, June). Controlling Perceived Depth in Stereoscopic Images. Proc. SPIE Stereoscopic Displays and Virtual Reality Systems VIII, 4297, 42–53. JOTA, R., NACENTA, M. A., JORGE, J. A., CARPENDALE, S., & GREENBERG, S. (2010). A comparison of ray pointing techniques for very large dis- plays. In Proceedings of graphics interface 2010 (pp. 269–276). Toronto, Ont., Canada: Canadian Information Processing Society. KEANE, B., & PYLYSHYN, Z. (2006). Is motion extrapolation employed in multiple object tracking? tracking as a low-level, non-predictive function. Cognitive Psychology, 52(4), 346–368. KERPHART, N. C. (1971). The slow learner in the classroom (2nd ed.). Columbus, OH: Merrill. KOESTER, H. H., LOPRESTI, E., & SIMPSON, R. C. (2005). Toward goldilocks’ pointing device: determining a ”just right” gain setting for users with physical impairments. In Proceedings of the 7th international acm sigaccess conference on computers and accessibility (pp. 84–89). New York, NY, USA: ACM. KOPPER, R., BOWMAN, D. A., SILVA, M. G., & MCMAHAN, R. P. (2010, Oc- tober). A human motor behavior model for distal pointing tasks. Int. J. Hum.-Comput. Stud., 68(10), 603–615. 52 References LEONARD, C., & PYLYSHYN, Z. (2003). Measuring the attentional demand of multiple object tracking (mot). Journal of Vision, 3(9), 582. LIU, G., AUSTEN, E., BOOTH, K., FISHER, B., ARGUE, R., REMPEL, M., & ENNS, J. (2005). Multiple-object tracking is based on scene, not retinal, coordinates. Journal of Experimental Psychology: Human Perception and Per- formance, 31(2), 235. MACKENZIE, I. S. (1991). Fitts’ law as a performance model in human-computer interaction. Unpublished doctoral dissertation, University of Toronto, Toronto, Ontario, Canada. MACKENZIE, I. S. (1992, March). Fitts’ law as a research and design tool in human-computer interaction. Human–Computer Interaction, 7(1), 91– 139. MACKENZIE, I. S., & RIDDERSMA, S. (1994). Effects of output display and control-display gain on human performance in interactive system. Be- haviour and Information Technology, 13, 328–337. OGLE, K. N. (1953, Oct). Precision and validity of stereoscopic depth percep- tion from double images. Journal of the Optical Society of America, 43(10), 906–913. PYLYSHYN, Z. W., & STORM, R. W. (1988). Tracking multiple independent targets: Evidence for a parallel tracking mechanism. Spatial Vision, 3, 179–197. SCHOFIELD, W. N. (1976). Do children find movements which cross the body midline difficult? Quarterly Journal of Experimental Psychology, 28(4), 571- 582. SCHOLL, B., & PYLYSHYN, Z. (1999). Tracking multiple items through occlu- sion: Clues to visual objecthood. Cognitive Psychology, 38, 259–290. SCHOLL, B., PYLYSHYN, Z., & FRANCONERI, S. (1999). When are featural and spatiotemporal properties encoded as a result of attentional allocation? 53 References Investigative Ophthalmology & Visual Science, 40(4), 4195. SHOEMAKER, G., TANG, A., & BOOTH, K. S. (2007). Shadow reaching: a new perspective on interaction for large displays. In Proceedings of the 20th annual acm symposium on user interface software and technology (pp. 53–56). New York, NY, USA: ACM. SHOEMAKER, G., TSUKITANI, T., KITAMURA, Y., & BOOTH, K. S. (in press). Two-part models capture the impact of gain on pointing performance. ACM Transactions on Computer-Human Inteaction. SOUKOREFF, R. W., & MACKENZIE, I. S. (2004, December). Towards a stan- dard for pointing device evaluation, perspectives on 27 years of fitts’ law research in hci. Int. J. Hum.-Comput. Stud., 61(6), 751–789. TRICK, L. M., GUINDON, J., & VALLIS, L. A. (2006). Sequential tapping in- terferes selectively with multiple-object tracking: Do finger-tapping and tracking share a common resource? The Quarterly Journal of Experimental Psychology, 59(7), 1188–1195. TSUKITANI, T., SHOEMAKER, G., BOOTH, K., TAKASHIMA, K., ITOH, Y., KI- TAMURA, Y., & KISHINO, F. (2011, April). A fitts law analysis of shadow metaphor mid-air pointing on a very large wall display. Information Pro- cessing Society of Japan Journal, 52(4), 1495–1503. VISWANATHAN, L., & MINGOLLA, E. (2002). Dynamics of attention in depth: Evidence from multi-element tracking. (Tech. Rep. Nos. CAS/CNS-98–012)). Boston: Boston University Center for Adaptive Systems, Department of Cognitive and Neural Systems. WANN, J. P., RUSHTON, S., & MON-WILLIAMS, M. (1995). Natural prob- lems for stereoscopic depth perception in virtual environments. Vision Research, 35(19), 2731 - 2736. WELFORD, A. T. (1968). Fundamentals of skill. London: Methuen. Wiimotelib - .NET managed library for the Nintendo Wii remote. (2012). Retrieved 54 from www.wiimotelib.org WOBBROCK, J. O., CUTRELL, E., HARADA, S., & MACKENZIE, I. S. (2008). An error model for pointing based on fitts’ law. In Proceedings of the twenty- sixth annual sigchi conference on human factors in computing systems (pp. 1613–1622). New York, NY, USA: ACM. YANTIS, S. (1992). Multielement visual tracking: Attention and perceptual organization. Cognitive Psychology, 24, 295–340. YUAN, C., PAN, H., & DALY, S. (2011). Stereoscopic 3d content depth tun- ing guided by human visual models. SID Symposium Digest of Technical Papers, 42(1), 916–919. ZHAI, S., KONG, J., & REN, X. (2004, December). Speed-accuracy tradeoff in fitts’ law tasks: on the equivalency of actual and nominal pointing precision. Int. J. Hum.-Comput. Stud., 61(6), 823–856. 55 Appendix A Pointing Experiment Data and Questionnaires Table A.1: Raw performance results for the experiment described in Chapter 2. The movements times (MT) were obtained by av- eraging over all subjects for each combination of Dv, Ds, A and W. Independent Observed Dv (cm) Ds (cm) A (cm) W (cm) We (cm) MT (ms) 110 110 25 5 6.84 1264 110 110 25 10 8.79 920 110 110 25 20 16.54 693 110 110 50 5 6.16 1623 110 110 50 10 11.09 1233 110 110 50 20 17.58 981 110 110 75 5 7.07 1830 110 110 75 10 12.09 1436 110 110 75 20 19.18 1131 110 220 25 5 10.50 1862 110 220 25 10 13.47 1249 110 220 25 20 22.18 872 110 220 50 5 11.64 2061 110 220 50 10 14.79 1426 110 220 50 20 23.70 1144 110 220 75 5 12.11 2280 Continued on next page 56 Appendix A. Pointing Experiment Data and Questionnaires Table A.1 – continued from previous page Independent Observed Dv (cm) Ds (cm) A (cm) W (cm) We (cm) MT (ms) 110 220 75 10 17.03 1743 110 220 75 20 26.04 1282 110 330 25 5 24.14 2000 110 330 25 10 22.99 1474 110 330 25 20 28.70 996 110 330 50 5 17.28 2165 110 330 50 10 22.11 1814 110 330 50 20 30.46 1315 110 330 75 5 19.70 2569 110 330 75 10 22.69 1961 110 330 75 20 29.58 1558 220 110 25 5 5.55 1372 220 110 25 10 10.82 1028 220 110 25 20 16.54 856 220 110 50 5 6.28 1679 220 110 50 10 9.59 1337 220 110 50 20 15.11 1050 220 110 75 5 6.51 2013 220 110 75 10 12.09 1623 220 110 75 20 17.58 1325 220 220 25 5 7.84 1400 220 220 25 10 11.35 1014 220 220 25 20 17.58 779 220 220 50 5 7.40 1713 220 220 50 10 10.82 1265 220 220 50 20 21.64 974 220 220 75 5 8.74 1910 220 220 75 10 13.24 1535 220 220 75 20 22.18 1179 220 330 25 5 9.71 1690 Continued on next page 57 Appendix A. Pointing Experiment Data and Questionnaires Table A.1 – continued from previous page Independent Observed Dv (cm) Ds (cm) A (cm) W (cm) We (cm) MT (ms) 220 330 25 10 14.79 1271 220 330 25 20 19.18 849 220 330 50 5 11.95 1805 220 330 50 10 14.79 1458 220 330 50 20 22.18 1099 220 330 75 5 13.12 2095 220 330 75 10 16.80 1628 220 330 75 20 24.18 1219 330 110 25 5 6.62 1477 330 110 25 10 9.93 1101 330 110 25 20 18.43 860 330 110 50 5 6.05 1976 330 110 50 10 10.54 1533 330 110 50 20 19.86 1153 330 110 75 5 6.73 2412 330 110 75 10 11.35 1953 330 110 75 20 22.18 1558 330 220 25 5 6.84 1405 330 220 25 10 12.56 1042 330 220 25 20 20.49 785 330 220 50 5 6.73 1643 330 220 50 10 10.82 1320 330 220 50 20 19.86 964 330 220 75 5 7.18 1949 330 220 75 10 12.33 1478 330 220 75 20 19.18 1154 330 330 25 5 9.46 1662 330 330 25 10 12.79 1064 330 330 25 20 17.58 771 330 330 50 5 9.71 1900 330 330 50 10 14.57 1345 Continued on next page 58 Appendix A. Pointing Experiment Data and Questionnaires Table A.1 – continued from previous page Independent Observed Dv (cm) Ds (cm) A (cm) W (cm) We (cm) MT (ms) 330 330 50 20 22.18 1034 330 330 75 5 9.71 2112 330 330 75 10 13.02 1660 330 330 75 20 22.70 1142 59 Appendix A. Pointing Experiment Data and Questionnaires Table A.2: ANOVA results for movement time (MT) Factor Corr. e dfe f f ect d ferror F p Partial η2 DV - 2 38 12.27 0.000 0.392 GG 0.749 1.50 28.46 12.27 0.000 0.392 HF 0.799 1.90 30.36 12.27 0.000 0.392 A - 2 38 369.68 0.000 0.951 GG 0.796 1.59 30.23 369.68 0.000 0.951 HF 0.857 1.71 32.55 369.68 0.000 0.951 W - 2 38 229.72 0.000 0.924 GG 0.554 1.11 21.06 229.72 0.000 0.924 HF 0.564 1.13 21.42 229.72 0.000 0.924 DV×DS - 4 76 36.78 0.000 0.659 GG 0.836 3.34 63.52 36.78 0.000 0.659 HF 1.000 4.00 76.00 36.78 0.000 0.659 DV×A - 4 76 11.20 0.000 0.371 GG 0.660 2.64 50.13 11.20 0.000 0.371 HF 0.776 3.10 58.95 11.20 0.000 0.371 DV×W - 4 76 7.78 0.000 0.291 GG 0.718 2.87 54.60 7.78 0.000 0.291 HF 0.860 3.44 65.36 7.78 0.000 0.291 A×W - 4 76 6.39 0.000 0.252 GG 0.880 3.52 66.90 6.39 0.000 0.252 HF 1.000 4.00 76.00 6.39 0.000 0.252 DS - 2 38 6.50 0.004 0.255 GG 0.923 1.85 35.08 6.50 0.005 0.255 HF 1.000 2.00 38.00 6.50 0.004 0.255 DS×A - 4 76 4.31 0.003 0.185 GG 0.777 3.10 59.03 4.31 0.007 0.185 HF 0.946 3.78 71.90 4.31 0.004 0.185 DS×W - 4 76 2.18 0.079 0.103 GG 0.681 2.73 51.78 2.18 0.107 0.103 HF 0.806 3.23 61.29 2.18 0.095 0.103 Continued on next page 60 Appendix A. Pointing Experiment Data and Questionnaires Table A.2 – continued from previous page Factor Corr. e dfe f f ect d ferror F p Partial η2 DV×DS×A - 8 152 4.35 0.000 0.186 GG 0.603 4.82 91.66 4.35 0.002 0.186 HF 0.833 6.67 126.63 4.35 0.000 0.186 DV×DS×W - 8 152 3.13 0.003 0.141 GG 0.642 5.14 97.64 3.13 0.011 0.141 HF 0.909 7.27 138.15 3.13 0.004 0.141 DV×A×W - 8 152 0.69 0.696 0.035 GG 0.640 5.12 97.35 0.69 0.633 0.035 HF 0.905 7.24 137.57 0.69 0.682 0.035 DS×A×W - 8 152 0.95 0.480 0.047 GG 0.637 5.10 96.85 0.95 0.456 0.047 HF 0.899 7.19 136.59 0.95 0.474 0.047 DS×DV×A×W - 16 304 1.14 0.320 0.056 GG 0.423 6.77 128.67 1.14 0.344 0.056 HF 0.682 10.91 207.34 1.14 0.334 0.056 GG = Greenhouse-Geisser; HF = Huynh-Feldt 61 Appendix A. Pointing Experiment Data and Questionnaires Version 2011-August-15  page 1/1 Interaction With Large Displays II Phase 1 Pre-Experiment Questionnaire Participant #     1. How old are you?   years   2. What is your gender? (tick one)   Male  Female  Other  Prefer not to indicate  3. Which is your dominant hand? (tick one)   Right  Left  Both  4. How much time do you spend per week using a computer? (tick one)   Less than 1 hour  1 to 3 hours  4 to 8 hours   More than 8 hours  5. Do you normally wear glasses or contact lenses? (tick one)   Yes If yes, what is your prescription?  __________________  I don’t know   No   6. Have you watched stereoscopic (3D) movies before?   Yes  No  Figure A.1: Pre-Experiment questionnaire for the pointing experiment de- scribed in Chapter 2 62 Appendix A. Pointing Experiment Data and Questionnaires Version: 2011-August-15  page 1/1  Interaction With Large Displays II Phase 1 Post-Experiment Questionnaire Participant #      1. Overall, what was the level of difficulty of the task? (Encircle one number)  (Easy)   1 2 3 4 5 (Impossible)   2. Did you employ any particular strategy in completing the task? Please explain.   _________________________________________________________________  _________________________________________________________________  _________________________________________________________________  _________________________________________________________________  _________________________________________________________________    3. Please write any other comments you have regarding your experience with this task:   _________________________________________________________________  _________________________________________________________________  _________________________________________________________________  _________________________________________________________________  _________________________________________________________________   Figure A.2: Post-Experiment questionnaire for the pointing experiment de- scribed in Chapter 2 63 Appendix B MOT Experiment Data and Questionnaires Table B.1: Raw performance results for the pilot study described in Chapter 3. Subject Viewing Condition (D) nTracked (nT) Accuracy 1 Flat 2 1 1 Flat 3 1 1 Flat 4 0.75 1 Flat 5 0.75 1 Stereo 2 1 1 Stereo 3 1 1 Stereo 4 0.75 1 Stereo 5 0.25 2 Flat 2 1 2 Flat 3 1 2 Flat 4 0.75 2 Flat 5 0.75 2 Stereo 2 0.75 2 Stereo 3 1 2 Stereo 4 0.75 2 Stereo 5 1 3 Flat 2 0.75 3 Flat 3 1 3 Flat 4 0.5 Continued on next page 64 Appendix B. MOT Experiment Data and Questionnaires Table B.1 – continued from previous page Subject Viewing Condition (D) nTracked (nT) Accuracy 3 Flat 5 0.25 3 Stereo 2 1 3 Stereo 3 1 3 Stereo 4 1 3 Stereo 5 0.5 4 Flat 2 0.75 4 Flat 3 0.75 4 Flat 4 0.75 4 Flat 5 0.75 4 Stereo 2 0.75 4 Stereo 3 0.75 4 Stereo 4 0.5 4 Stereo 5 0.75 5 Flat 2 1 5 Flat 3 0.5 5 Flat 4 1 5 Flat 5 0.5 5 Stereo 2 1 5 Stereo 3 1 5 Stereo 4 0.5 5 Stereo 5 0.75 65 Appendix B. MOT Experiment Data and Questionnaires  Version 2012-August-03  page 1/1 Interaction With Large Displays II Phase 2 Pre-Experiment Questionnaire Participant #     1. How old are you?   years   2. What is your gender? (tick one)   Male  Female  Other  3. How much time do you spend per week using a computer? (tick one)   Less than 1 hour  1 to 3 hours  4 to 8 hours   More than 8 hours  4. Do you normally wear glasses or contact lenses? (tick one)   Yes If yes, what is your prescription?  __________________  I don’t know   No   5. Have you watched stereoscopic (3D) movies before?   Yes  No  Figure B.1: Pre-Experiment questionnaire for the MOT experiment described in Chapter 3 66 Appendix B. MOT Experiment Data and Questionnaires  Version: 2012-August-03  page 1/1  Interaction With Large Displays II Phase 2 Post-Experiment Questionnaire Participant #      1. Overall, what was the level of difficulty of the task? (circle one number)  (Easy)   1 2 3 4 5 (Impossible)   2. Did you employ any particular strategy in performing the task? Please explain.   _________________________________________________________________  _________________________________________________________________  _________________________________________________________________  _________________________________________________________________  _________________________________________________________________    3. Please write any other comments you have regarding your experience with this task:   _________________________________________________________________  _________________________________________________________________  _________________________________________________________________  _________________________________________________________________  _________________________________________________________________   Figure B.2: Post-Experiment questionnaire for the MOT experiment described in Chapter 3 67

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
https://iiif.library.ubc.ca/presentation/dsp.24.1-0052210/manifest

Comment

Related Items