UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Moving target selection in interactive video Ilich, Michael Victor 2009

You don't seem to have a PDF reader installed, try download the pdf

Item Metadata

Download

Media
24-ubc_2010_spring_ilich_michael.pdf [ 3.73MB ]
Metadata
JSON: 24-1.0064925.json
JSON-LD: 24-1.0064925-ld.json
RDF/XML (Pretty): 24-1.0064925-rdf.xml
RDF/JSON: 24-1.0064925-rdf.json
Turtle: 24-1.0064925-turtle.txt
N-Triples: 24-1.0064925-rdf-ntriples.txt
Original Record: 24-1.0064925-source.json
Full Text
24-1.0064925-fulltext.txt
Citation
24-1.0064925.ris

Full Text

Moving Target Selection in Interactive Video  by Michael Victor Ilich B.A.Sc., Simon Fraser University, 2000  A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF  Masters of Applied Science in THE FACULTY OF GRADUATE STUDIES (Electrical and Computer Engineering)  The University Of British Columbia (Vancouver) December 2009 c Michael Victor Ilich, 2009  Abstract In this thesis, we present the results of a user study that compares three different selection methods for moving targets in 1D and 2D space. The standard Chaseand-Click method involves pursuing an onscreen target with the mouse pointer and clicking on it once directly over it. The novel Click-to-Pause method involves first depressing the mouse button to pause all onscreen action, moving the cursor over the target and releasing the mouse button to select it. The Hybrid method combines the initial pursuit with the ability to pause the action by depressing the mouse button, affording an optimization of the point of interception. Our results show that the Click-to-Pause and Hybrid methods results in lower selection times than the Chase-and-Click method for small or fast targets, while the Click-to-Pause technique is the lowest overall for small-fast targets. We integrate the more practical Hybrid method into a multi-view video browser to enable the selection of hockey players in a pre-recorded hockey game. We demonstrate that the majority of correct player selections were performed while the video was paused and that our display method for extraneous information has no effect on selection task performance. We develop a kinematic model that is based on movement speed and direction in 1D as an adjustment to the effective width and distance of a target. Our studies show that target speed assists users when a target is approaching, up to a critical velocity where the direction is irrelevant and speed is entirely responsible for the index of difficulty. In addition, we suggest that existing linear and discrete models of human motor control are inadequate for modeling the selection of a moving target and recommend the minimum jerk law as a guide for measuring human motor acceleration. By combining our empirical results from moving target selection ii  tasks in 1D with our theoretical model for motor control, we propose an extension to Fitts’ Law for moving targets in 2D polar space.  iii  Table of Contents Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  ii  Table of Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  iv  List of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  vii  List of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  viii  Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  xi  Dedication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  xii  Statement of Collaboration . . . . . . . . . . . . . . . . . . . . . . . . .  xiii  1  2  Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  1  1.1  Modeling Target Selection . . . . . . . . . . . . . . . . . . . . .  3  1.2  Moving Targets . . . . . . . . . . . . . . . . . . . . . . . . . . .  5  1.3  Existing Selection Techniques . . . . . . . . . . . . . . . . . . .  6  1.4  The Click-to-Pause Technique . . . . . . . . . . . . . . . . . . .  8  1.5  Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . .  9  1.6  Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  12  Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  14  2.1  Interactive Video . . . . . . . . . . . . . . . . . . . . . . . . . .  14  2.2  Target Selection . . . . . . . . . . . . . . . . . . . . . . . . . . .  19  2.3  Extending Fitts’ Law . . . . . . . . . . . . . . . . . . . . . . . .  22  2.4  Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  27  iv  3  Evaluating Interaction Techniques for Moving Target Selection . . .  29  3.1  Empirical Evaluation . . . . . . . . . . . . . . . . . . . . . . . .  32  3.1.1  Apparatus . . . . . . . . . . . . . . . . . . . . . . . . . .  33  3.1.2  Participants . . . . . . . . . . . . . . . . . . . . . . . . .  34  3.1.3  Procedure . . . . . . . . . . . . . . . . . . . . . . . . . .  34  3.1.4  Experimental Design . . . . . . . . . . . . . . . . . . . .  37  3.1.5  Performance Measures . . . . . . . . . . . . . . . . . . .  39  3.1.6  Results . . . . . . . . . . . . . . . . . . . . . . . . . . .  40  3.1.7  Discussion . . . . . . . . . . . . . . . . . . . . . . . . .  50  Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  57  Moving Target Selection in an Interactive Video Context . . . . . .  59  4.1  Empirical Evaluation . . . . . . . . . . . . . . . . . . . . . . . .  61  4.1.1  Apparatus . . . . . . . . . . . . . . . . . . . . . . . . . .  62  4.1.2  Procedure . . . . . . . . . . . . . . . . . . . . . . . . . .  63  4.1.3  Application Development . . . . . . . . . . . . . . . . .  65  4.1.4  Participants . . . . . . . . . . . . . . . . . . . . . . . . .  65  4.1.5  Experimental Design . . . . . . . . . . . . . . . . . . . .  66  4.1.6  Performance Measures . . . . . . . . . . . . . . . . . . .  66  4.1.7  Results and Discussion . . . . . . . . . . . . . . . . . . .  67  Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  71  Fitts’ Law Modeling . . . . . . . . . . . . . . . . . . . . . . . . . . .  73  5.1  Deterministic Versus Stochastic Modeling in Two Dimensions . .  74  5.2  Interception of Moving Targets . . . . . . . . . . . . . . . . . . .  75  5.3  Modeling the Data . . . . . . . . . . . . . . . . . . . . . . . . .  77  5.4  Directional Movement in 1D Space . . . . . . . . . . . . . . . . .  79  5.5  The Impact of Size, Speed and Direction on Movement Time . . .  84  5.6  Extension of Direction to Two Dimensions . . . . . . . . . . . . .  87  5.7  Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  89  Conclusions and Future Work . . . . . . . . . . . . . . . . . . . . .  91  6.1  Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  91  6.2  Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . .  93  3.2 4  4.2 5  6  v  6.3  6.4  Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  96  6.3.1  Refining Our Interactive Selection Methods Experiment .  96  6.3.2  Extending Our Work to 2D Polar Space . . . . . . . . . .  97  Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  98  Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100 A Trial Design for Chapter 3 . . . . . . . . . . . . . . . . . . . . . . . 106 B Ethics Approval . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110  vi  List of Tables 3.1  Responses to post-questions . . . . . . . . . . . . . . . . . . . .  54  4.1  Picture-in-picture questionnaire responses . . . . . . . . . . . . .  69  4.2  Scaled panel questionnaire responses . . . . . . . . . . . . . . . .  70  vii  List of Figures 3.1  State transition diagrams for methods of interaction, red transitions indicate where target is selected . . . . . . . . . . . . . . . . . .  32  3.2  Experimental apparatus for “capture the wisp” game . . . . . . .  33  3.3  Game acquisition tactics, red potion/chase (a,b), blue potion/pause (c,d) and green potion/hybrid (e,f) . . . . . . . . . . . . . . . . .  35  3.4  1D technique by size and speed . . . . . . . . . . . . . . . . . . .  42  3.5  1D mean times by size and speed for both techniques . . . . . . .  42  3.6  1D mean errors by size and speed for both techniques . . . . . . .  43  3.7  2D technique by size and speed . . . . . . . . . . . . . . . . . . .  44  3.8  2D mean times by size and speed for both techniques. C is chaseand-click, P is click-to-pause . . . . . . . . . . . . . . . . . . . .  45  2D mean errors by size and speed for both techniques . . . . . . .  46  3.10 2D mean times by size and speed for all three techniques . . . . .  47  3.11 2D mean errors by size and speed for all three techniques . . . . .  48  3.12 Technique chosen by target size, then speed . . . . . . . . . . . .  53  3.9  4.1  The interactive video browser, a selected hockey player has a blue bounding box . . . . . . . . . . . . . . . . . . . . . . . . . . . .  60  4.2  The picture-in-picture version of the interactive video browser . .  62  4.3  The scaled panel version of the interactive video browser . . . . .  62  4.4  Experimental apparatus for interactive video browser . . . . . . .  63  5.1  Kinematic diagram for target interception . . . . . . . . . . . . .  81  5.2  Position, velocity and acceleration for unconstrained motion . . .  82  5.3  1D chase analysis by size and speed . . . . . . . . . . . . . . . .  85  viii  5.4  Speed-direction relation for critical speed threshold . . . . . . . .  87  5.5  2D derivation of radial and angular speed . . . . . . . . . . . . .  89  6.1  Polar target specification, (a) radial velocity (b) angular velocity .  98  A.1 Sample XML file as input for a deterministic trial . . . . . . . . . 107 A.2 Sample XML file as input for a non-deterministic trial . . . . . . . 108 A.3 Sample angle diagram for average angle calculation from potion origin . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109  ix  List of Equations 1.1 Fitts’ law . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  4  2.1 Welford variation of Fitts’ law . . . . . . . . . . . . . . . . . . . . . .  23  2.2 Accot and Zhai 2D extension . . . . . . . . . . . . . . . . . . . . . . .  23  2.3 Probabilistic model of 2D pointing index of difficulty . . . . . . . . . .  24  2.4 Jagacinski’s moving target formula, Shannon formulation . . . . . . . .  24  2.5 Jagacinski’s moving target formula, Welford formulation . . . . . . . .  25  2.6 Hoffman’s moving target formula, Shannon formulation . . . . . . . . .  25  2.7 Hoffman’s moving target formula, Welford formulation . . . . . . . . .  25  2.8 Tresilian’s moving target interception formula . . . . . . . . . . . . . .  26  2.9 Faure’s equation for target pop-up and animation delays . . . . . . . . .  26  5.1 Fitts’ law in 2D with effective width . . . . . . . . . . . . . . . . . . .  74  5.2 Hoffman’s moving target formula, target moving towards . . . . . . . .  76  5.3 Hoffman’s moving target formula, target moving away . . . . . . . . .  76  5.4 Effective distance of 1D directional Fitts’ task . . . . . . . . . . . . . .  81  5.5 Effective width of 1D directional Fitts’ task . . . . . . . . . . . . . . .  81  5.6 First order position equation for minimum jerk law . . . . . . . . . . .  83  5.7 First order position equation for minimum jerk law . . . . . . . . . . .  83  5.8 Second order velocity equation for minimum jerk law . . . . . . . . . .  83  5.9 Third order acceleration equation for minimum jerk law . . . . . . . . .  83  5.10Critical velocity as a function of width . . . . . . . . . . . . . . . . . .  86  5.11Effective distance for target velocity relative to critical speed. . . . . . .  86  5.12Effective width for target velocity relative to critical speed . . . . . . .  86  x  Acknowledgements I would like to express my sincere gratitude to Dr. Sidney Fels and my colleagues in the HCT Lab, particularly Dr. Gregor Miller, Dr. Matthias Finke and Tony Tang. I learned a great deal during my time at UBC and enjoyed working with everyone.  xi  Dedication To my girlfriend, Shira, and my parents. Without their love and support, this would not have been possible.  xii  Statement of Collaboration This research is based on experiments designed by the author, Dr. Sidney Fels and Dr. Gregor Miller that were administered by the author. The author prepared the introduction in Chapter 1, performed the background research in Chapter 2 and wrote Chapters 3 and 4, presenting the results of each of the experiments. Dr. Sidney Fels and the author performed the trend analysis and modeling of the data in Chapter 5, while the author presented the results in Chapters 5 and 6.  xiii  Chapter 1  Introduction This thesis addresses the limitations of existing interaction techniques in target selection as applied to interactive video, where the targets consist of the dynamic content of the video. For example, interactive multi-view video footage of a hockey game has many players moving that may be hyperlinked; however, these fast moving targets are difficult to select due to their speed and relative size. In general, video consists of a mix of static shots, pan movements, multiple angles and scene transitions, where the state of video content is time variant. Target objects will be prone to movement, occlusion by each other and discontinuous visibility in a limited selection of scenes. There are currently no target selection methods for moving objects in a 2D scene that address these issues in interactive video applications. The limitations for advancement in multimedia research can be attributed to the lack of a predictive model for moving targets in a 2D scenario. We have developed a novel interaction technique that we call “Click-to-Pause”, for the selection of onscreen targets with a mouse pointer. Click-to-Pause enables users to pause a video by depressing the mouse button, which allows them to easily move the cursor to a stationary target. Once the user releases the mouse button, the target is selected and the playback of the video resumes. In contrast, without Clickto-Pause, the user must chase the target around the screen and click on it while it is moving to select it. We refer to this direct approach as the “Chase-and-Click” model. In addition, we condsider a hybrid approach in which targets are selected 1  when a user clicks directly on them, but pauses the video when the user clicks on an area of the screen not directly over a target. We refer to this hybrid approach as the “Chase-or-Pause” model, in which either of the previous two methods can be applied or some optimum combination. We first compare these three techniques empirically using a simple game-like environment to isolate the tradeoffs between the approaches. Our simple game consists of abstract moving targets of varying sizes, speeds and directions that can be captured using one of our techniques, that are enabled by game objects visualized as potions. This game structure allows us to establish a known starting location for the tests as described in Chapter 3. This experimental setup is similar to a typical Fitts’ law target selection study[15], extended to moving targets. Using data from this study, we analyze the movement times and selection errors to extend Fitts’ Law to include moving targets and predict the performance of each technique. We also performed a user study in a more realistic application environment that consisted of a multi-view video navigation tool. Our navigation tool integrates multi-angle video streams with annotated objects that can be selected during playback using our hybrid Chase-or-Pause method. Our novel interaction technique allows users to temporarily pause the video stream, select content within the video and resume playback on completion. The use of the Chase-or-Pause method in interactive video was inspired by Shneiderman[43] and Hutchins et al. [29] in their support for a direct engagement style of interface. We conclude our investigation with a qualitative study of the performance of the Click-to-Pause technique in two contrasting environments that display extraneous information as either a floating window overlay or as static peripheral panels around the borders of a scaled video. Our motivation for research is to address the developing need for interaction techniques that support non-linear navigation through the growing distribution of ondemand video sources such as digital cable, satellite and internet provision. As the library of online video is expanding on a daily basis, the need for a detailed classification system is apparent from the limited tagging capabilities offered by many popular video sharing sites. While the originators of digital video provide custom 2  meta tags, personalized annotations and hyperlinks to related media, there is no standard for addressing specific video content. Given the complexity of modern High Definition video, it is impractical to expect manual annotation of video content on a frame-by-frame basis; however, advances in computer vision will eventually provide a means of automating this process by providing tagged bounding regions that remain associated with objects throughout video playback. As direct interaction with video content is such a recent technological advance, there are currently no guidelines or design heuristics for selecting or manipulating video content. Pointing techniques that apply to the visible, static content of web applications are no longer valid in an environment where target objects are embedded in a dynamic, streaming video. The tracking of objects in interactive video environments can only be achieved through the development of a precise selection method that is resilient to the movement, occlusion and intermittent visibility of video content. Direct forms of interaction with common meta-content will be essential for navigating large video libraries.  1.1  Modeling Target Selection  When using a typical computer interface, target selection is a fundamental control operation that allows users to express specific goals. For graphical user interface (GUI) design, the common WIMP (Windows, Icons, Menus and Pointing Devices) interface forms the basis for communicating with all operating system environments, software applications and peripheral devices. PC users are expected to select from menu options, text fields or a group of icons on a daily basis. Such virtual pointing tasks can be modeled by the same mathematical representation that was derived by Paul Fitts[15] for physical tasks as a model of human movement. The movement time (MT) to acquire a target of width W from a distance D can be predicted using his formal relationship for defining the speed-accuracy tradeoff of directed movements in Equation 1.1.  3  MT = a + b log2  D +1 W  (1.1)  In this equation, a and b are empirically determined constants that represent the start/stop time of the pointing device and the speed of the pointing device, respectively. The logarithmic term is known as the index of difficulty, measured in bits and shown to be proportional to the distance of separation and inversely proportional to the target width. Researchers in HCI and Cognitive Psychology use this law to develop a predictive model for the movement time of pointing tasks in experimental interfaces. User interface developers use this law to evaluate human interface devices and establish control-display heuristics. Fitts’ law originally applied to stationary targets of a uniform shape in one dimension that are visible at the onset of the pointing task. This model fails to support the estimation of task completion time when targets are capable of movement, are of an irregular shape or are positioned in a two dimensional field. In recent years, researchers have attempted to extend Fitts law to account for each of these circumstances, approaching a more realistic scenario for pointing tasks in modern computing applications. The challenge associated with the development of an extension to the traditional Fitts’ law model is in determining which factors contribute to the index of difficulty and are consistent over a broad range of empirical data. The original model for Fitts’ law in one dimension is based on a speed-accuracy tradeoff, where the objective of speed is realized in the movement phase while covering a distance of D to the target. Conversely, the objective of accuracy is realized in the precision phase while attempting to arrive within a target width of W. From equation 1.1, we note that the movement time is directly proportional to the distance to the target and inversely proportional to the width of the target. Additional factors such as target height, angle, speed and direction must be studied to determine how they affect the movement time and how they relate to the target distance and width.  4  There are two distinct approaches to extending the traditional Fitts’ law model into 2D space and one successful approach to extending it to account for moving targets. The deterministic approach to modeling targets in 2D space uses euclidean geometry to add a factor of target height to the precision phase, in addition to the width[1]. Alternatively, the stochastic approach to modeling targets in 2D involves a generalization of the target shape and the cursor’s angle of approach using an empirical spread of hits along the axis of the target width[19]. A limited approach to the modeling of moving targets in 1D has involved a scaling of the width factor by a relative target speed of V, indicating a limited window for the precision phase[31]. This model has been refined by applying a position error to both movement phases, with a factor of direction resulting in two separate calculations[25]. The 2D deterministic, 2D stochastic and moving target extensions to Fitts’ law have all been demonstrated to be successful predictive models in their respective applications.  1.2  Moving Targets  The selection of moving targets is rarely seen in most Graphical User Interface (GUI) frontends that are developed for desktop programs, video on demand (VOD) systems, portable electronics or rich internet applications (RIA). The inherent challenge of this task indicates that it is well suited for a game environment, such as in a real-time strategy simulation[39]. The user perspective in such game scenarios is often ideal and the relative size and speed of target objects is maintained to enable users to easily select single or multiple targets with a simple mouse operation. In addition, the target objects in unit-based real-time strategy engines will move in deterministic patterns according to instructions issued by the player, assisting in the prediction of an ideal point of interception. Based on these observations, we attribute an increased index of difficulty in the Fitts’ law model to additional factors such as target speed, direction and the determinism of movement. Interactive video has introduced the moving target selection task as a requirement for video interfaces that are based on the selection and manipulation of content that is in constant motion during playback. Interactive video content is represented by  5  tagged selection regions within the video frame, for which the size, shape and position is dynamic over the video timeline. Interactive video objects can be selected as input criteria, manipulated as a playback controller or activated as a hyperlink to an online destination. In each of these cases, an inherent pointing task is required for which the target may be moving in predictive or non-predictive patterns, in various directions and at varying speeds. The most direct approach to this selection task would be to pursue the target and attempt to intercept it while in motion; however, a non-deterministic movement pattern, a faster target speed or a smaller visual representation may result in an exceedingly difficult task. Additional factors such as the presence of multiple targets in close proximity, target occlusion or travel out of view may result in a limited window of opportunity in which to complete the selection task. Based on this reasoning, it is apparent that an alternative approach to target selection may be desirable under these circumstances.  1.3  Existing Selection Techniques  In order to make target selection tasks easier, researchers have attempted to provide virtual enhancements[3] that improve user performance by reducing the completion time and the number of selection errors. In the Fitts’ law equation, the index of difficulty is defined as a logarithmic function that is directly proportional to the distance to the target and inversely proportional to the target width. Naturally, we perceive that in order to reduce the difficulty of the moving target selection task, we must either reduce the distance to the target[4] or increase the effective target width[37]. Based on our background research and questionnaire results, we suggest three major categories of virtual enhancement for target selection: pointer enhancement, target enhancement and task simplification. The enhancement of the pointer would involve increasing the effective area of the pointer, increasing the effective speed of the pointer or establishing an inherent affinity with the target. Increasing the effective area of the pointer would reduce  6  the movement distance to the target because of the increased contact area of the pointer. Increasing the effective speed of the pointer would reduce the movement time across large, unused areas of the screen. Establishing an inherent affinity between the pointer and the target could be done through an association based on proximity or recent, temporary contact. These enhancements are difficult to implement in dense clusters of target objects, due to the potential for multiple target selection and the need to optimize the speed-accuracy tradeoff of the task. The enhancement of the target would involve increasing the effective area of the target, decreasing the effective speed of the target or repositioning the target for an easier selection movement. Increasing the effective area of the target would reduce the movement distance for the pointer because of the increased contact area of the target. Decreasing the effective speed of the target minimizes the pursuit and enlarges the window of opportunity for interception. Repositioning the target for an easier selection movement would involve temporarily moving the target closer to the pointer, until selected. These enhancements are also difficult to implement as they result in target occlusion in dense clusters, delays for moving targets that are approaching the pointer and the loss of spatial context. We refer to task simplification as a generalization that involves removing one or more of the factors that are contributing to the overall task difficulty, decreasing the predicted movement time. As discussed in the previous section, the target size, the target speed and the determinism of movement are perceived to be the largest contributors to the effective index of difficulty. In interactive video, the target size and the determinism of movement are tied to the video content itself, although an overlay could be produced that would increase the effective target area on the screen. Instead, we consider a novel approach to the simplification of moving target selection tasks by removing the factor of target speed and reverting to a traditional Fitts’ law task involving a stationary target of width, W, located at a distance, D, from the pointer.  7  1.4  The Click-to-Pause Technique  The Click-to-Pause technique for interactive video allows users to temporarily pause a video source while interacting with target objects in the current view. This effectively removes the factor of target speed from the task of selecting video content on the basis of its distance from a pointer and its relative size in the frame. This is in contrast to the direct Chase-and-Click technique that involves movement of the pointer directly over a moving target and a carefully timed button press to select the target in the limited window in which the pointer is over the target. Removing the factor of target speed should theoretically reduce the effective movement time of the pointer because the pointer speed no longer needs to be coupled to the target speed; however, this theory fails to account for the direction of the moving target. In a study on the interception of moving targets in a 2D plane, Tresilian et al.[44] state that there are different control processes involved in interception than in pursuit. A user must pursue a moving target and match its position and speed if it is moving away from the pointer; however, a user must anticipate where to intercept a target if it is moving towards the pointer. The increased movement time of the pointer in a selection task can be directly attributed to the speed of a moving target if it is moving away from the pointer; however, the target speed can also assist in reducing the movement time of a pointer if it is moving towards the pointer by brining the interception point closer. From these observations, we conclude that there may be circumstances in which the Click-to-Pause technique does not reduce the effective movement time of a pointer in a moving target selection task. This observation was the motivation for creating a hybrid Chase-or-Pause technique, in which users would have the option to either chase an approaching target to a point of interception or pause a departing object to reduce the effective pursuit. In the context of our multi-view video browser, the hybrid technique enables users to pause video playback instantly with a button press from anywhere in the video frame. The freedom to move the pointer anywhere in the video frame prior to the button-down event allows the user to decide when to initiate the pause based on the distance to the target object and its trajectory. We hypthesize that users will adopt  8  an optimization strategy in selecting where to move the pointer and when to pause the target in order to find the most efficient point of interception. The Click-to-Pause technique can be defined by a re-arrangement of the sequence of events that would normally occur when selecting an object using the Chase-andClick technique. The Chase-and-Click technique consists of the initial movement of the pointer over the target object, a button-down event to initiate the selection followed by a button-up event to complete. This is in contrast to button-up selection, where the same chain of events occur except that selection is initiated only once the button is released. This is a common source of errors as suggested by Isokoski[30], due to the natural mental model of selection on initial contact. The Click-to-Pause technique consists of a button-down event, a movement of the pointer over the target object followed by a button-up event to select it. We anticipate that the order of events may initially affect user performance in selection tasks, due to the expectation of selection during a button-down event; however, we hope to counteract this effect by providing an extensive tutorial for Click-to-Pause operation.  1.5  Contributions  We developed a novel technique for the selection of moving targets in 2D that we call Click-to-Pause. This technique allows users to temporarily pause the target before performing the selection, eliminating the need to pursue the target while in motion, using the Chase-and-Click technique. In addition, we have developed a Hybrid Pause-or-Chase technique that enables users to adopt an optimization strategy based on the target speed, direction and determinism of movement. We conducted a user study designed to compare these three techniques, resulting in a large empirical data set that balanced factors such as dimension, target size, target speed and the determinism of the target’s motion. This data set was used to evaluate the impact of each technique on movement time in selection tasks and to develop a predictive model in 1D space with a plan for extension to 2D. The results  9  of our analysis indicated that:  • For 1D target selection, significant interaction effects were observed for technique by size, technique by speed and technique by size by speed on movement time. Chase-and-Click proved to be faster for medium-sized, largesized, slow-moving and moderately-moving targets and all combinations thereof. Click-to-Pause proved to be faster for small-sized and fast-moving targets and the combination of both. The Chase-and-Click technique resulted in substantially more errors (25.17%) than Click-to-Pause (1.34%). No significant main or interaction effects were observed for the target determinism of movement. • For 2D target selection, significant interaction effects were observed for technique by size, technique by speed and technique by size by speed on movement time. Chase-and-Click proved to be faster for medium-sized, largesized, slow-moving and moderately-moving targets and all combinations thereof. Click-to-Pause proved to be faster for small-sized and fast-moving targets and the combination of both. The Chase-and-Click technique resulted in substantially more errors (31.2%) than Click-to-Pause (1.58%). No significant main or interaction effects were observed for the target determinism of movement. • The graphs of technique versus size in 1D and 2D depict a crossover of Chase-and-Click and Click-to-Pause between small and medium, indicating a threshold below which Click-to-Pause outperforms Chase-and-Click. Likewise, the graphs of technique versus speed in 1D and 2D depict a crossover of Chase-and-Click and Click-to-Pause between moderately-fast and fast, indicating a threshold above which Click-to-Pause outperforms Chase-andClick. Speed has relatively no effect on movement time for Click-to-Pause trials, but makes Chase-and-Click tasks extremely difficult, especially when in combination with small targets. • In the second phase, significant interaction effects were observed for technique by size, technique by speed and technique by size by speed on move10  ment time. Chase-and-Click proved to be faster for large-sized, slow-moving and moderately-moving targets and all combinations thereof. Click-to-Pause and Hybrid proved to be faster for small-sized and fast-moving targets, while Click-to-Pause outperformed Hybrid for the combination of the two. The Chase-and-Click technique resulted in substantially more errors (29.17%) than Hybrid (6.91%) or Click-to-Pause (1.17%). No significant main or interaction effects were observed for the target determinism of movement. We discovered that target speed has relatively no impact on the Click-to-Pause technique, while target size follows a traditional Fitts’ trend for target width. In addition, small and fast targets made the Chase-and-Click technique extremely difficult, resulting in long movement times. The Hybrid Chase-or-Pause technique performed on-par with the Click-to-Pause technique for smaller or faster targets, but resulted in higher acquisition times for small-fast targets and a slightly higher error rate for all significant interactions. Optimization of pursuit movement and target pause were observed and occured more frequently with small and fast targets. These contributions are developed in Chapter 3. Once the Hybrid Chase-or-Pause technique was implemented in an interactive video browser, we observed that the vast majority of correct target selections were performed while the video was paused and that subjects were using an optimization strategy. The type of display mode in the video browser, scaled panel or picturein-picture, had no effect on task completion time. Subjects expressed a clear preference for a picture-in-picture mode of display over scaled panels for extraneous information and menus. These contributions are developed in Chapter 4. From our results, we identified the angle of approach and the direction of travel as key contributors to the index of difficulty and overall movement time in selection tasks. We indentified a kinematic model that accounts for human motor acceleration and the minimum jerk law is suggested as a means of empirically determining a formula or constant for acceleration. Analysis of 1D chase data revealed that target speed assists in selecting approaching targets, but dominates over direction beyond a critical speed for smaller targets. We provide a framework for extending 11  the directional model into 2D polar space using radial speed and angular speed that are calculated from Euclidean coordinates and speed vectors. These contributions are developed in Chapter 5.  1.6  Structure  The remainder of this thesis is organized first by an extensive literature review that analyzes related work in our field of research, followed by an evaluation of the three interaction techniques for selecting moving targets, a qualitative study of the integration of the Chase-or-Pause technique into an interactive video browser as well as an analysis of Fitts’ law models and data trends observed in our evaluation. This analysis is followed by a conclusion that summarizes our findings and contributions. The second chapter is a complete literature review that is structured by topic to include the following background research: (1) an analysis of interactive video and mechanisms used to navigate video, including hypervideo, video browser infrastructure and direct manipulation techniques, (2) a survey of enhanced target selection techniques that includes target enhancement, pointer enhancement and moving target acquisition, (3) a history of extensions to Fitts’ law that include a deterministic approach to bivariate pointing tasks, a stochastic approach to pointing tasks in two dimensions and mathematical models for moving targets. The third and fourth chapters are structured as follows: (1) a discussion of the design and implementation of each interaction method, (2) a description of the experimental design, (3) an outline of the procedure, (4) a presentation of the results, (5) a discussion of the overall outcome and (6) a conclusion. The fifth chapter provides an analysis of existing Fitts’ law models for the selection of targets in 2D space and moving targets, model validation with experimental data, the development of a kinematic model for human movement, an analysis of the factor of direction in 1D moving target tasks and a framework for applying a  12  directional model to 2D space using polar coordinates. The sixth chapter summarizes the contributions of this thesis and includes recommendations for future research.  13  Chapter 2  Related Work This chapter provides an overview of recent research in the areas of interactive video, target selection methodologies and extensions to the original Fitts’ law. One of the primary areas that our Click-to-Pause model addresses is the need for content selection techniques in interactive video, with examples of hypervideo implementations, video abstractions and direct manipulation theory. We compare the Click-to-Pause approach with existing target selection methods classified by cursor enhancements and target enhancements for both stationary and moving targets. Finally, we intend to model the selection of moving targets in 2D space, which will require an analysis of existing extensions to Fitts’ law, including the extension for bivariate pointing tasks and the extensions for modeling target movement in 1D or constrained 2D scenarios.  2.1  Interactive Video  Our main research focus has been in the area of interactive video and the theoretical approaches to the selection of moving targets in the form of dynamic content. Our research started with an investigation into the present state of hypervideo (or hyperlinked video), video annotation and video navigation mechanisms. Hypervideo adds a non-linear metadata structure to video that enables users to select content based on their personal interests. The use of video annotation has allowed users to create and share a personalized experience that can draw attention to specific  14  details in subsequent viewings. The importance of content sharing is apparent in the online study presented by Cesar et al.[6], confirming that the majority of online video retrieval is based on referral. Video navigation has traditionally consisted of standardized VCR-like analogies for rewind, play/pause, fast-forward and stop, with the addition of a time-based slider bar and global tags for content-based retrieval. Online services such as YouTube[47] and Asterpix[2] offer a limited form of interactive video with the inclusion of hyperlinks to related videos and supplemental content. With YouTube, temporal links appear as dialog boxes that are only accessible for a limited window of time in their custom flash player. In addition, users can link to temporal offsets within videos by specifying a timestamp in their commentary. Clikthrough[9] is an interactive tagging service that enables promoters to embed biographies and advertising links in music videos, with this static information appearing in a peripheral window. Asterpix[2] is a hypervideo service that provides a browser that enriches video by adding temporal links, appearing as time-sensitive regions of interest leading to user comments or related videos. However, these services fail to provide a means of uniquely identifying rich media content and establishing semantic relationships beyond a global set of tags to classify the video. There are currently no services that allow users to freely navigate through video, making time-specific annotations along the way and maintaining a record of their browsing activity that can be stored and shared. In recent years, several researchers have attempted to provide infrastructure for video navigation that would change it from a passive viewing experience to an active browsing task, akin to how users explore the World Wide Web. Some research has focused on abstracting the video viewing experience using techniques such as video skimming[8] and video summarization[24], while others have developed a structure for virtual pathways[41] and real-time editing techniques[6]. Cheng et al.[8] developed a unique video interaction technique called adaptive fastforwarding based on the paradigm of scenic car driving, in which speed is dictated by the complexity and relevance of scenery. This method of video skimming can be 15  seen in contrast to their previous research in still-image abstraction of key events in videos of weddings and sports. Cheng’s SmartPlayer application is an intelligent user interface that adapts playback speed to user preferences in content, slowing down for relevant material and fast-forwarding through material that is of little interest; however, adaptive fast-forwarding remains a linear form of video navigation, where playback speed is the only factor that users can control. Haubold et al.[24] developed a video browser with rich content cues known as the Video Audio Structure Text Multimedia (VAST MM) application. Within this application, users are able to browse video categories, peruse videos or video summaries and create public annotations or personal bookmarks. Audio content is extracted by running IBM’s ViaVoice to extract keywords from dialog that can be used to index subsections of the video based on semantic audio content. While VAST MM uses thumbnail snapshots, keywords from audio content and annotation text to provide rich summarization information, the navigation structure is still dependent on a timeline, using semantic visual content as anchor points for hyperlinks. Pea et al.[41] introduced the Digital Interactive Video Exploration and Reflection (DIVER) system that would enable users to create “virtual pathways” through a collection of video sources. The interface for Pea’s DIVER is based on a “looknotice-comment” model, in which the original source is displayed in one window, a magnified region of interest is displayed in a second and an index of thumbnails with attached commentary is displayed in a third. Users move the virtual camera viewfinder in the first window to generate a “spatial coordinate” with a region of interest and a “temporal coordinate” with a thumbnail of the corresponding frame. Although the DIVER enables personalization with a roaming viewfinder and thumbnail annotation, there is no provision for indexing semantic content or seamless navigation between video sources. Cesar et al.[6] developed a multimedia content sharing system that allowed users to synthesize video fragments, add their own annotations and share the resulting enriched video stream over a social networking site. The SMIL[28] 3.0 standard was 16  used as a description language for the video source, temporal entry and exit points as well as the accompanying annotations. In this approach, users produce structures that are analogous to edit decision lists (EDLs) used in video editing systems, with the addition of text annotations. Semantically enriched video is modeled with content wrappers that include a hyperlink to another video source, a navigation map and a set of annotations. Another approach to developing interactive video has been the incorporation of video content as an embedded control mechanism. The direct manipulation technique was adopted from a user interface design principle established by Ben Shneiderman [43], to provide analogies from the physical world in GUI development. One of the earliest examples of this technique is the introduction of the desktop metaphor for operating systems and office applications. Interactive video provides intuitive affordances in the form of video content, which can be manipulated as a mechanism for playback control of the video. Kimber et al.[33] adapted previous technologies including storyboard summarization and the Digital Object Tracking System [17] to produce Trailblazing, a video playback controller. Their prototype application was developed for multi-camera surveillance of an office space where people were represented by a bounding box overlay in each angle and by an icon in a floor plan composite of all position data. Kimber developed a direct manipulation interface by graphing a visualization path for the movement of each tagged person and enabling the user to scrub the video by dragging the person in the video window, or their icon in the floor plan view, along their respective path. The trailblazing application demonstrated that varying granularity of playback control could be achieved with objects of different speeds. In addition, a form of content-based retrieval by object position could be introduced by dragging an object to a specific location. Karrer et al.[32] subsequently introduced DRAGgable Object Navigation (DRAGON) for browsing through video by direct manipulation of video content along its natural movement path. Karrer suggests that the main motivation for direct manipulation in video playback is the lack of a mapping between human interface de17  vice (HID) movement and the video content with a traditional timeline slider. The DRAGON interface sought to provide a mechanism that would match the direction and movement amplitude of an object of interest to the direction and movement amplitude of a human interface device. While DRAGON outperformed a traditional timeline slider in fine-grained video search, navigation beyond the current scene was not possible. Dragicevic et al.[12] build on the concepts introduced by Karrer et al. by automating the extraction of motion in videos and developing a technique known as relative flow dragging. Using this technique, moving imagery is decomposed into the trajectories of individual pixels, allowing users to scrub over these trajectories and control playback of the video. DimP, the direct manipulation video player, was designed to allow direct manipulation of arbitrary video content along a relative pathway of motion, independent of background translation. In addition, curvilinear dragging techniques are provided by DimP to support mapping of linear HID movements to non-linear pixel trajectories that involve scale, arc-length and directional discontinuity. Goldman et al.[18] suggest an improvement to the aforementioned flow-based pixel processing methods, by introducing point particle tracking that is extended over a longer period of time than optical flow. Point particles that move together over time are grouped together to form larger object bodies that move and transform along multiple axes. Constraints are imposed on the range of motion of these particle objects and a starburst widget is used as a visualization of the range of motion for each particle. The starburst interaction technique allows a diversion from the linear temporal sequence of video, focusing on the spatial arrangement and orientation of video content. In general, the direct manipulation of video sequences is best suited to fine-grained navigation tasks, where a single object is being tracked in one scene. Faster moving objects tend to decrease the granularity of navigation by direct manipulation, while slower moving objects tend to increase the granularity, due to a shared controldisplay ratio and absolute timeline[33]. Coarse-grained navigation beyond the vis18  ibility of an object in a single scene is better suited by a temporal or semantic mechanism. The examples of interactive video discussed here limit the capabilities of effective reference or retrieval based on content, given that there is no direct connection with the actual content of the video stream. Our Click-to-Pause interaction technique relates to this topic on the basis of video content that can be selected, manipulated and adopted as criteria for subsequent operations. Our model provides a basis for non-linear navigation of video libraries that goes beyond the capabilities of the examples presented here and extends direct manipulation beyond the scope of the immediate playback sequence.  2.2  Target Selection  Our research encompasses the evaluation of a selection technique for moving targets in 2D space. In order to develop an effective technique for such a complex task, we must first survey existing techniques that offer virtual enhancements to the cursor or the target. By analyzing each technique in the context of moving targets, we hope to gain some insight into which elements would be most effective for our task. Balakrishnan[3] suggests that virtual pointing tasks are not limited to the same constraints as those in the physical world, from traditional cognitive psychology experiments. He conducted a survey of virtual pointing methods in HCI research, noting the benefits of both target expansion and control-display (CD) gain adaptation, and concluded that significant performance benefits could be realized by enhancing either the virtual pointer or the target itself. We present a brief survey of modern enhancement techniques that is organized by categories for pointer enhancement and target enhancement. The most common style of enhancement for the traditional point cursor has been the advent of the area cursor, an increase in the size of a cursor and its effective activation area. Grossman and Balakrishnan[20] built upon the concept of the area cursor with the introduction of the bubble cursor. Their technique is based on the  19  dynamic resizing of the cursor’s activation area, based on the number and proximity of targets, ensuring that only one is selected from any given cursor position. They demonstrated through two controlled experiments that there were distinct performance advantages for dynamic area cursors over the traditional point cursor. The bubble cursor can be viewed as proximity-dependent area cursor, which Chapuis et al.[7] attempted to match with the introduction of DynaSpot, a speeddependent area cursor. DynaSpot is speed-dependent in the sense that the activation area of the cursor is directly proportional to its movement speed. One might consider this to be a hybrid of a point and an area cursor, given that a slow or stationary DynaSpot behaves like a point cursor, while a fast moving DynaSpot behaves like an area cursor. Controlled experiments revealed that DynaSpot performed on par with bubble cursor, up to a maximum size. The main disadvantage of area cursors is the lack of precision in a densely populated field of potential targets, where the area cursor must constantly adapt to disambiguate selection. In addition, Gunn et al.[22] postulated that moving targets altered the size of the bubble cursor too frequently to be effective in dense fields of moving targets. For our purposes, the area cursor would be an inadequate enhancement as we anticipate that targets in video may be densely clustered and traveling at various speeds. An alternative to spatial enhancement of the cursor might be a localized mechanism for the disambiguation of target selection. Fekete et al.[14] recently investigated pattern matching as an alternative for this very purpose, using elliptical motions of the pointer to mimic a graphical tag on the target itself. While effective in selecting static targets in 2D space, the initial mimicry of a pattern block in motion followed by a secondary pie selection mode would be too complex for acquisition of all but the slowest moving targets. Research has also demonstrated the performance benefits of enhancing the target of a pointing task, while the point cursor retains its traditional form. As a result, such enhancements are generally based on the proximity of the target[37][5] in or20  der to disambiguate selection in a dense cluster of potential targets. We investigate recent forms of target enhancement, which we categorize as target feedback, target expansion and target association. Mould and Gutwin[39] investigated the application of target feedback on multiple moving targets in a game-like environment. Feedback can take several forms such as visual, auditory or tactile, of which only visual feedback was used by highlighting targets in an experiment that compared the effects on selection task performance for conditions of no feedback, target-only feedback and all-target feedback. Both forms of feedback significantly improved task completion, with a strong user preference for target-only feedback due to the perceived distraction of all-target feedback. Target-only feedback requires the prediction of user goals, while alltarget feedback can result in too many distracters amid the intended target. McGuffin and Balakrishnan[37] investigated target expansion in a study that compared selection task times for untiled expanding targets and tiled expanding targets, with and without true expansion in motor space. Their study demonstrated a distinct advantage of expanding targets over non-expanding, more efficient use of motor space with tiled targets and faster selection with true expansion in motor space. McGuffin acknowledges that they were unable to model the case of motor expansion with a fixed target edge, as previous research has not revealed how a shift in the target’s position will affect the index of difficulty. This is especially true for the case where a target begins moving only once the user has started to move a point cursor towards it. Baudisch et al.[5] questioned how effective target expansion would be in the case where users must select from a dense cluster of targets. They argued that without sufficient screen space to expand into, occlusion would occur and performance would be sacrificed. They introduced Starburst, a space-partitioning algorithm that identifies the available areas of screen space and partitions each target from the cluster into a different tessellated region on the screen. As with bubble cursor, this algorithm would not handle the movement of targets, as it would need to constantly regenerate the tessellations for the entire cluster. 21  Gunn et al.[22] recently developed two techniques for moving target selection, Comet Tails and Target Lock, both of which improved selection time over unassisted moving target selection. Comet Tails combines elements of both target feedback and target expansion by providing an activation area to reveal a trailing target expansion. Target Lock introduced the concept of target association, in which the cursor would associate with the last target that it rolled over for universal selection. We anticipate that dense clusters of fast moving targets will diminish any perceived benefit of these two techniques, given the degree of overlap from opposing comet tails and the small window of opportunity in which to engage a target lock between rollovers. This analysis of target selection techniques has provided a context for the development of our Click-to-Pause model as a novel method for the selection of moving targets. We have discussed the limitations of existing selection techniques with respect to moving targets, dense clusters of targets and occluded targets. Instead of enhancing the cursor or the target, we propose the temporary suspension of target motion in order to model our technique as a stationary target selection in 2D space.  2.3  Extending Fitts’ Law  In order to evaluate the effectiveness of target selection methods, we must first establish a metric that will form the basis for comparison with existing techniques. Our research encompasses the extension of Fitts’ law to moving targets in 2D space, for which we hope to gain some intuition on the derivation of such a model. Although there are no existing models for moving targets in 2D space, we hope to analyze the extension of Fitts’ law to 2D and to moving targets separately and combine the most effective elements of each. For this purpose, we present an overview of Fitts’ law, a classic ergonomics principle that models human movement for tasks of this nature. Fitts’ law was established in 1954 as a model in cognitive psychology to predict movement time in a simple pointing task[15]. Originally, this model was applied to physical interfaces and later used to measure task difficulty within  22  graphical user interfaces on computer screens. This relation is derived from human motor theory involving a distance-covering movement, D, followed by a series of corrective movements, based on a goal of width W[16]. This Shannon formulation is in contrast to the Welford form[46], in which the distance and width are represented as separate components in the index of difficulty, as shown in equation 2.1. MT = a + b log2 (D) + c log2  1 W  (2.1)  Where a, b and c are empirically determined constants, while D and W remain measures of distance and target width, respectively. In its original form, Fitts law was limited to stationary targets in one dimension; however, several researchers have attempted to refine this equation to extend Fitts’ law to two dimensions and to include the selection of moving targets. Extending Fitts’ law to two dimensions involves recognizing two independent axes of movement, an angle of incidence and a target area (a height component in addition to the previous width). MacKenzie and Buxton[35] proposed a substitution of either the apparent width, W , or the target height, H, for the width, W, in the original Fitts law equation. They demonstrated that rectangular targets produced a negative index of difficulty (ID, the logarithmic term) and corrected this by suggesting the Shannon formulation for the index. Accot and Zhai[1] focused on the rectangular shape of the target, applying amplitude and directional constraints to identify a deterministic model for movement time. Their equation for the movement time in a bivariate pointing task is shown in equation 2.2.  MT = a + b log2   D W  2  +η  D H  2   + 1  (2.2)  Where a, b, D and W are the same variables from the original Fitts law relation, while H represents the target height and η represents a free weight to account for a  23  different effect of the height component. By applying the model to data previously gathered in a study by Hoffman and Sheikh[26], a correlation coefficient of R2 > 0.95 was achieved; however, these early models failed to account for the angular orientation of the target[23] and the shape of the target[21]. Grossman and Balakrishnan[19] took a probabilistic approach to modeling 2D pointing that generalized the target shape and the cursor’s angle of approach using an empirical spread of hits along the axis of the target width. This theory was developed by modeling the index of difficulty as a function of the probability of hitting a target, as shown in equation 2.3,  IDPr = F  bnd f (X ,Y ) dY dX  (2.3)  R  Where the “bndf” is the bivariate normal density function, a model for the spread of hits based on the probability of hitting a target region, R. Hence, the index of difficulty varies with the shape of the target, R, as Grossman and Balakrishnan later demonstrated its application to targets of arbitrary shapes in a formal user study[21]. The modification of Fitts’ law to include moving targets was first attempted in cognitive psychology by Jagacinski et al.[31], who developed a model for the index of difficulty that accounted for the velocity of target movement, V, in 1D space. Using this velocity-based model, experimental data gathered over the course of multiple user studies produced two analytical solutions for movement time estimation (MT), in equation 2.4 and equation 2.5. MT = c + dD + e(V + 1)  MT = x + y log2  1 −1 W  VT 2D + z log2 +1 W W  24  (2.4)  (2.5)  Where c, d, e, x, y and z are empirically determined constants, D and W follow their traditional representations, V represents the target velocity and T represents the capture time constant, a measure of how long a cursor must be positioned over its target. The first equation establishes the proportionality of MT to D, V, and W, while the second equation follows the Welford model, separating the effects of D and V to revert to a traditional Fitts equation in the event that V = 0. Hoffman[25] explored linear control model theory as a basis for the proposal of two alternate models for the index of difficulty that would be compared against the experimental data gathered by Jagacinski et al.[31]. Hoffman derived first and second order equations for both the movement toward and away from the pointer origin. The first order equation is shown in both the single-component Shannon form of equation 2.6 and dual-component Welford form of equation 2.7. MT =  1 ln K  MT = a + b log2 D ±  V K  D ± VK V W 2 −K  − c log2  (2.6)  W V − 2 K  (2.7)  Where K, a, b and c are empirically determined constants and D, W and V follow their previous representations. When applied to the position and velocity control models of Jagacinski’s study, similar correlation coefficients were achieved and a critical speed for target acquisition was identified. The velocity control model and the dual-component Welford form of the equation realized a better fit with Jagacinski’s data. Tresilian and Lonergan[44] tested Jagacinski’s model for intercepting a moving target by designing a physical apparatus that consisted of a fixed baseball bat and a rubber target that moved freely along a perpendicular track, a distance of D from the bat. The experimenters varied the target size, the target speed and the strike  25  zone width to determine how subject reaction time and speed were affected by temporal precision constraints. Tresilian reported that subjects moved faster in interception tasks when the temporal precision demands were greater; however, the experimental results did not support Jagacinski’s model due to a perceived inverse proportionality between movement time and target speed. Tresilian suggested an alternate model in equation 2.8. MT = α + β D +  γ f (W ) V  (2.8)  Where α, β , and γ are empirically determined constants and D, W and V follow their previous representations. Tresilian attributes the necessity for a new model to the nature of the task, given that “pursue and capture” is significantly different from “anticipate and strike”. This is apparent from the concluding remark, “[the] interception of a target by pursuing it may involve different control processes than interception executed in an anticipatory fashion.” Faure et al.[13] demonstrated the “anticipate and strike” task classification in a recent study of the selection of targets that exhibit motion animation or pop-up behavior in applications such as Mac OSX Expos´e. They conducted a user study to evaluate the selection time and error rate of targets from each of these categories for a variety of transition delays, revealing no significant performance improvement over static targets, but a lower error rate in the latter case. The revised model for “anticipate and strike” tasks is illustrated by equation 2.9. MT = a + b log2  TV D +1 +c W D  (2.9)  Where a, b and c are empirically determined constants, D and W follow the previous representation and TV is a delay factor. The delay factor represents either the time delay before the target appears, or the duration of the target’s animation. A formal experiment revealed that the selection of static targets was significantly faster and less error prone than a similar task involving pop-up or animated targets, up to a maximum delay of 200ms. In addition, the extension to Fitts’ law achieved 26  an adjusted correlation coefficient of R2 = 0.9478, suggesting that this approach may provide some degree of accuracy for moving targets. Our research is directly related to the extended models of Fitts’ law as we are attempting to extend the Fitts’ law model to moving targets in 2D space. We have compared methods of extending the model for static targets in 2D space by analyzing deterministic models in euclidean space as well as a probabilistic model that is generalized to virtually any object shape. We also hope to gain insight into the extension of Fitts’ law to moving targets using examples of 1D movement tasks and constrained 2D planar movements tasks.  2.4  Summary  The examples of interactive video discussed here limit the capabilities of effective reference or retrieval based on content, given that there is no direct connection with the actual content of the video stream. Our Click-to-Pause interaction technique relates to this topic on the basis of video content that can be selected, manipulated and adopted as criteria for subsequent operations. Our model provides a basis for non-linear navigation of video libraries that goes beyond the capabilities of the examples presented here and extends direct manipulation beyond the scope of the immediate playback sequence. This analysis of target selection techniques has provided a context for the development of our Click-to-Pause model as a novel method for the selection of moving targets. We have discussed the limitations of existing selection techniques with respect to moving targets, dense clusters of targets and occluded targets. Instead of enhancing the cursor or the target, we propose the temporary suspension of target motion in order to model our technique as a stationary target selection in 2D space. Our research is directly related to the extended models of Fitts’ law as we are attempting to extend the Fitts’ law model to moving targets in 2D space. We have compared methods of extending the model for static targets in 2D space by analyzing deterministic models in euclidean space as well as a probabilistic model that is generalized to virtually any object shape. We also hope to gain insight into the extension of Fitts’ law to moving targets using examples of  27  1D movement tasks and constrained 2D planar movements tasks. In the next chapter, our first study investigates the advantage of temporarily transforming moving targets into static ones, at a user’s discretion, while the subsequent application demonstrates how this mechanism is applied to a multi-view video collection.  28  Chapter 3  Evaluating Interaction Techniques for Moving Target Selection The motivation for our research is to determine whether the Click-to-Pause technique for selecting moving targets in 2D space is more effective than the Chaseand-Click approach. Our goal is to implement this interaction method in an application that enables navigation through rich media spaces consisting of hyperlinked video. In order to assist with the selection of moving targets, we have developed Click-to-Pause as an interaction technique in which the user can temporarily pause a streaming video and interact with content objects in the visible frame. The Clickto-Pause method reduces the task of selecting a moving object to a traditional Fitts’ task involving the selection of a static object. This interaction method has applications in interactive television, online video navigation, security administration and sports entertainment. We have designed a game-like environment in which we have abstracted both the Chase-and-Click and Click-to-Pause techniques as magic potions that players must first acquire in order to catch elusive “Wisp” targets. Our game formed the basis for a user study to compare both of these techniques, based on the performance metrics of acquisition time and error rate. We have selected Chase-and-Click as 29  a control to determine whether it is more efficient that Click-to-Pause in specific cases. In order to balance each method for comparison, the use of the Click-toPause technique is restricted to a pause widget that is located at the same distance from a target as the starting point for the Chase-and-Click method. This allows us to measure performance over the same target distance, with the mouse button depressed and the target frozen in one case and the mouse free to pursue the moving target in the other. We developed a balanced selection of targets that vary in size, speed and determinism of movement for both 1D and 2D. The 1D trials will provide data that can be used to validate the original Fitts’ model for 1D pointing at static objects as well as the modified model for moving objects in 1D. In the second phase of our user study, we evaluate an additional interaction method that is a hybrid of the Chase-and-Click and Click-to-Pause methods, that we call “Chase-or-Pause”. With this method, users are free to hold down the mouse button from any location on the screen to pause the motion of a target. This allows users to initially pursue targets during a ballistic phase of movement, prior to engaging a pause operation to make a more precise selection. Our goal during the second phase of the experiment is to observe user behavior and determine whether users are applying an optimization strategy for homing in on a target prior to selection. In summary, the three interaction techniques are: 1. Chase-and-Click This mode of interaction serves as a reference for the comparison of the more novel approaches. In this mode, the user is unassisted in the pursuit of a moving target and must simply chase the target, predict its movement path and click on it while in motion. The state transition diagram is provided in Figure 3.1, where the Button Released state has no initial target selection and the user is in pursuit. Once a Button Down event occurs, an object is selected and retained upon a Button Up event. 2. Click-to-Pause In our novel approach to the acquisition of moving targets, the user is able to freeze all target motion by clicking on an pause activation area. With the mouse button depressed, the user can move the cursor over the paused object and release the button to select it. Releasing the button while the cursor is not over a valid target will simply resume the motion of 30  all targets in view. The state transition diagram is provided in Figure 3.1, with a Button Released state representing no initial selection while the target is in motion. A Button Down event pauses all action in the scene and the user is free to move the pointer over any target in view. A subsequent Button Up event with the pointer over a target will result in its selection. 3. Hybrid Chase-or-Pause We introduce a hybrid of the previous models in which the user is free to chase the target and reduce the initial movement distance, until they decide to click and hold the mouse button down anywhere in the scene and pause the target while making a corrective movement over it. In this model, the target can be acquired by either a Button Down or Button Up event, provided that the cursor is directly over the target. The state transition diagram is provided in Figure 3.1, with a Button Released state and no initial selection while the target is in motion. A Button Down event pauses all action in the scene and selects a target if the cursor was over it. A subsequent Button Up event will resume the action and select a target if the cursor was over it. In the event that a target was selected during the Button Down event, if the subsequent Button Up event occurs quickly enough, the pause will be negligible and the technique will resemble the Chase-and-Click mode. The Click-to-Pause interaction model is a deviation from the WIMP (Window, Icon, Menu, Pointer) convention of drag and drop, where the button-down event signifies target selection and the button-up event often signifies subsequent action such as relocation. We anticipate that this may initially interfere with the user’s mental model of drag and drop operations, given that a button-up event is reserved for selection and subsequent actions require an additional interaction mode. One exception to the role of the button up event is the analogy of area selection, where objects are selected once overlapped by the cursor rectangle. We anticipate an inherent penalty in performance during Click-To-Pause tasks due to the button state, as indicated by Isokoski[30].  31  Figure 3.1: State transition diagrams for methods of interaction, red transitions indicate where target is selected  3.1  Empirical Evaluation  In our game, entitled “Capture the Wisp”, the Chase-and-Click, Click-to-Pause and Chase-or-Pause interaction techniques were abstracted as game objects that we developed as “potions”. In order to ensure that users would start with their cursor in a known location that was a uniform distance from the target each trial, the user was required to move the cursor over a “potion” that was set at a constant distance from the target. In the game, users would highlight a potion of one of three colors: red (enabling the use of Chase-and-Click), blue (enabling the use of Click-to-Pause) or green (enabling the use of Chase-or-Pause). The target object was abstracted as a “wisp”, or a ball of light from folklore, that would start at a predetermined distance from the potion. Using this simple game, users were able 32  to capture wisp targets through a series of trials that provided a broad range of factors including size, speed and movement predictability, or determinism.  3.1.1  Apparatus  The user study was conducted on a Dell XPS notebook with a 2.60GHz Core2 Duo CPU with 4GB of RAM running Windows XP Professional using a Microsoft USB Wheel Mouse Optical device. For the purposes of this experiment, “Enhance pointer precision” was disabled and the pointer speed was set to 6 (of 10). The laptop LCD display was used at a resolution of 1024 x 640 pixels at a refresh rate of 60Hz. A standard Microsoft optical wheel mouse was used as the input pointing device and the Adobe Air environment was set to full screen while running the Flash program. Internet access and any potentially interfering services were disabled for the duration of the experiment. The experimental setup is illustrated in figure 3.2.  Figure 3.2: Experimental apparatus for “capture the wisp” game  33  3.1.2  Participants  We enlisted sixteen participants, 9 female and 7 male, to conduct the experiment over the course of a one hour session. All participants were experienced computer users with normal or corrected-to-normal vision and no color blindness. In addition, every participant was right handed and none of them had any limiting injury.  3.1.3  Procedure  Participants played the “Capture the Wisp” game to catch a moving wisp, represented as a white circle on the screen. Each wisp was initially protected by a shield that could be disabled by one of three mechanisms, represented by three different potions, a red, a blue and a green. The red potion simply removed the shield from the wisp, allowing the participant to click on it while in motion. The blue potion created a small blue “web” (24 pixels by 20 pixels) that the participant could click on to remove the wisp’s shield and freeze its movement. By holding down the mouse button while the cursor was situated over the web, participants could drag a thread from the web over the stationary wisp and release the mouse button to catch it. The green potion served as a combination of the two previous potions, removing the shield and allowing participants to click directly on the wisp while in motion; however, if the participant clicked anywhere else on the screen and held down the mouse button, it would freeze the wisp’s movement and allow them to drag diagonal crosshairs (a visual pause indicator) over the wisp and release the button to catch it. These three methods of interaction are illustrated in Figure 3.3 with the panel on the left (a,c,e) depicting the game state prior to potion activation and the panel on the right (b.d.f) depicting the method of target acquisition, d (and potentially f) representing a paused state. During gameplay, potions were activated by simply moving the mouse cursor over them, causing them to disappear and their effects to be immediately realized. Each potion was placed at the same distance from the wisp, although at an alternate  34  Figure 3.3: Game acquisition tactics, red potion/chase (a,b), blue potion/pause (c,d) and green potion/hybrid (e,f) angle for each two-dimensional trial. The main distinction between the blue and the green potion was the presence of the web, restricting the subjects to pause immediately prior to movement from the same distance as the red potion. Once a participant had moved the cursor over the blue potion, the web was placed at a precalculated distance from the potion, relative to the wisp position. Participants were unable to click directly on the wisp after activating a blue potion, but were required to click on the web, hold down the mouse button and move the cursor directly over the wisp with the button depressed. Likewise, participants were unable to freeze the wisp’s movement after activating the red potion, but had to click on the wisp in mid-motion.  35  The game was structured as a series of trials, each of which involved the capture of a single wisp using one of the prescribed methods based on potion type. Participants were asked to complete a tutorial consisting of twelve trials that included on-screen cues to indicate what the next logical action was, such as “Move the mouse over the potion” or “Click on the web to pause the wisp”. Upon completion of the tutorial, participants were directed to begin the game, consisting of two separate phases to first compare the performance of the red and blue potions, followed by an introduction to the green potion. The first phase was structured into twelve blocks of 24 trials each for a total of 288 trials overall. These twelve blocks were organized with an equal distribution of red and blue potions, interleaved to counter any potential learning or control effects on their direct comparison. Upon completion of the twelfth wave, participants engaged in a second tutorial to introduce the green potion, also consisting of twelve trials with on-screen cues. Upon completion of the second tutorial, participants were asked to complete a final block of 54 trials that presented only the green potion throughout. Unlike the previous twelve blocks that forced participants to rely on only one method of interaction per trial, the final one enabled them to click directly on the wisp, pause anywhere or devise a combination strategy. Further details on the design of the “Capture the Wisp” game including the procedure for generating trial data in XML form are included in Appendix A. After the second phase involving a final block of 54 trials testing the Hybrid Chaseor-Pause selection technique, subjects were asked to complete a multiple-choice questionnaire that consisted of a series of statements that were each rated on a Likert scale from 1, strongly disagree, to 5, strongly agree. The questionnaire consisted of eight qualitative statements about the red, blue and green potions that were intended to elicit an opinion as to the difficulty and efficiency of catching wisps with each type of potion.  36  3.1.4  Experimental Design  A repeated measures within-participant design was used as a quantitative comparison of performance between trials involving the red potion and trials involving the blue potion. The independent variable was the acquisition method represented by each potion, or Chase-and-Click by the red potion and Click-to-Pause by the blue potion. Over the course of the 288 trials, participants were presented with an even distribution of each potion, including 144 red and 144 blue. In addition, half of the 288 trials limited the wisp to movement in 1D to provide a dataset that could be used to validate existing models for tasks involving the selection of simple stationary and moving targets. We included three independent variables to determine the relationship between task difficulty and selection technique. These were counter-balanced to add a high degree of variability between trials. These variables included: • Size (W) The size of the Wisp in each trial, as measured by the width of its circle in pixels. [20 pixels, 36 pixels or 60 pixels] • Speed (V) The constant speed at which each Wisp traveled within each of the 288 trials. [100 pixels/sec, 175 pixels/sec or 250 pixels/sec] • Determinism of Movement (DM) The level of determinism in the movement pattern exhibited by the Wisp during each trial. [Non-deterministic or Deterministic] The first phase of the experiment was divided into separate 1D and 2D trials, in order to provide a basis for validating experimental data using previous Fitts’ law models. We averaged the angles at which the target wisps were oriented with respect to the potion using a procedure outlined in Appendix A. Size was varied as a comparison for the traditional Fitts tasks, while the distance from the pointer to the target was kept constant at 240 pixels throughout. We predicted that the Clickto-Pause technique would outperform Chase-and-Click for small targets (20 pixel width).  37  For moving targets, speed was perceived to be a strong factor in determining the index of difficulty and we predicted that the Click-to-Pause technique would outperform Chase-and-Click for fast targets (250 pixels/second). The degree of movement determinism established whether the wisp would move in a predictive pattern that followed Newton’s laws (rebounding off screen edges) or followed a random walk pattern according to predetermined waypoints. We predicted that the acquisition time of targets that exhibit deterministic movement patterns will be less than that of targets that exhibit non-deterministic movement patterns. To summarize, • H1 As target size is reduced, the Click-to-Pause technique will result in lower acquisition times than the Chase-and-Click technique. H1 As target size is reduced, Chase-and-Click will result in lower acquisition times or there will be no significant effect of technique. • H2 As target speeds are increased, the Click-to-Pause technique will result in lower acquisition times than Chase-and-Click technique. H2 As target speeds are increased, Chase-and-Click will result in lower acquisition times or there will be no significant effect of technique. • H3 For deterministic targets, acquisition times will be less than for nondeterministic ones. H3 For deterministic targets, acquisition times will be greater than for nondeterministic ones or there will be no significant effect of determinism. For the second phase, we measured all mouse movements and button events in order to quantitatively classify subject behavior for each trial. We made qualitative observations about the measured target acquisition strategies for each of the participants. Only the green potion was used throughout the final block and participants had the option to Chase-and-Click the Wisps or Click-to-Pause at any time without restriction. The 54 trials were conducted in the same way as the previous 288; however, only two-dimensional movement was investigated while an equal number of trials were distributed among each of the three Wisp sizes, each of the three Wisp speeds and between the two determinism conditions. The results from 38  the final block were compared with the last 54 2D trials from the previous phase for each of the preceding techniques. We predicted that the Hybrid Chase-or-Pause technique would result in a lower acquisition time than either the Chase-and-Click or Click-to-Pause techniques.  • H4 Trials involving the Chase-or-Pause Hybrid will result in lower acquisition times than for those involving either the Chase-and-Click or Click-toPause techniques. H4 Trials involving the Chase-or-Pause Hybrid will result in higher acquisition times than for those involving either Chase-and-Click or Click-to-Pause, or there will be no significant effect of technique.  3.1.5  Performance Measures  The dependent variables for this experiment included the acquisition time, measured from the activation of a potion to the moment the wisp was captured, and the number of errors, measured by the number of clicks or button releases that did not result in the wisp’s capture. In the second phase, the positions of both the mouse cursor and target Wisp were recorded every frame. In addition, each time at which a participant initiated a pause and the total acquisition time were recorded. The qualitative analysis consisted of a statistical measure of the distribution of participant behaviors among four categories: • Chase (C) A behavior in which the participant pursues the moving Wisp without using the option to pause (similar to the Red potion). • Pause Immediately (PI) A behavior in which the participant pauses the Wisp immediately and engages in a Click-to-Pause strategy (similar to the blue potion). • Hybrid (H) A behavior in which the participant initially pursues the moving Wisp, but exercises the option to pause as they get closer. 39  • Error Correction (EC) A behavior in which the participant has missed the Wisp and engages in smaller, Fitts subtasks to catch it. In order to automatically classify each trial, an algorithm was developed to classify a participant’s behavior based on the occurrence of a pause and the relative position of the mouse cursor. In the event that no pauses were observed, Chase mode was selected; if a pause was observed, the cursor position during the final pause was checked against the initial cursor and Wisp positions to determine if the participant chose to pause immediately (distance moved < remaining distance to the Wisp) or home-in on the Wisp (distance moved > remaining distance to the Wisp). In the event of an error, a button click or release that did not result in the capture of the Wisp, the error correction behavior was selected.  3.1.6  Results  For the first phase of the experiment, we conducted a 2 x 3 x 3 x 2 (technique x target size x target speed x determinism) repeated measures ANOVA on the movement time for each dimension. We split the data by dimension to examine the main and interactive effects of 1D and 2D tasks separately, due to the inequivalence of task complexity between the two spaces. We observed no significant main or interaction effects of determinism with the factors of target size, target speed and selection technique. A Bonferroni confidence level adjustment was made to prevent Type I error in pairwise comparison. One participant was removed from our analysis due to a misunderstanding with respect to the overall goals of the experiment, with an emphasis on speed and accuracy. Their performance was at least 2 standard deviations, and in several cases 3, from the mean of all movement times for each condition. As an example, they were 37% slower than the next slowest participant in the 1D Chase condition. Therefore, we report the results of 15 subjects for both phases of this experiment. For the 1D trials, a one-way repeated measures ANOVA was conducted to compare the total acquisition time of each Wisp for Chase-and-Click with Click-to40  Pause techniques of acquisition. Significant interaction effects were observed for technique by target size (F1.081,13.9 = 9.06, p = 0.008), technique by target speed (F2,13 = 6.975, p = 0.003) and technique by target size by target speed (F4,11 = 8.335, p < 0.0005). Post hoc tests involving a Bonferroni correction factor of β = 0.017 and pairwise comparison showed that Chase-and-Click resulted in lower movement times for medium-sized targets (133.74ms, 37% decrease) and slowmoving targets (173.16ms, 45.2% decrease) while Click-to-Pause resulted in lower movement times for small-sized targets (200.9ms, 17.8% decrease) and fast-moving targets (121.35ms, 16% decrease). With the 3-way interaction, Chase-and-Click resulted in lower movement times for small-slow targets (264.76ms, 47.7%), while Click-to-Pause resulted in lower movement times for small-moderate (371.1ms, 12.75% decrease) and small-fast (494.9ms, 18% decrease) targets. The mean movement times for technique by size and speed, normalized for intra-subject timing effects by subtracting the overall mean time by technique, are illustrated in Figure 3.4. The 3-way interaction between technique, target size and target speed illustrated the avantage of Click-to-Pause that was apparent for small-sized, fast-moving or combined small-fast targets over the course of the 1D trials. The 1D mean acquisition times split by size and speed, normalized for intra-subject effects by subtracting the overall mean time by technique, are illustrated in Figure 3.5 for the Chase-and-Click (C) and Click-to-Pause (P) techniques (negative movement times are a result of the normalization to zero). A significant main effect of technique was observed on the number of errors (F1,14 = 34.129, p < 0.0001); however, no significant interaction effects were observed between technique and target size or technique and target speed. Post hoc tests involving a Bonferroni correction factor of β = 0.017 and pairwise comparison showed that Click-to-Pause resulted in fewer errors (M = 3.933, SD = 0.918) compared with Chase-and-Click (M = 70, SD = 11.36). This result was consistent over each size, each speed and each size-speed combination. An error rate of 1.34% was observed for Click-to-Pause while 25.17% was observed for Chase-and-Click. The results of our 1D analysis of error by technique, size and speed are illustrated in 41  Figure 3.4: 1D technique by size and speed  Figure 3.5: 1D mean times by size and speed for both techniques  42  figure 3.6.  Figure 3.6: 1D mean errors by size and speed for both techniques For the 2D trials, a one-way repeated measures ANOVA was conducted to compare the total acquisition time of each Wisp for Chase-and-Click with Click-to-Pause techniques of acquisition. Significant interaction effects were observed for technique by target size (F1.199,13.8 = 16.619, p < 0.005), technique by target speed (F2,13 = 43.911, p < 0.005) and technique by target size by target speed (F4,11 = 37.379, p < 0.005). Post hoc tests involving a Bonferroni correction factor of β = 0.017 and pairwise comparison showed that Chase-and-Click resulted in lower movement times for medium-sized targets (191.1ms, 50.9% decrease), large-sized targets (217.6ms, 69.2% decrease), slow-moving targets (231ms, 54.98% decrease) and moderate-moving targets (122.4ms, 36% decrease) while Click-to-Pause resulted in lower movement times for small-sized targets (409ms, 13.73% decrease) and fast-moving targets (353.6ms, 12.59% decrease). With the 3-way interaction, Chase-and-Click resulted in lower movement times for medium-slow (392.8ms, 95% decrease), medium-moderate (161.1ms, 46.1% decrease), large-slow (197.7ms, 70.5% decrease), large-moderate (245.6ms, 77.6% decrease) and large-fast (209.7ms,  43  61% decrease) targets, while Click-to-Pause resulted in lower movement times for small-fast (1289.7ms, 46.54% decrease) targets. The mean movement times for technique by size and speed, normalized for intra-subject effects by subtracting the overall mean time by technique, are illustrated in Figure 3.7 (negative movement times are a result of the normalization to zero).  Figure 3.7: 2D technique by size and speed The 3-way interaction between technique, target size and target speed illustrated the advantage of Click-to-Pause for targets that were both small-sized and fastmoving over the course of the 2D trials. From the post hoc analysis, there is a strong interaction effect of technique by size by speed that resulted in a substantially increased movement time for small-fast targets. The 2D mean acquisition times split by size and speed, normalized for intra-subject effects by subtracting the overall mean time by technique, are illustrated in Figure 3.8 for the Chase-andClick (C) and Click-to-Pause (P) techniques (negative movement times are a result of the normalization to zero). 44  Figure 3.8: 2D mean times by size and speed for both techniques. C is chaseand-click, P is click-to-pause A significant main effect of technique was observed on the number of errors (F1,14 = 45.239, p < 0.0001); however, no significant interaction effects were observed between technique and target size or technique and target speed. Post hoc tests involving a Bonferroni correction factor of β = 0.017 and pairwise comparison showed that Click-to-Pause resulted in fewer errors (M = 3.933, SD = 0.918) compared with Chase-and-Click (M = 70, SD = 11.36). This result was consistent over each size, each speed and each size-speed combination. An error rate of 1.58% was observed for Click-to-Pause while 31.2% was observed for Chase-and-Click. The results of our 2D analysis of error by technique, size and speed are illustrated in figure 3.9. For the second phase of the experiment, we conducted a 3 x 3 x 3 x 2 (technique x target size x target speed x determinism) repeated measures ANOVA on the movement time for target acquisition comparing the Chase-and-Click, Click-to-Pause and Hybrid Chase-or-Pause techniques. We observed no significant main or interaction effects of determinism with the factors of target size, target speed and 45  Figure 3.9: 2D mean errors by size and speed for both techniques selection technique. A Bonferroni confidence level adjustment was made to prevent Type I errors in pairwise comparison. Significant interaction effects were observed for technique by target size (F1.617,13.383 = 10.172, p = 0.001), technique by target speed (F1.951,13.049 = 18.045, p < 0.005) and technique by target size by target speed (F2.688,12.312 = 5.903, p = 0.003). Post hoc tests involving a Bonferroni correction factor of β = 0.017 and pairwise comparison showed that Chase-and-Click resulted in the lowest movement times for slow-moving (M = -195.58, SD = 32.88, normalized), moderately-moving (M = -102.86, SD = 32.287, normalized)and large-sized (M = -377.256, SD = 51.78, normalized) targets. Click-to-Pause and the Hybrid technique resulted in lower movement times for small-sized (M = 244.18, SD = 53.86, normalized) and fast-moving (M = 11.536, SD = 25.687, normalized) targets, with no significant difference between them from pairwise comparison. With the 3-way interaction, Chase-andClick resulted in lower movement times for large-slow (M = -400.8, SD = 250.6, normalized), large-moderate (M = -469.7, SD = 50.47, normalized) and mediumslow (M = -339.7, SD = 190.6, normalized) targets. Click-to-Pause resulted in  46  the lowest movement time overall for small-fast targets (M = 186.52, SD = 53.5, normalized) over both the Hybrid (M = 351.491, SD = 55.27, normalized) and Chase-and-Click (M = 1158, SD = 204, normalized) techniques. The mean movement times for technique by size and speed, normalized for intra-subject effects by subtracting the overall mean time by technique, are illustrated in Figure 3.10 (negative movement times are a result of the normalization to zero).  Figure 3.10: 2D mean times by size and speed for all three techniques Significant interaction effects were observed on the number of errors for technique by size (F1.24,13.75 = 19.692, p < 0.005), technique by speed (F1.7,13.3 = 16.037, p < 0.005) and technique by size by speed (F2.14,12.8 = 6.9, p = 0.003) as a result in the overlap of Hybrid and Chase-and-Click techniques for mediumsized, slow-moving targets as well as the overlap of Click-to-Pause and Hybrid for small-sized, moderately-moving targets. Post hoc tests involving a Bonferroni correction factor of β = 0.017 and pairwise comparison showed that Clickto-Pause resulted in fewer errors (M = 2.53, SD = 1.3) compared with Hybrid 47  (M = 14.93, SD = 11.27) and Chase-and-Click (M = 63, SD = 22.48). An error rate of 1.17% was observed for Click-to-Pause, 6.91% for Hybrid and 29.17% for Chase-and-Click. Chase-and-Click consistently resulted in higher mean errors than Click-to-Pause for all target sizes, speeds and size-speed combinations; however, there proved to be no significant difference between the Chase-and-Click and Hybrid techniques for medium-sized, large-sized and slow-moving targets, including medium-slow and large-slow combinations. Our data indicates that subjects reverted to Chaseand-Click behavior using the Hybrid technique for targets of this nature, producing a similar pattern of errors. The results of our analysis of error by technique, size and speed are illustrated in figure 3.11.  Figure 3.11: 2D mean errors by size and speed for all three techniques Testing our hypotheses: • H1 As target size is reduced, Chase-and-Click will result in lower acquisition times or there will be no significant effect of technique. REJECTED 48  • H2 As target speeds are increased, Chase-and-Click will result in lower acquisition times or there will be no significant effect of technique. REJECTED • H3 For deterministic targets, acquisition times will be greater than for nondeterministic ones or there will be no significant effect of determinism. NOT REJECTED • H4 Trials involving the Chase-or-Pause Hybrid will result in higher acquisition times than for those involving either Chase-and-Click or Click-to-Pause, or there will be no significant effect of technique. NOT REJECTED In summary, our results were: • For 1D target selection, significant interaction effects were observed for technique by size, technique by speed and technique by size by speed on movement time. Chase-and-Click proved to be faster for medium-sized, largesized, slow-moving and moderately-moving targets and all combinations thereof. Click-to-Pause proved to be faster for small-sized and fast-moving targets and the combination of both. The Chase-and-Click technique resulted in substantially more errors (25.17%) than Click-to-Pause (1.34%). No significant main or interaction effects were observed for the target determinism of movement. • For 2D target selection, significant interaction effects were observed for technique by size, technique by speed and technique by size by speed on movement time. Chase-and-Click proved to be faster for medium-sized, largesized, slow-moving and moderately-moving targets and all combinations thereof. Click-to-Pause proved to be faster for small-sized and fast-moving targets and the combination of both. The Chase-and-Click technique resulted in substantially more errors (31.2%) than Click-to-Pause (1.58%). No significant main or interaction effects were observed for the target determinism of movement. • The graphs of technique versus size in 1D and 2D depict a crossover of Chase-and-Click and Click-to-Pause between small and medium, indicat49  ing a threshold below which Click-to-Pause outperforms Chase-and-Click. Likewise, the graphs of technique versus speed in 1D and 2D depict a crossover of Chase-and-Click and Click-to-Pause between moderately-fast and fast, indicating a threshold above which Click-to-Pause outperforms Chase-andClick. Speed has relatively no effect on movement time for Click-to-Pause trials, but makes Chase-and-Click tasks extremely difficult, especially when in combination with small targets. • In the second phase, significant interaction effects were observed for technique by size, technique by speed and technique by size by speed on movement time. Chase-and-Click proved to be faster for large-sized, slow-moving and moderately-moving targets and all combinations thereof. Click-to-Pause and Hybrid proved to be faster for small-sized and fast-moving targets, while Click-to-Pause outperformed Hybrid for the combination of the two. The Chase-and-Click technique resulted in substantially more errors (29.17%) than Hybrid (6.91%) or Click-to-Pause (1.17%). No significant main or interaction effects were observed for the target determinism of movement.  3.1.7  Discussion  For the first phase of the experiment, we observe that the Chase-and-Click technique resulted in lower acquisition times than Click-to-Pause in 1D for conditions involving a target size of 36px (medium) or 60px (large) as well as conditions involving a target speed of 100px/s (slow) or 175px/s (moderate). We infer that for larger and slower wisps, the overhead of clicking on a widget prior to the ballistic phase of movement outweighed the benefit of freezing the wisps. In addition, the area in which the wisp had to travel was more confined in the 1D trials and wisps were likely to rebound and approach the cursor voluntarily, an assistive phenomenon that would never be realized with Click-to-Pause. The Click-to-Pause technique resulted in lower acquisition times than Chase-andClick in 1D for conditions involving a target size of 20px (small) as well as conditions involving a target speed of 250px/s (fast).We infer that when stationary, the smaller wisps provide a sufficient width for error-free selection that is reduced to a 50  smaller effective width by the introduction of speed. Speed has very little impact on the Click-to-Pause technique, as can be seen by the relative plateau in figure 3.4. These graphs indicate that fast targets are more difficult with the Chase-and-Click technique, while figure 3.5 demonstrates that combined small and fast targets are substantially more difficult for the Chase-and-Click technique than any other conditions. In the 2D trials, the Chase-and-Click technique resulted in lower acquisition times than Click-to-Pause for target sizes of 36px (medium) and 60px (large) as well as speeds of 100px/s (slow) and 175px/s (moderate). This confirms the observation in 1D that the overhead of clicking on a widget prior to the ballistic phase of movement outweighed the benefit of freezing the wisps. The Click-to-Pause technique resulted in lower acquisition times than Chase-and-Click for a target size of 20px (small) as well as a target speed of 250px/s (fast). This confirms the observation from 1D that a reduction in the effective width of a moving target results in a more difficult Fitts’ task than the stationary case. In addition, we confirm that speed has very little impact on the Click-to-Pause technique, as observed by the relative plateau in figure 3.7, while figure 3.8 confirms that combined small and fast targets are substantially more difficult for the Chase-and-Click technique than any other conditions. The graphs in figures 3.4 and 3.7 indicate a crossing of the techniques at a target size between 20px (small) and 36px (medium) as well as at a target speed between 175px/s (moderate) and 250px/s (fast). This intersection of size and speed appears to define a threshold region where target size and speed dominate the movement time over the initial button-down operation incurred by the Click-to-Pause technique. Although the sequence of events for the Click-to-Pause technique is virtually the same as that of Chase-and-Click, the engaged muscles that keep the mouse button depressed limit the subject’s speed of movement with the mouse[30]. We observe that by restricting the pause operation to a confined location on the screen, users are unable to optimally pause during the target object’s movement, to minimize their own pointer distance. The cognitive overhead of a button-down operation prior to pointer movement establishes a minimum threshold for movement 51  time in every trial. For both the 1D and 2D trials, the Click-to-Pause technique resulted in substantially fewer errors that the Chase-and-Click technique, across all target sizes and speeds, as observed in figures ?? and ??. Errors were counted as either a button-down event that didn’t result in object selection for the Chase-and-Click technique or a buttonup event that didn’t result in object selection for the Click-to-Pause technique. In the Chase-and-Click trials, we conclude that subjects sacrificed accuracy for speed by attempting to click on wisps in rapid succession. In the Click-to-Pause trials, we conclude that subjects sacrificed speed for accuracy by performing a carefully controlled movement over the wisp before releasing the mouse button. We attribute the behavior observed in the Click-to-Pause trials to the muscles engaged to keep the mouse button depressed and the perception of a substantial penalty of returning to the blue web in the event of a miss. The results of the 1D and 2D trials for small sized and fast moving targets have forced us to reject the null hypotheses H1 and H2, due to the Click-to-Pause technique resulting in lower movement times for both conditions. We have failed to reject the null hypothesis H3 as there was no significant effect of determinism of movement on the mean acquisition time. This may be attributed to the implementation of non-deterministic movement patterns in a sequence of waypoints that the wisp followed. The further away that waypoints were plotted, the more predictive a single movement step was, resulting in a mixed deterministic movement that was dependent on the proximity of the next waypoint. The results of the second phase suggest that the occurrence of Chase-and-Click and Click-to-Pause strategies are consistent with observations made in the first phase, where users were prone to chase larger or slower moving targets while pausing the smaller or faster ones. The Click-to-Pause and Hybrid Techniques resulted in lower acquisition times than Chase-and-Click for small-sized, fast-moving and small-fast targets. For the combined small-fast targets, Click-to-Pause resulted in significantly lower acquisition times than the Hybrid technique, despite the speedfor-accuracy tradeoff. We hypothesize that there is a critical speed beyond which 52  interception of a moving target through optimization is less efficient than a single, precision-based movement over a stationary target. Crossman and Goodeve[11] describe pointer movement by two phases, a ballistic phase and a precision phase. The ballistic phase is defined by a distance-covering movement in which the cursor is moved along a fixed amplitude toward a target, while the precision phase is defined by sensory-feedback that allows the cursor to move in a short, corrective motion over the target. The Chase-and-Click technique consists of a single predictive ballistic movement, followed by a series of corrective precision movements based on target speed. The Click-to-Pause technique consists of a single ballistic movement to a known location followed by a short corrective precision phase that is dependent on target size. In the case of the hybrid strategy, users tended to optimize how their time was allocated to ballistic motion and corrective precision, based on the speed, size and determinism of motion for each target. In the second phase, the mouse and target position logs for each subject were analyzed in order to categorize the technique used for each trial. The results are summarized by the percentage of trials for which each technique was chosen by target size, then by target speed in figure 3.12.  Figure 3.12: Technique chosen by target size, then speed As predicted from the first phase, the Chase-and-Click technique is reserved for 53  larger and slower targets, while the Click-to-Pause technique is used more frequently as targets become smaller or faster. Observation of the Hybrid Chase-orPause technique was classified as a homing strategy in which subjects moved the mouse pointer over at least half the intial distance between the pointer and the target. Our data illustrated a trend among the trials of the second phase of the experiment, in which a homing strategy was used to pursue, pause and acquire targets. Figure 3.12 shows that for small (20px) targets, the Hybrid Chase-or-Pause technique was used more frequently than Chase-and-Click, based on the descriptive statistics of our behavioral categorization. The mean scores from the post-experiment questionnaire are listed by the eight statements that subjects were asked to rate in Table 3.1.  STATEMENT I caught wisps faster with the blue potion than with the red I made fewer mistakes with the blue potion than with the red When the wisp’s movement was predictable, the red potion made them easier to catch I preferred using the blue potion for smaller wisps I preferred using the red potion for slower wisps The faster wisps were too difficult to capture using the red potion The green potion made catching wisps easier than the blue With the green potion, I chose to chase wisps before freezing  MEAN 2.6  SD 1.36  3.8  1.18  4.6  0.5  4.3 4.4 4.1  1.14 0.63 1.06  4.1  1  3.1  1.39  Table 3.1: Responses to post-questions  From the mean response scores associated with these statements, we observe that subjects somewhat disagreed that they were able to catch wisps faster with the blue potion than with the red (M = 2.6, SD = 1.36); however, subjects agreed that they made significantly fewer mistakes with the blue potion than with the red (M = 3.8, 54  SD = 1.18). This is consistent with our results, as the blue potion reduced the number of errors from 25.17% to 1.34% for 1D trials and from 31.2% to 1.58% for 2D trials. With respect to the determinism of movement, subjects strongly agreed that the red potion made wisps easier to catch when their movement was predictable (M = 4.6, SD = 0.5), despite our analysis revealing no significant effect of movement determinism on either acquisition time or the number of errors. However, subjects agreed that the faster wisps were generally too difficult to capture using the red potion (M = 4.1, SD = 1.06), which is reflected in the data as a significant interaction effects of technique, size and speed with increased movement times and error rates for fast wisps. In order to determine subject preference for potion or technique based on the extremes of wisp size and speed, subjects were asked to rate their preference of using the blue potion for smaller wisps and the red potion for slower wisps. In each case, subjects were in agreement with the statement, revealing a preference for the hold technique in capturing smaller wisps (M = 4.3, SD = 1.14) and for the chase technique in capturing slower wisps (M = 4.4, SD = 0.63). With respect to the second phase, subjects agreed that the green potion made catching wisps easier than with the blue potion (M = 4.1, SD = 1). On the other hand, subjects neither agreed nor disagreed that they attempted to chase wisps before freezing them with the green potion (M = 3.1, SD = 1.39). This is contrary to our results, as the Hybrid technique performed as well as Click-to-Pause for smaller and faster wisps, but was outperformed by Click-to-Pause for small-fast wisps. Subjects were also asked to complete a post-experiment questionnaire that consisted of three long answer questions related to the advantages and disadvantages of the “freezing” potions and suggestions for improvement based on experience. When asked what the perceived strengths of the “freezing” potions were, eleven subjects noted that the blue and green potions made catching wisps much easier, particularly the small and fast ones. One subject commented that “it removed the uncertainty of wisp’s movements”, while “getting to it [without] predicting where it would be”. This indicates a general reduction in the perceived difficulty of the task, with the challenge attributed to the interception of a moving target and the 55  determinism of its motion. When asked what the perceived weaknesses of the “freezing” potions were, twelve subjects noted that it sometimes took much longer to freeze and drag the cursor over a wisp than to simply click on it in mid-motion. One subject commented that “clicking and dragging somehow [felt] slower than just moving the mouse”, while another suggested that they had to “slow down when dropping the thread”. This is indicative of the additional overhead of depressing the mouse button prior to movement and the reversal of a user’s conceptual model of selection on a button-down event, as discussed by Isokoski[30]. An interesting observation from one subject was that the blue and green potions gave “the illusion that you froze time so you’d aim slower”. This is a direct reflection of the finding by Tresilian and Lonergan[44] that subjects move faster when temporal precision demands are greater, due to the motor control system attempting to optimize spatio-temporal accuracy. Finally, subjects were asked to comment on how they might improve the effects of either the blue or green potions. One subject suggested that it shouldn’t be “necessary to drag the ’web’ directly onto the wisp, but only across it”, indicating that they recognized the speed-accuracy tradeoff of the task[15] and favored an emphasis on the ballistic phase of movement over the precision phase of correction[11]. We believe that the value of the precision phase of movement would be more apparent in a follow-up study involving a dense field of moving targets, in which the probability of crossing several in an effort to select only one posed an additional challenge. Three subjects suggested the creation of a potion to slow the wisps down instead of freezing them completely, suggesting a preference for a simplified chase task over a static object selection task. This suggestion supports an earlier observation that subjects view the mechanism associated with the blue and green potions as a burden that takes additional time, despite the fact that the target is stationary. As a result, subjects tend to favor aimed ballistic movements in the interception of easy targets over the control afforded by the precision movement in a simplified task. In addition, two subjects suggested that the distance between the potion and the 56  wisp should be reduced by either “attract[ing] the wisps” or “bring[ing] the wisp closer”, a method that is akin to the Drag-and-Pop or Drag-and-Pick techniques by Baudisch et al.[4]. This technique, while successful for static objects, may violate the spatial coherence of moving objects on the screen, as they are rearranged to be within spatial proximity of the cursor and subsequently replaced in their original trajectories. Seven subjects suggested some form of enhancement to either the cursor or the target, scaling either the size of the wisp or the effective area of the cursor, while one subject suggested creating a selectable trail behind the target wisp. These suggestions resemble the techniques discussed in Chapter 2, including area cursors[20][7], target expansion[36] and comet tails[22]. Static area cursors would be impractical for densely clustered targets and Gunn[22] confirms that adaptive area cursors, such as bubble cursor[20], would resize too frequently in such a field with moving targets. Enhancing the trail behind each wisp would result in occlusion or competition for screen space[5] in dense clusters of moving targets.  3.2  Conclusion  In this chapter, we presented the results of our experiment to evaluate the Clickto-Pause technique for the selection of moving targets. A comparison was made between Click-to-Pause and the Chase-and-Click technique for both 1D and 2D targets that varied in size, speed and determinism of movement. In both the 1D and 2D trials, the Click-to-Pause technique demonstrated lower target acquisition times for targets that were small in size (20px), fast moving (250px/s) or a combination of both. A threshold region was identified for target size and speed that resulted in lower acquisition times for Click-to-Pause due to factors of speed and size dominating the movement time over the additional pause operation. Overall, Click-to-Pause resulted in substantially fewer errors than the Chase-and-Click technique due to conflicting goals in the speed-accuracy tradeoff. No significant effect was observed for the determinism of movement on target acquisition time. The Hybrid Chase-or-Pause technique allowed subjects to arbitrarily pause moving  57  targets, indicating the potential for movement optimization. The Hybrid technique resulted in mean acquisition times similar to the Click-to-Pause method, except for small-fast targets in which the Click-to-Pause method resulted in even lower mean times. Subjects were observed applying a homing strategy in 16% of all successful target acquisitions, occurring more frequently than the Chase-and-Click approach for small targets. The experiment provided us with a balanced data set that included acquisition times for targets that were of varying sizes, speeds and levels of determinism in movement to be applied to the extension models for Fitts’ law. In the subsequent chapter, we will discuss the development of an interactive video browser for multi-view video feeds and its evaluation in an informal user study that compares two different display modes for information boxes and menus, while analyzing how often users apply the hybrid Chase-or-Pause technique.  58  Chapter 4  Moving Target Selection in an Interactive Video Context Our motivation for testing our Hybrid Chase-or-Pause technique in an interactive video context is to establish a novel mechanism for interacting directly with video content. There are currently no such mechanisms available, with the exception of a few experimental video players that use direct manipulation to control video playback by dragging dynamic content along its natural path[12][32]. A standard set of design heuristics will be necessary to address the selection of moving targets in the video, the occlusion of target objects and their intermittent visibility. We have developed a multi-view video navigation tool that incorporates our Hybrid Chaseor-Pause technique as a means of pausing video playback while selecting video objects from content in the paused frame. The video navigation environment is fully functional and incorporates feedback dialog, selection menus and navigation toolbars that are incorporated directly into the display of the video player. We are uncertain how our Chase-or-Pause interaction technique will be affected by the presence of these elements, given that each may affect the visibility of the video content. Our goal is to add structure to such extraneous information to prevent interruption of video playback or obstruction of its content. We have developed two different display modes to provide structure for extraneous information, including a scaled panel display and a picture-in-picture 59  display. The purpose of each display mode is to enable the selection of objects in the video frame, display object-specific information and provide context awareness of the current video state. For our second experiment, we conducted an informal user study that was meant to demonstrate the Hybrid Chase-or-Pause technique in a multi-view hockey video. In the study, subjects were given tasks such as selecting specific players on the ice, gaining additional information about them or adding them to a pin menu for subsequent retrieval. The interface consisted of a seek bar, camera angle button, pin menu for fast object retrieval and an information panel for object details. Although two minutes of hockey footage from nine different camera angles had been annotated, we implemented only the four most diverse angles in our browser, due to the hardware limitations of our experiment platform. The final interactive video browser is illustrated in Figure 4.1.  Figure 4.1: The interactive video browser, a selected hockey player has a blue bounding box  60  4.1  Empirical Evaluation  We developed two different display modes for the interactive video browser in order to qualitatively observe how the Hybrid Chase-or-Pause technique is utilized under different environmental constraints. The interactive video browser was developed as a fully functional application that engages users with peripheral windows and menus to provide additional information on or provide storage for selected players on the ice. We have developed a series of tasks based on the functional components of the browser and will assess whether one display mode affords an advantage over the other, with respect to task completion time. Our display modes consist of both a Picture-in-Picture display and a Scaled Panel display for peripheral windows and menus. The Picture-in-Picture display mode implements non-modal windows as floating transparencies that can be maneuvered around the display and closed by clicking on the familiar Windows “X” in the upper right-hand corner. Although minimization and scaling were not implemented, the windows can be translated to any point on the screen, outside a particular region of interest for a particular camera angle. A screenshot of the Picture-in-Picture display mode is shown in figure 4.2, with a player information dialog in the lower left-hand corner and the player pin menu in the upper right. The most apparent constraint that this environment imposes on our Chase-or-Pause interface is the occlusion of potential targets while the windows are visible on the screen. The Scaled Panel display mode scales the video in the foreground down by 33% from 960x540 to 640x360 and displays non-modal windows in the peripheral region along the right-hand and bottom of the screen. Either the player information window or the pin menu can be closed using the “X” button in the upper right-hand corner. A screenshot of the Scaled Panel display mode is shown in figure 4.3, with a player information dialog at the bottom and the player pin menu on the right. The most apparent constraint that this environment imposes on our Chase-or-Pause interface is the reduction of target sizes from 27 x 45 pixels to 18 x 30 pixels.  61  Figure 4.2: The picture-in-picture version of the interactive video browser  Figure 4.3: The scaled panel version of the interactive video browser  4.1.1  Apparatus  The user study was conducted on a Dell XPS notebook with a 2.60GHz Core2 Duo CPU with 4GB of RAM running Windows XP Professional. For the purposes of this experiment, “Enhance pointer precision” was disabled and the pointer speed was set to 6 (of 10). An external LCD display was used at a resolution of 1280 x 960 pixels at a refresh rate of 60Hz, while the laptop LCD display was used by the 62  experimenter for casual observation. A standard Microsoft optical wheel mouse was used as the input pointing device and the Flash player was set to full screen while running the program. The experimental setup is illustrated in figure 4.4.  Figure 4.4: Experimental apparatus for interactive video browser  4.1.2  Procedure  Participants were asked to complete a tutorial prior to starting the experiment, which introduced the onscreen widgets, the hockey players as selectable objects and the selection interface consisting of clicking directly on an object (Chase) or holding down the mouse button and selecting a stationary object by releasing the button (Pause). The tutorial also introduced the five task categories from which each participant was asked to complete two before proceeding to the experiment.  • Selection of a player: participants would need to select a player currently on the ice from a name provided. Participants could click directly on players or highlight each while in pause. 63  • Finding basic player information: once having selected a player, participants would need to click on their photo in the lower left or hold the mouse button down on top of the player to select from a pop-up menu to display the player information window. • Finding advanced player information: once a player is selected and their information window is displayed, participants would click the up arrow at the top of the window to display a full screen of player statistics. This modal window could be closed by a widget in the upper right corner. • Pinning a player: participants would need to select a player currently on the ice and use either the pin button or hold the mouse button down on top of the player to select from a pop-up menu to add that player to the pin menu on the right. • Pinning multiple players: this task serves as an extension to the last task, requiring participants to add a specific set of players on the ice to the pin menu, three tasks requiring 3 players and two tasks requiring 4, by selecting a player and either clicking on the pin button or holding the mouse button down on top of the player to select from a pop-up menu to add that player to the pin menu on the right. Players could be added in any order. Each task was preceded by a modal task description window that could be retrieved using the “New Task” button in the lower right. The task would be summarized in point form at the top of the screen and participants would be timed from the closure of the task dialog until the correct selection of a player, the display of requisite information or the population of the pin menu was complete. Players were offered breaks and were informed of their progress in-between tasks. Participants were encouraged to explore the video space by switching camera angles and maneuvering along the timeline using the seek bar to obtain a better view of the player object that was the target for the task. After the successful completion of each task, the state of the pin menu was cleared, the position in the timeline was reset and non-modal dialog windows were closed to establish a consistent state for  64  the onset of each new task. The experiment was structured into 2 blocks, one for the Scaled Panel version of the interactive video browser and one for the Picture-in-Picture version. Each block consisted of 25 trials that consisted of 5 tasks from each of the aforementioned categories that were presented in a random order that was consistent between participants. Subjects completed a multiple-choice questionnaire after each block as a form of self-evaluation of their performance and an evaluation of the display mode.  4.1.3  Application Development  We had previously recorded a hockey game at UBC using nine digital video cameras and transferred the video to a Mac Pro for editing, synchronization and export to Flash video format (.flv). In addition, we used a co-authored video annotation tool, written in Flash, to mark player positions in each of the nine camera views for a duration of two minutes during the first period of the game. This data was exported from a MySQL database in XML format and imported into our interactive browser to define selectable regions on the screen on a frame-by-frame basis. The tracking data for each hockey player was interpolated between key frames to provide a continuous region of interaction over them for the duration of the video.  4.1.4  Participants  We enlisted six participants, 2 female and 4 male, to participate in the experiment over the course of a one hour session. All participants were experienced computer users with normal or corrected-to-normal vision and no color blindness. Of the six participants, all in the age range of 18-39, each was a right handed mouse user and none of them had any limiting injuries. No subjects from the previous experiment in Chapter 3 were included in the six participants here. A pre-questionnaire established that the average self-rating of hockey knowledge was 2.5, on a scale from 0 to 5, or from novice to expert, respectively.  65  4.1.5  Experimental Design  A 2 x 5 (Display Mode x Task Type) repeated measures within-participant design was used as a quantitative comparison of task completion time between the Scaled Panel display mode and the Picture-in-Picture display mode. In addition, a log was maintained of all mouse down events, all mouse up events, all task completions that consisted of the application timestamp, the event type and the mouse position at the time of the event. This data was used to determine the timing between mouse down and mouse up events as an indication of whether hockey players were selected while the video was paused. We hypothesize that participants will achieve, on average, lower task completion times with the scaled panel mode over the picture-in-picture mode. In addition, we hypothesize that a greater number of correct target selections will be performed during a pause than with a click operation. To formalize our hypotheses:  • H1 Tasks performed in the Scaled Panel mode will result in lower completion times than those performed in Picture-in-Picture mode. H1 Tasks performed in the Picture-in-Picture mode will result in lower completion times or there will be no significant effect of display mode. • H2 Click-to-Pause and Chase-or-Pause will be used more frequently than Chase-and-Click. H2 Chase-and-Click will be used at least as frequently as Click-to-Pause or Chase-or-Pause.  4.1.6  Performance Measures  The primary dependent variable for this experiment was the task completion time for each of the five task categories, compared over 6 subjects between the two display mode conditions. This task measure, in seconds, accounts for a sequence of interactions that include identification and selection of hockey players, the display 66  of non-modal information windows and the addition of players to the pin menu. A repeated-measures one-way ANOVA was conducted to determine whether there was a significant effect of display mode on task completion time, a significant effect of task type on task completion time and an interaction effect between display mode and task type. In addition, a detailed log was maintained during both blocks of the experiment for each subject, tracking every mouse button event and the completion of each task. The current player selection was logged with each event, with a new player selection being associated with either a button down event, as a click, or a button up event, as a pause. If a new player was selected on a button up event, the time interval since the previous button down event was examined to determine whether a pause occurred. The standard interval for double-click timing is 500ms[40]; however, we selected a margin of 1 second as a measure above the experimental extreme of 750ms[30].  4.1.7  Results and Discussion  A within-subjects repeated measures ANOVA was conducted to determine whether the condition of display mode had a significnat effect on the task completion time. There was no significant main effect of display mode on task completion time (F1,5 = 0.006, p = 0.944), no interaction effects of display mode by task type (F4,2 = 0.643, p = 0.638) and neither the scaled panel mode (M=34.78s, SD=12.35) nor the picture-in-picture mode (M=35.01s, SD=11.75) resulted in lower task completion times. As a result, we did not reject the null hypothesis H1.  • H1 Tasks performed in the Picture-in-Picture mode will result in lower completion times or there will be no significant effect of display mode. NOT REJECTED A significant main effect was observed for task type on task completion time (F4,2 = 29.799, p < 0.005) as expected, due to the varying complexity of each 67  task category; however, no interaction effects were observed between task type and display mode. The tasks ranged in mean completion time as player select (M=16s, SD=6.41), basic player info (M=22.48s, SD=10.92), extended player info (M=27.44s, SD=12), single player pin (M=32.37s, SD=20.35) and multiple player pin (M=76.18s, SD=19.81). Of all the correct player objects selected within the scaled panel display mode, 89.43% were selected during a pause operation. Of all the player objects that were selected within the picture-in-picture display mode, 79.74% were selected during a pause operation, indicating that in both cases, participants were primarily utilizing either a Click-to-Pause or Hybrid Chase-or-Pause strategy. Overall, 84.4% of all correct player objects were selected during a pause operation. As a result, we rejected the null hypothesis H2.  • H2 Chase-and-Click will be used at least as frequently as Click-to-Pause or Chase-or-Pause. REJECTED The rejection of H1 indicated that the task completion time of subjects was relatively unaffected by the reduced size of the target player objects. This is counterintuitive to our results in the previous experiment in Chapter 3, as decreased target size was found to result in increased selection times for both Chase-and-Click and Click-to-Pause. We attribute the higher frequency of Click-to-Pause or Hybrid selection techniques over Chase-and-Click in the scaled panel display mode to the smaller target objects. As a result of the players in the video moving at varying speeds and in nondeterministic movement patterns, we were unable to split the data for target acquisition time by speed, distance or determinism. Errors were not recorded, due to the open-ended navigation environment and subjects were primed with a goal of speed. One factor that we anticipate may have contributed to higher completion time was the overall task complexity, which we attempted to counter-balance by introducing 68  tasks from five categories of difficulty in a random order, presenting the same order to all subjects. Although the most complex task involving the pinning of five different players took significantly longer than the simplest task involving a single player selection, no signficant interaction effect between task type and display mode was observed. After each display mode condition of the experiment, subjects were asked to complete a multiple choice questionnaire that consisted of a series of statements that were each rated on a Likert scale from 1, strongly disagree, to 5, strongly agree. As the order in which subjects completed the experiment was randomized and balanced, three of the six subjects completed the picture-in-picture display mode questionnaire between experiments and the scaled panel questionnaire after the final experiment, the other three in reverse order. The mean ratings for the picture-inpicture questionnaire responses are summarized in table 4.1, while those for the scaled-panel display mode are summarized in table 4.2. STATEMENT Picture-in-Picture mode allowed me to effectively follow the game I found the floating windows distracting I paused prior to selection more often than I clicked directly on a player Catching target players was difficult in picture-inpicture mode  MEAN 3.57  SD 1.51  2.43 4.14  1.62 1.21  1.57  0.78  Table 4.1: Picture-in-picture questionnaire responses  With respect to the picture-in-picture display mode, subjects agreed that the picturein-picture mode allowed them to follow the game (M = 3.57, SD = 1.51), while disagreeing with the notion that the floating overlay windows were too distracting (M = 2.43, SD = 1.62). In addition, subjects agreed that they tended to pause prior to player selection, rather than clicking on them in mid-motion (M = 4.14, SD = 1.21), while disagreeing with the notion that catching target players was difficult in this mode (M = 1.57, SD = 0.78). This is consistent with our experimental results that 69  STATEMENT The scaled panel mode allowed me to effectively follow the game I found the scaled hockey window more difficult to watch I paused prior to selection more often than I clicked directly on a player Catching target players was difficult in the scaled panel mode  MEAN 2.86  SD 1.46  3.86  1.34  3.71  1.38  3.14  1.57  Table 4.2: Scaled panel questionnaire responses  79.8% of all correct target selections in picture-in-picture mode were performed at the end of a pause operation, while each subject completed every task unassisted. On the other hand, in response to the scaled panel display mode, subjects neither agreed nor disagreed with the statement “the scaled panel mode allowed me to follow the game” (M = 2.86, SD = 1.46). Subjects agreed with the suggestion that the scaled hockey window was more difficult to watch (M = 3.86, SD = 1.34) than the picture-in-picture mode. With respect to the frequency of paused player selection, subjects agreed that this was their technique of choice (M = 3.71, SD = 1.38). This is consistent with our results, as click-to-pause accounted for 89.4% of all correct target selections in scaled panel mode. Subjects neither agreed nor disagreed that catching player targets was more difficult in scaled panel mode (M = 3.14, SD = 1.57), despite the 33% reduction in video size and the general opposition to the display layout. Subjects were also asked to complete a post-experiment questionnaire that consisted of three long answer questions that encouraged subjects to discuss their experiences with each of the display modes in the interactive browser application. When asked what the perceived strengths and weaknesses of the picture-in-picture mode of interaction were, four subjects emphasized their preference for full-screen video that was constantly maintained at the 1280x960 screen resolution. One subject commented that they were “more immersed in the hockey game with full screen”, 70  when it was central to the action rather than being reduced to a peripheral window during an interactive task. Only two subjects suggested that the overlay windows were “too obstructive” while performing tasks; however, one subject praised the ability to reposition the overlay windows and resize the player pin menu, stating “I have control over where things are at”. When asked about the strengths and weaknesses of the scaled panel display, five subjects expressed dissatisfaction with the scaled video, stating that it was “too small to follow play while looking at the player info”. One subject stated that “it’s annoying to use the image with continually changing size every time”, in reference to displaying player information or interacting with the pin menu and the reduction of the video. Overall, subjects were dissatisfied with the constant scaling of the video to 66% of its original size and the allocation of screen space for peripheral windows. When given a choice between the two display modes, five of the six subjects chose the picture-in-picture display, allowing them to “keep an eye on the game and not lose track of what’s happening while still getting additional details”. One subject requested a mix of the two modes and a stated a preference for consistency over gaining an advantage for following the game or reading peripheral statistics. As future work, a more quantitative approach to comparing video display modes for our interaction techniques for moving objects would need to incorporate shorter tasks with a carefully controlled sequence of videos with a consistent perspective and time base. A detailed comparison of the picture-in-picture and scaled panel display modes would require a reduced user interface with no advancement the timeline or option to switch perspectives. These elements would need to be automated and tailored towards specific goals of object identification, object selection and event tracking.  4.2  Conclusion  In this chapter, we developed an interactive video browser for multi-view video navigation to evaluate the usage of our Hybrid Chase-or-Pause technique in a real-  71  istic application. In addition, we compared two display environments, scaled panel and picture-in-picture, to determine the effects of video scaling and occlusion on task performance within the video frame. Our results indicated no significant effect of display mode on task completion time nor any interaction effects between display mode and task type. We analyzed a log of all mouse events and target selections to make observations of which interaction technique subjects chose most frequently in each display mode. Our results demonstrated that subjects selected player targets while paused more frequently than they clicked directly on a player in mid-motion. Subjects selected player targets while paused 84.43% of the time in the scaled panel mode and 79.74% of the time while in the picture-in-picture mode. User feedback in a post-questionnaire indicated a clear preference for the picture-in-picture display mode over the scaled panel approach. In the subsequent chapter, we present an analysis of the implications of our quantitative results from the first experiment on our approach to modeling Fitts’ law for a moving target in 1D and 2D space. We follow this with our conclusions, a declaration of contributions and a discussion of future work.  72  Chapter 5  Fitts’ Law Modeling This chapter explores several modern approaches to mathematically modeling a virtual pointing task based on proposed extensions to Fitts’ law that account for bivariate pointing in 2D space and target movement. We open with a discussion of the most general categorization of deterministic versus stochastic modeling and gradually assess which elements of each model are most applicable to our scenario for the pursuit and interception of moving targets in 1D and 2D space. We formulate a theory on the relationship of target speed and target direction with respect to the cursor, offering partial validation through the analysis of an observed trend in our empirical data gathered in the experiment from Chapter 3. Our motivation for studying the correlation of existing models for bivariate pointing and the pursuit of moving targets is to determine which characteristics of each accurately model real behavior in virtual pointing tasks. There are currently no modifications to Fitts’ law that model the selection of moving targets in two dimensional space. For validation purposes, the trials from the first phase of our experiment in Chapter 3 have been carefully divided to provide an equal number of cases in one dimension and two dimensions, as well as an equal number of Chase and Pause techniques for each dimension. The data gathered from all trials involving the Pause technique provide a basis for validation of traditional static Fitts’ law models, while the trials involving the chase technique provide a basis for validation of refined moving target models. 73  We begin with a discussion of the deterministic and stochastic approaches to modeling pointing tasks in two dimensions, followed by an overview of models for moving targets. We present our approach for modeling the data from our first experiment and discuss the limitations of existing models. By returning to first principles of kinematics in Euclidean space, we analyze a trend observed in our data for the selection of one dimensional moving targets and propose a method for extending this principle into two dimensions.  5.1  Deterministic Versus Stochastic Modeling in Two Dimensions  The approach to modeling pointing tasks in two dimensions has generally treated the width of a target in one dimension as an amplitude constraint that can be equally mirrored along a vertical axis. In this sense, researchers have defined an equivalent height parameter for targets that are situated at an angled distance from a cursor origin to the center of the object, in 2D space. The height and width of a target object were not initially deemed to be independent[35], as early modifications to Fitts’ law used the two parameters interchangeably and demonstrated that as one tended to infinity, the other became the sole contributor to the index of difficulty. Mackenzie and Buxton[35] adopted the original Fitts’ law equation with the widely accepted Shannon formulation[42]. The variation that they introduced was a substitution of an “effective width” for the conventional width parameter, consisting of either the minimum of the height or width, or the length of the intersecting vector through the target at a given angle. This relation is described by equation 5.1.  MT = a + b log2  D +1 We f f  (5.1)  Accot and Zhai[1] postulated that, while this model supported cases in which W = H or one tends to infinity, it could not support cases in which the width and 74  height became unequal, by elongating one or the other. They suggested that these parameters were independent, with W dominance leading to an “amplitude pointing” task and H dominance leading to a “directional pointing” task. Their empirical results of this theory were developed into a model that included both terms in the index of difficulty calculation and a separate weighting for the height term denoted by η in equation 2.2 from Chapter 2, repeated here. D 2 +η W  MT = a + b log2  D 2 +1 H  Grossman and Balakrishnan[19] sought to generalize pointing tasks in two dimensional space even further by proposing a probabilistic approach in which the index of difficulty was modeled as the probability of hitting a target in two dimensions given a normal distribution of possible hits. They were able to generalize over virtually any target shape and account for any approach angle using a bivariate normal density function to represent the shape in equation 2.3 from Chapter 2, repeated here.  IDPr = F (  R  bnd f (X ,Y ) dY dX )  This model has most recently been refined to empirically demonstrate its application to objects of arbitrary shape[21]. However, the modeling of pointing tasks involving moving targets has only been approached exclusively from one dimension using the basic principles of kinematics.  5.2  Interception of Moving Targets  One of the earliest approaches to extending Fitts’ law for moving targets was Jagacinski’s model involving a Welford formulation[46] that incorporated the target velocity into the precision phase term for W. The velocity term had no contribution to the ballistic phase, represented by the distance term, but contributed a vector (sign variant) term to the effective width. This relationship is illustrated in equation 2.4 from Chapter 2, repeated here.  75  MT = c + dD + e(V + 1)  1 W  −1  Hoffman[25] developed a first order model that employed continuous control of both position and velocity, which he validated using the data from Jagacinski’s previous study[31] for a strong fit. The first order model adopted a steady state position error (V/K) that was applied to a human movement model and found to affect both movement amplitude (D) and target width (W). The two variations of this model are presented for targets moving toward the cursor origin, in equation 5.2, and targets moving away from the cursor origin, in equation 5.3.  MT =  1 ln K  D + VK V W 2 −K  (5.2)  MT =  1 ln K  D − VK W V 2 −K  (5.3)  In order to test Jagacinski’s model further, Tresilian et al.[44] devised a physical system in which a bat was used to intercept a moving target given constraints of temporal precision and movement amplitude. The bat was fixed along an axis perpendicular to the trajectory of the target, varying only in its initial distance from the target. Results showed that subjects moved faster when temporal constraints were more demanding. The data could not be validated with Jagacinski’s original equation[31] and was fitted to equation 2.8 from Chapter 2, repeated here.  MT = α + β D + γ f V(W ) which was applied to the task of target interception in constrained 2D space. With respect to the question of direction, Tresilian[44] theorized that target movement toward or away from the origin must reflect different control processes. This is supported by the fit of Hoffman’s model to Jagacinski’s original data and indicates that in Euclidean space, there may be separate models for target movement toward 76  and away from a cursor’s point of origin.  5.3  Modeling the Data  In order to develop a model from our own empirical results, we decided to apply the Shannon formulation of the original Fitts’ law to our one dimensional trials involving Click-to-Pause (as the target was static) and apply Jagacinski’s formula, signed by direction, to our one dimensional trials involving Chase-and-Click (as the target was in 1D motion). Using the collective data from all 1D Pause trials, equation (1) was solved to reveal a = 403.875 and b = 119.554 for an r2 value of 0.504. These values were later improved by removing all non-deterministic movement trials and normalizing the data to produce a = 380.243 and b = 141.385 for an r2 value of 0.746. Similarly, we adopted Jagacinski’s equation for the movement time in the selection of moving targets and used the collective data from all 1D Chase trials, normalized and filtered to remove all cases involving non-deterministic target movement. Equation (2) was solved to reveal c = 528.1, d = 106.48 and e = 0.487 for an r2 value of only 0.093. We anticipated that errors may have altered the direction of the target movement in mid-pursuit and removed all trials in which an error count > 0 was observed. Despite our efforts, we were unable to improve upon the correlation of this model with our data, although we suspect that some cases may have been affected by the target rebounding off the edge of the screen, altering the nature of the task. For modeling our data for the Click-to-Pause technique in 2D space, the probabilistic approach seemed reasonable initially, given that the spread of hits could be applied to arbitrary points in 2D space. Furthermore , the two strongest advantages of the probabilistic approach to modeling 2D pointing tasks are the introduction of the angle of approach as well as the ability to apply the model to targets of virtually any shape. In order to progress to 2D bivariate pointing with the Pause technique, we adopted Grossman and Balakrishnan’s[19] probabilistic index of dif-  77  ficulty, IDProb . By applying the equation of a circle:  x 2 + y2 = r 2 We solve for y and substitute W/2 for radius to reveal our region of integration about the y axis.  W 2 − x2 2  y=±  To calculate the index of difficulty for capturing moving targets in 2D space, we revisited the theory of Hoffman’s[25] mathematical model and introduced a steady state position error of V, for a displacement of V x t. V, the target speed, as broken into x and y components,  ∆x 2 + t  V=  Vx =  ∆x t  Vy =  ∆y t  2  ∆y t  and as long as the target position is known at some time, t, this displacement of the target origin can be incorporated into a new IDProb , 2  IDx =  x − 2 2c(A− VKx ) 1 √ e Vx c(A− K ) 2π  −  IDy =  1 d A−  Vy L  √ e 2π 2  ( W − Vx ) IDPr = − 2W −KVx (2 K)  ( W2 ) −  −y2 − 2  ( W2 ) 78  y2 Vy 2d A− L  2  Vy L  −y2 −  Vy L  IDx IDy dydx  MTChase2D = a + bIDPr However, we were unable to solve for the constants in this equation given our data set for the Chase-and-Click technique in 2D space, after normalization and filtering of non-deterministic cases and errors. We were unable to identify any simplification to the process, given the time-dependent displacement defined by V. We anticipate that, although we were able to detect and filter errors, our data may have been affected by targets rebounding off four possible screen boundaries in two dimensions. We postulated that we could not reliably validate this model given the addition of a steady state error term, as this makes assumptions about human motor response that violate the Gaussian probability distribution. As a result, the boundaries of the hit spread defined by (-W/2, W/2) are no longer centered at the same location and applying a steady-state position error correction to the boundaries of an object’s dimensions cannot be validated with the model in its existing form. The resultant dispersion in the spread of hits as modeled by a normal distribution led us to further investigate the model in Euclidean space with an emphasis on the direction of the moving target as a factor. Given that Tresilian et al.[44] were unable to predict their empirical results from Jagacinski’s equation as well, we decided to revisit first principles in kinematics and analyze our data for one dimension involving the Chase-and-Click technique, resulting in the pursuit or interception of deterministically moving targets.  5.4  Directional Movement in 1D Space  Hoffman[25] addressed the need for a different model formulation for each direction of travel for a given target in 1D space, which was confirmed by Tresilian et al.[44] in their observation of disparate control processes in human movement by direction. Hoffman acknowledged that previous attempts to linearly model the human transient response were futile, due to the variations in the natural frequency and damping parameters between individuals. In response, a discrete movement 79  model was applied as a means of simulating human impulse response for a given set of kinematic equations to describe target behavior. In the context of our virtual pointing task, we classify the human response to movement of a target towards the cursor as an “anticipatory movement” and the response to a movement of the target away from the cursor as a “pursuit movement”[44]. Anticipatory behavior is exhibited by users in the event that they must intercept an oncoming target that is travelling in a direction that gradually brings it closer to the user’s point of origin. Alternatively, pursuit behavior is defined by target motion away from an origin for the user’s pointing device. This form of behavior can be observed when a target is travelling in a direction that is gradually moving it further from the user’s point of origin. In this case, the only means of intercepting the target is to pursue it, matching position and speed, or attempt to overtake it. When a target, t, is travelling in the direction of the user’s point of origin, Pu , we define a kinematic system as illustrated in Figure 5.1, in which the user, u, is at a distance D from a target that is initially moving toward the cursor at a speed, vt , of So . As indicated in the figure, the user is initially stationary, capable of a maximum acceleration (determined empirically) and at an initial position of 0 at the origin. The target moves at a constant speed of So , neither accelerating nor decelerating, from a starting position, pt , of D. The user’s objective is find an optimal position, d, at which the cursor meets the target in the shortest time possible, Tcoll .  d = Sot pt (t) = D − Sot pu (t) = 12 aut 2 pt (t) = pu (t) D − So t = 12 aut 2 t = − Saou ± 80  So2 a2u  + 2D au  Figure 5.1: Kinematic diagram for target interception  pt (t) = DE f f = D − So −  WEff =  So ± au  W So  So2 2D + a2u au  (5.4)  (5.5)  We derive the effective distance, De f f , and the effective width, We f f , for optimal target interception from the kinematic system for anticipatory behavior in the event that the target is moving towards the cursor. The effective movement distance, equation 5.4, and effective target width, equation 5.5, can be applied to a standard Fitts’law equation in either the Shannon formulation, involving a ratio of De f f /We f f , or the Welford formulation, involving a subtraction of logarithmic terms. The only parameter we are missing from our experimental data is the user acceleration, which we anticipate will be an empirically determined value that is left for future research. 81  Recent research by Lank et al.[34] has demonstrated that unconstrained ballistic motion follows the minimum jerk law, a mathematical model for voluntary movement by Hogan[27]. The law is based on the premise that the rate of change of acceleration, the derivative with respect to time being jerk, should be at a minimum to ensure a smooth transition in velocity over a directed movement. The minimum point of jerk is at the base of a parabola, or second order equation, which implies that the third derivative of jerk, pop, must be 0. The equations for the third order acceleration, fourth order velocity and fifth order position area plotted in Figure 5.2 for a unit distance and time of 1.  Figure 5.2: Position, velocity and acceleration for unconstrained motion For the system illustrated in figure 5.2, we assume an initial velocity, acceleration 82  and position of 0, as the cursor is stationary at the point of origin prior to any human limb response. These initial conditions provide us with a form of Hogan’s model of unconstrained motion for the pursuit of targets that are moving away from the cursor’s point of origin. Using the minimum jerk law for an initial target distance of D, we can calculate the effective distance that the user must travel in order to match the position and velocity of the target. The equations to describe the position and velocity using Hogan’s model are:  x(t) = D 10  t T  3  − 15  t T  4  +6  t T  5  (5.6)  x(t) = 10t 3 − 15t 4 + 6t 5  (5.7)  v(t) = x (t) = 30t 2 (t − 1)2  (5.8)  a(t) = v (t) = x (t) = 60t(2t − 1)(t − 1)  (5.9)  Equation 5.6 illustrates the more general form for Hogan’s model when attempting to cover a distance of D over a time base T. Substituting D = 1 and T = 1, equations 5.7 to 5.9 are unit normalized to generate the graph depicted in figure 5.2. This model defines the limitations of human movement under the minimum jerk law and provides a formulaic basis for empirical determination of human motor acceleration in future research. The acceleration factor, specific to individual subjects or interface devices, can then be applied to the effective distance in equation 5.4.  83  5.5  The Impact of Size, Speed and Direction on Movement Time  We have established the groundwork for future modeling of Fitts’ law for moving targets in one dimension by providing a kinematic model for describing human motor space and predictive target behavior; however, we have not yet established a distinction between modeling targets that are moving toward a cursor’s point of origin and those that are moving away from that point. The distinction in human motor performance is delineated by either an anticipatory behavior or a pursuit behavior for toward and away, respectively. If the factor of initial target distance is maintained as a constant, the remaining contributors to difficulty and movement time are the target size, target speed and direction of travel. In order to analyze the impact of target size and speed on the different control processes dictated by the alternate directions of the target in one dimension, we gathered all subject data for the 1D Chase-and-Click condition from our first experiment in Chapter 3. The data was first filtered for non-deterministic cases and then normalized to remove the intra-subject differences in speed. We divided the data into toward and away movements by comparing the initial and final cursor and target positions, then split it by size and speed. Our hypothesis is that speed was assistive for targets moving towards the cursor up to a threshold beyond which the direction didn’t matter and speed defined difficulty. The resulting graph of size vs. speed vs. direction is shown in Figure 5.3. From the plots in Figure 5.3, we observe that target size and target speed each contribute individually to the overall movement time, as we see an increasing trend in values from right to left (slow to fast) as well as from bottom to top (large to small). However, an interesting trend can be observed in the distinction between targets moving toward the cursor and targets moving away from the cursor. This trend can be viewed as a dispersion in movement times by direction at low to medium speeds and a convergence as the speeds of the targets increase to the left of the graph. While comparing the toward and away plots for large objects, we observe a steady  84  Figure 5.3: 1D chase analysis by size and speed and minimal difference in movement time with increased speed, indicating that the effects of direction and speed counteract one another and result in a similar index of difficulty for increased target widths. As we examine the plots for medium sized objects, we observe that the graphs disperse for slow and moderate speeds, indicating that speed is assistive in target acquisition due to the lower movement times observed for the toward plot. As the speed increases, the toward and away plots for medium sized objects converge and no advantage is realized from either direction. In the event of such convergence, we conclude that speed dominates direction as a contributing factor to the movement time, given that there is no visible effect of direction at that point. Upon inspection of the small target graphs, we observe a slight advantage of direction for targets moving away from the cursor at moderate speeds; however, as the speed increases, the plots converge and speed remains the dominant factor over direction in contributing to a higher movement time. From 85  these observations we infer that there is a critical speed at some point that is a function of the target width and establishes a threshold for the contribution of speed to the index of difficulty through the effective distance and effective width that it defines.  Vcrit = f (W )  DE f f =  WE f f =  D −Votacq if Vo < Vcrit if Vo > Vcrit  D  W  if Vo < Vcrit  W VVcrito  if Vo > Vcrit  (5.10)  (5.11)  (5.12)  In the event that the target speed is below the critical speed threshold, the effective target distance is reduced by distance covered in the time for the cursor to acquire the target, while the effective target width remains unaffected. In the event that target speed is above the critical speed threshold, the effective distance remains unaffected while the effective width is multiplied by a ratio of the critical speed to the actual target speed. When the target speed is below the critical threshold, direction dominates and only the effective distance is scaled to reflect either the distance travelled toward the cursor or the distance travelled away from it. This implies that the ballistic phase of movement, covering the target distance, is primarily affected by direction at low speeds. On the other hand, when the speed is above the critical threshold, speed dominates and the effective width, or effective window of opportunity for acquisition[45], is reduced by the ratio of the threshold to the higher speed. The dominance of speed implies that the precision phase of movement, homing in on the target, is primarily affected by speeds above the threshold. This scenario is illustrated in figure 5.4. 86  Figure 5.4: Speed-direction relation for critical speed threshold In summary, speed assists in reducing the movement time of target acquisition tasks for cases in which the target is moving towards the cursor below a certain speed threshold. Above this speed threshold, it makes no difference which direction the target is travelling in, the speed defines the difficulty. In addition, this effect is observed more frequently for smaller targets below a certain size threshold.  5.6  Extension of Direction to Two Dimensions  As an inspiration for future research efforts, we have devised a methodology for extending our direction-based Fitts’ model into two dimensions for bivariate pointing tasks involving moving targets. Traditionally, the index of difficulty for acquiring targets in 2D Euclidean space is defined by the distance from the cursor origin to the geometric center of the target and the width and height of that target. No research to date has addressed the lack of a model for moving targets in 2D space; however, the relative velocity of the target would most likely be described by a sum 87  of movement vectors. In order to apply the same modeling principles from our one dimensional analysis of direction, we must establish a directional factor in 2D space that is consistent in its effect on movement time and its interaction with target size and speed. The direction that the target is travelling in should have a direct and proportionate effect on the distance from the cursor’s point of origin. In 2D space, motion is defined by a vector at an arbitrary point (x, y), a distance D from the cursor origin with a magnitude of V. To preserve the analogy of movement directed toward or away from the cursor origin, we propose the development of a model in polar coordinates that defines target movement by a radial speed toward or away from the cursor origin and an angular speed that defines its rate of rotation. Supporting research for establishing a model that is based on movement angle can be found in Hancock and Booth[23], as validated by Grossman and Balakrishnan[19]. The impact that movement angle has on movement time for a pointing task is explained by the alternate muscle groups that are used to control movement in different directions as well as the variance in target width from non-uniform shapes[19]. We illustrate a sample calculation for converting a moving target in 2D Euclidean space to 2D polar space, with an angular speed consisting of the rate of changing movement angles and a radial speed consisting of the rate of changing distances from the origin to the target. We attempted to replicate our analysis from the previous section by converting the δ x and δ y values per frame for each target into a δ r radial speed using the method illustrated in Figure 5.5. By comparing the mean movement times between sizes and speeds for targets that were split by radial direction, we had hoped to discover a similar trend for critical speed threshold in 2D space. Unfortunately, we had not originally anticipated that direction would be such a significant factor and movement patterns in 2D space were determined by a randomized position on the screen (280 pixels from the origin) and the initial direction of target movement was determined by a balanced selection of random angles from each geometric quadrant. As a result of random positioning and movement angles, we had an extremely limited selection of radial 88  Figure 5.5: 2D derivation of radial and angular speed speeds to sort, normalize and compare. We were unable to identify the same trend in 2D space that was observed for moving targets in 1D space.  5.7  Conclusion  In this chapter, we presented a survey of modeling techniques for extending Fitts’ law into two dimensional space, most notably of deterministic and stochastic systems, in addition to techniques for refining Fitts’ law for the selection of moving targets. In our efforts to develop a model for moving targets in two dimensional space, we have identified the approach angle and target direction as contributing factors to the overall movement time, while noting that the existing linear and discrete models for human motor response were inadequate for our application as they do not reflect the level of realism necessary for the natural frequency and damping. We developed a kinematic model that described the effective target distance in terms of human motor acceleration and the effective target width in terms of a constant velocity. The minimum jerk law, defining the lowest rate of change in acceleration, was suggested as a means of deriving human motor acceleration empirically, due to its accuracy in predicting endpoints in human motor function. By  89  analyzing the data from our first experiment for the 1D chase condition, we have identified a trend that speed assists in the selection of targets moving towards the cursor below a critical speed for smaller objects, above which speed dominates difficulty. We extend our 1D directional model into 2D polar space by defining the radial and angular components of moving object trajectories in 2D Euclidean space. In the final chapter we present our conclusions from the two phases of the first experiment, the informal user study, the validation of Fitts’ law models and the trend identified through analysis of our experimental data. In addition, we outline our contributions and present future research recommendations for extending the Fitts’ law model for moving targets in two dimensions.  90  Chapter 6  Conclusions and Future Work In this chapter, we present a discussion of the results of our experiment to evaluate a novel interaction method for moving targets, our informal user study to implement our technique in an interactive video browser and the analysis of our data in an effort to develop a predictive model for future research. We also discuss the significance and contributions of the thesis, as well as a proposal for future research.  6.1  Discussion  In this thesis, we presented the results of our experiment to evaluate the Clickto-Pause technique for the selection of moving targets. A comparison was made between Click-to-Pause and the Chase-and-Click technique for both 1D and 2D targets that varied in size, speed and determinism of movement. In both the 1D and 2D trials, the Click-to-Pause technique demonstrated lower target acquisition times for targets that were small in size (20px), fast moving (250px/s) or a combination of both. A threshold region was identified for target size and speed that resulted in lower acquisition times for Click-to-Pause due to factors of speed and size dominating the movement time over the additional pause operation. Overall, Click-to-Pause resulted in substantially fewer errors than the Chase-and-Click technique due to conflicting goals in the speed-accuracy tradeoff. No significant effect was observed for the determinism of movement on target acquisition time.  91  The Hybrid Chase-or-Pause technique allowed subjects to arbitrarily pause moving targets, indicating the potential for movement optimization. The Hybrid technique resulted in mean acquisition times similar to the Click-to-Pause method, except for small-fast targets in which the Click-to-Pause method resulted in even lower mean times. Subjects were observed applying a homing strategy in 16% of all successful target acquisitions, occurring more frequently than the Chase-and-Click approach for small targets. The experiment provided us with a balanced data set that included acquisition times for targets that were of varying sizes, speeds and levels of determinism in movement to be applied to the extension models for Fitts’ law. For our second experiment, we developed an interactive video browser for multiview video navigation to evaluate the usage of our Hybrid Chase-or-Pause technique in a realistic application. In addition, we compared two display environments, scaled panel and picture-in-picture, to determine the effects of video scaling and occlusion on task performance within the video frame. Our results indicated no significant effect of display mode on task completion time nor any interaction effects between display mode and task type. We analyzed a log of all mouse events and target selections to make observations of which interaction technique subjects chose most frequently in each display mode. Our results demonstrated that subjects selected player targets while paused more frequently than they clicked directly on a player in mid-motion. Subjects selected player targets while paused 84.43% of the time in the scaled panel mode and 79.74% of the time while in the picture-in-picture mode. User feedback in a post-questionnaire indicated a clear preference for the picture-in-picture display mode over the scaled panel approach. In our analysis of Fitts’ law models, we presented a survey of modeling techniques for extending Fitts’ law into two dimensional space, most notably of deterministic and stochastic systems, in addition to techniques for refining Fitts’ law for the selection of moving targets. In our efforts to develop a model for moving targets in two dimensional space, we have identified the approach angle and target direction 92  as contributing factors to the overall movement time, while noting that the existing linear and discrete models for human motor response were inadequate for our application as they do not reflect the level of realism necessary for the natural frequency and damping. We developed a kinematic model that described the effective target distance in terms of human motor acceleration and the effective target width in terms of a constant velocity. The minimum jerk law, defining the lowest rate of change in acceleration, was suggested as a means of deriving human motor acceleration empirically, due to its accuracy in predicting endpoints in human motor function. By analyzing the data from our first experiment for the 1D chase condition, we have identified a trend that speed assists in the selection of targets moving towards the cursor below a critical speed for smaller objects, above which speed dominates difficulty. We extend our 1D directional model into 2D polar space by defining the radial and angular components of moving object trajectories in 2D Euclidean space.  6.2  Contributions  We developed a novel technique for the selection of moving targets in 2D that we call Click-to-Pause. This technique allows users to temporarily pause the target before performing the selection, eliminating the need to pursue the target while in motion, using the Chase-and-Click technique. In addition, we have developed a Hybrid Pause-or-Chase technique that enables users to adopt an optimization strategy based on the target speed, direction and determinism of movement. We conducted a user study designed to compare these three techniques, resulting in a large empirical data set that balanced factors such as dimension, target size, target speed and the determinism of the target’s motion. This data set was used to evaluate the impact of each technique on movement time in selection tasks and to develop a predictive model in 1D space with a plan for extension to 2D. The results of our analysis indicated that:  93  • For 1D target selection, significant interaction effects were observed for technique by size, technique by speed and technique by size by speed on movement time. Chase-and-Click proved to be faster for medium-sized, largesized, slow-moving and moderately-moving targets and all combinations thereof. Click-to-Pause proved to be faster for small-sized and fast-moving targets and the combination of both. The Chase-and-Click technique resulted in substantially more errors (25.17%) than Click-to-Pause (1.34%). No significant main or interaction effects were observed for the target determinism of movement. • For 2D target selection, significant interaction effects were observed for technique by size, technique by speed and technique by size by speed on movement time. Chase-and-Click proved to be faster for medium-sized, largesized, slow-moving and moderately-moving targets and all combinations thereof. Click-to-Pause proved to be faster for small-sized and fast-moving targets and the combination of both. The Chase-and-Click technique resulted in substantially more errors (31.2%) than Click-to-Pause (1.58%). No significant main or interaction effects were observed for the target determinism of movement. • The graphs of technique versus size in 1D and 2D depict a crossover of Chase-and-Click and Click-to-Pause between small and medium, indicating a threshold below which Click-to-Pause outperforms Chase-and-Click. Likewise, the graphs of technique versus speed in 1D and 2D depict a crossover of Chase-and-Click and Click-to-Pause between moderately-fast and fast, indicating a threshold above which Click-to-Pause outperforms Chase-andClick. Speed has relatively no effect on movement time for Click-to-Pause trials, but makes Chase-and-Click tasks extremely difficult, especially when in combination with small targets. • In the second phase, significant interaction effects were observed for technique by size, technique by speed and technique by size by speed on movement time. Chase-and-Click proved to be faster for large-sized, slow-moving 94  and moderately-moving targets and all combinations thereof. Click-to-Pause and Hybrid proved to be faster for small-sized and fast-moving targets, while Click-to-Pause outperformed Hybrid for the combination of the two. The Chase-and-Click technique resulted in substantially more errors (29.17%) than Hybrid (6.91%) or Click-to-Pause (1.17%). No significant main or interaction effects were observed for the target determinism of movement. We discovered that target speed has relatively no impact on the Click-to-Pause technique, while target size follows a traditional Fitts’ trend for target width. In addition, small and fast targets made the Chase-and-Click technique extremely difficult, resulting in long movement times. The Hybrid Chase-or-Pause technique performed on-par with the Click-to-Pause technique for smaller or faster targets, but resulted in higher acquisition times for small-fast targets and a slightly higher error rate for all significant interactions. Optimization of pursuit movement and target pause were observed and occured more frequently with small and fast targets. These contributions are developed in Chapter 3. Once the Hybrid Chase-or-Pause technique was implemented in an interactive video browser, we observed that the vast majority of correct target selections were performed while the video was paused and that subjects were using an optimization strategy. The type of display mode in the video browser, scaled panel or picturein-picture, had no effect on task completion time. Subjects expressed a clear preference for a picture-in-picture mode of display over scaled panels for extraneous information and menus. These contributions are developed in Chapter 4. From our results, we identified the angle of approach and the direction of travel as key contributors to the index of difficulty and overall movement time in selection tasks. We indentified a kinematic model that accounts for human motor acceleration and the minimum jerk law is suggested as a means of empirically determining a formula or constant for acceleration. Analysis of 1D chase data revealed that target speed assists in selecting approaching targets, but dominates over direction beyond a critical speed for smaller targets. We provide a framework for extending the directional model into 2D polar space using radial speed and angular speed that 95  are calculated from Euclidean coordinates and speed vectors. These contributions are developed in Chapter 5.  6.3  Future Work  As future work, we propose the design of an experiment to re-evaluate our three interaction techniques for the selection of moving targets in 2D space, using different factors that we hypothesize will contribute to the validation of a predictive model. In addition, we propose the design of a subsequent experiment to extend our 1D directional model into 2D polar space using angular and radial vectors to describe the motion of targets along a radial axis toward or away from a pointer positioned at the origin.  6.3.1  Refining Our Interactive Selection Methods Experiment  In this thesis, we developed an experiment to evaluate three interaction techniques for moving target selection in 1D and 2D that were representative of both stationary and moving target acquisition. We designed a set of trials to account for pre-selected factors including target size, speed and the determinism of movement. Over the course of our analysis, we discovered that there was no significant effect of the determinism of movement on the target acquisition time. Our efforts to model the movement time of a moving target selection task in 1D revealed a relationship between target size, speed and direction. Our data lacked a balanced distribution of targets moving toward the cursor and targets moving away from the cursor in 1D, due to non-deterministic targets unpredictably changing direction and deterministic targets rebounding off screen boundaries. As a result, we were unable to develop a model beyond the trend of speed assisting in target acquisition below a critical threshold for approaching targets. We recommend that the original experiment be redesigned to focus on specific research goals, including the validation of a 1D deterministic model involving factors of target size, speed and direction. Target movement would be initially restricted 96  to movement at angles that are increments of 45◦ to limit the motion of the target in 2D space and ensure an equal distribution of directions in all four euclidean quadrants relative to the pointer. The experiment would be administered with an exclusive goal of speed over accuracy by allowing targets to travel off screen, ending the trial if they are not acquired in time. This would allow us to ensure a consistent target trajectory for each trial, filtering failed trials and classifying those that remained. While this may reduce statistical power for analysis, the data for targets that were acquired onscreen would be more consistent. Finally, we would apply the minimum jerk law as our human motor response model in order to provide greater realism over linear and discrete movement models. With an empirically determined value for maximum human motor acceleration, a data set that balances target size, speed and direction as well as a kinematic formula that accounts for the speed-direction relationship, a suitable model could be tested for its predictive capabilities.  6.3.2  Extending Our Work to 2D Polar Space  In addition to the redesign of our experiment from Chapter 3 to account for target direction over movement determinism, we propse the design of an experiment to validate the extension our 1D directional model in 2D polar space. In our 1D scenario, we determined from our empirical results that direction defined an anticipatory movement for targets moving toward the pointer and an unconstrained pursuit movement for targets moving away. This was valid for target speed below a critical threshold, beyond which the direction had no significant effect on movement time, resulting in a speed dominance. We hypothesized that this relation could be demonstrated in 2D by describing target motion using polar vectors with respect to a pointer located at the polar origin. The radial velocity is representative of the directional component from 1D space, as it defines the speed toward or away from the polar origin of the pointer. Similarly, the angular velocity would be representative of the change in approach angle about a circle defined by the radial coordinate from the polar origin.  97  Our experimental design would isolate the factors of radial velocty and angular velocity by maintaining one as a constant while varying the other during a moving target selection task in two dimensions. One task would involve the acquisition of targets moving toward or away from the polar origin along radial trajectories at a constant angle. This would enable us to isolate the factors of speed and direction in 2D space, to determine whether our critical speed relationship for direction can be extended into 2D space. Another task would involve the acquisition of targets moving around the circumference of a circle with a constant radius projected from the polar origin of the pointer. This would enable us to isolate the factor of approach angle and determine how angular speed affects the movement time independent of the radial speed and direction. These factors could be combined to provide a complete model for the movement time to acquire a moving target in 2D space. A conceptual diagram of the virtual pointing tasks for each of these scenarios is illustrated in figure 6.1.  Figure 6.1: Polar target specification, (a) radial velocity (b) angular velocity  6.4  Summary  In this thesis we developed Click-to-Pause, a novel interaction technique for the selection of moving targets in 1D and 2D space. We presented the results of an 98  experiment designed to evaluate this technique against a direct Chase-and-Click approach as well as a hybrid of the two. During our analysis, we determined that Click-to-Pause resulted in lower acquisition times than Chase-and-Click for small sized and fast moving targets. Empirical graphs suggested that selection tasks using Click-to-Pause followed a standard Fitts’ model for target size, but were generally unaffected by target speed. The Click-to-Pause technique also resulted in substantially fewer errors than the Chase-and-Click technique. The Hybrid technique resulted in mean acquisition times similar to the Click-to-Pause method, except for small-fast targets in which the Click-to-Pause method resulted in even lower mean times. We demonstrated that subjects were adopting an optimization strategy of anticipatory homing movement for small and fast targets. We developed a multi-view video navigation tool with two different display modes that limited interaction with video content by scaling the size of the video image or occluding it with floating windows. Our empirical results revealed that the display mode had no effect on task completion time, but subjects expressed a clear preference for picture-in-picture mode. The results from our first experiment contributed to the validation of the original Fitts’ model for stationary target selection in 1D space and the identification of direction and approach angle as key contributors to the index of difficulty and movement time in moving target selection. We identified a relationship between speed and direction, such that speed assists in target acquisition for approaching targets up to a critical velocity, beyond which speed dominates difficulty. We devised an extension to this 1D directional model into 2D space using polar coordinates. Our research will provide the basis for future experiments that attempt to model moving target selection in 2D, using the predictive relation of speed and direction. A predictive model will enable the evaluation of alternative techniques for the selection of moving targets in interactive video. These empirical results will provide validation for a set of design heuristics for interactive video interfaces that incorporate dynamic content as selection criteria.  99  Bibliography [1] Accot, J. and Zhai, S. 2003. Refining Fitts’ law models for bivariate pointing. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Ft. Lauderdale, Florida, USA, April 05 - 10, 2003). CHI ’03. ACM, New York, NY, 193-200. → pages 5, 23, 74 [2] http://www.asterpix.com → pages 15 [3] Balakrishnan, R. 2004. Beating Fitts’ law: virtual enhancements for pointing facilitation. Int. J. Hum.-Comput. Stud. 61, 6 (Dec. 2004), 857-874 → pages 6, 19 [4] Baudisch, P., Cutrell, E., Robbins, D., Czerwinski, M., Tandler, P. Bederson, B., and Zierlinger, A. Drag-and-Pop and Drag-and-Pick: Techniques for Accessing Remote Screen Content on Touch- and Pen-operated Systems. In Proceedings of Interact 2003, Zurich Switzerland, August 2003, pp. 57-64. → pages 6, 57 [5] Baudisch, P., Zotov, A., Cutrell, E., and Hinckley, K. 2008. Starburst: a target expansion algorithm for non-uniform target distributions. In Proceedings of the Working Conference on Advanced Visual interfaces (Napoli, Italy, May 28 - 30, 2008). AVI ’08. ACM, New York, NY, 129-137. → pages 20, 21, 57 [6] Cesar, P., Bulterman, D. C., Jansen, J., Geerts, D., Knoche, H., and Seager, W. 2009. Fragment, tag, enrich, and send: Enhancing social sharing of video. ACM Trans. Multimedia Comput. Commun. Appl. 5, 3 (Aug. 2009), 1-27. → pages 15, 16 [7] Chapuis, O., Labrune, J., and Pietriga, E. 2009. DynaSpot: speed-dependent area cursor. In Proceedings of the 27th international Conference on Human Factors in Computing Systems (Boston, MA, USA, April 04 - 09, 2009). CHI ’09. ACM, New York, NY, 1391-1400. → pages 20, 57 100  [8] Cheng, K., Luo, S., Chen, B., and Chu, H. 2009. SmartPlayer: user-centric video fast-forwarding. In Proceedings of the 27th international Conference on Human Factors in Computing Systems (Boston, MA, USA, April 04 - 09, 2009). CHI ’09. ACM, New York, NY, 789-798. → pages 15 [9] http://www.clikthrough.com → pages 15 [10] Craik, Fergus and Tulving, E. The Oxford Handbook of Memory. Oxford University. Press, 2000. → pages [11] Crossman, E. and Goodeve, P. 1963. Feedback control of hand movement and Fitts’ Law. Quarterly Jouranl of Psychology, 1983, 35A, 251-278. → pages 53, 56 [12] Dragicevic, P., Ramos, G., Bibliowitcz, J., Nowrouzezahrai, D., Balakrishnan, R., and Singh, K. 2008. Video browsing by direct manipulation. In Proceeding of the Twenty-Sixth Annual SIGCHI Conference on Human Factors in Computing Systems (Florence, Italy, April 05 - 10, 2008). CHI ’08. ACM, New York, NY, 237-246. → pages 18, 59 [13] Faure, G., Chapuis, O., and Beaudouin-Lafon, M., Acquisition of Animated and Pop-Up Targets. In Proceedings of the 12th international Conference on Human-Computer Interaction (Uppsala, Sweden, August 24-28, 2009). INTERACT ’09 LNCS 372-385. → pages 26 [14] Fekete, J., Elmqvist, N., and Guiard, Y. 2009. Motion-pointing: target selection using elliptical motions. In Proceedings of the 27th international Conference on Human Factors in Computing Systems (Boston, MA, USA, April 04 - 09, 2009). CHI ’09. ACM, New York, NY, 289-298. → pages 20 [15] Fitts, P. M. (1954). The information capacity of the human motor system in controlling the amplitude of movement. Journal of Experimental Psychology, 47, 381-39 → pages 2, 3, 22, 56 [16] Fitts, P. M., & Peterson, J. R. (1964). Information capacity of discrete motor responses. Journal of Experimental Psychology, 67, 103-113. → pages 23 [17] Girgensohn, A., Kimber, D., Vaughan, J., Yang, T., Shipman, F., Turner, T., Rieffel, E., Wilcox, L., Chen, F., and Dunnigan, T. 2007. DOTS: support for effective video surveillance. In Proceedings of the 15th international Conference on Multimedia (Augsburg, Germany, September 25 - 29, 2007). MULTIMEDIA ’07. ACM, New York, NY, 423-432. → pages 17  101  [18] Goldman, D. B., Gonterman, C., Curless, B., Salesin, D., and Seitz, S. M. 2008. Video object annotation, navigation, and composition. In Proceedings of the 21st Annual ACM Symposium on User interface Software and Technology (Monterey, CA, USA, October 19 - 22, 2008). UIST ’08. ACM, New York, NY, 3-12. → pages 18 [19] Grossman, T. and Balakrishnan, R. 2005. A probabilistic approach to modeling two-dimensional pointing. ACM Trans. Comput.-Hum. Interact. 12, 3 (Sep. 2005), 435-459. → pages 5, 24, 75, 77, 88 [20] Grossman, T. and Balakrishnan, R. 2005. The bubble cursor: enhancing target acquisition by dynamic resizing of the cursor’s activation area. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Portland, Oregon, USA, April 02 - 07, 2005). CHI ’05. ACM, New York, NY, 281-290. → pages 19, 57 [21] Grossman, T., Kong, N., and Balakrishnan, R. 2007. Modeling pointing at targets of arbitrary shapes. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (San Jose, California, USA, April 28 - May 03, 2007). CHI ’07. ACM, New York, NY, 463-472. → pages 24, 75 [22] Gunn, T. J., Irani, P., and Anderson, J. 2009. An evaluation of techniques for selecting moving targets. In Proceedings of the 27th international Conference Extended Abstracts on Human Factors in Computing Systems (Boston, MA, USA, April 04 - 09, 2009). CHI EA ’09. ACM, New York, NY, 3329-3334 → pages 20, 22, 57 [23] Hancock, M. S. and Booth, K. S. 2004. Improving menu placement strategies for pen input. In Proceedings of Graphics interface 2004 (London, Ontario, Canada, May 17 - 19, 2004). ACM International Conference Proceeding Series, vol. 62. Canadian Human-Computer Communications Society, School of Computer Science, University of Waterloo, Waterloo, Ontario, 221-230. → pages 24, 88 [24] Haubold, A., Dutta, P., and Kender, J. R. 2008. Evaluation of video browser features and user interaction with VAST MM. In Proceeding of the 16th ACM international Conference on Multimedia (Vancouver, British Columbia, Canada, October 26 - 31, 2008). MM ’08. ACM, New York, NY, 449-458. → pages 15, 16 [25] Hoffmann, E. R., Capture of Moving Targets: a Modification of Fitts’ Law. Ergonomics, 1991, 34, 211–220. → pages 5, 25, 76, 78, 79 102  [26] Hoffmann, E. and Sheikh, I. 1994. Effect of varying target height in a Fitts movement task. Ergonomics 37, 6, 1071–1088. → pages 24 [27] N. Hogan, An organizing principle for a class of voluntary movements Journal of Neuroscience, Vol 4, pp. 2745–2754. → pages 82 [28] Hoschka, P. 2002. SMIL: an introduction. In ACM SIGGRAPH 2002 Conference Abstracts and Applications (San Antonio, Texas, July 21 - 26, 2002). SIGGRAPH ’02. ACM, New York, NY, 321-321 → pages 16 [29] Hutchins, E. L., Hollan, J. D., and Norman, D. A. 1985. Direct manipulation interfaces. Hum.-Comput. Interact. 1, 4 (Dec. 1985), 311-338. → pages 2 [30] Isokoski, P. 2006. Variability of throughput in pointing device tests: button-up or button-down?. In Proceedings of the 4th Nordic Conference on Human-Computer interaction: Changing Roles (Oslo, Norway, October 14 18, 2006). A. Mrch, K. Morgan, T. Bratteteig, G. Ghosh, and D. Svanaes, Eds. NordiCHI ’06, vol. 189. ACM, New York, NY, 68-77. → pages 9, 31, 51, 56, 67 [31] Jagacinski, R. J., Repperger, D. W., Ward, S. L., and Moran, M. S. (1980). A test of Fitts’ law with moving targets. Hum Factors, 22(2):225-233. → pages 5, 24, 25, 76 [32] Karrer, T., Weiss, M., Lee, E., and Borchers, J. 2008. DRAGON: a direct manipulation interface for frame-accurate in-scene video navigation. In Proceeding of the Twenty-Sixth Annual SIGCHI Conference on Human Factors in Computing Systems (Florence, Italy, April 05 - 10, 2008). CHI ’08. ACM, New York, NY, 247-250. → pages 17, 59 [33] D. Kimber, T. Dunnigan, A. Girgensohn, F. Shipman, T. Turner, and T. Yang. Trailblazing: Video playback control by direct object manipulation. In ICME, pages 1015–1018, 2007. → pages 17, 18 [34] Lank, E., Cheng, Y. N., and Ruiz, J. 2007. Endpoint prediction using motion kinematics. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (San Jose, California, USA, April 28 - May 03, 2007). CHI ’07. ACM, New York, NY, 637-646. → pages 82 [35] MacKenzie, I. S. and Buxton, W. 1992. Extending Fitts’ law to two-dimensional tasks. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Monterey, California, United States, May 03 - 07, 1992). P. Bauersfeld, J. Bennett, and G. Lynch, Eds. CHI ’92. ACM, New York, NY, 219-226 → pages 23, 74 103  [36] McGuffin, M. and Balakrishnan, R. 2002. Acquisition of expanding targets. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems: Changing Our World, Changing Ourselves (Minneapolis, Minnesota, USA, April 20 - 25, 2002). CHI ’02. ACM, New York, NY, 57-64. → pages 57 [37] McGuffin, M. J. and Balakrishnan, R. 2005. Fitts’ law and expanding targets: Experimental studies and designs for user interfaces. ACM Trans. Comput.-Hum. Interact. 12, 4 (Dec. 2005), 388-422. → pages 6, 20, 21 [38] McKoon, G., Ratcliff, R., & Dell, G. (1986). A critical evaluation of the semantic/episodic distinction. Journal of Experimental Psychology: Learning, Memory, and Cognition, 12, 295-306. → pages [39] Mould, D. and Gutwin, C. 2004. The effects of feedback on targeting with multiple moving targets. In Proceedings of Graphics interface 2004 (London, Ontario, Canada, May 17 - 19, 2004). ACM International Conference Proceeding Series, vol. 62. Canadian Human-Computer Communications Society, School of Computer Science, University of Waterloo, Waterloo,Ontario,25-32. → pages 5, 21 [40] Microsoft Developer Network, SetDoubleClickTime function, http://msdn.microsoft.com/enus/ → pages 67 [41] Pea, R., Mills, M., Rosen, J., Dauber, K., Effelsberg, W., and Hoffert, E. (2004). The DIVER project: Interactive digital video repurposing. IEEE Multimedia, 11(1), 54–61. → pages 15, 16 [42] Claude E. Shannon and Warren Weaver: The Mathematical Theory of Communication. The University of Illinois Press, Urbana, Illinois, 1949. ISBN 0-252-72548-4 → pages 74 [43] Shneiderman, B. 1984. The future of interactive systems and the emergence of direct manipulation. In Proc. of the NYU Symposium on User interfaces on Human Factors and interactive Computer Systems (New York, United States). Y. Vassiliou, Ed. Ablex Publishing Corp., Norwood, NJ, 1-28. → pages 2, 17 [44] Tresilian, J. R. & Lonergan, A. (2002) Intercepting moving objects: effect of temporal precision and movement amplitude. Experimental Brain Research, 142, 193-207 → pages 8, 25, 56, 76, 79, 80  104  [45] Tresilian, J. R., Plooy, A. & Marinovic, W. (2009) Manual interception in two dimensions: performance and space-time accuracy. Brain Research, 1250, 202-217. → pages 86 [46] Welford, A. 1968. The Fundamentals of Skill. Methuen, London, UK. → pages 23, 75 [47] www.youtube.com → pages 15  105  Appendix A  Trial Design for Chapter 3 Adobe Flash CS3 professional was used to develop the “Capture the Wisp” game in the Adobe Air 1.0 environment, which would allow local file access to a new XML file for each trial in the experiment. Each XML file contained parameters for the Wisp including the number of dimensions of movement, the wisp’s diameter, the wisp’s speed, the type of potion that would be available and whether the wisp’s movement pattern would be deterministic or not that trial. In addition, each XML file contained a placement coordinate for the potion as well as a starting coordinate for the wisp, balanced by quadrant from the potion origin. In the event that the movement pattern is deterministic, the wisp’s intial direction, defined by a randomized ∆x and ∆y combination and counterbalanced by direction along the x and y axes, is provided to establish an initial trajectory that is affected only by collisions with the screen boundaries. The x and y movement vectors, defined by ∆x and ∆y, must always sum to 1, which is the pixel distance travelled each frame, while the speed is controlled by the frame rate. A sample of the XML file for a trial involving a mid-sized target moving at fast speed in a deterministic movement pattern in 2D is illustrated in figure A.1. In the event that the movement pattern is non-deterministic, a set of 150 waypoints are included that were calculated using the random math function in an excel spreadsheet, within the limits of the screen boundaries. The initial object size, ob106  Figure A.1: Sample XML file as input for a deterministic trial ject speed and the enabled selection technique are set in the first Trial tag structure, while subsequent structures consist of additional waypoint coordinates that the object travels between. The flash program calculates the distance between waypoints and creates an appropriate step size each frame based on a vector length of 1, with speed controlled by the frame rate. Each time a wisp object reaches its destination, the flash code retrieves a new coordinate and calculates a new trajectory. A sample of the XML file for a trial involving a mid-sized target moving at fast speed in a non-deterministic movement pattern in 2D is illustrated in figure A.2. For 1D trials, the placement of the wisp is balanced between the two directions from the potion origin at the preset distance of 240 pixels; however, the 2D aspect added the complication of placing the wisp anywhere along a circle of radius 240 pixels from the potion origin to maintain the equivant distance or movement amplitude between trials. Using an initial random seed angle, a reflection about the X and Y axes and the average angle calculation of Angle = atan2 (∑ni=0 CosΘi , ∑ni=0 SinΘi ), we developed a running average of existing wisp angles to maintain the reflection of new angles across the origin. We illustrate the counterbalancing of angles by their sines and cosines in the circle depicted in figure A.3.  107  Figure A.2: Sample XML file as input for a non-deterministic trial The algorithm for determining the initial trajectory of each target started in quadrant 1 (Q1) and consisted of calculating a random number between -1 and 1 to describe the initial step along the x-axis, dx, followed by a calculation of the intial √ y step, dy = (1or − 1) ∗ 1 − dx2 . This was repeated for quadrant 2. Quadrant 3 consisted of multiplying the dx from quadrant 1 by -1 and the dy from quadrant 1 by -1. Quadrant 4 consisted of multiplying the dx from quadrant 2 by -1 and the dy from quadrant 1 by -1. This procedure balanced the relative trajectories between opposing quadrants centered around the potion origin. We have developed a method of counterbalancing factors such as technique (potion type), target size and target speed among the 288 experimental trials of the first phase of Chapter 3 and the 54 trials of the second phase. Given a random position for the potion, balanced by screen quadrant, we average angles over the placement of the wisp target around the origin of the potion coordinates. For deterministic trials, we balance the initial trajectory of the wisp in the opposite quadrant and allow the wisp to rebound of screen boundaries in perpetual motion at the initial speed. 108  Figure A.3: Sample angle diagram for average angle calculation from potion origin For non-deterministic trials, we define a series of waypoints that are followed in order for the duration of the trial. In doing so, we have created a balanced set of input trials for our 1D/2D Fitts’ law selection task abstracted as a game.  109  Appendix B  Ethics Approval  110  

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
http://iiif.library.ubc.ca/presentation/dsp.24.1-0064925/manifest

Comment

Related Items