UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Dual-task performance of visually guided action and perception Liu, Geniva 2005

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata

Download

Media
831-ubc_2005-105305.pdf [ 11.86MB ]
Metadata
JSON: 831-1.0099841.json
JSON-LD: 831-1.0099841-ld.json
RDF/XML (Pretty): 831-1.0099841-rdf.xml
RDF/JSON: 831-1.0099841-rdf.json
Turtle: 831-1.0099841-turtle.txt
N-Triples: 831-1.0099841-rdf-ntriples.txt
Original Record: 831-1.0099841-source.json
Full Text
831-1.0099841-fulltext.txt
Citation
831-1.0099841.ris

Full Text

DUAL-TASK PERFORMANCE OF VISUALLY GUIDED ACTION A N D PERCEPTION by GENIVA LIU B.A. (Honours), University of Alberta, 1997 M.A., University of British Columbia, 2001 A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY In THE FACULTY OF GRADUATE STUDIES (Psychology) THE UNIVERSITY OF BRITISH COLUMBIA October 2005 © Geniva Liu, 2005 ABSTRACT A currently influential theory of human vision (Milner & Goodale, 1995) posits the existence of two independent streams of visual processing: a ventral system for the conscious perception of objects and a dorsal system for the dynamic guidance of actions toward objects. This theory also proposes that action planning is controlled by the ventral stream, whereas action execution is controlled by the dorsal stream. What is left unspecified by this theory, however, is whether this functional independence also applies to the attentional mechanisms available to each system. To examine this question, a series of experiments was conducted to measure dual-task interference between tasks involving perception and action. One task required the visual identification of one object at the center of gaze, while the other task required concurrently pointing to a second object that appeared in the visual periphery. The online control functions of the dorsal stream were indexed by displacing the pointing target following movement initiation. Results indicated that successful performance of visual identification lead to interference in the planning, but not in the execution of action (Experiments 1, 2, 5, & 6), and this held for movements that were initiated before, during, or after presentation of the identification target (Experiment 3). When a speed stress was placed on pointing, dual-task costs were also observed in the execution and online control of action (Experiment 4). Furthermore, when action execution was forced to rely on ventral stream processes, either by spatially displacing the action from the target (Experiment 7) or by making the action based upon memory (Experiment 8), then visual identification interfered with both the planning and execution of action. These results support some independence in the attentional mechanisms used for ventral and dorsal tasks. Specifically, action planning shares attentional mechanisms with visual identification, but action execution as performed by the dorsal stream does not rely on these mechanisms. i i i T A B L E OF C O N T E N T S Abstract ii Table of Contents iii List of Figures v Acknowledgements vi Chapter 1: Introduction 1 Purpose of Thesis 1 The Two Purposes of Vision: Perception and Action 3 Stream Characteristics 8 Stream Interactions 10 Planning and execution of action 12 Attention and visually guided action tasks 15 Selective attention 16 Divided attention 19 Divided attention and action 21 Theories and Evidence Relevant to Dual-Task Perception and Action 24 Premotor Theory of Attention 25 Integrated Competition Hypothesis 29 Visual Attention Model 31 Present Questions 36 Experimental Overview 39 Chapter 2: Experiments 1 and 2 45 Experiment 1: Letter identification and pointing to a letter target 45 Methods 48 Results 54 Discussion 57 Experiment 2: Letter identification and pointing to a disc 61 Methods 61 Results 63 Discussion 66 Chapter 3: Experiments 3 and 4 67 Experiment 3: Negative and positive temporal lags 67 Methods 68 Results 70 Discussion 74 Experiment 4: Negative and positive temporal lags under deadline pressure 78 Methods 79 Results 80 Discussion 85 Interim Summary 91 IV Chapter 4: 93 Experiment 5: Kinematic measures of single- and dual-task pointing 93 Methods 96 Results 101 Discussion 106 Chapter 5: 109 Experiment 6: Spatially Remapped Pointing 109 Methods I l l Results 114 Discussion 119 Chapter 6: 122 Experiment 7: Comparing direct and indirect pointing methods 122 Methods 124 Results 127 Discussion 132 Experiment 8: Memory-based pointing 134 Methods 136 Results 139 Discussion 143 Chapter 7: General Discussion 147 Theoretical Implications 151 Relevance to existing research on the planning and execution of action 151 Relation to existing theories on Divided Attention 153 Relation to existing theories of perception and action 154 Remaining Questions & Directions for Future Work 157 Practical Implications 163 Final Remarks 164 References 165 V LIST OF FIGURES Fig. 1. Dorsal and ventral streams in the macaque 4 Fig. 2. Location of intraparietal sulcus, superior and inferior parietal lobules 14 Fig. 3. Sample trial sequence for Experiment 1 50 Fig. 4. Experiment 1 results: Letter identification and pointing to letter target 55 Fig. 5. Sample trial sequence for Experiment 2 62 Fig. 6. Experiment 2 results: Letter identification and pointing to disc 64 Fig. 7. Sample trial sequences for Experiment 3 70 Fig. 8. Experiment 3 results: Negative & positive temporal lags 72 Fig. 9. Experiment 4 results: Negative & positive lags under deadline pressure 82 Fig. 10. Sample trial sequence for Experiment 5 99 Fig. 11. Experiment 5 results: Behavioural measures 102 Fig. 12. Experiment 5 results: Kinematic measures 103 Fig. 13. Sample trial sequence for Experiment 6 112 Fig. 14. Experiment 6 results: Behavioural measures for spatial remapping 115 Fig. 15. Experiment 6 results: Kinematic measures for spatial remapping 116 Fig. 16. Experiment 7 results: Direct pointing versus indirect mousepointing 128 Fig. 17. Temporal sequence of events for Experiment 8 138 Fig. 18. Experiment 8 results: Memory-based pointing in immediate and delayed conditions 140 Chapter 1 1 C H A P T E R 1: I N T R O D U C T I O N Human vision allows us both to perceive and to interact with our surroundings. On the perceptual side, vision allows you to recognize a colleague, describe the room you are in, and to discriminate scissors from a spoon. Vision also guides action—it allows you to shake your colleague's hand, to walk around the room without bumping into the wall, and to pick up scissors differently than you would a spoon. In our waking lives, vision constantly serves the dual purposes of guiding both perception and action. Milner and Goodale (1995) have advanced a Perception-Action theory of two independent visual systems that instantiates these functions of perception and action in two distinct cortical systems: a ventral stream responsible for conscious perception (e.g., object identification, scene categorization) and a dorsal stream responsible for unconscious, online control of visually guided action (e.g., pointing, reaching, grasping). Both physiological and behavioural evidence support the existence of two streams of visual processing that subserve these dual functions of vision. In the realm of behavior, there is evidence that both visually guided perception and action tasks frequently require attention in order to operate efficiently. Purpose of Thesis Perception and action are frequently demanded at the same time. We read the news online while reaching for a coffee cup, or we monitor what is happening on the backburner of the stove while adding ingredients to the pot on the front burner. On a more safety-critical scale, while driving, we scan for cars, pedestrians, and other hazards while manipulating dashboard controls. Pilots monitor visual information inside and outside the cockpit while acting upon cockpit controls. However, it has been frequently demonstrated that humans cannot efficiently perform two tasks in a narrow Chapter 1 2 window of time. It is important to understand what the limits of performance are in multiple-task situations, as this knowledge is applicable to time- and safety-critical situations such as driving and aviation. Knowledge of the abilities and limitations of dual-task performance will allow for the design of environments and interfaces that minimize interference, allowing for quick responses and reduced error. The question of what happens to human behavior when we try to carry out two perceptual tasks at once is classic in the fields of visual perception and cognitive science, but little is known about our ability to carry out perception and action tasks concurrently. The Perception-Action theory provides a novel theoretical framework for approaching the question of how two tasks can or cannot be carried out simultaneously. As it has been demonstrated that there are two independent streams of visual processing, a question that arises is whether this functional independence also implies independence in the attentional mechanisms that subserve each stream. The focus of this thesis is on whether these two functions of visually guided perception and action can operate independently of each other in an attentionally demanding situation, or whether they compete for access to common sources of information and on operations performed by other regions of the brain. In pursuit of this question of dual-task performance, this dissertation describes a series of experiments that examine the consequences of performing perception and action tasks within the same narrow window of time. Before turning to the experiments in subsequent chapters, I will first review the current theories and behavioral evidence regarding the dual functions of vision in this introductory chapter. This will include: (a) the purposes of the two visual streams, their characteristics, and their interactions, (b) the planning and execution phases of action, (c) attention and visually guided action in selective and divided attention situations, and (d) theories and evidence relevant to Chapter 1 3 situations requiring both perception and action. This will be followed by a description of the present questions and an overview of the thesis experiments. The Two Purposes of Vision: Perception and Action The visual system has long been characterized as consisting of two anatomically and functionally separate streams. These streams are specialized for processing information for object identification and spatial localization. In early anatomical conceptions of two visual streams, the focus was upon pathways that diverged in the optic tract between the retina and the visual cortex (Held, 1970; Schneider, 1969; Trevarthen, 1968). The tectopulvinar pathway through the superior colliculus was thought to be specialized for location ('where'), while the geniculostriate pathway through the lateral geniculate nucleus was thought to be specialized for identification ('whaf). Later, Ungerleider and Mishkin (1982) shifted the focus in dual stream research to pathways that diverged after the primary visual cortex (VI). After initial processing in VI , visual information is projected in parallel along the ventral and dorsal pathways, which have been mapped out anatomically in the cortex of the macaque monkey (Ungerleider & Mishkin; Van Essen & DeYoe, 1995). The ventral stream projects from VI to the inferotemporal cortex, whereas the dorsal stream projects from VI to the posterior parietal cortex. The dorsal stream also receives input from the superior colliculus via the pulvinar. Figure 1 illustrates a schematic of the major routes of information flow from the retina to the ventral and dorsal streams in the macaque. Chapter 1 4 Figure 1. Dorsal and ventral streams in the macaque Illustration of the major routes by which retinal input reaches the dorsal and ventral streams. The diagram of the macaque brain (showing right hemisphere) illustrates the approximate pathways of the projections from the primary visual cortex (VI) to the posterior parietal cortex (dorsal stream) and the inferotemporal cortex (ventral stream). Abbreviations in the diagram are as follows: lateral geniculate nucleus, pars dorsalis (LGNd), pulvinar (Pulv), and superior colliculus (SC). Reprinted from Goodale, M . A . , & Humphrey, G. K . (1998), The objects of action and perception, Cognition, 67,181-207, with permission from Elsevier. Ungerleider and Mishkin (1982) based their theory largely on lesion studies involving the macaque monkey. Their proposal, in brief, was that the ventral stream processes information for object perception ('whaf), while the dorsal stream processes information about the spatial relations between objects ('where'). This has been one of the most influential theories as to how visual information is organized in the brain. The focus of the dorsal stream was on the perceptual function of apprehending spatial layout, rather than upon layout for the purposes of visuomotor control. However, this understanding of the functional differences between the ventral and dorsal streams was recast by Goodale and Milner (1992; Milner & Goodale, 1995). Chapter 1 5 They disagreed that the two streams simply evolved to process separate types of visual information for perception. Rather than focusing on the differences in visual information processed by either stream ('what' versus 'where'), they focused on the different functional outcomes of processing by each stream. In their conceptualization, the ventral stream transforms information for the conscious perception of objects ('whaf, e.g., recognition, identification, categorization), while the dorsal stream transforms information for the unconscious guidance of actions toward objects ('how', e.g., pointing, reaching, grasping). For instance, when looking in your desk drawer, ventral stream processing yields the ability to describe where a stapler is in relation to other objects, whereas dorsal processing allows you to grasp the stapler. In this example, both streams have information about location and shape of objects, but the division in labour is in the purposes for which each stream uses the information. This theoretical position regarding the functions of the ventral and dorsal streams will be referred to as the Perception-Action theory throughout this dissertation. The Perception-Action theory is supported by studies involving the neuroimaging of healthy humans performing various visual tasks (e.g., Grill-Spector, Kourtzi, & Kanwisher, 2001; Culham et al., 2003), studies of dissociations in the visually guided behavior of neuropsychological patients (Balint, 1909; Goodale, Milner, Jakobson, & Carey, 1991; Milner et al., 1991), and through dissociations in the behavior of healthy participants responding to visual illusions (Carey, 2001; Chua & Enns, 2005; Goodale, Pelisson, & Prablanc, 1986). Each class of evidence will be briefly considered in turn in the following paragraphs. Neuroimaging and neurophysiological data in humans is generally in agreement with these functional and anatomical distinctions between the two streams. The lateral occipital complex, a region of the occipitotemporal cortex in the ventral stream, is active Chapter 1 6 in object recognition (Grill-Spector et al., 2001). In the dorsal stream, areas centred around the intraparietal sulcus are active during visually guided actions. For instance, the anterior intraparietal (AIP) region has been implicated specifically in grasping (e.g., Binkofski et al., 1999; Culham et al., 2003). Compelling neuropsychological evidence for the dissociation between perception and action comes from patients with selective brain damage. Patients with optic ataxia (Balint, 1909) typically have damage to the superior portion of the posterior parietal cortex (dorsal stream). For the hemisphere that is contralateral to the lesion, these patients exhibit impaired dorsal functioning with intact ventral functioning (Jeannerod, 1988; Perenin & Vighetto, 1998). Such patients can use visual information to recognize and describe objects and they can even grasp these objects using tactile or proprioceptive information. However, they have deficits in their ability to use vision alone to reach for and grasp these same objects. Conversely, there are patients who show the opposite pattern of behaviour, exhibiting impaired ventral stream functions and intact dorsal stream functions. The best known example of this pattern of functioning is seen in patient DF. She sustained extensive damage to the ventrolateral regions of her occipital lobe, resulting in severe visual form agnosia (Goodale et al., 1991; Milner et al., 1991). While D F cannot identify or describe characteristics of visually presented objects (e.g., the orientation of a mailbox slot), she can use these characteristics to accurately direct actions towards the same objects (e.g., she can place a letter in the slot). Neuroimaging has confirmed that D F indeed has bilateral lesions that correspond to the location of the lateral occipital complex in healthy humans (James, Culham, Humphrey, Milner, & Goodale, 2003). In addition to neuroscientific evidence, the separability of perception and action has been demonstrated in the behavior of healthy participants. The two functions have Chapter 1 7 been dissociated in studies examining performance with static illusions and dynamic target displacements. First, there have been a variety of studies showing that when the perceptual system is fooled by visual illusions, the action system is not. For instance, while observers' verbal reports are biased by the Ebbinghaus size-contrast illusion (target disk surrounded by smaller disks appears larger than when same disk surrounded by larger disks), their grasp is scaled toward the actual, not the perceived, size of the disk (Aglioti, DeSouza, & Goodale, 1995; Haffenden & Goodale, 1998, 2000; Haffenden, Schiff, & Goodale, 2001). However, evidence from illusion studies has been criticized for using inappropriate measures and for an unnecessary mapping of the results onto the perception-action, ventral-dorsal distinction (e.g., Carey, 2001; Franz, 2001; Franz, Gegenfurtner, Bulthoff, & Fahle, 2000). A second line of behavioural evidence comes from studies dissociating the sensitivity of perception and action to dynamic target displacements (also referred to as target jumps or "double-step" targets). When performing an action toward a target object that is displaced during the movement, both saccadic and pointing movements are sensitive to target displacements. Specifically, participants will accurately look to or point to the new target location, even when they cannot consciously report the presence of the change (Bridgeman, Kirch, & Sperling, 1981; Bridgeman, Lewis, Heit, & Nagle, 1979; Chua & Enns, 2005; Fecteau, Chua, Franks, & Enns, 2001; Goodale, Pelisson & Prablanc, 1986). Thus, participants have access to the precise coordinates of the object to control action, even without perceptual awareness of unexpected changes in the target location. Chapter 1 8 Stream Characteristics In the context of the Perception-Action theory, not only are the purposes of each stream distinct, but the characteristics of each stream differ in ways that reflect the functions of each system. These differences are with respect to frames of reference, memory, speed, and consciousness. The ventral system codes object locations based on allocentric coordinates (in relation to the environment), which allows one to be conscious of the relative position of objects in the environment. The dorsal system codes location in egocentric coordinates (in relation to the observer), which allows one to act within and upon the environment in an accurate manner. The coordinate systems used for actions are effector-specific, such that the coordinates used for reaching are different from those used for saccades. Also, because these coordinates are continuously updated to effect visuomotor control as an individual moves through the environment, there is little need for the dorsal system to retain a memory of coordinates for past actions, whereas the ventral stream has a longer memory for object attributes in order facilitate identification and recognition of objects. In addition, the functions of the dorsal stream are generally completed much more rapidly than that of the ventral stream. Anatomically, this is reflected in the fact that the dorsal stream receives primarily magnocellular input, which is more rapidly transmitted than parvocellular input (Lamme & Roelfsema, 2000; Nowak and Bullier, 1997; Schmolesky et al., 1998). The ventral stream receives input from both the magno and parvo pathways. Behaviorally, the greater speed of the dorsal stream has been demonstrated with the finding that motor corrections to sudden target displacements can occur before perceptual awareness of the displacement can be consciously reported (Castiello, Paulignan, & Jeannerod, 1991). Chapter 1 9 As mentioned in the preceding discussion, the results of dorsal stream processing are also less accessible to consciousness than that of the ventral stream. Some of this evidence comes from the dissociation in conscious reports between dorsal and ventral lesion patients described earlier. For instance, patients with dorsal lesions can verbally describe the spatial characteristics of an object, such as its location and orientation, even if they cannot accurately act on these objects (Perenin & Vighetto, 1998). On the other hand, patients with ventral lesions can successfully act upon objects, even in the absence of the ability to consciously report the location and orientation of these objects (Milner et al., 1991). In healthy individuals, there is an ability to generally describe the features of an object related to its identity (e.g., shape, size, colour), which is information provided by the ventral stream. However, this information is imperfect such that individuals are not consciously aware of how this visual information is transformed in order to facilitate metrically precise motor actions (Goodale & Haffenden, 1998). The dorsal stream has access to this metrically precise information to guide accurate action. This dissociation between veridical perception and action is demonstrated through the unexpected target displacements described earlier, where observers can be unaware that a target has moved, yet can accurately saccade or point toward the final target location (Bridgeman et al., 1979,1981; Fecteau et al., 2001; Goodale et al., 1986). According to the Perception-Action model, this is in accordance with only the dorsal system being sensitive to target displacements. Conscious reports, however, are reflective of the contents of the ventral stream. In summary, the dorsal stream uses egocentric coordinates, has little to no memory, performs fast computations, and uses information that is not readily available to consciousness. On the other hand, the ventral stream uses allocentric coordinates, has Chapter 1 10 a memory for intrinsic object properties, is slower acting, and has results that can be made available to consciousness. Stream Interactions While the Perception-Action theory emphasizes the independence and differences between the two systems, it also proposes that the parallel functions of these pathways are typically coordinated to allow for seamless interaction between the two systems. This may come about because of ongoing system communication or 'crosstalk' (Lennie, 1998). This interaction may also arise because the dorsal system feeds its results back to area V I more quickly than the slower ventral stream can (Lamme, 2001), thus having a direct influence on the feedforward input of area V I to the ventral stream. However, underneath the smooth coordination between the parallel systems lies the fact that there are two independent streams of visual processing. The convergence of neuropsychological and behavioural evidence suggests that each stream can indeed function separately to accomplish tasks of perception (e.g., identification) or action (e.g., reaching) that can be considered to be attentionally demanding (e.g., Pashler, 1998a, Tipper, Howard, & Houghton, 1998). What has not yet been systematically explored is how these two systems are linked when they are required simultaneously. Research on dual-task performance indicates that it is generally quite difficult to perform two visual tasks concurrently without cost (Pashler, 1994; Shapiro, 2001). While the Perception-Action theory (Milner & Goodale, 1995) itself is not a theory of attention, it does have interesting and largely unexplored implications for the attentional costs and concurrent performance of perception and action tasks. There are two general classes of possibilities regarding the attentional mechanisms that subserve dorsal and ventral stream tasks. Chapter 1 11 One possibility is that dorsal and ventral streams do not share attentional mechanisms, such that there may be independent mechanisms or resources for each stream. The evidence for the anatomical and functional independence of the two streams, combined with the finding that dorsal functions can be accomplished with minimal awareness, has lead researchers to raise the possibility that each system can process a different source of visual information with little to no interference (Liu, Healey, & Enns, 2003; Norman, 2002). This raises the possibility of efficient multi-tasking if the appropriate tasks are combined. If the independence of the two streams is associated with independence in attentional processing for the two streams, then if one task relies primarily on ventral stream processes and another relies primarily on dorsal stream processes, then efficient task sharing may be possible. The other possibility is that there is a unitary attentional mechanism that both streams tap during task performance. Indeed, it has been proposed that there is a unitary, master attentional system (e.g., Mesulam, 1981; Posner & Petersen, 1990). In this case, the prediction is that dual tasks wi l l interfere with one another even if they separately rely on processes in the ventral and dorsal streams. The purpose of the present series of experiments was to test between the two possibilities of independence or interdependence, with a particular emphasis on how a perception task influences an action task as it unfolds. But first, to help the reader understand how ventral stream perception might influence visually guided action, there needs to be a discussion of how the dorsal and ventral streams are differentially involved in action. Carrying out an action includes both planning and execution of the movement, and the possibility has been raised that these two components of action are controlled by different neural regions. The next section w i l l describe these components Chapter 1 12 and the interpretation of their underlying neural substrate according to the Perception-Action theory. Planning and execution of action Human movement can be considered as being comprised of two distinct phases of neural processing: planning and execution (Schmidt & Lee, 1999). Planning consists of all the processes that occur prior to the initiation of any physical action, whereas execution consists of the processes involved in bringing the action to completion. As such, the duration of the planning phase can be assessed by measuring the period from the onset of a target to the onset of movement. This is known as reaction time or initiation time (the term that w i l l be used in this thesis). This measure is generally considered to index the time needed to complete the mental sub-processes of target identification, response selection and movement planning or preprogramming (Henry & Rogers, 1960). The time period from the onset of movement to its completion is known as movement time and it is considered to index the time needed for movement execution and any processes that are involved in controlling the movement during its execution. Total movement time is often further subdivided into two component phases: (1) an initial, ballistic period that reflects programming of the movement characteristics (e.g., movement direction, amplitude), and (2) a latter period that reflects refinement and error-correction of the movement, typically incorporating visual feedback in order to minimize the error between the effector and the target (Elliott, Helsen, & Chua, 2001; Woodworth, 1899). Execution performance is frequently measured using movement time and error variables. More recently, the advent of three-dimensional (3D) motion Chapter 1 13 analysis systems have allowed for the collection of more sensitive measures of execution such as movement velocity, acceleration, and trajectory. Within the Perception-Action theory (Goodale & Milner, 2004; Milner & Goodale, 1995), planning and execution of action are generally thought to be controlled by the ventral and dorsal streams, respectively. Planning of movement is thought to be a cognitive activity controlled by a perceptual system, including the ventral stream and right inferior parietal lobule (IPL), that allows for selection of the target object. It creates a representation of objects and their relationship to the environment. As such, it is more susceptible to cognitive influences such as semantics and memory. The execution and online control of action, on the other hand, is hypothesized to be controlled by the dorsal stream. This includes initial specification of movement parameters, such that visual information is transformed into the coordinates for action in the dorsal stream. In other words, the dorsal stream plays an important role i n controlling movement from the time that the execution of movement is initiated. The need for planning and execution to be more clearly understood within the Perception-Action theory has been recently highlighted by an alternative theory of action, the planning-control model, that also states that the planning and online control components of action are subserved by different visual representations (Glover, 2004). However, in the planning-control model, planning is linked to activity in the inferior parietal lobule (outside the dorsal and ventral streams), whereas control is linked to activity in the superior parietal lobule (SPL, in the dorsal stream). The Perception-Action theory, on the other hand, proposes planning is linked to processing in the ventral stream and the right inferior parietal lobule, whereas the initial movement parameters are controlled by dorsal stream processing, including the intraparietal Chapter 1 14 sulcus (IPS) and superior parietal lobule (Goodale & Milner, 2004). These regions of the brain are outlined in Figure 2 for both the macaque and the human brain. Figure 2. Location of intraparietal sulcus, superior and inferior parietal lobules Illustration of the location of the intraparietal sulcus (IPS), the superior parietal lobule (SPL), and the inferior parietal lobule (IPL) on a lateral view of (a) the macaque monkey brain and (b) the human brain. Adapted from Culham, J. C , & Kanwisher, N . G. (2001), Neuroimaging of cognitive functions in human parietal cortex, Current Opinion in Neurobiology, 11,157-163, with permission from Elsevier. One of the key differences between the planning and control model and the Perception-Action theory is that the former model claims that specification of movement parameters is accomplished in the inferior parietal lobule during the Chapter 1 15 planning phase, whereas in the Perception-Action model, this specification is accomplished by the dorsal stream. The focus of the experiments in this thesis is to understand the attentional interference effects of a perception task on the components of action in the context of the Perception-Action model. If action planning is controlled by a more cognitive, perceptually based system that includes the ventral stream, then a model of independent attentional control for ventral and dorsal streams would predict that a concurrent, attentionally demanding perception task (ventral) would interfere with action planning. At the same time, execution of an action, which is hypothetically controlled by the dorsal stream, should show no interference from a concurrent perception task. On the other hand, if the attentional mechanisms subserving both streams are held in common, then one would predict interference on both the planning and execution stages of action from a concurrent task of perception. Attention and visually guided action tasks. Before turning to a discussion of how attention for perception and action tasks might be coordinated, the attentional demands of action will be discussed. Research on attention can be broadly divided into questions of selective attention (selecting one stimulus amongst several to respond to) or divided attention (performing more than one task with one or more stimuli). While the focus of the experiments in this dissertation is upon divided attention for concurrent perception and action tasks, it is useful to briefly review evidence on selective attention in order to show that action can be considered to be an attentionally demanding process, as is the case for perception. Chapter 1 16 Selective attention One of the tasks of attention is to select certain perceptual input for further processing and to select a particular action (Lavie, 1995; Reason, 1990). However, research on attention typically focuses on perceptual tasks rather than action tasks. Visual perception tasks, such as detection and identification, have been repeatedly shown to be susceptible to interference from competing visual stimuli (e.g., Neisser, 1967; Treisman & Gelade, 1980). Compared to the large volume of research devoted to attention for perception, there is considerably less research devoted to attention for control of visually guided action, where movements are directed toward the target. This is an unfortunate case of neglect, because attention should be understood in the context in which we have evolved to function (Marr, 1982). One of the primary functions of vision is to allow us to interact with objects in the environment, through approach, avoidance or manipulation. Since the early 1990's, some progress has been made to redress this imbalance in research, and researchers have been able to further elucidate how attention is used for action. Allport (1987) describes the problem of attentional 'selection-for-action' as one in which there are several potential action targets in the environment, but motor actions are directed to only a single target. For instance, when picking berries from a bush, information about primarily one berry determines each reach into the bush. Information about other berries may influence reaching (e.g., another berry may be avoided), but information about other objects is temporarily decoupled from the control of action. Thus, there must be an attentional selection process in which spatial information about the target object (e.g., location, shape, size) is processed to a higher level so that actions can be accurately directed toward this particular object. Failures i n the coordination of Chapter 1 17 this information can be seen in action slips, where the correct action is performed on the wrong object, or the wrong action is performed on the correct object (Norman, 1981; Reason, 1979). In Allport's characterization of attention for action, distracting objects can interfere with action at the level of selecting the target. Importantly, this interference can be exhibited in the planning to execution stages. At the same time, this characterization of attention for action does not address what attentional requirements are necessary (if any) after target selection. To study the attentional limits of action, paradigms typically used to investigate the attentional limits of perception have been applied to the context of action. Demonstrations of distractor interference are one such example. The presence of distractors typically interferes with perceptual tasks such as identification by increasing response times and errors (e.g., Eriksen & Eriksen, 1974; Stroop, 1935). Presumably, this interference occurs because the distractors are also processed to a semantic level and therefore activate representations that compete with the target item for control of responding. In an action context, distractors may also activate competing action representations that compete with the target item for control of responding. In line with this idea, it has been shown that distractors affect reaching, pointing, and grasping only if the distractors have action- and task- relevant properties. Castiello (2001) showed that grasping and pointing were subject to attentional capture from abrupt onsets of 2D and 3D distractors, but this interference depended on whether the distractors could be grasped or pointed to. Movement execution of unspeeded grasping was clearly interfered with by distractors, as movement kinematics and duration were interfered with by 3D (graspable) but not 2D distractors (nongraspable). However, pointing was Chapter 1 18 interfered with by both 2D and 3D distractors (both can be pointed to). Thus distractors interfered with action when they possessed features relevant to the primary task. Similar task-relevant action interference has been shown for pointing in the presence of distractors along the movement path. Tipper, Lortie, and Baylis (1992) demonstrated that distractor lights increased total response times (initiation plus movement time) for speeded reaching toward a target light, but only if the distractors were presented along the movement path between the starting position of the hand and the target light. In a follow-up study, reaction time, movement time and the kinematics of reaching to a coloured block were shown to be interfered with by the presence of distractors (Tipper, Howard, & Jackson, 1997). However, the interference shown by Tipper et al. (1997) was only present when participants did not have advance knowledge of the target location before the movement cue was given. When the target location was specified before the movement cue was given, the interference disappeared. Tipper et al. thus hypothesized that the interference shown when the target location was unknown occurred at early target selection processes (showing up in initiation time), but continued to interfere as the movement unfolded (showing up in movement time and movement path). For unspeeded and speeded grasping of a target that had its location specified in advance of movement, Castiello (1996) similarly found an absence of movement interference when examining the effect of distractors. In general, the initiation time, duration, and kinematics of movement were not perturbed by the presence of distractors. Thus, it was possible that planning and selective attention processes directed toward the target were completed before any behaviour was executed. The prediction that would follow from this argument is that if the target changed location, Chapter 1 19 perhaps requiring a new target location to be selected, then interference would again appear. For the purposes of the present discussion, the important point to be gathered from the research on selective attention in action is that both planning and execution of action are susceptible to attentional interference from distractors, at least when they share characteristics with the movement target. Additionally, the possibility is raised that interference during target selection can manifest itself throughout the execution of movement. In other words, attentional demands will affect action on an ongoing basis during movement performance. Divided attention The previous section addressed the finding that there are performance costs to selectively attending to a perception or action target in the presence of distracting stimuli. There are also performance costs when participants have to divide their attention across multiple tasks. Typically, divided attention research addresses the concurrent performance of two tasks, referred to as a dual-task performance. Dual-task research has generally examined tasks that depend on the ventral stream, including visual detection, discrimination, and identification as indexed by explicit (conscious) reports. A common finding is that, when having to attend to two tasks simultaneously or in rapid succession, there is a performance decrement relative to performance of either task alone. Costs are usually seen as increases in error rates or reaction times. The task suffering most is usually the one given lowest priority by the participant, or the one that is attempted second when the tasks are performed in rapid sequence (Pashler, 1994; 1998b; Di Lollo, Kawahara, Zuvic, & Visser, 2001). For instance, when presented with a rapid sequence of visual stimuli containing two targets, correct responding to the first target frequently leads to impaired Chapter 1 20 responding to the second target. This impairment has been shown with the "attentional blink" paradigm. In this paradigm, targets are masked, responses are typically unspeeded, and the impairment is manifested in a decrease in accuracy for reporting the second target, particularly when the first and second targets are 100 to 500 ms apart (Raymond, Shapiro, & Arnell , 1992; Shapiro, Raymond, & Arnel l , 1994). This impairment can also occur when the two targets are in different spatial locations (e.g., Kristjansson & Nakayama, 2002; Visser, Zuvic, Bischof, & D i Lollo, 1999). Even simple perceptual tasks such as detection have been shown to be impaired when closely following letter identification (Joseph, Chun, & Nakayama, 1997). Dual-task interference has also been shown with the 'psychological refractory period' (PRP) paradigm, in which targets are not masked, responses are typically speeded, and the impairment is manifested in an increase in reaction time to the second target (Pashler, 1994). It is generally agreed that the impairment in second target performance is produced by a temporary unavailability of attentional processing that results from attending to the first target. To explain dual-task interference, there are three broad classes of theories on limitations to divided attention: bottleneck, resource, and crosstalk theories (Pashler, 1994; Pashler & Johnston, 1998). In bottleneck, or single-channel, theories, the primary idea is that certain mental operations necessary for task performance (e.g., response selection) can only be carried out sequentially. A bottleneck arises when two tasks require this critical mental operation at the same time. In resource theories, there are one or more pools of limited "resources" that are available for processing tasks and stimuli. Thus, when resources are devoted to one task, then they are less available for processing a second task. Unlike the predictions made by the bottleneck theory, processing for two tasks can occur in parallel, but the Chapter 1 21 processing is less efficient because fewer resources are available for one or both tasks. Resource theories postulate either a unitary resource (Kahneman, 1973) or multiple resources (Navon & Gopher, 1979; Wickens, 1980). The latter type of theory allows for the possibility that tasks that tap different resources will not interfere with one another. A third class of theories, crosstalk theories, suggests that performance limitations depend on the content of the information being processed. Specifically, if the tasks are more similar, then concurrent performance of such tasks would result in more impairment. On the other hand, if two tasks are quite different, then there should be less impairment. This is similar to the bottleneck idea in the sense that if two tasks require the same critical mental operations, then crosstalk should occur. Each class of theories makes different predictions for the functioning of concurrent ventral perception and dorsal action tasks. The following predictions also consider the idea that visual perception and action planning involve the ventral stream, whereas action execution involves the dorsal stream. If there is a unitary attentional bottleneck, then simultaneously attending to perception and action tasks should result in dual-task interference. This interference should occur regardless of whether perception overlaps with the planning or execution stages of action, if all stages require the same critical operation. If there are multiple resources for ventral and dorsal streams, then there should be minimal dual-task interference, particularly when the perception task overlaps with the execution of action. According to crosstalk theories, if the perception and action tasks are different enough, the two tasks might be able to run concurrently without interference, even at the planning stages of action. Divided attention and action While perceptual tasks have frequently been shown to interfere with one another when they must be performed in close temporal proximity, there has been extremely Chapter 1 22 limited research conducted on dual-task interference for visually guided action tasks. Evidence from the literature on bimanual reaching, however, is relevant to this discussion, as it involves two concurrent reaching movements to action targets. In general, this evidence shows that action planning is clearly susceptible to dual-task interference from concurrent action planning, whereas execution is sometimes interfered with by planning, and is not clearly interfered with by concurrent execution. For instance, Hazeltine, Diedrichsen, Kennerley, and Ivry (2003) compared bimanual pointing to two separate targets that had (a) both locations cued symbolically and indirectly by a letter, (b) both locations directly cued by the onset of the target (as will be used in the current thesis experiments), or (c) one location cued symbolically and the other cued directly. The temporal interval, or lag, between the onsets of each target cue was varied between 50 and 1000 ms. For pointing to the second target, the initiation time for symbolically cued actions was longer than the initiation time for directly cued actions, presumably because of the greater planning requirements needed to translate between the symbolic cue and the target location. Also, regardless of whether the first action was symbolically or directly cued, there was an impairment in the initiation time of the second pointing action, symbolically or directly cued, when a short lag intervened between targets (perceptual refractory period effect). This suggests that the planning of either a symbolically or directly cued action interferes with the concurrent planning and initiation of both symbolically and directly cued actions. This finding is consistent with the idea that action planning taps the same ventral stream processes for either directly or symbolically cued actions. A n interesting aspect of the perceptual refractory period effect shown by Hazeltine et al. (2003) was that the initiation time interference for directly cued second actions was greater when it was preceded by an symbolic, rather than direct, action cue. Chapter 1 23 Additionally, at the shortest lag, there was a cost for direct pointing movement time of the second action when it was preceded by a symbolic, rather than direct, action cue. Thus, both the planning and the execution of a direct action showed greater interference from the planning and/ or execution of symbolically, but not directly, cued actions. This is contrary to the hypothesis that a dorsal task (execution of directly cued actions) would show less interference from a ventral task (symbolically cued action planning) because they are independently controlled. Instead, it indicates that symbolically cued action planning w i l l lead to greater interference on a directly cued action, both at the level of planning and execution. This implies that directly cued action execution is not impervious to interference from a concurrent ventral task. However, Diedrichsen, Nambisan, Kennerley, and Ivry (2004) demonstrated results that supported the idea that the execution of action may not compete for attentional mechanisms in the same way that perceptual tasks typically do. They showed that with bimanual reaching to two separate, directly cued targets, online adjustments to targets that changed location during movement were just as efficient as unimanual pointing to targets that changed location. While initiation time was not reported, there was no interference in reaching movement time and error. Examination of kinematic data indicated that when one target moved and the other did not, that the trajectory of pointing to the stationary target was subtly influenced by displacement of the other target, but this deviation was quickly corrected and d id not show up as interference in pointing movement time or error. Thus, the execution of two direct actions can be controlled nearly (but not completely) independently to target displacements. The results of the preceding studies (Diedrichsen et al., 2004; Hazeltine et al., 2003) suggest that (a) the execution of directly cued action may be interfered with by Chapter 1 24 symbolically cued action planning, but (b) the execution of directly cued actions may be controlled almost independently from a concurrent, directly cued action. While the previous section on selective attention indicated that pointing and reaching actions can be subject to attentional interference (Castiello, 1996; Tipper et al., 1992), the evidence that the bimanual execution of actions can be carried out with minimal interference suggests that directly cued actions in divided attention situations may not be subject to the magnitude of attentional interference in the same way that ventral tasks are. In general, the previous sections have indicated that tasks of either perception or action are subject to attentional interference, at least when there is competition for target selection from distracting objects. Also, there is evidence that when dual-tasks are required, perceptual tasks are frequently prone to interference. Interestingly, however, the execution and online control of directly cued action does not appear to be clearly interfered with by the execution of concurrent, directly cued actions, but it is occasionally interfered with by action planning. Thus, it is still unclear how directly cued, visually guided actions (as are required in the current body of experiments) w i l l be affected in a dual-task context with a perceptual identification task. Theories and Evidence Relevant to Dual-Task Perception and Action The discussion w i l l now turn back to dual-task performance of perception and action in attentionally demanding situations. As research in attention for action has been traditionally neglected compared to attention for perception, the theories that address their concurrent performance are limited as well . Three theories w i l l be discussed that shed light on the interactions and interference that may occur when perception and action are demanded in a dual-task context. (1) The premotor theory of attention and its extensions (Rizzolatti, Riggio, Dascola, & Umilta, 1987; Rizzolatti, Chapter 1 25 Riggio, & Sheliga, 1994) allow for multiple circuits controlling attention for both dorsal and ventral stream tasks. (2) The integrated competition hypothesis (Duncan, 1996) states that attention emerges from multiple brain circuits converging to process a single object. (3) Finally, the Visual Attention Model (Schneider, 1995) postulates that a common visual attention mechanism serves both perception and action, and that this mechanism is necessarily bound to one object at a time. The limitation of these models in the current context, however, is that they are primarily designed to deal with target selection in the planning stages of action and do not address the execution of action. This thesis will directly address the question regarding dual-task interference in action execution. Before describing the experiments that were conducted, the three existing theories of concurrent perception and action will be reviewed so that readers will have a context for evaluating the results from the perspective of each of these theories. Premotor Theory of Attention The basic premise of the premotor theory (Rizzolatti et al., 1987, 1994) is that spatial attention is the result of preparing a motor action. Covert attention differs from overt action in that a separate "go" signal is needed for overt action. Action execution is therefore not necessary to produce orienting to specific spatial locations, only action preparation is. While this theory was initially formulated to address how attention is related to action, it has been extended to encompass attention for perception. The original theory postulates that attention is controlled in dorsal spatial-motor areas. Working against the assumption that there is a unitary, master attention system centered in the parietal lobe (e.g., Mesulam, 1981; Posner & Petersen, 1990), Rizzolatti and colleagues suggest that there are multiple circuits that can control attention in space. Mechanisms for spatial attention are thought to be located in "spatial pragmatic Chapter 1 26 maps," which are specialized sensorimotor circuits that are effector-specific representations of space. Spatial attention is the result of neuron facilitation within these spatial pragmatic maps; this facilitation is the consequence of preparing goal-directed movements in space, such as saccades or reaches. Furthermore, depending on task requirements, different pragmatic maps will become active, and spatial attention can arise from any pragmatic map that codes space. In other words, there is no unitary mechanism for spatial attention that is anatomically separate from these spatial maps. Evidence for different attentional circuits controlling different motor responses and regions of space comes from lesion studies of monkeys (Rizzolatti, Gentilucci, Matelli, 1985). For instance, lesions to areas 7a, 8, and the superior colliculus, which have neurons involved in eye movements (e.g., Goldberg & Bushnell, 1981; Sakata, Shibutani, & Kawano, 1982; Wurtz & Mohler, 1976), result in deficits in attending to extrapersonal space (far space; Rizzolatti et al.). On the other hand, lesions to areas 6 and 7b, which have neurons related to head and arm movements (e.g., Leinonen, Hyvarinen, Nyman, & Linnankoski, 1979; Rizzolatti, Scandolara, Matelli, & Gentilucci, 1981a, 1981b), cause deficits on focusing attention in peripersonal space (near space within reaching distance). The original formulation of the premotor theory was only meant to deal with attention to space and did not specify how attention for ventral stream tasks might be accomplished. More recently, Craighero, Fadiga, Rizzolatti, & Umilta (1999) proposed that the core idea of the premotor theory, that attention arises from the same neural circuits that control sensory transformations, can be applied to the ventral stream. The "pragmatic" representation of the visual world corresponds only to the functioning of the dorsal stream in the parietal lobe. However, a "semantic" representation of the world corresponds to the functioning of the ventral stream in the temporal lobe, and is Chapter 1 27 responsible for tasks such as detection and identification. Specifically, ventral stream processing gives rise to perceptual representations of the visual properties of objects and their meanings. Thus, the circuits that control processing of these properties may be what give rise to attention for ventral stream tasks. In support of this idea, Chelazzi, Miller, Duncan, & Desimone (1993) recorded the activity of neurons in inferotemporal cortex of monkeys performing a selection task (visual search). The monkeys were presented with a cue picture and subsequently had to make a saccade toward a target object amongst distractors. Importantly, the target was cued by identity, but not spatial location. Results indicated that neurons in the inferotemporal cortex had a strong and sustained response to the target, while responses to distractor objects was suppressed. Thus, the heightened activity of cells responsive to target identity allowed for detection of the cued object. This suggests that the attentional mechanisms for selecting and responding to an object based on identity, rather than space, are controlled by the same ventral stream circuits that perform analysis of relevant object features. This does not necessitate a control system that is separate from the neural circuits that process features relevant to the task. In a more general formulation of the ideas of the premotor theory, attention is the result of activating task-relevant sensorimotor circuits. Dorsal stream attention arises from circuits that control processing of motor responses such as a saccade, reach or grasp. Ventral stream attention arises from circuits that control processing of object representations such as identity. Importantly, the theory states that attention for any given task is derived from the activity of pragmatic (dorsal) and semantic (ventral) maps without the involvement of other anatomical structures. This allows for the existence of multiple circuits that control attention for both dorsal and ventral stream tasks. Chapter 1 28 The premotor theory has elements of a multiple resource theory by proposing that there are multiple circuits that control attention. This does not preclude the existence of a bottleneck within each attentional circuit. For instance, within the same circuit, attentional costs can arise from having to change a motor program (e.g., oculomotor; Rizzolatti et al., 1987). There is no specification, however, as to whether these circuits, pragmatic (dorsal) or semantic (ventral), can be activated concurrently without interference. The idea of multiple attentional centers allows for the possibility that these circuits can perform in parallel and that attention for different tasks can function in parallel. Thus, there may be no bottleneck when performing concurrent perception and action tasks. Presuming that dorsal and ventral streams as a whole process visual information in parallel, Milner and Goodale (1995) have suggested that the operation of multiple attention centers in both the dorsal and ventral streams is indeed a reasonable account of how attention works for each stream. But if and when action planning involves the ventral stream, then this would suggest that successful action involves both semantic and pragmatic map activation, as movement may require the semantic maps for selecting the object based on identity, and the pragmatic maps for coding the target location in space and specification of movement parameters. It should be noted, though, that the premotor theory does not have an explicit way of dealing with ongoing attention requirements during motor execution. In contrast to the possibility that dorsal and ventral streams have separate attention centers that might process information independently, other theories suggest that interference will arise when perception and action are directed to different objects. The integrated competition hypothesis (Duncan, 1996) and the Visual Attention Model (Schneider, 1995) will be considered in turn as they pertain to the behavioural consequences of dual-task perception and action. Chapter 1 29 Integrated Competition Hypothesis The integrated competition hypothesis proposed by Duncan (1996; Duncan, Humphreys, & Ward, 1997) states that objects compete in parallel for representation amongst multiple brain systems that respond to visual input (perceptual and motor, cortical and subcortical systems). During this competition, there will be interference in processing multiple objects. As one object becomes dominant in the processing of one system, this dominance tends to spread to other systems. Focused attention emerges as the different systems converge to process a single object that "wins" or dominates this competition. Depending on the task demands, many different types of properties (e.g., colour, location, shape) originating in different brain systems, be it visual or motor, can be used to control object selection. This initial bias then spreads to dominate other processing systems. This process of attending to an object is proposed to emerge over several hundreds of milliseconds, rather than just a few dozen milliseconds as proposed by high-speed models of attention (e.g., Treisman & Gelade, 1980). Within ventral stream functioning, support for the single-object perspective comes from evidence that two attributes of a single object can be identified concurrently without interference, whereas two attributes from two different objects cannot be handled concurrently without cost (Duncan, 1984). Across modalities, evidence for a benefit of attending to only one location has been taken to suggest that there may also be a single-object advantage. For instance, attending to visual and auditory events in the same side of space shows less interference than when attention is split across space (e.g., Spence & Driver, 1996). However, this evidence addresses locations, and not specifically objects. The evidence that is the most relevant to the current discussion of an object-Chapter 1 30 based perception-action coupling comes from studies testing the visual attention model (Schneider, 1995), which will be discussed in the next section. Similar to the premotor theory (Rizzolatti et al., 1987,1994), the integrated competition hypothesis has elements of multiple resource theories of dual-task performance, such that objects compete in parallel for attention amongst several different brain systems. In other words, both theories allow for multiple brain systems that process visual input for target selection. Where the integrated competition hypothesis differs from the premotor theory is that the former is explicit in proposing a single-capacity mechanism of attention, such that the nature of integrated competition will result in different systems processing the same object. As attention to an object emerges, there will be a bottleneck in processing that arises from this object-based convergence. The premotor theory, on the other hand, makes no explicit claims about whether or not different pragmatic and semantic maps will necessarily converge to process one object at a time. While Duncan's (1996) original theory is not specifically formulated to address action systems, a natural extension of this theory is that attention for visually guided perception and action should settle upon a single object. Ward (1999) developed a computational model that explicitly incorporates action into a model of visual selection. While this model makes no claims about what is occurring on a neural basis, it states that processing in perception and action systems have reciprocal influences upon one another, and that the result of this interaction is the gradual emergence of coordinated processing of a single object across multiple systems of perception and action. Thus, the core assumptions of the Ward model are quite similar to that of the integrated competition hypothesis, such that both theories assert that distributed processing will converge upon a single object. Chapter 1 31 The prediction from the integrated competition hypothesis for the ventral and dorsal streams is that there should be an object-specific coupling for perception and action tasks. Specifically, if the system has been biased to select a particular object for perceptual processing in the ventral stream, then this same object will gain dominance for processing in the dorsal stream, and vice versa. The integrated competition hypothesis should predict that dorsal processing would interfere with the ventral stream's ability to process a different object, leading to increases in latency and errors in perception. Conversely, processing in the ventral stream could interfere with selection and planning operations of the dorsal stream, which could also lead to increases in movement time and error for action. Interestingly, the integrated competition hypothesis suggests that attention will be coupled to a single object, regardless of which stream processes the target. Accordingly, two ventral stream tasks should interfere with one another, as long as they are referring to separate objects. For instance, assuming that action planning is a ventral task (Perception-Action model), then processing an object in the ventral stream for the purposes of identification should interfere with action planning to a second object. Thus, while the integrated competition hypothesis does not specifically deal with ventral and dorsal stream processing, it does predict that as long as the two systems are dealing with separate objects (as will be the case in the thesis experiments), there should be dual-task interference. Visual Attention Model The Visual Attention Model proposed by Schneider (1995; Schneider & Deubel, 2002) is the theory most explicitly formulated to address the attentional effects of concurrent perception and action in the ventral and dorsal streams, respectively. This model proposes that there are two purposes of visual attention, selection for perception, Chapter 1 32 which occurs in the ventral pathway (e.g., V4/ inferotemporal cortex), and selection for space-based action, which occurs in the dorsal pathway (e.g., posterior parietal cortex). Like the premotor theory, the model does not specify overt execution of action; instead, a separate and effector-specific 'go' signal is postulated to convert motor programs into executed actions. Like the integrated competition hypothesis, the Visual Attention Model states that attention settles on the processing of one object at a time. Selection is controlled by a common visual attention mechanism that gives priority to low-level visual representations of a single object in V I . This priority is then passed on for higher-level processing in the ventral and dorsal streams, where processing for conscious perception and visually guided action can occur in parallel. Because this model proposes that selection for perception and spatial-motor action is coupled by a common attentional mechanism, it predicts that programming a movement will bind conscious perceptual processing to the movement target and its location. Thus, if an object is selected for a reach or grasp movement, then perceptual processing is best for that object. Conversely, if an object is selected for perceptual analysis, then motor responses will be optimized toward this same object. According to this model, attending to two objects can only be accomplished serially. As such, visual perception of one object should interfere with simultaneous preparation of a motor response to a different object in a different location. In support of the Visual Attention Model, Deubel, Schneider, and Paprotta (1998) have demonstrated that, in a dual-task context, selecting an object for motor action also results in perceptual processing being coupled to that same object. The dual tasks involved discriminating a character amongst an alphanumeric string while simultaneously preparing a speeded left or right reaching movement to a target object within the alphanumeric string (i.e., characters were superimposed atop oval objects). Chapter 1 33 The location of the reaching target was indicated by a symbolic arrow cue defined by colour and shape. Regardless of whether the discrimination target was presented before reaching initiation or at reaching onset, discrimination performance was best and close to single-task discrimination levels when the discrimination target was the same object (or at least in the same location) as the reaching target. Performance deteriorated with increasing spatial displacement between the discrimination and reaching target. Reaching, however, was not subject to the same object-based coupling. Whether discrimination targets were presented before or at movement onset, performance measures of movement initiation time, amplitude and duration were independent of discrimination target location. In other words, movement was not interfered with regardless of whether the discrimination target was presented during the movement planning or execution phase. A similar pattern of results was found when grasping was used as the action task (Schiegg, Deubel, & Schneider, 2003). The fact that observers were instructed to prioritize the reaching task may account for the unperturbed reaching and grasping performance. Deubel et al. (1998) and Schiegg et al., in fact, did not expect any movement interference. However, discrimination performance was better than chance in all conditions, indicating that some level of attention had be directed to the discrimination target even when it was in a different location than the movement target. If action was absolutely coupled to perception at the object level, then dual-task reaching might be expected to suffer relative to single-task performance if any attention was directed toward the separate perceptual target. This was not observed. Thus, while Deubel et al. have shown that perception is interfered with by action, it is not as clear from this evidence whether action is necessarily interfered with by perception when dealing with two targets. Chapter 1 34 Deubel et al. (1998) suggest that even though they demonstrated that dorsal and ventral processing are coupled to the same object during execution (i.e., perceptual target presented at movement onset), execution of pointing can rely on stored motor programs. Specifically, action execution will be bound to the perceptual target in circumstances where the action target needs to be continuously monitored to evaluate movement success (i.e., by comparing the endpoints of the movement and the target position). However, for movements that are highly practiced or automatic, thus no longer requiring feedback control, attention to the movement target may not be required. In support of this, Paprotta, Schneider, & Deubel (in preparation, as cited in Schneider & Deubel, 2002) demonstrated that when reaching movements were blocked to the same location—in other words, when movements could become automatized—the object-based coupling between movement and discrimination disappeared. This also implies that if the pointing were to involve an online correction to a moving target, then the reaching system would need to engage selective attention and the coupling between movement and discrimination would return. One issue in using the Visual Attention Model evidence to speculate about whether action execution can continue without interference from perceptual tasks is that the experiments inspired by the Visual Attention Model always involved a target cue that possibly required more ventral processing than would be required for the direct onset of a target. The target cue was typically a centrally presented, coloured arrowhead that was used to cue the peripheral reaching target. Having to discriminate both shape and colour to determine a target location depends on putatively conscious processes of the ventral stream. It is possible that the action continues to recruit the ventral stream during execution when the target discrimination requires more ventral Chapter 1 35 processing. On the other hand, it may be the case that the dorsal stream controls action execution once target selection via the ventral stream is accomplished. Even if Deubel et al.'s (1998) reaching task was ventrally cued, planning and execution were still not interfered with by concurrent ventral discrimination demands. According to a version of the Perception-Action model that specifies that movement planning is ventral, movement planning should be interfered with by the perceptual discrimination. This was not the case in the Deubel et al. (1998) study. However, as discussed earlier, the priority of movement planning in the task instructions may have obviated any interference effects that could occur in planning. If movement was deemphasized as the primary task, then planning interference may start to be exhibited. The Visual Attention Model line of evidence implies that one cannot concurrently perform action and perception tasks without cost unless they are directed toward the same object. Complimentary evidence was shown by Castiello (1996) when participants had to reach for and grasp a target while consciously monitoring the number of times a second target was intermittently lit up during movement. Movement interference was seen in execution measures, particularly when the perceptual task was presented before the onset of movement, but interference also occurred when the perceptual task was presented later in movement. This interference did not occur when the reaching and counting targets were the same object, again suggesting that if attention needs to be split across two targets for ventral and dorsal stream tasks, then there will be dual-task interference effects. In general, the prediction from Visual Attention Model is similar to that from the integrated competition hypothesis: If the dorsal and ventral streams are dealing with two separate objects, then there will be interference in the dorsal action or ventral perception tasks. Additionally, it suggests that if a dorsal action is highly automatic Chapter 1 36 (e.g., movement blocked to a single location), then the dual-task cost for perception and action to two objects will not hold because attention is no longer needed for the action target and can be fully devoted to the perception target. On the other hand, if the action target moves during execution, then the Visual Attention Model would predict that the dorsal system would again need to attend to the action target to evaluate movement success; however, if participants are already attending to the perception target, then one should expect to see interference in action execution. Overall, none of the three theories—premotor, integrated competition, or Visual Attention Model —clearly predict a result where attentionally demanding dorsal and ventral stream tasks will not interfere with one another when referring to separate objects. While the premotor theory leaves open the possibility of efficient dual-task performance, the integrated competition hypothesis and Visual Attention Model clearly predict interference when dorsal and ventral stream tasks are dealing with two different objects. But in fairness, none of these theories was developed to address the effects of perception on the execution of action. On the other hand, if there are independent attentional mechanisms for the ventral and dorsal streams, then the framework of the Perception-Action theory predicts that perception should interfere with movement planning, but not execution and online control. Additionally, this conceptualization of the Perception-Action theory does not limit the attention of dorsal and ventral systems to only one object at a time. Present Questions The reviewed evidence suggests that both the planning and execution of visually guided action are susceptible to attentional limitations, but not under all situations. Additionally, it is unclear what happens to the planning and execution of action when Chapter 1 37 they are demanded concurrently with a perception task. Thus, the primary question addressed in this dissertation is whether the planning and execution components of visually guided pointing to one object are differentially affected by dual-task demands of a concurrent perception task to a separate object. Within the framework of the Perception-Action theory (Milner & Goodale, 1995), the prediction is that a ventral perception task should interfere with the planning of movement, but not the execution and online control to a separate object. This is based on the currently proposed idea that there is not a unitary mechanism of attention serving both ventral and dorsal streams, but separate mechanisms available to each stream for the control of task performance. This would be consistent with the predictions of a multiple resource or crosstalk theory of dual-task performance, such that ventral and dorsal tasks either tap separate attentional resources, or are dissimilar enough that they do not interfere. It is not consistent with the predictions of a bottleneck theory, which suggests that attentional processing is limited to one task at a time. In the context of the theories of perception and action that have been reviewed, the premotor theory allows for the possibility of multiple attentional control circuits for concurrent preparation of such tasks. The integrated competition hypothesis and the visual attention model, however, predict that attention is bound to one object at a time, and thus there should be dual-task interference because each task refers to a separate object. A n even finer-grained aspect of the present investigation is whether online adjustments of action to targets displaced during execution will be affected by dividing attention between two objects. According to the Perception-Action theory, the dorsal stream is specialized for dynamic control of movements. However, the bulk of the research on concurrent visually guided perception and action deals with stationary Chapter 1 38 targets. Thus, introducing unexpected target displacements on certain trials will be used to ensure that the online control function of the dorsal stream is being tapped. Theories that postulate an attentional bottleneck at the object level (integrated competition, Visual Attention Model) predict that if the dorsal system must update movement parameters to a displaced target that is a separate object from the perceptual target, there will be greater interference in the action execution because the attentional mechanism would not be freed from the perception task to evaluate the new position of the moving target. On the other hand, if the dorsal stream handles dynamic updating to the new position, and the dorsal stream has access to attentional mechanisms separate from the ventral stream, then there should not be interference. To examine this question, dual-task perception and action task performance with separate objects will be examined in the context of both moving and stationary targets. Once the relationship between perceptual identification and the simultaneous online control of action has been established, the thesis will investigate the consequences of having the action task guided by the conscious perception functions of the ventral stream. If there are unitary attentional mechanisms for each visual stream, and if the action task relies primarily on ventral stream processes, then a concurrent perception task should interfere with both planning and execution stages of movement. Thus, action tasks will be tested where participants are required to perform an action (a) to a symbolically cued location, (b) using an indirect mouse-pointing method of localization, and (c) using memory of the target location, manipulations that are designed to tap ventral stream functioning. These results will be compared to the patterns exhibited with direct, dorsally guided action to see if there are different effects on the planning and execution of action when the action relies primarily on the ventral stream. Chapter 1 39 Experimental Overview The basic approach taken in this research was to examine the consequences of performing a perception task (visual identification of an object at the center of gaze) upon a visually guided action task (pointing to a separate target object that appeared suddenly in the visual periphery). To ensure that the processing of the two tasks overlapped in time, the pointing target was presented at several different temporal intervals relative to the identification target. To ensure that the pointing task involved the online control of action, in some experiments the pointing target was displaced to a new location when the pointing action started (Experiments 1 to 4) or when the eyes had started moving to the target (Experiments 5 and 6). The main comparisons of interest were pointing in a dual-task context (concurrent with visual identification) versus pointing to the same targets in a single-task context (the same displays were presented but participants ignored the central identification stimuli). Any differences between measures of pointing performance in these two contexts could thus be attributed to the influence of participants being strategically set up perform these two tasks concurrently. Additionally, if task differences decreased as the temporal interval between the two tasks increased, it would suggest that differences were caused by the perception and action task competing concurrently for the same mechanisms. Having these two distinct measures of dual-task interference provided an opportunity to distinguish between more general interference deriving from simply being prepared to perform two different tasks (dual-task set effect) from the more specific effects of two tasks relying on the same short-duration micro-processes (concurrent competition effect, referred to as a concurrency cost throughout the thesis). Chapter 1 40 [Chapter 2] Experiment 1 sought to establish the basic behavioural effects of a perceptual letter identification task upon a pointing task, as well as establish that the perception task was attentionally demanding. Participants attempted to identify a central letter target as well as perform a speeded pointing movement to a peripheral letter target that was presented at various temporal intervals (Lags) after the central letter onset. Furthermore, participants were required to identify the peripheral letter in addition to pointing to it. In a control condition, participants ignored the central letter and were only required to point to and identify the peripheral letter. Results indicated that successful central letter identification led to deficits in peripheral letter identification when there was a short temporal lag between targets, but this deficit decreased as the temporal lag increased. Measures of pointing, however, were differentially affected by attending to the central letter. Initiation time for pointing (measured from pointing target onset) was the only measure to be adversely affected by short lags between the central and peripheral target. Movement time and pointing error were unaffected by the dual-task context and the temporal lag between targets, even for targets that were displaced at the onset of the pointing movement. These results suggested that concurrent identification of one target and pointing to another target interferes at the level of action planning or initiation, but not during the execution and online control of pointing toward the target. To rule out the possibility that the requirement to identify the peripheral letter was responsible for the delays in pointing initiation, Experiment 2 repeated the methods of Experiment 1, with the exceptions that the pointing target was now a disc and participants were not required to report on the appearance of this pointing target. The results were largely the same, such that there was a concurrent competition effect for pointing initiation time but no dual-task costs for pointing movement time or error. Chapter 1 41 [Chapter 3] Experiment 3 tested the possibility that the lack of dual-task costs in the execution and online control of pointing were caused by a rapid switching from central letter identification to pointing, such that participants were performing the tasks serially at all temporal lags. To reduce the possibility that participants could use this strategy, pointing targets in Experiment 3 were randomly presented either before or after the appearance of the central letter, so that participants could not anticipate which target would appear first. Regardless of this change, the results showed that the only measure of pointing that was impaired by the dual-task context and the temporal lag between tasks was pointing initiation time. Pointing movement time and error to stationary and displaced targets were again not negatively affected, even when the central target was presented while the hand was en route to the peripheral target. In Experiment 3, when the pointing target greatly preceded the letter target, participants had a tendency to postpone pointing initiation until a time close to the onset of the letter target, suggesting that they had a tendency to wait for the letter target before initiating pointing. Thus, Experiment 4 examined the consequences of encouraging participants to initiate pointing earlier than they would under the normal speeded pointing instructions of Experiment 3. An instructional deadline was implemented that required participants to initiate their pointing movements within 700 ms of pointing target onset. This would result in participants initiating pointing before they would naturally be ready to, and might possibly cut short the movement planning phase. The results showed that even though overall initiation time was reduced by the deadline instructions, the concurrency cost for dual-task initiation time was still present. Importantly, the deadline instructions now lead to a dual-task cost in pointing error, particularly for targets that were displaced. Chapter 1 42 [Chapter 4] Experiment 5 extended the findings of the preceding experiments by exploring the kinematics of pointing while participants were engaged in an attentionally demanding central visual perception task. The available setup to monitor the three-dimensional position of the hand prompted several small procedural changes to the perception and action tasks. For instance, the central task involved identification of the duration of a briefly flashed light, and target displacements were now triggered during eye movements, rather than upon pointing movement onset. Nonetheless, behavioural measures of pointing showed the same patterns as earlier observed, such that there was a concurrency cost in pointing initiation time, but no costs in pointing movement time or error. Kinematic measures of peak velocity, time to peak velocity, and time after peak velocity (deceleration time) also indicated no significant concurrency or dual-task set effects. [Chapter 5] The preceding experiments established that the information needed to plan a pointing action to a peripheral target and the information needed to identify a central target cannot be processed without interference, suggesting that the limiting factor is that both task components require the ventral stream. However, the information needed to execute pointing to a peripheral target that might be unexpectedly displaced and the information needed to identify a central target appears to be able to enter the dorsal and ventral systems and be processed without interference. The next series of experiments tested the prediction that if the action execution was also made to rely more upon the cognitive processes of the ventral stream, then interference in measures of both planning and execution of pointing should be observed. Experiment 6 was similar to Experiment 5, but the action task was changed so that the appearance of the peripheral stimulus indicated that participants should point Chapter 1 43 to a location that was spatially displaced from the peripheral stimuli. Thus, stimuli indicated the location of the pointing action in a symbolic way, which should recruit more ventral stream processing. For instance, a light stimulus appearing in one location indicated that participants were to point to a location displaced a few centimeters to the left, but a stimulus appearing in different location indicated that participants were to point a few centimeters to the right. However, results indicated that while pointing initiation time was elevated overall, pointing movement time and error were still unaffected by the dual-task context. This suggests that even with complex target selection instructions, once target selection has occurred, action execution is directed by the dorsal stream. [Chapter 6] Previous research has shown that indirect methods of indicating target location are less accurate than direct pointing, perhaps reflecting the contribution of less spatially precise ventral stream processes to target localization (Liu, Healey, & Enns, 2003). Experiment 7 compared the direct pointing method (hand points to the target location onscreen) of Experiments 1-5 with an indirect method (hand controls a computer mouse on a table that enables "pointing" with an onscreen cursor). Results indicated that the concurrency cost for pointing initiation time was greater for indirect than for direct pointing, and that indirect pointing error was greater in the dual-task context. Thus, when the action task was manipulated to rely more wholly on the ventral stream, costs from correct performance of the perceptual identification task were now observed in both the planning and execution of indirect pointing. Another way to reduce the involvement of the dorsal stream is to make pointing dependent upon visual memory (Milner & Goodale, 1995). Thus, Experiment 7 examined the consequences of dual-task pointing in a memory condition (target erased after being visible for a brief period), as opposed to the no-memory conditions (target Chapter 1 44 remains visible) that were used in the previous experiments. To make the memory demands even greater, pointing was compared in conditions without delay (pointing allowed immediately upon target onset) versus pointing after a delay of 2.5 seconds after target onset. Results indicated that in addition to the standard concurrency cost seen in pointing initiation time, there was now a general dual-task set cost in pointing error in both the immediate and delayed pointing conditions. Thus, results of Experiment 7 and 8 indicated that when pointing was made to rely on ventral stream processing, either because of indirect spatial mapping or because of memory demands, interference now occurred in the planning and execution of pointing when it was performed in conjunction with a visual perception task. Chapter 2 45 CHAPTER 2: EXPERIMENTS 1 AND 2 Experiment 1: Letter identification and pointing to a letter target The purpose of the first experiment was to examine the effects of perceptual letter identification on a visually guided pointing task. The primary question was whether any pointing measures would show the concurrency costs in errors and initiation time that are normally seen when two perceptual tasks are performed. A secondary purpose was to establish that the perceptual task was attentionally demanding. Pointing performance was compared under dual- and single-task conditions. In the dual-task condition, participants were required to monitor a centrally presented temporal stream of digits (10 items/second) for a letter target that was to be identified (ventral task). At the same time, they were asked to point to a second letter in the periphery that appeared unpredictably in space and time (dorsal task). To ensure that the online control of action could be distinguished from the mere execution of action to a remembered target (e.g., participants might store the peripheral letter location upon its appearance and respond to it only after identifying the central letter), pointing was compared for stationary targets (the target appeared in one location and remained there throughout the trial) versus displaced targets (the target moved unpredictably to a nearby location upon the onset of pointing). Additionally, to ensure that action planning could be distinguished from action execution, separate measures were taken for planning (initiation time) and execution (movement time and error). The task setup is a hybrid attentional blink paradigm and psychological refractory period paradigm (Arnell & Duncan, 2002; Jolicoeur, 1999; Pashler, 1994; Raymond, Shapiro, & Arnell, 1992). In both paradigms, interference is seen in performance to the second target if it is presented before processing of the first target is Chapter 2 46 complete. In the attentional blink paradigm, when two targets are presented in rapid succession, identification of the first target is associated with reduced accuracy for the second target for a period of approximately 500 ms. In the psychological refractory period paradigm, when two targets are presented in rapid succession, correct responding to the first target is associated with increased reaction or initiation time to the second target. In the current experiments, the perceptual task is not speeded and performance is measured in accuracy (typical of attentional blink tasks), but the pointing task is speeded and performance is measured in response initiation time (typical of psychological refractory period tasks) and accuracy (typical of attentional blink tasks). To establish that the central letter identification was attentionally demanding, participants were also required to identify the peripheral letter that was the target of pointing. If ventral and dorsal tasks tap independent attentional mechanisms, then the first letter identification (ventral) should interfere with the second letter identification (ventral) at short temporal lags, but should not interfere with pointing execution (dorsal). Thus, dual-task pointing performance will be compared with a single-task control where the central letter target is ignored and pointing and identification are only required of the peripheral letter target. Note that in Experiment 1, the single-task condition requires two responses (identification and pointing) to the peripheral target, whereas the dual-task condition requires two responses to the peripheral target as well as identification of the central target. The terms "single" and "dual" task are used for convenience in across-experiment comparisons of response requirements to the peripheral target alone (single) or to both the central and peripheral target (dual). In Experiment 2 and onward, the peripheral letter identification requirement is dropped from both single- and dual-task conditions. Chapter 2 47 If object identification and visually guided pointing involve shared attentional mechanisms, then letter identification should interfere with most measures of pointing at both the planning and execution level of pointing, provided the tasks are performed in close temporal proximity. This is the prediction derived from previous research on dual visual tasks (Shapiro, 2001). The integrated competition hypotheses (Duncan, 1996) and the visual attention model (Deubel et al., 1998; Schneider, 1995; Schneider & Deubel, 2002) also make the same prediction as long as the two tasks refer to separate objects, since both theories are based on a unitary and object-centered view of attention. Note that the visual attention model goes further to predict that highly practiced or preplanned movements may be immune from interference because they can be completed without feedback. However, movements that respond to dynamic changes in a scene should be most prone to interference, such that if the unitary, object-based mechanism of attention is focused on letter identification, it cannot simultaneously focus on the peripheral letter to complete the action to the new target location. Alternatively, if the execution of visually guided pointing is distinct from letter identification, as implied by the Perception-Action theory (Milner & Goodale, 1995), then concurrent letter identification should not interfere with certain components of pointing to a second object if there are separate processing mechanisms available to each of the ventral and dorsal streams. Given the Perception-Action distinction between the conscious planning of an action (involving the ventral stream) and its unconscious online control (involving the dorsal stream), pointing measures that index planning (e.g., initiation time) should be interfered with by a concurrent identification task, whereas pointing measures that index more purely dorsal stream functions (e.g., movement time, online modifications) should not show interference. Chapter 2 48 Methods Participants Participants were 14 right-handed undergraduate students (8 females, mean age 20.8 years) with normal or corrected-to-normal vision who volunteered in exchange for course credit at the University of British Columbia. Procedures for all studies described in this thesis were approved by the Behavioural Research Ethics Board of the University of British Columbia. Apparatus Participants were seated at a viewing distance of 57 cm from a 17" Elo Touchsystems monitor that used iTouch surface acoustic wave technology to register onscreen touches. The iTouch controller has a touchpoint density of 4096 x 4096, or over 15500 touchpoints/ cm 2. The standard deviation of error for positional accuracy is < 2 mm. The room was dimly lit to prevent lighting glare from appearing on the touchscreen monitor. Displays and data collection were controlled on a Power Macintosh computer (operating system 9.2.2) using MathWorks Matlab 5.2 software with Psychophysics Toolbox extensions (Brainard, 1997; Pelli, 1997). To create a more comfortable position for repeated pointing movements, the monitor was mounted on a stand that presented the screen tilted upward (28 degrees from the horizontal) at approximately waist-height; thus, participants were looking downward at the touchscreen. To provide a constant size of touchpoints, pointing responses were made with an Elo stylus that was held like a pen in the right hand. For conditions where letter identification was required, participants placed a keyboard on their lap and inputted letters with the left hand. Chapter 2 49 Stimuli A sample trial sequence is shown in Figure 3. A l l response stimuli were black items against a light gray background. The pointing home position was designated with an outline circle subtending 0.6° of visual angle. Visual fixation was designated with a square (0.4° across) centered 2.4° above the home position. A rapid serial visual presentation (RSVP) stream of items consisted of digit distractors and a letter target presented sequentially at visual fixation. Each item subtended approximately 0.6° (height) by 0.5° (width) and was presented in Arial font. Distractors were selected randomly with replacement from the digits 0 -9 , with the constraint that each selected distractor had to be different from either of the two preceding items. The central and peripheral targets were different letters selected from K, N , V, X, Y, or Z (selected on the basis of visual similarity). Each item in the stream was presented for 30 ms followed by a blank interval of 70 ms, yielding a presentation rate of 10 items per second. Five to ten distractors were presented before the central target letter and 14 distractors were presented after, with the peripheral target letter appearing at a temporal lag of 100, 300, or 700 ms after the onset of the central target. The peripheral letter target was presented for 100 ms before it was masked with a digit (0 - 9) that remained onscreen until the end of the trial. Peripheral targets could appear at positions 11,12, or 13 cm (or degrees of visual angle) to the right of visual fixation. These positions will be referred to as the near (11 cm), middle (12 cm), and far (13 cm) positions for Experiment 1. Note that throughout the experiments that comprise this thesis, 1 cm = 1 degree of visual angle with respect to the dimensions of all visual stimuli. Peripheral targets were equally divided between two conditions: (a) stationary targets, where the target that appeared at one of the three locations and remained there for the remainder of the trial, and (b) displaced targets, where the target initially Chapter 2 appeared at the middle position, but upon initiation of pointing, the target either jumped backward to the near position or forward to the far position. 50 Fixation K Time RSVP stream of items Letter target Pointing target 100, 300, 700 ms lag Pointing target masked Pointing target displaced upon movement initiation Point to final target location Figure 3. Sample trial sequence for Experiment 1 Schematic illustration of a typical trial sequence. A letter target appears within an rapid serial visual presentation (RSVP) stream of items presented at fixation, followed by peripheral pointing target that is masked after 100 ms. A displaced target trial is illustrated, in which the pointing target moves to a new location upon pointing initiation. On stationary trials, the pointing target remains in place upon pointing initiation. Report of the letter target is made following pointing completion. Procedure Both written and verbal instructions were given to participants. Participants were seated so that the midline of the body was aligned with visual fixation and the starting position for the hand. To initiate each trial, participants were instructed to gaze at fixation and touch the home position, which subsequently turned solid black to Chapter 2 51 indicate that the trial sequence would begin in 500 ms with the presentation of the central item stream. For both the dual and single-task conditions, participants were required to (a) maintain fixation at the location of the item stream and (b) keep their pointing hand at the home position until the onset of the peripheral letter target, upon which they were to look at and point to the peripheral target. If participants lifted their hand from the home position before the onset of the peripheral pointing target, a red " X " appeared above fixation to indicate their error. In the dual-task condition, participants were instructed to perform the following tasks: (a) to identify the central letter in the stream of items as accurately as possible, and (b) point to and identify the peripheral letter as quickly and as accurately as possible upon its appearance. Eye movements to the peripheral target were allowed. Participants were informed that sometimes the peripheral target would change location upon pointing initiation, but the task was always to point to the final target location. The exact percentage of jumps was not specified. Identifications of the central and peripheral letter targets were made with the left hand on the keyboard following completion of the pointing movement. Order of letter report was not constrained. Feedback for letter identification accuracy was presented with either "+" (correct) or "-" (incorrect) symbols that appeared for 800 ms to the left and right of fixation. The symbol on the left indicated accuracy for the central letter while the symbol on the right indicated accuracy for the peripheral letter. In the single-task condition, the displays were identical to the dual-task condition. However, participants were instructed to ignore the central stream of items while maintaining fixation, and had only to point to and identify the peripheral letter target. A l l other instructions and procedures were identical to the dual-task condition. Chapter 2 52 Dual- and single-task conditions were tested in separate blocks presented in counterbalanced order across participants. Within each block, trial conditions (temporal lag, target location, target displacement) were randomized. For each of the dual- or single-task conditions, participants (a) received block-appropriate instructions, (b) performed 20 practice trials or enough trials until participants were fluent with the task, (c) calibrated the touchscreen, and (d) performed 216 experimental trials for that block. Data Analyses In addition to recording central and peripheral letter identification error, several dependent measures were collected for pointing performance. Pointing measures collected were (a) initiation time (duration between pointing target onset and hand liftoff from home position), (b) movement time (duration between hand liftoff from home position and landing in the periphery), and (c) error (horizontal error in mm relative to final position of the pointing target). Measures of pointing error are presented in absolute constant error, which is a measure of the magnitude of the deviation from the target position without information regarding direction of deviation. Pointing is also presented in variable error, which is a measure of the consistency of pointing movements; this measure is calculated as the average standard deviation of errors from the final target location. With regards to the first measure of pointing error, results were presented in absolute constant error rather than constant error (magnitude of the deviation from the target position with information regarding direction of deviation) in order to allow collapsing of the magnitude of error on forward and backward jumps. Because participants had a tendency to overshoot backward displaced targets and undershoot forward displaced targets, this meant that for any analyses that collapsed across Chapter 2 53 direction of target displacement, displaced target error was close to zero. This would give a misleading conclusion that participants were quite accurate in pointing to displaced targets. Thus, in order to simplify interpretation of results, absolute constant error was used for analyses. Each measure was examined with a repeated-measures analysis of variance (ANOVA) that included within-participant factors of Task Number (dual, single), Lag (100, 300, 700), and Target Type (stationary, displaced). Data were collapsed over certain factors for selected dependent measures when they were not relevant to the analysis. In particular, only Lag was examined for central letter identification, and only Task Number and Lag were examined for peripheral letter identification and pointing initiation time. Trials were excluded from analysis if participants made anticipatory (initiation time < 100 ms) or late (initiation time > 2000 ms) pointing movements, or if movements were very short (movement time < 100 ms) or very long (movement time > 1000 ms). Short movement times indicated that participants lifted the stylus and touched down by fixation; long initiation and movement times indicated that participants were not making speeded responses. Trials were also excluded on the rare occasion that there was an error in writing to the file. In total, 2.2% of trials were excluded from initial analyses. To ensure that full attention had been paid to the central letter, peripheral letter accuracy and pointing movements were only analyzed when they followed correct letter identification. In addition, when the peripheral target was displaced, the final target location was always the nearest or the farthest position, so stationary and displaced targets were compared for these pointing locations. Chapter 2 54 Results The primary findings were that (a) central letter error was quite low and was unaffected by the temporal proximity to the pointing target, (b) peripheral letter error was increased by temporal proximity to the central letter target, (c) pointing initiation time to the peripheral letter target was impaired by temporal proximity to the central letter target, and (d) pointing movement time and error were unimpaired by temporal proximity to the central letter target. Letter Identification Performance Mean errors in central and peripheral letter identification are shown in Figure 4A. Participants successfully attended to the central letter identification task, making errors on only 3.5 % of trials. There was no effect of Lag on central identification performance, p > .71. Peripheral identification performance was examined for trials where participants successfully identified the central letter. Identification of the peripheral letter, which was also the target of the pointing action, was subject to a typical attentional blink pattern of increased error at short lags, as indicated by a Task Number x Lag interaction, F(2, 26) = 23.31, p_ < .0001. Errors decreased as lag increased for both dual-and single-task conditions; however, at short lags, peripheral letter error was much increased in the dual-task condition relative to the single-task condition. By Lag 700, error in the two conditions was equivalent, indicating that the central letter identification cost had disappeared. Chapter 2 Figure 4. Experiment 1 results: Letter identification and pointing to letter target A: Letter Identification % 5 c L U c .2 .4 I 2 _i o c .2 .1 t o a 0 £ o -#- Dual-Task Peripheral Letter - A - Single-Task Peripheral Letter - B - Dual-Task Central Letter 100 300 Lag (ms) 700 Pointing B: Initiation Time C: Movement Time 525 S 475 4 425 ± 375 H o a. 325 275 Dual-Task Single-Task 375 350 325 E 300-^ o S 275-^ 250 4 100 300 Lag (ms) 700 225 Dual-Task Displaced -O- Dual-Task Stationary -A- Single-Task Displaced - A - Single-Task Stationary 100 300 Lag (ms) 700 D: Absolute Constant Error L U o> § 4 c o Q. - • - Dual-Task Displaced -O- Dual-Task Stationary -A- Single-Task Displaced -A - Single-Task Stationary 100 300 700 Lag (ms) Chapter 2 56 Pointing performance Initiation time. Mean pointing initiation time relative to the onset of the pointing target is shown in Figure 4B. Error bars on this and all subsequent graphs represent one standard error of the mean. Pointing initiation time exhibited a pattern similar to that observed in peripheral letter identification, as indicated by a Task Number x Lag interaction, F(2, 26) = 24.64, p < .0001. Specifically, at short lags, dual-task initiation time was 95 ms longer than single-task initiation time, but by Lag 700, pointing initiation time in the two conditions no longer differed significantly. Additionally, initiation time decreased as lag increased in both dual- and single-task conditions, F(2, 26) = 81.30, p < .0001, indicating that there was a general effect of readiness on initiation time. Specifically, as time progresses from the onset of the trial, participants are more ready to point to the peripheral target when it appears, which manifests itself as a decrease in initiation time. Movement time. Mean pointing movement time is shown in Figure 4C. There was a trend for Task Number and Lag to interact for movement time, F(2,16) = 3.11, p < .07. However, rather than reflecting a dual-task cost in movement time, the trend indicated that as Lag increased from 100 to 700, dual-task movement time became faster (from 293 to 283 ms), whereas single-task movement time remained about the same (289 to 292 ms). Additionally, consistent with the need for pointing to incorporate the new target location during movement to a displaced target (Prablanc & Martin, 1992), movement time was 14 ms longer when the pointing target was displaced rather than stationary, F(l/13) = 16.22, p < .002. Error. Error measured in absolute constant error relative to the final target location is presented in Figure 4D. Preliminary analysis of constant error indicated that participants had a tendency to overshoot near targets and undershoot far targets (final Chapter 2 57 positions), particularly when targets were displaced upon pointing initiation. Reflecting this pattern, absolute constant error for displaced targets (4.8 mm) was greater than absolute constant error for stationary targets (2.2 mm), F(l/13) = 11.21, p < .006. In the displaced target condition, the level of error indicated that participants were partially correcting to the final target location. If participants were not correcting their movements, then absolute constant error should be at least at 10 mm (the distance of a target displacement), rather than 4.8 mm. Importantly, none of the factors of Task Type, F(l, 13) = 2.02, Lag, F(2, 26) = .25, nor the Task Type x Lag interaction, F(2, 26) = 1.15, had any effect on absolute constant error for either stationary or displaced targets. While error for dual- and single-task pointing showed no differences, it may have been the case that the dual-task pointing was less consistent than single-task pointing. Thus, variable error was examined in addition to absolute constant error. However, analyses showed that dual-task pointing variable error (3.9 mm) was actually more consistent that single-task pointing (4.5 mm) overall, F(l,13) = 8.19, p < .02. Additional effects showed that variability was greater for displaced targets (4.5 mm) than for stationary targets (3.9 mm), F(l, 13) = 21.87, p < .0005, and greater with increasing lag (increasing from 4.1 to 4.4 mm), F(2, 26) = 4.27, p < .03. Discussion Successful central letter identification had a detrimental effect on identification of the peripheral letter, which was also the target of the pointing. Specifically, peripheral letter identification was subject to a concurrency cost, such that at short lags, peripheral letter error was much elevated in the dual-task condition relative to the single-task condition. However, by Lag 700, errors in both conditions were equivalently low, Chapter 2 58 indicating that there was no longer any cost associated with correctly identifying the central letter. This suggests that as Lag increased in the dual-task condition, attention was freed up from the central letter identification task and made available for the peripheral letter identification task. The same concurrency cost from central letter identification was observed in the measure of movement planning (pointing initiation time). At short lags, dual-task initiation time was 95 ms longer than single-task initiation time, but by the longest lag tested, the two tasks no longer differed significantly. This suggests that the planning or initiation of a visually guided pointing action taps the same attentional mechanisms as those required for central letter identification, such that one cannot concurrently program a movement while processing a letter for identification. This is consistent with the prediction from the Perception-Action theory that both action planning and perceptual identification involve ventral stream processing. A different story emerged when considering measures of pointing execution and online control. In contrast to the interference observed in pointing initiation, movement time and error showed no measurable interference effects. Firstly, there was no evidence that movement time was prolonged in the dual-task relative to the single-task condition at any temporal lag. In other words, once pointing had been initiated, there was no task interference from the central or peripheral letter identification on movement time, even when targets were displaced. The only reliable effect observed was that movement time was longer for displaced targets, consistent with the need for pointing to incorporate the new target location during movement to a displaced target (Prablanc & Martin, 1992). Secondly, there was no evidence that pointing error was impaired in the dual-task context for either stationary or displaced targets, suggesting that once pointing was Chapter 2 59 underway, successful central letter identification did not impair it in any way, even when the target changed location. Interestingly, dual-task pointing was actually more consistent (as measured by variable error) than single-task pointing, suggesting a dual-task set enhancement for pointing. Given the prevailing evidence that dual-task performance is generally impaired relative to single-task performance, this dual-task enhancement is surprising. It suggests that on some measures, dual-task demands may actually result in a performance advantage. This issue will arise in subsequent experiments and be further addressed at that point. Overall, Experiment 1 demonstrated that successful letter identification interferes with the identification of a second letter in a peripheral visual location, and with initiation of a pointing action to this peripheral letter. However, no interference was evident in either the speed or accuracy of the ensuing pointing action to the peripheral letter. The finding that two letters cannot be identified without cost when they appear in close temporal proximity replicates many previous studies of the attentional blink (Shapiro, 2001; Shapiro, Arnell, & Raymond, 1997). The two new results here are that letter identification (1) interferes with the initiation of action, but (2) does not interfere with the execution and online control of action. The interpretation of the first result is that the subtask of planning a pointing action shares neurocognitive resources with the task of letter identification. We do not believe that it is merely the initiation of pointing that has been delayed by central letter identification. If there was simply a limit on action initiation during letter identification, and action planning was unaffected, then the cost in pointing initiation times should be a direct function of the lag that intervenes between the onset of the central and peripheral letters. For example, if it takes 195 ms of letter identification before pointing can be initiated (the Lag of 100 ms in Figure 2B plus the dual-task initiation time cost of Chapter 2 60 95 ms), then there should no longer be any measurable cost at Lag 300, as 195 ms has already elapsed. Since a cost still exists, it suggests that in addition to any limits on pointing initiation, there is also interference in planning the action simultaneously with letter identification. The interpretation of the second result of Experiment 1, that execution and online control of action can be accomplished without interference from letter identification, is based on the finding that Task Type and Lag had no influence on pointing time or error, even when the pointing target was displaced upon movement initiation. This means that the system guiding the hand registered the new target location and modified the action accordingly without interference from the letter identification task. Given that pointing execution was initiated well after presentation of the letter target (e.g., pointing initiation time is approximately 475 ms at Lag 100, indicating that pointing is initiated at about 575 ms after letter onset), one might argue that there was no interference from letter identification because letter processing was complete. However, it has been shown that attentional interference that occurs during the period of pointing initiation can continue to manifest itself during movement execution, particularly in movement time (Tipper et al., 1992). While this was not the case in the present experiment, the question of serial performance of letter identification and pointing execution will be directly addressed in Experiment 3. Before turning to the question of serial task performance, the next experiment addresses the possibility that the present delay in pointing initiation was influenced by the requirement to identify the peripheral letter. To see if peripheral letter identification played any role, Experiment 1 was repeated in all respects, with the exception that the pointing target was now a disc and that participants were not required to report on its Chapter 2 61 appearance. The results again showed dual-task costs for pointing initiation time but none for pointing movement time or error. Experiment 2: Letter identification and pointing to a disc The main purpose of the second experiment was to examine the effects of central letter identification upon pointing to a peripheral target in the absence of requiring identification of the second target. The reason for doing this was to rule out the possibility that the peripheral identification requirement was contributing to the concurrency effect in dual-task pointing initiation. Experiment 2 repeated the methods of Experiment 1 with the exception that the peripheral pointing target was now a solid disc instead of a letter. In both single- and dual-task conditions, participants were only required to look at and point to the target, and no longer had to report its identity. Any differences between Experiments 1 and 2 could therefore be attributed to the requirement to consciously attend to the identity of the pointing target. Methods Participants Participants were 14 right-handed students (11 females, mean age 19.9 years) with normal or corrected-to-normal vision who voluntarily participated in exchange course credit at the University of British Columbia. Stimuli and Procedure A sample trial sequence is shown in Figure 5. The stimuli and procedures were the same as in Experiment 1 with the exception of modifications to the peripheral pointing target. The peripheral pointing target was changed from a letter to a solid Chapter 2 62 black circle subtending 0.5°. After its onset, the peripheral target remained onscreen for the remainder of the trial. Additionally, participants no longer had to perform a letter identification with the peripheral pointing target; in both single- and dual-task conditions, instructions for the peripheral target were to simply to look at and point to it as quickly and as accurately as possible upon its onset. Time Fixation Light target flashed (15 or 60 ms) Letter target — Pointing target — 100, 300, 700 ms lag 3 ^ • Pointing target displaced upon movement initiation 9 Point to final target location Figure 5. Sample trial sequence for Experiment 2 Schematic illustration of a typical trial sequence containing a disc as a pointing target. A displaced target trial is illustrated, in which the peripheral target moves to a new location upon pointing initiation. Data Analyses The same exclusion criteria used in Experiment 1 were applied here, resulting in 3.2% of trials being excluded from initial analysis. A N O V A was used to examine dependent measures using the same factors and data inclusion criteria as described in Experiment 1, In further analyses, the data from Experiment 1 were combined with Chapter 2 63 Experiment 2 data and an A N O V A was run with Experiment (1, 2), as a between-participants factor and the same within-participants as described in Experiment 1. Results The main findings were that (a) as observed in Experiment 1, only pointing initiation time was impaired by temporal lag, whereas central letter error, pointing movement time, and pointing error measures remained unaffected by temporal lag, (b) pointing movement time was slower overall, and (c) pointing errors were decreased overall. Letter Identification Performance Mean errors in central letter identification performance are shown in Figure 6A. Letter identification was again quite successful, with mean errors at 2.9%. There was no effect of Lag on identification, p_ > .50. Additionally, performance did not differ from that in Experiment 1, p > .36. Pointing Performance Initiation time. Mean pointing initiation time is shown in Figure 6B. As observed in Experiment 1, there was a concurrency cost exhibited in pointing initiation time, as shown by the Task Number x Lag interaction, F(2, 26) = 31.03, p < .0001. Initiation time decreased overall as Lag increased in both dual- and single-task conditions. But at short lags, dual-task initiation time was 123 ms longer than single-task initiation time. By Lag 700, this difference was greatly attenuated. Comparison of initiation time performance to that in Experiment 1 revealed no differences, all p's > .36. Chapter 2 64 Figure 6. Experiment 2: Letter identification and pointing to disc A: Letter Identification 2 c 111 iS 4-1 24 2 .1 t o a. o £ o Dual-Task Central Letter 100 Pointing 300 Lag (ms) 700 B: Initiation Time 525 S 475 o E 425 - 375 at ° 325 Q. 275 Dual-Task Single-Task 100 300 Lag (ms) 700 450-, 425 j | 400 c o £ > o c 375 350 325 300 C: Movement Time -#- Dual-Task Displaced -O- Dual-Task Stationary -A- Single-Task Displaced - A - Single-Task Stationary 100 300 Lag (ms) 700 D: Absolute Constant Error -#- Dual-Task Displaced -O- Dual-Task Stationary -A- Single-Task Displaced - A - Single-Task Stationary 100 300 Lag (ms) 700 Chapter 2 65 Movement time. Mean pointing movement time is shown in Figure 6C. Movement times were longer for displaced targets, F(l, 13) = 19.64, p < .0008, as seen in Experiment 1. Comparison of movement time performance to that in Experiment 1 revealed a trend for movement time to be slower overall in Experiment 2 (373 ms) than in Experiment 1 (289 ms), F(l, 26) = 4.15, p < .06. Error. Mean absolute constant error is presented in Figure 6D. There was a trend for absolute constant error for displaced targets (2.5 mm) to be worse than performance for stationary targets (1.5 mm), F(l, 13) = 3.73, p < .08, but this difference was smaller than the significant 2.4 mm difference observed in Experiment 1. Thus, participants were showing a higher level of correction to final target location in this experiment. Additional interactions did not indicate any systematic dual-task set or concurrency costs. Dual-task error was at the same level as single-task error at all Lags except for Lag 700, where it was worse by 0.6 mm, Task Number x Lag, F(2, 26) = 5.27, p < .02. Comparison of absolute constant error to that in Experiment 1 revealed that errors were generally lower in Experiment 2, F(l, 26) = 4.25, p < .05. Examination of variable error also did not show any consistent dual-task costs. Pointing was less consistent for displaced targets (3.8 mm) than stationary targets (3.2 mm), F(l, 13) = 11.11, p < .001. An additional Task Number x Lag interaction, F(2, 26) = 4.62, p < .02, did not reflect any systematic advantage for either single- or dual-task variability. Comparison of variable error to that in Experiment 1 revealed that while there was a dual-task variable error advantage in Experiment 1, this was not the case for the present experiment, F(l, 26) = 4.53, p < .05. Chapter 2 66 Discussion Experiment 2 generally mirrored the patterns of Experiment 1. Even when the peripheral letter identification requirement was removed, the concurrency effect in pointing initiation still remained in the dual-task context. Additionally, there was still no dual-task effect on pointing movement time or error. Pointing errors were lower overall compared to Experiment 1, and the cost of correcting to displaced targets was diminished. With regards to the effect of reduced errors, it should be noted that movement times were longer overall in Experiment 2. Thus, there may simply have been a speed-accuracy tradeoff (Schmidt & Lee, 1999), such that longer movement times resulted in lower errors for their movements. Importantly, there were still no dual- vs. single-task costs in movement time or error. Overall, the findings suggest that the requirement to identify the pointing target influenced the overall speed and accuracy of pointing, but it did not contribute to a dual-task set cost or a time-sensitive concurrent competition cost on movement time and error. Thus, both Experiments 1 and 2 indicated that letter identification interferes with the initiation of action, but does not interfere with the execution and online control of action. However, an alternative interpretation of the lack of interference in movement time and accuracy seen in Experiments 1 and 2 is that the critical processes governing the online control of pointing occurred only after letter identification was complete. In other words, participants may have identified the central letter first and then rapidly switched to the pointing task. The next experiment examined this possibility. Chapter 3 67 CHAPTER 3: EXPERIMENTS 3 AND 4 Experiment 3: Negative and positive temporal lags The lack of impairment in the pointing execution and online control observed in Experiments 1 and 2 may have been the result of participants first performing the perceptual task of central letter identification and then rapidly switching to the visually guided action task. If the participants were executing pointing only once the letter task had been completed, then the tasks would be performed serially rather than in parallel. Indeed, when Deubel and colleagues (Deubel et al., 1998; Schiegg et al., 2003) observed a lack of interference of pointing or grasping on the task of alphanumeric discrimination, they attributed this finding to attentional selection having been completed for the action target and therefore being available for the discrimination target. Thus, to reduce the possibility that participants could use the strategy of postponing pointing until after letter identification, and to ensure that there were trials on which central letter identification was performed simultaneously with the online control of action, pointing targets were also presented at negative lags (the pointing target appeared prior to the central letter) that were randomly intermixed with positive lags selected from those tested in Experiments 1 and 2. Thus, participants would have to be prepared to perform either task first, instead of being prepared to perform letter identification first. This manipulation was also designed to create a range of pointing responses that occurred before, during, and after the letter presentation. The change in experimental design necessitated a few changes in methodology from the preceding experiments, including the requirement for participants not to move their eyes in response to the onset of the pointing target. Optimal accuracy of aiming movements is attained when gaze reaches the target before the hand (e.g., Mather & Chapter 3 68 Fisk, 1985; Neggers & Bekkering, 1999, 2000; Prablanc, Echallier, Komilis, & Jeannerod, 1979; Prablanc, Pelisson, & Goodale, 1986). Additionally, it has been shown that when participants can foveate the target rather than maintaining central fixation, online adjustments to displaced targets are more accurate and are initiated earlier in movement (Diedrichsen et al., 2004). Thus, it was expected that requiring the maintenance of fixation during pointing to a peripheral target would reduce overall pointing accuracy relative to performance in Experiment 2, where eye movements to the target were allowed. Nonetheless, this should not affect the presence of any accuracy differences between single- and dual-task contexts. If letter identification and pointing execution cannot be done concurrently, then dual-task interference in execution should now be observed in some of the negative lag conditions, when the hand is already in transit at the time that the central letter appears. In contrast, if the online control of pointing is independent from letter identification, then pointing movement time and accuracy should be unaffected by whether movements occur before, during, or after the appearance of the central letter. Methods Participants Participants were 14 right-handed students (12 females, mean age 21.6 years) with normal or corrected-to-normal vision who voluntarily participated in exchange course credit at the University of British Columbia. Chapter 3 69 Stimuli and Procedure The design was identical to Experiment 2 with the exception of modifications to the range of temporal lags presented and the requirement to maintain fixation throughout the trial. Dual- and single-task blocks each consisted of 240 trials. Sample trial sequences are shown in Figure 7. Whereas Experiments 1 and 2 only presented the pointing target at positive lags relative to the onset of the letter target, in the present experiment, negative lags were randomly intermixed with the positive lags. The intervals between onsets of the central letter and peripheral pointing target were -700, -100,100, and 700 ms. For negative lags, the pointing target would onset 700 or 100 ms before the letter target. For positive lags, the pointing target would onset 100 or 700 ms after the letter target. As in previous experiments, between five and ten distractors preceded the onset of the first target, which was either the peripheral disc (negative lags) or the central letter in the item stream (positive lags). After the onset of the first target, fourteen items appeared in the central item stream. At the appropriate temporal lag during these fourteen items, the second target appeared, which was either the central letter in the item stream (negative lags) or the peripheral disc (positive lags). Since the letter target appeared unpredictably either before or after the pointing target, participants were instructed to maintain their eyes at fixation during trial presentation. Otherwise, if participants made an eye movement to the pointing target before the letter target appeared, they would miss the subsequent letter target. Chapter 3 70 A: Negative Lag B: Positive Lag Time Fixation RSVP stream of items Pointing target — i Letter target — 1 -700, -100 ms lag Point to final target location Fixation RSVP stream of items Letter target — Pointing target —' 100, 700 ms lag Point to final target location Figure 7. Sample trial sequences for Experiment 3 Schematic illustration of a typical trial sequence for (a) Negative Lags, in which the pointing target appears at a variable interval before the letter target appears, and (b) Positive Lags, in which the pointing target appears at a variable interval after the letter target appears. A stationary target trial is illustrated, in which the pointing target remains in place upon pointing initiation. Data Analyses The same exclusion criteria used in Experiment 1 were applied here, resulting in 3.1% of trials being excluded from initial analysis. Repeated-measures A N O V A was used to examine dependent measures using the same factors and data inclusion criteria as described in Experiment 1, with the exception that for the factor of Lag, the levels were now -700, -100,100, and 700 ms. Results The main findings in Experiment 3 were that (a) central letter accuracy was unaffected by whether it was preceded or followed by the pointing target, (b) pointing initiation time was longest when the pointing target preceded the letter target, and initiation time decreased as the pointing target onset was delayed relative to the letter Chapter 3 71 onset, (c) pointing movement time and error were not negatively affected by the dual-task context, even when the letter target was presented while the hand was en route to the pointing target, and (d) pointing movement time was faster in the dual-task than the single-task condition at all Lags except for -700 ms. Letter Identification Performance Errors for central letter identification are presented in Figure 8A. As in previous experiments, errors were again very low (4.6%) and did not vary by Lag, F(3, 39) < 1. Pointing Performance Initiation time. Mean pointing initiation time, presented in Figure 8B, showed a lag-dependent cost similar to that observed in Experiments 1 and 2. Dual-task initiation time was elevated by over 330 ms from single-task initiation time at Lag -700, and this interference effect gradually diminished until it was absent at Lag 700, Task Number x Lag, F(3, 39) = 29.03, p < .0001. Movement time. Mean pointing movement time is shown in Figure 8C. Dual-task movement time was faster than single-task movement time at every lag except for -700 ms, Task Number x Lag, F(3, 39) = 7.25, p < .0007. Thus, facilitation for dual-task movement time was observed relative to single-task performance. Also, as observed in Experiments 1 and 2, movement time was longer (by 11 ms) for displaced than for stationary targets, F(l, 13) = 5.38, p < .04. Chapter 3 72 Figure 8. Experiment 3 results: Negative & positive temporal lags A: Letter Identification 2 5 o s Im. C LU C •2 .4 » .2 o c •2 1 t ' o Q. O £ 0 -700 Dual-Task Central Letter -100 100 Lag (ms) 700 Pointing B: Initiation Time 7 7 5 -t/l _E 6 7 5 -01 E i-ion 5 7 5 -TO 4 7 5 -o> c s o 3 7 5 -a. 275 : -700 Dual-Task Single-Task -100 100 Lag (ms) 700 E E > o B ) 375 350 H 325 H 300 275 250 i 225 C: Movement Time - • - Dual-Task Displaced -O- Dual-Task Stationary -A- Single-Task Displaced -A - Single-Task Stationary -700 -100 100 Lag (ms) 700 D: Absolute Constant Error 1 0 _ 8 E 8 6 LU Ol i * o a. -#• Dual-Task Displaced -O- Dual-Task Stationary -A- Single-Task Displaced - A - Single-Task Stationary -700 -100 100 Lag (ms) 7 0 0 Chapter 3 73 Errors. Pointing error measured in absolute constant error is presented in Figure 8D. Preliminary analyses on constant error again indicated that participants had a tendency to overshoot near targets and undershoot far targets (final positions). Analysis of absolute constant error indicated that participants were more accurate at pointing to stationary targets (3.2 mm) than to displaced targets (6.2 mm), F(l, 13) = 17.63, p < .002. In the displaced target condition, the level of error indicated that participants were partially correcting to the final target location. If participants were not correcting their movements, then absolute constant error should be at least at 10 mm (the distance of a target displacement), rather than 6.2 mm. Importantly, neither Task Type, Lag, nor their interaction had any effect on absolute constant error (all pis > .44). Examination of variable error supported this pattern, such that pointing was more consistent for stationary targets (4.9 mm) than for displaced targets (5.5 mm), F(l, 13) = 16.12, p < .002, but no other factors were significant (all p's > .24). Actual Pointing Initiation Relative to Letter Onset. The preceding analyses were premised on the assumption that pointing in the -700 Lag condition actually began prior to the onset of the central letter. However, actual initiation time varied across trials and participants, so those analyses may have masked interference effects that only occurred if the central target appeared after the pointing action had begun. To examine this possibility, trials were sorted into four bins according to when the pointing action was actually initiated relative to the onset of the central letter. Bin 1 (-600 ms to -200 ms) only included movements that were initiated well before letter onset, Bin 2 (-200 ms to +200 ms) included movements beginning near letter onset, Bin 3 (+200 ms to +600 ms) included movements beginning shortly after letter onset, and Bin 4 (+600 ms to +2000 ms) included movements that began long after letter onset. Thus, if letter processing interferes with movements that occur concurrently, then a concurrency cost should be Chapter 3 74 observed in Bin 2, when movements are occurring during letter presentation and processing. If being prepared to identify a letter interferes with movements in general (dual-task set effect), then a dual-task cost should be observed across all bins. Both pointing movement time and errors were analyzed with a repeated-measures A N O V A that examined the factors of Task Number, Target Movement, and Bin. Because of missing data, four subjects were excluded from the A N O V A . The results of these analyses were consistent with those already presented. There were no significant effects on movement time or error showing interference from letter identification as a function of Task Type or Bin (all p's > .18). As in the preceding analyses, there was a significant trend for dual-task movement time to be faster than single-task movement time at later bins, F(3, 27) = 4.37, p < .02, and pointing accuracy measured in absolute constant error was worse for displaced than for stationary targets, F(L 9) = 20.11, p<.002. Discussion Experiment 3 again showed that pointing initiation time was the only measure that was impaired by the dual-task context and the temporal lag between tasks. However, this experiment also demonstrated that pointing initiation time was even more delayed for negative than for positive lags, such that participants were reluctant to begin pointing to the peripheral target prior to the onset of the letter they were trying to identify. Nonetheless, when examining only the trials where participants were able to initiate a pointing movement at least 200 ms prior to the central letter onset, there was still no evidence that letter identification interfered with the online control of pointing. This suggests that only the planning involved in pointing to the initial target location shares neurocognitive resources with letter identification, and is therefore interfered Chapter 3 75 with. However, once the pointing action has been planned, it can be executed and even modified by changes in target location without any interference from simultaneous letter identification. It is also important to note that the task of identifying the letter is not simply being delayed for future completion during the online control of action; the digits following the letter circumvented this strategy by acting as backward masks (Brehaut, Enns, & Di Lollo, 1999; Giesbrecht & Di Lollo, 1998). Thus, the online control of pointing to stationary and displaced targets was not influenced by the concurrent identification of a letter in a temporal stream, even when the letter appeared while the hand was in transit and responding to an unpredictable change in the pointing target location. An unexpected finding in the present experiment was a clear facilitation of dual-task movement time, such that dual-task movement time was faster than single-task movement time except when the pointing target greatly preceded the letter target at Lag -700. This enhancement was only hinted at for positive lags in Experiment 2. This suggests that having to perform a dual-task confers a movement time advantage when participants have to do more tasks, as opposed to concentrating on pointing alone in the single-task control. It should not be considered a speed-accuracy tradeoff in dual-task performance, because this would predict that the faster dual-task movement time should lead to greater dual-task pointing errors. This was not the case, as dual- and single-task pointing errors were at the same level. Dual-task performance is generally worse or the same as single-task performance, and it is rare that dual-task performance is enhanced. However, Diedrichsen et al. (2004) found a similar enhancement in dual-task movement time when comparing direct unimanual (single-task) and bimanual (dual-task) pointing to targets that could be displaced. Brown and Jahanshahi (1998) also observed an Chapter 3 76 unexpected enhancement in dual-task performance of a peg placement task in Parkinson's patients when it was combined with a tapping task. For some skilled visually guided tasks, increased attention may interfere with the skilled action (Schmidt & Lee, 1999). Brown and Jahanshahi suggest that in a dual-task context where one task is largely automatic, attention to a separate task may result in excess attention being withdrawn from the automatic task; this results in an optimal level of attention being available for the automatic task. In other words, too much attention may be detrimental to a visually guided action. In the context of the present experiment, it is possible that the pressure of being configured to perform two tasks simultaneously results in a reduced, but optimal level of attention being available for pointing. Assuming that there are independent attention mechanisms for each stream, it may be the case that in the single-task context, both conscious ventral mechanisms and unconscious dorsal mechanisms are being devoted to action execution. When conscious attention is withdrawn from action execution in the dual-task context, then the enhancement in movement time can occur. This argument assumes optimal performance may not occur in the single-task context because ventral attention is being allocated to the pointing execution. However, because this movement time enhancement occurs even at Lag 700 when processing of the letter target is presumably complete and ventral attention is again available, this is not clear-cut evidence for withdrawal of ventral attention being beneficial to movement time. Additionally, this enhancement does not occur at Lag -700, when the letter target is most likely to be presented during action execution. Thus, it is not clear why the movement time enhancement is occurring. Overall, the results of the present experiment further support the ideas that (a) letter identification and pointing interfere at the level of action initiation or planning, (b) Chapter 3 77 these tasks do not interfere at the level of the on-line control of visually guided action, and (c) that there is a measurable advantage in movement time when participants are concurrently engaged in an identification task on a separate target. Up to this point, the experiments have allowed participants to pace themselves in performing letter identification and pointing, with the result that the two tasks interfere primarily in measures of pointing initiation. At one extreme, if the pointing target appears far in advance of the letter target, then pointing initiation is delayed by several hundred milliseconds. At the other extreme, if the letter target appears far in advance of the pointing target, then pointing initiation is as fast as when pointing is the only task to be performed. The monotonic decrease in pointing initiation time that occurs with an increasing lag between the appearance of the pointing and letter targets suggests that the pointing action must be fully planned before it can be carried out efficiently and concurrently with letter identification. However, in Experiment 3, when the pointing target greatly preceded the letter identification target, participants, on average, initiated pointing at a time very close to when the letter was being presented. One possibility is that in the Lag -700 condition, the planning phase was prolonged when participants were unsure whether the letter target would appear shortly (in 100 ms) or long (in 700 ms) after the appearance of the pointing target. If this is the case, then if participants were forced to initiate movement any earlier, then the planning phase would be cut short and there should be poorer performance that manifests itself in the pointing execution phase. Alternatively, planning could have been fully carried out in the Lag -700 condition, but participants were merely postponing initiation of pointing until they could anticipate if the letter target was upcoming. For instance, in the Lag -700 condition, if the letter target did not show up shortly after the pointing target, then participants knew that it would appear Chapter 3 78 long after and could then initiate the fully planned action. If this is the case, then forcing participants to initiate movement any earlier should not interfere with execution, as the movement is already fully planned. To test this hypothesis, the next experiment used an instructional deadline to encourage participants to initiate pointing even earlier than they would under the speeded pointing instructions of Experiment 3. Experiment 4: Negative and positive temporal lags under deadline pressure Experiment 4 examined the consequences of forcing participants to initiate their speeded movements earlier than they were inclined to. This was accomplished by imposing an instructional deadline on pointing initiation. Methods were identical to those used in Experiment 3, with the exception that participants were instructed to initiate pointing within a strict temporal deadline of 700 ms from the onset of the pointing target. This deadline was enforced through error feedback and through the recycling of trials on which actions were initiated too slowly. Participants therefore had to meet the deadline in order to complete the experiment in a reasonable amount of time. The deadline procedure limited the time that participants had to prepare the pointing action, particularly in the Lag -700 condition. Additionally, imposition of the deadline would ensure that in the Lag -700 condition, participants would be more likely to be executing pointing during the presentation of the letter target. Interfering with the spontaneous performance of pointing in conjunction with letter identification could lead to a few different outcomes. A limitation in initiating an action concurrent with letter identification would suggest that encouraging earlier initiation should not interfere with the execution of the action, as the action would already be fully planned. A limitation in planning an action concurrent with letter identification would suggest that if planning were to be truncated by the deadline Chapter 3 79 procedure, then this interference could carry over into the execution of pointing to the extent that there might be interference in both movement time and error. Thus, Experiment 4 created conditions where participants could not prepare or initiate their pointing actions according to the schedule that they would spontaneously produce under the speeded pointing conditions of Experiment 3. The same negative and positive lags that were tested in Experiment 3 were used here. Methods Participants Participants were 16 right-handed undergraduate students (13 females, mean age 19.5 years) with normal or corrected-to-normal vision who participated in exchange for course credit at the University of British Columbia. These participants had not previously participated in Experiment 3. Stimuli and Procedure The design was identical to that used in Experiment 3 with the exception of a deadline imposed upon the maximum time in which participants could initiate pointing after the onset of the peripheral pointing target. Participants were informed that if their pointing initiation time fell below 700 ms, the trial would be reused, thus prolonging the length of the experiment. Thus, participants were encouraged to initiate pointing as quickly and as accurately as possible upon pointing target onset, while still emphasizing the importance of correct letter identification performance. If the pointing initiation time was greater than 700 ms, then upon response completion, a 500 Hz error tone sounded for 500 ms and a message appeared that displayed response time and indicated that the trial would be presented again later in the experiment. Additionally, pointing responses that were initiated before the onset of Chapter 3 80 the peripheral target were punished with a red " X " and a message indicating that the trial would be presented again later in the experiment. Trials that did not fall into this initiation time window of 0 - 700 ms were recycled, such that they were presented randomly within the sequence of remaining trials. For both dual- and single-task experimental blocks, trials were presented until participants had completed a minimum of 192 trials (equally distributed within conditions) that fell within the correct response time window, up to a maximum of 240 trials each. The entire experiment took approximately one hour. Because of the trial recycling, participants performed an average of 208.5 trials in the dual-task block and 198.1 trials in the single-task block. Data Analyses 4.1% of trials were excluded from analyses because they did not meet the inclusion criteria defined in Experiment 1. An additional 2.1% of trials were excluded because they did not meet the pointing initiation time requirements of the current experiment. Other details of analyses were the same as described in Experiment 1. Results The primary findings were that (a) central letter accuracy was now affected when pointing was initiated before letter presentation, (b) pointing initiation time, while faster overall, was still longest when the pointing target greatly preceded the letter target, (c) pointing movement time was consistently faster in the dual-task context, and (d) pointing errors were worse in the dual-task context, particularly for displaced targets and for the greatest negative lag. Chapter 3 81 Letter Identification Performance Errors for central letter identification are presented in Figure 9A. Identification was quite good overall (4.8%), but there was now an effect of Lag on identification errors, F(3, 45) - 7.78, p < 0.0005. Errors were higher at the -700 ms Lag (11.3%) than they were at the -100, 100, or 700 ms Lags (2.0%, 2.9%, and 2.8%, respectively). Pointing Performance Initiation time. Mean pointing initiation time, presented in Figure 9B, again showed a lag-dependent cost. At Lag -700, dual-task initiation time was higher than single-task initiation time by over 120 ms, but this interference effect gradually diminished until it was absent at Lag 700, Task Number x Lag, F(3, 45) = 188.48, p < .0001. Movement time. Mean pointing movement time is shown in Figure 9C. Similar to the pattern observed in Experiments 2 and 3, dual-task movement time was faster than single-task movement time by an average of 49 ms, but now the faster dual-task movement times were present at the entire range of Lags that were tested, Task Number, F(l, 15) = 16.37, p < .002. There was no effect of Lag on movement time, nor did it interact with any other factor (all p's > .39). Chapter 3 82 Figure 9. Experiment 4 results: Negative & positive lags under deadline pressure A: Letter Identification s 5 C O l c •2 .4 o .3 T3 o> 9 _ l •* o c o a o £ 0 Dual-Task Central Letter -700 -100 100 Lag (ms) 7 0 0 Pointing B: Initiation Time C: Movement Time 775 -i £ 675 o £ g 575 S 4 7 5 ^ c | 375 275 Dual-Task Single-Task -700 -100 100 Lag (ms) 700 375 350 4 j | 325 E 300 > o S a 275 250 225 •#• Dual-Task Displaced -O- Dual-Task Stationary -A- Single-Task Displaced -A - Single-Task Stationary -700 -100 100 Lag (ms) 700 D: Absolute Constant Error 10 E £ o k_ LU 1 4 o a. 2H -700 Dual-Task Displaced -O- Dual-Task Stationary -A- Single-Task Displaced -A - Single-Task Stationary -100 100 Lag (ms) 700 Chapter 3 83 Errors. Pointing error measured in absolute constant error is presented in Figure 9D for stationary and displaced targets. Preliminary analyses on constant error again indicated that participants had a tendency to overshoot near targets and undershoot far targets (final positions). Mirroring previous findings, pointing was less accurate for displaced targets (6.7 mm) than for stationary targets (2.9 mm), F(l, 15) = 44.91, p < .0001. However, for the first time in the current series of experiments, it was observed that dual-task error (5.4 mm) was worse than single task error (4.2 mm), F(l/15) = 16.91, p < .001, but this dual-task error cost was greater for displaced targets than for stationary targets, Task Number x Target Movement, F(l , 15) = 14.67, p < .002. Simple effects confirmed the greater dual-task cost for online corrections, such that the dual-task cost was significant for displaced targets, F(l , 15) = 22.31, p < .0004, but only a trend for stationary targets, F(l/15) = 3.81, p < .07. This dual-task cost was also greatest at Lag -700, and decreased as lag increased, Task Number x Lag, F(3, 45) = 2.87, p < .05. Examination of variable error did not reflect the same absolute constant error patterns in Task Number or Lag. There was no effect of dual tasks on consistency, p > .42, and there was a trend for consistency to vary by Lag, F(3, 45) = 2.59, p < .07, but the effect was not a systematic increase or decrease with changes in Lag. The only effect in variable error that reflected the absolute constant error findings was that pointing was more variable for displaced targets (5.9 mm) than for stationary targets (5.2 mm), F(l, 15) = 17.66, p<.0009. Actual pointing initiation relative to letter onset. Pointing movement time and error for the current experiment were examined according to when pointing was initiated relative to letter presentation. Using the same procedure employed in Experiment 3, the trials were sorted into four bins according to when the pointing Chapter 3 84 action was actually initiated relative to the onset of the central letter. Bin 1 (-600 ms to -200 ms) only included movements that were initiated well before letter onset, Bin 2 (-200 ms to +200 ms) included movements beginning near letter onset, Bin 3 (+200 ms to +600 ms) included movements beginning shortly after letter onset, and Bin 4 (+600 ms to +2000 ms) included movements that began long after letter onset. Both pointing movement time and errors were analyzed with a repeated-measures A N O V A that examined the factors of Task Number, Target Movement, and Bin. Because of missing data, one subject was excluded from the A N O V A . The results were quite consistent with the preceding pointing analyses for this experiment. Analyses of movement time reflected earlier results indicating that dual- task movement time was faster than single-task movement time, F(l, 14) = 16.79, p < .002, and was faster for stationary rather than displaced targets, but not at the earliest initiation time bin, Target Movement x Bin, F(3, 42) = 3.49, p < .03. Analyses of absolute constant error revealed that dual-task errors were worse than single-task errors, F(l, 14) = 19.82, p < .0006 and that this dual-task cost was greater at the Bins 1 and 2, representing pointing movements initiated before and during letter presentation (2.3 mm and 2.1 mm dual-task cost in absolute constant error) than for Bins 3 and 4, representing pointing movements initiated after and long after (0.8 mm and 0.6 mm dual-task cost in absolute constant error), Task Number x Bin, F(3, 30) = 4.13, p < .02. Note that the data contributing to Bins 1 and 2 would have come from the Lag -700 condition. Additionally, the dual-task error cost was greater for displaced targets (difference of 2.1 mm) than for stationary targets (difference of 0.8 mm), Task Number x Target Movement, F(l, 14) = 4.81, p < .05. There was also greater error for displaced targets over both single- and dual-task conditions, F(l/14) = 36.36, p < .0001. Chapter 3 85 Discussion Because the results of the present experiment differed from previous experiments on all measures (pointing initiation time, movement time, error, and letter identification), each measure will be considered in turn in the following sections. Pointing Initiation Time The results showed that the deadline instructions were quite effective in that participants sped up their pointing initiation times so that they were well under the deadline of 700 ms. The lag-dependent initiation time cost was still present, such that there was a dual-task cost in pointing initiation that decreased as the presentation of the pointing target was increasingly delayed relative to the letter target. However, average initiation time was decreased overall in both the single and dual-task conditions. Even though single-task initiation time without the deadline instructions (in Experiment 3) were well under 700 ms, the additional instructional pressure led participants to speed up initiation times overall. This indicates that the instructional stress has a general effect across all conditions, even when increased initiation speed is not necessary (i.e., in Lags -100, 100, and 700). It is interesting that, under the deadline instructions, when the pointing target appeared well in advance of the letter target (Lag -700), subjects could not initiate the pointing movement at single-task levels, the same way that they could when the pointing target trailed the letter target by 700 ms. This suggests that the knowledge that letter identification would follow competes with the planning of pointing. This may have been a product of the fact that temporal lags were randomly intermixed, such that when the pointing target came up first, participants were still unaware of whether the letter would be presented shortly after or long after the pointing target. Thus, part of the Chapter 3 86 dual-task initiation time cost may have been caused by the uncertainty of whether concurrent letter identification would have to be accomplished while planning was still underway. By this account, an experiment in which Lags are blocked should show that dual-task initiation time would be at or near single-task levels when participants knew that letter identification would not be required until long after the pointing target was presented. Also, by this account, even larger negative lags would still lead to dual-task initiation time costs, as long as short and long negative lags were mixed. Pointing Movement Time As observed in Experiments 2 and 3, there was a dual-task set advantage for pointing movement times. Thus, when participants were prepared to perform dual tasks, there was ~50 ms of facilitation in movement time. What was different was that this facilitation was observed even at Lag -700, whereas it was not observed in Experiment 3. One possibility is that when participants were under more instructional stress and having more difficulty identifying the perceptual target (as indicated by the greater letter identification errors in Lag -700), attention was even more withdrawn from pointing, resulting in a general facilitation in pointing movement time across all lags. Thus, there could be a general facilitation in movement time when participants are prepared to perform two tasks rather than one. However, the fact that movement time facilitation is not seen in Experiments 1 and 2 does not clearly support this idea. Pointing Errors When participants initiated pointing earlier than they would under regular speeded instructions, there was a dual-task cost manifested as an increase in errors across all lags, particularly for targets that were displaced. This dual-task set effect Chapter 3 87 suggests that there is an overall error cost to the online control of pointing when participants are under pressure. In addition to the general dual-task set effect, the dual-task cost for online corrections was greatest when the pointing target greatly preceded the letter target (Lag -700), where planning is most truncated by the deadline instructions. This same cost at Lag -700 is seen in stationary targets, suggesting that impaired planning may be contributing to accuracy costs in those conditions. At the same time, the online control cost for displaced targets cannot wholly be attributed to impaired planning, because (a) it exists when in conditions where planning should not be as interrupted (Lags -100, 100, and 700), and (b) the dual-task cost is greatly attenuated for stationary targets, which should have an identical planning phase as for displaced targets (since displacement is triggered by movement onset). What accounts for the general impairment in online control across temporal lags? This could be an effect of being unable to incorporate visual feedback of the new pointing target location. Movement times were faster in the dual-task condition relative to performance in Experiment 3. Estimates of the time it takes to incorporate visual feedback to ensure accurate pointing between 100 and 260 ms (Keele & Posner, 1968; Zelaznik, Hawkins, & Kisselburgh, 1983). However, dual-task movement times in this experiment were around 250 ms, which makes it seem likely that participants had enough time to process the new target location that was triggered upon movement onset. Additionally, there was not merely a general speed-accuracy tradeoff that was contributing to the costs in pointing accuracy. Even with decreased initiation and movement times overall, there was comparable accuracy for single-task pointing in the Chapter 3 88 present experiment (2.9 mm) when compared to Experiment 3 (3.2 mm). The fact that pointing accuracy to displaced targets was generally worse in the dual-task condition, regardless of initiation or movement time (even when dual- and single-task initiation time are the same at Lag 700), supports the idea that the instructional pressure and the dual-task context interfered with online correction in a general way. Thus, it is possible that the online control of the dorsal stream breaks down only under high-stress situations when participants are in a dual-task context, set up to perform both perceptual identification and pointing. It has been shown that the dorsal stream is responsible for control of fast, automatic pointing movements (Pisella et al., 2000; Prablanc & Martin, 1992). Thus, one would expect that the dorsal stream could make quick corrections under these conditions, and indeed, participants could correct to single-task displaced targets (albeit with slower movement times). But the present results suggest that online control has its limitations, such that it will not operate efficiently when there is both an instructional stress and a dual-task context involving a ventral perception task. It may be the case that under stress, movements become more ballistic, reflecting more of the original movement plan and the reduced ability to correct to target displacements. This stress could be specific to initiation of pointing under a strict temporal deadline, or due to a more general interference of having to perform both tasks under stress. If the latter situation is the case, then performing the dual-tasks under other stressful situations (e.g., startling tones randomly presented, monetary penalties imposed) should also lead to interference in online control. As described above, the present increase in errors at Lag -700 for both stationary and displaced targets supports the idea suggested in Experiment 3, that action planning was prolonged by the dual-task requirements in the Lag -700 condition, rather than initiation being delayed. If only initiation were delayed and the movement was already Chapter 3 89 programmed, then encouraging participants to initiate pointing earlier should not have lead to a cost in errors. This was not the case in the present experiment. When initiation was forced to occur earlier, this lead to an increase in pointing errors, suggesting that action was initiated before it could be fully planned. This explanation assumes that interference in action planning carries out into the execution stages. It also assumes that movement parameters are at least partly specified by the ventral stream before the onset of movement (during planning). This is contrary to the claim of the Perception-Action model that specifies that movement parameters are specified solely by the dorsal stream. However, it could also mean that the dorsal stream cannot specify the movement parameters precisely until the more general ventral stream planning is complete. A n additional factor in the Lag -700 accuracy cost is that preparation for and processing of letter identification on these trials is occurring during movement execution. Examination of the initiation time binning analyses confirmed that the dual-task accuracy cost for both stationary and displaced targets was greatest when movements were initiated before or during letter presentation, rather than after letter presentation. However, the finding that concurrent letter identification did not interfere with pointing execution error when there was no deadline (Experiment 3) suggests that concurrency with identification is not the primary culprit in contributing to increased errors. A n alternative explanation for the concurrency cost at Lag -700, however, comes from comparing the pointing movement time in Experiments 3 and 4. Recall that pointing movement time in dual-task Lag -700 was elevated relative to all other lags; this is not the case in Experiment 4, as pointing movement time is invariant across lags in the dual-task condition. Thus, the deadline instructions may have had the effect of Chapter 3 90 creating a speed-accuracy tradeoff only in dual-task Lag -700, such that movement time was sped up, resulting in an accuracy decrease in pointing. Letter Identification Whereas Experiment 3 did not reveal any interference effects for pointing on letter identification, the present experiment did. Specifically, the deadline pressure also affected measures of letter identification, such that there was a dual-task cost in letter identification when the pointing target appeared before the letter (Lag -700) and action planning was "short-circuited" by the deadline instructions. This suggests that the increased deadline pressure on the pointing interfered with the ventral-stream processes available for planning, and also interfered with ventral stream processes affecting accuracy for letter identification (i.e., ventral processes interfered with in general). Another possibility is that the deadline pressure led participants to prioritize pointing over letter identification in order to initiate pointing within the temporal restrictions. Overall, all primary measures showed dual-task interference in Experiment 3, particularly at Lag -700. Planning of pointing was curtailed, dual-task pointing movement time was facilitated, pointing errors were increased, and letter identification was impaired. It seems that the heightened pressure to perform both letter identification and pointing reduced performance in both tasks. Importantly, the present results show that the dorsal stream control of online adjustments is vulnerable to interference from the general context of speed stress and being prepared to perform a perceptual identification task. Thus, the independence of the execution and online control of action from concurrent perception is not absolute, as conditions can be shown where it breaks down. Chapter 3 91 Interim Summary Experiments 1-4 are supportive of separate attentional mechanisms within a Perception-Action model of task interference, with regards to perceptual identification and pointing. In this model, information for the performance of two different visual tasks (pointing execution to a newly appearing target and identifying a central letter) appears to be able to enter the systems and be processed without interference. The supporting evidence for this idea is that there is no measurable interference in execution of each task when the two targets are simply presented in close temporal proximity to one another. On the other hand, there is interference between the planning of pointing and letter identification. Additionally, when participants are not allowed to fully plan their actions before having to initiate these actions, as in the deadline procedure (Experiment 4), then both letter identification and pointing errors begin to suffer. However, if planning of the pointing action is permitted to run to completion, then the online control of action can be accomplished even when it is occurring simultaneously with the presentation and processing of a central letter (Experiments 1-3). The evidence that this is actually online control, rather than merely pointing to a previously remembered target location (e.g., participants might store the peripheral target location upon its appearance and respond to it only after identifying the central letter), is that the pointing is sensitive to the unpredictable target displacements that occur after the pointing action has already been initiated. The remaining experiments explore the generality of the two-visual stream model of task interference by looking at the kinematics of pointing execution (Experiment 5) and exploring the consequences of making pointing execution rely more upon ventral, rather than dorsal stream, processing (Experiments 6-8). The prediction is Chapter 3 ' that as all phases of pointing rely more upon ventral stream processing, then a ventral task of identification is likely to cause interference in pointing execution. Chapter 4 93 CHAPTER 4: Experiment 5: Kinematic measures of single- and dual-task pointing Experiments 1 through 3 have established that, under normal instructions for speeded pointing, the execution and online control of pointing can be carried out concurrently with perceptual letter identification, but the planning of pointing cannot be carried out concurrently with letter identification. This was demonstrated through a robust interference of letter identification with pointing initiation time, but no interference of letter identification upon pointing movement time and accuracy. However, only behavioral measures of pointing (initiation time, movement time, and error) were collected because of the nature of the touchscreen setup used in the original experiments. The question remained open, then, as to whether kinematic measures of pointing were sensitive to the dual-task requirements in ways that were not revealed through the behavioural measures. For instance, Diedrichsen et al. (2004) showed that even though movement time and error for pointing to displaced targets were not affected by a dual-task context, the kinematics of pointing showed significant dual-task effects not reflected in the behavioural measures. Thus, Experiment 5 explored the kinematics of pointing to targets while participants were concurrently engaged in a concurrent perceptual identification task. The three-dimensional position of the hand was monitored continuously during the pointing action. The new measures analyzed were peak velocity, time to peak velocity, and time after peak velocity (deceleration time). The first two measures were used as an index of the initial impulse phase of movement, reflecting where the movement was initially planned to. The third measure of deceleration time was used to index the amount of time spent in the corrective phase of movement, where online adjustments usually occur. In other words, peak velocity and time to peak velocity were used as Chapter 4 94 indicators of planning, and deceleration time was used as an indicator of execution and online control. The equipment used to monitor these kinematics prompted several small procedural changes to both the central perception task and the pointing task. As only an LED board was available to present visual stimuli, the ventral perception task was modified to involve the discrimination of the duration of a brief flash of light (short versus long). Timing is considered attentionally demanding, such that if attention is diverted from a timing task, temporal intervals tend to be misperceived as shorter than they actually are (e.g., Brown & Boltz, 2004; Macar, Grondin, & Casini, 1994). Another change was that the distances over which the pointing action was made were increased in order to maximize use of the display surface area. A n increase in the distances of pointing action would prolong the pointing movement, which would allow more time for participants to incorporate visual feedback of the target position, particularly for displaced target trials. Thus, errors for pointing were expected to be reduced overall. A final change involved modifying the action task such that target displacements were triggered by eye movements, rather than by initiation of hand movements. This would allow for the observation of whether the impermeability of action execution would occur when the target jumps were triggered at a time other than at pointing initiation. For the current experiment, the results will show that that target jumps were triggered shortly after movement initiation, instead of at movement initiation. The target displacements were time-locked to eye movements to the target, a time during which a phenomenon known as saccadic suppression occurs. During saccades, the conscious sensitivity to target displacements is reduced or suppressed (Bridgeman, Hendry, & Stark, 1975; Bridgeman et al., 1979). In the previous experiments, Chapter 4 95 participants were informed that the targets would sometimes be displaced, but they were not told the specific percentage of target displacements. In the present experiment, participants were not informed of target displacements. Fecteau et al. (2001) have shown that awareness of target displacements does not affect the ability to correct to them. Thus, tying in target displacements to saccadic motion and making jumps less available to conscious awareness was not expected to have any effect on participants' ability to" correct to the new target location. Additionally, given that target jumps were now triggered by eye movements, only positive lags were tested (100, 300, 700) as in Experiments 1 and 2. Using negative lags would have meant that participants would invariably miss the central light duration target if they were to make eye movements to the peripheral target before the onset of the central light. However, Experiment 3 had already established the generality of perceptual interference on planning, but not the execution and online control of pointing movements, regardless of positive or negative lags. The interpretation of the Perception-Action model that has evolved so far suggests that, even with these modified tasks, measures that reflect pointing execution and online control (pointing movement time, error, deceleration time) should not show dual-task interference. With respect to the measures taken during execution that reflect planning (peak velocity, time to peak velocity), there are two possible outcomes. On one hand, since action planning has been demonstrably interfered with in a dual-task context, then the visual perception task may also impair these measures. On the other hand, given the finding that execution is not interfered with if the movement has been fully planned (Experiments 3 and 4), measures taken during execution may not show any significant sensitivity to the dual-task context. Chapter 4 96 Methods Participants Participants were 11 right-handed students (7 females, mean age 23.5 years) with normal or corrected-to-normal vision who voluntarily participated in exchange for $30 C D N at the University of British Columbia. Apparatus Participants were seated so that the midline of the body was aligned with visual fixation and the starting position for the hand. Viewing distance was fixed at 57 cm from the display using a chinrest. The room was dimly lit in a manner that prevented lighting glare from appearing on the pointing surface. Displays and data collection were controlled on a PC using custom-written QuickBasic 4.5 software in DOS. Stimuli for identification and pointing were presented through a translucent white plexiglass surface (45 cm high x 61 cm wide) that was tilted upward (approximately 13 degrees from the horizontal) at approximately waist-height; thus, participants were looking downward toward the display surface. Electrooculography. Horizontal movements of both eyes were monitored using electrooculography (EOG), which measures the differences in electrical potential between the front and the back of the eye. A disposable Ag-AgCL surface electrode (Kendall-LTP, Huntington Beach, CA) was placed at the outer canthus of each eye and a ground electrode was placed on the forehead. EOG signals were amplified (5-10 K) and band-pass filtered (0.1 - 30 Hz) using an A C preamplifier (Grass Instruments P511, Astromed). EOG signals were sampled at a rate of 500 Hz. The EOG signal was passed through an analog circuit that allowed online triggering of target jumps at approximately the midpoint of the primary saccade. Chapter 4 97 Kinematics. Pointing movements were made with a stylus held like a pen in the right hand. The stylus had an infrared marker placed on its tip. To collect the three-dimensional spatial coordinates of limb movements, position of the stylus was monitored using an Optotrak 3000 motion analysis system (Northern Digital) that was sampled at a rate of 500 Hz. The Optotrak 3000 has a spatial resolution of 0.01 mm at 2.25 metres and is accurate within .1 mm on the X and Y axes. For the experiments presently described, the Optotrak was set to sample at a temporal resolution of 500 Hz (every 2 ms). Error of pointing movements relative to the final target location was calculated in millimeters along the horizontal axis. Data reduction. EOG signals were low-pass filtered with a second-order dual-pass Butterworth filter (30 Hz cutoff). The onset of a saccade was defined as the lowest point in the EOG after which the signal increased toward the peak saccade (Carnahan & Marteniuk, 1994). The peak of the EOG signal was used to define the end of the primary saccade. The raw displacement data in the primary direction of movement (along the horizontal axis) were also low-pass filtered with a second-order dual-pass Butterworth filter (10 Hz cutoff). To determine instantaneous velocity, displacement was differentiated using a two-point central finite difference algorithm. To determine instantaneous acceleration, velocity was differentiated using the same algorithm. The beginning of the pointing movement was defined by the first velocity that was equal to or greater than 30 mm/s. The end of the movement was defined as when the stylus contacted the display panel. Stimuli & Procedure A sample trial is shown in Figure 10. Response stimuli were red LEDs presented at various locations under the display surface. Each LED created a circle of light Chapter 4 98 subtending 0.5°. The starting position for the pointing hand on all trials was designated with a yellow circle subtending 1.2 °. Visual fixation was designated with a 0.5° outline circle that was centered 2.4° above the starting hand position. Trials were initiated by the experimenter when the subject was gazing at fixation and EOG signals had settled down to baseline levels. Participants were required to maintain fixation until the appearance of the peripheral target, upon which they were to look at and point to the peripheral target as quickly and accurately as possible. A tone sounded to indicate the start of the trial. At a random interval between 1.5 and 2.5 sec later, the duration discrimination stimulus appeared at fixation. The duration discrimination stimulus appeared for either 15 ms ("short" duration) or 60 ms ("long" duration) At a 100, 300, or 700 ms Lag after onset of the central LED, an LED appeared in the periphery at a position either 25, 27.5, or 30 cm (or degrees of visual angle) to the right of visual fixation. These positions will be referred to as the near (25 cm), middle (27.5 cm), or far (30 cm) positions for Experiment 5. Peripheral targets were equally divided between two conditions: (a) stationary targets, where the target appeared randomly in one of the three locations and remained in that location for the remainder of the trial, and (b) displaced targets, where the target initially appeared in the middle position, but during the saccade, the target either jumped backward to the near position or forward to the far position. Target displacements were triggered at the midpoint of the primary saccade. Participants were given both written and verbal instructions at the start of each experimental session. In the dual-task condition, participants were instructed to perform the following tasks: (a) to identify the duration of the centrally presented light as accurately as possible, and (b) look and point to the peripheral light as quickly and as Chapter 4 99 accurately as possible upon its appearance. Participants were not informed that the pointing target would change location; they were merely instructed to point to the final target location. Time Fixation Light target flashed (15 or 60 ms) —i Pointing target 100, 300, 700 ms lag Pointing target displaced during saccade Point to final target location Figure 10. Sample trial sequence for Experiment 5. Schematic illustration of a typical trial sequence. A displaced target trial is illustrated. Verbal report of the light duration ("short" or "long") was made following completion of the pointing movement; the experimenter recorded the report and gave verbal feedback if participants made an error. At the end of the trial, a high tone sounded to indicate that participants could return their hand to the starting position. If participants started looking to the peripheral pointing target before its onset, the experimenter verbally delivered error feedback. Chapter 4 100 In the single-task condition, the displays and the response requirements were the same as in the dual-task condition, with the exception that participants ignored the centrally presented light and only had to point to the peripheral light. Dual- and single-task conditions were presented in separate sessions that were presented in counterbalanced order across participants; sessions were conducted on separate days. Before a dual-task session, practice trials were given for the light-duration discrimination alone, the pointing task alone, and then the duration discrimination and pointing task together, until the participants could successfully perform both tasks. Before a single-task session, practice trials were given for the pointing task alone. After the practice trials were given, accuracy was calibrated for each target position. For the experiment, participants completed 360 trials each for the dual- and single-task conditions. Rest breaks were given every 90 trials or upon request. Because of the nature of the equipment setup, the experimenter was present in the same room as the participant for the duration of the experiment. Data Analyses: Kinematic Measures New measures examined in this experiment were (a) peak velocity (PV, the rate of change in horizontal hand position with respect to time), (b) time to peak velocity (TtPV, duration between hand liftoff from home and attainment of peak velocity, (c) time after peak velocity (TaPV, duration of movement after attainment of peak velocity, used as an indicator of time spent in deceleration phase, where movement adjustments usually occur). Trials were excluded from analysis if participants made anticipatory eye movements (< 100 ms) or late eye movements (>1000ms), anticipatory pointing movements (< 100 ms), if there was an error recording from the stylus, and if there were Chapter 4 101 any irregularities in the EOG (low baseline, failure to trigger appropriate target displacement). In total, 5.7% of trials were excluded from initial analyses. Repeated-measures ANOVAs were used to analyze the dependent measures of light duration identification, pointing initiation time, movement time, error, variability, time to peak velocity, peak velocity, and time after peak velocity, using the same data inclusion criteria as described in Experiment 1. Three within-participants factors were examined: (a) Task Number: dual, single, (b) Lag: 100, 300, 700 ms, and (c) Target Movement: stationary, displaced. Data were collapsed over certain factors for selected dependent measures when they were not relevant to the analysis. In particular, only Lag was examined for central light discrimination, and only Task Number and Lag were examined for pointing initiation time. Results The primary findings from this experiment were that (a) light duration judgment was impaired by temporal proximity between the light duration and the pointing targets, (b) pointing initiation time to the peripheral target was impaired by temporal proximity to the central target, (c) pointing movement time and accuracy were unimpaired by temporal proximity to the central target, and (d) kinematic measures of pointing were unimpaired by temporal proximity to the central target. Chapter 4 102 Figure 11: Experiment 5 results: Behavioural measures A: Light Discrimination D: Absolute Constant Error -# - Dua l -Task Disp laced - O - Dua l -Task Stat ionary -A- S ing le-Task Disp laced - A - S ing le-Task Stationary 100 300 Lag (ms) 700 Chapter 4 103 Figure 12: Experiment 5 Results: Kinematic measures A: Peak Velocity 1500 1475 1450 -# - Dua l -Task Disp laced - O - Dua l -Task Stationary -A- S ing le-Task Disp laced ^ A - S ing le-Task Stationary B: Time To Peak Velocity 185 (A § 175 1 170 5 * 165 ro a o- 160 o 155 150 145 100 300 700 Lag (ms) C: Time after Peak Velocity -#• Dual -Task Disp laced - O - Dual -Task Stationary •A- S ing le-Task Disp laced - A - S ing le-Task Stationary 330 „ 320 (A § , 310 f 300 | 290 H 280 a ft 270 I 260 o E 250 i-240 i 230 Dual -Task Disp laced - O - Dual-Task Stationary -A- S ing le-Task Disp laced - A - S ing le-Task Stationary 100 300 700 Lag (ms) Chapter 4 104 Light Duration Tudgment Errors for light duration judgments, shown in Figure 11 A, were higher at Lag 100 (26.8%) than at Lag 300 or 700 (15.3%, 14.5%, respectively), F(2, 20) = 25.86, p < .0001. This suggested that at the shortest Lag, participants shifted attention to the pointing target when it appeared. Also note that chance performance on the light discrimination is 50%, so even when examining trials where participants made the correct judgment of light duration, there was a possibility that these responses may have been false positives. Indeed, examination of the proportion of "short" responses indicated that participants were biased to report "short" at Lag 100 (68%), a bias that was gone by Lag 300 (53%) and Lag 700 (48%). However, when additional analyses were conducted for pointing measures using all trials (regardless of light duration performance), this made no difference to the pattern of results. For comparability with Experiments 1 through 4, pointing performance will only be presented for trials on which the light duration judgment was correct. Pointing Performance Eye movements generally preceded hand movements. On average, when a target displacement occurred, it occurred 50 ms after the eye started moving and 21 ms after the hand started moving. Initiation time. Pointing initiation time is shown in Figure 11B. As observed in previous experiments, initiation time showed a concurrency cost. Specifically, dual-task initiation time was higher than single-task initiation time by 99 ms at Lag 100, but this difference was absent by Lag 700, Task Number x Lag, F(2, 20) = 24.30, p < .0001. Chapter 4 105 Movement time. Pointing movement time is shown in Figure 11C. Performing dual tasks had no effect on movement time, F(l, 10) < 1, and this did not vary by Lag, Task Number x Lag, F(2, 20) = 1.81, p > .19. The only effects observed were that movement time increased with increasing Lag, F(2, 20) = 23.65, p < .0001, and movement time was longer for displaced than for stationary targets, F(l, 10) = 11.56, p < .007. Errors. Errors measured in absolute constant error are presented in Figure H D . There were no effects of Task Number, nor did it interact with any other factors, all p's > .14. Errors had a tendency to be worse for displaced (3.9 mm) than for stationary (2.8 mm) targets, F(l, 10) = 4.67, p < .06. In the displaced target condition, the level of error indicated that participants were almost fully correcting to the final target location. If participants were not correcting their movements, then absolute constant error should be at least at 25 mm (the distance of a target displacement). Examination of variable error did not show any dual-task costs for Task Number or Lag. Variable error decreased with Lag for single tasks (from 5.9 mm to 5.0 mm at Lags 100 to 700), but not systematically for dual tasks (from 5.2 mm to 5.1 mm at Lags 100 to 700), Task Number x Lag, F(2, 20) = 3.58, p < .05. Additionally, pointing was less consistent for displaced targets (5.4 mm) than for stationary targets (4.8 mm), F(l , 10) = 7.01, p < .03. Peak velocity. PV is shown in Figure 12A. There was no main effect of Task Number on PV, F(l/10) = 1.15, p > .30. PV decreased with increasing Lag, F(2, 20) = 10.59, p < .0008, and this decrease was more pronounced for single-tasks than for dual-tasks, Task Number x Lag, F(2, 20) = 3.56, p < .05. Time to peak velocity. TtPV is shown in Figure 12B. There was no significant effect involving Task Number (all p's > .16). There appeared to be a trend for dual-task Chapter 4 106 TtPV to be higher than single-task TtPV, but the variability between subjects was too great to show an effect of any significance. Examination of the means of median values revealed the same pattern of effects. Time after Peak Velocity. TaPV is shown in Figure 12C. TaPV was greater for .displaced than stationary targets, F(l/10) = 13.53, p < .005. TaPV also increased with increasing Lag, F(2, 20) = 25.72, p < .0001. Discussion The behavioural measures generally mirrored the results of the previous experiments. There was dual-task interference evident in initiation time, but not in movement time or error. When pointing execution was subdivided into the two phases of acceleration and deceleration through examination of measures centering around peak velocity (the point at which acceleration changes to deceleration; Schmidt & Lee, 1999), there were no reliable effects of the dual-task context on kinematic measures of pointing. Specifically, kinematic measures did not clearly reveal any dual-task costs in indices of initial movement planning (peak velocity, time to peak velocity) or control (time after peak velocity). This is consistent with the findings from Experiments 1 to 3 that the execution of movement could be performed without any significant interference from the task of light duration identification. The finding that there were no significant dual-task costs on the kinematic indices of movement planning (peak velocity, time to peak velocity) supports the idea that, under normal instructions for speeding pointing, the movement has been fully planned to the original target location by the time that pointing is initiated. The suggestion that follows is that, under deadline pressure, movements will not be fully planned and dual-task interference will show up as significantly lower peak velocity Chapter 4 107 and higher time to peak velocity. Additionally, this finding again brings up the issue of whether initial movement parameters are specified by ventral planning or by the dorsal stream. Because these movement parameters were not significantly affected by the dual-task context, and given the present findings that visual perception does not affect dorsal stream execution, it is also possible that there was no interference because the dorsal stream determines these parameters. While the kinematic measures of the current experiment supported the idea that the online control of pointing can be carried out without any interference from a light duration identification task, there were some differences that were not observed in the preceding experiments. One new finding was a significant cost to light duration accuracy at the shortest lag. In contrast, for Experiments 1-3 where the same speeded pointing instructions were used, the perceptual letter identification task did not suffer at Lag 100 where identification and action planning were happening concurrently. However, the two tasks of light identification versus letter identification are not equated for difficulty, and the light identification may simply have been more difficult and more vulnerable to interference from a competing ventral stream process of planning. Given that temporal judgments have a tendency to be underestimated if attention is divided with another task, (e.g., Macar et al., 1994), the results are consistent with the idea that the planning of pointing was interfering with the concurrently demanded light judgment at the shortest lag. As the temporal separation between the tasks increased, this interference disappeared. Another difference in the results of this experiment was the absence of the dual-task movement time facilitation observed in Experiments 3-4. The fact that it was not present here under the same lag conditions that were previously tested does not Chapter 4 108 support the idea that the dual-task context leads to a decreased and optimal amount of attention being available for more efficient performance of the pointing task. Overall, the results of the present experiment demonstrated that successful light duration identification interfered with initiation of pointing to a peripheral letter, but did not interfere with the speed, accuracy, or kinematics of pointing execution. Chapter 5 109 CHAPTER 5: The remaining experiments were designed to examine if dual-task interference in the execution of planning could be observed if the pointing action was made to rely more upon the conscious processes of the ventral stream. This was tested by changing the pointing action so that it required a more complex remapping from the actual target location (Experiment 6), by changing the action so that it required a one-to-one, spatial translation from the target position (Experiment 7), and by making the pointing action rely more upon memory (Experiment 8). Experiment 6: Spatially Remapped Pointing The first experiment designed to make action execution rely on ventral stream processes involved making participants utilize a more complex decision rule in order to determine where to point. If the planning phase of pointing is dissociable from the execution of the of action, then it should be possible to influence the duration of planning in addition to influencing the execution phase. In some sense, this has already been demonstrated in that letter identification has produced a reliable lag-dependent cost in pointing initiation without negatively affecting measures of execution (Experiments 1-3, 5). As of yet, the experiments have not manipulated the complexity of the stimulus-response mapping, as participants always pointed directly to the onset of a peripheral target. However, by indicating the location for pointing in an indirect, symbolic manner, this may cause pointing execution to rely on ventral stream processing and lead to more action interference from a concurrent perceptual task. On the other hand, it may be possible that the ventral stream controls complex target selection and action planning, but the dorsal stream still controls action execution. In this latter case, there should not be interference from a concurrent perception task, as Chapter 5 110 long as the movement has been fully planned before initiation. However, if the mapping rule is changed during movement execution, the ventral stream should again become involved in selecting and planning a movement to a new target location, and interference should be seen in execution. Thus, Experiment 6 examined the consequences of increasing the difficulty of the movement planning by requiring a more complex stimulus and response mapping than was utilized in previous experiments. The experimental setup was identical to that used in Experiment 5, such that the perceptual task was judgment of light duration, and kinematics of pointing were collected in addition to behavioural measures. But whereas the peripheral lights in Experiment 5 served as the targets, in the present experiment, they served as symbolic target location cues. Specifically, a light on the right side of the pointing region indicated that participants were to point to a position on left side; a light on the left side indicated that participants were to point to a position on the right side, and a light in the middle indicated that participants were to point to the middle. Thus, there was a different spatial mapping rule for each target location. Additionally, eye movements to the target triggered target displacements on half of the trials, which meant that the spatial mapping rule would change during the trial. If the increased complexity of the stimulus-response mapping for the pointing action caused action execution to rely on ventral stream processes, it was expected that the light duration judgment would affect the measures of pointing execution in the dual-task context. If, however, the dorsal stream controlled action execution after more complex planning, then dual-task interference should not be exhibited unless the location cue changed position and planning to a new target location had to be initiated. Additionally, the increased complexity of the target-response mapping was expected to elevate pointing initiation time overall. The increased complexity was also Chapter 5 111 expected to increase the concurrency cost observed in dual-task pointing initiation time, as more difficult tasks are generally more prone to attentional interference (Pashler, 1998). Methods Participants Participants were 12 right-handed students (11 females, mean age 21.2 years) with normal or corrected-to-normal vision who participated in exchange for $30 C D N at the University of British Columbia. Apparatus, Stimuli and Procedure A sample trial is shown in Figure 13. The apparatus and displays were identical to those used in Experiment 5, with the exception that (a) pointing markers were added to the display panel and (b) a different indirect mapping rule was applied to each possible target location. The display panel is shown in Figure 10. Seven markers (blue circles subtending 0.5° each) were placed on the display in order to mark possible locations that participants could point to. The markers were placed at 2.5 cm (or degrees of visual angle) intervals at locations ranging from 20 to 35 cm to the right of the hand's starting position. The markers were aligned along the horizontal axis and were positioned 2 cm below where the target lights indicating the pointing position would appear. While the target lights would only appear at three possible positions above the markers (25, 27.5, and 30 cm to the right of the hand's starting position), the extra two flanking markers on each side of the target positions were used to create greater positional uncertainty as to where the target light would initially appear. Depending on the target light position, participants had to point to a designated blue marker using the following set of spatial remapping rules. If the light appeared Chapter 5 112 above the marker directly left of centre (at the 25 cm position), then participants had to point to the marker that was to the right of centre (at the 30 cm position). If the LED light appeared above the marker directly to the right of centre (at the 30 cm position), then participants had to point to the marker that was to the left of centre (at the 25 cm position). Finally, if the LED light appeared above the central marker (at the 27.5 cm position), then participants had to point to the marker that was at the centre position. Thus, there was a different pointing rule for each of the three possible pointing target light locations. Time Fixation Light target flashed (15 or 60 ms) — i 100, 300, 700 ms lag • • • Pointing target location cue —1 Pointing target displaced during saccade Point to target symbolized by location cue IT Figure 13. Sample trial sequence for Experiment 6. Sample spatial remapping trial depicting a condition in which the target location cue is displaced during the saccade. Here the target LED initially appears at the 27.5 cm position, then changes to the 25 cm position during the saccade. The correct pointing location is the 30 cm position. Pointing target lights were equally divided between conditions where the target was displaced or remained stationary during participants' saccades. For displaced Chapter 5 113 targets, target movements were divided equally between forward and backward target displacements. On forward displacement trials, targets either jumped from the centre position to the right position (27.5 cm to 30 cm) or from the left position to the centre position (25 cm to 30 cm). On backward displacement trials, targets either jumped from the centre position to the left position (27.5 cm to 25 cm) or from the right position to the centre position (30 cm to 27.5 cm). On stationary trials, targets appeared equally often at the left, centre, or right target positions. For both displaced and stationary target conditions, targets were equally likely to end up at one of the three target locations. For the experiment, participants completed a block of 360 trials each for the dual-and single-task conditions across two experimental sessions. Factors of lag, target position, and target movement were randomly intermixed within dual- or single-task blocks. Data Analyses The trial exclusion criteria defined in Experiment 5 were used to filter the present data, resulting in 5.5% of trials being excluded from initial analysis. Repeated-measures ANOVAs were used to analyze most dependent measures. Three within-participants factors were examined: (a) Task Number: dual, single, (b) Lag: 100, 300, 700 ms, and (c) Target Movement: stationary, displaced. Data were collapsed over certain factors for selected dependent measures when they were not relevant to the analysis. In particular, only Lag was examined for central light discrimination, and only Task Number and Lag were examined for pointing initiation time. Near, middle, and far positions were included in the analysis, as the correct final pointing position could have been any one of the three target positions. Additionally, in the dual-task condition, trials were included if participants correctly responded to the light duration target. Chapter 5 114 Where appropriate, the data from the current experiment were combined with Experiment 5 data (same displays but direct pointing) and an A N O V A was run with Experiment (5, 6) as a between-participants factor and the same within-participants as described in above. Results The primary findings were that (a) pointing initiation time was greatly elevated in this experiment, and (b) pointing movement time, accuracy, and kinematics were still unaffected by the dual-task context, even when targets were displaced to a new target location. Light Duration Discrimination Mean errors for light duration judgments are shown in Figure 14A. As observed in Experiment 5, errors were higher at Lag 100 (16.5%) than at Lag 300 or 700 (7.5%, 7.8%, respectively), F(2, 22) = 15.56, p < .0001. This suggested that at the shortest Lag, participants shifted their attention to the pointing target when it appeared, as in Experiment 5. Again, given that chance performance on the light discrimination is 50%, there was a possibility that "correct" responses may have been false positives. Indeed, examination of the proportion of "short" responses indicated that participants were biased to report "short" at Lag 100 (63%), a bias that was gone by Lag 300 (51%) and Lag 700 (46%). However, when additional analyses were conducted for pointing measures using all trials (regardless of light duration performance), this made no difference to the pattern of results. For comparability with Experiments 1 through 5, pointing performance will only be presented for trials on which the light duration judgment was correct. Chapter 5 115 Figure 14: Experiment 6: Behavioural Measures, Spatial Remapping A: Light Discrimination £ LU C ' o IB a 3-1 •S 2 S .1 a o Dual -Task Central Light 100 300 Lag (ms) 700 Pointing 520 _ 500 n g 480 j | 460 F 440 J 420 ra 5 400 1 380 j '•= 360 I 340 7 320 -300 B: Initiation Time C: Movement Time Dual-Task S ing le-Task 100 300 Lag (ms) 700 650 625 600 E 575 H a > o = 550 a e 525 H 500 100 -# - Dual -Task Disp laced - O - Dual -Task Stationary -A- S ing le-Task Disp laced - A - S ing le-Task Stationary 300 Lag (ms) 700 D: Absolute Constant Error 10-, E § LU i 4 o 0. Dual -Task Disp laced - O - Dua l -Task Stationary -A- S ing le-Task Disp laced - A - S ing le-Task Stationary 100 300 Lag (ms) 700 Figure 15: Experiment 6: Kinematic Measures, Spatial Remapping A: Peak Velocity 1250 1225-„ 1200-n I 1175 J 1 1 5 0 | 1125 > 1100 S 1075 Q. 1050 1025 H 1000 235 -230 -(ms) 225 ->, 220 -'o o 215-a > 210 -ro a 0- 205-O 200 -E 195 -190-185-420 „ 410 (A S 400 > •3 390 o » 380 H 370 o t 360 a 1 350 a .1 3 4 0 330 320 -#- Dua l -Task Disp laced - O - Dua l -Task Stationary -A- S ing le -Task Disp laced - A - S ing le -Task Stationary 100 300 Lag (ms) 700 B: Time To Peak Velocity 100 300 Lag (ms) 700 C: Time after Peak Velocity Dual -Task Disp laced - O - Dual -Task Stationary -A- S ing le-Task Disp laced - A - S ing le-Task Stationary Dual-Task Disp laced - O - Dual -Task Stationary A - S ing le-Task Disp laced - A - S ing le-Task Stationary 100 300 700 Lag (ms) Chapter 5 117 Comparison of light identification performance to that in Experiment 5 revealed errors were lower overall in the present experiment by 8%, F(l, 21) = 14.19, p < .002. Experiment did not interact with any within-participant factors, all p's > .41. Pointing Performance Eye movements generally preceded hand movements. Across all trials, when a target displacement occurred, in occurred at an average of 48.1 ms after the eye started moving and 54.8 ms before the hand started moving. Thus, the eye started moving, the target jumped, and then the hand started moving. This meant that on most displaced target trials, the target moved before pointing was initiated. Initiation time. Mean pointing initiation time is shown in Figure 14B. There was a concurrency cost in initiation time, such that dual-task initiation time was higher than single-task initiation time by 100 ms at Lag 100, but this difference diminished with increasing lag until it was absent by Lag 700, Task Number x Lag, F(2, 22) = 56.55, p < .0001. Comparison of initiation time performance to that in Experiment 5 revealed initiation time was higher overall in the present experiment by 78 ms, F(l/ 21) = 13.62, p < .002. Experiment did not interact with any within-participant factors, all p's > .1. Movement time. Mean pointing movement time is shown in Figure 14C. Movement time was 11 ms longer for displaced than for stationary targets, F(l , 11) = 10.88, p < .008. There was a tendency for dual-task movement time to be faster than single-task movement time, but this difference varied unsystematically across lag and final pointing position, Task Number X Lag x Final Pointing Position, F(4, 44) = 2.60, p < .05. Comparison of movement time performance to that in Experiment 5 revealed that movement time was longer overall in the present experiment by 125 ms, F(l, 21) = Chapter 5 118 6.51, p < .02. Additionally, movement time did not vary across lag in the present experiment, whereas it increased with increasing lag in Experiment 5, Lag x Experiment, F(2, 42) = 11.19, p < .0001. Experiment did not interact with any other within-participant factors, all p's > .1. Error. Mean absolute constant error is shown in Figure 14D. Participants were quite accurate at pointing to the final pointing position, such that they pointed to the location specified by each remapping rule, as opposed to the location of the target light. Errors decreased with increasing Lag, F(2, 22) = 7.13, p < .005. Additionally, there was a trend for errors to be greater for displaced targets than stationary targets, F(l, 11) = 3.77, P < .08, but the mean difference was less than .5 mm in magnitude. There were no effects of Task Number on absolute constant error, nor did it interact with any other factors (all p's > .28). Comparison of absolute constant error to that in Experiment 5 revealed a trend for absolute constant error to be lower in the present experiment by 1.0 mm, F(l, 21) = 4.10, p < .06. Additionally, there was a trend for the absolute constant error difference between stationary and displaced targets to be lower in the present experiment (.2 mm) than in Experiment 5 (1.1 mm), Target Movement x Experiment, F(l , 21) = 3.48, p <. 08. Experiment did not interact with any other within-participant factors, all p's > .1. Analysis of variable error did not show any significant effects, all p's > .09. Peak velocity. PV is shown in Figure 15A. There were no systematic or significant effects of Task Number or Lag, all p's > .09. Additional analyses of PV examined the effects of the cued target location that participants were supposed to point to. Results indicated that PV reflected the position of the target, rather than of the location cue. For instance, of the location cue appeared Chapter 5 119 in the far position, PV reflected planning of movements to the near position, which was where participants were instructed to point to. Time to peak velocity. TtPV is shown in Figure 15B. There was no effect of Task Number or Lag on TtPV, all p's > .13. Time after peak velocity. TaPV is shown in Figure 15C. TaPV was longer for displaced than for stationary targets, F(l, 11) = 10.86, p < .008,. This difference between displaced and stationary targets fluctuated unsystematically with Lag and Task Number, F(2, 22) = 4.16, p < .04. Discussion Even when the visually guided action was designed to recruit greater ventral stream processes with the introduction of more complex stimulus-response mapping rules, increased dual-task interference in both the planning and execution phases of pointing did not occur. With respect to the findings regarding the planning stage, this experiment showed that relative to Experiment 5 (direct pointing with kinematic measures) pointing initiation time was elevated for both dual- and single-task contexts by approximately 80 ms overall (compare Figures 9B and 12B). However, the magnitude of the concurrency cost was quite similar to that observed in Experiment 5, with a 100 ms dual-task cost at Lag 100 and no cost at Lag 700 for both experiments. Thus, increasing the complexity of the spatial mapping caused a prolongation of the planning phase of movement, but there was no exaggerated dual-task interference in this condition. The light discrimination task interfered with ventral planning as much in the spatial remapping condition (present experiment) as it did in the direct pointing condition (Experiment 5). This finding was unexpected from the perspective of most prior dual Chapter 5 120 task research, as more complex tasks are generally more prone to attentional interference. It may have been the case that the remapping rule for pointing did not add sufficient complexity to the pointing task to be observed as greater dual-task interference at Lag 100. As observed in Experiment 5, there were also no dual-task interference effects on any measures of execution in the present experiment. Both pointing movement time and error were unaffected by performance of the central light duration judgment, even when the target changed location. Additionally, kinematic measures of pointing were unaffected by the dual-task context. A reason for the lack of execution interference may have been that the blue target marker could have been selected as the action target before the initiation of pointing. Preliminary analyses revealed that eye initiation time for the present experiment was at the same levels as for Experiment 5, whereas pointing initiation time in the present experiment was elevated relative to performance in Experiment 5. This meant that participants may have been able to select the target location based on the remapping rule and then only initiate pointing when the action was fully planned. This suggests that the dorsal stream controls action execution even when the target is indicated by a symbolic cue that should rely even more on putatively ventral stream processes, compared to a situation where only a single target needs to be selected from the environment. For the case where results suggested that the dorsal stream controlled execution after complex planning, it was predicted that the target displacements during saccades would require planning to a new target location. Thus, interference should be seen in the action execution with displaced target location cues. But a consequence of the elevation in pointing initiation time without an elevation in eye initiation time was that Chapter 5 121 for the majority (80%) of displaced target trials, the displacement was triggered before the hand started moving, which meant that the displacement was triggered during the planning phase of the action. This allowed for the possibility that the new target location could be incorporated into the movement plan before execution. Future versions of this experiment should tie the target displacement to the onset of pointing so that participants cannot select the new target location before pointing onset. Overall, the results of this experiment indicate that even with a more complex stimulus-response mapping that determines where participants should point, as long as the target is in view and the action has been fully planned, then execution will rely on dorsal processes when participants can directly act upon the target. This suggests that in experiments where the action target is indicated by a symbolic cue, such as the colour- and shape-defined cues that were used in experiments supporting the Visual Attention Model (Schneider, 1995), once the target had been selected using ventral stream processes, execution was controlled at least in part by the dorsal stream. The next two experiments that follow are attempts to make action execution rely on ventral processes by displacing the action from the target, either spatially (acting in a space displaced from the target) or temporally (removing the target before participants can point to it). Chapter 6 122 CHAPTER 6: The dorsal stream is specialized for guiding direct actions in three-dimensional space (Milner & Goodale, 1995). In Experiments 1-5, participants were required to localize the target directly by pointing to a continuously visible, onscreen target in order to maximize the possibility that the action execution tapped the dorsal stream. Even in Experiment 6, participants were pointing directly to a visible target, which may have caused execution to rely primarily on dorsal processes. However, as the motor action becomes more removed from the target, such as by spatially displacing the action from the target, or by introducing a delay between target presentation and action (e.g., Westwood, Heath, & Roy, 2001), the action should become less controlled by the online processes of the dorsal stream and should be more influenced by conscious processes of the ventral stream. Thus, the hypothesis that follows is that if both tasks of central letter identification and peripheral pointing to the target are to rely more upon ventral stream processes, then there should be dual-task interference in both the planning and execution phases of pointing. Experiment 7: Comparing direct and indirect pointing methods One way to make pointing execution rely more heavily upon ventral stream processes is to spatially displace the action from the target. Liu, Healey, and Enns (2003) showed that in a dual-task context combining detection and localization tasks to a visual target, indirect localization (ventral, making a spatially mapped keyboard response) showed an impairment following detection (ventral), but direct localization (dorsal, pointing directly to the target location onscreen) did not. This implied that direct localization of the target did not compete for access with detection to the same attention-limited mechanisms, but indirect localization did. Chapter 6 123 If the model currently proposed for dual-task ventral and dorsal stream functioning is correct, then it should be possible to observe dual-task costs between letter identification and pointing execution by requiring participants to indicate the target location with an indirect action that relies more upon ventral stream processing. Thus, the present experiment compared the direct pointing method of previous experiments with the indirect method of pointing to the target with a cursor guided by a computer mouse, in two groups of participants. Mouse pointing does not allow participants to directly act on the target, as it involves a spatial translation between the action effector (the hand) and the target (onscreen). Thus, it should tap the more conscious, cognitive resources of the ventral perception stream (e.g., Chang & Ro, 2005; Liu et al., 2003). The procedure for both groups was similar to Experiment 2, with the exception that only stationary pointing targets were tested and pointing targets were presented over a larger spatial range. The greater spatial extent of target locations made the range of target locations tested more comparable to those used in Experiment 4 (direct pointing with behavioral and kinematic measures). If indirect pointing does indeed rely more on ventral stream processes, then we should see more generalized costs of letter identification, in both the planning and execution of indirect pointing. Specifically, successful letter identification should not only interfere with the initiation of pointing, but it should also show interference in movement time and accuracy of pointing. On the other hand, if there is nothing fundamentally different in the processing that guides direct versus indirect pointing, the pattern of results for indirect pointing should mirror those of direct pointing, such that there is dual-task interference in planning, but not in any measures of execution. Differences in equipment between the touchscreen used for direct pointing and the desktop computer used for indirect pointing made it inappropriate to directly Chapter 6 124 compare the magnitude of differences in execution between the direct pointing and indirect mouse pointing conditions. However, such technical limitations should not have invalidated the critical comparison of single- and dual-task differences within each experiment. Methods Participants Participants were 25 right-handed undergraduate students (14 females, mean age 20.2 years) with normal or corrected-to-normal vision who participated in exchange for course credit at the University of British Columbia. 12 individuals participated in the direct pointing condition and 13 individuals participated in the indirect mousepointing condition. Apparatus In the direct pointing condition, participants made responses using the same touchscreen setup as described in Experiment 1. In the indirect pointing condition, participants were seated at a viewing distance of 57 cm from a 17" CRT display. Displays and data collection were controlled by an eMac G4 using the same operating system and Matlab software used in the direct pointing condition. The eMac was placed upon a desk and mouse pointing was made with a Macintosh USB optical mouse placed on the desk surface in front of the participant. Mouse tracking acceleration was disabled ("mouse tracking" was set to the slowest speed possible) so that there was a 1:1 correspondence between the distance the mouse was moved on the desk and the distance that the mouse pointer moved onscreen. The mouse pointer was a black-and-white circle subtending 0.6° that had the following appearance: . For conditions where letter identification was required, Chapter 6 125 participants had a keyboard placed on the desk and were able to input letters with the left hand. The optical mouse does not translate fast, continuous movements across the screen when mouse tracking is set at a 1:1 correspondence for hand movementmouse pointer movement. However, this correspondence was desirable in order to have participants moving the pointing device (hand or mouse) the same physical distance across direct and indirect pointing conditions. The drawback was that on some trials, participants reported a tendency for the mouse pointer to halt partway through the movement. Thus, participants occasionally reported adopting the strategy of slowing down the movement, so that the mouse pointer would track their movements on a 1:1 basis, or they would pick up the mouse and reinitiate the movement. This had the effect of lengthening their movement times across all conditions. However, there was no reason to believe that this would add systematically different times to dual- and single-task conditions, but would instead add a random elevation to movement times for both conditions. Additionally, because there is a speed-accuracy trade-off for movement times, it might be expected that accuracy overall might be better in the mouse pointing group if movement times were elevated overall. Thus, movement time and accuracy would not be comparable between direct and indirect groups, but single- and dual-task movement time would still be comparable within groups. Between-group comparisons of letter identification and initiation time should be unaffected by the lengthening in indirect movement time. Stimuli Response stimuli were the same as in Experiment 2 with the exception that the central stream items subtended 0.6° (height) by 0.5° (width), and letter targets were Chapter 6 126 randomly selected from the letters of the alphabet, with the exception that I, O, Q, W, and Z were not used. Temporal lags between the central letter and peripheral target were -700, -100, 100, and 700 ms, as in Experiments 3 and 4. Additionally, to test targets over a wider range of eccentricities, peripheral targets could appear at positions 11,14, or 17° (or cm) to the right of fixation. These positions will be referred to as the near (11 cm), middle (14 cm) and far (17 cm) positions for Experiment 5. These peripheral targets were all stationary, such that when the target that appeared at one of the three locations, it remained there for the rest of the trial. Procedure Participants initiated each trial by gazing at fixation and touching the home position with the stylus (direct group) or clicking it with the mouse pointer (indirect group). Since the letter target would appear unpredictably either before or after the pointing target, participants were instructed to maintain their eyes at fixation during trial presentation for both direct and indirect pointing. Otherwise, if participants were looking at the pointing target before the letter target appeared, they would miss the subsequent letter target. For each of the dual- or single-task conditions, participants performed 20 practice trials or enough trials until they were fluent with the task, followed by 180 experimental trials. The entire experiment took about 45 minutes to complete. Data Analyses Trial exclusion criteria were the same as those defined in Experiment 1. For the direct pointing condition, 3.5% of trials were excluded from initial analysis. In the indirect pointing condition, 16.9% of trials were excluded. This higher number exclusions in the indirect condition arose because slight movements in the mouse at Chapter 6 127 fixation before the pointing target appeared were registered as initiation in pointing; indeed, examination of exclusions indicated that 16.0% of exclusions were because of anticipatory mouse movements, and these anticipations were equally likely in both single- and dual-task conditions. Where appropriate, dependent measures were analyzed with ANOVAs using a between-participants factor of Action Type (direct, indirect) and within-participants factors of (a) Task Number: dual, single, and (b) Lag: -700, -100, 100, 700 ms. Data were collapsed over certain within-participants factors for selected dependent measures when they were not relevant to the analysis. In particular, only Lag was examined for central letter identification. Additionally, as pointing movement time and error may not have been comparable across Action Type because of prolonged movement times in indirect mouse pointing, these results were analyzed separately for each Action Type. To maximize the possibility that any effects in pointing error are observed for trials on which participants were actually attending to the letter, dual-task results are reported for trials on which letter identification was successful. Results The primary findings were that (a) results for direct pointing mirrored the findings of previous experiments such that dual-task initiation time was impaired by temporal proximity between targets, dual-task movement time was facilitated overall, and dual-task error was unaffected, (b) indirect pointing initiation time showed an even greater concurrency cost in the dual-task context, (c) indirect pointing movement time did not differ between dual- and single-task contexts, and (d) indirect pointing error was impaired in the dual-task context. Chapter 6 128 Figure 16: Experiment 7 results: Direct pointing versus indirect mousepointing A: Letter Identification Direct Indirect * .5 % -2 • 1 1 C o a o £ o Dual-Task Central Letter -700 -100 100 700 Lag (ms) in c .9 .4 2 0) o .2 _i **-o c •I-1 o Q. o a 0 Dual-Task Central Letter -700 -100 100 Lag (ms) 700 Pointing B: Initiation Time Direct 900 • | 800 • £ 700-o 600 -500-= 400-o Q. 300-200 -Dual -Task S ing le-Task -700 -100 100 Lag (ms) 700 Indirect 900 -i | 800 700 H o 600 .2 % 500 • | 400 o °~ 300 200 -700 Dual -Task S ing le-Task -100 100 Lag (ms) 700 Chapter 6 129 C: Movement Time Direct 350 _ 325 <A E, 300 u E 275 - 250 a £ 225 v o 200 S O) 175 c ? 150 o °- 125 100 Dual -Task S ing le -Task -700 -100 100 Lag (ms) 700 Indirect 950 -925 -Hi E. 900 -0) £ 875 -850-§ E 825 -Qi > O 800 -S at 775 -c c 750 -o Q- 725 • 700 -Dual -Task S ing le -Task -700 -100 100 Lag (ms) 700 D: Absolute Constant Error Direct Indirect 10 E E £ 6 i 4 c o 0. Dual -Task S ing le -Task -700 -100 100 Lag (ms) 700 10 5 6 LU G l •S 4 c o 0. -700 -100 100 Lag (ms) Dual -Task S ing le -Task 700 Chapter 6 130 Letter Identification Performance Mean letter identification errors are shown in Figure 16A. Participants were able to successfully attend to the central letter identification task, with average errors at 4.9% in the direct pointing group and 4.2% in the indirect pointing group. While errors fluctuated by Lag for each action type, Action Type x Lag, F(3, 69) = 3.44, p < .03, there was no systematic increase or decrease in errors as Lag increased for either the direct or indirect pointing group. Pointing Performance Initiation time. Pointing initiation time, shown in Figure 16B, exhibited a concurrency cost for both direct and indirect pointing conditions, but the pattern varied in magnitude between direct and indirect pointing conditions, Task Number x Lag x Action Type, F(3, 69) = 3.66, p < .02. Specifically, the dual-task cost was greater for the indirect pointing group. At Lag -700, in the direct pointing group, dual-task initiation time was elevated by 242 ms from single-task initiation time, but in the indirect pointing group, dual-task initiation time was elevated by 443 ms from single-task initiation time. By Lag 700, these dual-task costs had disappeared for both groups. Notably, single-task initiation times were quite similar between the direct (323 ms) and the indirect (307 ms) pointing groups, and simple effects confirmed that single-task performance did not differ between pointing groups, p > .44. Movement time. Mean movement time is shown in Figure 16C. Movement times were not comparable between direct and indirect action groups because the difficulty in translating optical mouse movement lead to greatly elevated movement times in the indirect pointing group. Thus, movement time was analyzed separately for each action group using the within-participants factors of Task Number and Lag. Chapter 6 131 In the direct action group, dual-task movement time was faster than single-task movement time, F(l, 11) = 5.81, p < .04. Additionally, there was a trend for the dual-task advantage to increase with increases in lag, Task Number x Lag, F(3, 33) = 2.62, p < .07. This was similar to the increase in dual-task advantage observed in Experiment 3 where the design was similar with the addition of unexpected target displacements. A n additional effect in the direct action group showed that movement time decreased with increasing Lag, F(3, 33) = 9.96, p < .0001. In the indirect action group, there was no significant effect of Task Number on movement time, nor did Task Number interact with any other factors (all p's > .15). However, similar to the direct action group, movement time in the indirect action group did decrease with increasing Lag, F(3, 36) = 5.34, p < .004. Errors. Mean absolute constant error is shown in Figure 16D. Movement time is expected to affect errors overall in a speed-accuracy tradeoff, such that the longer the movement time, the more accurate the movement. Because movement times were so elevated in the indirect relative to the direct pointing condition, it was expected that this would contribute to accuracy being better in the indirect pointing overall, perhaps even to the point where it would attenuate any dual-task effects that might exist in accuracy. Thus, because the magnitude of pointing errors could not be reliably compared between Action Type, they were analyzed in separate repeated-measures ANOVAs for each group using the same factors as those in the movement time analyses. In the direct pointing condition, there was no effect of Task Number or Lag on absolute constant error, nor did these factors interact (p's > .13). Examination of variable error showed that for direct pointing, dual-task pointing was actually more consistent than single-task pointing, particularly at Lags -100 and 100, Task Number x Lag, F(3, 33) = 3.12,p<.01. Chapter 6 132 In the indirect pointing condition, dual-task errors were worse than single task-errors by 1.4 mm, F(l , 12) = 6.08, p < .03. There was no effect of Lag on indirect pointing error (p's > .22). Examination of variable error showed no effects of Task Number or Lag on pointing consistency (p's > .19). Discussion The results for this experiment showed that the effects for the direct pointing group were quite similar to the results observed in Experiments 2 and 3. Specifically, there was a dual-task cost in pointing initiation time that decreased as the temporal lag between target presentations increased, there was a general facilitation of dual-task set on pointing movement time, and pointing errors were not impaired in the dual-task context. In contrast, results for the indirect pointing group differed from previous experiments on every key measure. Specifically, the dual-task pointing initiation cost was significantly greater for indirect than for direct pointing, there was no significant difference in dual- and single-task movement time, and there was a dual-task cost in pointing errors. With respect to the greater dual-task initiation time cost for the indirect group, the cost cannot be attributed to greater difficulty in planning indirect movements in general; this possibility is ruled out by the fact that there are no significant differences in single-task initiation time between direct and indirect pointing groups. Instead, the results suggest that planning of indirect action shares even more resources with letter identification than direct action. This might be due to the extra spatial translation needed to map the mouse pointer to the hand position, whereas direct pointing does not require such a translation. Note that in the indirect pointing condition, average Chapter 6 133 initiation time at Lag -700 was 761 ms, which meant that the mousepointing was mostly initiated after letter presentation. Thus, participants generally had difficulty completing planning and initiating pointing before the target letter was presented. Taking into account the finding from Experiments 3 and 4 that preparation for and performance of letter identification prolongs and interferes with planning, rather than just initiation, this suggests an even greater prolongation of planning in the indirect pointing case. The other important finding was that there was a dual-task set cost in indirect pointing errors. This cost is consistent with the prediction that indirect pointing requires greater ventral stream visual processing, and thus will have a greater opportunity for interference from performance of letter identification. The fact that the cost was not lag-dependent (even when initiation time costs are lag-dependent) suggests that the dual-task interference on execution is quite general. This may be a result of the parameters for indirect movement execution being less accurately specified when participants are prepared to respond to the letter identification in that same context. Additionally, during execution, being set up to perform the letter identification may interfere with the ventral stream's ability to update indirect action to the target. In general, the results of the present experiment indicated that when the action task was designed to rely upon ventral visual processes by localizing a target through a mousepointer, the costs in action planning were even greater from a perceptual identification. Additionally, costs were now seen in action execution across all lags. This supports the idea that both the identification and action task now relied on the same mechanisms. Chapter 6 134 Experiment 8: Memory-based pointing The next experiment was designed to further test the prediction that a ventral identification task should interfere with pointing if the action is made to rely more heavily upon the conscious processes of the ventral stream. The method employed was to base the pointing upon memory of the target location. The dorsal and ventral streams differ in their memory for target locations (Goodale, Jakobson & Keillor, 1994; Milner & Goodale, 1995). The dorsal stream has little to no memory for the coordinates of a goal object. As an individual moves through the environment, the coordinates of the target location must be continually updated to effect visuomotor control. Because of this constant updating of coordinates, there is no need to retain memory of coordinates for past actions, whether or not they are executed. The positional memory of the dorsal system is estimated to range from 0 to 2 seconds (Bridgeman, Gemmer, Forsman, & Huemer, 2000; Elliott & Madalena, 1987; Goodale et al., 1994; Westwood, Heath, & Roy, 2000, 2001). The ventral stream, however, requires a memory of object attributes on a longer timescale in order to facilitate identification and recognition of objects. With respect to location, the ventral stream codes position of objects relative to the environment, rather than with respect to the observer. When there is a delay imposed between a view of the target and the action, movements are programmed using a stored representation of the target in relation to the environment. Generally, the suggestion is that the visuomotor mechanisms of the dorsal stream are engaged only when the action is required, and only if the target is visible at the time or has only recently disappeared. Otherwise, action is based on the perceptual representations of the ventral stream. It has been frequently shown that actions toward remembered targets are under control of the ventral system, whereas actions toward continuously visible targets are Chapter 6 135 under control of the dorsal system (e.g., Hu, Eagleson, & Goodale, 1999; Westwood, Chapman, & Roy, 2000; Westwood & Goodale, 2003; Westwood, Heath & Roy, 2000, 2001). In comparison to actions made to a visible target, actions made to a target that has been occluded before responses can be made are shown to have lower accuracy, greater variability, and slower movement times. These deficits are generally exaggerated when a delay is introduced between vision of the target and when participants are allowed to initiate movements (e.g., Mclntyre, Stratta, & Lacquaniti, 1997; Westwood et al.). According to the model of task interference that has been emerging from the previous experiments, if memory-based pointing is dependent on ventral stream processes, then dual-task costs in execution should be observed, in addition to the typical dual-task costs in pointing initiation that have been consistently observed thus far. Thus, the current experiment examined the consequences of dual-task pointing when it was performed in a memory-based condition (target was erased after 100 ms, before pointing could be initiated), rather than the full-vision conditions (target remains continuously visible during pointing) that had been tested up until this point. A second factor that was examined was the consequence of immediate versus delayed pointing relative to target onset. The latter condition should create an even greater memory interval between target presentation and pointing. The design of the experiment was similar to Experiment 2 where the pointing target was presented at positive lags of 100, 300, or 700 ms after presentation of the central letter target. However, the pointing target was only onscreen for 100 ms. Participants were allowed to initiate pointing after presentation of an auditory cue that was either presented simultaneous with pointing target onset (immediate pointing) or 2500 ms after pointing target onset (delayed pointing). Chapter 6 136 It was expected that making the target visible for only a brief period would lead to increased movement time and pointing errors because the target was not available to fully plan the movement. Thus, planning would have to be based partly upon memory of the target location. Additionally, the delayed pointing condition was expected to further increase pointing errors by making the movement rely more exclusively upon the memory-dependent processes of the ventral stream. Most importantly, these effects on errors were expected to be amplified in the dual-task context, because perceptual letter identification was expected tap the same mechanisms as those used for memory-based pointing execution. Methods Participants Participants were 12 right-handed students (7 females, mean age 29.2 years) with normal or corrected-to-normal vision who voluntarily participated at the University of British Columbia. Apparatus, Stimuli, and Procedure The design was similar to that of Experiment 2 with the exception of (a) changes to position and duration of the pointing target, (b) the requirement to maintain fixation throughout the trial, and (c) the introduction of a immediate and delayed pointing conditions that restricted whether participants could initiate movements when the pointing target was onset or only after a delay. To increase the memory demands, the spatial uncertainty of the pointing target location was increased. Pointing targets were the same dimensions as in Experiment 2, but target locations were randomly chosen to fall between 9° and 15° (or cm) to the right of visual fixation. Additionally, uncertainty was introduced in the vertical axis, such Chapter 6 137 that pointing target locations were randomly positioned between 1.4° above and 1.4° below the horizontal-axis position of visual fixation. A l l pointing targets remained stationary, such that they appeared for 100 ms in one position before being erased from the screen. A n illustration of the timeline of events for a trial is shown in Figure 17. The trial sequence for presentation of the central item stream, the central letter target, and the peripheral pointing target were the same as described in Experiment 2. Temporal lags between the central letter and peripheral target were 100, 300, and 700 ms. However, auditory cues (500 Hz tone presented for 50 ms) now signaled when participants were allowed to initiate pointing to the peripheral target. In the immediate pointing condition, the tone was presented simultaneously with the onset of the peripheral pointing target. In the delayed pointing condition, the tone was presented 2500 ms after the onset of the peripheral pointing target. For both conditions, if participants initiated pointing before the onset of the tone, a red " X " immediately appeared above fixation and a 220 Hz error tone was presented for 500 ms. Participants were also instructed to maintain their eyes at fixation during trial presentation. A digital video camera was used to monitor eye movements during the experiment. There were four blocks of trials (dual-task delay, dual-task no delay, single-task delay, single-task no delay) with 96 trials in each block. Order of blocks was counterbalanced across participants. Instructions and practice trials were given before the beginning of each block. The entire experiment took approximately one hour. Chapter 6 138 Letter Target Pointing Target Visual Targets 100 ms 100, 300, or 700 ms Movement Cue Immediate Pointing Delayed Pointing 2500 ms Figure 17. Temporal sequence of events for Experiment 8 Participants are given an auditory cue to initiate pointing movement either simultaneously with onset of the pointing target (immediate pointing) or 2500 ms after the onset of the pointing target (delayed pointing). Data Analyses Trials were filtered using the criteria defined in Experiment 1, with the additional criteria that trials were excluded if pointing movements were initiated <100 ms after the onset of the tone signaling that pointing could be initiated. This resulted in 2.6% of trials being excluded from initial analyses. Repeated-measures ANOVAs were used to analyze the dependent measures. Four within-participants factors were examined: (a) Task Number: dual, single, (b) Pointing Delay: immediate, delayed, and (c) Lag: 100, 300, 700 ms. For the dependent measure of central letter performance, only Lag and Pointing Condition were examined. Pointing data are presented for trials on which letter identification was correct. Chapter 6 139 Results The primary findings were that (a) pointing initiation time was impaired by temporal proximity between the letter and pointing target, particularly for the immediate pointing condition, (b) dual-task pointing movement time was facilitated when targets could be pointed to immediately, but not after a delay, (c) pointing movement time was slower in the delayed pointing condition, and (c) dual-task pointing accuracy was impaired in both the immediate and delayed pointing conditions. Letter Identification Performance Mean letter identification is shown in Figure 18A for immediate and delayed pointing conditions. Overall, letter identification performance was quite good (4.5%). There was a trend for errors to be lower at Lag 300 than Lag 100 or 700, F(2, 22) = 3.29, p < .06. There was no effect of Pointing Delay on letter identification, all pis > .22. Pointing Performance Initiation time. Mean pointing initiation time is shown in Figure 18B. There was a concurrency cost in dual-task initiation time (i.e., dual-task elevation that diminishes as Lag increases), but it only held for the immediate pointing condition, as shown by a Task Number x Lag x Pointing Delay interaction, F(2, 22) = 5.56, p < .02. Separate analyses of the immediate and delayed results confirmed that the concurrency cost was only present for the immediate pointing condition, Task Number x Lag, F(2, 22) = 20.59, p <.0001, but not for the delayed pointing condition, Task Number x Lag, F(2, 22) = 1.75, p > .19. There was a trend, however, for delayed pointing initiation time to be elevated in the dual-task condition, F(l , 11) = 4.71, p < .06. Chapter 6 Figure 18. Experiment 8 results: Memory-based pointing in immediate and delayed conditions A: Letter Identification Immediate Delayed 3 -2-o a o Dual -Task Centra l Letter 100 300 Lag (ms) 700 100 300 Lag (ms) 700 Pointing B: Initiation Time Immediate Delayed 480 i 280 J • • • • • — 100 300 . 700 100 300 700 Lag (ms) Lag (ms) Chapter 6 141 C: Movement Time Immediate Delayed Dual -Task S ing le -Task 100 300 Lag (ms) 700 100 300 Lag (ms) 700 D: Absolute Constant Error Immediate Delayed 2 2 ] 20 H 2-0-I , . , :—. . ^-100 300 700 100 300 700 Lag (ms) Lag (ms) Chapter 6 142 There was a general tendency for dual-task initiation time to be consistently slower than single-task initiation time, F(l, 11) = 18.59, p < .002, and for initiation time to decrease with increasing Lag, particularly for immediate pointing, F(2, 22) = 14.10, p < .0001. Movement time. Mean movement time is shown in Figure 18C. There was a trend for dual-task movement time to be faster than single-task movement time, but only in the immediate pointing condition, Task Number x Delay, F(l, 11) = 4.00, p < .08. Simple interactions confirmed that the trend for the dual-task movement time advantage was only present for the immediate pointing conditions, Task Number, F(l, 11) = 4.74, p <.06, but not for the delayed pointing condition, Task Number, F(l, 11) = .13, p > .72. An additional effect showed that immediate pointing had shorter movement times than delayed pointing, F(l, 11) = 6.47, p < .03. Error. Mean absolute constant error is shown in Figure 18D. Preliminary analyses of constant error indicated that participants had a general tendency to undershoot the targets, particularly for delayed pointing. This is consistent with a tendency for observers to mislocalize remembered targets towards fixation (e.g., Sheth & Shimojo, 2001). Analysis of absolute constant error indicated that errors were greater for dual tasks (15.5 mm) than single tasks (12.0 mm), F(i/11) = 6.92, p < .03. Delay also had a effect on error, such that delayed pointing errors (16.1 mm) were greater than immediate pointing errors (11.4 mm), F(l, 11) = 4.86, p < .05. However, the magnitude of the dual-task error cost was not greater for delayed pointing compared to immediate pointing, F(l, 11) < 1. Chapter 6 143 While error fluctuated by Lag, the effects were not in the direction of a systematic increase or decrease of error with increases in Lag, F(2, 22) = 4.80, p < .02. Analysis of variable error mirrored the delayed pointing cost, such that pointing was more variable for delayed pointing (11.0 mm) than for immediate trials (7.3 mm), F(l/ H) = 57.22, p<.0001. Discussion Immediate Pointing For immediate pointing, the results replicated previous experiments such that there was a concurrency cost in dual-task initiation time and facilitation for dual-task movement time. The primary difference, however, was the appearance of a dual-task set cost for pointing accuracy that did not vary by temporal lag between the two targets. There are a few possibilities as to why this dual-task set cost exists for immediate pointing. One possibility is that movement planning to briefly visible targets is more susceptible to interference from preparing for and processing letter identification in both immediate and delayed pointing conditions. The brief period during which the pointing target was visible meant that movement planning to the target location may not have been completed before the target disappeared, and thus planning was partly based upon memory of the target location. Indeed, there was a general decrease in accuracy for immediate pointing, even in the single-task context, relative to other experiments. Thus, when perceptual identification was required, it may have caused even greater interference in movement planning. This explanation assumes that interference carries into ventrally controlled execution as a result of interference during the planning stage of movement. However, this interference would have to be quite Chapter 6 144 general to the dual-task context, as it does not depend upon temporal lag between the identification and pointing target. A second possibility is that, in the immediate pointing condition, the memory demands for the letter identification or the pressure of the dual-task context interfered with memory for the target location during the execution of dual-task pointing, rather than simply interfering with planning. To distinguish between these two possibilities, that either planning or execution were interfered with by the letter identification, an experiment could be run in which the target remained onscreen longer to allow planning to run to completion. In that case, only execution would be likely to be based on memory. If there was no longer accuracy interference in the dual-task context, this would suggest that the currently observed interference came from interference during the planning phase. On the other hand, if there was still dual-task accuracy interference, this would suggest that the interference is occurring during the execution phase. A third possibility to explain the dual-task set cost in immediate pointing is that the faster movement time in the dual-task condition led to a speed-accuracy tradeoff in pointing. However, the fact that this dual-task movement time facilitation was observed in Experiments 3 and 4 without this cost in pointing accuracy suggests that this third possibility is unlikely. Overall, the fact that pointing error was subject to dual-task interference from ventral identification in the immediate pointing condition supports the idea that the ventral stream was involved in action execution. Delayed Pointing For delayed pointing, there was no concurrency cost for pointing initiation time, no dual-task movement time facilitation, and a dual-task cost in accuracy that was comparable in magnitude to the cost observed for immediate pointing. Chapter 6 145 With respect to the lack of concurrency cost in pointing initiation time, this was to be expected because by the time pointing was allowed to be initiated, 2500 ms has passed since the onset of the pointing target. This meant that the lags between central letter onset and the cue for pointing initiation were effectively 2600, 2800, and 3200 ms since the onset of the central letter target at Lags 100, 300, and 700, respectively. This should have been long enough for the central letter to have been identified in the dual-task context, as a dual-task cost in initiation time is usually absent by Lag 700 in immediate pointing conditions (Experiments 1-7). However, a trend for a dual-task set cost existed in delayed pointing initiation time. This suggests that there is still a generalized interference in planning that occurs when participants have to identify the central letter. This could be caused by heightened attentional requirements of preparing to respond to the letter or the ongoing need to hold an active representation of the letter identity in memory in the dual-task conditions. Another new effect that was observed in delayed pointing was the lack of dual-task movement time advantage compared to the advantage that was observed in immediate pointing. This suggests that in order for the dual-task movement time facilitation to occur to a briefly visible target, pointing must be allowed to be initiated as soon as possible after target presentation. Delayed pointing was less accurate than immediate pointing, which mirrors previous findings of increased error in memory-based, delayed action conditions (e.g., Westwood et al., 2001). Delayed pointing error also showed a dual-task set cost that did not differ in magnitude from the dual-task cost seen in immediate pointing. It was predicted that this cost should be greater in the delayed pointing relative to the immediate pointing, but this was not the case in the present data. It could be the case Chapter 6 146 that both immediate and delayed pointing execution rely equally upon the ventral stream, and are thus equally interfered with by the dual-task context. Generally, the results of the present experiment concurred with the interpretation that arose from the previous experiment, such that when pointing depends on ventral stream processing, either because of indirect spatial mapping (Experiment 7) or because of memory demands (current Experiment), it will be more likely to show interference in planning and execution when performed in conjunction with other ventral visual tasks. Chapter 7 147 CHAPTER 7: GENERAL DISCUSSION The purpose of the present set of experiments was to use the Perception-Action theory of Milner and Goodale (1995) as a novel guiding framework for exploring the attentional effects of performing dual tasks of visually guided perception and action. Previous research on dual-task limitations has repeatedly shown that when performing dual tasks of visual perception (e.g., identification), there are frequently costs in the ability to carry out both tasks as efficiently as one, particularly when the tasks are presented in rapid succession (e.g., Pashler, 1994; Shapiro, 2001). Previous studies in which the control of action has been paired with conscious report have come to a similar conclusion when each task refers to a separate object (Castiello, 1996; Deubel, Schneider, & Paprotta, 1998; Schiegg, Deubel, & Schneider, 2003). However, from a Perception-Action perspective, these studies do not tell the whole story. In particular, they do not test the consequences of simultaneously trying to perform a perception task with an action task requiring online control. According to the Perception-Action theory, perceptual tasks are controlled by the ventral stream. Action planning also involves the ventral stream as it is a cognitive activity that creates a representation of the target in relation to its environment. Action execution and online control, however, involve the dorsal stream, which transforms visual information into the precise target coordinates needed to act on the target. If perceptual identification and action planning are both ventral processes, and if there is a single mechanism that allows targets to be selected and processed for this stream, then perceptual identification and action planning should interfere with one another. At the same time, if action execution (including online control) is controlled by the dorsal stream, and if there is a separate attentional mechanism for the dorsal stream that enables target processing, then perceptual identification should not interfere with action Chapter 7 148 execution and online control. Alternatively, if there is a common attentional mechanism underlying both ventral and dorsal stream processing, then dual-task interference from perceptual identification should be manifested upon pointing at both planning and execution stages of the movement. To examine these predictions, perceptual identification, action planning, action execution, and online control were explored with a series of experiments that robustly demonstrated that a visual perception task consistently interfered with the planning, but not with the execution and online control, of a visually guided pointing task. Additional experiments showed that when pointing execution relied upon putatively ventral stream processes, execution also showed interference from the concurrent performance of a perception task. The following is a summary of the major findings of the experiments. 1. Performing a visual identification task results in a lag-dependent cost in the initiation time for pointing. When the pointing and identification targets are presented in close temporal proximity, there is a robust dual-task cost in initiation time that decreases as presentation of the pointing target is increasing delayed relative to the perceptual target (Experiments 1-8). Thus, participants cannot efficiently plan a pointing movement at the same time that they are processing a visual identification. The results suggest that the perceptual task competes with action planning for access to a common attentional mechanism that controls processing for the both tasks. 2. A correctly performed visual identification task does not lead to any impairment in pointing movement time relative to pointing in a single-task context (Experiments 1 - 8). Regardless of whether the pointing was initiated before, during, or after the presentation of the letter target, there is no interference in the movement time. The Chapter 7 149 only cost frequently seen in movement time is a tendency for movements to take longer when targets were displaced rather than stationary. Interestingly, there was an unexpected advantage for movement time observed in the dual-task context (Experiments 3, 4, 7, & 8); furthermore, this advantage was largely lag-independent. This suggests that the additional attentional demands of pointing in a dual-task context actually shorten movement times without a concomitant increase in pointing error. 3. Under normal instructions to carry out speeded pointing, a correctly performed visual identification task does not lead to any increases in pointing error when the pointing relies on the processes of the dorsal stream (Experiments 1-3, 5). Even when the target changes position at the onset of or during movement, there is no interference from performing the visual identification. As is the case with movement time, this lack of dual-task interference holds even when pointing is initiated before, during, or after the presentation of the letter target. In other words, the execution and the online control of pointing are unaffected by the attentional requirements of the ventral perception task. Examination of the movement kinematics during execution confirmed the lack of dual-task interference effects on a pointing that has been fully planned (Experiment 5). Even when the pointing is based on more complex target selection rules, execution is not interfered with (Experiment 6). This suggests that the mechanisms that allow for successful performance of visual identification do not overlap with those needed for movement execution. 4. When participants are under a deadline pressure to initiate pointing, and when the pointing target greatly precedes the identification target, the dual-task context leads to increases in pointing error for both stationary and displaced targets (Experiment 4). This suggests that costs in pointing execution will occur when there is insufficient Chapter 7 150 time to fully plan the pointing action in the dual-task context because of the deadline. Additionally, there are dual-task costs in pointing error to displaced targets across all temporal lags, reflecting a general cost in online control when participants are under the pressure of movement speed stress and a dual-task context. 5. When pointing relies on more putatively ventral stream functions requiring displaced spatial mapping (Experiment 7) or memory-based pointing (Experiment 8), there are dual-task costs in both the planning and execution of pointing. These results are in contrast to the finding that the execution of pointing is not interfered with when the task relies on direct, real-time control of the dorsal stream. This supports the idea that when both tasks rely on the ventral stream, there is interference in all aspects of pointing from the identification task. Overall, the present data show (a) dual-task interference between a perceptual task and action planning, (b) no interference of a perceptual task upon action execution under instructions merely to perform fast and accurate pointing, and (c) dual-task interference of a perceptual task upon action execution when the action is under extreme pressure or when it relies on ventral stream processes. These findings demonstrate a clear separation between the interference effects of a perception task on the planning versus the execution and online control of an action. Thus, the current body of experiments demonstrates that some aspects of visually guided action to one target can be performed concurrently, and without cost, with an attention-demanding perception task to a separate target. It is not because the pointing could be executed without evaluating the target location, because even when the target was displaced, the Chapter 7 151 execution of pointing could be carried out as quickly and as accurately under dual-task conditions as under single-task conditions. The current results also suggest that a perceptual task of either letter identification or temporal judgment will interfere with the planning phase of a visually guided pointing action, regardless of whether the action relies on more dorsal or ventral stream processing (Experiments 7 & 8). This supports the idea that planning a movement, in general, relies on perceptual processes that create a representation of objects and their relationship to the environment, and is therefore more susceptible to interference from other perceptual tasks that are supported by ventral stream processes. Theoretical Implications The following three subsections discuss the present results and the Perception-Action model in relation to theories of planning/execution, divided attention, and concurrent perception/ action that were covered in the introduction of this thesis. Relevance to existing research on the planning and execution of action The results of the body of experiments are supportive of the Perception-Action theory and its claim that some ventral and dorsal stream visual functions are independent, even in healthy human adults (Milner & Goodale, 1995). Specifically, it supports the idea that these streams are even independent in the mechanisms used for attentionally demanding tasks. The finding that letter identification interferes with action planning, but not with the execution and online modification of a planned limb action, can help to refine the theory. A recent alternative to the dual visual systems model (Glover, 2002, 2004) has highlighted the need for action planning and online control to be more clearly understood in the Perception-Action theory. The present Chapter 7 152 results do this by showing the clear dissociability of planning versus control of pointing with regard to an attention-demanding ventral stream task. The current experiments do not allow one to distinguish between the Perception-Action theory and the planning control model (Glover, 2004). Both models indicate that the planning of actions is more susceptible to cognitive influences, and both indicate that the control of action can be dynamically updated to changes in target locations, even after the action has been initiated. Where the models fundamentally differ is in where initial movement parameters (e.g., timing, velocity) are specified. In the Perception-Action theory, initial movement parameters are determined by the dorsal stream, whereas in the planning-control model, these parameters are determined by the IPL during the planning of the movement. In the planning-control model, the early phase of movement execution is determined by the planning of the perceptual system, and the control system takes over later in the movement. In the Perception-Action, model, the dorsal system is responsible for movement parameters right from the start of movement. To distinguish between these two alternatives using the current tasks, one could examine the effect of varying when pointing target displacements occur after movement onset. For target displacements that occur at movement onset, the planning-control model would predict that it should take measurable time for the online control system to take over and adjust movement to the new target location, whereas the Perception-Action model would predict that the dorsal system should rapidly adjust to the new target location. For target displacements that occur later in the movement, the planning-control model would predict that the control system would have taken over, and adjustments would happen much quicker relative to onset of the target displacement. The Perception-Action model, however, should not predict as large a difference in the Chapter 7 153 time it takes to adjust to a new target displacement, because the dorsal stream is responsible for specifying movement parameters both at the beginning and end of movement. Relation to existing theories on Divided Attention The current experimental results can be interpreted within each of the major theories of divided attention (bottleneck, resource, and crosstalk). A strict bottleneck theory of attention postulates that critical mental operations needed for target processing can only deal sequentially with multiple targets, regardless of which visual system controls responding to each target. Given that perceptual identification and action execution were shown to run concurrently in the present experiments, the results do not agree with such a model. Another possibility, however, is that there are separate bottlenecks for the ventral and dorsal streams. The results are consistent with at least a ventral stream bottleneck for identification and action planning. While concurrent dorsal tasks were not tested in this series of experiments, Diedrichsen et al.'s (2004) finding that concurrent bimanual pointing to stationary and displaced targets is largely independent suggests that the dorsal stream may not have a bottleneck in the execution of action. Resource theories hypothesize that there are one or more pools of limited attentional resources available for task processing. In this context, the current results indicate that dorsal processing does not share resources with the ventral stream. While there are limited resources for the ventral stream, as evidenced by the concurrency costs between identification and action planning, it is not as clear from the present evidence whether the independent functioning of the dorsal stream is a result of it having its own pool of resources, or whether it does not tap resources in the same way. Certainly, action execution does demand attention at some level, as shown by the evidence that Chapter 7 154 reaching can be perturbed by distractors (e.g., Tipper et al., 1992) and secondary tasks (e.g., Castiello, 1996). However, given that the control of action is largely automatic, it may be the case that action execution is not as attentionally demanding as perceptual tasks are. This issue will be revisited later in the discussion of "Remaining Questions and Future Work". The results can also be interpreted within the realm of crosstalk theories, which suggest that performance limitations are dependent upon the content of information being processed. The more similar the information, the more likely that interference would occur. Under this theory, the visual stream tapped in the task could be used as a metric of similarity, such that ventral stream tasks are more similar to one another than they are to dorsal stream tasks, and dorsal stream tasks are more similar to one another than they are to ventral stream tasks. However, within each stream, there should be a gradient of similarity as well, such that identical tasks will interfere most with one another, and interference will decrease with decreasing similarity. In the present data, letter identification produced an interference pattern on action planning that was similar to the interference pattern shown in a second letter identification (Experiment 1). While the two measures of letter identification and action planning are not directly comparable, given that two letter identifications are essentially the same type of task, the crosstalk model would predict much greater dual-task interference in this case than between letter identification and action planning. Thus, the results are not highly supportive of a crosstalk model of interference. Relation to existing theories of perception and action In the introduction to this thesis, three theories of perception and action were discussed: the premotor theory, the integrated competition hypothesis, and the visual attention model. The present data will now be considered for each theory in turn. Chapter 7 155 The premotor theory assumes there are multiple circuits that control attention, and that attention for a task is derived from the activity of pragmatic (dorsal) and semantic (ventral) maps without the involvement of other anatomical structures. According to the extended version of the premotor theory that includes perceptual tasks in addition to motor actions, the attentional mechanisms for the perceptual identification should occur in the ventral stream semantic maps that control identification, which are in the inferotemporal cortex. Attention for planning the pointing action, however, should activate separate spatial pragmatic maps in the parietal lobe that control pointing (e.g., Castiello & Paine, 2002). As the premotor theory does not address dual-task performance, it does not have a way to predict interference between these two maps when they are concurrently activated. It also does not explicitly address how attention is required during the ongoing execution of movement. However, if attention arising from pragmatic circuits is required on an ongoing basis during action execution, then the current results suggest that, indeed, sensorimotor and pragmatic circuits can run in parallel. The integrated competition hypothesis (Duncan, 1996) proposes that attention will emerge over a period of several hundred milliseconds and come to converge on a single object across multiple brain systems. This theory predicts that one could not efficiently deal with two separate objects at a time compared to dealing with only one object. While the theory itself is not designed to distinguish between the planning and execution stages of action, it would predict that as long as two objects were competing for attention, then there would be performance decrements. This pattern of results is seen in the current data for concurrent perceptual identification and action planning, but not for perceptual identification and action execution unless both tasks are within the purview of the ventral stream. As a key claim of the integrated competition Chapter 7 156 hypothesis is that selective attention will settle on one object at a time, regardless of the neural system involved, it does not allow for a way to predict why a ventral-stream action task would have its execution interfered with by identification of a separate object, but dorsal-stream action does not show this same interference. Thus, the reported results are not consistent with the predictions of the integrated competition hypothesis. The results of the current experiments cannot be satisfactorily dealt with by the Visual Attention Model either. The Visual Attention Model (Schneider, 1995) predicts that attentional selection is bound to one object at a time, regardless of whether an object is the target of an action or the focus of identification. For the execution of action, if the action target needs to be continuously monitored for movement success (such as when the target is displaced), then selective attention should be directed to the movement target. If selective attention is instead focused on the perceptual identification target, then we should see dual-task deficits in movement planning and in execution to displaced targets during concurrent performance of identification and action. However, this was not the case. Participants were clearly able to identify a letter without cost, even while they were successfully modifying pointing movements to a second object in response to an unpredictable change in its location. Additionally, as the Visual Attention Model does not elaborate a distinction between spatial-motor actions made directly toward the target and actions made in a more indirect manner (e.g., mousepointing, memory), it does not have a way of predicting the greater dual-task two-object interference when action relies more upon the ventral stream. Thus, the present results cannot be well accommodated by the Visual Attention Model either. Chapter 7 157 What accounts for the discrepancy between the evidence supporting a unitary limitation for concurrent ventral and dorsal tasks and the present evidence supporting independence in such concurrent tasks? As mentioned in the introduction, the bulk of the evidence in support of the unitary mechanism of attention involves actions cued symbolically by colour and shape (i.e., multiple stimuli available, target is designated from this stimuli set by symbolic cue), which should recruit more ventral stream processes. This may account for why evidence in support of Visual Attention Model for visually guided reaching finds an object- (or space-) based binding of attention, because even the execution of action activates ventral stream resources. The current experiments may have reduced the involvement of the ventral stream in planning and execution by having the target directly cued through its onset in the majority of the experiments. In this case, the coupling between perception and action execution has disappeared. On the other hand, in line with the idea that action planning is controlled by the ventral stream, it has been shown that dorsal stream responses can follow from a ventrally controlled cue (e.g., Bridgeman & Huemer, 1998). Indeed, in Experiment 6 with the spatial remapping rules for target selection, execution was not interfered with by the concurrent perception task. Thus, it should be further investigated whether and when the dorsal stream controls action execution when action planning and target selection have greater demands on the perceptual processes of the ventral stream. This question will be revisited in the following section. Remaining Questions & Directions for Future Work Having established that in the present experiments, ventral and dorsal stream tasks can function independently and without interference, we now turn to remaining questions and possibilities for future research raised by the current findings. These Chapter 7 158 questions involve (a) increasing the requirements of action planning, (b) the involvement of the dorsal stream before movement initiation, (c) attentional vs. nonattentional demands of action, (d) task priority, and (e) the generality of dorsal stream impermeability. Increasing the Requirements of Action Planning In the current set of experiments, with the exception of the spatial remapping condition (Experiment 6), only one pointing target was presented at a time when visually guided action was required. However, in the environment, we are frequently acting upon a target that is selected from amongst alternatives (e.g., pressing one elevator button out of eleven, grasping one apple out of many). This situation should require even greater action planning compared to actions planned to a single onset, as used in the present experiment. According the Perception-Action theory, target selection and action planning are part of the functioning of the ventral stream. The implication, then, is that once the target and response has been selected, then the dorsal stream controls the execution of movement. To further explore this idea, in experiments where a perceptual task is combined with an action task that first requires selection of a target amongst alternatives based on ventrally processed attributes (e.g., red target amongst green and blue targets, as used by Deubel et al. [1998]), we should observe even greater interference of perception on the action planning phase, but execution and online control should carry on unperturbed once planning is allowed to run to completion. On the other hand, if perceptual information for target selection is needed on an ongoing basis during action execution, then we should see dual-task interference again. Chapter 7 159 The involvement of the dorsal stream before movement initiation One of the claims of the Perception-Action model is that the dorsal stream is responsible for specifying the initial parameters of movement. Does this mean that in the current experiments, the action planning phase was actually reflective of dorsal programming of movement parameters, rather than a ventral planning of movement? Dorsal stream areas are not assumed to play a large role in action planning unless the target location has already been specified in advance (Goodale & Milner, 2004). However, if one were to assume that the planning phase in the current experiments primarily involved the dorsal stream's specification of the coordinates for action, then this would imply that the perceptual identification task was interfering with dorsal rather than ventral stream processes during initiation time. This, in turn, would be supportive of a unitary mechanism of attention for both streams, rather than the separate mechanisms that are currently postulated. This argument would also suggest that the reason that interference was not shown in dorsal action execution is because the perceptual identification interfered with dorsal action programming, and by the time execution was initiated, this unitary attention mechanism was freed up from the perceptual target processing. However, this argument for a unitary mechanism does not strongly hold, because the perceptual identification task did not interfere with dorsal action execution when the hand was in transit and adjusting to a new target location (Experiment 3). For this argument to hold, it would have to assume that action execution and online control is not attentionally demanding. This idea will be considered next. Attentional versus nonattentional demands of action In the present experiments, when action execution depends on the dorsal stream, it is impervious to interference from ventral stream tasks. The assumption thus far has Chapter 7 160 been that action execution is attentionally demanding, particularly when the action target is displaced. However, the current results do not fully tease apart whether the dorsal stream has its own attentional mechanisms, or whether it does not even use attention. As discussed in the introduction, the contents of dorsal stream processing are not readily available to consciousness. The idea that actions can be controlled without consciousness is intriguing because it has been proposed that consciousness may depend upon attention. Specifically, attention may be necessary in order for information to reach conscious awareness and be encoded in long-term memory (Allport, 1987; Posner, 1994). If it is the case that attention is necessary for consciousness, then it is possible that attention is only required for the conscious outcomes of the ventral stream, whereas the unconscious aspects of the action control may not require attention. Control of action has been described as being automatic. Automatic processing of stimuli is characterized as fast, not attentionally demanding, able to be completed in parallel with other stimulus processing, and nonvolitional in the sense that it is unavoidable. (Schneider & Shiffrin, 1977; Underwood & Everatt, 1996). Indeed, it has been shown that motor corrections to unexpected target displacements cannot be avoided even when participants are instructed to stop pointing in response to a target displacement (Pisella et al., 2000). Thus, if the dorsal stream does not require attention, then this may account for the finding that components of action controlled by the dorsal stream are immune to the effects of concurrent, attentionally demanding perception tasks. On the other hand, consciousness is not necessarily linked to attention. The immunity of dorsal-stream action to concurrent tasks does not have to imply that the dorsal stream does not require attention, but simply that the dorsal stream has separate Chapter 7 161 attentional mechanisms that are unavailable for consciousness and do not overlap with those required for the ventral stream. Thus, there are two possibilities why dorsal stream processing could be immune to perceptual-task interference; either dorsal processing does not demand attention, or this processing relies upon separate neural mechanisms. The possibility that the dorsal stream does not demand attention seems unlikely, given that action execution is interfered with by distractors that can be acted upon or that are presented during movement execution (e.g., Castiello, 1996, 2001). Given that action execution can be interfered with under these certain situations, the current data suggest that action does use attentional mechanisms that are independent from attention for perception. Also, if the dorsal stream does not demand attention, then concurrent dorsal stream action executions should not interfere with one another, as they are not tapping any limited attention mechanism. Diedrichsen et al. (2004) showed that reaching movements with the hand to one target showed slight trajectory deviations when the target for the other hand was displaced. However, this was the only measure that was significantly affected, as onset of movement adjustment, movement time, and error were unaffected. This suggests concurrent online control for two actions can operate with minimal interference, but interference nonetheless. Thus, while it is not definitively resolved as to what the attentional demands of dorsal action execution are, it seems likely that action does demand attention in some way. Task Priority Task priority was not explicitly manipulated in the current body of experiments. Participants were always encouraged to perform both tasks to the best of their ability so that no instructional priority was placed on one task over another. This differs from the Chapter 7 162 experiments conducted by Deubel, Schneider, and colleagues (e.g., Deubel et al., 1998) where priority was placed upon the action task over the perception task. It has not, however, been shown whether action execution would suffer if action is considered the secondary task. According to the emerging version of the Perception-Action model, if the streams do have access to independent attentional mechanisms, then assigning task priority to ventral perception task should still not lead to interference in action execution. Generality of Dorsal Stream Impermeability There are suggestions and evidence that the neural representations and circuits that control different types of visually guided action are effector specific (Milner & Goodale, 1995; Rizzolatti et al., 1985; Tipper et al., 1992). Additionally, different visually guided actions, such as pointing and grasping, are not equally interfered with by distractors; instead, interference appears to depend on the type of action required. Indeed, it has been suggested that pointing and grasping may be different at movement preplanning and kinematic levels (Carnahan, Goodale, Marteniuk, 1993; Robertson, Nico, & Hood, 1995, 1997), and that grasping may involve more online computation. Robertson et al. have also suggested that there is a different frame of reference and different attentional demands for pointing and grasping. If attention for dorsal stream tasks does not share mechanisms with ventral stream tasks, then even more complex action execution that requires more attention should not show interference from concurrent perception. On the other hand, if there is interference on more computationally intensive action tasks, this would suggest that the independence of the dorsal stream is limited to more simple tasks. In fact, interference in more complex tasks might suggest that tasks like reaching and pointing do not require attention. Chapter 7 163 Practical Implications Performing multiple tasks at once typically leads to performance decrements in one or both tasks. To enhance multitasking performance, one should try and design the environment to reduce interference between tasks. The results of the current body of experiments have implications for how multiple-task situations can be designed by outlining which systems can function in parallel without costs. This is particularly topical for situations where speed and accuracy of responding is important, such as in time and safety critical contexts like driving and aviation. In general, the results show that when tasks of visual identification are demanded in conjunction with visually guided action, there will be a specific pattern of interference upon the action. The time taken to plan the action will be consistently impaired if it overlaps with the processes of identification, but the time and accuracy of executing the action is not impaired. For example, when driving, this implies that when attending to the road for pedestrians and other hazards, this will not interfere with the actual execution of movement to the dashboard controls (e.g., to turn the radio on), nor will the execution of the movement interfere with the ability to visually attend to the road. On the other hand, the results highlight the fact that there will consistently be interference between visual identification and the plan to act upon the controls. Ideally, for time- and safety-critical situations, one should attempt to separate the two processes of identification and action planning as much as possible. However, insofar as action execution needs to be preceded by action planning, designers must be aware that it will be difficult to avoid this level of interference between perception and action. In terms of avoiding dual-task interference, another implication is that direct action upon the target is superior to indirect action (displaced from the target, memory based). Thus, when users need to visually identify targets, interfaces should require Chapter 7 164 direct action upon the controls, rather than an indirect action such as that controlled by a mouse pointer or another type of remote control. In general, the current results suggest that in environments that require multiple-task responding, performance costs can be reduced by designing tasks such that they would tap the ventral and dorsal streams, instead of tapping the same stream in the task requirements. Final Remarks The findings of this dissertation lend further support for the case of independent functioning for dorsal and ventral streams. Previous evidence has shown separability in the two streams using neuropsychological syndromes, neurophysiological data, and behavioural dissociations. The current evidence adds evidence for separability in the way that the two streams behave in a dual-task context, such that concurrent performance of ventral and dorsal tasks can be carried out in the same context without interference. 165 REFERENCES Aglioti, S., DeSouza, J. F. X., & Goodale, M . A. (1995). Size-contrast illusions deceive the eye but not the hand. Current Biology, 5, 679-685. Allport, A . (1987). Selection for action: Some behavioral and neurophysiological considerations of attention and action. In H . Heuer & A. F. Sanders (Eds.), Perspectives on perception and action (pp. 395-419). Hillsdale, NJ: Erlbaum. Arnell, K. M. , & Duncan, J. (2002). Separate and shared sources of dual-task cost in stimulus identification and response selection. Cognitive Psychology, 44,105-147. Balint, R. (1909). Seelenlahmung des Schauens, optische Ataxie, raumliche Storung der Aufmerksamkeit. Monatszeitschrift fur Psychiatrie und Neurologie, 25, 51-81. Binkofski, F., Dohle, C , Posse, S., Stephan, K. M. , Hefter, H. , Seitz, R. J., & Freund, H.-J. (1998). Human anterior intraparietal area subserves prehension. Neurology, 50,1253-1259. Brainard, D. H . (1997). The Psychophysics Toolbox, Spatial Vision, 10, 433-436. Brehaut, J. C , Enns, J. T., & Di Lollo, V. (1999). Visual masking plays two roles in the attentional blink. Perception & Psychophysics, 61, 1436-1448. Bridgeman, B., Gemmer, A., Forsman, T., & Huemer, V. (2000). Properties of the sensorimotor branch of the visual system. Vision Research, 40, 3539-3552. Bridgeman, B., Hendry, D., & Stark, L. (1975). Failure to detect displacement of the visual world during saccadic eye movements. Vision Research, 15, 719-722. Bridgeman, B., & Huemer, V. (1998). A spatially oriented decision does not induce consciousness in a motor task. Consciousness and Cognition, 7, 454-464. 166 Bridgeman, B., Kirch, M. , & Sperling, A. (1981). Segregation of cognitive and motor aspects of visual function using induced motion. Perception and Psychophysics, 29,336-342. Bridgeman, B., Lewis, S., Heit, & Nagle, M . (1979). Relation between cognitive and motor-oriented systems of visual position perception. Tournal of Experimental Psychology, Human Perception and Performance, 5, 692-700. Brown, S. W., & Boltz, M . G. (2002). Attentional processes in time perception: Effects of mental workload and event structure. Tournal of Experimental Psychology, Human Perception and Performance, 28, 600-615. Brown, R. G., & Jahanshahi, M . (1998). An unusual enhancement of motor performance during bimanual movement in Parkinson's disease. Tournal of Neurology, Neurosurgery, and Psychiatry, 64, 813-816. Carey, D. P. (2001). Do action systems resist visual illusions? Trends in Cognitive Sciences, 5,109-113. Carnahan, H. , Goodale, M. , & Marteniuk, R. (1993). Grasping versus pointing and the differential use of visual feedback. Human Movement Science, 12, 219-234. Castiello, U . (1996). Grasping a fruit: Selection for action. Tournal of Experimental Psychology: Human Perception and Performance, 22, 582-603. Castiello, U . (2001). The effects of abrupt onset of 2-D and 3-D distractors on prehension movements. Perception and Psychophysics, 63,1014-1025. Castiello, U., & Paine, M . (2002). Effects of left parietal injury on covert orienting of attention. Tournal of Neurology, Neurosurgery, and Psychiatry, 72, 73-76. Castiello, U., Paulignan, Y., & Jeannerod, M . (1991). Temporal dissociation of motor responses and subjective awareness. Brain, 114, 2639-2655. 167 Chang, E., & Ro, T. (2005). Inhibition of return in perception and action. Visual Cognition, 12, 443-472. Chelazzi, L., Miller, E. K., Duncan, J., & Desimone, R. (1993). A neural basis for visual search in inferior temporal cortex. Nature, 363, 345-347. Chua, R., & Enns, J.T. (2005). What the hand can't tell the eye: Illusion of space constancy during accurate pointing. Experimental Brain Research, 162,109-114. Craighero, L., Fadiga, L., Rizzolatti, G., & Umilta, C. (1999). Action for perception: A motor-visual attentional effect. Tournal of Experimental Psychology; Human Perception and Performance, 25,1673-1692. Culham, J. C , Danckert, S. L., DeSouza, J. F. X., Gati, J. S., Menon, R. S., & Goodale, M . A. (2003). Visually guided grasping produces fMRI activation in dorsal but not ventral stream brain areas. Experimental Brain Research, 153,180-189. Culham, J. C , & Kanwisher, N . G. (2001). Neuroimaging of cognitive functions in human parietal cortex. Current Opinion in Neurobiology, 11,157-163. Deubel, H., & Schneider, W. X. (1996). Saccade target selection and object recognition: Evidence for a common attentional mechanism. Vision Research, 36,1827-1837. Deubel, H., Schneider, W. X., & Paprotta, I. (1998). Selective dorsal and ventral processing. Evidence for a common attentional mechanism in reaching and perception. Visual Cognition, 5, 81-107. Di Lollo, V., Kawahara, J., Zuvic, S. M. , & Visser, T. A. W. (2001). The preattentive emperor has no clothes: A dynamic redressing. Journal of Experimental Psychology: General, 130, 479-492. 168 Diedrichsen, J., Nambisan, R v Kennerley, S. W., & Ivry, R. B. (2004). Independent on-line control of the two hands during bimanual reaching. European Tournal of Neuroscience, 19,1643-1652. Duncan, J. (1984). Selective attention and the organization of visual information. Tournal of Experimental Psychology: General, 113, 501-517. Duncan, J. (1996). Cooperating brain systems in selective perception and action. In T. & J. L. McClelland (Eds.), Attention and performance XVI: Information integration in perception and communication (pp. 549-578). Cambridge, M A : MIT Press. Duncan, J., Humphreys, G., & Ward, R. (1997). Competitive brain activity in visual attention. Current Opinion in Neurobiology, 7, 255-261. Elliott, D., Helsen, W. F., & Chua, R. (2001). A century later: Woodworm's (1899) two-component model of goal-directed aiming. Psychological Bulletin, 127, 342-357. Elliott, D., & Madalena, J. (1987). The influence of premovement visual information on manual aiming. The Quarterly Tournal of Experimental Psychology, 39A, 541-559. Eriksen, B. A., & Eriksen, C. W. (1974). Effects of noise letters upon the identification of a target letter in a non-search task. Perception and Psychophysics, 16, 143-149. Fecteau, J. H. , Chua, R., Franks, I., & Enns, J. T. (2001). Visual awareness and the on-line modification of action. Canadian Tournal of Experimental Psychology, 55,106-112. Franz, V. H . (2001). Action does not resist visual illusions. Trends in Cognitive Sciences, 5, 457-459. 169 Franz, V.H. , Gegenfurtner, K.R., Bulthoff, H.H., & Fahle, M . (2000). Grasping Visual Illusions: No Evidence for a Dissociation Between Perception and Action. Psychological Science, 11, 20-25. Giesbrecht, B., & Di Lollo, V. (1998). Beyond the attentional blink: Visual masking by object substitution. Tournal of Experimental Psychology: Human Perception and Performance, 24,1454-1466. Glover, S. (2004). Separate visual representations in the planning and control of action. Behavioral and Brain Sciences, 27, 3-78. Glover, S. (2002). Visual illusions affect planning but not control. Trends in Cognitive Sciences, 6, 288-292. Goldberg, M . E., & Bushnell, M . C. (1981). Behavioral enhancement of visual responses in monkey cerebral cortex: II. Modulation in frontal eye fields specifically related to saccades. Tournal of Neurophysiology, 46, 773-787. Goodale, M . A., & Haffenden, A. (1998). Frames of reference for perception and action in the human visual system. Neuroscience and Biobehavioral Reviews, 22,161-172. Goodale, M . A., & Humphrey, G. K. (1998). The objects of action and perception. Cognition, 67,181-207. Goodale, M . A., Jakobson, L. S., & Keillor, J. M . (1994). Differences in the visual control of pantomimed and natural grasping movements. Neuropsychologia, 32,1159-1178. Goodale, M . A., & A. D. Milner. (1992). Separate visual pathways for perception and action. Trends in Neurosciences, 15, 20-25. Goodale, M . A., & A. D. Milner. (2004). Plans for action. Behavioral and Brain Sciences, 27, 37-40. 170 Goodale, M . A., Milner, A . D., Jakobson, L. S., & Carey, D. P. (1991). A neurological dissociation between perceiving objects and grasping them. Nature, 349, 154-156. Goodale, M . A., Pelisson, D., & Prablanc, C. (1986). Large adjustments in visually guided reaching do not depend on vision of the hand or perception of target displacement. Nature, 320, 748-750. Grill-Spector, K., Kourtzi, Z., & Kan wisher, N . (2001). The lateral occipital complex and its role in object recognition. Vision Research, 41,1409-1422. Haffenden, A. M . , & Goodale, M . A., (1998). The effect of pictorial illusion on prehension and perception. Tournal of Cognitive Neuroscience, 10,122-136. Haffenden, A. M . , & Goodale, M . A. (2000). Independent effects of pictorial displays on perception and action. Vision Research, 40,1597-1607. Haffenden, A. M . , Schiff, K. C , & Goodale, M . A. (2001). The dissociation between perception and action in the Ebbinghaus illusion: Nonillusory effects of pictorial cues on grasp. Current Biology, 11,177-181. Hazeltine, E., Diedrichsen, J., Kennerley, S. W., & Ivry, R. B. (2003). Bimanual cross-talk during reaching movements is primarily related to response selection, not the specification of motor parameters. Psychological Research, 67, 56-70. Held, R. (1970). Two modes of processing spatially distributed visual stimuli. In F. O. Schmitt (Ed.), The neurosciences: Second study program (pp. 317-324). New York: Rockefeller University Press. Henry, F. M . , & Rogers, D. E. (1960). Increased response latency for complicated movements and a 'memory drum' theory of neuromotor action. Research Quarterly, 31, 448-458. 171 Hu, Y., Eagleson, R., & Goodale, M . A. (1999). The effects of delay on the kinematics of grasping. Experimental Brain Research, 126, 109-116. James, T. W., Culham, J., Humphrey, G. K., Milner, A . D., & Goodale, M . A . (2003). Ventral occipital lesions impair object recognition but not object-directed grasping: An fMRI study. Brain, 126, 2463-2475. Jeannerod, M . (1988). The neural and behavioral organization of goal-directed movements. New York: Oxford University Press. Jolicoeur, P. (1999). Concurrent response-selection demands modulate the attentional blink. Tournal of Experimental Psychology: Human Perception and Performance, 25,1097-1113. Joseph, J. S., Chun, M . M. , & Nakayama, K. (1997). Attentional requirements in a 'preattentive' feature search task. Nature, 387, 805-807. Kahneman, D. (1973). Attention and effort. Englewood Cliffs, NJ: Prentice Hall. Keele, S. W., & Posner, M . I. (1968). Processing of visual feedback in rapid movements. Tournal of Experimental Psychology, 77,155-158. Kristjansson, A., & Nakayama, K. (2002). The attentional blink in space and time. Vision Research, 42, 2039-2050. Mclntyre, J., Stratta, F., & Lacquaniti, F. (1997). Viewer-centered frame of reference for pointing to memorized targets in three-dimensional space. Tournal of Neurophysiology, 78,1601-1618. Leinonen, L., Hyvarinen, J., Nyman, G., & Linnankoski, I. (1979). I. Functional properties of neurons in lateral part of associative area 7 in awake monkeys. Experimental Brain Research, 34, 299-320. Lamme, V. A. (2001). Blindsight: The role of feedforward and feedback corticocortical connections. Acta Psychologica, 107, 209-228. 172 Lamme, V. A. F., & Roelfsema, P. R. (2000). The distinct modes of vision offered by feedforward and recurrent processing. Trends in Neurosciences, 23, 571-579. Lavie, N . (1995). Perceptual load as a necessary condition for selective attention. Tournal of Experiment Psychology: Human Perception and Performance, 18, 701-711. Lennie, P. (1998). Single units and visual cortical organization. Perception, 27, 889-935. Liu, G., Healey, C. G., & Enns, J. T. (2003). Target detection and localization in visual search: A dual systems perspective. Perception and Psychophysics, 65, 678-694. Macar, F., Grondin, S., & Casini, L. (1994). Controlled attention sharing influences time estimation. Memory and Cognition, 22, 673-686. Marr, D. (1982). Vision: A computational investigation into the human representation and processing of visual information. San Francisco, CA: W. H . Freeman. Mather, J. A., & Fisk, J. D. (1985). Orienting to targets by looking and pointing: Parallels and interactions in ocular and manual performance. The Quarterly Tournal of Experimental Psychology, 37A, 315-338. Mesulam, M . - M . (1981). A cortical network for directed attention and unilateral neglect. Annals of Neurology, 10, 309-325. Milner, A . D., & Goodale, M . A. (1995). The visual brain in action. London: Oxford University Press. Milner, A . D., Perrett D. L, Johnston, R. S., Benson, P. J., Jordan, T. R., Heeley, D. W., Bettucci, D., Mortara, F., Mutani, R., Terazzi, E., & Davidson, D. L. W. (1991). Perception and action in "visual form agnosia." Brain, 114, 405-428. Navon, D., & Gopher, D. (1979). On the economy of the human-processing system. Psychological Review, 86, 214-255. 173 Neggers, S. F. W., & Bekkering, H . (2000). Ocular gaze is anchored to the target of an ongoing pointing movement. Tournal of Neurophysiology, 83, 639-651. Neisser, U . (1967). Cognitive psychology. New York: Appleton Century-Crofts. Norman, D. A. (1981). Categorization of action slips. Psychological Review, 88,1-15. Norman, J. (2002). Two visual systems and two theories of perception: A n attempt to reconcile the constructivist and ecological approaches. Behavioral and Brain Sciences, 25, 73-144. Nowak, L . G., & Bullier, J. (1997). The timing of information transfer in the visual system. In K. S. Rockland, J. H . Kaas, & A. Peters, (Eds.), Cerebral Cortex: Vol. 12. Extrastriate Cortex in Primates (pp. 205-241). London: Plenum Press. Pashler, H . (1994). Dual-task interference in simple tasks: Data and theory. Psychological Bulletin, 116, 220-244. Pashler, H . (Ed.). (1998a). Attention. East Sussex, UK: Psychology Press. Pashler, H . E. (1998b). The psychology of attention. Cambridge, M A : MIT Press. Pashler, PL, & Johnston, J. C. (1998). Attentional limitations in dual-task performance. In H . Pashler (Ed.), Attention (pp. 155-189). East Sussex, UK: Psychology Press. Pelli, D. G. (1997). The VideoToolbox software for visual psychophysics: Transforming numbers into movies. Spatial Vision, 10, 437-442. Perenin, M . -T., & Vighetto, A. (1988). Optic ataxia: A specific disruption in visuomotor mechanisms. I. Different aspects of the deficit in reaching for objects. Brain, 111, 643-674. 174 Pisella, L., Grea, H., Tilikete, C , Vighetto, A., Desmurget, M . , Rode, G., Boisson, D., & Rossetti, Y. (2000). A n 'automatic pilot/ for the hand in human posterior parietal cortex: Toward reinterpreting optic ataxia. Nature Neuroscience, 3, 729-736. Posner, M . I. (1994). Attention: The mechanisms of consciousness. Proceedings of the National Academy of Sciences, 91, 7398-7403. Posner, M . I., & Petersen, S. E. (1990). The attention system of the human brain. Annual Review of Neuroscience, 13, 25-42. Prablanc, C., Echallier, J. F., Komilis, E., & Jeannerod, M . (1979). Optimal response of eye and hand motor systems in pointing at a visual target. II. Static and dynamic visual cues in the control of hand movement. Biological Cybernetics, 35,183-187. Prablanc, C , & Martin, O. (1992). Automatic control during hand reaching at undetected two-dimensional target displacements. Tournal of Neurophysiology, 67, 455-469. Prablanc, C , Pelisson, D., & Goodale, M . A. (1986). Visual control of reaching movements without vision of the limb. I. Role of retinal feedback of target position in guiding the hand. Experimental Brain Research, 62, 293-302. Raymond, J. E., Shapiro, K. L., & Arnell, K. M . (1992). Temporary suppression of visual processing in an RSVP task: An attentional blink? Tournal of Experimental Psychology: Human Perception and Performance, 18, 849-860. Reason, J. (1979). Actions not as planned: The price of automatization. In G. Underwood & R. Stephens (Eds.), Aspects of consciousness (Vol. 1) (pp. 67-89). London: Academic Press. Reason, J. T. (1990). Human error. New York: Cambridge University Press. 175 Rizzolatti, G., Gentilucci, M. , & Matelli, M . (1985). Selective spatial attention: One center, one circuit, or many circuits. In M . I. Posner & O. S. M . Marin (Eds.), Attention and performance XI (pp. 251-265). Hillsdale, NJ: Erlbaum. Rizzolatti, G., Riggio, L, Dascola, I., & Umilta, C. (1987). Reorienting attention across the horizontal and vertical meridians: Evidence in favor of a premotor theory of attention. Neuropsychologia, 25, 31-40. Rizzolatti, G., L. Riggio, and B. M . Sheliga. (1994). Space and selective attention. In C. Umilta & M . Moscovitch (Eds.), Attention and Performance XV (pp. 231-265). Cambridge, M A : MIT Press. Rizzolatti, G., Scandolara, C , Matelli, M . , & Gentilucci., M . (1981a). Afferent properties of periarcuate neurons in macaque monkeys. I. Somatosensory responses. Behavioral and Brain Research, 2, 125-146. Rizzolatti, G., Scandolara, C , Matelli, M. , & Gentilucci., M . (1981b). Afferent properties of periarcuate neurons in macaque monkeys. II. Visual responses. Behavioral and Brain Research, 2,147-163. Robertson, I., Nico, D., & Hood, B. M . (1995). The intention to act improves unilateral left neglect: Two demonstrations. Neuroreport: A n International Tournal for the Rapid Communication of Research in Neuroscience, 7, 246-248. Robertson, I., Nico, D., & Hood, B. M . (1997). Believing what you feel: Using proprioceptive feedback to reduce unilateral neglect. Neuropsychology, 11, 53-58. Sakata, H. , Shibutani, H. , & Kawano, K. (1982). Spatial properties of visual fixation neurons in posterior parietal association cortex of the monkey. Tournal of Neurophysiology, 43,1654-1672. 176 Schmolesky, M . T., Wang Y. C , Hanes, D.P., Thompson, K. G., Leutgeb, S., Schall, J. D., & Leventhal, A. G. (1998). Signal timing across the macaque visual system. Tournal of Neurophysiology, 79, 3272-3278. Schiegg, A., Deubel, H., & Schneider, W. X. (2003). Attentional selection during preparation of prehension movements. Visual Cognition, 10, 409-431. Schmidt, R. A., & Lee, T. D. (1999). Motor control and learning: A behavioral emphasis (3rd ed.). Champaign, IL: Human Kinetics. Schneider, G. E. (1969). Two visual systems. Science, 163, 895-902. Schneider, W. X. (1995). V A M : A neuro-cognitive model for visual attention control of segmentation, object recognition, and space-based motor action. Visual Cognition, 2, 331-375. Schneider, W. X., & Deubel, H . (2002). Selection-for-perception and selection-for-spatial-motor-action are coupled by visual attention: A review of recent findings and new evidence from stimulus-driven saccade control. In B. Hommel & W. Prinz (Eds.), Attention and Performance XIX (pp. 609-627) New York: Oxford University Press. Schneider, W., & Shiffrin, R. M . (1977). Controlled and automatic human information processing: I. Detection, search, and attention. Psychological Review, 84,1-66. Shapiro, K. (Ed.). (2001). The limits of attention: Temporal constraints in human information processing. Oxford, UK: Oxford University Press. Shapiro, K. L., Arnell, K. M . , & Raymond, J. E. (1997). The attentional blink. Trends in Cognitive Sciences, 1, 291-296. Shapiro, K. L., Raymond, J. E., & Arnell, K. M . (1994). Attention to visual pattern information produces the attentional blink in rapid serial visual presentation . Tournal of Experimental Psychology: Human Perception and Performance, 20, 357-371. 177 Sheth, B. R., & Shimojo, S. (2001). Compression of space in visual memory. Vision Research, 41, 329-341. Spence, C , & Driver, J. (1996). Audiovisual links in endogenous covert spatial attention. Tournal of Experimental Psychology: Human Perception and Performance, 22, 1005-1030. Stroop, J. R. (1935). Studies of interference in serial verbal reactions. Tournal of Experimental Psychology, 18, 643-662. Tipper, S. P., Howard, L. A., & Houghton, G. (1998). Action-based mechanisms of attention. Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences, 353,1385-1393. Tipper, S. P., Howard, L. A., & Houghton, G. (1999). Action-based mechanisms of attention. In G. W. Humphreys, J. Duncan, & A. Treisman (Eds.), Attention, space and action: Studies in cognitive neuroscience, (pp. 232-247). Oxford:_Oxford University Press. Tipper, S. P., Howard, L. A., & Jackson, S. R. (1997). Selective reaching to grasp: Evidence for distractor interference effects. Visual Cognition, 4,1-38. Tipper, S., Lortie, C , & Baylis, G. (1992). Selective reaching: Evidence for action-centered attention. Journal of Experimental Psychology: Human Perception and Performance. Treisman, A., & Gelade, G. (1980). A feature-integration theory of attention. Cognitive Psychology, 12, 97-136. Trevarthen, C. B. (1968). Two mechanisms of vision in primates. Psychologiche Forschung, 31, 299-337. Underwood, G., & Everatt, J. (1996). Automatic and controlled information processing: The role of attention in the processing of novelty. In O. Neumann & A. F. 178 Sanders (Eds.), Handbook of perception and action. Volume 3: Attention (pp. 185-227). r San Diego: Academic Press. Ungerleider, L. G., & Mishkin, M . (1982). Two cortical visual systems. In D. J. Ingle, M . A. Goodale, & R. J. W. Mansfield (Eds.), Analysis of visual behavior, (pp. 549-586). Cambridge, M A : MIT Press. Van Essen, D. C , and E. A . DeYoe. (1995). Concurrent processing in the primate visual cortex. In M . S. Gazzaniga (Ed.), The Cognitive Neurosciences (pp. 383-400). Cambridge, M A : MIT Press. Visser, T. A. W., Zuvic, S. M . , Bischof, W. F., & Di Lollo, V. (1999). The attentional blink with targets in different spatial locations. Psychonomic Bulletin and Review, 6, 432-436. Ward, R. (1999). Interactions between perception and action systems: A model for selective action. In G. W. Humphreys, J. Duncan, & A. Treisman (Eds.), Attention, space, and action: Studies in cognitive neuroscience (pp. 311-332). New York: Oxford University Press. Westwood, D. A., Chapman, C. D., & Roy, E. A . (2000). Pantomimed actions may be controlled by the ventral visual stream. Experimental Brain Research, 1330, 545-548. Westwood, D. A., & Goodale, M . A. (2003). Perceptual illusion and the real-time control of action. Spatial vision, 16, 243-254. Westwood, D. A., Heath, M . , & Roy, E. A . (2000). The effect of a pictorial illusion on closed-loop and open-loop prehension. Experimental Brain Research, 134, 456-463. Westwood, D. A., Heath, M. , & Roy, E. A. (2001). The accuracy of reaching movements in brief delay conditions. Canadian Tournal of Experimental Psychology, 55, 304-310. 179 Woodworm, R. S. (1899). The accuracy of voluntary movement. Psychological Review, 3, (3, Suppl. 13), 1-119. Wickens, C. D. (1984). Processing resources in attention. In R. Parasuraman & D. R. Davies (Eds.), Varieties of attention (pp. 63-101). Orlando, FL: Academic Press. Wurtz, R. PL, & Mohler, C. W. Organization of monkey superior colliculus: Enhanced visual response of superficial layer cells. Tournal of Neurophysiology, 39, 745-765. Zelaznik, H . N . , Hawkins, B., & Kisselburgh, L. (1983). Rapid visual feedback processing in single-aiming movements. Tournal of Motor behavior, 15, 217-236. 

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
http://iiif.library.ubc.ca/presentation/dsp.831.1-0099841/manifest

Comment

Related Items