UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Attention and oculomotor capture Hunt, Amelia R. 2005

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata

Download

Media
831-ubc_2005-105070.pdf [ 15.95MB ]
Metadata
JSON: 831-1.0099847.json
JSON-LD: 831-1.0099847-ld.json
RDF/XML (Pretty): 831-1.0099847-rdf.xml
RDF/JSON: 831-1.0099847-rdf.json
Turtle: 831-1.0099847-turtle.txt
N-Triples: 831-1.0099847-rdf-ntriples.txt
Original Record: 831-1.0099847-source.json
Full Text
831-1.0099847-fulltext.txt
Citation
831-1.0099847.ris

Full Text

ATTENTION A N D OCULOMOTOR C A P T U R E by A M E L I A R. HUNT B.Sc. Honours, Dalhousie University, 1999 M . A . The University of British Columbia, 2002 A THESIS SUBMITTED IN PARTIAL FULFILMENT OF THE REQUIREMEMTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY in THE F A C U L T Y OF G R A D U A T E STUDIES (Psychology) THE UNIVERSITY OF BRITISH C O L U M B I A August 2005 © Amelia R. Hunt, 2005 i i Abstract The investigations contained in this thesis explore distraction during visual search, with particular attention to what eye movements can reveal about the processes involved in visual search. A l l the experiments make use of the oculomotor capture effect, whereby an eye movement is misdirected toward a sudden onset distractor before being redirected to a target object. Fundamental differences between eye movements and attention in general, and between eye movements and manual responses in particular, support the current view that oculomotor capture is distinct from the more general effect known as attentional capture. Like oculomotor capture, attentional capture involves interference with visual search for a target by distractors. Unlike oculomotor capture, this interference is expressed as a delayed correct manual response, rather than a misdirected eye movement. The first study shows that resolution of oculomotor conflict between target and distractor responses takes place at or above the level of the superior colliculus, a midbrain structure involved in eye movement control. The second study explores the timecourse of eye movement and manual localization responses to targets in the presence of sudden onsets, and suggests that for both response types, capture reflects the quality of information about the visual display at a given point in time. The final study expands the oculomotor capture effect to search among emotional faces and finds that the eyes are captured by emotional faces more than neutral distractors only if emotion is task-relevant. Together, the research suggests that oculomotor capture is a specific instance of the more general attentional capture effect. It is proposed that differences and similarities between the two types of capture can be explained by the critical idea that the quality of information in a visual display changes over time, and that different response systems tend to access the same information at different moments in time. iii Table of Contents Abstract ii Table of Contents i i i List of Tables v List of Figures vi Acknowledgements vii Co-Authorship Statement 1...viii CHAPTER 1: Introduction 1 Voluntary and Reflexive Orienting of Attention 3 Attentional Capture 8 Voluntary and Reflexive Eye Movements 16 Oculomotor Capture 21 Attention and Eye Movements 27 Premotor Theory 27 Relationship between Eye Movements and Attention in Behavior 33 Shared Neurophysiology of Attention and Eye Movements 35 Summary and Conclusions 36 General Overview of Experiments 38 General Methods 40 References 43 CHAPTER 2: Integration of Competing Saccade Programs 55 Methods : 59 Results *. 60 Discussion 63 References 64 CHAPTER 3: Localization by Hand and Eye 67 Experiment 1 74 Methods 75 Results 78 Discussion 81 Experiment 2 82 Methods 82 Results... 83 Discussion of Experiment 1 and Experiment 2 85 Experiment 3 88 Methods 89 Results 90 Discussion 93 Experiment 4 93 IV Methods 94 Results 95 Discussion 101 General Discussion 102 References 108 CHAPTER 4: Attraction and Distraction by Emotional Stimuli 113 Experiment 1 120 Methods 120 Results 123 Discussion 126 Experiment 2 128 Methods 128 Results 130 Discussion 133 References 136 CHAPTER 5: General Discussion 139 Summary 140 Conclusions 144 Future Directions 149 References 152 APPENDIX A: Location Overlap and the Relative Effect of Onsets and Singletons 157 Experiment A l : Orange Target 158 Methods 158 Results 160 Experiment A2: Onset Target 163 Methods 163 Results 163 APPENDIX B: Tables of Complete Data from Chapter 3 166 Experiment 1 Results 167 Experiment 2 Results 168 Experiment 3 Results 169 Experiment 4 Results 171 Group 1: Target Duration of 150, 250 or 350ms 171 Group 2: Target Durations of 350, 400, and 500ms 173 APPENDIX C: Emotions Conveyed by the Stimuli used in Chapter 4 176 V List of Tables Chapter 3 Experiment 1 Results 167 Chapter 3 Experiment 2 Results 168 Chapter 3 Experiment 3 Results 169 Chapter 3 Experiment 4 Results 171 Chapter 4 Ratings of Emotional Faces 176 vi List of Figures Figure 1.1. Display used by Theeuwes, Kramer, Hahn and Irwin (1998) 22 Figure 1.2. Experimental setup used in the experiments in the present study 42 Figure 2.1. Illustration of the two-models of saccade control 56 Figure 2.2. Example of a trial sequence 58 Figure 2.3. Results: proportion of saccades for each of the RT bins 62 Figure 2.4. Results: saccadic RT 63 Figure 3.1. Illustration of the conditions 76 Figure 3.2. The reaction time to localize the target in Experiment 1 79 Figure 3.3. The direction of responses in Experiment 1 80 Figure 3.4. The reaction time to localize the target in Experiment 2 84 Figure 3.5. The direction of responses in Experiment 2 85 Figure 3.6. Capture in Experiments 1 and 2 as a function of reaction time 87 Figure 3.7. The reaction time to localize the target in Experiment 3 91 Figure 3.8. The direction of responses in Experiment 3 92 Figure 3.9. The reaction time to localize the target in Experiment 4 96 Figure 3.10. The direction of responses in Experiment 4 97 Figure 3.11. Capture in Experiment 4 as a function of reaction time 99 Figure 3.12. Eye movement capture as a function of reaction time 100 Figure 3.13. Function describing changes in information over time 103 Figure 4.1. Two typical trials from Experiment 1 121 Figure 4.2. The stimuli used in Experiment 1 123 Figure 4.3. Reaction time in Experiment 1 124 Figure 4.4. The destination of saccades in Experiment 1 125 Figure 4.5. The duration of fixation on the distractors in Experiment 1 126 Figure 4.6. The two types of target trials used in Experiment 2 130 Figure 4.7. The reaction time in Experiment 2 131 Figure 4.8. The destination of saccades in Experiment 2 132 Figure 4.9. The duration of fixation on the distractors in Experiment 2 133 Figure A . l . The conditions used in Experiments 1 and 2 159 Figure A.2. The reaction time to saccade to the target in Experiment 1 160 Figure A.3. The destination of saccades in Experiment 1 161 Figure A.4. The reaction time to saccade to the target in Experiment 2 164 Figure A.5. The destination of saccades in Experiment 2 165 vii Acknowledgements Thanks are due most of all to my mentor and friend, Alan Kingstone, for his infinite wisdom and tireless support. I am also grateful to everyone in the B A R lab for being such solid and supportive colleagues, and to the wise and generous faculty in the Psychology Department at U B C , especially Jim Enns, and also Todd Handy, Charlotte Johnston, Mark Schaller, Don Wilkie, and Romeo Chua (Human Kinetics) for their guidance. Thanks to my collaborators, Adrian von Muhlenen, Bettina Oik, and Robbie Cooper, and to all the undergraduate volunteers who helped with these and other research projects: Heather Booth, Craig Chapman, Vanessa Ho, Clara Hungr, and Michelle Lundigran, Thanks to NSERC and the Michael Smith Foundation for financial support. Finally, thanks to Brad Newman, and all my family and friends, for their unfailing love and encouragement. Co-authorship Statement I am the primary author on all the PhD work presented in this thesis. The work grew from discussions and collaborations with the following individuals: Drs. Alan Kingstone, Adrian von Miihlenen, and Bettina Oik; Graduate student Robbie Cooper (University of Stirling); Undergraduate volunteers Heather Booth, Vanessa Ho, Michelle Lundigran and Clara Hungr. 1 CHAPTER 1: An Introduction to Attention, Eye Movements, and Visual Search The visual world is a jumble of possibilities, with each edge, glitter, and movement indicating a potential value, danger, or distraction. The limited capacity of the human information processing system to deal with every single incoming visual event in full means that selective attention is intrinsically necessary, allowing information to be effectively filtered and prioritized according to expectations and goals. But selective attention also carries a risk, because while one aspect of the environment is selected for closer inspection, other more important things could be escaping notice. To prevent this from happening, there must also be a system in place for overriding current goals in the face of competing events that might signal something more important than the event on which attention is currently focused. For instance, a driver looking for a specific street would focus on visual items that are likely to provide information that will help to find the street, using voluntary attention to seek out items consistent with the shapes and locations of street signs. This task places the driver at risk of missing important information, such as traffic lights, pedestrians, and other cars. Presumably if something unexpected happened, like another car braking suddenly, attention is able to smoothly disengage from the task of finding the street sign in order to pick up on and react to this high-priority piece of information. This thesis is focused on understanding the delicate balance between attending to events because they are relevant to task goals, and attending to events because of their intrinsic perceptual properties. The attentional capture paradigm is well-suited to exploring the. tipping point between orienting to task-relevant events and orienting to perceptually salient distractions during visual search. In a typical attention capture experiment, a pre-defined target is 2 presented at the same time as task-irrelevant distractors, and participants are required to find the target and then perform some sort of discrimination task, and respond by pressing a key. The degree to which the distractors affect task performance is measured through changes in reaction time and accuracy to discriminate the target in the presence of distractors. The research presented in the current thesis uses a similar underlying logic, but the direction of eye movements is typically measured as well as, or instead of, manual responses. In these experiments, participants are instructed to execute an eye movement toward a pre-defined target as soon as they detect it, and targets are presented at the same time as task-irrelevant distractors. Previous investigations using this oculomotor capture methodology have shown that the eyes tend to be quite literally "captured" by perceptually-salient distractors, in that the eyes tend to be directed towards irrelevant items initially, and are later corrected to land on the target (e.g., Theeuwes, Kramer, Hahn and Irwin, 1998). This research relies heavily on the assumption that eye movements provide a rich, reliable, and direct measure of the locus of attention more generally. However, the exact relationship between attention and eye movements is the subject of some debate in the literature, and it is not clear to what extent the relationship between reflexive and voluntary eye movements can be used to draw conclusions about the relationship between reflexive and voluntary covert spatial attention. In this introduction, I will first discuss voluntary and reflexive attention, and review what attentional capture research has been able to reveal about the relationship between these two subtypes of attention. I will then discuss voluntary and reflexive eye movements, with a particular focus on what oculomotor capture research has uncovered about how voluntary and reflexive pathways are integrated.. In the third section, I will address the relationship 3 between attention and eye movements, and what this might reveal about the relationship between attentional and oculomotor capture. Voluntary and Reflexive Orienting of Attention Most people would probably agree that they are able to focus their eyes on one location in space and their visual attention on another location. There is no question that these introspections are accurate, in that there is a large body of evidence that attention can be focused on different peripheral locations while the eyes remain stationary (e.g. Posner, Nissen and Ogden, 1978; Posner, 1980). Shifts in spatial attention in the absence of eye movements are usually referred to as covert visual orienting, meaning there is no overt, observable behavior to indicate where exactly attention is directed. Instead, the locus of covert attention is typically mapped using faster reaction times and increased perceptual sensitivity at specific locations of the visual field as indicators of where attention is directed. This section of the introduction discusses the relationship between reflexive and voluntary spatial covert attention. Many studies of visual attention make use of metaphors, either explicitly or implicitly. The most commonly used metaphor in the studies reviewed here is the spotlight, with the locus of attention conceptualized as a beam that illuminates regions of the visual field for more detailed processing (e.g., Remington and Pierce, 1984; Kramer and Hahn, 1995). Other metaphors also exist, for example, attention is also thought of as a filter (Broadbent, 1957), as a binding agent (Treisman and Gelade, 1980) and as a zoom lens (Erikson and St. James, 1986). Metaphors are useful for communicating the basic functions of visual attention, and for generating ideas about how the neural processes subserving 4 attention might be structured. The attention-as-spotlight metaphor has framed many debates in the spatial attention literature (for example, whether you can "split the beam", the specific size and shape of the spotlight, and how the spotlight moves in space). Some aspects of the spotlight metaphor seem to be an adequate description of attention, for example when covert attention is allocated to a given location it literally increases perceptual sensitivity there (e.g., Cameron, Tai and Carrasco, 2002). But other aspects of the metaphor are inconsistent with how attention functions (for example, Driver and Baylis (1989) demonstrated that the "spotlight" can illuminate noncontiguous groups that share a perceptual feature). It is important when considering the debates about visual search to remain flexible and to recognize which components of various metaphors are helpful, and which are misleading. An empirical distinction between spatial attention as a goal-directed information filter versus attention as an early-warning system for unexpected events was first demonstrated by Posner and Cohen (1984), although as Posner noted previously (1978), the general division of behavior into reflexive processes and voluntary inhibition of those reflexes had been around for over a hundred years (p. 21). Posner defined a reflexive process as one that "may go on without the subject's intention or even despite intentions that it not take place" (1978, p. 90), using Stroop color-naming as an example of a reflexive process (word reading) that measurably interferes with intention (naming the color ink with which the word is written). His later demonstration that this reflexive/voluntary dichotomy exists in the context of orienting spatial selective attention (Posner and Cohen, 1984) was an important first step in what became a rich, active field of research towards a better understanding of how reflexive and voluntary processes share control over attentional orienting. The body of behavioral and 5 neurophysiological evidence in support of two distinct subsystems of attention has been steadily growing in the last twenty years (for a recent review, see Klein, 2004). In the context of attentional orienting in space, reflexive orienting of attention (also known as bottom-up, stimulus-driven, or exogenous orienting) occurs in response to a salient but irrelevant perceptual event, such as a sudden flash of light (Posner and Cohen, 1984). The typical behavioral marker of reflexive orienting is more efficient processing of perceptual events occurring at a stimulated (cued) location relative to other locations. For it to be thought of as purely reflexive orienting, attention must be allocated to the cued location regardless of any spatial predictiveness attached to the cue - that is, even if the cue has no informational value about the location of the impending target, and is irrelevant to the task being performed, processing at the cued location would be improved. In contrast, voluntary orienting occurs when allocating attention to a location in the field makes strategic sense in the context of the task being performed. The distinction between reflexive and voluntary orienting of attention has been supported by studies identifying several fundamental differences in the properties of attention when it is summoned by reflexive versus voluntary cues. One of the most important discoveries was that the processing advantages at the cued location conferred by reflexive orienting are short-lived, lasting only about 2-300 ms, and are followed by a longer period during which processing at the cued location is actually slower than in other parts of the visual field (Posner and Cohen, 1984). This effect is known as inhibition of return (IOR), and is thought to reflect a mechanism that discourages attention from revisiting recently-inspected locations (for a review, see Klein, 2000). In contrast to reflexive orienting, voluntary orienting of attention is not followed by inhibition of return. Other distinctions between voluntary and reflexive orienting were first demonstrated by Jonides (1981), who showed that reflexive orienting occurs after relatively short cue-target intervals and is less affected by memory load than voluntary orienting. Research since then has found numerous other differences between reflexive and voluntary covert orienting (e.g. Briand and Klein, 1987; Lu and Dosher, 2000; Taylor and Klein, 1998). Neurophysiological evidence suggests that unique networks of brain areas subserve voluntary and reflexive orienting processes. Corbetta and Shulman (2002) propose that the dorsofrontal parietal attention network, which involves dorsal posterior parietal areas, especially the intraparietal sulcus (IPS), and an area in or around the frontal eye field (FEF) in the frontal cortex, is responsible for the implementation of goals and expectations about incoming sensory signals and anticipated motor responses to them. A second network, the right frontoparietal network, involves the right temporoparietal junction (TPJ) and ventral frontal cortex (VFC), and is activated when an unexpected salient sensory event occurs in a previously unattended location. Corbetta and Shulman refer to this latter network as an independent "circuit-breaker", that interrupts the voluntary attention activity in the dorsofrontal parietal network to direct attention to unexpected sensory events. It is damage to right TPJ that is most consistently associated with unilateral spatial neglect, a syndrome in which patients lose the ability to orient attention to the left visual field (e.g. Vallar, 1998). Corbetta et al. suggest that damage to this region may disrupt the connection between the voluntary and reflexive orienting systems, causing abnormalities in spatial attention allocation. Spatial attention of any sort is most frequently associated with the parietal and frontal cortices, so the suggestion that distinct, independent regions within the parietal and frontal 7 cortex are involved in voluntary and reflexive orienting is particularly important. There is converging evidence to support the dissociation between the dorsofrontal and right frontoparietal networks. For example, Friedrich, Egly, Rafal and Beck (1998) found that patients with lesions in TPJ, unlike those with lesions in other areas of parietal cortex, were impaired at target detection, and showed a neglect-like pattern of impaired responses to events occurring in the left visual field. TPJ lesions did not affect the ability to use and maintain spatial expectancies. Further evidence for two independent parietal attention networks comes from Corbetta, Kincade, Ollinger, McAvoy, and Shulman (2000), who demonstrated a double-dissociation in fMRI activation associated with voluntary attention versus detection of targets in unexpected locations. They found that activity in the TPJ was elevated when a target was presented in an unattended location, and activity in the IPS was elevated when a location was voluntarily attended, independent of activity associated with the actual appearance of the target. Similar results were found by Serences, Shomstein, Leber, Golay, Egeth and Yantis (2005), who observed activity in TPJ and V F C that was predictive of involuntary attentional capture by irrelevant items that share properties with the target. Hopfinger, Buonocore and Mangun (2000) also found that unique areas of the parietal lobe were active in response to attention cues and to targets. Like Corbetta et al (2000), they found activity related to voluntary attention in IPS and the FEF. Unlike Corbetta et al., they associated the Superior Parietal Lobule (SPL), rather than TPJ, with target detection. This discrepancy could simply reflect differences in the task and stimuli used by the two research groups; the fact that they converge in their conclusions about brain activation in response to attentional cues but diverge with respect to activation in response to the target suggests that target-related activity, but not attention, varies according to the kind of target used, its timing, and the anticipated response. In any case, these two studies both suggest that there is a clear neurophysiological division between voluntary and reflexive orienting of attention. In summary, the behavioral dissociation between voluntary and reflexive orienting is reflected in neurophysiological and neuroimaging research, which suggests that two independent but coordinated networks underlie these two varieties of orienting. The question that attentional capture research attempts to address is how exactly these two networks are coordinated in visual search. Attentional Capture The balance between the benefits of goal-directed attention, also known as voluntary or endogenous attention, and the risks associated with missing other informative but task-irrelevant events, is an important underlying concept for the study of attentional capture. Given that voluntary, goal directed orienting of attention is a separate, independent process from the reflexive attraction of attention to potentially important events, one can ask, what sorts of information are important enough to interrupt ongoing processes in order to inspect them in more detail, and how does this interruption process occur? Attentional capture studies typically attempt to address this question by presenting an array of items through which participants must search to find a specific target. When the reaction time to detect the target is significantly slower or less efficient in the presence of specific kinds of distractors, it is assumed that voluntary attention to the target was overridden or "captured" by those distractors, thereby interfering with the goal-directed search process. Using this method, it should be possible to uncover the answers to these questions about how voluntary and reflexive attention contribute to visual search. 9 Straightforward answers have not been forthcoming, however, possibly because they have become entangled with the question of how attention actually operates in selecting items in the visual world, which is also the subject of some debate. Several theories have been proposed to explain how attention is directed in the visual field. One of the most influential of these is Feature Integration Theory (FIT), proposed by Treisman and Gelade (1980). They suggest that there are two stages involved in visual search. In the first stage, the scene is represented automatically and in parallel based on small set of basic dimensions. These dimensions are things like brightness, color, shape, and location, but they are in some sense free-floating features that are not conjoined to each other (for example, there could be basic perception of a yellow patch, but it would not be bound to a specific shape or location at this stage). Attention is serially allocated to specific regions of the display, and serves to conjoin various features present in that location into a cohesive object representation. Although this model has required revisions to accommodate the findings of subsequent studies (Treisman and Sato, 1990), it should be credited with initiating a new way of thinking about the function of attention. That is, attention does not simply speed stimulus detection or increase perceptual sensitivity, but can also influence how objects are formed and represented. FIT also reinforced the notion that there are pre-specified components of the visual environment that have a special "pre-attentive" status, and they are able to engage attention because of their inherent perceptual properties. From this research also emerged the general use of search slope as a diagnostic for serial versus parallel processing of the search display. When reaction time to detect the target increases systematically with the number of distractors in the display, one can infer that search of the distractors is performed in serial, 10 with each item in the display being inspected one at a time to determine if it is the target. If the reaction time does not increase, or increases very little, with the number of distractors, one can infer that the target "popped out" of the search display in parallel search, obviating the need for serial inspection of the distractors1. As an elaboration of FIT, Wolfe, Cave and Franzel (1989) suggested that guided search is able to make use of parallel search information in guiding the serial allocation of attention. While FIT assumes there is no relationship between the parallel and serial stages of visual search, the guided search model proposes that the information extracted from the visual display in parallel can then be used to guide attention to the target location. More specifically, the parallel processing of the display continues to occur after serial processing begins, and the movement of the serial spotlight of attention is not random across the display, but is instead guided by information from the ongoing parallel processing of the whole image. For example, if the task is to locate the red square among green squares and red circles, observers would be able to use color information obtained in parallel search stages to guide attention only to the subset of red items, rather than waste time inspecting green items which could not possibly be the target. Thus the observer is able to use specific feature dimensions to limit the number of items that serial search would need to inspect. FIT and similar models of selective attention have also re-invigorated a debate about the degree to which visual stimuli are processed prior to the allocation of attention (e.g. Broadbent, 1957; Duncan, 1980; Pashler, 1998). FIT brings to light the possibility that some components of the visual world are able to guide attention, and other components are only 1 Serial and parallel search are now more commonly thought to exist on a continuum (e.g. Duncan and Humpheys, 1989; Wolfe, 1998), although search slope is still a typical measure of the efficiency of search for a given pairing of targets and distractors. 11 fully processed after attention has been allocated to them. As is typical of many debates in cognitive psychology, the very fact that straightforward answers have not yet been produced is a good indication that the boundaries in question are flexible. Flexibility is indeed a component of many selection models. For instance, in Duncan's model of selective visual attention (1980; 1981), the degree to which elements are processed depends on the selection schedule, or (for the present purposes) the feature used to discriminate the target of attentional selection from the other items in the visual field. Duncan's concept of selection schedules was later elaborated into a formal theory of attention (e.g., Desimone and Duncan, 1995; Desimone, 1998) known as the biased competition model. Desimone and Duncan (1995) propose that selective attention reflects competition between neurons that represent incoming perceptual information. Competition can be biased in favor of one representation over another by bottom-up factors like stimulus contrast and luminance, or by top-down feedback mechanisms engaged by working memory. The ultimate winner of a series of competitions between various forms of representation gains control over behavior. The flipside of the underlying attentional properties that determine whether a target will "pop out" of a search display, is the attentional properties involved in trying to ignore a distractor with similar attention-grabbing effects. That is, when one out of several distractors is unique in terms of its color or shape or onset, it tends to interfere with search for a target relative to when all the distractors are uniform (e.g., Yantis and Jonides, 1984). The critical question that has been the topic of a great deal of investigation and debate is why exactly this interference occurs. On one end of the spectrum are explanations that emphasize the inherent properties of the stimulus itself (e.g., Theeuwes, 1992). These explanations suggest that 12 attention is initially allocated to locations in the visual world based purely on their perceptual salience. Salience itself is determined by the visual properties of the stimulus, such as a unique color or shape, or especially high or low luminance, relative to the rest of the display (Treisman and Gelade referred to these properties as "features", and when they are unique items in the display they are often referred to as "feature singletons"). On the other end of the spectrum of explanations is the notion that attention is allocated purely on the basis of voluntary goals of the observer, and capture by irrelevant distractors will occur only insofar as the distractors conform to the selection strategy being employed by the observer to detect the target (e.g., Folk and Remington, 1998; Folk, Remington and Johnston, 1992). These theories suggest that perceptual salience alone is not sufficient for capture to occur. Yantis and Jonides (1984) were among the first to show that search for a target letter was slower when an irrelevant abrupt onset was displayed. They initially assumed that this capture effect would occur regardless of the intentions of the observer, although in later work (Yantis and Jonides, 1990), they showed that capture was reduced or eliminated when a valid precue about the target location was presented prior to the actual target. They suggest that the irrelevant onset captured attention automatically, but that focused attention is able to override the effect of the sudden onset on search. Theeuwes (1992) also championed the notion that salient elements of the display capture attention in a purely stimulus-driven manner. He showed that a unique distractor interferes with search for the target even when the target and distractor never shared locations or features, and thus the distractor could be considered to be completely irrelevant to the task. Theeuwes' explanation for the longer search times in the presence of these irrelevant distractors was that reflexive attention moves to the most salient 13 item in the visual display, and then voluntary attention regains control and moves attention to the target location. Support for Theeuwes' (1992) explanation comes from investigations using IOR as a measure of where reflexive attention has been allocated (Godijn and Theeuwes 2002a). As discussed earlier in this introduction, IOR is observed following reflexive allocation of attention to a stimulus location, and not after voluntary allocation (Posner and Cohen, 1984). Theeuwes and Godijn observed slower responses when the target appeared at a location previously occupied by an irrelevant color singleton, consistent with the interpretation that attention had been reflexively allocated to the distractor and was then inhibited from returning there. In response to Theeuwes' (1992) claim that attention is automatically allocated to color singleton, Bacon and Egeth (1994) demonstrated that color singletons only capture attention when the task is to detect a singleton target. When the search task requires feature search (that is, serial search of the display elements), there was no effect of the color singleton on search. Bacon and Egeth argue that observers, instead of looking for the prespecified target, simply look for the odd-man-out in the display, which is possibly a more efficient way of performing the task when no distractor is presented. When a unique distractor is presented, however, the adoption of this so-called singleton-detection strategy results in confusion about which singleton is the target, and thus reaction time is slowed. Further research has similarly failed to produce attentional capture by feature singletons when the target is itself not a singleton (Folk and Annett, 1994; Gibson and Jiang, 1998; Yantis and Egeth, 1999). 14 Although there is scant evidence that feature singleton distractors capture attention during serial search, there is evidence that abrupt onsets have some kind of special status, in that they have been shown to capture attention regardless of what is defined as the target of search (Jonides and Yantis, 1988). However, it has been suggested that even capture by abrupt onsets may depend on the attentional set adopted by an observer. Folk et al. (1992) observed that abrupt onset cues only interfere with search when the best strategy for detecting the target is to use a feature that is shared with the onset distractor. They term this "contingent capture", and suggest that reflexive attention is always pre-configured to be allocated to items that are consistent with the current task goals. Items that are completely irrelevant, defined as those that have no visual features in common with the target, will not capture attention reflexively. In a further qualification of this theory, Folk, Remington and Wright (1994) acknowledge that stimulus salience is an important factor in attentional capture, but that the effects of salience follow the effects of attentional control settings, rather than the other way around as Theeuwes (1992) suggested. They argue that the attentional control setting determines whether or not a distractor will interfere with search, and following that, if the perceptual salience of target is greater than that of the distractor, attention will be allocated to the target first. Yantis (1993) agrees with Folk et al. (1992; 1994) that capture by feature singletons is contingent on attentional control settings, but he disagrees with their claim that this argument applies to abrupt onsets. He points out that although attentional control settings are able to override capture by feature singletons, abrupt onsets do seem to be the default setting, regardless of the task. This claim has been supported in a number of previous and subsequent studies (Franconeri, Hollingworth, and Simons, 2005; Jonides and Yantis, 1988; Yantis and 15 Hillstrom, 1993; Yantis and Jonides, 1996). In the context of Duncan's (1981) flexible selection schedules, abrupt onsets appear to be an especially efficient selection feature. To summarize this complex debate, it appears that attentional capture by unique but irrelevant feature singletons (Theeuwes, 1992) only occurs when observers adopt a singleton-detection mode (Bacon and Egeth, 1994). When it comes to abrupt-onset distractors, however, attentional capture appears to exist regardless of the type of strategy or the target involved (Yantis, 1993; Yantis and Jonides, 1996), although it can be overridden by attentional control settings (Folk et al., 1992; 1994; Yantis and Jonides, 1990). In fact, the arguments for stimulus-driven capture versus arguments for contingent or goal-driven capture are not that far apart. One extreme of the argument is that the initial allocation of attention is based on the salience of various visual features in the display, and later decisions about where to allocate attention are based on voluntary, goal-directed processes (Theeuwes, 1992). The other side argues that the initial allocation of attention to the search display is based on voluntary attentional control settings, and then influenced by the visual salience of features within the subset of features selected by visual attention (Folk et al., 1992; 1994). Most observations, however, remain somewhere in between these two claims, suggesting that attentional control settings are usually able to override stimulus-driven factors, but sometimes search strategy or sudden-onset event can tip the balance in favor of stimulus-driven factors. An outstanding issue related to this debate is the as-yet-unknown issue of why, exactly, responses are slower when an abrupt onset appears. One possibility is that attention is literally shifted to the onset location before it is shifted to the target location, and the discrimination of the target must await the arrival of attention at the target location before it can be performed (Theeuwes and Godijn, 2001). The other possibility is that the target and the distractor are both processed in parallel, and the presence of the onset increases the time it takes to filter out the rest of the display and focus on the target (Folk and Remington, 1998). The latter explanation does not predict that attention will be allocated specifically to the abrupt onset location, but rather that it will slow the movement of attention to the target. I will return to these issues in the discussion of the oculomotor capture literature later in this review. Voluntary and Reflexive Eye Movements Similar to voluntary and reflexive spatial attention, spatially-selective eye movements are often subdivided into two distinct subtypes. Voluntary eye movements are those executed in line with task goals and intentions, and reflexive eye movements are classified as those that occur to locations and events that are not relevant to the current task. A basic review of the physiology of the eye movement system follows, with special attention to two distinct pathways within the eye movement system which seem to map onto the distinction between voluntary and reflexive eye movements. The eyes are controlled by three pairs of muscles, only two of which are involved in the saccadic eye movement system (the third is the superior/inferior obliques, which are primarily involved in the rotation of the eyes in response to head tilting movements). The two pairs involved in saccadic eye movements are the lateral/medial rectus, controlling horizontal movement, and the superior/inferior rectus, controlling vertical movement. Most of the eye muscles are controlled via the oculomotor nerve (cranial nerve III), but the lateral rectus is controlled by the abducens nerve (nerve IV). To execute a lateral saccadic eye movement, the 17 paramedian pontine reticular formation (PPRF) sends bursts of impulses to the abducens nerve nucleus, causing contraction of the ipsilateral lateral rectus muscle and consequent lateral eye movement. To maintain fixation, the nucleus prepositus hyperglossi (NPH) sends a more tonic input to oculomotor centers that keeps the lateral eye position stable between bursts. A similar burst/tonic system controls vertical eye movements, with the rostal interstitial nucleus of the medial longitudinal fasciculus sending bursts of impulses to the trochlear nerve center, controlling superior rectus muscles, and the interstitial nucleus of Cajal providing tonic input to keep vertical position stable. Thus vertical and lateral eye movements are each controlled by independent "move" and "fixate" inputs. These "move" and "fixate' centers receive input from the superior colliculus (SC) of the midbrain, and from the Frontal Eye Fields (FEF) of the cortex. In studies of the eye movement system, the SC is the focus of discussion more frequently than other brain structures, despite the fact that it is just one component of a complex network of structures and pathways involved in the control of eye movements. It does stand out from other structures in the brain in many respects, not the least of which is its unmistakable appearance as a symmetrical pair of bumps on the dorsal surface of the midbrain. It has a clearly visible input, with a branch of the optic tract running into the structure on the dorsolateral surface, and outputs to the four abovementioned structures that mediate the circuitry of the eye muscles. The SC is thus unique from other structures in more than just appearance, having the compelling property of representing a relatively simple interface between sensory input and motor output, with the added advantage that the motor output is, generally speaking, an observable and easily measured behavioral response (eye movements). It also has the appealing quality that it is phylogenetically a very old visual 18 pathway, and consequently it is well-defined and structurally similar in a number of other animals. This opens avenues of research towards a better understand the inner workings of the superior colliculus relative to many other brain areas. Given all these qualities, it is not surprising that the SC has been the subject of a great deal of interest, study, insight, and speculation. Of course things are much more complicated upon closer inspection: the SC is a multilayered and multifunctional structure that receives both sensory and motor input from a wide range of cortical and subcortical brain areas. Moreover, the SC is not the only connection between visual input and the eye muscles, so eye movements cannot be interpreted as necessarily representing activity in the SC. The superficial and intermediate layers of the SC receive input directly from the retina. The deep layers receive input from the FEF, the parietal cortex, visual cortex and the substantia nigra. The deep layers send output to the brainstem areas involved in eye movements. The intermediate layers of each colliculus contain a spatial map that corresponds to the contralateral space, with the most medial region corresponding to the fovea, and more lateral regions corresponding to peripheral space. These regions provide input to the "move" and "fixate" centers in the brainstem, and as such have different firing characteristics. Medial regions of the SC are considered to be "fixation" cells, and they fire tonically while the eyes are fixated, and stop firing while the eyes are moving (Munoz and Wurtz, 1993). More lateral regions of the SC contain cells that release a burst of activity associated with the planning of an eye movement to a visually salient location, and a subpopulation of "buildup" neurons that also fire tonically starting just before fixation on a given point is broken, and appear to have a mutually inhibitory relationship with medial "fixation" cells (Munoz and Wurtz, 1995a and 1995b). The SC codes saccade programs in terms of vectors, with each location of 19 the SC representing a specific distance and trajectory. Two basic principles underlie most models of SC neurophysiology (e.g., Findlay and Walker, 1999; Godijn and Theeuwes, 2002b; Trappenberg, Dorris, Munoz, and Klein, 2001): first, there is lateral excitation, whereby cells in a 1mm radius are co-activated; second, there is remote inhibition, whereby cells outside that radius are inhibited. Eye movement behavior reflects this architecture, in that eye movements to one of two concurrent target locations tend to be faster when the targets are close together, and the eyes tend to land somewhere between the two locations (known as the center of gravity, or global, effect (Coren and Hoenig, 1972; Findlay, 1982). This is consistent with the notion that the activation of one location spreads to adjacent locations. When concurrent target locations are father apart, the execution of an eye movement to one of the locations tends to be slower than when the two points are closer together, and the landing position of the saccade is on one or the other point rather than somewhere in between (Walker, Deubel, Schneider and Findlay, 1997). This so-called remote distractor effect is consistent with the notion that there is mutual inhibition of remote activity within the SC. Surprisingly, the SC is not essential to eye movement behavior, at least not in the monkey. The FEF also contribute to eye movement behavior, and appears able to compensate when the SC is ablated, and vice versa (Schiller, Sandell, and Maunsell, 1987; Schiller, True and Conway, 1980). Only removal of both the FEF and the SC abolishes saccadic eye movements. The FEF, like the SC, code saccades in terms of vectors. The dorsomedial frontal eye fields, also known as the medial or supplementary eye fields (SEF, Schlag and Schlag-Rey, 1987), are also involved in eye movement control, and appear to represent specific orbital positions, or place codes, rather than vector codes. The exact role of the SEF 20 is not known, and lesions to this structure produce negligible deficits in eye movement behavior (Schiller and Chou, 1998). Many other areas of the brain also seem to be involved in eye movements, including regions of occipital cortex and parietal cortex, but they are not essential elements of the eye movement system, in the sense that eye movement behavior does not depend on these regions (Schiller, 1998). The FEF, SEF and other cortical regions involved in eye movements are typically thought of as part of a more recently-evolved motor system that enables sophisticated analysis and planning, relative to the more rudimentary oculomotor control system involving the SC. Indeed, the FEF and the SC were once thought to be the key structures in two independent pathways controlling eye movements, with the FEF responsible for planning voluntary eye movements in response to internally-generated goals and intentions, and the SC responsible for the reflexive saccades to salient visual events (Schiller et al., 1980; Becker and Juergens, 1979; Theeuwes et al., 1998). More recent research, however, suggests that in the undamaged brain, the cortical "voluntary" pathway converges on the SC and reflexive and voluntary inputs complete for control over eye movements on a common activation map within the intermediate layers of the SC (e.g., Findlay and Walker, 1999; Godijn and Theeuwes, 2002; Kopecz, 1995). Trappenberg et al. (2001) have developed a relatively successful neural model of competitive integration that describes both eye movement behavior and neural activity within the SC. In their model, voluntary and reflexive saccade goals converge and compete on a single layer of the SC through lateral interactions. Reflexive, "bottom-up" visual information reaches the SC quickly, and affects groups of cells corresponding to a given location. Slower activation in the map occurs in response to 21 probabilistic, strategic, or memory-based information, and can modulate the activation caused by reflexive input. In summary, current research suggests that voluntary and reflexive eye movements depend on two distinct pathways: A cortical pathway that involves occipital, parietal, and frontal areas, and seems to depend particularly on the FEF for successful execution of cortically-mediated (voluntary) eye movements; and a subcortical pathway that conducts visual information straight from the retina to the SC, where it converges on a common salience map to compete with other sensory signals for control over eye movement behavior. In the intact brain, the cortical pathway also converges on the SC, and enhances or competes with information from the retina there. However, the cortical pathway can also bypass the SC, and have direct control over the subcortical regions that control the eye muscles. Less is known about this direct pathway than the pathway involving the SC. Oculomotor capture Theeuwes et al. (1998) were the first to use eye movements in place of manual responses in an attentional capture experiment. Their motivation was simply that it is not yet known how eye movements are influenced by an abrupt onset distractor during search. In their experiment, six gray circles were shown equidistant from fixation, and after 1000ms, all changed to red except for one (see Figure 1.1). At the same time, on half the trials an additional red circle was added to the display at the same time as the target was revealed. The target contained either a 'c' or a reversed 'c', and the observer had to indicate the orientation of.the 'c' as quickly as possible. The 'c' was small enough that it had to be fixated to be 22 discriminated, necessitating a fast eye movement from the center to the target for the task to be completed. Figure 1.1. Display used by Theeuwes, Kramer, Hahn and Irwin (1998). In frame 1, 6 gray circles are shown around a central fixation point for 1000ms. In frame 2, 5 of the circles change to red (shown in black here). On half the trials, an additional circle is added to the display at one of the unoccupied positions. The task is to look at the gray circle as quickly as possible and discriminate a small letter shown inside the circle (letter in the actual experiment was smaller than the ones shown here). Theeuwes et al. (1998) observed that on trials where an irrelevant onset appeared, the eyes went to the onset before going to the target about half the time. They also replicated the earlier findings of Yantis and Jonides (1990), who found that precueing the impending location of the target abolished the effect of the onset on saccadic eye movements. They concluded, like Yantis and Jonides (1990), that capture by sudden onsets can be overridden by directed attention to the target location. They propose that there is independent, parallel programming of two saccades; one to the target, and one to the onset, and whichever one is complete first will gain control over the eye movement system. The valid location pre-cue gives the target-directed saccade program a head start on the onset-directed program, thus preventing the onset-directed program from reaching oculomotor control structures first. The notion of parallel programming of saccade programs is consistent with other research (McPeek, Skavenski and Nakayama, 2000, Theeuwes, Kramer, Hahn, Irwin and Zelinsky, 1999). However, the notion that these programs are completely independent is not 23 consistent with the more recent and well-developed models of saccade programming reviewed above (e.g., Findlay and Walker, 1999; Trappenberg et al., 2001). Indeed, in later research (Godijn and Theeuwes, 2001; also see Chapter 2 of the present thesis) it was demonstrated that capture results were more consistent with models that propose that saccade programs are not totally independent, but are integrated according to the strength and location of activation on a common map. The characteristics of this salience map are consistent with the neural architecture of the SC. These studies demonstrate the utility of the oculomotor capture paradigm for understanding how voluntary and reflexive saccade goals compete for control over the visual system, and how competing saccade programs are integrated to form a single stream of fixations during search. One important consideration, however, is the role of attention in this integration process. A question that needs to be addressed is whether oculomotor capture and attentional capture share a common cause, or if they are separate, independent phenomena. There are a priori reasons to believe they are separate, because the eye movement system has special access to incoming information from the visual world via the subcortical visual pathway through the SC. This pathway could give rise to oculomotor capture that is independent from attentional capture. On the other hand, eye movements and attention are closely coupled in visual search, so much so that oculomotor capture is often used as a tool for addressing some of the outstanding issues in visual search from the attention capture literature. An important instance in which oculomotor capture has been applied to questions about attentional capture is the issue of whether or not attention is literally allocated to an abrupt onset (Theeuwes, 1992), or whether it simply influences the effectiveness of 24 voluntary, nonspatial filters (Folk and Remington, 1998). The observation that the eyes go directly to the onset before they are redirected to the target would seem to suggest that attention is being literally allocated to the distractor location. If the delay observed in manual reaction time in attention capture research were due to a nonspatial filter, one would expect that saccadic responses might also be delayed, or that there would be more saccadic errors in the presence of an onset, but there would be no reason why saccades should be systematically directed to the onset location. The fact that saccades do tend to go to the onset before they go to the target on a substantial proportion of trials suggests that attention is in fact allocated directly to the onset location. There is an important fundamental assumption underlying this interpretation of the oculomotor capture results, however, which is that eye movements and attention shifts are in fact one and the same. Indeed, Theeuwes et al. (1998) explicitly note that in their explanation for why ocuolomotor capture occurs that they "assume that the shift of attention to the location initiated the programming of a saccade to the attended location" (page 383). Thus when the onset appeared, attention shifted to the onset location, and a saccade program to the onset location was programmed and then executed before the saccade program to the target location could be completed. As will be seen in the discussion of the relationship between eye movements and attention described below, this assumption is on rather shaky ground. Oculomotor capture research has also shown that the eyes are captured by abrupt onsets and luminance increases, but not by color singletons (Irwin, Colcombe, Kramer and Hahn, 2000). Irwin et al. note that this result is consistent with Yantis and Hillstrom (1994) and Yantis and Jonides (1996) who found that abrupt onsets have a special status in capturing attention. There have been challenges to this interpretation, however. Wu and Remington 25 (2003) systematically examined the effects of sudden onsets on both attentional capture (as measured by reaction time to discriminate the orientation of a target appearing at the cued location) and oculomotor capture (as measured by the deviation of eye movements to salient but irrelevant events). They found important differences between attention and eye movements, in that sudden onsets seemed to have a special significance for eye movements, irrespective of their potential relevance to the current task goals. For attentional orienting, sudden onsets influenced reaction time only when the onset had potential relevance to the task participants were performing. It is important to note that attentional and oculomotor capture methods in the work of Wu and Remington differed in other respects aside from the response system used to measure the effect of the sudden onsets. Attention capture was measured using manual discrimination responses, and oculomotor capture was measured using saccadic localization responses. Given the importance of observer goals and strategies shown to exist in previous attention capture research (Bacon and Egeth, 1994; Folk et al., 1992; 1994), it should not be assumed that a difference in task would have no effect on capture. A more direct comparison of manual and oculomotor capture was made by Ludwig and Gilchrist (2002), who compared the effects of an abrupt onset on manual button-press localization responses, directional mouse-movement responses, and saccadic eye movements. The task was to localize a color singleton target. They observed an effect of abrupt onsets on manual button-press responses when the onset was the same color as the target, but not when it was a different color from the target. For mouse-movement and saccadic responses, in contrast, the abrupt onset influenced responses even when it was a different color from the target. They concluded that the effect of the abrupt onset is actually independent of the 26 effector used to execute the response. The task itself does appear to be important, however, given that manual button-presses produce different capture than manual mouse movements, which explains why Wu and Remington (2003) found differences between eye movements and manual responses in their experiment. One important consideration in interpreting the results of both Ludwig and Gilchrist (2002) and Wu and Remington (2003) is that onset effects were observed for the most part in reaction time for manual responses, and in accuracy for saccades, and there was almost no effect of the onset on manual accuracy or saccadic reaction time. This difference in dependent measures of capture between the two response system makes it difficult, if not impossible, to cleanly compare the results from the two response types to determine if they were actually of a similar magnitude. A final important issue in drawing comparisons between oculomotor and attentional capture is that of time. A consistent finding in the oculomotor capture literature is that faster responses are associated with- an increase in the proportion of saccades directed towards the onset (Godijn and Theeuwes, 2002; Hunt et al., 2004). In a recent exploration of the effect of timecourse on oculomotor capture, van Zoest, Donk, and Theeuwes (2004) have shown that the relative salience of the target and distractors plays a critical role for responses that are executed very soon after the target is revealed, but at longer reaction times, the eyes are more likely to be allocated in accordance with top-down attentional control settings. It is thus important to keep in mind that differences between manual and saccadic responses could be due to these two responses being based on fundamentally different representations of the target location, but it is also possible that the changes in a single representation of the search display over time could account for differences between manual and saccadic responses, given that manual responses tend to be several hundred milliseconds slower than saccades. 27 In summary, oculomotor capture has been applied to issues in visual search in two ways. First, it has been used to form a better understanding of the relationship between voluntary and reflexive eye movement programs (e.g. Theeuwes et al., 1999; Godijn and Theeuwes, 2001; Hunt et al., 2004). The difficulty with using oculomotor capture in this way is that it is not clear how exactly attention influences this competition, if at all. Second, oculomotor capture has been used as a measure of how attention is allocated in a search display (e.g. Irwin et al., 2000; van Zoest et al., 2004). However, it is not yet known whether it is valid to equate eye movement and manual capture in this way, or if in fact they reflect fundamentally different processes. Attention and Eye Movements In the following section, I examine the relationship between attention and eye movements. I first briefly review the evidence that eye movement preparation and covert attention are in fact reflections of the same process (premotor theory). I then summarize the behavioral research concerning the effects of eye movements on attention, and the effects of attention on eye movements. Finally, I review the influence of attention on neural structures involved in eye movement programming. Premotor Theory One influential hypothesis about the relationship between attention and eye movements is that they are, in fact, one and the same. This very simple and intuitively appealing model was first advanced (and rejected) by Klein (1980), but has resurfaced with 28 the moniker premotor. theory of attention (Rizzolatti, Riggio, Dascola and Umilta, 1987). The basic notion behind premotor theory is that covert attention to a location is no more or less than the preparation of an eye movement to a given location. That is, when you attend to a location in space you are preparing an eye movement there, and when you prepare an eye movement to a location, you are attending to that location. This hypothesis has found some empirical support (e.g. Rizzolatti, Riggio, Dascola and Umilta, 1987; Craighero, Nascimben and Fadiga, 2004; Smith, Rorden and Jackson, 2004), but it has also failed to be supported in many circumstances (e.g., Klein, 1980; Klein and Pontefract, 1992; Hunt and Kingstone, 2003a). The term "premotor theory" arose from a study showing that attention to cues appearing in the visual field shared spatial anisotropics with eye movements (Rizzolati et al., 1987). They observed that attention was slower to cross the horizontal and vertical midlines than to move an equal distance that did not cross a midline. Reorienting the eyes across the midline involves activating different sets of muscles rather than simply modifying the activation of already-selected muscle sets, and they assumed that this change in muscle sets would have a measurable influence on reaction time. The emergence of this pattern even though the eyes were kept at fixation suggests that covert attention mirrors the behavior expected if the muscles driving the eyes were involved. Based on this result, Rizzolati et al. concluded that covert attention is shifted to a location when the saccadic program for moving the eyes to this location is ready to be executed. When a target appears in an unexpected location, the cost of reorienting attention represents the time needed to cancel one eye movement and prepare another. A fundamental assumption of this interpretation of their data, however, is that redirecting an eye movement is slower when it must cross the 29 horizontal or vertical meridian. A study by Reuter-Lorenz and Fendrich (1992) challenges this assumption, by demonstrating that the meridian effect exists for voluntary but not reflexive eye movements. This same dissociation is mirrored when covert attention is manipulated. Reuter-Lorenz and Fendrich argue that the meridian effect is only found when voluntary attention is manipulated because both covert and overt endogenous attention rely on the same internal representation of space, in which the meridians have some special emphasis. Another line of evidence taken in favor of a mutual dependence between eye movements and attention is the effect of eye position on visual orienting (Craighero, Nascimben and Fadiga, 2004). They found that when the eye was directed straight ahead, targets appearing in the nasal and temporal visual field under monocular viewing conditions showed similar benefits of valid spatial precues. When the eye was directed 40° in a temporal direction, temporal field cueing effects were no longer significant, and nasal field benefits remained significant. The authors claim this is "clear evidence of a strict dependence of attention on oculomotor processes in neurologic ally healthy subjects" (pg. 332), but temporal field cueing effects were numerically not much smaller when the eyes were deviated in a temporal direction then when they were directed straight ahead, and there was no significant interaction between field and cueing effects to support their claim. The weakness of the statistical effect suggests further research is needed before this finding can be taken as strong evidence in favor of premotor theory. Direct support for premotor theory also comes from patient A l , who has congenital opthalmoplegia and therefore lacks the ability to make saccadic eye movements (although she has previously been reported to show a similar pattern of fixations as unimpaired people by moving her head instead, Gilchrist, Brown and Findlay, 30 1997). This patient, relative to a group of age-matched controls, shows a reduced cueing effect for reflexive but not voluntary attention (Smith, Rorden and Jackson, 2004). They make no direct statistical comparisons between A l and normal individuals, however, and do not show the results from individual participants, which is necessary to demonstrate that A l is outside the range of normality among this group with respect to reflexive attention orienting. They also do not monitor eye movements in their control participants to ensure that they are not using eye movements to complete the task. Thus it is possible that control participants are employing a strategy that is not available to A l (that is, moving their eyes to the cue or the target). Finally, the authors test a range of cue-target intervals, but only fully report results from the shortest intervals. They do mention, however, that A l does show a pattern of IOR that is similar to normal control participants. Klein (1980) was the first to directly test the suggestion that visual attention and eye movement preparation are identical, a hypothesis he termed the oculomotor readiness hypothesis (OMRH). The O M R H was promptly rejected by Klein based on evidence described in the same paper. He showed that preparation of an eye movement to a given location was not accompanied by a corresponding shift of attention there. Likewise, a shift of spatial attention to a given location was not accompanied by the preparation of an eye movement there. When the oculomotor readiness hypothesis was later resurrected in the form of premotor theory, several researchers (e.g., Rizzolati et al., 1987) criticized Klein's rejection of oculomotor readiness as premature. Klein (1980) had assumed that participants had prepared eye movements to the instructed locations but failed to explicitly assess whether or not these eye movements had actually been prepared. Critics suggested that a failure on the 31 part of participants to actually prepare the instructed eye movement was the reason Klein failed to observe any effect of this preparation on his measure of attention. That flaw was corrected in a later experiment using a dual-task procedure (Klein and Pontefract, 1994). For one set of participants, the primary task was to execute a speeded eye movement to a cued location, but on a small proportion of trials participants would instead be required to detect a visual target as quickly as possible. For a second set of participants, the primary task was to detect a visual target, but on a small proportion of trials, participants had to instead execute a speeded saccade. The primary task always showed the expected cueing effect (assuring that participants were, in fact, using the cue to prepare for the upcoming task). On the secondary tasks, however, cueing effects were not observed, suggesting that the preparation of a saccade did not result in a shift in attention, and a shift in attention did not result in the preparation of an eye movement. The O M R H was rejected for a second time. Although Klein and Pontefract (1994) had improved on the methods of Klein (1980) and confirmed his conclusion, the dual task methodology they employed opened the door to a new criticism. In their experiment, the primary task always showed the expected cueing effects, and the secondary task did not. Participants may have been sacrificing speed in the secondary task to maximize performance in the primary task, thus masking any cueing effects in the secondary task. A later experiment (Hunt and Kingstone, 2003a) eliminated this concern by pairing a speeded saccadic response with an unspeeded manual discrimination task. The results using this method again confirmed the conclusions of Klein (1980): eye movements and volitional attention were functionally independent. Participants were no faster to discriminate a visual target at a location to which an eye movement had been 32 prepared, and participants were no faster to move their eyes to an attended location. The O M R H was rejected for a third time. Despite this evidence against it, it is not uncommon to hear premotor theory invoked as a basic assumption, especially in the context of oculomotor capture (e.g. Theeuwes et al., 1998). Part of the reason for this is that there is in fact a great deal of overlap in terms of the neural activation and characteristics of eye movements and attention, which is often taken as evidence in favor of premotor theory. When interpreting these results it is important to keep in mind that although similar neural activation and characteristics of eye movements and attention support the predictions of premotor theory, it cannot be concluded on the basis of this evidence alone that attention and eye movements are dependent processes. Assuming for a moment that eye movements and attention are completely independent, information that is relevant to attention is still also relevant to eye movements, with both systems giving a clear priority to visually salient events. It would also be critical to both attention and eye movements that they be tightly coordinated, because they usually explore the environment in tandem, rather than separately. Consequently, co-activation as a result of perceptual input that is important to both systems, and the interconnection of two independent but yoked systems, might be spuriously interpreted as evidence of a functional relationship. Finding similar characteristics between eye movements and attention alone also does not provide direct support for premotor theory. Attention and eye movements both perform similar functions, in that they select specific aspects of the perceptual world for more detailed processing according to both internal goals and incoming information. In order to perform this function efficiently, they may have solved functional problems in a similar fashion. Both clearly require a mechanism for selecting, moving, and engaging a given target, and also a 33 mechanism for interrupting that process when an event occurs that is potentially more important. It would make sense that two independent brain systems, subject to similar biological constraints, would adapt to environmental pressures in a similar way. Relationship between Eye Movements and Attention in Behavior Given that attention and the eyes are able to move independently, it is important to uncover what exactly the relationship between attention and eye movements is. How does movement of the eyes influence the locus of visual attention? How does the locus of visual attention influence movement of the eyes relative to other kinds of responses? As will be seen, the answer to these questions has not yet been determined, but some important steps to completing this picture have been made. In contrast with the evidence for independence when the eyes are stationary and attention is shifted to various locations, there is a great deal of positive evidence that just before the eyes move, attention tends to be shifted to the to-be-fixated location. For instance, Shepherd, Findlay and Hockey (1986) asked participants to respond to the appearance of a peripheral target with a button press while moving their eyes in the direction indicated by a central arrow cue. Reaction time to detect the target was faster when the target appeared in the location to which an eye movement was prepared, even when the appearance of the target preceded the saccade, suggesting that eye movement preparation can facilitate target detection at the to-be-fixated location. When the direction of the saccade and the direction of attention were pitted against each other, detection was still faster at the to-be-fixated location, suggesting that saccade preparation is a more powerful determinant of the locus of attention than the endogenous probability cue. Similar results were obtained by Hoffman and 34 Subramanian (1995), Kowler, Anderson, Dosher and Blaser (1995), McPeek, Maljkovic, and Nakayama (1999), Peterson, Kramer and Irwin (2004), and Posner (1980), who all found evidence that saccades and attention move in tandem when the eyes are actually moving. Thus there appears to be a link between attention and eye movements in that attention tends to be allocated to a location to which a saccade is executed. This result may initially seem at odds with demonstrations that attention and eye movement preparation are independent (Klein, 1980; Klein and Pontefract, 1992; Hunt and Kingstone 2003a). However, in these studies, attention was measured in conditions where eye movements were prepared but not executed, whereas in studies demonstrating an interdependence of attention and eye movements, eye movements were always executed. It is speculative, but plausible, to propose that there is some "point of no return", where eye movements can be prepared without attention up to a point, and then once the saccade preparation process passes a threshold where execution of the saccade is inevitable, attention tends to be allocated to the saccade target. I use the phrase "tends to be" here because an obligatory linkage has yet to be empirically demonstrated. The evidence reviewed above suggests that attention is allocated to the to-be-fixated location at least more often than it is not, but there is also evidence that voluntary attention can be shifted in a direction opposite to the to-be-fixated location when a target is very likely to appear there (Posner, 1980). Posner suggests that the eyes and attention move in tandem by default, but if there is a good reason for them to separate, they will. One can also ask the opposite question: what is the effect of attention on eye movements? There is ample evidence that at the very least, eye movements are sensitive to the same cues that attention is, in that eye movements are faster to cued locations than to 35 uncued locations (e.g., Crawford and Muller, 1992). But there is also some speculation that eye movements can operate outside the locus of attention. For instance, Hunt and Kingstone (2003b) showed that decreasing the luminance of the target increases IOR for manual responses, but has no effect on IOR for saccades. The explanation advanced to explain why luminance did not influence IOR among saccades was that saccades to the target (which was a sudden onset) were so rapid that they occurred before attentional effects had an opportunity to accrue. The subcortical visual pathway through the SC may make it possible to execute an eye movement before attention has been allocated to the to-be-fixated location when the target does not require directed attention to be detected. Shared Neurophysiology of Attention and Eye Movements One important shared characteristic of attention and eye movements is the conceptually similar bifuraction into reflexive and voluntary subtypes, although the division has quite distinct cortical and subcortical signatures in the two systems. As discussed earlier in this introduction, voluntary attention is primarily associated with the dorsofrontal parietal network, and reflexive attention with the right ventral frontoparietal network. In contrast, voluntary eye movements depend on cortical connections from the FEF to lower oculomotor structures, and reflexive eye movements depend on the subcortical SC. However, both attention networks also involve areas of the frontal lobe that are at or near the region in the human that is homologous to the primate FEF. The FEF is also associated with the planning of voluntary eye movements, and connects to both the SC and subcortical oculomotor control structures. Understanding the role of the SC and FEF in attention has thus been one of the major focuses of research trying to understand how they are linked. 36 Research in single-unit recording and stimulation in monkeys has linked the SC to attentional control, at least when it comes to reflexive attention (e.g. Kustav and Robinson, 1996; Ignashchenkova, Dicke, Haarmeier, and Their, 2004; Mtiller, Philiastides and Newsome, 2005). As far as evidence for a functional role of the SC in attention in humans goes, however, it is less convincing. For instance, the ability to orient covert attention to peripheral cues does not seems to depend on a functioning SC (Posner, Cohen and Rafal, 1982; Posner, Rafal, Choate and Vaughan, 1985; Sapir, Soroker, Berger and Henik, 1999), and various behavioural markers thought to depend on activity in the SC have been shown to influence eye movements, but not attentional orientating (e.g., Sumner, Adamjee and Mollon, 2002; Kingstone and Klein, 1993). When it comes to the FEF, there is evidence that the FEF influences the orienting of covert attention (e.g. Corbetta, Akbudak, Conturo et al., 1998; Nobre, Gitelman, Dias and Mesulam, 2000, Moore and Fallah, 2001). But there is also evidence that saccade preparation and attention allocation are distinct neural processes within the FEF (Murthy, Thompson, and Schall, 2001; Juan, Shotter-Jacobi and Schall, 2004), highlighting the important consideration that an overlap in some of the neural structures involved in attention and eye movements does not necessarily mean that attention and eye movements are dependent processes. Summary and Conclusions Attention and eye movements are both subdivided into voluntary and reflexive subtypes, and these divisions are supported by distinct neural signatures and behavioral 37 characteristics for the two systems. There is equivocal support from behavioral studies for a necessary dependence between eye movements and attention, although eye movements and shifts in the locus of attention have similar characteristics and are associated with similar patterns of activation. The most likely explanation for the current state of the literature is still in agreement with the explanation advanced by Posner 25 years ago: the eyes and attention tend to move together, but can separate when need arises. This explanation is also consistent with the strong, but not obligatory, links that have been observed between eye movement and attention neurophysiology. This state of affairs poses some problems for oculomotor capture research, which can be divided into two broad classes: one class of oculomotor capture research applies its results to the outstanding problems and issues in selective attention and attentional capture, and the other class applies its conclusions to the goal of understanding how competition is resolved specifically within the eye movement system, invoking references to structures like the SC and FEF in their explanation of capture effects. If oculomotor capture is in fact unique to the eye movement system, conclusions specific to competition in the eye movement system are warranted, but the utility of oculomotor capture for understanding problems in attention research will be seriously undermined. If oculomotor capture instead reflects the same underlying processes that are responsible for attentional capture, conclusions about attention based on oculomotor capture will be on solid ground, but the application of results from oculomotor capture to issues in the eye movement system may need to be revisited. One of the difficulties with drawing comparisons between oculomotor capture and attentional capture, as mentioned above, is that they differ in not only in how they are measured, but also in when they occur. That is, saccadic responses tend to be executed 38 hundreds of milliseconds faster than manual responses. The fact that eye movements are fast and manual responses are slow leads naturally to a consideration of speed-accuracy trade-offs. That is, responses that are executed quickly tend to be less accurate than responses that are executed more slowly, so that when any given factor influences reaction time, it is important to also examine any changes in accuracy that accompany those changes in reaction time, and vice-versa. The divergence of oculomotor and attentional capture in terms of their influence on accuracy and reaction time might reflect actual differences in their sensitivity and response to the abrupt onset, but they also might reflect a difference in where they fall on a speed-accuracy trade off function. This is an important consideration for the experiments reported in the present thesis, particularly in Chapter 3. General Overview of Experiments This thesis follows a manuscript-based format, with the following three chapters presenting three independent studies. Each study addresses some aspect of the issues discussed in this introduction, by examining the integration of voluntary and reflexive eye movements more closely (Chapter 2), by comparing eye movements and manual responses directly, under similar response time and target duration conditions (Chapter 3), and by exploring the effects of emotional stimuli on eye movements (Chapter 4). To describe these in more detail, Chapter 2 presents research into the role of the SC in goal-directed visual search. The predictions of two pre-existing models of how various saccade goals compete for control over eye movements are tested: in one, eye movements planned in accordance with voluntary goals take place in an independent pathway from eye movements that are directed 39 reflexively to sudden onset distractors, and whichever program reaches oculomotor control structures first gains control over eye movements. In the second model, voluntary and reflexive eye movements converge at the level of the SC, where they compete for control based on the strength and timing of their activation on a common map. The experiment finds clear support for the latter model, and also in the broader context of the current thesis, provides an example of how oculomotor capture can be used to answer questions about how the oculomotor system resolves competition. It is interesting to note, however, that the models tested in Chapter 2 are agnostic about the locus of attention. Indeed, in the final paragraph of the study, it is noted that competition between voluntary and reflexive events might also be resolved above the level of the SC. In Chapter 3, a direct comparison is made between eye movement and manual localization responses in a series of four experiments. Consistent with the extant literature, capture effects among manual responses are observed in reaction time but not in accuracy, and capture effects among saccadic responses are observed in response errors but not reaction time (Experiments 1 and 2). When reaction time deadlines are imposed on responses (Experiment 3), or the duration of the target is shortened to 150ms (Experiment 4), manual responses and saccadic responses are shown to behave very similarly, in that capture emerges in response errors rather than reaction time for both types of responses. These results strongly suggest that in fact eye movements and manual responses are both based on the quality of information about the target location, and response capture in the direction of a sudden onset is not unique to the eye movement system. In Chapter 4, the effect of emotional events on eye movements is explored in two experiments. Armed with the knowledge that eye movements and manual responses reflect 40 the same processes in the attentional capture paradigm, it is possible to use eye movements in the capture paradigm as a tool for understanding how attention prioritizes events in the visual field. Previous research using manual reaction time had found evidence that emotionally negative events have a special status in attracting attention, although other research had found evidence against this hypothesis. In this study, the path and timing of eye movements in the presence of emotional targets and distractors is explored. The results demonstrate that emotion can be used to find a target, but attention is not preferentially allocated to angry emotions over happy or neutral ones. In the final chapter of the thesis, the results of these experiments are summarized and integrated, and some of the issues discussed in this introduction are revisited in light of these findings. General Methods The general methodology used in the experiments reported in the present thesis is described below. Specific methodological details are summarized in the methods section for each experiment. Participants. A l l participants were recruited from U B C undergraduate classes. The majority of participants were given credit in psychology classes for participating in the experiment, but some were paid $10.00 per hour for their time. A l l had normal or corrected-to-normal vision and gave informed consent before participating in the experiment. A l l experimental protocols used are reviewed by the U B C Research Ethics Board, which complies with the Canadian Tri-Council Policy Statement on Ethical Conduct for Research Involving Humans. 41 Apparatus. The experimental setup used in the current thesis is shown in Figure 1.2. Eye movements were recorded using an EyeLink eye monitoring device (SMI Research). This head-mounted device records eye position every four milliseconds by way of two microcameras and an infrared head camera, which records the position of four infrared emitters mounted to each corner of the display monitor. Only the left eye was monitored in all experiments. Before each experiment began, participants underwent a nine-point calibration sequence, in which participants tracked a fixation dot that appeared in locations situated in a 3x3 grid on the monitor. Most participants performed the calibration sequence several times during an experiment, whenever there was a trial break or the calibration appeared to have drifted. The joystick was a symmetrical, 8-way digital joystick mounted in an aluminum box, 25x15x6 cm, and adhered to the table by Velcro strips. Experiment programs were written in C++ and displayed on a Pentium II running Windows 95 (Chapter 2) or Windows X P (Chapters 3 and 4). The experiment was displayed on a 17-inch, 80Hz monitor, viewed from a chinrest 57cm away. Eye movements were detected using a velocity criterion of at least 30° per second. In the experiments in Chapter 2, the experimenter was seated in the same room as the participant and monitored the participant's responses on a separate monitor on the other side of the room. For the experiments in chapters 3 and 4, the participant was seated alone in a smaller room and the experimenter could monitor their progress and responses from a monitor in a small, adjacent room. 42 Figure 1.2. Experimental setup used in the experiments in the present study. The observers wore a head-mounted eye monitor and viewed trial events on a monitor 57cm away. When joystick responses were collected, the joystick was affixed to the table with Velcro directly in front of the observer. Data Analysis. After the experiment, collected data were processed using a PERL script that pulled out relevant eye movements, manual response information, and trial information from the EyeLink data file and put them in a format suitable to be processed in spreadsheets and statistical software packages. This script also calculated the distance of saccade landing positions from the items displayed on a given trial. Only the first saccade executed on a given trial was analyzed in the studies in the present thesis, although the subsequent saccades were also recorded and stored. The first saccade was defined as the first saccade following the onset of the target to exceed a distance of 22.5 pixels in any one direction. Saccadic latency was recorded as the time from the onset of the target to the initiation of the first saccade. Unless stated otherwise, only correct trials were included in saccade latency analyses. Trials 43 were excluded from analysis when blinks occurred before the first eye movement, and when the eyes were not at the center of the display when the target appeared. Error bars are used in the graphical representations of the results of each study to illustrate the overall variability in the data against which the effects of various conditions were compared. For graphs representing the results of a within-subjects A N O V A s , the error bars were calculated using the methods recommended by Masson and Loftus (2003). They suggest using error bars based the actual error term from the within-subjects A N O V A , rather than the more typical method of using the standard error of each mean, because the latter contains between-subjects variability, which is actually discarded from within-subjects ANOVAs . When between-subjects comparisons are made (that is, in Chapter 4), the error bars used in the graphs were calculated using the standard error of the mean in each condition. References Bacon, W.F., & Egeth, H.E. (1994). Overriding stimulus-driven attentional capture. Perception & Psychophysics, 55, 485-496. Becker, W. & Juergens, R. (1979) An analysis of the saccadic system by means of double step stimuli. Vision Research, 19, 967-983. Briand, K . A . & Klein, R . M (1987). Is Posner's beam the same as Treisman's glue?: On the relation between visual orienting and feature integration theory. Journal of Experimental Psychology: Human Perception and Performance, 13, 228-241. Broadbent, D.E. (1957). A mechanical model for human attention and immediate memory. Psychological Review, 64, 205-215. 44 Cameron, E.L., Tai, J.C. and Carrasco, M . (2002). Covert attention affects the psychometric function of contrast sensitivity. Vision Research, 42, 949-967. Corbetta, M . , Akbudak, E., Conturo, T.E., Snyder, A .Z . , Ollinger, J .M., Drury, H.A., Linenweber, M.R., Petersen, S.E., Raichle, M.E. , Van Essen, D.C. & Shulman, G.L. (1998). A common network of functional areas for attention and eye movements. Neuron, 21, 761-773. Corbetta, M . , Kincade, J.M., Ollinger, J.M., McAvoy, M.P., & Shulman, G.L. (2000). Voluntary orienting is dissociated from target detection in human posterior parietal cortex. Nature Neuroscience, 3, 292-297'. Corbetta, M . & Shulman, G.L. (2002). Control of goal-directed and stimulus-driven attention in the brain. Nature Reviews Neuroscience, 3, 201-215. Coren, S., & Hoenig, P. (1972). Effect of non-target stimuli on the length of voluntary saccades. Perceptual and Motor Skills, 34, 499-508. Craighero, L. , Nascimben, M . & Fadiga, L . (2004). Eye position effects orienting of visuospatial attention. Current Biology, 14, 331-333. Crawford, T.L. & Miiller, H.J. (1992). Spatial and temporal effects of spatial attention on human saccadic eye movements. Vision Research, 32, 293-304. Desimone, R. (1998) Visual attention mediated by biased competition in extrastriate visual cortex. Philosophical Transactions of the Royal Society of London, 33, 1245-1255. Desimone, R. & Duncan, J. (1995). Neural mechanisms of selective visual attention. Annual Review of Neuroscience, 18, 193-222. Driver, J., & Baylis, G.C. (1989). Movement and visual attention: The spotlight metaphor breaks down. Journal of Experimental Psychology: Human Perception & 45 Performance, 15, 448-456. Duncan, J. (1980). The locus of interference in the perception of simultaneous stimuli. Psychological Review, 14, 272-300. Duncan, J. (1981). Directing attention in the visual field. Perception & Psychophysics, 30, 90-93. Duncan, J. & Humphreys, G. (1989)/ Visual search and stimulus similarity. Psychological Review, 96, 433-458. Erikson, C.W., & St. James, J.D. (1986). Visual attention within and around the field of focal attention: A zoom lens model. Perception and Psychophysics, 70, 225-240. Findlay, J .M. (1982). Global visual processing for saccadic eye movements. Vision Research, 22, 1033-1045. Findlay, J .M. & Walker, R. (1999). A model of saccade generation based on parallel processing and competitive inhibition. Behavioural Brain Sciences, 22, 661-621. Folk, C.L. & Annett, S (1994). Do locally defined feature discontinuities capture attention? Perception & Psychophysics, 56, 277-287. Folk, C.L. & Remington, R.W. (1998). Selectivity in distraction by irrelevant featural singletons: evidence for two forms of attentional capture. Journal of Experimental Psychology: Human Perception and Performance, 24, 847-858. Folk, C.L., Remington, R.W. & Johnston, J.C. (1992). Involuntary covert orienting is contingent on attentional control settings. Journal of Experimental Psychology: Human Perception and Performance, 18, 1030-1044. 46 Folk, C.L., Remington, R.W. and Wright, J.H (1994). The structure of attentional control: Contingent attentional capture by apparent motion, abrupt onset, and color. Journal of Experimental Psychology: Human Perception and Performance, 20, 317-329. Franconeri, S.L., Hollingsworth, A. and Simons, D.J. (2005). Do new objects capture attention? Psychological Science, 16, 275-281. Friedrich, F.J., Egly, R., Rafal, R.D., & Beck, D. (1998). Spatial attention deficits in humans: A comparison of superior parietal and temporal-parietal junction lesions. Neuropsychology, 12, 193-207. Gibson, B.S. & Jiang, Y . (1998). Surprise! An unexpected color singleton does not capture attention in visual search. Psychological Science, 9, 176-182. Gilchrist, I.D., Brown, V. & Findlay, J .M. (1997). Saccades without eye movements. Nature, 390, 130-131. Godijn, R. & Theeuwes, J. (2002a). Oculomotor capture and inhibition of return: Evidence for oculomotor suppression account of IOR. Psychological Research, 66, 234-246. Godijn, R. & Theeuwes, J. (2002b). Parallel programming of saccades: Evidence for a competitive integration model. Journal of Experimental Psychology: Human Perception & Performance, 28, 1039-1054. Hoffman, J.E. & Subramanian, B. (1995). The role of visual attention in saccadic eye movements, Perception & Psychophysics, 57, 787-795. Hopfinger, J.B., Buonocore, M . H . & Mangun, G.R. (2000). The neural mechanisms of top-down attentional control. Nature Neuroscience, 3, 284-291. Horowitz, G. & Newsome, W. (1999). Separate signals for target selection and movements specification in the superior colliculus. Science, 284, 1158-1161. 47 Hunt, A.R. & Kingstone, A . (2003a). Covert and overt voluntary attention: Linked or independent? Cognitive Brain Research, 18, 102-105. Hunt, A.R. & Kingstone, A. (2003b). Inhibition of return: Dissociating attentional and oculomotor components. Journal of Experimental Psychology: Human Perception & Performance, 29, 1068-1074. Hunt, A.R., Oik, B. , von Miihlenen, A. & Kingstone, A. (2004). Integration of competing saccade programs. Cognitive Brain Research, 19, 206-208. Ignashchenkova, A. , Dicke, P.W., Haarmeier, T. & Thier, P. (2004). Neuron-specific contribution of the superior colliculus to overt and covert shifts of attention. Nature Neuroscience, 7, 56-64. Irwin, D.E., Colcombe, A . M . , Kramer, A.F. & Hahn, S. (2000). Attentional and oculomotor capture by onset, luminance, and color singletons. Vision Research, 40, 1443-1458. Jonides, J. (1981). Voluntary versus automatic control over the mind's eye's movement. In J.B. Long & A.D. Baddeley (Eds.) Attention and Performance IX, pp 187-203. Hillsdale, NJ: Erlbaum. Jonides, J. & Yantis, S. (1988). Uniqueness of abrupt visual onset in capturing attention. Perception & Psychophysics, 55, 399^111. Juan, C.H., Shorter-Jacobi, S.M., & Schall, J.D. (2004). Dissociation of spatial attention and saccade preparation. Proceedings of the National Academy of Sciences, 101, 15541-15544. 48 Kingstone, A . & Klein, R . M . (1993). Visual offsets facilitate saccadic latency: Does predisengagement of attention mediate this gap effect? Journal of Experimental Psychology: Human Perception & Performance, 19, 1251-1265. Klein, R . M . (1980). Does oculomotor readiness mediate cognitive control of visual attention? In R. Nickerson (Ed.) Attention and Performance VIII, pp 259-276. New York: Academic Press. Klein, R . M . (2004). On the control of visual orienting. In M.I. Posner (Ed.) Cognitive Neuroscience of Attention, pp 29—44. New York, N Y : Guilford Press. Klein, R . M . (2000). Inhibition of return. Trends in Cognitive Science, 4, 138-147. Klein, R . M . & Pontefract, A . (1994). Does oculomotor readiness mediate cognitive control of visual attention? Revisited! In R. Nickerson (Ed.) Attention and Performance XV: Conscious and nonconscious information processing, pp. 333-350. Hillsdale, NJ: Erlbaum. Kopesz, K. (1995). Saccadic reaction time in gap/overlap paradigm: A model based on integration of intentional and visual information on neural, dynamic fields. Vision Research, 35, 2911-2925. Kowler, E., Anderson, E. Dosher, B. & Blaser, E. (1995). The role of attention in the programming of saccades. Vision Research, 13, 1897-1916, Kramer, A.F. , Hahn, S. (1995). Splitting the beam: Distribution of attention over noncontiguous regions of the visual field. Psychological Science, 6, 381-387. Kustov, A .A . , & Robinson, D.L. (1996). Shared neural control of attention shifts and eye movements. Nature, 384,12-11. 49 Lambert, A . , Norris, A . , Naiker, N . , & Aitken, V (2000). Effects of informative peripheral cues on eye movements: Revisiting William James' "derived attention". Visual Cognition, 7, 745-769. Lu, Z.L. & Dosher, B .A. (2000). Spatial attention: Different mechanisms for central and peripheral temporal precues? Journal of Experimental Psychology: Human Perception & Performance, 26, 1534-1548. Ludwig, C.J.H., & Gilchrist, I.D. (2002). Stimulus-driven and goal-driven control over visual selection. Journal of Experimental Psychology: Human Perception and Performance, 28(4), 902-912. Masson, M.E.J. & Loftus, G. R. (2003). Using confidence intervals for graphically based data interpretation. Canadian Journal of Experimental Psychology, 57, 203-220. McPeek, R .M. , Maljkovic, V. , and Nakayama, K . (1999) Saccades require focal attention and are facilitated by a short-term memory system. Vision Research, 39, 1555-1566. McPeek, R .M. , Skavenski, A. , Nakayama, K. (2000) Concurrent processing of saccades in visual search. Vision Research, 40, 2499-2516. Moore, T. & Fallah, M . (2001). Control of eye movements and spatial attention. Proceedings of the National Academy of Sciences, 98, 1273-1276. Moore, T. & Fallah, M . (2004). Microstimulation of the frontal eye fields and its effect on covert spatial attention. Journal of Neurophysiology, 91, 152-162. Miiller, J.R., Philiastides, M . G . & Newsome, W.T. (2005). Microstimulation of the superior colliculus focuses attention without moving the eyes. Proceedings of the National Academy of Sciences, 102, 524-529. 50 Munoz, D.P. & Wurtz, (1993). Fixation cells in monkey Superior Colliculus. I. Characteristics of cell discharge. Journal of Neurophysiology, 70, 559-575. Munoz, D.P. & Wurtz, (1995a). Saccade-related activity in monkey superior colliculus: I. Spread of activity during saccades. Journal of Neurophysiology, 73, 2334-2349. Munoz, D.P. & Wurtz, (1995b). Saccade-related activity in monkey superior colliculus: II. Characteristics of burst and buildup cells. Journal of Neurophysiology, 73, 2313-2333. Murthy, A., Thompson, K . G . & Schall, J.D. (2001). Dynamic dissociation of visual selection from saccade programming in frontal eye field. Journal of Neurophysiology, 86, 2634-2637. Nobre, A.C. , Gitelman, D.R., Dias, E.C., & Mesulam, M . M . (2000). Covert visual spatial orienting and saccades: overlapping neural systems. Neurolmage, 11, 210-216. Pashler, H . (1988). Familiarity and visual change detection. Perception and Psychophysics, 44, 369-378. Peterson, M.S., Kramer, A.F. & Irwin, D.E. (2004). Covert shifts of attention precede involuntary eye movements. Perception & Psychophysics, 66, 398^4-05. Posner, M.I. (1978). Chronometric Explorations of Mind. Hillsdale, NJ: Erlbaum. Posner, M.I. (1980). Orienting of attention. Quarterly Journal of Experimental Psychology, 32, 3-25. Posner, M.I. & Cohen, Y . (1984). Component of visual orienting. In H . Bouma and D. Bonwhuis (Eds.) Attention and Performance X, pp. 551-556. Hillsdale, NJ: Erlbaum. Posner, M.I. & Cohen, Y . & Rafal, R.D. (1982). Neural systems control of spatial orienting. Philosophical Transactions of the Royal Society of London, B298, 187-198. 51 Posner, M.I., Nissen, M J . and Ogden, W.C. (1977). Attended and unattended processing modes: The role of set for spatial location. In H . Pick (Ed.) Modes of Perception, Hillsdale, NJ: Lawrence Erlbaum Associates. Posner, M.I. , Rafal, R.D., Choate, L.S. & Vaughan, J. (1985). Inhibition of return: Neural basis and function. Cognitive Neuropsychology, 2, 211-228. Remington, R.D. and Pierce, L . (1984). Moving attention: evidence for time-invariant shifts of visual selective attention. Perception and Psychophysics, 35, 393-399. Reuter-Lorenz, P.A. and Fendrich, R. (1992). Oculomotor readiness and covert orienting: Differences between central and peripheral cues. Perception & Psychophysics, 52, 336-344. Rizzolati, G., Riggio, L . , Dascola, I. & Umilta, C. (1987). Reorienting attention across the horizontal and vertical meridians: Evidence in favor of a premotor theory of attention. Neuropsychologia, 25, 31-40. Sapir, A. , Soroker, N . , Berger, A . & Henik, A. (1999). Inhibition of return in spatial attention: Direct evidence for collicular generation. Nature Neuroscience, 2, 1053-1054 Schiller, P.H. (1998). The neural control of visually guided eye movements. In J. Richards (Ed.) Cognitive Neuroscience of Attention, pp. 3-50. Erlbaum: Mahwah, NJ. Schiller, P.H. & Chou, I. (1998). The effects of frontal eye field and dorsomedial frontal cortex lesions on visually guided eye movements. Nature Neuroscience, 1, 248-253. Schiller, P.H., Sandell, J.H., & Maunsell, J.H.R. (1987). The effect of frontal eye field and superior colliculus lesions on saccadic latencies in the rhesus monkey. Journal of Neurophysiology, 57, 1033-1049. 52 Schiller, P.H., True, S.D., and Conway, J.L. (1980). Deficits in eye movements following superior colliculus and frontal eye field ablations. Journal of Neurophysiology, 44, 1175-1189. Schlag, J. & Schlag-Rey, M . (1987). Evidence for a supplementary eye field. Journal of Neurophysiology, 57, 179-200. Serences, J.T., Shomstein, S., Leber, A .B . , Golay. X . , Egeth, H.E. & Yantis, S. (2005). Coordination of voluntary and stimulus-driven attentional control in human cortex. Psychological Science, 16, 114-122. Sheperd, M . , Findlay, J .M., and Hockey, R.J. (1986). The relationship between eye movements and spatial attention. The Quarterly Journal of Experimental Psychology, 38, 475-491. Smith, D.T., Rorden, C. and Jackson, S.R, (2004). Exogenous orienting of attention depends on the ability to execute eye movements. Current Biology, 14, 792-795. Sumner, P., Adamjee, T. & Mollon, J.D. (2002). Signals invisible to the collicular and magnocellular pathways can capture visual attention. Current Biology, 12, 1312-1316. Taylor, T.L., & Klein, R . M . (1998). On the causes and effects of inhibition of return. Psychonomic Bulletin and Review, 5, 625-643. Theeuwes, J. (1992). Perceptual sensitivity for color and form. Perception & Psychophysics, 43, 307-318. Theeuwes, J. & Godijn, R. (2001). Attentional and oculomotor capture. In C.I. Folk & B.S. Gibson (Eds.) Attraction, Distraction, and Action: Multiple Perspectives on Attentional Capture (pp. 121-150). Amsterdam: Elsevier. 53 Theeuwes, J., Kramer, A.F., Hahn, S. & Irwin, D.E. (1998). Our eyes to not always go where we want them to go: Capture of the eyes by new objects. Psychological Science, 9, 379-385. Theeuwes, J., Kramer, A.F. , Hahn S., Irwin D.E. & Zelinsky, G.J. (1999). Influence of attentional capture on oculomotor control. Journal of Experimental Psychology: Human Perception and Performance, 25, 1595-1608. Trappenberg, T.P., Dorris, M.D. , Munoz, D.P. & Klein, R . M . (2001). A model of saccade initiation based on competitive integration of endogenous and exogenous signals within the superior colliculus. Journal of Cognitive Neuroscience, 13, 256-271. Treisman, A. & Gelade, G. (1980). A feature-integration theory of attention. Cognitive Psychology, 12, 97-136. Treisman, A. & Sato, S. Conjunction search revisited. Journal of Experimental Psychology: Human Perception and Performance, 16, 459^-78. Vallar, G. (1998). Spatial hemineglect in humans. Trends in Cognitive Science, 2, 87-96. Van Zoest, W., Donk, M . & Theeuwes, J. (2004). The role of stimulus-driven and goal-driven control in saccadic visual selection. Journal of Experimental Psychology: Human Perception and Performance, 30, 746-759. Walker, R., Deubel, H. , Schneider, W.X. & Findlay, J .M. (1997). Effect of remote distractors on saccade programming: Evidence for an extended fixation zone. Journal of Neurophsyiology, 78, 1108-1119. 54 Wolfe, J .M., Cave, K.R., & Franzel, S.L. (1989). Guided search: An alternative to the feature integration model for visual search. Journal of Experimental Psychology: Human Perception and Performance, 15, 419^133. Wu, S.C. & Remington, R.W. (2003). Characteristics of covert and overt visual orienting: Evidence from attentional and oculomotor capture. Journal of Experimental Psychology: Human Perception and Performance, 29, 1050-1067. Yantis, S. (1993). Stimulus-driven attentional capture and attentional control settings. Journal of Experimental Psychology: Human Perception and Performance, 19, 676-681. Yantis, S. & Egeth, H.E. (1999). On the distinction between visual salience and stimulus-driven attentional capture. Journal of Experimental Psychology: Human Perception & Performance, 25, 661-676. Yantis, S. & Hillstrom, A.P. (1994). Stimulus-driven attentional capture: evidence from equiluminant visual objects. Journal of Experimental Psychology: Human Perception & Performance, 20, 95-107 Yantis, S. & Jonides, J. (1984). Abrupt visual onsets and selective attention: Evidence from visual search. Journal of Experimental Psychology: Human Perception & Performance, 10,601-620. Yantis, S. & Jonides, J. (1990). Abrupt visual onsets and selective attention: Voluntary versus automatic allocation. Journal of Experimental Psychology: Human Perception & Performance, 16, 121-134. Yantis, S. & Jonides, J. (1996). Attentional capture by abrupt onsets: New perceptual objects or visual masking? Journal of Experimental Psychology: Human Perception & Performance, 22, 1505-1513. 55 CHAPTER 2: Integration of Competing Saccade Programs 2 At any given moment there are numerous behaviors that one could perform. Some behaviors are signaled by the environment. For instance, braking a car is signaled by a traffic light changing from green to yellow. Other behaviors are guided by internal states, and they often compete with those signaled by the environment. For instance, a hurried driver might accelerate rather than brake a car to pass through a yellow light before it turns red. These competitions between externally- and internally-triggered behaviors play out time and again in one's everyday life. This is especially true for eye movements, which are made on average four times every second. Only one location can be fixated at any given moment, so one must choose to look at some items to the exclusion of others. The subset of items that are fixated can be characterized as "winners" of a competition for control of the eye movement system. The goal of the present study is to discover how and where this competition is resolved. Until recently the assumption has been that eye movements elicited by the external environment (often called reflexive saccades) and eye movements generated internally (often called voluntary saccades) are programmed independently but at the same time (Becker and Juergens, 1979; Theeuwes, Kramer, Hahn, Irwin & Zelinsky, 1999). The eye movement program that is completed first wins the competition, and the corresponding eye movement is generated. The physiology of the eye movement system is certainly consistent with this account. There are two separate pathways that can be activated at the same time; a 2 A version of this chapter has been published: Hunt, A.R., Oik, B. , von Muhlenen, A. & Kingstone, A. (2004). Integration of competing saccade programs. Cognitive Brain Research, 19, 206-208. 56 subcortical pathway associated with reflexive eye movements runs through the superior colliculus (SC) and a cortical pathway associated with voluntary eye movements runs through the frontal eye fields (FEF, Schiller, True and Conway, 1980). Both pathways converge on subcortical eye movement control structures (see Figure 2.1, top). Race Model Voluntary •Pathway Refiex|ve:: iRathway V .Control * Gpnf lict Model Volyntary, Pathway •(REFT' 'Reflexive-.'Patlfiway V Ocubmotof, "Control; Figure 2.1. Illustration of the two models proposed to account for the competition between reflexive and voluntary saccades. FEF = Frontal Eye Fields; SC = Superior Colliculus. An alternative to this "race model" account has recently been proposed (Trappenberg, Dorris, Munoz and Klein, 2001, Godijn & Theeuwes, 2002). Rather than a race between independent eye movement systems, the competition between reflexive and voluntary saccades is fought head-to-head within the SC (see Figure 2.1 bottom). The winner gains control of eye movements. The physiology also supports this conflict model as there are 57 strong connections between the FEF and the SC (Leichnetz, Spencer, Hardy, and Astruc, 1981). As the physiology supports either account, a direct behavioral test of these two proposals may be illuminating. It is well established that turning off a fixated light influences activity within the SC and causes eye movements to be made more rapidly (Dorris, Pare, and Munoz, 1997). According to the race model only the reflexive eye movement program passes through the SC. Thus only reflexive eye movements should benefit when a fixated light is extinguished. In contrast, according to the conflict model both reflexive and voluntary eye movement programs converge and compete directly in the SC. Thus extinguishing the fixated light would affect both reflexive and voluntary saccades equally. Past research has tried to assess the two models by examining the effect of fixation offset on voluntary saccades directed away from targets (antisaccades). Using this approach, the results have been equivocal (e.g., Reuter-Lorenz, Oonk, Barnes, and Hughes, 1995; Fischer and Weber, 1992). The present investigation instead turned to the oculomotor capture phenomenon (Theeuwes et al, 1999) as a promising avenue for understanding saccade competition. The display consisted of six orange circles surrounding a central fixation (see Figure 2.2). When all the circles except one changed to red, the task was to saccade to the uniquely colored item (the singleton). Concurrent with the color change, an additional red circle appeared in the display on half the trials. This sudden onset can trigger ("capture") reflexive saccades, often outside the observer's awareness (e.g., Theeuwes et al., 1999), thus eliciting a competition between a voluntary saccade to the target singleton and a reflexive 58 saccade to the sudden onset3. On half the trials the fixation was removed from the display at the same time as the singleton was revealed. The absence of a temporal gap between fixation offset and target onset minimizes the potential for warning effects, which may not be specific to the SC, to influence the results. If the race model is correct, capture should increase when the fixation point is removed. If the conflict model is correct, removing the fixation point should decrease reaction time, but should not influence capture. • Figure 2.2. Example of a trial sequence. 1. The trial begins with central fixation and 6 possible target locations. 2. After 1000 ms, all circles change color except one (the target). On half the trials, an additional red circle is added to the display (shown here). Also on half the trials, the fixation is removed as the target is revealed. The task is to quickly saccade to the color singleton. 3 It is important to acknowledge that "voluntary" and "reflexive", as used in this and the following chapters in this thesis, reflect the terminology of the oculomotor capture literature. They do not describe discrete categories based on the attributes of the stimuli themselves, but refer to an asymmetry in the relative power of color singletons and abrupt onsets in their ability to attract an eye movement when they are irrelevant. When an abrupt onset is the target, color singletons almost never attract an eye movement (see Appendix A for an empirical demonstration of this point), but when a color singleton is the target, eye movements are executed to irrelevant abrupt onsets on a significant number of trials. Saccades to the color singleton are thus considered more voluntary than saccades to the onset, but it is also the case that some proportion of saccades to the orange circle could be reflexive (especially when no onset distractor is presented) and some saccades to the onset could be considered voluntary (that is, an oberver could explicitly decide to saccade to the onset). Methods 59 General details of the participants, the eye monitor, the display and the experimental procedure are described in the General Methods section of the introduction. In the present study, each trial began when the participant pressed the spacebar while fixating a central crosshair. Six orange circles, 3.7° in diameter, were presented equidistant from the central fixation and evenly spaced around the circumference of an imaginary circle, 25° in diameter. The orange circles occupied only the odd clock positions (e.g., 1:00, 3:00, etc., see Figure 2.2). After 1000 ms all the circles except one changed to red. Which of the six circles remained orange was randomized from trial to trial. The task was to saccade to the orange color singleton (hereafter referred to as the "target"). Concurrent with the color change, on half the trials an additional red circle appeared in one of the even clock positions. This additional circle (hereafter referred to as the "onset") was irrelevant to the task. On half the trials the fixation was removed from the display at the same time as the target was revealed. The task was to move the eyes from the fixation point to the color singleton as quickly as possible. For half of the 28 participants in the experiment, an additional discrimination task was included in order to replicate the methods used in previous experiments (e.g. Theeuwes et al., 1999). Among this group of participants, the orange circles in the initial display each contained a figure 8 (0.4° x 0.2°). When the circles changed color, segments of the 8s were removed to form letters. The target letter always contained either a C or a reversed C, and the participant's task was to press the 7' key i f a C was presented, and the ' z ' key i f a reversed C was presented. The letter was sufficiently small that it could not be easily discriminated 60 without fixating it directly. In the group who did not perform the manual discrimination task, none of the circles contained digits or letters. Each participant completed 288 trials, divided into 4 blocks of 72 trials each. For each trial, the latency and accuracy of the first saccade executed after the onset of the target was recorded. The landing position of the saccade was classified as 1) voluntary i f it landed within 30° (in polar coordinates) of the target; 2) reflexive i f it landed within 30° of the distractor onset, or 3) an error i f it went elsewhere (see the general methods section in the introduction for more information on the detection and recording of saccades). For the half of participants who performed the discrimination task, manual reaction time and accuracy were also recorded. Results Manual discrimination task. Manual accuracy was 98.1%, and there were no main effects or interactions in accuracy. In a 2x2 within subjects A N O V A on manual reaction time, with onset (present or absent) and fixation point (on or off) as factors, there was a significant effect of the onset (reaction time was 1066.5ms when the onset was absent, and 1127.1 ms when the onset was present [F(l,13)=33.15, p<.001]). There was also a significant effect of fixation offset, with manual reaction time at 1086.9 when the fixation remained on, and 1106.7ms when it was removed, [F(l,13)=8.69, p<.05]. This result initially seems surprising, because a fixation offset effect is not typically observed among manual responses (e.g., Hunt and Kingstone, 2003b; Kingstone and Klein, 1993). However, it is not surprising that an effect that is usually thought of as specific to oculomotor responses would appear in manual responses under circumstances where the speed of the manual response depends on 61 the speed of the saccadic response, in that the item could not be discriminated until it was fixated. There was no interaction between the effect of the onset and the effect of the fixation offset on manual responses [F(1,13)<1]. Eye movement results. Eye movements among participants in the manual task condition tended to be slower than among the participants who did not perform the manual task, but the effect of the onset and the fixation offset were similar and so the results for saccades across the two groups were combined for greater power. The reflexive eye movement often won out over the voluntary one, resulting in a saccade to the irrelevant onset on 22% of the trials (see Figure 2.3). To measure the effect of fixation offset on the competition between reflexive and voluntary eye movement programs, the proportion of saccades that were captured by the abrupt onset distractor were examined as a function of fixation offset once the effect of the fixation offset on reaction time had been accounted for using a reaction time binning procedure. Faster saccades are associated with an increase in capture (e.g., Theeuwes et al., 1999). To isolate the effect of fixation offset on saccade destination from an effect of overall reduced reaction time, reaction time differences between fixation-on and fixation-off conditions were removed using a quartile splitting procedure. For each participant, all onset trials where the fixation remained on were divided into quartiles, regardless of saccade destination. After establishing these quartiles, the fixation-off trials were divided into these four pre-determined bins. Because fixation-off trials tended to be faster than the fixation-on trials, there were more fixation-off trials in the fastest reaction time bin (38.8% of all the fixation-off trials), with a decreasing proportion in the slower bins (to 16.8% in the slowest bin). This procedure approximately matches reaction time for fixation-on and fixation-off trials within each bin. The analysis revealed higher 62 proportions of reflexive saccades in faster reaction time bins [F(3,81)=32.76, p<.01], but neither the effect of the fixation offset [F(l ,27)<1], nor the interaction between fixation offset and reaction time bin [F(1,27)<1], approached significance. This indicates that fixation offset does not influence the proportion of saccades directed to the onset. • Errors • Voluntary • Reflexive g 70 OS « SO IS Fl« rn Bin 1 Fix Ft* on off Bin 2 • I 1 Fix Fix an ofl Bin 3 Fix Fix. on off Bin 4 Figure 2.3. The proportion of voluntary, reflexive, and erroneous saccades for each of the reaction time bins when the fixation remains on versus when it is removed. The number of saccades to the target increases with slower reaction time bins, but removing the fixation does not increase the proportion of saccades "captured" by the onset. Error bars in this and the following figure are calculated according to the method recommended by Masson and Loftus (2003) for representing within-subjects effects. A within-subjects A N O V A on saccadic reaction time revealed a significant effect of fixation offset (28 ms, [F(l,27)=23.34, p<.01], demonstrating its classic facilitative effect on saccadic reaction time (e.g., Reuter-Lorenz et al., 1995). There was no interaction between 63 fixation offset and the type of saccade [F(2,54)=2.04], indicating that the significant effect of the fixation offset was present for both reflexive saccades to the onset, and for voluntary saccades to the target. Among voluntary saccades, the fixation offset effect was not decreased by the occurrence of the sudden onset, suggesting that competition between saccade goals does not influence the magnitude of the FO effect (see Figure 2.4). • Fixation off Target Target Onset (no onset) {onset present) Figure 2.4. The reaction time of saccades when the fixation remains on and when it is removed for three types of saccades: 1) voluntary (target-directed) saccades in the absence of an onset 2) voluntary saccades in the presence of an onset and 3) reflexive (onset-directed) saccades. Discussion The race model predicted that reflexive saccades would be selectively facilitated by removing fixation, resulting in an increase in capture in this condition. Instead, we observed that saccades are just as likely to be captured by the onset when fixation is on than when it is off (Figure 2.3), supporting the existence of a common saccade map that is influenced by the inhibitory effects of the fixated stimulus. This direct behavioral test converges with other behavioral evidence confirming the predictions of theoretical and computational models that place the competition between voluntary and reflexive saccades within the SC (Trappenberg 64 et al, 2001; Godijn and Theeuwes, 2002). Although our results suggest that voluntary and reflexive saccade programs in our experiment converged on a common map, we are not claiming that an extra-collicular route for eye movement control cannot influence eye movement behavior. In fact, an alternative route must exist, given that in primates saccadic eye movements are observed even when the SC is ablated (Schiller et al., 1980). We have found evidence that competition between sensory targets takes place on a common map, either within the SC or at some earlier stage of processing. This is consistent with the predictions of the conflict model illustrated in Figure 2.1. One outstanding issue is that the conflict model of saccade control described in the current study has advanced the working hypothesis that competition between saccade goals occurs within the SC. The current results are consistent with this hypothesis, but they are also consistent with competition between saccade goals being resolved in one or more other neural regions, with the outcome of the competition routed through the SC. Future research is needed to better understand how the FEF, SC, and brain areas involved in the control of both saccades and visual attention are able to integrate and resolve competing information. References Becker, W., and Jurgens, R. (1979). A n analysis of the saccadic system by means of double step stimuli. Vision Research, 19, 967-983. Dorris, M.C. , Pare, M . , and Munoz, D.P. (1997). Neuronal activity in monkey superior colliculus related to the initiation of saccadic eye movements. Journal of Neuroscience, 17, 8566-8579. 65 Fischer, B. & Weber, H. (1992). Characteristics of "anti" saccades in man. Experimental Brain Research, 89, 415-424.10. Godijn, R. & Theeuwes, J. (2002). Programming of endogenous and exogenous saccades: evidence for a competitive integration model. Journal of Experimental Psychology: Human Perception & Performance 28, 1039-1054. Hunt, A.R. & Kingstone, A. (2003). Inhibition of return: Dissociating attentional and oculomotor components. Journal of Experimental Psychology: Human Perception & Performance, 29, 1068-1074. Kingstone, A. & Klein, R . M . (1993). Visual offsets facilitate saccadic latency: Does predisengagement of attention mediate this gap effect? Journal of Experimental Psychology: Human Perception & Performance, 19, 1251-1265. Leichnetz, GR, Spencer, R.F., Hardy, S.G., Astruc, K. (1981). The prefrontal corticotectal projection in the monkey; an anterograde and retrograde horseradish peroxidase study. Neuroscience, 6, 1023-1041. Masson, M . E. J., & Loftus, G. R. (2003). Using confidence intervals for graphically based data interpretation. Canadian Journal of Experimental Psychology, 57, 203-220. Reuter-Lorenz, PA, Oonk, H M , Barnes, L L , Hughes, HC (1995). Effects of warning signals and fixation point offsets on the latencies of pro- versus antisaccades: implications for an interpretation of the gap effect. Experimental Brain Research, 103, 287-293. Schiller, PH, True, SD, Conway, JL. (1980). Deficits in eye movements following frontal eye-field and superior colliculus ablations. Neurophysiology, 44,1175-89. Theeuwes, J., Kramer, A.F. , Hahn, S., Irwin, D.E., & Zelinsky, G.J. (1999). Influence of attentional capture on oculomotor control. Journal of Experimental Psychology: Human Perception & Performance, 25, 1595-1608. Trappenberg, T.P., Dorris, M.C. , Munoz, D.P., Klein, R . M . (2001). A model of saccade initiation based on the competitive integration of exogenous and endogenous signals in the superior colliculus. Journal of Cognitive Neuroscience,13, 256-271. 67 CHAPTER 3: Localization by hand and eye4 Selective attention is typically characterized as having two distinct subtypes: one subtype is variously named reflexive, exogenous, bottom-up and stimulus-driven attention, and the other subtype is referred to as voluntary, endogenous, top-down, and goal-directed attention. The bifurcation of selective attention into these subtypes has been supported by distinct function (e.g., Posner, 1980; Posner and Cohen, 1984; L u and Dosher, 2000; Taylor and Klein, 1998) and underlying neurophysiology (e.g. Corbetta and Shulman, 2002; Friedrich, Egly, Rafal and Beck, 1998; Serences, Shomstein, Leber, Golay, Egeth and Yantis, 2005) associated with reflexive and voluntary attention. An important question that is the subject of heated debate in the literature is how reflexive and voluntary attention function during search of the environment for a specific visual item, particularly in the face of a range of distracting events. Voluntary attention is typically thought of as a goal-directed filter that uses expectations about the target's perceptual features to enhance certain visual channels over others in order to isolate the target from the rest of the display. Reflexive attention, in contrast, is a mechanism that selects a region for closer inspection based on inherently attractive stimulus properties. The attentional capture paradigm is typically used as a method for measuring the relative contributions of voluntary and reflexive attention to visual search 4 This paper is currently in preparation for submission, under the working title "Localization by Hand and Eye: Effects of sudden onsets on goal-directed behavior". Hunt, von Muhlenen, and Kingstone, 2005. 68 by measuring the efficiency of search for a given target when it is paired with specific kinds of distractors. Several studies have emphasized the role of stimulus properties in guiding attention. Some have argued that unique visual features in the environment are able to "capture" attention reflexively, without being relevant to task goals. For instance, a single red item among green ones will interfere with search for a specific shape, even when color is irrelevant to the task (Theeuwes, 1992). Theeuwes argues that this is because attention is initially allocated to the most perceptually salient visual event in the display. Similarly, others have shown that abrupt onsets attract attention regardless of the task goals (e.g., Yantis and Jonides, 1984). The current explanation for the special status of abrupt onsets is that they represent the appearance of a new object, and thus have automatic priority for the system that orients attention (e.g., Enns, Austen, Di Lollo, Rauschenberger and Yantis, 2001, but see also Franconeri, Hollingsworth, and Simons, 2005). Thus, when a sudden onset appears, attention is automatically drawn to its location. Manual detection or discrimination responses are consequently slower in the presence of an onset, because they await the allocation of attention to the target location before they can be executed. Other studies have emphasized the role of voluntary goals and strategies in determining what kinds of distractors will and will not interfere with search. Attention tends to be distracted by items that share some perceptual feature with the target (e.g. Jonides and Yantis, 1988). This kind of distraction effect can be thought of as a byproduct of the attempts of voluntary attention to use available visual features to home in on a subset of goal-relevant items in the environment. In fact, there is some evidence that voluntary strategies are behind capture of attention by seemingly irrelevant singletons. Bacon and Egeth (1994) 69 demonstrated that when the target is a unique item (singleton), subjects tend to adopt a "singleton detection" strategy, and thus other singletons are distracting because they share with the target the characteristic of being unique, regardless of whether or not they share any specific features with the target. When the singleton detection strategy is no longer viable for finding the target, singleton distractors no longer impede search performance. There is also some debate about whether capture by sudden onsets can be considered purely reflexive, because the extent to which performance will be impeded by onsets depends on what has been defined as the target (e.g., Folk, Remington and Johnston, 1992; Folk, Remington and Wright, 1994). Folk and Remington (1998) have suggested that delayed responding in the presence of sudden onsets does not necessarily mean that attention has been allocated to the onset location. They propose that in the presence of irrelevant events, voluntary attention imposes a filter on the visual display in order to detect the target among the increased noise, and the imposition of this filter slows the detection of the target. According to this model of visual search, there is no allocation of spatial attention directly to the onset location. One barrier to understanding how attention functions in visual search is that the debate is mainly concerned with covert visual orienting, that is, orienting attention in the absence of any overt movement of the eyes and/or head. The locus of covert visual attention must be inferred using differences in the reaction time of manual discrimination responses. In recent years, many visual search studies have begun measuring eye movements as well as, or in place of, manual responses, to better understand how covert attention is allocated during search for a target. In the first of these oculomotor capture studies, Theeuwes, Kramer, Hahn, and Irwin (1998) and Theeuwes, Kramer, Hahn, Irwin and Zelinsky (1999) required participants to 70 make a saccadic eye movement from the center of the display to a color singleton target. On half the trials an irrelevant onset suddenly appeared at the same time as the target was revealed. Theeuwes et al. observed that the eyes were frequently directed towards the sudden onset first, and then redirected towards the target, a phenomenon known as oculomotor capture. They suggest that a reflexive shift of attention to the onset initiates the programming of an eye movement there, at the same time as a program to the color singleton is initiated in response to voluntary attention allocated to the target location, and whichever eye movement program is completed first wins control over the eye movement system. One of the assumptions of Theeuwes et al. (1999) is that the cause of attentional capture is the reflexive orienting of covert attention to the sudden onset. If the delay in responding associated with sudden onsets in manual tasks were due to the voluntary implementation of an attentional filter, one would not expect error saccades to be systematically directed towards the onset. Thus the existence of oculomotor capture supports the notion that attention is captured reflexively by sudden onsets. There is reason to question the assertion that oculomotor and attentional capture reflect the same underlying processes, however. Theeuwes, Kramer, Hahn and Irwin (1998) note that they ". . . assume that the reflexive shift of attention to the new object also initiated the programming of a saccade.. .to the location of the new object." (page 383). But covertly attending to a specific location in space does not necessarily elicit the programming of a saccade to that location (e.g., Hunt and Kingstone, 2003a), and saccades and manual responses have been shown in several instances to produce very different patterns of results (e.g. Hunt and Kingstone 2003a; Hunt and Klein, 2002; Posner, Nissen and Ogden, 1978; Sailer, Eggert, Ditterich, and Straube, 2002). Indeed, the architecture of the saccadic eye 71 movement system supports the possibility that oculomotor capture is a saccade-specific phenomenon. A branch of the optic tract feeds directly into the superior colliculus (SC), a midbrain structure known to be involved in eye movement control. A recent study by Godijn and Theeuwes (2002) used findings from oculomotor capture experiments as evidence that the integration of competing saccade programs takes place within the SC (see also the previous study). The notion that the SC plays an important role in oculomotor capture is difficult to reconcile with the assertion that oculomotor capture is tapping into the same attentional mechanisms as manual discrimination responses. Whether or not attentional and oculomotor capture reflect the same underlying processes is not yet known. Two studies to date have specifically addressed this issue, and reached opposite conclusions. Wu and Remington (2003) systematically compared oculomotor and attention capture, and found that eye movements were captured by sudden onsets but not by color singletons, while manual responses demonstrated attentional capture (that is, slower reaction time to discriminate the target) in the presence of both color singleton distractors and sudden onsets. They also note that the methodology used by Theeuwes et al (1998; 1999) encourages the use of a singleton-detection strategy, which brings to light the possibility that oculomotor capture is not a purely reflexive phenomenon. When the target itself was no longer a singleton, they observed smaller capture by sudden onsets among eye movement responses, and no capture among manual responses. They take their results as evidence that oculomotor capture by abrupt onsets is not mediated by the same underlying mechanism as attentional capture measured by manual responses. This conclusion is not supported by Ludwig and Gilchrist (2002), however, who demonstrated that capture by abrupt onsets does occur for manual mouse movement responses, but not for 72 manual button-press responses like the ones used by Wu and Remington (2003). Their results suggest that attentional and oculomotor capture are similar when both are measured using directional localization responses. It is not quite accurate to say that they are the same, however, because capture for manual responses was observed in reaction time differences, and saccadic capture was always observed in the direction of saccadic responses, but never in reaction time. This difference in the effect of the distractor on these two kinds of responses is representative of the oculomotor and attentional capture literatures as a whole, with oculomotor capture typically observed using the landing position of eye movements, and attentional capture measured using reaction time. These two measures make it difficult to directly compare capture for the two conditions. Indeed, Prinzmetal, McCool & Park (2005) have suggested that accuracy and reaction time may even reflect different underlying processes of attention. They propose that accuracy effects in attentional cueing experiments reflect the enhancement of perceptual processing caused by the voluntary allocation of attention. Reflexive attention, in contrast, does not influence response accuracy, because it does not affect perceptual processes. One might be tempted to infer based on this hypothesis that oculomotor capture reflects voluntary attentional orienting, while attentional capture reflects response competition brought on by purely reflexive processes. However, Prinzmetal et al. do note that the divergence of voluntary and reflexive attention in terms of their effect on accuracy may require a specific set of circumstances: there must be no pressure on reaction time, there must be no uncertainty about the target location, and the eyes must be monitored to ensure that it is covert, and not overt, attention that is being measured. Given that attentional and oculomotor capture experiments tend to violate most, if not all of these conditions, it is not clear whether their conclusions will apply to the current experiments. 73 Another major obstacle for interpreting differences between manual and saccadic responses is the time required to execute these two responses under most circumstances. Manual responses tend to be hundreds of milliseconds slower than eye movement responses. It is therefore plausible that by the time a manual response is initiated, the representation of the target location has changed from the time an eye movement would have been executed. As time passes, information about the visual display gradually accrues, which could have important influences on the response that is ultimately executed. Indeed, even within the eye movement domain, fast eye movements have been shown to be qualitatively different from slower movements in terms of how they respond to targets in the presence of salient distractors (Van Zoest, Donk and Theeuwes, 2005). To overcome these limitations, the current study contrasts attentional and oculomotor capture using manual localizing responses, rather than detection responses. Using an 8-way joystick, the manual response task is matched as closely as possible to the oculomotor localization task. If oculomotor and attentional capture effects are reflections of the same underlying visual search processes, manual localization responses should be similar to oculomotor responses observed in previous studies (e.g. Theeuwes et al, 1998). That is, responses should be directed toward the onset, particularly when they are executed soon after the target and onset are displayed. On the other hand, it is possible that eye movements and manual responses are distinct, and the effect of the onset is different for the two response types. If onsets have a special status for eye movements but not for attention, as Wu and Remington's (2003) results suggest, manual responses would show effects in reaction time (because of the imposition of an attentional filter in the presence of onsets, Folk and Remington, 1998), but no systematic bias to respond in the direction of the onset. 74 Experiment 1 In Experiment 1, oculomotor capture was compared to joystick localization responses under the same conditions, and measured within the same group of participants. Because a digital 8-way joystick was used to record manual responses, the number of circles presented in the initial display was reduced from 6 to 4. Previous research using four locations (e.g. Irwin, Colcombe, Kramer and Hahn, 2000) has found capture similar to that using six (e.g. Theeuwes et al, 1998), suggesting that the use of 4 locations is not likely to greatly influence the pattern of oculomotor responses relative to previous experiments. The introduction of a digital joystick to the oculomotor capture methodology highlights a problem with a typical capture experiment, which is that participants could eliminate, in advance, a subset of responses as always incorrect. As can be seen in Figure 3.1, each trial begins with 4 circles already in place, and the onset always appears in a location in between these circles. To illustrate the problem with a more extreme example, i f subjects were responding with verbal compass position responses instead of eye movements, they would quickly learn that "northwest" was not among the possible correct answers, and might become extremely unlikely to make this response regardless of whether an onset was presented there or not. When using a digital joystick, participants could use a similar strategy of inhibiting responses that cannot be correct in advance, reducing the potential for capture effects. This is a particular concern because it has been shown that spatially cueing the location of the impending target eliminates capture (Theeuwes et al., 1998), and the display inherently cues a subset of locations as correct. It is thus possible that researchers have actually been underestimating the magnitude of oculomotor capture in previous experiments, 75 because the location of the irrelevant onset is never associated with a correct response (see Appendix A for more on this point). One way to address this design issue is to present the display rotated by 45° on half the trials. By randomly intermixing displays in which the four initial circles are laid out in a diamond configuration with displays in which the four initial circles are laid out in a square configuration, all four responses are made equally likely across the experiment as a whole. However, this change alone does still allow for the strategy of eliminating a subset of possible responses on a trial-by-trial bases, because participants know that only positions occupied by a circle at the beginning of the trial will represent the set of possible correct responses. To counter this, two new conditions were included (see Figure 3.1). In the two original conditions, a color singleton appeared either with or without an irrelevant onset. In the two new conditions, the color singleton was itself a sudden onset, and it could appear with or without an additional irrelevant onset. Randomly intermingling these four conditions prevents participants from preparing to respond only to certain positions, because all positions are equally likely. Methods Nine participants completed one eye movement and one joystick block, each with 216 trials. The order of blocks was randomized across participants. Eye movements were recorded using an EyeLink eye tracker (see the General Methods section of Chapter 1 for more information). The four conditions illustrated in Figure 3.1 were randomly intermixed within each block. Four orange circles, 3.7° in diameter, were presented 12.5° from the central fixation and evenly spaced around the circumference of an imaginary circle. The 76 circles in the initial display could have one of two possible configurations, in the shape of either a diamond (as in Figure 3.1) or a square. In the eye movement block, participants fixated the central crosshair to trigger the onset of the trial, and the color singleton target was displayed 1000ms after a stable fixation was detected. The task was to fixate the color singleton as quickly as possible. The destination and latency of the first eye movement executed after the target was presented was recorded and analyzed. Figure 3.1. Illustration of the conditions used in the present study. The initial display is shown at the far left, and was presented for 1000ms before being replaced by one of the four displays shown on the right. Panels 1 and 2 are the two conditions used in previous capture experiment, with onset absent and present conditions shown. Panels 3 and 4 show the two additional conditions that were added to ensure that participants were not preparing for a subset of responses in advance. The diamond configuration is shown here; note that the display was also shown rotated 45° on half the trials. What was actually orange in the experiment is shown in black here, and the color red is represented in grey. In the joystick block, participants again fixated the central crosshair to trigger the onset of the trial, but instead of moving their eyes to the target they responded using a symmetrical, arcade-style, 8-way digital joystick. The joystick was built into a metal box, 6cm high, which was affixed to the table directly in front of the participant, who was instructed to use the dominant hand to respond. Inside the joystick mechanism were four switches, one at each of the compass positions, and the program recorded which switches 77 were triggered and when. Participants were instructed to keep their eyes fixated on the center fixation during the joystick block. When participants did not follow this instruction, the experimenter paused and re-instructed them to remain fixated on the center. Responses were classified as directed towards the target, the onset, or elsewhere. For the joystick, this classification process was straightforward because there are 8 discrete responses and the mapping between the response and the target location was clear. In the eye movement block, however, the eyes' landing position was continuous, and was generally scattered around the target or onset location. To classify saccades, the display was divided into eight 45° wedges, and if the saccade landed in the 45° wedge centered around the target, it was classified as a saccade to the target, if it landed in the 45° wedge centered around the onset, it was classified as a saccade to the onset, and if it landed anywhere else, it was classified as "elsewhere". It is important to have a measure of overall errors against which to compare the proportion of responses directed specifically toward the onset. General errors can occur because of poorly-planned responses and inattentiveness, and participants may be more prone to these kinds of errors in some conditions than in others. In the present series of experiments, on trials on which no onset is presented, there is one "correct" response, directed towards the target, and then 7 additional erroneous responses that the participant could make on any given trial. On onset trials, there is one "correct" response, one "capture" response (towards the onset) and then 6 additional erroneous "elsewhere" responses that the participant could make. If there is no tendency for participants to respond in the direction of the onset, one would expect the proportion of capture to be equal to roughly 1/6* of the proportion of responses made toward locations that contained neither the target nor the onset. 78 To compare whether responses towards the onset were greater than this value, the proportion of "elsewhere" responses for each participant in the onset present condition was divided by six, and this value used as an estimate of general error for comparison to the proportion of "onset" responses. The proportion of responses directed towards the onset must be significantly greater than the general errors in a paired t-test to conclude that responses were captured by the onset. In addition, when comparing the percent of capture across conditions, the estimate of general errors was first subtracted from the proportion of capture to control for changes in the general accuracy of responses across conditions. Results For the sake of simplicity, only the results of conditions 1 and 2 (see Figure 3.1) are reported in all experiments of the current study. In general, responses tended to be faster in conditions 3 and 4, because the target was both a sudden onset and a unique color, whereas in conditions 1 and 2 it was only a unique color. Capture by the irrelevant onset was never observed in condition 4, presumably because the target was visually much more salient than the irrelevant onset. This pattern was true of both eye movement and joystick responses. See Appendix A for more details on localization responses in conditions similar to conditions 3 and 4 in the present experiment. Appendix B provides tables of the complete set of data from all the experiments in the current study, including results from conditions 3 and 4. Eye Movement Block. The results in the eye movement block are shown in the left part of Figure 3.2. There was no effect of the onset on latency to saccade to the target [t(8)=0.42]. When the onset was presented, it captured eye movements on 13.7% of trials (see Figure 3.3). This value is less than is typically observed in oculomotor capture experiments (e.g., Theeuwes et al., 1998; Hunt et al., 2004), but it exceeded general error by 12.7%, a difference that is significant [t(8)=4.79, p<.01]. Eyes Joystick Figure 3.2. The reaction time to respond to the target in Experiment 1 is shown for both eye movement and joystick responses. Error bars in this and all other graphs illustrating within-subjects interaction effects in the present study are calculated using the pooled error term of the three factors and their interactions, according to the methods for illustrating within-subjects error described by Masson and Loftus (2003). There is a significant interaction of the onset effect with response type. Joystick Block. The right part of Figure 3.2 illustrates the results from joystick responses. There was a significant effect of the onset on the latency of correct joystick responses, with responses to the target being 547.2ms when the onset was absent and 599.3ms when it was present [t(8)=5.33, p<.001]. Responses were directed towards the onset when it appeared on only 2.7% of trials (see Figure 3.3), which when compared to general error was significant [t(8)=2.36, p=.046], but exceeded it by only 1.3%, suggesting consistent but not very strong capture by the irrelevant onset. 80 arises! ansa! croee oristsl a t o m ! present absent present Eyes Joystick Figure 3.3. The proportion of responses directed towards the onset, target, and elsewhere in the display is shown for both eye movement and joystick responses in Experiment 1. There is significantly more capture for eye movement than for joystick responses. The two horizontal lines in the onset present columns here and in all subsequent graphs represent the estimate of general error (which is equal to 1/6th of the "elsewhere" proportion; see the methods section for more details). Comparison of eye movement and joystick responses. The effect of the onset was also compared within participants across eye movement and joystick blocks in a 2x2 A N O V A . Latency in the eye movement block was significantly faster than latency in the joystick block, [F(l,8)=l 17.87, p<.001], and there was a significant main effect of the onset [F(l,8)=9.19, p<.05]. There was also a significant interaction [F(l,8)=39.59, p<.001], due to the fact that there was a large effect of the onset on joystick reaction time and not on saccadic reaction time. When the proportion of capture (minus general error) was compared between response types in a paired t-test, there were significantly fewer responses directed towards 81 the onset when participants responded with the joystick than when they responded with their eyes [t(8)=4.18, p<.01]5. Discussion The results of Experiment 1 suggest there is a very small but significant effect of the onset on joystick accuracy, and a robust effect of the onset on correct reaction time for joystick responses, with slower responses when an onset is presented. In contrast, eye movements show no effect of the onset on reaction time, but a robust effect on accuracy. The finding from the joystick block replicates the findings from attentional capture research (which is typically measured in manual choice reaction time), and the findings from the eye movement block replicates findings from oculomotor capture research (which is typically measured in the accuracy of localization responses). It is also noteworthy, however, that capture even among eye movements (13.7%) was quite low in this experiment relative to previous investigations (e.g. Godijn and Theeuwes, 2002; Theeuwes et al., 1999). Joystick localization responses are much slower than eye movements in the present experiment. This is perhaps not surprising, but this observation is nonetheless important because it highlights a problem in comparing attention and oculomotor capture, even when they are both localizing responses: they are executed at different times relative to the appearance of the onset and target. The onset may slow the reaction time of manual 5 Eye movements were also recorded in the joystick block. Subjects withheld a saccade for the entire duration of the trial on 65.6% of the trials, and this proportion was not significantly affected by the presence of an onset [t(8)=0.19]. The eyes went to the irrelevant onset on 3.1% of trials, which did not differ significantly from general error [t(8)= 1.56].When eye movements were executed towards the target, their mean latency was not significantly different from joystick latency (F(1,8)<1). 82 responses because the response is withheld until attention has been allocated to the target location. Eye movements are more likely to be executed quickly, and thus errors occur in the presence of an onset because the eye movement is released before attention has been shifted to the target location. Experiment 2 In Experiment 2, reaction time feedback was introduced, and participants were urged to keep reaction time as low as possible. Increased emphasis on speed is predicted to increase capture in the eye movement condition (e.g., Godijn & Theeuwes, 2002). If attention and oculomotor capture are subserved by the same underlying processes, increased reaction time pressure may also influence accuracy in the joystick condition, and increase the number of responses directed towards the onset. Methods The methods of Experiment 2 were similar to Experiment 1, except that reaction time feedback was added. After each trial, participants were shown their reaction time in milliseconds at the center of the display, and before each block, they were instructed to keep their reaction time as low as possible. Participants were also no longer discouraged from moving their eyes in the joystick block, and instead given no instructions about where to fixate. If the participant explicitly asked the experimenter where to fixate, the experimenter instructed the participant to do "whatever felt comfortable". Nine undergraduates participated in Experiment 2. 83 Results Eye Movement Block. There was no effect of the onset on the latency of saccades towards the target [t(8)=0.11]. Overall latency was numerically faster in this experiment than in the previous one, at 299.9ms, although in a between-group t-test comparing reaction time across the two experiments, this difference was not significant [t(16)=1.17]. Saccades were directed to the onset on 32.6% of trials in which it was presented, indicating a large increase in capture relative to the previous experiment. Joystick Block. There was a significant effect of the onset on joystick latency, with a reaction time of 493.5ms when the onset was absent and 524.3ms when it was present [t(8)=3.32, p<.05]. Latency is faster in this experiment than in the previous one, and in a between-subjects t-test, this difference is marginally significant [t( 16)= 1.99, p=.064], suggesting reaction time pressure was successful in reducing reaction time, at least among joystick responses. There was no effect of the onset on the proportion of responses correctly directed towards the target [t(8)=0.29], and only 3.1% of responses were directed towards the onset, a proportion that was not significantly different from general error [t(8)=0.33]. Relative to the previous experiment, the proportion of responses toward the onset is greater (3.1% versus 2.7%), but so is the overall proportion of responses toward other nontarget distractors (15.6% versus 7.9%), resulting in nonsignificant capture. •yes Joystick Figure 3.4. The reaction time to respond to the target in Experiment 2 is shown for both eye movement and joystick responses. There is a significant interaction of the onset effect with response type. Comparison of eye movement and joystick responses. Joystick and eye movement latency were included in a 2x2 A N O V A with response type (eye movement versus joystick) and onset (present versus absent) as factors. There was a significant effect of response type [F(l,8)=129.88, p<.001], with joystick responses slower than saccades, no main effect of the onset [F(l,8)= 1.62], and a significant interaction between response type and onset [F(l,8)=7.39, p<.05] generated by the significant effect of onset in the joystick condition and the lack of an onset effect in the eye movement condition. Not surprisingly, when the proportion of capture in the joystick and eye movement blocks were directly compared in a paired t-test (after correcting for general error), there was significantly more capture in the eye movement block [t(8)=4.52, p<.01], see Figure 3.5. 6 6 In the joystick block, eye movements were executed on 87.9% of the trials. Like Experiment 1, there was no significant difference in the latency of saccades in the joystick block versus joystick responses [F(1,8)<1], with eye movements in fact slightly slower (523.4) than joystick responses (508.9ms). The eye went to the onset on 13.2% of trials, 85 100 80 40 20 H 0 • m m • III 1 tlllJII --=1 • to o nset f~| to target • to efeewfieie onset onset absent present Eyes onset onset absent present Joystick Figure 3.5. The proportion of responses directed towards the onset, target, and elsewhere in the display is shown for both eye movement and joystick responses in Experiment 2. Joystick onset proportions are not significantly different from the estimate of general error (shown as the white line in the onset present columns). Discussion of Experiments 1 2 Eye movements show consistent capture in accuracy, and joystick responses show consistent capture in latency, similar to what has been observed in these two literatures previously. There was some evidence from Experiment 1 that joystick capture for localization responses does occur, but it was very small compared to eye movement accuracy, and did not increase from Experiment 1 to Experiment 2. In fact, in Experiment 2, which is significantly different from chance [t(8)=3.90, p<.01], and significantly greater than the proportion of onset-directed joystick responses [t(8)=4.89, p<.01]. The latency to move the eyes to the onset on these trials was also significantly faster than the joystick response latency (325.0 vs. 602.5ms [t(8)=2.90, p<.05]) suggesting that on these trials the eyes were directed to the onset first, and then a joystick response to the target usually followed. 86 it was no longer significantly different from general errors. There was no evidence that eye movement reaction time capture occurred. The results of experiments 1 and 2 taken together suggest that capture as measured by joystick response is unique from that measured by eye movement responses, in line with the observations of Wu and Remington (2004). Given that reaction time pressure had little effect on this pattern, one might be tempted to conclude that the source of the dissociation between these two response systems lies in a fundamental difference in how attention is influencing eye movements versus manual responses. However, the two types of responses have not yet been equated for speed, with joystick responses averaging over 200ms slower than eye movement responses, even with the pressure placed on reaction time in Experiment 2. It is still possible that the difference in pattern across the two response types is due to differences in the reaction time between the two response types. Generally speaking, fast responses tend to be less accurate, with accuracy increasing as responses are withheld for longer periods of time. This basic speed-accuracy trade-off function is an important consideration for any measure of performance, because it demands that reaction time and accuracy both be taken into account when assessing performance differences. To assess the role of speed-accuracy trade-offs in differences between attentional and oculomotor capture, the effect of reaction time on the proportion of capture was examined. To accomplish this, the data from the onset present trials of Experiments 1 and 2 were divided into quartiles based on reaction time. Quartile ranges were calculated separately for each participant and for each response type. The quartile to which a given trial belonged was then used as a factor in a two-way A N O V A on the proportion of capture (minus general 87 error), with the other factor being response type. The results are graphically represented according to the mean reaction time for each quartile (see Figure 3.6). 100 • -Experiment I Experiment^ •i Eye'Movement Responses Joystick-Responses? 200 300 400 500 fteaificm Time 600 700 800 Figure 3.6. The proportion of capture in Experiments 1 and 2 is shown as a function of reaction time quartile. Each square on the graph represents the mean reaction time for that quartile and the mean proportion capture for responses in that reaction time range. In Experiment 1, the significant effect of quartile [F(3,24)=10.17, p<.01] interacted with response type [F(3,24)=7.02, p<.01]. The same pattern was observed in Experiment 2, with a significant effect of quartile [F(3,24)=16.42, p<.01] and a significant interaction with response type [F(3,24)=15.61, p<.01]. The source of both of these interactions is clear from Figure 3.6: there is a large effect of quartile on saccadic reaction times, similar to that observed in previous oculomotor capture research [Experiment 1 F(3,24)=9.35, p<.001, Experiment 2 F(3,24)=17.39, p<.001]. There is no effect of quartile on joystick capture [Experiment 1 F(3,24)<1; Experiment 2 F(3,24)=2.27]. One might once again be tempted to conclude that oculomotor capture and attentional capture are unique, because speed of response has a different effect on capture for the two 88 types of responses. It is clear from Figure 3.6, however, that another interpretation is also possible. There is very little overlap in the reaction time distributions of the manual and saccadic responses. Indeed, from this graphical representation of the data it is easy to imagine that manual and saccadic responses share the same function of increasing capture with faster reaction times. That is, oculomotor and attentional capture are in fact reflections of the same underlying process, but fall on different points of the same speed-accuracy trade-off function. Experiment 3: Reaction Time Deadline In Experiment 3, the reaction time distribution of eye movement and joystick responses are brought closer together by introducing reaction time deadlines. Faster reaction times are associated with increases in capture among eye movement responses (e.g. Godijn and Theeuwes, 2002), but it is not known if capture among manual responses will be influenced by pressure on reaction time to a similar extent. If the difference between attention and oculomotor capture is due to speed-accuracy trade-offs, then when the reaction time differences are eliminated, joystick capture should emerge in accuracy instead of reaction time, and be similar in magnitude to eye movement capture. Alternatively, increasing reaction time pressure on joystick responses may reduce response accuracy overall, but not increase the proportion of joystick responses directed toward the onset. This would suggest that attentional and oculomotor capture are not simply sampling different timepoints of the same process, but instead measure different underlying processes. 89 Methods Twelve participants completed two sets of three blocks of trials, with each set of three beginning with a 500ms reaction time deadline in the first block, then a 400ms deadline in the second block, and a 350ms deadline in the final block. Half the participants completed a set of three blocks of eye movement trials followed by a set of three blocks of joystick responses, and the other half completed the sets of three in the opposite order, with 3 blocks of joystick trials followed by three blocks of eye movement trials. Each block contained at least 96 trials. The setup and task were similar to the previous two. The task was to execute a response in the direction of the target (the orange circle) as quickly as possible. In the eye movement block, participants were instructed to execute an eye movement from the central fixation crosshair to the orange target as quickly as possible. In the joystick block, participants moved an 8-way digital joystick in the direction of the target as quickly as possible. After each response, feedback was displayed for 500ms. If the response had been made within the deadline, the feedback message said either "Wrong" (if the joystick response was incorrect, or in the saccade block if the saccade landed outside a 45° wedge centered around the target), or "Right" (if the joystick response was correct, or if the saccade landed within the 45" wedge around the target). When the trial was "right" the reaction time in milliseconds was also displayed. If the response was executed after the deadline, the message displayed was "Too Slow". When the response was too slow, the trial was recycled. In the 350ms joystick response condition, it was very difficult for many participants to execute a response within the allotted time, and trials were recycled very frequently. To prevent this 90 block from running for a very long time for these participants, a limit of 192 was placed on the total number of trials in a given block. Six participants reached this limit, with an average of 83 trials completed within the deadline in the 350ms deadline block (the lowest number of trials completed in this block was 48)7. Results Reaction Time (joystick and saccade responses). Reaction time results were submitted to a 3-way within-subjects A N O V A with response (eye movement or joystick), reaction time deadline (500, 400, or 350ms) and onset (present or absent) as factors. The results are shown in Figure 3.7. There was a main effect of response [F(l,ll)=41.41, p<.001], a main effect of reaction time deadline, with faster reaction times when the deadline decreased, [F(2,22)=18.16, p<.001], and a main effect of onset [F(l,ll)=11.28, p<.001]. The main effect of onset was actually opposite to previous experiments, with faster responses when the onset distractor was present than when it was absent. There was an interaction of reaction time deadline with response type [F(2,22)=9.59], due to a larger effect of reaction time deadline on joystick reaction time than on saccadic reaction time (because eye movement responses are on average faster than 400ms in this task, there was not as much change across reaction time deadlines as there was for joystick responses, which tend to be slower than 400ms). There were no other significant interactions, including the interaction of reaction time deadline with the onset effect [F(1,11)<1]. 7 It was for this reason that no quartile analysis is shown for this experiment. The reduction in data for some participants, especially in the critical 350ms joystick condition, made further division of data into quartiles unfeasible. 91 600 r 500 h | onset absent • onset present 350 400 Eyes 500 1 Joystick Figure 3.7. The reaction time to respond to the target in Experiment 3 is shown for both eye movement and joystick responses. Responses had to be made within a deadline of 500, 400, or 350ms or the trial would time out. Accuracy (joystick and saccade responses). Figure 3.7 depicts the landing position of the first saccade executed after the target was presented. As in the previous experiment, the probability of a response landing specifically on the onset location by chance was calculated by taking one-sixth of the proportion of "elsewhere" responses. When the percent of capture trials (with general errors subtracted) was submitted to a 2-way A N O V A , with response type (eye movement or joystick) and target deadline (500, 400, or 350) as factors, there was no effect of response type, [F(l , l 1)=1.56], and a main effect of reaction time deadline [F(2,22)=5.16, p<.01], with responses becoming captured more frequently with decreasing reaction time deadline. The interaction of response type and reaction time deadline was not 92 significant [F(2,22)<1], suggesting that the pattern of increasing capture with decreasing deadline was similar for both eye movement and joystick responses8. The proportion of capture was significantly greater than chance in all three eye movement conditions [short t(ll)=4.90; medium t(ll)=4.04; long t(ll)=3.92] and in all three joystick conditions [short t(ll)=6.20; medium t(ll)=3.42; long t(ll)=3.37]. Eyes Joystick Figure 3.8. The proportion of responses directed towards the target, the onset, and to other locations on onset-present trials. Both eye movement and joystick responses had to be made within a deadline of 500, 400, or 350ms. It is in some respects surprising that there is a similar effect of reaction time pressure on eye movement and joystick responses in this experiment. Whereas joystick responses are typically slower than 500ms, making it difficult for participants to respond within shorter deadlines, eye movements tend to be faster than 350ms on average, and thus reaction time pressure had little effect on saccadic reaction time. One might have therefore expected that eye movements would show similar capture across deadline conditions. In Experiments 1 and 2, RT pressure likewise had no significant effect on RT for saccades, but there was an increase in the proportion of capture from 13.7% to 32.6%. Perhaps mean saccadic latency is already so fast that subjects are unable to respond any faster, but the overall sense of time pressure causes a shift in criteria, where subjects respond based on less complete information about the target location because of anticipated trial timeouts. 93 Discussion The results are consistent with the prediction that the differences between eye movement and joystick capture in Experiments 1 and 2 reflected a greater bias to sacrifice speed for accuracy among saccade responses than joystick responses. When reaction time differences are reduced by speeding joystick responses, the difference in capture is eliminated, with similar levels of capture in accuracy for both types of responses. This result suggests that the direction of both eye movement and joystick responses is based on the same underlying representation of the onset and target, and reflect the same locus of attention, when the response is executed within the same time period following the appearance of the target and the onset. Importantly, this pattern is not unique to the eye movement system, but is also observed among manual localization responses. Experiment 4 A response that is executed very quickly differs in at least two respects from a response that is executed more slowly. First, when pressured to respond quickly, overt responses might be poorly prepared, resulting in an increase in errors due to a failure in the accuracy of motor output to accurately reflect the target location. This explanation for capture is implied in the model developed by Godijn and Theeuwes (2001), in which information about the target location converges onto a saccade map within the superior colliculus, where it competes with information about the onset location. Top-down inhibition of specific nontarget locations prevents the automatic capture of sudden onsets by irrelevant 94 but salient visual events, and capture occurs when a response is executed before this inhibition has been fully instantiated. Second, the total processing duration of the target at the point when the response is executed is shorter when responses are executed quickly. The subsequent decrease in the amount of information about the target location on short- reaction time trials could cause an increase in responses to the onset due to uncertainty about the target location. Experiment 4 examines the effect of target information by shortening the target duration but no longer pressuring participants to respond within a certain deadline. This also allows for a simultaneous exploration of the effects of target information (by examining the effects of target duration) and response time (by examining capture effect in a reaction time quartile analysis). Methods There was a total of 30 participants in Experiment 4, which was similar to Experiment 3, except instead of manipulating reaction time deadline, target duration was manipulated. Each trial began with four orange circles around a central fixation crosshair. After 1000ms, the target was revealed when the distractor circles changed to red. After a set duration, the distractor circles changed back to orange, making it no longer possible to discriminate the target from the other circles. Twelve participants completed three blocks for each response type (eye movement versus joystick responses) with target durations of 150, 250, and 350ms, in that order. A second group of eighteen participants completed three blocks for each response type with a target duration of 350, 400, or 500ms, again in that order. The order of response type was counterbalanced across participants for both groups. 95 Results Reaction time (eye movement and joystick responses). Reaction time results were submitted to a 4-way mixed A N O V A , with response (eye movement or joystick), target duration (short, medium and long) and onset (present or absent) as within-subjects factors, and group as a between subjects factor. There was a main effect of response type [F(l,17)=181.85, p<.001], and a main effect of target duration [F(2,56)=6.64, p<.05]. There was no effect of the onset [F(l,28)=2.83], or of group [F(1,28)<1], but these factors were involved in interactions. There was an interaction of response type and the onset effect, with a larger effect of the onset on reaction time for joystick responses than for eye movements [F(l,28)=6.98, p<.05]. There was also a significant three-way interaction of response type, target duration, and group [F(2,56)=9.64, p<.001]. The source of this interaction can be seen in Figure 3.8: joystick reaction time increased with increasing target duration in the 150-350ms group [F(2,22)=23.81, p<.001], but not in the 350-500ms group [F(2,22)<1], while eye movement reaction time was not systematically influenced by target duration in either group [F(2,46)<1]. 96 Eye Target Duration Joystick Target Duration Figure 3.9. The reaction time to localize the target in Experiment 4 is shown for both eye movement and joystick responses. The orange color singleton target was revealed for 150ms to 350ms (one group) or for 350 to 500ms (another group), and then the other items would turn orange as well, hiding the target location. Accuracy (eye movement and joystick responses). The proportion of first responses directed towards the onset (with general error subtracted) was submitted to a 3-way mixed A N O V A , with response type (eye movements or joystick) and target duration (short, medium, and long) as within-subjects factors and group as a between-subjects factor. A l l three main effects were significant: response type, [F(l,28)=8.08, p<.01]; target duration [F(2,56)=3.91, p<.05], and group [F( 1.28)= 13.47, p<.001]. There was a significant two-way interaction of response type and target duration [marginally; F(2,56)=3.03, p=.056] and a significant interaction of group and target duration [F(2,56)=4.07, p<.05], and these were incorporated in a three-way interaction involving all three factors [F(2,56)=3.47, p<.05]. The source of this interaction is similar to that observed among reaction times, and can be seen in Figure 3.10; the proportion 97 of responses captured by the onset increases with decreasing target duration among joystick responses in the 150-350ms group [F(2,22)=5.61, p<.05], but not in the 350-500ms group [F(2,34)<1]. Eye movement capture is not influenced by target duration in either group [F(2,58)=1.42]. 100 80 _ 60 c E d 40 20 0 4 I ' to onset Id taigel to efeewhei ISO ZS'O 3S0 360 500 ISO 2S0 350 3SS 400 SCO Eye Target Duration Joystick Target Duration Figure 3.10. The proportion of eye movement and joystick responses directed towards the target, the onset, and to other locations on onset-present trials. The duration of the target was varied between 150 and 500 ms. When compared against general error, the proportion of capture was significant in all joystick conditions [group 1: 150ms target duration t(l 1)=4.82; 250ms target duration t(ll)=5.69; 350ms target duration in the t(ll)=5.29, all ps<.001; group 2: 350ms target duration t(17)=2.23; 400ms target duration t(17)=3.46, 500ms target duration t(17)=2.85, all 98 ps<.05]. The proportion of capture was also significantly greater than chance in all six eye movement conditions [group 1: 150ms target duration t(ll)=2.79; 250ms target duration t(ll)=5.69; 350ms target duration t(ll)=4.31, all ps<.05; group 2: [350ms target duration t(17)=3.74 400ms target duration t(17)=2.60; 500ms target duration t(17)=5.71 all ps<.05]. Quartile Analysis. The effect of response time and target duration were examined together in a mixed 4-way A N O V A with group, response type, target duration, and quartile as factors. Figure 3.11 shows the comparison of eye movements and joystick responses collapsed across response duration. There was a main effect of quartile [F(3,84)=5.66. p<.01], and quartile was involved in a four-way interaction with group, response type and target duration [F(6,168)=2.63, p<.05]. To break this down, joystick and eye movement conditions were analyzed separately. For joystick responses, there was a main effect of quartile [F(3,84)=2.08, p<.05] but it did not interact with group or target duration. The source of the main effect of quartile on joystick responses can be seen in Figure 3.11: the fourth quartile actually shows significantly more capture than the second [t(29)=2.48, p<.05] and third [t(29)=2.74, p<.05], and none of the other comparisons between quartiles are significant. It is important to note that this pattern of more capture in the fourth quartile appears to based primarily on the pattern across quartiles in the short duration group, although there is no significant interaction between group and quartile for the joystick responses [F(3,84)=2.08]. For the analysis of eye movement responses alone, there was also a main effect of quartile [F(3,84)=6.98, p<.001], but quartile was also involved in a three-way interaction with group and target duration [F(6,168)=2.78, p<.05]. 99 35 r 9 ' Long Target Duration Group, [ j Short Target DuiationGiuup Eye Movement Responses I 30 I 25 - \ 1 20 h 2 3 \ 1 i o -5 -0 I f © 200 300 500' BOO Figure 3.11.The proportion of capture for joystick and eye movement responses is shown as a function of quartile and group. In the long target duration group (black squares), targets were presented for 500, 400, or 350ms. In the short target duration group (gray squares), targets were presented for 350, 250, or 150ms. Each square represents both the mean reaction time in that quartile and the proportion of capture (minus an estimate of general error). To further break down the three-way interaction of target duration, group, and quartile among the eye movement responses, six one-way A N O V A s were conducted to examine the effect of quartile on capture at each of the six target durations (500, 400, and 350ms in one group, and 350, 250, and 150ms in the other group). There was a significant or marginally significant main effect of quartile for all target durations except the 150ms target duration [500ms F(3,51)=3.36, p<.05; 400ms F(3,51)=2.56, p=.065; 350ms (in longer duration group) F(3,51)=4.56, p<.01; 350ms (in shorter duration group) F(3,33)=3.06, p<.05; 250ms F(3,33)=2.55, p=.072]. In the 150ms target duration group, the effect of quartile was not significant [F(3,33)<1]. This pattern can be seen in Figure 3.12, which shows the effect of quartile for each of the duration groups in the eye movement condition. The short duration 100 (thick line) in the shorter target duration group (Figure 3.12B) is the one condition where the effect of quartile was not significant. 1 30 f 25 Target Duration — SOOms <400rt« J5C-ms 200 300 400 Reaction Time {ms) 500 600 35 1- 30 •ej'. •• • • 1 2 f 'TO:... ft; . ... 1 10 0 Target Duration 350mi ^Sprite 200 300 400 ReaGtion'Tjrne^ms) 600 Figure 3.12. The proportion of capture for eye movement responses is shown as a function of quartile and group. In the long target duration group (A), targets were presented for 500, 400, or 350ms. In the short target duration group (B), targets were presented for 350, 250, or 150ms. Each square represents both the mean reaction time in that quartile and the proportion of capture (minus an estimate of general error). 101 Discussion The manipulation of target duration in Experiment 4 revealed that capture increases among shorter target durations. This pattern mirrors the increase in capture among shorter reaction times observed in Experiment 3, and suggests that the amount of available information about the target location is a critical factor in whether or not capture will occur. As can be seen from Figure 3.10, shorter target durations did not systematically increase the number of general errors, but specifically influenced the proportion of responses directed towards the sudden onset. This pattern is also illustrated in Figure 3.11, with increased capture when the target duration is short (gray squares) relative to when the target is presented for a longer period of time (black squares). It is also evident from Figure 3.11 that eye movement capture decreases steeply as reaction time slows. Joystick capture, in contrast, reveals the opposite pattern, with capture increasing as reaction time slows. When plotted together, the proportion of capture for eye movement and joystick responses generates a U-shaped function across reaction times. This pattern can also be accommodated by the notion that the total amount of available information about the target location is a critical factor in whether or not capture will occur. Faster responses are captured because information about the target location has not yet had time to accrue, while slower responses are captured because the target location information has decayed significantly. One exception to the effect of decreasing capture across quartiles among eye movements described above is the very shortest target duration condition. As can be seen in Figure 3.12, there is a decrease in the proportion of capture in slower reaction time quartiles 102 in all conditions except the 150ms target duration condition, where it remains high across all quartiles. This result is also easily interpreted in terms of available information: when the target location information is removed very quickly, there is no longer any benefit to be gained by withholding a response, because no further information about the target location can be gathered during this interval. General Discussion The results of the current study suggest that a fruitful approach to understanding capture is in terms of the acquisition of information about the target location over time. As Posner (1978) describes in his seminal work Chronometric Explorations of Mind, a fundamental assumption of reaction time studies is that information about the visual environment is accrued over a measurable period of time, and the observer is able to access this information at different points in this accrual process. Thus a response executed soon after a target is displayed is more likely to be inaccurate, and later responses are more likely to be correct. A basic function describing this speed-accuracy trade-off process is shown in Figure 3.13. As time passes, the quality of information increases, until it reaches asymptote and there is no longer any benefit to accuracy associated with slower reaction times. 103 Figure 3.13. Function describing changes in information about the target location over time. As time passes, information about the target location increases, and then remains at asymptote. If the target is removed, there is decay of information about the target location. In most circumstances, saccades tend to be executed earlier in this curve than manual responses (their response time distributions along this curve are represented by the large gray bars). This model can be applied to the current results, and to the attentional capture phenomenon more generally. Presumably, the presence of a sudden onset creates greater uncertainty about which display element is the target. More detailed, higher-quality information about the target will have to be acquired before this uncertainly is resolved relative to when no onset is presented. Because it was the introduction of the onset to the display that created the additional uncertainty about the location of the target, when responses are executed before uncertainty about the target is resolved, they will often be directed towards the onset instead of toward the target. Experiments 1 and 2 demonstrated that joystick responses show consistent capture in latency, and eye movements show consistent capture in accuracy. This pattern replicates the general state of the literature, where attention capture tends to be measured in manual 104 reaction time differences, and oculomotor capture tends to be measured in eye movement accuracy. The reaction time distribution of saccadic and manual responses is represented by gray bars in Figure 3.13. Eye movement responses are initiated based on less information about the target location than manual responses, and as a result tend to be faster but less accurate. Manual responses tend to be executed at a point in the curve at which there is sufficient information about the target location that the response is reasonably accurate. The difference between manual and saccadic responses in terms of the curve shown in Figure 3.13 also provides an explanation as to why, in Experiments 1 and 2, eye movement responses showed an effect of response time quartile, and manual responses did not. Saccadic responses are executed at a point when the accrual of information about the target is still taking place, and thus small changes in reaction time will have a large effect on accuracy. Even very fast manual responses are typically executed after asymptote has been reached, and thus slower reaction times do not lead to increases in accuracy. This interpretation is especially evident in Figure 3.6, in which manual and saccadic capture are plotted as a function of reaction time. When forced to be executed faster, as they were in Experiment 3 of the present study, the manual response distribution moves leftward on the curve and becomes less accurate, and more similar to saccades. This result demonstrates that both manual and saccadic responses are directed toward the onset when they are executed soon after the target and onset appear, supporting the assertion that both respond based on the same information, but that this information produces different results when it is accessed at different points in time. When the duration of information about the target is manipulated directly, as it was in Experiment 4, an increase in capture under conditions of lower quality of information is 105 observed. At long target durations, eye movements showed more capture in accuracy and joystick responses showed more capture in reaction time, similar to the pattern observed in Experiments 1 and 2. In general, shortening the target duration produced an increase in capture. As illustrated by the dotted line in Figure 3.13, when the target is removed, the quality of information about its location begins to decay. This produces a U-shaped function of the proportion of capture across reaction time, with higher proportions of capture at very short and very long reaction times, like that observed in Figure 3.11. The decay of information about the target location can also explain the results from the 150ms target duration condition, in which saccades show no effect of quartile on the proportion of capture. A plausible explanation for this result is that when the target is removed after only 150ms, information about its location has not yet had time to reach asymptote. Thus saccades and manual responses alike have a high proportion of capture across their respective reaction time distributions9. The model of capture described above can also help to explain some of the patterns observed in other capture studies, if one assumes that some kinds of information will be picked up faster than others. For example, items that are higher in luminance will have a faster rate of information gain than items that are lower in luminance, and color singletons would have a faster rate of information gain because of the lack of competing stimulation from other locations in the display. In combination with differences in the speed of 9 It is interesting to note that 150ms seems to be a "magic number" in the attentional capture literature. Theeuwes, Atchley & Kramer (2000) displayed an irrelevant singleton at various points in time before the presentation of a target, and found that capture only occurs when the distractor is shown less than 150ms before the target. At longer temporal separations, the onset no longer interferes with discrimination of the target. A similar timecourse has been observed elsewhere (Kim and Cave, 1999). The current results converge with these findings, and suggest that it takes around 150ms for information about elements in the visual display to reach a maximum quality. 106 information accrual from different regions and objects, focused attention can also increase the rate of accrual of information from a selected location, object, or dimension (e.g. Hawkins, Shafto and Richardson, 1988). In other words, information from both salient and from visual features selected by attention would accrue more rapidly than information from the rest of the display. When salient and selected visual features work in concert, information about the target location will accrue very rapidly, and responses will be fast and accurate. Distractors that are very salient or similar to the target will increase the quality of information needed to identify the target, thereby slowing the decision time. As reviewed in the introduction, a source of debate in the attentional capture research is whether and when reflexive and voluntary processes will play the primary role in visual search. Underlying this debate is the notion that attention has a single locus, like a spotlight, that seeks out the target by moving from item to item, and can be moved strategically, based on expectations about the target, or it can be drawn to items reflexively, based on their intrinsic properties. This characterization of attention engenders debate about how voluntary and reflexive processes are able to wrest control of this search mechanism from one moment to the next. If one instead considers the search process in terms of accrual of information over time, the notion of reflexive and voluntary control as being mutually exclusive or serial processes becomes vestigial. The information-accrual model of capture also has the advantage of being generic enough to apply to any other situation where a target must be identified in the presence of distractors, including nonspatial situations, such as attention to global motion or auditory pitch. A secondary, but important, conclusion that can be drawn from the present study is that manual and eye movement responses reflect the same underlying process. This 107 conclusion will be welcome news to researchers who have been using oculomotor capture to understand the processes underlying visual search (e.g. Irwin, Colcombe, Kramer & Hahn, 2000; Theeuwes et al. 1999; van Zoest, et al., 2004). Eye movements are in fact able to sample a point in the target localization process that can provide information that is usually not provided by manual responses. Two important questions remain to be answered, however. One is why eye movements tend to be executed based on less information, while manual localization responses tend to be delayed until uncertainty about the target location has been resolved. A plausible explanation is that a manual response represents a larger investment of energy than an eye movement, and if the hand has been guided to an irrelevant location, it takes more energy and time to correct it than an eye movement. There is also a large payoff in moving the eyes in terms of information gain, where the higher-acuity fovea allows for more detailed information to be picked up from the fixated location. Presumably, valuable information is also gained from kinetic and tactile information that manual manipulation provides, but the cost in terms of time and energy is large enough that this system waits until after the accrual of visual information is complete before investing in moving towards a specific target. A second important outstanding question is how exactly attention influences the accrual of information about specific elements or features. This is a fundamental question, and researchers exploring the effect of attention on basic visual processes have begun providing important pieces of this puzzle. For instance, attention has been shown to enhance spatial resolution (Yeshurun and Carrasco, 1998), accelerate visual processing (Carrasco and McElree, 2001), and inhibit the processing of unattended signals (Dosher and Lu, 2000). Experiments exploring how selective attention influences and combines with basic perceptual 108 processes will likely also contribute greatly to a more complete understanding of visual search and attentional capture. References Bacon, W.F., & Egeth, H.E. (1994). Overriding stimulus-driven attentional capture. Perception & Psychophysics, 1994, 485^-96. Carrasco, M . & McElree, B. (2001). Covert attention accelerates the rate of visual information processing. Proceedings of the National Academy of Sciences. 98: 5363-5367. Corbetta, M . & Shulman, G.L. (2002). Control of goal-directed and stimulus-driven attention in the brain. Nature Reviews Neuroscience, 3, 201-215. Dosher, B. & Lu, Z.L. (2000). Noise exclusion in spatial attention. Psychological Science, 11, 139-146. Enns, J.T., Austen, E.L. , Di Lollo, V.D. , Rauschenberger, R., Yantis, S. (2001). New objects dominate luminance transients in setting attentional priority. Journal of Experimental Psychology: Human Perception and Performance, 27, 1287-1302. Folk, C.L. & Remington, R.W. (1998). Selectivity in distraction by irrelevant featural singletons: evidence for two forms of attentional capture. Journal of Experimental Psychology: Human Perception and Performance, 24, 847-858. Folk, C.L., Remington, R.W. & Johnston, J.C. (1992). Involuntary covert orienting is contingent on attentional control settings. Journal of Experimental Psychology: Human Perception and Performance, 18, 1030-1044. 109 Folk, C.L. , Remington, R.W. and Wright, J.H (1994). The structure of attentional control: Contingent attentional capture by apparent motion, abrupt onset, and color. Journal of Experimental Psychology: Human Perception and Performance, 20, 317-329. Franconeri, S.L., Hollingsworth, A. and Simons, D.J. (2005). Do new objects capture attention? Psychological Science, 16, 275-281. Friedrich, F.J., Egly, R., Rafal, R.D., & Beck, D. (1998). Spatial attention deficits in humans: A comparison of superior parietal and temporal-parietal junction lesions. Neuropsychology, 12, 193-207. Godijn, R. & Theeuwes, J. (2002). Programming of endogenous and exogenous saccades: Evidence for a competitive integration model. Journal of Experimental Psychology: Human Perception & Performance, 28, 1039-1054. Hawkins, H.L. , Shafto, M.G. , Richardson, K. (1988). Effects of target luminance and cue validity on the latency of visual detection. Perception & Psychophysics, 44, 484-92. Hunt, A.R. and Kingstone, A. (2003a). Covert and overt voluntary attention: Linked or independent? Cognitive Brain Research, 18, 102-105. Hunt, A.R. and Kingstone, A. (2003b). Inhibition of return: Dissociating attentional and oculomotor components. Journal of Experimental Psychology: Human Perception and Performance, 29 (5), 1068-1074. Hunt, A. R. & Klein, R. M . (2002). Eliminating the cost of task set reconfiguration. Memory and Cognition, 30 (4), 529-539. Hunt, A.R., Oik, B. , von Muhlenen, A. & Kingstone, A . (2004). Integration of competing saccade programs. Cognitive Brain Research, 19, 206-208. 110 Irwin, D.E., Colcombe, A . M . , Kramer, A.F. & Hahn, S. (2000). Attentional and oculomotor capture by onset, luminance, and color singletons. Vision Research, 40, 1443-1458. Jonides, J. & Yantis, S. (1988). Uniqueness of abrupt visual onset in capturing attention. Perception & Psychophysics, 55, 399-411. Kim, M.S. & Cave, K.R. (1999). Top-down and bottom-up attentional control: On the nature of interference from a salient distractor. Perception & Psychophysics, 61, 1009-1023. Lu, Z.L. & Dosher, B .A. (2000). Spatial attention: Different mechanisms for central and peripheral temporal precues? Journal of Experimental Psychology: Human Perception & Performance, 26, 1534-1548. Ludwig, C.J.H., & Gilchrist, I D . (2002). Stimulus-driven and goal-driven control over visual selection. Journal of Experimental Psychology: Human Perception and Performance, 25(4), 902-912. Masson, M . E. J., & Loftus, G. R. (2003). Using confidence intervals for graphically based data interpretation. Canadian Journal of Experimental Psychology, 57, 203-220. Prinzmetal, W., McCool, C. & Park, S. (2005). Attention: reaction time and accuracy reveal different mechanisms. Journal of Experimental Psychology; General, 143, 73-92. Posner, M.I. (1980). Orienting of attention. Quarterly Journal of Experimental Psychology, 32, 3-25. Posner, M.I. & Cohen, Y . (1984). Component of visual orienting. In H . Bouma and D. Bonwhuis (Eds.) Attention and Performance X, pp. 551-556. Hillsdale, NJ: Erlbaum. Remington, R.W., Johnston, J.C., & Yantis, S. (1992). Involuntary attentional capture by abrupt onsets. Perception & Psychophysics, 51, 279-290 I l l Sailer, U . , Eggert, T., Ditterich, J., and Straube, A . (2002). Global effect of a nearby distractor on targeting hand and eye movements. Journal of Experimental Psychology: Human Perception & Performance, 28, 1432-1446. Serences, J.T., Shomstein, S., Leber, A.B. , Golay. X . , Egeth, H.E. & Yantis, S. (2005). Coordination of voluntary and stimulus-driven attentional control in human cortex. Psychological Science, 16, 114-122. Taylor, T.L., & Klein, R . M . (1998). On the causes and effects of inhibition of return. Psychonomic Bulletin and Review, 5, 625-643. Theeuwes, J. (1992). Perceptual sensitivity for color and form. Perception & Psychophysics, 43, 307-318. Theeuwes, J., Atchley, P. & Kramer, A.F. (2000). On the time course of top-down and bottom-up control of visual attention. In S. Monsell and J. Driver (Eds.) Attention and Performance XVIII (pp. 105-124). Cambridge: MIT Press. Theeuwes, J. & Godijn, R. (2001). Attentional and oculomotor capture. In C.I. Folk & B.S. Gibson (Eds.) Attraction, Distraction, and Action: Multiple Perspectives on Attentional Capture (pp. 121-150). Amsterdam: Elsevier. Theeuwes, J., Kramer, A.F. , Hahn, S. & Irwin, D.E. (1998). The eyes do not always go where we want them to go: Capture of the eyes by new objects. Psychological Science, 9, 379-385. Theeuwes, J., Kramer, A.F. , Hahn, S., Irwin, D.E. & Zelinsky, G.J. (1998). Influence of attemtional capture on oculomotor control. Journal of Experimental Psychology: Human Perception & Performance, 25, 1595-1608. 112 van Zoest, W., Donk, M . & Theeuwes, J. (2004). The role of stimulus-driven control in saccadic visual selection. Journal of Experimental Psychology: Human Perception and Performance, 30, 746-759. Wu, S. & Remingston, R.W. (2003). Characteristics of covert and overt visual orienting: Evidence from attentional and oculomotor capture. Journal of Experimental Psychology: Human Perception & Performance, 29, 1050-1067. Yantis, S. & Jonides, J. (1984). Abrupt visual onsets and selective attention: Evidence from visual search. Journal of Experimental Psychology: Human Perception & Performance, 10,601-620. Yantis, S. & Jonides, J. (1996). Attentional capture by abrupt onsets: New perceptual objects or visual masking? Journal of Experimental Psychology: Human Perception & Performance, 22, 1505-1513. Yeshurun, Y . & Carrasco, M . (1998). Attention improves or impairs visual performance by enhancing spatial resolution. Nature, 396, 5 Nov. 72-75. 113 CHAPTER 4: Attraction and Distraction by Emotional Stimuli As we scan the environment for a specific item, voluntary visual attention enables selection of specific types of information to the exclusion of those items that are irrelevant to current task goals or intentions (e.g., Folk, Remingston & Johnston, 1992). For example, if you are looking for your friend in a crowd, your eyes might be drawn to people who have similar characteristics to your friend, such as a similar jacket or hair color. Your eyes might also be directed to certain kinds of events, even when they are irrelevant, because of intrinsic meaning or salience (e.g. Yantis & Jonides, 1984). Thus the reflection of sunlight off a shiny object might catch your eye, simply because it is bright, and not because it is in any way relevant to your intention to find your friend. The latter "reflexive" orienting system has been conceptualized as a "circuit-breaker" of voluntary attention, serving to interrupt ongoing voluntary attention in order to alert the system of a potentially important event (Corbetta and Shulman, 2002). The notion that reflexive attention serves as an alerting system has led to the question of what kinds of events are able to summon attention reflexively. If reflexive attention is an adaptive system that prevents behaviorally-relevant but unexpected events from escaping notice, one might predict that emotionally powerful or threatening events would have the power to attract attention. One approach to addressing this issue has been to measure the attentional effects of fear-inducing stimuli that are associated with phobias, such as snakes and spiders. Ohman, Flykt and Esteves (2001) found faster reaction time to detect fear-1 0 This paper is currently in preparation for submission, under the title "The effect of emotional stimuli on eye movements and attention", by Hunt, Cooper and Kingstone, 2005. 114 inducing targets11 (snakes and spiders), among neutral items (mushrooms and flowers) than to find a neutral target among fear-inducing items. This effect was more pronounced among participants with a specific snake or spider phobia. They suggest that fear-inducing stimuli are processed pre-attentively, that is, the threatening information can be extracted from a visual object without allocating attention to it. In comparison, in order to detect a neutral stimulus, attention must be allocated to each item in the display in a serial fashion in order to determine whether or not it is the target. A serious problem with this interpretation of their results is pointed out by Tipples, Young, Quinlan, Broks, and Ellis (2002), who note that snakes and spiders are animals, and flowers and mushrooms are not. Thus the difference between snakes/spiders and flowers/mushrooms may be due to an animate/inanimate difference rather than a threatening/nonthreatening difference. In support of their interpretation, Tipples et al. replicated the effect that reaction time is faster when detecting an animal among plants than when detecting a plant among animals, but also showed across several studies that reaction time to detect a threatening animal target among neutral animals is no faster than to detect a neutral animal among threatening animals. They point out that the subjective sense that attention is preferentially allocated to threatening events may arise not because attention is attracted to these stimuli, or because they are processed more quickly, but because once attention is has been directed to a threatening stimulus, it is harder to disengage from it than from neutral events. 1 1 Visual stimuli commonly associated with threat (for example, snakes, spiders, angry faces) are often termed "threatening" in an experimental context, even though they pose no actual threat to the participant. The underlying assumption is that if orienting to threatening-looking stimuli really is reflexive, it would be invariant even in these extremely artificial contexts. 115 Fox, Russo, Bowles and Dutton (2001) make a similar distinction between the power to attract attention, and the power to hold attention. They observe that threatening stimuli do not attract attention, but they do hold attention for longer than unthreatening stimuli, at least among people scoring high on anxiety scales. They observed that the time to detect a target presented in a location previously occupied by a threatening versus a neutral stimulus was the same, even for highly anxious individuals. When the target appeared at a different location than the threatening or neutral stimulus, reaction time to detect the target was delayed more for threatening than for neutral stimuli. This result suggests that attention is slower to disengage from a threatening stimulus than a neutral one. However, this pattern was only consistently observed among highly anxious individuals. Among those participants who scored low on an anxiety scale, slower disengagement of attention from threatening stimuli occurred only in a limited set of conditions. They suggest that anxiety increases the attentional dwell-time on threatening stimuli, rather like the "freezing" response to danger observed in other animals. They imply that this response is particular to highly anxious individuals, representing an orienting abnormality rather than a norm. Other research has explored the effect of facial expression on visual search, because faces convey emotional information such as threat and affiliation very quickly and efficiently (e.g. Ekman and Friesen, 1975). Hansen and Hansen (1988) were the first to introduce emotional expressions to a visual search task. Their logic was that if an angry or threatening face draws attention automatically, then reaction time to detect an angry face in a crowd of happy faces should be faster than to detect friendly faces in angry crowds. This is indeed what they observed. However, subsequent researchers have pointed out that the faces used by Hansen and Hansen in their experiment contained a critical confound, in that the high-116 contrast versions of photographs of facial expressions (adapted from Ekman and Friesen, 1975) had high-contrast black patches on the angry faces, but not on the happy faces. When this perceptually salient feature was removed from the photographs, a priority for angry faces in visual search was no longer observed (Purcell, Stewart & Skov, 1996). Studies exploring the effect of emotion on attention using schematic faces instead of photographs have replicated Hansen and Hansen's (1988) original effect. Eastwood, Smilek, and Merikle (2001) examined the reaction time to detect a positive or negative emotion embedded among from 7 to 19 neutral faces. An increase in the number of distractors is typically associated with an increase in reaction time to detect a target. This increase across the search display size, known as the search slope, can be used as an index of how quickly attention can filter out the distractors and home in on the target. Eastwood et al. (2001) reasoned that if attention gives priority to angry faces, the search slope to detect an angry face among neutral distractors should be shallower than the search slope to detect a happy face among neutral distractors. The results confirmed this prediction: negative faces were detected faster than positive ones, and search slopes were slightly but consistently steeper for happy faces than for angry faces. When faces were inverted, negative faces were still detected faster than positive ones, but there was no difference in slope. This result suggests that it was indeed the facial expressions causing the decrease in reaction time and slope for angry faces, and not a difference in detection of a down-curved line (if it were, there would be a reversal of the effect when the faces were inverted). Ohman, Lundqvist, and Esteves (2001) also observed a very small but significant effect of emotional expression of faces on visual search, with faster reaction times to detect a "threatening" (frowning) target than to detect a "friendly" (smiling) target. Thus there seems to be some evidence for faster 117 responses to negative emotional stimuli relative to positive. There was no interaction of emotion with set size (from 4 to 25 items), however, in contrast to the findings of Eastwood et al. (2001). One common feature of the studies above is that attention is typically measured using speeded detection responses. The locus of covert attention must therefore be inferred based on a nondirectional bivalent response. Slower reaction times are usually interpreted in these experiments as evidence that attention was delayed from being deployed to the target location because of reflexive capture or holding by threatening distractors, but no direct measure of attention is possible. Other factors besides attention could slow responses in the presence of threatening distractors, such as changes in strategy to detect the target, slower discrimination of the target features, and increased competition between possible responses. The mutually contradictory results produced in this literature might be the result of differences in how some other factors besides selective attention influence manual discrimination and detection responses. Measuring visual search using eye movements may provide an opportunity for a richer and more direct measure of attention. Covert attention tends to be deployed to the location to which an eye movement is about to be executed (e.g., Sheperd, Findlay, and Hockey, 1986; Hoffman and Subramanian, 1995; Kowler, Anderson, Dosher and Blaser, 1995; McPeek, Maljkovic, and Nakayama, 1999). Given that eye movements and attention tend to move together, eye movements can be assumed to be slower and more error-prone when attention is distracted by some task-irrelevant events. There is also evidence from the oculomotor capture literature (e.g. Theeuwes, Kramer, Hahn and Irwin, 1998) that salient events that occur during visual search can "capture" eye movements, even when the events 118 are irrelevant to the current task. Hunt, von Miihlenen and Kingstone (see the previous chapter of this thesis) demonstrated that the oculomotor capture effect reflects the same underlying processes as those causing slower manual reaction times in the presence of distracting events. If threatening or emotional events attract attention reflexively, and therefore slow reaction time, one would therefore also predict that a highly emotional event would capture eye movements more than emotionally-neutral events. Two studies have measured the effect of threatening events on eye movements (Hermans, Vansteenwegen, and Eelen, 1999; Miltner, Krieschel, Hecht, Trippe, and Weiss, 2004) but both used similar flower/mushroom/spider stimulus sets as Ohman, Flykt and Esteves (2001), and their findings are thus subject to the same criticism leveled by Tipples et al. that the threatening/unthreatening manipulation is confounded with an animate/inanimate distinction. It is interesting to note, however, that Miltner et al. (2004) observed that both eye movements and manual responses were actually faster for mushroom than spider stimuli, showing the opposite pattern from the original Ohman et al (2001a) study. Miltner et al. also observed that both manual responses and eye movements toward neutral stimuli were slowed by the presence of spiders among spider-phobics, but not among non-phobics. This result is similar to Fox et al. (2001) in that the attentional bias toward threatening stimuli is a unique characteristic of phobic individuals, and does not represent a typical attentional orienting response. To summarize the above results, the evidence that attention prioritizes emotionally negative or threatening events is suggestive, but not compelling, with some studies finding evidence in favor of this conclusion (e.g., Eastwood et al., 2001; Ohman, Lundqvist and Esteves, 2001) and others finding evidence that if threatening events do interfere with search, 119 they do so consistently only among people with a specific phobia (e.g., Tipples et al., 2002; Fox et al., 2001, Miltner et al., 2004). The present study explored the effect of emotional faces using eye movements to resolve the question of whether attention is attracted or held by threatening events. A single face target was shown among 5 distractors, equidistant from fixation, and the task was to move the eyes to the target face as quickly as possible. On half the trials, the 5 distractors were all neutral, but on the other half, one of the five distractors was a unique positive, negative, or neutral face. Using this methodology, four questions can be simultaneously addressed. 1) Wil l eye movements be directed towards negative emotional face targets faster than positive or neutral face targets? If so, this result would confirm the conclusions of Ohman et al (2001) who suggested that attention is indeed automatically allocated to threatening events. 2) Wil l erroneous eye movements towards a distractor (i.e., capture) be more frequent when the distractor displays a negative emotion than when it is positive or neutral? This would provide converging evidence that attention is reflexively attracted to threatening events (Ohman et al, 2001). 3) Wil l an irrelevant negative face distractor slow eye movements to a target more than an irrelevant positive or neutral face distractor? Distraction effects could occur for several reasons. If the answer to questions 1 and 2 above is "yes", then the distraction effect likely occurs because attention is reflexively drawn to the negative face distractors. If the answer to questions 1 and 2 above is "no", however, it is more likely that the distraction effect is due to slower 120 disengagement of attention from negative distractors, or some other factor such as target discriminability or strategy. 4) When eye movements are erroneously directed towards irrelevant faces, are they slower to disengage and redirect to the target when the distractor is a negative face than when it is positive or neutral? If so, this would provide the most direct evidence to date that attention is slower to disengage from a negative emotional stimulus than from positive or neutral stimuli. In the first experiment, the emotion and orientation of both target and distractor were varied between participants. In the second experiment, the target type was always emotionally neutral, and the distractor emotion and orientation varied within participants. The results of these two experiments together will shed light on the relative power of negative face stimuli to both attract and hold attention. Experiment 1 Methods Forty participants were randomly assigned to one of four groups. Each group completed one block of sixty trials. Groups were labeled according to the emotion of the target to which participants were instructed to move their eyes (see Appendix C for the results of a questionnaire assessing the emotions conveyed by the faces used in the current experiments). In the "happy" group, for example, each trial began with 6 upright neutral faces (see Figure 4.1). After 1000ms, one of the faces changed from neutral to a happy face. 121 On half the trials, a second face changed from neutral to angry. The task was to move the eyes from the central fixation crosshair to the happy face as quickly as possible, and the angry face was always irrelevant to this task for the participants in the happy group. The "angry" group participants were instructed to look towards the angry face among upright neutral faces, and on half the trials an irrelevant upright happy face was shown. 1 1000ms 2 -> Figure 4.1. Two typical trials from the "upright happy" group from Experiment 1. Each trial begins with 6 neutral faces, which appear when the participant presses the spacebar and a stable fixation is detected (1). After 1000ms, one of the neutral faces becomes the target (a happy face, in the case of the happy face group). On half the trials, the target face is displayed among only neutral faces, and on the other half of trials, an additional neutral face becomes a unique distractor (an angry face, in the case of the upright happy group). The task is to saccade to the target as quickly as possible, ignoring the other faces. In A, the target is shown alone, with no unique distractor. In B, a unique distractor (angry face) is revealed at the same time as the target. 122 The other two groups provided a control condition, in which the displays shown to the "happy" and "angry" groups were inverted, including the neutral face distractors. The display was inverted in order to reduce the emotional content of the items without changing their perceptual features. To better disguise the stimuli from being recognized or perceived as inverted schematic faces, participants were instructed to search for the "mushroom" or "goblet" among "tables" (see Figure 4.2). In order to reinforce this interpretation of the display, participants were given ten practice trials before the experimental block in which extra lines were added to the display to enhance their resemblance to non-emotional objects. During the experimental block, these additional lines were removed. After the experiment, participants filled out a questionnaire about the task and the resemblance of the objects to faces. Most participants had not noticed that the objects in the display were actually inverted faces. 123 Target Distractor Neutral Group; 1;l Happy." target Group 2: "Angry";target Group3:: Inverted Group Practice stimuli "Mush ropm'itarget Group 4: ^'G.obl^";target Figure 4.2. The stimuli used as the target, the neutral distractors, and the unique distractor in the four groups included in Experiment 1. The task was to fixate the target as quickly as possible, ignoring both neutral distractors and unique distractors. In the "mushroom" and "goblet" groups, participants were given practice trials prior to the experiment in which they were shown the inverted stimuli with extra lines included, in order to reinforce the interpretation of these stimuli as objects rather than faces. Participants with less than 60% correct or useable trials were discarded from analysis and replaced. Five participants were rejected for not meeting this criteria: two each from the inverted happy and upright angry groups, and one from the inverted angry group. The analyses were designed to address the four questions posed in the introduction. A mixed A N O V A with distractor (present or absent) as a within subjects factor and emotion (angry or happy) and orientation (upright and inverted) as between-subjects factors was used Results 124 to assess the effect of target and distractor type on reaction time. A between-subjects ANOVAs examined the proportion of saccades directed toward the distractor as a function of the emotion and orientation of the target and distractor. Finally, a third between-subjects A N O V A examined the duration of fixation on the distractor, also as a function of emotion and orientation. The results of the reaction time analysis are shown in Figure 4.3. There was a main effect of the distractor on saccadic reaction time, with faster saccades to the target when the distractor was absent than when it was present [F(l,36)=42.91]. There is also a significant interaction of orientation and distractor [F( 1,36)= 18.18], whereby the effect of the distractor is larger for upright faces than for inverted faces. When the reaction time to saccade to the target in the absence of a distractor is analyzed as a function of the target type, there is no significant effect of the target emotion [F(1,36)<1] or target orientation [F(1,36)<1]. £j| dfasfassta present fcWSft«J vol eft Target Type Figure 4.3. The reaction time of saccades directed towards the target is shown as a function of the target type and the presence or absence of a unique distractor. The unique distractor was always the same orientation as the target, but the opposite emotion (e.g., for those participants in the inverted angry target group, the distractor was an inverted happy face). Error bars are the standard error of the mean. 125 The analysis of the proportion of saccades erroneously directed towards the irrelevant distractor was similar to the reaction time analysis. Saccades were misdirected towards the irrelevant distractor on 26.2% of the trials in which it appeared, so attention was clearly drawn to the unique irrelevant items. The important question is whether the emotional content of the faces influenced the degree to which eye movements were captured. There was no evidence that this was the case, with no significant effect of orientation [F(l,36)=2.35], or emotion [F(1,36)<1], or an interaction [F(1,36)<1]. There was a numerical increase in capture to 32.0% when the angry face was the target and the happy face was the distractor, but this was not stable enough to be significant. happy lucqr *»gty Disiiactoe Type Figure 4.4. The proportion of saccades directed towards the unique distractor, the target, or to some other locations on the screen are shown as a function of the distractor type. Only distractor present trials are shown. Error bars are the standard error of the mean proportion of saccades directed towards the distractor. The eyes were not directed towards angry faces more than other types of stimuli. 126 Finally, the duration of fixation of trials on which the eyes went to the distractor was analyzed as a function of distractor type. There was no main effect of orientation [F(1,36)<1], emotion [F(1,36)<1], or an interaction [F(l,36)=1.37]. In the introduction it was predicted that if threatening stimuli hold attention longer than other kinds of stimuli, that there would be longer fixations when the eyes land on the angry face than on happy or inverted faces. This result does not support that prediction. 300* .250 g-200 i I too 3 Q so Q I I S 'artqry-Distracsof Type Figure 4.5. The duration of fixation on the unique distractor when saccades were erroneously directed towards it. The eyes did not fixate angry faces for longer than other kinds of faces. Discussion In response to the four questions posed in the introduction: 1) There was no evidence that eye movements were directed towards negative emotional faces faster than positive or neutral faces: This result is consistent with Tipples et al. (2001), Fox et al., (2001) and Miltner et al. (2004) in showing no attentional bias associated with negative faces. 127 2) Eye movements were frequently captured by the unique distractors, but the eyes were not directed towards the negative emotional distractors any more than other kinds of distractors. This converges with the reaction time evidence above to suggest that attention is not preferentially allocated to threatening emotional stimuli. 3) Upright distractors slowed reaction time to saccade to a target to a greater extent than inverted distractors. This is consistent with the results of Fox et al., who showed that threatening distractors interfered with detection of a target when they were presented in a different location than the target, but do not facilitate target detection when they are presented in the same location as the target. 4) There was no evidence that the eyes fixate upright or angry faces longer than neutral faces, arguing against the notion that attention is slower to disengage from emotional events. This result does not agree with the interpretation that attention is slower to disengage from negative distractors. This experiment has shown that saccades are not faster or more accurate to move towards angry faces than neutral and happy face targets. Evidence for the hypothesis that negative emotions hold attention for longer (Fox et al., 2001) is less clear-cut. On the one hand, emotional faces, happy and angry alike, interfere with detection of the target more than nonface distractors. On the other, the eyes were not slower to disengage from negative distractors than from positive or inverted distractors. 128 Experiment 2 Experiment 1 showed that search for a target is slower in the presence of emotional distractors, which previous research interpreted as evidence that attention is slower to disengage from these kinds of events (Fox et al., 2001). However, fixation duration was similar for all types of distractors, arguing against this interpretation. An alternative possibility for why emotional distractors would slow saccades to the target is that search for a target that is defined based on a specific emotion is slower when another emotional face appears in the search display. In other words, observers use emotional content as the feature that defines the target, and this is the most efficient strategy for detecting the target when all the distractors are neutral. When one of the distractors is also emotional, search is slower because additional time is required to discriminate the emotional target from the emotional distractor. Experiment 2 was designed to measure the effect of emotional and neutral distractors when emotion is not the defining attribute of the target. If the effects in Experiment 1 are due to the attention-grabbing properties of emotional face distractors, upright faces will interfere with search regardless of the target type. This experiment also allows for the comparison of distractor effects within participants instead of between participants, which will increase power to detect potential differences between distractors. Methods In this experiment, 10 participants completed 8 blocks of 60 trials. The display and series of events were similar to that of Experiment 1, except in the following respects. Each 129 participant experienced all 4 distractor types. The distractor type was constant within a block of trials (to match the conditions of Experiment 1), and could be an upright happy, inverted happy, upright angry, or inverted angry face (See Figure 4.2). In four of the eight blocks (one block for each distractor type), the target was a neutral face among inverted neutral faces, and in the other four blocks, the target was an inverted neutral face among upright neutral faces (see Figure 4.6). Within each block, the individual trial sequence was similar to Experiment 1, and a unique distractor was shown on half the trials. The two different types of target (inverted and upright neutral faces) were used because when both the target and unique distractor are upright or both are inverted, they are more similar, which was expected to increase capture relative to when one is upright and the other is inverted. Both inverted and upright targets were included in the experiment to balance out this potential effect. In the analyses of the reaction time, proportion capture, and duration of fixation, there was no main effect of target orientation and no interaction of either the distractor type or the distractor status (present or absent) with target orientation. For simplicity, target orientation was therefore not included as a factor in further analyses. 130 A 1 1000ms 2 B 1 1000ms 2 Figure 4.6. The two types of target trials used in Experiment 2 are shown. In A, each trial begins with 6 inverted neutral faces. After 1000ms, one of the neutral faces become an upright face (shown in the 3:00 position here), and on half the trials, an additional neutral face becomes a unique distractor (in this case, an upright angry face, but it could also be an inverted angry face, or an upright or inverted happy face). In B, each trial begins with 6 upright neutral faces. After 1000ms, one of the upright faces becomes an inverted neutral face (shown in the 9:00 position here), and on half the trials, an irrelevant unique distractor is shown (again, an upright angry face is shown in this Figure). Results Reaction time results were submitted to a three-way within-subjects A N O V A , with distractor status (absent or present), distractor orientation (upright or inverted), and distractor emotion (angry or happy) as factors. The results are shown in Figure 4.5. There was a main effect of the distractor status, with slower reaction times when the distractor was present than 131 when it was absent, [F(l,9)=30.04, p<.001], and a main effect of distractor orientation, with faster responses to the target in blocks with inverted distractors than blocks with upright distractors [F(l,9)=10.24, p<.05]. There were no significant interactions, including the interaction of the distractor orientation and the presence of absence of the distractor [F(1,9)<1], which is the interaction that was significant in the reaction time results from Experiment 1. In Experiment 2, upright face distractors slow reaction time to fixate the target more than inverted face distractors, but this seems to occur whether or not the distractor is actually presented. taratfad fevMl*d yprigtt saprtgpsi m&f hafW **SFV * * W Disirackx Typ© Figure 4.7. The reaction time of saccades directed towards the target is shown as a function of the distractor type and the presence or absence of a unique distractor. Error bars in the three figures depicting the results from Experiment 2 are calculated using the pooled error term of the three factors and their interactions, according to the methods for illustrating within-subjects error described by Masson and Loftus (2003). 132 The proportion of saccades directed to the distractor ("capture") was submitted to a 2x2 A N O V A , with distractor orientation (inverted or upright) and distractor emotion (angry or happy) as factors. The proportion of capture was larger when the distractor was happy than when it was angry [F(l,9)=5.98, p<.05], see Figure 4.7, but there was no effect of orientation and no interaction of distractor emotion and orientation [both Fs<l]. Distractor Type Figure 4.8. The proportions of saccades directed towards the distractor, the target, and to other locations are shown as a function of the distractor type. Finally, the duration of fixation on the distractor following capture was also analyzed in a 2x2 within-subjects A N O V A with distractor orientation and emotion as factors. One participant was omitted from this analysis because her very low capture rate (5.8%; the rest of the participants had a mean of 21.2% capture) meant she had very few observations per cell. The results of the A N O V A were similar to the proportion capture data, with a significant main effect of emotion [F(l,8)=5.37, p<.05], but no effect of orientation 133 [F(l,8)=3.09] and no interaction [F(1,8)<1]. This result supports the proportion analysis above in showing that not only are the eyes more likely to be captured by a happy face, but they are also likely stay on a happy face for longer before moving to the target. ;ll!|Mlll§pil! % ( trv titled-.:; nviiitcei happy arm Oisiiackx Type Figure 4.9. The duration of fixation on the distractor when saccades are initially directed towards the distractor instead of the target is shown. The eyes stayed on the happy faces for a longer period of time before correcting the saccade and moving to the target, regardless of orientation. Discussion In Experiment 2, in which the target type was constant and the distractor varied as a within-subjects variable, reaction time was slower overall when upright face distractors were expected, but not especially distracted by the presence of an upright face when they actually occurred. There also was an increase in the proportion of capture for happy faces, and an increase in the duration of fixation on happy face distractors. 134 The current experiments address an important distinction between a default or pre-existing bias to orient to emotional events, and the ability to use emotion to guide attention voluntarily. Upright emotional distractors had a significantly larger effect on reaction time than inverted faces in Experiment 1, but this result was not replicated in Experiment 2, in which the target was defined based on orientation instead of emotion. Instead, there was a main effect of orientation, with slower responses overall in blocks where the upright faces appeared, regardless of whether or not a distractor was actually presented. This pattern suggests that upright emotional faces may have been especially distracting in Experiment 1 because participants were voluntarily attending to emotional stimuli as a strategy for detecting the target. In Experiment 2, emotion was never relevant to the task goals, and upright emotional faces did not interfere with search for a neutral stimulus any more than inverted faces, although there was an overall increase in reaction time in blocks in which upright distractors were shown, even on trials where they did not actually appear. This result implies that the effect of upright faces on reaction time has more to do with strategy and expectancy about the search display than with reflexive attention to emotional events. Another interesting observation from Experiment 2 is that the happy faces interfere with search for a neutral target more than angry faces, regardless of whether they are inverted or not. One important consideration is that a "neutral" face does still have facial features that communicate an expression, although it does not have the strong emotional valence that the happy and angry faces have. Perhaps among the current stimuli, the quality of the neutral expression was closer to a happy face than to an angry face, thus the neutral and happy face were more easily confusable than the neutral and angry face. This confusion could result in the eyes being directed more often towards happy faces, and a longer fixation on the happy 135 face would be required to determine that it was not the target. The results of the questionnaire presented in Appendix C support this interpretation. Subjective ratings of the neutral faces were on average slightly higher on a happy scale than on an angry scale.' For the purposes of the present study, this result can at the very least be taken as evidence against the hypothesis that negative emotional stimuli have an attentional priority over positive stimuli. Together with the results of several experiments conducted by Tipples et al. (2002), Fox et al. (2000), and Purcell et al. (1996), none of whom found an effect of threatening events on attentional orienting in nonanxious observers, the present study presents a serious challenge to the notion of attentional capture by threat. The suggestion that threatening faces do not have the power to capture attention reflexively goes against the intuition that when one encounters a snake or a snarling dog or an angry face in a crowd, that there is a sense of being drawn to it automatically, and of recognizing and reacting to it immediately. However, there are many ways that threatening events could influence behavior besides through attention, any one of which would result in the experience of heightened awareness that was initially attributed to preferential attentional orienting by Hansen and Hansen (1988). Some of these alternatives are that people may be faster to process the identity of emotionally-charged stimuli, more sensitive to their details, faster to react to them, or more likely to remember them later. Future research is needed to shed light on which of these processes underlie our subjective experience of having our gaze drawn to, and held by, emotionally threatening events. 136 References Corbetta, M . & Shulman, G.L. (2002). Control of goal-directed and stimulus-driven attention in the brain. Nature Reviews Neuroscience, 3, 201-215. Eastwood, J.D., Smilek, D. & Merikle, P .M. (2001). Differential attentional guidance by unattended faces expressing positive and negative emotion. Perception & Psychophysics, 63,1004-1013. Ekman, P. & Friesen, W.V. (1975). Unmasking the face. Englewood Cliffs, NJ: Prentice-Hall. Enns, J.T. & Rensink, R.A. (1991). Preattentive recovery of 3-dimensional orientation from line drawings. Psychological Review, 98, 335-351. Folk, C.L., Remington, R., and Johnston, J.C. (1992). Involuntary covert orienting is contingent on attentional control settings. Journal of Experimental Psychology: Human Perception and Performance, 18, 1030-1044. Fox, E., Russo, R., Bowles, R., & Dutton, K. (2001). Do threatening stimuli draw or hold visual attention in subclinical anxiety? Journal of Experimental Psychology: General, 130,681-700. Hansen, C.H. & Hansen, R.D. (1988). Finding the face in the crowd: An anger superiority effect. Journal of Personality and Social Psychology, 54, 917-924. He, Z. J. and Nakayama, K. (1992). Surfaces versus features in visual search. Nature, 359,231-233. Hermans, D., Vansteenweger, D., & Eelen, P. (1999). Eye movement registration as a continuous index of attention deployment: Data from a group of spider-anxious students. Cognition and Emotion, 13, 419^-34. 137 Hoffman, J.E. & Subramanian, B. (1995). The role of visual attention in saccadic eye movements, Perception & Psychophysics, 57, 787-795. Kowler, E., Anderson, E. Dosher, B . & Blaser, E. (1995). The role of attention in the programming of saccades. Vision Research, 13, 1897-1916. McPeek, R. M . , Maljkovic, V. , & Nakayama, K. (1999). Saccades require focal attention and are facilitated by a short-term memory system. Vision Research, 39, 1555-1566. Miltner, W.H., Krieschel, S., Hecht, H. , Trippe, R., Weiss, T. (2004). Eye movements and behavioral responses to threatening and nonthreatening stimuli during visual search in phobic and nonphobic subjects. Emotion, 4, 323-339. Ohman, A. , Flykt, A. , & Esteves, F. (2001). Emotion drives attention: Detecting the snake in the grass. Journal of Experimental Psychology: General, 130, 466^1-78. Ohman, A. , Lundqvist, D., & Esteves, F. (2001). The face in the crowd revisited: A threat advantage with schematic stimuli. Journal of Personality and Social Psychology, 80, 381-396. Purcell, D.G., Stewart, A . L . , & Skov, R.B. (1996). It takes a confounded face to pop out of a crowd. Perception, 25, 1091-1108. Sheperd, M . , Findlay, J.M., and Hockey, R.J. (1986). The relationship between eye movements and spatial attention. The Quarterly Journal of Experimental Psychology, 38, 475^191. Theeuwes, J., Kramer, A.F. , Hahn, S. & Irwin, D.E. (1998). The eyes do not always do where we want them to go: Capture of the eyes by new objects. Psychological Science, 9, 379-385. 138 Tipples, J., Young, A.W., Quinlan, P., Broks, P. & Ellis, A.W. (2002). Searching for threat. The Quarterly Journal of Experimental Psychology, 55A, 1007-1026. Wolfe, J .M. (1996). Visual search. In H . Paschler (Ed.) Attention. London: University College London Press. Yantis, S. and Jonides, J. (1984). Abrupt visual onsets and selective attention: Evidence from visual search. Journal of Experimental Psychology: Human Perception & Performance, 10, 601-621. 139 CHAPTER 5: General Discussion In the present studies, I investigated voluntary search of the environment for a pre-specified target and interference with this search process by task-irrelevant distractors. Previously, two separate literatures have explored this question using a very similar methodology: in the oculomotor capture literature, the locus of attention is measured using the direction and timing of eye movements, and in the attentional capture literature, manual responses to visual targets are used to infer the locus of covert visual attention during search. In both cases, irrelevant distractors can interfere with search for the target, but in oculomotor capture experiments this interference is expressed as an incorrect eye movement in the direction of the irrelevant distractor, whereas in attentional capture experiments this interference tends to be expressed as slower reaction times to discriminate some aspect of the visual target. Oculomotor capture by irrelevant but perceptually salient events has previously been interpreted in two ways. It has been applied to the problems and issues of how reflexive and voluntary eye movement programs compete for control over eye movements, using a logic like that applied in Chapter 2 of the current thesis, in which oculomotor capture is used to test the predictions of two models of eye movement control. Oculomotor capture has also been applied to outstanding questions about how covert visual attention is guided in the visual field. The reasoning is that eye movements are able to provide a richer and more direct measure of the locus of attention than manual discrimination responses. This is the logic 140 behind Chapter 4 of the current thesis, in which the question of how emotional stimuli affect attentional orienting is tackled using eye movements instead of manual responses. These two applications of oculomotor capture appear to be mutually incompatible, in that oculomotor capture in the first instance is used to shed light on the eye movement system specifically, and in the second instance it is used to answer questions about attention, for which the underlying assumption is that the locus of attention during visual search would be the same regardless of the effector being used to measure that search. Chapter 3 addresses this issue to shed light on the relationship between oculomotor and attention capture. In what follows, I first briefly summarize the findings and implications of the current studies individually. I then discuss some general conclusions, implications, and limitations of the work as a whole. Finally, I advance some questions that ongoing and future research will need to address. Summary In Chapter 1,1 began by describing the literature supporting the division of covert orienting of attention into voluntary and reflexive subtypes. Voluntary orienting is usually associated with the dorsofrontal parietal network (Corbetta and Shulman, 2002), and functions in shifting attention based on expectancies and goals. The reflexive orienting network is activated in response to unexpected but perceptually salient events, and is usually associated with right temporoparietal areas and ventral frontal cortex. I also described attentional capture experiments, which explore the relative contribution of voluntary and reflexive processes in visual search. The major debate in this research is the degree to which 141 voluntary and reflexive processes contribute to search. On one side, the working hypothesis suggests that visual search is primarily stimulus-driven, or reflexive, with attention being allocated to the most salient events first (e.g., Theeuwes, 1992; Theeuwes and Godijn, 2003). On the other side, the claim is that visual search is driven by top-down, voluntary goals and expectations of the individual, and perceptually salient visual features will only interfere with search insofar as they necessitate more complex strategies and perceptual filters to find the target (e.g., Folk and Remington, 1998; Folk, Remington and Johnson, 1992). Although they are sometimes discussed as though they were discrete categories, the labeling of "voluntary" and "reflexive" depends heavily on the context. For example, a shift in attention toward a color singleton might be labeled "reflexive" under many circumstances, but a shift in attention to the same color singleton when it is defined as the target of search, and it is shown along with salient distractors, would be labeled "voluntary". In contrast to covert attention, voluntary eye movements are usually associated with the Frontal Eye Fields (FEF) and reflexive eye movements with the Superior Colliculus (SC). The results from oculomotor capture studies have shown that the eyes tend to be drawn to irrelevant sudden-onset distractors. This is usually taken as evidence in favor of the "reflexive-first" side of the debate, but it cannot be taken for granted that attentional capture and oculomotor capture are the same process. For one thing, voluntary and reflexive eye movements are supported by a different set of structures than those normally thought to be involved in reflexive and voluntary orienting of covert attention. For another, attention and eye movements have been shown to be behaviorally dissociable (e.g. Klein, 1980; Hunt and Kingstone, 2003). Nonetheless, eye movements and attention do appear to move together most of the time (Posner, 1980). The conclusion that can be drawn from the review in 142 Chapter 1 is that it is an important, but open, question whether the results from oculomotor capture research necessarily apply to attention capture more generally. In Chapter 2,1 laid out the predictions of two models of eye movement control and tested them using the effect of fixation offset on oculomotor capture. The results were straightforward and suggested reflexive and volitional competition for control over the eye movement system occurred at the level of the SC. Aside from contributing evidence in favor of the conflict model, and against the race model of eye movement control, this study also demonstrates how oculomotor capture research can help constrain models of eye movement control. In Chapter 3, I explored the relationship between oculomotor capture and attentional capture directly. It is not clear from previous research whether oculomotor and attentional capture are measuring the same underlying process of visual search, or if they are, as the previous chapter implies, qualitatively different phenomenon. One of the limitations of previous research for drawing conclusions about their relationship is that attentional and oculomotor capture tend to be measured using different tasks and dependent measures. In the experiments in Chapter 3,1 compared a manual localization task to a saccadic localization task using the same visual displays and within-subjects comparisons to minimize differences and allow for a more direct comparison between the two response types. Initially the results mirrored those observed in the literature: when measured using manual responses, irrelevant onsets slowed the reaction time to localize the target. When measured using saccades, irrelevant onsets influenced the direction of eye movements, but not their latency. I was able to show that the most likely reason for this difference is not that eye movements and manual responses are measuring different underlying processes, but that they are measuring changes 143 in the same process over time. When differences in reaction time between saccadic and manual responses are minimized by speeding manual responses, eye movement and joystick capture are both observed in the direction in which responses are executed. Similarly, when the duration of the target is very short, and information accrual over time is followed by information decay, eye movement and joystick capture were again very similar. The results support the notion that the eyes and the hand are both responding based on a shared representation of the target location, and both can contribute equally to an understanding of how attention is allocated during visual search. They also highlight the importance of taking into consideration the changes that occur over time that can affect the quality of information to which the observer is responding. In Chapter 4,1 built on the assumption that eye movements are a sensitive measure of how attention is allocated in the visual field. A common debate in attention revolves around which kinds of events attract attention because of intrinsic properties, and which will attract attention only when they are relevant to the current task. One influential and intuitively appealing suggestion is that attention is preferentially allocated to emotionally negative stimuli (e.g., Ohman, Flykt and Esteves, 2001). Using eye movements, I examined the effect of happy and angry face distractors on visual search and found evidence that, although emotion can be used voluntarily as a feature for detecting a target in the visual environment, there is no indication that eye movements and attention are preferentially allocated to negative emotional stimuli. 144 Conclusions Each study comprising the current thesis contributes a unique set of data that can help to establish an answer to the question of how visual search is accomplished. Together they suggest that eye movements can be a powerful tool for understanding how voluntary search strategies and expectations are integrated with incoming information from a dynamic visual environment. One obvious question that needs to be addressed, is what the effect of the fixation offset on capture observed in Chapter 2 really tells us, in light of the conclusions in Chapter 3, that oculomotor and attentional capture are both reflections of changes in the quality of information about the target identity. The conflict model (see Figure 2.1) suggests that various saccade goals converge on a common map within the SC and compete for control over the eyes based on timing, location, and strength of activation. This seems inconsistent with the notion that this competition is shared with the competition between voluntary and reflexive joystick localization responses. The conflict model does, in fact, require some revision to accommodate the notion that oculomotor capture reflects central, attentional processes that are accessible to all response systems. Based on the current results, a plausible modification to the model is that competition between voluntary and reflexive saccade targets takes place outside of the SC, and the results of this competition are then passed to the eye movement system12. The most plausible candidate for this role is the FEF, because of its shared role in spatial expectancies 1 2 The alternative outcome of this study was that the fixation offset could have selectively facilitated reflexive saccades, which would have supported the predictions of the race model instead of the conflict model. The race model cannot accommodate a competition between target and onset stimuli taking place outside of the eye movement system. 145 for both attention and eye movement. This possibility will be discussed further in the Future Directions section. Given that the SC also receives input directly from the retina, there may indeed be integration within the SC of visual information from the retina with input from other regions of the brain. The current results suggest that processes that are specific to the eye movement system are not the only, or even the major, contribution to capture by sudden onsets, but an alternative way to reconcile the conflict model with the conclusions of Chapter 3 is to propose that the SC itself plays a critical role in the integration of voluntary and reflexive orienting of covert attention as well as eye movements. This interpretation is less plausible, however, given the evidence that orienting of attention does not involve the SC (e.g., Sumner, Adamjee and Mollon, 2002; Posner, Cohen, and Rafal, 1982), can be dissociated from eye movements (e.g., Klein, 1980; Hunt and Kingstone, 2003), and is usually associated with the interplay of networks involving frontal and parietal cortices (e.g. Corbetta and Shulman, 2002). A major debate in the attention capture literature revolves around the issue of whether search is primarily goal-directed or stimulus-driven. The results of Chapter 3 suggest that oculomotor capture research can be used to shed light on attention control, and the results from oculomotor research have thus far been taken as evidence that search is primarily stimulus-driven (e.g., Theeuwes, Kramer^ Hahn, Irwin, and Zelinksy, 1999; Theeuwes & Godijn, 2001). The results of Chapter 3 also suggest that the primary determinant of capture for either response type is the quality of information about the target at the point when a response is executed. This supports neither a stimulus-driven nor a goal-directed account, but suggests instead that both contribute to the rate of information gain and to the quality of 146 information necessary to identify the target. This interpretation of the current results is most consistent with Desimone and Duncan's (1995) biased competition model of attention, wherein competitions between objects in the environment can be influenced by multiple factors, including properties of the visual objects themselves, and feedback mechanisms from working memory. Research that has explored the relative contributions of voluntary and reflexive processes in eye movement control converge on similar conclusions than those under the label "attentional capture", consistent with the notion that they reflect similar processes as attention (e.g., Irwin, Colcombe, Kramer, and Hahn, 2000; Theeuwes et al., 1999; Ludwig and Gilchrist, 2002). That is, voluntary goals and expectations are able to overpower salient distractors in most cases, but there does appear to be some specific default settings, whereby search is more efficient when targets are unique and/or very salient. These default setting can work to the observer's advantage when the target itself is very salient, but can work against the observer when distractors outweigh the target in terms of their perceptual saliency, because saliency alone cannot be used as the primary feature to identify the target. One of the dimensions that does not appear to attract attention or eye movements by default is negative emotion. Previous research has suggested that attention is reflexively captured by negative emotional stimuli (Ohman et al., 2001; Eastwood, Smilek and Merikle, 2001), although other research has not found evidence for this hypothesis, at least among nonanxious individuals (Tipples, Young, Quinlan, Broks and Ellis, 2002; Fox, Russo, Bowles and Dutton, 2001; Purcell, Stewart and Skov, 1996). Chapter 4 examined this unresolved debate using eye movements as a measure of how attention is allocated in the search display. The results suggest that emotion can be used as a strategy for filtering 147 distractors and homing in on the target, but there is no evidence that attention is attracted to negative emotional stimuli any more than neutral or positive stimuli (in fact, there was evidence that positive emotions were more distracting than negative emotions in the present study). One of the strengths of this study was that eye movements, relative to manual detection or discrimination responses, provided a better sense of the locus of attention in the visual display. In particular, I was able to measure both latency and the direction of eye movements, as well as how long they fixated different kinds of distractors, to provide a rich measure of how visual search is influenced by emotional faces. The results of Chapter 4 can also be interpreted in the context of the model of attentional capture proposed in Chapter 3. The hypothesis that attention is reflexively allocated to negative emotional stimuli (Ohman et al., 2001) could be instantiated in this model in two ways. The rate of information accrual about the identity and location of threatening stimuli could be faster than for other kinds of events. Alternatively, the threshold for responding to negative stimuli would be lower than for other kinds of stimuli. In either case, one would expect faster responses to angry face targets and a higher rate of capture by angry face distractors. Neither of these effects were observed. Instead, larger distractor effects by emotional faces occurred only when emotion was the defining feature of the target. When emotion is not the defining feature of the target, emotional distractors no longer interfere with identifying the target. This interpretation argues against an automatic tendency to orient to threatening or emotional stimuli, although it does suggest that emotion can be used to guide attention, and therefore implies that attention is not needed to identify or classify emotions. 148 One important limitation of any study that measures the effect of emotion on attention in a laboratory environment is that stimuli that under genuine circumstances might be emotionally-charged or threatening, are basically harmless. Labeling an angry schematic face presented as an element in a search display as "threatening" is almost certainly an overstatement. Even a photograph of an angry face embedded in a crowd of people lacks the context and impact of a real situation. Thus one weakness of the entire debate about whether negative emotions attracts attention reflexively is that when evidence against the hypothesis is found, it can always be criticized for not having sufficient external validity to demonstrate the true effect of emotion. When evidence in favor of this hypothesis is found, in contrast, one might be tempted to conclude that the effect of emotion was so powerful that it was expressed even under contrived circumstances. On the other hand, the problem of external validity also applies to positive results, in that they could just as easily be an artifact of the laboratory environment. This point as it applies to capture more generally is addressed further in the Future Directions section. As a whole, this thesis has contributed significantly to our understanding of selective attention during visual search. It has rejected the race model as a plausible description of how competition in the oculomotor system is resolved. It has shown that oculomotor capture and attentional capture both arise when responses are executed before information about the target location has had an opportunity to accrue. In doing so, it has also demonstrated that the previous division between oculomotor and attentional capture is in fact artificial. Finally, it has shown that attention is not reflexively attracted to emotionally negative events, while at the same time demonstrating the utility of the oculomotor capture paradigm for exploring questions about selective attention. 149 Future Directions The present work showed that attention and oculomotor capture should not continue to be thought of as distinct phenomena. This raises obvious questions about differences between eye movements and other kinds of responses observed in other areas of investigation (e.g. Sailer, Eggert, Ditterich and Straube, 2000; 2002; Hunt and Kingstone, 2003b). In general, comparison between different kinds of responses could be very a useful diagnostic for understanding which response effects reflect a shared representation that is available to multiple response systems, and which response effects arise because of the response type itself. For comparisons between response types to be truly diagnostic, however, it is critical that reaction time differences be removed. It is thus possible that many differences between response types observed previously were actually due to differences in reaction time, rather than differences in the architecture of the response systems. The current work demonstrates the importance of taking differences in reaction time into consideration when making these kind of comparisons between response types. On a similar note, researchers usually measure attention using manual button-press detection or discrimination responses. Manual responses are an easy task for participants to perform, experiments are simple and inexpensive to set up, and they afford an extremely efficient method for collecting large amounts of reaction time information in a short period of time. But it is well-established that the target of search and the task being performed on that target play an important role in whether or not a given distractor will influence responses (e.g. Bacon and Egeth, 1994; Folk et al., 1992). It has also been shown that manual button-press responses produce a different pattern of results than manual mouse movements (Ludwig and Gilchrist, 2002). The current research made an important connection between manual and saccadic localization responses, but the current conclusions cannot be automatically applied to manual detection or discrimination responses. Indeed, an extremely important issue that attention research as a whole must address is the degree to which the conclusions obtained using manual button-press responses really generalize to visual search in a broader variety of contexts. Another important question is how and where exactly does the attention system influence eye movements? There is clearly a tight coupling between attention and the eyes, in that they seem to move together unless there is a good reason for them to dissociate (Posner, 1980). The FEF appear to be an excellent place to start examining the question of how attention and eye movements are coupled. Voluntary eye movements are associated with the FEF, and voluntary attention is thought to be supported by the dorsofrontal network, involving the IPS and FEF. There is some evidence that shifts of covert attention depend on activity within the FEF (e.g., Moore and Fallah, 2004), although the FEF activity supporting covert attention appears to be distinct from that which supports eye movements (Murthy, Thompson, and Schall, 2001; Juan, Shorter-Jacobi and Schall, 2004). The FEF contain a map that corresponds to spatial expectancies, as well as populations of neurons that control eye movements in similar manner as the SC, with mutually inhibitory "move" and "fixate" neurons. Thus one possible explanation for the tight coupling of eye movements and attention is that both rely on the same spatial expectancy map housed within the FEF. Eye movements are typically thought to use the spatial information from FEF to prepare an eye movement program. The same spatial expectancy map could be used to select a region of space for more detailed processing, or to prepare a 151 reaching response. That is, the spatial expectancies map contained in FEF not only conveys spatial information to other FEF neurons that control the selection of eye movement programs, but it also sends information to visual processing centers that emphasize specific regions of visual cortex for more detailed processing. This hypothesis is consistent with the current findings, but it remains to be seen whether its predictions will be borne out in future studies. It is also important to comment that I am suggesting that this map would aid in coordinating exploration of the visual world by attention and the eyes, but I am not suggesting that all aspects of selective attention involve the FEF. Attention can be allocated to nonspatial properties, like specific colors, common motion, and auditory qualities like pitch. The FEF would be just one part of a larger neural network subserving selective attention more generally, involving multiple areas of parietal and frontal cortex associated with voluntary attention (e.g. Corbetta and Shulman, 2002) and/or working memory (e.g. Desimone, 1998). A final important issue that needs to be resolved is the problem of understanding what kinds of things attract attention reflexively, not just in the contrived environment of the laboratory, but also during exploration of the world in rich, dynamic environments. This issue is especially apparent in Chapter 4, in which emotion was manipulated using schematic face stimuli presented in a search display with neutral and inverted faces. In the context of the literature on emotion and attention, this chapter answers some important questions. But in a larger context it brings to the fore the issue of external validity, which is critical for understanding attentional capture across a variety of situations. This thesis began with an example of a driver searching for a street sign reorienting to crucial but unexpected events, such as another car braking. The attentional capture 152 phenomenon is intended to explore these kinds of circumstances, in which attention disengages from an ongoing task and switches to an important but unexpected event, or in which attention remains focused on the current task in spite of salient but irrelevant distractors. Attentional capture research should therefore be able to predict, and be shaped by, the outcome of real-world situations such as these. For example, accident reports would be a valuable source of information about what kinds of situations lead to attention being focused on critical events, and which lead to attention missing important information because it was distracted by irrelevant events or overly-focused on an ongoing task. These kinds of steps have not yet been taken in the context of attentional capture, but when they are, they will surely lead to a richer and more complete understanding of attention. References Bacon, W.F., & Egeth, H.E. (1994). Overriding stimulus-driven attentional capture. Perception & Psychophysics, 1994, 485-496. Bell, A .H . , Fecteau, J.H. & Munoz, D.P. (2004). Using auditory and visual stimuli to investigate the behavioral and neuronal consequences of reflexive covert orienting. Journal of Neurophysiology, 91, 2172-2184. Corbetta, M . & Shulman, G.L. (2002). Control of goal-directed and stimulus-driven attention in the brain. Nature Reviews Neuroscience, 3, 201-215. Desimone, R. (1998) Visual attention mediated by biased competition in extrastriate visual cortex. Philosophical Transactions of the Royal Society of London, 33, 1245-1255. 153 Desimone, R. & Duncan, J. (1995). Neural mechanisms of selective visual attention. Annual Review of Neuroscience, 18, 193-222. Eastwood, J.D., Smilek, D. & Merikle, P .M. (2001). Differential attentional guidance by unattended faces expressing positive and negative emotion. Perception & Psychophysics, 63,1004-1013. Fecteau, J.H., Bell, A . H . & Munoz, D.P. (2004). Neural correlates of the automatic and goal-driven biases in orienting spatial attention. Journal of Neurophysiology, 92, 1728-1737. Folk, C.L. & Remington, R.W. (1998). Selectivity in distraction by irrelevant featural singletons: evidence for two forms of attentional capture. Journal of Experimental Psychology: Human Perception and Performance, 24, 847-858. Folk, C.L., Remington, R.W. & Johnston, J.C. (1992). Involuntary covert orienting is contingent on attentional control settings. Journal of Experimental Psychology: Human Perception and Performance, 18, 1030-1044. Fox, E., Russo, R., Bowles, R., & Dutton, K. (2001). Do threatening stimuli draw or hold visual attention in subclinical anxiety? Journal of Experimental Psychology: General, 130, 681-700. Godijn, R. & Theeuwes, J. (2002). Parallel programming of saccades: Evidence for a competitive integration model. Journal of Experimental Psychology: Human Perception & Performance, 28, 1039-1054. Hunt, A.R. & Kingstone, A. (2003). Covert and overt voluntary attention: Linked or independent? Cognitive Brain Research, 18, 102-105. 154 Hunt, A.R. & Kingstone, A. (2003b). Inhibition of return: Dissociating attentional and oculomotor components. Journal of Experimental Psychology: Human Perception & Performance, 29, 1068-1074. Irwin, D.E., Colcombe, A . M . , Kramer, A.F. & Hahn, S. (2000). Attentional and oculomotor capture by onset, luminance, and color singletons. Vision Research, 40, 1443-1458. Juan, C.H., Shorter-Jacobi, S.M., & Schall, J.D. (2004). Dissociation of spatial attention and saccade preparation. Proceedings of the National Academy of Sciences, 101, 15541-15544. Klein, R . M . (1980). Does oculomotor readiness mediate cognitive control of visual attention? In R. Nickerson (Ed.) Attention and Performance VIII, pp 259-276. New York: Academic Press. Ludwig, C.J.H., & Gilchrist, I.D. (2002). Stimulus-driven and goal-driven control over visual selection. Journal of Experimental Psychology: Human Perception and Performance, 28(4), 902-912. Moore, T. & Fallah, M . (2004). Microstimulation of the frontal eye fields and its effect on covert spatial attention. Journal of Neurophysiology, 91, 152-162. Murthy, A. , Thompson, K . G . & Schall, J.D. (2001). Dynamic dissociation of visual selection from saccade programming in frontal eye field. Journal of Neurophysiology, 86, 2634-2637. Ohman, A. , Flykt, A. , & Esteves, F. (2001). Emotion drives attention: Detecting the snake in the grass. Journal of Experimental Psychology: General, 130, 466^178. 155 Posner, M.I. (1980). Orienting of attention. Quarterly Journal of Experimental Psychology, 32, 3-25. Posner, M.I. & Cohen, Y . & Rafal, R.D. (1982). Neural systems control of spatial orienting. Philosophical Transactions of the Royal Society of London, B298, 187-198. Purcell, D.G., Stewart, A.L . , & Skov, R.B. (1996). It takes a confounded face to pop out of a crowd. Perception, 25, 1091-1108. Sailer, U . , Eggert, T., Ditterich, J., and Straube, A. (2000). Spatial and temporal aspects of eye-hand coordination across different tasks. Experimental Brain Research, 134, 163-173. Sailer, U . , Eggert, T., Ditterich, J., and Straube, A. (2002). Global effect of a nearby distractor on targeting hand and eye movements. Journal of Experimental Psychology: Human Perception & Performance, 28, 1432-1446. Sumner, P., Adamjee, T. & Mollon, J.D. (2002). Signals invisible to the collicular and magnocellular pathways can capture visual attention. Current Biology, 12, 1312-1316. Theeuwes, J. (1992). Perceptual sensitivity for color and form. Perception & Psychophysics, 43, 307-318. Theeuwes, J. & Godijn, R. (2003). Attentional and oculomotor capture. In C.I. Folk & B.S. Gibson (Eds.) Attraction, Distraction, and Action: Multiple Perspectives on Attentional Capture (pp. 121-150). Amsterdam: Elsevier. Theeuwes, J., Kramer, A.F. , Hahn S., Irwin D.E. & Zelinsky, G J . (1999). Influence of attentional capture on oculomotor control. Journal of Experimental Psychology: Human Perception and Performance, 25, 1595-1608. 156 Tipples, J., Young, A.W., Quinlan, P., Broks, P. & Ellis, A .W. (2002). Searching for threat. The Quarterly Journal of Experimental Psychology, 55A, 1007-1026. Wu, S.C. & Remington, R.W. (2003). Characteristics of covert and overt visual orienting: Evidence from attentional and oculomotor capture. Journal of Experimental Psychology: Human Perception and Performance, 29, 1050-1067. Yantis, S. & Jonides, J. (1990). Abrupt visual onsets and selective attention: Voluntary versus automatic allocation. Journal of Experimental Psychology: Human Perception & Performance, 16, 121-134. 157 APPENDIX A: Location overlap and the relative effect of sudden onset and color singleton distractors The two experiments in this section supplement the experiments presented in the thesis by addressing two outstanding questions. Firstly, throughout the experiments in the present thesis, responses towards the color singleton target are referred to as "voluntary", and responses towards the onset are referred to as "reflexive". But one can question whether a saccade to the color singleton is purely voluntary, in that a uniquely-colored item is quite a salient visual feature, and may draw an eye movement regardless of whether or not it is relevant to the current task. In Experiment 1 below, the color singleton is the target and a irrelevant onset appears on half the trials, similar to the methods used throughout the thesis. In Experiment 2, in contrast, the onset itself is the target, and an irrelevant color singleton appears on half the trials. The results demonstrate that the onset captures eye movements when the color is the target, but the color singleton does not capture attention when the onset is the target. While this does not demonstrate definitively that saccades to the singleton are purely voluntary and the onset are purely reflexive in nature, it does demonstrate that there is an asymmetry in the relationship between these two events, where the onset singleton is a more powerful distractor than the color singleton. Hence, consistent with the oculomotor capture literature, eye movements to the color singleton in the presence of the onset can be conceived of as voluntary and eye movements to the onset distractor can be conceived of as reflexive. The second important purpose here is to explore the effect of the two additional conditions that were added in Chapter 3 and Chapter 4. As mentioned in the introduction to 158 Experiment 1 in Chapter 3, it is possible that previous investigations of oculomotor capture have actually underestimated the extent to which a sudden onset captures eye movements because an onset appears only in locations that are never occupied by a target. It is not known whether participants use this information to selectively inhibit eye movements towards the onset locations. Here I explore this question by first presenting participants with a block of trials in which the target and onset never share locations (e.g., as in Chapter 2 of the current thesis) and then in the second block presenting those two conditions again, randomly intermixed with trials in which the target is both a color singleton and an onset (e.g. as in Chapter 3 of the current thesis). The first and second block can then be compared to measure the degree to which the addition of these two new conditions influences performance on the original conditions. The results show that both the effect of the onset on saccadic latency, and the proportion of saccades directed towards the onset, do not increase from the first to the second block of trials. This result suggests that when the onset is sometimes relevant to the task, there is no systematic increase in the influence of the onset on saccades to the target. Experiment A l : orange target Methods Twelve participants completed 72 trials in block 1 and 144 trials in block 2. In block 1, conditions 1 and 2 shown in Figure 1 were randomly intermixed. The trial begins when the participant presses the space bar and a stable fixation is detected. Six orange circles are presented, and after 1000 ms, all change to red except for one, which is the target. On half the 159 trials, an additional circle is added to the display at one of the six unoccupied positions. The task is to move the eyes from the central fixation crosshair to the orange singleton target. Display when trial starts (1000ms) Experiment 1 Experiment 2 Condition 1 Condition 2 Condition 3 Condition 4 Figure A.1. The different conditions used in Experiments 1 and 2 are illustrated. At the beginning of the trial, the display was always six orange (shown in gray) circles equidistant from fixation. After 1000ms, the display changed according to one of the 4 types of conditions per experiment. In condition one, the target was shown alone (an orange target in E1, and an onset in E2). In condition two, the target was shown together with an irrelevant distractor (a red onset in E1 and an orange singleton in E2). Only these two conditions were included in the first block of trials. In the second block of trials, conditions 3 and 4 were also included. In condition 3, the target was an onset (E1) or the target was orange (E2), and no unique distractor appeared. In condition 4, the target was an onset (E1) or orange (E2) and the unique irrelevant distractor was presented. 160 In Block 2, two additional conditions were intermixed with those shown in Block 1. Target can now appear as an onset (thus in block 1 not all onsets are irrelevant). In condition 3, the target appears as an additional orange circle. In condition 4, there are two onsets, one orange and one red. Results Latency of first saccade. In Block 1, there was a significant effect of the onset on the latency of saccades going directly to the target (see Figure A.2), t(l,ll)=2.93, p<.05. In Block 2, a 1-way A N O V A on all four conditions is significant F(3)=35.23, p<.001, due to the fact that saccades are faster to move to the target when it is an onset than when it is not. onset distractor absent |~1 onset distractor present m C1 C2 C1 C2 C3 C4 Block 1 Block 2 Figure A.2. The reaction time to saccade to the target in Experiment 1 is shown as a function of condition and block. See the methods section and Figure A.1 for a description of the four conditions. 161 In a two-way A N O V A comparing the latency of saccades in conditions 1 and 2 in blocks 1 and 2, there is a main effect of onset distractor F ( l , l 1)=13.74, p<.01, but no effect of block (F(1,11)<1). The interaction was not significant F ( l , l l )= 3.40, p=.0922. If it were significant, it would indicate that the effect of the onset on latency was larger in Block 1 than in Block 2, which is opposite from what would be observed if the introduction of targets in onset positions increases capture. Destination of first saccade. In block 1, there was a significant effect of the onset on the percent of saccades directed towards the target first, with 78.6% going to the target when it appeared alone, and 53.2% going to the target when it appeared with an onset, t(ll)=8.35 (see Figure A.3). On 24.8% of trials in which is appeared, the first saccade was directed towards the onset. ,.o 100 so 4) 1 I 60 o c o 40 j i 20 1 Q 0 C1 C2 Block 1 C1 C2 C3 Block 2 C4 to elsewhere j j to target to onset Figure A.3. The proportion of saccades directed to the onset, target, and elsewhere in the onset-distractor present trials of Experiment 1 are shown as a function of condition and block. 162 In the second block, when all four conditions were analyzed in a one-way A N O V A , there was a significant effect of condition on the percent of saccades going to the target first F(3)=49.05, p<.001. This occurred because condition 2 differs from the other three (all ps>.001) and the other three conditions do not differ from each other. In other words, the irrelevant onset does not influence responses in condition 4. On 31.4% of the trials in condition 2, the eyes went to the onset, but on only 3% of trials in condition 4 did the eyes go to the onset before going to the target. A 2x2 A N O V A on the proportion of saccades going to the target within conditions 1 and 2 only was used to explore the effect of block (1 or 2) and condition (onset present versus absent). There was a main effect of the onset (F(l,ll)=190.36, p<.001), but no effect of block (F(1,11)<1 and no interaction (F(l,ll)=1.02). A t-test on the percent of saccades directed to the onset in block 1 and block 2 was not significant t(l 1)=1.37. It is in the direction that would be expected if the added conditions in block two would serve to increase capture, with 24.8% capture in block 1 and 31.4% in block 2, but when examined on a subject-by-subject bases, there are six positive and six negative differences among individual participants, suggesting there is no systematic increase in capture from the first block to the second. Conclusions. The results demonstrate that responses in conditions 1 and 2 are, in fact, not influenced by the occurrence of the other two conditions. That is, capture by the onset does not vary as a function of whether or not it is interleaved with conditions where the target can appear as an onset. 163 Experiment A2: onset target Methods Twelve new participants were recruited for Experiment 2. Experiment 2 was similar to Experiment 1, with each participant completing 72 trials in block 1 and then 144 trials in block 2. In block 1, the display begins with 6 orange circles. After 1000 ms, all six circles change color, and an additional red circle is added to the display, which is the target. In condition 1, the red onset target is shown alone. In condition two, the onset target is shown at the same time as all the circles change color except one, and this color singleton is irrelevant to the task. In Block 2, an additional two conditions are intermixed with conditions 1 and 2. The target can now be orange (so not all orange items are irrelevant). In condition 3, the target appears as an additional orange circle. In condition 4, there are two orange items, one of which is an onset (see Figure A . l for an illustration). Results Latency of first saccade. A paired t-test on the effect of the onset on saccadic latency in Block 1 revealed a significant effect of the orange singleton, t(l,ll)=4.41, p<.001, but opposite to the expected direction, with faster saccades to the onset when the orange singleton was presented than when the onset appears alone. The reason for this facilitative effect of the onset is not known, but it may be that the extra color in the display had an alerting effect that facilitates overall reaction time. In Block 2, there is no effect of the onset on latency, F(3,33)=2.03. 164 orange distractor absent orange distractor present rr C1 C 2 Block 1 C1 C2 C3 C4 Block 2 Figure A.4. The reaction time to saccade to the onset target in Experiment 2 is shown as a function of condition and block. See the methods section and Figure A.1 for a description of the four conditions. In a 2-way A N O V A on the latency of conditions 1 and 2 alone in blocks 1 and 2 reveals no effect of the orange distractor, F ( l , l 1)=1.05, and no effect of block, F ( l , l 1)=2.56), but the interaction was significant F ( l , l l )= 10.63, p=.01. This interaction indicates that in the first block, the presence of the orange distractor speeded response time by 14.6ms, and in the second block, it had no effect. Destination of first saccade. In Block 1, there was no effect of the onset on the proportion of saccades directed towards the target, t(ll)=0.43. Only 1.1% of saccades were directed to the orange singleton when it was presented. Results were similar in Block 2, with no effect of condition on saccades to the target onset, F(3,33)=1.70. A 2-way A N O V A comparing 165 conditions 1 and 2 only in block 1 and 2 revealed no significant effects of block or condition on the proportion of saccades directed towards the target, and no interaction (all Fs<l). 100 80 60 S Q o g TJ O 4 0 c o £ 20 H C1 C2 Block 1 C1 C2 C3 Block 2 C4 [~1 to elsewhere to target to singleton Figure A.5. The proportion of saccades directed to the onset, target, and elsewhere in the onset-distractor present trials of Experiment 1 are shown as a function of condition and block. Conclusions. There was no evidence that the orange color singleton attracted attention or saccades when the onset was the target. 166 APPENDIX B: Tables of complete data from Chapter 3 In the following tables, the results from the conditions run in Chapter 3 are reported in full. In these experiments, four conditions were included, but for the sake of simplicity, only the first two of these were analyzed in the main text. Here the results from the conditions in which the target was an onset are also reported. Generally speaking, the results mirror those observed in Appendix A, in which responses to the target when it is an onset are faster, more accurate, and less influenced by the presence of an irrelevant onset. Presumably, the added salience of the target when it is both an onset and a unique color reduces the relative impact of the onset, which is an onset but not a unique color (see, for example, Figure A.3, conditions 3 and 4). The table format is the same throughout: the two conditions analyzed in Chapter 3 are shown first (the conditions where the target was static), followed by the results from the two conditions where the target was an onset, which were not analyzed in Chapter 3. Reaction times are shown in milliseconds followed by standard errors. The percent of responses directed towards three regions of the screen (the target, the onset, and other) are also shown. See the methods section of the introduction to the current thesis and the methods of Chapter 3 for more details about how these regions are defined. For the destination results, the totals for a given condition may add up to slightly more or less than 100% due to rounding error. 167 Experiment 1 results Reaction Time (standard error) Eye Movements Joystick Movements Static Target, Onset Absent 335.6 (26.7) 547.2 (23.4) Static Target, Onset Present 332.1 (27.7) 599.3 (27.7) Dynamic Target, Onset Absent 305.8 (29.2) 529.4 (24.7) Dynamic Target, Onset Present 312.8 (29.8) 551.5 (28.7) Percent of Responses Eye Movements To Onset To Target To Other Static Target, Onset Absent — 92.4 (1.8) 7.6(1.8) Static Target, Onset Present 13.7 (2.6) 80.7 (2.4) 5.6(1.5) Dynamic Target, Onset Absent — 94.9 (2.9) 5.1 (2.9) Dynamic Target, Onset Present 2.3 (0.7) 89.1 (4.0) 8.7 (3.6) Joystick Movements To Onset To Target To Other Static Target, Onset Absent — 93.5 (1.3) 6.5 (1.3) Static Target, Onset Present 2.7 (0.8) 89.2 (2.5) 7.9(1.9) Dynamic Target, Onset Absent — 93.1 (1.9) 6.9 (5.8) Dynamic Target, Onset Present 0.0 (0.0) 93.9 (2.0) 6.1 (6.1) 168 Experiment 2 results Reaction Time (standard error) Eye Movements Joystick Movements Static Target, Onset Absent 300.7(11.1) 493.5 (16.8) Static Target, Onset Present 299.0 (15.3) 524.3 (24.3) Dynamic Target, Onset Absent 233.3 (9.6) 442.5 (10.5) Dynamic Target, Onset Present 248.1 (9.8) 451.9 (9.6) Percent of Responses Eye Movements To Onset To Target To Other Static Target, Onset Absent — 86.0 (3.0) 14.0 (3.0) Static Target, Onset Present 32.6 (8.1) 59.3 (7.7) 8.1 (1.3) Dynamic Target, Onset Absent — 96.0 (1.4) 4.0 (1.4) Dynamic Target, Onset Present 5.0 (4.4) 87.1 (2.0) 7.9(1.3) Joystick Movements To Onset To Target To Other Static Target, Onset Absent — 80.5 (2.5) 19.5 (2.5) Static Target, Onset Present 3.1 (4.7) 81.2 (2.4) 15.6(1.9) Dynamic Target, Onset Absent — 80.6 (1.9) 19.4(1.9) Dynamic Target, Onset Present 0.0 (0.0) 83.2 (2.3) 16.8 (2.3) 169 Experiment 3 Results Reaction Time (standard error) 350ms reaction time deadline Eye Movements Joystick Movements (RTD) Static Target, Onset Absent 250.3 (6.4) 306.7 (14.7) Static Target, Onset Present 248.7 (5.9) 297.5 (17.3) Dynamic Target, Onset Absent 229.9 (7.1) 318.9 (8.8) Dynamic Target, Onset Present 234.2 (7.0) 314.3 (13.3) 400ms RTD Eye Movements Joystick Movements Static Target, Onset Absent 258.4 (9.2) 355.1 (8.9) Static Target, Onset Present 257.6 (10.2) 345.6 (4.8) Dynamic Target, Onset Absent 236.7 (7.8) 350.7 (3.2) Dynamic Target, Onset Present 242.9 (7.7) 361.0 (3.3) 500ms RTD Eye Movements Joystick Movements Static Target, Onset Absent 281.7 (10.0) 411.4(19.7) Static Target, Onset Present 273.6 (11.6) 403.6 (22.4) Dynamic Target, Onset Absent 248.7 (9.8) 396.1 (10.0) Dynamic Target, Onset Present 255.5 (10.1) 407.2(13.1) Percent of Responses Eye Movements: 350ms RTD To Onset To Target To Other Static Target, Onset Absent — 71.2(5.2) 28.8(5.2) Static Target, Onset Present 31.3 (5.4) 48.4 (5.7) 20.3 (4.4) 170 Dynamic Target, Onset Absent — 85.3 (5.4) 14.7 (5.4) Dynamic Target, Onset Present 5.0 (4.4) 73.9 (5.3) 17.3 (5.5) Eye Movements: 400ms RTD To Onset To Target To Other Static Target, Onset Absent — 75.2 (5.2) 24.8 (5.2) Static Target, Onset Present 24.1 (5.1) 57.4 (6.7) 18.6 (4.7) Dynamic Target, Onset Absent — 85.5 (4.5) 14.5 (4.5) Dynamic Target, Onset Present 3.3 (1.2) 74.1 (6.3) 22.6 (5.4) Eye Movements: 500ms RTD To Onset To Target To Other Static Target, Onset Absent — 82.6 (5.3) 17.4 (5.3) Static Target, Onset Present 22.8 (4.7) 54.9 (5.9) 22.4 (4.2) Dynamic Target, Onset Absent — 79.1 (6.5) 20.9 (6.5) Dynamic Target, Onset Present 4.1 (1.6) 68.3 (7.1) 27.6 (6.9) Joystick Movements: 350ms RTD To Onset To Target To Other Static Target, Onset Absent — 44.2 (6.9) 55.8 (7.9) Static Target, Onset Present 29.1 (3.7) 40.4 (5.4) 30.4 (4.7) Dynamic Target, Onset Absent — 60.2 (6.7) 37.6 (7.0) Dynamic Target, Onset Present 6.7 (2.1) 43.2 (6.4) 46.3 (6.4) Joystick Movements: 400ms RTD To Onset To Target To Other Static Target, Onset Absent — 68.1 (7.3) 31.9 (7.3) Static Target, Onset Present 17.2 (3.4) 54.4 (6.0) 27.6 (5.7) Dynamic Target, Onset Absent — 75.7 (5.6) 24.3 (5.6) Dynamic Target, Onset Present 4.2 (1.6) 69.4 (5.3) 26.5 (3.9) 171 Joystick Movements: 500ms RTD To Onset To Target To Other Static Target, Onset Absent — 80.8 (6.8) 22.3 (7.0) Static Target, Onset Present 19.5 (4.8) 62.6 (7.1) 19.0 (4.7) Dynamic Target, Onset Absent — 81.6 (5.1) 20.5 (5.5) Dynamic Target, Onset Present 2.9 (2.5) 76.0 (5.6) 25.0 (4.3) Experiment 4 Results Group 1: Target duration of 150, 250 or 350ms Reaction Time (standard error) 150ms Target Deadline (TD) Eye Movements Joystick Movements Static Target, Onset Absent 354.8 (22.6) 477.9 (18.0) Static Target, Onset Present 351.8 (23.6) 481.7 (18.5) Dynamic Target, Onset Absent 327.0 (24.8) 442.0 (13.5) Dynamic Target, Onset Present 332.0 (20.8) 456.4(17.1) 250ms TD Eye Movements Joystick Movements Static Target, Onset Absent 344.8 (21.6) 485.0 (12.8) Static Target, Onset Present 332.9 (24.7) 498.2 (16.0) Dynamic Target, Onset Absent 308.4 (21.9) 450.0 (13.3) Dynamic Target, Onset Present 307.8 (18.8) 478.8 (18.5) 350ms TD Eye Movements Joystick Movements Static Target, Onset Absent 340.6 (25.5) 577.4 (13.6) Static Target, Onset Present 345.6 (24.9) 579.4 (17.2) 172 Dynamic Target, Onset Absent 289.4 (18.9) 565.4 (17.0) Dynamic Target, Onset Present 324.1 (29.5) 558.5 (12.3) Percent of Responses Eye Movements: 150ms TD To Onset To Target To Other Static Target, Onset Absent — 91.9 (2.4) Static Target, Onset Present 22.5 (3.8) 62.9 (4.0) Dynamic Target, Onset Absent — 90.8 (3.6) Dynamic Target, Onset Present 2.2 (0.7) 86.7 (2.8) 8.1 (2.4) 14.6 (3.9) 9.2 (3.6) 11.1 (2.6) Eye Movements: 250ms TD To Onset To Target To Other Static Target, Onset Absent — 92.5 (2.0) Static Target, Onset Present 15.9 (2.7) 74.6 (4.7) Dynamic Target, Onset Absent — 94.3 (2.1) Dynamic Target, Onset Present 1.0 (0.7) 94.1 (1.7) 7.5 (2.0) 9.5 (2.8) 5.7 (2.1) 4.9(1.4) Eye Movements: 350ms TD To Onset To Target To Other Static Target, Onset Absent — 92.6 (2.9) Static Target, Onset Present 22.5 (3.9) 67.8 (4.2) Dynamic Target, Onset Absent — 95.1(1.1) Dynamic Target, Onset Present 2.2 (0.8) 86.9 (3.7) 7.4 (2.9) 9.6 (2.5) 4.9(1.1) 10.9 (3.2) Joystick Movements: 150ms TD To Onset To Target To Other 173 Static Target, Onset Absent — 88.0 (3.6) 12.0 (3.6) Static Target, Onset Present 23.7 (7.7) 64.4 (7.3) 11.8(2.5) Dynamic Target, Onset Absent — 88.6 (2.7) 11.4 (2.7) Dynamic Target, Onset Present 2.4 (1.0) 83.0 (3.4) 14.6 (2.7) Joystick Movements: 250ms TD To Onset To Target To Other Static Target, Onset Absent — 89.8 (1.5) 10.2(1.5) Static Target, Onset Present 10.7 (2.5) 75.4 (3.6) 13.9 (2.3) Dynamic Target, Onset Absent — 88.1 (1.9) 11.9(1.9) Dynamic Target, Onset Present 0.7 (0.5) 88.4 (2.6) 10.9 (2.6) Joystick Movements: 350ms TD To Onset To Target To Other Static Target, Onset Absent — 76.6 (3.3) 23.4 (3.3) Static Target, Onset Present 8.0(1.3) 73.6 (3.7) 18.5 (2.9) Dynamic Target, Onset Absent — 77.9 (4.3) 22.1 (4.3) Dynamic Target, Onset Present 1.2 (0.7) 78.4 (3.8) 20.4 (3.6) Group 2: Target Durations of 350, 400, and 500ms Reaction Time (standard error) 350ms Target Duration (TD) Eye Movements Joystick Movements Static Target, Onset Absent 317.8 (8.9) 509.7 (16.9) Static Target, Onset Present 310.6 (17.5) 551.5 (24.9) Dynamic Target, Onset Absent 270.5 (7.7) 524.7 (23.0) Dynamic Target, Onset Present 286.6 (9.4) 512.9 (19.6) 174 400ms TD Eye Movements Joystick Movements Static Target, Onset Absent 315.8(9.1) 521.7 (27.8) Static Target, Onset Present 335.9(17.5) 543.9 (28.1) Dynamic Target, Onset Absent 284.3 (8.6) 527.1 (39.4) Dynamic Target, Onset Present 295.3 (8.6) 531.5 (29.1) 500ms TD Eye Movements Joystick Movements Static Target, Onset Absent 336.5 (14.0) 496.6 (22.4) Static Target, Onset Present 331.9(14.8) 537.6 (31.3) Dynamic Target, Onset Absent 311.8 (13.7) 478.4 (23.5) Dynamic Target, Onset Present 313.5 (12.8) 511.2 (26.2) Percent of Responses Eye Movements: 350ms TD To Onset To Target To Other Static Target, Onset Absent — 94.7(1.5) 5.1 (1.5) Static Target, Onset Present 11.2(1.8) 79.8 (3.3) 9.1 (2.1) Dynamic Target, Onset Absent — 93.1 (1.9) 6.9(1.9) Dynamic Target, Onset Present 1.4(0.6) 85.5 (2.5) 13.1 (2.3) Eye Movements: 400ms TD To Onset To Target To Other Static Target, Onset Absent — 93.1 (2.1) 6.9 (2.1) Static Target, Onset Present 10.7 (3.1) 76.3 (3.8) 13.6 (2.9) Dynamic Target, Onset Absent — 91.5 (1.6) 9.0 (1.7) Dynamic Target, Onset Present 2.6 (0.9) 85.3 (2.0) 12.6 (2.0) 175 Eye Movements: 500ms TD To Onset To Target To Other Static Target, Onset Absent — 92.0(1.8) 8.0(1.8) Static Target, Onset Present 11.9 (2.6) 76.8 (2.8) 11.0 (2.2) Dynamic Target, Onset Absent — 91.8 (2.2) 8.0 (2.2) Dynamic Target, Onset Present 1.9 (0.6) 82.5 (3.6) 14.7 (3.3) Joystick Movements: 350ms TD To Onset To Target To Other Static Target, Onset Absent — 73.4 (3.3) 26.6 (3.3) Static Target, Onset Present 6.6(1.6) 71.6 (3.6) 21.8 (2.3) Dynamic Target, Onset Absent — 68.5 (4.0) 31.5 (4.0) Dynamic Target, Onset Present 0.8 (0.4) 70.0 (3.7) 29.2 (3.5) Joystick Movements: 400ms TD To Onset To Target To Other Static Target, Onset Absent — 67.6 (3.5) 32.4 (3.5) Static Target, Onset Present 8.7 (1.4) 68.6 (3.5) 22.6 (3.0) Dynamic Target, Onset Absent — 70.9 (3.4) 29.1 (3.4) Dynamic Target, Onset Present 0.5 (0.3) 73.4 (3.2) 26.1 (3.1) Joystick Movements: 500ms TD To Onset To Target To Other Static Target, Onset Absent — 80.3 (2.5) 19.7 (2.5) Static Target, Onset Present 7.4(1.7) 76.1 (2.8) 16.4(1.9) Dynamic Target, Onset Absent — 78.9 (3.4) 21.1 (3.4) Dynamic Target, Onset Present 0.2 (0.2) 79.3 (2.5) 20.5 (2.5) 176 APPENDIX C: Emotions Conveyed by the Stimuli used in Chapter 4 The emotions conveyed by the happy, angry, and neutral faces used in Chapter 4 were checked using a brief questionnaire administered to a naive group of participants who did not participate in either of the experiments reported in Chapter 4. The questionnaire asked participants to rate the three faces on a 7-point scale for the degree to which they appeared to be happy, angry, scared, surprised, disgusted, and sad. The order in which the faces were listed was counterbalanced across the 12 participants who completed the questionnaire. The happy face was rated highest on the "happy" scale, and the angry face was rated highest on the "angry" scale. There were also elements of surprise in the happy face, and elements of disgust and sadness in the angry face. It is also interesting to note that the neutral face is seen as slightly more happy than angry. Average ratings of the three faces used in Chapter 4 on six emotional dimensions. Face Type Happy Angry Neutral Rating Scale Happy 5^ 83 LOO 3^ 25 (from 1 to 7) Angry 1.08 5.75 2.17 Surprised 2.75 1.42 2.42 Disgusted 1.08 3.42 1.92 Sad 1.50 3.58 2.08 Scared 1.42 1.92 1.83 

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
http://iiif.library.ubc.ca/presentation/dsp.831.1-0099847/manifest

Comment

Related Items