UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Crossmodal interactions in stimulus-driven spatial attention and inhibition of return: evidence from… MacDonald, John J. 1999

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata

Download

Media
831-ubc_1999-389367.pdf [ 8.09MB ]
Metadata
JSON: 831-1.0099469.json
JSON-LD: 831-1.0099469-ld.json
RDF/XML (Pretty): 831-1.0099469-rdf.xml
RDF/JSON: 831-1.0099469-rdf.json
Turtle: 831-1.0099469-turtle.txt
N-Triples: 831-1.0099469-rdf-ntriples.txt
Original Record: 831-1.0099469-source.json
Full Text
831-1.0099469-fulltext.txt
Citation
831-1.0099469.ris

Full Text

CROSSMODAL INTERACTIONS IN STIMULUS-DRIVEN SPATIAL ATTENTION AND INHIBITION OF RETURN: EVIDENCE FROM BEHAVIOURAL AND ELECTROPHYSIOLOGICAL MEASURES By JOHN J. MCDONALD B.A., Simon Fraser University, 1993 M.A., The University of British Columbia, 1996 A THESIS SUBMITTED IN PARTIAL FULFILMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY in THE FACULTY OF GRADUATE STUDIES (Department of Psychology) We accept this thesis as conforming to the required standard THE UNIVERSITY OF BRITISH COLUMBIA April 1999 © John Joseph McDonald, 1999 In p r e s e n t i n g t h i s t h e s i s i n p a r t i a l f u l f i l m e n t of the requirements fo r an advanced degree at the U n i v e r s i t y of B r i t i s h Columbia, I agree tha t the L i b r a r y s h a l l make i t f r e e l y a v a i l a b l e fo r reference and s tudy . I f u r t he r agree tha t p e r m i s s i o n fo r ex t ens ive copying of t h i s t h e s i s fo r s c h o l a r l y purposes may be granted by the head of my department or by h i s or her r e p r e s e n t a t i v e s . I t i s understood tha t copying or p u b l i c a t i o n of t h i s t h e s i s fo r f i n a n c i a l g a i n s h a l l not be a l lowed wi thout my w r i t t e n p e r m i s s i o n . Department of PS^OHOIOGM The U n i v e r s i t y of B r i t i s h Columbia Vancouver, Canada Date I?, flfiRll ffl 11 ABSTRACT Ten experiments examined the interactions between vision and audition in stimulus-driven spatial attention orienting and inhibition of return (IOR). IOR is the demonstration that subjects are slower to respond to stimuli that are presented at a previously stimulated location. In each experiment, subjects made go/no-go responses to peripheral targets but not to central targets. On every trial, a target was preceded by a sensory event, called a "cue," either in the same modality (intramodal conditions) or in a different modality (crossmodal conditions). The cue did not predict the location of the target stimulus in any experiment. In some experiments, the cue and target modalities were fixed and different. Under these conditions, response times to a visual target were shorter when it appeared at the same location as an auditory cue than when it appeared on the opposite side of fixation, particularly at short (100 ms) cue-target stimulus onset asynchronies (Experiments 1A and IB). Similarly, response times to an auditory target were shorter when it appeared at the same location as a visual cue than when it appeared at a location on the opposite side of fixation (Experiments 2A and 2B). These crossmodal effects indicate that stimulus-driven spatial attention orienting might arise from a single supramodal brain mechanism. IOR was not observed in either crossmodal experiment indicating that it might arise from modality specific mechanisms. However, for many subjects, IOR did occur between auditory cues and visual targets (Experiments 3A and 3B) and between visual cues and auditory targets (Experiment 4A and 4B) when the target could appear in the same modality as the cue on half of the trials. Finally, the crossmodal effects of stimulus-driven spatial attention orienting on auditory and visual event-related brain potentials (ERPs) were examined in the final two experiments. Auditory cues modulated the ERPs to visual targets and visual cues modulated the ERPs to auditory targets, demonstrating that the mechanisms for spatial attention orienting cannot be completely modality specific. However, these crossmodal ERP effects were very different from each other indicating that the mechanisms for spatial attention orienting cannot be completely shared. I l l TABLE OF CONTENTS Abstract 1 1 List of Tables vi List of Figures vii Acknowledgments x 1. INTRODUCTION 1 2. BACKGROUND TO BEHAVIOURAL EXPERIMENTS 3 2.1 Spatial Attention Orienting 3 2.1.1 Covert Spatial Attention Orienting to Visual Stimuli 3 2.1.2 Covert Spatial Attention Orienting to Nonvisual Stimuli 7 2.1.3 Mechanisms and Consequences of Covert Spatial Attention Orienting 12 2.2 Inhibition of Return 14 2.2.1 Mechanisms of Inhibition of Return 15 2.2.2 Consequences of Inhibition of Return 17 2.2.3 Inhibition of Return to Nonvisual Stimuli 18 2.3 Modality Specificity of Spatial Attention and IOR 21 2.3.1 Intersensory Interactions in Perception and Performance 25 2.3.2 Multimodal Convergence in the Brain 28 2.3.3 Crossmodal Interactions in Spatial Attention 31 2.3.4 Crossmodal Interactions in Inhibition of Return 36 3. AIMS OF PRESENT STUDY 40 4. GENERAL METHODS 43 4.1 Stimuli and Apparatus 44 4.2 Procedure and Design 45 4.3 Eye Movement Monitoring 46 4.4 Data Analysis 47 iv 5. EXPERIMENTS IA AND IB 49 5.1 Method 50 5.2 Results 51 5.3 Discussion 54 6. EXPERIMENTS 2A AND 2B 58 6.1 Method •••• 60 6.2 Results 60 6.3 Discussion 64 7. EXPERIMENTS 3A AND 3B 72 7.1 Method 73 7.2 Results 74 7.3 Discussion 79 8. EXPERIMENTS 4A AND 4B 82 8.1 Method 82 8.2 Results 83 8.3 Discussion 89 9. BACKGROUND TO ELECTROPHYSIOLOGICAL EXPERIMENTS 97 9.1 Event-Related Brain Potentials (ERPs) 98 9.1.1 ERP Recording and Measurement 98 9.1.2 Making Inferences from ERPs 100 9.1.3 Why Use ERPs to Study Attention? 103 9.2 Attentional Modulation of Visual ERPs 104 9.3 Attentional Modulation of Auditory ERPs 107 9.4 ERP Measurement of Crossmodal Attention 109 10. AIMS OF ELECTROPHYSIOLOGICAL EXPERIMENTS 111 V 11. ELECTROPHYSIOLOGICAL METHODS 114 11.1 EEG Recording 114 11.2 Artifact Rejection and Averaging 115 11.3 Statistical Analyses 115 12. EXPERIMENT 5 116 12.1 Method 116 12.2 Results 118 12.3 Discussion 127 13. EXPERIMENT 6 130 13.1 Method 131 13.2 Results 133 13.3 Discussion 138 14. GENERAL DISCUSSION 139 14.1 Spatial Attention Orienting: Modality Specific or Supramodal? 145 14.2 Can a Strongly Supramodal Account be Salvaged? 151 14.3 Strategic Control Factors '. 154 14.4 Inhibition of Return: Modality Specific or Supramodal? 158 14.5 Future Cognitive Neuroscience Experiments 160 References 164 vi LIST OF TABLES Table 1 Mean Response Time (M; in milliseconds), Standard Error (SE), and Percent Errors (%E) as a Function of Cue Validity and Stimulus Onset Asynchrony (SOA) in Experiments 1A and IB 2 Mean Response Time (M; in milliseconds), Standard Error (SE), and Percent Errors (%E) as a Function of Cue Validity and Stimulus Onset Asynchrony (SOA) in Experiments 2A and 2B 3 Mean Response Time (M; in milliseconds), Standard Error (SE), and Percent Errors (%E) as a Function of Cue Validity and Stimulus Onset Asynchrony (SOA) in Experiments 3A and 3B 4 Mean Response Time (M; in milliseconds), Standard Error (SE), and Percent Errors (%E) as a Function of Cue Validity and Stimulus Onset Asynchrony (SOA) in Experiments 4A and 4B 5 Mean Response Time (M; in milliseconds), Standard Error (SE), and Percent Errors (%E) as a Function of Cue Validity and Stimulus Onset Asynchrony (SOA) in Experiment 5 6 Mean Response Time (M; in milliseconds), Standard Error (SE), and Percent 134 Errors (%E) as a Function of Cue Validity and Stimulus Onset Asynchrony (SOA) in Experiment 6 Page 52 62 76 vii LIST OF FIGURES Figure Page 1 Schematic diagrams of the possible relationships between the visual, 23 auditory, and somatic spatial attention mechanisms. (A) Three separate visual, auditory, and somatic spatial attention mechanisms. (B) Three interacting visual, auditory, and somatic spatial attention mechanisms. (C) A single, supramodal spatial attention mechanism. 2 The crossmodal interactions in stimulus-driven spatial attention that have 37 been demonstrated to date. The visual-on-auditory and auditory-on-visual cue effects are absent from the illustration because there is conflicting data about their occurrence. 3 Example of stimulus display sequences in Experiments 1 to 4. The variable 48 stimulus onset asynchrony (SOA) between cue and target is obtained by adding the cue duration (70 ms) with the variable time interval that immediately follows. 4 Mean response times (RTs; in milliseconds) as a function of the cue-target 55 stimulus onset asynchrony (SOA) in Experiment 1. 5 Mean response times (RTs; in milliseconds) as a function of the cue-target 65 stimulus onset asynchrony (SOA) in Experiment 2. 6 Mean response times (RTs; in milliseconds) as a function of the cue-target 78 stimulus onset asynchrony (SOA) in Experiment 3. 7 Mean response times (RTs; in milliseconds) as a function of the cue-target 88 stimulus onset asynchrony (SOA) in Experiment 4. 8 Individual subjects' cue effects (invalid - valid difference) in Experiments 93 3A, 3B, 4A, and 4B. The intramodal and crossmodal effects are shown separately for each experiment, both with eye-position monitoring (denoted A) and without eye-position monitoring (denoted B). The cue effects are for the 900-ms stimulus onset asynchrony (SOA) in all conditions except for the V-V condition, where the 500-ms SOA is shown. Negative cue effects denote the presence of IOR. viii Figure Page 9 Interactions between a hypothetical spatial attention mechanism and the 96 visual and auditory "what" systems. Excitatory pathways are shown as open circles and inhibitory pathways are shown as filled circles. Note, only a few functionally important connections are shown. LGN = lateral geniculate nucleus, VI etc. = areas of visual cortex, CN = cochlear nucleus, SO = superior olive, IC = inferior colliculus, MGN = medial geniculate nucleus, A l etc. = areas of auditory cortex. The broken connection between the posterior parietal and superior colliculus indicates other brain areas (e.g., substania nigra pars reticulata) mediating the pathway. 10 Idealized event-related potential to an auditory stimulus plotted on a log- 102 time scale to show the auditory brainstem responses (peaks I-V), the mid-latency cortical components (N0, P0, N a , Pa, Nb), the late transient cortical components (PI, NI, P2), and the late task-related components (Nd, N2, P3, SW). Reprinted from Hillyard (1993). 11 Event-related potentials (ERPs) to visual target stimuli preceded 100-300 ms 122 by spatially uninformative auditory cues in Experiment 5. The waveforms shown were corrected by the adjacent response (Adjar) filter to remove distortion caused by the cue ERP (see text for details). Following the adjar procedure, the ERPs were averaged across 10 subjects and across left and right targets. The solid line represents the ERP on valid-cue trials and the dashed line represents the ERP on invalid-cue trials. 12 ERPs to visual target stimuli preceded 900-1100 ms by spatially 123 uninformative auditory cues in Experiment 5. The waveforms shown were averaged across 10 subjects and across left and right targets. The solid line represents the ERP on valid-cue trials and the dashed line represents the ERP on invalid-cue trials. 13 The original ERPs elicited by the visual target at the Cz site (left), along with 125 the estimated residual cue responses (middle) and adjar-corrected ERPs (right). All waveforms were averaged across 10 subjects. The solid line represents the ERP on valid-cue trials and the dashed line represents the ERP on invalid-cue trials. Note that cue validity differences are observed in both the uncorrected and corrected waveforms. Figure Page Event-related potentials (ERP) to auditory target stimuli preceded 100-500 ms by spatially uninformative visual cues in Experiment 6. The waveforms shown were averaged across 12 subjects and across left and right targets. The solid line represents the ERP on valid-cue trials and the dashed line represents the ERP on invalid-cue trials. X ACKNOWLEDGMENTS The present thesis was carried out in the Psychophysics Laboratory in the Department of Psychology, University of British Columbia, Vancouver, Canada. I express my warmest thanks to my supervisor, Dr. Lawrence Ward, for providing me the opportunity and the encouragement for this research. He also read several drafts of the thesis and provided me with astute comments, for which I am grateful. It has been a pleasure learning from one of the world's finest psychophysicists. I also express my thanks to Drs. Vincent Di Lollo and James Enns for many valuable discussions and for their inspiring approach to scientific research. I express my gratitude to colleagues Christian Richard, Kent Kiehl, and David Prime for their friendship, support, and collaboration. I wish them all luck in their future endeavors. This work was financially supported by a post-graduate research scholarship from the Natural Sciences and Engineering Research Council of Canada (NSERC) and from NSERC grants awarded to Dr. Ward. Finally, I thank my family for their support. I especially thank my wife, Debby Giassi McDonald, for the inspiration, support, and patience that she has given me over the years. 1 1 INTRODUCTION Humans live in a dynamic environment that consists of many objects occurring at various positions in space and time. Coherent perceptual, cognitive, and motor functioning therefore requires selective processing of relevant sensory events at the expense of less relevant ones. Indeed, only a small portion of the information reaching our sensory systems is used to construct an internal representation of the external world. The cognitive process that enables us to preferentially process relevant sensory information is called selective attention (for reviews, see LaBerge, 1990; Johnston & Dark, 1986). Consider, for example, a predatory animal's task of searching for its prey. This task must be accomplished skillfully and efficiently in order to ensure the survival of the predator. In some situations, the predator can locate its prey easily, such as when the prey wanders across an open field. In these situations, finding the prey can be accomplished simply by monitoring changes in the most relevant sensory modality. In other situations, however, the predator must monitor changes in several modalities simultaneously in order to find the prey. The existence of multiple sensory systems affords considerable perceptual flexibility, thereby increasing the likelihood that the predator will successfully locate its prey. Thus, in a field of tall grass, a cat might monitor changes in visual (e.g., the sight of rustling grass) and auditory (e.g., the sound of rustling grass) stimulation in order to locate a mouse. Of course, once the prey has been located, pursuit must be accurately directed in order for the hunt to be successful. The existence of a stable, unified representation of the external world increases the likelihood that subsequent pursuit of the prey will be directed accurately. 2 As the above example indicates, adaptive organisms must accomplish two important goals. First, they must monitor changes in the external world in order to detect meaningful events. The ability of many organisms to monitor such changes is maximized by the existence of multiple sensory systems that each provides a unique means of perceiving the environment. Second, organisms must maintain the perception of a stable external world so that their actions can be coordinated readily and accurately with other objects in the environment. Consequently, the brain is faced with the momentous task of integrating signals from seemingly separate modalities into the unitary perception that we experience. Converging evidence from studies of individual sensory modalities indicates that selective attention is critically important for accomplishing both of the goals listed above. In particular, it is well known that salient visual events can attract attention to their spatial location (Jonides, 1981) and that, in the absence of attention, visual features can be integrated incorrectly to form illusory objects (Treisman & Gelade, 1980). Whereas these and other findings are indeed significant, research on individual modalities can lead to only a partial understanding of the way humans represent the external world and of how such representations influence our behaviour. For example, classical studies of attention have examined our ability to selectively listen to spoken messages in the absence of visual input (e.g., Cherry, 1953). However, seeing the articulatory movements of the speaker's lips can dramatically improve speech perception during face-to-face conversation, particularly in noisy situations (Sumby & Pollack, 1954). The influence of one modality (e.g., vision) on selective processing of signals in another modality (e.g., audition) demonstrates that attention must be coordinated across the senses. Consequently, 3 it is important to gain a better understanding of the mechanisms that coordinate selective attention across different sensory modalities and of the consequences of such coordination on brain activity, perception, and behaviour. 2 BACKGROUND TO BEHAVIORAL EXPERIMENTS 2.1 Spatial Attention Orienting Over a century ago, James (1890) characterized one type of attentional process as the "accommodation or adjustment of the sensory organs" (p. 434). This characterization implies that overt changes of the sensory organs (i.e., movements of the eyes) usually occur when we attend to different objects in the external world. However, James (1890) proposed that selective attention could also produce changes "within ideational centers concerned with the object to which the attention is paid" (p. 434). So, whereas we usually look at objects to which we are attending, we can also attend to them without any change in eye position. In modern terminology, orienting attention to spatial locations without adjustment of the sensory organs is called covert spatial attention orienting (Posner, 1980). More generally, cover spatial attention orienting refers to the allocation of limited-capacity mental processes to selected spatial locations in the absence of changes in eye position. 2.1.1 Covert Spatial Attention Orienting to Visual Stimuli In the past two decades, researchers have learned a great deal about the processes involved in selective attention by studying how humans covertly orient their attention to different 4 locations in visual space (for recent reviews, see Klein, Kingstone, & Pontefract, 1992; Wright & Ward, 1998). Laboratory studies of visual spatial attention have commonly used the spatial cueing paradigm to examine the effects of orienting attention on stimulus processing. In this paradigm, an initial stimulus, called the "cue," is used to direct attention to a specific location prior to the appearance of a target stimulus to which a response is made. The cue is said to be valid if it provides correct information about the target's location and invalid if it provides incorrect information about the target's location. In principle, a cue can also be "neutral" if it provides no information about the target's location. However, the appropriate use of a neutral cue depends on many factors, including whether or not it provides the same alerting effect as the valid and invalid cues (Jonides & Mack, 1984). Many spatial cueing experiments involve the presentation of a symbolic cue that is informative about the likely location of the forthcoming target. For example, a leftward pointing arrow would be a valid cue for a target appearing to the left of the arrow and would be an invalid cue for a target appearing to the right of the arrow. In such studies, the subjects are typically told that the target will appear at the validly cued location on most trials (e.g., 80%) and that they should focus their attention at that location. The general result of these experiments is that the subjects respond to the target both more quickly and more accurately when it is presented at a validly cued location than when it is presented at an invalidly cued location. This symbolic cue effect, which occurs in the absence of eye movements, is taken as an indication that a shift of attention is initiated prior to the appearance of the target. A more detailed analysis reveals that the response times (RTs) are faster on valid-cue trials and slower on invalid-cue trials than on 5 neutral-cue trials (in symbolic cue experiments, a common neutral cue is an arrow that points in both directions). The performance advantage on valid-cue trials relative to neutral-cue trials indicates that there is a benefit (or facilitatory effect) of shifting attention to the location indicated by the cue, whereas the performance decrement on invalid-cue trials relative to neutral-cue trials indicates that there is also a cost (or inhibitory effect). The benefits and costs of symbolic spatial cues have been demonstrated across a wide range of tasks involving simple RT, identity-based choice RT, and location-based choice RT, indicating that shifting spatial attention has general effects on visual performance (e.g., Jonides, 1981; Posner, 1980). Other spatial cueing experiments involve the prior presentation of a salient visual stimulus at or near a potential target location. These stimuli are called direct (or peripheral) cues, and they usually involve abrupt onsets or luminance increments. In principle, however, a direct cue could be any visual event that signals a possible target location simple on the basis of its appearance. Thus, in contrast, to symbolic spatial cues, direct cues do not have to be informative (that is, predictive) about the location of the target in order to influence RT performance. Rather, they can influence performance even when the subjects are given explicit instructions to ignore them (e.g., Jonides, 1981). Under some direct cue conditions, subjects respond to the target both more quickly and more accurately when it appears at the validly cued location than when it appears at the invalid cue location. Despite the similarity of these effects and those produced by symbolic spatial cues, several investigators have discovered important differences between them (for recent reviews, see Klein et al., 1992; Wright & Ward, 1998). The major difference is that the time courses of the facilitatory and inhibitory effects of direct and symbolic spatial cues are 6 different (e.g., Muller & Rabbitt, 1989). The facilitatory effect of spatially uninformative direct cues is largest when the stimulus onset asynchrony (SOA) between the cue and target is short (100-150 ms), whereas the facilitatory effect of symbolic cues is largest when the SOA is longer (> 400 ms). Moreover, the relative facilitation produced by uninformative direct cues is replaced at longer cue-target intervals by a relative inhibitory effect, called "inhibition of return" (discussed in Section 2.2), whereas symbolic cues fail to produce this effect. In addition, the effects of symbolic cues are more susceptible to interference from concurrent memory tasks (e.g., Jonides, 1981) relative to the effects of direct cues. These differences indicate that there are two ways in which attention can be oriented in space. One way, initiated by symbolic cues, is slow and voluntary, whereas the other way, initiated by direct cues, is faster and more automatic (Jonides, 1981; Muller & Rabbitt, 1989). The effects of such symbolic cues are said to be goal-driven (endogenous) because they are dependent on voluntary, or top-down, processes. In contrast, the effects of direct cues are said to be stimulus-driven (also called exogenous) because they occur when the cue is uninformative, are less susceptible to interference, and cannot be voluntarily suppressed. Of course, a mixture of goal-driven and stimulus-driven effects can sometimes arise, such as when a direct cue is made spatially informative so that, like a symbolic cue, it indicates the likely location of the target. Some believe that goal-driven and stimulus-driven effects reflect different ways in which the same attentional processes are brought to bear on the target stimuli at selected spatial locations (e.g., Jonides, 1981; Muller & Rabbitt, 1989; Posner, 1980), whereas others believe that they reflect two different types of attentional processes (e.g., Briand & Klein, 1987). 7 2.1.2 Covert Spatial Attention Orienting to Nonvisual Stimuli Although the majority of spatial cueing studies have examined the effects of visual cues on processing of subsequent visual stimuli, there is growing evidence that spatial attention can also influence processing of stimuli from other sensory modalities. For example, when a spatially informative auditory cue is followed by an auditory target, RTs to the target are shorter on valid-cue trials than on invalid-cue trials. Such effects occur in tasks involving identity-based (McDonald & Ward, in press; Mondor & Zatorre, 1995) and location-based (Bedard, Massioui, Pillon, & Nandrino, 1993; Quinlan & Bailey, 1995; Spence & Driver, 1994) choice responses. However, there is conflicting evidence regarding the effect of informative spatial cueing on simple RT to sounds. Small cue effects have been observed in some auditory simple RT tasks (Bedard et al., 1993; Buchtel, Butter, & Ayvasik, 1996; Quinlan & Bailey, 1995) but not in others (Buchtel & Butter, 1988; Posner, 1978). Several researchers have argued that spatial attention typically does not affect simple RTs to sounds because such responses can occur before sound localization takes place (Posner, 1978; Rhodes, 1987; Spence & Driver, 1994). The rationale for this argument is that sound location is not mapped topographically by the auditory receptors, which are located in the cochlea. Instead, the single available spatial dimension of the auditory receptor surface, distance along the cochlea, is used to encode spectral frequency (i.e., pitch). This "tonotopic" representation is preserved throughout most of the auditory system. The location of a sound source must therefore be computed by specialized, location-sensitive neurons in subcortical brain areas, such as the superior olives, and the inferior and superior colliculi, based on differences between the ears in 8 the phase or intensity of the sound. Of these brain areas, only the superior colliculus has a well defined spatiotopic representation like that of the visual system. Most neurons in this midbrain structure, however, respond to sensory events in multiple modalities. Thus, the superior colliculus is not a modality-specific auditory area. In light of the fact that the auditory system is organized primarily with respect to spectral frequency rather than spatial location, some researchers have examined the effects of uninformative spatial cueing in location-based auditory discrimination tasks (McDonald & Ward, in press; Spence & Driver, 1994; Ward, 1994). However, the interpretation of the data from standard location-based tasks, which require explicit responding in the direction of the target, can be weakened by nonattentional explanations for the observed cue effects. The most serious problem is that the usual facilitatory effect of a spatial cue can often be attributed to response-level effects arising from stimulus-response compatibility. This is particularly problematic when both the cue and target appear on the left or right side of fixation and subjects are required to make left-right discrimination responses based on the location of the target. In this situation, subjects might respond faster on valid-cue trials than on invalid-cue trials simply because the cue activates the correct response on valid-cue trials. Indeed, irrelevant location information can have dramatic effects on choice RT in many situations, such as in the well-known Simon paradigm (for a review, see Simon, 1990). In this paradigm, subjects respond to a target stimulus on the basis of some nonspatial dimension (e.g., colour) that is assigned to left and right manual responses. The location of the target also varies but is nonetheless irrelevant to the response. The Simon effect is the demonstration that subjects respond faster when the target location 9 corresponds to the location of the assigned response than when it does not. For example, Simon and Small (1969) instructed subjects to press a left button for a low-frequency tone and a right button for a high-frequency tone. Correct responses to the low-frequency tone were on average 60 ms faster when the tone was heard in the left ear rather than in the right ear. Similarly, correct responses to the high-frequency tone were on average 62 ms faster when the tone was heard in the right ear rather than in the left ear. This effect indicates that there is a natural tendency to react in the direction of a stimulus even when the location of that stimulus is irrelevant. Two approaches have been developed to eliminate the possibility of response priming in auditory spatial cueing experiments. In one approach, used by Spence and Driver (1994), auditory cues are presented on left and right sides of fixation while targets are presented above or below the cued location, either on the same side of fixation ("valid" trial) or on the opposite side ("invalid" trial). Subjects are then required to make choice responses based on the target's elevation, so that the response dimension (up-down) is orthogonal to the cued dimension (left-right). Using this technique, which they called "orthogonal-cueing," Spence and Driver (1994) found that elevation-based responses were faster when the cue and target appeared on the same side of fixation rather than opposite sides. These results provide converging evidence that spatial attention can influence the processing of sounds because they cannot be attributed to response-priming effects. In another approach, used by McDonald and Ward (in press), the decision to make a go/no-go response is based on the spatial location of the target stimulus. For example, listeners might respond by pressing a single button to targets emanating from either of two peripherally 10 located speakers but not to targets emanating from a centrally located speaker. The rationale for this technique is that the requirement to respond to targets appearing at just certain locations ensures the use of location-sensitive neurons in making responses, but it does not allow cues to activate responses differentially on valid-cue and invalid-cue trials. McDonald and Ward (in press) called this approach the "implicit spatial discrimination" technique. Based on the anatomical considerations noted by other researchers, McDonald and Ward (in press) hypothesized that the spatial locations of auditory stimuli must be relevant to the experimental task in order for spatial cue effects to occur in audition. This hypothesis, called the spatial-relevance hypothesis, was based on the idea that making spatial locations relevant to the task induces the use of location-sensitive neurons to inform the response made to the auditory target, and that only these neurons are sensitive to the effects of spatial cues. Importantly, McDonald and Ward (in press) used the implicit spatial discrimination paradigm to test this hypothesis without the possibility of response-priming by the cue. Consistent with the spatial-relevance hypothesis, a spatially uninformative auditory cue influenced response latencies to an auditory target when the decision to respond was based on the spatial location of the target but not when the decision to respond was based on the spectral frequency or onset of the target. As in visual tasks using uninformative spatial cues, response times were faster on valid-cue trials than on invalid-cue trials when the stimulus onset asynchrony (SOA) between cue and target was 100 ms. Given the effect of spatial relevance on the spatial cue effects in audition, it is appropriate to consider in more detail the criteria that establish whether a given task is spatial or nonspatial. 11 The most direct criterion is whether the decision to respond is based on the spatial location of the target or on some nonspatial feature. On the basis of this criterion, a spatial task is one in which the decision to respond is based on the physical spatial location of the target with respect to the observer ("egocentric" space), whereas a nonspatial task is one in which the decision to respond is based on a nonspatial feature of the target. Although this criterion is intuitive, there are at least two potential problems with it. First, under some conditions, it is possible to pay attention to the spatial location of the target even when the decision to respond is based on a nonspatial feature. In particular, subjects might be able to attend to the spatial location of the target when making simple reactions to it (i.e., responding according to the onset of the target stimulus) because such responses are made rather effortlessly. One way to overcome this problem would be to make the nonspatial task sufficiently difficult so that subjects could not possibly attend to the location of the target and perform the task adequately. Second, there are interactions between some spatial and nonspatial features in the auditory and visual modalities. For example, under some conditions, a sequence of tones that is presented in ascending or descending order of spectral frequency is heard as correspondingly ascending or descending in space (e.g., Roffler, 1968). Similar interactions between "what" and "where" have been observed in vision, both in terms of the perception of features and in terms of anatomical connections (e.g., Zeki, 1993). In the absence of any such interactions, however, one can be fairly certain that a given feature is nonspatial. For example, although there are interactions between spectral frequency and elevation (i.e., vertical) localization, there are few, if any, interactions between spectral frequency and azimuthal (i.e., horizontal) localization. The 12 absence of auditory spatial cue effects on implicit auditory frequency discrimination RTs in several recent experiments, even though the same cues influenced implicit spatial discrimination RTs (McDonald & Ward, in press), provides converging evidence that such discriminations can be nonspatial. Most visual nonspatial features, however, with the possible exception of colour, probably interact to some degree with spatial location. 2.1.3 Mechanisms and Consequences of Covert Spatial Attention Orienting An important distinction has been made between the mechanisms (causes) and consequences (effects) of attention orienting, particularly from a neuroscientific perspective (e.g., LaBerge, 1995; Posner, 1995). Following previous work, a mechanism of attention orienting is defined in this thesis as an integrated system of brain areas that is involved in shifting attention to different locations, or objects, in space. An anatomical area in which neural activity is, to some extent, specific to attention-orienting processes is very likely to be part of such a mechanism (Posner, 1995). By comparison, attentional modulation of activity in brain areas that usually perform sensory, perceptual, or motor computations is defined as a consequence of attention orienting. These latter attentional modulations occur in several cortical areas and are "expressed" as enhancements of activity at the attended location relative to activity at the unattended location (LaBerge, 1995). Three brain areas - the posterior parietal cortex (PPC), superior colliculus, and pulvinar nucleus of the thalamus - have been associated with covert shifts of spatial attention in the visual modality. These three areas are believed to form a coordinated mechanism, often called the 13 posterior attention system (Posner, Petersen, Fox, & Raichle, 1988), for shifting attention within the visual environment. Each area is hypothesized to perform the computations for a single, unique mental operation that, in combination, modulates the neural activity in cortical areas that are involved in the sensory and perceptual processing of visual stimuli. Importantly, these features are shared by other recent accounts of attention orienting and focusing (e.g., LaBerge, 1995). According to Posner and colleagues (for a review, see Posner & Raichle, 1994), the PPC disengages attention from the current focus of attention, the superior colliculus shifts attention to a new location, and the pulvinar nucleus of the thalamus re-engages attention at the new location. Most of the evidence for these hypothetical attentional roles comes from a detailed analysis of the performance deficits following selective brain injury in humans (Posner, Rafal, Choate, & Vaughan, 1985; Posner, Walker, Friedrich, & Rafal, 1987; Rafal, Posner, Freidman, Inhoff, & Bernstein, 1988). Importantly, qualitatively different patterns of deficits are observed following damage to the three brain areas noted above. Patients with unilateral parietal lesions perform in spatial cueing tasks as if they were impaired in their ability to disengage attention from an ipsilesional stimulus (i.e., one that is located in the visual field that is ipsilateral to their lesion) in order to attend to a contralateral stimulus. In comparison, patients with degenerative damage to the superior colliculus perform as if they were impaired in their ability to shift attention, whereas patients with thalamic lesions perform in the same tasks as if they were impaired in their ability to engage attention (cf. Posner & Raichle, 1994). These findings are generally consistent with evidence from single-cell (e.g., Robinson, Bowman, & Kertzman, 1995; Petersen, Robinson, & Keys, 1985) and behavioural (e.g., Rafal, Henik, & Smith, 1991) studies that suggest that 14 neurons in these brain areas have an attentional function. 2.2 Inhibition of return Another phenomenon that has been discovered using the spatial cueing paradigm is called inhibition of return (IOR; Maylor, 1985; Maylor & Hockey, 1985; Posner et al., 1985; Posner & Cohen, 1984; Tassinari, Aglioti, Chelazzi, Marzi, & Berlucchi, 1987). IOR is the demonstration that subjects are slower to respond to a target that is presented at the location of a spatially uninformative cue than when it is presented at a different location. General interest in this effect stems from the belief that it is somehow related to selective attention. One attentional conceptualization of IOR is as follows. Although subjects maintain eye fixation throughout the experiment, the peripheral cue causes a momentary shift of attention to its location, thereby facilitating target processing at the cued location for a short period (Jonides, 1981; Posner & Cohen, 1984). However, subjects quickly return their attention to the centre point of eye fixation because the cue does not accurately predict the location of the target or because special incentives are provided to focus attention at fixation. For example, many researchers present a salient stimulus at fixation immediately after the cue but before the target. This stimulus is called a central reorienting event. Once attention is reoriented away from the cued location, an inhibitory process develops that biases attention from returning to the cued location. Thus, if the target is presented after attention has been returned to fixation, more time is required to shift attention back to the cued location than is required to shift attention to a new location. On this basis, the IOR effect is related to attention in two ways. First, orienting attention to, and then 15 away from, the cued location causes inhibition to develop at the cued location. This issue pertains to the mechanism of IOR. Second, the inhibition at the cued location influences subsequent attentional orienting to the cued location, thereby suppressing perceptual processing there. This issue pertains to the consequence of IOR. 2.2.1 Mechanisms of Inhibition of Return Support for an attentional mechanism of IOR comes primarily from the fact that it occurs when eye fixation is maintained at the centre of the display. In addition, the IOR effect is usually preceded at short (< 200 ms) SO As by a relative facilitation of responses for valid-cue trials, indicating that the cue summoned attention to its location. Moreover, at least some attention-demanding secondary tasks disrupt the facilitatory and inhibitory effects roughly equally (Maylor, 1985). On this basis, several investigators have argued that the IOR effect arises from spatial attention orienting to, or away from, the cued location (e.g., Maylor, 1985; Maylor & Hockey, 1985). One potential problem for this interpretation is that goal-driven shifts of attention in response to central symbolic cues do not produced IOR in fixating subjects (Posner & Cohen, 1984; Rafal, Calebresi, Brennan, & Sciolto, 1989). This raises the possibility that IOR is caused by nonattentional processes. Posner and Cohen (1984) speculated that IOR arises directly from sensory stimulation, whereas others have suggested that it arises from activation (or suppression) of the oculomotor system (Rafal et al., 1989; Tassinari et al., 1987). Interestingly, the IOR effect is observed when subjects move their eyes to the cued location in the absence of peripheral stimulation, for example in response to symbolic cues 16 (Posner et al., 1985; Rafal et al., 1989). Thus, it appears that IOR can arise "endogenously" from the generation of eye movements as well as "exogenously" from peripheral stimulation. Rafal et al. (1989) proposed that preparation of an eye movement is necessary and sufficient to produce IOR in both situations. To test this hypothesis, Rafal et al. (1989, Experiment 4) compared simple RT to visual targets under three symbolic-cue conditions. In the eyes-fixed condition, participants maintained eye fixation throughout the task and pressed a button when the target appeared. In the saccade-execution condition, participants moved their eyes to the cued location and then back to fixation (in response to a central reorienting event) and pressed a button when the target appeared. In the saccade-preparation condition, participants prepared to make a saccade to the cued location but did not execute it. IOR was observed in both saccade conditions but not in the eyes-fixed condition. These results provide some support for the hypothesis that saccadic preparation is involved in generating IOR. An alternative explanation is that suppression of the natural tendency to look toward the location of a peripheral cue causes the IOR effect (Tassinari et al., 1987). This account, like the previous one, is based on the assumption that a peripheral sensory event normally elicits a saccadic eye movement to its location (in other words, it causes an eye movement to be prepared and normally executed). According to Tassinari et al., participants prevent this orienting response by generating a command to suppress orienting toward the location of the cue. This command is presumed to last long enough to affect orienting to a subsequent target stimulus presented at the cued location. On this basis, participants detect targets appearing at or near the cued location more slowly because the oculomotor command that suppresses saccadic eye 17 movements to the cued location also influences manual responding to stimuli in that direction. Importantly, however, both oculomotor explanations assume that direct visual cues cause subjects to prepare saccadic eye movements reflexively and that it is this saccadic preparation, or the need to suppress it, that produces IOR. 2.2.2 Consequences of Inhibition of Return The previous section considered the possible mechanisms that might be involved in producing the IOR effect. A separate question concerns the processes that are inhibited by IOR once it is established (Rafal & Henik, 1994; Reuter-Lorenz, Jha, & Rosenquist, 1996). The IOR effect may reflect an inhibition of the perceptual processes at the cued location or it may simply be a delay in initiating a motor response (either oculomotor or manual). The first proposal is consistent with the characterization of IOR as an attentional effect (i.e., that it influences attentional orienting to the cued location) because covert orienting of attention is also known to influence sensory-perceptual processing. The latter proposal is inconsistent with this characterization. Although the popular interpretation of the IOR effect claims that attentional processes are suppressed at previously cued locations, several findings indicate that IOR may be more closely associated with responding than with attention. For example, uninformative visual cues have no influence on temporal-order judgments at long SOAs (Maylor, 1985; Posner et al., 1985). By comparison, there is evidence for a strong temporal-order bias in favor of the cued location at short SOAs when the cue is uninformative (Maylor, 1985) and at long SOAs when the cue is 18 highly informative (Stelmach & Herdman, 1991). Under these latter conditions, subjects report seeing a stimulus at the cued location before seeing a stimulus in the opposite hemifield even when the two stimuli appear simultaneously. These results are taken as evidence that attention facilitates visual encoding so that a stimulus at the cued location has "prior entry" to the perceptual system (Sternberg & Knoll, 1973). That IOR fails to have the opposite effect on temporal-order judgments suggests that it does not influence sensory-perceptual processes (Maylor, 1985). In contrast to the above findings, some recent observations indicate that IOR can influence perceptual processes under some conditions. First, IOR occurs in tasks requiring speeded temporal-order judgments although it appears to be reduced by the appearance of multiple targets (Gibson & Egeth, 1994). Second, recent electrophysiological evidence indicates that IOR is associated with reduced perceptual processing in occipital cortical areas (McDonald, Ward, & Kiehl, in press). Taken together, these recent observations provide convincing evidence that visual IOR at least partially reflects modulations in perceptual processes. 2.2.3 Inhibition of Return to Nonvisual Stimuli The various explanations for IOR predict that IOR should arise from any external event that summons attention to its spatial location or normally elicits a saccadic eye movement. Although most previous studies on orienting have focused on vision, it is known that auditory and somatic stimuli can summon attention (Butter, Buchtel, & Santucci, 1989; McDonald & Ward, in press; Kilgard & Merzenich, 1995; Spence & Driver) and elicit saccadic eye 19 movements (Jay & Sparks, 1987a, 1990) to their spatial locations. Given such findings, it is reasonable to expect that auditory and somatic stimuli will also produce IOR under some conditions. Because spatial attention appears to influence auditory performance only when spatial relevance is established, the attentional explanation predicts that IOR should occur in audition only when the decision to respond is based on the spatial location of the target (cf. McDonald & Ward, in press). In contrast, the oculomotor explanations predict that IOR should occur in audition whenever subjects make or prepare to make saccadic eye movements to the cued location. Thus, according to these latter accounts, then, IOR might occur in both spatial and nonspatial tasks because unexpected sounds appear to prime the oculomotor system reflexively in both conditions (Jay & Sparks, 1987b, 1990). Despite the above reasons to predict the occurrence of IOR in auditory tasks, none was observed in a recent series of experiments involving location-based choice responses (Spence & Driver, 1994). Drawing parallels with some findings in the visual modality, Spence and Driver (1994, 1997, 1998a) attributed these null results to the use of a choice-response task, speculating that IOR is less reliable in tasks involving choice responses than ones involving simple reactions (cf. Terry, Valdes, & Neill, 1994; but see Pratt, Kingstone, & Khoe, 1997). More recently, IOR has been observed in several experiments involving simple reactions to auditory stimuli (Spence & Driver, 1998a; Tassinari & Berlucchi, 1995; Schmidt, 1996). This pattern of results is exactly opposite to what would be predicted from an attentional account, at least according to the spatial relevance hypothesis. Indeed, several researchers have proposed that auditory IOR is mediated by saccadic preparation rather than by covert shifts of attention (Reuter-Lorenz & Rosenquist, 20 1996; Schmidt, 1996; Tassinari & Berlucchi, 1995). However, an alternative explanation exists for the observation of the inhibitory effect in spatial cueing tasks. Namely, subjects might respond to the target more slowly on valid-cue trials than on invalid-cue trials because of the need to inhibit a manual response to the cue (Harvey, 1980). This explanation is certainly possible given the natural tendency to react in the direction of a peripheral stimulus (Simon, 1990, Simon & Small, 1969). Thus, manual response suppression might have a larger effect on responses to subsequent targets when they appear in the same direction as the cue than when they appear in the opposite direction. This explanation is similar to the oculomotor-suppression explanation for IOR except that it assumes that the. inhibition arises from the need to suppress a manual response rather than from the need to suppress an oculomotor response. Consequently, inhibition arising from manual response suppression is contrasted with IOR on the basis that the two effects are produced by different mechanisms. One way to control for manual response suppression is to require subjects to respond to every stimulus in a series, thereby eliminating the necessity of withholding responses to the cues (Maylor & Hockey, 1985). This technique is called the target-target method. In such experiments, IOR is measured by comparing response latencies according to the spatial relation between the locations of successive targets. For example, subjects respond more slowly to a visual target when it appears at the location of a preceding visual target than when the two targets appear at different locations (Maylor & Hockey, 1985; Terry et al., 1994). This inhibitory effect between visual targets presumably reflects "genuine" IOR because it cannot be related to manual-21 response inhibition. By comparison, no inhibitory effect occurs when subjects make simple reactions to successive auditory targets in the absence of intervening cues, indicating that the inhibitory effect found in auditory simple RT experiments does reflect manual-response suppression (McDonald & Ward, in press; Spence & Driver, 1998a). Two recent auditory studies reported IOR that cannot arise from manual response suppression. In one, IOR occurred between successive auditory targets when they were randomly intermixed with visual targets (Spence & Driver, 1998a). Because the conditions used in this experiment were identical to those used in a purely auditory experiment in which no IOR was apparent except for the inclusion of visual trials, Spence and Driver (1998a) argued that target-modality uncertainty was a critical factor for determining whether IOR occurs in audition. However, in another study, IOR was observed in several purely auditory experiments that did not include visual trials (McDonald & Ward, in press). In this latter study, IOR occurred between auditory cues and targets and between successive auditory targets in the absence of eye movements when the decision to respond was based on spatial, but not nonspatial, properties of the target. Because the sensory and oculomotor explanations predict the occurrence of IOR regardless of the particular task, these findings are most consistent with the attentional explanation for IOR. 2.3 Modality Specificity of Spatial Attention Orienting and IOR The observation of similar spatial cue effects in different sensory modalities has led to the consideration of whether spatial attention is controlled by supramodal brain mechanisms rather 22 than by modality-specific ones (Buchtel & Butter, 1988; Butter, et al., 1989; Farah, Wong, Monheit, & Morrow, 1989; Hillyard, Simpson, Woods, Van Voorhis, & Mtinte, 1984; Mondor & Amirault, 1998; Spence & Driver, 1997; Spence, Nicholls, Gillespie, & Driver, 1998; Ward, 1994; Ward, McDonald & Golestani, 1998a). Addressing this issue is critically important in developing a complete understanding of spatially selective attention. Figure 1 illustrates three different types of relationships that could exist between the visual, auditory, and somatic spatial attention (and IOR) mechanisms. In the top diagram (Figure 1 A), spatial attention orienting in the various modalities involves entirely separate brain mechanisms that operate independently on representations of modality-specific space (Mondor & Amirault, 1998; Posner, 1978). According to this hypothesis, called the separate-mechanisms hypothesis, attention orients to the location of an event in one modality without evoking a corresponding shift in any other modality. For example, based on previous physiological and neurological data, Mondor and Zatorre (1995) suggested that the superior colliculus might be involved in visual, but not auditory, spatial attention orienting, whereas the inferior colliculus might be involved in auditory, but not visual, spatial attention orienting, thereby constituting two separate attention orienting mechanisms. Another possibility, illustrated in Figure IC, is that spatial attention orienting in different modalities involves the same brain mechanism(s) that operate on representations of multimodal space (Buchtel & Butter, 1988; Butter et al., 1989; Farah et al., 1989). According to this hypothesis, called the supramodal-mechanisms hypothesis, the mechanisms for spatial attention are completely shared among the various sensory modalities 23 AUDITION VISION TOUCH ( Attention >v X i T i e c h a n i s m / ( Attention >v ^ m e c h a n i s r r i / ( Attention ^ \ ^ m e c h a n i s r r i / Figure 1. Schematic diagrams of the possible relationships between the visual, auditory, and somatic spatial attention mechanisms. (A) Three separate visual, auditory, and somatic spatial attention mechanisms. (B) Three interacting visual, auditory, and somatic spatial attention mechanisms. (C) A single, supramodal spatial attention mechanism. 24 in order to selectively process multimodal events and to coordinate attention across different modalities. One possibility is that the brain structures that have been implicated in visual attention orienting on the basis of neurophysiological research - the P P C , superior colliculus, and pulvinar nucleus - actually control attention orienting in several modalities. In any event, a strong supramodal-mechanisms hypothesis would predict that the spatial attention mechanism should operate in the same manner regardless of which modalities are involved. Consequently, orienting attention to a specific location would have the same effect on the processing of a stimulus occurring there whether attention was initially summoned to that location by a stimulus in the same modality or in a different modality. A slightly weaker supramodal hypothesis would allow for some different operations to occur when multiple modalities are involved because additional time would be required to translate different sensory signals into a common coordinate frame. A n intermediate position, illustrated in Figure IB , is that spatial attention orienting involves modality-specific mechanisms that interact at least in some conditions. According to this position, called the linked-mechanisms hypothesis, modality-specific mechanisms of attention are "linked" such that shifts of attention in one modality tend to produce shifts in other modalities (Farah et al., 1989; Spence & Driver, 1997). It is unclear how the links between modality-specific mechanisms would be established however. Determining whether the mechanisms for spatial attention are separate, linked, or supramodal depends in part on the existence of intersensory interactions. For present purposes, an intersensory interaction is defined as an effect of information presented in one modality on the processing of information presented in a different modality (cf. Welch & Warren, 1986). Such 25 intersensory interactions are known to exist in the control of overt orienting behaviours, particularly with respect to stimulus localization (for a review, see Stein & Meredith, 1993). For example, the presence of a visual stimulus can strongly influence the localization of auditory and somatic stimuli. These effects arise because neural signals in different sensory pathways may be integrated in the brain, either by converging upon the same individual neurons (e.g., Stein & Meredith, 1993) or by firing in temporal synchrony (Massaro, 1998). Importantly, the existence of intersensory interactions in the control of stimulus-driven attention and IOR would provide strong evidence that the mechanisms for those phenomena cannot be entirely modality-specific. Depending on the specific patterns of interactions that occur, the mechanisms might be separable but linked or entirely supramodal. However, if no intersensory interactions exist, then the mechanisms cannot be supramodal or even tightly linked. 2.3.1 Intersensory Interactions in Perception and Performance Intersensory interactions are known to influence the detectability of suprathreshold stimuli. Normally, simple manual RT is about 30-40 ms faster to a sound than to a light, even when the stimulus intensities are matched psychophysical^ (Goldstone, 1968). This auditory superiority presumably reflects the fact that the processing that is done by the retina is slower than the processing that is done by the inner ear. However, it is known that when a light precedes a sound by the difference in detectability so that their input to more central brain areas is simultaneous, the reaction to the combined stimulus is often faster than the reactions to either stimulus presented alone (Hershenson, 1962; Morell, 1968). This multisensory enhancement is 26 believed to reflect either a summation of individual sensory signals in the brain or a general warning effect of one stimulus on the other. It is also known that the auditory superiority in simple RT tasks is not observed in choice RT tasks when the target modality is made uncertain by randomly intermixing auditory and visual signals (Colavita, 1974). If both auditory and visual stimuli are presented simultaneously on a small portion of trials, observers respond much more often by pressing the visual response key than by pressing the auditory response key. Thus, whereas auditory responses are normally quicker than visual responses, there is a clear visual superiority over audition when the modality of the target is uncertain. This visual superiority is believed to reflect intersensory interactions resulting from changes in attentional set (Colavita & Weisberg, 1979; Egeth & Sager, 1977). Even more dramatic intersensory interactions occur in tasks involving spatial localization. Although vision, audition, proprioception, and touch provide information concerning the spatial position of external objects, vision is clearly the dominant spatial modality in humans. Such dominance is clearly reflected in the resolution acuity of the various spatial modalities. For example, the resolution acuity of vision is about 1 min of arc at the fovea (e.g., Howard, 1982), whereas the minimum audible angle for sounds presented directly in front of the listener (0° azimuth) is about 1-2° (e.g., Mills, 1958). Visual dominance of spatial judgments is also reflected in the timing and metrics of localization responses, which are both faster and more accurate for visual stimuli than for nonvisual stimuli (e.g., Simpson, 1979). It is no surprise, then, that intersensory interactions involving spatial localization are generally dominated by vision. For example, the mere presence of a textured visual field improves the accuracy of 27 auditory localization, even for sounds appearing outside of the observer's field of vision (Shelton & Searle, 1980). This intersensory effect partially reflects the fact that vision provides a "frame of reference" from which auditory localization judgments can be made (Fisher, 1964). However, the effect does not occur when eye movements are prevented, nor do eye movements made in the dark facilitate auditory localization (Piatt & Warren, 1972). These findings suggest that auditory localization judgments are made with joint reference to visual spatial representations and oculomotor activity. Intersensory interactions in spatial localization are often examined in the laboratory by placing normally congruent sources of sensory information in conflict with one another. Such discrepancies sometimes occur outside of the laboratory, such as when the voice of a movie actress comes from a speaker to one side of the theater or when the apparent voice of a dummy comes from a nearby ventriloquist who speaks without lip movements. These phenomena show quite dramatically that when visual and auditory sensory modalities provide spatially discrepant information, observers tend to hear the sound as emanating from the source of visual input. For obvious reasons, the effect of visual spatial information on auditory localization is often called the "ventriloquism effect" (Howard & Templeton, 1966, p. 361). More generally, such effects are called intersensory biases (Pick, Warren, & Hay, 1969). One of the first empirical demonstrations of intersensory bias was the finding that displacement of the visual field by a prism causes one's felt hand position to be strongly biased toward the displaced visual image (Hay, Pick & Ikeda, 1965). This finding led other researchers to look for similar effects between different pairs of the spatial senses. The results of these studies suggest a hierarchical dominance 28 of intersensory bias: vision biases both audition and proprioception, proprioception biases audition but not vision, and audition biases neither vision nor proprioception (e.g., Pick et al., 1969). 2.3.2 Multimodal Convergence in the Brain Electrophysiological studies have identified several brain areas that could form the basis of the intersensory interactions discussed in the preceding section. These areas are considered to be multimodal because (1) they contain neurons that are responsive to sensory signals in different modalities, (2) they receive input from several modality-specific sensory areas, and (3) lesions to these areas produce deficits involving more than one modality. Brain areas that exhibit these properties have been identified in the cat (Stein & Meredith, 1993), guinea pig (King & Palmer, 1985), and monkey (Streicher & Ettlinger, 1987), and recent neuroimaging studies have revealed homologous areas in humans (Banati et al., 1997). These areas include the deep layers of the superior colliculus, parts of the PPC, and the medial pulvinar nucleus of the thalamus. As noted earlier, these brain areas are believed to be part of a mechanism for shifting attention to different locations of external space. The implication of their multimodal properties, of course, is that spatial attention might shift to locations within multimodal space rather than to locations within modality-specific space. Neurons with multimodal properties have been well documented in the superior colliculus of the cat (for a review, see Stein & Meredith, 1993). This midbrain structure contains several layers that are usually categorized as either superficial or deep based on their 29 physiological and anatomical characteristics. Input to the deep layers, which contain many multimodal neurons, comes from several cortical and subcortical structures (for a review, see Huerta & Halting, 1984). Substantial descending afferents come from two multimodal areas of the cortex. One area, called the lateral suprasylvian area (LS), surrounds the suprasylvian sulcus. The other surrounds the anterior extrasylvian sulcus (AES) and contains separate visual (AEV), auditory (Field AES), and somatosensory (SIV) areas, as well as multimodal areas (Wallace, Meredith, & Stein, 1993). Other cortical inputs to the multimodal superior colliculus layers originate in the PPC and the frontal eye fields, at least in primates. The first pathway is believed to be involved in visual attention orienting and the second is believed to be involved in eye movements (Sparks & Hartwich-Young, 1989). Indeed, many multimodal neurons in the superior colliculus project to the brainstem nuclei that control orientation of the eyes and head (Meredith, Wallace, & Stein, 1992). Others project to thalamic nuclei, such as the pulvinar nucleus and, from there, to the PPC and other sensory cortical areas. As a result of the multisensory convergence noted above, sensory stimuli in different modalities can evoke responses from the same individual deep-layer neurons (Meredith & Stein, 1986b). Some cells respond optimally to concurrent stimuli in two spatial modalities (e.g., vision and audition, vision and touch) and others respond to concurrent stimuli in three modalities. The visual, auditory, and somatosensory receptive fields of these neurons are organized similarly, thereby aligning the representations of modality-specific space. This common organization suggests that the neurons in the deep-layer superior colliculus represent multimodal space. Additionally, whereas some neurons simply receive convergent inputs from 30 different modalities, many others integrate these inputs. This gives rise to sensory experiences that are dramatically different from the simple sum of the individual signals. In general, stimuli that appear to have "common causality" (Stein & Meredith, 1993, p. 123) enhance the responsiveness of these neurons, whereas stimuli that do not appear to have common causality depress their responsiveness. Consider, for example, a deep-layer neuron that is responsive to both visual and auditory stimuli. Under some conditions, such a neuron might respond more vigorously to a combined visual-auditory stimulus than to either individual stimulus. In other conditions, the same neuron might respond less vigorously to a combined visual-auditory stimulus than to an effective individual stimulus. These changes are often much larger than the simple sum of the unimodal responses and are determined by the spatial and temporal relations between the individual stimuli (Meredith, Nemitz, & Stein, 1987; Meredith & Stein, 1986a). In particular, combinations of stimuli appearing at the same or nearby locations produce responses that are enhanced far above the best unimodal response (response enhancement), whereas combinations of stimuli appearing at different locations produce responses that are depressed far below the best unimodal response (response depression). There is a fairly long temporal window within which such effects can occur, so that information reaching the various sensory systems at different times can still interact in the brain. Importantly, most multimodal neurons in the deep layers of the superior colliculus, and particularly those that exhibit response enhancement and depression, project to the motor systems in the brainstem and spinal cord that control eye, ear, and head orientations to sensory 31 stimuli (Stein & Meredith, 1985; Meredith et al., 1992; Wallace et al, 1993). It is therefore quite reasonable to expect that the integrated multisensory signals influence overt behaviour in meaningful and predictable ways. Consistent with these expectations are observations that superior-colliculus-mediated behaviours, including orienting and localization, depend upon the same principles that govern multisensory integration at the neuronal level (Stein, Meredith, Huneycutt, & McDade, 1989; Stein, Huneycutt, & Meredith, 1988). For example, a cat's overt orienting response to a light can be enhanced by a spatially concordant sound and depressed by a spatially discordant sound. Thus, through the integration processes in the superior colliculus, external stimuli that are presumed to be related (i.e., have common causality) facilitate overt orienting behaviours, whereas those that are presumed to be unrelated inhibit overt orienting behaviours. 2.3.3 Crossmodal Interactions in Spatial Attention One way to examine the modality specificity of spatial attention mechanisms is to observe the effects of spatial cues presented in one modality on the response latencies to targets presented in a different modality. Posner and colleagues (Posner, 1978; Posner, Davidson, & Nissen, 1976a; Posner, Nissen, & Ogden, 1978) were the first to use this crossmodal cueing technique, albeit with only limited success. They examined the effects of a symbolic visual cue, an arrow, on RTs to visual, auditory, and somatic targets. Consistent with other early findings, strong symbolic-cue effects were observed in tasks involving detection or discrimination of visual targets. In contrast, the same visual cue failed to influence response times to auditory and 32 somatic targets, except when subjects were required to discriminate between two different somatic targets (Posner et al., 1976a). These data are suggestive of a separate visual spatial attention mechanism. However, it is difficult to make strong inferences about the modality specificity of spatial attention on the basis of Posner et al.'s (1976a) experiments. One complication is the general difficulty in obtaining crossmodal spatial cue effects in nonspatial tasks. In particular, given that the auditory system is primarily arranged according to spectral frequency, the null visual-auditory effect might simply reflect the insensitivity of simple reaction and identity-based choice response measures to attention. Although these are reliable measures for examining spatial attention in intramodal visual tasks, they are not reliable for examining such effects in intramodal auditory tasks. In general, spatial cue effects are measurable only in auditory tasks involving location-based decisions. Presumably, this is because nonspatial auditory decisions can be made without reference to location-sensitive, and thus spatial-attention-sensitive, auditory representations (McDonald & Ward, in press). Extending this reasoning to crossmodal situations, one might expect to find evidence for visual-auditory interactions in spatial attention only in tasks involving spatial decisions. More recent studies of crossmodal attention have addressed the modality specificity of spatial attention mechanisms by using direct spatial cues rather than symbolic cues. In one such study, Farah et al. (1989) examined the effects of spatially uninformative auditory and visual cues on simple RT to visual targets in patients with damage to the right parietal lobe. Their results showed that RTs were greatly elevated for invalidly cued visual targets appearing on the contralesional side of fixation regardless of the modality of the cue. As mentioned previously, 33 this performance pattern has been interpreted in terms of a specific impairment in the ability to disengage attention from an ipsilesional stimulus in order to shift it to a contralateral stimulus (Posner et al., 1987). Thus, Farah et al.'s (1989) results indicate that patients were impaired in their ability to disengage attention from an auditory cue in order to shift it to a contralesional visual target. Farah et al. (1989) postulated the existence of a supramodal spatial attention mechanism that is engaged by both auditory and visual stimuli because a crossmodal impairment appears to falsify the separate mechanisms account described earlier. Interestingly, they also argued against a separable-but-linked mechanisms explanation, whereby activation of the auditory attention mechanism by the auditory cue leads to activation of the visual attention mechanism, because the auditory cue apparently activated the visual attention mechanism automatically and therefore the visual attention system did not appear to be truly modality-specific. Intersensory interactions in spatial attention have also been observed in several recent direct cueing experiments involving neurologically normal individuals (Buchtel & Butter, 1988; Butter et al., 1989; Mondor & Amirault, 1998; Spence & Driver, 1996, 1997; Spence et al., 1998; Ward, 1994; Ward et al., 1998a, 1998b). These studies have revealed symmetrical crossmodal cue effects between visual and somatic stimuli (Butter et al., 1989; Spence et al., 1998) and between somatic and auditory stimuli (Spence et al., 1998). For example, Butter et al. (1989) found that spatially informative somatic cues can influence simple RT to visual targets, and that spatially informative visual cues can influence simple RT to somatic targets. These crossmodal cue effects occurred across a wide range of cue-target SOAs, including very short ones (< 150 34 ms). Although it is possible that these effects arise from a combination of stimulus-driven and goal-driven effects, given the use of spatially informative direct cues, Spence et al. (1998) found the same pattern of results in a subsequent series of experiments using spatially uninformative cues. Thus, there is converging evidence against the separate mechanisms hypothesis for stimulus-driven spatial attention orienting in the somatic and visual modalities. The symmetrical interactions between visual and somatic stimuli and between auditory and somatic stimuli provide some support for the supramodal mechanisms hypothesis, which proposes that the same mechanism is involved in stimulus-driven spatial attention orienting in all spatial modalities. However, the interactions between vision and audition observed so far do not lend support for this position. In one recent target identification study, spatially uninformative auditory and visual cues produced strong cueing effects on intramodal trials but not on crossmodal trials (Mondor & Amirault, 1998). Making the cue spatially informative did produce small but significant crossmodal cue effects in this study and, on this basis, Mondor and Amirault (1998) concluded that such effects depend on the engagement of goal-driven spatial attention mechanisms. Under different experimental conditions, others have shown that there are asymmetrical cue effects between auditory and visual stimuli. Using spatially informative cues, Buchtel and Butter (1988) found that auditory cues influenced simple RTs to visual targets but that visual cues did not influence RTs to auditory targets. The same asymmetry was observed in a later uninformative spatial cueing study using the orthogonal-cueing paradigm described earlier (Spence & Driver, 1997). In one experiment, uninformative auditory cues were followed by 35 either auditory or visual targets. Choice responses to both the auditory and visual targets were faster when the cue and target appeared on the same side of fixation than when they appeared on opposite sides. In other experiments, uninformative visual cues were followed by either auditory or visual targets. Consistent with Buchtel and Butter's (1988) prior observations, responses to the visual target were faster on the side of the cue, but responses to the auditory target were unaffected by the location of the visual cue. Spence and Driver (1997) concluded from these findings that there are asymmetrical links between audition and vision such that stimulus-driven spatial attention orienting in audition causes attention orienting in vision, whereas the reverse trend does not occur. There are some indications that the opposite asymmetry can occur under different experimental conditions (Ward, 1994; Ward, McDonald, & Lin, 1998b). In Ward's experiments, subjects made choice responses based on the location of an auditory target in one experiment and a visual target in another experiment. In each task, the target was preceded by a spatially uninformative visual or auditory cue, by both cues, or by no cues. Thus, in direct contrast to most of Spence and Driver's (1997) experiments, the modality of the target was certain, and the modality of the cue was uncertain (and very complex) in Ward's experiments. The major crossmodal findings were that the visual cue influenced response times to the auditory target, facilitating them at short SOAs and inhibiting them at longer SOAs, but that the auditory cue failed to influence response times to the visual target. Thus, the data reported by Ward (1994) showed a clear visual-on-auditory effect and the opposite asymmetry compared to the one observed by Spence and Driver (1997). These findings are consistent with the asymmetrical 36 intersensory interactions involving spatial localization, outlined in Section 1.3.1, which are generally dominated by vision (for a review, see Posner, Nissen, & Klein, 1976b). Ward (1994; Ward et al., 1998a, 1998b) concluded that spatial cue effects between vision and audition are dominated by vision under his particular experimental conditions because visual localization is so much more precise than auditory localization. The known crossmodal interactions in stimulus-driven spatial attention orienting are summarized in Figure 2. The possible lack of crossmodal interaction between visual and auditory stimuli, either in one direction (Buchtel & Butter, 1988; Spence & Driver, 1997; Ward, 1994) or in both directions (Mondor & Amirault, 1998), is an important finding in terms of the modality specificity of the underlying mechanisms involved in spatial attention because it would argue against the strong supramodal mechanism hypothesis. Nevertheless, because of the inconsistencies noted above, the separate, linked, and supramodal hypotheses all remain viable. 2.3.4 Crossmodal Interactions in Inhibition of Return The possibility of IOR occurring crossmodally has been examined in some recent investigations (Reuter-Lorenz et al., 1996; Reuter-Lorenz & Rosenquist, 1996; Spence & Driver, 1998a, 1998b; Tassinari & Campara, 1996; Ward, 1994; Ward et al., 1998a, 1998b). In most cases, simple RT has been used to measure crossmodal IOR because it is known to be a sensitive measure of visual IOR. Under various conditions, IOR has been observed in simple RT tasks involving visual cues and somatic targets, somatic cues and visual targets, visual cues and auditory targets, and auditory cues and visual targets (Reuter-Lorenz et al., 1996; Reuter Lorenz 37 T A R G E T MODALITY VISION AUDITION TOUCH Figure 2. The crossmodal interactions in stimulus-driven spatial attention that have been demonstrated to date. The visual-on-auditory and auditory-on-visual cue effects are absent from the illustration because there is conflicting data about their occurrence. 38 & Rosenquist; 1996; Spence & Driver, 1998b; Tassinari & Campara, 1996). In addition, symmetrical IOR effects have been observed in simple RT tasks between consecutive auditory and visual targets in the absence of intervening cues. These symmetrical crossmodal effects indicate that IOR might be controlled by supramodal, or at least tightly linked, mechanisms. One possibility, mentioned earlier, is that IOR arises from activation or suppression of the oculomotor system. Such an oculomotor mechanism would be best construed as a supramodal one because the oculomotor system is engaged by sensory signals in several modalities. However, two important issues must be addressed before any convincing argument can be made about the supramodal properties of IOR. One issue concerns the appropriateness of simple RT for measurement of crossmodal IOR. Although location information is inherent in the visual and somatic modalities, it must be derived by specialized neurons in the auditory modality. Indeed, both the facilitatory and inhibitory effects of spatial cueing within audition depend upon the active involvement of the specialized, location-sensitive auditory neurons in performing the experimental task (McDonald & Ward, in press). One might reasonably expect that audio-visual and auditory-somatic IOR might similarly depend on the explicit involvement of location-sensitive neurons, which is clearly not provided by most simple RT tasks. At first glance, the observation of IOR between auditory and visual stimuli appears to disconfirm this prediction. However, previous studies demonstrate that audio-visual IOR does not occur in simple RT tasks except under certain conditions. For example, auditory cues generate IOR in visual simple RT tasks when eye movements are made but not when eye movements are prevented (Reuter-Lorenz & Rosenquist, 1996). Audio-visual IOR has been observed for simple 39 RT in the known absence of eye movements but only when auditory and visual targets are randomly intermixed in a target-target design (Spence & Driver, 1998a; Spence et al., 1998). This procedure might establish spatial relevance indirectly by necessarily involving visual spatial representations. A second issue concerns the existence of some alternative explanations for the inhibitory effects in simple RT tasks. The target-target experiments mentioned above discount one such explanation, involving inhibition of a response to the cue. Unfortunately, another methodological concern can be raised about these and all other simple RT studies of IOR. Specifically, it is possible that the inhibitory spatial cueing effects on simple RT reflect response criterion shifts rather than genuine IOR effects (Shaw, 1980). Indeed, it is well known that facilitatory cue effects can occur because subjects reduce the amount of evidence required to respond to targets presented at the cued location (i.e., they lower their response criterion). By the very same argument, inhibitory cue effects might occur at long SOAs because subjects increase the amount of evidence required to respond to targets presented at the cued location. Thus, different techniques are required to adequately address the modality specificity of IOR. Several techniques have been developed to address the potential confound of response criterion effects. Signal detection measures have been used with some success to study visual spatial attention (e.g., Downing, 1988; Luck, Hillyard, Mouloua, Woldorff, Clark, & Hawkins, 1994). Such measures are not easily implemented in auditory and crossmodal tasks because spatial attention has little or no effect on the detection of sounds. Another way to address response criterion effects is simply to require a choice response so that a criterion shift would 40 lead to a speed-accuracy trade-off (Klein, Brennan, & Gilani, 1987; Spence & Driver, 1994, 1997). For example, a lower criterion in a given condition would lead to faster responses and more errors because less evidence is actually being used to generate the response. Spence and Driver (1994, 1997) used this strategy with some success in their auditory and crossmodal experiments by requiring subjects to discriminate between two different target elevations. However, although there was no indication of any criterion shifts in these experiments, there was also no evidence for IOR occurring intramodally or crossmodally. The absence of IOR between auditory cues and targets and between auditory cues and visual targets is particularly surprising because there were significant facilitatory cue effects at short SOAs in both conditions. In addition to these null findings, several location-based choice RT experiments by Ward (1994; Ward et al., 1998b) failed to observe IOR in any crossmodal condition, including ones that produced large facilitatory effects at short SOAs. Given these null findings, one might conclude that genuine IOR does not occur when the cue and target are in different modalities. This would imply that modality-specific mechanisms, such as purely sensory ones, could cause the IOR effect. 3 AIMS OF PRESENT STUDY Given the inconsistencies of the previous crossmodal experiments, the present study sought to clarify the modality specificity of stimulus-driven spatial attention and IOR by examining the crossmodal interactions between visual and auditory stimuli. One major goal was to further investigate whether symmetrical interactions occur between vision and audition in the 41 control of stimulus-driven spatial attention orienting. The experiments reported in this thesis each used a variant of the implicit spatial discrimination technique that was effective at ascertaining facilitatory and inhibitory spatial cue effects within audition (McDonald & Ward, in press). This technique (1) ensures the involvement of location-sensitive auditory neurons by making the spatial location of the target relevant to the participant's task, (2) eliminates the possibility of response priming by the cue by requiring the same response on valid-cue and invalid-cue trials, and (3) presents cue and target stimuli at the same location, thereby allowing multimodal neurons to respond maximally. In addition, eye position was monitored for some subjects in each experiment in order to control for the possibility that overt orienting contributed to the observed cue effects. Thus, in the experiments reported here, any observed crossmodal spatial cue effects could be more unambiguously attributed to the involvement of covert stimulus-driven spatial attention mechanisms. A second goal was to examine crossmodal IOR in a location-based choice RT task. One advantage of such a task was that it allowed for the assessment of possible criterion shifts by examining the trends in the error data. Specifically, a reverse trend in the error data would leave an observed cue effect open to criterion-based explanations, whereas a consistent trend (or no trend) would discount such explanations. Because there is no similar way to rule out criterion-based explanations for cue effects obtained in simple RT tasks, none of the previous demonstrations of crossmodal IOR can be characterized as being definitive. Another advantage of a location-based choice RT task arises from the fact that uninformative spatial cues typically do not influence nonspatial judgments in auditory (McDonald & Ward, in press; Spence & 42 Driver, 1994) and audio-visual (Ward et al., 1998a) situations. Again, these null effects probably occur because nonspatial auditory judgments can be based entirely on nonspatial auditory representations. Consistent with this reasoning, one might also expect to find larger audio-visual IOR in location-based tasks that ensure the involvement of spatial representations. A final advantage of such a task is that it affords the opportunity to test all of the various explanations for IOR. In contrast, simple RT experiments fail to address the attentional account of IOR because spatial attention orienting typically fails to occur in simple RT tasks. Therefore, it is not surprising that several researchers have speculated on the basis of these studies that IOR arises from activation or suppression of the oculomotor system (Reuter-Lorenz & Rosenquist, 1996; Spence & Driver, 1998a, 1998b; Tassinari & Campara, 1996). Although an oculomotor mechanism of IOR is possible under some conditions, an attentional mechanism is possible under others (for a review, see McDonald, 1998). Accordingly, the present study also sought to clarify the possible mechanism(s) of IOR by testing specific predictions stemming from the oculomotor and attentional explanations. A third, and closely related, goal of the present study was to determine some of the situational factors that can influence the crossmodal interactions in spatial attention and IOR. This was done to resolve some of the inconsistencies of previous crossmodal investigations and to provide more convincing support for an attentional interpretation of the crossmodal cue effects. The behavioural experiments reported here examined whether making the modality of the target stimulus more or less certain would influence the crossmodal cue effects. This particular manipulation was chosen partly because previous studies indicated that target modality 43 uncertainty is an important factor for intramodal spatial attention effects (Posner et al., 1978) and IOR (Spence & Driver, 1998). A fourth goal of the present study was to examine the neural consequences of crossmodal spatial attention in order to distinguish between the various hypothesis concerning the modality specificity of stimulus-driven spatial attention orienting. Importantly, the supramodal-mechanism hypothesis predicts that auditory and visual spatial cues should produce effects at similar stages of auditory and visual information processing, whereas the linked-mechanisms hypothesis does not make this prediction. This critical prediction was tested in Experiments 5 and 6 by recording event-related brain potentials from subjects while they performed in crossmodal spatial cueing tasks. The event-related potential technique is described later in sections (Section 9 and 11). 4 G E N E R A L M E T H O D S Following the appearance of a spatial cue, subjects made speeded manual responses to targets appearing from the left or right side of fixation (GO trials) but not to targets appearing at fixation (NO-GO trials). The cue was always spatially uninformative so as to minimize the influence of goal-driven attention mechanisms. Except where noted, the cue preceded the target by 100, 500, or 900 ms on any given trial. In the first two experiments, the cue and target stimuli appeared in different modalities on every trial. Visual cues preceded auditory targets in Experiment 1, and auditory cues preceded visual targets in Experiment 2. In these experiments, subjects always knew which modality was relevant and which was irrelevant because the cue and target modalities were both fixed. This was not the case in the next two experiments, where the 44 modality of the target was made uncertain by randomly intermixing auditory and visual targets. In Experiment 3, visual cues preceded visual targets on half of the trials (intramodal trials) and auditory targets on the other half (crossmodal trials). In Experiment 4, auditory cues preceded randomly intermixed auditory (intramodal trials) and visual (crossmodal trials) targets. This design permitted the evaluation of several important predictions regarding the modality specificity of the mechanisms controlling spatial attention orienting and IOR. The first four experiments each consisted of two separate conditions. In one condition, subjects were told to look directly at the fixation LED and to avoid making eye movements during the trials. However, eye position was not actually monitored in these subjects. In the other condition, eye position was monitored by recordings of the horizontal electrooculogram (EOG). This was done to ensure that changes in eye position could not account for any cue effects that were observed. The last two experiments extended the critical findings of Experiments 1 and 2 by recording event-related potentials in similar crossmodal spatial cueing tasks. Visual cues preceded auditory targets in Experiment 5, whereas auditory cues preceded visual targets in Experiment 6. The electrophysiological methods for these experiments will be described in Section 10. 4.1 Stimuli and Apparatus All of the experiments were conducted in a darkened, sound-attenuated chamber (183 cm x 193 cm x 197 cm) with a background sound level of 35 dB SPL. The chamber contained an 45 adjustable chin-rest and a response box placed on a small table. An array of three speakers was aligned horizontally across the front wall of the chamber. The central speaker was positioned 100 cm directly in front of the chin-rest, which was used to minimize head movements and reduce muscle tension. The outer speakers were positioned 25° to the left and right of the central speaker. Green light-emitting diode (LEDs; each 0.3 °) were fixed to the centre of each speaker to serve as location markers. The central LED also served as a fixation point. The auditory cue and target stimuli were generated, amplified, and gated by a custom-built, digitally controlled sound generator. The auditory cue consisted of two 30-ms broad-band noise bursts separated by a 10-ms silent interval and measured 70 dB (SPL) from the subject's ear position. These cues were designed to be highly localizable in space. The auditory target was a 1000-Hz pure tone presented for 50 ms at 75 dB (SPL). The visual cue was a 70-ms flash of two green LEDs that were aligned horizontally around one of the three location markers and spaced 3.5° apart. The visual target was a 50-ms flash of two red LEDs that were aligned vertically around one of the location markers and spaced 3.50 apart. Subjects responded to target stimuli by a pressing a button on the response box with their right index finger. Responses times were measured in milliseconds by an independent hardware timer on a custom input-output board. Stimulus presentation, timing, and data acquisition were controlled by a Hewlett Packard 486-66 microcomputer running custom software. 4.2 Procedure and Design The three location markers were illuminated throughout each experimental session. 46 Subjects were instructed to look directly at the central LED at all times during a block of trials. Each trial began with a brief offset of the fixation LED. After a 550-ms delay, a cue stimulus was presented with equal probability from one of the three stimulus locations. A target was then presented following a variable SOA, again with equal probability from one of the three stimulus locations. Subjects were told to respond as quickly as possible to peripheral targets (GO trials) but to withhold their response to central targets (NO-GO trials). Eight hundred ms after a correct response, the fixation LED blinked again to signal the start of the next trial. A 500-ms error tone sounded if the subject made a response on a NO-GO trial or if the subject did not make a response within 1500 ms on a GO trial. The intertrial interval was 2000 ms for all trials. Subjects were informed that the probability of the target occurring at the cued location was equal to the probability of the target occurring at each of the other two locations. Trials in which a peripheral target was preceded by same-location or opposite-location cues are referred herein to as valid-cue and invalid-cue trials, respectively. Trials in which a peripheral target was preceded by a centre cue are called centre-cue trials. The event sequences for these trials are shown in Figure 3. 4.3 Eye Movement Monitoring Eye movements were monitored in Experiments IB, 2B, 3B, 4B, 5 and 6 by recording the horizontal EOG using tin electrodes placed 1 cm lateral to the left and right outer canthi. The EOG activity was amplified with a bandpass of 0.1 - 30 Hz and continuously digitized at a rate of 128 Hz. Electrode impedance was kept below 5 kQ. Eye position was calibrated at the 47 beginning of each test session by having subjects make several saccades to the left and right speaker locations. Trials contaminated by eye movements (> 2.5°) were discarded prior to analysis. In addition, the EOG was also averaged during the cue-target interval to determine whether or not subjects made eye movements to the cued location. Participants were disqualified if their average EOG deviation over the cue-target interval exceeded ±2 uV (also see Section 10). 4.4 Data Analysis In all the experiments, the first block of trials was treated as practice and was not analyzed. Response times less than 100 ms or greater than 1000 ms were also discarded. Trials on which no response was made to a peripheral target were treated as errors. Median RTs were calculated separately from the remaining trials for each subject in each validity x SOA condition. Statistical analyses were performed on the resulting data by using repeated measures analyses of variances (ANOVAs) with validity (valid and invalid) and SOA (100, 500, and 900; except where noted) as within-subject factors. Central-cue conditions were not analyzed because of the difficulty of interpreting "neutral" baseline conditions (Jonides & Mack, 1984), particularly in studies using direct cues (Wright, Richard, & McDonald, 1994). Huynh-Feldt-corrected degrees of freedom were used to determine probability values for all factors with more than two levels when the sphericity assumption was violated. Following the initial analyses, the RT and accuracy data were pooled across all subjects with eye-position monitoring as a between-subjects variable. Specific hypotheses were tested by comparing the valid-cue and invalid-cue conditions at various SOAs using the Bonferroni inequality to control the familywise error rate (set at 0.05 48 A o • ® • o 44 SPEAKER O TARGETLEDS ® FIXATION LED • CUELEDS Figure 3. (A) Stimulus apparatus used in all of the experiments. (B) Sequence of events on a single trials in the experiments. An auditory cue - visual target trials is illustrated. Note that the cue-target stimulus onset asynchrony is obtained by adding the cue duration with the variable time interval that follows. 49 unless otherwise noted). To reduce the possibility of making a type I error, the number of j comparisons was limited to two (Experiments 3-6) or three (Experiments 1 and 2). However, one-tailed tests were done when appropriate (e.g., to test for a facilitatory cue effect at the 100-ms SOA). Finally, confidence intervals were also calculated for the invalid-valid RT differences using the three-way MSe term. 5 EXPERIMENTS 1A AND IB Experiments 1A and IB examined whether the presentation of a spatially uninformative auditory cue would influence RTs to a subsequent visual target when the modality of both stimuli were certain. If, qn the one hand, a salient auditory stimulus orients attention in such a way that processing of a subsequent visual stimulus at or near its location is not facilitated, then the auditory cue should fail to influence the performance on the visual task. This would indicate that the mechanisms for stimulus-driven spatial attention orienting are at least partially modality specific. If, on the other hand, a salient auditory stimulus orients attention in such a way that processing of a subsequent visual stimulus at or near its location is facilitated, then the auditory cue should influence the performance on the visual task. This would indicate that the mechanisms for stimulus-driven spatial attention orienting are either linked or supramodal. The auditory-on-visual cue effect has been examined under these very simple cue environments in two previous experiments. Reuter-Lorenz and Rosenquist (1996) found that simple detection responses to visual targets were faster on valid-cue trials than on invalid-cue trials when the SOA between cue and target was short (100-200 ms). The same cues, however, 50 failed to produce IOR at longer SOAs unless subjects made eye movements toward and then away from the cued location. In contrast, Spence and Driver (1998b) did observe IOR between auditory cues and visual targets in a similar simple RT task when a central auditory event was used to reorient attention back to fixation. They argued that Reuter-Lorenz and Rosenquist's failure to demonstrate such an effect could have been due to a long-lasting facilitation at the cued location. However, eye position was not monitored in Spence and Driver's (1998b) experiment, and so their significant IOR results might simply reflect overt shifts of gaze rather than covert shifts of attention. In addition, neither experiment eliminated the possibility of criterion shifts because simple RT, rather than choice RT, was measured. 5.1 Method Subjects Twenty-three new subjects (13 females and 10 males) were recruited from the University of British Columbia to participate. All subjects reported having normal hearing and normal or corrected-to-normal vision. Thirteen subjects between the ages of 17 and 23 (mean age =19 years) took part in Experiment 1A and 10 subjects between the ages of 17 and 25 (mean age = 22 years) took part in Experiment IB. One subject in Experiment IB was left-handed. Procedure Subjects were required to respond to peripheral visual targets following spatially uninformative auditory cues. Each subject first participated in a block of 27 practice trials with 51 the experimenter present in order to provide feedback and to answer any questions. Following the practice trials, each subject participated in 756 experimental trials separated by 1-min rest periods into 28 blocks. The testing session lasted approximately 1 hr. 5.2 Results All of the subjects performed the visual spatial discrimination task with over 90% accuracy. Responses occurred on only 4.0% of the NO-GO trials in Experiment 1A and 3.8% of the NO-GO trials in Experiment IB. Approximately 1.0% and 2.5% of the GO trials were discarded from Experiments 1A and IB, respectively, because the RTs were below 100 ms or above 1500 ms. Approximately 0.5% of the trials were discarded in Experiment IB due to excessive eye movement. Median RTs were calculated separately from the remaining data for each subject in each of the nine validity x SOA conditions. The intersubject means of these median RTs and the corresponding error rates are shown in Table 1. In the initial analyses of the RT data, there was a significant main effect of SOA in both experiments with the longer RTs occurring at the 100-ms SOA [Experiment 1 A: F(2,24) = 25.6, p < 0.0005; Experiment IB: F(2,18) = lA,p = 0.004]. The interpretation of this effect is made clear from inspection of Table 1, which reveals that the mean RTs generally decreased as the SOA increased. This pattern is common in spatial cueing experiments and has been interpreted as a temporal warning effect of the cue (e.g., Ward, 1994). Responses are generally faster at longer SOAs because there is less temporal uncertainty as to when the target will occur. There was also a significant main effect of validity in both experiments, with subjects responding more 52 Table 1 Mean Response Time (M; in milliseconds), Standard Error (SE), and Percent Errors (%E) as a Function of Cue Validity and Stimulus Onset Asynchrony (SOA) in Experiments 1A and lB. f SOA Exp Cue Effect Cue Type 100 500 900 M SE %E M SE %E M SE %E 1A A-V Valid 460 32 0.7 441 30 0.5 425 24 1.3 Central 504 30 1.4 463 28 0.4 444 26 0.9 Invalid 493 29 1.1 449 29 0.4 428 24 1.4 IB A-V Valid 404 27 0.4 393 24 0.9 391 24 1.6 Central 428 26 0.5 405 24 0.4 399 28 0.2 Invalid 427 29 0.9 411 27 0.2 388 26 0.9 1A+B A-V Valid 436 22 0.5 420 20 0.7 410 17 1.4 Central 471 21 1.1 438 19 0.4 424 19 0.5 Invalid 465 21 1.1 433 20 0.4 410 18 1.3 Key to abbreviations: Exp = Experiment; A-V = auditory - visual 53 quickly on valid-cue trials than on invalid-cue trials [Experiment 1 A: F(2,12) = 11.3, p = 0.006; Experiment IB: F(2,9) = 6.7,p = 0.03]. In Experiment 1 A, this difference was large (33 ms) at the 100-ms SOA but was negligible at the 500 and 900 ms intervals (8 and 3 ms, respectively), resulting in a significant validity x SOA interaction [F(2,24) = 10.4, p = 0.001]. A similar interaction was observed in Experiment IB [F(2,18) = 6.5, p = 0.009] although the invalid -valid RT difference was still fairly large (18 ms) at the 500-ms SOA. The data from Experiments 1A and IB were then pooled and analyzed in a mixed ANOVA to determine whether the monitoring of eye position influenced any of the within-subjects effects. There was a larger temporal warning effect in Experiment 1A than in Experiment IB giving rise to a marginally significant SOA x eye position monitoring interaction [F(2,42) = 3.0,p = 0.061]. However, the main effect of eye-position monitoring was nonsignificant [F(l,21) = 1.5, p = 0.23], as were all of the interactions involving that factor [validity x eye-position monitoring: F(l, 21) < 1; SOA x validity x eye-position monitoring: F(2,42) = 2.1,p = 0.14]. As expected based on the separate ANOVAs, the main effects for validity and SOA as well as the validity x SOA interaction were all highly significant [validity: F(2,21) = 17.3,p < 0.0005; SOA: F(2,42) = 29.1,p < 0.0005; validity x SOA: F(2,42) = 14.3,p < 0.0005]. Planned comparisons between the mean RTs for valid and invalid cue trials at each of the SOAs showed that the mean cue effect was significant at the 100-ms SOA [29 ms; p(20 < p. < 38) - .95] and the 500-ms SOA [13 ms; ^ (4 < u < 22) = .95], but not at the 900-ms SOA [0 ms; p(-9 < p: < 9) = .95]. Figure 4 summarizes these combined RT data. The analysis of the error data revealed a significant main effect for SOA, with the lowest 54 error rates occurring at the 500-ms SOA [F(2,42) = 2.8, p = 0.08]. None of the other effects or interactions approached significance [all F's<Y\. Although the error percentages on GO trials in Experiments 1A and IB were very low, Table 1 reveals some interesting trends. At the 100-ms SOA, subjects made slightly fewer errors on valid-cue trials than on invalid-cue trials indicating that a speed-accuracy trade off did not occur. Consequently, the facilitatory cue effect at the 100-ms SOA cannot be attributed to a shift in the decision criterion. In contrast, subjects made slightly more errors on valid-cue trials than on invalid-cue trials at the 500-ms SOA. This raises the possibility that the 13-ms advantage for valid-cue trials over invalid cue trials might have arisen from a shift in the response criterion. Namely, subjects might have used a relatively less conservative criterion on valid-cue trials than on invalid-cue trials, thereby making their responses faster but less accurate. 5.3 Discussion The subjects in Experiments 1A and IB performed the visual spatial discrimination more quickly when the auditory cue and visual target appeared at the same location than when they appeared on opposite sides of fixation. This facilitatory cue effect was largest at the 100-ms SOA, was present at a reduced magnitude at the 500-ms SOA, and was completely absent at the 900-ms SOA. The cue effect at the 100-ms SOA was not due to a criterion shift because responding at that interval was both faster and more accurate on valid-cue trials than on invalid-cue trials. Moreover, and unlike previous studies (e.g., Spence & Driver, 1997), the present study also ruled out any contribution of overt orienting to the observed auditory-on-visual cue 55 Figure 4. Mean response times (RTs; in milliseconds) as a function of the cue-target stimulus onset asynchrony (SOA) in Experiment 1. Valid-cue trials are shown as fdled squares and invalid-cue trials are shown as unfilled squares. 56 effect by monitoring eye position (in Experiment IB). Thus, it seems reasonable to conclude that, under the present conditions, auditory cues do affect responses to visual targets. One possible inference that can be drawn from these findings is that the auditory cue orients spatial attention such that processing of the subsequent visual target is facilitated at the validly cued location, inhibited at the invalidly cued location, or both. As noted earlier, this seems to indicate that the mechanisms for stimulus-driven spatial attention orienting are not entirely modality-specific. Although it is tempting to speculate that spatial attention orienting involves completely supramodal mechanisms, the evidence reviewed in the Introduction suggests that this is not the case. In particular, recent experiments indicate that visual cues fail to influence responses to auditory targets (Mondor & Amirault, 1998; Spence & Driver, 1997). Consequently, it is critical to investigate whether visual cues can influence RTs to auditory targets using the present stimulus and task parameters before making any specific claims about the modality specificity of the stimulus-driven spatial attention orienting mechanisms. This was done in Experiments 2A and 2B. Despite the clear and strong facilitatory effects at the shorter cue-target intervals, there was no evidence for IOR at the longer intervals in the present experiments. These null results parallel those reported by Reuter-Lorenz and Rosenquist (1996), who found no evidence for IOR between auditory cues and visual targets in a series of simple RT tasks. As noted earlier, however, auditory cues did elicit IOR in that study when subjects first moved their eyes to the cued location and then returned their gaze to a central fixation light prior to the appearance of the visual target. Based on their findings Reuter-Lorenz and Rosenquist (1996) concluded (1) that 57 explicit oculomotor activation, but not spatial attention orienting, is critical for producing IOR to auditory cues, and (2) that, unlike visual stimuli, auditory stimuli do not engage the oculomotor system reflexively. These conclusions, although intriguing, are inconsistent with recent findings of IOR in intramodal auditory experiments in the known absence of eye movements (McDonald & Ward, in press; Tassinari & Berlucchi, 1995). Thus, it is possible that the null IOR findings in the present experiments as well as in Reuter-Lorenz and Rosenquist's (1996) experiment were caused by something other than a lack of reflexive oculomotor activation by the auditory cue. One possibility is that the lack of IOR was caused by persisting attentional facilitation at the cued location. This seems somewhat unlikely because the facilitatory cue effect, most likely caused by spatial attention orienting, vanished completely by the 900-ms SOA in Experiments 1A and IB. Nevertheless, it is worth noting that there were substantial cue effects at the 500-ms SOAs when eye position was monitored. This long-lasting facilitation might reflect a slower time course for stimulus-driven spatial attention orienting between auditory cues and visual targets or that, despite instructions to the contrary, subjects voluntarily sustained their attention at the cued location. Clearly, IOR would be abolished in both situations. This raises the possibility that IOR might exist crossmodally under other circumstances, as suggested by the several simple RT experiments mentioned in the Introduction. For example, Spence and Driver (1998b) observed IOR between auditory cues and targets when an audiovisual event was presented at fixation to draw attention away from the cued location. Unfortunately, these results are difficult to interpret because changes in the decision criterion cannot be ruled out. Another, possibly related, explanation for the lack of IOR concerns the possibility that 58 crossmodal IOR is under strategic control. There have been several indications that different stimulus and task parameters influence behavioural performance in spatial cueing tasks, particularly when multiple modalities are involved. For example, Posner (1978) showed that the facilitatory effects of symbolic visual cues are reduced when the modality of the target is uncertain. He argued that attentional selection based on modality can take precedence over, and thus strategically alter, attentional selection based on spatial location. More recently, it has been argued that various strategic factors might affect the ability of visual and auditory cues to orient spatial attention crossmodally (Ward et al., 1998a, 1998b). The IOR effect might be particularly sensitive to such factors because ample time is available between the presentation of the cue and target to implement a particular attentional set or strategy. These various explanations are tested in Experiments 3A and 3B. Before that, however, the opposite crossmodal condition, involving auditory cues and visual targets, will be examined. 6 E X P E R I M E N T S 2 A A N D 2 B Experiments 2 A and 2B examined whether the presentation of a spatially uninformative visual cue would influence RTs to a subsequent auditory target when both the cue and target modalities were certain. As in Experiments 1A and IB, this design permitted the evaluation of two critical predictions. If, on the one hand, a salient visual stimulus orients attention in such a way that processing of a subsequent auditory stimulus at or near its location is facilitated, then the visual cue should influence the performance on the current auditory task. Given the results of Experiments 1A and IB, this would indicate that stimulus-driven spatial attention orienting 59 might be controlled by a supramodal mechanism that acts on multimodal representations of space (cf. Ward et al, 1998a). If, on the other hand, a salient visual stimulus orients attention in such a way that processing of a subsequent auditory stimulus at, or near, its location is not facilitated, then the visual cue should not influence the performance on the current auditory task. This would indicate that crossmodal spatial attention orienting is dominated by audition under these, and possibly other, conditions (cf. Spence & Driver, 1997). Moreover, such results would certainly indicate that the mechanisms for stimulus-driven spatial attention orienting are at least partially modality-specific. Two previous studies have addressed this issue when the cue and target modalities were both certain. In one, subjects made responses based on the auditory target's elevation more quickly when appeared on the same side of fixation as the visual cue than when the cue and target appeared on opposite sides (Spence and Driver, 1997, Experiment 5). This facilitatory effect was observed at unusually long SOAs (200 and 700 ms but not 100 ms) and there was no subsequent IOR effect. In another study, subjects made simple detection responses to auditory targets more slowly when the cue and target appeared on the same side of fixation than when they appeared on opposite sides (Reuter-Lorenz et al., 1996). This IOR effect occurred at long SOAs (1000 and 1300 ms) following a central reorienting event (a flicker of the fixation point). These findings suggest that visual cues can influence RTs to auditory targets under some conditions, facilitating them at short SOAs and inhibiting them at longer SOAs. However, there has been no evidence to date for a facilitatory visual-on-auditory effect in the known absence of eye movements. In fact, Spence and Driver (1997, Experiment 6) found that under their 60 experimental conditions, the facilitatory effect of visual cues on auditory elevation responses occurred only when eye position was not monitored. This raises the possibility that all visual-on-auditory cue effects are caused by eye movements rather than spatial attention shifts, at least when the cue and target modalities are certain. 6.1 Method Subjects Twenty-seven students (19 females and 8 males) from the University of British Columbia participated in Experiment 2. All subjects reported having normal hearing and normal or corrected-to-normal vision. Seventeen subjects between the ages of 17 and 22 (mean age =19 years) took part in Experiment 2A and 10 subjects between the ages of 18 and 26 (mean age = 21 years) took part in Experiment 2B. All subjects were right-handed. Procedure Each subject first participated in a block of 27 practice trials with the experimenter present in order to provide feedback and to answer any questions. Following the practice trials, each subject participated in 756 experimental trials separated by 1-min rest periods into 28 blocks. The testing session lasted approximately 1 hr. 6.2 Results All of the subjects performed the auditory spatial discrimination task with over 85% 61 accuracy. The percentage of NO-GO errors was 7.3% in Experiment 2 A and 8.1% in Experiment 2B. Approximately 3.5% of the GO trails were discarded from each experiment prior to analysis because the RTs were below 100 ms or above 1500 ms. An additional 1.8% percent of the trials were discarded in Experiment 2B due to excessive eye movement. Median RTs were calculated separately from the remaining data for each subject in each of the nine validity x SOA conditions. The intersubject means of these median RTs and the corresponding error rates are shown in Table 2. In the analyses of the RT data, there was a significant main effect of SOA in both experiments indicating that the visual cue produced a temporal warning effect [Experiment 2A: F(2,32) = 38.1,/? < 0.0005; Experiment 2B: F(2,18) = 20.6,/? < 0.0005]. Also, in Experiment 2A, mean RTs were shorter on valid-cue trials than on invalid-cue trials at all three SOAs, resulting in a significant main effect of validity [F(2, 32) = 8.7,/? = 0.001]. This effect did not reach significance in Experiment 2B, as the invalid-valid RT difference was large only at the first SOA [F(l, 9) = 2.8, p = 0.13]. There was a significant validity x SOA interaction in both experiments, suggesting that the validity effect depended on SOA in both cases [Experiment 2A: F(2, 32) = 8.7,p = 0.001; Experiment 2B: F(2,18) = 4.1,p = 0.05]. The data from Experiments 2A and 2B were then pooled and analyzed in a mixed ANOVA to determine whether the monitoring of eye position influenced any of the within-subjects effects. As expected on the basis of the separate analyses, there was a marginally significant interaction between validity, SOA, and eye-position monitoring suggesting that the validity effect was larger at the 500-ms and 900-ms SOAs when eye position was not monitored 62 Table 2 Mean Response Time (M; in milliseconds), Standard Error (SE), and Percent Errors (%E) as a Function of Cue Validity and Stimulus Onset Asynchrony (SOA) in Experiments 2A and 2B.f SOA Exp Cue Effect Cue Type 100 500 900 M SE %E M SE %E M SE %E 2A V-A Valid 467 21 1.3 426 19 0.7 430 18 0.7 Central 517 18 4.3 478 20 1.1 448 22 0.9 Invalid 511 19 1.4 468 20 0.4 445 19 0.4 2B V-A Valid 477 38 0.4 433 34 0.5 438 31 0.7 Central 521 40 5.0 455 38 0.4 429 32 0.2 Invalid 518 39 1.3 442 32 0.9 443 38 1.1 2A+B V-A Valid 471 19 0.9 428 17 0.6 433 16 0.7 Central 518 18 4.6 468 18 0.7 441 18 0.7 Invalid 513 18 1.3 458 17 0.5 444 18 0.7 •fKey to abbreviations: Exp = Experiment; V-A = visual - auditory 63 than when eye position was monitored [F(2,50) = 2.4, p = 0.10]. None of the other effects involving the eye-position monitoring factor, including the main effect, was significant. In both experiments, RTs were much shorter on valid-cue trials than on invalid-cue trials at the 100-ms SOA. This effect also occurred at the 500-ms SOA in Experiment 2A, but was smaller at that interval in Experiment 2B. The effect was reduced at the 900-ms SOA in both experiments although it was still present in Experiment 2A. Planned comparisons between the mean RTs for valid and invalid cue trials at each of the SOAs showed that the mean cue effect was significant at the 100-ms SOA [42 ms; ^ (30 < p < 54) = .95] and the 500-ms SOA [30 ms effect; p(18 < p < 42) = .95 ] but not at the 900-ms SOA [11 ms effect; p(-l < p. < 23) = .95]. Figure 5 summarizes the combined RT data. There were few errors on GO-trials indicating that subjects could easily localize the auditory target. Subjects made fewer than 2% errors in most conditions of Experiments 2 A and 2B. However, the error rate was higher when the target followed very shortly after a central cue. This higher error rate (4.3% in Experiment 2A and 5.0% in Experiment 2B) most likely reflects a different decision strategy on these trials. In particular, the requirement to withhold responses to targets appearing at central fixation might have caused some confusion, particularly at short SOAs, as to whether the cue or target appeared there on centre-cue trials. Consistent with this interpretation, the RTs were, on average, longer in the centre-cue condition. Thus, neither the RT data nor the error data were analyzed in this condition because of the potential change in decision strategy resulting from the task demands. Despite the overall low error rates, the increased error rate in the centre-cue condition 64 does indicate that changes in decision strategy can be detected in these experiments. However, as shown in Table 2, subjects made slightly fewer errors on valid-cue trials than on invalid-cue trials at the 100-ms SOA. This trend is consistent with RT data indicating that a speed-accuracy trade-off did not occur. Consequently, the facilitatory cue effect at the shortest SOA cannot be attributed to a shift in criterion. The analysis of the error rates revealed no significant main effects or interactions. 6.3 Discussion The data from Experiments 2A and 2B demonstrate unequivocally, and in contrast to recent suggestions (e.g., Mondor & Amirault, 1998; Spence & Driver, 1997), that a spatially uninformative visual cue can facilitate responses to an auditory target on valid-cue trials relative to invalid-cue trials. This effect was not due to overt eye movements because trials on which eye position deviated from the central fixation point were discarded from the analysis (Experiment 2B). Likewise, the effect was probably not due to a change in the decision criterion because there was no evidence for a speed-accuracy trade off. In light of these careful considerations, it is tempting to conclude from the results of Experiments 2A and 2B that the spatially uninformative visual cue oriented attention in such a way that processing of the auditory targets was facilitated on valid-cue trials relative to invalid-cue trials. Together with the results of Experiments IA and IB, then, the results of Experiments 2 A and 2B indicate that stimulus-driven spatial attention orienting might be accomplished by a single, supramodal mechanism that works in the same manner regardless of the particular spatial modalities involved. The existence 65 Figure 5. Mean response times (RTs; in milliseconds) as a function of the cue-target stimulus onset asynchrony (SOA) in Experiments 2. Valid-cue trials are shown as filled squares and Invalid-cue trials are shown as unfilled squares. 66 of such a mechanism would indicate that spatial attention orienting is strongly supramodal. Again, despite the strong and significant facilitatory cue effect, IOR did not occur between the visual cue and auditory target. This outcome is somewhat surprising because the visual cue appeared to be sufficient to summon attention, as indicated by the facilitatory cue effect, and to engage the oculomotor system, as indicated by previous behavioural and physiological studies of the oculomotor system (e.g., Jay & Sparks, 1987b, 1990; Posner & Cohen, 1980). Although it is conceivable that auditory stimuli, such as the cue used in Experiments 1A and IB, do not reflexively engage oculomotor processes (cf., Reuter-Lorenz & Rosenquist, 1996), the various oculomotor accounts of IOR are based on the proposal that salient visual stimuli, such as the cues used in Experiments 2A and 2B, do engage the oculomotor system reflexively, causing overt orienting responses to be prepared but then later suppressed due to the instructions to maintain eye fixation. If such an account was correct, then a salient but uninformative visual cue should produce IOR regardless of the target's modality. That IOR failed to occur in the present experiments indicates either that the visual cue did not engage the oculomotor system or that such oculomotor processes do not invariably produce IOR. On the assumption that the visual cue did engage the oculomotor system, the present null findings are suggestive of a nonoculomotor account of IOR. Of course, the alternative explanations for the lack of IOR to the auditory cue in Experiments 1A and IB also must also be considered here. Namely, IOR, possibly arising from oculomotor-based mechanisms, might have been masked by a long lasting facilitatory effect of attention. Notice from Table 2 that the facilitatory effect of the visual cue was indeed sustained much longer than in previous intramodal auditory and visual 67 experiments. Alternatively, and as mentioned earlier, crossmodal IOR might be under strategic control and might occur in different experimental conditions. Despite the symmetrical crossmodal facilitation, observed for the first time in this investigation, it remains possible that spatial attention orienting involves a weakly supramodal mechanism, in the sense defined earlier, or even separable-but-linked mechanisms, rather than a strongly supramodal one. Indeed, it is not immediately apparent why a supramodal mechanism would fail to orient attention crossmodally under some conditions. To review, all previous studies of stimulus-driven spatial attention orienting involving auditory and visual stimuli have observed null cue effects in at least one direction. For example, Spence and Driver (1997), who found an asymmetry favoring audition, asserted that visual cues have no influence on stimulus-driven covert auditory orienting despite their own data suggesting the possibility of such an effect (when eye position was not monitored). They offered one possible explanation for this asymmetry in terms of anatomical differences in the brain, implying that the absence of a visual-on-auditory effect arises completely from differences in neural "hardware" (cf. Klein, 1977). An alternative explanation would attribute the occasional absence of this, or any other, crossmodal effect to strategic control factors. This latter explanation has some appeal because it is well known that intramodal attention effects are under strategic control (Egeth & Yantis, 1997; Folk, Remington, & Johnston, 1992). Even stimulus-driven attention effects, which often occur without intent, are still determined by the observer's "plan of action" (Egeth & Yantis, 1997, p. 276). This is amply demonstrated by the observation that attentional capture by salient visual events can be prevented by focusing attention elsewhere in the visual field (Yantis & Jonides, 68 1990). Another example is the critical influence of spatial relevance on auditory spatial attention (McDonald & Ward, in press). These examples demonstrate that intramodal spatial attention effects are contingent upon task-induced "attentional control settings" (Folk et al., 1992). Such strategic control settings might also modulate the crossmodal effects of uninformative spatial cues (Ward et al., 1998a, 1998b). There are several notable differences in the experimental conditions under which crossmodal cue effects have been examined. The most obvious differences involve the actual tasks that were used. Whereas significant crossmodal cue effects occurred in location-based discrimination tasks (Spence & Driver, 1997; Ward, 1994; Ward et al., 1998a, 1998b), they invariably failed to occur in identity-based discrimination tasks (Mondor & Amirault, 1998; Ward et al., 1998a). Importantly, the inability of spatial cues to influence performance in tasks involving auditory stimuli does not rule out the possibility of crossmodal cue effects in other situations. Rather, it suggests that such tasks are not always appropriate for investigating spatial attention because they do not ensure the involvement of location-sensitive auditory neurons. Another difference is the cue and target environments used. Ward (1994) used a very complicated cue environment, with auditory and visual stimuli appearing either alone, together, or not at all. In each experiment, however, subjects responded to targets presented in just a single modality; thus, the cue modality was uncertain (and quite complex) and the target modality was certain. In contrast, Spence and Driver (1997) presented cues from a single modality and usually required subjects to respond to randomly intermixed auditory and visual targets. In these experiments, then, the cue modality was certain and the target modality was 69 uncertain. Given the different pattern of results obtained in the two experimental situations, it seems possible that the certainty of the target stimulus can affect crossmodal attention orienting by influencing task-induced strategies. Importantly, there are some existing data indicating that strategy can influence the crossmodal interactions in attention and performance. Klein (1977) demonstrated that the normal visual dominance over kinesthesis in detection and localization performance is due to a strategy to attend to the visual modality rather than a hardwired relationship between the sensory modalities and the attention mechanism. He found that the visual dominance effect depended on the subjects knowing in advance of every trial that there will be a bimodal (visual and kinesthetic) stimulus. When subjects were instructed to attend to one modality, conflicting visual and kinesthetic information interfered with performance equally. These results suggest that vision controls awareness and performance only when a particular attentional strategy is adopted. Generally consistent with the above findings is the observation that spatial cue effects at least sometimes depend on the subjects knowing the modality of the target in advance of every trial. Posner et al. (1978) showed that the benefits and costs of symbolic spatial cues are substantially reduced when the modality of the target is uncertain than when it is certain. They required subjects to make speeded discriminations concerning the intensity of a visual or somatic target following a symbolic visual cue. The cue was either a "neutral" plus sign, an arrow pointing to the left or right side of fixation, or a letter (V or T) denoting the possible modality of the target. The cue was valid on 80% of the trials except when it was neutral, where it was followed equally often by all possible targets. The results showed that the visual and somatic 70 modality cues both produced substantial costs (25-40 ms) and benefits (20 ms) compared to the neural condition. This indicates that subjects can selectively attend to signals in one modality or the other. The spatial cues produced much smaller RT effects, both in comparison to the modality cue effects and the spatial cue effects that are found when subjects know the modality of the target in advance (e.g., Posner, 1978). Posner et al. (1978) speculated that spatial cue effects occur only when the modality of the target is known in advance because such knowledge "allows the subject to focus attention on a particular location in internal space" (p. 156). So far, the present experiments show that substantial facilitatory spatial cue effects can occur between auditory and visual cues and targets when the modality of the target is known in advance. The next two experiments examined whether similar effects would be obtained when the modality of the target is not known in advance. Following Posner et al.'s (1978) data, the most straightforward prediction would be that both intramodal and crossmodal spatial cue effects would be reduced in these target-modality uncertainty experiments. However, more specific hypotheses are required for the present investigation because it involved very different spatial cueing procedures. In particular, it might be possible to account for both the facilitatory and inhibitory effects of uninformative spatial cues in terms of specific strategies induced by variations in target modality certainty. One such account is outlined below. On every trial of a spatial cueing experiment, subjects must suppress their response to the cue and prepare to respond quickly to the target. One consequence of this process is the temporal warning effect, which indicates that a certain amount of time is required to fully prepare for the appearance of the target. Another is the IOR effect, which indicates that subjects sometimes shift 71 attention away from the cued location in preparation for the target. This latter effect seems to occur only when subjects reorient their attention to a neutral (nontarget) location following the appearance of the cue. In intramodal visual tasks, attention returns to fixation either reflexively or in response to a central reorienting event (Maylor, 1985; Rafal et al., 1989). In intramodal auditory tasks, attention also shifts away from the location of a spatially uninformative auditory cue even in the absence of a central reorienting event (McDonald & Ward, in press). Presumably, reorienting occurs in both modalities because subjects know that sustaining attention at the cued location will hinder their performance on a substantial portion of the trials. Notice, however, that such performance decrements would be considerably reduced when the cue and target dimensions do not overlap. Thus, there is far less incentive to actively reorient attention away from the cue when the required response is based on a different form of information. Consistent with this idea are several failures to observe auditory and visual IOR when the response is based on nonspatial (e.g., McDonald & Ward, in press; Terry et al., 1994) or nonazimuthal (Spence & Driver, 1997) discriminations. Moreover, other selective inhibitory effects also seem to be associated with relevant but not irrelevant distracter properties (e.g., negative priming; Tipper, Weaver, & Houghton, 1994). Based on the above account, a reasonable hypothesis for the absence of IOR in Experiments 1 and 2 is that subjects did not actively reorient their attention to a neutral location following the cue. A likely reason for this strategy is that because the subjects were told in advance that the cue and target would be in different modalities, they could prepare for the target without necessarily orienting away from the cue. This view predicts that IOR should be obtained 72 when the target modality is uncertain because the subjects must adopt a more aggressive strategy for dealing with the appearance of the sometimes interfering cue. It also predicts that the facilitatory effect should be smaller and less sustained because subjects are actively reorienting away from it. 7 EXPERIMENTS 3A AND 3B In Experiments 3A and 3B, a spatially uninformative auditory cue was followed by a visual target on half of the trials and by an auditory target on the remaining trials. The visual and auditory target trials were randomly intermixed so that subjects would not know the modality of the target prior to its appearance. These conditions were similar to Spence and Driver's (1997) experiment involving auditory cues and auditory and visual targets, where significant facilitatory cue effects were observed for both auditory (24 ms) and visual (33 ms) target trials at the 100-ms SOA. Based on these findings, Spence and Driver concluded that uninformative sounds produce stimulus-driven visual attention orienting. However, the cue effects observed in that experiment are open to nonattentional explanations as well. Spence and Driver experimentally discounted one such explanation involving relative target elevation judgments based on the cue elevation (such judgments would be easier when the cue and target were on the same side of fixation rather than on opposite sides) by showing that participants could not judge the elevation of the cue sound. However, eye position was not monitored in their experiment, and thus overt orienting might have contributed significantly to the observed spatial cue effects. This possibility is often disregarded in studies of stimulus-driven spatial attention because the cue effects are observed at 73 short SOAs, when the target appears before an eye movement can be made. However, even under these conditions, the manual response normally occurs after such an eye movement can be made. Thus, it is still quite possible that overt orienting could influence responding to the target. Moreover, it has been shown that the attentional facilitation occurring at the cued location is larger when an eye movement to that location is eventually made (Posner & Cohen, 1980). This suggests that additional, oculomotor-based, mechanisms are involved on trials in which eye movements are elicited compared to when they are not (Klein et al., 1992). If so, Spence and Driver's (1997) crossmodal effect would have few implications for purely covert spatial attention orienting. Although a large auditory-on-visual cue effect did occur in Experiments IA and IB when the modality of the target was certain, the strategic account given earlier, in which observers must actively reorient away from the cued location when the cue and target can appear in the same modality, predicts that the cue effect will be smaller, less sustained, and eventually replaced by IOR when the modality of the target is uncertain. These predicted results differ dramatically from those observed by Spence and Driver (1997) and they inform different models of the underlying attention mechanisms. 7.1 Method Subjects Twenty-two new subjects (15 females and 7 males) were recruited from the University of British Columbia to participate. All subjects reported having normal hearing and normal or corrected-to-normal vision. Twelve subjects between the ages of 18 and 22 (mean age = 20 74 years) took part in Experiment 3 A and 10 subjects between the ages of 17 and 23 (mean age = 20 years) took part in Experiment 3B. All subjects were right-handed. Procedure and Design The subjects were told to respond to all peripheral targets and to withhold their response to all central targets, regardless of the modality. They were informed that the cue did not predict the location or the modality of the target. Subjects ran in two separate sessions that were run on different days. Each session lasted approximately 1 hr and consisted of 756 trials that were separated by 1-min rest periods into 14 blocks. Half of the trials consisted of an auditory cue and auditory target (A-A) and the other trials consisted of an auditory cue and visual target (A-V). These trial conditions were randomly intermixed within the same experimental blocks for every subject. Subjects participated in at least 54 practice trials at the start of each session with the experimenter present in order to provide feedback and to answer any questions. 7.2 Results All of the subjects in Experiments 3 A and 3B performed the spatial discrimination task with over 90% accuracy. Responses occurred on 3.5% and 3.4% of the NO-GO trials in Experiments 3A and 3B, respectively, with more NO-GO errors on A-A trials (5.1%) than on A-V trials (1.7%). Fewer than 1% of the GO trials were discarded from each experiment because the RTs were below 100 ms or above 1500 ms, and fewer than 1% were discarded from Experiment 3B due to excessive eye movement. Median RTs were calculated separately from 75 the remaining data for each subject in each of the 18 validity x target modality x SOA conditions. The intersubject means of these median RTs and the corresponding error rates are shown in Table 3. The RT data were first analyzed in separate repeated measures ANOVA for each experiment with target modality (visual and auditory) as an additional within-subjects factor. These analyses revealed a significant main effect of SOA in both experiments, with subjects again slowest to respond at the shortest SOA [Experiment 3A: F(2,22) = 45.9,p < 0.0005; Experiment 3B: F(2,18) = 20.1,p = 0.001]. The main effect for validity did not approach significance, suggesting that the RTs did not differ overall between valid and invalid cue trials [Experiment 3A: F< 1; Experiment 3B: F( 1,9) =1.4,/? = 0.3]. However, the interaction between SOA and validity was highly significant [Experiment 3A: F(2,22) = 17.1,/? < 0.0005; Experiment 3B: F(2,18) = 7.9,/? = 0.004]. At the 100-ms SOA, subjects responded more rapidly to targets on valid-cue trials than on invalid-cue trials. At the 900-ms SOA, subjects responded more rapidly on invalid-cue trials. In Experiment 3B, subjects responded more slowly to auditory targets than visual targets [F(l,9) = 10.4,/? = 0.01]. This difference was largest at the 100-ms SOA, giving rise to a significant interaction between target modality and SOA [F(2,18) = 4.8,/? = 0.04]. There was also a significant interaction between target modality and validity in Experiment 3B [F(2,18) = 7.9,/? = 0.005]. However, the three-way interaction between target modality, SOA and validity was nonsignificant [F(2,22) = 1.4,/? = 0.27], suggesting that the pattern of auditory cue effects across the SOAs was similar in the A-A and A-V conditions. None of the effects involving target modality was significant in Experiment 3A [target modality: 76 Table 3 Mean Response Time (M; in milliseconds), Standard Error (SE) and Percent Errors (%E) as a Function of Cue Validity and Stimulus Onset Asynchrony (SOA) in Experiments 3A and 3B.f SOA Exp Cue Effect Cue Type 100 500 900 M SE %E M SE %E M SE %E 3A A-A Valid 449 12 0.5 408 11 1.1 421 11 0.7 Central 461 14 1.3 391 13 0.5 400 14 0.2 Invalid 463 18 0.5 396 14 0.7 394 16 0.5 A-V Valid 438 14 0.9 402 14 0.5 416 16 0.4 Central 452 13 1.1 404 14 0.5 399 14 0.4 Invalid 445 12 1.3 405 15 0.5 388 15 0.5 3B A-A Valid 502 32 0.5 459 22 0.7 465 23 0.9 Central 531 31 2.3 473 21 0.2 453 19 0.4 Invalid 535 32 0.5 472 23 0.5 452 25 0.4 A-V Valid 446 21 0.2 428 18 0.4 424 16 0.4 Central 465 21 0.4 423 15 0.5 412 19 0.2 Invalid 463 20 0.5 424 17 0.5 414 14 0.4 3A+B A-A Valid 473 . 17 0.5 431 13 0.9 442 11 0.5 Central 493 17 1.8 428 15 0.4 424 12 0.2 Invalid 496 19 0.4 431 15 0.7 421 10 0.4 A-V Valid 442 12 0.5 414 12 0.5 420 13 0.4 Central 458 12 0.7 413 10 0.5 405 13 0.2 Invalid 453 11 0.9 414 10 0.5 400 15 0.5 +Key to abbreviations: Exp = Experiment; A-A = auditory-auditory; A-V= auditory - visual 77 F<1; target modality x validity: F<\; target modality x SOA: F(2,22) = 2.4,/? = 0.11]. The data from Experiments 3 A and 3B were then pooled and analyzed in a mixed ANOVA to determine whether the monitoring of eye position influenced any of the within-subjects effects. This analysis revealed a significant interaction between eye-position monitoring and target modality indicating that RTs were longer for auditory targets than visual targets in the monitored group but not in the unmonitored group [F(l,20) = 5.4, p = 0.03]. None of the other interactions involving the eye-position monitoring factor was significant. As expected from the separate analyses, the main effect of SOA was highly significant [F(2,40) = 55.6, p < 0.0005] with the longest RTs at the shortest SOA. However, as shown in Figure 6, the temporal warning effect did not appear to be equivalent on valid and invalid cue trials. On invalid-cue trials, the RTs decreased monotonically as a function of SOA for both A-A and A-V trials, indicating that the1 auditory cue produced the usual temporal warning effect. On valid-cue trials, the RTs initially decreased but then increased again at longer SOAs. As noted earlier, this up-turn on valid-cue trials indicates the presence of IOR at the longer SOAs (cf. Posner & Cohen, 1984). Consistent with this interpretation, there was a significant interaction between validity and SOA [F(2,40) = 24.9, p < 0.0005]. For both A-A and A-V conditions, the mean RTs were shorter on valid-cue trials than on invalid-cue trials at the 100-ms interval and longer on valid-cue trials than on invalid-cue trials at the 900-ms SOA. These cue effects were subjected to Bonferroni t-tests with the familywise error rate set at 0.10 and the target condition defining the experimental unit. Such tests were one-tailed for the 100-ms cue effects to test specifically for faster RTs on valid-cue than on invalid-cue trials. These reasonably conservative tests revealed that the 78 S T I M U L U S O N S E T A S Y N C H R O N Y ( M S ) Figure 6. Mean response times (RTs; in milliseconds) as a function of the cue-target stimulus onset asynchrony (SOA) in Experiments 3. Valid-cue trials are shown as filled squares and invalid-cue trials are shown as unfilled squares. 79 facilitatory and inhibitory cue effects were significant in both the A-A condition (+23 ms at the 100-ms SOA; -21 ms at the 900-ms SOA) and the A-V condition (+11 ms at the 100-ms SOA; -20 ms at the 900-ms SOA). However, it should be noted that the relatively small facilitatory cue effect in the A-V condition would not have been significant at the more conservative familywise error rate of 0.05, and a 95% confidence interval for that effect did include zero [A-V: p(-3 < p. < 25) = 0.95 at the 100-ms SOA and p(-6 < p. < -34) = 0.95 at the 900-ms SOA; A-A: p(9 < u < 34) = 0.95 at the 100-ms SOA and p(-7 < p < -35) = 0.95 at the 900-ms SOA ]. Still, given that Bonferroni tests are conservative, it seems reasonable to conclude that the auditory cue did have a small but reliable effect on RTs to the visual target at the 100-ms SOA. In the analysis of the combined error data, there were no significant main effects or interactions [all F's < 1]. However, inspection of Table 3 reveals that subjects made slightly fewer errors on valid-cue trials than on invalid-cue trials at the 100-ms SOA for most of the conditions, indicating that a speed-accuracy trade off did not occur at that interval. Similarly, in most conditions, there was a trend for more errors and longer RTs on invalid-cue trials than valid-cue trials at the longer SOAs. Thus, neither the facilitatory cue effect nor the inhibitory cue effect can be attributed to changes in the response criterion. 7.3 Discussion A spatially uninformative auditory cue influenced RTs to both auditory and visual targets in Experiments 3A and 3B. The data obtained in the A-A condition replicate the findings of a recent series of intramodal auditory spatial cueing experiments in which a spatially 80 uninformative auditory cue was followed on every trial by an auditory target (McDonald & Ward, in press). In both cases, RTs were shorter on valid-cue trials than on invalid-cue trials when the cue-target SOA was 100 ms, and longer on those same trials when the cue-target SOA was 900-ms. Thus, the present findings provide further evidence that salient but uninformative auditory events elicit covert spatial attention orienting and IOR when the locations of the stimuli are relevant to the observers' task. In addition, because a central reorienting event was not used in the present experiments, the data also confirm previous observations that auditory IOR can occur without explicit demands to reorient attention to fixation following the appearance of the cue (McDonald & Ward, in press). The data obtained in the A-V condition confirm recent conclusions that covert spatial attention orienting to the location of an irrelevant sound affects the processing of subsequent visual stimuli (Reuter-Lorenz & Rosenquist, 1996; Spence & Driver, 1998a, 1998b; Experiments 1A and IB). However, the pattern of cue effects observed in the present experiments differed in several ways to previously reported auditory-on-visual cue effects. First, the RT advantage for valid-cue trials at the 100-ms SOA was substantially smaller in the present experiment than in several previous ones. In particular, the mean advantage for valid-cue trials over invalid-cue trials was 29 ms [p(20 < p < 38) = .95] in Experiment 1 (A and B combined), whereas it was only 11 ms [p(-3 < u. < 25) = 0.95] in the present experiment. This difference cannot be attributed to variations in goal-driven attentional processes because the cue was spatially uninformative in both cases. In fact, Experiments 1A and IB were identical to the present experiments, except that the former had only visual target trials (modality certain), whereas the 81 latter had auditory and visual target trials intermixed (modality uncertain). Second, the auditory cue did have quite a substantial inhibitory effect on the RTs to the visual target at the 900-ms SOA in the present experiment but not in Experiment 1. This effect was not likely caused by changes in the response criterion because there was no evidence for a speed-accuracy trade off. Likewise, it did not arise from overt orienting because eye position was monitored in a group of subjects to ensure that they did not move their eyes away from the central fixation point. Thus, the present experiments demonstrate unequivocally the existence of IOR between auditory cues and visual targets in the absence of eye movements. That such an effect failed to occur in Experiments 1A and IB indicates that the existence of auditory-on-visual IOR depends critically on target modality uncertainty, at least in the absence of a central reorienting event. Both of these findings appear to confirm the hypothesis of strategic control factors in stimulus-driven crossmodal spatial attention orienting and IOR. One possible strategic factor, which was noted earlier, involves attentional reorienting. According to this account, subjects actively reorient attention away from the cued location, so as to minimize interference, only when the cue and target can occur from the same modality. This account is broadly consistent with several previous results that show (1) that stimulus-driven attention orienting depends on task-induced strategic control factors, (2) that IOR fails to occur when attention is sustained at the cue location, (3) that the facilitatory and inhibitory cue effects occurring within audition depend on the strategic involvement of the proper spatial representations of the cue and target, and (4) that other, possibly attentional, effects such as negative priming depend on overlap between the cue and target dimensions. The next experiment addressed whether the strategic 82 control factors induced by target modality uncertainty can influence the crossmodal effects of visual cues. 8 EXPERIMENTS 4A AND 4B In Experiments 4A and 4B, a spatially uninformative visual cue was followed by an auditory target on half of the trials and by a visual target on the remaining trials. As in Experiments 3A and 3B, the auditory and visual target trials were randomly intermixed so that subjects would not know the modality of the target prior to its appearance. The conditions were thus similar to Spence and Driver's (1997) experiments involving visual cues and visual and auditory targets. In those experiments Spence and Driver (1997) found that spatially uninformative visual cues failed to influence auditory targets and concluded that "visual events do not induce covert auditory orienting when eye movements are prevented" (p. 17). Although the results of Experiment 2 disconfirmed this conclusion, it is likely that the facilitatory cue effect will be reduced in the present experimental conditions because a different strategy, like the one outlined earlier, must be adopted to perform the task in the presence of the sometimes interfering cue. Importantly, the V-A trials in these experiments were identical to those in Experiments 2A and 2B, permitting a comparison between the visual-on-auditory effects under target-modality certainty and uncertainty conditions. 8.1 Method Subjects 83 Twenty new subjects (15 females and 5 males) were recruited from the University of British Columbia to participate. All subjects reported having normal hearing and normal or corrected-to-normal vision. Ten subjects between the ages of 18 and 43 (mean age = 26 years) took part in Experiment 4A and ten subjects between the ages of 18 and 22 (mean age = 20 years) took part in Experiment 4B. One subject in Experiment 4A was left-handed. Procedure The subjects were told to respond to all peripheral targets and to withhold their response to all central targets, regardless of the modality. They were informed that the cue did not predict the location or the modality of the target. Subjects ran in two separate sessions that were run on different days. Each session lasted approximately 1 hr and consisted of 756 trials separated by 1-min rest periods into 14 blocks. Half of the trials consisted of a visual cue and visual target (V-V) and the other trials consisted of a visual cue and auditory target (V-A). These trial conditions were randomly intermixed within the same experimental blocks for every subject. Subjects participated in at least 54 practice trials at the start of each session with the experimenter present in order to provide feedback and to answer any questions. 8.2 Results All of the subjects in Experiments 3 A and 3B performed the spatial discrimination task with over 85% accuracy. Responses occurred on 5.3% and 3.8% of the NO-GO trials in Experiments 4A and 4B, respectively, with more NO-GO errors on V-A trials (5.7%) than on V-84 V trials (1.8%). Approximately 3% and 6% of the GO trials were discarded from Experiments 4A and 4B, respectively, because the RTs were below 100 ms or above 1500 ms. Another 0.5%, were discarded in Experiment 4B due to excessive eye movement. Median RTs were calculated separately from the remaining data for each subject in each of the 18 validity x target modality x SOA conditions. The intersubject means of these median RTs and the corresponding error rates are shown in Table 4. As in Experiments 3A and 3B, the RT data were first analyzed in separate repeated measures ANOVA for each experiment with target modality (visual and auditory) as an additional within-subjects factor. There was no main effect of target modality in either experiment (both Fs < 1) indicating that the overall RTs to visual and auditory targets did not differ reliably. The usual main effect of SOA did occur in both experiments, with subjects again slowest to respond at the shortest SOA [Experiment 4A: F(2,18) = 15.9;p = 0.002; Experiment 4B: F(2,18) = 14.0;p = 0.001]. There was also an interaction between target modality and SOA in Experiment 4B, with a larger temporal warning effect occurring in the V-A condition than in the V-V condition [F(2,18) = 7.9,p = 0.004]. The main effect for validity did not approach significance in these experiments indicating that there was no overall difference between the RTs on valid and invalid cue trials [F < 1 for both experiments]. However, there was a significant interaction between SOA and validity in both experiments suggesting that there were different patterns of cue effects at the various SOAs [Experiment 4A: F(2,18) = 6.8, p = 0.02; Experiment 4B: F(2,18) = 4.3,p = 0.03]. At the 100-ms SOA, the RTs were shorter on valid-cue trials than invalid-cue trials in both the V-V and V-A conditions. This facilitatory cue effect was reduced or 85 Table 4 Mean Response Time (M; in milliseconds), Standard Error (SE) and Percent Errors (%E) as a Function of Cue Validity and Stimulus Onset Asynchrony (SOA) in Experiments 4A and 4B.f SOA Exp Cue Effect Cue Type 100 500 900 M SE %E M SE %E M SE %E 4A V-V Valid 482 25 0.4 455 21 0.0 459 23 0.2 Central 493 26 0.5 463 23 0.0 460 23 0.0 Invalid 494 28 0.7 434 22 0.2 450 25 0.0 V-A Valid 479 30 0.0 431 27 0.2 452 23 0.5 Central 511 35 0.4 441 33 0.2 422 33 0.2 Invalid 491 38 0.0 434 33 0.0 438 32 0.0 4B V-V Valid 518 20 1.3 497 17 1.1 518 13 1.4 Central 544 16 1.3 508 16 0.4 517 14 1.4 Invalid 544 15 0.7 490 20 1.1 509 15 3.0 V-A Valid 548 17 1.1 494 16 0.5 500 13 0.7 Central 567 22 2.0 490 15 0.7 496 14 1.1 Invalid 558 15 2.1 489 14 1.4 502 11 1.1 4A+B V-V Valid 500 16 0.8 476 14 0.5 488 14 0.8 Central 518 16 0.9 485 14 0.2 489 15 0.7 Invalid 519 17 0.7 462 16 0.6 479 16 1.5 V-A Valid 513 19 0.5 462 17 0.4 476 14 0.6 Central 539 21 1.2 466 19 0.4 459 20 0.6 Invalid 525 21 1.1 461 18 0.7 470 18 0.5 ^ey to abbreviations: Exp = Experiment; V-V visual - visual; V-A = visual - auditory 86 reversed at the longer SOAs for both the V-V and V-A conditions and thus the target modality x SOA x validity interaction did not reach significance [Experiment 4A: F(2,18) = 2.4, p = 0.12; Experiment 4B: F(2,18) = 2.2,p = 0.14]. However, Table 4 reveals that, although the facilitatory cue effect diminished in the V-A condition, it did not reverse substantially in Experiment 4B. These results indicate that, when subjects are prevented from making eye movements, a spatially uninformative visual cue might fail to produce IOR for auditory targets. The data from Experiments 4A and 4B were then pooled and analyzed in a mixed ANOVA to determine whether the monitoring of eye position affected any of the within-subjects effects. The main effect of eye-position monitoring was marginally significant in this analysis [F(l, 18) = 3.7,p = 0.07], with shorter RTs occurring in the eye-position unmonitored group (458 ms) than in the eye-position monitored group (514 ms). Also, as expected from the separate analyses, the main effect of SOA was highly significant [F(2,36) = 29.7, p < 0.0005]. This effect can be seen clearly in Figure 7. The mean RTs were longest at the shortest cue-target SOA regardless of the target's modality indicating that the visual cue had a temporal warning effect on both V-V and V-A trials. In most conditions, there was a rather sharp decrease in RT between the 100-ms and 500-ms SOAs. However, this trend reversed between the 500-ms and 900-ms SOAs, where the time to respond to both the visual and auditory targets increased substantially. Figure 7 also shows that the mean RT differences between valid and invalid cue trials varied as a function of SOA [F(2, 36) = 10.4,/? < 0.0005]. The mean RTs were somewhat shorter on valid-cue trials than on invalid-cue trials at the 100-ms interval in both the V-V and V-A conditions. This facilitatory cue effect reversed at the 500 and 900-ms SOAs in the V-V condition and at the 900-ms in the V-A condition. The inhibitory cue effect was larger in the V-V than in the V-A condition although this interaction did not reach significance [F(2, 36) = 2.3, p = 0.12], As suggested by the separate analyses, the modality * SOA * validity x eye-position monitoring interaction approached significance [F(2,36) = 2.3,p = 0.11]. To test specific hypotheses, the cue effects were subjected to Bonferroni t-tests with the familywise error rate set at 0.10 and the target condition defining the experimental unit. Such tests were one-tailed for the 100-ms cue effects to test specifically for faster RTs on valid-cue than on invalid-cue trials. The cue effects were tested at the 100 and 500 ms SOAs in the V-V condition and at the 100 and 900 ms SOAs in the V-A condition because previous studies indicate that IOR occurs later in experiments involving auditory stimuli (e.g., McDonald & Ward, in press). These reasonably conservative tests revealed that the facilitatory cue effect was significant at the 100-ms SOA in both the V-V condition [+19 ms; p(+7 < p < +31) = 0.95] and the V-A condition [+12 ms; p(0 < u < +24) = 0.95]. Thus, it seems reasonable to conclude that the visual cue did have reliable facilitatory effects on RTs at the 100-ms SOA in both conditions. In contrast, the inhibitory cue effect was significant at the 500-ms SOA in the V-V condition [-14 ms; p(-26 < p. < -2) = 0.95] but not at the 900-ms SOA in the V-A condition [-6 ms; p(-18 < u < +6) = 0.95]. In the analysis of the pooled error data, there was a significant main effect of eye-position monitoring, with the monitored group making more errors than the unmonitored group [F(l, 18) = 10.7,/> = 0.004]. The three-way interaction between SOA, target modality, and eye-position monitoring was also significant [F(2, 36) = 4.9, p = 0.016]. None of the other main effects or interactions was significant. Inspection of Table 4 reveals that there was no overall difference 550 Figure 7. Mean response times (RTs; in milliseconds) as a function of the cue-target stimulus onset asynchrony (SOA) in Experiments 4. Valid-cue trials are shown as fdled squares and invalid-cue trials are shown as unfilled squares. 89 between the percent of errors made on valid and invalid cue trials at the 100-ms SOA in the V-V condition (0.8% vs. 0.7%). There was a trend for fewer errors on valid-cue trials than on invalid-cue trials at the 100-ms SOA in the V-A condition (0.5% vs. 1.1%). The RT advantages for valid-cue trials over invalid-cue trials at the 100-ms SOAs probably do not reflect changes in the decision criterion because a speed-accuracy trade-off occurred failed to occur in both conditions. In contrast, although there was very little difference between the percentages of errors made on valid and invalid cue trials at the 500-ms SOA in the V-V condition (0.5% vs 0.6%), a lower error percentage did occur on valid-cue trials than on invalid-cue trials at the 900-ms SOA in that condition (0.7% vs. 1.6%). This indicates that, whereas the RT difference at the 500-ms SOA probably reflects IOR, the RT difference at the 900-ms SOA might have arisen simply because subjects used a more conservative decision criterion on valid-cue trials than on invalid-cue trials. 8.3 Discussion In Experiments 4A and 4B, a spatially uninformative visual cue influenced RTs to both visual and auditory targets. The pattern of results obtained in the V-V condition is consistent with several previous studies of intramodal visual spatial cueing (e.g., Posner & Cohen, 1984; Rafal et al., 1989). At the 100-ms SOA, subjects responded more quickly to the visual target when it appeared at the validly cued location than when it appeared at the invalidly cued location. This cue effect reversed at the two longer SOAs, where subjects responded more slowly to the visual target when it appeared at the validly cued location. The facilitatory cue effect observed at the short SOA is commonly attributed to an abrupt shift of attention toward the 90 cued location, whereas the inhibitory effect observed at the longer SOAs is commonly attributed to the occurrence of IOR (Posner & Cohen, 1984). Because a central reorienting event was not used in the present experiments, the data also confirm several previous observations that visual IOR can occur without explicit demands to reorient attention to fixation following the appearance of the cue (Maylor, 1985; Rafal et al., 1989; Tassinari et al., 1987). The occurrence of the uninformative visual cue produced a similar effect on RTs in the V-A condition at the 100-ms SOA. In this condition, subjects responded more quickly to the auditory target on valid-cue trials than on invalid-cue trials. This facilitatory effect occurred in both the unmonitored subjects (Experiment 4A) and the monitored subjects (Experiment 4B) indicating that overt orienting did not contribute to the effect. Moreover, because there was no sign of a speed-accuracy trade off at the 100-ms SOA, the facilitatory cue effect cannot be attributed to a shift in the decision criterion. These results, together with the results of Experiment 2, disconfirm previous conclusions about the absence of a visual-on-auditory cue effect (Mondor & Amirault, 1998; Spence & Driver, 1997). Comparisons of the data obtained in Experiments 2 A/B and 4 A/B reveal that the facilitatory visual-on-auditory effect at the 100-ms SOA was around four times smaller in Experiment 4 (+12 ms) than it was in Experiment 2 (42 ms), and its time course was substantially shortened. These differences, like those observed in the A-V condition (see Experiments 3 A and 3B), must be due to the inclusion of intramodal trials because the experiments were identical in all other respects. It is possible that the effect of target-modality uncertainty on the facilitatory visual-on-auditory effect might partially account for the apparent 91 lack of such an effect in some other experiments where the modality of the target was also uncertain (e.g., Mondor & Amirault, 1998, Experiment 1; Spence & Driver, 1997, Experiments 3, 4, and 6). Importantly, however, as the results of Experiment 1 clearly demonstrate, the reduction of such an effect under these conditions, does not definitively rule out its presence in other conditions. Consistent with the predictions made earlier, the IOR effect also occurred in the V-A condition when eye position was not monitored (Experiment 4A). Surprisingly, however, this effect disappeared when eye position was monitored (Experiment IB) indicating that overt orienting might have contributed to the IOR effect in the unmonitored subjects. One possible implication of this finding is that, whereas IOR can occur between auditory cues and visual targets, it cannot occur between visual cues and auditory targets, at least when eye movements are prevented. This conclusion would be suggestive of separable but linked mechanisms for auditory and visual IOR. On this basis, then, IOR would not likely arise from oculomotor processes because such processes are arguably supramodal. Similarly, IOR would not likely arise from spatial attention orienting because the symmetrical facilitatory cue effects suggest the existence of a supramodal spatial attention mechanism. A closer inspection of the individual subjects' performance in Experiments 3 and 4 suggests several important findings regarding the existence of intramodal and crossmodal IOR and the effects of eye-position monitoring on those effects. The individual subjects' median cue effects for the long SOA condition in Experiments 3 and 4 are shown in Figure 8. Although there are too few observations to run statistical tests, some tentative conclusions can be reached 92 because the median cue effects were based on many trials and were thus stable measures of the individual subjects' performance. First, as can be seen from Figure 8, a substantial portion of the subjects in each cue-target condition exhibited negative cue effects, both when eye position was monitored and when it was unmonitored, indicating that IOR occurred for those subjects. In Experiment 3A (unmonitored), 8 of 12 subjects (67%) had IOR on A - A trials and 11 of 12 subjects (92%) had IOR on A - V trials. In Experiment 3B (monitored), 9 of 10 (90%) had IOR on A - A trials and 6 of 10 (60%) had IOR on A - V trials. In Experiment 4A (unmonitored), 8 of 10 subjects (80%) had IOR on V - V trials and 7 of 10 subjects (70%) had IOR on V - A trials. Finally, in Experiment 4B (monitored), 6 of 10 subjects (60%) had IOR on V - V trials and 4 of 10 subjects (40%) had IOR on V - A trials. Second, the patterns of cue effects across subjects differed substantially in monitored and unmonitored subjects. In general, there were many more large cue effects, including some positive ones, in the unmonitored subjects than in the monitored subjects. Additionally, the cue effects appeared to be clustered around two or three different values in the monitored subjects, whereas they were distributed more uniformly in the unmonitored subjects. In each of the crossmodal conditions, there was a group of subjects whose mean cue effect was approximately -24 ms, and a second group of subjects whose mean cue effect was approximately +9 ms. There was also a third group of subjects in the V - A condition whose mean cue effect was +36 ms. A similar group of subjects was not observed in the A - V condition. These findings suggest that eye-position monitoring did affect the intramodal and crossmodal cue effects at the longer SOAs. The primary effect of such monitoring was an 93 CO LU O z LU CC LU LL. LL. Q Q —I 2 60n 4(H • • 20H • • • • • 0-r- • a aa a -20-• • • • • • • • • -40-1 -60H -80H -100 J B B B B A-A A-V V-V V-A EXP 3A/B EXP 4A/B Figure 8. Individual subjects' cue effects (invalid - valid difference) in Experiments 3A, 3B, 4A, and 4B. The intramodal and crossmodal effects are shown separately for each experiment, both with eye-position monitoring (denoted A) and without eye-position monitoring (denoted B). The cue effects are for the 900-ms stimulus onset asynchrony (SOA) in all conditions except for the V-V condition, where the 500-ms SOA is shown. Negative cue effects denote the presence of IOR. 94 overall reduction in the extremely large positive and negative cue effects. One reasonable explanation for this effect is that the unmonitored subjects made eye movements toward the cued location on a portion of the trials. A large positive (facilitatory) cue effect might have arisen if a subject made a saccade toward the cued location and did not saccade back to the central fixation point prior to the appearance of the target. In contrast, a large negative (IOR) effect may have arisen if a subject made a saccade toward the cued location and did saccade back to the central fixation point prior to the appearance of the target. This explanation is consistent with a recent study by Kingstone and Pratt (in press) that found that intramodal visual IOR was larger on trials with eye movements than on trials without eye movements. An important implication of this finding in that IOR might arise from very different mechanisms in the presence and absence of overt eye movements. Importantly, these findings also suggest that IOR can occur between visual and auditory stimuli in the absence of eye movements when the target modality is uncertain. However, as indicated by the bimodal, and sometimes trimodal, distributions of the individual subjects' cue effects, neither intramodal nor crossmodal IOR arises for all subjects in these experiments. One likely explanation is that some subjects did not actively reorient attention away from the cued location, even though the target modality was uncertain, causing them to have large facilitatory effects. However, given that the same magnitude of IOR occurred in a portion of subjects in each of the crossmodal conditions, it seems reasonable to conclude that there are symmetrical IOR effects between the auditory and visual modalities. Thus, IOR, like spatial attention orienting, might arise from a supramodal mechanism rather than from modality-specific 95 mechanisms. The model in Figure 9 represents one way that a supramodal mechanism might be involved in spatial attention orienting within and between the visual and auditory modalities (see also Ward et al., 1998). This model is based on Posner's posterior attention system, except that it postulates interactions between the spatial attention mechanism and both visual and auditory "what" systems. According to this model, visual and auditory features are encoded preattentively in modality-specific, functionally segregated cortical areas (i.e., "feature maps") but are suppressed by tonic activity in the PPC (cf. Treisman & Gormican, 1988). The PPC also suppresses activity of the superior colliculus so that reflexive covert and overt orienting is inhibited. Localized activity in the pulvinar nucleus suppresses this tonic PPC activity at one location, enhancing all of the sensory features at that location, allowing them to be properly combined into a perceptual object. Thus, the pulvinar nucleus engages attention at one location by suppressing the inhibitory PPC activity (mechanism), thereby disinhibiting the sensory features at the attended location (consequence). When an auditory or visual cue appears, the pulvinar's suppression of the PPC is removed at the cued location as is PPC's suppression of the superior colliculus. The former operation disengages attention (i.e., reestablishes PPC suppression of feature maps) and the latter operation allows a new centre of superior colliculus activity to shift attention to a new location. Finally, the pulvinar's suppression of the PPC is restored at the cued location, re-engaging attention there. Consequently, the sensory features of a subsequent target stimulus appearing at the validly cued (attended) location are enhanced relative to those of a target stimulus appearing at an invalidly cued (unattended) location. Of course, in 96 F R O N T A L C O R T E X A> ;? A1 • A2 > MGN t C N ^ S O ^ I C COCHLEA AUDITORY SYSTEM POSTERIOR PARIETAL PULVINAR NUCLEUS DEEP SC VISUAL SC L Z Z SPATIAL ATTENTION MECHANISM V4 •IT V1 • V3 I. ; LGN 1 j - RETINA VISUAL SYSTEM Figure 9. Interactions between a hypothetical spatial attention mechanism and the visual and auditory "what" systems. Excitatory pathways are shown as open circles and inhibitory pathways are shown as filled circles. Note, only a few functionally important connections are shown. LGN = lateral geniculate nucleus, VI etc. = areas of visual cortex, CN = cochlear nucleus, SO = superior olive, IC = inferior colliculus, MGN = medial geniculate nucleus, A l etc. = areas of auditory cortex. The broken connection between the posterior parietal and superior colliculus indicates other brain areas (e.g., substania nigra pars reticulata) mediating the pathway. 97 order for such a mechanism to be entirely supramodal, the various attentional operations (disengage, shift, engage) must be accomplished by the multimodal neurons in the PPC, superior colliculus, and pulvinar nucleus, such that precisely the same operations are performed in order to shift attention within vision and audition. An important feature of this model is that it differentiates clearly between the mechanism and the consequences of spatial attention orienting. The mechanism consists of three interacting brain structures that are known to play important roles in visual spatial attention, whereas the consequences take place in modality-specific auditory and visual cortical areas. This leads to a specific prediction about the crossmodal interactions between auditory and visual cues and targets: A strongly supramodal mechanism should influence the activity of neurons in the same modality-specific brain areas that are involved in the sensory or perceptual processing of the target stimulus regardless of the modality of the cue. Experiments 5 and 6 sought to test this prediction. 9 BACKGROUND TO ELECTROPHYSIOLOGICAL EXPERIMENTS Several recent technological advances permit the investigation of the neural processes that are involved in spatial attention orienting. Experiments 5 and 6 used one electrophysiological measure, called the event-related brain potential (ERP), to determine whether spatial attention orienting in response to uninformative auditory and visual cues influences the sensory encoding of a target presented in a different modality. Before discussing the specific hypotheses in more detail, the ERP method and its application to selective attention 98 research is described in this section. 9.1 Event-related Brain Potentials (ERPs) The recording of ERPs has been by far the most successful and widely used human physiological measurement technique owing to its high temporal resolution, relative ease of use, and general accessibility. Because ERPs are usually recorded by electrodes placed on the scalp, the technique provides a noninvasive means of measuring brain activity from humans while they perceive external stimuli, make decisions, and generate overt responses. 9.1.1 ERP Recording and Measurement An ERP is an electrical change that arises from neural activity in association with some evoking stimulus or event. Such electrical changes reflect the flow of ions across cellular membranes during synaptic activity. In order to be apparent at the surface of the scalp, a large population of neurons having a similar orientation must be synchronously active (Nunez, 1981). In this situation, the individual electrical fields sum to yield a dipolar field that, if orientated properly, passes through the intervening fluids and tissue to produce a current measurable at the scalp. These currents are measured by placing a pair of electrodes on the scalp and amplifying the voltage difference between the two electrode locations. In this way, any fluctuations that are common to both electrodes, such as those produced by electrical lines or gross muscle movements, are canceled. Typically, in multichannel recordings, each channel records the difference between one scalp electrode and a reference electrode. This is called the common 99 reference recording procedure. The common reference site is chosen so that it is relatively uninfluenced by the neural electricity of interest. Unfortunately, because there is no scalp location that is totally inactive with respect to brain electricity, the EEG measured from the sites of interest usually consists of some fluctuations from that are due to the reference site. Some commonly used reference sites include the mastoids, earlobes, and nose. The measurement of an ERP begins with the specification of an epoch of EEG that is time-locked to the event of interest. This epoch contains voltage fluctuations that are specifically related to the event (i.e., the "signal") as well as voltage fluctuations that are unrelated to it (i.e., the "noise"). Because the signal is generally much smaller than the noise, it is difficult to distinguish the ERP from the background EEG activity within any single epoch. For this reason, signal processing techniques are required to extract the signal from the noise. The most common signal processing technique involves repeated presentations of a stimulus so that the ERP can be averaged over many trials. The background activity that is not time-locked to the event cancels out over repeated trials, thereby revealing the underlying response. This procedure assumes that the noise changes from trial to trial whereas the actual response to each stimulus presentation is the same. In this ideal situation, the noise will be inversely proportional to the square root of the number of samples in the average (see Picton, Lins, & Scherg, 1995). Another technique for increasing the signal-to-noise ratio involves limiting the frequencies included in the ERP. Such filtering works best when the frequency content of the signal differs substantially from that of the noise. The resulting ERP waveform consists of several positive and negative peaks that are 100 related to various aspects of sensory, cognitive, and motor processing. These peaks are described in terms of their polarity and either their latency or ordinal value. For example, the negative peak that occurs 100 ms after the appearance of an auditory event is called the N100 or the NI because it is the first large negative peak in the auditory-evoked ERP waveform. The latencies of such peaks are usually expressed as the time interval between the stimulus and the maximum value of the peak. Amplitudes are measured with respect to the mean voltage within some interval preceding the appearance of the stimulus (baseline-to-peak method) or to the amplitude of some other peak in the waveform (peak-to-peak method). 9.1.2 Making Inferences from ERPs The voltage that is recorded at any given instance by a pair of electrodes located on the surface of the scalp reflects the activity of several different sources in the brain. Similarly, the voltage at different scalp locations often reflects the activity of the same underlying sources. Both of these problems arise because the electrical fields produced within the brain propagate through brain and scalp tissues to quite distant locations. Thus, a peak in the ERP waveform measured over some scalp location might reflect (1) the activity of a single neural generator located near that location, (2) the activity of a single neural generator located far from the electrode site, or (3) the activity of multiple generators located in various regions of the brain with fields that sum to be maximal at a particular latency and scalp location. These ambiguity makes it difficult to interpret the ERP deflections directly. Instead, researchers must identify separable components of the ERP waveform on the basis of physiological and psychological 101 criteria (for a review, see Coles and Rugg, 1995). Some researchers define an ERP component in terms of the particular aspect of information processing that it reflects (Donchin, Ritter, & McCallum, 1978), whereas others define it more in terms of the underlying physiological source (e.g., Naatanen and Picton, 1987). Although both approaches stress the importance of appropriate experimental manipulations to determine which feature of the ERP varies predictably as a function of some independent variable (e.g., stimulus probability), the first, more psychological, approach is at least potentially separate from the underlying neural generator. In practice, however, both physiological and psychological criteria are often used simultaneously in the identification of ERP components. For example, Donchin et al. (1978) suggested that a component should be defined by its polarity and scalp distribution, which implicate a particular neural source, as well as its latency and sensitivity to experimental manipulation, which implicate a psychological function. This combined approach to component identification is directly related to the two major goals of cognitive ERP research: to identify components in the ERP waveform that are related to particular information processing functions and to localize their generators in the brain. Several well-known ERP components are shown in the idealized auditory-evoked waveform in Figure 10. The peaks labeled I-V are auditory brainstem responses that reflect processing along the auditory nerve and subcortical relay nuclei. Following these very short-latency responses are several cortical responses including the "mid-latency" components (N0, P0, N a , Pa, Nb), the "late transient" components (Pl, N l , P2), and the later, task-related components (Nd, N2, P3, SW). The shortest latency components are obligatorily elicited by the appearance 102 S T I M U L U S T I M E ( m s e c ) O N S E T Figure 10. Idealized event-related potential to an auditory stimulus plotted on a log-time scale to show the auditory brainstem responses (peaks I-V), the mid-latency cortical components (N0, P0, N a , Pa, Nb), the late transient cortical components (Pl, Nl , P2), and the late task-related components (Nd, N2, P3, SW). Reprinted from Hillyard (1993). im en 103 of an appropriate stimulus and, for the most part, depend on physical properties such as stimulus tensity and modality. These exogenous components can be contrasted with longer-latency dogenous components that are determined primarily by the task-induced goals of the observer (Donchin et al., 1978). This useful distinction is oversimplified because most sensory-related components are influenced to some extent by cognitive factors (e.g., attention) and most late components are influenced by physical properties of the stimulus. It is important to note that a substantial portion of neural activity does not produce electrical fields that are measurable from the surface of the scalp (Vaughan & Arezzo, 1988). Distant electrical fields will not be produced by weak or asynchronous activity, or by the activity of neurons that are aligned suboptimally. Consequently, whereas one can make strong conclusions about differential brain activity when ERP differences are obtained, one cannot make strong conclusions about the lack of differential brain activity when ERP differences are not obtained (Rugg & Coles, 1995). This problem is exacerbated when ERPs are measured from relatively few scalp locations because substantial effects might exist but simply go unnoticed. In general, null findings are interpretable only when there are strong pre-existing hypotheses about the possible size of an effect and when a sufficiently dense electrode array is used. 9.1.3 Why use ERPs to Study Attention? As the previous experiments demonstrate, behavioural measures provide useful information about the mechanisms and consequences of attention. There is however an obvious advantage of physiological measures such as ERP recordings for clarifying the neural basis of 104 attention. Because ERPs reflect moment to moment fluctuations in identifiable brain areas, they can provide critical information about the relative sequencing of neural events that is difficult to infer from traditional behavioural measures or from other, less temporally precise, neuroimaging techniques. The ERP technique thus provides complementary information that can be used to inform psychological theories of attention. Consider, for example, the issue of whether attention influences the transmission of sensory information at early or late stages of information processing. Theories of early selection propose that selective attention controls the transmission of sensory information at the initial stages of processing where stimulus features are registered and perceptual representations are constructed (e.g., Downing, 1988). In comparison, theories of late selection propose that selective attention acts at later, post-perceptual stages by biasing responses or allocating decision processes in favor of the attended input (e.g., Shaw, 1980). There is now convincing evidence that spatial attention can modulate both early "exogenous" and late "endogenous" components, thereby providing evidence that selection can occur at several stages of processing (Luck et al., 1994). These ERP findings are briefly reviewed in the remainder of this section (for more complete reviews, see Hillyard, Mangun, Woldorff, & Luck, 1995; Naatanen, 1992). 9.2 Attentional Modulation of Visual ERPs The electrophysiological consequences of spatial attention have been examined extensively in the visual domain. The major finding of these studies is that attending to a spatial location produces amplitude enhancements of three sensory-evoked components of the ERP 105 elicited by visual stimuli at the attended location: the posterior PI (80-140 ms), anterior NI (130-190 ms), and posterior NI (140-200 ms) components (e.g., Eason, Harter, & White, 1969; Mangun & Hillyard, 1990, 1991; Mangun, Hillyard, & Luck, 1993; Neville & Lawson, 1987; Rugg, Milner, Lines, & Phalp, 1987). Such amplitude modulations occur without appreciable changes in latency and are observed when attention is either voluntarily sustained over a long series of trials (e.g., Mangun et al., 1993) or shifted on a trial-by-trial basis in response to a symbolic cue (e.g., Mangun & Hillyard, 1991). The earliest attention-sensitive component, the posterior PI, appears to reflect neural processing in the vicinity of the fusiform gyrus (extrastriate area V4; Clark & Hillyard, 1996; Heinze, Mangun, Burchert, Hinrichs, Scholz, Miinte, Gos, Scherg, Johannes, Hundeshagen, Gazzaniga, & Hillyard, 1994). The neural generators of the attention-sensitive negative components have been more difficult to identify because of their broad distributions over the scalp. The scalp topography of the posterior NI reveals several different foci indicating that it might have multiple extrastriate cortical generators. The scalp topography of the anterior NI reveals a voltage maximum over frontal cortex but its broad distribution over the scalp is consistent with a forward-pointing generator located within occipito-temporal or occipito-parietal cortex. Thus, based on the latencies and distributions of the early PI and NI components, researchers have suggested that attention can modulate the flow of information in several different areas of extrastriate visual cortical areas (for a recent review, see Mangun, 1995). A few studies have recently examined the effects of direct cues on the ERPs elicited by subsequent visual targets (Anllo-Vento, 1995; Eimer, 1994a; Hopfinger & Mangun, 1998; McDonald et al., in press). In one, subjects made identity-based choice responses to a visual target that was preceded by a spatially uninformative visual cue (Hopfmger & Mangun, 1998). At short (67-267 ms) SOAs, targets appearing at the validly cued location elicited faster responses and a larger Pl component than did targets appearing at the invalidly cued location. This Pl effect was remarkably similar to those observed in sustained attention and symbolic cueing paradigms indicating that stimulus-driven and goal-driven spatial attention mechanisms might have the same consequences on the flow of information within extrastriate visual cortex. Similar enhancements of the sensory-evoked ERP components have been reported at longer (700 ms) SOAs following the presentation of a spatially informative visual cue (Anllo-Vento, 1995). Finally, reductions of the posterior Pl and P2 components have been observed at longer (> 500 ms) SOAs when the cue is spatially uninformative (McDonald et al., in press). One consideration that affects the interpretation of these studies is that sensory interactions between cue and target are likely to be different on valid-cue trials than they are on invalid-cue trials, and thus the ERP effects of direct cues might reflect sensory rather than attentional processes. Depending on many situational parameters, sensory processes might enhance or suppress perceptual processing of stimuli appearing at the cued location whether or not attention is engaged there. On the one hand, neurons stimulated by a cue might continue to respond above their background rate until the target appears, resulting in more vigorous responding to the target (sensory summation). On the other hand, such neurons might be responding below their background rate when the target appears, resulting in less vigorous responding to the target (sensory refractoriness). In light of these potential sensory effects, 107 additional information is required in order to interpret direct cue effects in terms of attentional processes (cf. McDonald et al., in press). 9.3 Attentional Modulation of Auditory ERPs Most of the ERP studies of auditory selective attention have involved dichotic listening procedures in which listeners are required to attend to tones presented in one ear and ignore tones presented in the other ear. The relevant and irrelevant tones typically differed from each other in spectral frequency as well as location so as to make them easily discernable. Recent studies of this type show that attention can influence relatively early stages of auditory processing. The earliest attention effect consists of a more positive waveform between 20 and 50 ms post-stimulus for attended tones than for unattended tones (Woldorff & Hillyard, 1991). Attempts to localize the source of this "P20.5o effect" using combined ERP and event-related magnetic field (ERF) recordings indicate that it is generated in the primary auditory cortical areas. This effect reflects an initial attentional selection based on the physical properties of the stimuli and is consistent with a sensory gain mechanism of attention. Subsequent portions of the ERP are negatively displaced for attended stimuli in relation to unattended stimuli (Hansen & Hillyard, 1980; Naatanen, 1990). This negative difference (Nd) consists of two portions. The early Nd (Nde) is largest at frontocentral scalp sites and reaches a maximum between the Nl and P2 peaks. The late Nd (Nd,) is largest at frontal scalp sites and reaches a maximum 300-350 ms after the onset of the auditory stimulus. There is some theoretical and empirical disagreement about whether the Nd e includes an amplification of an exogenous Nl component or subcomponent 108 (Hillyard, Hink, Schwent, & Picton, 1973) or whether it reflects separate processes that are activated only by the attended stimulus (Naatanen & Michie, 1979). The former hypothesis is consistent with a sensory gain mechanism of attention that modulates the flow of sensory information, whereas the later hypothesis would be suggestive of some other attentional mechanism. However, there is general agreement that the Nd, is caused by a separate endogenous process, called the processing negativity (Naatanen et al., 1978), which may be generated in frontal cortex or in some deeper structure (Giard, Perrin, Pernier, & Peronnet, 1988). In particular, the Nd, might reflect the maintenance and rehearsal of an attentional trace, an actively formed neuronal representation of the physical features (e.g., location, frequency) of the attended stimulus (Naatanen, 1990; Naatanen & Michie, 1979). All of the above mentioned studies used variations of the sustained attention paradigm, in which listeners attend to tones in the relevant ear over a long period of time. Recently, however, other studies have examined the effects of auditory spatial attention using the spatial cueing paradigm, in which the location of the target is cued on a trial-by-trial basis (e.g., Schroger, 1993, 1994; Schroger & Eimer, 1993, 1997). In these experiments, a symbolic cue that was presented at fixation indicated the most likely side of the target. Consistent with the results obtained in the sustained attention experiments, auditory targets elicited more negative ERPs on valid-cue trials than on invalid-cue trials. An early Nd (Ndl) was largest at parietal electrode sites and had a peak latency of about 160 ms (i.e., after the auditory NI). Importantly, its scalp distribution differed from the Nd e observed in sustained attention paradigms suggesting that it might reflect different selection processes. A later Nd (Nd2) was largest at central and frontal electrode sites 109 and had a peak latency of about 300 ms. Based on these findings, Schroger and Eimer (1997) concluded that spatial selection in trial-by-trial cueing situations occurs at "intermediate" stages of information processing rather than at very early (sensory) or very late (response) stages. 9.4 ERP measurement of crossmodal attention Previous ERP studies of crossmodal attention have addressed two separate issues. One issue is whether the performance benefits caused by attending to a specific modality (cf. Posner et al., 1978) arise from changes in modality-specific brain areas or supramodal ones (Alho, Woods, & Algazi, 1994, Alho, Woods, Algazi, & Naatanen, 1992; Eimer & Schroger, 1998; Hackley, Woldorff, & Hillyard, 1990; Woods, Alho, & Algazi, 1992, 1993). In one such study, Woods et al. (1992, Experiment 2) presented random sequences of visual and auditory stimuli and told subjects to respond to infrequent targets in one modality or the other. The primary comparisons were between the ERPs in one modality when that modality was attended or unattended. Attention to the auditory stimuli resulted in enhancements to the auditory ERPs over central scalp sites, including an early enhanced negativity (the "Nda", or negative auditory difference) and a later enhanced positivity (the "Pda", or positive auditory difference). Attention to the visual stimuli resulted in enhancements to the visual ERPs over more posterior scalp sites, including an early occipital Pdv (positive visual difference) and a later temporal NcL. (negative visual difference). Importantly, the scalp distributions of these attention effects suggest that attending to a specific modality results in processing modulations in modality-specific cortical regions (Woods et al., 1992; see also Eimer & Schroger, 1998). These conclusions are consistent 110 with the findings of an earlier PET investigation where directing attention toward a specific modality was found to enhance the regional cerebral blood flow within several modality-specific cortical regions (Roland, 1982). The second issue is whether directing attention to a particular spatial location can influence the processing of stimuli in an irrelevant modality (Eimer & Schrbger, 1998; Hillyard et al., 1984). If separate, modality-specific mechanisms are involved in spatial attention orienting then the direction of spatial attention should have no effect on the ERP waveforms elicited by stimuli in an irrelevant modality. Alternatively, if supramodal mechanisms are involved in spatial attention orienting then the direction of spatial attention should have a substantial effect on the ERP waveforms elicited by stimuli in an irrelevant modality. Hillyard et al. (1984) tested these predictions in a sustained attention experiment in which random sequences of auditory and visual stimuli were presented at the same spatial locations 30° to the left and right of fixation. Subjects in a "visual group" were instructed to attend to the visual modality and to respond to an infrequent target flash whenever it appeared at the attended location and subjects in an "auditory group" were instructed to attend to the auditory modality and to respond to an infrequent target tone whenever it appeared at the attended location. Hillyard et al. (1984) found that the anterior Nl component of the ERP to the visual stimuli were larger at the attended location than at the unattended location, regardless of whether subjects attended to the visual or auditory modality. Similarly, the ERPs elicited by the auditory stimuli were more negative at the attended location than at the unattended location, regardless of whether subjects attended to the auditory or visual modality. In both cases, however, the attention effects were attenuated when I l l the stimulus appeared in the irrelevant modality. Based on these findings, Hillyard et al. (1984) concluded that goal-driven spatial attention orienting is neither entirely modality specific nor entirely supramodal (see also Eimer & Schroger, 1998). 10 AIMS OF ELECTROPHYSIOLOGICAL EXPERIMENTS The final two experiments investigated the crossmodal interactions in stimulus-driven spatial attention orienting by recording brain electricity from subjects participating in crossmodal spatial cueing experiments. One goal of these experiments was to examine, for the first time, the electrophysiological consequences of stimulus-driven spatial attention orienting when the cues and targets appear in different modalities. As reviewed in the previous section, there is convincing evidence that intramodal visual and auditory spatial attention can, under some conditions, influence the neural activity in modality-specific brain areas that are involved in sensory or perceptual processing, leading several researchers to conclude that selective processing of the external world is mediated at relatively early stages of the perceptual systems. The present ERP experiments tested whether crossmodal spatial attention can similarly influence the neural activity in modality-specific brain areas, particularly when stimulus-driven mechanisms are involved. For example, can a spatially uninformative auditory cue affect the neural responses to a subsequent visual target in modality-specific visual brain areas? Similarly, can a spatially uninformative visual cue affect the neural responses to a subsequent auditory target in modality-specific auditory brain areas? These questions were addressed in Experiments 5 and 6, respectively. 112 A related goal of the final two experiments was to address the modality specificity of the stimulus-driven spatial attention orienting mechanism(s) using electrophysiological measures. Although the RT data from Experiments 1 to 4 suggest that such mechanisms might be strongly supramodal, it is unlikely that any single behavioural measure can adequately determine the modality specificity of the spatial attention mechanism. In particular, because RT measures do not specify in exact detail the level of processing at which the attentional effects take place, it is difficult to determine whether the symmetrical cue effects between auditory and visual stimuli on behavioural performance are really equivalent, as would be predicted by a strongly supramodal mechanism, or whether they differ in ways that would indicate that they arise from different underlying mechanisms. Given the different ecological arguments for the various crossmodal cue effects (cf. Spence & Driver, 1997; Ward, 1994), it is quite conceivable that auditory and visual spatial cues influence processing of different-modality targets at qualitatively different stages. The ERP technique can help address this issue because it provides information about the relative sequencing of neural events and about the possible brain areas that are involved. Accordingly, this technique was used in Experiments 5 and 6 in order to distinguish between the various hypotheses concerning the modality-specificity of stimulus-driven spatial attention orienting. Some previous ERP investigations of crossmodal attention have indicated that a supramodal mechanism should produce identical effects regardless of the modality involved. For example, Woods et al. (1992) proposed that "a single, supramodal attentional system should produce similar ERP manifestations for analogous attentional operations in auditory and visual 113 modalities" (p. 341). They argued that the modality-specific ERP modulations that they observed for intramodal auditory and visual attention, which were presumably generated in auditory and visual cortices, respectively, ruled out the possibility that attentional selection of a modality depends on a single, supramodal mechanism. However, it is unclear why a supramodal mechanism would influence processing only within modality-nonspecific (i.e., multimodal) brain areas. It is perhaps more conceivable that supramodal mechanisms would influence the neural activity in modality-specific cortical areas rather than modality-nonspecific ones. In particular, the hypothetical mechanism illustrated in Figure 9 orients attention multimodally whereby activation of the mechanism by either an auditory or visual stimulus produces a relative enhancement of perceptual processing within both modalities. On this basis, the present study assumed that a supramodal mechanism should modulate processing in functionally similar areas of the auditory and visual systems whereas modality-specific mechanisms might modulate processing in functionally different brain areas. Functionally similar brain areas are defined herein as analogous areas of the visual and auditory modalities. For example, if spatial attention influences the neural activity in secondary visual areas, then it should also influence the neural activity in secondary auditory areas. Because visual cues can, under some conditions, influence processing of visual targets in extrastriate cortex, the strong supramodal mechanism hypothesis predicts that an auditory cue can also influence the processing of visual targets in extrastriate cortex. Similarly, because auditory cues can, under some conditions, influence processing of auditory targets in auditory cortex, the strong supramodal mechanism hypothesis predicts that a visual cue can also influence 114 the processing of auditory targets in auditory cortex. Of course, such effects might occur later than in intramodal visual cases because extra time might be required to translated the different-modality stimuli into a common framework. However, if functionally similar effects fail to occur entirely, then spatial attention orienting cannot be strongly supramodal under those particular conditions. Experiments 5 and 6 tested whether spatially uninformative auditory and visual cues produce functionally similar or functionally different crossmodal ERP effects under the conditions that where used in Experiments 1 and 2. 11 ELECTROPHYSIOLOGICAL METHODS 11.1 E E G Recording The EEG was recorded by 30 tin electrodes mounted in an elastic cap (ElectroCap International). The electrodes were positioned at frontal/central (FP1, FPz, FP2, F7, F3, Fz, F4, F8, FC3, FC4), temporal/central (T7, C3, Cz, C4, T8), parietal/central (CP3, CP4, P3, Pz, P4), and temporal/parietal/occipital (P7, P03, POz, P04, P8, P07, 01, OZ, 02, P08) scalp locations (American Electroencephalographic Society, 1991). An electrode placed at the centre of the right mastoid bone was used as reference for all other electrodes. The horizontal EOG was recorded by electrodes placed 1 cm lateral to the external canthi of both eyes. Electrode impedances were kept below 5 kQ. The EEG and EOG signals were amplified with a 0.1 -100 Hz (-12 dB/oct; 3-dB attenuation) bandpass, continuously digitized at a rate of 256 samples per second, and stored on disk for off-line averaging. 11.2 Artifact Rejection and Averaging The continuous EEG/EOG data files were scrutinized for artifacts prior to averaging. The experimenter discarded trials that were contaminated by horizontal eye movements or blinks, defined roughly as ±30 pV deflections on the horizontal EOG channel, and blinks, defined roughly as ±30 uV deflections at FP1. Trials contaminated by muscle activity or amplifier blocking were also discarded. The waveforms from the remaining trials were then averaged for each subject in 1400-ms epochs that started 200 ms before the target stimulus. The averages were then digitally low-pass filtered (30Ffz) to remove high-frequency noise produced by muscle movements and external electrical sources. 11.3 Statistical Analyses The EEG was averaged separately for all combinations of location (left and right) and validity (valid and invalid), resulting in four ERPs for each subject at each electrode site. The amplitudes of some of the most prominent deflections were then quantified as either the mean or largest peak (when clearly discernable) within a specified latency. Both mean and peak amplitude measures were determined with respect to the mean voltage within a 200-ms pre-target baseline period. Attention effects on the amplitude measures were analyzed in a separate repeated measures ANOVAs with location, validity, and recording hemisphere (when appropriate) as within-subject factors. These statistical analyses were performed at a small number of electrode sites to control for possible type I errors. Specific comparisons were then made using Bonferroni t-tests. 116 12 EXPERIMENT 5 Experiment 5 examined the effects of a spatially uninformative auditory cue on the ERPs elicited by a subsequent visual target. The separate mechanisms hypothesis predicts that the auditory cue should have no influence on the processing of the visual target. This outcome seems unlikely given the results of Experiments IA, IB, 3A, and 3B, where significant facilitatory cue effects were observed. In contrast, the supramodal mechanisms hypothesis predicts that the auditory cue should have a strong influence on the processing of the visual target. According to a strongly supramodal account, such as the one described earlier (Figure 9), the auditory cue should modulate processing in extrastriate cortical areas because spatially uninformative visual cues are known to modulate processing in those areas (Hopfinger & Mangun, 1998; McDonald et al., in press). 12.1 Method Subjects Twelve students (7 females and 5 males) from the University of British Columbia participated in Experiment 5. All subjects were between the ages of 18 and 28 (mean age = 21 years), were right-handed, and reported having normal hearing and normal or corrected-to-normal vision. Procedure and Design As in Experiments 1A and IB, subjects were required to respond to a peripheral visual 117 target following a spatially uninformative visual cue. However, in this experiment, the SOA varied randomly between 100 and 300 ms and between 900 and 1100 ms (uniform distributions). In order to reduce the total number of trials, central cues were omitted from the experimental design and the proportion of centre target (NO-GO) trials was reduced from 33 percent to 20 percent. Thus, following completion of the practice trials, each subject participated in 400 GO trials and 100 NO-GO trials separated by 1-min rest periods into 25 blocks. The testing session lasted approximately 45 min. Removal of ERP overlap A pilot study involving ERP recordings to the auditory cue stimulus indicated that the auditory cue was sufficiently intense to elicit an ERP that could overlap and distort the ERP to the subsequent visual target, particularly at short cue-target SOAs. Several steps were thus taken in order to control for the possibility that the ERP differences between valid-cue and invalid-cue conditions were caused by such distortion. First, the SOA was randomly varied across 200-ms ranges so that the ERP responses elicited by the cue would partially cancel out during averaging. This SOA "jitter" has been shown to approximate a low-pass filter that removes all of the peaks with periods that are shorter than the jitter width (Woldorff, 1993). The 200-ms range was chosen because it was long enough to filter out some prominent components of the auditory cue ERP, and it was short enough to produce substantial stimulus-driven cue effects. Second, the adjacent-response (Adjar) filter procedure (Woldorff, 1993) was used to estimate and subtract the residual cue ERP overlap from the target-elicited ERP on short-SOA trials. This process was 118 done noniteratively (Adjar Level 1) because the averaged cue ERPs on long SOA trials provided a relatively undistorted estimate of the cue ERPs on short SOA trials. The residual cue ERP overlap was then estimated for each subject by convolving their cue ERP with the relevant SOA event distribution. The estimated residual cue ERP overlap was then subtracted from the target ERP for each subject to yield relatively undistorted target ERP waveforms. Statistical Analyses The EEG was averaged separately for all combinations of validity (valid and invalid), target location (left and right), and SOA (100-300 and 900-1100), resulting in eight ERPs for each subject at each electrode site. The effects of the experimental variables on the amplitude measures for the Pl (90-160 ms), Nl (160-210 ms), and Nd2 (200-400 ms) components were then assessed at lateral posterior sites in separate repeated-measures ANOVAs with validity, target location, SOA, and recording hemisphere as factors. In addition, the effects of the experimental variables on the mean amplitude within the anterior Nd2 (200-400 ms) and late positive difference (Pdl, 420-520 ms) latency ranges were analyzed at midline anterior sites in separate repeated-measures ANOVAs with validity, target location, and SOA as factors. The RT data were analyzed in a repeated-measures ANOVA with validity, target location, and SOA as factors. 12.2 Results Behavioral Results 119 All of the subjects in Experiment 5 performed the visual spatial discrimination task with over 90% accuracy. Responses occurred on only 3.4% of the NO-GO trials. Approximately 1% of the GO trials were discarded prior to analysis due to excessive eye movement and an additional 2.3% was discarded because the RTs were below 100 ms or above 1500 ms. Median RTs were calculated separately from the remaining data for each subject in each of the nine validity * SOA conditions. The intersubject means of these median RTs and the corresponding error rates are shown in Table 5. The difference between RTs to targets appearing to the left (466 ms) and right (468 ms) of the fixation point was not significant and there were no significant interactions involving target location [all Fs < 1]. However, there was a significant main effect of SOA [F(l,l 1) = 11.2, p = 0.007], with subjects slower to respond at the 100-300 ms SOA (479 ms) than at the 900-1100 ms SOA (457 ms). There was also a significant main effect of validity [F(l,l 1) = 6.7, p = 0.03], with subjects faster to respond on valid-cue trials (463 ms) than on invalid-cue trials (471 ms). This difference was larger at the 100-300 ms SOA than at the 900-1100 ms SOA, giving rise to a significant validity x SOA interaction [F(l,l 1) = 12.4,p = 0.005]. Pairwise comparisons between the mean RTs for valid and invalid cue trials at each of the SOAs showed that the mean cue effect was significant at the 100-300 ms SOA [18 ms;p{\ < u < 35) = .95] but not at the 900-1100 ms SOA [-1 ms;/>(-18 < p. < 16) = .95]. There were no significant main effects or interactions in the analysis of the error data. Inspection of Table 5 indicates that there were no consistent trends in the error data. Although subjects did make slightly more errors on valid-cue trials than invalid-cue trials for targets 120 Table 5 Mean Response Time (M; in milliseconds), Standard Error (SE), and Percent Errors (%E) as a Function of Cue Validity and Stimulus Onset Asynchrony (SOA) in Experiment 5.+ SOA Cue Target Cue 100 - 300 900- 1100 Effect Location Type M SE %E M SE %E A-V Left Valid 467 24 0.8 455 23 0.6 Invalid 487 24 0.6 454 21 0.6 Right Valid 471 24 0.4 458 20 0.4 Invalid 488 21 0.6 457 19 0.4 Total Valid 469 24 0.6 457 22 0.5 Invalid 487 23 0.6 456 20 0.5 fKey to abbreviations: A-V = auditory - visual 121 appearing on the left side of fixation, this was offset by the reverse trend for targets on the right. Thus, as in Experiments 1 to 4, the facilitatory cue effect observed at the 100-ms SOA cannot be attributed to a shift in the decision criterion. ERP Results The adjar-corrected ERPs elicited by the visual target in the 100-300 ms SOA condition are shown in Figure 11, and the (uncorrected) ERPs elicited by the visual target in the 900-1100 ms SOA condition are shown in Figure 12. In both SOA conditions, the ERPs were characterized by multiple spatially and temporally overlapping components. At posterior sites (e.g., P07/8), the ERP waveform included a Pl peak around 120 ms after target onset and an Nl peak around 180 ms after target onset. These peaks were larger at scalp sites contralateral to the target stimulus because each visual hemifield projects to the contralateral side of the visual cortex. At more anterior sites (e.g., FC3/4), the ERP waveform included a symmetrically distributed Nl peak around 160 ms after target onset. Finally, there was a broadly distributed P3 component that was largest at parietal scalp locations (e.g., P3/4). Removal of ERP distortion. Figure 13 displays the original (uncorrected) ERPs elicited by the visual target in the 100-300 ms SOA condition, along with the corresponding residual cue response that was estimated and removed. A substantial residual cue response did occur despite the 200-ms SOA jitter, and thus the uncorrected target-elicited ERPs contained overlapping activity generated by the cue as well as the target. However, the adjar procedure was determined 122 IPSI LATERAL MIDLINE CONTRALATERAL Figure 11. Event-related potentials (ERPs) to visual target stimuli preceded 100-300 ms by spatially uninformative auditory cues in Experiment 5. The waveforms shown were corrected by the adjacent response (Adjar) filter to remove distortion caused by the cue ERP (see text for details). Following the adjar procedure, the ERPs were averaged across 10 subjects and across left and right targets. The solid line represents the ERP on valid-cue trials and the dashed line represents the ERP on invalid-cue trials. 123 IPSILATERAL MIDLINE CONTRALATERAL FP1/2 FPZ FP1/2 Figure 12. ERPs to visual target stimuli preceded 900-1100 ms by spatially uninformative auditory cues in Experiment 5. The waveforms shown were averaged across 10 subjects and across left and right targets. The solid line represents the ERP on valid-cue trials and the dashed line represents the ERP on invalid-cue trials. 124 to be effective at removing most of this overlapping activity because the adjar-corrected ERPs resembled the undistorted ERPs obtained at the 900-1100 ms SOA. Moreover, the main effects of interest, described below, were observed in the original waveforms as well as in the adjar-corrected waveforms indicating that the adjar filter procedure did not artificially create any ERP differences. Cue validity effects on ERPs to visual targets. As can be seen in Figure 11, there were substantial cue validity effects on the ERPs elicited by the visual target in the 100-300 ms SOA condition. The most prominent effect was as a larger negativity on valid-cue trials than on invalid-cue trials beginning approximately 200 ms after the onset of the target and continuing for 200-250 ms. This Nd was reflected by a statistically significant main effect for cue validity in the 200-400 ms interval at midline fronto-central [Fz: F(l , l 1) = 9.1,/? = 0.01; Cz: F(l,l 1) = 24.1,/; < 0.0005] and lateral occipital [P07/8: F(l , l 1) = 16.2,/? = 0.002] scalp locations. Although present for both left and right targets, the Nd was larger for left versus right targets at some sites [Fz: F(l , l 1) = 5.2,/? = 0.04; P07/8: F(l , l 1) = 4.3,/? = 0.06]. At posterior sites, the Nd was distributed over scalp sites contralateral to the target stimulus [location x validity x hemisphere: F(l , l 1) = 16.4,/? = 0.002], whereas, at anterior sites, it was more widely distributed and symmetrical, and was followed by a positive difference in which the mean ERP amplitude in the 400-500 ms interval was more positive on valid-cue trials than on invalid-cue trials [Fz: F(l , l 1) = 5.5,/? = 0.04; Cz: F(l , l 1) = 7.3,/? = 0.02]. Similar but smaller ERP differences were observed in the 900-1100 ms SOA condition, as one might expect given the absence of any cue validity effects on RT performance (Figure 12). As for the Nd, there was a significant SOA x 125 ADJAR FILTERING PROCESS APPLIED TO THE LEFT VISUAL TARGET ERPS Figure 13. The original ERPs elicited by the visual target at the Cz site (left), along with the estimated residual cue responses (middle) and adjar-corrected ERPs (right). All waveforms were averaged across 10 subjects. The solid line represents the ERP on valid-cue trials and the dashed line represents the ERP on invalid-cue trials. Note that cue validity differences are observed in both the uncorrected and corrected waveforms. 126 location x validity x hemisphere interaction at P07/8 [F(l,l 1) = 5.2,p = 0.04] and a marginally significant SOA x validity interaction at Fz [F(l,l 1) = 3.7,p = 0.08]. There were no significant main effects of cue validity on the amplitudes of the shorter-latency PI and NI peaks at posterior sites [Fs < 1]. However, there was a significant location x validity interaction for the posterior PI [P07/8: F(l,l 1) = 17.5,/? = 0.002; P7/8: F(l,l 1) = 13.2, p = 0.004], reflecting the fact that the PI was larger on valid-cue than on invalid-cue trials for right-field targets but was smaller on valid-cue trials than on invalid-cue trials for left-field targets. A similar location x validity interaction was observed for the posterior NI [P07/8: F(l,l 1) = 5.2,/? = 0.04; P03/4: F(l,l 1) = 8.5,/? = 0.01], reflecting the fact that the NI was larger on valid-cue than on invalid-cue trials for left-field targets but was smaller on valid-cue trials than on invalid-cue trials for right-field targets. Electrooculograms In this experiment, trials that were contaminated by deviations in the horizontal EOG were discarded prior to the analyses in order to eliminate the confounding effects of overt orienting, particularly in the direction of the auditory cue. However, because EOG cannot reliably detect very small (< 2°) changes in eye position, it is possible that subjects made small eye movements on some of the trials that were analyzed. Thus, the grand-averaged EOG recordings were examined during the cue-target interval to determine whether there were any consistent trends for subjects to look in the direction of the cue. The EOG deflections failed to exceed ± 2 pV indicating that the deviation of the eyes away from the fixation light was small. 127 On the basis of EOG recordings from additional subjects, who made eye movements to lights located 1.15° left and right of the central fixation LED, it is estimated that the subject's eye position was within 0.15° of the central LED during the entire cue-target interval. 12.3 Discussion Despite the small changes in the experimental conditions, noted earlier, the behavioural results in this experiment replicated the short-SOA facilitatory effect that was observed in Experiments IA and IB. Specifically, in the short (100-300 ms) SOA condition, subjects made faster responses to the visual target when it appeared at the location of the auditory cue (valid-cue trials) than when it occurred on the opposite side of fixation (invalid-cue trials). This effect was observed for both left and right targets and was accompanied by several changes in the target-elicited ERP waveforms. As expected, both the behavioural and ERP cue validity effects were reduced or absent in the longer SOA condition. The ERP effects observed in the present crossmodal experiment differed from the standard effects observed in previous studies of intramodal visual spatial attention. In intramodal visual spatial attention experiments, the early Pl and Nl components are typically larger for stimuli at attended locations than for stimuli at unattended locations. Although such effects are larger and more robust in sustained visual attention paradigms, they have been observed in several transient visual attention experiments using symbolic (e.g., Eimer, p994b; Mangun & Hillyard, 1991) and direct (Hopfinger & Mangun, 1998) visual cues. For example, in Hop finger and Mangun's (1998) study, a visual target that was preceded by a spatially uninformative visual 128 cue elicited a significantly larger PI component on valid-cue trials than on invalid-cue trials. Notably, this effect occurred when the cue-target SOA was short (67-267 ms) and was accompanied by faster response times on valid-cue trials than on invalid-cue trials. These findings are consistent with a sensory gain mechanism of attention that amplifies the information flow within extrastriate visual cortex originating from the attended area of the visual field (LaBerge, 1995; Mangun & Hillyard, 1990). On this basis, Hopfinger and Mangun (1998) concluded that "when attention is reflexively attracted by a sensory event, cortical visual processing is subsequently altered." However, in the present crossmodal experiment, a spatially uninformative auditory cue failed to influence the early, sensory-evoked components of the visual evoked ERP. This does not necessarily indicate that the auditory cue failed to influence the information processing in visual cortex but it does suggest that any such effect probably occurred after the initial sensory processes in those areas were completed. The earliest effects of the auditory cue began around 200 ms after the appearance of the visual target and were manifested as relatively long-lasting Nds. Namely, the ERPs elicited by the visual target were more negative on valid-cue trials than on invalid-cue trials starting around 200 ms. One such Nd was observed over lateral occipital cortical areas and thus might reflect modulations of relatively late processes within extrastriate visual cortex. Such crossmodal effects might conceivably occur only after sufficient time has allowed for input to extrastriate visual cortex from higher polysensory areas (e.g., the PPC). This interpretation is broadly consistent with the neural-specificity theory of attention that proposes that visual attention is accomplished by facilitating neuronal activity within feature-specific visual pathways (Harter & 129 Aine, 1984). According to this theory, stimuli that are processed by the facilitated neuronal population (e.g., neurons responding to stimuli at the attended location) elicit a negative shift that Harter and Aine (1984) called a selection negativity. This selection negativity is believed to occur after the initial sensory-evoked ERP activity under some conditions and to involve the tectopulvinar-parietal pathway when selection is based on spatial location. One criticism of the neural-specificity theory is that modulations of a sensory processing should affect the earlier components of the target-elicited ERP rather than producing an entirely new component (Naatanen, 1986, 1992). However, this criticism does not apply to the present crossmodal selection negativity because there is reason to expect longer-latency ERP modulations when the cue and target are in different modalities. Thus, the present ERP findings provide some converging support the neural-specificity theory and extend it by showing that a selection negativity can occur (1) when the cue and target are in different modalities, and (2) when stimulus-driven, rather than goal-driven, mechanisms of attention are involved. As can be seen in Figure 11, the Nd in the 200-400 ms interval also extended anteriorly at midline sites. Although a deep dipolar source might possibly generate both the frontocentral and lateral occipital Nds, the two effects most likely arise from different neural generators. This conclusion is supported by the observation that, whereas the frontocentral Nd was widely distributed and symmetrical, the lateral occipital Nd was narrowly distributed and contralateral. The frontocentral Nd might have arisen from a forward-pointing dipolar source in parietal cortex, a dipolar source in frontal cortex, or a deeper dipolar source that generated electrical fields that spread to the frontal scalp. Indeed, the frontocentral Nd appears to be similar to the Nd, that 130 occurs in sustained auditory attention tasks and that arises in part from modality nonspecific sources (Giard et al, 1988). Thus, like the Ndl, the frontocentral Nd effect observed in the present experiment might reflect the maintenance of an attentional trace of the attended (i.e., validly cued) target or the further processing of the attended target. 13 EXPERIMENT 6 Experiment 6 examined the effects of a spatially uninformative visual cue on the ERPs elicited by a subsequent auditory target. If stimulus-driven spatial attention orienting mechanisms are strongly supramodal, then the crossmodal cue validity effects on the target ERPs should be functionally similar to those observed in Experiment 5. In particular, the visual cue should produce effects that are functionally similar to the Nds observed over lateral occipital sites and frontocentral sites in the previous experiment. Thus, one might expect to observe differences between the ERPs on valid-cue and invalid-cue trials that are generated in modality-specific auditory brain areas (functionally similar to the lateral occipital Nd) and in modality-nonspecific brain areas (functionally similar to the frontocentral Nd). Such effects might be Nds themselves, or they might be somewhat different, depending on the characteristics of the neural generators involved. For example, a modulation of any sensory response in auditory cortex might be functionally similar to the selection negativity over lateral occipital cortex (cf. Harter & Aine, 1984). Alternatively, if stimulus-driven spatial attention orienting mechanisms are modality-specific but interactive, then functionally different cue validity effects should be observed in the present experiment. 131 13.1 Method Subjects Ten students (4 females and 6 males) from the University of British Columbia participated in Experiment 5. All subjects were between the ages of 19 and 21 (mean age = 21 years), were right-handed, and reported having normal hearing and normal or corrected-to-normal vision. Procedure and Design As in Experiments 2A and 2B, subjects were required to respond to peripheral auditory targets following spatially uninformative visual cues. However, in this experiment, the SOA varied randomly between 100 and 500 ms (uniform distribution) in order to focus on the facilitatory cue effect at relatively short SOAs and to reduce ERP distortion caused by overlapping responses to successive stimuli. In order to reduce the total number of trials, central cues were omitted from the experimental design and the proportion of centre target (NO-GO) trials was reduced from 33 percent to 20 percent. Thus, following completion of the practice trials, each subject participated in 400 GO trials and 100 NO-GO trials separated by 1-min rest periods into 25 blocks. The testing session lasted approximately 45 min. Removal of ERP overlap There were fewer potential problems of ERP overlap in Experiment 6 than in Experiment 5 because the visual cue stimulus elicited a smaller ERP response than did the auditory cue 132 stimulus. Nevertheless, in order to control for the possibility that the ERP differences between valid and invalid cue conditions were caused by such distortion, the SOA was randomly varied across a large interval so that the ERP responses elicited by the cue would partially cancel out during averaging. A 100-500 ms interval, was chosen here because substantial cue effects were observed at both the 100-ms and 500-ms SOAs in Experiment 2. The Adjar fdter procedure was not used because the SOA interval was wide enough to remove most of the distortion from the cue ERP. Statistical Analyses The EEG was averaged separately for all combinations of validity (valid and invalid), target location (left and right), and SOA (100-300 and 900-1100), resulting in eight ERPs for each subject at each electrode site. The effects of the experimental variables on the mean amplitude within the Nde (100-200 ms), Ndl (200-300 ms), and Pdl (400-500 ms) latency ranges were analyzed at midline anterior sites in separate repeated ANOVAs with validity and target position as within-subject factors. An additional ANOVA was done on the mean amplitude in the 90-120 ms interval at temporal (T7/T8) sites to assess the effects of the experimental variables on the ERP activity generated in the auditory cortex. Lastly, additional analyses were done in two consecutive intervals (200-300 ms; 300-500 ms) of the ERP waveforms to assess the effects of the experimental variables on the mean ERP amplitudes at lateral posterior sites. The recording hemisphere was used as a within-subject factor for the ANOVAs done at lateral sites. 133 13.2 Results Behavioral Results All of the subjects performed the visual spatial discrimination task with over 90% accuracy. Responses occurred on only 2.5% of the NO-GO trials. Fewer than 1% of the GO trials were discarded prior to analysis due to excessive eye movement and an additional 1.7% was discarded because the RTs were below 100 ms or above 1500 ms. Median RTs were calculated separately from the remaining data for each subject in each of the eight validity x SOA conditions. The intersubject means of these median RTs and the corresponding error rates are shown in Table 6. The difference between RTs to targets appearing to the left (429 ms) and right (422 ms) of the fixation point was not significant [F(l,9) = 1.2;p = 0.3]. Also, none of the interactions involving target location were significant [all Fs <= 1]. However, as in all previous experiments, there was a significant main effect of SOA [F(l,9) = 52.9,p < 0.0005], with subjects slower to respond at the 100-300 ms SOA (444 ms) than at the 300-500 ms SOA (406 ms). There was also a significant main effect of validity [F(l,9) = 13 A,p = 0.005], with subjects faster to respond on valid-cue trials (412 ms) than on invalid-cue trials (438 ms). This difference was somewhat larger at the 100-300 ms SOA than at the 900-1100 ms SOA, giving rise to a significant interaction between SOA and validity [F(l,9) = 6.0,p = 0.04]. However, pairwise comparisons between the mean RTs for valid and invalid cue trials at each of the SOAs showed that the mean cue effect was significant at both the 100-300 ms SOA [30 ms; p(\2 < p < 48) = .95] and the 900-1100 ms SOA [20 ms;p(2 < u < 38) = .95]. 134 Table 6 Mean Response Time (M; in milliseconds), Standard Error (SE), and Percent Errors (%E) as a Function of Cue Validity and Stimulus Onset Asynchrony (SOA) in Experiment 6.+ SOA Cue Target Cue 100 - 300 300 - 500 Effect Location Type M SE %E M SE %E V-A Left Valid 431 25 0.4 398 24 1.2 Invalid 463 30 1.0 423 28 0.8 Right Valid 427 27 0.8 395 23 0.6 Invalid 455 27 1.4 411 28 0.8 Total Valid 429 26 0.6 397 24 0.9 Invalid 459 29 1.2 417 28 0.8 Key to abbreviations: V-A = visual - auditory 135 The analysis of the error data failed to reveal any significant main effects or interactions. However, inspection of Table 6 indicates that subjects made fewer errors on valid-cue trials than on invalid-cue trials in the 100-300 ms SOA condition. Thus, the facilitatory cue effect observed in this condition cannot be explained in terms of a shift in the decision criterion. The reverse trend occurred for targets on the left side of fixation in the 300-500 ms SOA condition, leaving open the possibility of a criterion shift explanation, but this speed-accuracy trade off was offset for targets on the right side of fixation. ERP Results The grand-averaged ERPs to the auditory target, shown in Figure 14, were characterized by multiple spatially and temporally overlapping components. At anterior sites, the waveforms included a widely and symmetrically distributed NI peak around 100 ms after target onset, a P2 peak around 200 ms after target onset, and a N2 peak around 280 ms after target onset. Previous studies indicate that both the NI and P2 peaks have multiple generators including modality specific ones in the auditory cortices and modality nonspecific ones in frontal motor or premotor areas (Naatanen, 1992; Naatanen & Picton, 1987). A bimodal negative peak was observed at temporal (T3/4) sites with the first negative peak around 90 ms (N90), the second negative peak around 150 ms (N150), and an intermediate positive-going wave around 105 ms (P105). These temporal peaks are similar to those giving rise to the temporal "T-complexes" (Wolpaw & Penry, 1975), which are believed to be generated in auditory association cortex in the superior temporal gyrus (Naatanen & Picton, 1987). Lastly, a symmetrically distributed late positive deflection 136 IPSILATERAL MIDLINE CONTRALATERAL FP1/2 FPZ FP1/2 Figure 14. Event-related potentials (ERP) to auditory target stimuli preceded 100-500 ms by spatially uninformative visual cues in Experiment 6. The waveforms shown were averaged across 12 subjects and across left and right targets. The solid line represents the ERP on valid-cue trials and the dashed line represents the ERP on invalid-cue trials. 137 (LPD) was observed at posterior sites. Although this peak might be related to the P3 component, its parieto-occipital maximum was posterior to the typical parieto-central P3 maximum. As one might expect given the strong cue validity effects on RT performance, there were noticeable cue validity effects on the ERPs elicited by the auditory target. As can be seen in Figure 14, the P105 peak was slightly larger on valid-cue trials than on invalid-cue trials over contralateral temporal (T3/4) sites but this effect was not statistically significant. In contrast, the mean amplitude during the Nde latency range, which began around the latency of the midline Nl peak, was actually smaller (less negative) on valid-cue trials than on invalid-cue trials at Fz [F(l,9) = 5.6,/? = 0.04] but not at Cz [F(l,9) = 1.8,/? = 0.21]. Surprisingly, there was no cue validity effect on the mean amplitude within the Ndl latency range at midline frontal or central sites [Fs < 1]. However, substantial cue validity effects were observed at lateral posterior sites (P07/8 and P7/8). Within the 200-300 ms interval, the ERP waveforms were substantially more positive on valid-cue trials than on invalid cue trials at ipsilateral sites [P7/8: F(l,9) = 48.3,/? < 0.005; P07/8: F(l,9) = 37.0,/? < 0.005]. As can be seen in Figure 14, the N2 peak that is present at anterior sites in both valid-cue and invalid-cue conditions is reduced or altogether absent at ipsilateral posterior sites in the valid-cue condition. This indicates that the ERP difference in the 200-300 ms interval was most likely due to an enhanced positivity on valid-cue trials that overlaps the N2 peak at ipsilateral posterior sites. A similar positive difference was observed at ipsilateral posterior sites in the 300-500 ms interval [P7/8: F(l,9) = 42.5,/? < 0.005; P07/8: F(l,9) = 25.0,/? = 0.001]. However, unlike the earlier interval, a polarity reversal occurred in the 300-500 ms interval, where the ERP waveform was more positive on valid-cue trials than on 138 invalid-cue trials in the ipsilateral hemisphere and more negative on valid-cue trials than on invalid-cue trials. This suggests that cue validity might have affected different neural generators in the 200-300 ms and 300-500 ms intervals. Lastly, cue validity influenced the mean ERP amplitude within the Pdl latency range. This effect was widely distributed over anterior sites, although it was statistically larger at sites that were ipsilateral to the target stimulus [F3/4: F(l,9) = 22.7,p = 0.001]. The latency and broad scalp distribution of this is effect closely resembled the late positive difference that was observed in Experiment 5. Electrooculo grams The grand-averaged EOG recordings were examined during the cue-target interval to determine whether there were any consistent trends for subjects to look in the direction of the cue. As in the previous experiment, the EOG deflections failed to exceed ± 2 uV indicating that the deviation of the eyes was quite small, indicating that the subject's eye position was within 0.15° of the central LED during the entire cue-target interval. 13.3 Discussion The behavioural data obtained in this experiment replicated the facilitatory cue effect that occurred in Experiments 2A and 2B. Specifically, subjects made faster responses to the auditory target when it appeared at the location of the visual cue (valid-cue trials) than when it occurred on the opposite side of fixation (invalid-cue trials). This effect was observed for both left and 139 right targets and was accompanied by several changes in the target-elicited ERP waveforms. However, apart from the late positive difference over the frontal scalp, these effects did not appear to be functionally similar to those observed in Experiment 5. The major effect of cue validity on the target-elicited ERPs was an enhanced positivity on valid-cue trials over ipsilateral electrode sites, which began around 200 ms after the appearance of the target and continued for at least 300 ms. The later portion of this positive difference might have been related to the P3 component, which is believed to reflect categorical perceptual processes that are at least partially modality-nonspecific. However, P3 differences are usually broadly and symmetrically distributed over parietal-central electrode sites, whereas the positive difference in this experiment was ipsilaterally distributed over more posterior sites. Although it is conceivable that this positive difference, particularly the early portion, might have been generated in some higher auditory brain area, this is obviously very speculative. Additional experimentation and source analyses are required to determine the possible neural generator(s) of the ipsilateral positive differences. Interestingly, there were no Nds between the ERPs on valid-cue and invalid-cue trials in the present experiment. Such effects do occur in intramodal auditory (e.g., Schroger, 1993, 1994; Schroger & Eimer, 1993, 1997) and visual (Eimer, 1993, 1994b, 1996) experiments involving symbolic cues. 14 GENERAL DISCUSSION Previous research has investigated the modality specificity of covert spatial attention 140 orienting by assessing the behavioural interactions between cues and targets presented in different combinations of modalities. This research has led to a consistent picture of spatial attention orienting within vision, audition, and touch (intramodal attention orienting), as well as between most of these modalities (crossmodal attention orienting). However, the relatively few studies of crossmodal spatial attention orienting using auditory and visual stimuli have reported inconsistent effects. In studies of stimulus-driven attention orienting, null effects have been reported by some authors (Mondor & Amirault, 1998) and asymmetrical effects of both possible types have been reported by others (Spence & Driver, 1997; Ward, 1994; Ward et al., 1998b). In some cases, visual cues influenced RTs to auditory targets but auditory cues failed to influence RTs to visual targets.(Ward, 1994; Ward et al., 1994b), whereas, in other cases, the opposite asymmetry occurred (Spence & Driver, 1997). The above results have important implications for the modality specificity of covert spatial attention orienting because any null effect implies that the mechanisms for stimulus-driven spatial attention orienting may not be entirely supramodal (see Figure 1). Mondor and Amirault (1998) argued on the basis of their null effects that there are separate mechanisms for stimulus-driven spatial attention orienting in vision and audition. In contrast, Spence and Driver (1997) argued on the basis of their asymmetrical effects that such modality-specific mechanisms are "linked" so that auditory events engage the visual attention orienting mechanism but that visual events fail to engage the auditory attention orienting mechanism. Finally, Ward (1994) argued on the basis of the opposite asymmetry that a supramodal mechanism might control stimulus-driven spatial attention orienting in some tasks (e.g., auditory) but not in others (e.g., 141 visual). The crossmodal effects observed in these prior studies are all vulnerable to alternative explanations. First, Mondor and Amirault's (1998) null effects might have been due to the fact that the choice response was based on target identity (e.g., colour, sound frequency) rather than target location. As noted earlier, several previous studies have demonstrated that auditory stimuli do not typically elicit spatial attention orienting in nonspatial tasks (e.g., McDonald & Ward, in press). Second, stimulus-response compatibility effects, such as response priming (e.g., Simon & Small, 1969), might have contributed to Ward's (1994) crossmodal effects because subjects in his experiments made explicit left-right discrimination judgments. So, for example, subjects in the auditory task might have responded faster on valid-cue trials than on invalid-cue trials simply because the lateralized cue activated the correct response. Spence and Driver (1997) even outlined a response-priming explanation for the null auditory-on-visual effect, although a detailed analysis of it reveals that it fails to explain all of Ward's results (cf. Ward et al., 1998b). Third, overt orienting might have contributed to Spence and Driver's (1997) auditory-on-visual cue effect and to Ward's (1994) visual-on-auditory cue effect because eye position was not monitored in either case. Given these ambiguities, the present study examined the modality specificity of covert spatial attention orienting and IOR by assessing further the interactions between auditory and visual cues and targets. The implicit spatial discrimination task was used in each experiment to eliminate the possible influence of response priming by the cue and to ensure the use of location-sensitive neurons in making responses (cf. McDonald & Ward, in press). Additionally, eye 142 movements were monitored in some experiments (Experiments IB, 2B, 3B, 4B, 5 and 6) but not in others (Experiments 1 A, 2A, 3A, 4A) to examine whether eye-position monitoring would influence any of the observed cue effects. Finally, the auditory and visual cues were always spatially uninformative so that any obtained cue effects would be attributable to mechanisms of stimulus-driven attention orienting. Five important results emerged from the present experiments. First, strong symmetrical facilitatory cue effects were obtained between auditory and visual stimuli when the cue and target modalities were both certain. The auditory-on-visual cue effect was largest at the 100-ms SOA, although it was also significant at the 500-ms SOA. It was also slightly smaller in monitored subjects (Experiment IB) than in unmonitored subjects (Experiment 1 A) at least at the 100-ms SOA but this difference was not statistically reliable. The visual-on-auditory cue effect lasted for at least 500 ms after the appearance of the cue, which is somewhat longer than the stimulus-driven cue effects obtained in previous intramodal visual and auditory experiments. However, this cue effect was substantially smaller in the monitored subjects (Experiment 2B) than in the unmonitored subjects (Experiment 2A) at the 500-ms SOA, suggesting that overt orienting might have contributed to the long-SOA results. In contrast, eye-position monitoring had little or no influence on the magnitude of the visual-on-auditory effect at the 100-ms SOA. On the basis of these findings, it is reasonable to conclude that, under these conditions, (1) uninformative auditory cues orient spatial attention covertly and in a stimulus-driven manner so as to facilitate the processing of visual targets appearing at the validly cued location relative to those appearing at other locations, and (2) uninformative visual cues orient spatial attention 143 covertly and in a stimulus-driven manner so as to facilitate the processing of auditory targets appearing at the validly-cued location relative to those appearing at other locations. These conclusions are, of course, contrary to previous claims that auditory or visual cues, or both, fail to affect the processing of targets appearing in a different modality (Driver & Spence, 1998; Mondor & Amirault, 1998; Spence & Driver, 1997; Spence et al., 1998; Ward, 1994). Second, crossmodal IOR was observed at the longer cue-target SOAs when the modality of the target was uncertain. These effects were generally larger in the absence of eye position monitoring than in the presence of eye position monitoring. Auditory cues produced approximately equal amounts of IOR in the A-A and A-V conditions (Experiments 3 A and 3B). In contrast, visual cues produced more IOR in the V-V condition than in the V-A condition, particularly in monitored subjects (Experiments 4A and 4B). Surprisingly, IOR was not apparent in the average results between visual cues and auditory targets when eye position was monitored, raising the possibility that the inhibitory effect arises from separable-but-linked auditory and visual mechanisms. However, an inspection of the individual subjects' cue effects revealed that substantial IOR effects did occur for some subjects, but not others, in both the V-A and A-V conditions. This demonstrates that IOR can sometimes occur between visual cues and auditory targets as well as between auditory cues and visual targets. A third and related finding was the effect of target modality uncertainty on the crossmodal cue effects. To recount, strong facilitatory effects were observed in the absence of IOR when the simplest possible cue and target environments were used, that is, when a single cue was presented from single modality (e.g., vision) and a single target was presented from a 144 single, but different, modality (e.g., audition). These crossmodal facilitatory effects were substantially smaller when the modality of the target was made uncertain and, for many subjects, were followed by IOR at the longest SOA. Such differences suggest that crossmodal cue effects, like intramodal ones, might be governed by strategic control factors (cf. Ward et al., 1998a). Fourth, the auditory-on-visual facilitatory effect was associated with several cue validity effects on the target-elicited ERP waveforms (Experiment 5). One of these effects was a larger negativity on valid-cue trials than on invalid-cue trials over the lateral occipital scalp, consistent with generation in extrastriate visual areas such as V4 and inferior temporal cortex. This effect began after the initial sensory-related ERP components (PI and NI) and it was interpreted as a selection negativity similar to those observed in intramodal visual attention studies (Harter & Aine, 1984). Its relatively late onset most likely reflects input from multimodal brain areas, such as the posterior parietal cortex, which can process stimuli in multiple modalities. Another, possibly separate, effect was a larger negativity on valid-cue trials than on invalid-cue trials over the midline fronto-central scalp. This effect was similar to the late Nd effects that have been observed in several other attention paradigms and, therefore, most likely reflects additional processing of targets presented at the validly cued location. A later positive difference was observed over the fronto-central scalp, although its source and significance were unclear. Fifth, and finally, the visual-on-auditory facilitatory effect was also associated with cue validity effects on the target-elicited ERP waveforms (Experiment 6). However, most of these effects appeared to be functionally different than those in the opposite crossmodal condition. In particular, there were no negative differences observed between valid-cue and invalid-cue trials 145 over the frontal, central, or temporal scalp regions. The primary effect was a larger positive peak on valid-cue trials than on invalid-cue trials over the ipsilateral occipito-parietal scalp beginning approximately 200 ms after the appearance of the target and continuing for several hundred ms. The interpretation of this effect is unclear. On the one hand, it might reflect a modulation of neural activity in some higher auditory, or perhaps multimodal, cortical area. On the other hand, it might reflect a modulation of a component or subcomponent in the P3 family. Another effect was a late positive displacement of valid-cue trials relative to invalid-cue trials over the frontal scalp. This effect did appear to be functionally similar to the late positive displacement that occurred in the opposite crossmodal condition, but its late onset suggests that it did not contribute significantly to the cue validity effects on RT performance. 14.1 Spatial attention orienting: Modality Specific or Supramodal? The RT data alone indicate that, under the present experimental conditions, stimulus-driven spatial attention orienting might involve a single, supramodal mechanism. A hypothetical supramodal mechanism, which was based on the posterior attention system, was outlined in Section 8.3 (also see Figure 9). This model lead to three key predictions. First, it predicted that any sensory event, regardless of the modality, that engages the spatial attention mechanism should influence the processing of visual targets in similar ways. Second, it predicted that any sensory event, regardless of the modality, that engages the spatial attention mechanism should influence the processing of auditory targets in similar ways. Third, it predicted that spatial attention orienting in response to auditory or visual cues should have functionally similar effects 146 on the processing of visual and auditory targets. Thus, on the basis of this last prediction, if spatial attention orienting influences the processing of visual targets in modality-specific visual cortical areas, then it should influence the processing of auditory targets in functionally similar auditory cortical areas. Alternatively, if spatial attention orienting influences the processing of visual targets in higher-order multimodal brain areas, then it should influence the processing of auditory targets in functionally similar, or perhaps identical, higher-order multimodal brain areas. There is little question that goal-driven spatial attention orienting can have functionally similar effects on the processing of visual and auditory stimuli, both at early, modality-specific stages and later, multimodal stages of processing. Spatial attention influences the modality-specific P 2 0. 5 0 peak of the auditory ERP (Woldorff & Hillyard, 1991) and the modality-specific Pl and Nl peaks of the visual ERP (e.g., Mangun et al., 1993), particularly in sustained attention paradigms. In addition, symbolic spatial cues can produce attention-related negative differences in later modality-nonspecific portions of the ERPs to both auditory (Schroger, 1993,1994; Schroger & Eimer, 1993, 1997) and visual (Eimer, 1993, 1994b, 1996) targets. The similarities of these ERP effects supports the view that the goal-driven spatial attention orienting mechanisms are not entirely separate. Eimer and Schroger (1998) examined whether the mechanisms of goal-driven spatial attention orienting are modality-specific or supramodal in two crossmodal ERP experiments using symbolic visual cues. In both experiments, auditory and visual stimuli appeared randomly and with equal probability from the validly cued location or from the invalidly cued location. Subjects were instructed to attend to one modality and to respond to infrequently presented 147 targets at the validly cued location. In the attend-vision condition, visual stimuli appearing at validly cued locations elicited larger Pl and Nl components at lateral occipital sites and more negative ERPs in the Ndl and Nd2 latency ranges at midline sites than did visual stimuli appearing at invalidly cued locations. Similar Nds to visual stimuli occurred at midline sites in the attend-audition condition suggesting that goal-driven spatial attention orienting had similar effects on intermediate visual processing regardless of which modality was relevant to the task. Similar Nd effects were also observed for the auditory targets, both in the attend-audition condition and in the attend-vision condition when the auditory and visual stimuli appeared from the same locations. These findings are generally consistent with the supramodal mechanisms hypothesis. However, they are not generally consistent with the type of supramodal mechanism outlined in 8.3, which modulates processing in modality-specific auditory and visual cortical areas, because the crossmodal ERP effects were centreed over modality-nonspecific cortical areas. Moreover, the Nd2 effects were somewhat larger in the intramodal conditions than in the crossmodal conditions, indicating that some modality-specific mechanisms might also contribute to goal-driven crossmodal effects (cf. Hillyard et al., 1984; Posner et al., 1978). The present study used the ERP technique to address the modality specificity of stimulus-driven spatial attention orienting. Although very few prior ERP studies have examined the electrophysiological consequences of such attention shifts, there is evidence that an uninformative visual spatial cue can amplify the amplitude of the lateral occipital Pl component, which is generated in extrastriate visual cortex (Hopfinger & Mangun, 1998). On this basis, it was predicted that if the mechanisms of stimulus-driven attention orienting are strongly 148 supramodal then an uninformative auditory spatial cue should also amplify the amplitude of the lateral occipital P l component under the appropriate experimental conditions. Such an effect was not observed in Experiment 5. Instead, beginning after the P l and N l components, the target-elicited ERP was more negative on valid-cue trials than on invalid-cue trials. As noted previously, one such negative difference occurred over the lateral occipital scalp. This suggests that, under the present conditions, orienting attention to the location of an uninformative auditory cue affects the neural processing of a subsequent visual target in extrastriate cortex but only after the initial sensory processing of the target stimulus has been done. A possible explanation for the relatively long latency of the crossmodal effect is that it depends on reafferent projections from multimodal cortical neurons, whereas the intramodal effect does not. This implies that the intramodal and crossmodal attention effects might to some extent depend on different mechanisms. This proposal is inconsistent with the strong supramodal position. Similarly, the ERP data obtained in the V-A condition of Experiment 6 also failed to provide convincing evidence for the strong supramodal position. Again, the supramodal position predicted that the visual cue should have influenced processing of the auditory target in modality-specific auditory cortical areas as well as in modality-nonspecific brain areas. The former effect would have been functionally similar to the negative difference observed over the lateral occipital scalp in the A-V condition, whereas the latter effect would have been functionally similar to the ERP differences observed over more anterior scalp regions. Although a late positive difference was observed over anterior scalp regions in both crossmodal conditions, the earlier effects obtained in the V-A condition appeared to be functionally different from those 149 in the A-V condition. In particular, there were no ERP differences over temporal or fronto-central scalp sites that would have reflected differential processing in known auditory cortical areas. Such effects have previously been observed in sustained auditory attention paradigms (e.g., Woldorff & Hillyard, 1991) although we don't know whether these effects occur in uninformative direct auditory cueing studies. The ipsilateral positive difference observed over occipito-parietal areas in the present study might have been generated in some higher-order auditory cortical area, but such effects are usually observed more anteriorly. Differences in the amplitude of the P3 component are also usually observed over more anterior scalp areas. Given its posterior distribution, the ipsilateral positive difference most likely arises in multimodal regions of the parietal cortex. On the basis of the ERP data of Experiments 5 and 6, it seems reasonable to conclude that the mechanisms of stimulus-driven spatial attention orienting are not strongly supramodal, at least under the experimental conditions used in the present study. This is somewhat surprising because symmetrical crossmodal cue effects were observed at short SOAs under these experimental conditions, particularly when the cue and target modalities were both certain. A tentative suggestion is that the mechanism of spatial attention orienting is only weakly supramodal. Similar to the strong supramodal mechanism, a weakly supramodal mechanism would involve the multimodal cells in the PPC, superior colliculus, and pulvinar nucleus, and would be engaged by both visual and auditory stimuli. In principle, however, visual and auditory stimuli might engage such a mechanism differently because modality-specific processes are required to transform the various sensory signals into a common, multimodal frame of reference. 150 Moreover, such a mechanism might have slightly different consequences depending on the manner it was engaged, such that crossmodal effects might occur at later stages of processing than intramodal effects. There are at least two alternatives to the weakly supramodal position. First, the results of the present experiments might arise from a hierarchical attention system that consists of both modality-specific and supramodal orienting mechanisms. This type of hierarchical system was described by Posner (1990). The existence of modality-specific attention mechanisms makes functional sense because attention can be oriented to modality-specific features of auditory (e.g., sound frequency) and visual (e.g., colour) features. However, Posner (1990) speculated further that there might be modality-specific mechanisms for spatial attention orienting in vision, audition, and touch. Still, the hierarchichal position, like the weakly supramodal position, would attribute any crossmodal cue effect to a purely supramodal mechanism such as the one outlined above. Second, the results of the present experiments might arise from separable but reciprocally linked mechanisms for stimulus-driven attention orienting in vision and audition (Figure IB). Just how such links might be instantiated is somewhat unclear at present. Spence and Driver (1997) speculated that they might involve different spatial representations within the superior colliculus. In particular, they hypothesized that stimulus-driven visual attention orienting might arise directly from activity within the purely visual superficial layers of the superior colliculus and that stimulus-driven auditory attention orienting might arise directly from the multimodal deeper layers. Accordingly, any shift of auditory attention would be accompanied by a shift in 151 visual attention, whereas a shift of visual attention could be done independently of auditory attention. However, notice that this proposal actually accounts for the auditory-on-visual cue effect in terms of a shared, multimodal attention mechanism. Specifically, Spence and Driver's (1997) neurophysiological account attempts to explain spatial attention orienting in terms of two neural mechanisms - a modality-specific visual one and a supramodal one - with no apparent links between them (also see Ward, 1994) and attributes the crossmodal cue effects to the supramodal mechanism. Thus, there are several possible accounts for the behavioural and ERP data obtained in the present study. Most of these accounts attribute the observed crossmodal cue effects to some sort of supramodal mechanism; they differ primarily on the existence of other, modality-specific mechanisms. At present, the weakly supramodal account might be preferable to the others on the basis that it is simpler and more readily falsifiable. 14.2 Can a Strongly Supramodal Account be Salvaged? The strongly supramodal account, which postulates that auditory, visual, and tactile stimuli produce identical shifts of attention, cannot account for the functionally different ERP effects observed in the present study. However, the present findings do not rule out the possibility that a strongly supramodal spatial attention orienting mechanism might be engaged in other experimental conditions. Although there were several advantages for using the implicit spatial discrimination task in the present crossmodal experiments, such a task might not be the optimal one for obtaining cue validity effects on the early, modality-specific ERP components. 152 Indeed, attention-related modulations of the Pl and Nl components of the visual ERP have failed to occur in several intramodal visual attention experiments involving symbolic spatial cues. In an experiment by Hillyard, Miinte, and Neville (1985), subjects were required to attend to a symbolically cued location while a series of five visual stimuli was presented randomly at that location and at a location on the opposite side of fixation. Consistent with the ERP effects of sustained visual attention, the lateral occipital Pl elicited by the fifth stimulus in the series was larger on valid-cue trials than on invalid-cue trials. However, a similar Pl enhancement failed to occur for the first four stimuli. Such results indicate that attention modulation of the early visual ERP components are less reliably obtained in spatial cueing experiments than in sustained attention experiments. More recent studies have demonstrated that the effects of symbolic cues on the visual Pl and Nl components depend at least partially on various task characteristics (e.g., Eimer, 1993, 1994b, 1996). The role of the response assignments was examined in a series of experiments by Eimer (1994b). In some experiments, responses were required only to infrequent target stimuli at the validly cued location. These response assignments were roughly equivalent to those used in sustained attention paradigms. In another experiment, responses were required to target stimuli at both validly and invalidly cued locations. Enhanced Pl and Nl components to validly cued stimuli were obtained in the experiments in which invalidly cued stimuli did not require a response and thus could be completely ignored. In contrast, there was no significant Pl validity effect when stimuli at both validly and invalidly cued locations required a response. The Nl validity effect was also substantially reduced in this latter condition. According to Eimer 153 (1994b), these results suggest that different attentional strategies might be employed in the sustained attention paradigm and in the standard spatial cueing paradigm. In the former case, attention can be actively and fully engaged at the expected location whereas, in the latter case, attention might be only weakly engaged at the validly cued location, or perhaps partially divided among the validly and invalidly cued locations. One implication of Eimer's (1994b) findings that is especially pertinent to the present investigation is that the engagement operation of the posterior attentional system - attributed to the pulvinar nucleus of the thalamus - might play a less prominent role in some spatial cueing paradigms than in sustained attention paradigms. Consequently, one can speculate whether evidence for a strongly supramodal spatial attention mechanism could be obtained in different conditions that, like Eimer's (1994b) modified cueing tasks, would allow the subject to fully engage attention at the validly cued location. A related issue is whether any location-based choice RT task would suffice to observe effects of uninformative spatial cues on the ERP components generated in auditory cortical areas. Such areas are organized with respect to spectral frequency and the majority of cells in those areas are insensitive to spatial location. Consistent with these neurophysiological findings, several early auditory cortical ERP components have tonotopically organized generators (e.g., Woods, Alho, & Algazi, 1991). On this basis, it is reasonable to speculate that the stimulus-driven effects of spatial cues on modality-specific auditory ERP components would be better observed in tasks that involve frequency-based judgments. For example, it is possible to modify the implicit spatial discrimination paradigm such that subjects discriminate between low-154 frequency and high-frequency target tones appearing at peripheral locations and ignore all target tones appearing at the central location. An analogous visual task might involve subjects discriminating between visual targets of various colours at peripheral but not central locations. If iminformative spatial cues affected the amplitude of modality-specific ERP components elicited by different-modality targets in such situations, then a strongly supramodal account of spatial attention orienting would be applicable. Clearly, further experiments addressing this issue are needed. 14.3 Strategic Control Factors Based on different patterns of results, Mondor and Amirault (1998) concluded that crossmodal spatial cue effects are dependent on the involvement of goal-driven spatial attention mechanisms. To review their major findings, no significant crossmodal spatial cue effects occurred in an experiment where both the cue and target modalities were uncertain and the cue was spatially uninformative. In contrast, significant crossmodal spatial cue effects did occur in a second experiment where the number of cue-target SOAs was reduced (from two to one), and where the cue was made spatially informative. These methodological changes were made to increase the "predictability" of the cue-target relation and, as argued by Mondor and Amirault, to increase the relative contribution of goal-driven processes. These significant crossmodal cue effects were replicated and extended in a third experiment where the predictive relation between the cue and target locations was maintained and where both the cue and target modalities were made certain. Mondor and Amirault concluded from these findings that there are some links 155 between the neural systems mediating goal-driven attention orienting but not between the neural systems mediating stimulus-driven attention orienting. They did, however, quality this conclusion somewhat by speculating that a small but nonsignificant effect of uninformative auditory cues on visual RTs could have arisen from crossmodal interactions at the level of the superior colliculus. Their major conclusion, however, was that crossmodal cueing effects are primarily dependent upon goal-driven attention orienting mechanisms. The present findings disconfirm Mondor and Amirault's (1998) claim regarding the dependence of crossmodal spatial cue effects on the involvement of goal-driven spatial attention orienting mechanisms because large effects were obtained in the absence of any predictive relation between the cue and target locations. However, the variations of the crossmodal effects across experiments involving spatially uninformative cues indicate that strategic factors do play an important role in stimulus-driven crossmodal attention orienting. The crossmodal facilitatory cue effects that were found in the present study were smaller when the target modality was uncertain than when it was certain, presumably because of differences in the subjects' strategic framing of the tasks. Notice that subjects must avoid making responses to the cue in all spatial cueing experiments. Therefore, the cue causes a certain amount of interference in all such tasks. There is noticeably less interference when it is known in advance that the cue and target will appear in different modalities because the stimuli do not share any features except the irrelevant feature of spatial location. The facilitatory effect of the cue might be relatively large in this situation because subjects have little or no reason to inhibit a shift of attention to the cue. However, the interference of the cue increases when the cue and target can appear in the same 156 modality because, in this case, the stimuli share task-relevant characteristics (e.g., on A-A trials, both stimuli are brief sounds). The facilitatory effect of the cue might be smaller in this situation because subjects actively inhibit shifts of attention to the cue in order to prepare for the appearance of the target. The effect of target modality uncertainty demonstrates that stimulus-driven crossmodal spatial cue effects are under strategic control. During the past decade, there have been several studies on the influence of strategic, or "top-down," factors on the stimulus-driven attention orienting abilities of various visual features (for a recent review, see Egeth & Yantis, 1997). Some have argued that the ability of any stimulus property to capture attention is dependent on the establishment of attentional control settings for that property that reflect the task-induced behavioural goals of the observer (e.g., Folk et al., 1992). In support of this position, Folk et al. (1992) found that a spatially uninformative visual cue affects RTs to a subsequent visual target only when the cue and target are defined by the same feature (e.g., colour or abrupt onset). On this basis, it might actually seem counterintuitive that stronger facilitatory cue effects were observed in the present study when the cue modality was irrelevant to the task (i.e., target modality certain) than when the cue modality was relevant (i.e., target modality uncertain). However, in the present series of experiments, the critical target feature was its location rather than its colour or spectral frequency. Therefore, the present results do not contradict Folk et al.'s (1992) general claim about contingent attentional capture. At this point, it is appropriate to speculate on why audiovisual effects sometimes fail to occur. As discussed by Ward et al. (1998a, 1998b), a combination of strategic and perceptual 157 factors appears to have been responsible. First, Mondor and Amirault's (1998) null effects seem to be consistent with the establishment of attentional control settings because the subjects in that experiment made responses based on the target's colour or spectral frequency. As a result, subjects might have established attentional control settings for these modality-specific features, thereby reducing the attention capturing capabilities of cues in a different modality. With respect to the lack of an auditory-on-visual cue effect, previous experiments show that for auditory spatial cue effects to appear it is crucial that the experimental task engage the proper spatial representations of the cue and target (McDonald & Ward, in press). Thus, auditory cues might fail to influence responses to visual targets when the cue environment is complex, as in Ward's (1994, Ward et al., 1998b) experiments, because subjects do not process the spatial location property of the auditory cues. When cues are uninformative and often conflicting and targets are all visual, it suffices to process the more directly-encoded property of frequency content of the auditory cue in order to register it as a nontarget stimulus and use its appearance as a cue to prepare for the target. The more laborious spatial location processing of the auditory cue is purposely avoided in order to reduce interference of the cue on the target and thus visual spatial attention is not affected by the location of the auditory cue. On the other hand, it is possible that the failure of visual cues to affect responses to auditory targets, as reported by Spence and Driver (1997), arose because their auditory target elevation task was not sensitive to the azimuthal orienting of spatial attention by a visual cue. In other words, when cue and target are relatively far apart on valid-cue trials, as they were in the elevation task, attentional effects can be expected to be weak, whereas the possibility of sensory 158 or perceptual interactions remains (because of centre-surround organization of the relatively large receptive fields of multimodal neurons, cf. Stein & Meredith, 1993). 14.4 Inhibition of Return: Modality Specific or Supramodal? The IOR effect, like the facilitatory effect, appeared to be influenced by strategic control factors in the present experiments. Specifically, whereas there was no evidence for any crossmodal IOR when the target modality was certain, there was evidence for IOR, both between auditory cues and visual targets and between visual cues and auditory targets, when the target modality was uncertain. It is likely that the failure of auditory and visual cues to produce crossmodal IOR when the target modality was certain occurred because subjects did not actively reorient attention away from the cued location, as discussed above. The inclusion of some intramodal targets caused the cue modality to become relevant, and thus, distracting, causing many subjects to orient their attention away from the cued location in order to prepare for the appearance of the target. This interpretation is consistent with several intramodal visual and auditory spatial cueing studies that indicate that IOR does not occur when subjects voluntarily sustain their attention at the cued location. The tentative conclusion that IOR arises at least in part from a supramodal mechanism must be tempered for two reasons. First, crossmodal IOR failed to occur in some subjects, particularly when eye position was monitored. In fact, there was no overall IOR effect in the V-A condition in Experiment 3B, raising the possibility that visual cues do not reliably elicit IOR for auditory targets. Second, the crossmodal IOR effects were substantially larger when eye 159 position was unmonitored than when it was monitored. This indicates that crossmodal IOR might arise from different mechanisms in the presence and absence of eye movements. For example, in the absence of any eye movements, IOR might arise entirely from spatial attention orienting to, and then away from, the cued location. However, when subjects make eye movements in the direction of the cued location, IOR might arise from the joint activity of an spatial attention orienting mechanism and an oculomotor mechanism. Importantly, one of these mechanisms might be supramodal whereas the other might be modality-specific. It is possible that crossmodal IOR might be observed more reliably when attention is reoriented to central fixation by a sensory event occurring there following the presentation of a peripheral cue. Spence and Driver (1998b) showed that auditory cues can produce IOR in simple visual RT tasks when an audio-visual event is presented between cue and target (Experiment 1) or between successive targets in the absence of cues (Experiment 2). Their study was motivated in part by Posner and Cohen's (1984) original argument that IOR might occur more reliably if attention is drawn away from the cued location, either endogenously via task demands (e.g., more targets appearing at fixation) or exogenously via sensory stimulation. It is tempting to generalize Spence and Driver's conclusions to the present crossmodal situations. However, reliable IOR effects have been obtained in every modality tested to date whether or not central reorienting events are used (Maylor, 1985; McDonald & Ward, in press; Rafal et al., 1989; Tassinari & Campara, 1996). Thus, although the central reorienting paradigm is effective in generating inhibitory effects, genuine IOR occurs in vision, audition, and touch even if no manipulation, exogenous or otherwise, is used to summon attention back to the centre of the 160 display. Moreover, the observations of IOR in the absence of a central reorienting event are clearly less susceptible to alternative explanations than are the observations of IOR in the presence of such an event. One potential problem with the central reorienting paradigm that is often overlooked is that the inhibitory effect might arise from perceptual effects such as apparent motion. Given these considerations, it would be both surprising and disappointing if crossmodal IOR depended on the use of central reorienting events. 14.5 Future Cognitive Neuroscience Experiments It is worth noting that asymmetries in crossmodal spatial cue effects were previously taken as prima facie evidence for separate-but-linked spatial attention mechanisms. However, as the above discussion implies, such asymmetries might also arise from strategic manipulation of a single, supramodal mechanism. In fact, the supramodal view and the separable-but-linked view make similar behavioural predictions. On the one hand, the supramodal mechanism view can account for null crossmodal cue effects in terms of strategic manipulations of the spatial attention orienting mechanism. On the other hand, the separate-but-linked view can conceptually account for symmetrical crossmodal cue effects by assuming that attention often shifts simultaneously but independently in different modalities. The differences between the various explanations of these crossmodal cue effects might only be found using other neurophysiological and neuroimaging techniques such as single-cell recording and fMRI. The supramodal mechanisms hypothesis predicts that several multimodal brain structures, such as the PPC, superior colliculus, and pulvinar nucleus should be engaged in 161 intramodal auditory and visual attention tasks as well as in crossmodal attention tasks. In contrast, the separate-but-linked hypothesis predicts that modality-specific brain structures should be engaged during intramodal attention tasks. The inferior colliculus might be involved in shifting auditory attention (Mondor & Zatorre, 1995), whereas the purely visual layers of the superior colliculus might be involved in shifting visual attention (Spence & Driver, 1997). Importantly, the linked mechanisms hypothesis makes the crucial prediction that, because such mechanisms are linked, both modality-specific auditory and visual brain mechanisms should be active during the appropriate intramodal or crossmodal attention-shifting tasks. On the basis of the separate mechanisms given above, a salient visual cue should activate the superifical layers of the superior colliculus (visual attention mechanism) as well as the inferior colliculus (auditory attention mechanism). Similarly, an auditory cue should activate the inferior colliculus as well as the superficial layers of the superior colliculus. The existing neurophysiological evidence indicates that such links do not exist between these specific brain areas. Additional experiments will be necessary to determine whether such links might exist between other modality-specific brain areas (e.g., between separate auditory and visual spatial maps in the thalamus or PPC). Of course, much remains unknown about spatial attention and IOR, both with respect to their neural specificity and with respect to other important issues. The present paper illustrates how future studies might address these issues from a cognitive neuroscience prospective. These studies would provide the insights that would be necessary to develop ideal cognitive neuroscientific models of spatial attention orienting and IOR. The primary goal of such models and of cognitive neuroscience in general is to understand in a more complete way how the mental 162 operations involved in human perception, cognition, and behaviour arise from neural operations. It is possible to specify the critical features that an ideal cognitive neuroscientific model of spatial attention orienting should possess. First, it should accurately describe the pathways that are involved in sensory, perceptual, cognitive, and attentive processing. Existing neuroanatomical evidence indicates that these pathways are quite complex, involving reciprocal connections between several functionally specific brain regions (e.g., Hubel & Weisel, 1977; Zeki, 1993). An attempt was made in the current paper to highlight some of the pathways that might be particularly important for spatial attention orienting in vision and audition (see Figure 1). Second, an ideal model should separate between the neural mechanisms of attention and their consequences (e.g., LaBerge, 1995; Posner, 1995). The mechanisms of attention are presumed to consist of multiple brain regions that each have a unique attentional function, whereas the consequences of attention are believed to occur as relative enhancements of neural activity in other brain areas, including those involved in the processing of sensory features. In the present study, this distinction led to the prediction that a supramodal mechanism would have functionally similar consequences in modality-specific cortical areas. Third, an ideal model should specify how the data obtained in behavioural studies of humans could arise from the neural mechanisms and consequences. Fourth, an ideal model should specify how the neural mechanism can be reconfigured by strategic (top-down) control. Finally, an ideal model is one that can be implemented computationally so that various situations can be simulated. An attempt was made in the present paper to describe a model for stimulus-driven attention orienting that can be engaged by auditory as well as visual sensory events. This model was based on the multimodal cells in the P P C , superior colliculus, and pulvinar nucleus, and was strongly supramodal insofar as it was implemented in the same fashion, producing identical effects, regardless of the particular modalities involved. Whether or not this type of model is viable w i l l depend on data from future neuroanatomical, neurophysiological, and behavioural studies. 164 REFERENCES Alho, K., Woods, D.L., Algazi, A. (1994). Processing of auditory stimuli during auditory and visual attention as revealed by event-related potentials. Psvchophysiology, 31,469-479. Alho, K., Woods, D.L., Algazi, A., & Naatanen, R. (1992). Intermodal selective attention. II. Effects of attentional load on processing of auditory and visual stimuli in central space. Electroencephalography and Clinical Neurophysiology. 82, 356-368. American Electroencephalographic Society (1991). Guidelines for standard electrode position nomenclature. Journal of Clinical Neurophysiology. 8, 200-202. Anllo-Vento, L. (1995). Shifting attention in visual space: The effects of peripheral cueing on brain cortical potentials. International Journal of Neuroscience. 80. 353-370. Banati, R.B. et al., (1997). The functional anatomy of sensory integration in man. Neuroimage. 5, S213. Bedard, M.A., El Massioui, F., Pillon, B., & Nandrino, J.L. (1993). Time for reorienting of attention: A premotor hypothesis of the underlying mechanism. Neuropsychologia, 31. 241-249. Bradshaw, J.L., Howard, M.J., Pierson, J.M., Phillips, J., & Bradshaw, J.A. (1992). Effects of expectancy and attention in vibrotactile choice reaction time tasks. Quarterly Journal of Experimental Psychology. 44A, 509-528. Briand, K.A., & Klein, R.M. (1987). Is Posner's "beam" the same as Treisman's "glue"?: On the relation between visual orienting and feature integration theory. Journal of Experimental Psychology: Human Perception and Performance, 13. 228-241. Broadbent, D.E. (1958). Perception and communication. London: Pergamon Press. Buchtel, H.A., & Butter, C M . (1988). Spatial attention shifts: Implications for the role of polysensory mechanisms using visual and tactile stimuli. Neuropsychologia, 26,499-509. Buchtel, H.A., Butter, C M . , & Ayvasik, B. (1996). Effects of stimulus source and intensity on covert orientation to auditory stimuli. Neuropsychologia. 34. 979-985. Bushnell, M . C , Goldberg, M.E., & Robinson, D.L. (1981). Behavioral enhancement of visual responses in monkey cerebral cortex. I. Modulation in posterior parietal cortex related to selective visual attention. Journal of Neurophysiology. 46, 755-772. 165 Butter, C M . , Buchtel, H.A., & Santucci, R. (1989). Spatial attention shifts: Further eveidence for the role of poly sensory mechanisms using visual and tactile stimuli. Neuropsvchologia. 27. 1231-1240. Cherry, E.C. (1953). Some experiments on the recognition of speech, with one and two ears. Journal of the Acoustical Society of America. 25, 975-979. Clark, V.P., & Hillyard, S.A. (1996). Spatial selective attention effects early extrastriate but not striate components of the visual evoked potential. Journal of Cognitive Neuroscience. 8, 387-402. Clohessy, A:, Posner, M.I., Rothbart, M.K., & Vecera, S.P. (1991). The development of inhibition of return in early infancy. Journal of Cognitive Neuroscience. 3, 345-350. Colavita, F.B. (1974). Human sensory dominance. Perception & Psychophysics. 16, 409-412. Colavita, F.B., & Weisberg, D. (1979). A further investigation of visual dominance. Perception & Psychophysics. 25, 345-347. Coles, M.G.H. & Rugg, M.D. (1995). Event-related brain potentials: An introduction. In M.D. Rugg & M.G.H. Coles (Eds.), Electrophysiology of mind (pp. 1-26). New York: Oxford University Press. Danziger, S., Fendrich, R., & Rafal, R.D. (1997). Inhibitory tagging of locations in the blind field of hemianopic patients. Consciousness and Cognition. 6, 291-307. Donchin, E., Ritter, W., & McCallum, W.C. (1978). Cognitive psychophysiology: The endogenous components of the ERP. In E. Callaway, P. Tueting, & S.H. Koslow (Eds.), Event-related brain potentials in man, (pp. 349-441). New York: Academic Press. Downing, C.J. (1988). Expectancy and visual-spatial attention: Effects on perceptual quality. Journal of Experimental Psychology: Human Perception and Performance. 14, 188-202. Eason, R., Harter, M., & White, C. (1969). Effects of attention and arousal on visual evoked cortical potentials and reaction times in man. Physiology and Behavior, 4, 283-289. Egeth, H.E., & Sager, L.C. (1977). On the locus of visual dominance. Perception & Psychophysics. 22, 77-86. Egeth, H.E., & Yantis, S. (1997). Visual attention: Control, Representation, and time 166 course. Annual Review of Psychology. 48, 69-97. Eimer, M. (1993). Spatial cueing, sensory gating and selective response preparation: An ERP study of visual-spatial orienting. Electroencephalography and Clinical Neurophysiology, 88, 408-420. Eimer, M. (1994a). An ERP study on visual spatial priming with peripheral onsets. Psvchophysiology. 31. 154-163. Eimer, M. (1994b). "Sensory gating" as a mechanism for visuo-spatial orienting: Electrophysiological evidence from trial-by-trial cueing experiments. Perception & Psychophysics. 55, 667-675. Eimer, M. (1996). ERP modulations indicate the selective processing of visual stimuli as a result of transient and sustained spatial attention. Psychophvsiology, 33, 13-21. Eimer, M., & Schroger, E. (1998). Effects of intermodal attention and cross-modal links in spatial attention. Psvchophysiology, 35, 313-327. Eriksen, CW. & Yeh, Y.Y. (1985). Allocation of attention in the visual field. Journal of Experimental Psychology: Human Perception and Performance. 11, 583-597. Farah, M.J., Wong, A.B., Monheit, M.A., & Morrow, L A. (1989). Parietal lobe mechanisms of spatial attention: Modality-specific or supramodal? Neuropsychologia. 27, 461-470. Fisher, G.H. (1964). Spatial localization by the blind. American Journal of Psychology, 77,2-14. Folk, C.L., Remington, R.W., & Johnston, J.C (1992). Involuntary covert orienting is contingent on attentional control settings. Journal of Experimental Psychology: Human Perception and Performance. 18. 1030-1044. Fox, E. (1995). Negative priming from ignored distracters in visual selection: A review. Psychonomic Bulletin & Review, 2. 145-173. Giard, M.-H., Perrin, F., Pernier, J., & Peronnet, F. (1988). Several attention-related waveforms in auditory areas: a topographic study. Electroencephalography and Clinical Neurophysiology. 69. 371-384. Gibson, B.S., & Egeth, H. (1994). Inhibition and disinhibition of return: Evidence from 167 temporal order judgments. Perception & Psychophysics, 56, 669-680. Goldstone, S. (1968). Reaction time to onset and termination of lights and sounds. Perceptual and Motor Skills. 27. 1023-1029. Hackley, S.A., Woldorff, M.G., & Hillyard, S.A. (1990). Cross-modal selective attention effects on retinal, myogenic, brainstem, and cerebral evoked potentials. P s vchophvsio lo gv. 27, 195-208. Hansen, J.C., & Hillyard, S.A. (1980). Temporal dynamics of human auditory selective attention. Psvchophvsiologv. 25, 316-329. Harman, C , Posner, M.I., Rothbart, M.K., Thomas-Thrapp, L. (1994). Development of orienting to locations and objects in human infants. Canadian Journal of Experimental Psychology. 48, 284-300. Harter, M.R., & Aine, C.J. (1984). Brain mechanisms of visual selective attention. In R. Parasuraman & D. R. Davies (Eds.), Varieties of attention (pp. 293-321). New York: Academic Press. Harvey, N. (1980). Non-informative effects of stimuli functioning as cues. Quarterly Journal of Experimental Psychology. 32, 413-425. Hay, J.C., Pick, H.L., Jr., & Ikeda, K. (1965). Visual capture produced by prism spectacles. Psychonomic Science. 215-216. Heinze, H.J., Mangun, G.R., Burchert, W., Hinrichs, H., Scholz, M., Munte, T.F., Gbs, A., Scherg, M., Johannes, S., Hundeshagen, H., Gazzaniga, M.S., & Hillyard, S.A. (1994). Combined spatial and temporal imaging of brain activity during visual selective attention in humans. Nature. 372, 543-546. Hershenson, M. (1962). Reaction time as a measure of intersensory facilitation. Journal of Experimental Psychology. 63, 289-293. Hillyard, S.A., Hink, R.F., Schwent, V.L., & Picton, T.W. (1973). Electrical signs of selective attention in the human brain. Science, 182, 177-180. Hillyard, S.A., Mangun, G.R., Woldorff, M.G., & Luck, S.J. (1995). Neural systems mediating selective attention. In M.S. Gazzaniga (Ed.), The cognitive neurosciences (pp. 665-681). Cambridge: MIT Press. 168 Hillyard, S.A., Munte, T.F., & Neville, H.J. (1985). Visual-spatial attention, orienting, and brain physiology. In M.I. Posner & O.S. Marin (Eds.), Attention and Performance XI (pp. 63-84). Hillsdale, NJ: Erlbaum. Hillyard, S.A., Simpson, G.V., Woods, D.L., Van Voorhis, S., & Munte, T.F. (1984). Event-related brain potentials and selective attention to different modalities. In F. Reinoso-Suarez & C. Ajmone-Marsan (Eds.), Cortical Integration (pp. 395-414). New York: Raven Press. Hopfinger, J.B., & Mangun, G.R. (1998). Reflexive attention modulates processing of visual stimuli in human extrastriate cortex. Psychological Science. 9, 441-447. Howard, LP. (1982). Human visual orientation. New York: Wiley. Howard, LP., & Templeton, W.B. (1966). Human spatial orientation. New York: Wiley. Hubel, D.H., & Wiesel, T.N. (1977). Functional architecture of macaque visual cortex. Proceedings of the Royal Society. London (B). 198. 1-59. Huerta, M.F., & Harting, J.K. (1984). The mammalian superior colliculus: Studies of its morphology and connections. In H. vanegas (Ed.), Comparative neurology of the optic tectum (pp. 687-773). New York: Plenum. James, W. (1890). The principles of psychology. Volume 1. New York: Dover Publications. Jay, M.F., & Sparks, D.L. (1987a). Sensorimotor integration in the primate superior colliculus. I. Motor convergence. Journal of Neurophysiology. 57, 22-34. Jay, M.F., & Sparks, D.L. (1987b). Sensorimotor integration in the primate superior colliculus. II. Coordinates of auditory signals, Journal of Neurophysiology. 57, 35-55. Jay, M.F., & Sparks, D.L. (1990). Localization of auditory and visual targets for the initiation of saccadic eye movements. In M.A. Berkley & W. Stebbins (Eds.), Comparative perception, volume 1: Basic mechanisms (pp. 351-374). New York: Wiley. Johnson, M.H., Posner, M L , & Rothbart, M.K. (1991). Components of visual orienting in early infancy: Contingency learning, anticipatory looking and disengaging. Journal of Cognitive Neuroscience. 3, 335-344. Johnston, W.A. & Dark, V.J. (1986). Selective attention. Annual Review of Psychology, 37, 43-75. 169 Jonides, J. (1981). Voluntary versus automatic control over the mind's eye's movement. In J.B. Long & A.D. Baddeley (Eds.), Attention & Performance IX (pp. 187-203). Hillsdale, NJ: Erlbaum. Jonides, J., & Mack, R. (1984). On the cost and benefit of cost and benefit. Psychological Bulletin. 96, 29-44. Kilgard, M.P., & Merzenich, M.M. (1995). Anticipated stimuli across the skin. Nature. 373. 663. Kinchla, R.A. (1992). Attention. Annual Review of Psychology. 43, 711-742. King, A.J., & Palmer, A.R. (1985). Integration of visual and auditory information in bimodal neurones in the guinea-pig superior colliculus. Experimental Brain Research, 60. 492-500. Kingstone, A., & Pratt, J. (in press). Inhibition of return is composed of attentional and oculomotor processes. Klein, R.M. (1977). Attention and visual dominance: A chronometric analysis. Journal of Experimental Psychology: Human Perception and Performance, 3, 365-378. Klein, R.M., Brennan, M., & Gilani, A. (1987, November). Covert cross-modality orienting of attention in space. Paper presented at the annual meeting of the Psychonomic Society, Seattle, WA. Klein, R.M., Kingstone, A., & Pontefact, A. (1992). Orienting of visual attention. In K. Rayner (Ed.), Eve movements and visual cognition: Scene perception and reading (pp. 46-65). New York: Springer-Verlag. LaBerge, D. (1990). Attention. Psychological Science, I, 156-162. LaBerge, D. (1995). Attentional processing: The brain's art of mindfulness. Cambridge: Harvard University Press. Luck, S.J., Hillyard, S.A., Mouloua, M., Woldorff, M.G., Clark, V.P., & Hawkins, H.L. (1994). Effects of spatial cuing on luminance detectability: Psychophysical and electrophysiological evidence for early selection. Journal of Experimental Psychology: Human Perception and Performance. 20, 887-904. Mangun, G.R. (1995). Neural mechanisms of visual selective attention. 170 Psvchophysiology. 32. 4-18. Mangun, G.R., & Hillyard, S.A. (1990). Electrophysiological studies of visual selective attention in humans. In A.B. Scheibel & A.F. Wechsler (Eds.), Neurobiology of higher cortical function (pp. 271-295). New York: Guilford Press. Mangun, G.R., & Hillyard, S.A. (1991). Modulations of sensory-evoked brain potentials provide evidence for changes in perceptual processing during visual-spatial priming. Journal of Experimental Psychology: Human Perception and Performance, 17, 1057-1074. Mangun, G.R., Hillyard, S.A, & Luck, S.J. (1993). Electrocortical substrates of visual selective attention. In D. Meyer & S. Kornblum (Eds.), Attention and Performance XIV (pp. 219-243). Cambridge: MIT Press. Massaro, D.W. (1998). Perceiving talking faces: From speech perception to a behavioral principle. Cambridge, MA: MIT Press. May, CP., Kane, M.J., & Hasher, L. (1995). Determinants of negative priming. Psychological Bulletin. 118. 35-54. Maylor, E. A. (1983). Components of orienting in visual space. Unpublished doctoral thesis, University of Durham. Maylor, E.A. (1985). Facilitatory and inhibitory components of orienting in visual space. In M.I. Posner & O. Marin (Eds.), Attention and Performance XI (pp. 189-207). Hillsdale, NJ: Erlbaum. Maylor, E.A. & Hockey, R. (1985). Inhibitory component of externally controlled orienting in visual space. Journal of Experimental Psychology: Human Perception and Performance. I i , 777-787. McDonald, J.J. (1998). Is IOR related to attention? Unpublished manuscript. McDonald, J.J., & Ward, L.M. (in press). Spatial relevance determines facilitatory and inhibitory effects of auditory covert spatial orienting. Journal of Experimental Psychology: Human Perception and Performance. McDonald, J.J., Ward, L.M., & Kiehl, K.A. (in press). An event-related brain potential study of inhibition of return. Perception & Psychophysics. Meredith, M.A., Nemitz, J.W., & Stein, B.E. (1987). Determinants of multisensory 171 integration in superior colliculus neurons. I. Temporal factors. Journal of Neuroscience. 10, 3215-3229. Meredith, M.A., & Stein, B.E. (1986a). Spatial factors determine the activity of multisensory neurons in cat superior colliculus. Brain Research, 365. 350-354. Meredith, M.A., & Stein, B.E. (1986b). Visual, auditory and somatosensory convergence on cells in superior colliculus results in multisensory integration. Journal of Neurophysiology, 56, 640-662. Meredith, M.A., Wallace, M.T., & Stein, B.E. (1992). Visual, auditory and somatosensory convergence in output neurons of the cat superior colliculus: Multisensory properties of the tecto-reticulo-spinal projection. Experimental Brain Research, 88. 181-186. Mills, A.W. (1958). On the minimum audible angle. Journal of the Acoustical Society of America. 32. 132-134. Mondor, T.A. & Amirault, K.J. (1998). Effect of same- and different-modality spatial cues on auditory and visual target identification. Journal of Experimental Psychology: Human Perception and Performance. 24, 745-755. Mondor, T.A. & Zatorre, R.J. (1995). Shifting and focusing auditory spatial attention. Journal of Experimental Psychology: Human Perception and Performance, 21, 387-409. Moran, J. & Desimone, R. (1985). Selective attention gates visual processing in the extrastriate cortex. Science. 229. 782-784. Morell, L.K. (1968). Cross-modality effects upon choice reaction time. Psychonomic Science, 11, 129-130. Muller, H.J., & Rabbitt, P.M.A. (1989). Reflexive and voluntary orienting of visual attention: Time course of activation and resistence to interruption. Journal of Experimental Psychology: Human Perception and Performance. 15. 315-330. Naatanen, R. (1990). The role of attention in auditory information processing as revealed by event-related brain potentials. Behavioral and Brain Sciences. 13. 201-288. Naatanen, R. (1992). Attention and brain function. Hillsdale, NJ: Erlbaum. Naatanen, R., & Michie, P.T. (1979). Early selective-attention effects on the evoked potential: A critical review and reinterpretation. Biological Psychology. 8, 81-136. 172 Naatanen, R., & Picton, T.W. (1987). The Nl wave of the human electric and megnetic response to sound: A review and analysis of the component structure. Psvchophvsiology, 24, 375-425. Neill, W.T. (1977). Inhibitory and facilitatory processes in attention. Journal of Experimental Psychology: Human Perception and Performance. 3, 444-450. Neville, H.J., & Lawson, D. (1987). Attention to central and peripheral visual space in a movement detection task: An event-related potential and behavioral study. I. Normal hearing Adults. Brain Research. 405. 253-267. Nunez, P. (1981). Electric fields of the brain. New York: Oxford University Press. Petersen, S.E., Robinson, D.L., & Keys, W. (1985). Pulvinar nuclei of the behaving rhesus monkey: Visual responses and their modulations. Journal of Neurophysiology. 54, 867-886. Pick, H.L., Warren, D.H., & Hay, J.C. (1969). Sensory conflict in judgements of spatial direction. Perception & Psychophysics. 6, 203-205. Picton, T.W., Lins, O.G., & Scherg. M. (1995). The recording and analysis of event-related potentials. In F. Boiler & J. Grafman (Eds.), Handbook of Neuropsychology. Vol 10 (pp 3-73). Amsterdam: Elsevier. Piatt, B.B., & Warren, D.H. (1972). Auditory localization: The importance of eye movements and a textured visual environment. Perception & Psychophysics. 12,245-248. Posner, M.I. (1978). Chronometric explorations of mind. Ney York: Oxford University Press. Posner, M.I. (1980). Orienting of attention. Quarterly Journal of Experimental Psychology. 32, 3-25. Posner, M.I. (1990). Hierarchical distributed networks in the neuropsychology of selective attention. In A. Caramazza (Ed.), Cognitive neuropsychology and neurolinguistics: Advances in models of cognitive function and impairment (pp. 187-210). Hillsdale, NJ: Erlbaum. Posner, M.I. (1995). Attention in cognitive neuroscience: An overview. In M. S. Gazzaniga (Ed.), The cognitive neurosciences (pp. 615-624). Cambridge: MIT Press. 173 Posner, M.I. & Cohen, Y. (1980). Attention and the control of movements. In G.E. Stelmach & J. Requin (Eds.), Tutorials in motor behavior (pp. 243-258). New York: North-Holland. Posner, M.I. & Cohen, Y. (1984). Components of visual orienting. In H. Bouma & D.G. Bouwhuis (Eds.), Attention and Performance X (pp. 531-556). Hillsdale NJ: Erlbaum. Posner, M.L, Davidson, B.J., & Nissen, M.J. (1976a, November). The process of stimulus detection. Paper presented at the annual meeting of the Psychonomic Society, St. Louis, MO. Posner, M.L, Nissen, M.J., & Klein, R.M. (1976b). Visual dominance: An information-processing account of its origins and significance. Psychological Review. 83, 157-171. Posner, M.L, Nissen, M.J., & Ogden, W.C. (1978). Attended and unattended processing modes: The role of set for spatial location. In H.L. Pick & I.J. Saltzman (Eds.), Modes of perceiving and processing information (pp. 137-157). Hillsdale, NJ: Erlbaum. Posner, M.L, Petersen, S.E., Fox, P.T., & Raichle, M.E. (1988). Localization of cognitive operations in the human brain. Science, 240, 1627-1631. Posner, M.L, Rafal, R.D., Choate, L.S., & Vaughan, J. (1985). Inhibition of return: Neural basis and function. Cognitive Neuropsychology, 2, 211-228. Posner, M.L, Raichle, M. (1994). Images of mind. New York: Scientific American Library. Posner, M.L, Walker, J.A., Friedrich, F.J., & Rafal, R.D. (1987). Effects of parietal injury on covert orienting of attention. Journal of Neuroscience. 4, 1863-1874. Pratt, J., Kingstone, A., & Khoe, W. (1997). Inhibition of return in location and identity based choice decision tasks. Perception & Psychophysics, 59, 964-971. Quinlan, P.T. & Bailey, P.J. (1995). An examination of attentional control in the auditory modality: Further evidence for auditory orienting. Perception & Psychophysics, 57, 614-628. Rafal, R.D., Calebresi, P.A., Brennan, C.W., & Sciolto, T.K. (1989). Saccade preparation inhibits reorienting to recently attended locations. Journal of Experimental Psychology: Human Perception and Performance. 15. 673-685. Rafal, R.D. & Henik, A. (1994). The neurology of inhibition: Integrating controlled and automatic processes. In D. Dagenbach & T.H. Carr (Eds.), Inhibitory processes in attention, 174 memory, and language (pp. 1-51). New York: Academic Press Rafal, R.D., Henik, A., & Smith, J. (1991). Extrageniculate contributions to reflex visual orienting in normal humans: A temporal hemifield advantage. Journal of Cognitive Neuroscience. 3, 322- 328. Rafal, R.D., Posner, M.I., Freidman, J.H., Inhoff, A.W., & Bernstein, E. (1988). Orienting of visual attention in progressive supranuclear palsy. Brain. 111. 267-280. Reuter-Lorenz, P.A., Jha, A.P., & Rosenquist, J.N. (1996). What is inhibited in inhibition of return? Journal of Experimental Psychology: Human Perception and Performance. 22, 367-378. Rhodes, G. (1987). Auditory attention and the representation of spatial information. Perception & Psychophysics. 42. 1-14. Robinson, D.L., Bowman, E.M., & Kertzman, C. (1995). Covert orienting of attention in macaques. II. Contributions of parietal cortex. Journal of Neurophysiology, 74, 698-712. Robinson, D.L., & McClurkin, J.W. (1989). The visual superior colliculus and pulvinar. In R.H. Wurtz, & M.E. Goldberg (Eds.), The neurobiology of saccadic eve movements. New York: Elsevier. Roland, P.E. (1982). Cortical regulation of selective attention in man: a regional cerebral blood flow study. Journal of Neurosphysiology. 48, 1059-1078. Roffler, S. (1968). Factors that influence the localization of sound in the vertical plane. Journal of the Acoustical Society of America. 43. 1255-1259. Rugg, M.D. & Coles, M.G.H. (1995). The ERP and cognitive psychology: Conceptual issues. In M.D. Rugg & M.G.H. Coles (Eds.), Electrophysiology of mind (pp. 27-39). New York: Oxford University Press. Rugg, M.D., Milner, A.D., Lines, C.R., & Phalp, R. (1987). Modulation of visual event-related potential by spatial and nonspatial visual selective attention. Neuropsvchologia, 25, 85-96. Schmidt, W.C. (1996). 'Inhibition of return' without visual input. Neuropsvchologia. 34, 943-952. Schroger, E. (1993). Event-related potentials to auditory stimuli following transient shifts 175 of spatial attention in a Go/Nogo task. Biological Psychology, 36, 183-207. Schroger, E. (1994). Human brain potential signs of selection by location and frequency in an auditory transient attention situation. Neuroscience Letters, 173, 163-166. Schroger, E., & Eimer, M. (1993). Effects of transient spatial attention on auditory event-related potentials, Neuroreport. 4, 588-590. SchrOger, E., & Eimer, M. (1997). Endogenous covert spatial orienting in audition: "Cost-benefit" analyses of reaction times and event-related potentials. The Quarterly Journal of Experimental Psychology. 5OA. 457-474. Shaw, M.L. (1980). Identifying attentional and decision-making components in information processing. In R.S. Nickerson (Ed.), Attention and Performance VIII (pp. 277-296). Hillsdale, NJ: Erlbaum. Shelton, B.R., & Searle, C L . (1980). The influence of vision on the absolute identification of sound-source position. Perception & Psychophysics. 28. 589-596. Simion, F., Valenza, E., Umilta, C , & Dalla Barba, B. (1995). Inhibition of return in newborns is temporo-nasal asymmetrical. Infant Behaviour & Development. 18, 189-194. Simon, J.R. (1990). The effects of an irrelevant directional cue on human information processing. In R.W. Proctor & T.G. Reeve (Eds.), Stimulus-response compatibility: An integrated perspective (pp. 31-86). Amsterdam: North-Holland. Simon, J.R., & Small, A.M. JR., (1969). Processing auditory information: Interference from an irrelevant cue. Journal of Applied Psychology. 53, 433-435. Simpson, W.E. (1979). Latency of locating lights and sounds. Journal of Experimental Psychology. 93, 169-175. Sparks, D.L., & Hartwich-Young, R. (1989). The deep layers of the superior colliculus. In R.H. Wurtz & M.E. Goldberg (Eds.), The neurobiology of saccadic eve movements (pp. 213-256). New York: Elsevier. Spence, C.J., & Driver, J. (1994). Covert spatial orienting in audition: Exogenous and endogenous mechanisms facilitate sound localization. Journal of Experimental Psychology: Human Perception and Performance. 20, 555-574. Spence, C.J., & Driver, J. (1996). Audiovisual links in endogenous covert spatial 176 orienting. Journal of Experimental Psychology: Human Perception and Performance, 22,1005-1030. Spence, C.J., & Driver, J. (1997). Audiovisual links in exogenous covert spatial orienting. Perception & Psychophysics. 59. 1-22. • Spence, C.J., & Driver, J. (1998a). Auditory and audiovisual inhibition of return. Perception & Psychophysics. 60. 125-139. Spence, C.J., & Driver, J. (1998b). Inhibition of return following an auditory cue: The role of central reorienting events. Experimental Brain Research. 118. 352-360. Spence, C.J., Nicholls, M.R., Gillespie, N., & Driver, J. (1998). Cross-modal links in exogenous covert spatial orienting between touch, audition, and vision. Perception & Psychophysics. 60, 544-557. Spitzer, H., Desimone, R., & Moran, J. (1988). Increased attention enhances both behavioral and neuronal performance. Science. 240. 338-340. Stein, B.E., Huneycutt, W.S., & Meredith, M.A. (1988). Neurons and behavior: The same rules of multisensory integration apply. Brain Research. 448, 355-358. Stein, B.E., & Meredith, M.A. (1993). The merging of the senses. Cambridge, MA: MIT Press. Stein, B.E., Meredith, M.A., Huneycutt, W.S., & McDade, L. (1989). Behavioral indices of multisensory integration: Orientation to visual cues is affected by auditory stimuli. Journal of Cognitive Neuroscience. I, 12-24. Stelmach, L.B. & Herdman, C M . (1991). Directed attention and the perception of temporal order. Journal of Experimental Psychology: Human Perception and Performance. 17, 539-550. Sternberg, S. & Knoll, R.L. (1973). The perception of temporal order: Fundamental issues and a general model. In S. Komblum (Ed.), Attention and Performance IV (pp. 629-685). New York: Academic Press. Streicher, M., & Ettlinger, G. (1987). Crossmodal recognition of familiar and unfamiliar objects by the monkey: The effects of ablation of polysensory neocortex or of the amygdaloid complex. Behavioral Brain Research. 23, 95-107. 177 Sumby, W.H., & Pollack, I. (1954). Visual contribution to speech intelligibility in noise. Journal of the Acoustical Society of America. 26, 212-215. Tassinari, G., Aglioti, S., Chelazzi, L., Marzi, C.A., & Berlucchi, G. (1987). Distribution in the visual field of the costs of voluntarily allocated attention and of the inhibitory after-effects of covert orienting. Neuropsvchologia. 25, 55-71. Tassinari, G., Aglioti, S., Chelazzi, L., Peru, A., & Berlucchi, G. (1994). Do peripheral non-informative cues induce early facilitation of target detection? Vision Research, 34,179-189. Tassinari, G., & Berlucchi, G. (1993). Sensory and attentional components of slowing of manual reaction-time to non-fixated visual targets by ipsilateral primes. Vision Research. 33, 1525-1534. Tassinari, G., & Berlucchi, G. (1995). Covert orienting to non-informative cues: Reaction time studies. Behavioural Brain Research. 71. 101-112. Taylor, T.L., & Klein, R.M. (in press). On the causes and effects of inhibition of return. Psychonomic Bulletin and Review. Terry, K.M., Valdes, L.A., & Neill, W.T. (1994). Does "inhibition of return" occur in discrimination tasks? Perception & Psychophysics. 55, 279-286. Tipper, S.P., & Cranston, M. (1985). Selective attention and priming: Inhibitory and facilitatory effects of ignored primes. Quarterly Journal of Experimental Psychology. 37A, 591-611. Tipper, S.P., Weaver, B., & Houghton, G. (1994). Behavioural goals determine inhibitory mechanisms of selective attention. Quarterly Journal of Experimental Psychology, 47A, 809-840. Treisman, A. (1960). Contextual cues in selective listening. Quarterly Journal of Experimental Psychology. 12, 242-248. Treisman, A., & Gelade, G. (1980). A feature integration theory of attention. Cognitive Psychology. 12. 97-136. Treisman, A. & Gormican, S. (1988). Feature analysis in early vision: Evidence from search asymmetries. Psychological Review. 95, 15-48. Valenza, E., Simion, F., & Umilta, C. (1994). Inhibition of return in newborn infants. Infant Behavior and Development. 17. 293-302. 178 Vaughan, H.G., Jr., & Arezzo, J.C. (1988). The neural basis of event-related potentials. In T.W. Picton (Ed.), Handbook of Electroencephalography and Clinical Neurophysiology. Revised Series. Vol. 3. Human Event-Related Potentials (pp. 45-96). New York: Elsevier. Wallace, M.T., Meredith, M.A., & Stein, B.E. (1993). Converging influences from visual, auditory, and somatosensory cortices onto output neurons of the superior colliculus. Journal of Neurophysiology. 69. 1797-1809. Ward, L.M. (1994). Supramodal and modality-specific mechanisms for stimulus-driven shifts of auditory and visual attention. Canadian Journal of Experimental Psychology, 48, 242-259. Ward, L.M., McDonald, J.J., & Golestani, N. (1998a). Cross-modal control of attention shifts. In R.D. Wright (Ed.) Visual attention (pp. 232-268). New York: Oxford University Press. Ward, L.M., McDonald, J.J., & Lin, D. (1998b). On asymmetries in cross-modal spatial attention orienting. Manuscript submitted for publication. Weiskrantz, L. (1986). Blindsight: A case study and implications. New York: Oxford University Press. Welch, R.B., & Warren, D.H. (1986). Intersensory interactions. In K.R. Boff, L. Kaufman, & J.P. Thomas (Eds.), Handbook of Perception and Human Performance. Vol. 1: Sensory Processes and Perception (pp. 1-36). New York: Wiley. Woldorff, M.G. (1993). Distortion of ERP averages due to overlap from temporally adjacent ERPs: Analysis and correction. Psvchophysiology. 30. 98-119. Woldorff, M.G., & Hillyard, S.A. (1991). Modulation of early auditory processing during selective listening to rapidly presented tones. Electroencephalography and Clinical Neurophysiology. 79, 170-191. Wolpaw, J.R., & Penry, J.K. (1975). A temporal component of the auditory evoked response. Electroencephalography and Clinical Neurophysiology, 39, 609-620. Woods, D.L., Alho, K., & Algazi, A. (1991). Brain potential signs of feature processing during auditory selective attention. Neuroreport. 2,189-192. Woods, D.L., Alho, K., & Algazi, A. (1992). Intermodal selective attention. I. Effects on event-related potentials to lateralized auditory and visual stimuli. Electroencephalography and Clinical Neurophysiology. 82, 341-355. 179 Woods, D.L., Alho, K., & Algazi, A. (1993). Intermodality selective attention: Evidence for processing in tonotopic auditory fields. Psychophysiology. 30, 287-295. Wright, R.D., & Richard, C M . (1996). Inhibition-of-return at multiple locations in visual space. Canadian Journal of Experimental Psychology. 54, 324-327. Wright, R.D., Richard, C M . , & McDonald, J.J. (1995). Neutral location cues and . cost/benefit analysis of visual attention shifts. Canadian Journal of Experimental Psychology. 49, 540-548. Wright, R.D., & Ward, L.M. (1998). The control of visual attention. In R. D. Wright (Ed.), Visual attention (pp 132-186). New York: Oxford University Press. Wurtz, R.H., & Albano, J.E. (1980). Visual-motor function of the primate superior colliculus. Annual Review of Neuroscience, 3, 189-226. Yantis, S., & Jonides, J. (1990). Abrupt visual onsets and selective attention: Voluntary versus automatic allocation. Journal of Experimental Psychology: Human Perception and Performance. 16, 121-134. Zeki, S. (1993). A vision of the brain. Oxford: Blackwell. 

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
https://iiif.library.ubc.ca/presentation/dsp.831.1-0099469/manifest

Comment

Related Items