Arts, Faculty of
Psychology, Department of
DSpace
UBCV
Blundon, Elizabeth Gwynne
Master of Arts - MA
University of British Columbia
We conducted four experiments comprised of sequential auditory and visual searches in order to further explore the generalizability of the search asymmetry phenomenon to different sensory modalities, and to the sequential presentation of items in search arrays. It has been shown that search time to identify targets that contain features that distractors don’t have (feature-present targets) is faster than search time to identify targets that are missing features that distractors have (feature-absent targets). In Experiment 1 participants listened to auditory oddball sequences, consisting of two types of five-tone runs: the flat run, which consisted of five pure tones of the same frequency (the feature-absent target), and the change run, which consisted of four pure tones of the same frequency, followed by a fifth tone of a different frequency (the feature-present target). In some sequences the change runs were common and the flat runs were rare (the feature-present condition), while in other sequences these roles were reversed (the feature-absent condition). Experiments 2, 3 and 4 used the same protocol, however the visual stimuli consisted of rings (annuli) that differed by some feature (colour in Experiment 2, contrast in Experiment 3, and shade in Experiment 4). In all four experiments participant reaction times (RT) and electrophysiological (P300) responses to rare target patterns were recorded. In Experiments 1, 2 and 3, the reaction time and P300 latencies to identify feature-present targets were significantly faster than those to feature-absent targets, suggesting strong similarities between simultaneous visual search, and sequential auditory and visual search. What’s more, P300 responses to feature-present targets exhibited strong characteristics of both P3a and P3b subcomponents, while feature-absent responses only resembled that of the P3b. By contrast, the results of the fourth experiment were inconclusive. In Experiment 4 the saliency of the feature difference in the change runs was significantly reduced compared to that of the first three experiments, yielding longer reaction times and weaker P300 responses. Implications for the current understanding of search strategies associated with easy (feature-present) and difficult (feature-absent) searches, as well as the locus of the search asymmetry phenomenon, are discussed.
Search through time is like search through space: behavioural and electrophysiological evidence by Elizabeth Gwynne Blundon B.Sc., The University of British Columbia, 2011 A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF ARTS in THE FACULTY OF GRADUATE AND POSTDOCTORAL STUDIES (Psychology) THE UNIVERSITY OF BRITISH COLUMBIA (Vancouver) August 2015 © Elizabeth Gwynne Blundon, 2015 ii Abstract We conducted four experiments comprised of sequential auditory and visual searches in order to further explore the generalizability of the search asymmetry phenomenon to different sensory modalities, and to the sequential presentation of items in search arrays. It has been shown that search time to identify targets that contain features that distractors don’t have (feature-present targets) is faster than search time to identify targets that are missing features that distractors have (feature-absent targets). In Experiment 1 participants listened to auditory oddball sequences, consisting of two types of five-tone runs: the flat run, which consisted of five pure tones of the same frequency (the feature-absent target), and the change run, which consisted of four pure tones of the same frequency, followed by a fifth tone of a different frequency (the feature-present target). In some sequences the change runs were common and the flat runs were rare (the feature-present condition), while in other sequences these roles were reversed (the feature-absent condition). Experiments 2, 3 and 4 used the same protocol, however the visual stimuli consisted of rings (annuli) that differed by some feature (colour in Experiment 2, contrast in Experiment 3, and shade in Experiment 4). In all four experiments participant reaction times (RT) and electrophysiological (P300) responses to rare target patterns were recorded. In Experiments 1, 2 and 3, the reaction time and P300 latencies to identify feature-present targets were significantly faster than those to feature-absent targets, suggesting strong similarities between simultaneous visual search, and sequential auditory and visual search. What’s more, P300 responses to feature-present targets exhibited strong characteristics of both P3a and P3b subcomponents, while feature-absent responses only resembled that of the P3b. By contrast, the results of the fourth experiment were inconclusive. In Experiment 4 the saliency of the feature difference in the change runs iii was significantly reduced compared to that of the first three experiments, yielding longer reaction times and weaker P300 responses. Implications for the current understanding of search strategies associated with easy (feature-present) and difficult (feature-absent) searches, as well as the locus of the search asymmetry phenomenon, are discussed. iv Preface This thesis is original and unpublished work by the author, Elizabeth Blundon, under the supervision and guidance of Dr. Lawrence Ward. The research program was designed jointly by Ms. Blundon and Dr. Ward, and executed solely by Ms. Blundon. Data was collected primarily by Ms. Blundon, with some assistance from a Research Assistant. Data was analyzed exclusively by Ms. Blundon. v Table of contents Abstract .................................................................................................................................... ii Preface ..................................................................................................................................... iv Table of contents ..................................................................................................................... v List of tables............................................................................................................................ vi List of figures ......................................................................................................................... vii Acknowledgements ............................................................................................................... xii Chapter 1: Introduction ......................................................................................................... 1 Attention and visual search ............................................................................................................. 2 Beyond the visual modality .............................................................................................................. 9 The present work ............................................................................................................................ 14 Chapter 2: Auditory search asymmetry (Experiment 1) .................................................. 15 Methods ........................................................................................................................................... 17 Results.............................................................................................................................................. 22 Discussion ........................................................................................................................................ 27 Chapter 3: Visual sequential search asymmetry (Experiments 2, 3 and 4) ..................... 31 Methods ........................................................................................................................................... 33 Results.............................................................................................................................................. 38 Discussion ........................................................................................................................................ 45 Chapter 4: General discussion ............................................................................................. 49 Bibliography .......................................................................................................................... 59 Appendix ................................................................................................................................ 66 Experiment 1 Scalp ERPs (auditory, 500 Hz and 1000 Hz tones) .............................................. 66 Experiment 2 Scalp ERPs (colour, yellow and blue rings) ......................................................... 68 Experiment 3 Scalp ERPs (contrast, black and white rings) ...................................................... 70 Experiment 4 Scalp ERPs (shade, light grey and dark grey rings)............................................ 72 vi List of tables Table 1. Experiment 1 behavioural and electrophysiological results. Results are based on the average of all individual differences between conditions. Significant differences are indicated with an asterisk (*). Effect size (es) estimates are only included for significant differences. ........................................ 23 Table 2. Experiment 2 behavioural and electrophysiological results. Results are based on the average of all individual differences between conditions. Significant differences are indicated with an asterisk (*). Effect size (es) estimates are only included for significant differences. ........................................ 39 Table 3. Experiment 3 behavioural and electrophysiological results. Results are based on the average of all individual differences between conditions. Significant differences are indicated with an asterisk (*). Effect size (es) estimates are only included for significant differences. ........................................ 40 Table 4. Experiment 4 behavioural and electrophysiological results. Results are based on the average of all individual differences between conditions. There were no significant differences, therefore no effect sizes were estimated. .................................................................................................................. 41 vii List of figures Figure 1. Example of search asymmetry stimulus, taken from Treisman & Souther (1985). In the feature-present condition (A) the target is a Q (circle with a line), while the distractors are O’s. In the feature-absent condition (B), the target is an O, while the distractors are Q’s. Typically the feature-present target is found significantly more quickly than the feature-absent target. ................................. 4 Figure 2. Experiment 1 pure-tone stimuli. Stem-down high notes were 1000 Hz and stem-up low notes were 500 Hz. Flat runs consisted of five pure tones of the same frequency, while change runs consisted of four pure tones of the same frequency, followed by a fifth tone of a different frequency. All runs consisted of a combination of 500-Hz and 1000-Hz tones, which generated two versions of each type of run, one in which 1000-Hz tones comprised the first four tones in a change run (change-down, B) and all the tones in the flat run (flat-high, B), and the same for the 500-Hz tones (change-up and flat-low, A). Each tone was 50 ms in duration, and successive tones in a run were separated by 100 ms of silence .................................................................................................................................. 19 Figure 3. Experiment 1 sequence design. Participants each heard four sequences, consisting of two feature-present sequences and two feature-absent sequences. Stem-down high notes were 1000 Hz and stem-up low notes were 500 Hz. Rare runs are targets to be detected in the longer sequence of common runs. Green borders are around common runs, orange borders are around rare runs. Dashed lines are around flat runs, solid lines are around change runs. ............................................................. 20 Figure 4. Grand Average (across all subjects) ERPs at Pz to the four run types when they were rare or common and the respective difference waves. Each panel compares the difference waves for feature-present and feature-absent conditions for run types in which the first four tones in the run were identical. (A) Change-up (feature-present) compared to flat-low (feature-absent). (B) Change-down (feature-present) compared to flat-high (feature-absent). Time 600 ms refers to the onset of the last tone of the run. ERPs at more electrode sites are displayed in Figures A1-A4. ................................... 25 Figure 5. Annulus stimuli. All ring colours in the figure are approximations of ring colours presented to participants. Yellow ring RGB values were (255,255,0), blue rings were (0,0,255), black rings were viii (0,0,0), white rings were (255,255,255), lighter grey rings were (101,101,101) and darker grey rings varied between (65,65,65) and (59,59,59). Background shades were (200,200,200) for Experiments 2 and 4, while background shades were (128,128,128) (colours in the figure are not exactly the same as those used in the experiments). ............................................................................................................ 35 Figure 6. ERPs at Pz to the four run types per experiment (12 total) when they were rare or common and the respective difference waves. Each panel compares the difference waves for feature-present and feature-absent conditions for run types in which the first four rings in the run were identical. The stimuli that the ERPs are in response to are in next to each ERP. ERPs at more electrode sites are displayed in Figures A5-A16................................................................................................................ 45 Figure 7. Scalp maps of the change run difference waves, calculated by subtracting the ERP to each stimulus type when it was common from that to the identical stimulus type when it was rare, between 100 ms and 550 ms post stimulus, for all four experiments. ................................................................ 54 Figure 8. Scalp maps of the flat run difference waves, calculated by subtracting the ERP to each stimulus type when it was common from that to the identical stimulus type when it was rare, between 100 ms and 550 ms post stimulus, for all four experiments. ................................................................ 54 Figure 9. P300 difference waves for feature-present (magenta) and feature-absent (green) midline electrode sites for all four experiments................................................................................................. 55 Figure A1. Experiment 1 scalp ERPs, feature-present condition. Difference waves (black, dashed) were determined by subtracting common flat-high ERPs (blue) from rare change-down target ERPs (red). ..................................................................................................................................................... 66 Figure A 2. Experiment 1 scalp ERPs, feature-absent condition. Difference waves (black, dashed) were determined by subtracting common change-down ERPs (blue) from rare flat-high target ERPs (red). ..................................................................................................................................................... 66 ix Figure A3. Experiment 1 scalp ERPs, feature-present condition. Difference waves (black, dashed) were determined by subtracting common flat-low ERPs (blue) from rare change-up target ERPs (red). .............................................................................................................................................................. 67 Figure A4. Experiment 1 scalp ERPs, feature-absent condition. Difference waves (black, dashed) were determined by subtracting common change-up ERPs (blue) from rare flat-low target ERPs (red). .............................................................................................................................................................. 67 Figure A5. Experiment 2 scalp ERPs, feature-present condition. Difference waves (black, dashed) were determined by subtracting common flat-blue ERPs (blue) from rare change-blue to yellow target ERPs (red). ........................................................................................................................................... 68 Figure A6. Experiment 2 scalp ERPs, feature-absent condition. Difference waves (black, dashed) were determined by subtracting common change-blue to yellow ERPs (blue) from rare flat-blue target ERPs (red). ........................................................................................................................................... 68 Figure A7. Experiment 2 scalp ERPs, feature-present condition. Difference waves (black, dashed) were determined by subtracting common flat-yellow ERPs (blue) from rare change-yellow to blue target ERPs (red). ................................................................................................................................. 69 Figure A8. Experiment 2 scalp ERPs, feature-absent condition. Difference waves (black, dashed) were determined by subtracting common change-yellow to blue ERPs (blue) from rare flat-yellow target ERPs (red). ................................................................................................................................. 69 Figure A9. Experiment 3 scalp ERPs, feature-present condition. Difference waves (black, dashed) were determined by subtracting common flat-black ERPs (blue) from rare change-black to white target ERPs (red). ................................................................................................................................. 70 Figure A10. Experiment 3 scalp ERPs, feature-absent condition. Difference waves (black, dashed) were determined by subtracting common change-black to white ERPs (blue) from rare flat-black target ERPs (red). ................................................................................................................................. 70 x Figure A11 Experiment 3 scalp ERPs, feature-present condition. Difference waves (black, dashed) were determined by subtracting common flat-white ERPs (blue) from rare change-white to black target ERPs (red). ................................................................................................................................. 71 Figure A12. Experiment 3 scalp ERPs, feature-absent condition. Difference waves (black, dashed) were determined by subtracting common change-white to black ERPs (blue) from rare flat-white target ERPs (red). ................................................................................................................................. 71 Figure A13. Experiment 4 scalp ERPs, feature-present condition. Difference waves (black, dashed) were determined by subtracting common flat-dark ERPs (blue) from rare change-dark to light target ERPs (red). ........................................................................................................................................... 72 Figure A14. Experiment 4 scalp ERPs, feature-absent condition. Difference waves (black, dashed) were determined by subtracting common change-dark to light ERPs (blue) from rare flat-dark target ERPs (red). ........................................................................................................................................... 72 Figure A15. Experiment 4 scalp ERPs, feature-present condition. Difference waves (black, dashed) were determined by subtracting common flat-light ERPs (blue) from rare change-light to dark target ERPs (red). ........................................................................................................................................... 73 Figure A16. Experiment 4 scalp ERPs, feature-absent condition. Difference waves (black, dashed) were determined by subtracting common change-light to dark ERPs (blue) from rare flat-light target ERPs (red). ........................................................................................................................................... 73 Figure A17. Both figures are taken from Polich (2007). (Top left) A schematic representation of the proposed front-parietal attention mechanisms and temporo-parietal memory encoding networks presumed to underlie the P3a and P3b respectively. (Top right) Examples of grand average P3a and P3b ERP responses compared to reaction times, as well as P3a and P3b amplitude and latency scalp topographies. (Bottom right) Illustrations of P300 responses to a single stimulus (left), to an oddball stimulus (middle), to an infrequent salient distractor stimulus (right, P3a) and to a non-salient target stimulus (right, P3b). ............................................................................................................................ 74 xi Figure A18. Experiments 2 and 3, possible P2 (FCZ) and inversion (OZ) for all change runs, both rare and common. The positivity (and inversion) to rare runs is much earlier than that to common runs. No such effect is observed in response to flat runs, so are not shown in the figure. .................................. 75 Figure A19. Experiment 1, negativity (MMN) in response to all change runs, both rare and common. A large positivity (P3a) follows the negativity to rare change runs (red), but not common change runs (blue). No such effect is observed in response to the flat runs, so are not shown in the figure. ........... 76 xii Acknowledgements The author would like to thank Dr. Lawrence Ward for his continued support and mentorship, and Tricia Sernande for assisting in data collection. 1 Chapter 1: Introduction This thesis will describe four experiments that seek to investigate the relationship between auditory and visual search, as well as the relationship between search through arrays of stimuli that are presented all at once (simultaneously) versus one at a time (sequentially). This was done to expand upon current literature on the cross-modal (visual versus auditory) generalizability of various search phenomena that have thus far been primarily limited to a specific kind of search task, namely identifying one or more visual targets among a static array of distractor items. With the discovery of the similarities between analogous auditory and visual search tasks (replicated and expanded on in Experiment 1) (Cusack & Carlyon, 2003), current understanding of search phenomena, and the subsequent models that have been developed to account for them, may need to be re-interpreted. Namely, since auditory stimuli are typically presented sequentially, rather than simultaneously, the boundaries of the search phenomena may need to include search through time as well as search through space. Therefore, Experiments 2 and 3 described in this thesis seek to examine the relationship between search through sequentially presented visual stimuli and search through simultaneously presented visual stimuli. Finally, Experiment 4 was performed in order to explore how sequential search can help us to understand the mechanism (attention vs perception) of these search phenomena, and how the search strategies established by search through space could generalize to search through time. In all four experiments behavioural (reaction time) and electrophysiological (ERP) measures were used to compare behaviour associated with search through temporally distributed stimulus arrays with established behaviours associated with search through simultaneously presented stimulus arrays. 2 Attention and visual search Soccer fans know that at the FIFA world cup, many players are supplied with very brightly-coloured footwear. Imagine in game 1 only a few players wear their brightest sneakers, while most other players wear their regular season, scuffed up trainers. The individuals wearing their brightest footwear seem to pop out among the other players, and are very easy to follow around the field. However, in game 2 most players decide to wear their brightest footwear, while only a few continue to wear their trainers. Those in the regular season trainers are very difficult to spot and are easily lost in the melee. Though the contrast between the two sets of footwear is the same in both games, differentiating between players (based on shoe colour alone) is not as easy in game 2 as it was in game 1. This asymmetrical ease of search in a dynamic environment is the focus of this thesis. What is known about attention and scene analysis is very heavily based on investigations into search behaviour through static visual arrays. Treisman proposed her influential feature integration theory (FIT) based on studies that characterized how subjects identified targets in an array of simultaneously presented visual stimuli (Treisman & Gelade, 1980). She reported that, depending on the feature composition of the items in the array, some target items could be detected pre-attentively, whereas detection of others required focused attention (Treisman & Gormican, 1988). Much of the evidence for her theories arose from her observations of asymmetries in visual search, a phenomenon where a target A among distractors B is faster to identify than target B among distractors A (Treisman & Souther, 1985). This asymmetry is typically observed when one of the items in the array contains a feature that the others are missing. For example, if target A contains a feature that distractors B are missing, target A is easy to find, and speed to identify target A does not 3 increase much, if at all, as the set size (the number of distractors) increases (see Figure 1). This is called the feature-present condition. However, since target B is missing a feature that target A has, target B is much harder to identify, and speed to identify target B increases as a function of set size. This is called the feature-absent condition. In this case, it is believed that target A is identified pre-attentively, and is said to “pop out” among distractors B (Treisman & Gelade, 1980; Treisman A., 1982). In contrast, identification of target B is said to require serial, self-terminating search, and requires focused attention to be directed toward each item in the array until the target is found. It was initially believed that item A’s additional feature produced greater perceptual activity relative to item B (Weber’s law, Gescheider, 1997), and this difference in perceptual activity was the cause of the asymmetry (Treisman & Gormican, 1988). There are, however, many cases where asymmetries are observed even with feature singletons or items that consist of only one feature. For example, C’s are more quickly identified among O’s than is the converse, a line at an angle is more quickly identified among vertical or horizontal lines than is the converse, and a prototypical colour like red is more quickly identified among a related non-prototypical colour like magenta than is the converse (Treisman & Gormican, 1988). It was later shown that an asymmetry is observed when pairing familiar and unfamiliar stimuli whose perceptual activities are equal (Wang, Cavanagh, & Green, 1994; Shen & Reingold, 2001; Rauschenberger & Yantis, 2006). Wang et al (1994) showed that inverted Ns pop out among regular Ns, while regular Ns require serial search to identify among inverted Ns, suggesting that the asymmetry is more broadly attributable to differential ease of processing of the items in the array, which could account for the asymmetry observed when pairing feature singletons. In short, if the distractors are 4 easy to process and thus inhibit, the target is likely to pop out. If the distractors are difficult to process, however, identification of the target will require more directed attention. Figure 1. Example of search asymmetry stimulus, taken from Treisman & Souther (1985). In the feature-present condition (A) the target is a Q (circle with a line), while the distractors are O’s. In the feature-absent condition (B), the target is an O, while the distractors are Q’s. Typically the feature-present target is found significantly more quickly than the feature-absent target. 5 Wolfe expanded on Treisman’s work by looking at the relationship between the pre-attentive stage of search and the attentive stage (Wolfe, Cave, & Franzel, 1989; Wolfe, 1994; Wolfe, 1998), or rather how the pre-attentive stage guides the way in which we direct our attention across a visual scene. In the guided search (GS) model, bottom-up activation of the search array generates an activation map, and areas with the highest activation draw the most attention. Activation is a function of stimulus dissimilarity, so the items in the array that are least like their neighbors generate the highest activation, and subsequently draw the most attention. Activation is also a function of search goals, so top-down information about the target features will change the activation map, enhancing activation for relevant features and inhibiting activation for irrelevant features. These activation maps guide deployment of attention from the highest peak of activation to the lowest. In this sense, GS is a more flexible all-purpose search strategy that combines parallel with serial search, and bottom-up with top-down information, as opposed to the stricter dichotomy between these search strategies suggested by FIT. In contrast, Dosher et al. (Dosher, Han, & Lu, 2004) called into question the necessity for a two-process model of search to account for typical search asymmetry phenomena. They suspected that hard searches took longer to perform because during difficult search tasks participants made more overt attention shifts (eye movements) than during easy search tasks, but that covert attention (attention allocation in the absence of eye movements) mechanisms might operate differently. Dosher et al. investigated this distinction by conceptually replicating a typical search asymmetry study but limiting participant search time so as to engage only covert attention shifts. Participants were shown search displays very briefly (50 ms or 100 ms), and they were forced to indicate whether or not they had seen a target at 6 various time lags from when the arrays were first shown to them. Their detection accuracy and response times were compared between easy and difficult search, for the two display times, for small or large set sizes, and across all time lags. Dosher et al. found no significant differences in response time between any of these conditions. They did find, however, that response accuracy was higher for easy searches than for difficult searches. They concluded that covert attention mechanisms are likely parallel in nature, and are similarly successful at identifying targets in easy or difficult search tasks. Search asymmetries in this framework arise from unequal discriminabilities of the targets in their respective contexts. Furthermore, Dosher et al. fit both a probabilistic parallel and a serial search model to the data they collected, and found that the probabilistic parallel search model accounted for the observed search phenomena in the absence of eye movements, while the serial model over-predicted the response time increase as set size increased. Though they acknowledge that some hybrid models (serial and parallel) might account for the data just as well as the probabilistic parallel model, a model heavily influenced by serial processing would not account for the data as well as even a simple parallel search model. Like GS, the Dosher et al. model calls into question the usefulness of requiring two, distinct search strategies to account for search asymmetry phenomena. Many more studies have been conducted to investigate the boundaries of the asymmetry phenomenon (e.g., Duncan & Humphreys, 1989; Humphreys & Muller, 2000; von Grünau & Dubé, 1994; Wang, Zhang, He, & Jiang, 2010) and many other models have been proposed to characterize the phenomenon (e.g., Duncan & Humphreys, 1992; Li, 1999). None of the latter have achieved the prominence of FIT, GS, or the Dosher et al. model, and so will not be discussed further here. 7 Neural correlates of these visual search phenomena have been extensively researched using electrophysiological (Schubö, Schröger, & Meinecke, 2004), hemodynamic (Leonards, Sunaert, Van Hecke, & Orban, 2000) and TMS measures (Ashbridge, Walsh, & Cowey, 1997) in humans, and implanted electrodes in macaque monkeys (Bichot, Rossi, & Desimone, 2005). Electrophysiological measures are particularly useful in studying search asymmetry as they can easily be compared with behavioural reaction times. EEG recordings provide millisecond accuracy in detecting neurological responses to stimuli, which can be averaged to show peaks and troughs with functional meanings related to a presented stimulus. These neurological responses are called Event-Related-Potentials (ERP), because they represent characteristic changes in potential difference (voltage) between scalp electrodes that are temporally related to a specific event. ERPs are typically oscillatory in nature, and the different peaks and troughs are called components. One component, called the P300, is of particular interest to the investigation of search asymmetry phenomena, as it is typically associated with target identification and context updating, its latency often covaries with reaction time, and its amplitude and shape is modulated by task demands (Polich, 2007). Therefore, the P300 can be used to index the neurological response associated with identifying a target in an array. The P300 is also thought to contain sub-components, called the P3a and P3b, but these will be discussed in more detail in the General Discussion. Of special interest here is a study by Luck and Hillyard (1990), who performed a conceptual replication of a typical search asymmetry study to investigate electrophysiological responses to feature-present and feature-absent search targets. Their initial goal was to use the P300 to help confirm whether different search strategies were being used in the different search conditions. Since P300 amplitude has been shown to increase with decreasing 8 stimulus probability (that is, the less likely a stimulus is to occur the greater P300 amplitude in response to its identification) (see Polich, 2007 for review), they predicted that for parallel search the P300 amplitude would remain constant, whereas for serial search the P300 amplitude would increase as a function of set size. By definition, a parallel process only requires one decision point to identify the target. A serial process, however, requires multiple decision points leading up to target identification. From the participant’s perspective, with each decision point in the serial process the probability of the stimulus occurring decreases, which should predict a larger P300 amplitude if the target is found. Therefore, if a parallel process is truly being used to identify the feature-present target, P300 amplitude should remain relatively constant regardless of set size. In contrast, if a serial process in being used to identify the feature-absent target, P300 amplitude should increase as a function of set size. This is exactly what they found for response-locked (epochs marked by the onset of the response) P300 ERPs: P300 amplitude to the feature-present targets were largely unaffected by increasing set size, while the response-locked P300 amplitude to the feature-absent targets increased as a function of set size. In addition, they found that the stimulus-locked (epochs marked by the onset of the stimulus) P300 response to identifying the feature-present target elicited a P300 that was larger, earlier and sharper than the P300 response to identifying the feature-absent target. They found that both P300 latencies and reaction times to the feature-present target remained fairly constant regardless of set size, while P300 latencies and reaction times to the feature-absent target increased as a function of set size. What’s more, stimulus-locked P300 amplitudes actually decreased as a function of set size for the feature-absent condition, while amplitudes in the feature-present condition remained constant (this is the opposite effect to 9 the response-locked P300 ERPs, and will become relevant in the next paragraph). These latency and reaction time results are consistent with Treisman’s original behavioural study, and together with the amplitude results provide electrophysiological evidence that different strategies are employed in these different conditions. Subsequent electrophysiological research, however, has called this interpretation into question. Notably, Wolber & Wascher (2003) found converging evidence of both stimulus-locked and response-locked P300 amplitudes remaining constant during parallel search (they used feature singletons in their parallel search task). For serial search (form singletons and conjunction search), however, they found that both stimulus-locked and response-locked P300 amplitudes decreased as a function of set size, which is inconsistent with the response-locked P300 results reported by Luck & Hillyard (1990). Since P300 amplitude has been found to decrease with increasing processing demands of the task (Kok, 2001; Polich, 2007), they interpreted this result as evidence that only a single search mechanism is used for all search tasks, consistent with the Dosher et al. (2004) model, and that there is no qualitative difference between easy feature and difficult conjunction search per se. With these results, it is still unclear whether one or two search strategies can account for the search asymmetry phenomenon. Beyond the visual modality The cross-modal generalizability of these data and models has been investigated as well. Several studies have confirmed analogous asymmetries in haptic search (Sathian & Burton, 1991; Whang, Burton, & Shulman, 1991; Overvliet, Mayer, Smeets, & Brenner, 2008; Plaisier, Bergmann Tiest, & Klappers, 2008), but these will not be reviewed in this thesis. Most relevant here is that Cusack & Carlyon (2003) found that the auditory system 10 was subject to a similar type of “search” asymmetry as the visual system. They created a type of auditory soundscape consisting of tones distributed in frequency and time (for 2 seconds), either made up of pure tones of various frequencies with or without a frequency-modulated (warble-tone) target, or of modulated tones with or without a pure-tone target. Participants were more accurate in detecting the warble-tone target among the pure tone distractors than in detecting the pure-tone target among the warble tone distractors. In the context of a number of other variations designed to rule out peripheral auditory processing mechanisms as the source of this asymmetry, they concluded that Treisman’s (1988) approach (two search strategies) was the best explanation for the auditory search asymmetry they had observed. Nonetheless there are significant differences between the paradigm of Cusack and Carlyon (2003) and that of Triesman and Souther (1985). Most important among these, are that Cusack and Carlyon used accuracy in target detection as a proxy of attention focusing, whereas Treisman and Souther used search speed as their proxy. In the Treisman and Souther paradigm, accuracy was near 100% for all conditions, suggesting that the identifiability of the targets in each condition was comparable, or at least not different enough to affect accuracy, whereas the identifiability of the Cusack and Carlyon target stimulus clearly was greater in the feature-present condition than in the feature-absent condition. As Cusack and Carlyon mentioned, informational masking (Pollack, 1975; Watson, 1987) could have been partly responsible for the asymmetry observed in their “soundscape” experiments (see Chapter 2 for further discussion on this point). A few studies have used the mismatch negativity (MMN; the negative difference between ERPs to infrequent, unpredictable stimuli and those to frequent, highly predictable stimuli around 100-250 ms after stimulus onset) to investigate whether the origins of the 11 asymmetry arises from early perceptual (automatic) or attention mechanisms (Bishop, O'reilly, & McArthur, 2005; Timm, Weise, Grimm, & Schröger, 2011). Because the MMN is associated with automatic low-level feature difference detection (Näätänen, Paavilainen, Rinne, & Alho, 2007), Bishop et al. (2005) argued that if the asymmetry were early perceptual in origin there would be a difference in MMN amplitude between feature-absent (unmodulated tone targets) and feature-present (modulated tone targets) conditions, both when participants attend and when they do not attend to the stimuli. If the effect arises from differences in attention deployment, however, then an MMN asymmetry should only be observed when participants attend to the stimuli, and presumably no MMN asymmetry would be observed when participants do not attend. Bishop et al. (2005) reported the presence of a robust MMN asymmetry both when participants attended and when they did not, implying that an asymmetry in low-level feature processing exists. They did not include a behavioural analysis in their investigation, however, so no claims could be made about the link between the MMN and the different reaction times observed in traditional asymmetry studies. Timm et al. (2011) replicated the Bishop et al. (2005) results including a behavioural measure, but in their study the MMN response did not seem to predict the behavioural results. Despite only finding an MMN in the feature-present condition, their hit and false alarm rates were consistent across both conditions, suggesting that the MMN is not related to the mechanism that governs the accuracy of the behavioural response in this task. Although these studies provide compelling evidence of the automatic nature of detection of feature-present targets, they cannot account for feature-absent detection. This further suggests two search strategies are used for feature-present and feature-absent search, supporting Treisman’s original FIT and/or Wolfe’s GS model. 12 The similarities between visual and auditory (and haptic) search asymmetry phenomena, however, may call for some re-interpretation of the FIT, GS and Dosher’s probabilistic parallel search models. All of these models consist of some kind of pre-attentive, parallel, search process. However, in the auditory versions of these tasks stimuli are primarily presented sequentially, which precludes participants from being able to perform the usual kind of parallel search. Considering the similarities with visual search, this calls into question the assumption that parallel search is the strategy that is associated with quick target identification. Perhaps a more generalizable strategy is that diffuse attention is all that is required to identify highly salient targets among less salient distractors. This was Treisman’s original interpretation of the locus of the effect; though she acknowledged that pre-attentive processing implies a process that occurs before attention is engaged, she suggested that “so-called preattentive search is really search in which attention is distributed widely over the whole display rather than narrowly focused and directed serially to one object or subgroup at a time (Treisman & Gormican, 1988, pg 43).” This interpretation is generalizable to all sensory modalities, and it could mean that so-called parallel search is simply a consequence of applying diffuse attention to a spatial array with multiple items. Since the initial development of FIT and competing search models it would appear that this interpretation of the locus of the effect has been overlooked, but in light of the auditory search asymmetry results perhaps this interpretation needs to be re-explored. An important consideration here is how the auditory system perceives auditory objects. Unlike visual stimuli, spatial location is not sufficient to define boundaries between identical auditory objects (Kubovy & Van Valkenburg, 2001). Kubovy and Van Valkenburg used the following example to illustrate this point: whereas two identical spotlights shone in 13 opposite sides of a room will be perceived as two visual objects, two identical tones presented from two different spatial locations will be perceived as the same tone. The only way for these identical tones to be distinguished is for them to be staggered in time, or, in other words, played in sequence. This illustrates that it’s not possible for two identical simultaneous auditory events to be heard as distinct objects when they differ only in their spatial location, they must differ in time to be heard as distinct auditory objects. In this sense time is to identical auditory objects as space is to identical visual objects. Therefore, it is unlikely that one could reproduce the same effect if one were to present an auditory target and homogeneous distractors simultaneously, as all the distractors would be perceived as one event, and the target event would always pop out when it differed in frequency or some other characteristic from the distractors. Of course, visual search through time, or at least through sequentially presented visual stimuli, is eminently possible, as the Rapid Serial Visual Presentation (RSVP) literature confirms (Lawrence, 1971; Broadbent D. E., 1977). A typical RSVP protocol involves participant identification of a target among rapid serial presentation of visual stimuli in the same location. An interesting phenomenon that arises in such tasks, which will not be reviewed here, is the so-called “attention blink,” which seems to imply that attention deployment through time is affected when a target is detected in a serial stream (Shapiro, Raymond, & Arnell, 1997). Though the RSVP protocol has been used to study target identification indexed by unique physical (Botella & Eriksen, 1992) and semantic (Broadbent & Broadbent, 1987) features, as well as illusory conjunctions (McLean, Broadbent, & Broadbent, 1983), so far it has not been used to investigate search asymmetry phenomena per se. 14 Another relevant ERP study demonstrated that attending to a particular temporal position in a sequence can modulate the MMN and the P300 responses to oddball stimuli (Sanders & Astheimer, 2008). When attention was directed to the moment at which an oddball stimulus tone occurred, the MMN and P300 responses to the oddball were enhanced relative to when the oddball occurred at an unattended moment in the sequence. Clearly attention can be distributed in time as well as in space (e.g., Large & Jones, 1999; Jones et al., 2002). Therefore, an attention approach to auditory search in time appears to be a reasonable framework within which to consider our findings. The present work The similarities between auditory and visual search raise two important questions: 1) are the similarities between sequential and simultaneous search unique to the auditory system, or could this effect generalize to vision as well, and 2) if not unique to the auditory system, what is the locus of the effect, perceptual or attentional, that would allow this effect to generalize to multiple modalities and multiple methods of presentation? Four experiments were conducted to answer the above questions. The first (Chapter 2) is a conceptual replication of the auditory version of the search asymmetry tasks. The second and third (Chapter 3) were modified versions of the auditory experiment, but using visual stimuli instead of auditory stimuli. These experiments serve to confirm whether sequential search asymmetry is an artifact of the auditory system, or if it generalizes to the visual system as well. The fourth experiment (Chapter 3) serves as a first step into investigating the locus of the effect, attention vs perception. 15 Chapter 2: Auditory search asymmetry (Experiment 1) This chapter describes a new auditory analog of the visual search asymmetry first reported by Treisman and Souther (Treisman & Souther, 1985). The experiment was based on a study by Bekinschtein and colleagues (Bekinschtein, et al., 2009), that revised the standard mismatch negativity (MMN) task into one that required conscious attention, involving detecting rare tone patterns among more common ones, and another that did not, involving only simple changes in tone frequency within the tone patterns themselves. The simple changes in tone frequency within patterns generated a standard MMN ERP response without sustained attention (because it occurred in mind-wandering or distracted conscious controls, cf. Näätänen, et al., 2007) or without intact consciousness (because it occurred in vegetative and minimally conscious patients). Changes in tone patterns across longer sequences of such patterns, however, generated a P300 ERP response only in conscious controls (and in a few minimally conscious patients) who were asked to count rare patterns, and not at all in mind-wandering or distracted conscious controls or in any vegetative patients asked to count rare patterns. This suggests that detecting this kind of pattern change is not automatic and does require consciousness and sustained focal attention. The Bekinschtein et al. (2009) protocol was ideal to investigate auditory search asymmetry because it consists of two different rare auditory pattern detection tasks that in some ways seem more analogous to the visual search tasks designed by Treisman and Souther (1985) than are the previous auditory search asymmetry studies considered in chapter 1 (Cusack & Carlyon, 2003; Bishop, O'reilly, & McArthur, 2005; Timm, Weise, Grimm, & Schröger, 2011). First, the auditory stimuli used to generate the sets of oddball 16 arrays in Bekinschtein et al. (2009) contain some critical similarities to the visual search sets used by Treisman & Souther (1985). The auditory pattern sets consist of two almost identical auditory objects, each comprised of a run of five pure tones. The fifth tone of each object is either identical to the previous four (termed a flat run) or is different from them in acoustic frequency (a feature change, termed a change run). These auditory stimuli aren’t completely analogous to the visual stimuli, however, as what differs between the visual stimuli is the addition of a feature, not a feature change. Indeed, the acoustic frequency of the tones in our stimuli is what Treisman and Souther (1985) called a ‘substitutive feature,’ in which the absence of a particular feature in a stimulus implies the presence of another, for example colour in visual stimuli. Nonetheless, acoustic frequency is mapped in subcortical areas and in the primary auditory cortex to a spatially-arrayed tonotopic pitch map (e.g., Pickles, 1988) in which activation of different pitch regions corresponds nicely to activations of different regions in the shape maps in visual cortex (Treisman & Gormican, 1988). Thus, finding a rare change run among common flat runs should be easy, because it contains a single different pitch feature that pops out in the pitch map; this is similar to finding a Q among an array of O’s. But finding a rare flat run among common change runs should be much more difficult because the change runs repeatedly and persistently activate two regions of the pitch map, and it is difficult to discriminate from them a stimulus that only activates a single one of those regions when both regions could still be somewhat active. This is like finding an O among an array of Q’s. Second, Bekinschtein et al. (2009) established that the P300 waveform is generated in response to successful identification of the global pattern change. They did not collect behavioural (reaction time) data, however, nor did they separate their P300 results into 17 feature-present and feature-absent conditions. Instead they collapsed the data across those conditions. To illustrate this, the visually analogous situation would be to treat search for a Q among O’s as equivalent to search for an O among Q’s, and to report a single P300 response to identification of all targets regardless of whether they were a Q or an O. As was mentioned in Chapter 1, Luck and Hillyard conceptually replicated Treisman and Souther’s original visual search task and found different P300 responses to the easy (for example, search for a Q among O’s) and difficult (search for an O among Q’s) visual search tasks (Luck & Hillyard, 1990). Given the apparent differences in attention demands of the two auditory pattern search tasks described above, it seems likely that there should be different ERP responses to the two auditory pattern types (changing patterns embedded in unchanging ones and unchanging patterns embedded in changing ones), as in the two visual array types in the Luck and Hillyard (1990) study. Thus, unlike Bekinschtein et al. (2009), we analyzed the reaction time and P300 response for the two pattern types separately to uncover evidence of search asymmetry in the auditory domain. Methods Participants. Data were collected initially from 20 participants. From 15-20 participants has been shown in previous studies in our lab and others to yield reliable EEG data for either ERP or connectivity analyses given the numbers of stimulus trials in the study design. Data collection was stopped after the indicated number of participants had been included. Data from three participants were excluded because of excessive noise in their EEG. Thus, the analysis to be described is based on 17 participants (10 female, mean age 23.1 years). The experimental protocol was approved by the University of British Columbia 18 Behavioural Research Ethics Board in accordance with the provisions of the World Medical Association Declaration of Helsinki. Participants gave informed consent and were offered monetary compensation ($10/hr) for their participation. Participants were all right-handed and reported no hearing or neurological difficulties. Stimuli. Auditory stimuli consisted of two types of five-tone runs, called flat runs and change runs (see Figure 2). Flat runs consisted of five pure tones of the same frequency, while change runs consisted of four pure tones of the same frequency, followed by a fifth tone of a different frequency. All runs consisted of a combination of 500-Hz and 1000-Hz tones, which generated two versions of each type of run, one in which 1000-Hz tones comprised the first four tones in a change run and all the tones in the flat run (change-down and flat-high, see Figure 2B), and the same for the 500-Hz tones (change-up and flat-low, see Figure 2A). Each tone was 50 ms in duration, and successive tones in a run were separated by 100 ms of silence. Hann-filtered tones were generated using MATLAB (MathWorks, Natick, USA) and were administered binaurally at 70 dB through insert earphones (E.A.R. 3A) in a sound-attenuating chamber. Tone runs were generated using Audacity (Sourceforge). Each run lasted 650 ms from the onset of the first tone to the offset of the last tone. Pauses between the offset of the final tone of a given run and the onset of the first tone of the next run varied randomly from 700 ms to 1000 ms. Stimuli were presented and responses registered using Presentation software (Neurobehavioural Systems, Berkeley, CA, USA). 19 Figure 2. Experiment 1 pure-tone stimuli. Stem-down high notes were 1000 Hz and stem-up low notes were 500 Hz. Flat runs consisted of five pure tones of the same frequency, while change runs consisted of four pure tones of the same frequency, followed by a fifth tone of a different frequency. All runs consisted of a combination of 500-Hz and 1000-Hz tones, which generated two versions of each type of run, one in which 1000-Hz tones comprised the first four tones in a change run (change-down, B) and all the tones in the flat run (flat-high, B), and the same for the 500-Hz tones (change-up and flat-low, A). Each tone was 50 ms in duration, and successive tones in a run were separated by 100 ms of silence20 Figure 3. Experiment 1 sequence design. Participants each heard four sequences, consisting of two feature-present sequences and two feature-absent sequences. Stem-down high notes were 1000 Hz and stem-up low notes were 500 Hz. Rare runs are targets to be detected in the longer sequence of common runs. Green borders are around common runs, orange borders are around rare runs. Dashed lines are around flat runs, solid lines are around change runs. Procedure. Once participants were comfortably seated in the sound-attenuating chamber, they were familiarized with the stimuli and protocol. Each participant heard four extended oddball sequences of tone runs (about 3.5 mins per sequence) in randomized order per subject (see Figure 3). Each sequence began with 30 instances of the common run. From then on the rare run was presented on a random 20% of occasions among 80% common runs. In each sequence rare runs were heard between 18 and 30 times. There were always at least 2 common runs before and after each rare run. The four sequences of runs consisted of the following: (1) common flat-low, rare change-up; (2) common change-up, rare flat-low; (3) common flat-high, rare change-down; (4) common change-down, rare flat-high. Participants 21 were instructed to click a mouse every time they heard a pattern oddball (rare run), and their response times were recorded. It should be noted that it was made clear to participants that they were only to respond when they heard a change in the global pattern, i.e., a rare run, not every time they heard a tone that differed from the previous tone. EEG recording and ERP extraction. EEG signals were digitized at 500 Hz (National Instruments, Inc., Vaudreuil-Dorion, QC, Canada) from a 60-channel electrode cap (Electrocap, Inc., Eaton, OH, USA, International 10-10 placement) referenced to the right mastoid. Before digitization EEG signals were amplified and analog bandpass filtered from 0.1 Hz to 100 Hz (SA Instrumentation, San Diego, CA, USA). Eye movements were recorded with four periocular electrodes. All electrode impedances were kept below 10 k (input impedance of the amplifier was > 2 g). EEG data were analyzed using EEGLAB software (Delorme & Makeig, 2004). First they were down-sampled to 250 Hz and re-referenced to average reference. Artifacts were rejected using independent component analysis (runica algorithm, EEGLAB; (Viola, et al., 2009). The continuous EEG record was then epoched for ERP analysis from -1000 to +2000 ms relative to the onset of the first tone of each run. The remaining ERP analysis was conducted using ERPLAB (Lopez-Calderon & Luck, 2014), ERP analysis software that runs in EEGLAB. Shortened segments of these epochs were selected for display purposes, particularly when only the final tone in a run was of interest. All epochs around rare runs, and the epochs around the common run that appeared immediately before each rare run were extracted. This ensured equal numbers of common- and rare-run-epochs for the latter analyses. Finally, ERPs were low-pass filtered at 30Hz before plotting. 22 Results In what follows, to be consistent with the vision terminology, we call the two sequences in which the rare runs were the flat runs (flat-low and flat-high) the ‘feature-absent’ condition, and those in which the rare runs were the change runs (change-up and change-down) the ‘feature-present’ condition. Mean response times, hits, false alarms, and ERP latencies and amplitudes were compared with dependent means t-tests with df=16 for all comparisons. ERP latencies were determined by taking the peak latency (the latency at the highest amplitude) of the difference wave between 200-800 ms post stimulus onset per subject, then averaging across all subjects. Amplitudes were determined by taking the average amplitude of the difference wave over an interval 20 ms to either side of the peak per subject, then averaging across subjects. For effect size we used es = √(𝑡2 (𝑡2 + 𝑑𝑓)⁄ ) (Field, 2009). 23 Table 1. Experiment 1 behavioural and electrophysiological results. Results are based on the average of all individual differences between conditions. Significant differences are indicated with an asterisk (*). Effect size (es) estimates are only included for significant differences. Target Run RT (ms) Hit (%) FA (%) Lat (ms) Am (µV) Change-Up (CU) 386 93.9 0.1 338 7.7 Flat-Low (FL) 542 91.5 0.4 456 6.3 Difference (FL – CU) 156 2.4 0.3 119 1.5 p 0.0003* 0.71 0.21 0.002* 0.20 t(16) (es) 4.63(0.76) 0.38(N/A) 1.30(N/A) 3.65(0.69) 1.36(N/A) Change-Down (CD) 386 95.7 0.2 349 7.4 Flat-High (FH) 556 95.1 0.7 464 6.3 Difference (CD – FH) 170 0.6 0.5 116 1.1 p 0.00006* 0.88 0.02* 0.007* 0.30 t(16) (es) 4.40(0.74) 0.15(N/A) 2.55(0.54) 3.12(0.62) 1.07(N/A) Behavioural Results. Table 1 summarizes the behavioural (and electrophysiological) results of the experiment. The response times to correctly detect the rare target run (hits) were calculated from the onset of the fifth tone in the run, separately for the two feature-absent conditions and the two feature-present conditions. In each case the latency was significantly longer for the response to the target run in the feature-absent condition than in the corresponding feature-present condition. Corresponding conditions were those who shared stimuli (for example flat-low and change-up) but while one stimulus (flat-low) was common in one condition (feature-present), it was rare in the other (feature-absent), and vice versa. Correct responses were defined as a response to a rare target run that occurred before 24 the onset of the final tone of the following non-target run. This time window varied between 1300 ms and 1900 ms after the onset of the final tone in a target run. Mean percent correct detections for the feature-absent conditions were not significantly different from mean percent correct detections for the corresponding feature-present condition. False alarms, responses to common (non-target) runs, were very infrequent, less than 0.5% (1 in 200) overall, ruling out a large speed-accuracy tradeoff. Indeed in over 70% of the blocks in the individual data no false alarms at all occurred, rendering a signal detection analysis uninformative. P300 ERP Results. We computed difference waves from the ERPs to the final tone (onset at 600 ms relative to the onset of the first tone) of each run type when that run type was rare and when it was common. These difference waves reveal the ERP effect of detecting a feature-present target or feature-absent target in the absence of any physical differences between the stimuli. The ERPs to the last tones in each run type, and the respective difference waves comparing the ERPs to the identical run type when it was rare and when it was common, are displayed in Figure 4 for the Pz electrode, and in Figure A1-A4 for all electrodes. Thus, for example, for a feature-present target we compared the average ERP elicited by the last tone of the change-up run when that run was common, to the ERP elicited by the last tone of the change-up run when that run was rare. Presumably this would reflect the presence of a P300-type wave to the last tone in the change-up run when that run was rare (among common flat-low runs) and the absence of a P300-type wave when that run was common (among rare flat-low runs). Similarly, such a comparison of the flat-low run when it was rare (among common change-up runs) to when it was common (among rare change-up runs) would reveal the effect of detection of a feature-absent target. 25 Figure 4. Grand Average (across all subjects) ERPs at Pz to the four run types when they were rare or common and the respective difference waves. Each panel compares the difference waves for feature-present and feature-absent conditions for run types in which the first four tones in the run were identical. (A) Change-up (feature-present) compared to flat-low (feature-absent). (B) Change-down (feature-present) compared to flat-high (feature-absent). Time 600 ms refers to the onset of the last tone of the run. ERPs at more electrode sites are displayed in Figures A1-A4. 26 Figure 4 demonstrates that there was a clear P300-like response peaking between about 300 ms and 500 ms post onset of the fifth (last) tone in each run (900 ms and 1100 ms in the figure) to all runs when they were rare (blue lines) and no such response when those same runs were common (red lines). The (black) difference waves for each stimulus type reflect mainly this P300 response, as can be seen from the near indistinguishability of the black and blue lines in the figure. The shapes of the difference waves, the latencies of the peaks of the P300 responses, and the latencies of the difference waves differed substantially, however, between the flat (feature-absent) and change (feature-present) runs. In general, the feature-present target peaked significantly earlier, approximately 118 ms, than its corresponding feature-absent target (cf. Table 1). On the other hand, the amplitudes of the difference waves were not significantly different in general. Finally, we also analyzed the differences between the areas under the P3 curves (numerical integration of the curves) and the results were similar to those just detailed and so are not reported here. In addition to the difference in latency just described, the P300 response differed in scalp topography between feature-present and feature-absent conditions (see Figures A1-A4). In both feature-present targets (change-up and change-down) the P300 had the earliest and sharpest peak at medial frontal sites, and its amplitude was roughly the same from Pz to FCz electrodes. In both feature-absent conditions (where flat-low and flat-high were rare), however, the amplitude of the broader and later difference wave was largest at Pz, quickly attenuated at more frontal sites, and was completely absent at FCz. 27 Discussion The results of Experiment 1 constitute additional evidence for search asymmetry in the auditory domain. Participants were asked to identify auditory targets among arrays of distractors while their response times and electrophysiological responses were recorded. Our behavioural results were similar to those of a visual search task designed by Treisman and Souther; target objects containing a feature-difference that distractors lacked (feature-present condition) were identified faster than target objects that were missing a feature-difference that distractors possessed (feature-absent condition). Our ERP results also were highly similar to those of Luck and Hillyard (1990) for visual stimuli. Overall these results are consistent with Treisman’s feature integration theory (Treisman & Gelade, 1980); the different feature of the feature-present target stands out in the cortical pitch map from the single-featured distractors making detection fast and easy, but the multiple-featured distractors in the feature-absent condition require focused attention to process because they do not stand out in any feature map, slowing target identification. By subtracting the response when each stimulus was common from the response when the same stimulus was rare, we can see the P300 response to the target category in the absence of any activity generated by task-irrelevant feature changes inherent in the stimuli. Figure 4 shows that there was virtually no P300 response to the common runs (red lines in the figure) in both feature-present and feature-absent conditions, regardless of whether they were flat or change runs. Therefore, the difference waves (black lines in Figure 4) overlap considerably with the P300 responses to the rare stimuli (blue lines). This suggests that the P300 responses to the targets exclusively reflect processing of the category change. The differences between the difference waves for feature-present and feature-absent conditions, 28 and their different scalp topographies, will be discussed further in Chapter 4 in the context of the experiments to be discussed in Chapter 3. Treisman and Souther (1985) and Luck and Hillyard (1990) interpreted the differences in response times and P300 latencies between the conditions as indicative of participants using different search strategies for each of the conditions: pre-attentive parallel search in the visual feature-present condition and serial self-terminating search in the visual feature-absent condition. Because auditory stimuli in the present experiment were presented in series rather than simultaneously, however, there was no opportunity for our participants to use the parallel search strategy, nor could their search be self-terminating as participants were not free to search among all distractors at their leisure. This aspect of our study might have been unavoidable. Unlike visual stimuli, spatial location is not sufficient to define boundaries between identical auditory objects (Kubovy & Van Valkenburg, 2001; this was discussed in Chapter 1, but will be repeated here for the sake of clarity). Kubovy and Van Valkenburg used the following example to illustrate this point: whereas two identical spotlights shone in opposite sides of a room will be perceived as two visual objects, two identical tones presented from two different spatial locations will be perceived as a single tone. The only way for identical tones to be distinguished is for them to be staggered in time, or, in other words, played in sequence. This illustrates that it’s not possible for two identical simultaneous auditory events to be heard as distinct objects when they differ only in their spatial location, they must differ in time to be heard as distinct auditory objects. In this sense time is to auditory objects as space is to visual objects. Therefore, it is unlikely that one could reproduce the same effect if one were to present identical auditory distractors simultaneously, 29 as they would be perceived as one event, and the target event would always pop out when it differed in frequency or some other characteristic from the distractors. Cusack and Carlyon (2003) attempted to get around this last problem by creating stimuli consisting of four to thirty-two, 250-ms-long tones, of varying frequencies randomly distributed over the span of 2 seconds. While most of the tones in these trials necessarily overlapped in time, their varying frequencies likely created a blended heterogeneous soundscape, particularly on the trials with 32 distractors, against which the target stimulus could (or could not) be heard. While this simulates a simultaneous auditory search array better than our study design does, it comes at the cost of not having a set of uniquely distinguishable identical distractors, a key feature in typical visual search paradigms. In addition, as Cusack and Carlyon mentioned, informational masking could play a role in modulating the difficulty of the search for the target. In particular, they argued that increasing performance with an increasing number of distractors could be the result of release from informational masking as the soundscape becomes more homogeneous (Oh & Lufti, 1998). Thus it appears to be difficult to arrange auditory stimuli in such a way that the search for a target is strictly analogous to visual search. Indeed, we feel that it is questionable whether any auditory version of the visual search task can be called a search task at all; instead it most closely resembles a sequential stimulus identification task. It is possible, however, that the behavioural and electrophysiological similarities between these tasks reflect a difference in attention distribution strategies that can be applied to search through both space and time, and in all sensory domains. Perhaps in feature-present conditions (and their analogs) a diffuse attention strategy can be applied, where because participants are looking (or listening) for a specific 30 but rarely occurring salient feature, they need only attend superficially to each stimulus. And, perhaps in feature-absent conditions a more focused attention strategy is applied, where participants attend to every individual stimulus in order to integrate all features for identification. It is possible that these strategies manifest in parallel and serial self-terminating search only when perceptual objects are presented simultaneously, or rather when the task requires participants to search through space rather than through time. A plausible analogous strategy is that auditory attention is distributed temporally, particularly tuned to the target tone of each stimulus as it’s presented in a predictable rhythm (Large & Jones, 1999; Jones, Moynihan, MacKenzie, & Puente, 2002). This rhythmic cue could facilitate deployment of attention to each target individually, much like that of serial search in vision. This strategy could be crucial to identifying the feature-absent target, as deployment of focused attention at the correct temporal position of every stimulus is required to identify whether a run is common or rare. Attention does not need to be voluntarily deployed to the end of each stimulus in the feature-present condition, however, as the presence of the unique tone regardless of which temporal position it is in is itself sufficient to identify the target. Thus, in identification of a feature-present auditory target, attention can be deployed diffusely in time, as it can be deployed diffusely in space in search for a feature-present visual target. 31 Chapter 3: Visual sequential search asymmetry (Experiments 2, 3 and 4) Chapter 2 presents behavioural and electrophysiological evidence of search asymmetry in auditory search tasks using patterns of five-tone runs. These data may be inconsistent with the feature integration theory (Treisman & Souther, 1985). Since it was required that the auditory stimuli be presented in series (Kubovy & Van Valkenburg, 2001) there was no opportunity to use parallel search in the feature-present condition, yet behavioural (reaction times) and electrophysiological data (P300 responses) were consistent with those of the visually analogous task believed to require parallel search. Therefore, it would appear that when stimuli are presented sequentially, a necessarily serial search strategy can generate behavioural and electrophysiological signatures previously believed to be indicative of parallel search. This effect could be unique to the auditory system, or a different interpretation of the locus (attention vs perception) of the search asymmetry effect may be required to accommodate the similarities between serial and simultaneous search. We approached this question by designing a visual analog to the sequential auditory search described in Chapter 2 and using it to explore conditions that could help to distinguish between attentional and perceptual explanations for the search asymmetry. The two different auditory stimulus sequences make very different demands upon attention. Detecting rare patterns that include a tone change (change runs) requires only diffuse voluntary (top-down) ongoing attention to the sequence of unchanging patterns, as orienting to the rare pattern easily recruits bottom-up attention when the deviant tone in a rare changing pattern is encountered. In contrast, detection of the rare patterns with no tone change (flat runs) requires sustained, voluntary (top-down) focused attention to each pattern 32 in the sequence comprised of patterns with a tone change, as there is no tone deviant to recruit bottom-up attention in each unchanging pattern (cf. Wright & Ward, 2008). A similar explanation could be applied to both simultaneous and sequential search through visual patterns. On the other hand, the asymmetry effect could be more perceptual in origin. For tone sequences, the additional area of the tonotopic map activated by the change tone would be easily noticed within the overall activity pattern on the auditory cortex (more discriminable), whereas the slight decline in activity in one area in an overall pattern of activation of two areas would be harder to notice (be less discriminable) and would require more scrutiny/processing (e.g., detailed comparison processes either across the tonotopic map or across time in one area of it). Again, a similar explanation could be applied to either sequential or simultaneous visual search. That is, unique elementary features are easily detectable in the feature maps of visual cortex (V1-V4), but search for the absence of such features requires scrutiny because of the additional comparison processes that must occur to notice a slight decline in activation in one specific locus of the map. For our visual analog of the auditory runs we created sequences of common and rare flat and change runs of visual stimuli that required serial search just as did the auditory runs of Chapter 2. The first two experiments (named Experiments 2 and 3, as the auditory experiment in chapter 2 is called Experiment 1) confirmed that serial visual search yields comparable results to auditory search, showing that the auditory search results aren’t unique to the auditory system. Experiment 2 used colour differences (yellow and blue), while Experiment 3 used contrast differences (black and white) to explore sequential search asymmetry. Both colour and contrast are substitutive features (Treisman & Gormican, 1988), 33 where the absence of one implies another, therefore Experiments 2 and 3 were performed to make sure that any asymmetry found was not unique to a specific visual feature, but could generalize to all substitutive features. The third experiment (Experiment 4) served to test the attention orienting strategies proposed by Wright and Ward (2008). The intention of Experiment 4 was to force participants to voluntarily focus their attention to both common and rare runs, regardless of condition, by reducing the salience of the feature-present target. If detecting the feature-present target only required diffuse ongoing attention, then making the targets nearly indistinguishable from the distractors would force participants to focus their attention to every stimulus in both the feature-present and feature-absent conditions, thus eliminating the asymmetry. Methods Participants: Data were collected from 19 participants, who all participated in Experiments 2, 3 and 4. From 15-20 participants has been shown in previous studies in our lab and others to yield reliable EEG data for either ERP or connectivity analyses given the numbers of stimulus trials in the study design. Data collection was stopped after the indicated number of participants had been included. Data from 3 participants were excluded due to excessive noise or stimulus presentation error. The analysis is based on 16 participants (10 female, mean age 21.7 year). Participants were offered monetary compensation ($10/hr) for their participation. Participants were all right handed and reported no vision or neurological difficulties. Stimuli: Stimuli in each experiment consisted of two types of five ring (annulus) runs, called “flat runs” and “change runs”. Flat runs consisted of five identical rings, whereas 34 change runs consisted of four identical rings, followed by a fifth ring that differed by some feature. The fifth ring of the change runs differed in colour in Experiment 2, in contrast in Experiment 3, and grey-scale value in Experiment 4. Runs consisted of a combination of 2 dimensions of the same feature; yellow (RGB = 255,255,0; luminance = 229.6 candela/m2) and blue (0,0,255; 219.3) in Experiment 2, black (0,0,0; 3.4) and white (255,255,255; 380.3) in Experiment 3, and two different shades of grey in Experiment 4, where the lighter shade was always (101,101,101; 58.3), while the darker shade varied between (65,65,65; 41.1) and (51,51,51; 37.7) to accommodate each participant’s ability to discriminate between the flat and change runs using these low contrast colours (one participant, however, required greater discriminability, and thus Experiment 4 was calibrated to (111,111,111; 68.5) as the lighter shade, and (35,35,35; 34.3) and the darker shade for them). These combinations generated two versions of each type of run (see Figure 5). In Experiment 2 the flat runs consisted of either five yellow rings or five blue rings, while the change runs consisted of four yellow rings and one blue ring, or four blue rings and one yellow ring. In Experiment 3 the flat runs consisted of either five white rings or five black rings, while the change runs consisted of four white rings and one black ring, or four black rings and one white ring. Finally, in Experiment 4 the flat runs consisted of either five lighter grey rings or five darker grey rings, while the change runs consisted of four lighter grey rings and one darker grey ring, or four darker grey rings and one lighter grey ring. Rings were presented against an intermediate grey background; RGB values of the background were (200,200,200) for Experiments 2 and 4, and (128,128,128) for Experiment 3. Background shades were not consistent across all experiments because we tried to choose a background shade that would be in equal contrast against each coloured annulus. Unfortunately a single background colour could not achieve 35 that for all 6 colours. Each ring was presented for 100ms, and rings were separated by 50 ms of blank screen. Rings were generated and presented using Presentation software (Neurobehavioural Systems, Berkeley, CA, USA). Each run lasted 700 ms from the initial presentation of the first ring to the end of the last ring. Pauses between each run varied randomly from 700 ms to 1000 ms. Stimuli appeared on a 18.5-in. CRT monitor running at 77 Hz, placed at a viewing distance of approximately 60 cm from the participant. Figure 5. Annulus stimuli. All ring colours in the figure are approximations of ring colours presented to participants. Yellow ring RGB values were (255,255,0), blue rings were (0,0,255), black rings were (0,0,0), white rings were (255,255,255), lighter grey rings were (101,101,101) and darker grey rings varied between (65,65,65) and (59,59,59). Background shades were (200,200,200) for Experiments 2 and 4, while background shades were (128,128,128) (colours in the figure are not exactly the same as those used in the experiments). 36 Procedure: The design used for each experiment was identical to that of Experiment 1 (see Figure 3 for a schematic of the design). The protocol used in Experiments 2, 3, and 4 was also very similar to that described in Chapter 2, save for a few differences. Once participants were comfortably seated approximately 60cm from the monitor in the sound-equalizing chamber, they were familiarized with the stimuli and protocol. Next they participated in a short calibration for Experiment 4. During the calibration 16 runs (2 of each type) were presented in the same random order, and participants were asked to verbally identify each run by saying “flat” or “change”. The shades of grey were adjusted as necessary so that each participant could identify every run with near perfect (no less than 94%, 15/16 correct) accuracy. The calibration was repeated until every participant reached this accuracy threshold. This calibration was only performed for Experiment 4 because during a behavioural pilot run of this protocol participants reported that discriminating the stimuli in Experiments 2 and 3 was very easy without adjustment, while discriminating the stimuli in Experiment 4 was overall much more difficult, and the difficulty varied across participants. Each experiment consisted of 4 oddball sequences (about 4 mins each) presented in randomized order. Each sequence consisted of two runs, one common and one rare. Sequences began with 30 instances of the common run. Thereafter rare runs were presented randomly among common runs. Rare runs were presented on 20% of occasions, and common runs were presented on 80% of occasions, so in each sequence participants saw between 21 and 29 rare runs in each sequence (while this range is accurate for almost all participants and all sequences, in Experiment 3 one participant saw 16 rare runs in one sequence, while in Experiment 4 a different participant saw 39 rare runs in one sequence). There were always at least two common runs between each rare run. Participants were instructed to click a mouse 37 every time they saw a rare run, and their reaction times were recorded. It should be noted that it was made clear to participants that they were only to respond when they saw a rare run, not every time they saw a ring that differed from the previous ring. EEG recording and ERP extraction: EEG recording and ERP extraction were identical to that of Experiment 1 in Chapter 2. It will be repeated, however, for the sake of clarity. EEG data were amplified and analog bandpass filtered from 0.01 Hz to 100 Hz (SA Instrumentation, San Diego, CA, USA). Data were digitized (National Instruments, Inc., Vaudreuil-Dorion, QC, Canada) at 500 Hz from a 60-channel electrode cap (Electrocap, Inc., Eaton, OH, USA, International 10-10 placement) referenced to the right mastoid. Eye movements were recorded with four periocular electrodes. Input impedance of the amplifier was > 2 gΩ, so electrode impedances were kept below 10 kΩ. Initial EEG analysis was conducted using EEGLAB (Delorme & Makeig, 2004) software. Data were down-sampled to 250 Hz, digitally lowpass filtered at 60 Hz, and re-referenced to average reference. Artifacts were rejected using independent component analysis (runica algorithm, EEGLAB; (Viola F. C., et al., 2009). The remaining ERP analysis was conducted using ERPLAB (Lopez-Calderon & Luck, 2014), ERP analysis software that runs in EEGLAB. The continuous EEG data were epoched from -200 to +1000 milliseconds relative to the onset of the last ring. Only the common runs that appeared immediately before every rare run were extracted; this was to make sure there were equal numbers of common and rare runs used for later analysis. Finally, ERPs were low-pass filtered at 30 Hz before plotting. 38 Results The following analyses were identical to those used in Chapter 2, however aspects of the analyses will be repeated for the sake of clarity. Again, to be consistent with the vision terminology, we call the sequences in which the rare runs were the flat runs (all rings were the same colour) the ‘feature-absent’ condition, and those in which the rare runs were the change runs (the last ring was a different colour) the ‘feature-present’ condition. Again, mean response times, hits, false alarms, and ERP latencies and amplitudes were compared with dependent means t-tests with df=15 for all comparisons. ERP latencies were determined by taking the peak latency (the latency at the highest amplitude) of the difference wave between 200-800 ms post stimulus onset per subject, then averaging across all subjects. Amplitudes were determined by taking the average amplitude of the difference wave over an interval 20 ms to either side of the peak per subject, then averaging across subjects. For effect size we used es = √(𝑡2 (𝑡2 + 𝑑𝑓)⁄ ) (Field, 2009). In general, the results for Experiment 2 (colour) and Experiment 3 (contrast) were very similar, so they will be described together. In contrast, the results for Experiment 4 were different, so will be discussed separately in each category. 39 Table 2. Experiment 2 behavioural and electrophysiological results. Results are based on the average of all individual differences between conditions. Significant differences are indicated with an asterisk (*). Effect size (es) estimates are only included for significant differences. Target Run RT (ms) Hit (%) FA (%) Lat (ms) Am (µV) Change Yellow to Blue (Y2B) 389 97.4 0.2 350 5.8 Flat Yellow (Y) 488 96.5 0.4 442 4.5 Difference (Y – Y2B) 99 2.4 0.3 92 1.4 p 0.0000001* 0.53 0.30 0.02* 0.05 t(15) (es) 9.60(0.93) 0.64(N/A) 1.07(N/A) 2.70(0.57) 2.09(N/A) Change Blue to Yellow (B2Y) 377 96.6 0.1 371 5.1 Flat Blue (Blu) 500 96.7 0.8 499 4.4 Difference (Blu – BY) 123 0.2 0.7 128 0.7 p 0.000001* 0.95 0.006* 0.0009* 0.35 t(15) (es) 8.15(0.90) 0.06(N/A) 3.24(0.64) 4.13(0.73) 0.95(N/A) 40 Table 3. Experiment 3 behavioural and electrophysiological results. Results are based on the average of all individual differences between conditions. Significant differences are indicated with an asterisk (*). Effect size (es) estimates are only included for significant differences. Target Run RT (ms) Hit (%) FA (%) Lat (ms) Am (µV) Change White to Black (W2B) 474 99.2 0.05 409 6.7 Flat White (W) 521 97.0 0.6 474 4.9 Difference (W – W2B) 48 2.3 0.6 66 1.8 p 0.006* 0.19 0.003* 0.04* 0.02* t(15) (es) 3.21(0.64) 1.39(N/A) 3.49(0.67) 2.31(0.51) 2.72(0.57) Change Black to White (B2W) 382 99.5 0.2 373 6.7 Flat Black (Bla) 474 95.0 1.1 470 4.7 Difference (Bla – B2W) 92 4.5 0.9 97 2.0 p 0.000002* 0.07 0.04* 0.002* 0.001* t(15) (es) 7.51(0.89) 1.97(N/A) 2.21(0.50) 3.71(0.69) 3.90(0.71) 41 Table 4. Experiment 4 behavioural and electrophysiological results. Results are based on the average of all individual differences between conditions. There were no significant differences, therefore no effect sizes were estimated. Target Run RT (ms) Hit (%) FA (%) Lat (ms) Am (µV) Change Light to Dark (L2D) 519 70.5 2.0 506 4.1 Flat Light (L) 549 67.5 2.9 541 3.3 Difference (L – LD) 30 3.0 0.9 35 0.9 p 0.25 0.53 0.49 0.44 0.14 t(15) (es) 1.21(N/A) 0.65(N/A) 0.72(N/A) 0.80(N/A) 1.56(N/A) Change Dark to Light (D2L) 540 72.6 1.9 521 4.3 Flat Dark (D) 561 78.0 2.2 580 3.8 Difference (D – DL) 21 5.4 0.3 59 0.5 p 0.57 0.37 0.71 0.12 0.39 t(15) (es) 0.57(N/A) 0.93(N/A) 0.38(N/A) 1.64(N/A) 0.89(N/A) Behavioural Results. The response times to correctly detect the rare target run (hits) were calculated from the onset of the fifth ring in the run, separately for feature-absent conditions and the feature-present conditions for all three experiments (6 of each in total). Tables 2 and 3 summarize the behavioural results of Experiments 2 and 3, respectively. In general, Experiments 2 and 3 yielded similar results to those of the auditory Experiment 1 described in Chapter 2. In Experiments 2 and 3 the reaction time to identify the feature-absent target was significantly longer than to identify the corresponding feature-present target. Corresponding conditions were those that shared stimuli (for example flat yellow) but 42 whereas in one condition that stimulus was common (feature-present), in the other condition it was rare (feature-absent), and vice versa for the change yellow to blue stimulus. Correct responses were defined as a response to a rare target run that occurred before the onset of the final ring of the following non-target run. This time window varied between 1400 ms and 2000 ms after the onset of the final ring in a target run. Mean percent correct detections were at ceiling (around 97%) for both the feature-absent and feature-present conditions, and were not significantly different. False alarms, responses to common (non-target) runs, were very infrequent, less than 1.2% (1 in 84) overall, ruling out a large speed-accuracy tradeoff. Similar to Experiment 1, in over 68% of the blocks in Experiment 2 and in over 60% of the blocks in Experiment 3 of the individual data no false alarms at all occurred, rendering a signal detection analysis uninformative. In Experiment 4, the latency responses to the feature-present and feature-absent targets did not differ significantly (Table 4). Mean percent correct detections also did not differ significantly between feature-present and feature-absent conditions, however they were much lower (around 70%) than those of Experiments 2 and 3. False alarms, though larger than in Experiments 2 and 3, were still very infrequent, less than 3%. P300 ERP results at Pz: As in the Chapter 2 analysis, we computed the difference waves from the ERPs to the final ring of each run type when that run type was rare and when it was common. The ERPs to the last rings (onset at 600 ms relative to the onset of the first ring) in each run type, and the respective difference waves comparing the ERPs to the identical run type when it was rare and when it was common, are displayed in Figure 6 for the Pz electrode, and in Figures A5-A16 for all electrodes. 43 Figure 6 demonstrates that the P300 responses in Experiments 2 and 3 are very similar to those of the Experiment 1 in Chapter 2. In both experiments there was a clear P300-like response peaking between about 300 ms and 500 ms post onset of the fifth ring to both feature-present and feature-absent targets (red lines, solid for feature-present and dashed for feature-absent targets) and no such response when those same runs were common (blue lines, solid for feature-present and dashed for feature-absent common runs). As in Experiment 1, the difference waves (black, solid for feature-present and dashed for feature-absent, rare minus common runs) for each stimulus type are nearly indistinguishable from their corresponding target response in red, because of the low activity generated by the common runs. The shapes of the difference waves, the latencies of the peaks of the P300 responses, and the latencies of the difference waves differed substantially, however, between the flat (feature-absent) and change (feature-present) runs. In general, the feature-present target peaked significantly earlier, approximately 96 ms (between 66 ms and 128 ms) than its corresponding feature-absent target (cf. Tables 2 and 3). On the other hand, the amplitudes of the difference waves were not substantially different between feature-present and feature-absent targets in Experiment 2, they were significantly different in Experiment 3 (see Table 3). We also analyzed the differences between the areas under the P3 curves (numerical integration of the curves) and the results were similar to those just detailed and so are not reported here. In Experiment 4, the P300 responses are quite different (see Figure 6). There appears to be a P300-like response in both feature-present and feature-absent targets, though they peak much later (between about 400 ms and 700 ms post onset of the fifth ring) and their amplitudes are greatly attenuated compared to those from Experiments 2 and 3. Because of 44 the greater variability in the responses to both the target and common runs compared to those of Experiments 2 and 3, the difference waves for each stimulus type are more distinguishable from their corresponding target responses in red, though they are still very similar. The shapes of the difference waves are both broad and smooth, though the feature-present responses are slightly “peakier” than those of the feature-absent responses. Though visually the feature-present target appears to peak earlier and with higher amplitude than the feature-absent target, the peak latencies, amplitudes and areas under the curve are not significantly different (see Table 4). P300 topographies: In Experiments 2 and 3, The P300 response was greatest at the parietal scalp sites for both feature-present and feature absent targets. In the feature-present conditions the P300 had sharpest peak at the parietal sites, while the response was quickly attenuated beyond centro-parietal and occipital sites. In addition, an early positivity was observed peaking first at the frontal then central sites about 150 ms post stimulus. In the feature-absent conditions the scalp distribution showed a similar pattern to the feature-present P300 response, where the amplitude was also largest at parietal sites and quickly attenuated beyond centro-parietal and occipital sites, but the shapes of the waves were much broader and shallower. No early frontal positivity was observed to feature-absent targets. In Experiment 4 the P300 responses to both feature-present and feature-absent targets were greatest at the parietal scalp sites, however the response was much more quickly attenuated, and did not extend beyond parietal sites. Both P300 responses were broad and shallow, though more so to the feature-absent than feature-present targets. There was no evidence of an early frontal positivity to either feature-present or feature-absent targets. 45 Figure 6. ERPs at Pz to the four run types per experiment (12 total) when they were rare or common and the respective difference waves. Each panel compares the difference waves for feature-present and feature-absent conditions for run types in which the first four rings in the run were identical. The stimuli that the ERPs are in response to are in next to each ERP. ERPs at more electrode sites are displayed in Figures A5-A16. Discussion The results of Experiments 2 and 3 constitute evidence for search asymmetry using sequentially presented visual stimuli. Participants were asked to identify visual target patterns among arrays of distractors while their response times and electrophysiological responses were recorded. Our behavioural results were similar to those of a visual search task designed by Treisman and Souther (1985), as well as the results observed in the analogous auditory Experiment 1 presented in Chapter 2: target objects containing a feature-difference that distractors lacked (feature-present condition) were identified faster than target objects that were missing a feature-difference that distractors possessed (feature-absent condition). Our 46 ERP results were also highly similar to those of Luck and Hillyard (1990) for simultaneously presented visual stimuli, and the auditory stimuli used in Experiment 1. These results provide evidence that the similarities observed between sequentially presented auditory and simultaneously presented visual stimuli are not unique to the auditory system; searching for targets of different feature compositions appears to use similar strategies, or at least yield similar behavioural and electrophysiological results, regardless of whether the items in the arrays are visual or auditory, or are presented all at once or one at a time. The intention of Experiment 4 was to investigate whether the locus of the search asymmetry effect was at the level of perception or attention. It was hypothesized that by reducing the salience of the feature difference present in the change run, participants would need to focus their attention to all stimuli, regardless of whether they were searching for the feature-present or feature-absent target. Should the asymmetry disappear under these conditions, this would constitute evidence of the locus of the effect being at the level of attention (Wright & Ward, 2008). However, should the asymmetry persist, the locus should most likely be at the level of perception. The results of Experiment 4 are mixed. While visually the P300 responses to the feature-present targets appear to be sharper, earlier, and larger than the P300 responses to the feature-absent targets, they are not significantly different on any of those measures. What’s more, reaction time results were not significantly different, suggesting that the asymmetry appears to have disappeared under these conditions. However, given the high variability in the electrophysiological (s latency = 125.5, s amplitude = 2.97) and behavioural data (s RT= 121.5) relative to that of Experiments 2 and 3 (s latency = 103.6, s amplitude = 2.88, s RT = 89.9), it is possible that the present study does not have enough power to illustrate these 47 differences statistically. While clear differences in behavioural and electrophysiological responses were observed by the same subjects with the same number of trials in Experiments 2 and 3, more subjects or more trials may be necessary to see these same differences in Experiment 4 given the reduced saliency of the feature difference. Aside from collecting more data for Experiment 4, there are other potential ways to uncover the locus of the effect using a similar protocol. Past experiments investigating the MMN have used a roving auditory oddball paradigm, in which auditory arrays of single simple tones would periodically change in frequency; for example participants might hear 6 tones at 500 Hz, then 4 tones at 700 Hz, then 5 tones at 350 Hz, etc (Garrido, et al., 2008; Haenschel, Vernon, Prabuddh, Gruzelier, & Baldeweg, 2005; Cowan, Winkler, Teder, & Näätänen, 1993). A similar roving adaptation of Experiment 1 could be ideal to investigate the locus of the search asymmetry effect. In this adaptation, the stimulus arrays would include multiple kinds of feature-present and feature-absent stimuli, consisting of different combinations of tones. In other words, auditory arrays would consist of flat runs of five 500-Hz tones, five 600-Hz tones, etc, and change runs would consist of four 500-Hz tones and one 700-Hz tone, another of four 600-Hz tones and one 300-Hz tone, etc. Whereas auditory arrays would be of the same oddball format that was used in all four of the present experiments, where one type of run would always be common and the other type of run would always be rare, the tonal composition of the runs would vary throughout the sequences. This would mean that there would be no specific feature that participants could listen for to identify the rare runs among the common runs, there would only be salient feature differences that could identify the targets from the common runs. As in Experiment 4, participants would also need to focus their attention to each stimulus in order to identify the 48 targets regardless of which stimulus, flat or change, they were listening for. Unlike Experiment 4, however, the stimuli in the roving paradigm do maintain the salient feature difference that was present in our Experiments 1, 2 and 3. By using this roving adaptation, we could more likely determine the locus of the effect with the same number of trials and participants as in Experiments 1, 2 and 3. 49 Chapter 4: General discussion Taken together, the results of Experiments 1, 2 and 3 provide strong evidence that important properties of search through sequentially presented auditory arrays (Experiment 1), and visual arrays (Experiments 2 and 3) strongly resemble those of search through static, briefly-presented, visual arrays (Treisman & Souther, 1985) and analogous, very brief, auditory arrays (Cusack & Carlyon, 2003). In particular, searches through the sequentially-presented arrays exhibited a kind of asymmetry of response that was first discovered in searches through static visual arrays (Treisman & Souther, 1985). Search times and P300 latencies to identify targets were significantly faster in the feature-present condition than in the feature-absent condition. These results suggest that the search strategies used to search through sequentially presented stimuli may be very similar to the search strategies used to search through static visual arrays. What’s more, it is clear from these similarities that a simple conceptualization of “parallel search” might not be a useful way to think about the search strategy associated with identifying feature-present targets, as even when stimuli are presented one at a time, when parallel search is impossible, we see search outcomes similar to when stimuli are presented simultaneously when parallel search is theoretically possible. It remains unclear, however, whether the locus of the asymmetry effect is at the level of attention or perception, and further investigation will be required in order to tease these apart. In addition to their latencies, so are the shapes of the P300 responses very different between feature-present and feature-absent conditions. Figures 4 and 6 show that the P300 ERP to the feature-present target is initially larger and sharper, but broadens out at the same rate as that to the feature-absent target. These different shapes suggest that the neurological processes underlying detection of these targets may be slightly different. Recently, it has 50 been discovered that the P300 may contain sub-components that are associated with distinct neurological processes (Polich, 2007). One component, the P3a, is most clearly revealed in association with detection of task irrelevant oddballs, or “distractors”, which are rare, highly salient, unexpected stimuli that are not targets and therefore typically do not incur a response from participants (Polich & Comerchero, 2003). The P3a is typically quite sharp and can occur as early as 200 ms post stimulus onset. It is strongest at centro-parietal electrode sites, and can extend as far anterior as FZ. The P3a is thought to be stimulus-driven, and is believed to originate from fronto-parietal attention mechanisms that are involved in orienting attention to highly salient, unexpected stimuli (Corbetta, Patel, & Shulman, 2008). A second component, the P3b, is much broader and later than the P3a, and is associated with task-relevant oddballs, or target, identification. Its amplitude is maximal over parietal sites, but is quickly attenuated as it travels away from PZ. It is believed to originate from temporo-parietal networks, which are thought to reflect memory and stimulus encoding mechanisms associated with context updating (Polich, 2007). Though each of these sub-components has been investigated on its own, it is believed that “…every ‘P300’ is composed of the P3a and P3b subcomponents, but the resulting ERP scalp topographies vary with the stimulus and task conditions that elicit them (Polich, 2007, Figure 8 caption, pg 2138).” (cf., e.g., Polich, 2007, Figures 1, 5 and 8, or see Figure A17). In other words, P300 shapes and latencies are highly sensitive to task demands, because different tasks will require different demands on attention (P3a) and memory (P3b), which will generate P3a and P3b responses at different strengths. It would appear, then, that the different task demands of identifying the feature-present and feature-absent targets do in fact elicit different contributions of these neural 51 responses in our experiments. The feature-present condition seems to elicit characteristics of both the P3a and P3b response, namely the sharp, early peak of the P3a but with a slow return to baseline, taking approximately 400 ms to reach the zero line, which is characteristic of the P3b. This is consistent with the current understanding of the conditions under which the P3a and P3b are typically observed: the feature-present target is not only a task-relevant oddball (P3b), but also contains a salient perceptual oddball that is both infrequent and unpredictable (P3a). We will use the study design of Experiment 1 (auditory) to illustrate this in detail. As per the methods described in Bekinschtein (2009) (on which all 4 experiments were based), in every sequence the target stimulus was heard at a 20% probability (approximately between 18 and 30 times per block). Given that each stimulus consisted of 5 tones (in Experiment 1), in any feature-present sequence of, e.g., 100 trials, participants would hear on average 480 instances of the same tone (500 Hz if the common run was flat low, 1000 Hz if the common run was flat high), and only 20 instances of the other tone (1000 Hz if the common run was flat low, 500 Hz if the common run was flat high). Not only were these infrequent and unpredictable tones also highly perceptually salient given the tonal context, they also happened to index the target stimulus. Therefore it’s not surprising that the feature-present target P300 response is characteristic of both the P3a and P3b, because the target shares properties with the types of stimuli with which these responses are typically associated. This same logic can also be applied to the results of both Experiments 2 and 3. In contrast, the broad and relatively flat shapes of the P300 ERPs in response to the feature-absent targets, in addition to their scalp topographies, suggest that they are mostly P3b ERPs, which is also consistent with the literature (e.g., Polich, 2007). In contrast with 52 the feature-present target, the feature-absent target does not contain a highly perceptually salient feature change; there isn’t anything perceptually infrequent or unpredictable indexing the feature-absent target. It is simply made up of five of the most frequent stimuli (tones or rings) instead of only four of them. Therefore it’s not surprising that the feature-absent target does not elicit a P3a, as the P3a is most strongly associated with responses to infrequent, unpredictable, perceptually-salient stimuli (Polich, 2007). Given the immediate sensory (visual or acoustic) context of the feature-absent condition, the feature-absent target does not contain a stimulus that would be expected to elicit a P3a response. The feature-absent common run, however, does contain a relatively infrequent, perceptually salient stimulus change at the end of each common change run, so one might be inclined to expect that each feature-absent common run would produce a P3a response. Using the previous example (Experiment 1), in any block of 100 trials where the feature-absent run was the rare target, participants would hear on average 420 instances of the same tone (500 Hz if the common run was change up and 1000 Hz if the common run was change down) and 80 instances of the other tone (1000 Hz if the common run was change up and 500 Hz if the common run was change down). Though the perceptually-salient stimulus in this case is also relatively infrequent, in the context of the feature-absent condition it is nonetheless highly predictable, occurring on 80% of trials, and therefore should not generate a P3a. This is indeed what was found in the present study, as can be seen in Figures 4 and 6, where the responses to the common change runs in the feature-absent condition are generally around baseline and virtually indistinguishable from the responses to the common flat runs in the feature-present condition, which do not contain such perceptually-salient stimuli. Taken together, every common (change) run of the feature-absent-condition contains an infrequent, perceptually-53 salient, but predictable stimulus that doesn’t elicit a P3a, whereas every rare target (change) run of the feature-present condition contains an infrequent, perceptually-salient, unpredictable stimulus (that also serves to identify the target) that does elicit a P3a. It has been shown also that the P300 ERP is sensitive to the discriminability of the target in a 2-stimulus oddball paradigm (Polich, 2003). Polich compared P300 responses to infrequent target stimuli that were both easy and difficult to discriminate from standards. The P300 responses to targets that were easy to discriminate were also much earlier, sharper, and larger than the P300 responses to the targets that were difficult to discriminate, which were later, smaller and broader. Though not the central focus of that study, those results are consistent with what was found in Experiments 2 and 3, which suggests that these particular morphologies are characteristic of easy vs difficult task demands. An infrequent, unpredictable target that is easy to discriminate from standards is also likely to be highly perceptually salient, and thus might also elicit a stronger P3a sub-component than would a target that is very similar to the standards. This could be the reason for the difference in morphologies found by Polich (2003) for easy and difficult tasks. 54 Figure 7. Scalp maps of the change run difference waves, calculated by subtracting the ERP to each stimulus type when it was common from that to the identical stimulus type when it was rare, between 100 ms and 550 ms post stimulus, for all four experiments. Figure 8. Scalp maps of the flat run difference waves, calculated by subtracting the ERP to each stimulus type when it was common from that to the identical stimulus type when it was rare, between 100 ms and 550 ms post stimulus, for all four experiments. 55 Figure 9. P300 difference waves for feature-present (magenta) and feature-absent (green) midline electrode sites for all four experiments. The distinction between P3a and P3b is reflected somewhat in the scalp topographies as well. Figures 7 and 8 show the scalp maps of the change run and flat run difference waves (these are the difference waves calculated by subtracting the ERP to each stimulus type when it was common from that to the identical stimulus type when it was rare) between 100 ms and 550 ms post stimulus, respectively. The scalp topographies for the feature-absent targets are consistent with the P3b: they are maximal over parietal sites, and are quickly attenuated as the response spreads from PZ (Figure 8). This can be seen in Figure 9 as well, which shows the P300 difference waves for feature-present (magenta) and feature-absent (green) midline electrode sites. The response is maximal at PZ, but by CZ the response has been attenuated 56 by about half, and it is completely absent at FCZ and inverted at FZ. The topographies for the feature-present target, however, are consistent with the presence of a strong P3a in Experiment 1. Figure 7 shows that between 200 and 250 ms the response is strongest at central sites, then progressively spreads in the posterior direction until it reaches its maximum at parietal sites between 300 and 350 ms. This is also reflected in Figure 9, as the positivity seen at FZ around 200 ms post stimulus travels back to PZ over the course of about 100 ms, increasing in amplitude as it spreads backward. For Experiments 2 and 3, however, the P300 response is much more restricted to the parietal sites, and is quickly attenuated as it spreads in the anterior direction, more so than the P3a would be. Perhaps the feature differences in Experiments 2 and 3 are not salient enough to elicit the P3a-like topography that is evident in Experiment 1. In contrast, both feature-present and feature-absent responses in Experiment 4 show characteristic P3b topography, the only difference being that the feature-present target reaches a slightly larger maximum than the feature-absent target. Though, for the most part, Experiments 1, 2 and 3 have generated very comparable results, it is clear from Figure 9 that there are some key differences in ERP topography, aside from the P3a topographical differences. For one, in Experiments 2 and 3 the feature-present condition difference wave shows a very early positivity, beginning at approximately 150 ms after stimulus onset, which is maximal at frontal electrodes and slowly attenuated as it spreads in the posterior direction toward PZ. Though it’s possible that this could be a very early P3a-like component, given its early latency it is unlikely to be in the P300 family. It is more likely to be a visual P2 component, as it peaks at frontal sites around 200 ms post stimulus onset, and, as can be seen in Figure 7, there is a negativity at posterior sites at around the same time (Key, Dove, & Maguire, 2005). A visual P2, however, is typically 57 associated with stimulus detection, regardless of its global context, so one would assume that both the common and rare runs would generate a visual P2, and subsequently the effect would be subtracted out when calculating the difference wave. Figure A18 shows the P2 at FZ and inversion at Oz in Experiments 2 and 3 to all stimuli, regardless of condition, It can be seen there is evidence of this effect in both the common and rare run, but it is much earlier in the rare run, so the subtraction failed to cancel out the effect. This further suggests that the early positivity is likely to be a visual P2. Similarly, the Experiment 1 change run difference wave shows an early, frontal negativity that is maximal at central sites and spreads anterior until FZ, but is quickly attenuated posteriorly (Figure 9), and is inverted at the temporal sites (Figure 7, 200-250 ms). This is most likely an MMN, which is an automatic response to novel auditory stimuli in an array (Näätänen, Paavilainen, Rinne, & Alho, 2007). Similar to the P2, the MMN is an automatic response to whenever a novel sound is heard, so an MMN should be generated by both common and rare change runs regardless of condition, and therefore one would expect that this effect would be canceled out by the difference wave. However, as can be seen in Figure A19, an MMN has been generated to all change runs regardless of whether they were common or rare, but again due to the latency difference of the MMN between these two conditions the effect failed to be canceled out in the difference wave. Finally, this work, like many models of visual search developed in response to FIT (Treisman & Gormican, 1988), call into question the utility of needing two distinct search strategies to explain ambiguous search phenomena. As many have pointed out (Wolfe, 1994; Dosher, Han & Lu, 2004, Wolber & Wascher, 2003) the strict distinction between serial and parallel search may be artificial, and in many cases could be a byproduct of experiment 58 design and/or stimulus design characteristics. Furthermore, perhaps the difference in P300 responses to feature-present and feature-absent targets are not in themselves consequences of identifying targets using two distinct search strategies, but are simply consequences of performing easy (feature-present) vs difficult (feature-absent) searches, arising from differences in target saliency (Polich, 2007) or distractor complexity (Duncan & Humphreys, 1989; Shen & Reingold, 2001). Even after over 50 years of investigation into search phenomena (Neisser, Novick, & Lazar, 1963; Neisser, 1967; LaBerge, 1973), still more work needs to be done to answer these questions. 59 Bibliography Ashbridge, E., Walsh, V., & Cowey, A. (1997). Temporal aspects of visual search studied by transcranial magnetic stimulation. Neuropsychologia, 35(8), 1121-1131. Bekinschtein, T., Dehaene, S., Rohaut, B., Tadel, F., Cohen, L., & Naccache, L. (2009). Neural signature of the conscious processing of auditory regularities. Proceedings of the National Academy of Sciences of the United States of America, 106(5), 1672-1677. Bichot, N., Rossi, A., & Desimone, R. (2005). Parallel and serial neural mechanisms for visual search in macaque area V4. Science, 308(5721), 529-534. Bishop, D., O'reilly, J., & McArthur, G. (2005). Electrophysiological evidence implicates automatic low-level feature detectors in perceptual asymmetry. Cognitive Brain Research, 24(1), 177-179. Botella, J., & Eriksen, C. (1992). Filtering versus parallel processing in RSVP tasks. Perception & psychophysics, 51(4), 334-343. Broadbent, D. E. (1977). The hidden preattentive processes. American Psychologist, 32, 109-118. Broadbent, D., & Broadbent, M. (1987). From detection to identification: response to multiple targets in rapid serial visual presentation. Perception & Psychophysics, 42(2), 105-113. Corbetta, M., Patel, G., & Shulman, G. (2008). The Reorienting System of the Human Brain: From Environment to Theory of Mind. Neuron, 58(3), 306-324. 60 Cowan, N., Winkler, I., Teder, W., & Näätänen, R. (1993). Memory prerequisites of mismatch negativity in the auditory event-related potential (ERP). Journal of Experimental Psychology: Learning, Memory, and Cognition, 19(4), 909-921. Cusack, R., & Carlyon, R. (2003). Perceptual Asymmetries in Audition. Journal of Experimental Psychology: Human Perception and Performance, 29(3), 713-725. Delorme, A., & Makeig, S. (2004). EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. Journal of Neuroscience Methods, 134(1), 9-21. Dosher, B., Han, S., & Lu, Z. (2004). Parallel processing in visual search asymmetry. Journal of experimental psychology. Human perception and performance, 30(1), 3-27. Duncan, J., & Humphreys, G. (1989). Visual search and stimulus similarity. Psychological Review, 96(3), 433-458. Duncan, J., & Humphreys, G. (1992). Beyond the search surface: visual search and attentional engagement. Journal of experimental psychology. Human perception and performance, 18(2), 578-593. Field, A. P. (2009). Discovering Statistics Using SPSS. Third Edition. London: Sage Publications. Garrido, M., Friston, K., Kiebel, S., Stephan, K., Baldeweg, T., & Kilner, J. (2008). The functional anatomy of the MMN: a DCM study of the roving paradigm. NeuroImage, 42(2), 936-944. Gescheider, G. A. (1997). Psychophysics: the fundamentals. Mahwah, NJ: Lawrence Erlbaum Associates. 61 Haenschel, C., Vernon, D. J., Prabuddh, D., Gruzelier, J. H., & Baldeweg, T. (2005). Event-related brain potential correlates of human auditory sensory memory-trace formation. Journal of Neuroscience, 25(45), 10494–10501. Humphreys, G., & Muller, H. (2000). A Search Asymmetry Reversed by Figure-Ground Assignment. Psychological Science, 11(3), 196-201. Jones, M., Moynihan, H., MacKenzie, N., & Puente, J. (2002). Temporal aspects of stimulus-driven attending in dynamic arrays. Psychological science : a journal of the American Psychological Society / APS, 13(4), 313-319. Key, A., Dove, G., & Maguire, M. (2005). Linking brainwaves to the brain: an ERP primer. Developmental neuropsychology, 27(2), 183-215. Kok, A. (2001). On the utility of P3 amplitude as a measure of processing capacity. Psychophysiology, 38(3), 557-577. Kubovy, M., & Van Valkenburg, D. (2001). Auditory and visual objects. Cognition, 80(1-2), 97–126. LaBerge, D. (1973). Attention and the measurement of perceptual learning. Memory and Cognition, 1, 268-276. Large, E., & Jones, M. (1999). The Dynamics of Attending: How People Track Time-Varying Events. Psychological Review, 106(1), 119-159. Lawrence, D. H. (1971). Two studies of visual search for word targets with controlled rates of presentation. Perception & psychophysics, 10, 85-89. Leonards, U., Sunaert, S., Van Hecke, P., & Orban, G. (2000). Attention mechanisms in visual search - an fMRI study. Journal of Cognitive Neuroscience, 2, 61-75. 62 Li, Z. (1999). Contextual influences in V1 as a basis for pop out and asymmetry in visual search. Proceedings of the National Academy of Sciences of the United States of America, 96(18), 10530-10535. Lopez-Calderon, J., & Luck, S. J. (2014). ERPLAB: An open-source toolbox for the analysis of event-related potentials. , 8:213. doi: 10.3389/fnhum.2014.00213. Frontiers in Human Neuroscience, 8(213), 1-14. Luck, S. J., & Hillyard, S. (1990). Electrophysiological evidence for parallel and serial processing during visual search. Perception & Psychophysics, 48(6), 603-617. McLean, J., Broadbent, D., & Broadbent, M. (1983). Combining attributes in rapid serial visual presentation tasks. The Quarterly journal of experimental psychology Section A, Human experimental psychology, 35(1), 171-186. Näätänen, R., Paavilainen, P., Rinne, T., & Alho, K. (2007). The mismatch negativity (MMN) in basic research of central auditory processing: A review. Clinical Neurophysiology, 118(12), 2544-2590. Neisser, U. (1967). Cognitive Psychology. New York: Appleton-Century-Crofts. Neisser, U., Novick, R., & Lazar, R. (1963). Searching for ten targets simultaneously. Perceptual and Motor Skills, 17, 955-961. Oh, E., & Lufti, R. (1998). Nonmonotonicity of informational masking. The Journal of the Acoustical Society of America, 104(6), 3489-3499. Overvliet, K., Mayer, K., Smeets, J., & Brenner, B. (2008). Haptic search is more efficient when the stimulus can be interpreted as consisting of fewer items. Acta Psychologica, 127(1), 51-56. 63 Pickles, J. O. (1988). An Introduction to the Physiology of Hearing (2nd Ed.). San Diego: Academic Press. Plaisier, M., Bergmann Tiest, W., & Klappers, A. (2008). Haptic pop-out in a hand sweep. Acta Psychologica, 128(2), 368-377. Polich, J. (2007). Updating P300: An Integrative Theory of P3a and P3b. Clinical Neurophysiology, 118(10), 2128-2148. Polich, J., & Comerchero, M. (2003). P3a from visual stimuli: Typicality, task, and topography. Brain Topography, 15(3), 141-152. Pollack, I. (1975). Auditory informational masking. Journal of the Acoustical Society of America, 57(S5). Rauschenberger, R., & Yantis, S. (2006). Perceptual encoding efficiency in visual search. Journal of Experimental Psychology-General, 135(1), 116-131. Sanders, L., & Astheimer, L. (2008). Temporally selective attention modulates early perceptual processing: Event-related potential evidence. Perception & Psychophysics, 70(4), 732-742. Sathian, K., & Burton, H. (1991). The role of spatially selective attention in the tactile perception of texture. Perception & Psychophysics, 50(3), 237-248. Schubö, A., Schröger, E., & Meinecke, C. (2004). Texture segmentation and visual search for pop-out targets. An ERP study. Cognitive Brain Research, 21(3), 317-334. Shapiro, K., Raymond, J., & Arnell, K. (1997). The attentional blink. Trends in cognitive sciences, 1(8), 291-296. Shen, J., & Reingold, E. (2001). Visual search asymmetry: the influence of stimulus familiarity and low-level features. Perception & Psychophysics, 63(3), 464-475. 64 Timm, J., Weise, A., Grimm, S., & Schröger, E. (2011). An Asymmetry in the Automatic Detection of the Presence or Absence of a Frequency Modulation within a Tone: A Mismatch Negativity Study. Frontiers in Psychology, 2(189), 1-8. Treisman, A. (1982). Perceptual Grouping and Attention in Visual Search for Features and for Objects. Journal of experimental psychology. Human perception and performance, 8(2), 194-214. Treisman, A. M., & Gelade, G. (1980). A feature-integration theory of attention. Cognitive Psychology, 12, 97-136. Treisman, A., & Gormican, S. (1988). Feature analysis in early vision: evidence from search asymmetries. Psychological Review, 95(1), 15-48. Treisman, A., & Souther, J. (1985). Search Asymmetry: A Diagnostic for Preattentive Processing of Separable Features. Journal of Experimental Psychology: General, 114(3), 285-310. Viola, F. C., Thorne, J., Edmonds, B., Schneider, T., Eichele, T., & Debener, S. (2009). Semi-automatic identification of independent components representing EEG artifact. Clinical Neurophysiology, 120, 868-877. von Grünau, M., & Dubé, S. (1994). Visual search asymmetry for viewing direction. Perception & Psychophysics, 56(2), 211-220. Wang, L., Zhang, K., He, S., & Jiang, Y. (2010). Searching for life motion signals. Visual search asymmetry in local but not global biological-motion processing. Psychological Science, 21(8), 1083-1089. Wang, Q., Cavanagh, P., & Green, M. (1994). Familiarity and pop-out in visual search. Perception & Psychophysics, 56(6), 495-500. 65 Watson, C. S. (1987). Uncertainty, informational masking, and the capacity of immediate auditory memory. In W. A. Yost, & C. S. Watson, Auditory Processing of Complex Sounds (pp. 267-277). Hillsdale, NJ: Lawrence Erlbaum Associates, Inc. Whang, K., Burton, H., & Shulman, G. (1991). Selective attention in vibrotactile tasks: detecting the presence and absence of amplitude change. Perception & Psychophysics, 50(2), 157-165. Wolber, M., & Wascher, E. (2003). Visual search strategies are indexed by event-related lateralizations of the EEG. Biological Psychology, 63(1), 79-100. Wolfe, J. (1994). Guided Search 2.0-A revised model of visual search. Psychonomic Bulletin & Review, 1(2), 202-238. Wolfe, J. (1998). What can 1 million trials tell us about visual search? Psychological Science, 9(1), 33-39. Wolfe, J., Cave, K., & Franzel, S. (1989). Guided search: an alternative to the feature integration model for visual search. Journal of experimental psychology. Human perception and performance, 15(3), 419-433. Wright, R. D., & Ward, L. M. (2008). Orienting of Attention. New York: Oxford University Press. 66 Appendix Experiment 1 Scalp ERPs (auditory, 500 Hz and 1000 Hz tones) Figure A1. Experiment 1 scalp ERPs, feature-present condition. Difference waves (black, dashed) were determined by subtracting common flat-high ERPs (blue) from rare change-down target ERPs (red). Figure A 2. Experiment 1 scalp ERPs, feature-absent condition. Difference waves (black, dashed) were determined by subtracting common change-down ERPs (blue) from rare flat-high target ERPs (red). 67 Figure A3. Experiment 1 scalp ERPs, feature-present condition. Difference waves (black, dashed) were determined by subtracting common flat-low ERPs (blue) from rare change-up target ERPs (red). Figure A4. Experiment 1 scalp ERPs, feature-absent condition. Difference waves (black, dashed) were determined by subtracting common change-up ERPs (blue) from rare flat-low target ERPs (red). 68 Experiment 2 Scalp ERPs (colour, yellow and blue rings) Figure A5. Experiment 2 scalp ERPs, feature-present condition. Difference waves (black, dashed) were determined by subtracting common flat-blue ERPs (blue) from rare change-blue to yellow target ERPs (red). Figure A6. Experiment 2 scalp ERPs, feature-absent condition. Difference waves (black, dashed) were determined by subtracting common change-blue to yellow ERPs (blue) from rare flat-blue target ERPs (red). 69 Figure A7. Experiment 2 scalp ERPs, feature-present condition. Difference waves (black, dashed) were determined by subtracting common flat-yellow ERPs (blue) from rare change-yellow to blue target ERPs (red). Figure A8. Experiment 2 scalp ERPs, feature-absent condition. Difference waves (black, dashed) were determined by subtracting common change-yellow to blue ERPs (blue) from rare flat-yellow target ERPs (red). 70 Experiment 3 Scalp ERPs (contrast, black and white rings) Figure A9. Experiment 3 scalp ERPs, feature-present condition. Difference waves (black, dashed) were determined by subtracting common flat-black ERPs (blue) from rare change-black to white target ERPs (red). Figure A10. Experiment 3 scalp ERPs, feature-absent condition. Difference waves (black, dashed) were determined by subtracting common change-black to white ERPs (blue) from rare flat-black target ERPs (red). 71 Figure A11 Experiment 3 scalp ERPs, feature-present condition. Difference waves (black, dashed) were determined by subtracting common flat-white ERPs (blue) from rare change-white to black target ERPs (red). Figure A12. Experiment 3 scalp ERPs, feature-absent condition. Difference waves (black, dashed) were determined by subtracting common change-white to black ERPs (blue) from rare flat-white target ERPs (red). 72 Experiment 4 Scalp ERPs (shade, light grey and dark grey rings) Figure A13. Experiment 4 scalp ERPs, feature-present condition. Difference waves (black, dashed) were determined by subtracting common flat-dark ERPs (blue) from rare change-dark to light target ERPs (red). Figure A14. Experiment 4 scalp ERPs, feature-absent condition. Difference waves (black, dashed) were determined by subtracting common change-dark to light ERPs (blue) from rare flat-dark target ERPs (red). 73 Figure A15. Experiment 4 scalp ERPs, feature-present condition. Difference waves (black, dashed) were determined by subtracting common flat-light ERPs (blue) from rare change-light to dark target ERPs (red). Figure A16. Experiment 4 scalp ERPs, feature-absent condition. Difference waves (black, dashed) were determined by subtracting common change-light to dark ERPs (blue) from rare flat-light target ERPs (red). 74 Figure A17. Both figures are taken from Polich (2007). (Top left) A schematic representation of the proposed front-parietal attention mechanisms and temporo-parietal memory encoding networks presumed to underlie the P3a and P3b respectively. (Top right) Examples of grand average P3a and P3b ERP responses compared to reaction times, as well as P3a and P3b amplitude and latency scalp topographies. (Bottom right) Illustrations of P300 responses to a single stimulus (left), to an oddball stimulus (middle), to an infrequent salient distractor stimulus (right, P3a) and to a non-salient target stimulus (right, P3b). 75 Figure A18. Experiments 2 and 3, possible P2 (FCZ) and inversion (OZ) for all change runs, both rare and common. The positivity (and inversion) to rare runs is much earlier than that to common runs. No such effect is observed in response to flat runs, so are not shown in the figure. 76 Figure A19. Experiment 1, negativity (MMN) in response to all change runs, both rare and common. A large positivity (P3a) follows the negativity to rare change runs (red), but not common change runs (blue). No such effect is observed in response to the flat runs, so are not shown in the figure.
Thesis/Dissertation
eng
Psychology
Vancouver : University of British Columbia Library
Attribution-NonCommercial-NoDerivs 2.5 Canada
Graduate
Search through time is like search through space : behavioural and electrophysiological evidence
Text