UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Visual, vestibular, and proprioceptive contributions to the perception of upright Corbett, Jennifer E. 2007

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Notice for Google Chrome users:
If you are having trouble viewing or searching the PDF with Google Chrome, please download it here instead.

Item Metadata

Download

Media
831-ubc_2007-317549.pdf [ 6.73MB ]
Metadata
JSON: 831-1.0100617.json
JSON-LD: 831-1.0100617-ld.json
RDF/XML (Pretty): 831-1.0100617-rdf.xml
RDF/JSON: 831-1.0100617-rdf.json
Turtle: 831-1.0100617-turtle.txt
N-Triples: 831-1.0100617-rdf-ntriples.txt
Original Record: 831-1.0100617-source.json
Full Text
831-1.0100617-fulltext.txt
Citation
831-1.0100617.ris

Full Text

V I S U A L , V E S T I B U L A R , A N D P R O P R I O C E P T I V E C O N T R I B U T I O N S T O T H E P E R C E P T I O N O F U P R I G H T J E N N I F E R E. C O R B E T T B. A . , The Pennsylvania State Universi ty, 2000 M . A . , The Universi ty of Bri t ish Columbia , 2004 A D I S S E R T A T I O N S U B M I T T E D I N P A R T I A L F U L F I L M E N T O F T H E R E Q U I R E M E N T S F O R T H E D E G R E E O F D O C T O R O F P H I L O S O P H Y i n T H E F A C U L T Y O F G R A D U A T E STUDIES (Psychology) T H E U N I V E R S I T Y O F BRITISH C O L U M B I A August , 2007 © Jennifer E. Corbett, 2007 11 Abstract This dissertation examines the integration of visual and bodi ly inputs for the perception of upright. Normal ly , we effortlessly integrate external cues from surfaces i n the visual environment and internal bodi ly position signals to accurately determine which way is "up." However, when sensory signals conflict, we base our perceptions of gravitational upright on visual references i n the external environment. While tilting the body further decreases the ability to accurately perceive upright i n the presence of a tilted visual context, the precise ways i n which multisensory inputs to upright are integrated under these circumstances remain undetermined. Chapter 2 examines this question by isolating the effects of tilting the head or the whole body on perceived upright. Findings reveal a hierarchy of sensory contributions to the perception of upright where visual cues are weighted most heavily, followed by vestibular input about head position, and then proprioceptive signals about the position of the body when both visual and vestibular cues are unreliable. Chapter 3 then examines the amount of time necessary to overcome tilt illusions induced by local context i n close proximity to a stimulus as compared to spatially remote global context. Results show different time courses for each type of i l lusion, and suggest the existence of distinct mechanisms involved i n each. Chapter 4 follows wi th an ERP investigation to show that vestibular context influences later, post-perceptual stages of visual processing, as indexed by a modulation of the P3 E R P component. The thesis concludes wi th a summary of the main findings and outstanding questions, followed by an expanded discussion of how visual, vestibular, and proprioceptive inputs are hierarchically integrated to maintain a stable percept of our dynamic surroundings. Table of Contents Abstract i i Table of Contents i i i List of Tables . . . . v List of Figures v i Acknowledgements v i i Co-Authorship Statement v i i i CHAPTER 1: Introduction to the perception of upright 1 Purpose of thesis 2 Muitisensory contributions to upright perception 6 Visua l context for upright 7 Bodi ly context for upright 9 Muitisensory integration for upright perception? 12 General overview of experiments 14 References '. 17 CHAPTER 2: Observer pitch and roll influence the Rod and Frame Illusion 22 Introduction 22 Experiment 1 24 Methods 25 Results 28 Discussion . 32 Experiment 2 34 Methods 35 Results 35 Discussion 36 Experiment 3 * 36 Methods 37 Results 37 Discussion ..38 General Discussion 38 References 40 CHAPTER 3: When do we know which way is "up?" More time is needed to resolve global than local orientation illusions 41 Introduction 41 Experiment 1 48 Methods 49 Results 53 Discussion 55 Experiment 2 56 Results 58 Discussion .....60 General Discussion 60 References 65 iv CHAPTER 4: Electrophysiological evidence for a post-perceptual locus of visual and vestibular effects on perceived upright 69 Introduction • 69 Experiment 1 72 Methods 73 Results 77 Discussion 81 Experiment 2 82 Methods • 82 Results 83 Discussion • 86 General Discussion 87 References 92 CHAPTER 5: General Discussion 95 Remaining Questions 96 Practical Implications 102 Closing Remarks • 106 References 108 List of Tables 2.1. Mean rod settings i n Chapter 2, Experiment 1 vi Lis t of Figures 2.1. Participant's eye view of the rod and frame apparatus i n Chapter 2 27 2.2. Mean rod and frame i l lusion i n Chapter 2, Experiment 1 30 2.3. Mean rod and frame i l lusion i n Chapter 2, Experiment 2 35 2.4. Mean rod and frame i l lusion i n Chapter 2, Experiment 3 37 3.1. Examples of (a&c). congruent and (b&d). incongruent trials i n Chapter 3 43 3.2. Schematic diagram of the time-course of a trial i n Chapter 3, Experiment 1 51 3.3. Mean T I / R F I i n each S A O i n Chapter 3, Experiment 1 54 3.4. Schematic diagram of the time-course of a trial i n Chapter 3, Experiment 2 57 3.5. Mean T I / R F I i n each S O A i n Chapter-3, Experiment 1 58 4.1. Examples of (a), incongruent and (b). congruent trials i n Chapter 4 74 4.2. Mean correct response times for RFI observers i n Chapter 4, Experiment 1 78 4.3. Grand-average ERP waveforms i n Chapter 4, Experimentl 79 4.4. Mean correct response times for RFI observers i n Chapter 4, Experiment 2 84 4.5. Grand-average E R P waveforms i n Chapter 4, Experiment 2 85 V l l l Co-Authorship Statement I am the primary author on all of the work presented in this PhD thesis. This dissertation grew from discussions and collaborations with Dr. James T. Enns and Dr. Todd. C. Handy. 1 CHAPTER 1: INTRODUCTION TO MULTISENSORY INTEGRATION FOR UPRIGHT PERCEPTION H o w do we know which way is "up?" Normal ly we are stationed on level surfaces v iewing upright environments. Under these circumstances, we effortlessly integrate visual cues from the external environment wi th our bodies' "felt" vestibular and proprioceptive senses to accurately determine gravitational upright. However, when these sensory inputs conflict, such as when the visual environment is tilted, when Our heads are tilted, or when our whole bodies are tilted, we can experience striking misperceptions of upright. Mystery cabins and other roadside attractions provide entertaining demonstrations of how the orientation of visible surfaces i n the external environment can overrule internal vestibular and proprioceptive cues to upright. These tilted shacks are usually located i n a heavily wooded area wi th no visible horizon. When visitors peer into one of these dimly-l i t cabins, they can witness a range of "gravity defying" phenomenon, such as people and objects appearing to stand at impossible angles, while balls and bottles rol l uphi l l . Usually there is a local legend claiming that the tilted cabin sl id down a nearby h i l l to rest over a magnetic anomaly i n the Earth that disrupts the laws of gravity (e. g. w w w .oregonvor lex.com.). However, vis ion scientists have demonstrated that the phenomena experienced at these tourist attractions are actually visual illusions that result because observers misperceive the direction of gravity based on the orientation of visual cues inherent i n the tilted lines of the cabin's walls, floor, and ceiling (Shimamura & Prinzmetal, 1999). This dissertation proposes that our heavy dependence on visual input responsible for these illusions is part of an efficient process 2 of integrating multisensory inputs to aid us in adapting to changes in the orientation of the body, the visible environment, or both, in order to maintain a veridical perception of the world around us. Purpose of thesis While entertaining in their own right, mystery cabins offer first-hand demonstrations of how our perceptions of upright are influenced by the visible context of the surrounding environment. This thesis contains three sets of experiments using simplified laboratory versions of the illusion at work in mystery cabins to investigate how multisensory inputs are integrated to perceive gravitational upright. In addition to increasing our knowledge of human sensory and perceptual functioning, understanding how we use visual and bodily cues to upright when multisensory inputs conflict can lead to numerous improvements in human safety. Knowledge gained from studies of how humans perceive upright has already helped to make a number of advancements in safety and improvements to human-machine interactions. One example occurred during the National Transportation Safety Board's investigation of the Christmas Eve 1968 crash of Allegheny Airlines Flight 736, which killed 20 of the 47 passengers onboard (1968-AAR70-04). The plane crashed while attempting to land on the "haunted" Runway 32 at the Bradford Regional Airport in Pennsylvania. Runway 32 was the site of several fatal landing crashes since it opened in 1961. Based on previous reports that the tilt of the surrounding environment (or cabin) biases the perceived orientation of the objects within it (Asch & Witkin, 1948a), investigators concluded that approaching pilots unknowingly relied on the 60° tilt of the surrounding hillside (in the absence of a visible horizon) to perceive the orientation of the oncoming runway. Pilots therefore perceived the level runway as slanting downward in 3 the opposite direction of the upward slanting hillside they mistook for level ground. This report called for the runway to be flanked by rows of lights that converged inward, but appeared to form a rectangular landing area from a pilot's approaching perspective. Based on this new visual context to counteract the visual illusion of the level runway tilting in the opposite direction of the upward slanting hillside, pilots now underestimated the slope of Runway 32 and have landed safely with little incident since. The same visual illusions responsible for these fatal crashes are also responsible for the phenomena experienced at mystery cabins. In the simplest form of these illusions, surrounding a rod with a tilted frame causes observers to misperceive the orientation of the rod in the opposite direction of the frame (the Rod and Frame Illusion, or RFI) (Asch & Witkin, 1948a). The RFI demonstrates our heavy reliance on visual context for perceiving the direction of gravitational upright. Even when we are receiving accurate vestibular feedback about the position of the head with respect to gravity, and proprioceptive feedback about the pull of gravity and the position of the body relative to gravity, such as when we are viewing the interior of a mystery cabin from an outside, level vantage point, we mainly rely on the tilt of the visual frame, leading to remarkable illusions of gravitational upright. Given the importance of understanding how we weight visual and bodily signals to perceive upright, it is necessary to examine several gaps in our current understanding of multisensory integration for upright perception. First, tilting observers increases the degree to which perceived upright is biased by visual context (Asch & Witkin, 1948b), such as when standing inside tilted mystery cabins or landing on a level runway, However, the precise ways in which visual, vestibular, and proprioceptive inputs are weighted under these circumstances have not been clearly outlined. Inside mystery cabins, tourists are supported by the tilted floor of the cabin, but can right their heads 4 with respect to gravity to actively maintain an upright posture, but pilots approaching a runway at an angle are strapped into an aircraft, passively tilting their whole bodies. It is possible that pilots in twisting aircrafts who cannot right their heads with respect to gravity are subject to even more overwhelming misperceptions of upright. Recent studies indicate that orientation perception is influenced by visual context in at least two stages of processing; local spatial interactions in early stages and longer-range interactions in later stages (Dyde & Milner, 2002; L i & Matin, 2005). Specifically, local orientation context has been demonstrated to bias perceptions and actions to a target stimulus, while global orientation context only biases perceptions of the target's orientation. Visually guided actions to targets within tilted global contexts are corrected online in accordance with an accurate representation of gravitational upright (Li & Matin, 2005). Dyde & Milner (2002) proposed that local spatial interactions influence processing before the division of visual processes into "what" ventral and "how" dorsal streams (Milner & Goodale, 1995), and long-range interactions influence visual processes after this division only in the "what" (ventral) stream. Therefore, the later influence of global context allows for dorsal processes, thought to be critical for guiding on-line reaching, to be unaffected or corrected online over the course of processing (Dyde & Milner, 2002; L i & Matin, 2005). While this implies different time courses for orientation illusions induced by local and global tilted contexts, this proposal has not been empirically confirmed. This is because almost nothing is known about the time course of global context effects compared to the broad literature on the time course of local orientation illusions. In addition, the post-perceptual locus of global context effects implied by these findings has also not been directly confirmed. Therefore, this dissertation aims to address three unresolved issues regarding muitisensory integration for the perception of upright: 5 Aim 1: To identify the separate the contributions of visual, vestibular, and proprioceptive inputs to visually perceived upright. Aim 2: To determine whether there are different time courses for local and global visual orientation illusions. Aim 3: To determine whether global orientation context affects perceptual or post-perceptual stages of visual processing, and when bodily contributions are integrated to help counteract these global tilt illusions. Answering the first aim by identifying the contributions of each type of sensory input will help us to predict how perceived upright will be affected in a given situation. The second aim to determine the time courses of local and global orientation illusions will help to pin-point when visual orientation context must be presented at the local or global level in order to correct or induce misperceptions of upright. The third aim to directly assess whether global orientation context and corrective bodily signals are integrated during perceptual or post-perceptual stages of processing will allow for prediction of when bodily input is most influential in overcoming or inducing global orientation context affects. Meeting these aims will advance our theoretical understanding of multisensory orientation perception, and can lead to the development of techniques to re-weight the relative contributions of these multisensory inputs to improve the reliability of our perceptions of upright. This manuscript-style dissertation addresses these three aims sequentially, in three independent studies following this introductory chapter. Starting in Chapter 2, the effects of vestibular and proprioceptive inputs on the perception of upright are isolated 6 by: a), tilting observers' whole bodies, b). tilting only their heads, and c). tilting only their bodies while their heads remain upright to measure the effects on the degree to which a tilted frame biases their perceptions of upright (Aim 1). Chapter 3 continues by determining how much time observers need to overcome visual tilt illusions induced by local context in close proximity to a stimulus as compared to global context spatially remote from an isolated stimulus (Aim 2). Chapter 4 follows with an event-related potential (ERP) investigation to examine whether perceptual or post-perceptual cortical activity is associated with perceiving the correct orientation of a stimulus in a tilted global context when vestibular inputs about head position are intact versus disrupted via head tilt (Aim 3). Chapter 5 concludes with a summary of the main findings and limitations of the present study, followed by a list of the outstanding questions posed by the findings. The dissertation ends with an expanded discussion of how the present research informs our understanding of multisensory integration to maintain a veridical perception of gravitational upright. Multisensory contributions to the perception of upright Before describing the specific experiments conducted, the remainder of this introductory chapter outlines the literature regarding the three main aims of this thesis to expose the gaps in our current understanding of multisensory integration for upright perception. To this effect, the next section provides a review of relevant findings to date intended to stress the importance of determining the individual influences of vestibular and proprioceptive inputs to visually perceived upright, as well as the importance of determining when visual and bodily inputs influence perceived upright over the course of processing. This chapter ends with an overview of how the forthcoming studies address each aim of the dissertation. 7 Visual context for upright We base our perceptions of upright on the horizontal and vertical lines inherent in the visible surfaces of the external environment (Howard, 1982). We treat even a single horizontal line or a plane lying parallel to the pull of gravity as the "horizon," and evaluate the orientation of surrounding objects with respect to these simple reference frames (MacDougall, 1903). Our heavy reliance on visual information leads to striking misperceptions of visual and bodily upright, such as "antigravity" hills. Visitors to these wooded stretches of highway like Confusion Hi l l in southwestern Pennsylvania find that when they put their cars into neutral on a stretch of road that appears to slant upwards, they experience rolling forward or uphill against the pull of gravity. Using laboratory models, Bressan and colleagues (2003) demonstrated that this illusion results due to heavy dependence on visual inputs to perceive the direction of gravitational upright. Specifically, observers perceive the orientation of a downward sloping stretch of road flanked by two sharply upward slanting stretches of road as sloping upward, despite the felt pull of gravity on their bodies indicating the true direction of upright. In this sense, antigravity hills are similar to the fatal misperceptions of gravitational upright responsible for the crash of Allegheny Airlines Flight 732. Another similar misperception can be experienced when driving away from Lake Erie on Interstate 79 in Pennsylvania: the road slants sharply upward, causing passengers to perceive the level lake rising into the sky behind them. The effects of displacing the entire visual field have been studied using prism goggles. Helmholtz (1871) first reported that observers wearing prism goggles adapt their reference frames for determining upright to match a visual field tilted by approximately 45°. Most observers reported that the displaced visual field was upright 8 after only several minutes of exposure, and later reported that their upright bodies were tilted wi th respect to the prismatically displaced visual field they had adopted as their new context for gravitational upright. These effects of pr ism vis ion were later empirically documented i n perceptual estimates of upright and motor tasks assessing the perceived orientation of the body (Morant & Beller, 1965; Cohen, 1973; Choe & Welch, 1974; Mikaelian, 1974). Tilted frames or rooms also control the perceived orientation of objects wi th in them, as wel l as the perceived orientation of the body, -even when the body is upright and aligned wi th the pu l l of gravity. When observers adjust a rod inside a tilted room to vertical, not only do their rod judgments consistently err i n the opposite direction of the tilted room (Asch & Witk in , 1948a), but wi th prolonged exposure, vertical observers eventually perceive the tilted room as upright and feel as if they are tilted (Passey &. Guedry, 1949; Passey, 1950). A moving visual field can induce such strong misperceptions of upright that observers experience a complete full-body rotation when they are actually upright and stationary. For example, H o w a r d and Childerson (1994) found that the majority of observers tested experienced a complete 360° body rotation inside the rotating room facility at York University. This type of reorientation i l lusion is so convincing that when observers were ly ing supine on their backs, rotating the room around the body axis of symmetry caused them to perceive themselves as standing upright (Groen, Jenkin, & Howard , 2002), emphasizing how visual cues to upright override competing bodi ly inputs to form our perceptions of gravitational upright. In addition, observers can accurately estimate upright when their heads or whole bodies are tilted and they are v iewing an upright environment (Ong & Kessinger, 1971; Goodendough, et. al, 1981), just as patients wi th bilateral vestibular damage do not experience misperceptions of upright when viewing 9 an upright visual field (Anastasopoulos, et. al., 1999). Similarly, when Jenkin and colleagues (2003; 2004) asked observers to adjust the shading of a disk surrounded by a tilted frame in accordance with illumination from above, adjustments erred in the a. direction of the frame, but when observers' bodies were tilted and the frame was upright, they made negligible errors. Taken together, these studies provide strong evidence that visual signals override competing bodily signals for perceiving the direction of gravitational upright and the orientation of the body with respect to gravity. Bodily context for upright While visual references to upright are most important, we continue to perceive upright in the dark, with our eyes closed, and when visual input is impaired or absent (as for blind patients). A range of studies have investigated how we use bodily inputs to determine upright when visual cues to upright are unavailable. For example, observers misperceive upright in the opposite direction of the body when they are slightly tilted sideways with their eyes closed (the E-Effect Muller, 1916; Guerraz, Poquin, Luyat, & Ohlmann, 1998), and they misperceive upright in the same direction of the body when lying 90° sideways or supine with their eyes closed (the Aubert, or A-Effect, Aubert,. 1961; Templeton, 1973). Specifically, the direction of errors in perceived upright reverses when the body is tilted past 60° from vertical, such that tilting the body at an angle under 60° causes misperceptions of upright in the direction of body tilt, and tilting the whole body more than 60° causes misperceptions in the direction opposite of body tilt (Howard, 1986; Kaptein & Van Gisbergen, 2004). Tilting only the head also results in E-Effects such that perceived upright is biased in the direction of head tilt in the absence of a visual field (Witkin & Asch, 1948). As the head cannot be tilted past approximately 60° while the body remains upright, it is likely that this reversal in the direction of perceived upright 10 reflects a shift in the spatial coordinate system used when the body is tilted past the point where balance is able to be maintained without concurrent visual support (Howard, 1982; 1986). When observers are exposed to a tilted visual frame of reference, bodily inputs have a secondary influence on perceived upright. For example, tilting the head, slightly tilting the body, or having observers lie 90° sideways while performing RFI tasks increases the illusion, or the degree to which they misperceive upright in the direction of the tilted frame (Guerraz, Poquin, & Ohlmann, 1998; Asch & Witkin, 1948b; Wade, 1969; Ong & Kessinger, 1971; Bischof, 1974). Similarly, when observers adjust the shading of a disk according to illumination from above, they make larger errors in the direction of a surrounding tilted frame when their bodies are also tilted compared to when their bodies are upright (Jenkin, Dyde, Zacher, Jenkin, & Harris, 2003). Tilting the head increases the illusory effects of a rotating room on visually perceived upright to the extent that observers wi l l fall over if not restrained (Allison, Howard, & Zacher, 1999; Merker & Held, 1981; Young, Oman, & Dichgans, 1975; Guerraz, et. al., 1998). When a mild electrical current is applied behind the ears to disrupt vestibular inputs (Galvanic Vestibular Stimulation, or GVS), observers experience involuntary body sway and similar overwhelming misperceptions of upright in the direction of a tilted or prismatically-displaced visual field (Kennedy, et. al., 2003; Cauquil & Day, 1998). Similar results have been obtained by injecting ice water into the ear canal to disrupt vestibular inputs (Caloric Vestibular Stimulation), (Yamamoto, et. al, 2002). Vestibular disorders also lead to the same perceptual consequence of a decreased ability to correctly determine gravitational vertical in the presence of a tilted visual frame of reference (Anastasopoulos, et. al., 1999; Hafstrom et. al., 2004; Yardley, 1990). Finally, Telford, 11 Howard , and O h m i (1995) have demonstrated that observers can determine the direction of "straight ahead," or heading when provided wi th only visual input, but vestibular and proprioceptive information about the position of the head and body i n space are both necessary i n order for them to make accurate movements forward. Taken together, these studies demonstrate that vestibular and proprioceptive inputs regarding the felt pu l l of gravity play a secondary role to visual contributions i n perceiving the direction of upright. When vestibular and proprioceptive inputs about gravity are absent, as in microgravity, observers become solely dependent on visual input and adjust the orientation of a rod wi th respect to a rotating visual field regardless of how their bodies are positioned (Cheung, Howard , & Money, 1990), suggesting that visual input also overrules remaining proprioceptive "head-on-trunk" input. Furthermore, when visual input is sparse or absent i n microgravity (as i n the interior of some spacecrafts), observers experience spontaneous reorientation illusions such that their perceptions of upright w i l l unexpectedly and arbitrarily shift from time to time (Kornilova, 1997). These unpredictable reorientation illusions further stress the importance of visual input over proprioceptive information about the relative positions of the head and feet. Observers i n the microgravity stage of parabolic flight report losing all sense of spatial orientation, or floating aimlessly, when they are blindfolded, but a sense of upright is immediately restored when their bodies contact a surface of the interior of the aircraft, or when they are strapped into a seat (Lackner 1992; Lackner & Graybiel , 1979). Similarly, Goodendough and colleagues (1981) have demonstrated that blindfolded observers i n microgravity also lose their sense of upright, but it is immediately restored by any tactile gravity cue, such as light pressure applied to the bottom of the foot. Together these studies suggest that head-on-trunk proprioceptive input about the inter-positioning of 12 body parts does not aid in perceiving upright when visual input and gravitational force are absent, but proprioceptive input about the slant of the supporting surfaces does play a subordinate role in the perception of upright under these circumstances. Overall, these studies suggest that observers can use stabilizing vestibular information about the position of the head and proprioceptive feedback about the orientation of supporting surfaces in the environment, but not proprioceptive input about the inter-positioning of body parts to determine which way is "up" when visual input is unreliable. Multisensory integration for upright perception? The studies outlined so far demonstrate that observers rely primarily on visual input to determine upright, but vestibular and proprioceptive inputs about the felt pull of gravity play subordinate roles. Yet, the individual influences of vestibular and proprioceptive inputs in the presence of a tilted context have not been isolated for comparison in the same study. Determining these relative weightings wil l allow us to predict how observers wil l behave when the visual input is unreliable and vestibular and proprioceptive inputs are compromised, for example when tilted pilots approach tilted runways, when astronauts experience reorientation illusions, and when postural stability and visual acuity decline with age. Falls are the leading cause of visits to the emergency room and the number one reason for accidental deaths in United States seniors age 65 and older (Anderson & Smith, 2005). Elderly patients' decreased ability to sense the direction of gravity is the most critical factor to increasing their risk of falling (Girardi, Konrad, Amin, & Hughes, 2001). There are two key cognitive differences between elderly fallers and healthy young observers; compared to controls, elderly patients exhibit decreased postural control, and an associated increased reliance on visual cues to gravity when determining upright 13 (Lajoie & Gallagher, 2004; Turano, Rubin, Herdman, Chee, & Fried, 1994). Importantly, visual acuity shows a marked decline after the age of 65, (Gittings & Fozard, 1986). Therefore, elderly patients have only a degraded representation of the visual world and their perceptions of upright become influenced by bodily fluctuations associated with declines in postural control, leading to discontinuities in their perceptions of an upright world. They can no longer sense the direction of gravity to effortlessly maintain their balance, leading to their increased risk of falling. A n understanding of how healthy young observers integrate sensory inputs to perceive upright can aid in determining the specific cognitive deficits behind the increased risk of fatal falls in elderly populations. In fact, one research group has already begun to design GVS systems to control posture by electrically stimulating the vestibular organs to induce involuntary postural sway (Maeda, et. al, 2005). Addressing the first aim of this dissertation wil l contribute to this important applied research question by determining how vestibular and proprioceptive deficits may individually contribute to patients' decreased postural stability and increased dependence on visual input, in effort to inform the development of techniques to decrease their risk of fatal falls. It may be possible to prevent falls in the elderly by correcting or inducing local and global context illusions during key points in the course of processing. Therefore, answering the second aim to determine the time course of local and global context illusions can aid in determining how soon after exposure to a tilted context local or global factors that can overcome these misperceptions must be implemented. The final aim of this dissertation to determine whether visual and bodily inputs become integrated during perceptual or post-perceptual stages of processing speaks to techniques that we may be able to develop to present visual and bodily cues to upright to affect perceptions in a desired manner at a given point in time. These findings can lead to improvements in current techniques, such 14 as GVS postural control systems. In addition, an understanding of the time course of global and local context effects and when the relative contributions of bodily inputs become integrated with these contexts could lead to similar improvements in navigation and flight landing software, as well other situations where observers must quickly switch between multiple sensory contexts for upright. General overview of experiments This dissertation integrates the results of three independent studies and allows us to draw important conclusions about how observers use visual input in conjunction with other sensory inputs over the course of processing to perceive gravitational upright. The first aim of the dissertation, to determine how observers weight visual and bodily inputs to upright, was addressed in a series of three experiments presented in Chapter 2. In all three experiments, observers adjusted the orientation of a rod surrounded by a tilted frame to visually perceived upright when their heads and whole bodies were tilted in several positions. The tilted frame biased the perceived orientation of the rod to an increased extent when observers' heads or whole bodies were tilted from gravitational vertical (Experiment 1). Yet, observers perceived the tilt of the rod in the opposite direction of the tilted frame to the same extent when their whole bodies were upright, and when only their bodies were tilted but their heads remained upright (Experiment 2). Tilting observers' heads while holding the position of their bodies' upright with respect to gravity also increased the magnitude of the visual illusion, as a function of the slant of the tilted supporting surface (Experiment 3). Taken together, these findings point to a sensory hierarchy for upright perception, where visual input has top priority, followed by vestibular signals, and then proprioceptive inputs when visual and vestibular inputs become unreliable. 15 After determining the relative weighting of visual and bodi ly contributions to upright, two experiments i n Chapter 3 were conducted to compare the time course of local and global context effects i n order to predict when perceived upright w i l l become influenced by each type of visual input. Observers determined the clockwise or counterclockwise orientation of an isolated rod distantly surrounded by a tilted frame, or a central grating immediately surrounded by tilted grating while their reaction times and error rates were recorded. Results support Dyde & Milner ' s two-stage model of orientation perception where local context illusions result dur ing early stages of processing and global illusions are manifest during later stages of processing, after the split of the ventral visual stream for perception and the dorsal stream for visually guided action. U p o n finding different time courses for local and global visual context illusions, two studies i n Chapter 4 used event related potentials (ERPs) to record the cortical activity associated wi th the perception of the rod i n RFI displays to provide direct confirmation of a post-perceptual locus of the il lusion. Repeating this experiment wi th observers' heads tilted to disrupt vestibular inputs inversely modulated the time observers needed to correctly determine the tilt of the rod and the associated cortical activity, suggesting that vestibular inputs to upright help to correct the i l lusory effects of a tilted global context dur ing post-perceptual stages of processing. After all three aims of the dissertation are addressed, Chapter 5 concludes wi th a summary of the main findings regarding multisensory influences on perceived upright. This main discussion is followed by an outline of several outstanding questions raised by the present findings and the limitations l ikely to be encountered i n future upright 16 perception studies. A n expanded discussion of the theoretical implications follows this summary, with particular attention to how our dependence on visual input is calibrated by a felt sense of gravity to allow for a continuous perception of an upright world. 17 References Al l i son , R., Howard , I. P., & Zacher, J. E. (1999). Effect of field size, head motion, and rotational velocity on rol l vection and illusory self-tilt i n a tumbling room. Perception, 28, p. 299-306. Anastasopoulos, D. , Bronstein, A . , Haslwanter, T., Fetter, M , & Dichgans, J. (1999). The role of somatosensory input for the perception of ver t ica l ly . Annals of the New York Academy of Science, 871, p. 379-383. Anderson, R. N . , & Smith, B. L . (2005). Deaths: Leading causes for 2002. National Vital Statistics Reports, 53(17), 19 pages. Asch, S. E., & Witk in , H . A . (1948a). Studies in space orientation: I. Perception of the upright wi th displaced visual fields. Journal of Experimental Psychology, 38, p. 325-337. Asch, S. E. , & Witk in , H . A . (1948b). Studies i n space orientation: II. Perception of the upright wi th displaced visual fields and wi th body tilted. Journal of Experimental Psychology, 38, p. 455-477. Aubert, H . (1861). A n apparent important turn of objects wi th inclination of the head to the right or left. Archives of Pathological Anatomy, 20, p. 381-393. Bischof, N . (1974). Optic vestibular orientation to the vertical. I N J. J. Kornhuber (Ed.), Handbook of sensory physiology, V o l . 6, Pt. 2, pp. 155-190. N e w York: Springer-Verlag. Bressan, P., Garlaschelli, L . , & Barracano, M . (2003). Antigravi ty hil ls are visual illusions. Psychological Science, 14, p. 441-449. Bridgeman, B., Peery, S., & Anand , S. (1997). Interaction of cognitive and sensorimotor maps of visual space. Perception & Psychophysics, 59(3), p. 456-469. Cauqui l , A . S. & Day, B. L . (1998). Galvanic vestibular stimulation modulates voluntary movement of the human upper body. Journal of Physiology, 513(2), p. 611-619. Cheung, B. S. K , Howard , I. P., & Money, K . E . (1990). Visually-induced tilt during parabolic flights. Experimental Brain Research, 81, p. 391-397. Choe, S. C , & Welch, R. B. (1974). Variables affecting the intermanual transfer and decay of pr ism adaptation. Journal of Experimental Psychology, 102, p. 1076-1084 Cohen, M . M . (1973). Visual feedback, distribution of practice, and intermanual transfer of pr ism aftereffects. Perceptual and Motor Skills, 37, p. 599-609. Donchin, E., & Coles, M . G . H . (1988). Is the P300 component a manifestation of context updating? The Behavioral and Brain Sciences, 11, p. 355-372. 18 Dyde, R. T., & Milner , D . A . (2002). Two illusions of perceived orientation: One fools all of the people some of the time; the other fools all of the people all of the time. Experimental Brain Research, 144 (4), p. 518-527. Girardi , M v Konrad, H . R., A m i n , M . , & Hughes, L . F. (2001). Predicting fall risks i n an elderly population: Computer dynamic posturography versus electronystagmography test results. Laryngoscope, 111(9), p. 1528-1532. Gittings, N . S., & Fozard, J. L . (1986). Age related changes i n visual acuity. Experimental Gerontology, 21(4-5), p. 423-433. Goodendough, D . R., Oltman, P. K. , Sigman, E., & Cox, P. W . (1981). The rod-and-frame i l lusion i n erect and supine observers. Perception & Psychophysics, 29(4), p. 365-370. Groen, E. L . , Jenkin, H . L. , & Howard , I. P. (2002). Perception of self-tilt i n a true and illusory vertical plane. Perception, 31(12), p. 1477-1490. Guerraz, M . , Poquin, D. , Luyat, M . , & Ohlmann, T. (1998). Head orientation involvement i n assessment of the subjective vertical during whole body tilt. Perceptual & Motor Skills, 87, p. 643-648. Hafstrom, A . C. A . , Fransson, P. A . , Karlberg, M . , & Magnusson, M . (2004). Ipsilesional visual field dependency for patients w i th vestibular schwannoma. Neuroreport, 15(14), p. 2201-2204. Helmholtz , H.E.F . von, (1871/1962). Treatise on physiological optics, 3 r d Edit ion. Southhall, J .P .C , Trans. (Eds.), Dover, N e w York: N Y . pp. 242-270. Howard , I. P. (1982). Human visual orientation. N e w York: Wiley. Howard , I. P: (1986). The perception of posture, self motion, and the visual vertical. In K . R. Boff, L . Kaufman, & J. P. Thomas, Handbook of perception and human performance (p. 1-50). N e w York: Wiley. Howard , I. P., & Childerson, L . (1994). The contribution of motion, the visual frame, and visual polarity to sensations of body tilt. Perception, 23(7), p. 753-762. Jenkin, H . L . , Dyde, R. T., Jenkin, M . R., & Harris, L . R. (2003). In Vir tua l Reality, which way is up? Proceedings from International Conference on Artificial Reality and Telexistence (ICAT), Japan. Jenkin, H . L. , Dyde, R. T., Jenkin, M . R., & Harris, L . R., (2004). Pitching up i n IVY. Proceedings from International Conference on Artificial Reality and Telexistence (ICAT), Korea. Jenkin, H . L . , Dyde, R. T., Zacher, J. E., Jenkin, M . R., & Harris, L . R. (2003). M u l t i -sensory contributions to the perception of up: Evidence from i l luminat ion judgments [Abstract]. Journal of Vision, 3(9), p. 638a. . 19 Kaptein, R. G., & Van Gisbergen, J. A. M . (2004). Interpretation of a discontinuity in the sense of verticality at large body tilt. Journal of Neurophysiology, 91(5), p. 2205-2214. Kennedy, P. M . , Carlsen, A . N . , Inglis, J. T., Chow, R., Franks, I. M . , & Chua, R. (2003). Relative contributions of visual and vestibular information on the trajectory of human gait. Experimental Brain Research, 153, p. 113-117. Kornilova, L. N . (1997). Orientation illusions in spaceflight. Journal of Vestibular Research, 7(6), p. 429-439. Lackner, J. R. (1992). Multimodal and motor influences on orientation: Implications for adapting to weightless and virtual environments. Journal of Vestibular Research, 2, p. 307-322. Lackner, J. R., & Graybiel, A. (1979). Parabolic flight: Loss of sense of orientation. Science, 206, p. 1105-1108. Lajoie, Y., & Gallagher, S. P. (2004). Predicting falls within the elderly community: Comparison of postural sway, reaction time, the Berg balance scale, and the Activities-specific Balance Confidence (ABC) scale for comparing fallers and non-fallers. Archives of Gerontology and Geriatrics, 38(1), p. 11-26. L i , W., & Matin, L. (2005). Two wrongs make a right: Linear increase of accuracy of visually-guided manual pointing, reaching, and height-matching with increase in hand-to-body distance. Vision Research, 45, p. 533-550. MacDougall, R. (1903). The subjective horizon. Psychological Review Monograph, 4, p. 145-166. Maeda, T., Ando, H. , Amemiya, T., Inami, M . , Nagaya, N . , & Sugimoto, M . (2005). Shaking the world: Galvanic Vestibular Stimulation as a novel sensation interface, Association for Computing Machinery's Special Interest Group on Graphics and Interactive Techniques (ACM SIGGRAPH), Los Angeles, CA. Merker, B. H. , & Held, R. (1981). Eye torsion and the apparent horizon under head tilt and visual field rotation. Vision Research, 21, p. 543-547. Mikaelian, H . H . (1974). Intersensory generalization and. conditioning in adaptation to visual tilt. The American Journal of Psychology, 87(1-2), p. 197-202. Milner, A . D., & Goodale, M . A. (1995). The visual brain in action. London: Oxford University Press. Morant, R. B., & Beller, H . K. (1965). Adaptation to prismatically rotated visual fields. Science, 148, p. 530-531. Muller, G. E. (1916). Above the Aubert Phenomenon. Journal of Sensory Physiology, 49, p. 109-246. 20 National Transportation Safety Board (1968-AAR70). Aircraft Accident Report. Allegheny Airlines, Inc. Convair 530, N5802, near the Bradford Regional Airport, December 24, 2968. Bureau of Aviation Safety, Washington: DC. Ong, J., & Kessinger, D. J. (1971). Perception of vertically with a rod and frame apparatus. American Journal of Optometry & Archives of American Academy of Optometry, 48(8), p. 662-666. Passey, G. E. (1950). The perception of the vertical: IV. Adjustment to the vertical with normal and tilted visual frames. Journal of Experimental Psychology, 40, p. 738-745. Passey, G. E., & Guedry, F. E. Jr. (1949). The perception of the vertical: Adaptation effects in four planes. Journal of Experimental Psychology, 39, p. 700-707. Shimamura, A . P., & Prinzmetal, W. (1999). The Mystery Spot illusion and its relation to other visual illusions. Psychological Science, 10, p. 501-507. Telford, L., Howard, I. P., & Ohmi, M . (1995). Heading judgments during active and passive self-motion. Experimental Brain Research, 104(3), p. 502-510. Templeton, W. B. (1973). The role of gravitational cues in the judgment of visual orientation. Perception & Psychophysics, 14, p. 451-457. Turano, K., Rubin, G. S., Herdman, S. J., Chee, E., & Fried, L. P. (1994). Visual stabilization of posture in the elderly: Fallers vs. nonfallers. Optometry and Vision Science, 71(12), p. 761-769. Wade, N . J. (1969). The effect of stimulus line variations on visual orientation with head upright and tilted. Australian Journal of Psychology, 21, p. 177-185. Witkin, H . A., & Asch, S.E. (1948). Studies in space orientation: III. Perception of the upright in the absence of a visual field. Journal of Experimental Psychology, 38, p. 603-614. Yardley, L. (1990). Contribution of somatosensory information to perception of the visual vertical with body tilt and rotating visual field. Perception & Psychophysics, 48, p. 131-134. Yamamoto, K., Mamoto, Y., Imai, T., Hirasaki, E., & Kubo, T. (2002). Effects of caloric vestibular stimulation on head and trunk movements during walking. Gait & Posture, 15(3), p. 274-281. \ Young, L. R., Oman, C. M . , & Dichgans, J. M . (1975). Influence of head orientation on visually induced pitch and roll sensation. Aviation, Space, and Environmental Medicine, 46, p. 264-269. CHAPTER 2: OBSERVER PITCH AND ROLL INFLUENCE 22 THE ROD AND FRAME ILLUSION1 Introduction Observers misperceive the orientation of a vertical rod when it is v iewed in the context of a tilted frame (the Rod and Frame Illusion, or RFI). The pitch and roll of the surrounding surfaces have independent influences on this i l lusion (Nelson & Prinzmetal, 2003). Experiment 1 measured the RFI when the pitch and rol l of the floor that supported the observer was varied, and the observer was either seated i n a chair or standing upright. There were additive influences of pitch and rol l on the RFI of seated but not standing observers. Experiments 2 and 3 decoupled body roll and head ro l l in order to isolate the vestibular and proprioceptive contributions to these effects. The results are interpreted i n support of a hierarchy of influence on the RFI: V i sua l input is given top priority, followed by vestibular input, and then proprioceptive input. 1 A version of this chapter is published: Corbett, J. E., & Enns, J. T. (2006). Observer pitch and roll influence the rod and frame illusion. Psychonomic Bulletin & Review, 13(1), p. 160-165 23 This study examines how visual, vestibular, and proprioceptive sensory mechanisms combine to influence the perception of upright. Carnival funhouses and roadside attractions that distort our sense of upright give us firsthand experience in how these sensory systems interact. For example, inside a tilted "mystery cabin," people may appear to change in size as they move from one location to another, objects may appear to balance at impossible angles, and balls may seem to roll uphill (Shimamura & Prinzmetal, 1999). Previous researchers have used the Rod and Frame Illusion (RFI) in the laboratory to demonstrate our heavy reliance on vision in these types of circumstances. In the simplest type of experiment, surrounding a vertical rod with a tilted frame causes observers to misperceive the orientation of the rod in the direction of the frame (Asch & Witkin, 1948a). The RFI is even stronger when observers are rolled to one side, as when they are inside a mystery cabin (Asch & Witkin, 1948b), suggesting that observers rely even more on vision when their vestibular and proprioceptive senses are disturbed. A similar conclusion has been drawn with regard to induced disequilibrium and illusions such as the Zollner, Poggendorff, and Ponzo illusions (Prinzmetal & Beck, 2001). The most is known about visual influences on the RFI. For example, a recent study compared the effects of changing an environment with regard to pitch (up and down slant of the viewed surfaces with respect to the viewer) and roll (side to side tilt of the viewed surfaces with respect to the viewer (Nelson & Prinzmetal, 2003). The authors concluded that environmental pitch and roll contribute independently to the RFI. But pitch and roll of the viewed environment are not the only possible sources of disequilibrium in mystery cabins. Observers themselves are in an environment with surfaces that are tilted causing them (their bodies) to be pitched (rotated about an imaginary axis running through the observer's line of sight) and rolled (rotated about an 24 imaginary axis that runs through the observer's ears). Yet little is known about these influences on the RFI, including whether observer pitch and roll have effects similar to those of visually perceived pitch and roll of the environment (Nelson & Prinzmetal, 2003). Experiment 1 We began by measuring the RFI in six conditions: three levels of observer roll (0°, or upright; +20° roll, congruent with the tilted frame; -20° roll, incongruent with the tilted frame) combined with two levels of observer pitch (0°, or upright; -20° pitch, backward with respect to the frame). In pilot experiments, we tested 0° and -20° frame positions, in addition to a forward pitch condition. There was never any illusion in the 0° condition, regardless of observer pitch or roll, pointing once again to the dominance of vision in this illusion. Furthermore, because the results for +20" and forward pitch were indistinguishable from those for -20° and backward pitch, respectively, we restricted ourselves to the latter conditions in order to maximize statistical power. Our choice of a 20° magnitude for pitch and roll was based on two considerations: (1). The RFI is maximized when the visual frame is tilted between 15° and 22° (Shimamura & Prinzmetal, 1999), and (2). Roll of the head or body results in a shift from an environmentally based sense of upright to an ego-based one when roll exceeds 60° (Howard, 1982, 1986; Young, 1984). In addition to examining observer pitch and roll, we addressed several other unresolved issues. For example, it is not clear whether there is a difference between rolling observers congruently and incongruently, with respect to the frame. From the perspective that proprioceptive disequilibrium heightens the observer's reliance on vision, congruent and incongruent roll should both lead to the same large RFI. However, 25 Witkin and Asch (1948) reported that observers who were rolled incongruently with the visual frame experienced a larger RFI than did those who were rolled congruently. However, the mean difference in observers' judgments was small and not statistically significant. Furthermore, the observers were sitting in a chair, and the frame was viewed in an otherwise dark room; these two conditions have not been studied systematically. Importantly, sitting in a chair limits the corrective actions an observer can make in order to maintain an upright posture and therefore may limit access to the relevant proprioceptive information. Similarly, viewing the frame in the dark limits the use of other visual references, such as a visible horizon, that might otherwise be used to calibrate a sense of upright (Bressan, Garlaschelli, & Barracano, 2003). To examine the roles played by these other factors, we tested (1). observer roll that was congruent and incongruent with the tilt of the visual frame, (2). seated observers (as in Witkin & Asch, 1948, and Asch & Witkin, 1948b) and standing observers (as in Asch & Witkin, 1948a), and (3). dark viewing (only the rod and. frame were illuminated, as in Nelson & Prinzmetal, 2003; Witkin & Asch, 1948) and light viewing (as in Shimamura & Prinzmetal, 1999). Methods Participants Eighteen individuals (10 females and 8 males, 18-35 years old) at the University of British Columbia participated voluntarily in two sessions. Five were undergraduates recruited from the human subject pool in. the Department of Psychology, and they received extra course credit. Thirteen graduate students and postdoctoral fellows were offered refreshments for their participation. A l l of the participants had normal or corrected-to normal vision and none reported visual or proprioceptive disorders. 26 The thirteen graduate students and postdoctoral fellows participated in both a light and a dark session. However, the five undergraduates who were initially tested with the room lights on were unavailable for a second session. Therefore, eighteen observers participated in a first testing session in the light and thirteen of those participated in the second session in the dark, where only the rod and frame were illuminated. Apparatus A rod and frame box made of translucent white Plexiglas (57 cm long X 31 cm high X 31 cm wide) was used to induce the RFI, as illustrated in Figure 2.1. During testing, the rod could be rotated 360°, but the square frame was fixed at an orientation of +20" relative to horizontal. A protractor, displayed on a. disc mounted on the rear of the box, and visible only to the experimenter, indicated the rod's deviation from upright (0°). The elevation of the box was adjusted so that it was at eye level when the observers were seated or standing. The position of the box was adjusted for each observer so that approximately 5 cm. separated the eyes of the observer from the front edge of the box. The observers were supported either on a level floor (for the no-roll and no-pitch conditions) or on a 125-cm2 wooden platform that was slanted 20°, relative to the floor. This platform could be rotated into positions marked on the floor in order to create active conditions for roll only, pitch only, or combined pitch and roll. A chair was bolted onto the platform for the passive viewing conditions. The platform and chair could be rotated independently to produce passive conditions for roll only, pitch only, or combined pitch 27 and roll . We note that although pitch and roll were not fully mechanically independent in this apparatus, the apparatus d id permit variation along each of these two dimensions. The participants wore a patch over one eye, so that all adjustments were made wi th their preferred eye. upright Figure 2.1. Participant's eye view of the rod and frame apparatus._The frame is rolled +20° to create the rod and frame illusion. When the rod is vertical with respect to gravity (0°, solid line), it appears tilted to observers because their perception of upright errs in the direction of the tilted frame (dashed line). Procedure The participants peered inside the box and instructed the experimenter to rotate the rod unti l it was "vertical," defined as " i n line wi th the pul l of gravity." The participants were given a demonstration of how the rod could be rotated before testing began; testing d id not begin until all questions regarding the procedure had been answered. The experimenter was unable to see the rod inside the box and could only see the protractor mounted on the rotating disc. The experimenter looked away from the box each time a participant asked for the disc to be rotated. The participants were given 28 up to 1. minute to refine their adjustments on each trial. The initial position of the rod alternated from trial to trial, beginning initially to the far right or far left of vertical. The participants were first tested with the room lights on. Because there were 5 cm between their eyes and the front edge of the box, there was a possibility for peripheral vision of the room to play a role. Participants made rid adjustments in each of two (active-standing or passive-sitting) postural conditions. The order of postural conditions was randomly determined. In the active condition, the participants stood on the platform. The experimenter, who stood at the other end of the box in order to move the rod • according to the instructions of the participant, was able to make sure that the participant's body and head maintained an upright position and that his or her head did not move away from the box at any time. In the passive condition, the participants were seated in a chair that included a headrest to ensure that the head, was kept in consistent orientation with the trunk. In each postural condition the platform or chair apparatus were positioned in one of the six possible postural combinations of body roll (0° no roll, +20° roll with the frame, or -20° in the opposite direction of the frame) and pitch (0° no pitch,-20° backwards pitch). The testing session in the dark occurred on a separate day. In this condition, a small bulb illuminated the interior of the rod and. frame box to ensure that the rod and frame were as visible as they were in the lights-on condition. Each participant adjusted the rod a total of 12 times in each session (3 roll X 2 pitch X 2 postural conditions). Results The mean settings (degrees of tilt from vertical) are shown in Table 2.1. These data were examined by an A N O V A involving the factors of observer roll (none, congruent, incongruent), observer pitch (none, pitched), postural balance (active, passive), and lighting (light, dark). 29 Table 2.1. Mean rod settings (in degrees from vertical) in Experiment 1. Standard errors are in parentheses. O B S E R V E R R O L L O B S E R V E R P I T C H None Congruent Incongruent A C T I V E OBSERVERS - L I G H T N= =18 None Backward 2.03 1.58 (0.84) 2.67 "(1.15) (0.95) 2.64 (1.10) 1.75 1.92 (1.21) (0.94) A C T I V E OBSERVERS - D A R K N= =13 None Backward 3.19 3.04 (0.81) 3.54 (0.76) (1.00) 3.08 (0.91) 3.08 2.50 (0.92) (1.14) PASSIVE OBSERVERS - L I G H T N= =18 None Backward 3.61 1.11 (0.84) 5.39 (0.92) (0.98) 4.98 (1.24) 0.89 0.31 (L09) (0.84) PASSIVE OBSERVERS - D A R K N= =13 None Backward 2.92 3.62 (1.10) 7.81 (1.00) (0.86) 5.77 (1.19) 4.15 3.08 (1.07) (0.95) 30 14 -j to 12 -CD \ CD 10 -CD 8 -T J Ro 6 -T J CD •*-» 4 -co rj 2 -< 0 --? -Active Observer Roll o 0°None • + 20° Congruent ± - 20° Incongruent 0° 20° Observer Pitch Passive 0° 20° Observer Pitch Figure 2.2. Mean rod and frame illusion in Experiment 1. Influences of pitch and roll of the floor in active and passive observers. Error bars indicate plus and minus one standard error of the mean. Figure 2.2 illustrates the main findings. First, there was a significant RFI in both the active and passive conditions: Participants were never able to adjust the rod to 0° of orientation i n the context of the tilted visual frame (all p's>.01). Second, manipulations of observer roll and pitch each had an influence on the RFI for passive observers. Congruent observer rol l increased RFI, whereas incongruent roll decreased the il lusion, when compared wi th the no-roll condition. Observer pitch, on the other hand, decreased the RFI. Third, the influences of observer rol l and pitch were independent, meaning that the effects of congruent and incongruent rol l were the same, regardless of whether the observer was pitched or not. Fourth, although observers i n the active condition were subject to the RFI, rod settings were not influenced significantly by variations in the roll or pitch of the platform. Fifth, whether the room lights were on or off had little influence 31 on these findings. A single exception concerned the incongruent rol l condition for passive observers i n the light. Under these conditions, incongruent rol l resulted in an RFI that was even smaller than i n the no-roll baseline condition. Otherwise, incongruent roll resulted in essentially the same illusion as d id the no-roll condition. These conclusions were supported by A N O V A s . A preliminary analysis of the thirteen participants who were tested both i n the light and i n the dark revealed no significant main effect of light, and no interactions of light X activity, light X. pitch, light X roll X pitch, or light X activity X rol l X pitch (all p's>.05). The only significant interactions involving light were light X roll, F(2,24)=4.786, p<.02, and light X activity X roll , F(2,24)=5.52, p<.01, which could both be attributed to a decreased i l lusion in the incongruent rol l condition, but only when the participants were tested under passive postural control i n the light. Under these conditions, the RFI was significantly smaller than it was i n the no-roll control conditions, F(l,24)=9.46, p<.01. In order to maximize the statistical power for the consideration of observer pitch and roll , the remaining analyses were conducted on the data from all eighteen observers who were tested i n the light. This A N O V A revealed significant interactions of activity X rol l , f(2,34)^6.31, p<.01, and activity X pitch, F(l,17)=4.64, p<.05, but no three-way interaction of activity X roll X pitch. Examination of these interactions indicated that the influences of observer rol l and pitch were significant in the passive postural condition but not i n the active condition (all p's>.20). A simple effects A N O V A i n the passive condition revealed significant main effects of observer rol l , F(2,34)=19.47, p<.01, and observer pitch, F( 1,17)= 11.39, p<.01, but no interaction of rol l X pitch. The main effect of roll was subdivided further into a significantly larger i l lusion for congruent rol l than for no roll , F(l,34)=8.64, u<.01, and a marginally smaller i l lusion for incongruent rol l than for no rol l , f(I,34)=2.89, p<A0. 32 Discussion The main finding was that observer pitch and roll had independent influences on the RFI when observers were seated. The absence of a statistical interaction under these conditions is similar to the results of Nelson and Prinzmetal (2003), who found that pitch and roll of the visual environment contributed independently to observers' estimates of upright, even when observers were stationed on a flat surface. Together wi th the present finding, this suggests that separate mechanisms are at play i n the perception of these two dimensions of space, regardless of whether they are perceived via visual, vestibular, or proprioceptive inputs. Experiment 1 also contributed data to unresolved issues regarding the RFI. First, on the question of the congruent versus incongruent observer roll , the data are clear i n showing that the largest i l lusion occurs when observers are oriented congruently wi th the frame. When observers are rolled incongruentiy, die RFI is similar to that measured when observers are not rolled at all . One explanation is that the congruent rol l condition is most similar to the conditions of everyday viewing, i n which the visual cues i n the environment agree wi th the proprioceptive cues regarding upright; it is the reliance on visual cues under these everyday circumstances that contributes to the large i l lusion in the congruent experimental conditions. In contrast, when observers are rolled incongruentiy wi th the frame, the discrepancy between these two sources of information may prompt them to consult their vestibular and proprioceptive signals more carefully, permitting compensation for the tilted visual frame. Support for this interpretation can be found i n the interactions observed between light, postural activity, and observer roll . Recall that when observers were tested in the 33 light, the incongruent roll condition resulted in a smaller RFI than when they were tested in the dark. This is consistent with observers using proprioception to compensate for the RFI normally experienced under observer roll. These signals would have greater support in the light condition, in which the rest of the testing room was partially visible and could be used to calibrate these cues, than in the dark condition, in which only the rod and frame were visible. Additional support for the idea that observers were weighting their visual and proprioceptive cues differentially comes from the small but significant reduction in the RFI for pitched observers. When seated observers adjusted the rod, they experienced a smaller visual tilt illusion than when they were sitting upright. Pitching a seated person backward (or forward) in a. chair gives him or her additional points of somatosensory contact (back and legs), possibly allowing the individual to read his or her body position better (mckiding roll) than when the individual is seated upright. The second issue concerns ambient visual lighting. For the most part, whether or not observers were able to see the room through peripheral vision had no effect on the RFI. This means that the stimuli considered "relevant" to the observer are restricted to the rod and frame apparatus. The one exception to this rule, already mentioned, concerns passively seated observers who are rolled incongruentiy with respect to the tilted frame. In this situation, it appears that peripheral vision helped observers compensate to some extent for the roll of the frame by allowing them to consult their proprioceptive signals relative to their ambient visual surroundings. A final issue concerns seated as opposed to standing observers. The results showed that observers who were actively maintaining their balance experienced a robust RFI. However, their illusion was not influenced by the pitch and roll of the environment. In contrast, when observers were seated, variations in observer orientation had a large 34 and significant influence. There are at least two explanations for this difference. First active observers may have access to more accurate proprioceptive information than do passive observers, simply by virtue of the fact that they must use this information to maintain their balance. If so, then the reduced effects of observer pitch and roll for active observers are consistent with the compensation account already described: Proprioception is weighted more heavily when it is also being used for the purpose of standing upright. A n alternative account is that the heads of active observers remained upright with respect to gravity, whereas the heads of passive observers were rolled. This postural adjustment by the active observers may have allowed them to experience the rod and frame in a way more similar to the way it is experienced by observers who are supported on a horizontal surface. We consider this issue in the next experiments. Experiment 2 In Experiment 1, seated observers were influenced by the pitch and roll of the floor, but standing observers were not. Yet these conditions differed not only in whether participants made active postural adjustments; the heads of standing observers were upright, whereas those of seated observers were roiled. This may be important, because head orientation strongly influences the vestibular sense. Body balance is dependent, to a large extent, on the semicircular canals and otolith organs of the. inner ear (Wilson & Melvill Jones, 1979). Perhaps the sensitivity to pitch and roll in seated observers occurred because their vestibular sense of gravity was disrupted. If so, then we should be able to remove the observed proprioceptive influences on the RFI by allowing seated observers to hold their heads upright with respect to gravity while the chair in which they are seated is pitched and rolled. Alternatively, if the vestibular system is not the principle sensory mechanism being disrupted by head and body roll, we should obtain the same 35 results as in Experiment 1 (passive condition) even when the head is held upright. M e t h o d -Twelve individuals (6 females and 6 males, 18-35 years old) participated in a procedure identical to that used i n the passive condition (light viewing) i n Experiment 1, wi th the exception that the headrest was removed from the chair and all of the participants were instructed to keep their heads upright during the session. The experimenter, standing at the other end of the box, was able to make sure that the participant's head was maintained in an upright position using a horizontal reference line on the wal l behind the participant. Results 14 * 1 2 2 10 CD jo 8 £ 6 if) .5, 2 < „ Observer Roll o 0°None • + 20° Congruent * - 20° Incongruent 0 ° 20° Observer Pitch Figure 2.3. Mean rod and frame illusion in Experiment 2. Passively pitched and rolled observers with vertical head orientation. Error bars indicate plus and minus one standard error of the mean. , Figure 2.3 shows two main findings. First, there was a significant RFI i n each 36 condition of the experiment, all p's<.01, that was similar in magnitude to the RFI measured in the active condition of Experiment 1, F<1. Second, manipulations of observer roll and pitch on seated observers no longer had any influence on the RFI, all F's for main effects and interactions--. 1. Discussion The results suggest that the proprioceptive influences on the RFI observed in Experiment 1 occurred because the vestibular system, was disrupted by rolling the head. When observers were able to hold their heads upright in the present experiment, there was still a significant RFI, but it was no longer influenced by the position of the observer's body relative to the pull of gravity. In other words, the results were now .indistinguishable from those in the active condition of Experiment 1, in which observers were standing upright on a pitched and. rolled platform. Experiment 3 As a final step, we asked whether disrupting the vestibular system alone was sufficient to reinstate the proprioceptive effects observed in the passive condition of Experiment 1. We tested this by measuring the RFI while participants were standing upright on the pitched and rolled platform, but with the requirement that they roll their heads 20° to one side, so their heads would be congruent with the visual frame. If vestibular disruption was responsible for the proprioceptive influences on the RFI, then this manipulation should .reinstate them. Alternatively, if the combination of body and head roll was necessary to cause the proprioceptive influences in Experiment 1, then a simple roll of the head should not reveal any proprioceptive influences. 37 M e t h o d The twelve participants from Experiment 2 were tested i n a procedure identical to that used i n the active condition (light viewing) i n Experiment 1, wi th the exception that they aligned their heads wi th the tilt of the frame. The experimenter monitored the head position of all of the observers relative to a 20" reference line on the wal l directly behind the participants' heads. 5 Residts Figure 2.4 shows two main findings. Firs t there was a significant RFI i n each condition of the experiment all p's<.01, that was much larger in magnitude than was the RFI measured i n the active condition i n Experiment 1, F(l,23)=24.60, p<.001. Second, manipulations of observer rol l and pitch on standing observers with, rolled heads had no measurable influence on the RFI, all F's for main effects and inter action s<l.5. 14 _ 12 CO CD a) 10 CD CD -a 8 J T3 O DH T3 CD -»—» < 4 2 J -2 Observer Rol l o 0 ° N o n e • + 2 0 ° Congruent A - 2 0 ° Incongruent 0 ° 2 0 ° Observer Pitch 38 Figure 2.4. Mean rod and frame illusion in Experiment 3. Active observers with head orientation rolled +20°. Error bars indicate plus and minus one standard error of the mean. Discussion The proprioceptive influences on the RFI observed in the passive condition of Experiment 1 appear to depend on both passive seating and a rolled head. As we saw in Experiment 3, simply rolling the observer's head so that it was congruent with the visual frame was not sufficient to invoke the proprioceptive influences on the RFI from the pitched and rolled platform that was supporting the participants. Experiment 2 showed that passive seating on a pitched and rolled chair was also not sufficient in itself to induce these effects. Only when the observer's head was rolled (to disrupt the vestibular sense of upright) and when the observer was seated in a chair that was pitched and rolled (to accentuate the proprioceptive inputs concerning the unusual environment) was the RFI influenced by both the pitch and roll of the floor supporting the observers. General Discussion The consistent RFI that was measured in three experiments points to the large extent to which our perception of tipright depends on vision. This reliance on vision becomes even stronger when the vestibular sense is disrupted by the act of rolling one's head to the side (as demonstrated in Experiment 3). In contrast, merely rolling one's body, without a corresponding roll of the head, does not result in a stronger-than-normal illusion (Experiment 2), presumably because the vestibular system can still be used to inform judgments about upright based on an upright head position. Finally, when both the vestibular system and the proprioceptive sense of upright are disturbed (Experiment 1, passive condition), the visual illusion (RFI) becomes sensitive to proprioceptive signals regarding gravitational upright. 39 This pattern of data suggests a hierarchy of control signals with.regard to the perception of upright. In this hierarchy, vision is the dominant sensory system, providing an illusion that is never completely overcome under any circumstance when the larger visual context is a tilted frame. The vestibular system has secondary status, in that it is permitted to modulate the visual input. This is presumably why the RFI is so large when the head is tilted and why it is reduced by the simple act of holding the head upright. Finally, the proprioceptive system does contribute to visual judgments of upright, but only if the vestibular sense has first become unreliable because of a head position that is less than vertical. The finding of independent effects of observer roll and pitch in seated observers (Experiment 1) bears a superficial resemblance to the report that the pitch and roll of viewed surfaces contribute separately to the RFI (Nelson & Prinzmetal, 2003). A promising lead for future studies of this independence is the observation that the vestibular system (semicircular canals and otolith organs) is organized to permit independent registration of the pitch and roll of the head (Wilson & Melvill Jones, 1979). Along the same lines, recordings from muscles of the legs and trunk show separate responses to perturbations along the pitch and roll axes of rotation (Carpenter, Allum, & Honegger, 1999). Moreover, vestibular neurons signaling pitch and roll maintain their distinctiveness in the visuomotor areas of the brain to which they project (Carpenter et al., 1999). It is therefore possible that functionally separate mechanisms for the visual perception of pitch and roll may have come about, in an evolutionary sense, from their connection to the more primitive vestibular and proprioceptive systems from which they emerged. If so, this suggests that future research wil l continue to find independent mechanisms for the perception of pitch and roll, regardless of the sensory system being studied. 40 References Asch, S. E., & Witkin, H . A. (1948a). Studies in space orientation: I. Perception of the upright with displaced visual fields, journal of Experimental Psychology, 38, p. 325-337. ' ' • Asch, S. E., & Witkin, H . A . (1948b). Studies i n space orientation: II. Perception of the upright with, displaced visual fields and with body tilted. Journal of Experimental Psychology, 38, p. 455-477. Bressan, P., Garlaschelli, L., & Barracano, M . (2003). Antigravity hills are visual illusions. Psychological Science, Carpenter, M . G., Allum, J. H. j . , & Honegger, F. (1999). Directional sensitivity of stretch reflexes and balance corrections for normal subjects in the roll and pitch planes. Experimental Brain Research, 129, p. 93-113. Howard, I. P. (1982). Human visual orientation. New York: Wiley. Howard, I. P. (1986). The perception of posture, self motion, and the visual vertical. In. K. R. Boff, L. Kaufman, & J. P. Thomas (Eds.), Handbook of perception and human performance: Vol. 1. Sensory processes and perception (pp. 1.8.1-18.62). New York: Wiley. Nelson, R., & Prinzmetal, W. (2003). On the relation between roll and pitch. Visual Cognition, 10, p. 715-728. Prinzmetal, W., & Beck, D. M . (2001). The tilt-consistency theory of visual illusions. Journal of Experimental Psychology: Human Perception & Performance, 27, p. 206-217. Shimamura, A . P., & Prinzmetal, W. (1999). The Mystery Spot illusion and its relation to other visual illusions. Psychological Science, 10, p. 501-507. Wilson, V. J., & Melvill Jones, G. (1979). Mammalian, vestibular physiology. New York: Plenum. Witkin, H . A., & Asch, S. E. (1948). Studies in space orientation: IV. Further experiments on perception of the upright with displaced visual fields. Journal of Experimental Psychology, 38, p. 762-782. Young, L. R. (1984). Perception of the body in space: Mechanisms. In J. M . Brookhart & V. B. Mountcastle (Section Eds.) & I. Da.rian-Smi.th (Vol. Ed.), Handbook of physiology: Section 1. The Nervous System: Vol. 3. Sensory Processes (pp. 1023-1066). Bethesda, M D : American Physiological Society. 41 CHAPTER 3: WHEN DO WE KNOW WHICH WAY IS "UP?" MORE TIME IS NEEDED TO RESOLVE GLOBAL THAN LOCAL ORIENTATION ILLUSIONS2 Introduction Dyde & Milner (2002) propose orientation perception is influenced in two stages of processing: local spatial interactions in early stages and longer-range interactions in later stages. We test this hypothesis by varying the interval between a flashed target and surrounding context in two orientation illusions: The Tilt Illusion (Tl) involves dense gratings in close proximity; the Rod and Frame Illusion (RFI) involves a solitary bar surrounded by a distant frame. We also vary whether the context is flashed briefly (Experiment 1) or remains visible (Experiment 2). In support of Dyde & Milner's two-stage model, results show 1). only the Tl is influenced by a briefly flashed context in advance of the target, 2). both illusions are strongest when the context and inducer appear simultaneously, and 3). the frame in the RFI must be visible for at least 800 ms to cause an illusion with asynchronous presentation. 2 A version of this chapter is under review: Corbett, J. E., Handy, T. C , & Enns, J. T. (submitted). When do we know which way is up? More time is needed to resolve global than local orientation illusions. Vision Research. 42 We are largely unaware in everyday life that our perception of upright is determined by subtle interactions between multiple sources of information. Yet studies of unusual environments such as mystery cabins (Shimamura & Prinzmetal, 1999), rotating rooms (Allison, Howard & Zacher, 1999), and magnetic hills (Bressan, Garlaschelli, & Barracano, 2003) highlight the complex interactions that occur between visual, vestibular, and proprioceptive sources of information regarding "upright." Here we examine the time course of perceptual interactions between the orientation of a surrounding visual context and the perceived orientation of a central target. The study was motivated by a recently proposed hypothesis (Dyde & Milner, 2002) that there are separable influences of local (short range) spatial interactions between a stimulus and context in proximity and global (long range) spatial interactions between a stimulus and a distal surrounding context on upright perception (Dyde & Milner, 2002). Dyde & Milner (2002) tested healthy human participants in both the Tilt Illusion (Tl) and the Rod and Frame Illusion (RFI) using measures of perception and visually guided action. The stimulus configurations for these two illusions are illustrated in Figure 3.1. T l displays consisted of a circular central target grating surrounded by a larger circular grating; RFI displays consisted of a large distant frame centered around a small central rod. These two display configurations differed primarily in that the Tl allowed for faster, short-range spatial interactions between the registered orientation of the context and the target, whereas the RFI depended on longer-range spatial interactions for its effects. ' The main finding was that the T l influenced both perceptual judgments of the orientation of the central target grating and visually guided reaching toward the central target. The RFI, on the other hand, had an influence only on perceptual judgments, leaving reaching to the central rod unaffected by the orientation of the surrounding frame. Dyde & Milner (2002) interpreted these results to suggest that the Tl had an 43 influence on relatively early stages of visual processing (perhaps even as early as area V I , where receptive fields are small and sensitivity to local orientation is high), whereas the RFI was manifest during later stages in processing (possibly in extrastriate visual regions in the ventral visual stream, where receptive fields are larger and not as sensitive to local edge orientation). ...for a c lockw ise tilted test Cong ruen t Incongruent T l Figure 3.1. Examples of (a) congruent and (b) incongruent Tl stimuli, and (c) congruent and (d) incongruent RFI stimuli for a clockwise tilted test. Participants perceive the orientation of the test as biased in the opposite direction of the inducing surround, such that the tests in panels (a) and (c) appear more tilted from gravitational upright than their identically tilted counterparts in panels (b) and (d). 44 L i and Matin (2005) added to this conclusion by showing that pointing actions to the rod in RFI displays are initiated in accordance with the illusory visual perception, but are corrected on-line and completed accurately in agreement with upright posture and the felt pull of gravity. This is still in keeping with Dyde and Milner's (2002) proposal that the Tl influences processing before the division of visual processes into "what" and "how" streams (Milner & Goodale, 1995), and the RFI influences visual processes only in the "what" (ventral visual stream). The later influence of context in the RFI therefore allows dorsal processes, thought to be critical for guiding on-line reaching, to be unaffected by the surrounding frame, which only biases interactions in the ventral stream. Dyde & Milner's (2002) hypothesis of early short-range and later long-range interactions in orientation perception makes a clear prediction about the relative time course of contextual influences. That is, although both illusions occur when the surrounding context and the central target are presented simultaneously, presenting these stimuli asynchronously should lead to very different outcomes. Specifically, when the surrounding context is briefly presented before the central target, only the Tl should result in visual adaptation effects. This is because the Tl is believed to involve local spatial interactions between the small receptive fields of area V I and should have sufficient time to occur with brief flashes. The RFI, in contrast should not produce visual adaptation effects because its interactions are thought to occur through longer-range, more object-invariant representation, and should not have enough time to occur with brief, asynchronous presentations. Before describing the experiments we conducted to test these hypotheses, we wi l l briefly review what is known about possible dissociations 45 between the Tl and RFI, including the possibility that their interactions occur on a different time scale. Separate mechanisms in the T l and RFI? Although the Tl and RFI each show the strongest illusion when the context is tilted approximately 15° from vertical (Gibson & Radner, 1937; Over, Broerse, & Crassini, 1972; Beh, Wenderoth, & Purcell, 1971), there are several hints that these illusions are dissociable. For example, the RFI occurs with large frames (e. g. Dyde & Milner, 2002; Asch & Witkin, 1948), whereas T l studies typically employ smaller and closer inducing surrounds (e. g. Wolfe, 1984; Wenderoth & Van der Zwan, 1989). Increasing the retinal size of the frame increases the magnitude of the RFI, or the degree to which the frame biases perceived upright (Ebenholtz, 1977; Ebenholtz & Callan, 1980). In addition, the outer-most frame defines the RFI, such that an upright rod appears tilted when surrounded by a small upright frame and a large tilted frame, but the same rod appears upright when surrounded by a small tilted frame and a large upright frame (Spinelli, Antonucci, Daini, Fanzon, & Zoccolotti, 1995; DiLorenzo & Rock, 1982). These results suggest that large displays outside the range of local inhibitory connections induce the largest RFIs. In contrast, the magnitude of the Tl decreases when there is a spatial gap between the center and surround (Antonucci, et. at., 1995; Wenderoth, van der Zwan, & Williams, 1993; Wenderoth, Johnstone, & van der Zwan, 1989; Wenderoth & Johnstone, 1988; Virsu & Taskinen, 1975). This all suggests that while a similar illusory outcome may occur in Tl and RFI displays, with the target appearing to tilt away from the surrounding orientation, short-range mechanisms may be responsible for the T l and longer-range mechanisms for the RFI. 46 The time course of orientation illusions Much more is known about the time course of the Tl than of the RFI. Note that the Tl is also referred to in the literature as the Tilt Adaptation/ After Effect (if the inducing and test gratings are presented asynchronously) or Simultaneous Tilt Illusion (if the inducer and test are presented in unison). In some of the earliest time course studies of the Tl (Gibson, 1933; Gibson & Radner, 1937) participants were first adapted to a tilted grating for a brief period (several ms to several sees) before judging the orientation of a test grating in the same location. The illusion increased in magnitude along with adaptation time, and was largest when the inducer and target were presented simultaneously. This general pattern has been reported in many other more recent studies, with some reporting an asymptotic illusion with a 200 ms interval between inducing and target presentations (Durrant & Clifford, 2006; Wolfe, 1984; Matin, 1974). Matin (1974) also left the inducing context continuously visible for 3000 ms in two observers. Both observers experienced the largest illusion when inducer and test stimulus were simultaneously presented, though only one of them showed a strong illusion with a large asynchrony between stimuli. Matin (1974) interpreted this as suggesting that motion transients contribute to the illusion, with the simultaneous onsets of inducing context and targets leading to the largest effects. Other studies have explored the factor of stimulus duration (as opposed to stimulus asynchrony). These generally report that the Tl increases with decreasing exposure durations of a simultaneous inducer-target display of up to 10 ms (Wenderoth & van der Zwan, 1989; Wenderoth, van der Zwan, & Johnstone, 1989; Calvert & Harris, 1985; Wenderoth & Johnstone, 1988). However, this interpretation has been contested. 47 O'Toole (1979) proposed that longer exposure durations increased the magnitude of the TI because the central grating was masked by the surrounding context, making it difficult to actually see the target. Clifford and Harris (2005) countered this interpretation by reporting that the TI can be measured with an after-effect, even though the target was masked to such an extent that it could not be discriminated with explicit reports from the participant. A handful of TI studies have varied the inducing duration and the duration of the test flash in the same experiment (Calvert & Harris, 1988; Gibson & Radner, 1937; Harris & Calvert, 1989; Sekular & Littlejohn, 1974; Wolfe, 1984; Wenderoth & van der Zwan, 1989). These can be summarized by three "rules of thumb": the TI is greatest when 1). the inducing context is visible for a longer period of time, 2). when the target stimulus is flashed for a proportionately shorter period of time, and 3). when the inducing context and target appear in close temporal and spatial proximity. This third point is consistent with the involvement of local spatial contour interactions underlying the TI, while the first two points^ are consistent with inducing context serving to "condition" the system to establish a new, short-term, reference for the perception of upright. Finally, no studies have reported an illusion when the inducing context is presented immediately after the presentation of the target, regardless of whether the inducing context is flashed briefly (Durrant & Clifford, 2006; Matin, 1974) or remains visible after it appears (Matin, 1974). This is consistent with the illusion occurring because of rapid-acting forward adaptation effects, rather than because of masking effects that might occur through stimulus integration or interruption of processing. This large literature on the time course of the TI stands a sharp contrast to the lack of corresponding research on the time course of the RFL In light of mounting evidence that these illusions are mediated by different mechanisms, with local inhibition 48 underlying the T l and long-range interactions sub-serving the RFI, it is important to examine whether there are also differences i n the time course of each type of i l lusion. The present study Here we compared the time course of the T l and the RFI for the first time. To be able to compare the two illusions as closely and as sensitively as possible, we modified the participant's task slightly from the traditional one of assessing the Point of Subjective Vertical (PSV) by having participants adjust the target orientation unti l it appears gravitationally upright. Instead, we recorded response time and accuracy for the task of discriminating whether the central target orientation was Counterclockwise (CCW) (left) or Clockwise (CW) (right) from upright, as defined by gravity. Us ing response time and accuracy measures allowed us to be sensitive to the i l lus ion Over a larger range of performance levels. For instance, even when participants are able to perform accurately i n a task, differences i n their response times may belie a differential speed of processing. Similarly, two response times that are similar i n duration may be h id ing two tasks that differ i n accuracy. Fol lowing Dyde & Milner (2002), our hypothesis was that the T l and the RFI wou ld follow different processing time courses. In particular, we hypothesized larger forward-acting adaptation effects for the T l than the RFI because, of the greater involvement of faster, local spatial interactions i n the TL Experiment 1: Brief presentations of inducing context and target orientation Experiment 1 compared the T l and RFI when the inducing context was flashed briefly either before, during, or after a brief flash of the central target. O n any given trial, the target could be tilting by 8°, either C W W or C W , and it could be surrounded by an inducing context orientation that was leaning 15° either C C W or C W . Congruent trials 49 were considered those i n which target and inducer were both tilting i n the same direction, and incongruent trials were those i n which target and inducer were leaning in opposite directions, as illustrated i n Figure 3.1. Differences i n the mean correct response time (RT) or mean percentage correct (PC) between these two types of trials were used to index an il lusion. Specifically, participants should take longer to respond correctly and have reduced accuracy on congruent displays (Figures 3.1a&c) relative to incongruent displays (Figures 3.1b&d). O n congruent trials the inducing context makes it appear as though the target is less tilted or tilted slightly opposite the direction i n which the inducing context is oriented, and on incongruent trials the context highlights the mismatched orientation of target. Note that a baseline measurement of test orientation discrimination without the surrounding inducer is not necessary wi th this task, as i n the traditional PSV task, because congruent and incongruent conditions can be averaged over C C W and C W tilting context orientations, thus averaging out any biases participant's might have to respond i n one direction or the other when they are uncertain. Methods Participants. In total, twenty-nine undergraduate students from the University of British Columbia voluntarily participated i n a one-hour session i n exchange for extra credit i n a department Psychology course. Two separate sets of observers participated i n TI and RFI sessions, and the illusions were never intermixed, such that TI inducers were never presented wi th RFI tests or visa versa. Fifteen observers (10 women and 5 men, aged 18-23) participated i n the TI task and fourteen (8 women and 6 men, aged 18-23) participated i n the RFI task. A l l had normal or corrected-to-normal vis ion and none reported any vestibular or proprioceptive disorders. 50 Task/instructions to participants. O n each trial participants indicated whether the test stimulus tilted C C W (left) or C W (right) of gravity-defined vertical by pressing one of two lateralized response keys ("z" or "?") on a keyboard w i th the left or right index finger, respectively. The experimenter instructed them to fixate on a 0.1° dot i n the center of the display between trials, and that on each trial, a small grating (Tl) or a short rod (RFI) wou ld appear i n the center of the screen. Participants were asked to indicate the direction i n which the test grating or rod was leaning as quickly and accurately as possible, regardless of the orientation of any objects surrounding the test. Trials began at a fixed interval following each response and participants were instructed to respond even if they were uncertain. Trial sequence Each observer participated in four blocks of 64 trials for a total of 256 trials. We began by manipulating the duration between onset of the inducing context and test stimulus (Stimulus Onset Asynchrony, or SOA) and measuring the effects on the RT and P C for each type of i l lusion. O n each trial, participants either saw the inducer followed by the test (positive SO As) , the inducer and test at the same time (simultaneous onset, 0 ms SOA) , or the test followed by the inducer (negative SO As) . In each T l or RFI session, wi th equal probability i n each trial, we tested five positive S O A s (1600 ms, 800 ms, 400 ms, 200 ms, and 100 ms), a simultaneous 0-ms SOA, and two negative S O A s (-100 ms and -200 ms). We recorded RT and P C from the offset of the test i n each trial. Figure 3.2 provides a schematic diagram of the time-course of a trail i n each S O A i n Experiment 1 wi th brief flashes of the inducer and test. 51 S O A (ms) Inducer Test 1600 SS9 800 m 400 200 • 100 1 -100 r hi': : -200 • Figure 3.2. Schematic diagram of the time-course of a tr ial i n Experiment 1 for each S O A (ms) when the inducer flashed for 100 ms. 52 Displays: Stimuli were drawn in black on a white background and appeared in the center of the screen. From a viewing distance of 50 cm, the diameter of the surrounding frame in the Tl was 12.4° of visual angle and one side of the square RFI frame also subtended 12.4° of visual angle. Both T l gratings were composed of 5 cycle per degree sine waves. To elicit the maximum illusions (Beh, Wenderoth, & Purcell, 1971; Wenderoth & Beh, 1977), the inducing frame or grating was tilted 15° CCW or 15° CW with equal probability on every trial. While holding the size of the inducer constant, we minimized the size of the test grating to 2° in T l displays to ensure that local interactions caused the illusion, and maximized the gap between the rod and frame in RFI displays, with a 1.15° rod to further increase the likelihood that global interactions were responsible for the illusion. Each type of test stimulus tilted 8° CCW or CW from vertical at random with equal probability on each trail. Procedure: In each illusion condition (Tl or RFI) each possible combination of inducer (CCW or CW), test (CCW or CW), and SOA (1600 ms, 800 ms, 400 ms, 200 ms, 100 .ms, 0 ms, -100 ms, or -200 ms) appeared at random with equal probability in each block. The lights remained off for the entire duration of each session to minimize the vertical context of the experimental room, which has been reported to decrease tilt illusions (Spinelli, et. al., 1995; Zoccolotti, et. al., 1997; Purves & Howe, 1994; Stoper & Cohen, 1989; Cian, Raphel, & Barraud, 2001; Ebenholtz & Utrie, 1982). To cover the horizontal and vertical lines of the monitor, we affixed a black cardboard annulus to the monitor casing to form a circular viewing window with a 20° inner and a 35° outer diameter. Viewing was binocular without restraint so as not to introduce any extraneous proprioceptive or vestibular inputs to gravity. 53 Each participant completed four blocks of sixteen practice trials. A t the end of each practice block, a percent error rate appeared i n the center of the screen. A s we were primarily concerned wi th the time needed to re-gain an accurate perception of upright i n the presence of a tilted context, we required participants to achieve an average accuracy of at least 80% i n order to proceed to the experimental trials. Each trial was terminated when the participant made a manual response, or was timed-out after 5000 ms. N o participant timed-out i n more than 3% of the total trials. Results Figure 3.3 shows the mean RT and P C data. For the T l , having the inducing context flash briefly before the target led to an i l lusion that grew monotonically as the interval between inducing context and target was reduced from 800 to 0 ms, wi th the largest i l lusion occurring when they both flashed simultaneously. For the RFI, a significant i l lusion was measured only when the context and the target were presented simultaneously. These conclusions were supported by mixed analyses of variance ( A N O V A ) examining the between-participant factor of Display (Tl, RFI), and the within-participant factors of Context-Target Congruency (Congruent, Incongruent) x Temporal Asynchrony (-1600, -800, -400, -200, -100, 0 100, and 200 ms). Simple effects tests were used to examine the pattern of data separately for each display type and planned comparisons involving Fisher's L S D test were used examine each i l lusion at indiv idual time intervals. 54 o congruent incongruent 11 oo -E CD 1000 -£ 900 " a V) c o 800 • Q. QJ 700 " *-> U <u 600 " t_ o u 500 • c TO o> 400 -1 TI - i i i-¥ 11 oo -1000 -900 -800 " 700 600 1 500 400 RFI 1600 800 400 200 100 0 -100 -200 1600 800 400 200 100 0 -100 -200 O O c o t o Q. O CL -i r 1600 800 400 200 100 0 -100 -200 i 1 1 1 1 1 1 r-1600 800 400 200 100 0 -100 -200 SOA (ms) Figure 3.3. Experiment 1 when the inducer visible for 100 ms: Influences of congruent (dashed line) and incongruent (solid line) frame tilts on: a). TI mean correct response time, b). RFI mean correct response time, c). TI mean accuracy, and d). RFI mean accuracy. Error bars indicate ±1 SEM. In addit ion to main effects of all three factors (F's>12.784 / p's<.001), A N O V A s indicated that the TI was greater for a larger number of temporal asynchronies than the RFI. In particular, there was a significant interaction between Display x Congruency (RT: F(l,27)=29.136, p<.001; PC: F(l,27)=14.564, p=.001), and a significant interaction between Display x Temporal Asynchrony (RT: F(7,189)=16.076, /?<.001; PC: F(7,189)=8.513, p<.001). The 0 ms S O A led to the largest effects for both TI and RFI displays, reflected by an interaction between Congruency x Temporal Asynchrony (RT: F(7,189)=27.029, p<.001; 55 PC: F(7,189)=49.396, p<.001). This effect at 0 ms was larger in T l displays than in RFI displays, indicated by a three-way interaction of Display x Congruency x Temporal Asynchrony (RT: F(7,189)=14.068, p<.001; PC: F(7,189)=21.148, p<.001. Planned comparisons for the Tl displays indicated a significant illusion (Congruent more difficult than Incongruent) in the RT data for all intervals between 800 ms and 0 ms, inclusive (p's <.05), and in the PC data for all intervals between 400 ms and 0 ms, inclusive (p's <.05). In addition, the PC data showed significant differences in the -100 ms and-200 ms interval that were opposite in direction to the illusion (p's < .05), probably reflecting a backward masking effect of the inducer on the target. That is, when the inducing context was flashed immediately following the central target, the visibility of the target was reduced by meta-contrast masking mechanisms, and so participants tended to make a response in keeping with the orientation of the inducing context. This would serve to reduce accuracy specifically on incongruent trials. The same planned comparisons for the RFI displays indicated a significant illusion only when the inducing context and target were flashed simultaneously (0 ms interval), in both the RT and PC data (p's <.05). In addition, there were significant differences in the opposite direction for RT in one of the intervals (1600 ms) and for PC in one interval (200 ms) (p's <.05). Discussion Flashing the inducing context briefly and asynchronously with the target had very different effects on the Tl and the RFI. For the Tl , flashing the context immediately before the central target led to the expected adaptation effect. Namely, the illusion grew monotonically between 800 ms and 0 ms, consistent with the rapid adaptation effects others have reported for T l displays (Durrant & Clifford, 2006; Harris & Calvert, 1989; 56 Wolfe, 1984; Matin, 1974; Sekular & Littlejohn, 1974; Gibson & Radner, 1937), and consistent with the hypothesis that the Tl acts via short-range spatial interactions in very early stages of visual processing (Dyde & Milner, 2002). The RFI showed no similar adaptation effects of this kind. A significant illusion occurred with these displays only when the context and target were flashed simultaneously (0 ms interval). This is consistent with the hypothesis that the RFI acts via long-range spatial interactions in later stages of visual processing and thus is not susceptible to low-level adaptation effects (Dyde & Milner, 2002). One factor that may have contributed to the absence of any forward-acting adaptation effects in the RFI in this experiment was that the inducing context was only presented for 100 ms. This short exposure to the inducing frame may not have been long enough for long-range interactions to become active before the display was erased from the screen. In the next experiment, we allowed the inducing context to remain on view, following its onset, until the participant responded. Experiment 2: Extended presentations of the inducing context The participants, task, design, and procedure of Experiment 2 were identical to those of Experiment 1, with the single exception that the inducer remained visible until the end of each trial. Participants were tested in this experiment immediately following their completion of Experiment 1. Figure 3.4 provides a schematic diagram of the time-course of a trial for each SOA. 57 S A O (ms) Inducer j e s t 1600 800 400 200 100 -100 -200 Figure 3.4. Schematic diagram of the time-course of a tr ial i n Experiment 2 for each S O A (ms) when the inducer was continuously v is ib le un t i l end of the tr ial . 58 o congruent incongruent <u 1000 E § 800 CL TI 1600 800 400 200 100 0 -100 -200 1100 ,1000 900 800 b. RFI 1600 800 400 200 100 0 -100 -200 O Q. O 1 .9 " .8 -.7 " .6 -.5 .4 1 .9 i .8 .7 ^ .6 .5 .4 5 - * -d. 1600 800 400 200 100 1600 800 400 200 100 -100 -200 SOA (ms) Figure 3.5. Experiment 2 when the inducer remained on view until the end of each trial: Influences of congruent (dashed line) and incongruent (solid line) frame tilts on a). TI mean correct response time, b). RFI mean correct response time, c). TI mean accuracy, and d). RFI mean accuracy. Error bars indicate ±1 SEM. Results The RT and P C data are shown i n Figure 3.5. Overall , these data are i n accordance wi th the results of Experiment 1, showing a rapid forward bias induced by the local grating i n TI displays, but not by the frame i n RFI displays. These conclusions were t supported by mixed A N O V A s examining the between-participant factor of Display (TI, RFI), and the within-participant factors of Context-Target Congruency (Congruent, 59 Incongruent) x Temporal Asynchrony (-1600, -800, -400, -200, -100, 0 100, and 200 ms). Simple effects tests were used to examine the pattern of data separately for each display type, and planned comparisons involving Fisher's LSD test were used examine each illusion at individual temporal asynchronies. As in Experiment 1, A N O V A s revealed main effects of all three factors on RT and PC (F's>5.011, p's<.001). However, the T l was no longer as prominent with prolonged exposure to the inducer in Experiment 2, showing only a marginal interaction between Display and Congruency in the PC data, F(l,27)=3.048, p=.092, and eliminating the significant interaction between these factors in the RT data for short flashes of the inducer (Experiment 1). A l l other two- and three-way interactions had p's <.05, as in Experiment 1. Planned comparisons for T l displays revealed a significant illusion (Congruent more difficult than Incongruent) in the RT data for the 400 ms and 200 ms intervals (p's <.05), and in the PC data for the intervals between 400 ms and 0 ms, inclusive (p's <.05). However, there was no longer an RT effect at the 100 ms and 0 ms SOAs for prolonged (Experiment 2) versus brief (Experiment 1) exposure to the inducer. The PC data again showed a significant difference in the -100 ms interval in the opposite direction to the illusion (p < .05), likely reflecting a backward masking effect of the inducer on the target. The same planned comparisons for the RFI displays again showed a significant illusion when the inducing context and target onset simultaneously (0 ms SOA), in both RT and PC data (p <.05) as in Experiment 1 with a briefly flashed inducer. However, prolonged exposure to the inducer in Experiment 2 now caused a significant illusion in the PC data at the 1600 ms and 800 ms SOAs (p's <.05). 60 Discussion The main finding in Experiment 2 was a moderate effect of the frame that emerged in the PC data when the frame was continuously visible for 800 ms or more in advance of the rod. There was again an illusion with simultaneous presentations for both illusions, and a forward-acting TI when the inducing grating remained on view approximately 400 ms in advance of the test grating. As in previous investigations of the TI time course (Wenderoth & van der Zwan, 1989; Wenderoth, van der Zwan, & Johnstone, 1989; Calvert & Harris, 1985; Wenderoth & Johnstone, 1988), the illusion was weaker with prolonged (Experiment 2) versus brief (Experiment 1) exposure to the inducing grating. The results of Experiment 2 provide further support for Dyde and Milner's (2002) proposal that orientation perception is influenced in two stages of processing, by early local spatial interactions and later, long-range global orientation effects. The local inducing context in the TI again exerted a forward-acting bias on the perceived orientation of the rod. As predicted, when the frame in RFI displays remained on view for an extended duration (800 ms or longer), the long-range interactions associated with the global orientation context of the frame had sufficient time to'bias the perceived orientation of the forthcoming rod. General Discussion The main finding of this study is that the Tilt Illusion (TI) has a different time course than the Rod-and-Frame Illusion (RFI). According to Dyde and Milner (2002), the TI occurs early on in processing, affecting both ventral and dorsal inputs for perceptions and visually-guided actions, respectively, via rapid local adaptation. The RFI occurs later in ventral brain areas after the bifurcation of the two visual streams via longer-range and therefore slower acting influences. Early local interactions in TI displays are supported 61 by our findings that: 1). the inducing grating i n TI displays always affected the time needed to correctly determine the orientation of the test grating regardless of whether it flashed briefly up to 800 ms before the onset of the test grating (Experiment 1), 2). appeared simultaneously wi th the test grating (Experiments 1 and 2), or 3). remained visible up to 400 ms before the test grating (Experiment 2). Longer-range global interactions i n RFI displays are supported by our findings that when the global frame was not part of the same temporal event as the, rod, as i n simultaneous, 0 ms S O A conditions, the frame only affected the perceived orientation of the rod when it remained continuously on v iew for at least 800 ms beforehand (Experiment 2). The absence of an i l lusion for either TI or RFI displays i n the negative S O A conditions (both i n Experiments 1 and 2) confirms previous reports that an inducer appearing after a test has offset does not affect the perceived orientation of the test (Matin, 1974; Durrant & Clifford, 2006). The monotonically increasing effect of the inducing grating i n T I displays i n Experiment 1, and the more moderate effect of the frame wi th sustained presentation of the inducing grating up to about 400 ms before the test i n Experiment 2, support the proposal of a rapid time course for the local interactions involved i n the TI. The lack of an effect i n asynchronous RFI displays without several hundred milliseconds of sustained sensory support suggest a relatively slower time course for the long-range global effects involved i n the RFI. Different time courses suggest that different spatial frames of reference may be involved i n each il lusion. Large displays outside the range of local inhibitory connections induce the largest RFI's, such that an upright rod appears tilted when surrounded by a small upright frame and a large tilted frame, but the same rod appears upright when surrounded by a small tilted frame and a large upright frame (Spinelli, Antonucci, Daini , Fanzon, & Zoccolotti, 1995; DiLorenzo & Rock, 1982). In contrast, the magnitude of the TI decreases when a spatial gap between the center and 62 surround impedes local interactions (Antonucci, et. at., 1995; Wenderoth, van der Zwan, & Will iams, 1993; Wenderoth, Johnstone, & van der Zwan, 1989; Wenderoth & Johnstone, 1988; V i r su & Taskinen, 1975). Furthermore, Zoccolotti and colleagues (1997) found that when a rod is surrounded by two frames, the outer-most frame determines perceived upright when large luminous displays are presented i n an otherwise dark room, but that the inner-most frame determines perceived upright when small displays are presented i n a well-l i t environment. Based on these findings, Zocollotti et. al. (1997) proposed that environment-centered allocentric representations of space are mainly affected by the most global frame of reference, whereas egocentric observer-centered representations are most influenced by local orientation context. Zocollotti et. al.'s (1997) interpretation is consistent wi th Dyde and Milner ' s (2002) findings that the local inhibit ion responsible for the T l affects perceptions and actions early on at the receptive field level, but that the RFI only affects the perceived orientation of objects later i n processing after the dorsal and ventral streams split. This proposal is also supported by the present findings that the local grating i n T l displays induced a rapid-forward acting bias on the perceived orientation of the test grating, but that no long range effects of the frame occurred i n the RFI displays unless the global frame was visible for several hundred milliseconds i n advance of the rod. Specifically, the T l occurs before the two visual streams split and therefore has a early, feed-forward effect on both allocentric and egocentric spatial representations. O n the other hand, allocentric representations buil t mainly from ventral inputs to perceive the global orientation of the surrounding environment are most affected by the longer-range interactions i n RFI displays, after the split of the two visual streams for perception and visually guided 63 action. The later interactions i n RFI displays i n the ventral stream have little effect on visual inputs i n the dorsal stream, largely responsible for an accurate egocentric representation of space to visually guide actions to the rod. Local egocentric representations formed mainly i n the dorsal stream and global allocentric representations formed mainly i n the ventral stream may explain why visually guided reaches to the rod i n RFI displays start wi th respect to the global frame, but are rapidly corrected over the course of the movement to correspond to the local orientation of the rod (Li & Matin, 2005). Reaching i n the case of the T l is governed from the outset by local orientation context, and therefore an egocentric frame of reference. O n the other hand, reaching i n the case of the RFI begins w i th an allocentric frame of reference, as observers first v iew the RFI displays and form perceptual judgments about the orientation of the rod. Yet, when observers reach for the rod, they switch from an allocentric representation influenced by the global frame to an egocentric representation, rapidly localized online by dorsal visual stream (and proprioceptive) feedback to come into line wi th the rod. Studies showing that the RFI is affected by vestibular and proprioceptive disturbances (Corbett, & Enns, 2006; Prinzmetal & Beck, 2001), but that the T l is relatively unaffected by these manipulations (Wenderoth & Burke, 2006) are also consistent wi th rapid local context effects on egocentric reference frames and slower global context effects on allocentric reference frames. Also consistent is the f inding that large or rotating displays induce ocular torsion, or involuntary eye-movements that accompany vestibular stimulation (e. g. Dichgans & Brandt, 1974; Bischof, 1974; Merker & H e l d , 1981; Guerraz, 64 Poquiri, & Ohlmann, 1998). These findings correspond with the idea that the slower mechanisms involved in correcting for global context effects are informed by visual, vestibular, and proprioceptive inputs, whereas purely visual mechanisms are involved in local context effects. The present finding of different time courses for these two orientation illusions has important implications for many situations in which it is critical to predict the onset of visual bias in perceived upright. Local context illusions that rapidly affect both perceptions and actions require rapid local corrections, whereas global illusions that do not affect actions can be corrected by several hundred milliseconds of exposure to a global context for upright. For example, astronauts in microgravity, where visual signals are even more dominant than usual because of the absence of gravity-based vestibular and proprioceptive information, frequently report sudden reversals in the direction of perceived upright, which they refer to as "the downs" (Kornilova, 1997), triggered by unpredictable influences of available information in a sparse visual environment. Simply viewing a colleague in an upside-down position can cause the astronaut to experience a sudden reversal in the surfaces perceived as the ceiling and floor (Oman, Howard, Smith, Beall, Natapoff, Zacher, & Jenkins, 2003). Determining how rapidly such reorientation illusions onset would aid in determining whether factors that can overcome these illusions must be geared towards an egocentric or allocentric spatial coordinate system, and therefore implemented at the local or global level, respectively. In summary, the present study provides strong support in the temporal domain for Dyde & Milner's (2002) proposal that local spatial interactions influence the perception of upright during early stages of processing and that longer-range spatial interactions influence later stages of processing, after the separation between the specialized dorsal and ventral visual streams. 65 References Allison, R.S., Howard, LP., & Zacher, J.E. (1999). Effect of field size, head motion, and rotational velocity on roll vection and illusory self-tilt in a tumbling room. Perception, 28, p. 299-306. Antonucci, G., Fanzon, D., Spinelli, D., & Zoccolotti, P. (1995). Visual factors affecting the rod-and-frame illusion: Role of gap size and frame components. Perception, 24(10), p. 1119-1130. Asch, S. E., & Witkin, H . A. (1948). Studies in space orientation: I. Perception of the upright with displaced visual fields. Journal of Experimental Psychology, 38, p. 325-337. Beh, H. , Wenderoth, P., & Purcell, A. T. (1971). The angular function of a rod-and-frame illusion. Perception & Psychophysics, 9, p. 353-355. Bischof, N . (1974). Optic vestibular orientation to the vertical. IN J. J. Kornhuber (Ed.), Handbook of sensory physiology, Vol. 6, Pt. 2, pp. 155-190. New York: Springer-Verlag. Bressan, P., Garlaschelli, L., & Barracano, M . (2003). Antigravity hills are visual illusions. Psychological Science, 14, p. 441-449. Calvert, J. E. & Harris, J. P. (1985). Tilt illusion decreases as presentation time increases. Perception, 14(1), A29. Calvert, J. E., & Harris, J. P. (1988). Spatial frequency and duration effects on the • tilt illusion and orientation acuity. Vision Research, 28(9), p. 1051-1059. Cian, C , Raphel, C , Barraud, P.A. (2001). The role of cognitive factors in the rod-and-frame effect. Perception, 30(12), p. 1427-1438. Clifford, C. W. G , & Harris, J. A. (2005). Contextual modulation outside of awareness. Current Biology, 15, p. 574-578. Dichgans, J., & Brandt, T. (1974). The psychophysics of visually induced perception of self-motion and tilt. In F.O. Schmitt (Ed.), The neurosciences: Third study program (pp. 123-129). Cambridge, M A : MIT Press. DiLorenzo, J. R., & Rock, I. (1982). The rod-and-frame effect as a function of the righting of the frame. Journal of Experimental Psychology: Human Perception & Performance, 8(4), p. 536-546. Durant, S., & Clifford, C. W. G. (2006). Dynamics of the influence of segmentation cues on orientation perception. Vision Research, 46, p. 2934-2940. Dyde , R. T., & Milner, D. (2002). Two illusions of perceived orientation: One fools all of the people some of the time; the other fools all of the people all of the time. Experimental Brain Research, 144, p. 518-527. 66 Ebenholtz, S. M . (1977). Determinants of the rod and frame effect: The role of retinal size. Perception and Psychophysics, 22(6), p. 531-538. Ebenholtz, S. M . , & Callan, J. W. (1980). Modulation of the rod and frame effect: Retinal angle vs. apparent size. Psychological Research, 42(4), p. 327-334. Ebenholtz, S. M . , & Utrie, J. W. Jr. (1982). Inhibition of the rod-and-frame effect of circular contours. Perception & Psychophysics, 32(2), p. 199-200. Gibson, J. J. (1933). Adaptation, after-effect, and contrast in the perception of curved lines. Journal of Experimental Psychology, 16, p. 1-31. Gibson, J. J. & Radner, M . (1937). Adaptation, after-effect, and contrast in the perception of tilted lines. I. Quantitative studies. Journal of Experimental Psychology, 20, p. 453-469. Guerraz, M . , Poquin, D., & Ohlmann, T. (1998). The role of head-centric spatial reference with a static and kinetic visual disturbance. Perception and Psychophysics, 60(2), p. 287-295. Harris, J. P., & Calvert, J. E. (1989). Contrast, spatial frequency, and test duration effects on the tilt aftereffect: Implications for underlying mechanisms. Vision Research, 29(1), p. 129-135. Kornilova, L. N . (1997). Orientation illusions in spaceflight. Journal of Vestibular Research, 7(6), p. 429-439. L i , W., & Matin, L. (2005). Two wrongs make a right: Linear increase of accuracy of visually-guided manual pointing, reaching, and height-matching with increase in hand-to-body distance. Vision Research, 45, p. 533-550. Matin, E. (1974). Light adaptation and the dynamics of induced tilt. Vision Research, 14, p. 255-265. Merker, B. H. , & Held, R. (1981). Eye torsion and the apparent horizon under head tilt and visual field rotation. Vision Research, 21, p. 543-547. Milner, A . D., & Goodale, M . A. (1995). The visual brain in action. London: Oxford University Press. Oman, C. M . , Howard, I. P., Smith, T., Beall, A . C , Natapoff, A., Zacher, J. E., & Jenkin, H . L. (2003). The role of visual cues in microgravity spatial orientation. J. Buckey, J. Homick (Eds.). Neurolab Spacelab Mission: Neuroscience Research in Space, N A S A SP-2003-535, p. 69-81. O'Toole, B. (1979). Exposure-time and spatial-frequency effects in the tilt illusion. Perception. 8(5), p. 557-564. 67 Over, R., Broerse, J., & Crassini, B. (1972). Orientation Illusion and masking in central and peripheral vision. Journal of Experimental Psychology, 96(1), p. 25-31. Prinzmetal, W., & Beck, D. M . (2001). The tilt-constancy theory of visual illusions. Journal of Experimental Psychology: Human Perception and Performance, 27(1), p. 206-217. Purves, D., & Howe, C. Q. (2004). The statistics of natural scene geometry predict the perception of angles and line orientation [Abstract]. Journal of Vision, 4(8), p. 40a. Sekular, R., & Littlejohn, J. (1974). Tilt aftereffect following very brief exposures. Vision Research, 14, p. 151-152. Shimamura, A . P., & Prinzmetal, W. (1999). The Mystery Spot illusion and its relation to other visual illusions. Psychological Science, 10, p. 501-507. Spinelli, D., Antonucci, G., Daini, R., Fanzon, D., Zoccolotti, P. (1995). Modulation of the rod-and-frame illusion by additional external stimuli. Perception, 24(10), p. 1105-1118. Stoper, A . E., Cohen, M . M . (1989). Effect of structured visual environments on apparent eye level. Perception & Psychophysics, 46(5), p. 469-475. Virsu, V. & Taskinen, H . (1975). Central inhibitory interactions in human vision. Experimental Brain Research, 23(1), p. 65-74. Wenderoth, P. & Beh, H . (1977). Component analysis of orientation illusions. Perception, 6, p. 57-75. Wenderoth, P. & Burke, D. (2006). Testing the tilt-constancy theory of visual illusions. Perception, 35(2), p. 201-213. Wenderoth, P. & Johnstone, S. (1988). The differential effects of brief exposures and surrounding contours on direct and indirect tilt illusions. Perception, 17, p. 165-176. Wenderoth, P., Johnstone, S., & van der Zwan, R. (1989). Two-dimensional tilt illusions induced by orthogonal plaid patterns: Effects of plaid motion, orientation, spatial separation, and spatial frequency. Perception, 18, p. 25-38. Wenderoth, P., & van der Zwan, R. (1989). The effects of exposure duration and surrounding frames on direct and indirect tilt aftereffects and illusions. < Perception & Psychophysics, 46(4), p. 338-44. Wenderoth, P., van der Zwan, R., & Johnstone, S. (1989). Orientation illusions induced by briefly flashed plaids. Perception, 18, p. 715-728 Wenderoth, P., van der Zwan, R., & Williams, M . (1993). Direct evidence for competition between local and global mechanisms of two dimensional orientation illusions. Perception, 22, p. 273-286. 68 Wolfe, J. (1984). Short test flashes produce large tilt aftereffects. Vision Research, 24(12), p. 1959-1964. Zoccolotti, P., Antonucci, G., Daini, R., Martelli, M . L., & Spinelli, D. (1997). Frame-of-reference and hierarchical-organization effects in the rod-and-frame illusion. Perception, 26(12), p. 1485-1494. 69 CHAPTER4: ELECTROPHYSIOLOGICAL EVIDENCE FOR A POST-PERCEPTUAL LOCUS OF VISUAL AND VESTIBULAR EFFECTS ON PERCEIVED UPRIGHT Introduction Humans rely on a hierarchy of multisensory inputs to perceive "up," such that a rod appears tilted in the opposite direction of a surrounding frame (the Rod and Frame Illusion, RFI), but vestibular and proprioceptive inputs help to counteract the illusion. Behavioral evidence suggests that the RFI is manifest during later, post-perceptual stages of processing. To test this proposal, we varied the orientation of the frame and measured the cortical activity associated with the perceived orientation of the rod using event-related potentials (ERPs). P3 amplitudes were larger when the frame was tilted to accentuate the orientation of the rod compared to when the orientation of the frame caused the rod to appear less tilted (Experiment 1). Head tilt, which impairs orientation perception for behavioral measures, delayed the onset and attenuated the amplitude of the frame's effect on the P3 (Experiment 2). Results support a post-perceptual locus for multisensory integration in the presence of a tilted visual context. 3 A version of this chapter is under review: Corbett, J. E., Enns, J. T., & Handy, T. C. (submitted). Electrophysiological evidence for a post-perceptual locus of visual and vestibular effects on perceived upright. Experimental Brain Research. 70 Normal ly , we effortlessly integrate external cues from surfaces i n the visual environment and internal bodily position signals to accurately determine which way is "up." However, when sensory signals conflict, visual input can bias our perceptions of gravitational upright. Tilted mystery cabins provide first-hand demonstrations of the misperceptions that can result when visual, vestibular, and proprioceptive cues to upright are i n conflict. Visitors to these tourist attractions experience objects and people resting at impossible, gravity-defying angles due to the misleading references to upright given by the tilted walls, floor, and ceiling of the cabin. In the simplest form of these illusions, observers perceive the orientation of a rod i n the context of a surrounding tilted frame, such that the rod appears tilted i n the opposite direction of the frame (the Rod and Frame Illusion, or RFI) (Asch & Witk in , 1948). Til t ing observers' heads or whole bodies from gravitational upright increases the magnitude of the RFI (Corbett & Enns, 2006), suggesting that observers reference vestibular input about the position of the head wi th respect to gravity i n order to override misleading visual information and accurately judge the orientation of the rod. Behavioral evidence suggests that this muitisensory integration occurs during later, post-perceptual stages of processing. Dyde and Mi lner (2002) found that while observers misperceived the position of stimuli presented inside a tilted frame, they could accurately point to these same stimuli. Similarly, Bridgeman and colleagues (1997) reported that an offset frame always caused a target to appear displaced i n the opposite direction (the Roelofs Effect), but half of the participants tested made accurate reaching movements to the target. L i and Mat in (2005) later demonstrated that pointing movements to st imuli inside tilted frames are initiated wi th respect to the illusory visual perception, but terminate i n accordance wi th an accurate perception of gravitational upright, suggesting that the RFI does not affect visually-guided actions. Taken together, 71 these findings suggest that vestibular and visual inputs are integrated by brain areas active later during post-perceptual stages of processing, after the bifurcation of the dorsal and ventral visual streams, thought to process visual information for perception and action, respectively (Milner & Goodale, 1995). v In addition to behavioral evidence i n favor of a post-perceptual locus for such multisensory integration, there is also evidence against the possibility of an earlier locus for global context illusions. Namely, if the RFI is accomplished by early visual processes involving interactions between receptive fields or low-level edge detectors, then the i l lusion should result from a small tilted proximal frame and decrease as the retinal image of the rod and frame becomes larger, extending past the range of receptive fields. O n the contrary, increasing the retinal size of the frame increases the magnitude of the il lusion, or the degree to which the frame biases perceived upright (Ebenholtz, 1977; Ebenholtz & Callan, 1980). Similarly, the most global, or outer-most visual context defines upright i n the external environment such that an upright rod appears tilted when surrounded by a small upright frame and a large tilted frame, but the same rod appears upright when surrounded by a small tilted frame and a large upright frame (Spinelli, Antonucci, Daini , Fanzon, & Zoccolotti, 1995; DiLorenzo & Rock, 1982). Both of these findings suggest that longer-range interactions based on global shape perception, not earlier local interactions between adjacent edges, are mainly responsible for the RFI. Despite this growing body of evidence that tilted visual references i n the surrounding environment affect post-perceptual stages of processing, the late temporal locus impl ied by these finding has yet to be directly confirmed. Towards addressing this issue, event-related potentials (ERPs) yie ld high-resolution measurements of the time-course of neural activity associated wi th perceptual and post-perceptual processes without the added time required to plan and execute motor responses necessary for 72 behavioral reaction time measures. Therefore, we took advantage of this temporal precision, i n conjunction wi th behavioral measures, to investigate whether this type of muitisensory integration occurs during early, or post-perceptual stages of processing. Specifically, we recorded ERPs to the onset of a flashed rod surrounded by a tilted frame while observers determined whether the rod was tilted Counterclockwise (CCW) or Clockwise (CW) from gravitational upright. While humans must reference vestibular and proprioceptive inputs multiple times over the course of processing to prevent the body from tipping over wi th each slight change of viewpoint, the present investigation is specifically concerned wi th one such time during the course of processing when visual and vestibular inputs are integrated to overcome global tilt illusions such as the RFI. Experiment 1 In Experiment 1, we used ERPs i n conjunction w i th behavioral measures to examine whether observers overcome environmental tilt i l lusions dur ing early, perceptual stages of processing or later, post-perceptual stages. Neura l activity associated wi th early, sensory and perceptual-level visual processing is indexed by changes i n the amplitude of the PI component (the first positive deflection typically observed i n ERP waveforms 100 ms after stimulus onset), and the N l component (the first negative-going deflection typically observed 200 ms after stimulus onset) (Luck & Hi l lya rd , 1995). O n the other hand, changes i n the amplitude of the later P3 component, or the third positive-going deflection typically found 300 ms after stimulus onset, are associated wi th post-perceptual processes such as work ing memory (Donchin & Coles, 1988), or decision certainty (Johnson, 1986). 73 To test between an early, perceptual, or later, post-perceptual locus of the RFI, we used a version of the task i n which a tilted frame could induce two different percepts of a physically identical rod; one that aids discrimination of the rod's physical tilt and one that impairs rod orientation discrimination. When the rod and frame are incongruently oriented (Figure 4.1a), the mismatch i n orientation should accentuate the tilt of the rod, making it appear even more tilted from upright than it physically is. O n the other hand, when the rod and frame are tilted congruently, the orientation of the rod is biased i n the opposite direction of the tilted frame causing the rod to appear less tilted, upright, or even slightly tilted i n the opposite direction than it physically is (Figure 4.1b). Examining the cortical activity elicited by the rod as a function of the orientation of the frame for trials i n which observers correctly determined the orientation of the rod allowed us to examine whether the RFI is manifest during early or post-perceptual stages of processing. Modulat ions of the amplitude of earlier PI and /or N I components elicited by the rod as a function of the orientation of the frame w o u l d suggest that the RFI directly influences sensory/perceptual-levels of visual processing. Conversely, differences i n the amplitude of the P3 component i n response to the rod as a function of the orientation of the frame wou ld provide direct electrophysiological support for the proposed late temporal locus of global tilt illusions such as the RFI. Methods Participants Forty-one right-handed undergraduate students (22 women and 19 men, aged 18-28) at the University of British Columbia voluntarily participated i n a two-hour session i n 74 exchange for $20 (CAD). A l l participants had normal or corrected-to-normal vision, and none reported any neurological disorders. A l l procedures and protocols were in accordance with the University of British Columbia's Institutional Review Board. Figure 4.1. Examples of (a), incongruent and (b). congruent trials. Observers determined whether the rod was tilted CCW (left) or CW (right) from gravitational vertical (In this example, " C W " (right) was the correct response). Although the two rods are physically identical, the perceived orientation of the rod is biased in the opposite direction of the frame, such that the rod in panel (a) surrounded by an incongruentiy tilted frame appears more tilted from gravitational upright than the rod in panel (b) surrounded by a congruently tilted frame. On each trial, observers were required to indicate whether the top of a rod, flashed briefly inside a square frame, was tilted CCW (left) or CW (right) from gravitational upright (Figure 4.1). The experimenter informed participants that they should judge the tilt of the rod on each trial, regardless of the orientation of the surrounding frame, and instructed them to make a prompt decision even if they were unsure of the rod's tilt, but to respond as accurately as possible. a) . b ) . Task 75 A p p a r a t u s Observers sat approximately 65 cm away from a Macintosh monitor, which displayed stimuli generated by a Pentium P C using the V A P P stimulus presentation system (http:/ /nilab.psychiatry.ubc.ca/vapp/). We took several steps to increase the l ikelihood that observers w o u l d rely on the frame i n the RFI displays to perceive the direction of upright instead of using the upright context given by surfaces i n the external laboratory. To al low observers to v iew complete RFI displays without seeing any of the horizontal or vertical edges of the monitor that may have provided them wi th an alternative visual reference to upright, we mounted a black cardboard annulus wi th an inner radius of approximately 20° over the rectangular monitor to create a viewing w i n d o w wi th inner and outer circular contours. In addition, to l imit the availability of the upright context of the experimental room, we switched off the room lights several minutes before beginning each experimental session, and lights remained off throughout the entire duration of the experiment i n order to further l imit any upright references inherent i n the surrounding laboratory. D e s i g n , Stimuli, a n d P r o c e d u r e Observers participated i n seven blocks of sixty trials for a total of 420 trials each. A l l st imuli were black on a white background. Each trial began wi th a square frame subtending 10° visual angle displayed i n the center of the circular v iewing window. We presented the rod and frame i n temporal succession to record the neural activity associated wi th the onset of the rod alone as a function of frame orientation. The frame was either tilted 15° C C W or 15° C W from gravity. To avoid confounding neural activity associated wi th the perception of the rod stimulus wi th neural activity associated wi th expectancies from a learned temporal "rhythm" of repeated rod and frame durations, we 76 varied the length of the frame presentation before and after the 100 ms rod stimulus on each trial. Behavioral response time data collected during pilot studies showed that the context of the tilted frame had the largest effect on the perceived orientation of the rod when it was visible 1200 ms before the 100 ms rod stimulus appeared. Therefore, 1000 ms - 1400 ms after the onset of the frame, a rod subtending 1.5° visual angle appeared for 100 ms i n the center of the frame. We chose to use a small rod and a relatively large surrounding frame to allow for a large spatial gap between the rod and frame to maximize the l ikel ihood that local interactions could not account for any observed differences i n correct response times or cortical activity. The rod was tilted 8° C C W from gravity at random on half of the trials i n each block, and 8° C W from gravity on the other half of trials. The frame remained on the screen for 800 ms - 1200 ms after the rod had offset. O n each trial, observers pressed the left button on a hand-held controller wi th the left thumb if the rod was tilted C C W , or the right button wi th the right thumb if the rod was tilted C W from gravity, and we allowed them up to 3000 ms after the offset of the frame to make their choice. We measured their response time and accuracy using a remote computer l inked to the hand-held controller. The four possible combinations of frames ( C C W or C W ) and rods ( C C W or CW) were presented at random wi th equal probability i n each block. Because we were interested i n the amount of time observers needed to overcome the RFI and thus to correctly resolve the tilt of the rod w i th respect to gravitational upright, participants first completed a block of sixty practice trials wi th at least 85% accuracy before they were permitted to begin the experimental trials. 77 Electrophysiological Recording Concurrently wi th , behavioral measures of RT and accuracy, we recorded electroencephalograms (EEGs) from multiple scalp locations using 20 tin electrodes mounted i n an elastic cap. A l l E E G activity was recorded wi th reference to the left mastoid, amplified using a Grass Instruments, M o d e l 12 Neurodata Acquis i t ion System wi th a band pass of 0.1-30 H z (half-amplitude cut-offs), and digit ized on-line at a sampling rate of 256 samples-per-second. To ensure proper eye fixation, vertical and horizontal electro-oculograms (EOGs) were also recorded; the vertical E O G s from two electrodes, one inferior to the left eye and one inferior to the right eye, and the horizontal E O G s from two electrodes on each of the left and right outer canthi. A l l electrode impedances were kept below five kQ. Off-line, computerized artifact rejection was used to eliminate trials during which detectable eye movements (> 1°), blinks, muscle potentials, or amplifier blocking occurred. For each participant, E E G s were time-locked to a 3000 ms epoch centered around the presentation of the rod stimulus on each trial. Subsequently, al l EEGs were algebraically re-referenced to the average of the left and the right mastoid signals, and filtered wi th a low-pass Gaussian filter (25.6 H z half^ amplitude cut-off) to eliminate high-frequency artifacts i n the waveforms. Results Behavioral RFI Criterion A s some observers are less dependent on the orientation of the surrounding visual field to determine upright than others (Witkin, 1949), we used a behavioral criterion to distinguish observers that were subject to the RFI. We collapsed the data over C C W and 78 C W rod tilts to compare incongruent trials (observers should respond faster if the rod appears more tilted than it physically is, Figure 4.1a) to congruent trials (observers should respond slower if the rod looks less tilted or even upright, Figure 4.1b). 560 CO — 540 co CD CO c O CL co 0 o CD O O 520 500 480 460 T r Incongruent Tr ia ls C o n g r u e n t Tr ia ls Figure 4.2. Mean correct response times to incongruent and congruent trials for participants who were subject RFI in Experiment 1. Error bars indicate ±1 SEM. Thirty-seven of the forty-one participants met our behavioral criterion such that they correctly responded faster to incongruent trials relative to congruent trials, r(35)=8.52, p<.001 (Figure 4.2). The same pattern held for response accuracy, such that observers responded wi th higher average levels of accuracy i n incongruent trials as compared to congruent trials, r(35)=5.48, p<.001. However, we required participants to achieve an average accuracy of at least 85% i n order to proceed to the experimental trials because we were primarily concerned wi th the ability to accurately perceive upright i n the presence of a tilted context. Consequently, accuracy was at ceiling and not analyzed further. 79 -4 C z 0 4 8 12 14 o o o o o o o o o o o o o o o o o o o o o CM v T - C M C O T M D ( O N C O Figure 4.3. Grand-average ERP waveforms at lateral occipital and midline parietal electrode sites for incongruent trials (dashed line) and congruent trials (solid line) in Experiment 1. The modulations of the P3 amplitude over central parietal electrode sites C z and P z are outlined with solid rectangles, and the lack of corresponding effects in PI and N l amplitudes over lateral occipital sites 0 „ and O z are outlined with dashed rectangles. 80 Electrophysiological Data We collapsed the resultant ERPs for correct trials across C C W and C W rod tilts, and then used participants' averaged waveforms to derive the grand-averaged waveforms for incongruent and congruent trials. We analyzed the mean amplitudes of the P I , N I , and P3 components i n the grand-averaged ERPs by computing mean amplitudes for each component relative to a 200 ms pre-stimulus baseline, and then identifying the latency of the peak amplitude of each waveform for each electrode and condition of interest. Specifically, we compared incongruent and congruent ERPs over lateral and central occipital electrode sites Tg, OU O z , and 0 2 (scalp locations where the PI and N I are maximal), and over central parietal sites C z and P z (scalp locations where the P3 is maximal) (Coles & Rugg, 1995). For data points entered into the following statistical analyses, we then measured the amplitude of the single-subject waveforms wi th in each electrode and condition during the 20 ms w i n d o w surrounding the corresponding peak amplitude for each component i n the grand average (see Handy, 2005). For observers considered i n ERP analyses, no more than 10% of trials were rejected due to artifacts i n each of the four conditions resulting from all possible combinations of the two rod positions ( C C W and C W ) and the two frame positions ( C C W and CW) . The grand-averaged waveforms for observers who were subject to the RFI showed an increase i n the amplitude of the P3 component for incongruent trials relative to congruent trials over central parietal sites C z and P z (Figure 4.3). A s the P3 modulation appeared maximal around 400 ms, this was confirmed by a single-factor repeated-measures A N O V A on the mean P3 amplitudes from 390 ms - 410 ms at electrodes C z and Pz, which showed a significant main effect of Frame Til t (Incongruent, Congruent), 81 F(l,36)=25.81, p<.001. Two additional repeated-measures A N O V A s confirmed that there were no significant effects of Frame Tilt on the amplitudes of the P I (100 ms - 120 ms), or the N I (180 ms - 200 ms) over lateral occipital electrode sites, T 5 , T^ Ov O z , and 0 2 (both F's <1, both p's >.535). Whi le only 4 of the 41 observers were not subject to the RFI, their indiv idual and grand-averaged waveforms showed no modulation of any components. Discussion With regard to the question of whether the RFI is manifest during perceptual or post-perceptual stages of processing, the results of Experiment 1 are consistent wi th a post-perceptual locus. We found that the orientation of the surrounding frame inversely modulated the amplitude of the P3 i n response to the rod and correct manual response times, wi th larger P3 amplitudes for incongruent trials relative to congruent trials, but longer response times for congruent trials relative to incongruent trials. A s the P3 occurs sequentially later i n the time-course of visual processing than the P I or N I , these findings imply that the orientation of the rod biases post-perceptual stages of processing, presumably through access to intact vestibular input (and to a lesser ' extent proprioceptive input) when visual cues are unreliable. If the modulation of P3 amplitude found i n Experiment 1 indexes a post-perceptual locus for the RFI, then manipulations that influence behavioral measures of the ability to accurately perceive upright should result i n a corresponding manipulation of the P3. Specifically, tilting the head disrupts stabilizing vestibular inputs and increases the degree to which the orientation of a vertical rod appears tilted i n the opposite direction of a surrounding frame (Corbett & Enns, 2006). It follows that if the observed P3 modulation is an index of the influence of the tilted frame on the perceived orientation of the rod, head tilt should also affect this E R P component. A t the same time, the RFI task in 82 Experiment 1 may not have been challenging enough to affect earlier sensory and perceptual P l / N l components, but under more perceptually-demanding conditions wi th the head tilted, these components may show an effect of the RFI. Therefore, we also wanted to confirm that the absence of perceptual-level effects i n Experiment 1 could not simply be ascribed to low perceptual demands. Experiment 2 In Experiment 2, we compared ERPs and correct response times for incongruent trials and congruent trials when observers' heads were upright and when their heads were tilted. If the increased amplitude of the P3 for incongruent versus congruent trials i n Experiment 1 reflects a post-perceptual locus for the RFI, then head tilt should increase the amount of time needed to correctly discriminate the tilt of the rod and decrease the effect of the frame on the amplitude of the corresponding P3 component. Methods Twenty-two right-handed undergraduate students (11 women and 11 men, aged 18-27) at the University of British Columbia voluntarily participated i n a two-hour session i n exchange for $20 (CAD) . A l l participants had normal or corrected-to-normal vision, and none reported any neurological disorders. A l l behavioral and E R P methods i n Experiment 2 were identical to Experiment 1, except that each observer i n Experiment 2 participated i n two conditions, head upright and head tilted 15°, wi th five consecutive blocks of sixty trials for a total of 300 trials for each observer i n each condition. The order of conditions (head upright and head tilted) and the direction of head tilt ( C C W and CW) was counterbalanced between participants, and they each completed one block of 60 practice trials wi th at least 85% accuracy at the start of each condition. The experimenter 83 continuously monitored the position of each participant's head wi th reference to a 15° line positioned on the wa l l behind the observer, adjusted to be in-line wi th the center of each individual 's head. Observers were able to right their heads between head tilted blocks i n order to help ensure their comfort. Results Nineteen of the twenty-two observers met the behavioral response time criterion for experiencing the RFI, such that when their heads were upright and they were viewing a tilted frame, they correctly responded faster to incongruent rods versus congruent rods. For each of these observers, no more than 10% of trials were rejected due to artifacts i n each of the eight conditions resulting from all possible combinations of the two head positions (upright and tilted), the two rod positions ( C C W and C W ) , and the two frame positions ( C C W and C W ) . Behavioral data Observers who were subject to the RFI when their heads were upright were also subject to the i l lusion when their heads were tilted (Figure 4.4). Pilot data showed no evidence that the C C W or C W direction of head tilt had dissimilar effects on behavioral or electrophysiological responses, so we collapsed the experimental data over C C W and C W head tilts for each participant. Head tilt slowed correct responses, as confirmed by a 2 x 2 repeated-measures A N O V A , which revealed significant main effects of Frame Tilt (Incongruent, Congruent), F(l,18)=39.00, p<.001, and Head Tilt (Upright, Tilted), F(l,18)=20.17, p<.001, on manual response times for correct trials. 84 620 600 C/3 £ IB 0 CO c o C L <s> CD 580 560 Z. 540 o CD L _ 1— o o 520 500 480 460 Incongruent Tr ia ls CZ] Congruen t Tr ia ls H e a d H e a d Upright Ti l ted Figure 4.4. M e a n correct response times to incongruent and congruent trials for participants who were subject RFI i n the head upright and head tilted conditions of Experiment 2. Error bars indicate ±1 S E M . Electrophysiological Data A s in Experiment 1, for observers i n Experiment 2 who were subject to the RFI, the amplitude of the P3 elicited by the rod stimulus over central parietal sites C z and P z was larger for incongruent rod and frame trials compared to congruent rod and frame trials when the head was upright. However, this effect of frame position on P3 amplitude was attenuated, such that incongruent amplitudes were decreased when observers' heads were tilted 15° C W or C C W from gravitational upright. In addition, head tilt delayed the onset and abbreviated the duration of the P3 by approximately 150 ms i n the head tilted versus head upright condition (Figure 4.5). 8 5 Figure 4.5. Grand-average E R P waveforms i n the (a), head upright, and (b). head ti l ted conditions i n Experiment 2 at m id l ine parietal electrode sites for incongruent trials (dashed line) and congruent trials (solid line). So l id rectangles outline the P3 modula t ion found i n head upright trials and dashed rectangles outl ine the attenuated and delayed modulat ion for head t i l ted trials. 86 These results were supported by an omnibus A N O V A to concurrently examine the magnitude and time-course of the resultant P3s. We compared the effects of Frame Til t (Incongruent, Congruent), Time W i n d o w (250 - 300 ms, 300 - 350 ms, 350 - 400 ms, and 400 - 450 ms), and Head Tilt (Upright, Tilted) on the mean amplitudes at electrode sites C z and P z . This analysis revealed significant main effects of each factor (Frame Tilt: F(l,18)=11.90, p<.005; Time Window: F(3,54)=39.02, p<.001; Head Tilt: F(l,18)=3.97, p<.05), interactions between Time W i n d o w and Frame Tilt (F(3,54)=3.91, p<.05), and Time W i n d o w and Head Tilt (F(3,54)=3.06, p>.05), and a marginally significant three-way interaction (F(3,54)=2.56, p=.0641). Planned comparisons revealed a significant effect of Frame Tilt at each of the four consecutive time windows i n the head upright condition (all F's>4.77, all p's<.494), but only the 400 - 450 ms time w i n d o w showed a significant effect of Frame Til t on the amplitude of the P3 when observers' heads were tilted (F(l,18)=8.85, p<.005). Two additional repeated-measures A N O V A s confirmed that there were no significant effects of Frame Tilt or Head Tilt on the amplitudes of the P I (100 ms - 120 ms), or the N l (180 ms - 200 ms) over lateral occipital electrode sites, T y T^ O v O^ and 0 2 (all F's <1, all p's >.412). A s i n Experiment 1, no modulations of any components were observed for the three participants i n Experiment 2 who were not subject to the RFI. Discussion Our results show that observers are less able to successfully determine the orientation of the rod when their heads are tilted to disrupt access to vestibular input compared to when their heads are upright. A s i n Experiment 1, observers in both conditions of Experiment 2 perceived the tilt of the rod as biased i n the opposite direction of the tilted frame. However, as evidenced by the increased time needed for correct responses and the attenuated amplitude and latency of the P3 for incongruent versus 87 congruent trials i n the head tilted condition, head tilt made the orientation of the rod harder to determine. These results converge wi th our earlier behavioral findings (Corbett & Enns, 2006) that l imit ing access to vestibular inputs via head tilt impairs the ability to determine upright when the visual frame of reference is misaligned wi th gravitational vertical. Therefore, Experiment 2 provided further support for the proposal that the RFI is manifest during post-perceptual processing, and that the lack of P I or N I effects are not simply due to an insensitivity to perceptual influences i n our paradigm. General Discussion Based on dissociations between the effects of the RFI on perceptions and visually-guided actions, Dyde and Milner (2002) proposed that a tilted frame biases the perceived orientation of a rod it surrounds during post-perceptual stages of processing. Here we conducted two experiments that provide direct electrophysiological support for this proposal. In Experiment 1, we found that the amplitude of the P3 component elicited by the rod was larger when the rod and frame were incongruently versus congruently tilted, providing electrophysiological evidence that the RFI is manifest dur ing later, post-perceptual stages of processing. However, the possibility remained that the lack of PI or N I results obtained i n Experiment 1 may have reflected a lack of sensitivity to these components i n our RFI task. Therefore, i n Experiment 2, we repeated the procedure wi th observers' heads tilted, a manipulation known to decrease the ability to perceive upright when measured behaviorally (Corbett & Enns, 2006). Results were i n accordance wi th behavioral findings and showed that head tilt significantly attenuated the amplitude and latency of the P3 i n response to incongruent versus congruent rods, further suggesting that vestibular context for upright is integrated wi th visual context dur ing later, post-perceptual stages of processing. The effect of head tilt on the amplitude and latency of 88 the P3 strengthened our claim that the RFI is indexed by the P3 and helped to counter the argument that our RFI task was simply not sensitive to P I or N l components. Therefore, P3 effects, but no earlier perceptual P I or N l effects across two experiments provide further support for Dyde and Milner 's (2002) proposal that the RFI occurs during post-perceptual stages of processing. Whi le the observed modulation of the P3 component is consistent wi th a post-perceptual locus of the RFI, the question remains as to what this modulat ion reveals about the sensory interactions underlying this contextual effect on perceived upright. The P3 is generally believed to index differences i n task difficulty. For example, Palmer, Nasman, and Wilson (1994) reported that the amplitude of the P3 i n response to a same-different letter pair classification task using physically identical stimuli decreased as a function of increasing task difficulty manipulated v ia instructions to participants to compare letters based on physical identity (low difficulty), name identity (medium difficulty), or category identity (vowels/consonants) (high difficulty). In addition, difficult discriminations tend to produce later P3s (Hi l lyard & Kutas, 1983; Verleger, 1997). A l o n g these lines, difficult discriminations are generally made wi th a higher degree of uncertainty or "equivocation," another factor that has been found to modulate the amplitude of the P3 such that smaller P3s are associated wi th more uncertain or equivocal decisions (Johnson, 1986). These studies suggest that our f inding of an increased P3 amplitude for rods i n incongruent frames resulted because the rod appeared more tilted and was "easier" to discriminate, as opposed to congruent trials when the frame caused the rod to appear less tilted. Our behavioral results also support this proposal, as observers correctly determined the orientation of the rod faster i n "easier," incongruent trials compared to congruent trials. When the head was tilted to disrupt vestibular inputs to upright, rod tilt discriminations were more difficult for both 89 incongruent and congruent trials, s lowing , correct responses and attenuating the amplitude and latency of the P3 for incongruent rods found for observers wi th upright heads. Therefore, the modulation of the P3 observed i n the present results l ikely reflects differences i n task difficulty such that "harder" rod tilt discriminations produce smaller P3s. In addition to indexing task difficulty, modulations of the amplitude of the P3 have also been associated wi th bi-stable percepts elicited by ambiguous figures such as the Necker Cube. Several studies have found an increase i n this later positivity corresponding to perceptual reversals of bi-stable figures (Kornmeier & Bach, 2003; Basar-Eroglu, et. al., 1993; Isoglu-Alkag, ef al, 1998). In light of these findings, the increased P3 found for incongruent trials i n the present study may reflect a reversal i n perceived upright that is easier to accomplish when the visible surround accentuates the tilt of the rod compared to a context tilted congruently wi th the rod, making it harder to discriminate which direction that the rod is tilted. Whi le our data support a post-perceptual locus i n visual-vestibular interactions, visual processing is known to interact wi th other sensory modalities, most notably auditory inputs, at a perceptual level. Most recently, Mishra , Martinez, Sejnowski, and Hi l lya rd (2007) found electrophysiological evidence of a perceptual interaction between visual and auditory inputs. using a paradigm i n which a single flash of light was presented between two brief auditory tones, causing most observers to report seeing a second il lusory flash of light. They reported an early modulat ion of bilateral cortical activity over auditory and visual electrode sites 30 - 60 ms after the second sound, which was larger i n amplitude for observers who saw the il lusory flash more frequently. These results are similar to previous findings of early perceptual interactions between visual and auditory inputs i n electrophysiological investigations (Talsma & Waldorff, 2005; 90 Calvert, Spence, & Stein, 2004; Teder-Salejarvi, et. al., 2002; Molho lm , et. al., 2002; McDona ld , Teder-Salejarvi, & Hi l lya rd , 2000; Giard & Peronnet, 1999). Compared to our present findings of a post-perceptual locus for multisensory interactions, these studies suggest that multisensory integration can occur dur ing different stages of processing depending on the sensory systems involved, such that visual and auditory senses mainly interact during perceptual processing, whereas visual and vestibular inputs involved the RFI seem to be integrated during post-perceptual stages of processing. The present work suggests that when confronted wi th a tilted visual frame of reference, observers can accurately determine upright by referencing intact vestibular context for upright, as indexed by a modulation of the post-perceptual P3 component when the head is upright and an attenuation and delay of the modulat ion when the head is tilted. While our conclusions speak to the integration of visual and vestibular inputs to perceive upright, we d id not directly manipulate vestibular inputs. We note that head tilt also alters proprioceptive inputs from the neck muscles and that some may even argue that vestibular input gives a true measure of upright when the head is tilted, as the fluid i n the vestibular organs is constantly acted upon by gravity. Whereas countless studies have employed electrical stimulation of the vestibular organs to examine the effects of vestibular disruption on perceived upright, the electrophysiological effects reflected by changes i n cortical activity i n response to vestibular manipulations such as head tilt are not yet well-understood. Importantly, head tilt and direct manipulations of vestibular input, such as Galvanic Vestibular Stimulation (GVS), and vestibular disorders lead to the same perceptual consequence of decreased ability to correctly determine gravitational vertical (Young, Oman, & Dichgans, 1975; Merker & Held , 1981; Guerraz, Poquin, & Ohlmann, 1998; Bischof, 1974; Cauqui l & Day, 1998; Hafstrom et. al., 2004; Yardley, 1990; Kennedy et. al., 2003). In addition, while vestibular fluid returns to a resting position 91 aligned wi th gravitational vertical shortly after the head is tilted, different populations of hair cells l ining the vestibular organs are stimulated when the head is upright versus when it is tilted, further justifying head tilt as a method to introduce vestibular conflict wi th gravitational vertical. In summary, our findings provide strong support for Dyde & Milner ' s (2002) proposal that visual tilt illusions such as the RFI affect post-perceptual stages of processing. In addition, the present work is further evidence for a sensory hierarchy for the perception of upright, where visual input automatically has top priority, followed by a later influence of vestibular input about the position of the head wi th respect to gravity (Corbett & Enns, 2006). Establishing our perceptions of upright based on visual references i n the external environment renders us less sensitive to the constantly fluctuating positions of our bodies, and allows us to maintain a stable percept of an upright wor ld . However, when visual input is unreliable, we can reference intact bodi ly position cues such as vestibular input about the position of the head i n order accurately perceive upright. Therefore, muitisensory integration for the perception of upright is likely part of an efficient process, which allows us to perceive the environment as stable and upright despite our bodies' constantly changing positions wi th in that environment. To conclude, our results suggest that visitors to tilted mystery cabins w i l l experience the most striking illusions as soon as they enter the cabin, and may do we l l to hold onto the available handrails, at least for the first few hundred milliseconds. 92 References Asch, S. E., & Witkin , H . A . (1948). Studies i n space orientation: I. Perception of the upright wi th displaced visual fields. Journal of Experimental Psychology, 38, p. 325-337. Basar-Eroglu, C , Striiber, D. , Stadler, M . , & Kruse, E. (1993). Multistable visual perception induces a slow positive E E G wave. International Journal of Neuroscience, 73, p. 139-151. Bischof, N . (1974). Optic vestibular orientation to the vertical. In J. J. Kornhuber (Ed.), Handbook of Sensory Physiology, V o l . 6, Pt. 2, pp. 155-190. N e w York: Springer-Verlag. Bridgeman, B., Peery, S., & Anand, S. (1997). Interaction of cognitive and sensorimotor maps of visual space. Perception & Psychophysics, 59(3), p. 456-469. Calvert, G . A . , Spence, C , & Stein, B. E. (Eds.) (2004). The Handbook of Multisensory Processes. Cambridge, M A : MIT Press. Cauqui l , A . S. & Day, B. L . (1998). Galvanic vestibular stimulation modulates voluntary movement of the human upper body. Journal of Physiology, 513(2), p. 611-619. Coles, M . G . H . , & Rugg, M . D . (1995). Event-related potentials: A n introduction. In M . D . Rugg & M . G. H . Coles (Eds.) Electrophysiology of Mind: Event- Related Potentials and Cognition. N e w York: Oxford. Corbett, J. E., & Enns, J. T. (2006). Observer pitch and ro l l influence the rod and frame il lusion. Psychonomic Bulletin & Review, 13(1), p. 160-165. DiLorenzo, J.R., & Rock I. (1982). The rod-and-frame effect as a function of the righting of the frame. Journal of Experimental Psychology: Human Perception & Performance, 8(4), p. 536-546. Donchin, E., & Coles, M . G . H . (1988). Is the P300 component a manifestation of context updating? The Behavioral and Brain Sciences, 11, p. 355-372. Dyde, R. T., & Milner , D . A . (2002). Two illusions of perceived orientation: One fools all of the people some of the time; the other fools all of the people al l of the time. Experimental Brain Research, 144(4), p. 518-527. Ebenholtz, S. M . (1977). Determinants of the rod and frame effect: The role of retinal size. Perception and Psychophysics, 22(6), p. 531-538. Ebenholtz, S. M . , & Callan, J. W . (1980). Modula t ion of the rod and frame effect: Retinal angle vs. apparent size. Psychological Research, 42(4), p. 327-334. Giard , M . H . , & Peronnet, F. (1999). Audi tory-visual integration during mult imodal object recognition i n humans: A behavioral and electrophysiological study. Journal of Cognitive Neuroscience, 11(5), p. 473 - 490. 93 Guerraz, M . , Poquin, D. , & Ohlmann, T. (1998). The role of head-centric spatial reference wi th a static and kinetic visual disturbance. Perception and Psychophysics, 60(2), p. 287-295. Hafstrom, A . C. A . , Fransson, P. A . , Karlberg, M . , & Magnusson, M . (2004). Ipsilesional visual field dependency for patients w i th vestibular schwannoma. Neuroreport, 15(14), p. 2201-2204. Handy, T. C. (2005). Event-related potentials: A methods handbook. Cambridge: MIT Press. Hi l lya rd , S. A . , & Kutas, M . (1983). Electrophysiology of cognitive processing. Annual Review of Psychology, 34, p. 33-61. Isoglu-Alkac, U . , Basar-Eroglu, C , Ademoglu, A . , Demiralp, T., Miener, M . , & Stadler, M . (1998). Analysis of the electroencephalographic activity dur ing Necker cube reversals by means of the wavelet transform. Biological Cybernetics, 79, p. 437-442. Johnson, R. Jr. (1986). A triarchic model of P300 amplitude. Psychophysiology, 23, p. 367-384. Kennedy, P. M . , Carlsen, A . N . , Inglis, J. T., Chow, R., Franks, I. M . , & Chua, R. (2003). Relative contributions of visual and vestibular information on the trajectory of human gait. Experimental Brain Research, 153, p. 113-117. Kornmeier, J., & Bach, M . (2003). Early neural activity i n Necker-cube reversal: Evidence for low-level processing of a gestalt phenomenon. Psychophysiology, 41(1), p. 1-8. L i , W., & Mat in , L . (2005). Two wrongs make a right: Linear increase of accuracy of visually-guided manual pointing, reaching, and height-matching w i th increase i n hand-to-body distance. Vision Research, 45, p. 533-550. Luck, S. J., & Hi l lya rd , S. A . (1995). The role of attention i n feature detection and conjunction discrimination: A n electrophysiological analysis. International Journal of Neuroscience, 80(1-4), p. 281-297. McDona ld , J. J., Teder-Salejarvi; W . A . , & Hi l lya rd , S. A . (2000). Involuntary orienting to sound improves visual perception. Nature, 407, p. 906-908. Merker, B. H . , & Held , R. (1981). Eye torsion and the apparent horizon under head tilt and visual field rotation. Vision Research, 21, p. 543-547. Milner , A . D. , & Goodale, M . A . (1995). The Visual Brain in Action. London: Oxford University Press. 94 Mishra , J., Martinez, A . , Sejnowski, T.J., & Hi l lyard , S. A . (2007). Early cross-modal interactions i n auditory and visual cortex underlie a sound-induced visual il lusion. Journal of Neuroscience, 27(15), p. 4120 - 4131. Molho lm, S., Ritter, W., Murray, M . M . , Javitt, D . C , Schroeder, C. E., & Foxe, J. J. (2002). Muitisensory auditory-visual interactions during early sensory processing in humans: A high-density electrical mapping study. Cognitive Brain Research, 14(1), p. 115 -128. Palmer, B., Nasman, V . T., & Wilson, G . F. (1994). Task difficulty: Effects on ERPs i n a same-different letter classification task. Biological Psychology, 38(2-3), p. 199-214. Spinelli , D. , Antonucci, G. , Daini , R., Fanzon, D . , & Zoccolotti, P. (1995). Modula t ion of the rod-and-frame i l lusion by additional external stimuli. Perception, 24(10), p. 1105-1118. Teder-Salejarvi, W . A . , McDona ld , J. J., D i Russo, F., & Hi l l ya rd , S. A . (2002). A n analysis of audio-visual crossmodal integration by means of event-related potential (ERP) recordings. Cognitive Brain Research 14, p. 106-114. Verleger, R. (1997). O n the utility of the P3 latency as an index of mental chronometry. Psychophysiology, 34(2), p. 131-156. Wi tk in , H . A . (1949). Perception of body position and the position of the visual field. Psychological Monographs, 63, p. 1-63. Yardley, L . (1990). Contribution of somatosensory information to perception of the visual vertical wi th body tilt and rotating visual field. Perception & Psychophysics, 48, p. 131-134. Young, L . R., Oman, C. M . , & Dichgans, J. M . (1975). Influence of head orientation on visually induced pitch and rol l sensation. Aviation, Space, and Environmental Medicine, 46, p. 264-269. 95 C H A P T E R 5: G E N E R A L D I S C U S S I O N This dissertation examined several aspects of a cognitive mechanism responsible for integrating multisensory inputs to arrive at a veridical perception of upright under a variety of circumstances. Three sets of experiments determined the relative weighting of visual, vestibular, and proprioceptive cues when participants judged the orientation of a visual stimulus presented i n a tilted context, and explored how these sensory inputs become integrated over the time-course of processing to form our perceptions of gravitational upright. The following is a summary of how the main findings addressed each of the aims outlined i n the introduction of this dissertation: A i m 1: Visual , vestibular, and proprioceptive inputs are integrated i n a hierarchical fashion, such that perceived upright is influenced by vestibular input when visual cues to upright are unreliable, and by proprioceptive signals when visual and vestibular inputs are unreliable (Chapter 2). A i m 2: a). Local orientation context i n close proximity to a target influences the target's perceived orientation when flashed briefly beforehand, b). both local and global orientation contexts affect the perceived orientation of a target when presented simultaneously w i th the target, but c). global context must be visible for at least 800 ms to cause an i l lusion when presented asynchronously wi th a target. These findings are predicted by Dyde and Milner ' s (2002) two-stage theory of orientation perception, wi th local spatial interactions during early stages and longer-range interactions during later stages (Chapter 3). A i m 3: Cortical activity associated wi th the perceived orientation of a rod, as measured by event-related potentials (ERPs), varied as a function of the orientation of a surrounding frame i n later, post-perceptual stages of processing. Head tilt, which impairs 96 orientation perception for' behavioral measures, delayed the onset and attenuated the amplitude of the frame's effect on the P3, supporting the proposal of a post-perceptual locus for multisensory integration correcting for the visual i l lus ion induced by the tilted global frame. (Chapter 4). . Remaining questions Overall , this dissertation concludes wi th the proposal that we perceive upright wi th respect to a hierarchy of sensory inputs, integrated at multiple points over the course of processing. Out l ining the relative weighting of these inputs, and pinpointing key times that they are integrated during the course of processing has raised several outstanding questions about how we maintain a veridical perception of upright addressed i n the fol lowing five sub-sections: Separate vestibular and proprioceptive contributions to upright perception? Can the effects of proprioceptive and vestibular manipulations be separated i n ERP and RT measures to further examine the independent contributions of each sensory input? First, the present studies conclude that vestibular inputs play a secondary role i n forming and maintaining perceptions of upright, and that proprioceptive inputs about the felt slant of the supporting surface only affect perceptual judgments of upright when visual and vestibular inputs are unreliable. When observers judged the orientation of a rod, the orientation of the surrounding frame affected cortical activity i n post-perceptual stages of processing, as indexed by a modulation of the P3 component. Vestibular inputs about the position of the head wi th respect to gravity helped to correct this i l lusion, as the P3 effect was reduced when the head was tilted (Chapter 4). Importantly, behavioral experiments i n Chapter 2 demonstrated that head tilt decreased the ability to accurately 97 determine gravitational upright when viewing a tilted frame, confirming that the effects of head tilt on the amplitude of the P3 component observed i n Chapter 4 reflect a decreased ability to discriminate the orientation of the rod. Given additional behavioral findings i n Chapter 2 that disrupting access to both vestibular and proprioceptive signals increased the magnitude of the RFI, it remains to be determined whether such manipulations also cause a corresponding attenuation of the P3 effect. In contrast, tilting observers' bodies but al lowing them right their heads wi th respect to gravity may reinstate the P3 effect found wi th upright heads, consistent w i th findings i n Chapter 2 that disrupting only trunk-based proprioceptive signals does not. affect the degree to which perceived upright is biased by a tilted visual frame of reference. Separate time courses of vestibular and proprioceptive contributions? Are proprioceptive influences integrated during perceptual or post-perceptual stages of processing? Both head and whole body tilt increase the degree of error in rod adjustments (Chapter 2), and head tilt increases RT for correct rod orientation discriminations and delays the onset of the associated P3 modulat ion (Chapter 4). Therefore, tilting the whole body to disrupt vestibular and proprioceptive inputs may also increase RT i n correct rod orientation discriminations and delay the onset of the post-perceptual P3 effect. Alternatively, proprioceptive inputs may affect earlier perceptual stages of processing, as we must constantly make tiny postural adjustments i n order to compensate for our ever-changing positions wi th in the environment. If this is the case, then tilting observers' whole bodies should cause the amplitude of earlier, sensory/perceptual E R P components such as the P I or N l , to become affected by proprioceptive inputs about the slant of the surfaces that are supporting observers. This outcome w o u l d also be consistent wi th the proposed hierarchy of muitisensory 98 integration for upright perception, wi th earlier and perhaps even more numerous references to proprioceptive inputs that are not as influential as visual inputs to perceived upright, but nevertheless aid i n providing feedback to maintain balance. Finally, tilting the body but not the head d id not increase the degree of error i n the ability to determine upright i n Chapter 2, but measuring the RT and associated cortical activity for correct rod orientations wi th only the body tilted may uncover effects of proprioceptive inputs not detected by the rod adjustment task. Challenges i n separating vestibular from proprioceptive contributions Whi le the present work was able to utilize the temporal precision afforded by ERPs to pinpoint the post-perceptual locus of visual and vestibular integration i n the afferent stream of processing, an investigation using ERPs to examine the temporal effects of disrupting both proprioceptive and vestibular inputs must overcome additional methodological obstacles. Til t ing the entire body w i l l require several untested modifications to E R P methods. First, using ERPs i n the present work to examine observers seated wi th tilted heads d id not require any special instrumental or methodological considerations beyond the experimenter monitoring that observers maintained the correct tilted head posture. However, a study where ERPs are recorded from tilted observers w i l l require that these observers maintain this tilted posture for the long duration of a repeated-measures study. Such manipulations are usually accomplished by having participants lie sideways wi th their whole bodies resting on a table or other horizontal surface. However, the question arises of how to institute such postural manipulations without al lowing subjects to rest their heads against the supporting surface. This is important, because doing so w o u l d cause one side of the cap containing the net of sensory electrodes recording cortical activity to rest on the surface, 99 disrupting the recording of underlying brain activity. If participants were made to lie horizontally, but not allowed to contact their heads wi th a supporting surface, this considerable postural strain on the neck could introduce muscle-related noise into the E E G signal, causing recordings to become too erratic to al low for a theoretical interpretation of the underlying neural activity. Therefore, a less temporally precise behavioral approach, such as the correct RT RFI task without corresponding ERP measures, may have to be used to help further determine the effects of disrupting proprioceptive and vestibular inputs on perceived upright. The role of ind iv idual differences? A large number of studies published during the 1970s examined the relationships between indiv idual differences i n personality traits and cognitive strategies associated wi th the degree to which an indiv idual references the context of the surrounding environment. In particular, W i t k i n (1969) distinguished between "field dependent" observers who tend to use external cues, or field references, for perception of the upright, and "field independent" observers who tend to rely less on external field cues, and instead to rely to an increased extent on internal bodily cues to perceive upright. Wi tk in (1977) further proposed that field dependence and independence are associated wi th two styles of learning and personality, wi th field dependent observers focused on broad concepts and social feedback and field independent observers focused on specific details and less influenced by social feedback. RFI measures of field dependence have also been reliably associated wi th differences i n culture/race and sex. In particular, Easterners/Asians and females tend to be more field dependent than Westerners/Caucasians or males (Adevai, Silverman, & McGough , 1970; Ji, Peng, & Nisbett, 2000; M c G i l l i g a n & Barclay, 1974; Witk in , 1977). Whi le not a focus of the present 100 investigation, it is worth noting that consistent differences between cultures /races and sexes were observed while conducting the experiments contained i n this dissertation. In particular, Chinese and female observers tended to be more field dependent than Canadian or male observers i n both the traditional RFI adjustment task i n Chapter 2, and the RT and accuracy RFI tasks i n Chapters 3 and 4. Future systematic investigations along these lines can help to better predict how a given indiv idual w i l l perceive upright during different multisensory conditions What is the role of attention i n the perception of upright? It has long been known that field dependent and independent observers have different patterns of gaze i n space, such that field independent observers are better able to maintain fixation on the rod i n RFI displays than field dependent observers (Blowers & O'Connor, 1978). However, Blowers and O'Connor 's (1978) proposal that field independent subjects are better able to selectively attend to the rod and ignore the orientation of the frame than field dependent observers has not been tested. There is also evidence that observers can be cued by haptic stimulation to attend to a bodi ly context for upright. For example, proprioceptive feedback can affect the perceived orientation of a visual texture gradient such that observers w i l l perceive the orientation of the gradient i n the direction of gravity signaled by a fingertip touch (Jeka & Lackner, 1994). In addition, tilting observers increases the RFI or the effects of a rotating room, (Asch & Witk in , 1948a&b, Chapter 2; Al l i son , Howard , & Zacher, 1999), but haptic feedback given by pressure applied to the bottom of the feet restores these illusions to a baseline magnitude measured when observers are upright (Goodendough, Oltman, Sigman, & Cox, 1981). Patients wi th bilateral vestibular loss report feeling "cloudy-headed," or mentally disoriented, due to the extra effort needed to attend to v isual input i n order to 101 maintain balance, reducing the amount of attention available for other cognitive tasks (Yardley et. a l , 2001; Anderson et. al., 2003; Pellecchia, 2003; Redfern et. al., 2003). Together, these studies suggest that attention plays a role i n whether perceptions of upright are based on visual or bodi ly inputs either by selecting/ignoring aspects of the scene, or by being cued to a given sensory modality. A new task for measuring the time to correctly discriminate orientation? Previous investigations of visual context illusions have employed the technique of estimating the Point of Subjective Vertical (PSV) used i n Chapter 2 (e.g. Asch & Wi tk in 1948 a&b; Wenderoth & Johnstone, 1988; Wenderoth, van der Zwan , & Johnstone, 1989). However, the present work also developed a modified version of the RFI task to measure RT and accuracy for two percepts of the same rod's orientation as a function of the orientation of the surrounding frame. Importantly, this task allows for a measure of the associated time course not given by traditional PSV tasks, wh ich only estimate the magnitude of error i n rod adjustments caused by a tilted surrounding frame. The new modified RT task also allows for a measure of the time course of cortical activity associated wi th the two percepts of the rod induced by the tilt of the surrounding frame, -a measure that cannot be localized using the traditional PSV task. Similar tasks have been developed i n which observers determine whether the top ends of two rods i n side-by-side RFI displays point inward or outward, as a function of the inward or outward tilt of both surrounding frames (e. g. Calvert & Harris, 1988). However, these tasks do not allow for a measure of cortical activity associated wi th a single rod stimulus required for ERP experiments. A s measures of correct RT may uncover differences i n perceived upright not able to be detected i n the PSV task, and the magnitude of error estimated by 102 the PSV task may not be reflected i n the RT task, future investigations may benefit from using both tasks i n conjunction. Practical implications In addition to these theoretical implications, the present work also has several practical implications for human safety, health, and quality of life addressed i n the remaining three sub-sections. H o w to improve the reliability of multisensory signals Studies of multisensory integration for the perception of upright have particularly useful implications for flight and navigation, microgravity space missions, and patients wi th sensory impairments. The findings i n Chapter 2 that head tilt increases reliance on visual input when determining upright, but the slant of the supporting surface affects perceived upright when the whole body is tilted are important considerations for designers of navigation and flight landing software. In attempt to prevent further fatal airline crashes such as the one described i n Chapter 1, the present research suggests that several measures can be taken to increase the reliability of multisensory inputs to gravity. For instance, an observer who must override conflicting bodi ly inputs about the direction of gravity, such as a pilot descending to approach the runway below, may benefit from tilting the head i n line wi th the approaching terrain. A l ign ing the visual and vestibular senses i n this manner may prove to increase dependence on visual inputs to upright and the corresponding ability to override conflicting bodi ly signals about the felt pu l l of gravity, or lack thereof as i n microgravity missions. In addition, methods to improve the reliability of multisensory inputs for perceiving upright, such as G V S postural control systems currently under development 103 i n Japan (Maeda, et. al, 2005), should take measures to account for cultural and sex differences i n field dependence when marketing to different cultures and genders. Finally, attentional resources may be able to be allocated to improve the reliability of multisensory inputs using peripheral cues i n postural control systems and flight landing and navigation instrument panels, such as flashes of light i n the location where a tilted frame of reference is about to appear. There is mounting evidence that covert transient attention (rapid, stimulus-driven attention to events i n the visual periphery without eye movements from fixation) improves performance, speeds processing, and enhances the sensory representation at the attended location (Carrasco, Penpeci-Talgar & Eckstein, 2000; Carrasco & McElree, 2001; Carrasco, L ing , & Read, 2004). However, this type of attention has also been demonstrated to impair performance for tasks i n which an enhanced representation is not advantageous. For example, attention impairs texture segmentation at foveal locations where acuity becomes too fine to discriminate global orientations of offset texture patches, causing the observer to attend to ind iv idual elements /textons (Yeshurun & Carrasco, 1998). Given these costs and benefits of attention, I am currently investigating whether cueing the location where a rod w i l l appear inside a tilted frame improves or impairs performance when determining upright as judged by errors i n rod settings and the rate at which performance increases from chance to asymptotic levels. Such investigations w i l l help to determine whether attending to a tilted context improves or impairs an observers' ability to accurately perceive upright. 104 Extending these findings to patient populations A s wi th al l upright perception studies using neurologically normal adults, our results can be used to predict how observers' representations of upright are formed when the relevant physiological mechanisms supporting the visual and bodi ly contributions to upright are merely disrupted for some time, but this sensory machinery remains fully intact and otherwise functional. Therefore, it is possible that being permanently deprived of a sensory input to upright causes observers to re-weight remaining sensory contributions to upright differently than they w o u l d if their access to such sensory inputs was merely disrupted for a brief period of time. For example, tilting the head only disrupts access to vestibular signals about gravity, but patients wi th bilateral vestibular lesions have a complete loss of vestibular functioning. However, findings from bl ind and vestibular lesion patient populations suggest that our results do apply to circumstances where a given sensory modality is completely non-functional. For example, b l ind patients must rely on intact vestibular and proprioceptive inputs to functionally compensate for a lack of visual input i n a similar manner that participants i n the present work rely on vestibular and proprioceptive inputs when presented w i th a tilted frame. In addition, the present finding that tilting observers' heads increased the RFI mirror findings by Guerraz and colleagues (2001) who have shown that patients wi th bilateral vestibular lesions exhibit an increased dependence on visual input to perceive upright. When asked to adjust the tilt of a rod to vertical wi th respect to gravity, these patients w i l l adjust the rod to match the orientation of a surrounding tilted frame to an even greater extent than neurologically normal individuals whose heads are tilted. Just as normal observers, when these patients are blindfolded, the degree of sway i n their postures normally associated wi th a lack of visual input becomes increased, but precision contact of the fingertips w i l l cause them to perceive upright i n the direction of 105 proprioceptive feedback (Jeka & Lackner, 1994; Lackner, et. al. 1999; Jeka, et. al, in press). These findings suggest that patients wi th permanent vestibular damage perceive upright as biased by proprioceptive inputs regarding the slant of supporting surfaces when visual inputs are unreliable i n the same manner as tilted observers i n the present study. Therefore, while the present manipulations do not completely remove visual, vestibular, or proprioceptive contributions to upright, results from the literature concerning patients wi th permanent deficits of these sensory inputs support extending the current results to help predict how observers wi th partial or complete sensory impairments w i l l misperceive the direction of upright under different circumstances. This could lead to techniques to improve the quality of life for patients with bilateral vestibular lesions. For instance, these patients experience a recurring sense of dizziness and disorientation similar to motion sickness. Gureraz and colleagues (2001) proposed that patients experience vertigo and nausea because visual context that is not supported by feedback from the vestibular balance organs similar to the sensory conflict that results when neurologically normal observers become i l l when v iewing a stable visual reference such as the page of a book while i n transit. The present findings suggest that these patients, as we l l as sick commuters, wou ld benefit by looking outside at a passing reference wi th in the moving visual scene to corroborate vestibular signals that the body is i n motion. Ways to reduce falls i n the elderly Findings of a sensory hierarchy for the perception of upright suggest that elderly patients who exhibit an increased loss of control over bodi ly inputs discussed i n Chapter 1 may benefit from environments wi th strong cues to visual upright, such as rooms and hallways wi th sharply contrasting walls and ceilings, designed reflect the felt slant of the 106 supporting floor. For example adding luminous tape (not visible i n normal or daylight conditions) to the contours of hallways and rooms may aid these patients i n maintaining an upright posture when they must navigate about their surroundings wi th attenuated or absent visual input, such as when walk ing to the bathroom i n the middle of the night. Sharpened visual references to upright may aid them i n overcoming decreasingly accurate bodi ly inputs, thus stabilizing posture and reducing the risk of fatal falls. C los ing remarks This dissertation provides converging behavioral and electrophysiological evidence that a hierarchy of visual, vestibular, and proprioceptive sensory signals governs our perceptions of upright. The dominant visual signals i n this hierarchy reflect the most invariant objects i n the visual environment that we interpret as references to the direction of gravity. Large horizontal surfaces represent the ground or the horizon, and large vertical surfaces belong to the defining structures perpendicular to the pu l l of gravity, such as buildings or mountains. When we experience sudden changes i n the orientation of supporting surfaces, these global environmental signals help to maintain a veridical perception of upright, as wel l as a vertical posture based on this veridical perception. However, when these global references are tilted from gravitational vertical, we must rely on vestibular signals about the pu l l of gravity, and finally proprioceptive influences when both visual and vestibular signals are disrupted i n attempt to maintain an upright posture. This hierarchical integration of muitisensory signals represents a robust system for determining "upright" i n the environment in which we have evolved, such that under normal conditions visual, vestibular, and proprioceptive systems accurately inform us 107 about the true direction of gravitational upright. However, when these sensory signals conflict, we default to visual inputs inherent i n the most global and invariant references i n the external environment, instead of less stable vestibular and proprioceptive inputs. In conclusion, we know relatively little about the important problem of how these sensory signals are weighted and integrated under different circumstances. In that sense, this thesis is only a beginning. However, the present findings do offer a final piece of practical advice to the thousands of tourists who visit tilted mystery cabins each year and pay to experience the misperceptions of gravity that arise when sensory inputs to upright conflict. Given the present findings that accurate orientation perception is most impaired when the head or the whole body is tilted i n the presence of a titled visual field, tourists wi th strong stomachs are advised to tilt their heads when inside these tilted cabins to disrupt both vestibular and proprioceptive cues to gravity, causing the most impressive illusions. However, they should be cautioned to hold onto the handrails attached to the walls i n case they become too overwhelmed by the strong illusions they can now experience. 108 References Adeva i , G. , Silverman, A . J., & McGough , W . E. (1970). Ethnic differences i n perceptual testing. International Journal of Social Psychiatry, 16, p. 237-239. Al l i son , R., Howard , I. P., & Zacher, J. E. (1999). Effect of field size, head motion, and rotational velocity on rol l vection and illusory self-tilt i n a tumbling room. Perception, 28, p. 299-306. Anderson, G. , Hagman, J., Taliarizadeh, R., Svedberg, A . , & Larsen, H . C. (2003). Dual-task study of cognitive and postural interference i n patients wi th vestibular disorders. Otology & Neurotology, 24, p. 289-293. Asch, S. E., & Witk in , H . A . (1948a). Studies in space orientation: I. Perception of the upright wi th displaced visual fields. Journal of Experimental Psychology, 38, p. 325-337. Asch, S. E. , & Witk in , H . A . (1948b). Studies i n space orientation: II. Perception of the upright wi th displaced visual fields and wi th body tilted. Journal of Experimental Psychology, 38, p. 455-477. Blowers, G . H . & O'Connor, K . P. (1978). Relation of eye movements to errors on the Rod-and-Frame Test. Perceptual & Motor Skills, 46, p. 719-725. Bridgeman, B., Peery, S., Anand, S. (1997). Interaction of cognitive and sensorimotor maps of visual space. Perceptual Psychology, 59, p. 456-469. Calvert, J. E., & Harris, J. P. (1988). Spatial frequency and duration effects on the tilt i l lusion and orientation acuity. Vision Research, 28(9), p. 1051-1059. Carrasco, M . , L ing , S. & Read, S. (2004). Attention alters appearance. Nature Neuroscience, 7, p. 308-313. Carrasco, M . & McElree, B. (2001). Covert attention accelerates the rate of visual information processing. Proceedings of the National Academy of Sciences, 98, p. 5363-5367. Carrasco, M . , Penpeci-Talgar, C. & Eckstein, M . (2000). Spatial attention increases contrast sensitivity across the CSF: Support for signal enhancement. Vision Research, 40, p. 1203-1215. Dyde, R. T., & Milner , D . A . (2002). Two illusions of perceived orientation: One fools all of the people some of the time; the other fools al l of the people all of the time. Experimental Brain Research, 144 (4), p. 518-527. Goodendough, D . R., Oltman, P. K. , Sigman, E., & Cox, P. W . (1981). The rod-and-frame i l lusion i n erect and supine observers. Perception & Psychophysics, 29(4), p. 365-370. 109 Guerraz, ML, Yardley, L . , Bertholon, P., Pollak, L . , Rudge, P., Gresty, M . A . , & Bronstein, A . M . (2001). Visual vertigo: Symptom assessment, spatial orientation and postural control. Brain, 124(8), p. 1646-1656. Howard , I. P. (1982). Human visual orientation. N e w York: Wiley. Jeka, J. J., & Lackner, J. R. (1994). Fingertip contact influences human postural control. Experimental Brain Research, 100, p. 495-502. Jeka, J. J., Saffer, M . , Al l i son , L. , Zhang, Y. , Carver, S., & Kiemel , T. (in press). Sensory re-weighting wi th translational visual st imuli i n young and healthy elderly adults: The role of state dependent noise. Experimental Brain Research. Ji, L . , Peng, K . , & Nisbett, R. E. (2000). Culture, control, and perception of relationships i n the. environment. Journal of Personality and Social Psychology, 78, p. 943-955. Lackner, J. R., Diz io , P., & Jeka, J. (1999). Precision contact of the fingertip reduces postural sway of individuals wi th bilateral vestibular loss. Experimental Brain Research, 126, p. 459-466. L i , W., & Mat in , L . (2005). Two wrongs make a right: Linear increase of accuracy of visually-guided manual pointing, reaching, and height-matching wi th increase i n hand-to-body distance. Vision Research, 45, p. 533-550. Maeda, T., Ando , H . , Amemiya , T., Inami, M . , Nagaya, N . , & Sugimoto, M . (2005). Shaking the wor ld : Galvanic Vestibular Stimulation as a novel sensation interface, Association for Computing Machinery's Special Interest Group on Graphics and Interactive Techniques (ACM SIGGRAPH), Los Angeles, C A . McGi l l igan , R.P., & Barclay, A . G . (1974). Sex differences and spatial ability factors i n Witkin's "differentiation" construct. Journal of Clinical Psychology, 30(4), p. 528-532. Pellechia, G. L . (2003). Postural sway increases w i th attentional demands of concurrent cognitive task. Gait and Posture, 18, p. 29-34. Redfern, M . S., Talkowski, M . E., Jennings, J. R., & Furman, J. M . (2004). Cognitive influences i n postural control of patients wi th unilateral vestibular loss. Gait and Posture, 19(2), p. 105-114. Wenderoth, P. & Johnstone, S. (1988). The differential effects of brief exposures and surrounding contours on direct and indirect tilt illusions. Perception, 17, p. 165-176. Wenderoth, P., van der Zwan, R., & Johnstone, S. (1989). Orientation illusions induced by briefly flashed plaids. Perception, 18, p. 715-728. Wi tk in , H . A . (1969). Social influences in the development of cognitive style. N e w York: Rand M c N a l l y . 110 Witk in , H . A . , Moore, C. A . , Goodenough, D . R., & Cox, P. W . (1977). Field-dependent and field-independent cognitive styles and their educational implications. Review of Educational Research, 47(1), p. 1-64. Yardley, L. , Gardner, M . , Bronstein, A . , Davies, R., Buckwel l , D. , & Luxon, L . (2001). Interference between postural control and mental task performance i n patients wi th vestibular disorder and healthy controls. Journal of Neurology, Neurosurgery, and Psychiatry, 71, p. 48-52. Yeshurun, Y . & Carrasco, M . (1998). Attention improves or impairs visual performance by enhancing spatial resolution. Nature, 396(5), p. 72-75. 

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            data-media="{[{embed.selectedMedia}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
https://iiif.library.ubc.ca/presentation/dsp.831.1-0100617/manifest

Comment

Related Items