UBC Faculty Research and Publications

Breaking the Fourth Wall of Cognitive Science : Real World Social Attention and the Dual Function of… Risko, Evan F.; Richardson, Daniel |q (Daniel C.); Kingstone, Alan 2016

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata


52383-Risko_E_et_al_Breaking_fourth_wall.pdf [ 383.05kB ]
JSON: 52383-1.0305737.json
JSON-LD: 52383-1.0305737-ld.json
RDF/XML (Pretty): 52383-1.0305737-rdf.xml
RDF/JSON: 52383-1.0305737-rdf.json
Turtle: 52383-1.0305737-turtle.txt
N-Triples: 52383-1.0305737-rdf-ntriples.txt
Original Record: 52383-1.0305737-source.json
Full Text

Full Text

                                  Corresponding Author  Evan F. Risko Department of Psychology,  University of Waterloo,  200 University Ave. West, Waterloo, ON N2L 3G1  Email: efrisko@uwaterloo.ca   WORD COUNT: 2112 REFERENCES: 34Dual Function of Gaze 1  RUNNING HEAD: THE DUAL FUNCTION OF GAZE              Breaking the Fourth Wall of Cognitive Science: Real World Social Attention and the Dual Function of Gaze    Evan F.  Risko1, University of Waterloo Daniel C. Richardson, University College London Alan Kingstone, University of British Columbia                          Dual Function of Gaze 2  Abstract  Research in cognitive science typically places a boundary between the participant and the stimuli they are asked to process. While this separation affords experimental control, it can also severely limit the generalizability of the conclusions that are drawn. Here we review new evidence that some conclusions that have been drawn about social attention do not extend beyond the laboratory. They fundamentally misrepresent how social attention operates in natural social contexts. Critically, these difficulties have led to renewed interest in the dual function of gaze - when in authentic social situations the eyes both collect information from the environment (an encoding function) and communicate one's mental states to others (a signalling function)– which traditional social attention paradigms arguably have failed to capture. We review this recent work and discuss the utility of adopting more naturalistic methods in cognitive science.  Keywords: Social Attention, Gaze, Ecological Validity                     Dual Function of Gaze 3  Introduction The ‘fourth wall’ in theatre is the illusory barrier that exists between actors and the audience. It allows the audience to believe that the stage is a world apart from theirs, and allows the actors to overlook the fact that others are observing them. In the mid-19th century, dramatists experimented with ‘breaking the fourth wall’, having actors acknowledge and speak to the audience members. The jarring effects are still used today in television shows such as “The Office” and “House of Cards”. Experiments in social attention, just like the theatre, employ a similarly convenient, illusory barrier between participants and the stimuli they are presented. For example, participants are not expected to interact with the faces presented to them, or worry about what the stimuli might think of them. Yet these experiments are routinely thought to serve as effective representations of real life situations where people interact with others who can see, respond to and form opinions of them. In this review we present experimental evidence that when the fourth wall of cognitive science is broken - when the stimuli can look back at the participant - common results can be turned on their head and this can lead to fundamental new insights and directions for research.  INSERT FIGURE 1 HERE  Social Attention Research: People Attend Differently to Real People than Images of People Much of our life is spent thinking about other people -- both during our waking hours and when we dream.  Research on human attention seems to support this strong interest in people.  There are numerous studies indicating that when individuals are presented with photos or videos Dual Function of Gaze 4  containing people, there is a profound tendency to look at people rather than other objects (Fletcher-Watson, Findlay, Leekam, & Benson, 2008; Yarbus, 1967). This focus on people -- and especially their eyes (e.g., see Birmingham & Kingstone, 2009; Frischen, Bayliss, & Tipper, 2007 for recent reviews) appears to include a deep interest in what others might be interested in. The prevailing explanation for these findings is that people's eyes provide an unusually rich source of information, revealing their mental and emotional state as well as exposing where and to what their attention is committed to in their environment (e.g., Baron-Cohen, Wheelwright, & Jolliffe, 1997; Emery, 2000).  Indeed, this tendency to attend to the face and eyes of other people, and to follow where their gaze is directed, appears to be partially automatic and subserved by brain regions that are preferentially tuned for stimuli that look like eyes (Batki, Baron-Cohen, Wheelwright, Connellan, & Ahluwalia, 2000; Friesen & Kingstone, 1998; Frischen et al., 2007; Langton, Watt, & Bruce, 2000; Laidlaw, Risko & Kingstone, 2011; Levy, Foulsham, & Kingstone, 2012).  Against this background of laboratory research, it was remarkable to discover that when an image of a person is replaced with a real person, a profoundly different pattern of looking behaviour emerges. Two studies are particularly instructive on this point.  Laidlaw, Foulsham, Kuhn, and Kingstone (2011) demonstrated that when people were in a room with a stranger sitting in a chair across from them, they were disinclined to look at that person. Indeed, they were more likely to look at the chair if it was empty than if it contained a person.  However, when a video of that stranger sitting in a chair was presented on a computer screen participants were biased to look at the person's image, consistent with previous laboratory studies.  Gallup, Chong, and Couzin (2012a) extended the Laidlaw study, demonstrating that individuals were also disinclined to look at where other real people were looking. Gallup et al. (2012a) placed an Dual Function of Gaze 5  object in a busy hallway and monitored individuals’ gaze behavior with respect to that object. When face-to-face with an individual that looked at the object, people were actually less likely to look at the object than if no one had looked at the object at all (see also Gallup, Hale, Sumpter, Garnier, Kacelnik, Krebs, & Couzin, 2012b). In sum, Laidlaw et al. (2011) and Gallup et al. (2012a, 2012b) found patterns of gaze in natural social contexts that were the opposite of what would have been expected given demonstrations in the laboratory that individuals have a strong, automatic tendency to look at, and follow, the gaze of images in face-to-face situations.  These discrepancies between social attentional phenomena observed in the laboratory and in natural social contexts demonstrated that the paradigms that yielded the former were failing to capture some critical attribute of social attention – an idea that is currently drawing much interest (e.g., Pfeiffer, Vogeley & Schilbach, 2013; Schilbach, 2015; Schilbach, Timmermans, Reddy, Costatt, Bente, Schlicht, & Vogeley, 2013). We have recently suggested that what this research failed to capture was the dual function of gaze (Gobel & Richardson, 2014; Jarick & Kingstone, 2015; Laidlaw, Risko, & Kingstone, in press; Nasiopoulos, Risko, & Kingstone, in press; Richardson, & Gobel, in press; Risko, & Kingstone, 2015).  The Dual Function of Gaze The dual function of gaze was captured succinctly by Argyle and Cook (1976) nearly 40 years ago when they argued that “Whenever organisms use vision, the eyes become signals as well as channels” (p. xi).  In other words, the eyes both gather information (i.e., act as a channel) and communicate information to others (i.e., act as a signal). This dual function provides a straightforward explanation for why discrepancies arise with respect to how individuals attend to images of people versus real people. When looking at a representation of a social stimulus, Dual Function of Gaze 6  individuals need not worry about what their own gaze might be communicating to the image. This luxury is absent when the “stimuli” are real people. From this perspective, the Laidlaw et al. (2011) and Gallup et al. (2012a, 2012b) results could be interpreted as reflecting a kind of “civil inattention” wherein individuals in an authentic social context inhibit exhibiting “staring” or looking behavior in general that could be interpreted by strangers to communicate interest (Goffman, 1963; Zuckerman, Miserandino, & Bernieri, 1983). There is no reason to enact this social norm when looking at an image of a person and as a result, the ability to predict attentional behavior in authentic social contexts where gaze simultaneously plays both roles becomes limited.   Recent research has addressed this research limitation by manipulating a participant’s belief about whether or not their own gaze is viewed by another person. This manipulation captures the differences in attentional behavior between when the participant’s gaze functions as a channel, acquiring information, or as both a channel and signal to the person being viewed. For example, Gobel and Richardson (2014) had participants look at videos of people of different social rank and made judgements about them. As they viewed the targets, a camera recorded participants’ faces. There were two conditions. In the two-way condition, participants believed that the targets would later be shown the recordings of the participants’ faces. In other words, the participants looked at the target faces believing that the targets would later be watching them back. In the one-way condition, the participants believed that the recordings of their own faces would not be seen by anyone. Participants' looking behaviour was influenced by the social rank of the targets, but this effect was reversed across the one- and two-way conditions. When looking at targets of high social rank, they looked at the eyes more in the one-way than in the two-way condition. The opposite trend was present when participants looked at targets of low social rank. Dual Function of Gaze 7  One interpretation is that there are norms associated with social rank, include avoiding eye contact with higher-ranking individuals (De Waal, 1989; Emery, 2000), which is only active when gaze has a signalling function. In other words, when participants thought that they were both a viewer and would be viewed. In a similar vein, Risko and Kingstone (2011) placed participants in a room with a provocative stimulus (i.e., a swimsuit calendar) and manipulated, using a mobile eye tracker, whether individuals thought that their gaze was being monitored. Critically, when individuals did not think their gaze was being monitored, they almost always looked at the provocative stimulus but when wearing an eye tracker these looks decreased significantly. The authors interpreted this change in looking behaviour as a kind of gaze based impression management (e.g., “I am not interested in that kind of material”). In recent work, Nasiopoulus et al. (2014) replicated this effect and also demonstrated that (a) this gaze-based impression management disappeared over time once participants forgot about wearing the glasses (i.e., they readily looked at the provocative calendar) and (b) it reappeared if participants were subtly reminded about the eye tracker (i.e., they avoided looking at the calendar). The latter results suggest that maintaining an active representation of at least some feature of the social context might be necessary for social norm based influences on gaze. These examples draw attention to the need to develop a deeper understanding of the social norms of looking, given these norms largely shape what we attend to in authentic social contexts. While there are methodological challenges in leaving the laboratory, progress is being made. For example, Wu, Bischof and Kingstone (2013; 2014) surreptitiously recorded individuals’ gaze behavior as they ate with another person (sometimes a confederate) and documented the norms of looking associated with that common social situation. For example, in Dual Function of Gaze 8  pairs when one person looks down to signal that a bite of food is forthcoming, the other person tends to look away.  Having identified the gaze patterns present in a naturalistic interaction, future work can now focus on the factors that modulate this particular social attentional behavior. This strategy of first examining a phenomenon of interest in situ and following it up with more controlled experiments (i.e., cognitive ethology; Kingstone, 2009; Kingstone, Smilek, & Eastwood, 2008) will likely play a more important role as social attention research takes more seriously the dual function of gaze.  Future Directions These results clearly demonstrate that the real and implied presence of another can substantially alter patterns of gaze in social contexts. But the basic mechanisms underlying these effects remain to be understood. One direction is to integrate these results with current models of visual attention. Perhaps the information search and communicative functions of gaze are unique, top-down, contributors to the control of visual attention (e.g., Navalpakkam, & Itti, 2005; Wolfe, 2007).  Another compelling question is how the dual function of gaze relates to overt attention (i.e., a shift of attention that involves an eye/head/body movement) and covert attention (i.e., a shift of attention that does not involve an eye/head/body movement). If the fact that our eyes communicate information to others constrains our gaze selection, then (presumably) this constraint would apply only to overt shifts of visual attention. This raises the interesting possibility that overt and covert visual attention could be differentially sensitive to the actual or implied presence of others (see Kuhn et al., 2016 for recent evidence in support of this proposition). From a dual function perspective, a shift in covert attention could aid in the pick up of visual information from a given location without overtly communicating that an attention shift Dual Function of Gaze 9  to that location has occurred. This circumvention of the signalling function could represent one of the primary reasons our covert attentional systems evolved.   Conclusion: Beyond Social Attention There is a clear utility – we argue, a necessity - to considering the importance of ecological validity in cognitive science research.  While we have focused on social attention research here, there is good reason to suspect similar benefits in other areas of research. For example, research using more “naturalistic” stimuli and tasks has contributed significantly to a greater emphasis on top-down (e.g., task sensitive) vs. bottom-up (e.g., stimulus driven) approaches to modelling gaze (e.g., Chen & Zelinsky, 2006; Tatler, Hayhoe, Land, & Ballard, 2011; Turano, Geruschat, & Baker, 2003). In addition, like in many of the examples reviewed above, patterns of gaze can change across live and simulated putatively non-social contexts (e.g., Dicks, Button, & Davids, 2010).  As such, it could be beneficial to re-evaluate the confidence that we place in principles derived solely from research that strongly tips the scales in favor of experimental control over ecological validity and, when appropriate, consider tipping the scales in favor of the latter. Unless cognitive scientists are prepared to break the fourth wall of cognitive science, conclusions about social attention may never extend beyond the laboratory cubicle.       Dual Function of Gaze 10  Acknowledgements This research was supported by a Social Sciences and Humanities Research Council of Canada (SSHRC) insight grant to EFR and AK and funding from the Canada Research Chairs program to EFR.                    Dual Function of Gaze 11  Notes 1. Correspondence may be directed to EFR, Department of Psychology, University of Waterloo, 200 University Ave. West, Waterloo, ON N2L 3G1. Email: efrisko@uwaterloo.ca                 Dual Function of Gaze 12  References Baron-Cohen, S., Wheelwright, S., & Jolliffe, A. T. (1997). Is there a" language of the eyes"? Evidence from normal adults, and adults with autism or Asperger syndrome. Visual Cognition, 4, 311-331. Batki, A., Baron-Cohen, S., Wheelwright, S., Connellan, J., & Ahluwalia, J. (2000). Is there an innate gaze module? Evidence from human neonates. Infant Behavior and Development, 23, 223-229. Birmingham, E., & Kingstone, A. (2009). Human social attention: A new look at past, present and future investigations. Annals of the New York Academy of Sciences, 1156, 118–140. Chen, X., & Zelinsky, G. J. (2006). Real-world visual search is dominated by top-down guidance. Vision research, 46, 4118-4133. De Waal, F. B. M. (1989). Chimpanzee politics: Power and sex among apes. Baltimore, MD: John Hopkins University Press. Emery, N. J. (2000). The eyes have it: the neuroethology, function and evolution of social gaze. Neuroscience & Biobehavioral Reviews, 24, 581-604. Fletcher-Watson, S., Findlay, J. M., Leekam, S. R., & Benson, V. (2008). Rapid detection of person information in a naturalistic scene. Perception, 37, 571-583. Friesen, C. K., & Kingstone, A. (1998). The eyes have it! Reflexive orienting is triggered by nonpredictive gaze. Psychonomic Bulletin & Review, 5, 490-495. Frishen, A., Bayliss, A. P., & Tipper, S. P. (2007). Gaze cueing of attention: Visual attention, social cognition, and individual differences. Psychological Bulletin, 133, 694–724. Gallup, A. C., Chong, A. and Couzin, I. D. (2012a). The directional flow of visual information transfer between pedestrians. Biology Letters, 8, 520–522. Dual Function of Gaze 13  Gallup, A.C., Hale, J.J., Sumpter, D.J.T., Garnier, S., Kacelnik, A., Krebs, J.R., & Couzin, I.D. (2012). Visual attention and the acquisition of information in human crowds. Proceedings of the National Academy of Sciences of the United States of America, 109, 7245-7250. Gobel, M. S., Kim, H. S., & Richardson, D. C. (2015). The dual function of social gaze. Cognition, 136, 359-364. Goffman, E. (1963). Behavior in public places. Free Press: New York, New York. Jarick, M., & Kingstone, A. (2015). The duality of gaze: Eyes extract and signal social information during sustained cooperative and competitive dyadic gaze. Frontiers in Psychology, 6, 1423. Kingstone, A. (2009). Taking a real look at social attention. Current Opinion in Neurobiology, 19, 52-56. Kingstone, A., Smilek, D., & Eastwood, J. D. (2008). Cognitive ethology: A new approach for studying human cognition. British Journal of Psychology, 99, 317-340. Kuhn, G., Teszka, R., Tenaw, N. & Kingstone, A. (2016). Don't be fooled! Top-down inhibition of attentional orienting in response to social cues in face-to-face and video context. Cognition, 146, 136-142. Laidlaw, K. E. W., Risko, E. F., & Kingstone, A. (2012). A new look at social attention: Orienting to the eyes is not (entirely) under volitional control. Journal of Experimental Psychology: Human Perception and Performance, 38, 1132–1143. Laidlaw, K.E.W., Risko, E.F., & Kingstone, A. (in press). Levels of complexity and the duality of gaze: How social attention changes from lab to life. In S.D. Obhi, & E.S. Cross (Eds.), Shared Representations: Sensorimotor Foundations of Social Life. Cambridge University Press. Dual Function of Gaze 14  Laidlaw, K.E.W., Foulsham, T., Kuhn, G., & Kingstone, A. (2011). Potential social interactions are important to social attention. Proceedings of the National Academy of Sciences, 108, 5548-5553. Langton, S. R., Watt, R. J., & Bruce, V. (2000). Do the eyes have it? Cues to the direction of social attention. Trends in Cognitive Sciences, 4, 50-59. Levy, J., Foulsham, T., & Kingstone, A. (2012). Monsters are people too. Biology Letters, 9, 20120850. Nasiopoulos, E., Risko, E. F. & Kingstone, A. (in press). Social attention, social presence, and the dual function of gaze. In A. Puce, & B.I. Bertenthal (Eds.), The Many Faces of Social Attention: Behavioral and Neural Measures. Springer: New York, NY. Nasiopoulos, E., Risko, E.F., Foulsham, T., & Kingstone, A. (2014). Wearable computing: Will it make people prosocial? British Journal of Psychology, 106, 209-216. Navalpakkam, V., & Itti, L. (2005). Modeling the influence of task on attention. Vision Research, 45, 205–231. Pfeiffer, U. J., Vogeley, K., & Schilbach, L. (2013). From gaze cueing to dual eye-tracking: novel approaches to investigate the neural correlates of gaze in social interaction. Neuroscience & Biobehavioral Reviews, 37, 2516-2528. Richardson, D. C, & Gobel, M.S. (in press). Social attention. In J. Fawcett, E. F. Risko, & A. Kingstone (Eds.), The Handbook of Attention. MIT Press: Cambridge, MA. Richardson, D. C., Street, C. N. H., Tan, J. Y. M., Kirkham, N. Z., Hoover, M. A., Ghane, & Cavanaugh, A. (2012). Joint perception: Gaze and social context. Frontiers in Human Neuroscience, 6, 194. Dual Function of Gaze 15  Risko, E.F., & Kingstone, A. (2011). Eyes wide shut: Implied social presence, eye tracking and attention. Attention, Perception, and Psychophysics, 73, 291-296. Risko, E. F., & Kingstone, A. (2015). Attention in the wild: Visual attention in complex, dynamic and social environments. In J. Szalma et al. (Eds.), Cambridge Handbook of Applied Perception Research. Cambridge University Press: Cambridge, UK. Risko, E. F., Laidlaw, K., Freeth, M., Foulsham, T., & Kingstone, A. (2012). Social attention with real versus reel stimuli: Toward an empirical approach to concerns about ecological validity. Frontiers in Human Neuroscience, 6, 143. doi:10.3389/fnhum.2012.00143 Schilbach, L. (2015). Eye to eye, face to face and brain to brain: novel approaches to study the behavioral dynamics and neural mechanisms of social interactions. Current Opinion in Behavioral Sciences, 3, 130-135. Schilbach, L., Timmermans, B., Reddy, V., Costatt, A., Bente, G., Schlicht, T., & Vogeley, K. (2013). Toward a second-person neuroscience. Behavioral and Brain Sciences, 36, 393-414. Soto, D., Hodsoll, J., Rotshtein, P., & Humphreys, G. W. (2008). Automatic guidance of attention from working memory. Trends in Cognitive Sciences, 12, 342-348. Wu, D.W.-L., Bischof, W.F. & Kingstone, A. (2013). Looking while eating: The importance of social context to social attention. Scientific Reports, 3, 2356. Wu, D.W.-L., Bischof, W.F. & Kingstone, A. (2014). Natural gaze signaling in a social context. Evolution & Human Behavior, 35, 211-218. Wolfe, J. M. (2007). Guided search 4.0: Current progress with a model of visual search. In Integrated models of cognitive systems, W. D. Gray (Ed). Oxford University Press: New York, NY. Dual Function of Gaze 16  Yarbus, A.L. 1967. Eye Movements and Vision. Plenum Press: New York, NY. Zuckerman, M., Miserandino, M., & Bernieri, F. (1983). Civil inattention exists—in elevators. Personality and Social Psychology Bulletin, 9, 578-586.                      Dual Function of Gaze 17  Recommended Readings Kingstone, A., Smilek, D., & Eastwood, J. D. (2008). Cognitive ethology: A new approach for studying human cognition. British Journal of Psychology, 99, 317-340. ● A representative review of arguments for the utility of adopting more ecological methods in cognitive science in general. Included in the same issue are responses to the article and the author’s response to those responses. Schilbach, L., Timmermans, B., Reddy, V., Costatt, A., Bente, G., Schlicht, T., & Vogeley, K. (2013). Toward a second-person neuroscience. Behavioral and Brain Sciences, 36, 393-414. ● A recent and comprehensive argument for the use of paradigms that more closely approximate an authentic social interaction in research on social cognition. Richardson, D. C, & Gobel, M.S. (in press). Social attention. In J. Fawcett, E. F. Risko, & A. Kingstone (Eds.), The Handbook of Attention. MIT Press: Cambridge, MA. ● A comprehensive, highly accessible overview of research on social attention. Argyle, M., & Cook, M. (1976). Gaze and mutual gaze. Cambridge: Cambridge University Press. ● Classic, early work on gaze that emphasized its dual function Pönkänen, L.M., Alhoniemi,A., Leppänen, J.M., & Hietanen, J.K. (2011). Does it make a difference if I have eye contact with you or with your picture? An ERP study. Social, Cognitive, and Affective Neuroscience, 6, 486–494. ● Empirical work providing evidence that how we respond to direct vs. averted gaze is also modulated by whether the gazer is a real person or picture of a person. Dual Function of Gaze 18   Figure 1.  The fourth wall in cognitive science represents a barrier between participants and stimuli that we argue limits the conclusions that can be drawn from many experimental studies, particularly in social attention (e.g., faces on a screen, unlike real people, cannot look back at the participant). As such, by breaking the fourth wall (e.g., by using real people as stimuli) new insights can be gained. 


Citation Scheme:


Citations by CSL (citeproc-js)

Usage Statistics



Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            async >
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:


Related Items