Arts, Faculty of
Psychology, Department of
DSpace
UBCV
Bryce, Courtney Aryn
Doctor of Philosophy - PhD
University of British Columbia
The stress response is a co-ordinated reaction to real or perceived threats. One arm of the response is the hypothalamic-pituitary-adrenal axis, initiated by corticotropin-releasing factor (CRF). Centrally-active CRF mediates many of the rapid behavioural stress responses. Indeed, CRF mediates, and exogeneous CRF mimics, the acute stress-induced bias away from more valuable rewards requiring greater effort. In Chapter 1, we discuss relevant research comparing acute stress and central CRF infusion at the circuit level and as it relates to different cognitive domains. In Chapters 2 and 3, we examine how acute stress and increased central CRF activity alter cognitive flexibility in both sexes, and risk/reward decision making in males. Using a probabilistic reversal learning (PRL) task, results suggested that acute stress impaired PRL in males, but CRF slightly facilitated flexibility across both sexes, with minimal overall sex differences. Using two decision making tasks, we found that acute stress increased reward sensitivity and CRF impaired optimal risky choice with external cues, whereas neither manipulation altered risky choice when reward contingencies were internally generated. Moreover, CRF markedly reduced motivation in all tasks. When viewed together, we found that CRF hyperactivity altered behaviour in a manner more similar to chronic than acute stress manipulations. In search of potential mechanisms, we probe the role of centrally-active CRF in altering ventral tegmental area dopamine neuron activity in Chapter 4, revealing that CRF increased dopamine neuron population activity. Chapter 5 examines the role of increased mesolimbic dopamine signaling on effort-related decision making. Here, we found increased nucleus accumbens (NAc) D2 receptor activity biased choice away from the more valuable, but more physically onerous, reward in a manner parallel to central CRF treatment. Collectively, these latter results suggest that increased CRF signaling enhances tonic dopamine activity, enhancing mesolimbic dopamine tone acting on NAc D2 receptors to reduce effort choice. Enhanced CRF activity is found in those with depression and other stress-related disorders. Therefore, culmination of these experiments point to a key role in CRF hyperactivity, perhaps via interaction with dopamine signaling, in perturbing behaviour within cognitive domains known to be altered in depression.
THE ROLE OF CORTICOTROPIN-RELEASING FACTOR AND ACUTE STRESS IN MODULATING COGNITIVE FLEXIBILITY AND DECISION MAKING: PHYSIOLOGICAL AND BEHAVIOURAL INTERACTIONS WITH MESOLIMBIC DOPAMINE by Courtney Aryn Bryce B.A. (High Honours), The University of Saskatchewan, 2013 M.A., The University of British Columbia, 2015 A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY in THE FACULTY OF GRADUATE AND POSTDOCTORAL STUDIES (Psychology) THE UNIVERSITY OF BRITISH COLUMBIA (Vancouver) March 2020 © Courtney Aryn Bryce, 2020 ii The following individuals certify that they have read, and recommend to the Faculty of Graduate and Postdoctoral Studies for acceptance, the dissertation entitled: The role of corticotropin-releasing factor and acute stress in modulating cognitive flexibility and decision making: physiological and behavioural interactions with mesolimbic dopamine submitted by Courtney Bryce in partial fulfillment of the requirements for the degree of Doctor of Philosophy in Psychology Examining Committee: Stan Floresco, Psychology Supervisor Joanne Weinberg, Cellular and Physiological Sciences Supervisory Committee Member Frances Chen, Psychology Supervisory Committee Member Liisa Galea, Psychology University Examiner Victor Viau, Cellular and Physiological Sciences University Examiner Additional Supervisory Committee Members: John Mantsch, External Examiner, Biomedical Sciences Supervisory Committee Member Ujendra Kumar, Chair, Pharmaceutical Sciences Supervisory Committee Member iii Abstract The stress response is a co-ordinated reaction to real or perceived threats. One arm of the response is the hypothalamic-pituitary-adrenal axis, initiated by corticotropin-releasing factor (CRF). Centrally-active CRF mediates many of the rapid behavioural stress responses. Indeed, CRF mediates, and exogeneous CRF mimics, the acute stress-induced bias away from more valuable rewards requiring greater effort. In Chapter 1, we discuss relevant research comparing acute stress and central CRF infusion at the circuit level and as it relates to different cognitive domains. In Chapters 2 and 3, we examine how acute stress and increased central CRF activity alter cognitive flexibility in both sexes, and risk/reward decision making in males. Using a probabilistic reversal learning (PRL) task, results suggested that acute stress impaired PRL in males, but CRF slightly facilitated flexibility across both sexes, with minimal overall sex differences. Using two decision making tasks, we found that acute stress increased reward sensitivity and CRF impaired optimal risky choice with external cues, whereas neither manipulation altered risky choice when reward contingencies were internally generated. Moreover, CRF markedly reduced motivation in all tasks. When viewed together, we found that CRF hyperactivity altered behaviour in a manner more similar to chronic than acute stress manipulations. In search of potential mechanisms, we probe the role of centrally-active CRF in altering ventral tegmental area dopamine neuron activity in Chapter 4, revealing that CRF increased dopamine neuron population activity. Chapter 5 examines the role of increased mesolimbic dopamine signaling on effort-related decision making. Here, we found increased nucleus accumbens (NAc) D2 receptor activity biased choice away from the more valuable, but more physically onerous, reward in a manner parallel to central CRF treatment. Collectively, these latter results suggest that increased CRF signaling enhances tonic dopamine activity, iv enhancing mesolimbic dopamine tone acting on NAc D2 receptors to reduce effort choice. Enhanced CRF activity is found in those with depression and other stress-related disorders. Therefore, culmination of these experiments point to a key role in CRF hyperactivity, perhaps via interaction with dopamine signaling, in perturbing behaviour within cognitive domains known to be altered in depression. v Lay Summary Real or perceived threats cause the body to release various factors making up the stress response. Accumulation of these factors, built up over time by repeated exposure can lead to disorders such as depression. Depression is characterized by feelings of sadness and loss of pleasure. However, impairments in motivation and cognition, including cognitive flexibility and decision making, are also detrimental to depression outcomes. Current antidepressants are ineffective for some sufferers and fail to address motivational and cognitive deficits. Thus, identifying stress factors that are increased in depression and alter cognition is critical for developing new antidepressants. Here, we examined the role of acute stress, with a focus on how an increase in one particular stress factor, contributes to alterations in cognition and motivation. Results indicated that increasing this stress factor impaired motivation and decision making, areas known to be affected in depression. vi Preface Experimental chapters (2-4) were conducted in the laboratory of Dr. Stan B. Floresco in the Department of Psychology at the University of British Columbia. Experiments were designed by Courtney A. Bryce (C.A. Bryce) and Dr. Stan B. Floresco (S.B. Floresco). Data collection was conducted by C.A. Bryce and undergraduate students under her direct supervision. Data were analyzed and written by C.A. Bryce, with support from S.B. Floresco. Chapter 1: A version of the section entitled, “Physiological interactions between CRF and DA” was included in part in a previously published review paper. This review was conceived and conducted by S. Hupalo, with writing by C.A. Bryce, D.A. Bangasser, C.W. Berridge, R.J. Valentino, and S.B. Floresco. Only the section that was written directly by C.A. Bryce with the assistance of S.B. Floresco was included in the Chapter 1 section. This version was published in the following form: Hupalo, S., Bryce, C.A., Bangasser, D.A., Berridge, C.W., Valentino, R.J., & Floresco, S.B. (2019). Corticotropin-releasing factor (CRF) circuit modulation of cognition and motivation. Neuroscience and Biobehavioral Reviews, 103, 50-59. https://doi.org/10.1016/j.neubiorev.2019.06.010 Chapter 3: A version of this chapter has been submitted for publication. C.A. Bryce conceived, analyzed data, and wrote the manuscript with S.B. Floresco. A.J. Adalbert and M.M. Claes aided in experimental work and M. van Holstein provided manuscript feedback. This version was published in the following form: Bryce, C. A., Adalbert, A. J., Claes, M. M., van Holstein, M., & Floresco, S. B. (2020). Differential effects of corticotropin-releasing factor and acute stress on risk/reward decision-making. Neurobiology of Learning and Memory, 169. https://doi.org/10.1016/j.nlm.2020.10716 vii Chapter 5: A version of this chapter has been published. C.A. Bryce conceived, analyzed data, and wrote the manuscript with S.B. Floresco. This version was published in the following form: Bryce, C.A., & Floresco, S.B. (2019). Alterations in effort-related decision-making induced by stimulation of dopamine D1, D2, D3, and corticotropin-releasing factor receptors in nucleus accumbens subregions. Psychopharmacology, 236(9), 2699-2712. https://doi.org/10.1007/s00213-019-05244-w C.A. Bryce performed all surgeries, and conducted behavioural training and testing with assistance from A.J. Adalbert, M.M. Claes, and N. Symonds (undergraduate volunteers). C.A. Bryce wrote the dissertation, with input from S.B. Floresco. All experimental protocols were approved by the Animal Care Committee (ACC), University of British Columbia, and conducted in compliance with guidelines provided by the Canadian Council on Animal Care (CCAC). ACC certificate numbers: A10-0197, A14-0210, or A18-0242 viii Table of Contents Abstract ......................................................................................................................................... iii Lay Summary ................................................................................................................................ v Preface ........................................................................................................................................... vi Table of Contents ....................................................................................................................... viii List of Tables ............................................................................................................................... xii List of Figures ............................................................................................................................. xiii List of Abbreviations ................................................................................................................. xiv Acknowledgements .................................................................................................................... xvi Chapter 1: Introduction ............................................................................................................... 1 1.1 Acute stress response: Focus on the HPA axis and CRF systems ............................................ 2 1.2 Behavioural effects of acute stress and CRF ........................................................................... 8 1.21 Learning and memory ............................................................................................................ 9 1.22 Attention ........................................................................................................................... 15 1.23 Working memory ................................................................................................................ 16 1.24 Cognitive flexibility ............................................................................................................. 18 1.25 Motivation and reward/negative feedback processing ................................................................. 22 1.26 Decision making ................................................................................................................. 27 1.3 Physiological interactions between CRF and dopamine........................................................ 31 1.4 Sex differences...................................................................................................................... 34 1.41 Sex differences in the stress response ...................................................................................... 34 1.42 Baseline sex differences in cognitive functioning ....................................................................... 35 1.43 Sex differences in the effects of stress on cognition .................................................................... 37 1.5 Clinical implications: Emphasis on depression ..................................................................... 39 1.6 Objectives............................................................................................................................. 41 ix Chapter 2: Modulation of cognitive flexibility in a probabilistic reversal learning task by CRF and acute stress in male and female rats ......................................................................... 45 2.1 Introduction ......................................................................................................................... 45 2.2 Methods ............................................................................................................................... 47 2.21 Animals............................................................................................................................. 47 2.22 Apparatus .......................................................................................................................... 47 2.23 Surgery ............................................................................................................................. 48 2.24 Initial lever pressing training ................................................................................................. 48 2.25 Probabilistic reversal learning ................................................................................................ 49 2.26 Drugs, microinfusion protocols, and acute restraint stress ............................................................ 50 2.27 Female estrous cycle ............................................................................................................ 52 2.28 Histology ........................................................................................................................... 53 2.29 Data analysis ...................................................................................................................... 53 2.3 Results .................................................................................................................................. 56 2.31 Central CRF infusion ........................................................................................................... 56 2.32 Restraint stress .................................................................................................................... 60 2.4 Discussion............................................................................................................................. 65 2.41 Cognitive flexibility ............................................................................................................. 65 2.42 Feedback sensitivity............................................................................................................. 68 2.43 Motivation ......................................................................................................................... 70 2.44 Sex differences ................................................................................................................... 72 2.45 Conclusion ......................................................................................................................... 73 Chapter 3: Differential effects of CRF and acute stress on different forms of risk/reward decision making ........................................................................................................................... 79 3.1 Introduction ......................................................................................................................... 79 3.2 Methods ............................................................................................................................... 81 3.21 Animals............................................................................................................................. 81 3.22 Apparatus .......................................................................................................................... 82 3.23 Risk/reward decision making tasks ......................................................................................... 82 x 3.24 Drugs and microinfusion protocols ......................................................................................... 86 3.25 Histology ........................................................................................................................... 87 3.26 Data analysis ...................................................................................................................... 87 3.3 Results .................................................................................................................................. 93 3.31 Probabilistic discounting ....................................................................................................... 93 3.32 Blackjack task .................................................................................................................... 96 3.4 Discussion............................................................................................................................. 98 3.41 CRF and acute stress effects on risky choice ............................................................................. 99 3.42 Motivational effects of CRF and acute stress .......................................................................... 103 3.43 Conclusion ....................................................................................................................... 104 Chapter 4: CRF dose-dependently increases dopaminergic and non-dopaminergic activity in the VTA ................................................................................................................................. 114 4.1 Introduction ....................................................................................................................... 114 4.2 Methods ............................................................................................................................. 117 4.21 Extracellular electrophysiology recordings ............................................................................. 117 4.22 Drug infusions .................................................................................................................. 118 4.23 Histology ......................................................................................................................... 119 4.24 Data analysis .................................................................................................................... 119 4.3 Results ................................................................................................................................ 120 4.31 Dopamine neural activity in the VTA .................................................................................... 120 4.32 Non-dopamine neural activity in the VTA .............................................................................. 121 4.4 Discussion........................................................................................................................... 122 Chapter 5: Alterations in effort-related decision making induced by stimulation of dopamine D1, D2, D3, and CRF receptors in NAc subregions ............................................. 129 5.1 Introduction ....................................................................................................................... 129 5.2 Methods ............................................................................................................................. 132 5.21 Initial lever pressing training ............................................................................................... 132 5.22 Effort-related decision making ............................................................................................. 132 xi 5.23 Reward magnitude discrimination......................................................................................... 134 5.24 Surgery ........................................................................................................................... 134 5.25 Drugs and microinfusion protocols ....................................................................................... 134 5.26 Histology ......................................................................................................................... 136 5.27 Data analysis .................................................................................................................... 136 5.3 Results ................................................................................................................................ 137 5.31 D1 receptor stimulation ...................................................................................................... 137 5.32 D2/3 receptor stimulation .................................................................................................... 138 5.33 D3 receptor stimulation ...................................................................................................... 140 5.34 CRF................................................................................................................................ 141 5.35 Baseline choice differences across NAc core and shell groups .................................................... 142 5.4 Discussion........................................................................................................................... 144 5.41 D1 receptor stimulation ...................................................................................................... 145 5.42 D2/3 receptor stimulation .................................................................................................... 146 5.43 CRF stimulation ................................................................................................................ 149 5.44 Conclusion ....................................................................................................................... 151 Chapter 6: General Discussion ................................................................................................ 159 6.1 Overview of findings .......................................................................................................... 159 6.2 Implications of acute stress and central CRF on cognitive flexibility and risky choice ....... 162 6.21 Reward and negative feedback sensitivity .............................................................................. 169 6.22 Motivation ....................................................................................................................... 173 6.23 Sex differences in cognitive flexibility, feedback sensitivity, and motivation ................................. 176 6.3 Implications of central CRF signaling on VTA dopamine physiology and consequences of increased mesolimbic dopamine activity on effort choice ......................................................... 179 6.4 Experimental limitations and future directions .................................................................. 184 6.5 Relevance to depression and other stress-related disorders ................................................ 193 6.6 Conclusion.......................................................................................................................... 200 References .................................................................................................................................. 201 xii List of Tables Table 1. Mean (range) number of wins and losses on win-stay | good odds, lose-shift | good odds, and lose-shift | poor odds trials after treatment with vehicle (aCSF), CRF, no stress or one-hour restraint stress. ..................... 106 Table 2. Mean (S.E.M) locomotor activity (photobeam breaks) and number of omissions (averaged over trials) after treatment with vehicle (aCSF), CRF, no stress or one-hour restraint stress. .................................................. 107 Table 3. Mean (S.E.M) win-stay/lose-shift ratios as a function of the recent outcome of the preceding trial ) after treatment with vehicle (aCSF), CRF, no stress or one-hour restraint stress. .................................................. 108 Table 4. Mean (S.E.M) locomotor activity (photobeam breaks) and number of omissions (over 48 trials) after treatment with vehicle (saline) or D1, D2, D3 receptor agonists or CRF during effort-discounting. .................. 152 xiii List of Figures Figure 1. Central CRF infusion reduced motivation and feedback sensitivity in males during acquisition. ........... 75 Figure 2. Central CRF infusion reduced motivation and negative feedback sensitivity during performance. ........ 76 Figure 3. Restraint stress impaired flexibility in males during acquisition. .................................................... 77 Figure 4. Restraint stress reduced motivation and feedback sensitivity in females during performance. .............. 78 Figure 5. Probabilistic discounting and Blackjack task designs. ................................................................ 109 Figure 6. Central CRF infusion reduced motivation but not choice on probabilistic discounting. ..................... 110 Figure 7. Restraint stress slightly reduced motivation but not choice on probabilistic discounting.................... 111 Figure 8. Central CRF infusion reduced optimal risky choice on the cue-guided Blackjack task. ..................... 112 Figure 9. Restraint stress slightly increased risky choice and reward sensitivity on the external cue-guided Blackjack task. .............................................................................................................................................. 113 Figure 10. High dose central CRF increased tonic, but not phasic, dopamine neuron firing. ........................... 127 Figure 11. Low dose central CRF increased GABAergic and unclassified neuron firing. ............................... 128 Figure 12. NAc core and shell placements. ........................................................................................... 153 Figure 13. D1 receptor stimulation in the NAc core or shell does not alter effort discounting. ......................... 154 Figure 14. D2/3 receptor stimulation in the NAc reduced high effort choice. ............................................... 155 Figure 15. Effects of D3 receptor stimulation in the NAc core and shell on effort discounting. ....................... 156 Figure 16. Effects of CRF infusion in the NAc core and shell on effort discounting. ..................................... 157 Figure 17. Baseline effort choice in the NAc core and shell groups. .......................................................... 158 xiv List of Abbreviations 5-CSRT 5-choice choice serial reaction time task 5-HT serotonin ACTH adrenocorticotropic-releasing hormone aCSF artificial cerebrospinal fluid ANOVA analysis of variance ANS autonomic nervous system AP anteroposterior BLA basolateral amygdala BNST bed nucleus of the stria terminalis CGT Cambridge gambling task cAMP cyclic adenosine monophosphate CeA central amygdala CRF corticotropin-releasing factor CRF1R corticotropin-releasing factor receptor 1 CRF2R corticotropin-releasing factor receptor 2 CORT corticosterone or cortisol DRN dorsal raphe nucleus DV dorsoventral EPSC excitatory post-synaptic current FR fixed ratio GABA gamma-aminobutyric acid GDT game of dice test GR glucocorticoid receptor HPA hypothalamic-pituitary-adrenal HR high reward lever ICV intracerebroventricular IGT Iowa Gambling Task IPSC inhibitory post-synaptic current ITI intertrial interval LC locus coeruleus xv LR low reward lever ML mediolateral mPFC medial prefrontal cortex MR mineralocorticoid receptor NAc nucleus accumbens OFC orbitofrontal cortex PFC prefrontal cortex PRL probabilistic reversal learning PVN paraventricular nucleus of the hypothalamus UCN urocortin VTA ventral tegmental area xvi Acknowledgements First, I would like to thank my supervisor, Dr. Stan Floresco, for his support and guidance throughout this process. Thank you for being an outstanding mentor. Your advice and insight have undoubtedly made me a better writer and a better scientist. Although grad school was challenging, I had the time of my life. Creating a learning environment where everyone was instilled with a sense of belonging is a real credit to your leadership. I am grateful to the other members of my supervisory committee, Dr. Joanne Weinberg and Dr. Frances Chen, who have provided valuable insights in building this thesis. I would also like to extend my thanks to Dr. Catherine Rankin for her excellent questions and advice during the comprehensive exam. This thesis is connected to a much more grounded literature because of her input. In particular, I would like to thank both Dr. Weinberg and Dr. Rankin for pushing me into what should be the future of neuroscience, where both males and females are used in all preclinical work. The basis on which we build an understanding of the brain should be built atop a foundation of equality. One of the biggest gifts of this program has been the lasting friendships that have been made along the way. I would like to thank all of the members of the Floresco lab, past and present for their support and friendship: Patrick Piantadosi, Meagan Auger, Nicole Jenni, Debra Bercovici, Maric Tse, Gemma Floresco, Colin Stopper, Josh Larkin, Ryan Tomm, and Mieke van Holstein. You have made the bad times bearable and the good times the best of my life. Thanks for all the laughs. Special thanks to Maric Tse for helping me learn the tricks of electrophysiology. I will also be forever indebted to the undergraduates who helped me with the behavioural experiments. Thank you Jade, Mona, and Nicola; I know you’ll do great things. I am also grateful to the wonderful animal care staff, especially Lucille Hoover, Alice Chan, and Anne xvii Cheng. And of course, I have to thank the rats for their service to the advancement of science. I hope the cause is worthy of their sacrifice. My greatest thanks are to my family. My parents did not attend university and we did not have a lot of money growing up, but they gave me the world. They put up with my interminable curiosity, my fierce sense of justice, and most of all they believed in me. To my dad: I thought you were the smartest person in the world and I wanted to be like you; I hope I’ve made you proud. To my mom: your love and support know no depths and I’m forever grateful for everything you’ve done for me; I see your sacrifice and I am forever in your debt. Thank you also to my in-laws for making such a wonderful son and for all the emotional support you’ve given me. I count myself lucky to have two sets of parents. Finally, I have to thank my partner, Latif. I hope I am worthy of your unending love and support. Thank you for encouraging me to follow my dreams and believing in me when I didn’t believe in myself. Most of all, thank you for seeing me as your equal. And for always making me laugh. 1 Chapter 1: Introduction The acute stress response is galvanized in reaction to real or perceived threats in the environment. Although stress is perceived as negative in our society, the stress response orchestrates a complex array of factors to ensure survival. One such system is the hypothalamic-pituitary-adrenal (HPA) axis that receives descending and ascending inputs that alert the hypothalamus to threats. The main conductor of the HPA axis is corticotropin-releasing factor (CRF), initiating a chain of signaling cascades that terminate in increased glucocorticoid secretion from the adrenal glands. It is notable that CRF not only drives HPA axis activation, but also acts outside of the hypothalamus to mediate the rapid actions of stress within relevant brain regions. Decades of research point to a complicated relationship between stress and cognition. For some cognitive processes, such as attention and working memory, the preponderance of evidence suggests that acute stress impairs cognitive function (Butts et al., 2011; Del Arco et al., 2007; Devilbiss et al., 2012; Diamond et al., 1996; Luethi et al., 2009; Olver et al., 2015; Sänger et al., 2014; Schoofs et al., 2008; Shansky et al., 2006; Skosnik et al., 2000). For others, such as learning, memory, cognitive flexibility, and decision making, the effects of acute stress are multifaceted and depend on the timing of the stressor relative to the memory formation or myriad other factors including sex differences (Bryce & Floresco, 2016; Bryce & Howland, 2015; Butts et al., 2013; Diamond et al., 2006; Fields et al., 2014; Preston et al., 2007; Roozendaal et al., 2006). The relevant intermediary between stress and cognition has been elucidated in some cases, with learning, memory, and working memory, mediated in part by glucocorticoids and/or CRF (Bardgett et al., 1994; Buchanan & Lovallo, 2001; Deak et al., 1999; Hupalo & Berridge, 2016; Roozendaal et al., 2003; Roozendaal et al., 2006). Relatively less is known about how acute stress mediates alterations in cognitive flexibility and decision making, which are also 2 mediated in large part by dopamine signaling (Floresco, St. Onge, Ghods-Sharifi, & Winstanley, 2008; Klanker, Feenstra, & Denys, 2013). Moreover, enhanced CRF activity has been found in stress-related disorders, such as schizophrenia, obsessive compulsive disorder, post-traumatic stress disorder, and depression (Altemus et al., 1992; Banki et al., 1987; Banki et al., 1992; Bremner et al., 1997; Nemeroff et al., 1984). Therefore, the experiments detailed in this Thesis sought to elucidate how acute stress and CRF signaling are involved in regulating cognitive flexibility and decision making, cognitive processes that are known to be affected in conditions associated with CRF hyperactivity (Austin, Mitchell, & Goodwin, 2001; Cella, Dymond, & Cooper, 2010; Dickstein et al., 2010; Han et al., 2012; Ilonen & Leinonen, 2000; Lee et al., 2012; Pulcu et al., 2014; Ravnkilde et al., 2002; Treadway et al., 2012). This thesis also aims to clarify the actions of increased CRF signaling on dopamine neuron physiology to understand how potential interplay between CRF and dopamine may lead to cognitive and motivational deficits. 1.1 Acute stress response: Focus on the HPA axis and CRF systems The acute stress response involves myriad factors working in concert to orchestrate the response to real or perceived threats in the environment, to return the organism to homeostasis, and to activate energy availability to ensure survival. In 1936, Hans Selye first characterized the physiological consequences of what came to be known as “stress”, which is generally defined as any condition that seriously perturbs the physiological or psychological homeostasis of an organism (Selye, 1936). This adaptive system can be divided into two main systems of organization that respond on different timescales: the autonomic nervous system (ANS) that drives quick action to marshal resources on the scale of seconds to minutes and the HPA axis, that allows for sustained action over minutes to hours (or longer in the case of genomic actions). 3 The canonical ‘fight-or-flight’ response was first characterized by Walter Cannon in the early twentieth century and best encapsulates the ANS (Cannon, 1915). The sympathetic nervous system, an arm of the ANS, involves descending inputs from the limbic system and dorsomedial hypothalamus to brainstem and spinal cord regions interacting with ascending inputs from preganglionic sympathetic neurons in the intermediolateral cell column of the spinal cord. These ascending and descending sympathetic inputs project to pre- or paravertebral ganglia that in turn project to chromaffin cells of the adrenal medulla (Habib, Gold, & Chrousos, 2001; Ulrich-Lai & Herman, 2009). Activation of the sympathetic nervous system generally increases circulating levels of epinephrine (via the adrenal medulla) and norepinephrine (via the sympathetic nerves) which increases heart rate, causes peripheral vasoconstriction, and mobilizes energy stores (Habib et al., 2001; Ulrich-Lai & Herman, 2009). The parasympathetic nervous system, the other arm of the ANS, acts in response to stressors via craniosacral preganglionic nuclei which trigger postganglionic nuclei located in or near the end organs they innervate (i.e. abdominal viscera) (Ulrich-Lai & Herman, 2009). The parasympathetic nervous system generally acts in opposition to the actions of the sympathetic nervous system to conserve energy and slow heart rate. The ANS releases epinephrine and norepinephrine in the body; however, neither epinephrine nor norepinephrine originating from peripheral systems cross the blood-brain barrier (Ulrich-Lai & Herman, 2009). As such, the brain contains its own norepinephrine system, with the majority of norepinephrine-expressing neurons located in the locus coeruleus (LC). LC norepinephrine-expressing neurons innervate the entire neuroaxis of the brain (Foote, Bloom, & Aston Jones, 1983). The central norepinephrine system is robustly activated by a diverse array of acutely stressful stimuli, thus mediating the stress-induced ANS response within the brain (Morilak et al., 2005). In addition, the central norepinephrine system exerts top-down control 4 over the descending ANS via brainstem regions, eventually increasing peripheral epinephrine from the adrenal glands (Habib et al., 2001; Morilak et al., 2005). The ANS works in concert with the HPA axis, which is the neuroendocrine limb of the stress response. The seat of the HPA axis is the paraventricular nucleus of the hypothalamus (PVN), which receives input from various brain regions activated by the perception of stressors in the environment. Dorsomedial parvocellular neurons in the PVN synthesize CRF and vasopressin, and project to hypophyseal portal capillaries in the external zone of the median eminence (Habib et al., 2001). CRF activates CRF1 receptors on corticotrope cells in the anterior pituitary to synthesize pro-opiomelanocortin-related peptides, including adrenocorticotropic-releasing hormone (ACTH), melanocyte-stimulating hormones, and -endorphin (Aguilera & Liu, 2012; Herman et al., 2016). CRF-induced ACTH secretion from the anterior pituitary into the bloodstream acts primarily to synthesize and release glucocorticoids from the adrenal glands. Vasopressin, acting on posterior pituitary receptors, synergistically enhances the ability of CRF to produce ACTH but does not independently increase ACTH production from the pituitary (Habib et al., 2001). Glucocorticoids, the final product of the HPA axis, are released into the bloodstream, acting throughout the body and brain, to mediate the stress response. Acute stress-induced glucocorticoid release slows digestion, reduces appetite, suppresses the immune system and reproduction, and mobilizes energy stores to prime the organism for reaction to the stressor (Herman et al., 2016). The primary glucocorticoid in humans is cortisol, whereas in rodents the main glucocorticoid is corticosterone (CORT). CORT binds to two cognate receptors throughout the body and the brain. The high affinity mineralocorticoid (MR) receptor is typically 90 percent occupied at basal CORT levels, whereas the glucocorticoid receptor (GR), due to its lower 5 affinity, is only activated with potentiated CORT levels, such as during episodes of acute stress (Reul & de Kloet, 1985). Increased CORT, via lower affinity GR activation, provides a vital negative feedback mechanism on the HPA axis, within the hypothalamus, ventral hippocampus, and prefrontal cortex (PFC), to suppress further CORT secretion (Ulrich-Lai & Herman, 2009). The HPA axis involves top-down and bottom-up regulation converging onto CRF-expressing neurons in the PVN. The PVN receives direct excitatory innervation from the nucleus of the solitary tract, which is also a relay for limbic inputs to drive ANS activity, therefore the nucleus of the solitary tract acts as an integrator of ANS and HPA axis activity (Ulrich-Lai & Herman, 2009). The anteroventral part of the dorsomedial hypothalamus and the arcuate nucleus project to the PVN providing intrahypothalamic excitation. Additionally, the anterior bed nucleus of the stria terminalis (BNST), infralimbic PFC, central amygdala (CeA), basolateral amygdala (BLA), dorsal raphe nucleus (DRN), and spinal cord nuclei provide stress-induced excitation of the HPA axis (Ulrich-Lai & Herman, 2009). Suppression of the HPA axis includes mostly GABAergic inhibition via the medial preoptic area, ventrolateral dorsomedial hypothalamus, local neurons in the peri-PVN region, posterior BNST, lateral septum, prelimbic PFC, and ventral hippocampus. Limbic regions typically do not interact directly with the PVN, rather intervening synapses relay information from the PFC, amygdala, and hippocampus via GABA-rich cells in the BNST and/or hypothalamus (Herman et al., 2003). Here, information from stress-excitatory regions are combined with information from stress-inhibitory regions, ensuring limbic information is integrated before reaching primary stress effectors, such as the PVN. Excitatory glutamatergic projections from the hippocampus and the prelimbic PFC are converted into inhibition of the PVN via trans-synaptic inhibition with projections through the GABAergic 6 BNST and/or periPVN. The infralimbic PFC and CeA innervate the nucleus of the solitary tract providing excitation to both HPA and ANS effectors, resulting in an integration between the different limbs of the stress response. Alternatively, amygdala neurons are mostly GABAergic, therefore stress-induced excitation of the PVN likely results from trans-synaptic disinhibition (Herman et al., 2016; Ulrich-Lai & Herman, 2009). Research on the HPA axis changed when, in 1981, Wylie Vale and colleagues discovered 41-amino acid stress-related neuropeptide, corticotropin-releasing hormone or factor (Vale et al., 1981). This peptide was initially classified as a hormone due to the central role of CRF in initiating the HPA axis via release into the median eminence enroute to the anterior pituitary. That said, PVN neurons expressing CRF also project to distal regions such as the ventral tegmental area (VTA) (Rodaros et al., 2007) and the CRF peptide is expressed extrahypothalamically within the BNST, CeA, VTA, and PFC (Grieder et al., 2014; Swanson et al., 1983), mediating a wider variety of responses to stress than initially understood. Therefore, this neuropeptide is also known as a factor, but CRF and CRH are typically used interchangeably. The CRF peptide activates two cognate receptors: CRF1 and CRF2 receptor. CRF1R is widely expressed in cortical and subcortical brain regions, including in the PFC, nucleus accumbens (NAc), and amygdala, and in mesencephalic regions, such as the dopamine-expressing VTA (Bittencourt & Sawchenko, 2000; Tan et al., 2017; Van Pett et al., 2000). Conversely, CRF2R is more restricted to subcortical and brainstem regions (Bittencourt & Sawchenko, 2000; Tan et al., 2017; Van Pett et al., 2000). Both receptors, however, are widely expressed in peripheral tissues (Hillhouse & Grammatopoulos, 2006). These receptors are class B1 subfamily of membrane-bound proteins that belong to the family of seven transmembrane G 7 protein-coupled receptors, encoded by 15 genes in humans (Hillhouse & Grammatopoulos, 2006). Structurally, CRF1R and CRH2R share ~70% identical amino acid components, with substantial divergence at the N terminus (Hillhouse & Grammatopoulos, 2006). Upon agonist binding, these receptors alter their structural conformation and signal through activation of heterotrimeric G proteins, regulating a diverse network of intracellular systems. CRF1R is a cognate receptor for both CRF as well as urocortin I (UCNI), which is in the family of CRF-related peptides (including UCNII and UCNII), with equivalent high affinity (Hillhouse & Grammatopoulos, 2006). Alternatively, CRF2R binds all the UCNs (UCNI, UCNII, UCNII) with significantly higher binding affinity than CRF, indicating that UCNs may be its natural ligands (Hillhouse & Grammatopoulos, 2006). Initial investigations of the effects of CRF on the HPA axis demonstrated that activation of pituitary CRF1Rs causes a substantial cyclic adenosine monophosphate (cAMP) response via stimulation of Gs proteins, indicating that CRF1R may be a Gs-coupled receptor (Millan et al., 1987). Gs-coupled receptor activation cleaves Gs from the G and G trimeric complex. Activated Gs binds to adenylyl cyclase to produce the second messenger cAMP, activating the cAMP-dependent protein kinase (or protein kinase A). Indeed, most studies show that both CRF1R and CRF2R primarily stimulate the adenylyl cyclase/cAMP pathway via Gs protein-coupled activation (Chen et al., 1993; Grammatopoulos et al., 1999; Lovenberg et al., 1995). Both receptors, however, can activate other G-subunits, including Go, Gq/11, Gi1/2, and Gz, which appears to be tissue- and region-dependent (Blank et al., 2003; Grammatopoulos et al., 2001; Hillhouse et al., 2002; Hillhouse & Grammatopoulos, 2006). 8 1.2 Behavioural effects of acute stress and CRF One essential differentiating characteristic of stress is its duration. Acute stress is defined as stress that is on the timescale of minutes to hours, whereas chronic stress is defined as stress that persists for several days, weeks, or even months. In preclinical research, acute stress can be modelled in various ways. Acute stress can be assayed in rodents using the following non-exhaustive list of stress manipulations: exposure to predator odour, restraint, forced swim, exposure to cold, social defeat, and exogenous CORT and/or epinephrine/norepinephrine-increasing injections. In opposition, chronic stress manipulations in preclinical animal models typically include chronic social defeat, where the animal is subjected daily to a dominant aggressor that attacks the animal into submission over days or weeks, or chronic unpredictable or restraint stress, where the animal is subjected to a number of different stressors that occur in an unpredictable order or is restrained for hours per day for days or weeks, respectively. Alternatively, acute stress in humans is typically manipulated using psychological stress such as public speaking (Trier Social Stress Test), physical stress such as placing one hand in cold water (cold pressor stress) or exogenous administration of stress hormones such as CORT. Studies regarding chronic stress in humans are usually correlative in nature, as chronic stress is typically not manipulated due to ethical concerns. Not only are there numerous stress manipulations but there are other factors that widely differ across studies, including variations in the manipulation itself, such as the duration of the stressor and the time between stress exposure and testing. Studies also vary by subject, such as the sex, age, species, or (rodent) strain used. It is unsurprising then that acute stress can produce complicated, sometimes opposing, results dependent on many different factors. 9 1.21 Learning and memory Memories are formed in three stages: 1) encoding involves taking in the information or learning, 2) the nascent memory trace is stabilized during consolidation, 3) the memory is retrieved. The memory trace can also be reactivated during memory retrieval and can re-enter an unstable state so that reconsolidation is required to re-stabilize the memory. Acute stress may interact with all of these processes, depending on when the stress occurs. The hippocampus, located within the medial temporal lobe, is a crucial brain node in the circuitry underling memory formation. The hippocampus ensures information transfer from short- to long-term memory and is involved in all stages of memory processing, including encoding, consolidation, retrieval, and reconsolidation. Not only is it vital for memory, the hippocampus provides substantial inhibition on the HPA axis (Ulrich-Lai & Herman, 2009). Furthermore, hippocampal neurons are particularly sensitive to stress hormones, with formative work by McEwen, de Kloet, and Sapolsky, and others finding that prolonged CORT exposure or stress causes neuronal atrophy in the hippocampus (de Kloet, Joëls, & Holsboer, 2005; McEwen & Milner, 2007; Sapolsky, 2003). The anatomically neighboring amygdala provides glutamatergic inputs to the hippocampus and is also important in memory processing (Roozendaal, McEwen, & Chattarji, 2009). However, work by Chattarji and colleagues found that stress increases, rather than reduces, dendritic growth and spine density in BLA pyramidal neurons (Govindarajan et al., 2006; Mitra et al., 2005; Vyas, Jadhav, & Chattarji, 2006). Therefore, both the hippocampus and BLA are important structures in understanding how acute stress alters learning and memory. Cross species studies involve elucidating the timing and mechanism by which acute stress manipulations alter different aspects of memory formation. As such, subjects are exposed to 10 acute stress either prior to encoding, indicating alterations in either encoding or consolidation, during consolidation, which would rule out alterations in encoding, or prior to retrieval, indicating perturbations in memory retrieval. Memory tasks in animals can involve object recognition, spatial memory for a hidden platform, or fear memory for a context or tone that was previously associated with aversive stimuli (typically foot shocks). In humans, memory tasks typically rely on declarative memory and involve remembering a series of stimuli, including lists of words. Exposure to acute stress or pharmacological stress in humans prior to encoding facilitates (Domes et al. 2002; Schwabe et al., 2008; Smeets et al., 2007) or impairs (Cazakoff, Johnson, & Howland, 2010; Diamond et al., 2006; Elzinga, Bakker, & Bremner, 2005; Kirschbaum et al., 1996) both short- and long-term memory encoding and/or consolidation. These complex effects are consistent with animal work showing that predator odour exposure stress immediately prior to encoding impaired spatial memory in a water maze task (Diamond et al., 2006) but acute stress prior to context-dependent fear conditioning enhances fear expression, which is dependent on global and hippocampal CRF signaling (Blank et al., 2002). Therefore, both short- and long-term memory show complex stress effects with studies finding an impairment or improvement depending on many factors. However, for the majority of studies it is difficult to distinguish between stress-induced alterations in encoding (i.e. learning) and consolidation. To rule out stress-induced alterations in encoding, subjects can be exposed to acute stress following encoding. In opposition to acute stress prior to encoding, decades of research indicates that acute stress or pharmacological administration of stress hormones shortly after encoding consistently enhances memory consolidation and subsequent retrieval across species (Andreano & Cahill, 2006; Beckner et al., 2006; Buchanan & Lovallo, 2001; Cahill, Gorski, & Le, 2003; 11 Cahill & McGaugh, 1998; Gold & Van Buskirk, 1975; Introini-Collison et al., 1992; Roozendaal & McGaugh, 1996; Roozendaal et al., 2006; Smeets et al., 2008). The stress-induced consolidation enhancement may be dependent on both increased CORT activity and emotional arousal, as acute stress (Cahill et al., 2003) or exogeneous CORT (Buchanan & Lovallo, 2001) administration immediately following encoding increased recall of emotionally-arousing, but not neutral, pictures. Perhaps the emotional arousal required for enhanced consolidation is mediated by the ANS system. In support, studies in rodents show that enhanced consolidation requires arousal-induced norepinephrine activation in the BLA as CORT administration immediately following exposure to novel object recognition increased memory for the object, which was blocked by a systemic or intra-BLA norepinephrine antagonist (Roozendaal et al., 2006). Increased consolidation is not dependent on valence, as aversive memories in an inhibitory-avoidance task are dose-dependently enhanced by exogeneous epinephrine administration when administered immediately post-training but prior to testing (Cahill & McGaugh, 1998; Gold & Van Buskirk, 1975; Introini-Collison et al., 1992; Roozendaal & McGaugh, 1996). Therefore, increased stress hormones shortly after encoding facilitate memory consolidation and subsequent retrieval. Conversely, acute or pharmacological stress immediately prior to retrieval or following retraining impairs memory retrieval and reconsolidation, respectively (Cai et al., 2006; Cazakoff et al., 2010; de Quervain et al., 2000; de Quervain et al., 1998; Kuhlmann, Piel, & Wolf, 2005; Maroun & Akirav, 2008; Roozendaal et al., 2003; Schwabe & Wolf, 2010; Smeets et al., 2008; Wang et al., 2008; Zhao et al., 2009). Stress-induced impairments may be due to CORT actions in the hippocampus as intra-hippocampus CORT agonist infusion prior to water maze spatial testing dose-dependently impairs memory retrieval (Roozendaal et al., 2003). Additionally, 12 stress-induced reconsolidation is mediated by GR activation in the BLA, as intra-BLA GR antagonist infusion blocks stress-induced impairments in reconsolidation in object recognition (Maroun & Akirav, 2008) or morphine-dependent conditioned place preference (Wang et al., 2008) paradigms. Taken together, this complicated pattern of effects indicates that, mediated by stress hormone actions within the hippocampus and BLA, acute stress impairs or enhances memory when experienced prior to encoding, but consistently facilitates and impairs memory when experienced immediately following encoding or prior to retrieval, respectively. One model attempts to explain these seemingly discrepant findings suggesting that these effects are due to the biphasic action of stress hormones, particularly CORT (Joëls, Pu, Wiegert, Oitzl, & Krugers, 2006). As stated in earlier sections, CORT actions are both rapid via non-genomic pathways or delayed via genomic pathways. Therefore, early stress responses, including CRF, norepinephrine, and rapid CORT actions favor the encoding of relevant information by increasing neuronal activity. On the other hand, delayed genomic CORT actions would reduce the processing of new information by suppressing neuronal activity. Thus, it is assumed that stress enhances memory when it is experienced in the context and in temporal proximity of encoding. However, memory is impaired when the stress occurs out of the encoding context or at greater temporal distance from encoding. Given the relatively rapid actions of CRF following acute stress, it is interesting to note that CRF actions mediate some aspects of stress-induced memory alterations and both the CRF peptide and its cognate receptors are expressed in brain regions that mediate memory processing, including the hippocampus and amygdala (Joëls & Baram, 2009; Tan et al., 2017; Van Pett et al., 2000). Indeed, CRF signaling is required for memory encoding and/or consolidation for both aversive and appetitive memories (Deak et al., 1999; Heinrichs et al., 1996; Heinrichs, 2003; 13 Hubbard et al., 2007; Radulovic et al., 1999). Specifically, CRF may mediate fear conditioning as intracerebroventricular (ICV) CRF antagonist administration prior to learning prevents fear expression 24 hours later (Deak et al., 1999), indicating that tonic CRF signaling is required for learning and/or memory consolidation in this paradigm. CRF acts within the BLA specifically, as nonselective or CRF1R-specific antagonist administration prior to fear conditioning (Hubbard et al., 2007) or post inhibitory avoidance training (Roozendaal et al., 2002) impairs fear expression 48 hours later. Perhaps CRF antagonist administration blocked fear conditioning by reducing aversion to foot shocks. However, this is unlikely given that the CRF antagonist did not alter latency to escape foot shock in a shuttle box escape test (Deak et al., 1999). CRF signaling is also required for short-term retention of appetitive memories, with ICV CRF antagonist administration immediately prior to social recognition learning reducing memory retention 30 or 120 mins post encoding (Heinrichs, 2003). Exogenous CRF treatment both centrally and within certain brain nuclei, such as the hippocampus and amygdala, also alter memory processing in a similar and divergent manner to acute stress manipulations. For instance, increased CRF signaling differentially alters long-term memory retention in a regionally-dependent manner, with memory retention enhanced by intra-hippocampus CRF infusion, but impaired by intra-lateral septum CRF infusion, immediately prior to tone- or context-dependent fear conditioning (Radulovic et al., 1999). Interestingly, increasing CRF signaling may induce learning deficits, as transgenic mice that globally overexpress CRF show deficits in learning on a spatial water maze task (Heinrichs et al., 1996). Taken collectively, these studies reveal that CRF signaling is required for memory encoding and/or consolidation and enhanced CRF signaling displays regionally specific effects that may 14 impair learning and/or memory when administered globally or within the lateral septum, but enhance memory within the hippocampus. Similar to acute stress, enhanced CRF activity, particularly within the BLA, facilitates memory consolidation regardless of whether the memory is appetitive or aversive, using contextual fear conditioning, inhibitory avoidance, or sexual reinforcement paradigms (Lee & Sung, 1989; Liang & Lee, 1988; Ma et al., 1999; Radulovic et al., 1999). However, CRF-induced memory enhancement is regionally-dependent, as ICV CRF infusion or CRF infusion into the hippocampus following tone- or context-dependent fear conditioning (Radulovic et al., 1999) or into the dentate gyrus of the hippocampus (Ma et al., 1999) following inhibitory avoidance training, increased fear expression, whereas CRF infusion into the lateral septum reduced fear expression 24 hours later (Radulovic et al., 1999). Collectively, these studies reveal that CRF activity is necessary for memory consolidation. Yet, exogenously enhancing CRF signaling alters memory in a regionally-specific manner, with increased CRF signaling globally, and within the BLA and hippocampus, enhancing memory consolidation regardless of whether the task is appetitive or aversive, whereas CRF signaling within the lateral septum impairs memory consolidation. Relatively less is known about how CRF immediately prior to testing affects memory retrieval and reconsolidation. One study, however, finds that intra-BLA CRF infusion one hour prior to testing increases fear expression 24 hours after fear conditioning (Isogawa, Bush, & Ledoux, 2013), indicating that, in opposition to acute stress, CRF signaling within the BLA prior to testing may facilitate memory retrieval. That said, intra-BLA CRF may specifically enhance fear expression, and not necessarily alter memory retrieval. Future studies should investigate the role of global CRF activity in memory retrieval and reconsolidation. 15 1.22 Attention Attention requires cognitive control mediated in large part by the PFC (Nyberg, 2018). Attention is typically assayed in humans using change or signal detection tasks that require participants to ignore irrelevant stimuli. In these tasks, participants respond for certain stimuli (go trials) and withhold response when stimuli is changed or absent (no-go trials). In rodents, the gold-standard for attention tasks is the 5-choice serial reaction time (5-CSRT) task. In this task, rodents are required to nose poke one of five, briefly illuminated, nose poke apertures, with shorter illumination periods requiring more attention. Attention scores for both human and rodent tasks are measured using the number (or percentage) correct detections or reaction time for correct detections. Early studies on stress and attention found that acute stress narrowed the focus of attention, with more perceptual errors occurring following the threat of electric shock (Kohn, 1954). Indeed, more recent studies find that acute stress impairs attention. For instance, exposure to acute stress prior to the task impaired attention, reducing accuracy and increasing reaction time (Olver et al., 2015; Sänger et al., 2014; Skosnik et al., 2000), which may be dependent on CORT, as CORT levels negatively correlated with attention scores (Bohnen et al., 1990; Skosnik et al., 2000). Both acute and chronic stress impair attention as self-reported stress during an exam period impaired both sustained and divided attention (Liston, McEwen, & Casey, 2009; Vedhara et al., 2000), indicating that attention processing is particularly susceptible to stress manipulations. Increased CRF signaling may mediate stress-induced attentional impairments as CRF infusion, like stress, impairs attention (Cole et al., 2016; Ohmura et al., 2009; Van’t Veer et al., 2012). Specifically, ICV CRF (0.25-1g) infusion reduced accuracy in the 5-CSRT test (Van’t 16 Veer et al., 2012) and reduced vigilance score on a sustained attention task that required rats to discriminate between signaled and non-signaled cues (Cole et al., 2016). CRF-induced attentional impairments are dose-dependent as very low ICV CRF doses (0.1g) enhance attention, whereas higher CRF doses (0.25-1g) consistently impair attention in the 5-CSRT test (Cole et al., 2016; Ohmura et al., 2009; Van’t Veer et al., 2012). Therefore, attentional impairments following stress manipulations and increased central CRF signaling are consistent and conserved across species, as numerous studies show that both manipulations impair the ability to maintain attention using a variety of behavioural tasks in both humans and rodents. 1.23 Working memory Working memory is an executive function reliant on the PFC to guide goal-directed behaviour (Arnsten, 2009). Not only does the PFC play a major role in coordinating the stress response, but neurons within the PFC are highly sensitive to the effects of acute stress and undergo substantial stress-induced remodelling of dendritic processes (Cerqueira et al., 2007; Holmes & Wellman, 2009; Liston et al., 2009). As such, PFC-dependent executive functions are exquisitely sensitive to acute stress and alter working memory processes (Holmes & Wellman, 2009; Shields et al., 2016; Yuen et al., 2009). Working memory is the ability to temporarily hold and manipulate information. The most common working memory task in humans is the reading span task in which participants read a collection of sentences out loud and are instructed to memorize the last word of each sentence. Sentences are added until participants are unable to recall the last word of each sentence, with participants scored on the number of sentences remembered. Another popular working memory task used in human studies is the n-back task in which participants are given a string of numbers and instructed to respond yes or no if the presented stimuli is the same as one presented n-trials 17 prior. Working memory load is manipulated by increasing the n to 2-back or 3-back and percentage correct responses are used as the measure of working memory. In rodents, working memory is typically measured using variants of a delayed alternation task. In one version rodents run a radial arm maze with half of the arms baited with reward. Following a delay, rodents run the maze a second time but on this test trial the arms that were previously non-baited now contain reward and the previously baited arms contain no reward. The T-maze version is similar but only one arm of the T-maze is baited with food reward and on each trial the rodent is required to enter the opposite arm that was previously baited. Working memory is measured by the number of errors made (i.e. entering arms that were baited on the previous trial). The vast majority of cross-species studies find that acute, chronic, or pharmacological stress impairs working memory (Butts et al., 2011; Del Arco et al., 2007; Devilbiss et al., 2012; Diamond et al., 1996; Luethi et al., 2009; Mizoguchi et al., 2000; Davies et al., 2013; Olver et al., 2015; Schoofs et al., 2008; Shansky et al., 2006; Shields et al., 2015; Shields et al., 2016). However, some have found working memory improvements (Duncko et al., 2009; Yuen et al., 2009). One reason for this discrepancy could be that working memory enhancements are found when the stressor is relatively mild (Yuen et al., 2009) or does not strongly activate the HPA axis (Duncko et al., 2009), implicating that increased CORT levels are responsible for impaired working memory. Indeed, working memory impairments may be CORT-dependent as acute or chronic systemic or intra-PFC CORT treatment impairs working memory (Bardgett et al., 1994; Roozendaal et al., 2004), likely due to stress- and CORT-induced dendritic retraction of PFC neurons (Cerqueira et al., 2007; Wellman, 2001). The PFC is also an important region for CRF-induced alterations in working memory. Although not as thoroughly investigated, emerging studies by Craig Berridge and group indicate 18 that ICV CRF infusion dose-dependently impairs working memory in rodents on a spatial working memory task (Hupalo & Berridge, 2016). Similar to the effects of CORT, CRF-induced impairments are likely due to actions within the PFC. Indeed, CRF and CRF1R are both expressed throughout the PFC (Swanson, Sawchenko, Rivier, et al., 1983; Van Pett et al., 2000) and intra-PFC CRF infusion mimics ICV CRF impairments in working memory, specifically within the caudal dorsomedial PFC (Hupalo & Berridge, 2016). Furthermore, chemogenetic activation of CRF-expressing neurons in the PFC similarly impairs working memory dependent on local CRF activity, as intra-PFC CRF receptor antagonism attenuates this impairment (Hupalo et al., 2019). Given the similarities between acute, chronic, and pharmacological stress on attention and working memory, PFC-related cognition appears particularly sensitive to the impairing effects of stress. As such, both CORT and CRF signaling globally and within the PFC similarly impair working memory function. 1.24 Cognitive flexibility Cognitive flexibility, like working memory, is an executive function mediated by the PFC, particularly within medial (mPFC) and orbital (OFC) subregions (Birrell & Brown, 2000; Floresco, Block, & Tse, 2008; Ghods-Sharifi, Haluk, & Floresco, 2008; McAlonan & Brown, 2003). Cognitive flexibility is the ability to flexibly adapt to changes in the environment and is underpinned by various underlying cognitive processes, such as learning, memory, attention, and working memory. Cognitive flexibility can be divided into two, relatively distinct, subcategories that are mediated by independent brain circuitries: OFC-dependent reversal learning (McAlonan & Brown, 2003) and mPFC-dependent set-shifting (Floresco, Block, & Tse, 2008). Stimulus-response reversal learning is measured on a variety of tasks but involves discriminating between two stimuli, one of which is reinforced. When subjects meet a certain discrimination criteria, the 19 stimulus-reward contingencies are reversed so that the previously rewarded stimuli is now not rewarded and vice versa. Alternatively, extra-dimensional set-shifting involves discriminating between stimuli based on one task dimension, with one stimuli response reinforced. Following discrimination to criteria, the task shifts so that responses are reinforced on a separate task dimension (i.e. from correct responding to the position of a cue light over a lever to correct responding to a spatial position independent of the cue). The type of stimulus can vary between and within species, with odour, visual, auditory, and response set (i.e. left vs right), stimuli dimensions typical for cognitive flexibility tasks. Furthermore, reversal learning tasks can be reinforced on a deterministic schedule, where every correct response is rewarded, or a probabilistic schedule, where the majority of ‘correct’ responses are rewarded (typically 80%) but a minority of ‘incorrect’ responses are also rewarded (typically 20%). The ability to switch within or between stimulus dimensions rely on distinct neural circuitries. The OFC is a critical brain region for both deterministic and probabilistic reversal learning (PRL) (Dalton et al., 2016; Ghods-Sharifi, Haluk, & Floresco, 2008; McAlonan & Brown, 2003), as the OFC keeps track of current incentive value of reward-paired stimuli (Bechara, Damasio, & Damasio, 2000; Clark, Cools, & Robbins, 2004). Conversely, set-shifting does not involve the OFC (Ghods-Sharifi et al., 2008; McAlonan & Brown, 2003), but rather is mediated in part by the mPFC (Birrell & Brown, 2000; Floresco, Block, & Tse, 2008). Subcortical regions are also differentially involved in cognitive flexibility tasks as PRL is mediated in part by the NAc shell (Dalton, Phillips, & Floresco, 2014), with this subregion involved in guiding behaviour under reward uncertainty (Floresco, 2015), whereas set-shifting relies on the NAc core (Floresco et al., 2006), and deterministic reversal learning involves the dorsal medial striatum (Castañé et al., 2010). 20 Acute stress effects on cognitive flexibility are similarly complex, impairing or facilitating flexibility depending on myriad factors (Hurtubise & Howland, 2017). For instance, in male rodents, 30 mins of restraint or elevated platform stress, or 10 mins of forced swim stress prior to testing facilitates deterministic reversal learning on a variety of tasks (Bryce & Howland, 2015; Dong et al., 2013; Graybeal et al., 2011; Thai, Zhang, & Howland, 2013). However, 15 mins of tail pinch stress in the same context as behavioural testing had no effect on reversal learning (Butts et al., 2013). Importantly, acute stress-induced facilitation of reversal learning does not appear mediated by CORT, as systemic GR or MR antagonist administration prior to stress has no effect on performance (Bryce & Howland, 2015; Thai et al., 2013). In contrast, 15 mins of in-context tail pinch stress prior to testing impaired set-shifting (Butts et al., 2013), but 30 mins of restraint stress prior to testing was without effect on set-shifting (Thai et al., 2013). Inconsistent effects of acute stress on flexibility are difficult to explain given the differences in stressor duration, timing, intensity, and context, among other factors (Hurtubise & Howland, 2017). Furthermore, these tasks are intrinsically different in terms of difficulty, with set-shifting more cognitively demanding than deterministic reversal learning. Thus, stress may enhance, impair, or show null effects depending on cognitive load among myriad other factors. If stress becomes more prolonged or chronic, however, cognitive flexibility becomes invariably impaired on both reversal learning and set-shifting tasks (Bondi et al., 2008, 2010; Hurtubise & Howland, 2016; Jett et al., 2015; Jett et al., 2017; Lapiz-Bluhm et al., 2009). Acute stress effects on cognitive flexibility are also multifaceted in humans. The gold standard for cognitive flexibility tasks in humans is the Wisconsin Card Sorting Test, which is similar to set-shifting in that participants are asked to sort cords based on an unknown dimension (colour, quantity, shape). Once learned, the matching dimension is changed (i.e. from colour to 21 quantity). Using this task, researchers found that psychosocial stress enhanced (Gabrys et al., 2019) or impaired (Shields et al., 2016) cognitive flexibility, reducing or increasing the number of perseverative errors, respectively. Stress-induced CORT levels negatively correlated with perseverative errors when cognitive flexibility was enhanced but not impaired (Gabrys et al., 2019; Shields et al., 2016), indicating that increased CORT may enhance flexibility under certain conditions. This is hard to reconcile given a meta-analysis of CORT administration on cognitive flexibility was without effect (Shields et al., 2015), suggesting that increased CORT does not induce a reliable effect on this form of cognitive flexibility. The impairing effects of acute stress may instead rely on the ANS as systemic norepinephrine antagonist administration blocks the stress-impairing effects on cognitive flexibility (Alexander et al., 2007). These conflicting results may also be due to sex differences. Indeed, psychosocial stress impaired cognitive flexibility in men, but not women (Shields et al., 2016) and in preclinical studies, although the majority of studies used exclusively male rodents, one study found that male rodents are more sensitive than females to the impairing effects of acute stress on cognitive flexibility (Laredo et al., 2015). That said, potential sex differences in acute stress effects on cognitive flexibility have yet to be explicitly tested in preclinical models. Furthermore, no clinical or preclinical studies to date have assessed how acute stress alters reversal learning when the outcomes are probabilistic. Chapter 2 of this thesis will address these gaps in knowledge, specifically interrogating the role of acute stress on cognitive flexibility in both male and female rats on a PRL task that has more recently been back-translated from a human PRL task for use in preclinical animal models (Bari et al., 2010; Dalton et al., 2014). Given the lack of effect of GR/MR antagonists and inconsistent correlation with CORT levels, this thesis will also investigate underlying mechanisms of the stress response that may be 22 driving alterations in flexibility. One candidate mechanism is CRF, as work by Rita Valentino’s lab reveals that increased central CRF signaling impaired cognitive flexibility, with very low ICV CRF doses (0.03g) impairing set-shifting and relatively higher CRF doses (0.1g) impairing reversal learning (Snyder et al., 2011). Cognitive flexibility is mediated in part by the norepinephrine system (Lapiz & Morilak, 2006) and CRF acts in the LC to increase activity of norepinephrine-expressing neurons (Curtis et al., 1997). Interestingly, low dose CRF infusion (0.002g) into the LC that moderately increased neuronal discharge and activated the mPFC facilitated set-shifting with no effect on reversal learning, whereas higher dose intra-LC infusion (0.02g) that maximally increased neuronal discharge and did not activate the PFC was without effect on set-shifting but facilitated reversal learning (Snyder et al., 2011). Work by the Valentino group also shows that CRF interacts with serotonin (5-HT) in the dorsal raphè nucleus (DRN) to alter cognitive flexibility. Specifically, intra-DRN CRF infusion (0.03g) at a dose that reduced 5-HT levels in the mPFC improved set-shifting but had no effect on reversal learning (Snyder et al., 2015). Together, these results indicate that increasing LC or reducing DRN activity and subsequent NE potentiation or 5-HT suppression of terminal release into the mPFC improves set-shifting but not reversal learning, which is not dependent on the mPFC (Floresco et al., 2008). This may also be why acute stress exerts complex effects on cognitive flexibility, as it depends on the regional activity of CRF and the degree of NE and/or 5-HT activation and terminal release. 1.25 Motivation and reward/negative feedback processing Motivation is a fundamental component to consider when investigating the role of stress in cognitive functioning as any perceived deficit in learning, memory, or attention may be due to an underlying motivational impairment. Motivation is typically understood in terms of appetitive 23 motivation for reward. Although the concepts of motivation and reward are interlinked, Kent Berridge’s group has been instrumental in delineating motivation for reward or ‘wanting’ from reward processing or ‘liking’ the reward itself (Berridge & Robinson, 2003), with stress disrupting both of these processes (Kumar et al., 2014). An early correlative study found that stress alters reward processing as U.S. Army Cadets reported a reduced ability to experience pleasure following a stressful exam period (Berenbaum & Connelly, 1993). More recent experimental evidence found that acute stress directly reduced the sweet taste of rewards (Al’absi et al., 2012) and reduced activation in regions known to process reward such as the dorsal striatum, amygdala, and cingulate cortex when presented with primary food reward (Born et al., 2010). Moreover, in rodents, chronic stress reduced reward preference in a sucrose preference test (Hollon, Burgeno, & Phillips, 2015; Krishnan et al., 2007; Papp, Willner, & Muscat, 1991; Tye et al., 2013). Instrumental behaviour is similarly altered by stress. Using a signal detection task, where participants identify differentially rewarded stimuli, researchers found that threat-of-shock stress reduced monetary reward responsiveness (Bogdan & Pizzagalli, 2006) and sensitivity to monetary reward feedback in a probabilistic selection task (Berghorst et al., 2013), perhaps by reducing activation of reward-related brain regions, including the dorsal striatum, mPFC, and OFC, in response to monetary reward feedback (Porcelli, Lewis, & Delgado, 2012; Treadway et al., 2013) or anticipation (Kinner, Wolf, & Merz, 2016). In rodents, acute stress reduced motivation to work for reward on a progressive ratio task, where exponentially increasing lever presses are required for one sugar pellet reward (Wanat et al., 2013) and reduced conditioned place preference for drug reward (Papp et al., 1991). However, separating motivation from reward processing is difficult as both processes are required for these tasks. That said, using an 24 effort-related decision making task, our group found that acute restraint stress reduced preference for larger vs smaller rewards, but only when the larger reward required more effort to obtain and not when the effort costs were equal (Shafiei, Gray, Viau, & Floresco, 2012). Taken collectively, this research indicates that stress reduces reward responsiveness and motivation across species and tasks. Although stress may reduce motivation and reward processing for primary (i.e. food) and secondary (i.e. monetary) reward, stress increases motivation for drug reward, enhancing reinstatement of drug seeking in animals with a history of self-administering drugs of abuse (Koob & Kreek, 2007; Marinelli & Piazza, 2002), which depends on CRF signaling (Blacktop et al., 2011; Erb et al., 2006; Koob, 2010; Shaham et al., 1998; Shalev, Erb, & Shaham, 2010). However, drug experience hijacks the natural reward pathway, so enhanced motivation for drug reward may be due to drug-dependent alterations in reward processing (Kreek & Koob, 1998). Although severe stressors can cause a reduction in reward sensitivity and motivation (Berghorst et al., 2013; Bogdan & Pizzagalli, 2006; Wanat et al., 2013), relatively mild stressors can enhance motivation for highly palatable food. Intuitively, this makes sense when viewing the role of stress in overeating and obesity. Indeed, exposure to a mild acute stressor increased the consumption of foods with high fat or sugar content (Lemmens et al., 2011; Oliver, Wardle, & Gibson, 2000; Rutters et al., 2009), by enhancing ‘wanting’ but not ‘liking’ of food rewards (Lemmens et al., 2011) and increasing activity in reward-related brain regions (Serfling et al., 2019). Similarly, in preclinical studies, pharmacological stress with the norepinephrine-enhancing drug yohimbine increased reinstatement of food seeking for sugar reward (Calu et al., 2013; Cifani et al., 2012; Liu, 2015), which also depends on CRF signaling (Ghitza et al., 2006; Liu, 2015). Together these studies indicate that mild stress may act via the CRF/norepinephrine 25 system to enhance motivation for highly salient rewards such as drugs of abuse or high fat or sugar rewards. Appetitive motivation is typically thought of as approaching reward. Yet, motivation can also be understood as the avoidance of aversive stimuli or experiences. Relatively less is known about this type of motivation. However, there are some indications that acute stress also reduces aversive motivation. In humans, acute stress reduced the use of negative feedback in a probabilistic selection task where participants were required to use negative feedback to correctly avoid a stimuli that was comparatively less rewarding (Petzold et al., 2010), perhaps due to stress-induced dampening of mPFC activity during feedback about monetary loss (Treadway et al., 2013). Similarly, using a conditioned taste aversion task, where animals were given a nausea-inducing agent to reduce subsequent preference for sucrose solution, researchers found that forced swim or foot shock stress prior to and following the aversive compound reduced conditioned taste aversion (Bourne et al., 1992; Revusky & Reilly, 1989), indicating that acute stress may reduce learning about aversive stimuli that should be avoided. Therefore, acute stress can reduce motivation and reinforcement sensitivity regardless of valence. However, acute stress can also enhance reward sensitivity and motivation for highly palatable rewards and drugs of abuse, perhaps by hijacking the natural reward pathway. Pioneering studies by Wylie Vale, George Koob, and others reveal that CRF exerts complex effects on appetitive and aversive motivation independent of HPA axis activation. For instance, low dose central CRF infusion (0.5g) induced conditioned place preference or aversion, increasing or decreasing time spent in the CRF-paired chamber, respectively (Cador et al., 1992; Heinrichs, Britton, & Koob, 1991), whereas high dose central CRF infusion (5g) consistently induced conditioned place aversion (Heinrichs et al., 1991). That said, most CRF 26 doses reduce operant responding for food on fixed and variable intervals and reduce spontaneous food intake in food-deprived animals (Britton et al., 1982; Britton et al., 1986; Glowa et al., 1992; Jones et al., 1998). Furthermore, work by our group and others has shown that CRF signaling is necessary for stress-induced motivational deficits as intra-VTA CRF antagonist infusion blocked— and central and intra-VTA exogeneous CRF infusion mimicked— the acute stress-induced reduction in motivation to work for sugar pellet reward on a progressive ratio schedule of reinforcement (Bryce & Floresco, 2016; Wanat et al., 2013). Interestingly, humans that are homozygous for the A allele single nucleotide polymorphism in the CRF1R gene, associated with enhanced CRF1R expression and increased risk for depression, demonstrated reduced reward response bias and suppressed mPFC and OFC activation in response to reward following acute stress compared to participants that expressed the G allele (Bogdan et al., 2011), indicating that stress-induced alterations in reward processing in humans may also be due to the CRF system. Reduced reward motivation may be regionally dependent, as NAc CRF induced conditioned place preference, increased exploration of a novel object (Lemos et al., 2012), accelerated social bond formation (Lim et al., 2007), and enhanced incentive motivation for reward cues (Peciña, Schulkin, & Berridge, 2006). NAc shell CRF also reduced sucrose preference and induced immobility in a forced swim test (Chen et al., 2012). However, severe stress switches CRF action in the NAc from the appetitive to aversive (Lemos et al., 2012). Therefore, forced swim stress in the previous task may have altered the valence of CRF in this experiment. Collectively, this body of work reveals that CRF is well-poised to mediate the alterations in stress-induced motivational impairments. Increased CRF signaling modulates motivation and reward processing, in a circuit-specific manner, and increasing CRF activity 27 within certain nodes may induce differential effects on motivation compared to acute stress or global increases in CRF. CRF most likely alters these processes via interactions with the mesolimbic dopamine system given CRF within dopamine cell body and terminal regions alters motivation and reward and dopamine plays an crucial role in these processes (Wise, 2004). 1.26 Decision making Acute stress and CRF modulate complex evaluative processes involving assessments of costs and benefits to optimize rewards of greater subjective value. Costs can differ in terms of the length of the delay, probability of receiving reward, and amount of effort required, among other factors. Considerable advances have been made in delineating the neural circuits mediating cost/benefit decision-making. Certain nodes within this circuitry appear essential for decision making involving a variety of costs and may allow organisms to overcome costs to maximize overall benefits. As such, signaling within the NAc, amygdala, and PFC are required to overcome uncertain, delayed, or more effortful costs (Floresco et al., 2008, 2009; Winstanley & Floresco, 2016). Relatively less is known about how stress influences decision making. More recent findings suggest that the effect of stress on decision making depends in part on the type of cost involved. For example, a meta-analytic study in humans revealed that acute stress reduced choice of larger rewards associated with a longer delay (Fields et al., 2014), most prominently in those that showed increased CORT (Kimura et al., 2013). However, our group has shown that neither acute restraint stress nor CORT administration altered delay discounting in rats (Shafiei et al., 2012), indicating this effect may not generalize across species. In contrast, acute, chronic, or pharmacological stress increased choice of the risky, disadvantageous option in various tasks where the larger rewards were delivered in a probabilistic manner (Koot et al., 2013; Morgado et 28 al., 2015; Nobrega et al., 2016; Preston et al., 2007; Putman et al., 2010; Simonovic et al., 2017; Starcke et al., 2008; Wemm & Wulfert, 2017; however see Gathmann et al., 2014). However, the majority of these studies are conducted in humans using a variety of tasks that use external cues to guide decisions or require participants to use feedback to internally generate risk/reward probabilities to guide subsequent action. Given the dearth of preclinical research, Chapter 3 of this thesis will address how acute stress and CRF modulate different forms of decision-making under reward uncertainty and elucidate how internal vs external information pertaining to risk/reward contingencies alter such effects. The ability to overcome effort costs in the pursuit of larger or ‘better’ rewards is critically dependent on mesolimbic dopamine signaling, particularly in the NAc. Early effort-related decision making studies used T-maze tasks that required rats to choose between one arm that was baited with a small reward and the other arm that required the rat to scale a physical barrier to obtain a larger reward. Operant tasks of effort-related decision making were then developed that required animals to choose between rat chow that was freely available or lever pressing on a fixed schedule for more palatable sugar reward. Seminal work by John Salamone and group demonstrated that systemic dopamine receptor blockade with haloperidol or intra-NAc dopamine antagonism or depletion diminished choice of larger, more costly rewards in the T-maze and concurrent choice effort tasks (Cousins & Salamone, 1994; Floresco, Tse, & Ghods-Sharifi, 2008; Nowend et al., 2001; Salamone et al., 1991, 1994). More recent operant tasks have been utilized by our group where rats choose between a low effort/low reward lever (2 sugar pellets), and a high effort/high reward lever (4 sugar pellets), with the effort requirement increasing over blocks of trials (2, 5, 10, and 20 presses). Using this task, our group found that increasing dopamine tone via systemic amphetamine administration has a biphasic effect on effort choice, 29 with low doses increasing and high doses reducing choice of the larger, more effortful option (Floresco et al., 2008). Decisions involving choices between easily obtained, less valuable rewards vs. more valuable ones that require greater effort to obtain appear to be particularly sensitive to stress manipulations. For example, acute restraint stress markedly shifted choice away from large rewards requiring more effort to obtain and increased decision latencies, yet this is not mediated by glucocorticoids, as CORT administration did not recapitulate these effects (Shafiei et al., 2012). Instead, CRF antagonism blocked restraint stress-induced reduction in effort choice, demonstrating that stress-related increases in CRF diminish preference for larger rewards associated with a greater effort cost (Bryce & Floresco, 2016). Parallel to acute stress, exogeneous central CRF infusion markedly shifted choice away from larger rewards, but only when these rewards required more effort, as increased CRF activity had no effect on choice for larger vs smaller rewards of equal costs (Bryce & Floresco, 2016). Thus, rather than reducing the incentive value of larger rewards, increased CRF transmission may amplify the perceived effort costs required to obtain them. Consistent with this hypothesis, increased CRF activity reduced responding for rewards delivered on a progressive ratio schedule of reinforcement (Bryce & Floresco, 2016; Wanat et al., 2013). Interestingly, central CRF increased deliberation time irrespective of differential effort costs in a similar manner to acute stress (Bryce & Floresco, 2016; Shafiei et al., 2012), indicating that the ability of acute stress to induce ‘indecisiveness’ and increase processing times for action selection is also mediated in part by increased central CRF transmission. One key difference was that central CRF, but not acute stress, increased the number of trials that were omitted, but only when the larger reward required more effort (Bryce 30 & Floresco, 2016; Shafiei et al., 2012), suggesting that CRF may reduce the motivation to perform the task when access to the more preferred reward is impeded. Seeking to identify the neural loci where CRF may be acting to alter motivation, we targeted the midbrain dopamine neurons in the VTA, given the well-established role that dopamine plays in promoting preference of the larger, costlier rewards (Cousins & Salamone, 1994; Floresco, Tse, & Ghods-Sharifi, 2008; Nowend et al., 2001; Salamone et al., 1991, 1994). Furthermore, CRF is released in the VTA during episodes of acute stress (Wang et al., 2005) and stress-induced perturbations in motivation are mediated by activation of CRF receptors within the VTA (Wanat et al., 2013). As such, our group and others have found that intra-VTA CRF shifts choice away from larger, more effortful rewards and reduces instrumental responding on a progressive ratio, similar to acute stress and central CRF infusion (Wanat et al., 2013; Bryce and Floresco, 2016). However, intra-VTA CRF does not completely recapitulate the effects of central CRF or acute stress in that intra-VTA CRF does not increase choice latencies. It is notable that effects of acute stress on choice latency appear to be mediated by enhanced dopamine transmission, as increased choice latencies following acute stress was blocked by dopamine antagonist administration (Shafiei et al., 2012). Although the terminal region(s) where enhanced dopamine may mediate alterations in choice latencies is currently unknown, stimulation of D2 receptors in the PFC or BLA increased choice latencies during other types of cost/benefit decision making (Larkin, Jenni, & Floresco, 2016; St Onge, Abhari, & Floresco, 2011). Additionally, acute stress and central CRF increased extracellular dopamine signaling in the NAc and PFC (Abercrombie et al., 1989; Dunn & Berridge, 1990; Dunn, 1988; Dunn & Berridge, 1987; Holly, Debold, & Miczek, 2015; Matsuzaki, Takamatsu, & Moroji, 1989; Puglisi-Allegra et al., 1991), whereas intra-VTA CRF infusion reduced PFC but increased NAc dopamine 31 signaling (Kalivas, Duffy, & Latimer, 1987). Therefore, stress-induced increases in CRF transmission may impede decision processing times by enhancing terminal release via mechanisms independent of its actions within the VTA. These observations highlight that stress acts via CRF signaling to modulate motivational and decision-making functions in a circuit-specific manner and that increasing CRF activity in certain nodes of these circuits may induce differential effects on these behaviours compared to acute stress or global increases in CRF. Superficially, the effects of CRF on effort choice appear to mimic a reduction in dopamine tone (Bryce & Floresco, 2016; Cousins & Salamone, 1994; Floresco et al., 2008; Nowend et al., 2001; Salamone et al., 1991, 1994). However, excessive increases in dopamine activity can also reduce preference for larger rewards associated with a greater effort cost, as systemic administration of higher doses of amphetamine (Floresco et al., 2008), which increase dopamine tone, and overexpression of striatal D2 receptors (Filla et al., 2018), shift preference away from larger, more costly rewards during effort-related decision making. Given the crucial role of NAc dopamine in overcoming effort costs (Cousins & Salamone, 1994; Nowend et al., 2001; Salamone et al., 1994; Salamone et al., 1991), it is notable that no study to date has delineated the receptor-specific role of increased dopamine activity in the NAc. Additionally, stress is one condition that can enhance dopamine activity, and this is modulated in part by CRF (Lemos et al., 2012). As such, Chapter 5 will elucidate how increasing activity at dopamine D1, D2, and D3 receptors, and CRF activity within different subregions of the NAc alters effort-related decision making. 1.3 Physiological interactions between CRF and dopamine To varying degrees, dopamine activity modulates all functions discussed in detail in the above sections, including learning and memory (Puig, Antzoulatos, & Miller, 2014), attention 32 (Nieoullon, 2002), working memory (D’Esposito & Postle, 2015), cognitive flexibility (Klanker et al., 2013), motivation (Wise, 2004), and decision-making (Floresco et al., 2008). Furthermore, acute stress alters dopamine functioning, increasing VTA dopamine activity (Valenti, Lodge, & Grace, 2011) and subsequent terminal release (Holly et al., 2015). Mounting evidence suggests that CRF mediates the relationship between stress and dopamine signaling (Holly et al., 2016; Wanat et al., 2013; Wang, 2005). This idea is supported by anatomical observations demonstrating strong interactions between dopamine and CRF systems (Kelly & Fudge, 2018). In rodents, the VTA produces local CRF (Grieder et al., 2014) and receives CRF innervation from the lateral BNST, CeA, and PVN (Dabrowska et al., 2016; Rodaros et al., 2007), and both CRF1Rs and CRF2s are localized within the dopamine cell body region of this structure (Tan et al., 2017; Van Pett et al., 2000). Moreover, the NAc contains local CRF cell bodies (Merchenthaler, 1984; Swanson et al., 1983) and fibers that may originate from the thalamic paraventricular nucleus, BNST, BLA, and mPFC (Itoga et al., 2019). Tyrosine-hydroxylase-immunoreactivity co-localizes with CRF fibers and both CRF1R and CRF2R subtypes in the NAc (Lemos et al., 2012), suggesting that CRF is well poised to modulate dopamine terminal release. A survey of the literature on CRF-dopamine interactions reveal a complex relationship between these systems. For example, in vitro studies demonstrate that CRF can excite VTA dopamine neurons (Korotkova et al., 2006), although as has been observed with other monoamine systems, these effects show bimodal, dose-dependent actions with higher concentrations reducing excitatory currents on dopamine cells (Williams, Buchta, & Riegel, 2014). Conversely, CRF can increase inhibitory currents driven by D2 and GABAB receptors on dopamine neurons (Beckstead et al., 2009) and increase firing of VTA GABA neurons 33 (Korotkova et al., 2006), making it unclear if the net effect of VTA CRF receptor activation is an increase or decrease in dopamine neural firing and terminal release. Indeed, other in vivo studies suggest that CRF may exert an inhibitory tone on VTA dopamine neurons. This is supported by observations that central administration of a CRF1 receptor antagonist increases the number of spontaneously active dopamine neurons (Lodge & Grace, 2005), which drives slower, extrasynaptic or “tonic” dopamine levels within the NAc (Floresco et al., 2003). With respect to CRF effects on different modes of dopamine release, microdialysis studies demonstrated that centrally-administered CRF increased tonic dopamine release in the NAc (Dunn & Berridge, 1987; Kalivas & Duffy, 1995; Matsuzaki et al., 1989), and blockade of VTA CRF2Rs attenuate increased mesoaccumbens dopamine release induced by social defeat stress (Holly et al., 2015). Moreover, administration of CRF on NAc slices in vitro potentiated evoked dopamine release, suggesting that CRF may facilitate dopamine transmission via local mechanisms within this nucleus (Lemos et al., 2012). On the other hand, intra-VTA CRF reduced phasic accumbal dopamine release evoked by stimulation of PPTg inputs to the VTA and attenuated reward-associated phasic dopamine release in rats working for food reward on a progressive ratio (Wanat et al., 2013). These studies illuminate the complexity of CRF actions on dopamine neuron physiology and release, which may arise from different experimental conditions or suggest that CRF differentially shapes tonic vs phasic dopamine activity. With these issues in mind, Chapter 4 of this Thesis will address the role of increased central CRF signaling on VTA dopamine neuron physiology in vivo, with the specific aim to clarify interplay between CRF and dopamine on behavioural outcomes. 34 1.4 Sex differences 1.41 Sex differences in the stress response Both sexes show similar patterns of HPA axis activation in response to acute stress. That said, animal studies consistently find that females show a greater physiological stress response than males as noted by elevated CORT levels after exposure to a range of stressors (Handa et al., 1994; Iwasaki-Sekino et al., 2009). This fits with the differential role of sex hormones in altering HPA axis responses, with testosterone suppressing, and estrogen enhancing, circulating CORT levels (Brownlee et al., 2005; Plager et al., 1964). Female sex hormones cycle, and this cyclicity can result in differential responses to stress depending on cycle stage, with stages that are characterized by high levels of estrogen increasing stress reactivity compared to stages that are characterized by low estrogen. Estrogens also regulate CRF gene and promoter activity by increasing CRF gene expression in the PVN (Ochedalski et al., 2007), pointing to the role of gonadal hormones in mediating the sex differences in HPA axis activity. In humans, however, there is either a slight increase in male CORT levels following acute stress compared to females, or no appreciable sex difference in the stress-induced CORT response (Kirschbaum et al., 1999; Uhart et al., 2006), likely due to differences in oral contraception, age, medications, and other factors not readily controllable in human studies. Females are also more sensitive to CRF than males (Bangasser et al., 2010; Wiersielis et al., 2016). Indeed, females display increased CRF gene expression in the PVN and CeA in response to acute stress (Iwasaki-Sekino et al., 2009) and behaviourally, females show increased CRF-induced grooming behaviour compared to males (Wiersielis et al., 2016). Outside of the HPA axis, exogenous central CRF infusion activates different neural circuitries, with females showing more neuronal activation in the periaqueductal grey and BNST, whereas males 35 demonstrate greater activation in the mPFC, amygdala, and DR (Salvatore et al., 2018). Female rats also demonstrate greater coupling of Gs to the CRF1 receptor than male rats, indicating that female CRF1Rs in the LC would signal primarily via the cAMP/protein kinase A pathway (Bangasser et al., 2010). Moreover, CRF1R internalizes following stress in males, whereas for females CRF1Rs become more prominent on the plasma membrane following stress (Bangasser et al., 2010). Increased CRF1Rs on the plasma membrane coupled to Gs proteins would increase the excitability of neurons following stress, indicating that females may be more sensitive to the effects of stress and CRF administration due to enhanced CRF1R expression. Enhanced stress responsivity may also be why females are more susceptible to stress-related disorders than males (Bangasser & Valentino, 2014). 1.42 Baseline sex differences in cognitive functioning Meaningful sex differences in executive functions are largely absent in both human and animal studies. Indeed, cross-species studies reveal no sex differences in attention in humans (Giambra & Quilter, 1989; Pletzer, Harris, & Ortner, 2017) or rodents (Cole et al., 2016; Papaleo et al., 2012). However, performance of females was impaired compared to males with increasing task difficulty (Bayless et al., 2012; Jentsch & Taylor, 2003) and females typically displayed slower reaction times overall (Giambra & Quilter, 1989; Papaleo et al., 2012; Pletzer et al., 2017). Similarly, there are limited sex effects on working memory. In humans, conflicting evidence shows a female or male advantage depending on what task of working memory is employed, with females performing better on spatial working memory tasks (Duff & Hampson, 2001; Grissom & Reyes, 2019; Rahman, Abrahams, & Jussab, 2005; Voyer, Voyer, & Saint-Aubin, 2017). On the other hand, in animal studies males performed better in spatial working 36 memory tasks (Gibbs & Johnson, 2008; Grissom & Reyes, 2019; Seymoure, Dou, & Juraska, 1996) or there were no sex differences on working memory (Bimonte & Denenberg, 2000; West, Maynard, & Leasure, 2018). Some have interpreted these conflicting results to suggest that if any sex differences do exist they might depend on the type of task used and strategy required, with females typically using a more allocentric approach in spatial working memory tasks which would give them an advantageous in certain tasks over males (Grissom & Reyes, 2019). Sex differences in decision making ability also depend on numerous factors including the type of cost involved. Several studies reveal no sex differences when there is a longer delay for the more rewarding option in a delay discounting task (Cross et al., 2011; Weafer et al., 2015). In terms of effort costs, one study in humans finds that females prefer the less rewarding option that requires less physical effort to obtain compared to males in an effort-related decision making task (Treadway et al., 2009). Although sex differences were not explicitly tested, female rats shifted preference toward the more effortful option following ovariectomy, indicating female sex hormones may reduce effort choice (Uban et al., 2012). In regards to uncertainty or risk in decision making, human studies find that either males tend to outperform females (Overman & Pierce, 2013; van den Bos, Homberg, & de Visser, 2013) or there are no sex differences (Kim et al., 2015; Van Der Plas et al., 2009). While there are no apparent sex differences when the task involves reward omission (Ashenhurst, Seaman, & David Jentsch, 2012; Doi, Nishitani, & Shinohara, 2015; Grissom & Reyes, 2019; Orsini & Setlow, 2017; Sidlauskaite et al., 2018), females will avoid the option associated with frequent explicit punishment such as foot shock, unpalatable food, or monetary loss, as females are more sensitive to punishment than males (Orsini & Setlow, 2017; Orsini et al., 2016; van den Bos et al., 2012). Therefore, studies that use punishment find that males play more risky than females (Lighthall, Mather, & Gorlick, 2009; 37 Mather & Lighthall, 2012; Orsini & Setlow, 2017; Orsini et al., 2016; Van den Bos et al., 2009). Collectively, this body of research suggests that any sex differences found in executive functioning and decision making domains may be driven by differing strategies or preferences between the sexes. This has led to an emergent literature suggesting that sex differences in cognition should be re-evaluated and are perhaps not as pervasive as previously anticipated (Grissom & Reyes, 2019). 1.43 Sex differences in the effects of stress on cognition Given sex differences in executive function and HPA axis output, it is unsurprising that acute stress may differentially modulate aspects of high-level cognition for males and females. Although stress effects have been previously characterized and detailed in the attention section above, these studies have used exclusively male populations or included both male and female subjects but did not report if sex was a significant covariate. In regards to CRF, one study in rodents found that central CRF dose-dependently impaired attention in both male and female rats (Cole et al., 2016). In females, CRF-induced impairments were modulated by circulating ovarian hormones. Specifically, females in proestrous and estrous phases of their cycle, which are characterized by elevated ovarian hormone levels, were resistant to the disruptive effect of CRF on sustained attention. This fits with other studies that consistently found females in higher estradiol phases of their cycle are protected from the effect of stress on cognition (Maki et al., 2015; Schoofs & Wolf, 2009; Zeidan et al., 2011). Acute stress differentially alters working memory and cognitive flexibility depending on sex. For instance, acute stress enhanced working memory in males (Schoofs et al., 2013) but either impaired (Schoofs et al., 2013) or had no effect (Cornelisse et al., 2011) on working memory in females. In parallel, acute stress in humans or chronic stress in rodents impairs 38 flexibility in males but not females (Laredo et al., 2015; Shields et al., 2016). However, one study found that chronic restraint stress in rodents impaired flexibility in females but had no effect in males (Grafe et al., 2017). These effects do not appear to be CORT-dependent as a recent meta-analysis found that sex was not a significant covariate on either working memory or cognitive flexibility alterations following exogenous CORT administration in humans (Shields et al., 2015). Moreover, exogeneous CORT modulated overall working memory performance but there was no overall effect of CORT on cognitive flexibility (Shields et al., 2015). Given the conflicting results of stress on cognitive flexibility and the lack of effect of CORT in mediating these effects, investigating the interplay between sex and stress, with particular emphasis on identifying potential stress mediators, on cognitive flexibility would be prudent. As such, Chapter 2 will address how acute stress and CRF alter cognitive flexibility using a translatable PRL task in both sexes. In terms of decision making, sex differences in response to acute stress have not been investigated in delay- or effort-based tasks. However, in tasks involving uncertainty previous work, particularly in humans, shows that males tend to play more risky than females at baseline and acute stress can exacerbate this sex difference (Grissom & Reyes, 2019; Lighthall et al., 2009; Mather & Lighthall, 2012). That said, sex differences in risky choice appear to be present only when the task involves explicit punishments, such as foot shock or monetary loss, but not when negative feedback is limited to the absence of reward (Grissom & Reyes, 2019; Lighthall et al., 2009; Mather & Lighthall, 2012). Therefore, the ability of acute stress to alter risky decision making may depend on how the task employs negative feedback (Grissom & Reyes, 2019). 39 1.5 Clinical implications: Emphasis on depression Comparing acute stress and CRF effects on any measure is difficult given the non-parallel research literature, in that acute stress effects are possible to investigate in both humans and rodents; however, studies investigating CRF effects in humans are extraordinarily rare. Not only is there is a relatively smaller literature on the effects of CRF on cognition and behaviour generally, acute CRF effects in humans are particularly difficult to probe given that CRF is a large molecule peptide that does not cross the blood-brain barrier (Martins, Kastin, & Banks, 1996). Administering or measuring CRF in human subjects would need to be conducted via spinal tap and is, therefore, impractical for research purposes. Although small peptide non-specific and receptor-specific CRF antagonists have been developed for clinical trials, they have rarely been used for research on cognition. Given the practical limits, one potential way to assess CRF effects in humans is to use clinical populations that endogenously produce more CRF than typical subjects. Increased CRF peptide in the cerebrospinal fluid is found in many stress-related disorders, such as schizophrenia (Banki et al., 1987), obsessive compulsive disorder (Altemus et al., 1992), post-traumatic stress disorder (Bremner et al., 1997), and bipolar and unipolar depression (Banki et al., 1987, 1992; Nemeroff et al., 1984); the latter of which is a good framework in which to view potential cognitive symptoms given the abundance of research on cognitive effects in depression. Although affective symptoms are the traditional hallmarks of depression, the cognitive symptoms of depression are particularly debilitating. Indeed, individuals with depression show a constellation of cognitive deficits, including impairments in explicit visual and verbal memory, (Austin et al., 1999, 2001), psychomotor slowing, and impaired attention (Lee et al., 2012; Ravnkilde et al., 2002). Additional impairments are consistently found in working memory and, 40 more inconsistently, in cognitive flexibility (Austin et al., 2001; Dickstein et al., 2010; Ilonen & Leinonen, 2000; Lee et al., 2012; Ravnkilde et al., 2002; Reppermund et al., 2009; Taylor Tavares et al., 2008). Those who suffer from depression also appear to show more fundamental deficits in reward and punishment processing with maladaptive responses to punishment and reward hyposensitivity using a range of cognitive tasks (Eshel & Roiser, 2010). Similar to acute stress and increased CRF signaling, reward hyposensitivity is consistently linked to depression (Henriques & Davidson, 2000; Henriques, Glowacki, & Davidson, 1994; McFarland & Klein, 2009; Pizzagalli et al., 2005, 2009), due in part to reduced ventral striatum activation to positive stimuli anticipation and receipt (Liu et al., 2014; Zhang et al., 2013). Participants with depression also demonstrate altered sensitivity to punishment. Indeed, individuals with depression show an increased probability of making an error given an error on the previous trial, interpreted as a ‘catastrophic response to perceived failure’ (Beats et al., 1996), and demonstrate diminished ability to utilize both positive and negative reinforcement to guide learning (Chase et al., 2010; Holmes & Pizzagalli, 2007). Taken together, these studies indicate that depression dampens sensitivity to reinforcement, irrespective of valence. Alterations in feedback sensitivity or motivation may underlie decision-making deficits in depression. Depression is associated with a shift in choice away from more preferred yet delayed rewards (Pulcu et al. 2014) and rewards that involve uncertainty (Cella, Dymond, & Cooper, 2010; Clark et al., 2011; Han et al., 2012; Taylor Tavares et al., 2007). Moreover, decision-making appears to be particularly impaired when the more rewarding choice requires more physical effort to obtain (Treadway et al., 2012). Collectively this research illustrates how decision making is dysregulated in depression regardless of the type of cost involved. 41 Depression is a debilitating disorder that deprives sufferers of pleasure, motivation, and cognitive abilities. Given that depression increases CRF signaling, it would be of interest to identify the potential contributions of central CRF in altering behaviour within specific cognitive domains known to be altered in depression. As discussed in detail in above sections, increased CRF signaling impairs motivation (Bryce & Floresco, 2016; Wanat et al., 2013), and PFC-dependent executive functioning, including attention (Van’t Veer et al., 2012) and working memory (Hupalo & Berridge, 2016), in a pattern similar to the constellation of symptoms observed in depression. However, the role of CRF signaling in translatable tasks of cognitive flexibility and decision making involving uncertainty have yet to be addressed. In light of the recent initiative to shift away from ‘modeling’ complex human disorders in preclinical studies, future studies should aim to identify symptom clusters that characterize dissociable aspects shared by many different disorders. By elucidating the role of increased CRF signaling in altering specific domains of cognition we can better understand how CRF hyperactivity may underlie symptom clusters common to many stress-related disorders, such as depression, schizophrenia, obsessive compulsive disorder, and post-traumatic stress disorder. 1.6 Objectives Although acute stress modulation of cognition has been thoroughly examined in a wide range of species and using various tasks, there remain some pressing gaps in knowledge. Notably, acute stress differentially alters cognitive flexibility based on a variety of factors. However, these tasks use deterministic outcomes, which are rarely encountered in daily life. A cognitive flexibility task with probabilistic outcomes has recently been developed for use in preclinical models, which would allow us to better probe the underlying circuitry involved in the modulation of stress on cognitive flexibility. Furthermore, we have recently delineated the role 42 of acute stress, mediated by CRF signaling, in decision making involving effort costs (Bryce & Floresco, 2016; Shafiei et al., 2012). Whether this relationship extends to other types of costs, particularly involving probability or uncertainty, has yet to be addressed. Given that increased CRF activity in the VTA, which contains dopamine-expressing neurons, alters effort choice similar to acute stress and central CRF (Bryce & Floresco, 2016), one would expect that these effects are related to how acute stress and CRF modulate dopamine signaling. Studies find that bath applied CRF to VTA slices has complex, and sometimes dichotomous, effects on VTA dopamine neuron physiology (Korotkova et al., 2006; Ungless et al., 2003; Williams et al., 2014), and CRF in the VTA in intact animals reduces phasic mesoaccumbal dopamine release in a stimulus-dependent manner (Wanat et al., 2013). That said, no study to date has assessed how increasing widespread CRF signaling by infusing this neuropeptide into the ventricular system may alter VTA dopamine physiology. This would aid in our understanding of how central CRF activity alters effort choice, and potentially other cognitive functions. Given the prominent role of dopamine in altering effort choice (Cousins, Sokolowski, & Salamone, 1993; Floresco et al., 2008; Nowend et al., 2001), it is noteworthy that research has not focused on how increasing dopamine signaling in downstream mesoaccumbal terminal regions via dopamine receptor stimulation or CRF infusion, which is known to increase dopamine levels (Lemos et al., 2012), would affect effort choice. Due to the present ambiguity regarding the role of stress in high-level cognitive function and the lack of potential intermediating factors, we investigated how acute stress alters cognitive flexibility and decision making, with a particular focus on CRF signaling and the interplay between CRF and dopamine physiology. Chapter 2: Examined the role of increased central CRF and acute stress on cognitive flexibility in both sexes. During these experiments, animals were either tested prior to task acquisition or 43 during performance on a PRL task. In this task, one lever was designated as the ‘correct’ lever and, when chosen, elicited a sugar pellet reward with 80% probability, whereas, the other lever was designated as the ‘incorrect’ lever and resulted in reward with 20% probability. Once learned to criterion, these contingencies were reversed so that the previously ‘correct’ lever was now ‘incorrect’ and vice versa. Flexibility was assessed by examining the number of reversals and errors made during the testing session. Animals underwent CRF infusion or one-hour of restraint stress either prior to task acquisition or once animals were trained. This experiment was designed to assess how acute stress and CRF may similarly or differentially modulate cognitive flexibility and probabilistic reinforcement learning in male and female animals. Chapter 3: Examined the role of increased central CRF and acute stress on two different tasks of risk/reward decision making. Based on the results of Chapter 2 and the lack of sex differences in risk/reward decision making in human and rodent models when the risk involved reward omission, we only assessed how CRF and stress affected decision making in males. Animals in this experiment were trained on a probabilistic discounting or Blackjack task that required internal generation or reliance on external cues, respectively, to guide choice between a small/certain or a large/risky reward. On a probabilistic discounting task, the probability of obtaining the larger reward increased or decreased systematically over blocks of trials (100-6.25%). On a cue-guided Blackjack task, reward probabilities (50% or 12.5%) were signaled by discriminative auditory cues. Animals underwent CRF infusion or one-hour of restraint stress prior to testing, with choice of the more rewarding but riskier option the main variable of interest. The design of this experiment allowed us probe the effects of acute stress and CRF on risky decision making involving probabilistic outcomes. 44 Chapter 4: Examined the role of increased central CRF on VTA dopamine neuron activity in anaesthetized male rats. The effects of intraventricular CRF infusions at doses that were shown to be behaviourally active were examined on different parameters of dopamine neuron activity using extracellular, single-unit recordings. We found and recorded putative dopamine and GABA neurons as well as neurons we were unable to classify. The main variables of interest were neuronal firing rate, the number of dopamine neurons per electrode track as a measure of tonic dopamine neuron population activity, and phasic bursting activity. This experiment was devised to further clarify how centrally infused CRF modulates dopamine neuron physiology in vivo. Chapter 5: The electrophysiological results of Chapter 4 prompted us to examine the role of NAc dopamine and CRF receptor activation on effort-related decision making. Male rats were well-trained on an effort discounting task wherein they chose between a low effort/low reward vs a high effort/high reward lever where the effort requirement increased over trial blocks (2-20 presses). Dopamine D1, D2/3, or D3 receptor agonists, or CRF, were infused into the NAc core or shell prior to testing. This experiment allowed for the dissection of how dopamine receptor activity drives alterations in effort choice, at the level of the NAc. 45 Chapter 2: Modulation of cognitive flexibility in a probabilistic reversal learning task by CRF and acute stress in male and female rats 2.1 Introduction The acute stress response orchestrates multiple interacting systems, including the HPA axis, to respond to real or perceived threats in the environment. Though adaptive, acute stress can impair some facets of cognition such as memory retrieval (Cazakoff et al., 2010; Howland & Cazakoff, 2010; Kim & Diamond, 2002) and attention (Liston et al., 2009; Olver et al., 2015). In both humans and pre-clinical animal models, the effects of acute stress on PFC-mediated cognition is more variable. As discussed in detail previously, processes such as working memory and cognitive flexibility can either be impaired (Butts et al., 2011, 2013; Devilbiss et al., 2012; Diamond et al., 1996; Laredo et al., 2015; Luethi, 2009; Davies et al., 2013; Olver et al., 2015; Shields et al., 2015, 2016) or facilitated (Bryce & Howland, 2015; Duncko et al., 2009; Gabrys et al., 2019; Graybeal et al., 2011; Thai et al., 2013; Yuen et al., 2009) by stress depending on the task, stressor, and timing among other factors. CRF initiates the HPA axis; however, the neuropeptide and its cognate receptors are also widely expressed outside of the HPA axis (Bittencourt & Sawchenko, 2000; Tan et al., 2017; Van Pett et al., 2000), mediating many of the cognitive effects of stress (Cador et al., 1992; Koob, 2010; Müller et al., 2003). Similar to acute stress, exogeneous central CRF infusion impairs attention (Van’t Veer et al., 2012) and working memory (Hupalo & Berridge, 2016). Likewise, central CRF impairs cognitive flexibility during reversal learning, where stimulus-reward contingencies are reversed so that the previously non-reward stimulus is now rewarded and vice versa, and impairs extradimensional set-shifting, requiring a strategy switch to focus attention on a separate task dimension, in an attentional set-shifting task (Snyder et al., 2011). 46 Stress-induced alterations in PFC-mediated cognition, including working memory and cognitive flexibility, may be dependent on sex. For instance, one study found that acute stress impaired working memory in females but facilitated working memory in males (Schoofs et al., 2013). Conversely, other studies have found that working memory was impaired by stress in both males and females (Schoofs et al., 2008; Shansky et al., 2006). Somewhat surprisingly, research on sex differences in stress-induced alterations in cognitive flexibility is relatively limited. However, cross-species studies show that social stressors impair flexibility in males, but not females (Laredo et al., 2015; Shields et al., 2016). The cognitive flexibility literature regarding the role of acute stress and CRF hyperactivity have focused primarily on situations where outcomes are deterministic. Yet, real world circumstances are rarely so simplistic, in that a “correct” action may not always be reinforced. Recently, rodent cognitive flexibility assays have been developed that use probabilistic reinforcement, mirroring human tasks (Bari et al., 2010). Previous studies using PRL tasks in humans have found that depressed patients, who show elevated CRF levels (Banki et al., 1987; Nemeroff et al., 1984), are more sensitive to misleading negative feedback (Taylor Tavares et al., 2008) or commit slightly more errors during reversal phases (Dickstein et al., 2010; however see Dombrovski et al., 2010; Murphy et al., 2003). It is unclear whether these effects observed in depressed patients are driven in part by increased CRF activity. Moreover, whether acute stress modulates PRL has also yet to be addressed. Therefore, the primary goal of the current set of experiments was to probe how acute stress and enhanced CRF activity affect performance on a PRL task (Bari et al., 2010; Dalton et al., 2014, 2016). A secondary aim was to investigate if these manipulations induced any potential differences in males vs females. Finally, in light of the fact that human PRL tasks are typically conducted in a manner where participants learn the 47 initial probabilistic discrimination and reversal in the same session, we additionally considered how stress and CRF influence flexibility prior to acquisition of the task. 2.2 Methods 2.21 Animals Squads of 12-24 male and female Long–Evans rats (Charles River Laboratories) were maintained on a 12 h light/dark cycle. Rat colonies were segregated by sex. Following one week of acclimatization to the colony, animals underwent intra-cranial surgery (in the CRF group) and were individually housed. Once recovered, rats were food restricted and maintained at 85–90% of ad libitum weight in age-matched rats throughout training. Weight was monitored daily and rat chow was provided at the end of each training day. Water was provided ad libitum for the duration of the experiment. Care was taken to limit the animals used and all testing was done in accordance with the Canadian Council of Animal Care and the Animal Care Committee of the University of British Columbia. 2.22 Apparatus Behavioural training was conducted in operant chambers (30.5 x 24 x 21 cm; Med-Associates, St Alban, VT, USA) within a sound-attenuating box and equipped with a fan for ventilation and limiting extraneous noise. The chamber was fitted with a central food receptacle that dispensed sweetened reward pellets (45 mg; Bioserv) with two retractable levers on either side of the food receptacle. The chambers were illuminated by a single 100mA house light located in the top-center of the wall opposite the levers. All experimental data were recorded by a computer connected to the chambers via an interface. Sensors located along the length of the box measured the number of photo beam breaks, used as an index of locomotor activity. The 48 operant chambers were connected to a personal computer via an interface to record experimental data. 2.23 Surgery Prior to training, rats in groups that were assigned to receive ICV CRF infusion were partially anaesthetized with ketamine (50 mg/kg, IP)/xylazine (5 mg/kg) 10 minutes prior to isofluorine gas anesthesia and analgesic (anafen, 10mg/kg, SC) administration. Rats were placed into a stereotaxic frame and secured with earbars to maintain a flat skull. Isofluorine and oxygen were administered via nose cone maintained between 2- 4% to ensure surgical plane. Rats were implanted with a unilateral cannula lowered to 1 mm dorsal to the right lateral ventricle (coordinates: [AP -1.0 mm from bregma, ML - 1.8 mm from bregma, DV -2.5 mm from dura]) secured in place with skull screws and dental acrylic. Post-operative procedures included daily weighing and subcutaneous analgesic administration (anafen 10 mg/kg) for at least 2 days following surgery. 2.24 Initial lever pressing training Lever training commenced following at least 5 days of food restriction. On the day prior to training, ~40 pellets were placed into the home cage of pair-housed rats. During the first day of lever training, one of two levers was extended for the duration of the session and 2-3 crushed sugar pellets were placed on the lever prior to the rat being placed into the chamber. Initial lever training began with at least two days of FR1 training, in a counterbalanced manner, where rats pressed the lever once for one sugar pellet to a criterion of 60 presses in 30 minutes on one lever followed by FR1 training on the opposite lever. Rats were subsequently trained on a simplified version of the full task, which required rats to press the retractable lever where one sugar pellet was delivered on 50% of trials. Sessions consisted of 90 training trials with 45 presses required 49 on each lever. Trials were initiated with the illumination of the houselight and the extension of one of the two levers and separated with a 40s ITI. For this and all other phases of training, trials were scored as omissions if the rat failed to respond to the lever within 10 seconds following lever extension. If this occurred, the lever retracted, the houselight turned off, and a 40 second ITI began. Rats were trained to a criterion of 80 successful trials (i.e. < 10 omissions) for 2 consecutive days, which took approximately 5 days of training. Rats were either tested the following day on the acquisition version of the PRL task or trained on the performance version of the PRL task until behavioural stability was achieved. 2.25 Probabilistic reversal learning Separate squads of animals were trained on one of two versions of the PRL task. Initial training was identical across groups; however, animals in the “acquisition” version were tested prior to experiencing the PRL task whereas rats in the “performance” version were trained on the task until they displayed stable patterns of choice. Rats in the acquisition experiments were tested on a single, 240 trial version of the task, whereas those in the performance experiments received daily sessions of 200 trials. In both versions, trials began every 15 s with illumination of the house-light and 3 s later, both levers were extended. At the start of the session, one of the two levers was randomly designated as the “correct” lever and the other lever was the “incorrect” lever. A response on the “correct” lever delivered one sugar pellet reward on 80% of trials but nonreinforced on 20% of trials. Conversely, “incorrect” responses were predominately nonreinforced, with 80% of trials omitting reward; however, a minority delivered sugar pellet reward, with 20% of trials reinforced. Failure to respond within 10 s of insertion (i.e., trial omission) led to retraction of the levers and termination of the houselight. When rats selected the “correct” lever on 8 consecutive 50 trials (regardless of whether a correct choice was reinforced) the reward contingencies were reversed so that the “correct” lever became the “incorrect” lever and vice versa. For the acquisition version, contingencies reversed after at least 80 completed trials during the initial discrimination, even if a rat made 8 consecutive correct choices prior to this. This procedure was instituted to ensure that each rat completed the same number of trials during the initial acquisition of action/outcome contingencies to assess how our manipulations affected probabilistic reinforcement learning. For the performance version of the task, contingencies switched immediately after 8 consecutive correct choices, and this was repeated until the end of the session. In the performance version, daily training sessions were continued until rats achieved stable patterns of choice, defined as at least 3 reversals completed per session for 2 consecutive days (typically requiring 11-12 training sessions). 2.26 Drugs, microinfusion protocols, and acute restraint stress In one series of experiments, separate groups of animals were tested on the PRL task following ICV CRF infusion. Animals in these groups first received a mock infusion on the day prior to their first microinfusion test day, during which stylets were removed from the guide cannula and replaced with injectors for the duration of the subsequent infusion (approximately 4 mins); however, no infusion was given. Infusions were administered unilaterally into the right lateral ventricle via a 30-gauge infusion needle protruding 0.8mm past the guide cannula. The infusion needle was connected to polyethelene tubing, which was attached to a microinfusion pump set to infuse at a rate of 0.84 μl/min. The injector was left in for an additional 1 min to allow for diffusion. Rats were left in their home cage for 10 mins prior to behavioural testing. CRF or vehicle (artificial cerebrospinal fluid, aCSF) was infused at a volume of 3 μl. CRF was sonicated until dissolved in aCSF and protected from light. CRF doses were selected from 51 our previous study that found that 3μg CRF infusion into the lateral ventricle reduces choice of larger rewards linked to a greater effort cost on an effort discounting task (Bryce & Floresco, 2016). However, comparing males and females using the same CRF dose may provide only limited information given that females are more sensitive to the effects of CRF than males (Bangasser et al., 2010; Wiersielis et al., 2016). Although we conducted a pilot study on the first squad of males tested on the PRL performance version of the task and found no effects of a lower CRF dose (1g), this does not necessarily mean that females might not have been affected by lower CRF doses. Given the fact that high-dose (3g) but not low-dose (1g) CRF altered effort choice (Bryce & Floresco, 2016) and that there are sex differences in response to acute stress on cognitive flexibility (Laredo et al., 2015; Shields et al., 2016), the current study was conducted to understand the role of high-dose CRF on cognitive flexibility in both sexes. Therefore, we did not conduct any further testing at lower CRF doses. In a separate series of experiments, groups of animals were tested on the PRL task following acute restraint stress. The first day of restraint stress testing was a baseline test day where animals were placed on carts in their home cage and wheeled into the room that would subsequently be the room where they would be restrained. They remained in this room for the duration of the subsequent restraint (1 hr), prior to being placed in the operant chamber for testing. A table fan was turned on to circulate air and limit extraneous noise. The following day, animals were wheeled into the same room where each rat was placed into a Plexiglas restrainer (83 × 133 × 197 mm; Harvard Apparatus, Massachusetts, USA). A table fan was aimed at the restrainers to prevent hyperthermia as described by Bryce and Floresco (2016). After 1 hour, rats were removed from restrainers and placed back into their home cage for an additional 10 mins, after which they were placed into the operant chamber for testing. 52 The acquisition experiments used a between-subjects design whereby animals were split into two groups based on initial training performance, half of which received vehicle (3l aCSF) or ‘no stress’ and half received CRF (3g) or one hour of restraint stress, 10 mins prior to the first day of the full PRL task. The performance experiments used a within-subjects design with rats that were well-trained on the PRL task prior to testing. On the first day, separate groups of animals received either 3l vehicle (aCSF) infusion or ‘no stress’ and the second day received either 3g CRF infusion or one hour of restraint stress, 10 mins prior to being placed into the operant chamber for testing on the PRL task. 2.27 Female estrous cycle All female rats were lavaged to determine estrous cycle. To do so, a pipette tip filled with saline was inserted into the vagina, which was flushed with saline, recaptured, and dispensed onto gel-coated glass slides. When dry, slides were stained with Cresyl Violet and cover slipped with Permount. Estrous cycle was determined via microscopic identification of cells. Four stages (pro-estrus, estrus, metestrus, and diestrus) were categorized based on the abundance of nucleated or corneated epithelial cells and leukocytes. Female rats in the acquisition experiment were lavaged once following testing so as not to interfere with behavioural results. Female rats in the trained experiment were lavaged for three days prior to testing to habituate rats to the experience. Rats were lavaged immediately after each test day. Upon microscopic examination of the slides, we noticed that the vast majority of females were in low ovarian hormone stages (diestrus and metestrus, n = 54) with a small minority in high ovarian hormone stages (pro-estrus and estrus, n = 4) of the estrous cycle. However, it is not uncommon for females who are food restricted to cycle abnormally and stay in low ovarian 53 hormone stages of their estrous cycle (Terry et al., 2005). As we did not have equal numbers in each stage, we were unable to add this factor as an additional between-subjects factor in our ANOVAs. Therefore, females were analyzed with Estrous Cycle (pro-estrus, estrus, metestrus, diestrus) at the time of testing as a covariate in the ANOVA. That said, we found no effect of Estrous Cycle on any measure when sex differences were present (all Fs < 3.20, all Ps > 0.09) on any task, therefore data regarding estrous cycle will not be discussed further. 2.28 Histology Following testing, rats were killed with CO2 and brains were removed and fixed in 4% formalin solution. Brains were sectioned at 50 μm, mounted on gel-coated slides and Nissl stained using Cresyl Violet. One-hundred and ninety-eight rats were trained and tested, with 86 rats receiving surgery. Of these, 7 animals were removed due to lateral ventricle placements that were too dorsal (n=5) or too medial (n=2), and 1 animal was removed due to incomplete infusion rounds, leaving 78 rats for analysis in the CRF groups and 112 rats for analysis in the stress groups. 2.29 Data analysis The primary variables of interest were the number of reversals completed per session and number of errors committed, both analyzed as a function of the number of trials completed. This was to accommodate for an increase in the omission rates that could be induced by our manipulations, as a reduction in the number of reversals may be due to impaired task performance or merely due to rats completing fewer trials. Specifically, reversal and error data were transformed by taking these values and dividing by the total number of trials minus the number of omissions made. This value was then multiplied by 100 to obtain a metric of reversals or errors/100 completed trials. Reversals/100 trials was not assessed for acquisition versions of 54 the PRL task as the number of reversals animals were able to complete was limited. Additional analyses assessed the number of errors committed to achieve the first discrimination and first reversal. Note that in each of these experiments, CRF or stress treatments tended to increase omission rates, and this included some rats that made more than 100 errors, making analysis of their choice data difficult. In these instances, we eliminated the data from these rats and focused on rats that completed a sufficient number of trials on CRF/stress test days. We also analyzed choice-by-choice win-stay and lose-shift data according to the outcome. We separately analyzed reward and negative feedback sensitivity when the animals chose correctly or incorrectly on the current trial given a win or loss on the previous trial. Win–stay ratios assessed the probability that a rat followed a rewarded choice with another choice of the same type (correct or incorrect), calculated from the number of trials on which a rat chose the correct/incorrect lever after being rewarded on the preceding trial, divided by the total number of rewarded correct or incorrect choices. Alternatively, lose–shift ratios assessed the probability that a rat switched choices following negative feedback (i.e., reward omission) on the preceding trial, calculated from the number of trials on which a rat switched responding to the other lever after reward omission for a correct or incorrect choice on the previous trial, divided by the total number of non-rewarded correct/incorrect choices. Choice latency (time between lever insertion and press) was analyzed separately based on choice (correct or incorrect choice trials). The number of trial omissions (i.e., trials where no response was made within 10 s of lever insertion) was analyzed overall and according to task phase: initial discrimination and following first reversal. As these trials were not equally distributed over the session, we transformed the omissions per phase into an omission rate, taking the number of omissions for the initial discrimination divided by the number of trials need to complete the initial discrimination. 55 Similarly, the number of omissions made following the first reversal was divided by the number of trials needed for the initial discrimination subtracted by the total number of trials in the session. All variables in the acquisition group were analyzed using a two- or three-way ANOVA, with Treatment (vehicle vs CRF or no stress control vs restraint stress) and Sex (male vs female) as between-subjects factors. All variables in the performance group were analyzed using a two- or three-way ANOVA, with Treatment (vehicle vs CRF or control vs restraint stress) as a within-subjects factor and Sex (male vs female) as a between-subjects factor. Errors and omission rates were analyzed by Phase (initial discrimination vs first reversal). However, there was always a significant main effect of Phase in these analyses, with errors and omissions higher in the reversal phase as opposed to the discrimination phase, and so this will not be further discussed. Win-stay and lose-shift tendencies, and choice latency were initially analyzed by Choice (correct vs incorrect) as an additional within-subjects factor. Initial analyses found no main effect of Choice on latencies in either manipulation group when first acquiring the task (all Fs < 1, all Ps > 0.35), but a main effect of Choice on latencies for both manipulation groups during performance (CRF group: F(1,27)= 13.93, p < 0.001; Restraint stress group: F(1,54)= 18.38, p < 0.001), with slower reaction times when choosing the incorrect compared to the correct choice. Likely, these differential results were due to extended training in the performance groups, indicating that animals had some knowledge about the nature of the task in these groups and were more “indecisive” prior to eventually choosing incorrectly. That said, these initial analyses did not reveal any interactions with the Choice variable on choice latency and no main effects or interactions with the Choice variable on win-stay lose-shift tendencies (all Fs < 1.62 all Ps > 0.21), indicating that any changes in these measures were comparable after both correct and 56 incorrect choices. Therefore, for simplicity, we analyzed win-stay lose-shift ratios and choice latency (collapsed across correct/incorrect choices) using a three-way ANOVA with Sex and Treatment as within-subjects variables for choice latency, and Ratio Type (win-stay vs lose-shift) as an additional within-subjects variable for win-stay lose-shift ratios. We found a significant main effect of Ratio Type for all analyses, with win-stay tendencies always higher than lose-shift tendencies so this will not be further discussed. 2.3 Results 2.31 Central CRF infusion 2.31.1 Acquisition (between-subjects comparisons). Fifty-six animals were trained and tested prior to PRL acquisition (Fig 1A) but 7 were removed due to missed placements, leaving 49 animals (n=29 males; n=20 females) for analysis. When analyzing all animals in the CRF acquisition group, we found that CRF increased omission rates (Fig 1B), with a significant effect of Treatment (F(1,45)= 6.05, p < 0.05). Over the entire session, rats receiving vehicle treatments made 7.04 +/- 2.73 omissions, whereas those receiving CRF made 25.23 +/- 5.80 omissions. However, CRF differentially affected omission rates depending on Phase and Sex (Treatment x Phase x Sex interaction: F(1,45)= 4.04, p < 0.05). Looking at Fig 1B, we can see that CRF treatment slightly increased omission rates during the initial discrimination of the task but markedly increased omission rates following the first reversal in male rats (Treatment x Phase interaction: F(1,27)= 8.64, p < 0.005); main effect of Treatment during initial discrimination: (F(1,27)= 7.83, p < 0.01); main effect of Treatment during reversal phase: (F(1,27)= 9.06, p < 0.001). However, this profile was not observed in females (main effect of Treatment, Treatment x Phase interaction: all Fs < 1 all Ps > 0.50) and there was no main effect of Sex (F(1,45) < 1, n.s.) or interaction between Sex x Phase (F(1,45) < 1, n.s.). Together this pattern of results 57 indicate that, at least in males, CRF hyperactivity markedly increased the tendency for rats not to make a choice following a change in reward contingencies. Additionally, CRF treatments increased reaction time (Fig 1C), in a manner dependent on Sex (Treatment x Sex interaction: F(1,45)= 5.32, p < 0.05). Simple main effects found that CRF significantly increased choice latencies in males (F(1,27)= 7.85, p < 0.01), but not females (F(1,18) < 1, n.s.). However, there were no main effects of Sex (F(1,45)= 2.18, n.s.). Together these results indicate that increased CRF signaling reduced motivation in males, but not females, increasing the number of trials omitted and, on trials that were completed, increasing the latency to make a choice. Of all the rats that received CRF infusions, 8 animals made more than 100 trial omissions, making analysis of their choice data problematic. Data from these rats were removed from subsequent analyses, leaving 41 animals (n=23 males; n=18 females). Analyses of error data revealed that CRF infusion had no effect on error rate over the entire session (main effect of Treatment, main effect of Sex, Treatment x Sex interaction: all Fs < 2.43 all Ps > 0.13, Fig 1D) or during the initial discrimination or first reversal phases of the session in either sex relative to vehicle (main effect of Treatment, Main effect of Sex, Treatment x Phase, Sex x Phase, Treatment x Sex x Phase interactions: all Fs < 1.16 all Ps > 0.29, Fig 1E). Therefore, we conclude that CRF hyperactivity had no effect on initial probabilistic reinforcement learning or probabilistic reversals. Although CRF treatment did not alter the number of errors committed during the acquisition and reversal of the rules, it was still of interest to assess how choice might differ from trial to trial based on the outcome of the previous trial. Analysis of the win-stay lose-shift data revealed a significant Treatment X Sex interaction (F(1,37)= 5.88, p < 0.05). Simple main effects 58 found that CRF reduced win-stay, lose-shift tendencies in males (F(1,21)= 10.08, p < 0.01), but not in females (F(1,16) < 1, n.s., Fig 1F). However, there was no main effect of Sex (F(1,37) < 1, n.s.) or Treatment (F(1,37)= 1.60, n.s) and no interaction with Ratio Type (Treatment x Ratio, Sex x Ratio, Treatment x Sex x Ratio: all Fs < 1 all Ps > 0.50). Together these data suggest that increased CRF signaling in males reduced the impact that previously rewarded or non-rewarded actions had on subsequent action selection, reducing both win-stay and lose shift tendencies. In comparison, this CRF effect was not observed in female rats. 2.31.2 Performance (within-subjects comparisons). A separate group of 30 rats (n=18 males; n=12 females) were trained on the PRL task for 11 days prior to testing (Fig 2A). One rat did not complete all rounds of infusions, leaving 29 rats (n=17 males; n=12 females) for subsequent analyses. When we analyzed the acquisition data, we found no effect of Sex on the mean number of reversals completed during initial training (F(1,21)= 2.11, n.s., Fig 2B), indicating that both sexes learned the PRL task at comparable rates. As was observed in the acquisition experiment, CRF infusion significantly increased the number of trials omitted in both sexes (main effect of Treatment: F(1,27)= 9.02, p < 0.01; Treatment x Sex interaction: F(1,27)= 2.00, n.s., Fig 2C), that depended on phase (Treatment x Phase interaction: F(1,27)= 4.78, p < 0.05). Simple main effects revealed that CRF slightly increased omission rates during initial discrimination (F(1,28)= 5.80, p < 0.05), but this effect was dramatically increased once reward contingencies changed during the reversal phase (F(1,28)= 9.29, p < 0.001, Fig 2C). However, in contrast to the null effect of Sex in the acquisition experiment, we found a main effect of Sex during performance (F(1,27)= 4.59, p < 0.05, Fig 2C, inset), whereby females made more omissions than males across treatment conditions, which was dependent on phase (Sex x Phase interaction: F(1,27)= 4.43, p < 0.05), but 59 not a three-way Treatment x Sex x Phase interaction (F(1,27)= 1.86, n.s.). Simple main effects found that there were no sex differences during initial discrimination (main effect of Sex: F(1,27)= 1.23, n.s.), whereas during the reversal phase, females omitted significantly more than males across treatment groups (main effect of Sex: F(1,27) = 4.88, p < 0.05). This pattern of results indicates that CRF hyperactivity markedly increases the tendency to omit trials following a change in reward contingencies in males regardless of training but in both sexes with extended training. Thus, instead of potentially making an incorrect choice, animals would rather not choose at all. Moreover, females might be more sensitive to changing reward contingencies than males as they were more likely to omit choice during reversal than discrimination phases, at least when they were well-trained on the task. CRF infusion also increased choice latencies (Fig 2D) (main effect of Treatment: F(1,27)= 5.09, p < 0.05, Fig 2C) that, although visually appeared to be driven by an increase in males, did not depend on Sex (Treatment x Sex interaction: F(1,27)= 2.04, n.s.). However across treatment conditions, females were slower to make a choice than males (main effect of Sex: (F(1,27)= 16.95, p < 0.001). When taken with previous research on central CRF, these results point to a major role for central CRF hyperactivity in slowing deliberation times across tasks (Bryce & Floresco, 2016). Data from 6 animals that omitted more than 100 trials after CRF infusions were removed from analyses, leaving data from n=23 animals (n=13 males; n=10 females). Neither CRF treatment nor Sex had any effect on the number of errors committed (main effect of Sex, Treatment x Sex interaction: all Fs < 1.48 all Ps > 0.24, Fig 2E). Similarly, there was no main effects or interactions of Sex, Treatment, or Phase (all Fs < 1.43 all Ps > 0.24, Fig 2F). 60 When we analyzed the number of reversals per 100 completed trials, we found a slight increase in the number of reversals made following CRF infusion (Fig 2G). However this effect only trended towards significance (main effect of Treatment: F(1,21)= 3.26, p= 0.08), and did not depend on Sex (main effect of Sex, CRF x Sex interaction: all Fs < 1 all Ps > 0.60). Visual inspection of these data revealed that this effect appeared to be driven primarily by males. Thus, while the number of errors was unaffected, increased CRF signaling slightly enhanced flexibility in terms of the number of reversals completed. We then analyzed how CRF treatments affected reward and negative feedback sensitivity (Fig 2H). We found a significant Treatment X Ratio interaction (F(1,21)= 7.48, p < 0.05). Simple main effects show that, in contrast to the acquisition version, CRF slightly, but non-significantly, increased win-stay tendencies (F(1,22) = 2.38, n.s.) but significantly reduced lose-shift tendencies (F(1,22)= 5.56, p < 0.05). However, we did not find a main effect of Treatment or Sex or an interaction between the two variables (main effect of Treatment, main effect of Sex, Treatment x Sex interaction: all Fs < 1 all Ps > 0.52). Taken together, this pattern of results indicate that CRF had a minor effect on flexibility, increasing the number of reversals without affecting errors, which was likely due to the slight increase in reward sensitivity and substantial reduction in negative feedback sensitivity, with CRF reducing the tendency of following up a loss on the previous trial with a switch to the opposing lever. 2.32 Restraint stress 2.32.1 Acquisition (between-subjects comparisons). Separate groups of rats were trained and tested n=56 rats (n=32 males; n=24 females) prior to PRL acquisition (Fig 3A) and all were included in subsequent analyses. First, we found that one-hour of restraint stress had no effect on omission rate (main effect of Treatment, Sex x Treatment, Sex x Phase, Treatment x Phase, Sex 61 x Treatment x Phase interactions: all Fs < 2.36 all Ps > 0.13, Fig 3B). However, similar to previous experiments, we found that females omitted more trials than males (main effect of Sex: F(1,51)= 25.34, p < 0.001, Fig 3B, inset). This indicates that acute stress, as opposed to CRF hyperactivity, does not alter the number of trials omitted. Additionally, restraint stress did not alter average choice latency (Fig 3C), as the analyses failed to yield significant effects of Treatment (F(1,52)= 1.54, n.s.; Sex x Treatment interaction: F(1,52) < 1, n.s.). However, females were slower to react than males across treatment conditions (F(1,52)= 34.95, p < 0.001). Collectively, these results indicate that females complete fewer trials and are slower to decide on trials that they did complete relative to males. However acute stress did not differentially affect these measures of motivation. As was done in the previous experiments, data from 7 animals that omitted more than 100 trials were removed from choice analysis (n=49 total; n=26 male; n=23 females). Interestingly, in this experiment we found a significant Treatment X Sex interaction (F(1,45)= 4.53, p < 0.05, Fig 3D). Partitioning this interaction further revealed that restraint stress increased the number of errors committed in males (F(1,24)= 6.36, p < 0.05), but had no effect in females (F(1,21) < 1, n.s.). When averaged across sex, there was no main effect of restraint stress (F(1,45)= 1.84, n.s.). However, due to the stress-induced increase in errors found in males, when analyzing errors across treatment conditions, males committed more errors than females (main effect of Sex: F(1,45)= 9.63, p < 0.01). In contrast, when examining the effects of restraint on errors committed during the first discrimination and reversal phases we did not find any effects (main effect of Treatment, Stress x Phase interaction, Sex x Treatment x Phase interaction: all Fs < 3.34 all Ps > 0.07, Fig 3E). However, we again found that when averaging across treatment groups, males committed more errors than females (main effect of Sex: F(1,45)= 10.80, p < 0.01). Visual 62 inspection of the data indicates that this may be caused again by the stress-induced increase in errors in males, particularly following the first reversal, although this is not reflected statistically. Thus, restraint stress slightly increased the error rate in males but not females. Restraint stress did not alter reward or negative feedback sensitivity in either sex relative to control animals (main effect of Treatment, Sex x Treatment, Ratio x Treatment, Sex x Ratio x Treatment: all Fs < 1.52 all Ps > 0.22, Fig 3F). However, we found a main effect of Sex (F(1,45)= 19.01, p < 0.001, Fig 3F, inset), whereby females were less sensitive to immediate feedback than males. Therefore, in contrast to the effects of CRF infusion, acute stress did not alter sensitivity to probabilistic reinforcement but impaired performance by increasing the number of errors committed in males but not females. Moreover, females were less likely than males to use feedback on the previous trial to inform choice behaviour on subsequent trials, without altering flexibility. 2.32.2. Performance (within-subjects comparisons). A separate group of 56 animals were trained for 11 days on the PRL task prior to testing (n=32 males; n=24 females, Fig 4A). Contrary to animals in the CRF performance group, animals in the stress performance group showed significant sex differences in training, with females completing fewer reversals per 100 completed trials over the course of training than males (F(1,21)= 7.49, p < 0.05, Fig 4B), indicating that females were slower to learn the PRL task than males. Once animals entered the testing phase of the experiment, we found that restraint stress significantly increased omission rate (main effect of Treatment: F(1,53)= 12.70, p < 0.01, Fig 4C), but this depended on Sex (Sex x Treatment interaction: F(1,53)= 5.96, p < 0.05) and Phase (Treatment x Phase interaction: F(1,53)= 6.66, p < 0.05), but not a three-way interaction (Sex x Treatment x Phase interaction: F(1,53) < 1, n.s.). First, simple main effects for the Treatment X 63 Phase interaction show that stress did not increase omission rate in the discrimination phase (F(1,55)= 2.60, n.s.). However, stress significantly increased omission rate in the reversal phase (F(1,54)= 9.41, p < 0.01, Fig 4C, inset-left). Second, simple main effects for the Sex X Treatment interaction found that stress did not increase the omission rate in males (F(1,31)= 3.18, n.s.) but significantly increased the omission rate in females (F(1,23)= 8.24, p < 0.01). We again found that females omitted significantly more than males across treatment groups (main effect of Sex: F(1,53)= 41.85, p < 0.001, Fig 4C, inset-right), driven by phase (Phase x Sex interaction F(1,53)= 46.18, p < 0.001). Simple main effects found that females omitted slightly more trials during initial discrimination than males (F(1,54)= 9.03, p < 0.01). However, females omitted dramatically more trials during the reversal phase compared to males (F(1,54)= 41.94, p < 0.001). In contrast to acquisition, we found that acute stress had little effect during initial discrimination but markedly increased omissions following a reversal in reward contingencies. However, this effect was driven by females omitting more trials following acute stress relative to males. Moreover, females were more sensitive to changes in reward contingencies than males as they were more likely to omit trials during reversal than discrimination phases across both manipulations when well-trained but not during acquisition. Restraint stress also increased choice latency (Fig 4D) (main effect of Treatment: F(1,54)= 7.67, p < 0.01) that did not depend on Sex (Sex x Treatment interaction: F(1,54) < 1, n.s.). Similar to acquisition, however, females had a slower reaction time relative to males (main effect of Sex: F(1,54)= 116.44, p < 0.001, Fig 4D, inset). Therefore, acute stress increased choice latencies in both sexes and females were slower to respond compared to males. With respect to choice data, we removed data from 3 animals that omitted more than 100 trials, leaving n=53 animals (n=32 males; n=19 females). Analysis of the data from the 64 remaining rats revealed no effect of Treatment on errors (main effect of Treatment, Sex x Treatment interaction: all Fs < 1.12, all Ps > 0.30). However, we found that females committed more errors than males across treatment groups (F(1,51)= 81.15, p < 0.001, Fig 4E). Restraint stress also had no effect on the number of errors on initial discrimination or first reversal phases (main effect of Treatment, Treatment x Sex interaction, Treatment x Phase interaction, Treatment x Sex x Phase interaction: all Fs < 1.91 all Ps > 0.17, Fig 4F). Similar to the average number of errors, females committed more errors than males across treatment groups (main effect of Sex: F(1,51)= 5.84, p < 0.05), that did not depend on Phase (Sex x Phase interaction: F(1,51) < 1, n.s.). When viewed collectively, females made less errors when acquiring the task (although it might be driven by stress-induced increase in errors in males), but committed more errors when well-trained on the task. This was perhaps due to differential learning of the task as females were completing fewer reversals than males by the end of training (Fig 4B). In keeping with the error data, we found no effects of Treatment (F(1,51) < 1, n.s., Fig 4G) or Sex (F(1,51)= 1.07, n.s ; Sex x Treatment interaction: F(1,51)= 1.58, n.s.) on the number of reversals completed, indicating that acute stress did not alter flexibility using this measure. Even though we did not find any effect on flexibility, we assessed how stress may alter sensitivity to probabilistic reinforcement. Here, we found a significant Treatment X Sex interaction (F(1,51)=3.57, p < 0.05, Fig 4H). Simple main effects found no effect of stress in males (main effect of Treatment: F(1,31) < 1, n.s.), but a main effect of stress in females (main effect of Treatment: F(1,20)= 4.50, p < 0.05), with stress reducing both win-stay and lose-shift tendencies relative to control treatments. However, there was no main effect of Treatment (F(1,51)= 2.74, n.s.) or Ratio or interactions between the two variables (Treatment x Ratio, Sex x Ratio, Treatment x Sex x Ratio: all Fs < 1 all Ps > 0.41). Curiously, we also found a main effect 65 of Sex (F(1,51) = 31.90, p < 0.001, Fig 4H, inset) on win-stay lose-shift tendencies, in a similar manner to the acquisition group. Thus, acute stress in females, but not males, reduced the tendency to follow up a rewarded trial with choice on the same lever and also reduced the probability of switching to the other lever when the previously chosen lever was non-rewarded. Taken collectively with the CRF data, this behavioural pattern indicates that males may be more sensitive to increased CRF signaling, whereas females may be more sensitive to acute stress in terms of feedback sensitivity. 2.4 Discussion The present series of experiments provide evidence that central CRF infusion and acute stress distinctly alter performance on various measures of the PRL task. Increased CRF signaling reduced both reward and negative feedback sensitivity in males, but not females, during the initial acquisition of the PRL task. In comparison, during performance on the PRL task, CRF treatments reduced negative feedback in both sexes and slightly facilitated cognitive flexibility, as evidenced by an increase in the number of reversals completed. On the other hand, acute stress selectively increased the number of errors committed in males, but not females, during PRL acquisition. During performance, acute stress reduced both reward and negative feedback sensitivity in females compared to males. Both CRF and acute stress markedly reduced motivation to perform the task in both sexes. To this point, both manipulations dramatically increased the number of trials omitted, particularly following the first reversal when reward contingencies changed, and CRF and acute stress increased decision latencies. 2.41 Cognitive flexibility CRF hyperactivity and acute stress differentially altered flexibility in the PRL task that depended on task structure. First, CRF altered cognitive flexibility in a limited capacity, slightly 66 facilitating flexibility by increasing the number of reversals completed, with the no effect on the number of errors committed. However, CRF did not alter flexibility when first learning the task contingencies during acquisition. This is in opposition to previous work showing that central CRF infusion impairs reversal learning and extradimensional set-shifting in an attentional set-shifting task in males by increasing the number of errors (Snyder et al., 2011). The reason for these discrepant results is unclear. One explanation is that CRF alters flexibility when outcomes are deterministic such as in the attentional set-shifting task, whereas CRF has no effect when outcomes are probabilistic. Alternatively, divergent effects could be due to differences in CRF dose as the previous study used doses that were orders of magnitude lower than the doses used here (0.030-0.300g vs 3g CRF). That said, we did pilot a lower dose (1g) in males without effect (results not shown) and previous studies found no effect of lower CRF doses (0.25 or 1g) on an effort-related decision-making task (Bryce & Floresco, 2016). Therefore, although plausible, it is unlikely that the different CRF doses produced opposing effects on the current compared to the former study. It is interesting to compare these findings with those from humans with stress-related disorders associated potentiated CRF signaling such as depression (Banki et al., 1987; Nemeroff et al., 1984). These studies find that individuals with depression show cognitive flexibility impairments on WCST or set-shifting tasks that use deterministic outcomes (Austin et al., 2001; Ilonen & Leinonen, 2000; Lee et al., 2012; Reppermund et al., 2009) similar to chronic stress (Bondi et al., 2010; Bondi et al., 2008; Hurtubise & Howland, 2016; Jett et al., 2015; Jett et al., 2017; Lapiz-Bluhm et al., 2009). However, when tasks use probabilistic reinforcement in a human version of the PRL task, individuals with depression show no overall impairment (Dombrovski et al., 2010; Murphy et al., 2003; Taylor Tavares et al., 2008), a trend toward an 67 impairment on a subsample of young participants (Dickstein et al., 2010), or a significant impairment in a subset of patients who have attempted suicide (Dombrovski et al., 2010). Notably, cognitive flexibility tasks that use deterministic outcomes find impairments both in those suffering from depression and following central CRF infusion in animals (Snyder et al., 2011). Yet, when probabilistic reinforcement is employed, there are no consistent effects in either case. Taken together, this may indicate that hyperactive CRF signaling selectively alters flexibility in depression, particularly when outcomes are deterministic. In opposition to CRF, acute stress caused a slight disruption in the acquisition of the PRL task, increasing the overall number of errors committed. This finding adds to the literature of stress-induced deficits in cognitive flexibility (Butts et al., 2013; Laredo et al., 2015; Shields et al., 2015, 2016) but contrasts with others reporting facilitation of this process after acute stress (Bryce & Howland, 2015; Gabrys et al., 2019; Graybeal et al., 2011; Thai et al., 2013). The reason for these discrepant findings may be due to the differences in testing across groups, with different timing, duration, and types of stressors. Additionally, it is notable that acute stress only affected error rates when probabilistic reward information about different response outcomes were first acquired, and not when these outcomes were well-learned. Therefore, one possibility is that acute stress impairs flexibility when first learning the task structure, but has no effect on flexibility once the nature of this task is established. These divergent findings may also be due to these manipulations working on different underlying mechanisms. For instance, inactivating the prelimbic mPFC facilitates flexibility in the PRL task (Dalton et al., 2016) in a similar manner to central CRF infusion indicating that increasing CRF signaling may alter flexibility by suppressing activity within the mPFC. That said, bath applying CRF to mPFC slices increases the excitability of pyramidal neurons (Liu et 68 al., 2015), enhancing rather than suppressing mPFC activity. However, it is unclear if centrally applied CRF in a behaving animal would have the same effect. Alternatively, inactivation of the medial or lateral OFC or the NAc shell impair flexibility in PRL tasks and may be possible sites of action for acute stress-induced impairments (Dalton et al., 2014, 2016). Intriguingly, citalopram, which increases 5-HT levels, exerts bidirectional effects on PRL, facilitating flexibility in a similar manner to central CRF administration at high doses but impairing flexibility in a similar manner to acute restraint stress at low doses (Bari et al., 2010). This is an intriguing possibility as CRF increases the firing rate of 5-HT neurons (Lowry et al., 2000), inducing 5-HT release throughout terminal regions. Although both central CRF infusion and acute restraint stress increase 5-HT levels (Haleem & Parveen, 1994; Singh et al., 1992), whether these manipulations do so differentially is currently unclear. Therefore, perhaps CRF hyperactivity acts directly or indirectly to reduce mPFC activity, whereas acute stress acts to reduce activity in the OFC or NAc shell. Alternatively, both of these manipulations may alter 5-HT signaling with differing magnitudes, that in turn differentially modify cognitive flexibility. 2.42 Feedback sensitivity Although effects of CRF on standard metrics of flexibility in the PRL task were limited, increased CRF signaling consistently reduced negative feedback sensitivity, while having a more limited effect on reward sensitivity. Specifically, CRF reduced reward sensitivity when the probabilistic contingencies were first being acquired in males but not females. In contrast we found no effects of acute stress on this measure in males on either PRL task, whereas acute stress in females reduced both reward and negative feedback sensitivity, mirroring the effects of CRF infusion in males on this measure. 69 Although CRF and acute stress showed minor differences in terms of sex and task experience, the direction of the effects was the same. That is, both CRF and stress reduced reward and negative feedback sensitivity. These effects are in keeping with other studies finding acute stress-induced alterations in reward and negative feedback sensitivity (Bogdan & Pizzagalli, 2006; Kumar et al., 2014; Lemos et al., 2012; Petzold et al., 2010; Porcelli et al., 2012). Additionally, reduced reward sensitivity is consistent with CRF-induced impairments in motivation and reward-related behaviours, including reducing food consumption, choice of larger rewards requiring more effort, and motivation to work for sugar reward (Bell et al., 1998; Bryce & Floresco, 2016; Cador et al., 1992; Glowa et al., 1992; Heinrichs et al., 1991; Wanat et al., 2013). Reduced feedback sensitivity may be mediated by interactions between CRF and dopamine given the prominent role of dopamine in motivation and reward processing (Wise, 2004) and the CRF-induced reduction in phasic mesolimbic dopamine release in response to rewards (Wanat et al., 2013). As CRF increases dopamine neuron firing rate (Korotkova et al., 2006) and potentiates dopamine release (Matsuzaki et al., 1989), increasing dopamine signaling in terminal regions may serve to obscure dips in phasic dopamine release in response to reward omission (Fiorillo, Newsome, & Schultz, 2008), leading to reduced learning from negative feedback. Another potential candidate mechanism for reducing feedback sensitivity is 5-HT. This is an intriguing possibility as CRF increases the firing rate of 5-HT neurons (Lowry et al., 2000), inducing 5-HT release throughout terminal regions and high doses of exogenous 5-HT reduce negative feedback sensitivity (Bari et al., 2010) in a similar manner to increased CRF signaling. These results are also notable considering that feedback sensitivity is impaired in humans with depression. Indeed, participants suffering from depression show reductions in reward 70 responsiveness (Bogdan & Pizzagalli, 2006; Henriques & Davidson, 2000; Henriques et al., 1994; Pizzagalli et al., 2005, 2009) and use of negative feedback (Beats et al., 1996; Petzold et al., 2010). Depression also impairs the ability to utilize reinforcement to guide learning (Chase et al., 2010; Holmes & Pizzagalli, 2007), perhaps by blunting neural responses to both reward and loss (Treadway et al., 2013; Ubl et al., 2014). Therefore, potentiated CRF signaling may drive depression-related deficits in feedback processing. 2.43 Motivation Increased CRF signaling impaired motivation on both versions the PRL task, as evidenced by an increase in trial omissions. Interestingly, rats omitted trials only slightly during initial discrimination but dramatically omitted trials once reward contingencies were reversed. This was true whether animals had previously experienced a reversal before (performance version) or not (acquisition version), indicating that regardless of training or experience, CRF causes task disengagement, particularly when reward contingencies change. This is interesting considering that previous studies find a ‘catastrophic’ response to errors in depression, with participants substantially more likely to commit subsequent errors given an error on the previous trial (Beats et al., 1996; Elliott et al., 1997). Therefore, when the rules of the task change, or when experiencing consistent negative feedback, rather than attempting to change strategies, animals become disengaged, even though choosing incorrectly is more advantageous (20% chance of reward) than not choosing at all. CRF hyperactivity also caused animals to be more ‘indecisive’. This is in keeping with previous studies that found ICV CRF treatment consistently increased choice latencies using numerous assays (Beard et al., 2015; Bryce & Floresco, 2016; Van’t Veer et al., 2012) and is consistent with psychomotor slowing, a characteristic symptom of depression (Lemke, 1999; 71 Murphy et al., 2001; Schrijvers et al., 2008; Sobin & Sackeim, 1997). These results indicate that increased CRF signaling may play a key role in the amotivational symptoms of depression (Grahek et al., 2019). Similar to CRF, acute stress impaired motivation, albeit in a more limited capacity. Acute stress had no effect on omissions or latency when first acquiring the task. One explanation for the lack of motivational effects during acquisition may be that encountering a novel task with new rules of engagement may be a mild stressor at baseline, with an additional stressor unable to further alter motivation. However, acute stress increased omission rates and choice latencies when animals were well-trained. In contrast to CRF, acute stress increased omissions driven in large part by females relative to males. The reason for this discrepancy is unclear. Conceivably, males may be more sensitive to CRF whereas females may be more sensitive to acute stress. This is unlikely, however, as most studies show that females are more sensitive to the effects of CRF than males (Bangasser et al., 2010; Wiersielis et al., 2016). Perhaps these sex differences are due to differential activation of brain regions in response to central CRF infusion. This is a possibility as females show increased connectivity between the periaqueductal grey and BNST, whereas males show increased connectivity between different regions, such as the mPFC and amygdala, following central CRF infusion (Salvatore et al., 2018). Given that these tasks involve activation of the mPFC (Dalton et al., 2016), differential activation of this region in males and females may lead to distinct CRF-induced behaviours. In regards to acute stress, females appear to be more sensitive to the effects of acute stress, with increased ACTH and CORT following identical acute stressors in female compared to male animals (Babb et al., 2013; Iwasaki-Sekino et al., 2009). Thus, acute stress-induced potentiation of circulating CORT levels in females may independently alter some aspects of motivation and cognition. 72 2.44 Sex differences Overall, we found nuanced sex differences in response to CRF and acute stress manipulations. As stated in the preceding sections, we found that CRF differentially affected males, whereas acute stress differentially affected females on motivational and feedback sensitivity related aspects of PRL tasks; however acute stress only impaired flexibility in males but not females. Notably, there were no effects of estrous cycle on any measure. These null effects may add to a growing literature indicating that cycling ovarian hormones have fewer effects on cognition than originally anticipated (Conrad et al., 2004; Epting, 1998; Stackman et al., 1997). That said, given that females were food restricted and not cycling normally, alterations on these measures by estrous cycle stage may not have become evident. Indeed, most females were in low ovarian hormone stages (diestrus and metestrus) at time of testing. Although, investigating the role of estrous cycle stage on cognition was outside of our current scope, future experiments should include estrous cycle as a variable with relatively equal numbers of females in each stage of the cycle in order to fully determine how fluctuations in ovarian hormones alter cognitive flexibility and motivation. Interestingly, we found different baseline effects dependent on if the groups were previously subjected to surgery (CRF group) or not (stress group). For instance, females were less sensitive to feedback at baseline than males during acquisition and following training if they had not undergone surgery (i.e. in the stress groups) but not if they had undergone surgery (i.e. in the CRF groups). This may also be why females completed fewer reversals during initial training than males in the stress group but not in the CRF group. One explanation for these effects could be that the stress of surgery alters long-term sensitivity to feedback in females, although this has not been explicitly tested. 73 More consistently, we found baseline sex differences in motivation. Overall, females appeared to show reduced motivation to perform the task compared to males, omitting more trials and taking longer to make a choice at baseline. Notably, females omitted at a similar rate as males during initial discrimination but omitted markedly more than males following a change in reward contingencies. This effect was consistent for both manipulations during PRL performance, but not acquisition. However, it may be that omission rates were high for both males and females during acquisition and so sex differences were not as evident. This is interesting considering that previous studies employing various operant chamber tasks have found that females omit more trials (Bayless et al., 2012; Orsini et al., 2016), respond less on fixed ratio responding for sugar reward, and take longer to deliberate when choosing risky (Orsini et al., 2016). Together suggesting that females are less motivated to perform appetitively-motivated operant tasks, or are more easily disengaged, particularly when the rules of the task change. 2.45 Conclusion The current series of experiments reveal that increased CRF signaling slightly facilitates cognitive flexibility during performance on the PRL task, in contrast to lower doses of CRF that impair flexibility with deterministic outcomes (Snyder et al., 2011). Consistent with reward and negative feedback deficits in depressed patients (Beats et al., 1996; Chase et al., 2010; Eshel & Roiser, 2010; Holmes & Pizzagalli, 2007), enhanced CRF neurotransmission consistently reduced feedback sensitivity, demonstrating a role for CRF in mediating depression-related reductions in reinforcement sensitivity. On the other hand, acute restraint stress selectively impaired PRL during acquisition in males, providing further evidence that acute stress impairs cognitive flexibility in males but not females (Laredo et al., 2015; Shields et al., 2016). Acute 74 stress also reduced both reward and negative feedback sensitivity in females dependent on task experience providing nuance for previous work showing stress-induced alterations in feedback sensitivity (Bogdan & Pizzagalli, 2006; Kumar et al., 2014; Lemos et al., 2012; Petzold et al., 2010; Porcelli et al., 2012). Notably, both CRF and acute stress reduced motivation, particularly following a switch in reward contingencies, consistent with motivational deficits in depression (Grahek et al., 2019; Lemke, 1999; Murphy et al., 2001; Schrijvers et al., 2008; Sobin & Sackeim, 1997), indicating that CRF hyperactivity may play a key role in this symptom of depression. Taken collectively, these experiments reveal the complex nature of stress and stress-related mechanisms in perturbing complex features of cognition such as cognitive flexibility and more basic probabilistic reinforcement and how sex-differences further complicate these effects. One of the major strengths of the current study was to assess potential sex differences in both male and female rats. That said, the impetus for our current series of experiments was based on our previous work indicating that both acute stress and central CRF altered effort-related decision making in male rats and so our main goal was to extend these findings. Therefore, while additional sex differences would be of interest to pursue, the rest of this thesis will focus on males. 75 Figure 1. Central CRF infusion reduced motivation and feedback sensitivity in males during acquisition. a) Experimental timeline. b) CRF infusion increased omission rates for the initial discrimination and reversal phases and c) increased average choice latency (in seconds) in males but not females. d) CRF infusion had no effect on the number of errors per 100 completed trials or the e) number of errors during the initial discrimination and first reversal phase. f) Win-stay, lose-shift tendencies were reduced in males following CRF infusion with no effect in females. 76 Figure 2. Central CRF infusion reduced motivation and negative feedback sensitivity during performance. a) Experimental timeline. b) Initial training was similar for males and females. c) CRF infusion slightly increased omission rate during initial discrimination and substantially increased omission rate during the reversal phase and females had a higher omission rate than males (inset). d) CRF infusion increased choice latencies and females had a slower latency than males. e) CRF had no effect on the number of errors per 100 completed trials or f) the number of errors committed during the initial discrimination and first reversal phases. g) CRF infusion slightly increased the number of reversals per 100 completed trials and h) significantly reduced negative feedback sensitivity. 77 Figure 3. Restraint stress impaired flexibility in males during acquisition. a) Experimental timeline. b) Restraint stress had no effect on omission rate for the initial discrimination and reversal phases of the task but females had a higher omission rate than males (inset). c) Restraint stress had no effect on average choice latencies but females were slower than males. d) Restraint stress increased the total number of errors per completed 100 trials in males, but not females. e) However, there was no effect of restraint stress when dividing the number of errors into initial discrimination and first reversal phases. f) There was no effect of restraint stress on win-stay lose-shift tendencies but females were less sensitive to feedback than males (inset). 78 Figure 4. Restraint stress reduced motivation and feedback sensitivity in females during performance. a) Experimental timeline. b) Females completed fewer reversals per session over the course of training. c) Restraint stress increased overall omission rate, driven by an increase in the reversal phase (inset-left). Restraint stress increased the omission rate in females relative to males and females omitted more across groups (inset-right). d) Restraint stress increased choice latencies and females were slower than males. e) Restraint stress had no effect on the number of errors per 100 completed trials but females committed more errors than males (not shown). f) Restraint stress had no effect on errors during initial discrimination and first reversal but females committed more errors than males (not shown). g) Restraint stress also had no effect on the number of reversals per 100 trials. h) Restraint stress reduced win-stay lose-shift tendencies in females, but not males, and females were less sensitive to feedback than males (inset). 79 Chapter 3: Differential effects of CRF and acute stress on different forms of risk/reward decision making 3.1 Introduction Stress and cost/benefit decision making share a reciprocal relationship in that decision-making can sometimes be stressful and stress can influence decision-making. Over the past 20 years, considerable advances have been made in delineating the neural circuits mediating cost/benefit decision-making in humans and animals, yet, how stress and its potential underlying mechanisms may influence these decisions remains to be fully explored. Recent findings suggest that the effect of stress on decision making depend in part on the type of cost involved. For example, studies in humans found that stress-induced potentiation of CORT reduces choice of larger rewards associated with a longer delay (Fields et al., 2014; Kimura et al., 2013) but increases choice of larger rewards delivered in a probabilistic manner (Preston et al., 2007; Putman et al., 2010; Simonovic et al., 2017; Starcke et al., 2008; Wemm & Wulfert, 2017). Similarly, stress-related disorders such as depression are associated with a shift in choice away from the more preferred yet delayed rewards (Pulcu et al., 2014) or those requiring more physical effort to obtain (Treadway et al., 2012). In addition, preclinical studies in rats have shown that one hour of acute restraint stress or intra-cranial infusion of the stress-related neuropeptide, CRF, reduces preference for larger rewards that require more effort, but not delayed rewards (Bryce & Floresco, 2016; Shafiei et al., 2012). These findings are notable in light of the fact that depression is associated with increased CRF (Banki et al., 1987; Nemeroff et al., 1984), which suggest that perturbations in certain forms of cost/benefit decision making may be driven by excessive CRF activity. Another type of decision making entails evaluations involving reward uncertainty. In this regard, studies across species indicate that acute stress, administration of stress hormones, or 80 stress-related disorders are associated with complex and sometimes differential alterations in risk/reward decision-making. For example, acute stress impaired learning to choose advantageous options and increased choice of the risky, disadvantageous options on the IGT, in which participants choose between decks of cards with differing reward and punishment probabilities (Preston et al., 2007; Simonovic et al., 2017; Wemm & Wulfert, 2017). Interestingly, in a rodent version of the IGT, exogenous CORT administration or exposure to learned helplessness led to more risky choices in male rats (Koot et al., 2013; Nobrega et al., 2016). On the other hand, there have been some reports that depressed patients tend to choose from the more risky, disadvantageous decks (Cella et al., 2010; Han et al., 2012; Moniz et al., 2016; Must et al., 2006), whereas others have either failed to observe these effects (Deisenhammer et al., 2018; Gorlyn et al., 2013; Jollant et al., 2016; McGovern et al., 2014) or observed reductions in risky choice (Smoski et al., 2008). Decisions under uncertainty often involve considerations of risk and ambiguity. To disentangle risk from ambiguity, studies in humans have used tasks that explicitly inform participants of the probabilities associated with the outcomes. Using these types of task, studies have shown that acute stress or treatment with exogenous CORT increases risky choice (Starcke et al., 2008; Putman et al., 2010). Yet, acute stress did not affect risky choice on the CGT, another type of task in which participants are provided information about reward probabilities (Gathmann et al., 2014). On the other hand, depressed patients and subsets who have attempted suicide tend to make poorer quality decisions on the CGT (Clark et al., 2011; Rubinsztein et al., 2006; Taylor Tavares et al., 2007). In line with these data, preclinical studies in rodents using a cued version of the IGT have revealed that chronic unpredictable stress led to more lower risk choices, even when choosing risky was more favourable (Morgado et al., 2015). Together the 81 majority of studies across species indicate that acute stress or potentiated stress hormones increase risky choice in various paradigms, perhaps by altering reward processing (Bogdan & Pizzagalli, 2006; Kumar et al., 2014; Lemos et al., 2012; Porcelli et al., 2012) or negative feedback sensitivity (Petzold et al., 2010). As mentioned above, we have recently identified the stress neuropeptide, CRF, as a necessary and sufficient mediator of the stress-induced effects on choice when decisions involve effort costs, shifting bias away from more costly rewards (Bryce & Floresco, 2016). The present study was designed to expand on these findings and elucidate how increased CRF activity or acute stress may influence cost/benefit decision making in situations involving reward uncertainty. Specifically, we assessed how central CRF infusions or acute restraint alters risky choice, using two distinct assays that measured risk/reward decision-making guided either by internal representation of action outcome contingencies or external cues that guide choice behaviour. To this end, we used a probabilistic discounting task, which requires animals to keep track of previous rewarded outcomes in order to modify choice based on changing probabilities of reward (Jenni et al., 2017; Stopper et al., 2013), and a recently developed Blackjack task (Floresco et al., 2018), in which auditory cues inform the decision maker about the relative probability of obtaining larger, risky rewards. 3.2 Methods Most experimental procedures were similar to those from Chapter 2. Thus, only deviations from the previous procedure will be described here. 3.21 Animals Separate cohorts of male Long Evans rats were utilized for all experiments, and weighed between 250-275 g prior to training. Rats were initially pair housed and food restricted to 85% of 82 free-feeding weight following one week of acclimatization. Rats underwent stereotaxic surgery following behavioural training and were subsequently individually housed for recovery and testing. 3.22 Apparatus The operant chambers were fitted with a speaker located in the top-center of the wall opposite the levers. The speaker was connected to a programmable sound generator (ANL-926, Med Associates), through which auditory stimuli were delivered. 3.23 Risk/reward decision making tasks 3.23.1 Probabilistic discounting. Following initial training (see Chapter 2 methods), one group of rats was trained on a probabilistic discounting task as described previously (St. Onge et al. 2010; Stopper et al. 2013) (Fig 5A). Rats received daily sessions, 5-7 days per week, consisting of 90 trials, separated into five blocks of 18 trials. Trials started every 40 s with illumination of the house light and 3 s later, one or both levers were extended. One lever was designated the large/risky lever, the other the small/certain lever, which remained consistent throughout training. If the rat did not respond within 10 s of lever extension, the house light would turn off and the chamber was reset to the intertrial state and the trial was scored as an omission. Choice of either lever caused both levers to retract, the house light would remain on for another 4s and then turn off until the next trial commenced. Choice of the small/certain lever always delivered one pellet with 100% probability, whereas choice of the large/risky lever delivered four pellets but with the odds of receipt varying depending on the particular trial block. Upon a large/risky win, four pellets were delivered 0.5s apart. Each of the five probability blocks consisted of eight-forced choice trials where only one lever was presented (four trials for each, randomized in pairs) enabling animals to learn the reward magnitude and probability of receiving 83 reinforcement over each block. This was followed by 10 free-choice trials, where both levers were extended and the rat had to decide whether to choose the small/certain or the large/risky lever. The probability of obtaining four pellets after pressing the large/risky lever was varied systematically in either descending or ascending order across the five blocks, with separate groups of rats trained on the descending or ascending variant. For the descending variant of the task, the large reward probability was initially 100%, then decreased to 50%, 25%, 12.5%, and 6.25%. Conversely, the ascending variant started at 6.25%, then increased to 12.5%, 25%, 50%, and 100%. When the odds of obtaining the large reward was 100% or 50%, this option would be more advantageous, however in the 25% block, both options had equal long-term utility. For the other two blocks of trials, the 12.5% and 6.25% blocks, risky choice was disadvantageous and therefore the small/certain lever had greater utility. Rats were trained for ~ 15 days, after which they displayed stable patterns of choice behaviour, determined by analyzing data from three consecutive sessions with a two-way repeated measures ANOVA, with day and odds as factors. Choice behaviour was deemed stable if there was no main effect of Day and no Day x Odds interaction (p > 0.10). They were then subjected to surgery, retraining and testing, as described in subsequent sections. 3.23.2 Blackjack task. A separate group of rats was initially trained on the Blackjack Task (Fig 5B). These rats were initially trained to lever press as above, followed by reward magnitude training as detailed previously (Floresco et al. 2018). The first phase consisted of 2-3 days of training on a 48-trial task that was divided into four blocks of two forced-choice trials followed by 10 free-choice trials. Trials were initiated every 40 s with insertion of one or two levers. Pressing one lever always delivered four pellets, whereas the other lever always delivered one pellet, with the lever associations remaining consistent throughout subsequent training. Rats 84 were then trained for another 2-3 days on a modified version of reward magnitude discrimination with sessions consisting of 72 trials divided into four blocks of eight forced-choice and 10 free-choice trials. Choosing the small reward lever always delivered one pellet, whereas choice of the large reward lever delivered four pellets with 50% probability. The following day, training on the Blackjack task commenced and consisted of two phases. The initial phase of Blackjack training was a forced-choice version, wherein sessions consisted of 52 trials; the first 32 trials were forced-choice (randomized in pairs, 16 trials with each lever) followed by 20 free-choice trials, where both levers were inserted. Once a group of rats displayed stable choice behaviour, they were trained on the second and final, free-choice version of this task, which consisted of 40 free-choice trials (27 mins), which were identical to the initial phase. For both forced- and free-choice versions of the Blackjack task, trials began every 40 s with house light illumination and presentation of one of two distinct auditory cues (3kHz pure tone or white noise at 80 dB). Both cues were presented an equal number of times pseudorandomly over the session (randomized in pairs). Three seconds after the trial started, one or both levers were extended into the chamber. Choice of the small/certain lever immediately turned off the auditory cue and delivered one pellet with 100% probability, irrespective of which cue was presented. Choice of the large/risky lever could deliver four or zero pellets. The probability of obtaining the larger reward on a particular trial was indicated by which auditory cue was presented. One auditory cue was associated with “good-odds” trials, where a risky choice delivered the large reward with 50% probability. The other auditory cue signaled “poor-odds” trials, where a risky choice was rewarded with 12.5% probability. Therefore, choosing the large/risky option was more advantageous on good odd trials, whereas on poor odd trials, the 85 small/certain option had greater utility. Auditory cues associated with good vs poor odd trials were counterbalanced across rats and remained consistent over the duration of the experiment. Choice of either lever caused both levers to retract. If the rat chose the large/risky option and received a reward, the cue and house light remained on during the delivery of the four pellets and turned off 3s after choice. Large/risky choices that did not deliver reward extinguished the house light immediately and the auditory cue was terminated 2s after the choice. The extended presentation of the auditory cue was intended to facilitate learning of the predictive value of these cues and the likelihood of the different outcomes associated with these choices. Following an omission, both levers were retracted and the house light and auditory cue were extinguished. On forced-choice trials when the large/risky lever was extended, the auditory cue indicated the respective probability of obtaining the large reward on that trial (50% or 12.5%). For forced-choice trials where the small/certain lever was inserted, an equal number of each auditory cue was presented, so that rats could learn that a response on this lever always delivered one pellet, irrespective of the auditory cue presented. Rats were trained for 12-17 days on the forced-choice version of the task until they displayed stable patterns of choice, using analyses similar to those used to determine stability for the probabilistic discounting task. Rats then progressed to the final phase of training (40 free-choice trials) for another 5-6 days until they again displayed stable patterns of risky choice over 3 days. Rats then underwent stereotaxic surgery and were retrained on the task (the first day with the forced-choice version, followed by the free-choice version) for at least 5 days until stable choice behaviour was re-established. Rats subsequently received counterbalanced drug and stress manipulations prior to testing on the free-choice version of the Blackjack task. For the analysis, we included data from rats that displayed rational patterns of choice on good odds trials (i.e.: ≥50% choice of the risky option on these trials) during baseline 86 and vehicle test days. Eight rats did not meet this inclusion criterion and their data was not included in the analysis. 3.24 Drugs and microinfusion protocols Rats underwent intra-cranial surgery (see Chapter 2 methods) following behavioural training. Rats recovered from surgery and were retrained to stability on the probabilistic discounting or Blackjack task for at least 5 days until they showed stable baseline choice for 3 consecutive days. One or two days prior to drug testing, rats received a mock infusion where the stylet was removed from the guide cannula and a stainless-steel injector was inserted for the duration of the subsequent infusion; however, no infusion was delivered. The day after showing stable choice, the group received its first microinfusion test day. Two doses were chosen for these experiments. The higher CRF dose was based on previous work in our lab finding that 3μg CRF reduced choice of the more preferable but more costly reward in an effort discounting task (Bryce & Floresco, 2016). Although lower CRF doses did not significantly alter choice during effort discounting or alter flexibility during pilot studies in the PRL task described in Chapter 2, initial tests in risk/reward decision making tasks suggested that there may be dose-dependent effects on risky choice. As such, adding a second, lower dose in the analysis to probe potential CRF dose effects was prudent to understand how different doses of this compound may interact to alter choice behaviour involving reward uncertainty. All tests used a within-subjects design. CRF or vehicle (aCSF) was infused at a volume of 3 μl for high dose infusion and 1 μl for low dose infusion. The differential volume was due to the lower solubility of CRF in aCSF (1μg/1μl). Fresh CRF was prepared prior to the first CRF infusion, aliquoted into smaller doses and frozen until the second test day when it was unthawed 87 immediately prior to infusion. CRF doses in the probabilistic discounting and Blackjack task were administered in a counterbalanced order across rats, using a within-subjects design. On the first day of testing, rats received counterbalanced infusions of either one of two volumes of aCSF (1 or 3μl). The following day, rats received the same dose of CRF that corresponded to the volume on the previous test day (i.e. if rats received 1μl vehicle the first day they would receive 1μg CRF the following day, whereas if they received 3μl vehicle the first day they would receive 3μg CRF the following day). Rats were retrained on the task for at least three days or until they re-attained stable patterns of choice behaviour for three consecutive days prior to receiving the second round of vehicle and then CRF infusions. After infusions, rats were given at least one week of retraining to re-stabilize choice behaviour prior to undergoing restraint stress testing. At least one week after the final round of vehicle/CRF infusions, we tested the effects of acute restraint stress on performance of the probabilistic discounting or Blackjack tasks. 3.25 Histology Following testing, rats were euthanized with CO2 and brains were removed and fixed in 4% formalin solution. Brains were sectioned at 50 μm, mounted on gel-coated slides and Nissl stained using Cresyl Violet. Sixty-one rats were trained and tested on separate behavioural tasks. Six animals were removed from the CRF analysis due to lateral ventricle placements that were too dorsal (n = 4) or too medial (n = 2), leaving 54 rats for analysis. Animals that had missed placements were included in the subsequent restraint stress analysis. 3.26 Data analysis 3.26.1 Probabilistic discounting. The primary dependent variable for all experiments was proportion of choices on the large, risky lever. For the probabilistic discounting task this entailed 88 computing the proportion choice on the large risky lever for each block of free-choice trials factoring out omissions. For each block, we divided the number of risky choices by the number of successfully completed trials (i.e. when the rat made a choice on either lever). Additional measures included choice latencies, defined as time between lever insertion and choice, number of trial omissions and locomotor activity (indexed by photobeam breaks). We also conducted choice-by-choice analyses to clarify how our manipulations affected choice as a function of the outcome of a preceding trial (reward or non-reward). The proportion of win-stay trials was calculated from the number of times a rat chose the large/risky lever after choosing the risky option on the preceding trial and obtaining the large reward (a win), divided by the total number of free-choice trials where the rat obtained the larger reward. Changes in win-stay performance were used as an index of reward sensitivity. Alternatively, lose-shift performance was calculated from the number of times a rat shifted choice to the small/certain lever after choosing the risky option on the preceding trial and was not rewarded (a loss), divided by the total number of free-choice trials resulting in a loss. Changes in lose-shift performance functioned as an index of negative-feedback sensitivity. Win-stay and lose-shift analyses were conducted for all trials across the five blocks. We were unable to conduct a block-by-block analysis of these data because there were many instances where rats either did not select the large/risky lever or did not obtain the large reward at all during the latter blocks. Risky choice data was analyzed using a three- or four-way mixed ANOVA. For animals receiving CRF infusion, choice data was analyzed with an ANOVA model consisting of Dose (low vs high volume/dose for vehicle/CRF infusions), Treatment (vehicle vs CRF), and Probability Block (100, 50, 25, 12.5, 6.25% odds of receiving reward, irrespective of the order presented) as a within-subjects factors and Order variant (ascending or descending odds) as a 89 between-subjects factor. Omission data were analyzed in a similar manner, except here, the block factor was Block number (1st to 5th block, irrespective of the reward probability in that block). This was done because it became apparent that omission rates tended to increase later in a test session, irrespective of the task variant animals were trained on. In these analyses, main effects of Treatment would indicate a difference between vehicle and CRF treatments irrespective of dose, whereas Treatment x Dose interactions would indicate that one dose of CRF caused a greater change of a particular variable vs another. Data from the restraint stress tests were analyzed with Treatment (no stress baseline vs one-hour restraint), and Trial block as within-subjects factors and Order variant as a between-subjects factor. In these analyses, the main effect of block was always significant (p < 0.01) and will not be reported further. Win-stay/lose-shift data was analyzed with ANOVA models using Dose, Treatment, and Feedback response (win-stay vs lose-shift) as within-subjects factors for the CRF experiment, whereas in the stress experiment these data were analyzed with Treatment and Feedback type as within-subjects factors. Choice latency was averaged over the test session as there was no significant effect of latency based on whether the animal chose risky or certain options (data not shown) and was, along with locomotor data, analyzed with a two-way ANOVA with Dose and Treatment as within-subjects factors in the CRF experiments, and a one-way ANOVA, with Treatment as a within-subjects factor for the restraint stress experiment. 3.26.2 Blackjack task. For the Blackjack task, the primary dependent variable of interest was the proportion of choices of the large/risky option on good and poor-odds trials, factoring out trial omissions. Choice latency was again averaged over the course of the test session as we found no significant differences between treatments on this measure when partitioned based on 90 choice (risky vs certain) or trial type (good vs poor odds; data not shown). Additional measures included the number of trial omissions and locomotor activity. For the Blackjack task, we conducted two types of win/stay lose/shift analyses. The first type was similar to that detailed above, and assessed how the outcome of the most recent risky choice influenced subsequent choice (hereafter referred to as “Recent Outcome”). Win-stay ratios were calculated by taking the number of risky choices following receipt of the larger reward (a risky win) divided by the total number of larger rewards obtained. Lose-shift ratios were calculated as the number of small/certain choices following a non-rewarded risky choice (risky loss) over the total number of non-rewarded choice trials. For these analyses, we further subdivided win-stay and lose-shift ratios based on the odds the animals faced during a particular choice (good vs poor), irrespective of the odds they faced on the preceding choice. Accordingly, the odds on the current trial could either be the same or different from the odds on the previous outcome trial. In contrast to the probabilistic discounting task, where the odds on the next trial in a sequence are most often the same as the odds in the previous trial, the probability of receiving the large reward on a particular Blackjack trial is independent of the outcome on the preceding trial. Therefore, a reliance on a win-stay or lose-shift strategy based on the outcome of the previous trial was not necessarily advantageous in the context of the Blackjack task. To account for this, we conducted another type of win-stay/lose shift strategy which we refer to as “Same Odds” win-stay/lose-shift behaviour. This analysis computed how the outcome of a risky choice on a good or poor-odds trial influenced choice on the next trial of the same type (good or poor odds). For instance, if a rat chose risky on a good odds trial and was rewarded, we assessed win-stay behaviour on the next good odds trial, which due to the structure of the task could be 1-3 trials later in the choice sequence. If a rat chose the risky lever on a subsequent 91 good odds trial, the response was scored as a “win-stay | good odds” trial. To analyze this type of win-stay/lose-shift behaviour (hereafter referred to as “Same Odds”), we first calculated the number of trials on which each animal selected the risky lever as a function of the Odds (good vs. poor) and Outcome on that trial (win vs. loss), resulting in four conditions: win | good odds, loss | good odds, win | poor odds, and loss | poor odds. Looking at all animals, there were very few wins on poor odds trials (i.e.; being rewarded for a risky choice on a poor odds trial). Given that only 2-3 of those trials would be rewarded if rats chose risky on all 20 poor odds trials in a session (20% probability of reward), and that the risky lever is not the advantageous choice on poor odds trials, the number of risky wins on this trial type would be expected to be low. Indeed, none of the rats received more than two risky wins on a poor odds trial throughout all test sessions, therefore, we were unable to analyze data for “win-stay | poor odds” condition. Instead, “win-stay | good odds”, “lose-shift | good odds”, and “lose-shift | poor odds” data were analyzed separately. Win-stay | good odds ratios were calculated as the number of risky wins on a good odds trial followed by another risky choice on the next good odds trial in the sequence, divided by the total number of risky wins on good odds trials. Lose-shift | good odds and lose-shift | poor odds ratios were calculated as the number of non-rewarded risky choices followed by certain choices on good odds (or poor odds) trials divided by the total number of non-rewarded good odds (or poor odds) trials. A difficulty in conducting these analyses was that, across the different test sessions, there were some rats that responded only 0-2 times on a particular trial type, which we felt was an insufficient number to include in a particular analysis. To overcome this complication, we only analyzed data from animals that experienced a trial type (Feedback x Odds) at least 3 times 92 during a session. Thus, each of these analyses only included a subset of rats that experienced a particular trial type at least 3 times under all test conditions (see Supplementary Table 1). For the CRF experiments, choice, omissions, and the Recent Outcome win-stay/lose-shift data were analyzed using a two- or three-way within-subjects ANOVA, with Dose (low vs high dose vehicle and drug), Test (vehicle vs drug), and Trial type (good vs poor-odds trials) as within-subjects factors. Analysis of the choice data revealed that rats chose the large/risky option more often on good vs poor odds trials (all Ps < 0.001), demonstrating they were able to distinguish good and poor odds trial probabilities from their respective auditory cues, and as such will not be reported further. Choice latency and locomotion were analyzed using a two-way ANOVA, with Dose and Test as within-subjects factors. Lastly, the Same Odds win-stay/lose-shift data were analyzed using three separate ANOVAs: “win-stay | good odds”, “lose-shift | good odds”, and “lose-shift | poor odds” with Dose and Treatment as within-subjects factors. For the restraint stress experiment, choice data were analyzed with a two-way ANOVA, with Test (no stress vs one-hour restraint stress) and Trial type as within-subjects factors. Choice latency and locomotion were analyzed with one-way ANOVAs. Same Odds win-stay/lose-shift data were analyzed using three separate ANOVAs: “win-stay | good odds”, “lose-shift | good odds”, and “lose-shift | poor odds” with Treatment as a within-subjects factor. Lastly, for all of analyses of the CRF infusion data, there were no differences in performance following infusions of 1 l vs 3 l vehicle (all Ps > 0.10). For clarity, each figure presents the average vehicle data alongside data obtained on test days where rats received 1 and 3 g CRF. 93 3.3 Results 3.31 Probabilistic discounting 3.31.1 Central CRF infusion. Of the 34 rats that were tested on the probabilistic discounting task, 1 was unable to complete all 4 rounds of testing and 4 were removed due to missed placements, leaving data from n=29 rats included in the analyses. This comprised separate groups of animals that were trained on one of the two order variants of the task in which odds of receiving the large risky reward either decreased (descending order, n=16) or increased over blocks of trials (ascending order, n=13). Infusions of CRF induced a marked increase in the number of trial omissions (Fig 6A). Analysis of these data revealed a significant main effect of Treatment (F(1,27) = 10.26, p < 0.01) but no main effect or interactions with the Dose factor (F(1,27) < 1, n.s.; Treatment x Dose, Dose x Block, Treatment x Dose x Block interactions: all Fs < 1.95 all Ps > 0.17), indicating that both doses induced a comparable increase in omissions relative to control treatments. Interestingly, the analysis also yielded a Treatment X Block interaction (F(4,108) = 4.55, p < 0.01). As displayed in Figure 6A, CRF infusions did not significantly alter omissions relative to vehicle during the first block (F(1,28) = 2.07, n.s.), but did significantly increase omissions in the latter trial blocks (all Ps < 0.05). Notably there were no interactions with the Order Variant factor (all Fs < 1.96 all Ps > 0.11, Fig 6A), indicating that the increase in omissions in the latter part of the sessions occurred irrespective of whether the probability of obtaining the larger reward was higher or lower compared to the first block. Thus, CRF infusions increased the tendency for rats to not make a choice, with this effect being more prominent later in the test session. Both doses of CRF also increased average choice latency, as indicated by a main effect of Treatment (F(1,27) = 4.27, p < 0.05, Fig 6B) but no effects of Dose (main effect of Dose and 94 Treatment x Dose interaction: Fs < 1 Ps > 0.78) or Order (three way interaction: F(1,27) = 3.10, n.s.). Interestingly, CRF infusion had no effect on locomotion in this task (all Fs < 3.16 all Ps < 0.09, Table 2). Of the 29 rats tested, 5 did not make any choices in at least one block of 10 trials during at least one of their test sessions (4 in the descending and 1 in the ascending groups), making analysis of their choice data problematic. As such we analyzed choice data on the remaining 24 rats that made a sufficient number of choices in each of the blocks (n = 12 in each group). In contrast to the effects of CRF on trial omissions and choice latencies, these treatments did not alter risky choice (main effect of Treatment, main effect of Dose, Treatment x Dose interaction: all Fs < 1.37, all Ps > 0.26, Fig 6C). There were also no effects of the Order variant or any interactions with this factor (all Fs < 1.09 all Ps > 0.31) or probability block (all Fs < 1.69 all Ps > 0.16, Fig 6C, inset). Despite the lack of effect of CRF treatments on overall levels of risky choice, we further probed how these treatments might alter sensitivity to reward and negative feedback. In keeping with the lack of effect on choice, CRF infusion also did not affect win-stay or lose-shift behaviour on this task (all Fs < 2.38 all Ps > 0.20, Table 3). 3.31.2 Restraint stress. Rats were then retrained for at least one week until they re-established baseline choice performance. Thirty-three animals were tested and included in subsequent analysis. We analyzed choice data as above with order of odds (ascending: n=14; descending: n=19) as a between-subjects factor and probability block as a within-subjects factor. Acute stress significantly increased the number of trial omissions (Fig 7A), with a significant main effect of Treatment (F(1,31) = 8.94, p < 0.01). There was also a significant main effect of Block (F(4,124) = 4.73, p < 0.01), with omissions increasing over blocks of trials, yet 95 there was no Treatment X Block interaction (F(4,124) = 1.47, n.s.), indicating that stress increased omissions regardless of block. There was also no Treatment X Order interaction (F(1,31) < 1, n.s.), again suggesting that the increase in omissions in the latter blocks of the session occurred regardless of whether the probability of obtaining the larger reward was higher or lower compared to the first block. Thus, unlike CRF, acute stress increased trial omissions over all blocks of trials, including in the first block of the session. Restraint stress did not affect choice latency (main effect of treatment: F(1,31) = 2.87, n.s.; Treatment X Order interaction (F(1,31) = 3.42, n.s., Fig 7B). Additionally, acute stress reduced locomotion (F(1,31)=32.25, p < .001, Table 2). Together, these data indicate that, unlike increased CRF signaling, acute stress had no effect on deliberation time but reduced locomotion. Of the 33 rats tested, 8 did not make any choices in at least one block of 10 trials during at least one of their test sessions (5 in the descending and 3 in the ascending groups). As such, we analyzed the choice data on the remaining 25 rats that made a sufficient number of choices in each of the blocks (14 in the descending group and 11 in the ascending group). We found that one hour of restraint stress did not significantly affect risky choice (Fig 7C). Specifically, we found no main effect of Treatment (F(1,23) < 1, n.s.), and no Treatment X Order interaction (F(1,23) < 1, n.s., Fig 7C, inset). Additionally, we did not find any interaction with Block (Treatment x Block, Treatment x Block x Order: all Fs < 1 Ps > 0.52), suggesting that, similar to CRF infusion, acute stress had no effect on risky choice in this assay. Reward and negative feedback sensitivity were also not affected by restraint stress (Table 3). Specifically, there was no main effect of Treatment (F(1,23) < 1, n.s.) or any significant interaction with the other factors (all Fs < 2.63, Ps > 0.12) indicating that acute stress did not induce any discernable change in sensitivity to recently rewarded or non-rewarded choices. 96 3.32 Blackjack task 3.32.1 Central CRF infusion. Data from 17 rats that displayed optimal patterns of choice under baseline/vehicle conditions and had an accurate cannula placement were included in the analysis. Analysis of risky choice revealed that CRF infusion caused a selective decrease in risky choice on good odds trials, without affecting choice on poor odds trials (Fig 8A). Specifically, the analysis produced a significant Treatment x Odds interaction (F(1,16)= 5.65, p < 0.05), but no main effects of Treatment, Dose or interactions with these terms (all Fs < 2.93 all Ps > 0.11). The Treatment x Odds interaction was driven by the fact that both doses of CRF reduced risky choice on good odds trials (F(1,16) = 7.33, p < 0.05) while at the same time, having no effect on poor odds trials (F(1,16) < 1, n.s.). CRF infusion increased choice latency in a dose-dependent manner (Fig 8B). Specifically, the analysis revealed a significant Treatment by Dose interaction (F(1,16) = 7.16, p < 0.05), in the absence of main effects of Treatment (F(1,16) = 5.53, p < 0.05) or Dose (F(1,16) = 2.06, n.s.). Simple main effects further confirmed that only the 3μg dose (F(1,16) = 6.48, p < 0.05) but not the 1μg dose (F(1,16) < 1, n.s.) increased choice latency. Furthermore, in contrast to the effects observed in the probabilistic discounting experiment, CRF infusion reduced locomotion during the Blackjack task (main effect of Treatment (F(1,16) = 10.96, p < 0.01; interaction with Dose: all Fs < 1.06 Ps > 0.32, Table 2). Yet, these treatments did not affect trial omissions (all Fs < 2.4 all Ps > 0.15, Table 2). To further investigate whether CRF-induced reductions in risky choice on good odds trials was associated with changes in responding to recently rewarded/non-rewarded choices, we assessed how the outcome after a risky choice influenced action selection on the subsequent trial in the choice sequence. CRF did not alter win-stay or lose-shift behaviour, as there were no 97 significant effect of Treatment (F(1,16) < 1, n.s.) Dose (F(1,16) = 3.20, n.s.), or Treatment x Dose interactions (F(1,16) = 3.02, n.s., Table 3). Furthermore, there were no effects of Treatment and/or Dose on Feedback or Trial type (all Fs < 2.80 Ps > 0.11, Table 3). Thus, as was observed in the probabilistic discounting experiment, increased CRF activity did not alter the manner in which recently rewarded or non-rewarded risky choices influenced subsequent action selection. In the Blackjack task, the odds of obtaining the larger reward are signaled by external cues and vary on a trial-by-trial basis in that the odds on the current trial are not necessarily the same as those on the previous trial. We therefore conducted a second win-stay/lose-shift analysis that assessed how the outcome of a risky choice influenced choice on the next trial of the same type (i.e. good or poor odds). CRF infusion did not affect win-stay behaviour on good odds trials (win-stay | good odds: F(1,14) < 1, n.s., Fig 8C-left) nor did it affect lose-shift behaviour on these trials (lose-shift | good odds: F(1,14) = 2.29, n.s., Fig 8C-middle). Furthermore, CRF did not alter sensitivity to losses from one poor odds trial to the next (lose-shift | poor odds: F(1,7) < 1, n.s., Fig 8C-right). In each of these analyses, there were no significant effects of Dose or Treatment x Dose interactions (all Fs < 1.83 all Ps > 0.20). Together with the choice data, these analyses indicate that CRF signaling reduced risky choice when external cues signaled the probability of obtaining larger reward were relatively high. However, this did not appear to be driven by to reduced sensitivity to rewarded choices or increased sensitivity or non-rewarded ones. 3.32.2 Restraint stress. Eighteen rats were retrained on the Blackjack task for at least one week until they re-established stable choice behaviour and then rats were subjected to acute, one-hour restraint stress. This included data from 17 rats used in the above-mentioned analysis, and another rat with inaccurate surgical placement. In contrast to the effects of CRF, restraint stress 98 tended to increase risky choice on good and poor odds trials (Fig 9A). Despite this apparent increase, analysis of the choice data revealed a main effect of Treatment that only approached statistical significance (F(1,17)= 3.32, p = 0.08), with no Treatment x Odds interaction (F(1,17) = 1.32, n.s., Fig 9A). In this experiment, stress did not affect choice latency (F(1,17)= 1.13, n.s.; Fig 8B), or number of trials omitted (F(1,17) < 1, n.s., Table 2), but significantly reduced locomotion (F(1,17) = 22.51, p < 0.001, Table 2). When we examined how outcomes of a risky choice influenced decision making on the subsequent trial, we found no effect of stress (F(1,17) < 1, n.s.), irrespective of Feedback response (Treatment x Feedback interaction: F(1,17) = 2.52, n.s.) or Trial type (Treatment x Odds interaction: F(1,17) < 1, n.s.; 3-way interaction: F(1,17) = 2.49, n.s., Table 3). When we assessed how the outcome of a risky choice influenced choice on the next trial of the same type, we observed that stress increased win-stay tendency across good odds trials (win-stay | good odds: F(1,17) = 7.79, p < 0.05, Fig 9C-left). However, stress had no effect on negative feedback sensitivity on either good odds trials (lose-shift | good odds: F(1,15) < 1, n.s., Fig 9C-middle) or poor odds trials (lose-shift | poor odds: (F(1,10) < 1, n.s., Fig 9C-right). Thus, acute stress increased the influence that rewarded risky choices on a good odds trial had on action selection on subsequent good odds trials. 3.4 Discussion Central CRF infusion and acute restraint stress differentially altered risk/reward decision making in a manner dependent on whether decisions were guided by internal representations of reward contingencies or by external cues that informed of the probability of obtaining larger rewards. Neither manipulation altered risky choice in the probabilistic discounting task, where choice was guided by internally-generated information about reward history. In stark contrast, 99 CRF infusion and acute stress altered risky choice on the Blackjack task, where auditory stimuli informed animals of whether the odds of obtaining larger rewards were good or poor. Interestingly, CRF infusions reduced risky choice selectively when the odds of receiving the larger reward were advantageous, whereas acute restraint stress tended to increase choice of the risky option. We also observed differences between CRF infusion and acute stress on other performance measures across tasks, most notably that the former consistently increased decision latencies. 3.41 CRF and acute stress effects on risky choice The lack of effect of increased CRF signaling or acute stress on risky choice in the probabilistic discounting task stands in notable contrast to how these manipulations affect other forms of cost/benefit decision making. Specifically, both CRF administration and acute stress reduced choice of larger rewards associated with a greater effort cost, but not those delivered after a longer delay (Bryce & Floresco, 2016; Shafiei et al., 2012). This combination of findings suggests that, at least on the surface, CRF and stress may have a somewhat selective effect on effort-related decision making. The lack of effect of stress on probabilistic discounting also lies in contrast to other reports that stress can under some conditions increase risky choice in human and rodent studies on versions of the IGT (Koot et al., 2013; Nobrega et al., 2016; Preston et al., 2007; Simonovic et al., 2017; Wemm & Wulfert, 2017). These differential effects may be related to differences in design and information processed across the various tasks. For example, different versions of the IGT used with humans and rats require subjects to hold risk/reward information about four different choice options, which may tax working memory processes to a greater degree than the two options used in the probabilistic discounting task here. Indeed, cross-species studies show 100 that increased circulating stress hormones or increased CRF impairs working memory functions (Duncko et al., 2009; Hupalo & Berridge, 2016; Roozendaal, 2004; Shields et al., 2015; Uribe-Mariño et al., 2016). On the other hand, the probabilistic discounting task also requires flexible adjustments in choice biases in response to changes in risk/reward contingencies over a session. In this regard, previous work in humans and animals have shown that increased CRF or acute stress can either impair or improve performance on tasks that require cognitive flexibility, such as set-shifting and reversal learning (Bryce & Howland, 2015; Butts et al., 2013; Gabrys et al., 2019; Graybeal et al., 2011; Laredo et al., 2015; Shields et al., 2016; Snyder et al., 2011; Thai et al., 2013). Notably, these classic tests of cognitive flexibility typically use deterministic reinforcement, in that correct or incorrect choices result in reward or no reward with 100% certainty, as opposed to probabilistic reward contingencies used here. Thus, CRF and stress may have a more pronounced effect on cognitive flexibility when outcomes are deterministic rather than probabilistic. Another important consideration when comparing the present results to previous studies on how stress may affect decision making pertains to the type of stressor rats were exposed to. For example, studies demonstrating increased risky choice have either used inescapable foot shock (Nobrega et al., 2016) or CORT injections (Koot et al., 2013). Of particular note, treatment with the α2 adrenergic autoreceptor antagonist yohimbine, which has been used as a pharmacological stressor, impaired adjustments in choice biases on a probabilistic discounting task similar to the one used here (Montes et al., 2015). These findings indicate that the manner in which stress may alter risk/reward decision making is also dependent in part on the type of stressor used. In contrast to the lack of effect of CRF or acute stress on probabilistic discounting, these manipulations differentially altered risky choice in the Blackjack task, where external auditory 101 cues inform the decision making about the relative probability of obtaining larger/risky rewards. Increased CRF signaling induced a suboptimal pattern of choice, causing a shift in bias away from the large/risky option only when the odds of receiving the large reward was favourable. These findings complement Morgado et al. (2015), showing that chronic unpredictable stress reduced risky choice even under conditions where risky options had greater utility. Notably, individuals with depression, which is associated with increased CRF activity (Banki et al., 1987; Nemeroff et al., 1984), tend to make poorer quality decisions on the CGT, which also provides subjects with explicit information about reward probabilities (Clark et al., 2011; Rubinsztein et al., 2006; Taylor Tavares et al., 2007). Why central CRF impaired risky choice when guided by external cues but not internal representations of reward history is unclear. It is possible that these treatments may have impaired discrimination between the different auditory cues used to inform decision making. However, such an effect would be expected to alter choice biases on both good and poor odds trials. Thus, the selective reduction of risky choice on good odds trials induced by increased CRF renders this explanation unlikely. Alternatively, it is interesting to note that CRF-induced reductions in risky choice occurred on those trials associated with the greatest amount of uncertainty as to whether or not the larger reward may be received (i.e. 50%). Interestingly, stress-related disorders such as depression, anxiety, and post-traumatic stress disorders are also characterized by intolerance for uncertainty (Carleton et al., 2012; Nelson, Shankman, & Proudfit, 2014; Wright, Lebell, & Carleton, 2016). Thus, while additional research is required to further clarify how increased CRF activity may alter cue-guided behaviours, the present results suggest that one effect of this treatment may be to reduce tolerance for uncertainty. 102 In contrast to CRF treatments, acute stress caused a slight, and more generalized increase in risky choice. This finding adds to the plethora of cross-species studies demonstrating that acute stress (at least in males), increases risky choice using various paradigms (Koot et al., 2013; Nobrega et al., 2016; Preston et al., 2007; Putman et al., 2010; Simonovic et al., 2017; Starcke et al., 2008; Wemm & Wulfert, 2017). Notably, acute stress also caused a significant increase in reward sensitivity, in that rats were more likely to follow a rewarded risky choice with another risky choice across good odds trials. This increase in reward sensitivity fits with previous studies showing that acute stress increased reward- and approach-related behaviour to drug and highly palatable food reward (Lemmens et al., 2011; Marinelli, Aouizerate, Barrot, Le Moal, & Piazza, 1998; Marinelli & Piazza, 2002; Oliver et al., 2000; Rutters et al., 2009; Serfling et al., 2019). This being said, it is curious that acute stress enhanced reward sensitivity when choice was guided by external cues but had no effect on the probabilistic discounting task. This suggests that the ability of stress to enhance reward sensitivity to influence subsequent action selection may be sensitive to task structure. While it remains unclear why CRF and stress induce differential effects on choice in the Blackjack task, our impression of these findings is that the suboptimal choice behaviour induced by CRF more closely resembles the effects of chronic stress and profiles of decision making in patients with depression. Notably, patients with depression have increased circulating CRF levels (Banki et al., 1987; Nemeroff et al., 1984) and central CRF overexpression in genetic rodent models mirrors some of the behavioural effects of chronic stress (Dirks et al., 2002). Therefore, exogenously increasing CRF signaling may lead to more chronic stress-like behavioural manifestations, which may include suboptimal reward-related decision making. In comparison, 103 restraint stress alters choice in a manner comparable to other acute stress manipulations in both rodents and humans. 3.42 Motivational effects of CRF and acute stress Central CRF infusion and acute stress also exerted differential effects on certain motivational aspects of risk/reward decision making. Most notably, CRF infusion induced consistent increases in choice latency across both risk/reward decision making tasks. This result is similar to the effects of this treatment on decision latencies during effort-related decision making, which also increased deliberation periods when rats chose between smaller, easily obtainable rewards and larger ones associated with a greater effort cost (Bryce & Floresco, 2016). Furthermore, central CRF infusions have been reported to increase choice latencies in attentional tasks (Beard et al., 2015; Van’t Veer et al., 2012). From these data it is apparent that increased central CRF transmission slow processing times for action selection across a wide variety of tasks. Conversely, acute stress had no effect on choice latency on either task, in direct contrast to stress-induced increases in choice latencies during effort discounting (Bryce & Floresco, 2016; Shafiei et al., 2012). Juxtaposition of these findings suggest that acute stress does not uniformly increase decision latencies, but rather, the ability of acute stress to increase ‘indecisiveness’ depends on the type of cost being processed and that decisions involving physical effort costs may be particularly susceptible to stress. Excessive CRF signaling and acute stress also increased the number of trial omissions during performance of the probabilistic discounting task, but not on the Blackjack task. It is important to note that effects of CRF or stress on trial omissions during probabilistic discounting were more prominent in the latter parts of the test session. This profile may reflect a form of decision fatigue, rather than a general reduction in motivation to engage in the task. Taking this 104 into consideration, a parsimonious explanation for the lack of effect of these treatments on omissions during the Blackjack task may simply be that rats were given fewer trials (40) compared to the discounting task (90). A complementary explanation for these differential effects may be that the auditory cues used were presented before animals were able to make a choice and may have had an invigorating effect on responding, adding additional incentive for animals to re-engage with the levers following the inter-trial interval. 3.43 Conclusion In summary, the present findings provide novel insight into the differential ways that increased CRF or acute stress can alter cost-benefit decision making involving reward uncertainty. Central CRF administration had no effect on risky choice when decisions were guided by internal representation of risk/reward contingencies, but lead to suboptimal, risk averse patterns of choice when decisions were guided by external cues. Alternatively, acute restraint stress did not affect risky choice during probabilistic discounting, but enhanced reward sensitivity during cue-guided decision making. These results reveal how different mechanisms of the stress response can act in similar ways when decision making involves effort costs (Bryce & Floresco, 2016) but produce divergent effects when decision making involves risk of reward omission or uncertainty. It is notable that animals subjected to chronic stress and patients with depression display increased central CRF (Banki et al., 1987; Chappell et al., 1986; Nemeroff et al., 1984). Both of these conditions are associated with suboptimal patterns of decision making (Baradell & Klein, 1993; Clark et al., 2011; Morgado et al., 2015; Rubinsztein et al., 2006; Taylor Tavares et al., 2007), similar to the effects of CRF reported here. As such, this research may aid in identifying the neurochemical mechanisms that may underlie perturbations in cost/benefit decision making observed in stress-related disorders such as depression. 105 Dopamine signaling is particularly important in overcoming various costs required for optimal decision making (Stopper & Floresco, 2015; Winstanley & Floresco, 2016). Although we did not find that stress manipulations altered risk/reward decision making in a manner similar to fluctuations in dopamine tone in the probabilistic discounting task (St. Onge et al., 2010; St Onge & Floresco, 2009), dopamine alterations have not yet been assessed using the external cue-guided Blackjack task. Therefore, stress manipulations may indeed rely on dopaminergic mechanisms to alter risk/reward decision making when external cues are present. Furthermore, both central CRF and acute stress alter decision making involving effort costs, which parallel alterations in effort choice following manipulations that reduce dopamine tone (Bryce & Floresco, 2016; Salamone & Correa, 2012). Previous work finds that blocking the CRF1R increases tonic dopamine activity in the VTA, indicating that CRF exerts tonic inhibition on VTA DA neurons. However, no studies to date have investigated the role of increased central CRF signaling in VTA dopamine neuron activity. Therefore, the following chapter will uncover the role of central CRF on VTA dopamine neuron physiology. 106 Table 1. Mean (range) number of wins and losses on win-stay | good odds, lose-shift | good odds, and lose-shift | poor odds trials after treatment with vehicle (aCSF), CRF, no stress or one-hour restraint stress. Blackjack Mean # of Wins Mean # of Losses win-stay | good odds win-stay | poor odds lose-shift | good odds lose-shift | poor odds CRF 1μl Vehicle 8.0 (3-12) N/A 7.4 (3-15) 11.1 (4-18) 3μl Vehicle 9.1 (4-13) 7.1 (4-12) 10.4 (4-14) 1μg CRF 8.3 (3-14) 6.7 (4-12) 11.1 (3-19) 3μg CRF 7.8 (4-16) 6.4 (3-12) 10.9 (3-15) Restraint Stress Baseline 6.7 (3-12) 6.6 (3-13) 6.6 (3-15) Stress 9.1 (3-14) 6.3 (3-13) 7.3 (3-15) 107 Table 2. Mean (S.E.M) locomotor activity (photobeam breaks) and number of omissions (averaged over trials) after treatment with vehicle (aCSF), CRF, no stress or one-hour restraint stress. ## indicates main effect of Treatment (CRF) significance at p < 0.01 vs average vehicle.**indicates significance at p < 0.01 ***indicates significance at p < 0.001 vs no stress. Probabilistic discounting Blackjack Locomotion # of omissions Locomotion # of omissions CRF 1μl Vehicle 1457 (98) 6.0 (2.4) 1209 (79) 0.1 (0.1) 3μl Vehicle 1481 (145) 3.6 (1.4) 1252 (74) 0.0 (0) 1μg CRF 1427 (125) 8.9 (3.3)## 997 (78)## 0.2 (0.2) 3μg CRF 1433 (155) 11.2 (3.4)## 905 (80)## 1.5 (0.9) Restraint Stress Baseline 1556 (126) 7.6 (2.5) 1441 (99) 0.1 (0.1) Stress 930 (102)*** 14.5 (4.1)** 970 (87)** 0 (0) 108 Table 3. Mean (S.E.M) win-stay/lose-shift ratios as a function of the recent outcome of the preceding trial ) after treatment with vehicle (aCSF), CRF, no stress or one-hour restraint stress. Probabilistic Discounting Blackjack Win-Stay Lose-Shift Win-Stay Lose-Shift CRF 1μl Vehicle 0.88 (0.02) 0.49 (0.05) 0.51 (0.07) 0.44 (0.05) 3μl Vehicle 0.83 (0.03) 0.52 (0.06) 0.49 (0.06) 0.47 (0.05) 1μg CRF 0.85 (0.03) 0.43 (0.05) 0.47 (0.07) 0.41 (0.06) 3μg CRF 0.85 (0.03) 0.45 (0.05) 0.55 (0.07) 0.53 (0.06) Restraint Stress Baseline 0.85 (0.03) 0.43 (0.04) 0.36 (0.05) 0.60 (0.07) Stress 0.82 (0.05) 0.49 (0.06) 0.49 (0.06) 0.52 (0.07) 109 Figure 5. Probabilistic discounting and Blackjack task designs. a) The top illustration depicts the probabilities of obtaining the small/certain reward vs the large/risky reward for both order variants (ascending and descending odds trials). Below is the format for a single free-choice trial on the probabilistic discounting task including reward contingencies associated with responding on either lever. b) The top illustration depicts the probabilities of obtaining the small/certain vs the large/risky reward on good vs poor odds trials, which were signaled by one of two auditory cues. Below is the format for a single free-choice trial on the Blackjack task including reward contingencies associated with responding on either lever. 110 Figure 6. Central CRF infusion reduced motivation but not choice on probabilistic discounting. a) ICV CRF administration significantly increased the number of trials omitted, driven by increases in later trial blocks. b) These treatments also increased choice latencies. c) Neither dose of CRF altered risky choice, irrespective of the order with which reward probabilities changes across blocks (insets). There was no difference on risky choice between the high and low volume vehicle infusion (inset). d) CRF also did not alter win-stay, lose-shift tendencies. Stars indicate level of significance (one star, p<0.05, two-stars p < 0.01) vs average vehicle. For this and all other figures, error bars represent S.E.M. 111 Figure 7. Restraint stress slightly reduced motivation but not choice on probabilistic discounting. a) Restraint stress significantly increased trial omissions over the entire session. b) Restraint stress had no effect on choice latency. c) Risky choice was unaffected by restraint stress, irrespective of the order in which odds changed across blocks (insets). d) Win-stay, lose-shift tendencies were also unaltered by restraint. Stars indicate level of significance vs baseline (two-stars p < 0.01). 112 Figure 8. Central CRF infusion reduced optimal risky choice on the cue-guided Blackjack task. a) ICV CRF administration significantly reduced risky choice on trials when cues signaled the odds of receiving the large reward were good, but had no effect on poor-odds trials. There was no difference on risky choice between the high and low volume vehicle infusion (inset). b) CRF infusion also significantly increased choice latency but only at the 3μg dose. c) Similarly, win-stay/lose shift behaviour analyzed in terms of how the outcome of a risky choice influenced choice on the next trial of the same type for good odds (left and middle) and poor odds trials (right) was unaffected by CRF treatment. Each graph represents data from subsets of rats that experienced a sufficient number of win or loss trials, respectively. Stars indicate level of significance vs average vehicle (one-star p < 0.05). 113 Figure 9. Restraint stress slightly increased risky choice and reward sensitivity on the external cue-guided Blackjack task. a) Restraint stress tended to increase risky choice irrespective of the odds. b) Restraint had no effect on choice latency. c) However, restraint stress increased the tendency to follow up a risky win on a good odds trial with another risky choice on the next good odds trial (left), but had no effect on lose-shift tendencies on good odds (middle) or poor odds trials (right). Stars indicate level of significance vs baseline (one-star p < 0.05). Crosses denote p < 0.10. 114 Chapter 4: CRF dose-dependently increases dopaminergic and non-dopaminergic activity in the VTA 4.1 Introduction The previous two chapters revealed that CRF hyperactivity diminishes motivation. That is, exogeneous CRF treatment increases the number of trials omitted and choice latency across multiple tasks (Bryce & Floresco, 2016). As mesolimbic dopamine plays a key role in motivation (Berridge, 2012; Salamone & Correa, 2012; Wise, 2004), interactions between CRF and dopamine may lead to motivational impairments. Indeed, anatomical observations demonstrate strong interactions between CRF and dopamine systems (Kelly & Fudge, 2018). Therefore, the purpose of this chapter was to clarify how centrally-administered CRF may perturb dopamine neuron activity. The stress-related neuropeptide, CRF, initiates the HPA axis, but also acts outside the HPA axis to mediate many of the behavioural effects of stress (Cador et al., 1992; Koob, 2010; Müller et al., 2003). The CRF peptide and its cognate stimulatory Gs- or Gq-coupled CRF1Rs and CRF2Rs are expressed throughout the brain, including within the mesocorticolimbic dopamine system (Aguilera & Liu, 2012; Bittencourt & Sawchenko, 2000; Tan et al., 2017; Van Pett et al., 2000). The central hub of this circuitry is the VTA, which houses dopamine, glutamate, and GABA neurons and neurons that express a combination of two or more neurotransmitters (Morales & Margolis, 2017). In rodents, the VTA locally produces CRF (Grieder et al., 2014) and receives CRF innervation from the lateral BNST, CeA, and PVN (Dabrowska et al., 2016; Rodaros et al., 2007), and both CRF1Rs and CRF2Rs are localized within the dopamine cell body region of this structure (Tan et al., 2017; Van Pett et al., 2000). Moreover, centrally-infused CRF increases activation within the mesocorticolimbic system by enhancing immunoreactivity to 115 the immediate early gene, c-Fos, indicating that administration into the lateral ventricle system has direct or indirect actions on the VTA, PFC, and NAc (Bittencourt & Sawchenko, 2000). CRF modulation of VTA dopamine neurons has been well-characterized in vitro, revealing that CRF acts in complex, often opposing ways and can increase or decrease excitatory and/or inhibitory transmission on VTA dopamine neurons through numerous mechanisms. For example, bath application of CRF promotes post-synaptic excitation of VTA dopamine neurons dependent on CRF2R and CRF-BP, enhancing NMDA receptor-dependent excitatory post-synaptic currents (EPSCs) and increasing the firing rate via CRF1Rs (Korotkova et al., 2006; Ungless et al., 2003; Wanat et al., 2008). In addition to the excitatory actions, CRF has also been shown to potentiate inhibitory signaling on VTA dopamine neurons enhancing both dopamine and GABA-B-mediated inhibitory post-synaptic current (IPSC) amplitude (Beckstead et al., 2009; Riegel & Williams, 2008). More recent experiments have illuminated a bi-phasic pre-synaptic action of CRF on glutamate and GABA release, potentiating EPSCs via pre-synaptic CRF1Rs on glutamate terminals but dampening EPSCs via CRF2R activation of GABA terminals to release GABA onto pre-synaptic GABA-B receptors on glutamatergic terminals (Williams et al., 2014). In contrast, relatively little is known about how CRF may affect GABA neurons; however, one study found that bath applied CRF in vitro potentiated the firing rate of GABA neurons (Korotkova et al., 2006). That said, in vitro preparations sever many afferent inputs to the VTA which may alter the pre- and post-synaptic effects of CRF within an intact brain and may lead to divergent results. For instance, when manipulated in intact animals, acute stress, intra-ventricular, and intra-VTA CRF infusion potentiate dopamine release in the NAc and PFC (Dunn & Berridge, 1987; Holly & Miczek, 2016; Kalivas & Duffy, 1995; Kalivas et al., 1987; Matsuzaki et al., 1989). CRF is 116 released into the VTA in response to stress (Wang et al., 2005) and subsequently reduces motivation to work for reward (Wanat et al., 2013). Critically, this behavioural effect is dependent on phasic dopamine release in the NAc via afferent input from the PPTg in response to reward delivery but not reward cues (Wanat et al., 2013). Moreover, central and intra-VTA infusions of exogenous CRF markedly shift choice biases away from larger rewards requiring multiple lever presses in favor of smaller ones delivered after a single press in an effort-discounting task (Bryce & Floresco, 2016). CRF also acts downstream in the mesolimbic dopamine circuit at the level of the NAc, where pre-synaptic CRF receptors are located on VTA dopamine terminals to facilitate local dopamine release, mediating appetitive behaviour such as conditioned place preference, invigoration of instrumental reward seeking and social bond formation (Lemos et al., 2012; Lim et al., 2007; Peciña et al., 2006). When viewing these findings with those from in vitro neurophysiological studies, it is unclear if the net effect of VTA CRF receptor activation in vivo is an increase or decrease in dopamine neural firing and terminal release. Indeed, other in vivo studies suggest that CRF may exert an inhibitory tone on VTA dopamine neurons, with central administration of a CRF1R antagonist increasing the number of spontaneously active dopamine neurons (Lodge & Grace, 2005), which drives slower, extrasynaptic or ‘tonic’ dopamine levels within the NAc (Floresco et al., 2003). Given these conflicting effects, illuminating how CRF hyperactivity may affect dopamine neuron activity in intact animals is important to elucidate the functional role of CRF on behaviour. Therefore, our primary aim was to clarify how increased central CRF signaling interacts with the mesolimbic dopamine system, with a specific focus on dopaminergic and GABAergic neuron physiology within the VTA. 117 4.2 Methods 4.21 Extracellular electrophysiology recordings Male Long Evans rats (300 – 600 g) were anesthetized with chloral hydrate (400 mg/kg, i.p.) and maintained by supplemental administration of chloral hydrate (intra-jugular vein) as required to maintain suppression of withdrawal reflex in response to toe pinch. Once anaesthetized, rats were placed in a stereotaxic apparatus and core body temperature of 37°C was sustained by a thermostatically-controlled heating pad. CRF (1 or 3g) or vehicle (aCSF; 1 or 3l) was infused via insertion of a 23-gauge cannula (coordinates: [AP -1.0 mm from bregma, ML - 1.8 mm from bregma, DV -2.5 mm from dura]). A burr hole was also drilled above the contralateral VTA (coordinates: [AP -5.3mm from bregma, ML -0.8 mm from bregma]) and dura carefully removed. Extracellular recording microelectrodes (2.0mm OD borosilicate glass capillary tubing, ~1m tip diameter, impedance 5–10 M) were filled with 2 M NaCl containing 2% Pontamine sky blue. The signal was amplified and filtered (50-1000 Hz) using an X-Cell3+ microelectrode amplifier (FHC). Action potential data were acquired, discriminated from noise, and analyzed using Spike2 software (CED) on a computer with an analog/digital interface (micro 1401 mkII). Extracellular glass recording electrodes were lowered through the VTA (-6.5 to -8.5 mm ventral from brain surface) using a hydraulic microdrive in a predetermined pattern of 5-9 vertical passes or tracks separated by 200m. Putative dopamine and GABA neurons were characterized primarily based on waveform morphology and firing rate. Putative dopamine neurons were identified using established electrophysiological waveform criteria characterized by a long-duration bi-or triphasic action potential, often with a break between the initial segment and somatodendritic spike components (Grace & Bunney, 1983; Ungless & Grace, 2012). In 118 contrast, GABA neurons were identified by high firing frequency and narrow spike width. If neurons presented a stable firing pattern but could not be classified by waveform morphology or firing rate as dopamine or GABA neurons they were analyzed separately as ‘unclassified’ neurons. Once isolated, neuron activity was recorded for 1 to 3 min. Initial waveform recordings used to categorize neurons were obtained using a low-end filter set at 50 Hz for at least 15s. The use of this lower filter setting permits a more complete analysis of the waveform which enables the ability to distinguish dopaminergic cells from other cell types (Ungless & Grace, 2012). Previous juxticellular recording/labeling experiments using these parameters have shown that ~90% of neurons fitting this criteria are dopaminergic (Ungless & Grace, 2012; Ungless, Magill, & Bolam, 2004). Using in vivo single-cell extracellular recording techniques, we measured three characteristics of dopamine neuron activity in the VTA: (1) the overall population activity, or the number of dopamine neurons that are spontaneously active per electrode track, (2) the basal firing rate, and (3) the percentage of action potentials occurring in bursts and the average number of spikes within a burst. A burst is defined as the occurrence of two consecutive spikes with an interspike interval < 80 ms, with the termination of a burst defined as two spikes with an interspike interval >160 ms (Grace & Bunney, 1983). GABA and unclassified neurons were analyzed only in terms of firing rate. Once the experiment was terminated, the recording site was marked via ejection of Pontamine sky blue dye from the tip of the recording electrode. 4.22 Drug infusions Separate groups of 8-10 male Long Evans rats received infusions of behaviourally-relevant doses of vehicle (aCSF; 1 or 3l) or CRF (1 or 3g) into the left lateral ventricle. These doses were chosen based on previous published work and those from preceding chapters demonstrating 119 that CRF at these doses alter behaviour, including effort and risk/reward decision-making as well as altering choice latencies and task engagement (Bryce & Floresco, 2016). Two CRF doses were chosen to look at potential dose-dependent effects. However, CRF has a solubility of 1g/1l, therefore, we used two different volumes similar to previous behavioural experiments (Bryce & Floresco, 2016). Once the cannula above the lateral ventricle was lowered into place and the recording electrode was immediately dorsal to the VTA, a 30-gauge injector was inserted into the cannula 0.8mm ventral to the tip of the cannula to allow fluid to disperse into the lateral ventricle. Separate groups of rats received the low vehicle (aCSF, 1l) or CRF (1g/1l) dose, or the high vehicle (aCSF, 3l) or CRF (3g/3l) dose, infused at a rate of 0.22l/min. Injectors were kept in place throughout the duration of the recording experiment to allow for diffusion and to minimize disruption. 4.23 Histology Once the recording experiment was terminated, rats were overdosed with chloral hydrate (IV) and brains were removed and fixed with formalin solution (4% formaldehyde) for at least 48h prior to slicing. Brains were sliced into 50m thick coronal sections and mounted on gel-coated slides. Once dry, slides were stained with Cresyl Violet to verify placement of VTA recording electrode and lateral ventricle cannulation sites via microscope, with reference to a stereotaxic atlas (Paxinos & Watson, 2013). 4.24 Data analysis The main variables of interest were the number of spontaneously active dopamine cells per track, dopamine neuron firing rate, percentage spikes in burst, and average number of spikes in burst. To rule out the possibility that alteration in the number of dopamine cells per track was due to differences in the number of tracks completed we also assessed the number of tracks 120 completed in each group. Other variables included GABA firing rate and unclassified firing rate, which were cells that could not be classified as dopamine or GABA, and non-dopamine firing rate, which combined GABA and unclassified cells. Data was averaged within each animal then within group and compared across groups in a between subject’s design. Only animals that had at least 5 tracks were included in the analysis. For simplicity, we combined the 1l and 3l aCSF vehicle data given that there were no significant differences on any measure between these groups (all Fs < 1, all Ps > 0.20). These data were analyzed with one-way ANOVAs with Drug Treatment (combined aCSF vehicle, 1g, or 3g CRF) as a between-subjects variable. Post hoc analyses were conducted when significant main effects were present using Dunnett’s test, comparing drug groups to the control group (combined aCSF vehicle). Additionally, the distribution of dopamine and non-dopamine neuron firing rates were assessed using the Kruskall-Wallis nonparametric test for independent samples with more than 2 groups. 4.3 Results 4.31 Dopamine neural activity in the VTA Data was collected from 258 dopamine neurons from 32 rats (15 combined vehicle group, 9 low CRF group, 8 high CRF group). As Figure 10A depicts, we found that central CRF infusion increased the number of spontaneously active dopamine cells per track (F(2,29)= 5.80, p < 0.01). Multiple comparisons further revealed that this was driven by an increase in dopamine cells per track following the 3g CRF infusion (p < 0.005), but not following the 1g CRF infusion (p > 0.05). This increase in spontaneously active dopamine cells per track was not due to any differences between groups on the number of tracks completed (F(2,29) < 1, n.s.). The CRF-induced increase in dopamine neuron population activity was accompanied by a significant increase in average dopamine firing rate (F(2,29)= 3.09, p < 0.05, Fig 10B), with Dunnett’s test 121 again finding a significant effect of the 3g CRF dose (p < 0.05), but not the 1g CRF dose (p > 0.05). In contrast to the effects of CRF on population activity and firing rate, we found no effect of CRF on phasic burst activity of dopamine cells, indexed by the percentage of spikes in burst (F(2,29) < 1, n.s., Fig 10C) or average number of spikes in burst (F(2,29)< 1, n.s., Fig 10D). The increase in spontaneously active dopamine neuron activity may be due to CRF releasing dopamine neurons from tonic inhibition, which would increase the number of spontaneously active dopamine neurons firing at a slower frequency. To clarify, we created a frequency distribution of the firing rate of dopamine neurons for each of the 3 groups (combined vehicle, 1g CRF, and 3g CRF). Looking at this distribution plotted in Figure 10E, it is apparent that CRF actually reduced the proportion of neurons firing at a lower frequency and increased the proportion of neurons firing at a higher frequency. However we did not find a significant difference between drug group populations using the nonparametric Kruskal-Wallis Test (2 = 0.16, n.s., Fig 10E). Taken collectively, our results show that high dose CRF alters certain parameters of VTA dopamine neural activity. Specifically, these treatments increased measures associated with tonic dopamine activity by increasing the number of spontaneously active dopamine neurons and shifting these neurons into a higher firing frequency, whereas CRF did not alter phasic dopamine activity. 4.32 Non-dopamine neural activity in the VTA While searching for dopamine neurons within the VTA, we also occasionally found putative GABAergic cells (82 GABA cells from 21 animals: 9 combined vehicle, 7 low CRF, 5 high CRF) and cells that we could not classify using our strict inclusion criteria, termed ‘unclassified’ cells (105 unclassified cells from 29 animals: 13 combined vehicle, 8 low CRF, 8 122 high CRF). Analyzing putative GABAergic cells, we found that central CRF infusion increased firing rate (F(2,17)= 5.83, p < 0.05, Fig 11A). In contrast to dopamine neuron firing rate, this was driven by the 1g CRF dose (p < 0.01), whereas after treatment with 3g CRF, this effect was lower and did not achieve statistical significance (p > 0.05). CRF infusion also increased the firing rate of unclassified neurons (F(2,26)= 3.44, p < 0.05, Fig 11B), again due to increases following the 1g CRF dose (p < 0.05), but not the 3g CRF dose (p > 0.05). We again looked at the firing rate frequency distribution of non-dopamine neurons but did not find a significant difference between populations using the nonparametric Kruskal-Wallis Test (2 = 0.13, n.s., Fig 11C). Together these results show that, in opposition to the effects of high dose CRF on VTA dopamine neuron physiology, a lower CRF dose increased the firing rate of GABAergic and unclassified neurons. 4.4 Discussion Given the multifaceted physiological and functional effects of CRF on the mesocorticolimbic dopamine system, the current experiment aimed to clarify how increased central CRF signaling alters VTA dopamine and non-dopamine neuron firing in intact animals, which may provide an improved understanding of how CRF affects behaviour. Collectively, the current results reveal that the complexity of actions involve dose-dependent effects on dopamine and non-dopamine neurons in the VTA. More specifically, high dose CRF alters tonic dopamine activity, increasing the number of spontaneously active dopamine neurons and firing rate, whereas low dose CRF alters non-dopamine activity, increasing the firing rate of GABAergic and unclassified cells in the VTA. Increased tonic dopamine activity fits with previous work showing that two hours of acute restraint stress increases population activity driven by the ventral hippocampus (Valenti et al., 123 2011), indicating that high dose central CRF administration and acute restraint stress increase tonic dopamine activity in a similar manner. Population activity of VTA dopamine cells is mediated by the infralimbic PFC and ventral subiculum of the hippocampus (Grace, 2016; Lodge & Grace, 2006a; Patton et al., 2013; Zimmerman & Grace, 2016) and appears to regulate the slower changing extrasynaptic tonic dopamine release in regions like the NAc, which can be measured with microdialysis (Floresco et al., 2003). Therefore, CRF- and stress-induced increases in population activity would likely increase tonic mesocorticolimbic dopamine release, which is consistent with previous work showing that acute stress and CRF infusion potentiate dopamine release in the NAc and PFC (Dunn & Berridge, 1987; Holly & Miczek, 2016; Kalivas & Duffy, 1995; Kalivas et al., 1987; Matsuzaki et al., 1989). It is possible that the CRF-induced increase in spontaneous dopamine activity observed after ICV treatment is mediated via direct actions within the hippocampus (Floresco, Todd, & Grace, 2001; Valenti et al., 2011) or driven by inputs from the infralimbic PFC (Patton et al., 2013) or medial septum (Bortz & Grace, 2018), as enhancing activity in these regions similarly increased population activity. That said, the possibility that CRF altered dopamine activity via direct actions in the hippocampus is unlikely given that this manipulation tended to reduce average dopamine firing rate by increasing the proportion of low firing neurons (Floresco et al., 2001), which is at odds with our present findings. Increased tonic dopamine activity is in contrast to previous work indicating that central CRF signaling exerts tonic inhibition on spontaneous dopamine neuron activity, as a CRF1R antagonist increased the number of spontaneously active dopamine neurons (Lodge & Grace, 2005). The higher CRF dose used here, that appears to mimic the effects of acute restraint stress, may act primarily on CRF2Rs to increase spontaneous dopamine activity. However, an in vitro 124 study found that amplifying CRF1R activity increases EPSCs, whereas amplifying CRF2R activity reduces EPSCs on VTA dopamine neurons (Williams et al., 2014). That said, it is currently unclear if blocking CRF2Rs would similarly or differentially impact population activity in intact animals. Therefore, future studies should pharmacologically probe the CRF receptor dynamics on VTA dopamine population activity. Interestingly, we found that in addition to increasing dopamine neuron population activity, high dose CRF also shifted dopamine neurons into an increased firing rate, which is in line with numerous in vitro studies demonstrating that bath applied CRF onto VTA slices increases dopamine neuron EPSCs (Ungless et al., 2003; Williams et al., 2014) and firing rate (Korotkova et al., 2006). As such, the effects of CRF hyperactivity on dopamine neuron firing rate may be mediated through direct actions of this neuropeptide on dopamine neurons within the VTA, which is plausible as the VTA expresses CRFRs (Tan et al., 2017; Van Pett et al., 2000) and centrally infused CRF alters VTA activity, as measured by increased immunoreactivity to the immediate early gene, c-Fos (Bittencourt & Sawchenko, 2000). In contrast to tonic dopamine activity, we found increased CRF neurotransmission was without effect on phasic dopamine neuron activity. Phasic activity is indexed by measuring burst firing, which occurs in rapid response to reward and reward-related stimuli and is driven by glutamatergic input originating in the mPFC and PPTg and gated by the laterodorsal tegmentum (Grace et al., 2007; Lodge & Grace, 2006b). Burst firing regulates synaptic phasic dopamine release to efferent regions and can be measured via fast-scan cyclic voltammetry (Floresco et al., 2003). These null results are notable given the reduction in certain forms of phasic dopamine release in the NAc following intra-VTA CRF administration (Wanat et al., 2013). However, this was activity-dependent, dampening phasic dopamine release in the NAc in response to reward, 125 but not reward-related cues, driven by PPTg stimulation. These results appear contradictory, however, perhaps effects on phasic dopamine depend on the site of action. As such, CRF in the VTA does not appear to alter spontaneous phasic dopamine activity, but rather reduces phasic dopamine release when particular inputs to the VTA are stimulated. Additionally, CRF neurotransmission potentiated non-dopamine neuron firing rate; however, not at the dose that increased dopamine neuron firing rate. That is, low, but not high, dose CRF increased the firing rate of putative GABAergic neurons and neurons that were not classified as either dopamine or GABA. Although the literature on GABA neurons is more limited, the current results are in agreement with one study showing bath applied CRF increased the firing rate of VTA GABA neurons in vitro (Korotkova et al., 2006). Given the parallel findings between experimental protocols, it is plausible that the dose-dependent increase in firing rate of both dopamine and non-dopamine neurons occurs at the level of the VTA, although the exact mechanism, including pharmacological effects at CRF receptors, requires further investigation. The present results aid in clarifying previous behavioural alterations. That is, high doses of centrally-administered CRF at the same dose that increases dopamine population activity and firing rate also reduces choice of the more rewarding option requiring more effort to obtain and increases choice latency, whereas CRF at the lower dose that increased non-dopamine firing rate had no effect on effort choice or latency (Bryce & Floresco, 2016). Conceivably, centrally-infused CRF at high doses reduces effort choice by increasing tonic dopamine signaling, particularly in the NAc, which is principally involved in overcoming effort costs (Cousins & Salamone, 1994; Cousins, Sokolowski, & Salamone, 1993; Floresco, Tse, & Ghods-Sharifi, 2008; Nowend et al., 2001; Salamone et al., 1994; Salamone et al., 1991). That said, the effects 126 of CRF on effort-related decision-making appear to mimic a reduction in dopamine tone (Bryce & Floresco, 2016; Floresco et al., 2008; Salamone et al., 1991). However, enhanced dopamine activity can also reduce preference for larger rewards associated with a greater effort cost. For instance, systemic administration of high-dose amphetamine, which increases dopamine tone, also shifts preference away from larger rewards during effort discounting (Floresco et al., 2008). Likewise, overexpression of striatal D2Rs reduces effort-related choice (Filla et al., 2018). Taken together, it is tempting to speculate that reductions in motivated responding induced by excessive CRF activity are driven by increased mesoaccumbens dopamine levels, a hypothesis that will be tested in the subsequent chapter. 127 Figure 10. High dose central CRF increased tonic, but not phasic, dopamine neuron firing. a) High (3g), but not low (1g) CRF infusion, increased the number of dopamine cells per track and b) increased the firing rate of dopamine neurons. c) ICV CRF infusion had no effect on the percentage of spikes in burst or d) the average number of spikes in burst. e) ICV CRF infusion did not alter the distribution of dopamine neuron firing rates. 128 Figure 11. Low dose central CRF increased GABAergic and unclassified neuron firing. a) Low (1g), but not high (3g) CRF infusion, increased GABAergic firing rate and b) increased firing rate of unclassified neurons. c) Central CRF infusion did not alter the distribution of non-dopamine neuron firing rates. 129 Chapter 5: Alterations in effort-related decision making induced by stimulation of dopamine D1, D2, D3, and CRF receptors in NAc subregions 5.1 Introduction It is well-established that dopamine modulates decision making involving evaluation of options that differ in terms of their relative costs and benefits. Among the various types of choices a decision maker may face, those that involve effort costs are particularly sensitive to pharmacological manipulations of dopamine activity. Systemic treatment with dopamine antagonists invariably reduce preference for larger rewards associated with a greater effort cost (Cousins, Wei, & Salamone, 1994; Floresco et al., 2008; Salamone et al., 1994; Salamone et al., 1991). Moreover, this form of decision making is perturbed in various illnesses linked to alterations in dopamine functioning such as depression and schizophrenia, as individuals with these disorders tend to bias choices away from larger rewards requiring more effort (Treadway et al., 2012; Treadway et al., 2015). Mesoaccumbens dopamine projections to the NAc are particularly important in overcoming effort costs in favor of larger rewards. Reducing dopamine within the NAc shifts choice away from options associated with greater effort costs even though these options would yield larger or more palatable rewards (Salamone et al., 1994; Salamone et al., 1991; Sokolowski & Salamone, 1998). The NAc can be partitioned into a lateral core and medial shell region, and inactivation studies have shown that activity within the NAc core rather than the shell plays a primary role in overcoming effort costs (Ghods-Sharifi & Floresco, 2010). However dopamine receptor activity is necessary in both subregions as blocking either D1 or D2 receptors in either NAc core and shell shift choice away from more effortful options (Nowend et al., 2001). In contrast to the multitude of studies demonstrating that reducing NAc dopamine diminishes the preference for larger or more preferred rewards associated with greater effort 130 costs, there is a surprising paucity of studies characterizing how increasing dopamine activity affects effort-related decision making. The majority of these studies have used systemic administration of drugs that promote dopamine release, such as amphetamine, and yielded varying effects depending on the behavioural procedures used. Using a concurrent choice procedure, Cousins and colleagues (1994) reported that amphetamine reduced preference to press a lever multiple times for a preferred reward pellet, but also reduced consumption of freely-available chow. Other studies using procedures involving choice between different magnitudes of rewards have shown that increasing dopamine release either biases choices towards larger, more costly rewards in a T-maze task (Bardgett et al., 2009) or exerts biphasic, dose dependent effects on an effort discounting task conducted in operant chambers (Floresco et al., 2008). Fewer studies have examined how more selective activation of mesoaccumbens dopamine pathways affect effortful responding. Intra-NAc infusions of amphetamine can increase responding for food on a progressive ratio schedule (Zhang et al., 2003), but the dopamine receptor mechanisms underlying these effects are unclear. In this regard, overexpression of striatal D2 receptors reduces preference to work harder for more preferred rewards (Filla et al., 2018), and overexpression of striatal D3 receptors reduces instrumental responding for food delivered on a progressive ratio (Simpson et al., 2014). Nevertheless, a comprehensive analysis of how increasing activity at different dopamine receptors in distinct regions of the NAc influences effort-related decision making is lacking. Clarification of these issues may provide insight into the mechanisms underlying alterations in these types of decisions that occur during situations that promote excessive mesoaccumbens dopamine activity. Stress is one condition that can enhance dopamine activity, and this is modulated in part by the neuropeptide CRF. CRF exerts complex effects on the activity of midbrain dopamine 131 neurons, with the majority of studies reporting that it can increase dopamine neuron firing rate in vitro (Beckstead et al., 2009; Korotkova et al., 2006; Riegel & Williams, 2008; Ungless et al., 2003; Wanat et al., 2008; Williams et al., 2014). Likewise, central or intra-VTA CRF increases tonic levels of dopamine in the NAc and prefrontal cortex (Dunn & Berridge, 1987; Holly et al., 2015; Kalivas et al., 1987; Matsuzaki et al., 1989). However, intra-VTA administration of CRF also reduces reward-associated phasic dopamine release in the NAc (Wanat et al., 2013). On the other hand, CRF can also augment evoked mesoaccumbens via local mechanisms within the NAc, which in turn can promote a conditioned place preference and augment the ability of Pavlovian stimuli to invigorate reward-related responding (Peciña et al., 2006; Lemos et al., 2012). In contrast, CRF infusions into the lateral ventricles or the VTA reduces lever pressing for food delivered on a progressive ratio schedule and also shifts choice bias towards smaller, lower cost rewards (Bryce & Floresco, 2016; Wanat et al., 2013). However, it is unclear how CRF in the NAc may influence these processes. With these issues in mind, the present study was conducted to provide a thorough characterization of how increasing dopamine and CRF activity within different subregions of the NAc may influence effort-related decision making. Specifically, we examined the effects of intracranial infusions of selective D1, D2 and D3 receptor agonists, or CRF within the NAc core and shell on this form of decision making, assessed with an operant based effort-discounting procedure wherein rats chose between smaller, easy to obtain rewards vs larger rewards associated with a greater effort cost. 132 5.2 Methods Some experimental procedures were similar to those from Chapter 2 and 3. Thus, only deviations from the previous procedure will be described here. Separate cohorts of 12-24 male Long Evans rats were utilized for all experiments. 5.21 Initial lever pressing training Following food restriction and FR1 training (see Chapter 2 methods), rats were trained on a simplified version of the full task, which required rats to press the retractable lever within 10 s of lever extension. Sessions consisted of 90 training trials with 45 presses required on each lever. Trials were initiated with the illumination of the houselight and the extension of one of the two levers and separated with a 40s ITI. Trials were scored as omissions if the rat failed to respond to the lever within 10 s following lever extension. If this occurred, the lever retracted, the houselight turned off, and the 40 s ITI began. On the other hand, if the rat responded to the lever within 10 s, the rat received one pellet. Rats were trained to a criterion of 80 successful trials (i.e. < 10 omissions) for 2 consecutive days, which took approximately 5 days of training. 5.22 Effort-related decision making Following initial lever pressing training, rats were trained on the effort-related decision making task in separate cohorts of 16-24 animals, 5-7 days a week. The procedures were similar to those described previously (Bryce & Floresco, 2016). Briefly, daily 32-min training sessions consisted of 48 trials separated into 4 blocks of 2 forced-choice trials followed by 10 free-choice trials. For all sessions and blocks, one counterbalanced lever was designated as the low reward lever (LR) and the other lever was designated as the high reward lever (HR). Every 40 s, the houselight was illuminated and 3 s later, one or both levers were inserted into the chamber. If the LR lever was chosen, both levers would retract and two pellets would be delivered. Conversely, 133 the first press on the HR lever caused the LR lever to retract, while the HR lever remained inserted. Additional lever presses were required until the ratio required for the current block was completed. For the initial block, the effort requirement to obtain the larger reward was 2 presses, and then 5, 10 and 20 presses over subsequent blocks. Upon completion of the required number of presses, the lever retracted and 4 pellets were delivered, 0.5 sec apart. If, on the rare occasion a rat failed to complete the required number of presses on the HR lever within 25 s of extension, the lever retracted and no pellets were dispensed but choice was still incorporated into the analysis. If, however, rats failed to make a response on either lever within 25 s of its insertion, an omission was scored and both levers retracted. Other measures included choice latencies (i.e. the time between lever extension and initial selection), rates of pressing on the HR lever (averaged across blocks), and photobeam breaks as an index of locomotor activity. Groups of rats were typically run in squads of 16-24. The initial phase of the experiment tested rats with cannulae that were implanted in the NAc core. Subsequent groups comprised of rats that included those with cannulae implanted in the shell and core. Rats were trained until as a group, they met two criteria: 1) chose HR lever during the first trial block on at least 75% of the free-choice trials, and 2) demonstrated stable baseline levels of discounting for 3 consecutive days. Stability was analyzed using a 3 x 4 repeated measures analysis of variance (ANOVA) with Training Day (3) and Trial Block (4) as within-subjects factors. The animals were judged to have achieved stability of choice behaviour when effect of Trial Block was significant at the p < 0.10 level but there was no main effect of Training Day (p > 0.10) and no Training Day x Trial Block interaction (p > 0.10). Rats took an average of 20 days to reach this criterion. 134 5.23 Reward magnitude discrimination A priori, we determined that if a particular manipulation reduced choice of the HR option, we would test the effects of this manipulation on a simpler, reward magnitude discrimination. In these experiments, separate cohorts of rats underwent initial lever pressing training as above, and were subsequently trained on a task that equated cost across the larger and smaller reward. Specifically, one press on either the HR or LR lever delivered 4 or 2 pellets, respectively throughout the session. The task was structured similar to the effort-discounting task, with sessions consisting of 48 trials, with 12 trials per block (2 trials at beginning of each block were forced choice trials and 10 were free choice trials). Rats were trained for approximately 10 days on this task at which point they predominately preferred the HR lever (~90% choice). 5.24 Surgery Surgery was conducted in a similar manner to Chapter 2. However, the location of cannulae placement differed. Specifically, two surgeries were performed. Separate group of rats were implanted with bilateral cannulae lowered to 1 mm dorsal to the either the NAc core (coordinates: [AP +1.6 mm from bregma, ML +/- 1.8 mm from bregma, DV -6.5 mm from dura) or NAc shell (coordinates: [AP +1.3 mm from bregma, ML +/- 1.0 mm from bregma, DV -6.5 mm from dura]) secured in place with skull screws and dental acrylic. 5.25 Drugs and microinfusion protocols Rats recovered from surgery and were (re)trained to stability on the effort discounting or reward magnitude discrimination task for at least 5 days until they showed stable baseline choice for 3 consecutive days. One or two days prior to drug testing, rats received a mock infusion where stylets were removed from guide cannulae and stainless-steel injectors were inserted for the duration of the subsequent infusion, but no infusion was delivered. The day after showing 135 stable discounting, the group received its first microinfusion test day. All drug tests used a within-subjects design. Drugs or vehicle were infused at a volume of 0.3μl per hemisphere for dopaminergic drugs/saline and 0.5μl per hemisphere for CRF and aCSF. The difference in volumes used for dopamine drugs vs CRF was due to the lower solubility of the latter compound in aCSF (1μg/1μl). The dopaminergic agonists were chosen because they have been shown to alter decision making when administered systemically and affect other executive functions, including decision making, when infused directly into the NAc (Haluk and Floresco 2009; St Onge and Floresco 2009; Stopper et al. 2013). The dopaminergic drugs and doses used were the D1 receptor agonist SKF 81297 (0.2 and 2μg; Tocris Bioscience), the D2/D3 agonist quinpirole (1 and 10μg; Sigma-Aldrich), and the D3-preferring agonist PD 128907 (1.5 and 3μg; Tocris Bioscience). PD 128,907 was selected for the current study as this compound, in comparison with quinpirole, has a considerably higher selectivity for D3 relative to D2 receptors (450 times; Bristow et al. 1996; Sautel et al. 1995). Separate groups of rats received infusions of one dose of human/rat CRF (0.5 μg; Tocris Bioscience). The dose of CRF was selected as previous studies found that CRF at this dose reduces effort choice in the same effort discounting task and breakpoint in the progressive ratio task when infused into the VTA and elicits conditioned place preference and enhances cue-triggered motivation when infused into the NAc (Bryce and Floresco 2016; Lemos et al. 2012; Peciña et al. 2006; Wanat et al. 2013). All drugs used in the current study were dissolved in physiological saline (dopamine agonists) or aCSF (CRF infusions), sonicated until dissolved, and protected from light. Infusions were administered bilaterally via 30-gauge infusion needles protruding 0.8mm past guide cannulae, connected to polyethelene tubing attached to a microinfusion pump. The 136 pump was set to infuse at a rate of 0.4 μl/min. The injectors were left in for an additional 1 min to allow for diffusion. Rats were left in their home cage for 10 mins prior to behavioural testing. Separate groups of rats were allocated for testing each of the drugs and were matched for baseline choice behaviour. Drug doses were administered in a counterbalanced order across rats, using a within-subjects design. On separate test days, rats received counterbalanced infusions of either one of two doses of each dopaminergic drug and saline or CRF and aCSF. A maximum of 3 infusions were administered to minimize damage that can occur with repeated infusions. Rats were retrained on the task for at least one day or until they attained behavioural stability before receiving subsequent infusions. If there was a greater than 15% variation in HR choice relative to baseline performance on the day after an infusion test, rats received an extra day of retraining prior to testing. 5.26 Histology Figures 11A-D represent the ventral point of accurate cannula placements in the NAc core and shell in the effort discounting and reward magnitude tasks for those rats infusions with agonist infusions targeting D1 (Fig 12A), D2 (Fig 12B), D3 (Fig 12C) receptors, and CRF (Fig 12D). Twenty animals were removed from analyses due to placements outside of the NAc core or shell, with animals removed because placements were asymmetric (n=12), dorsal (n=6), or anterior/ventral (n=2) to the NAc. These missed placements are depicted in Figure 12E. 5.27 Data analysis The main dependent variable in these experiments was the proportion choice of the high effort/high reward option, factoring out trial omissions. Additional measures included choice latency, defined as the time between lever insertion and choice, and pressing rates on the HR lever, which was calculated by averaging the number of presses made on this lever per second, 137 averaged across all blocks. The number of trial omissions and locomotor activity (indexed by photobeam breaks) were also recorded and analyzed. Choice data were analyzed using two-way repeated measures ANOVAs, with Drug Dose and Block as two within-subjects factors. In these analyses, the main effect of Block was always significant (p < 0.01) and will not be reported further. All other performance variables were analyzed with a one-way within-subjects ANOVA. Data analyses were performed separately for each drug group and each NAc subregion. Multiple comparisons using Tukey’s post hoc test were used where applicable. 5.3 Results 5.31 D1 receptor stimulation 5.31.1 NAc core. Twenty-four rats with cannulae implanted into the intra-NAc core were initially prepared to receive infusion of the D1 receptor agonist prior to an effort discounting session. Two rats died prior to testing and 3 had placements outside of the NAc core, resulting in data from n=19 rats. SKF 81297 did not significantly affect choice behaviour (main effect of Treatment, F(2,36)= 1.60, n.s.; Treatment x Block interaction F(6,108)= 1.34, n.s., Fig 13A), choice latency (F(2,36)= 1.26, n.s., Fig 13B), or pressing rate (F(2,36)= 1.97, n.s., Fig 13C). However, these treatments were behaviourally active, in that they caused a significant increase in locomotion (F(2,36)= 12.52, p < 0.01) after treatment with the low (p < 0.05) and the high dose (p < 0.01) relative to vehicle infusion (Table 4). Additionally, infusions of SKF 81297 caused a slight, but statistically significant increase in the number of omissions at the higher, 2 g dose (F(2,36)= 5.21, p < 0.01 and Tukey’s, p < 0.01, Table 4). 5.31.2 NAc shell. A separate group of 14 rats received infusions of SKF 81297 in the NAc shell but the data from 4 were removed due to missed placements, leaving a final n=10 rats. 138 Analysis of the choice data yielded no effect of SKF 81297 on choice (F(2,18) < 1, n.s., Treatment x Block interaction F(6,108)=1.89, n.s., Fig 13D). Likewise, these treatments did not alter choice latency (F(2,18)= 1.07, n.s., Fig 13E), and, in contrast with D1R stimulation in the core, had no effect on locomotion (F(2,18)= 1.11, n.s., Table 4) or number of omissions (F(2,18) < 1, n.s., Table 4). Interestingly, SKF 81297 infusion significantly reduced pressing rate on the HR lever, at the highest dose (F(2,18)= 4.52, p < 0.05 and Tukey’s, p < 0.05, Fig 13F). Collectively, these data indicate that increased stimulation of D1 receptors in the NAc does not alter effort-related decision making nor does it affect deliberation times. However, these treatments induced differential effects on other performance measures, with NAc core D1 stimulation increasing locomotion and omissions, whereas similar treatments in the NAc shell reduced rates of lever pressing. 5.32 D2/3 receptor stimulation 5.32.1 NAc core. Eleven rats initially received intra-NAc core infusions of the D2/3 receptor agonist quinpirole. Data from one rat was removed due to inaccurate placement, leaving a total of n=10 animals for the analyses. In contrast to the null effect of D1 receptor stimulation, quinpirole markedly biased choice away from the large reward associated with a greater effort cost (Fig 14A). The ANOVA on these data revealed a significant main effect of Treatment (F(2,18)= 13.75, p < 0.001); Treatment x Block interaction F(6,54) < 1, n.s.). Post hoc comparisons confirmed that both the lower 1 g and higher 10 g dose reduced choice of the HR option across all blocks (p < 0.01, Fig 14A). The high dose of quinpirole also increased choice latencies (F(2,18)= 4.54, p < 0.05 and Tukey’s, p < 0.05, Fig 14B). Yet, even though animals were slower to make a choice and were less likely to select the HR lever, once they did choose this lever, quinpirole did not affect pressing rate on the HR lever (F(2,18) < 1, n.s., Fig 14C) or 139 locomotion (F(2,18) < 1, n.s., Table 4). These treatments also caused a slight, but nonsignificant, increase in the number of omissions (F(2,18)=3.64, n.s., Table 4). 5.32.2 NAc shell. Another 13 rats received infusions of quinpirole in the NAc shell. One rat did not complete testing and 3 were removed due to missed placements, leaving a final n=9 rats for the analyses. Mirroring the effect of NAc core D2 receptor stimulation, NAc shell quinpirole infusions reduced preference for the larger, yet more costly reward (F(2,16)= 7.26, p < 0.01; Treatment x Block interaction F(6,48) < 1, n.s., Fig 14D). However, this effect was only apparent after treatment with the higher, 10 g dose (p < 0.05). As was observed following NAc core infusions, quinpirole in the NAc shell significantly increased choice latencies at the higher dose (F(2,16)= 4.94, p < 0.05 and Tukey’s, p < 0.05, Fig 14E) but had no effect on pressing rate (F(2,16) < 1, n.s., Fig 14F), nor did it affect locomotion (F(2,16)= 1.56, n.s., Table 4) or omissions (F(2,16) < 1, n.s., Table 4). Taken together, these findings indicate that increasing NAc D2 receptor activity in both subregions of the NAc biases choice away from the more rewarding but costly option and increases deliberation times, with the NAc core appearing to be more sensitive to these treatments. However, these effects did not appear to be attributable to more general disruptions in motivational processes, as quinpirole infusions did not affect pressing rates once the higher reward option was chosen. 5.32.21 Reward magnitude discrimination. In light of the fact that quinpirole caused a marked reduction in preference for larger rewards associated with a greater effort cost, a subsequent experiment was conducted to ascertain if these treatments caused a more general reduction in preference for larger rewards. Seven well-trained rats received infusion of the 10 µg dose of quinpirole into NAc core prior to performing a reward magnitude discrimination, wherein rats chose between a larger and smaller reward, both of which were delivered after one 140 lever press. In contrast to the effects on effort discounting, NAc core quinpirole did not affect choice (F(1,6)= 3.48, n.s., Fig 14G), choice latencies (F(1,6) < 1, n.s., Fig 14G, inset), locomotion (saline=1130 ±147, quinpirole=1189±168) or omissions (saline=0.1 ±0.1, quinpirole=0.6±0.6) (both Fs (1,6) < 1, n.s.). Similarly, NAc shell quinpirole administration (n=7) also did not affect choice of the HR lever (F(1,6)= 1.40, n.s., Fig 14H), choice latencies (F(1,6) < 1, n.s., Fig 14H, inset), locomotion (saline=1132 ±110, quinpirole=1006±148; F(1,6)= 3.76, n.s.) or omissions (saline=0.0, quinpirole=0.0; F(1,6) < 1, n.s.). Collectively, these results indicate that the effects of NAc D2 receptor stimulation on effort-related decision making are not attributable to reduced preference for larger rewards. Moreover, the lack of effect on decision latencies in this experiment suggest that the increased deliberation times during effort-related decisions did not reflect a general psychomotor slowing. 5.33 D3 receptor stimulation 5.33.1 NAc core. Quinpirole activates both D2 and D3 receptors and as such, it was unclear whether the effects of this compound on effort-related choice was driven by stimulation of one or both of these receptors (Sautel et al., 1995). To disambiguate this further, a separate group of rats received infusions of PD 128,907, an agonist that has higher selectivity for the D3 receptor. Fourteen rats were tested in this experiment with 2 animals removed from the analysis due to cannula placements outside of the NAc core. Inspection of the choice data from the remaining 12 rats showed that these treatments actually caused a slight increase in choice of the high-effort option at the lower, 1.5 µg dose, but not at the higher, 3 µg dose. In fact, 10 of the 12 animals showed an increase preference for the HR option after treatment with the lower dose. However, formal analysis of these data indicated that this effect was not statistically reliable (F(2,22)=2.28, n.s.; Treatment x Block interaction F(6,66)= 1.19, n.s., Fig 15A). PD 128,907 administration did 141 not alter choice latencies (F(2,22)= 1.64, n.s., Fig 15B), pressing rate on the HR lever (F(2,22) < 1, n.s., Fig 14C) locomotion (F(2,22) < 1, n.s., Table 4) or number of omissions (F(2,22)= 1.14, n.s., Table 4). Thus, more preferential activation of D3 receptors in the NAc core alters effort-related behaviours in a manner that was distinct from the effects of quinpirole, which activates both receptors, with D3 receptor stimulation slightly increasing preference for larger, more costly rewards. 5.33.2 NAc shell. A separate group of 13 rats were tested in this group. One rat failed to complete testing and the placement of another fell outside of the NAc shell, leaving a final n=11 rats. When infused into the NAc shell PD 128,907 had no effect on choice (F(2,20)=1.80, n.s.; Treatment x Block interaction F(6,60) < 1, n.s., Fig 15D), choice latencies (F(2,20)= 1.07, n.s., Fig 15E), locomotion (F(2,20)= 1.12, n.s., Table 4), omissions (F(2,20) < 1, n.s., Table 4), or pressing rate (F(2,20) < 1, n.s., Fig 15F). 5.34 CRF 5.34.1 NAc core. Previous studies have shown that increasing CRF activity in the NAc core can induce a dopamine-dependent conditioned place preference and augments mesoaccumbens dopamine release in vitro (Lemos et al., 2012). Given these findings, it was of interest to us to assess whether increasing CRF activity in the NAc may alter effort-related choice in a manner that was similar to activation of dopamine receptors in this nucleus. To investigate this, 15 rats were trained and tested on the effort discounting task. The effects of CRF infusions into the NAc core on choice were quite distinct compared to the effects of dopamine receptor agonists, in that it caused a subtle, but noticeable flattening of the discounting curve (Fig 16A). Analysis of these data did not reveal a main effect of CRF on choice (F(1,14) < 1, n.s.). Indeed, when the choice data were averaged across all blocks, rats chose the HR option as often after CRF treatment 142 (67% +/- 6) as they did under control conditions (67% +/- 5). However, the analysis did yield a significant Treatment x Block interaction (F(3,42)= 4.82, p < 0.01). Simple main effects further confirmed that intra-NAc core CRF reduced choice of the HR option in the first block, when effort costs were low (2 presses; F(1,14)= 7.10, p < 0.05) but actually increased choice of this option when effort costs were high (20 presses; F(1,14)= 6.50, p < 0.05). These effects on choice were not accompanied by any alterations in choice latencies (F(1,14) < 1, n.s., Fig 16B), pressing rate (F(1,14)= 2.47, n.s., Fig 16c), locomotion (F(1,14) < 1, n.s., Table 4) or number of omissions (F(1,14)= 1.08, n.s., Table 4). 5.34.2 NAc shell. Twenty-one rats with cannulae implanted in the shell were trained and tested on the effort discounting task, however 6 placements were outside of the NAc shell leaving a final n=15 rats in this experiment. In contrast to the effects of CRF infusion into the NAc core, similar infusions into the shell subregion did not alter choice or any other behavioural measure (all Fs (1,14) < 1, n.s., Fig 16 D,E,F and Table 4). 5.35 Baseline choice differences across NAc core and shell groups Across all of the experiments in this study, there was an apparent trend that under control conditions, rats that were implanted with cannulae in the NAc shell displayed a greater preference for the HR option compared to those in the NAc core groups. Given that NAc core, but not shell, inactivation (Ghods-Sharifi & Floresco, 2010) reduces effortful choice, we wanted to assess if these differences in choice were due to cannulae implantation. To explore this, we first compared the average choice data from 3 days prior to surgery for all rats that were later implanted with cannulae in the core or shell, and compared it to the average choice data from 3 days following surgery. These data were analyzed with a three-way ANOVA, with Region (core vs shell) as a between-subjects factor and Phase (pre- vs post-infusion) and Block as two within- 143 subjects factors. As is apparent in Figure 17A, we observed that prior to any surgical implantation, rats that were to be allocated to the shell groups chose the HR option more frequently than those allocated to the core groups. The analysis of these data revealed a significant main effect of region (F(1,99)= 12.34, p < 0.001). Notably, there was no significant main effects of pre- vs post-surgery phase (F(1,99) < 1, n.s.), Region x Phase (F(1,99) =2.20, n.s.) or three-way interactions (F(3,297)= 1.51, n.s.). Therefore, the increased preference for the HR option observed in rats with shell vs core cannulae appeared to be due to pre-existing differences in baseline levels of performance. It is unclear what the underlying causes of these differences were, but it may be partially attributable to the fact that rats in the core group required significantly more days of training to achieve stable patterns of choice prior to surgery (mean=19 days, SD=1.53) compared to the shell group (mean=17 days, SD=0.95; F(1,99)= 41.48, p < 0.001). After surgery, rats required an average of 9 days to re-achieve stable patterns of choice prior to infusion, with rats receiving a mock infusion at least one day prior to their first infusion test day. To probe the differences between the groups further, we conducted a similar analysis taking the average effort choice of the first 3 days post-surgery and the average of the 3 days immediately prior to the first infusion test days. Again, we found that there was a significant main effect of region (F(1,99)= 28.53, p < 0.001), with those in the shell groups showing increased choice compared to those in the core condition (see Figure 17B). Contrary to above, we found a significant main effect of phase (F(1,99)= 23.16, p < 0.001), with rats showing an increase in overall choice of the high effort option during baseline sessions prior to testing compared to choice during the 3 days after surgical recovery. However, this effect appeared to be driven by further increase in preference for the high effort option in rats in the shell group, as 144 suggested by the significant three-way interaction (F(3,297)= 2.63, p < 0.05). Partitioning this interaction with two, two-way ANOVAs, we found that for rats in the core groups, there was no effect of Phase (F(1,55)= 2.07, n.s.) or Phase x Block interaction (F(3,165) < 1, n.s.). In contrast, for rats in the shell groups, simple main effects analyses revealed a significant main effect of Phase (F(1,44)= 24.10, p < 0.001) and Phase x Block interaction (F(3,132)= 4.19, p < 0.01), with these rats choosing the HR option more frequently during baseline training sessions prior to test, compared to post-surgery. Thus, with additional training, rats with cannulae implanted in the NAc shell tended to show an additional drift in baseline levels of choice, whereas those with core groups did not. Although it is again unclear what may have caused these differences, it is notable that cannulae implanted in the NAc core would have caused comparatively more damage to the overlying dorsal striatum relative to shell cannulae. Nevertheless, the results of these analysis suggest that these differences in preference for the high-effort/large reward option for rats in the core vs shell groups in these experiments appeared to be due to pre-existing differences between the groups, with the effects of cannulae implantation in different subregions of the NAc playing at most, a minor role. 5.4 Discussion The primary objective of this study was to characterize how increased activity at different dopamine receptors within subregions of the NAc alter effort-related decision making. A secondary aim was to compare these effects to those induced by increased CRF activity within these nuclei, given that CRF can enhance mesoaccumbens dopamine release. The main findings were that increased activation of D2-like receptors with quinpirole in either the NAc core or shell markedly shifted choice away from larger, more costly rewards and increased deliberation times. In contrast, activation of D1 or D3 receptors did not affect choice, but did alter other aspects of 145 behaviour such as locomotion and rates of lever pressing. On the other hand, infusions of CRF into the NAc core alter choice in a manner distinct from those induced by dopamine receptor agonists, flattening the discounting curve. 5.41 D1 receptor stimulation Normal D1 receptor tone in the NAc facilitates effort-related choice, as blockade of these receptors in either the core or shell subregions reduces preference for more palatable rewards associated with a greater effort cost, assessed with a concurrent choice procedure (Nowend et al., 2001). However, the present results indicate that pharmacological stimulation of these receptors neither increases nor decreases effort-related choice. Infusions of a D1 receptor agonist into the NAc core did increase locomotor activity, confirming that these treatments were behaviourally active. Moreover, similar treatments in the NAc shell caused a slight but statistically significant reduction in rates of lever pressing, suggesting that under some conditions, excessive D1 receptor activity in this nucleus may dampen response vigor. The lack of effect of intra-NAc infusions of SKF 81297 on effort-related choice contrasts with the effects of this treatment on other forms of cost/benefit decision making. Specifically, D1 receptor stimulation in the NAc optimizes choice behaviour during risk/reward decision making assessed with a probabilistic discounting procedure, increasing preference for larger/risky rewards when their probabilities were high and reducing risky choice when they were low (Stopper et al. 2013). These differential effects highlight that the manner in which dopamine activity in the NAc influences reward-related decision making can differ in a manner dependent in part on the type of costs (effort vs uncertainty) that are being evaluated when choosing between rewards of different magnitudes. 146 5.42 D2/3 receptor stimulation In contrast to the above-mentioned findings, infusions of quinpirole in either subregion of the NAc, which activates both D2 and D3 receptors, markedly shifted choice biases away from larger rewards associated with a greater effort cost. These treatments also increased deliberation times, indicating that excessive activation of these receptors can also slow decision speeds. Importantly, these treatments did not alter rates of pressing on the HR lever, locomotion or trial omissions, nor did they affect any behavioural measure when rats were tested on a reward magnitude discrimination where they chose between larger vs smaller rewards with equated effort costs. Furthermore, intra-NAc infusions of the same doses of this compound did not alter probabilistic discounting (Stopper et al., 2013), again demonstrating differences in the dopamine receptor mechanisms that regulate distinct forms of cost/benefit decision making. Taken together, the effects of intra-NAc quinpirole cannot be attributed to non-specific disruptions in motivational or discrimination processes. The NAc core appeared to be more sensitive to the effects of quinpirole on choice, as infusions of a higher dose in either the NAc core or shell was effective at altering behaviour, whereas a lower dose was only effective in the core. Notably, blockade of D2 receptors in either subregion of the NAc also reduced the tendency to exert more effort to obtain more palatable rewards (Nowend et al., 2001). From these data, it is apparent that NAc D2 receptor activity modulates effort-related decision making in the form of an “inverted-U” shaped function, wherein diminished or excessive activation of these receptors reduces the tendency to pursue larger or more preferred rewards associated with a greater effort cost. The effects of intra-NAc quinpirole reported here complement other findings that systemic or intracranial administration of this drug can perturb certain motivational processes. 147 For example, systemic treatment with this drug tended to reduce responding for food under a progressive ratio schedule (Depoortere et al., 1996). Similarly infusions of quinpirole into the OFC reduced motivation to engage in a 5-CSRT task, increasing the number of omissions and latency to respond (Winstanley et al., 2010). Likewise, systemic treatment with D2 receptor agonists reduced choice of larger rewards that may also deliver punishment (Blaes et al., 2018). These findings, in addition to the present data demonstrate that excessive activation of D2 receptors with this compound either systemically or in different forebrain regions can exert disruptive effects on various aspects of motivated responding. It is of particular interest to compare the present results to those from a recent study by Filla and colleagues (2018), that examined how overexpression of postsynaptic striatal D2 receptors affected performance on a variety of effort and value-related decision making tasks. In that study, mice that were engineered to overexpress postsynaptic striatal D2 receptors showed normal adjustments in choice in response to shifts in the relative value (i.e. palatability) of different rewards. On the other hand, overexpression of these receptors markedly reduced preference to repeatedly initiate the action of lever pressing to obtain more-preferred rewards. The authors posited that excessive activity at striatal D2 receptor sites perturbs effort-related choice by increasing sensitivity to effort costs, a conclusion that is entirely consistent with the effects of intra-NAc quinpirole infusions reported here. An important consideration when evaluating the behavioural effects of administration D2 receptor agonists into the NAc is that these compounds activate both postsynaptic receptors on NAc neurons as well as pre-synaptic autoreceptors on dopamine terminals (Baik, 2013; Meador-Woodruff et al., 1989). It is therefore unclear if the effects on effort-related choice are due to excessive stimulation of post-synaptic D2 receptors, suppression of NAc dopamine release, or 148 both. Thus, activation of dopamine autoreceptors by quinpirole may explain why choice behaviour elicited by D2 receptor activation mirrors that of reduced NAc dopamine tone (Nowend et al., 2001; Salamone et al., 1994; Salamone et al., 1991). That said, two lines of evidence suggest that the effect of D2 stimulation may not be driven primarily by reduced dopamine release. First, as mentioned above, overexpression of postsynaptic striatal D2 receptors altered effort-related decision making in a manner similar to intra-NAc quinpirole infusions (Filla et al., 2018). Second, dopamine release is also modulated by D3 autoreceptors, as administration of the D3 receptor agonist PD 128,907 reduces dopamine tone in the NAc (Sokoloff et al., 1990; Zapata & Shippenberg, 2002; Zapata et al., 2001). It follows that if the effects of quinpirole were driven primarily by an autoreceptor-mediated reduction in mesoaccumbens dopamine tone, treatment with a D3 receptor agonist that also reduces dopamine tone would be expected to induce similar pattern of results. Instead, intra-NAc infusions of a D3 receptor agonist did not recapitulate any of the effects of quinpirole treatment, as discussed below. In light of these considerations, it is reasonable to propose that the effects of intra-NAc quinpirole on effort-related decision making were driven more by excessive activation of postsynaptic D2 receptors, rather than presynaptic autoreceptors or D3 receptors. In contrast to the effects of quinpirole, infusion of the D3-preferring receptor agonist PD 128,907 in the NAc core did not significantly alter choice behaviour or latencies, although the lower dose tended to increase preference for larger, more costly rewards. Moreover, similar treatments in the NAc shell had no effect on any behavioural measure. Again, it is interesting to point out that the lack of effect of D3 receptor stimulation on effort discounting contrasts with a marked reduction in preference for large, uncertain rewards during probabilistic discounting (Stopper et al., 2013). 149 5.43 CRF stimulation It is notable that the effects of D2 receptor stimulation on effort discounting reported here bear a striking resemblance to those of acute stress or increased CRF activity (Bryce & Floresco, 2016; Shafiei et al., 2012). Thus, similar to the effects of intra-NAc quinpirole, one-hour of restraint stress, or infusion of CRF into the ventricles or VTA reduce tolerance for effort costs and increase decision latencies (Bryce & Floresco, 2016; Shafiei et al., 2012). These manipulations can also increase dopamine tone within the NAc (Dunn & Berridge, 1987; Holly et al., 2015; Kalivas & Duffy, 1995; Kalivas et al., 1987; Matsuzaki et al., 1989) and Chapter 4 found that central CRF infusion increased tonic VTA dopamine neuron activity. These findings, juxtaposed with the present results suggest that reductions in effort-related choice induced by stress and/or global increases CRF activity may be driven by increases in mesoaccumbens dopamine tone that in turn may cause excessive activation of D2 receptors and shift bias away from larger, more costly rewards. CRF activity in the NAc has been shown to evoke dopamine release (Lemos et al., 2012). As such, it was of interest to compare the effects of NAc dopamine receptor stimulation on effort discounting to those induced by intra-NAc infusions of CRF. Somewhat surprisingly, intra-NAc core CRF infusions did not cause an overall reduction in effort-related choice. Instead, this manipulation induced a subtle flattening of the effort-discounting curve, with rats preferring the more rewarding option less when effort costs were low but preferring this option more when effort costs were high. Thus, CRF treatment did not alter overall levels of choice of the larger more effortful lever when averaged across the session, but instead induced a more static profile of choice, with rats appearing to be less sensitive to changes in effort cost that occurred over the course of the session. This stands in marked contrast to the effects of CRF infusions in the 150 ventricles or VTA, which both caused a reduction in choice of higher effort option during effort discounting (Bryce & Floresco, 2016). As such, these data highlight that increased CRF activity in different brain regions can induce disparate effects on motivated behaviours and dopamine activity (Lemos et al., 2012; Wanat et al., 2013). Although the mechanisms through which CRF in the NAc core altered effort discounting remains unclear, it may be related to its ability to potentiate mesoaccumbens dopamine release. Thus, increasing dopamine tone via CRF infusions may interfere with signals pertaining to reward saliency (Redgrave et al., 1999) or changes in the amount of rewards received over time (St Onge et al., 2012). Interestingly, CRF microinjections into the NAc core can induce a conditioned place preference in naive rats (Lemos et al., 2012). Although speculative, it is possible that CRF-induced potentiation of NAc dopamine release may have blunted the ability of rats to integrate information about low vs high effort costs associated with larger rewards that is used to bias choice towards or away from that option. That said, the finding that intra-NAc core CRF did not recapitulate the effects of any dopamine receptor agonist suggests that these effects may be due to a combination of effects on dopamine transmission, post synaptic CRF receptors, and other neurotransmitters such as acetylcholine (Chen et al., 2012). Infusions of CRF into the NAc shell had no effect on choice or any other behavioural measure during effort discounting. Note that perturbations in effort-related choice has been observed in humans afflicted with stress-related disorders, such as depression, which is also associated with increased CRF activity (Nemeroff et al., 1984; Treadway et al., 2012). In this regard, it is interesting to note that intra-NAc shell infusions of CRF can elicit a depressive-like effect in the forced swim test (Chen et al., 2012). These findings, combined with the lack of effect on effort-related choice reported here suggest that if increased CRF activity contributes to 151 the etiology of depression, it may promote different symptoms of this disorder (e.g.; anergia, helplessness) via actions in distinct brain nuclei. 5.44 Conclusion The present findings provide novel insight into the complex mechanisms through which dopamine receptor activity shapes effort-related decision making, in a receptor-dependent manner. Excessive activation of NAc dopamine D2 receptors diminishes preference for larger, more costly rewards and slows deliberation times, but spares other aspects of motivated behaviour. In contrast, D1 receptor stimulation in the NAc shell can reduce response vigor, whereas D3 receptor stimulation slightly increased effort choice, without affecting other aspects of decision making. These effects were distinct from the more subtle alterations in choice induced by increased CRF activity. Clarifying the neurochemical underpinnings of effort-related decision making is of particular relevance, considering that perturbations in these functions have been reported in a variety of disorders, including depression (Treadway et al., 2012), schizophrenia (Treadway et al., 2015), and obesity (Mata et al., 2017). The reduced tendency to exert greater effort to obtain larger rewards has commonly been attributed to diminished dopamine activity, given the substantial body of preclinical studies demonstrating that reduction in dopaminergic tone shifts preference away from preferred yet more costly rewards. However, the present findings suggest that, at least in some instances, motivational impairments associated with some disorders may in fact be driven by excessive, rather than reduced activation of D2 receptors within the NAc. As such, specifying the different types of neurochemical abnormalities underlying anergia may aid in developing distinct treatment strategies for these symptoms across different disorders. 152 Locomotion # of Omissions NAc Core NAc Shell NAc Core NAc Shell SKF 81297 saline 1133 (73) 1357(116) 1.8 (1.1) 0.0 (0) 0.2 µg 1365 (94)* 1457(156) 2.4 (1.1) 0.0 (0) 2.0 µg 1567 (93)** 1281(118) 3.3 (1.2)** 0.0 (0) Quinpirole saline 1120 (118) 1258 (201) 0.0 (0) 0.0 (0) 1.0 µg 1169 (165) 1489 (176) 0.5 (0.4) 0.2 (0.2) 10.0 µg 1158 (186) 1440 (167) 7.5 (3.6) 1.2 (1.2) PD 128,907 81297 saline 1021 (114) 1163 (107) 0.3 (0.1) 0.2 (0.1) 1.5 µg 1121 (121) 1344 (208) 0.3 (0.1) 0.1 (0.1) 3.0 µg 1070 (108) 1218 (125) 2.9 (2.5) 0.0 (0) CRF aCSF 1156 (96) 1405 (138) 0.1 (0.1) 0.2 (0.1) 0.5 µg 1224 (155) 1271 (142) 1.8 (1.7) 0.5 (0.2) Table 4. Mean (S.E.M) locomotor activity (photobeam breaks) and number of omissions (over 48 trials) after treatment with vehicle (saline) or D1, D2, D3 receptor agonists or CRF during effort discounting. * indicates significance at p < 0.05 **indicates significance at p < 0.01 vs saline. 153 Figure 12. NAc core and shell placements. Locations of infusions into the NAc core (squares) and shell (circles) of a) the D1 receptor agonist SKF 81297 b) the D2/3 receptor agonist quinpirole, c) the D3-preferring agonist PD 128,907 and d) CRF, and e) locations of missed placements outside of the NAc core (squares) or shell (circles). 154 Figure 13. D1 receptor stimulation in the NAc core or shell does not alter effort discounting. a) Proportion choice of the HR lever as a function of the amount of effort (lever presses) required to obtain the larger reward across the four trial blocks after infusions of vehicle or SKF 81297 in the NAc core. b-c) choice latencies and pressing rate were also unaffected. SKF 81297 in the NAc shell had no effect on d) choice or e) choice latencies, but did the higher dose did reduce f) pressing rate. Stars denotes p<0.05 vs vehicle 155 Figure 14. D2/3 receptor stimulation in the NAc reduced high effort choice. a) NAc core quinpirole infusions dose-dependently reduced preference for the HR option at both the low (1g) and high (10g) dose. The high dose also b) increased choice latencies but c) had no effect on pressing rate. d) NAc shell quinpirole infusions reduced choice of the HR option at the higher dose (10g) and e) increased choice latencies, but f) had no effect on pressing rate. g-h) infusions of quinpirole (10g) into either NAc subregion did not affect choice during reward magnitude discrimination, wherein rats choice between larger vs smaller rewards of equal effort costs. These treatments also did not affect choice latencies (insets). 156 Figure 15. Effects of D3 receptor stimulation in the NAc core and shell on effort discounting. Effects of D3 receptor stimulation in the NAc core and shell on effort discounting. a) Infusions PD 128,907 into the NAc core did not alter choice significantly, but did cause a slight increase at the lower dose (1.5g) b) These treatments also did not alter choice latencies c) or pressing rate d) NAc shell PD 128,907 infusions did not alter effort discounting, e) choice latency or f) pressing rate. 157 Figure 16. Effects of CRF infusion in the NAc core and shell on effort discounting. a) Infusions into the NAc core reduced choice of the HR option when the effort requirement was low, and increased it when it was high. These treatments did not affect b) choice latencies or c) pressing rates. d) Similar CRF infusions in the NAc shell had no effect on choice, and also did not affect e) choice latencies or f) pressing rate. 158 Figure 17. Baseline effort choice in the NAc core and shell groups. a) Animals in the NAc shell group showed a higher preference for the high effort choice at baseline compared to animals in the NAc core group, that was not altered by cannula implantation. b) Those in the NAc shell but not core group further increased effort choice immediately prior to first infusion compared to immediately following surgery. 159 Chapter 6: General Discussion The present series of experiments described in this Thesis have interrogated how acute stress and exogenous manipulations of CRF alter cognition, primarily concerned with cognitive flexibility and decision making. In doing so, we identified potential interactions with dopamine that may partially mediate the relationship between CRF and cognition. As such, we further probed changes in dopamine within the mesolimbic system in response to increased CRF signaling and connected these findings with alterations in decision making. In the following discussion, the implications of these findings will be discussed, with a specific focus on comparing the key differences and similarities between acute stress and CRF in altering component processes of cognition, the interactions between CRF and dopamine on physiology and cognition, and providing insight into how increased CRF signaling may play a role in symptom clusters common to stress-related neuropsychiatric disorders. 6.1 Overview of findings In Chapter 2, we assessed the role of acute stress and CRF hyperactivity on cognitive flexibility with probabilistic outcomes in male and female rats. The primary findings were that acute stress and CRF had a minor effect on flexibility and probabilistic reinforcement learning, albeit in opposing directions. Acute stress impaired performance during acquisition of the task in males, whereas increased CRF activity slightly enhanced flexibility in both sexes with extensive training. The current findings extend previous literature demonstrating that acute stress impairs cognitive flexibility with deterministic outcomes in males, but not in females (Laredo et al., 2015; Shields et al., 2016). When integrated with previous literature, these results indicate that acute stress impairs flexibility in males regardless of reinforcement schedule. It is notable that impairments in flexibility were only found when acute stress was administered prior to task 160 acquisition. Therefore, it is plausible that acute stress impeded learning. However, this is unlikely given that acute stress did not increase the number of errors committed during the initial discrimination phase when animals learn the contingencies of the two response options. Rather, it appears that acute stress subtly increased error rates over the course of the reversal stage of the task. In regards to CRF, the current findings are at odds with the only previous study regarding the role of central CRF effects on cognitive flexibility, in which they found that CRF impaired flexibility in a deterministic set-shifting task (Snyder et al., 2011). This may be due to differences in CRF doses, reinforcement schedules, or training protocols. The higher central CRF dose used in the current study facilitated flexibility when the odds were probabilistic in well-trained animals in comparison to a lower central CRF dose used in the previous study which impaired set-shifting and reversal learning when the reinforcement was deterministic in animals first exposed to the changing contingencies. Future work should establish how these factors specifically influence flexibility. In Chapter 3, we again found differential effects of acute stress and central CRF on risk/reward decision making as well as conflicting results based on how contingencies were presented. Neither manipulation altered risky choice when contingencies were internally generated via previous feedback information. Juxtaposed with our previous finding that both acute stress and central CRF reduce choice of the more preferable option associated with a higher effort cost, the current results demonstrate that stress manipulations are specifically sensitive to effort costs but do not alter other forms of internally-generated cost/benefit decision making. However, when these odds were externally cued, CRF signaling reduced optimal risky choice by reducing preference for the more advantageous option, whereas acute stress slightly increased risky choice, irrespective of odds. Therefore, the ability of stress and central CRF to 161 influence both cognitive flexibility and risk/reward decision making depends, to a large extent, on experimental methods such as training and task structure. Nevertheless, even with these considerations, it appears as though acute stress and central CRF exert opposing behavioural outcomes which will be discussed in further detail below. It should also be emphasized that central CRF infusion likely impaired motivation by amplifying ‘indecision’ and disengagement. This was true regardless of whether the task involved uncertainty, such as the cognitive flexibility and risk/reward decision making tasks, or when the task was deterministic and the cost was physical exertion, such as in the effort-related decision making task (Bryce & Floresco, 2016). Moreover, increased choice latencies are a consistent characteristic following central CRF administration (Beard et al., 2015; Van’t Veer et al., 2012) and in depressed participants (Lemke et al., 1999; Sobin & Sackeim, 1997) in a variety of tasks. Therefore, CRF hyperactivity may be a key mediator in amotivational aspects of stress and depression. Given the primary function of dopamine in cost/benefit decision making (Nowend et al., 2001; Salamone et al., 2012; Stopper & Floresco, 2015) and the complex effects of mesocorticolimbic dopamine activity in response to CRF activity (Beckstead et al., 2009; Lodge & Grace, 2005; Ungless et al., 2003; Williams et al., 2014), it was previously unclear how centrally-acting CRF would alter dopamine activity. As such, we identified the role of central CRF on VTA dopamine neuron activity in Chapter 4 and, in Chapter 5, we clarified how increased dopamine activity within distinct subregions of the NAc influences decision making. Results of these studies demonstrated that increased central CRF signaling increased tonic dopamine activity in the VTA in a similar manner to acute restraint stress (Valenti et al., 2011). However, acute stress also increased phasic dopamine activity but did not affect firing rate 162 (Valenti et al., 2011), whereas central CRF had no effect on phasic dopamine activity but increased dopamine firing rate. It is notable that both acute stress and central CRF similarly altered choice in an effort-related decision making task (Bryce & Floresco, 2016), which is mediated in large part by mesolimbic dopamine (Aberman & Salamone, 1999; Nowend et al., 2001; Salamone et al., 1994). Therefore, it is tempting to speculate that acute stress and central CRF signaling reduce effort choice by increasing tonic mesolimbic dopamine activity. Evidence in support of this hypothesis finds that increasing activity at the D2 receptor in the NAc reduces effort choice in a parallel manner to both acute stress and central CRF as detailed in Chapter 5. 6.2 Implications of acute stress and central CRF on cognitive flexibility and risky choice The impetus for investigating the role of acute stress and central CRF on cognitive flexibility and risk/reward decision making tasks was derived from our earlier study revealing that both acute stress and central CRF administration reduce choice of the more preferable option when this option requires more physical effort to obtain but not when costs are equal (Bryce & Floresco, 2016). This led us to question how these two manipulations alter other forms of cost/benefit decision making and if alterations in choice are due to changes in cognitive flexibility. Putting these results together, we find an interesting pattern of results with important implications. For the sake of comparison, only effects in males will be discussed in the following three sections. Comparisons between males and females will be detailed in the sex differences section below. First, we found that alterations in decision making following central CRF administration are not due to changes in the ability to flexibly adapt to changing circumstances. That is, reductions in effort and risk choice following CRF infusion do not appear to be due to altered cognitive flexibility. Evidence for this is derived from Chapter 2, which finds that the same CRF 163 dose that reduces choice of the more preferable option in effort-related and external cue-guided risk tasks, slightly facilitates flexibility in a PRL task. Moreover, the same CRF dose has no effect on risky choice when the contingencies systematically change over the course of the session with animals keeping track of previous outcomes to guide subsequent choice behaviour in a probabilistic discounting task. This null effect holds true when odds on the risky lever start good and progressively get worse over the session and when odds start poor and get progressively better over the session. If a manipulation caused an impairment in flexibility, one would expect risky choice to be preferable over the course of the session when odds of choosing risky start at 100 percent and decrease over blocks of trials. Conversely, we would expect risky choice to remain low over the course of the session when odds of choosing risky start at 6.25 percent and increase over blocks of trials. Such impairments are found with systemic increases in dopamine tone via amphetamine administration (St. Onge et al., 2010) and inactivation of the mPFC (St. Onge & Floresco, 2010). Collectively, these experiments provide evidence that central CRF modulates decision making independently from alterations in cognitive flexibility. On the other hand, acute stress impairs cognitive flexibility in males. Therefore, it is plausible that acute stress may alter decision making by impairing cognitive flexibility. Impaired flexibility may be why acute stress increased overall risky choice irrespective of odds in the external cue-guided Blackjack task. That is, acute stress may have increased perseveration on the risky lever, regardless of whether the auditory cue indicated greater expected utility. However, one would expect that if acute stress increased perseveration on the risky option regardless of odds, that risky choice would also be increased in the probabilistic discounting task. Instead, we find that acute stress did not alter risky choice when odds changed from good to poor or when odds changed from poor to good over the course of the session in a probabilistic discounting 164 task. Therefore, it is unlikely that either central CRF or acute stress altered decision making via perturbations in cognitive flexibility. Second, the manner in which acute stress and central CRF alter decision making depends on the type of cost involved. For certain costs, both acute stress and central CRF administration show similar effects. Specifically, given the choice between a small reward requiring one lever press or a large reward requiring 2-20 lever presses, both of these manipulations shift preference away from the larger, costlier reward (Bryce & Floresco, 2016). The reduction in effort choice is not due to alterations in reward processing, discrimination between rewards of different magnitudes, or general amotivation, as neither manipulation alters choice between a small reward and a large reward when the costs are equated (i.e. one lever press each). Rather, central CRF reduces effort choice by reducing the motivation to work for reward as evidenced by reduced willingness to lever press for sugar pellet reward in a progressive ratio task in a similar manner to chronic stress manipulations (Olausson et al., 2013). Acute stress, however, did not alter responding on a progressive ratio task in our lab (Shafiei et al., 2012), indicating that central CRF and acute stress may alter effort choice in the same direction via distinct mechanisms or that effort choice is more sensitive to stress alterations than impairments in motivation. Although effort costs are particularly sensitive to stress manipulations, the influence of other types of costs are more nuanced. Previous work in our lab finds that acute stress is without effect when the choice is between a small, immediate reward and a large, delayed reward (Shafiei et al., 2012). Decision making alterations have been influenced, sometimes in opposing directions, by acute stress or central CRF or neither manipulation. Therefore, it is unlikely that central CRF would alter delayed choice. However, future studies should investigate the role of CRF in delay discounting to rule out this possibility. 165 As it pertains to risky decision making, previous studies in humans consistently demonstrate that acute stress manipulations increase risky choice (Preston et al., 2007; Simonovic et al., 2017; Wemm & Wulfert, 2017). In rodents, inescapable foot shock (Nobrega et al., 2016) and CORT injections (Koot et al., 2013) also increase risky choice. However, treatment with the pharmacological stressor, yohimbine, impaired adjustments in risky choice biases on a probabilistic discounting task similar to the one used here (Montes et al., 2015), indicating that the manner in which stress may alter risk/reward decision making is also dependent in part on the type of stressor used. Therefore, the current series of experiments explored the role of acute stress on risky choice using two separate risk/reward decision making tasks given that this had yet to be assessed in preclinical rodent models. Here we found that alterations in risky choice following acute stress and central CRF depend on the manipulation and whether the task involved internally integrating information about reward from previous feedback or was guided by external cues signaling reward probability. First, we found that neither central CRF nor acute stress have any effect when the choice is between a small, certain reward and a large, risky reward with reward probabilities systematically ranging from 6.25 to 100 percent in ascending or descending order. These may differ from previous effects found in humans using the IGT for various reasons. One such reason may be related to differences in design and information processed across the various tasks. For example, different versions of the IGT used with humans and rats require subjects to hold risk/reward information about four different choice options, which may tax working memory processes to a greater degree than the two options in the probabilistic discounting task used here. Indeed, cross-species studies show that increased circulating stress hormones or increased CRF impair working memory functions 166 (Duncko et al., 2009; Hupalo & Berridge, 2016; Roozendaal, 2004; Shields et al., 2015; Uribe-Mariño et al., 2016). When auditory cues guided choice in the Blackjack task, both acute stress and CRF altered risky choice, albeit in opposing directions. Similar to probabilistic discounting, this task involves choice between a small, certain option and a large, risky option. Unlike probabilistic discounting, auditory cues signaled two trial types; in one trial type the risky choice was advantageous at 50 percent probability of reward, whereas the other trial type indicated that risky choice was disadvantageous at 12.5 percent probability of reward. Mirroring previous cross-species work, we found that acute stress increased risky choice. Importantly, acute stress did not improve or impair decision making per se, as risky choice was increased during trials in which this option was both advantageous and disadvantageous in terms of accruing the most rewards. Conversely, CRF impaired optimal decision making, as risky choice was only reduced when auditory cues signaled that this option was the most advantageous. Putting these results together with previous work indicates that central CRF does not alter the ability to distinguish between rewards of differing magnitudes (Bryce & Floresco, 2016) or between auditory cues signaling different risk probabilities, as animals were still able to inhibit risky choice when the auditory cue signaled this option was disadvantageous. Rather, CRF appears to specifically bias preference away from the most advantageous option when externally cued. It is also plausible that CRF reduced tolerance for uncertainty given that this option delivered reward with a maximally uncertain probability of 50 percent, indicating that CRF may mediate perturbations in optimal choice involving uncertainty, perhaps by reducing tolerance for uncertainty. When taking the cognitive flexibility and risk/reward decision making results together, it is notable that acute stress and central CRF exert opposing effects on behaviour in males on both 167 tasks, which can tell us important information about potential divergent mechanisms of action. That is, central CRF slightly facilitates flexibility and impairs optimal risky decision making with external cues, whereas acute stress impairs flexibility and slightly facilitates risky decision making with external cues. Analyzing previous studies reveals that prelimbic mPFC inactivation enhances flexibility in the PRL task (Dalton et al., 2016) and reduces optimal risky choice on the Blackjack task (van Holstein et al., 2019) in a similar manner to central CRF infusion, indicating that increasing CRF signaling may alter flexibility and risky choice by suppressing activity within the mPFC. Indeed, central CRF infusion reaches the mPFC, as previous work shows that central CRF induced activation of the immediate early gene, c-Fos, in the mPFC (Bittencourt & Sawchenko, 2000). That said, bath applied CRF to mPFC slices increases the excitability of pyramidal neurons (Liu et al., 2015), enhancing rather than suppressing mPFC activity. However, it is unclear if centrally applied CRF in a behaving animal would have the same effect. Importantly, central CRF does not alter probabilistic discounting in a similar manner to mPFC inactivation, which increases or decreases risky choice dependent on whether the odds are presented in ascending or descending order (St. Onge & Floresco, 2010). The ability of CRF to dampen mPFC activity may be stimulus-dependent, thus exerting divergent effects depending on the underlying circuitry involved in the task. Therefore, the possibility for central CRF to dampen mPFC activity or output to downstream regions, altering PRL and cue-guided risky decision making, but not risky choice during probabilistic discounting, cannot be conclusively ruled out. Alternatively, suppressing medial or lateral OFC or NAc shell activity impaired PRL flexibility and may be possible sites of action for acute stress-induced impairments (Dalton et al., 2014; Dalton et al., 2016). The role of the OFC has yet to be elucidated in regards to risky choice on the Blackjack task, but NAc shell inactivation increased risky choice on this task (Floresco et 168 al., 2018), in a similar manner to acute stress. Unlike acute stress, however, NAc shell inactivation reduced risky choice on the probabilistic discounting task (Stopper & Floresco, 2011). Collectively, it is tempting to speculate that increasing CRF activity acts directly or indirectly to reduce mPFC activity, whereas acute stress via unknown mechanisms acts to reduce OFC and/or NAc shell activity, that in turn differentially alters cognitive flexibility and risky choice in certain contexts. Another intriguing candidate mechanism for differential stress and CRF alterations is 5-HT. As reported above, alterations in 5-HT may underlie the difference in PRL flexibility, with low doses of citalopram, which increases 5-HT levels, impairing and high doses of citalopram facilitating flexibility (Bari et al., 2010), similar to acute stress and central CRF, respectively. Indeed, ICV CRF infusion (3g) at the same dose used in the current study increased tryptophan hydroxylase levels (Singh et al., 1992), which is the rate limiting enzyme in 5-HT synthesis, and acute restraint stress increased the 5-HT synthesis rate in various brain regions in rats (Haleem & Parveen, 1994). Whether high dose central CRF increases 5-HT to a greater extent than acute restraint stress is unclear given the differences in methodology between studies. As it relates to risk/reward decision making, previous work has found that a 5-HT agonist impaired risky choice on a rodent gambling task, which is a task similar to the human IGT, shifting preference away from the optimal risky choice (Zeeb, Robbins, & Winstanley, 2009). Although difficult to translate to the current risk/reward decision making tasks, when taken at face value, this reduction in optimal risky choice appears similar to the influence of central CRF on the external cue-guided Blackjack task. Together, it is conceivable that alterations in 5-HT activity may be the mechanism that underpins differences between acute stress and central CRF effects on PRL and risk/reward decision making. 169 Finally, the divergent effects of central CRF and acute stress may be that high dose central CRF acts more similarly to chronic than acute stress. Indeed, central CRF overexpression in genetic rodent models mirrors chronic stress (Dirks et al., 2002) and CRF-induced suboptimal choice behaviour on the external cue-guided risk/reward decision making task more closely resembles the effects of chronic, rather than acute, stress (Morgado et al., 2015). Although the role of chronic stress on PRL has yet to be investigated, prolonged or chronic stress impairs cognitive flexibility on deterministic reversal learning and set-shifting tasks by increasing the number of errors (Bondi et al., 2010; Bondi et al., 2008; Hurtubise & Howland, 2016; Jett et al., 2015; Jett et al., 2017; Lapiz-Bluhm et al., 2009). This appears to be in opposition to our results as we found that CRF slightly enhanced flexibility by increasing the number of reversals but had no effect on the number of errors committed. Given the lack of studies on PRL in preclinical models, however, it is difficult to say if increased CRF signaling is consistent with chronic stress or depression on this task. As such, future studies should assess the role of chronic stress in PRL. In comparison, restraint stress alters risky choice on the cue-guided Blackjack task and performance on the PRL task in a manner comparable to other acute stress manipulations in both rodents and humans (Butts et al., 2013; Koot et al., 2013; Laredo et al., 2015; Nobrega et al., 2016; Preston et al., 2007; Putman et al., 2010; Shields et al., 2015; Shields et al., 2016; Simonovic et al., 2017; Starcke et al., 2008; Wemm & Wulfert, 2017). Therefore, exogenously increasing CRF signaling may lead to more chronic stress-like behavioural manifestations in some cognitive domains, which may include suboptimal cost/benefit decision making. 6.21 Reward and negative feedback sensitivity Given that both cognitive flexibility and risk/reward decision making tasks involve probabilistic reinforcement, we are able to assess how central CRF and acute stress alter 170 sensitivity to reward and negative feedback. The influence of central CRF on reward sensitivity was nuanced, as this manipulation only reduced sensitivity to reward when first learning about the PRL task during acquisition. Given that there were no alterations in reward sensitivity during risk/reward decision making or during PRL testing when the task was well-learned indicates that CRF only reduces reward sensitivity during acquisition of the task when gathering information about the different action outcome associations and not when these associations have already been established. Reward hyposensitivity is consistent with previous work finding acute stress induced reductions in reward sensitivity (Bogdan & Pizzagalli, 2006; Kumar et al., 2014; Lemos et al., 2012; Petzold et al., 2010; Porcelli et al., 2012) and CRF-induced impairments in motivation and reward-related behaviours, including reducing food consumption, choice of larger rewards with more effort, motivation to work for sugar reward, and conditioned place preference (Bell et al., 1998; Bryce & Floresco, 2016; Cador et al., 1992; Glowa et al., 1992; Heinrichs, Britton, & Koob, 1991; Wanat, Bonci, & Phillips, 2013). Although the mechanism was not assessed, perhaps interactions between CRF and dopamine dampen reward sensitivity given the prominent role of dopamine in motivation and reward processing (Wise, 2004) and the CRF-induced reduction in phasic mesolimbic dopamine release in response to rewards (Wanat et al., 2013). More consistently, central CRF markedly reduced negative feedback sensitivity on both PRL task versions, and on risk/reward decision making tasks, although the latter effects were not significant, indicating that central CRF alters negative feedback processing particularly during PRL when both choice options are probabilistically reinforced with rewards of the same magnitude. Reduced ability to use negative feedback to guide subsequent behaviour is consistent with studies finding acute stress in humans reduces negative feedback sensitivity in a 171 probabilistic reinforcement task (Petzold et al., 2010), which may be due to stress-induced suppression of mPFC activity during feedback about monetary loss (Treadway et al., 2013), and reduced conditioned taste aversion following acute stress in rodents (Bourne et al., 1992; Revusky & Reilly, 1989). Collectively indicating that CRF may reduce learning from negative feedback in a manner similar to acute stress. Why CRF reduces the tendency to follow a reward omission with a switch in the response option to a greater degree during PRL is currently unclear. However, a parsimonious interpretation is that PRL tasks provide potential for more negative feedback given 200 trials in which both levers are probabilistically reinforced as compared to 40 (Blackjack) or 90 (probabilistic discounting) trials in which only one lever is probabilistically reinforced in risk/reward decision making tasks. As such, negative feedback is critical for optimal PRL for the duration of the session as reward omission is the only signal that a switch in reward contingencies has occurred. Importantly, negative feedback also occurs on the minority of ‘correct’ trials in PRL tasks. Therefore, CRF may alter the ability to use negative feedback information to keep track of previous action-outcome associations over a sustained period of time when both levers have been probabilistically reinforced. On the other hand, negative feedback becomes more crucial on risk/reward decision making tasks during trials in which the risky lever is no longer advantageous (i.e. during the 12.5 and 6.25 percent blocks or trial type) and thus requires a switch in preference away from this option. Therefore, CRF may subtly alter negative feedback sensitivity that only emerges with an increased volume of trials, or CRF may alter the ability to hold information about response options that require learning from reward omission, particularly when both options have been associated with reward omission. 172 Interestingly, prelimbic mPFC inactivation (Dalton et al., 2016) and high dose systemic administration of the 5-HT-enhancing drug, citalopram (Bari et al., 2010), both reduce negative feedback sensitivity. Above we identified both prelimbic mPFC and high dose 5-HT signaling as mechanisms that facilitate cognitive flexibility in a similar manner to central CRF infusion. Thus, CRF hyperactivity may alter flexibility and negative feedback sensitivity by altering mPFC and/or 5-HT activity. However, similar to reductions in reward sensitivity, reduced negative feedback sensitivity may also be mediated by interactions between CRF and dopamine. As Chapter 4 demonstrates, high dose central CRF increased tonic VTA dopamine activity, which would increase mesolimbic dopamine release in terminal regions (Dunn & Berridge, 1987; Kalivas & Duffy, 1995; Matsuzaki et al., 1989). Increased mesolimbic dopamine tone may serve to obscure dips in phasic dopamine release in response to reward omission (Fiorillo et al., 2008), leading to reduced learning from negative feedback. In contrast, acute stress increased reward sensitivity, although the only significant increase was on good odds trials during the external cue guided Blackjack task. Increased reward sensitivity fits with previous studies showing that acute stress increased reward- and approach-related behaviour for drug reward (Marinelli et al., 1998; Marinelli & Piazza, 2002; Serfling et al., 2019) and mild acute stress increases both ‘liking’ and motivation for highly palatable foods in humans (Lemmens et al., 2011; Oliver et al., 2000; Rutters et al., 2009), but is in disagreement with other studies consistently finding that acute stress reduces reward sensitivity (Bogdan & Pizzagalli, 2006; Kumar et al., 2014; Lemos et al., 2012; Petzold et al., 2010; Porcelli et al., 2012). Alternatively, acute stress did not significantly alter negative feedback sensitivity, with no consistent trend in altering this measure in either direction when the tasks are viewed collectively. The null effect is in contrast to the reduced ability to use negative feedback to 173 correctly avoid a less rewarding stimuli following acute stress in humans (Petzold et al., 2010) and dampening of mPFC activity during feedback about monetary loss (Treadway et al., 2013). This suggests that the ability of stress to modify subsequent action selection based on the integration of reward and reward omission feedback may be sensitive to task structure. 6.22 Motivation Motivation is a vital component in all cognitive domains, as unmotivated behaviour may obscure alterations in cognition. This is especially important when probing the role of stress on cognition as stress and stress-related mechanisms can independently alter motivation (Bryce & Floresco, 2016; Shafiei et al., 2012; Wanat et al., 2013). Unsurprisingly, both acute stress and central CRF impaired motivational aspects of both PRL and decision making tasks, although in distinct patterns. More specifically, we revealed that central CRF impaired motivation by increasing the number of trials omitted both when first acquiring and when extensively trained on the cognitive flexibility task. This effect was more dramatic, however, once reward contingencies had switched, indicating that CRF causes task disengagement when the rules of the game change. Instead of switching strategies and trying the other lever option, animals stop playing the game, even though choosing either lever would be more advantageous given that they both are probabilistically reinforced. This is perhaps why increased CRF signaling causes reduced sensitivity to negative feedback, given that instead of switching to the other lever following reward omission, animals choose neither option. Increased CRF activity also increased the number of trials omitted in the probabilistic discounting, but not the external cue-guided blackjack task. In the probabilistic discounting task, CRF tended to increase task disengagement towards the end of the session, regardless of the order the odds were presented, which may 174 reflect a form of decision fatigue, rather than a general reduction in motivation to engage in the task. When analyzing the effects of CRF on all three tasks, a parsimonious explanation for the lack of effect on omissions during the Blackjack task may simply be that rats were given fewer trials (40) compared to the PRL (200 or 240) and probabilistic discounting task (90), indicating that motivational impairments occur when animals are subjected to more trials. Indeed, an accumulation of negative feedback may frustrate subjects into disengagement. However, these effects are also consistent with CRF impairing motivation when PRL and probabilistic discounting tasks involve an increased cognitive load that is concomitant with many trials taxing working memory and attention. Given the CRF-induced impairments to working memory and attention (Hupalo & Berridge, 2016; Van’t Veer et al., 2012), this is a strong possibility. That said, increased CRF signaling also increased trial omissions when the choice was between a smaller, easily accessible reward and a larger reward requiring more effort and causes disengagement with lever pressing for reward earlier in a progressive ratio task than in vehicle treatment (Bryce & Floresco, 2016). Therefore, increased CRF activity may cause a general reduction in motivated responding when the task requires more cognitive or physical effort. In contrast, by using external cues to signal probability of reward or by equating effort costs between two lever options (Bryce & Floresco, 2016), cognitive and physical load is reduced and animals are less likely to be impaired by CRF treatments. Similarly, these auditory cues were also presented before animals were able to make a choice and may have had an invigorating effect on responding, incentivizing animals to re-engage with the levers following the inter-trial interval. These explanations are not mutually exclusive and may work in concert to impair motivation depending on task structure. 175 CRF treatment also increased choice latencies in cognitive flexibility and both decision making tasks. This result is similar to the effects of CRF treatment on decision latencies during effort-related decision making, which also increased deliberation periods when rats chose between smaller, easily obtainable rewards and larger ones associated with a greater effort cost. CRF hyperactivity caused ‘indecision’ even when the choice was between lever pressing for a small reward and a larger reward of equal cost, even though there was no effect on choice (Bryce & Floresco, 2016), indicating that CRF independently modulates latencies and choice behaviour. Furthermore, central CRF infusions have been reported to increase choice latencies in attentional tasks (Beard et al., 2015; Van’t Veer et al., 2012). When viewed collectively, this pattern indicates that central CRF activity causes ‘indecision’ that is independent from effects on other aspects of cognition, including effort choice, risky choice, or flexibility, and is consistent regardless of task structure. Acute stress impaired motivation, albeit in a more limited capacity. Acute stress was without effect on trial omissions or choice latency when acquiring the PRL task but slightly increased trial omissions following a change in reward contingencies and slightly increased choice latencies when animals were well-trained on the PRL task. In terms of decision making tasks, acute stress increased trial omissions but had no effect on choice latency during probabilistic discounting, whereas neither omissions nor latency was altered by acute stress in the Blackjack task. In previous experiments, our lab has found that acute stress increased choice latencies when the choice was between a smaller and larger reward with or without differential effort costs (Bryce & Floresco, 2016; Shafiei et al., 2012). Juxtaposition of these findings suggest that that acute stress does not uniformly increase decision latencies, but rather, the ability 176 of acute stress to increase ‘indecision’ depends on the type of cost being processed and that decisions involving physical effort costs may be particularly susceptible to stress. 6.23 Sex differences in cognitive flexibility, feedback sensitivity, and motivation Within certain cognitive domains, including cognitive flexibility, previous studies have presented sex differences in baseline and stress-induced cognition. However, in Chapter 2 we found minimal sex differences on a PRL task. First, at baseline there were no sex differences in flexibility in animals exposed to central CRF infusion in either training protocol. Conversely, in animals that were subjected to acute stress, females committed fewer errors overall than males during acquisition. However, this appears to be driven by a stress-induced impairment in males, with a null effect of stress in females, and no difference at baseline. This is consistent with previous cross-species work that finds acute stress impairs cognitive flexibility in males but not females (Laredo et al., 2015; Shields et al., 2016). Perplexingly, we found an overall impairment in females as compared to males, with an increased error rate, when animals were extensively trained, and reduced sensitivity to feedback, on both versions of the PRL task in the stress condition. The reason for the former effect is unclear, but appears to be a sex difference in training on the PRL task. That is, females completed fewer reversals over the course of initial training, and although there was no sex difference in the number of reversals at testing, we did find that the number of errors committed during testing was higher in females relative to males. One possible explanation for the difference in reinforcement sensitivity between CRF and stress groups is that animals in the CRF group were subjected to intracranial surgery shortly prior to testing and the stress of recovery may have altered long-term stress-related processes differentially for females compared to males that subsequently interfered with sensitivity to reinforcement. Indeed, numerous animal studies find that females are more sensitive to stress 177 than males, increasing ACTH and CORT levels with identical stressors (Babb et al., 2013; Iwasaki-Sekino et al., 2009). That said, perhaps a more parsimonious explanation is that the groups of females in the CRF and stress group were from different litters with perhaps dissimilar experiences prior to arriving at our facility and these differences between studies are an artifact of between group differences not easily accounted for in the current context. Most notably, we found that females were more ‘indecisive’ and more likely to disengage with the PRL task at baseline, appearing to be less motivated to perform the task than males. Moreover, females were more sensitive to the changing nature of the task, omitting more trials when reward contingencies switched compared to males. Indeed, females take more time to respond and omit more trials using various cognitive tasks that tax attention or require animals to hold probabilistic action-outcome contingencies in mind (Bayless et al., 2012; Orsini et al., 2016). A possible explanation for higher task disengagement may be due to hypersensitivity to punishment in females relative to males (Grissom & Reyes, 2019; Orsini et al., 2016), although this is not likely given that the only ‘punishment’ used in the present study was reward omission. Finally, it is notable that estrous cycle stage did not alter motivation or flexibility in females. These null effects may add to a growing literature indicating that cycling ovarian hormones have fewer effects on cognition than originally anticipated (Conrad et al., 2004; Epting, 1998; Stackman et al., 1997). However, as mentioned in Chapter 2, females were food restricted and not cycling normally, with most females in low ovarian hormone stages of their cycle. Therefore, potential alterations due to cyclicity may not be evident using the current methods. Future studies should aim to specifically address how estrous cycle stage alters cognitive flexibility and motivation. 178 In addition to baseline sex differences, we found few stress or CRF-induced alterations based on sex. First, we found a differential effect of sex on motivation depending on the manipulation. Specifically, motivational impairments were driven primarily by males following central CRF infusion, increasing choice latency and omissions. In contrast, impaired motivation was driven primarily by females following acute stress, increasing choice latency and number of omissions. Moreover, CRF consistently reduced reward and negative feedback in males, whereas CRF only reduced negative feedback in females with extensive training. Conversely, in well-trained animals acute stress reduced overall sensitivity to feedback in females but not males. Although CRF and acute stress showed minor differences in terms of sex and task experiences, the direction of effects is the same. That is, both CRF and stress reduced reward and negative feedback sensitivity, in keeping with previous findings (Bell et al., 1998; Bogdan & Pizzagalli, 2006; Bryce & Floresco, 2016; Cador et al., 1992; Glowa et al., 1992; Heinrichs et al., 1991; Kumar et al., 2014; Lemos et al., 2012; Petzold et al., 2010; Porcelli et al., 2012; Wanat et al., 2013). CRF-induced increases in omissions and latencies is driven, almost exclusively, by an increase in males, with little effect in females. This may add an important caveat to the literature on the motivational impairments of CRF in that it may only be true in males. However, females displayed higher baseline omission rates and latencies compared to males, which may have occluded potential effects of CRF manipulations on these measures of motivation. Why males are more impacted by CRF and females by acute stress is currently unknown. As stated in Chapter 2, sex differences may be due to differential activation of brain regions following CRF infusion, with activation more prevalent in the periaqueductal grey and BNST in females, whereas activation in the mPFC and amygdala are more predominant in males (Salvatore et al., 2018). Given that these tasks involve activation of the mPFC (Dalton et al., 179 2016), differential activation of this region in males and females may lead to distinct CRF-induced behaviours. Conversely, females are more sensitive to the effects of acute stress than males, with increased ACTH and CORT following acute stress in females compared to males (Babb et al., 2013; Iwasaki-Sekino et al., 2009). Thus, acute stress-induced potentiation of circulating CORT levels in females compared to males may independently alter some aspects of motivation and cognition. Future research should explore these possibilities. 6.3 Implications of central CRF signaling on VTA dopamine physiology and consequences of increased mesolimbic dopamine activity on effort choice Among the various types of costs (e.g. delays, uncertainty) that a decision-maker may consider in pursuit of different rewards, decisions involving choices between easily obtained but less valuable rewards vs. more valuable ones requiring greater effort to obtain appear to be particularly sensitive to increases in CRF neurotransmission. We have previously shown that central CRF infusion markedly shifts choice bias in favor of smaller rewards delivered after a single press and away from larger rewards obtainable after multiple lever presses in an effort discounting task (Bryce & Floresco, 2016). Similarly, one-hour of restraint stress induces a similar effect on effort choice (Shafiei et al., 2012). Yet this effect is not mediated by glucocorticoids, as CORT treatment does not recapitulate the effects of stress on this form of decision-making (Shafiei et al., 2012). Instead, we found that non-specific CRF antagonist administration prevents the effects of restraint stress on effort choice, demonstrating that stress-related increases in CRF diminish preference for larger rewards associated with a greater effort cost (Bryce & Floresco, 2016). Notably, neither increased CRF neurotransmission nor restraint stress has any effect on choice of larger vs. smaller rewards of equal costs. Thus, rather than reducing the incentive value of larger rewards or altering discrimination between rewards of 180 differing value, increased CRF activity may amplify the perceived effort costs required to obtain them. Consistent with this idea, central CRF infusion also reduced responding for rewards delivered on a progressive ratio schedule of reinforcement (Bryce & Floresco, 2016; Wanat et al., 2013). The ability to overcome effort costs to purse more valuable rewards is critically dependent on mesolimbic dopamine signaling, particularly in the NAc. As such, dopamine depletion or dopamine receptor blockade in the NAc reduced choice of larger, more costly rewards in a manner similar to increased CRF activity (Cousins & Salamone, 1994; Cousins et al., 1993; Floresco et al., 2008; Nowend et al., 2001; Salamone et al., 1991, 1994). Therefore, it is plausible that CRF reduces motivation to pursue more costly rewards via actions on dopamine activity. Indeed, anatomical observations validate interactions between the CRF and dopamine systems (Kelly & Fudge, 2018). However, bath applied CRF on VTA slices alters dopamine activity in a complex manner, altering excitatory and inhibitory activity, as detailed in Chapter 1 (Beckstead et al., 2009; Korotkova et al., 2006; Ungless et al., 2003; Williams et al., 2014). Moreover, CRF altered different modes of dopamine release, with microdialysis studies demonstrating that central CRF treatment increased tonic dopamine release in the NAc (Dunn & Berridge, 1987; Kalivas & Duffy, 1995; Matsuzaki et al., 1989) and CRF within the NAc facilitated local dopamine transmission (Lemos et al., 2012), whereas intra-VTA CRF reduced phasic NAc dopamine release evoked by stimulation of PPTg inputs (Wanat et al., 2013). Given the complex and somewhat opposing manners by which CRF influences dopamine activity, whether and how CRF modulates dopamine transmission to reduce effort-related choice remains an intriguing question. On the surface, the manner in which CRF reduced effort-related choice and motivated responding suggests that these effects are driven by reductions in 181 dopamine tone (Bryce & Floresco, 2016; Floresco et al., 2008; Salamone et al., 1991). However, excessive increases in dopamine activity can also reduce preference for larger rewards associated with a greater effort cost. Specifically, systemic administration of higher doses of amphetamine, which increased dopamine tone, also shifts preference away from larger rewards during effort discounting in a similar manner to enhanced CRF signaling (Floresco et al., 2008). Likewise, overexpression of striatal D2 receptors reduced effort choice (Filla et al., 2018). Therefore, it is plausible that reductions in motivated responding induced by excessive CRF activity are driven by increased mesoaccumbal dopamine levels. A prudent first step in understanding how increased CRF neurotransmission may alter behaviour via interactions with dopamine signaling is to clarify how central CRF infusion modulates VTA dopamine neuron activity. Chapter 4 of this Thesis addresses this question, finding that higher CRF doses that alter effort choice increase spontaneously active dopamine neuron activity, while lower CRF doses that do not alter effort choice have no effect on spontaneous dopamine activity. It is plausible that increased spontaneous activity that enhances extrasynaptic dopamine levels within the NAc (Floresco et al., 2003) mediate the reduction in effort choice following exogeneous CRF administration. In support of this hypothesis, Chapter 5 found that increasing activity at the D2 receptor in both the NAc core and shell reduced effort choice in a strikingly similar manner to central CRF treatment. When taking the full behavioural pattern into account, alterations induced by central CRF administration and NAc D2 receptor agonist administration are remarkably similar. Notably, both CRF hyperactivity and NAc D2 receptor stimulation slow decision latencies and lead to disengagement by increasing the number of trials omitted (although the small number of animals in the NAc core D2 receptor agonist infusion group meant that this effect was only trending 182 toward increased omissions). This is unsurprising, considering that dopamine appears to mediate the effect of acute stress on decision latencies, as acute stress increased deliberation times, which was blocked by systemic administration of a dopamine receptor antagonist prior to stress exposure (Shafiei et al., 2012). These effects are not exclusive to effort discounting, as central CRF infusion and activation of D2 receptors in the PFC or BLA increase response latencies during other types of cost/benefit decision making or attention tasks (Larkin et al., 2016; St Onge et al., 2011). As Chapter 4 details, central CRF at the same dose that increased the number of omissions and choice latency, enhanced the number of spontaneously active VTA dopamine neurons, which would increase tonic dopamine signaling in downstream regions (Floresco et al., 2003). Therefore, increased extrasynaptic dopamine in terminal regions may slow decision latencies, occasionally to such a degree that trials are omitted. When viewed collectively with choice data, we speculate that CRF hyperactivity increases tonic dopamine activity in the VTA, which enhances extrasynaptic dopamine tone in downstream terminal regions, including the NAc, acting on D2 receptors within this nucleus to bias choice away from choice options associated with more physical effort, slow decision latencies, and heighten task disengagement. The ability of CRF to alter behaviour depends on the location of action within the mesolimbic dopamine system. For instance, we previously found that CRF infusion directly into the VTA shifted choice away from larger, more costly rewards in an effort discounting task, similar to central CRF infusion, while other studies found that intra-VTA CRF reduced instrumental responding on a progressive ratio (Bryce & Floresco, 2016; Wanat et al., 2013). However, enhancing CRF activity within the VTA does not completely recapitulate the behavioural effects of central CRF treatment, as the latter increased decision latencies and trial omissions, whereas the former does neither. One reason for the discrepancy in choice and 183 motivational components of this task may be that central CRF and intra-VTA CRF act on different mechanisms of the mesolimbic dopamine response. For instance, central CRF may reduce effort choice and increase decision latencies and trial omissions by increasing spontaneous dopamine neuron activity within the VTA which would result in potentiated tonic dopamine signaling. Although phasic dopamine activity was not altered with central CRF infusion, CRF within the VTA may reduce effort choice via phasic dopamine alterations as this manipulation dampens phasic dopamine release in the NAc in response to reward, which reduces instrumental responding on a progressive ratio task (Wanat et al., 2013). Moreover, central CRF infusion increased the firing rate of dopamine neurons, perhaps via direct action in the VTA, as bath applied CRF increased the firing rate of VTA dopamine neurons (Korotkova et al., 2006). Therefore, central and intra-VTA CRF infusion may alter effort choice by increasing the firing rate of VTA dopamine neurons and/or dampening phasic NAc dopamine release, whereas motivational perturbations such as latency and omissions may be perturbed by increasing tonic dopamine activity. Although alterations in tonic dopamine activity in response to intra-VTA CRF infusion have yet to be investigated, differential behavioural profiles in the effort-related decision making task following central vs intra-VTA CRF treatment may be driven by actions that alter tonic vs phasic dopamine activity. CRF within mesolimbic terminal regions shift effort choice in a distinct manner. As Chapter 5 illustrates, CRF infusions in the NAc core make rats less sensitive to changes in effort costs and induce a more static pattern of choice, with rats preferring the larger reward less when costs are low and preferring this option more when effort costs are high. As mentioned above, this may be related to the ability of CRF in the NAc to enhance mesoaccumbens dopamine release (Lemos et al., 2012). Thus, potentiated dopamine tone via CRF infusions may interfere 184 with signals pertaining to reward saliency (Redgrave et al., 1999) or changes in the amount of rewards received over time (St Onge et al., 2012). Although speculative, it is possible that CRF-induced potentiation of NAc dopamine release may have blunted the ability of rats to integrate information about low vs high effort costs associated with larger rewards that is used to bias choice towards or away from that option. However, the fact that intra-NAc CRF did not recapitulate the effects of any dopamine receptor agonist suggests that these effects may be due to a combination of effects on dopamine transmission, post-synaptic CRF receptors, and other neurotransmitters such as acetylcholine (Chen et al., 2012). In contrast to the NAc core, CRF in the NAc shell had no effect on any component of the effort discounting task. Together, these observations serve to highlight the circuit-specific manner in which CRF modulates dopamine-dependent motivation and decision-making functions in that increasing CRF activity in certain nodes of these circuits may induce differential effects on these behaviours compared to acute stress or global increases in CRF. 6.4 Experimental limitations and future directions While the current series of experiment primarily aimed to shed light on how acute stress and the stress-related neuropeptide CRF perturb cognitive functions including flexibility and decision making, with a secondary aim to understand interactions between the CRF and dopamine systems on decision making, a number of methodological issues should be considered. First, it is unclear if one hour of restraint stress is similar in magnitude to either low (1g) or high (3g) dose CRF used in our experiments. This is due to the difficulty in measuring large peptides, such as CRF, in microdialysis probes, especially in the ventricular system of behaving animals. One group has measured CRF in tissue via in vivo microdialysis in response to social stress and found that during acute social defeat stress, CRF levels in the posterior, but not 185 anterior VTA, are potentiated compared to baseline levels (Holly et al., 2016). Another way to assess CRF levels, albeit indirectly, is by measuring immunoreactivity to the CRF peptide or mRNA precursor ex vivo. Using in-situ immunoreactivity, previous studies have found that acute stress increased CRF-immunoreactivity in various brain regions, including the PVN, mPFC, CeA, BNST, and VTA (Chappell et al., 1986; Imaki et al., 1991; Mamalaki et al., 1992; Wang et al., 2005). However, as it concerns the present experiments, increased CRF-immunoreactivity does not mean that CRF infusion into the lateral ventricular system at the doses currently used would be similar in nature or magnitude to endogenous CRF released in response to acute stress. We can only assess how these manipulations affect behaviour and are, therefore, unable to conclude that the CRF doses used in the present experiment are similar to what would endogenously be released in response to one-hour of acute restraint stress. Although we did not explicitly test CORT levels for either manipulation, assessing how both central CRF infusion at the relevant doses and one-hour of restraint stress alter peripheral CORT levels may offer insight into the relative magnitude of CRF infusion and restraint stress. Our group found that one-hour restraint stress performed identically to the present experiments in male rats increased plasma CORT levels to ~300ng/ml at 30 mins and ~500ng/ml at 60 mins following release from restraint, which would coincide with behaviour in the timeline of our PRL tasks (Shafiei et al., 2012). Another study found that ICV infusion of 1 or 10g CRF in male rats increased plasma CORT levels with a similar magnitude of ~500ng/ml at approximately 40 mins after infusion (Campbell et al., 2004). Therefore, although these stress manipulations were not directly compared, these results provide some evidence that central CRF infusion at the doses tested and one-hour of restraint stress are at least somewhat comparable in terms of HPA axis activation and subsequent CORT release at the time of behavioural testing. 186 Moreover, we posit that alterations in behaviour following stress manipulations in these tasks were not due to increased CORT levels. One reason is due to timing. That is, CRF and CORT mediate fast vs slow reactions to stress, respectively. The time course is such that the stress signal is initiated in the brain and released into the bloodstream, where it acts on the adrenal glands to produce CORT, which then cycles back to the brain to influence neural function and as part of a negative feedback loop. This circuitous route is a lengthier process than the duration of our behavioural tasks which are all under an hour in length. Therefore, we posit that the majority of behavioural effects we saw in these tasks were not due to stress-induced alterations in CORT, but rather mediated by more rapid stress factors that are expressed in the brain and can be activated within minutes, such as CRF. Secondly, our group previously found that central CRF but not CORT activity mediated the acute restraint stress-induced reduction in choice of the larger reward requiring more effort. Specifically, treatment with a CRF antagonist attenuated acute stress effects on effort choice in an effort discounting task (Bryce & Floresco, 2016; Shafiei et al., 2012). In contrast, exogeneous CORT administration at doses higher than endogenously circulating plasma CORT levels following one-hour restraint stress had no effect on effort choice (Shafiei et al., 2012), whereas central CRF infusion at the high-dose used in the current PRL experiments, mimicked the effects of acute stress, reducing effort choice and increasing deliberation times. Thirdly, CRF mediates many of the fast-acting behavioural effects of stress, independent of HPA axis activation (Cador et al., 1992; Dunn & Berridge, 1990) and, as detailed above, both central CRF infusion and one-hour restraint stress activate the HPA axis to a similar degree (Campbell et al., 2004; Shafiei et al., 2012). With all of this in mind, we suggest that the effects on PRL and decision making following central CRF and acute stress 187 manipulations in the current series of experiments are due to rapid-acting mechanisms within the brain, including increased CRF activity and interactions between CRF and dopamine systems. As we posit that CORT does not alter behaviour in these tasks in males, the fact that females show higher baseline and stress-induced plasma CORT levels (Handa et al., 1994; Iwasaki-Sekino et al., 2009), lessens the concern for stress manipulations differentially altering behaviour in males and females due to differential CORT activation. Moreover, females differ in plasma CORT levels depending on estrous cycle phase, with females in higher ovarian hormone phases showing increased CORT response compared to lower ovarian hormone phases (Carey et al., 1995). Given that females in our study were food restricted and thus primarily in low ovarian hormone phases of the estrous cycle at testing, one would expect that sex differences in baseline and stress-induced CORT response to be more limited than if females were cycling normally. As we only included high-dose CRF in the PRL study, any potential effects of CRF at lower doses that may have had more nuanced effects on behaviour in males and females may have been obscured. The CRF dose we chose presently was based primarily on our previous finding that high-dose CRF (3g) caused similar alterations to one-hour restraint stress during effort-related decision making. We included a lower dose in the risk/reward decision making tasks to test for any potential dose effects and found, again, that alterations were primarily driven by the higher dose (3g). However, we did not formally include the lower dose when assessing cognitive flexibility in the PRL task for a few reasons. Namely, we conducted a pilot study on the first squad of males tested on the PRL performance version of the task and found no effects of the lower CRF dose (1g). Additionally, because animals were tested during their first experience of the PRL task, this experiment required a between-subjects analysis. As such, we tested 8 groups in the PRL acquisition version alone, with manipulations of acute stress (no 188 stress and restraint stress groups) and CRF (vehicle and CRF infusion), required in both sexes. Adding an additional 4 groups to test a lower dose did not appear prudent given previous null findings. That said, the CRF doses we chose are on the high end of what previous experiments have used. In our view this is a positive, as we found effects at high CRF doses that may be similar to more chronic stress-like behaviours. However, we may have missed more subtle effects found at very low CRF doses. For instance, one study found ICV CRF (0.030-0.300g) impaired reversal learning and set-shifting with deterministic outcomes in male rats (Snyder et al., 2011). Furthermore, we treated both males and females with the same high CRF dose. However, females are more sensitive to some of the effects of CRF than males (Bangasser et al., 2010; Wiersielis et al., 2016). That said, males and females show similar dose-dependent impairments in a sustained attention task when treated with low CRF doses (0.1, 0.5, or 1g) (Cole et al., 2016), suggesting that at least on some cognitive tasks, males and females show similar behavioural alterations to the same central CRF dose. Although we are unable to exclude the possibility that these extremely low doses, particularly in females, may have altered behavioural measures on the PRL task, the dose we chose was based on previous work by our group on effort discounting in male rats and presently clarified the role of high-dose CRF on cognition and how it may relate to more severe or chronic stress manipulations. However, future experiments should investigate possible dose-dependent CRF effects on behaviour in male and female rats, with the potential for differential CRF effects at lower doses, particularly in females. Finally, CRF infusion and acute stress may produce opposing effects due to differences in time course. That is, we infused CRF into the lateral ventricle 10 mins prior to behavioural testing, which would presumably act on neurons within a matter of minutes. Indeed, 1g CRF infusion into the lateral ventricle stimulated c-Fos immunoreactivity in widespread brain regions 189 as early as 30 mins post administration (earliest time point tested) (Bittencourt & Sawchenko, 2000). On the other hand, plasma CORT levels peak within 30 mins of stress onset (Livezey, Miller, & Vogel, 1985); however, this is a peripheral measurement. Acute restraint stress would be expected to result in a slower, endogenous, accumulation of CRF and other stress mediators within brain regions as compared to direct CRF infusion. Therefore, we are unable to exclude the possibility that differences in acute stress and central CRF infusion may be due to differences in time course. Future studies should directly compare CRF expression following exogeneous central CRF infusion and one-hour acute restraint stress over time to better understand these dynamics. Although central infusion is a good first-pass in understanding how enhancing central CRF signaling can alter cognitive processes, this method does not tell us where or how CRF may be acting to elicit such perturbations. Our previous studies found that CRF within the VTA may mediate some actions, such as effort choice. However, intra-VTA and intra-NAc CRF did not completely recapitulate central CRF effects in the effort discounting task, indicating that CRF may act outside of the VTA or NAc to induce the consistent central CRF-induced motivational impairments. Indeed, it is possible that CRF is acting outside of the VTA to enhance tonic dopamine signaling, subsequently altering motivation and choice. Future studies should assess the regional specificity of CRF effects, with possibilities focused on the neural circuitry known to be involved in shifting tonic dopamine activity including the mPFC, BLA, and hippocampus (Grace, 2016; Lodge & Grace, 2006; Patton et al., 2013; Zimmerman & Grace, 2016). Furthermore, central CRF infusion enhances tonic dopamine activity, which we posit would subsequently increase activity at NAc D2 receptors to reduce effort choice. This possibility should be specifically tested, perhaps by infusing CRF centrally while blocking D2 receptors in 190 the NAc. However, this may not tell us much considering that NAc D2 receptor antagonist administration independently reduces effort choice (Nowend et al., 2001). Additionally, stimulation of D2 receptors using exogeneous agonist administration does not alter activity in the same manner as endogenous mesolimbic dopamine release. Indeed, increasing mesolimbic dopamine release via central CRF infusion would act on all dopamine receptors present, with varying affinity, not exclusive to D2 receptors. That said, technology is currently unable to separately manipulate tonic dopamine activity without affecting phasic activity as well. Therefore, the present study points to where dopamine may be primarily acting to facilitate behavioural effects, but prospective studies should attempt to understand the differential role of tonic vs phasic endogenous dopamine signaling within the mesolimbic circuitry and how this may impact behavioural outcomes. CRF infusion into the lateral ventricle does not give us good spatial-, temporal-, or receptor-specificity. As explained in detail in Chapter 1, CRF acts on two receptors. Current studies do not indicate which receptor is primarily involved in mediating alterations in motivation and choice. Therefore, future studies using pre-treatment with a CRF1R or CRF2R antagonist should be conducted to ascertain the pharmacological effects on choice, motivation, and physiological effects. Moreover, recent advancement of genetic techniques, including optogenetics and chemogenetics, could aid in probing CRF effects with spatial and temporal specificity. However, these manipulations are more difficult to implement for researchers wanting to manipulate large peptides such as CRF. This is for reasons inherent to how CRF is expressed and released. First, stimulating CRF-expressing neurons with light releases small-molecule transmitters with a 1:1 ratio, whereas large-molecule peptide release will only occur with repeated stimulation. Second, CRF peptide is typically co-expressed with other 191 neurotransmitters such as GABA and glutamate and other peptides such as vasopressin and somatostatin (Gallopin et al., 2006; Grieder et al., 2014; Habib et al., 2001; Kubota et al., 2011; Tagliaferro & Morales, 2008). Therefore, experiments that stimulate CRF-expressing neurons should use CRF receptor antagonists to ensure that effects seen are due to CRF and not to neurotransmitter activity. Using this approach, one study found that, similar to exogenous intra-PFC CRF infusion, chemogenetically-stimulating CRF-expressing PFC neurons impaired working memory, which was blocked by intra-PFC CRF antagonist administration, indicating that impairments are due to enhanced local CRF activity (Hupalo et al., 2019). Future studies should use a similar chemo- or opto-genetic approach to probe how increasing CRF signaling within neural circuitries alters mesocorticolimbic physiology and cognition, with a particular focus on decision making involving effort or uncertainty. Finally, we only included females in one chapter of experiments. Although we did not explicitly set out to test sex differences in our experiments, the addition of females in the PRL study was an added layer of analysis that was warranted in assessing cognitive flexibility given the divergent effects of stress on this measure in males and females in prior studies (Laredo et al., 2015; Shields et al., 2016). Our current findings indicated minimal sex differences in stress- and CRF- induced alterations in flexibility, with the majority of effects found in baseline motivational differences between males and females. Given that the main impetus of these studies was to extend previous findings on acute stress and CRF effects on effort-related decision making in males, we chose to use exclusively males for the remainder of our studies. This lack of female inclusion should not suggest that we are arguing that males should be used as the standard in all experiments and females should be relegated to special interest experiments on sex differences. Indeed, the concern that females are difficult to study given potential estrous cycle- 192 dependent effects has been overstated, particularly when using food restriction which reduces cycling in females, and the fact that male hormones also cycle, but are rarely measured. As we used exclusively males in the latter chapters, we can only speculate as to how females may have been affected in the risk/reward decision making tasks and in terms of dopamine neuron physiology. Females may be less motivated to perform on the risk/reward decision making tasks and, as such, show baseline increases in omissions and choice latencies similar to what we found in the PRL tasks. Yet, there is a theoretical basis underpinning the idea that females would perform in similar manner to males on risk/reward decision making tasks, as sex differences in risky choice are largely absent in tasks using reward omission (Grissom & Reyes, 2019). Although stress manipulations may alter risky choice, evidence indicates that if males and females differ on stress-induced alterations in risky choice that this may be in terms of degree and not direction (Grissom & Reyes, 2019). Additionally, we posit that females may be less likely to choose the more effortful option in the effort-related decision making task due to lessened motivation in our hands and previous work in humans, with females omitting more trials, particularly when the task gets more difficult, and choosing the larger, more effortful reward less often as compared to males (Treadway et al., 2009). Indeed, previous studies from our group have shown that females tend to choose the high effort option less frequently compared male rats (based on archival data), although interestingly, this effect dissipated after ovariectomy (Uban et al., 2012). That said, we again speculate that neither acute stress nor CRF infusion would differentially alter choice and that various intra-NAc dopamine receptor agonist administration would most likely not differentially alter choice in males and females on the effort-related decision making task. 193 Research regarding the influence of stress on dopamine neuron physiology is relatively scant. One study finds that, although there were no sex differences in dopamine neuron activity at baseline, chronic mild stress reduced dopamine neuron population activity in both sexes; however, the effect was more pronounced in females compared to males (~50% vs ~75% reduction) (Rincón-Cortés & Grace, 2017). Given the similar directionality in males and females, we postulate that central CRF may also increase dopamine neuron population activity in females but to a greater extent in females as compared to males. Moving forward, however, studies should include both sexes, even if there is not a clear theoretical argument as to whether or not they may be differentially impacted; one could either find sex differences, which is notable, or increase their sample size. Taken collectively, the addition of females in Chapter 2 extends the literature on stress-induced impairments in cognitive flexibility in males, but not females, to include cognitive flexibility with probabilistic outcomes. That said, future work should assess how females may be affected by stress and CRF on decision making and dopamine neuron physiology. 6.5 Relevance to depression and other stress-related disorders Depression is a complex disorder characterized by a constellation of affective, motivational, and cognitive symptoms. Traditionally thought of as a mood disorder, depression involves debilitating sadness coupled with an inability to derive pleasure from ordinarily pleasurable experiences. Although low mood is the hallmark symptom of the disorder, cognitive deficits are particularly devastating as impairments in cognitive flexibility, executive functioning and decision making render the sufferer unable to reverse the cycle of negative thought and behavioural patterns. Moreover, motivational impairments are all-encompassing and prevent sufferers from taking the steps necessary to get help. Traditional anti-depressants are tested and 194 approved based on their ability to improve mood. Therefore, current treatments for depression do not address cognitive or motivational perturbations of the disorder. Moreover, traditional antidepressants are ineffective for a large minority of patients, even when the focus is exclusively on treating mood symptoms. Perhaps the ineffectiveness of treatment options is due to the heterogeneity of the disorder. More recent classifications indicate that depression may be the umbrella term for similar symptom clusters (Gold, 2015; Tylee et al., 1999). As such, preclinical work should move away from ‘modeling’ complex human disorders with heterogenous phenotypes, and instead identify symptoms common across many related disorders. Hyperactive stress responses are shared by numerous stress-related disorders, with post-traumatic stress disorder, obsessive compulsive disorder, schizophrenia, bipolar disorder, and depression showing enhanced CRF activity (Altemus et al., 1992; Banki et al., 1987, 1992; Bremner et al., 1997; Nemeroff et al., 1984). Additionally, these disorders also share aberrant cognitive processing (Bangasser & Kawasumi, 2015; Gruner & Pittenger, 2017; Lee & Park, 2005). Therefore, identifying the potential contributions of CRF hyperactivity in altering behaviour within specific cognitive domains known to be altered in depression and other stress-related disorders may prove more fruitful in treating these cognitive impairments across traditionally siloed disorders. Although our main focus was to better understand how acute stress and CRF alter cognition from a basic science perspective, it was our secondary aim to compare these effects to those found in stress-related disorders, particularly depression. When comparing cognitive flexibility, previous work in humans found that individuals with depression are impaired on cognitive flexibility tasks using the WCST or set-shifting tasks (Austin et al., 2001; Ilonen & Leinonen, 2000; Lee et al., 2012; Reppermund et al., 2009). However, using a similar PRL task 195 to the one used presently, individuals with depression show no overall impairment (Dombrovski et al., 2010; Murphy et al., 2003; Taylor Tavares et al., 2008), a trend toward an impairment on a subsample of young participants (Dickstein et al., 2010), or a significant impairment in a subset of patients who have attempted suicide (Dombrovski et al., 2010). That said, in the human PRL version initial discrimination and reversal stages occur in the same session and so either learning or flexibility may be altered. Our current results demonstrate that when animals were exposed to increased CRF signaling prior to PRL task acquisition, we found no marked alterations in flexibility. Although the role of chronic stress on PRL has yet to be investigated, prolonged or chronic stress impairs cognitive flexibility on deterministic reversal learning and set-shifting tasks (Bondi et al., 2010; Bondi et al., 2008; Hurtubise & Howland, 2016; Jett et al., 2015; Jett et al., 2017; Lapiz-Bluhm et al., 2009). Taken together, the majority of studies find no marked impairments in cognitive flexibility using PRL tasks either in individuals with depression or with increased CRF signaling in rodents. Therefore, increased CRF signaling is consistent with chronic stress and depression on cognitive flexibility tasks in which the outcomes are deterministic. However, it is difficult conclude if this holds true for tasks employing probabilistic outcomes given the discrepant findings and myriad explanations as to why both CRF and depressed populations may have resulted in null effects. Those suffering with depression are particularly susceptible to dysregulation of decision making processes when deciding between various response options associated with differential effort costs. Indeed, individuals with depression suffer from a multitude of energy-related deficiencies, including a substantial lack of motivation, in which they are unwilling or unable to expend effort in exchange for pleasurable experiences (Salamone et al., 2015; Stahl, 2002; Tylee et al., 1999). Mirroring increased CRF signaling in rodents (Bryce & Floresco, 2016), depressed 196 patients tested on a back-translated version of the effort-related decision making task used in the present study demonstrate a pronounced reduction in preference for larger rewards associated with greater effort costs (Treadway et al., 2012). Notably, this effect has been recently extended to sufferers of bipolar disorder and schizophrenia (Zou et al., 2019), disorders which are also characterized by CRF hyperactivity (Banki et al., 1987, 1992; Nemeroff et al., 1984), perhaps pointing to a shared etiology and underscoring the need for treating symptom clusters across disorders. In regards to decision making involving uncertainty, we revealed that CRF specifically altered optimal risky choice, reducing preference for the larger reward when auditory cues indicated that this option was advantageous similar to chronic stress manipulations (Morgado et al., 2015). Notably, this option was also associated with the most uncertainty as the probability of reward was 50 percent. Impairments in optimal risky decision making and intolerance for uncertainty are also found in individuals with depression (Cella et al., 2010; Clark et al., 2011; Han et al., 2012; Taylor Tavares et al., 2007). Collectively, these results point to a major role for increased CRF activity in mediating alterations in decision making involving effort and uncertainty seen in depression, with CRF reducing the willingness to expend effort for more preferred rewards and impairing optimal choice involving uncertainty. Additionally, using PRL and risk/reward decision making tasks, we found alterations in sensitivity to feedback following CRF infusion. More specifically, enhanced CRF activity reduced reward and negative feedback sensitivity. Feedback hyposensitivity is consistent with reductions in reward sensitivity (Beats et al., 1996; Bogdan & Pizzagalli, 2006; Chase et al., 2010; Eshel & Roiser, 2010; Henriques & Davidson, 2000; Henriques et al., 1994; Holmes & Pizzagalli, 2007; Pizzagalli et al., 2005, Pizzagalli et al., 2009) and negative feedback sensitivity 197 (Beats et al., 1996; Petzold et al., 2010) in individuals with depression, diminishing the ability to utilize both positive reinforcement to guide learning (Chase et al., 2010; Holmes & Pizzagalli, 2007), perhaps by blunting neural responses to both reward and loss (Treadway et al., 2013; Ubl et al., 2014). Therefore, potentiated CRF signaling may drive depression-related deficits in feedback processing. Finally, we found that increased CRF signaling impaired motivation across all tasks. Notably, high dose CRF caused task disengagement, increasing the number of trials omitted during tasks of cognitive flexibility and decision making. This holds true whether the task involved uncertainty, like the PRL and probabilistic discounting tasks, or whether the task involved galvanizing physical effort, like in the effort discounting task (Bryce & Floresco, 2016). Disengagement tends to occur when the rules of the task change, when encountering an obstacle like a switch in reward contingencies, or with an accumulation of negative feedback when the task involves uncertainty. Notably, individuals with depression show a ‘catastrophic’ response to errors, with depressed subjects substantially more likely to commit subsequent errors given an error on the previous trial (Beats et al., 1996; Elliott et al., 1997). Moreover, increased CRF activity slows processing speed and leads to ‘indecisiveness’ by increasing response time in tests of cognitive flexibility, decision making and attention (Beard et al., 2015; Bryce & Floresco, 2016; Van’t Veer et al., 2012). This is in agreement with increased decision latencies on various tests of executive functioning and psychomotor slowing found in individuals with depression (Murphy et al., 2001; Lemke et al., 1999; Schrijvers et al., 2008; Sobin & Sackeim, 1997). Given that abnormal increases in central CRF activity have been implicated in the pathophysiology of depression (Nemeroff et al, 1984; Banki et al, 1987; Hauger et al, 2009; Binder et al, 2009) the 198 present findings provide support for the notion that hyperactive CRF transmission may be a central force underlying the motivational impairments in depression (Grahek et al., 2019). Although it is plausible that acutely increasing CRF transmission alters aspects of cognition by activating the stress response and in doing so acts in a similar manner to acute stress, this possibility was not bared out in the current results. On PRL and risk/reward decision making tasks, acute stress and restraint stress altered flexibility and risky choice in opposing directions with restraint stress mirroring previous studies of acute stress (Butts et al., 2013; Koot et al., 2013; Laredo et al., 2015; Preston et al., 2007; Putman et al., 2010; Shields et al., 2015, 2016; Simonovic et al., 2017; Starcke et al., 2008; Wemm & Wulfert, 2017), whereas CRF hyperactivity paralleled previous studies of chronic stress or depression (Beats et al., 1996; Bogdan & Pizzagalli, 2006; Cella et al., 2010; Chase et al., 2010; Clark et al., 2011; Elliott et al., 1997; Eshel & Roiser, 2010; Han et al., 2012; Henriques & Davidson, 2000; Henriques et al., 1994; Holmes & Pizzagalli, 2007; Nobrega et al., 2016; Petzold et al., 2010; Pizzagalli et al., 2009, 2005; Taylor Tavares et al., 2007). Motivational impairments appear particularly sensitive to stress manipulations as both CRF and acute stress lead to aberrant motivation, albeit the latter more inconsistently. Notably, chronic, but not acute, stress increases CRF expression (Chappell et al., 1986; Imaki et al., 1991; Mamalaki et al., 1992) and genetic CRF overexpression mirrors some of the behavioural effects of chronic stress (Dirks et al., 2002). Therefore, a parsimonious interpretation is that exogenously increasing CRF signaling may lead to more chronic stress-like behavioural manifestations in some cognitive domains, with prolonged or sustained CRF activity leading to pathological outcomes. We should also note the potential implications for addiction, given the interactions between CRF and dopamine discussed at length here and the role of these mechanisms in drug addiction. 199 Animal models of addiction investigate drug craving by using reinstatement of drug seeking following self-administered drug exposure and subsequent extinction. Reinstatement can be initiated by exposure to the drug cue or by stress manipulations. CRF is argued to be the primary factor in the transition between early addiction states, which are characterized by positive reinforcement mechanisms related to drug-induced dopamine release, into later addiction stages, characterized by negative reinforcement mechanisms, in which individuals with addiction use drugs to reduce the adverse, anxiety-inducing symptoms associated with chronic drug abuse (Koob, 2008, 2010). Briefly, both central and regional CRF activity, particularly within the VTA, have been shown to mediate stress- and cue-induced reinstatement, and exogeneous CRF treatment induces reinstatement of drug reward to all drugs of abuse including cocaine, heroin, methamphetamine, and alcohol in animals models of addiction (Blacktop et al., 2011; Buffalari et al., 2012; Chen et al., 2014; Galesi et al., 2016; Koob, 2008; Mantsch et al., 2008; Moffett & Goeders, 2007; Shaham et al., 1997). Moreover, dopamine activity is principally involved in the etiology, maintenance, and relapse in addiction, with self-stimulation of drugs of abuse enhancing tonic dopamine signaling in mesocorticolimbic terminal regions (Mantsch et al., 2016; Shaham & Stewart, 1995). Indeed, CRF/dopamine mechanisms in reinstatement are linked, insofar as stress increases CRF signaling in the VTA, which increases local glutamate signaling on dopamine neurons (Wang et al., 2005), subsequently potentiating downstream dopamine release (Wise & Morales, 2010). Therefore, our current findings demonstrating that centrally-active CRF increases tonic dopamine neuron activity fits with these previous findings and suggests that CRF receptor antagonists may be an effective addiction treatment, although current clinical trials with CRF1R antagonists have shown mixed results (Contoreggi, Lee, & Chrousos, 2013; Spierling & Zorrilla, 2017; Zorrilla et al., 2013). 200 6.6 Conclusion Overall, the current results provide insight into how acute stress and CRF signaling modulate aspects of cognition, underscoring the differential roles of these manipulations on cognitive flexibility, feedback sensitivity, and decision making involving uncertainty. Furthermore, they point to a specific role for CRF in amotivational aspects of behaviour. When viewed collectively, it is striking to highlight that enhanced CRF activity acts more similarly to chronic stress manipulations and alterations found in sufferers of depression, particularly as it relates to optimal risky choice and feedback sensitivity, than to manipulations of acute stress, indicating that hyperactive CRF neurotransmission may play a key role in altering these cognitive domains in individuals with depression. However, in other domains acute stress and increased CRF activity similarly perturb behaviour, suggesting that motivational effects and effort choice are particularly susceptible to stress manipulations. Although further work is needed to clarify the specific circuitry, deficits in motivation and effort choice caused by CRF hyperactivity may be due increased tonic dopamine signaling, particularly via enhanced stimulation at the D2 receptor. Finally, this work highlights the need for better treatments for individuals suffering from stress-related disorders that try to address individual symptom profiles across disorders rather than attempting, and failing, to address the full spectrum of symptoms found within traditional diagnoses. 201 References Abercrombie, E. D., Keefe, K. A., DiFrischia, D. S., & Zigmond, M. J. (1989). Differential effect of stress on in vivo dopamine release in striatum, nucleus accumbens, and medial frontal cortex. Journal of Neurochemistry, 52(5), 1655–1658. https://doi.org/10.1111/j.1471-4159.1989.tb09224.x Aberman, J. E., & Salamone, J. D. (1999). Interference with accumbens dopamine transmission makes rats more sensitive to work requirements but does not impair primary food reinforcement. Behavioural Pharmacology, 92(2), 545–552. https://doi.org/10.1097/00008877-199908001-00202 Aguilera, G., & Liu, Y. (2012). The molecular physiology of CRH neurons. Frontiers in Neuroendocrinology, 33(1), 67–84. https://doi.org/10.1016/j.yfrne.2011.08.002 Al’absi, M., Nakajima, M., Hooker, S., Wittmers, L., & Cragin, T. (2012). Exposure to acute stress is associated with attenuated sweet taste. Psychophysiology, 49(1), 96–103. https://doi.org/10.1111/j.1469-8986.2011.01289.x Alexander, J. K., Hillier, A., Smith, R. M., Tivarus, M. E., & Beversdorf, D. Q. (2007). Beta-adrenergic modulation of cognitive flexibility during stress. Journal of Cognitive Neuroscience, 19(3), 468–478. https://doi.org/10.1162/jocn.2007.19.3.468 Altemus, M., Pigott, T., Kalogeras, K. T., Demitrack, M., Dubbert, B., Murphy, D. L., & Gold, P. W. (1992). Abnormalities in the regulation of vasopressin and corticotropin releasing factor secretion in obsessive-compulsive disorder. Archives of General Psychiatry, 49(1), 9–20. https://doi.org/10.1001/archpsyc.1992.01820010009002 Andreano, J. M., & Cahill, L. (2006). Glucocorticoid release and memory consolidation in men and women. Psychological Science, 17(6), 466–470. https://doi.org/10.1111/j.1467-9280.2006.01729.x Arnsten, A. F. T. (2009). Stress signalling pathways that impair prefrontal cortex structure and function. Nature Reviews. Neuroscience, 10(6), 410–422. https://doi.org/10.1038/nrn2648 Ashenhurst, J. R., Seaman, M., & David Jentsch, J. (2012). Responding in a test of decision-making under risk is under moderate genetic control in the rat. Alcoholism: Clinical and Experimental Research, 36(6), 417–425. https://doi.org/10.1111/j.1530-0277.2011.01701.x Austin, M., Mitchell, P., & Goodwin, G. M. (2001). Cognitive deficits in depression : Possible implications for functional neuropathology. British Journal of Psychiatry, 178, 200–206. https://doi.org/10.1192/bjp.178.3.200 Austin, M., Mitchell, P., Wilhelm, K., Parker, G., Hickie, I., Brodaty, H., … Milic, M. (1999). Cognitive function in depression : a distinct pattern of frontal impairment in melancholia ? Psychological Medicine, 29(May), 73–85. Babb, J. A., Masini, C. V., Day, H. E. W., & Campeau, S. (2013). Sex differences in activated 202 corticotropin-releasing factor neurons within stress-related neurocircuitry and hypothalamic-pituitary-adrenocortical axis hormones following restraint in rats. Neuroscience, 234, 40–52. https://doi.org/10.1016/j.neuroscience.2012.12.051 Baik, J.-H. (2013). Dopamine signaling in reward-related behaviors. Frontiers in Neural Circuits, 7(October), 152. https://doi.org/10.3389/fncir.2013.00152 Bangasser, D. A., Curtis, A., Reyes, B. A. S., Bethea, T. T., Parastatidis, I., Ischiropoulos, H., … Valentino, R. J. (2010). Sex differences in corticotropin-releasing factor receptor signaling and trafficking: potential role in female vulnerability to stress-related psychopathology. Molecular Psychiatry, 15(9), 877, 896–904. https://doi.org/10.1038/mp.2010.66 Bangasser, D. A., & Kawasumi, Y. (2015). Cognitive disruptions in stress-related psychiatric disorders: A role for corticotropin releasing factor (CRF). Hormones and Behavior, 76, 125–135. https://doi.org/10.1016/j.yhbeh.2015.04.003 Bangasser, D. A., & Valentino, R. J. (2014). Sex differences in stress-related psychiatric disorders: Neurobiological perspectives. Frontiers in Neuroendocrinology, 35(3), 303–319. https://doi.org/10.1016/j.yfrne.2014.03.008 Banki, C. M., Bissette, G., Arato, M., O’Connor, L., & Nemeroff, C. B. (1987). CSF corticotropin-releasing factor-like immunoreactivity in depression and schizophrenia. American Journal of Psychiatry, 7(144), 873–877. Banki, C. M., Karmacsi, L., Bissette, G., & Nemeroff, C. B. (1992). CSF corticotropin-releasing hormone and somatostatin in major depression: Response to antidepressant treatment and relapse. European Neuropsychopharmacology, 2(2), 107–113. https://doi.org/10.1016/0924-977X(92)90019-5 Baradell, J. G., & Klein, K. (1993). Relationship of life stress and body consciousness to hypervigilant decision making. Personality Processes and Individual Differences, 64(2), 267–273. Bardgett, M. E., Depenbrock, M., Downs, N., Points, M., & Green, L. (2009). Dopamine modulates effort-based decision making in rats. Behavioral Neuroscience, 123(2), 242–251. https://doi.org/10.1037/a0014625 Bardgett, M. E., Taylor, G. T., Csernansky, J. G., Newcomer, J. W., & Nock, B. (1994). Chronic corticosterone treatment impairs spontaneous alternation behavior in rats. Behavioral and Neural Biology, 61(2), 186–190. https://doi.org/10.1016/S0163-1047(05)80074-3 Bari, A., Theobald, D. E., Caprioli, D., Mar, A. C., Aidoo-Micah, A., Dalley, J. W., & Robbins, T. W. (2010). Serotonin modulates sensitivity to reward and negative feedback in a probabilistic reversal learning task in rats. Neuropsychopharmacology, 35(6), 1290–1301. https://doi.org/10.1038/npp.2009.233 Bayless, D. W., Darling, J. S., Stout, W. J., & Daniel, J. M. (2012). Sex differences in attentional processes in adult rats as measured by performance on the 5-choice serial reaction time task. 203 Behavioural Brain Research, 235(1), 48–54. https://doi.org/10.1016/j.bbr.2012.07.028 Beard, C., Donahue, R. J., Dillon, D. G., Van’t Veer, A., Webber, C., Lee, J., … Pizzagalli, D. A. (2015). Abnormal error processing in depressive states: A translational examination in humans and rats. Translational Psychiatry, 5(January), e564. https://doi.org/10.1038/tp.2015.54 Beats, B., Sahakian, B. J., & Levy, R. (1996). Cognitive performance in tests sensitive to frontal lobe dysfunction in the elderly depressed. Psychological Medicine, 26, 591–603. Bechara, A., Damasio, H., & Damasio, A. R. (2000). Emotion, decision making and the orbitofrontal cortex. Cerebral Cortex, 295–307. https://doi.org/10.1093/cercor/10.3.295 Beckner, V. E., Tucker, D. M., Delville, Y., & Mohr, D. C. (2006). Stress facilitates consolidation of verbal memory for a film but does not affect retrieval. Behavioral Neuroscience, 120(3), 518–527. https://doi.org/10.1037/0735-7044.120.3.518 Beckstead, M. J., Gantz, S. C., Ford, C. P., Stenzel-Poore, M. P., Phillips, P. E., Mark, G. P., & Williams, J. T. (2009). CRF enhancement of GIRK channel-mediated transmission in dopamine neurons. Neuropsychopharmacology, 34(8), 1926–1935. https://doi.org/10.1038/npp.2009.25 Bell, S. M., Reynolds, J. G., Thiele, T. E., Gan, J., Figlewicz, D. P., & Woods, S. C. (1998). Effects of third intracerebroventricular injections of corticotropin-releasing factor (CRF) on ethanol drinking and food intake. Psychopharmacology, 139(1–2), 128–135. https://doi.org/10.1007/s002130050697 Berenbaum, H., & Connelly, J. (1993). The effect of stress on hedonic capacity. The Journal of Abnormal Psychology, 102(3), 474–481. https://doi.org/10.1509/jmr.15.0319 Berghorst, L. H., Bogdan, R., Frank, M. J., & Pizzagalli, D. A. (2013). Acute stress selectively reduces reward sensitivity. Frontiers in Human Neuroscience, 7, 1–15. https://doi.org/10.3389/fnhum.2013.00133 Berridge, K. C. (2012). From prediction error to incentive salience: Mesolimbic computation of reward motivation. European Journal of Neuroscience, 35(7), 1124–1143. https://doi.org/10.1111/j.1460-9568.2012.07990.x Berridge, K. C., & Robinson, T. E. (2003). Parsing reward. Trends in Neurosciences, 26(9), 507–513. https://doi.org/10.1016/S0166-2236(03)00233-9 Bimonte, H. A., & Denenberg, V. H. (2000). Sex differences in vicarious trial-and-error behavior during radial arm maze learning. Physiology and Behavior, 68(4), 495–499. https://doi.org/10.1016/S0031-9384(99)00201-2 Birrell, J. M., & Brown, V. J. (2000). Medial frontal cortex mediates perceptual attentional set shifting in the rat. Journal of Neuroscience, 20(11), 4320–4324. 204 Bittencourt, J. C., & Sawchenko, P. E. (2000). Do centrally administered neuropeptides access cognate receptors?: An analysis in the central corticotropin-releasing factor system. The Journal of Neuroscience, 20(3), 1142–1156. Blacktop, J. M., Seubert, C., Baker, D. a, Ferda, N., Lee, G., Graf, E. N., & Mantsch, J. R. (2011). Augmented cocaine seeking in response to stress or CRF delivered into the ventral tegmental area following long-access self-administration is mediated by CRF receptor type 1 but not CRF receptor type 2. The Journal of Neuroscience, 31(31), 11396–11403. https://doi.org/10.1523/JNEUROSCI.1393-11.2011 Blaes, S. L., Orsini, C. A., Mitchell, M. R., Spurrell, M. S., Betzhold, S. M., Vera, K., … Setlow, B. (2018). Monoaminergic modulation of decision-making under risk of punishment in a rat model. Behavioural Pharmacology, 29(8), 745–761. https://doi.org/10.1097/FBP.0000000000000448 Blank, T., Nijholt, I., Eckart, K., & Spiess, J. (2002). Priming of long-term potentiation in mouse hippocampus by corticotropin-releasing factor and acute stress: Implications for hippocampus-dependent learning. Journal of Neuroscience, 22(9), 3788–3794. Blank, T., Nijholt, I., Grammatopoulos, D. K., Randeva, H. S., Hillhouse, E. W., & Spiess, J. (2003). Corticotropin-releasing factor receptors couple to multiple G-proteins to activate diverse intracellular signaling pathways in mouse hippocampus: Role in neuronal excitability and associative learning. Journal of Neuroscience, 23(2), 700–707. Bogdan, R., & Pizzagalli, D. A. (2006). Acute stress reduces reward responsiveness: Implications for depression. Biological Psychiatry, 60(10), 1147–1154. https://doi.org/10.1016/j.biopsych.2006.03.037 Bogdan, R., Santesso, D. L., Fagerness, J., Perlis, R. H., & Pizzagalli, D. A. (2011). Corticotropin-releasing hormone receptor type 1 (CRHR1) genetic variation and stress interact to influence reward learning. Journal of Neuroscience, 31(37), 13246–13254. https://doi.org/10.1523/jneurosci.2661-11.2011 Bohnen, N., Houx, P., Nicolson, N., & Jolles, J. (1990). Cortisol reactivity and cognitive performance in a continuous mental task paradigm. Biological Psychology, 31(2), 107–116. https://doi.org/10.1016/0301-0511(90)90011-K Bondi, C. O., Jett, J. D., & Morilak, D. A. (2010). Beneficial effects of desipramine on cognitive function of chronically stressed rats are mediated by α1-adrenergic receptors in medial prefrontal cortex. Progress in Neuro-Psychopharmacology and Biological Psychiatry, 34(6), 913–923. https://doi.org/10.1016/j.pnpbp.2010.04.016 Bondi, C. O., Rodriguez, G., Gould, G. G., Frazer, A., & Morilak, D. A. (2008). Chronic unpredictable stress induces a cognitive deficit and anxiety-like behavior in rats that is prevented by chronic antidepressant drug treatment. Neuropsychopharmacology, 33(2), 320–331. https://doi.org/10.1038/sj.npp.1301410 Born, J. M., Lemmens, S. G. T., Rutters, F., Nieuwenhuizen, A. G., Formisano, E., Goebel, R., & 205 Westerterp-Plantenga, M. S. (2010). Acute stress and food-related reward activation in the brain during food choice during eating in the absence of hunger. International Journal of Obesity, 34(1), 172–181. https://doi.org/10.1038/ijo.2009.221 Bortz, D. M., & Grace, A. A. (2018). Medial septum differentially regulates dopamine neuron activity in the rat ventral tegmental area and substantia nigra via distinct pathways. Neuropsychopharmacology, 43(10), 2093–2100. https://doi.org/10.1038/s41386-018-0048-2 Bourne, M. J., Calton, J. L., Gustavson, K. K., & Schachtman, T. R. (1992). Effects of acute swim stress on LiCl-induced conditioned taste aversions. Physiology and Behavior, 51(6), 1227–1234. https://doi.org/10.1016/0031-9384(92)90313-Q Bremner, J. D., Licinio, J., Darnell, A., Krystal, J. H., Owens, M. J., Southwick, S. M., … Charney, D. S. (1997). Elevated CSF corticotropin-releasing factor concentrations in posttraumatic stress disorder. American Journal of Psychiatry, 154(5), 624–629. https://doi.org/10.1176/ajp.154.5.624 Bristow, L. J., Cook, G. P., Gay, J. C., Kulagowski, J. J., Landon, L., Murray, F., … Hutson, P. H. (1996). The behavioural and neurochemical profile of the putative dopamine D3 receptor agonist, (+)-PD 128907, in the rat. Neuropharmacology, 35(3), 285–294. https://doi.org/10.1016/0028-3908(96)00179-7 Britton, D. R., Koob, G. F. I., Vale, J. R. W., & Diego, S. (1982). Intraventricular corticotropin-releasing factor enhances behavioral effects of novelty. Life Sciences, 31, 363–367. Britton, D. R., Varela, M., Garcia, A., & Rosenthal, M. (1986). Dexamethasone suppresses pituitary-adrenal but not behavioral effects of centrally administered CRF. Life Sciences, 38, 211–216. Brownlee, K. K., Moore, A. W., & Hackney, A. C. (2005). Relationship between circulating cortisol and testosterone: Influence of physical exercise. Journal of Sports Science and Medicine, 4(1), 76–83. Bryce, C. A., & Floresco, S. B. (2016). Perturbations in effort-related decision-making driven by acute stress and corticotropin-releasing factor. Neuropsychopharmacology, 41, 2147–2159. https://doi.org/10.1038/npp.2016.15 Bryce, C. A., & Howland, J. G. (2015). Stress facilitates late reversal learning using a touchscreen-based visual discrimination procedure in male Long Evans rats. Behavioural Brain Research, 278, 21–28. https://doi.org/10.1016/j.bbr.2014.09.027 Buchanan, T. W., & Lovallo, W. R. (2001). Enhanced memory for emotional material following stress-level cortisol treatment in humans. Psychoneuroendocrinology, 26(3), 307–317. https://doi.org/10.1016/S0306-4530(00)00058-5 Buffalari, D. M., Baldwin, C. K., Feltenstein, M. W., & See, R. E. (2012). Corticotrophin releasing factor (CRF) induced reinstatement of cocaine seeking in male and female rats. 206 Physiology and Behavior, 105(2), 209–214. https://doi.org/10.1016/j.physbeh.2011.08.020 Butts, K. A., Floresco, S. B., & Phillips, A. G. (2013). Acute stress impairs set-shifting but not reversal learning. Behavioural Brain Research, 252, 222–229. https://doi.org/10.1016/j.bbr.2013.06.007 Butts, K. A., Weinberg, J., Young, A. H., & Phillips, A. G. (2011). Glucocorticoid receptors in the prefrontal cortex regulate stress-evoked dopamine efflux and aspects of executive function. Proceedings of the National Academy of Sciences, 108(45), 18459–18464. https://doi.org/10.1073/pnas.1111746108 Cador, M., Ahmed, S. H., Koob, G. F., Le Moal, M., & Stinus, L. (1992). Corticotropin-releasing factor induces a place aversion independent of its neuroendocrine role. Brain Research, 597(2), 304–309. https://doi.org/10.1016/0006-8993(92)91487-Y Cahill, L., Gorski, L., & Le, K. (2003). Enhanced human memory consolidation with post-learning stress: Interaction with the degree of arousal at encoding. Learning and Memory, 10(4), 270–274. https://doi.org/10.1101/lm.62403 Cahill, L., & McGaugh, J. L. (1998). Mechanisms of emotional arousal and lasting declarative memory. Trends in Neurosciences, 21(7), 294–299. https://doi.org/10.1016/S0166-2236(97)01214-9 Cai, W. H., Blundell, J., Han, J., Greene, R. W., & Powell, C. M. (2006). Postreactivation glucocorticoids impair recall of established fear memory. Journal of Neuroscience, 26(37), 9560–9566. https://doi.org/10.1523/JNEUROSCI.2397-06.2006 Calu, D. J., Kawa, A. B., Marchant, N. J., Navarre, B. M., Henderson, M. J., Chen, B., … Shaham, Y. (2013). Optogenetic inhibition of dorsal medial prefrontal cortex attenuates stress-induced reinstatement of palatable food seeking in female rats. Journal of Neuroscience, 33(1), 214–226. https://doi.org/10.1523/JNEUROSCI.2016-12.2013 Campbell, B. M., Morrison, J. L., Walker, E. L., & Merchant, K. M. (2004). Differential regulation of behavioral, genomic and neuroendocrine responses by CRF infusions in rats. Pharmacology Biochemistry and Behavior, 77(3), 447–455. https://doi.org/10.1016/j.pbb.2003.12.010 Cannon, W. B. (1915). Bodily changes in pain, hunger, fear, and rage: An account of recent researches into the function of emotional excitement. New York/London: Appleton Century Crofts. Carey, M., Deterd, C., de Koning, J., Helmerhorst, F., & de Kloet, E. (1995). The influence of ovarian steroids on hypothalamic-pituitary- adrenal regulation in the female rat. The Journal of Endocrinology, 144(2), 311–321. https://doi.org/10.1677/joe.0.1440311 Carleton, N. R., Mulvogue, M. K., Thibodeau, M. A., McCabe, R. E., Antony, M. M., & Asmundson, G. J. G. (2012). Increasingly certain about uncertainty: Intolerance of uncertainty across anxiety and depression. Journal of Anxiety Disorders, 26(3), 468–479. 207 https://doi.org/10.1016/j.janxdis.2012.01.011 Castañé, A. A., Theobald, D. E. H., & Robbins, T. W. (2010). Selective lesions of the dorsomedial striatum impair serial spatial reversal learning in rats. Behavioural Brain Research, 210(1), 74–83. https://doi.org/10.1016/j.bbr.2010.02.017 Cazakoff, B. N., Johnson, K. J., & Howland, J. G. (2010). Converging effects of acute stress on spatial and recognition memory in rodents: A review of recent behavioural and pharmacological findings. Progress in Neuro-Psychopharmacology and Biological Psychiatry, 34(5), 733–741. https://doi.org/10.1016/j.pnpbp.2010.04.002 Cella, M., Dymond, S., & Cooper, A. (2010). Impaired flexible decision-making in major depressive disorder. Journal of Affective Disorders, 124(1–2), 207–210. https://doi.org/10.1016/j.jad.2009.11.013 Cerqueira, J. J., Mailliet, F., Almeida, O. F. X., Jay, T. M., & Sousa, N. (2007). The prefrontal cortex as a key target of the maladaptive response to stress. Journal of Neuroscience, 27(11), 2781–2787. https://doi.org/10.1523/JNEUROSCI.4372-06.2007 Chappell, P. B., Smith, M. A., Kilts, C. D., Bissette, G., Ritchie, J., & Nemeroff, C. B. (1986). Alterations in corticotropin-releasing factor-like immunoreactivity in discrete rat brain regions after acute and chronic stress. The Journal of Neuroscience, 6(10), 2908–2914. Chase, H. W., Frank, M. J., Michael, A., Bullmore, E. T., Sahakian, B. J., & Robbins, T. W. (2010). Approach and avoidance learning in patients with major depression and healthy controls: Relation to anhedonia. Psychological Medicine, 40(3), 433–440. https://doi.org/10.1017/S0033291709990468 Chen, N. A., Jupp, B., Sztainberg, Y., Lebow, M., Brown, R. M., Kim, J. H., … Lawrence, A. J. (2014). Knockdown of CRF1 Receptors in the Ventral Tegmental Area Attenuates Cue- and Acute Food Deprivation Stress-Induced Cocaine Seeking in Mice. Journal of Neuroscience, 34(35), 11560–11570. https://doi.org/10.1523/JNEUROSCI.4763-12.2014 Chen, R., Lewis, K. A., Perrin, M. H., & Vale, W. W. (1993). Expression cloning of a human corticotropin-releasing-factor receptor. Proceedings of the National Academy of Sciences of the United States of America, 90(19), 8967–8971. https://doi.org/10.1073/pnas.90.19.8967 Chen, Y. W., Rada, P. V., Bützler, B. P., Leibowitz, S. F., & Hoebel, B. G. (2012). Corticotropin-releasing factor in the nucleus accumbens shell induces swim depression, anxiety, and anhedonia along with changes in local dopamine/acetylcholine balance. Neuroscience, 206, 155–166. https://doi.org/10.1016/j.neuroscience.2011.12.009 Cifani, C., Koya, E., Navarre, B. M., Calu, D. J., Baumann, M. H., Marchant, N. J., … Hope, B. T. (2012). Medial prefrontal cortex neuronal activation and synaptic alterations after stress-induced reinstatement of palatable food seeking: A study using c-fos-GFP transgenic female rats. Journal of Neuroscience, 32(25), 8480–8490. https://doi.org/10.1523/JNEUROSCI.5895-11.2012 208 Clark, L., Cools, R., & Robbins, T. W. (2004). The neuropsychology of ventral prefrontal cortex: Decision-making and reversal learning. Brain and Cognition, 55(1), 41–53. https://doi.org/10.1016/S0278-2626(03)00284-7 Clark, L., Dombrovski, A. Y., Siegle, G. J., Butters, M. A., Shollenberger, C. L., Sahakian, B. J., & Szanto, K. (2011). Impairment in risk-sensitive secision-making in older suicide attempters with depression. Psychology and Aging, 26(2), 321–330. https://doi.org/10.1037/a0021646 Cole, R. D., Kawasumi, Y., Parikh, V., & Bangasser, D. A. (2016). Corticotropin releasing factor impairs sustained attention in male and female rats. Behavioural Brain Research, 296, 30–34. https://doi.org/10.1016/j.bbr.2015.08.023 Conrad, C. D., Jackson, J. L., Wieczorek, L., Baran, S. E., Harman, J. S., Wright, R. L., & Korol, D. L. (2004). Acute stress impairs spatial memory in male but not female rats: Influence of estrous cycle. Pharmacology Biochemistry and Behavior, 78(3), 569–579. https://doi.org/10.1016/j.pbb.2004.04.025 Contoreggi, C., Lee, M. R., & Chrousos, G. (2013). Addiction and corticotropin-releasing hormone type 1 receptor antagonist medications. Annals of the New York Academy of Sciences, 1282(1), 107–118. https://doi.org/10.1111/nyas.12007 Cornelisse, S., van Stegeren, A. H., & Joëls, M. (2011). Implications of psychosocial stress on memory formation in a typical male versus female student sample. Psychoneuroendocrinology, 36(4), 569–578. https://doi.org/10.1016/j.psyneuen.2010.09.002 Cousins, M. S., & Salamone, J. D. (1994). Nucleus accumbens dopamine depletions in rats affect relative response allocation in a novel cost/benefit procedure. Pharmacology, Biochemistry and Behavior, 49(1), 85–91. https://doi.org/10.1016/0091-3057(94)90460-X Cousins, M. S., Sokolowski, J. D., & Salamone, J. D. (1993). Different effects of nucleus accumbens and ventrolateral striatal dopamine depletions on instrumental response selection in the rat. Pharmacology, Biochemistry and Behavior, 46(4), 943–951. https://doi.org/10.1016/0091-3057(93)90226-J Cousins, M. S., Wei, W., & Salamone, J. D. (1994). Pharmacological characterization of performance on a concurrent lever pressing/feeding choice procedure: Effects of dopamine antagonist, cholinomimetic, sedative and stimulant drugs. Psychopharmacology, 116(4), 529–537. https://doi.org/10.1007/BF02247489 Cross, C. P., Copping, L. T., & Campbell, A. (2011). Sex differences in impulsivity: A meta-analysis. Psychological Bulletin, 137(1), 97–130. https://doi.org/10.1037/a0021591 Curtis, A. L., Lechner, S. M., Pavcovich, L. A., & Valentino, R. J. (1997). Activation of the locus coeruleus noradrenergic system by intracoerulear microinfusion of corticotropin-releasing factor: Effects on discharge rate, cortical norepinephrine levels and cortical electroencephalographic activity. Journal of Pharmacology and Experimental Therapeutics, 209 281(1), 163–172. D’Esposito, M., & Postle, B. (2015). The cognitive neuroscience of working memory. Annual Review of Psychology, 66, 115–142. https://doi.org/10.1093/acprof:oso/9780198570394.001.0001 Dabrowska, J., Martinon, D., Moaddab, M., & Rainnie, D. G. (2016). Targeting corticotropin-releasing factor projections from the oval nucleus of the bed nucleus of the stria terminalis using cell-type specific neuronal tracing studies in mouse and rat brain. Journal of Neuroendocrinology, 28(12). https://doi.org/10.1111/jne.12442 Dalton, G. L., Phillips, A. G., & Floresco, S. B. (2014). Preferential involvement by nucleus accumbens shell in mediating probabilistic learning and reversal shifts. Journal of Neuroscience, 34(13), 4618–4626. https://doi.org/10.1523/JNEUROSCI.5058-13.2014 Dalton, G. L., Wang, N. Y., Phillips, A. G., & Floresco, S. B. (2016). Multifaceted contributions by different regions of the orbitofrontal and medial prefrontal cortex to probabilistic reversal learning. The Journal of Neuroscience, 36(6), 1996–2006. https://doi.org/10.1523/jneurosci.3366-15.2016 Davies, D., Molder, J. J., Greba, Q., & Howland, J. G. (2013). Inactivation of medial prefrontal cortex or acute stress impairs odor. Learning & Memory, 20(12), 665–669. De Kloet, E. R., Joëls, M., & Holsboer, F. (2005). Stress and the brain: From adaptation to disease. Nature Reviews Neuroscience, 6(6), 463–475. https://doi.org/10.1038/nrn1683 de Quervain, D. J. F., Roozendaal, B., Nitsch, R. M., McGaugh, J. L., & Hock, C. (2000). Acute cortisone administration impairs retrieval of long-term declarative memory in humans. Nature Neuroscience, 3(4), 313–314. https://doi.org/10.1038/73873 de Quervain, D., Roozendaal, B., & McGaugh, J. L. (1998). Stress and glucocorticoids impair retrieval of long-term spatial memory. Nature, 394(August), 787–790. Deak, T., Nguyen, K. T., Ehrlich, A. L., Watkins, L. R., Spencer, R. L., Maier, S. F., … Gold, P. W. (1999). The impact of the nonpeptide corticotropin-releasing hormone antagonist antalarmin on behavioral and endocrine responses to stress. Endocrinology, 140(1), 79–86. https://doi.org/10.1210/endo.140.1.6415 Deisenhammer, E. A., Schmid, S. K., Kemmler, G., Moser, B., & Delazer, M. (2018). Decision making under risk and under ambiguity in depressed suicide attempters, depressed non-attempters and healthy controls. Journal of Affective Disorders, 226, 261–266. https://doi.org/10.1016/j.jad.2017.10.012 Del Arco, A., Segovia, G., Garrido, P., de Blas, M., & Mora, F. (2007). Stress, prefrontal cortex and environmental enrichment: Studies on dopamine and acetylcholine release and working memory performance in rats. Behavioural Brain Research, 176(2), 267–273. https://doi.org/10.1016/j.bbr.2006.10.006 210 Depoortere, R., Perrault, G., & Sanger, D. J. (1996). Behavioural effects in the rat of the putative dopamine D3 receptor agonist 7-OH-DPAT : Comparison with quinpirole and apomorphine. Psychopharmacology, 124, 231–240. Devilbiss, D. M., Jenison, R. L., & Berridge, C. W. (2012). Stress-induced impairment of a working memory rask: Role of spiking rate and spiking history predicted discharge. PLoS Computational Biology, 8(9). https://doi.org/10.1371/journal.pcbi.1002681 Diamond, D. M., Campbell, A. M., Park, C. R., Woodson, J. C., Conrad, C. D., Bachstetter, A. D., & Mervis, R. F. (2006). Influence of predator stress on the consolidation versus retrieval of long-term spatial memory and hippocampal spinogenesis, 16, 571–576. https://doi.org/10.1002/hipo Diamond, D. M., Ingersoll, N., Fleshner, M., & Rose, G. M. (1996). Psychological stress impairs spatial working memory: Relevance to electrophysiological studies of hippocampal function. Behavioral Neuroscience, 110(4), 661–672. https://doi.org/10.1037/0735-7044.110.4.661 Dickstein, D. P., Finger, E. C., Brotman, M. A., Rich, B. A., Pine, D. S., Blair, J. R., & Leibenluft, E. (2010). Impaired probabilistic reversal learning in youths with mood and anxiety disorders. Psychological Medicine, 40(7), 1089–1100. https://doi.org/10.1017/S0033291709991462 Dirks, A., Groenink, L., Bouwknecht, J. A., Hijzen, T. H., Van Der Gugten, J., Ronken, E., … Olivier, B. (2002). Overexpression of corticotropin-releasing hormone in transgenic mice and chronic stress-like autonomic and physiological alterations. European Journal of Neuroscience, 16(9), 1751–1760. https://doi.org/10.1046/j.1460-9568.2002.02245.x Doi, H., Nishitani, S., & Shinohara, K. (2015). Sex difference in the relationship between salivary testosterone and inter-temporal choice. Hormones and Behavior, 69, 50–58. https://doi.org/10.1016/j.yhbeh.2014.12.005 Dombrovski, A. Y., Clark, L., Siegle, G. J., Butters, M. A., Ichikawa, N., Sahakian, B., & Szanto, K. (2010). Reward/punishment reversal learning in older suicide attempters. American Journal of Psychiatry, 167(6), 699–707. https://doi.org/10.1176/appi.ajp.2009.09030407.REWARD/PUNISHMENT Domes, G., Heinrichs, M., Reichwald, U., & Hautzinger, M. (2002). Hypothalamic–pituitary–adrenal axis reactivity to psychological stress and memory in middle- aged women: High responders exhibit enhanced declarative memory performance. Psychoneuroendocrinology, 27, 843–853. https://doi.org/0306-4530/02/$ Dong, Z., Bai, Y., Wu, X., Li, H., Gong, B., Howland, J. G., … Wang, Y. T. (2013). Hippocampal long-term depression mediates spatial reversal learning in the Morris water maze. Neuropharmacology, 64, 65–73. https://doi.org/10.1016/j.neuropharm.2012.06.027 Duff, S. J., & Hampson, E. (2001). A sex difference on a novel spatial working memory task in humans. Brain and Cognition, 47(3), 470–493. https://doi.org/10.1006/brcg.2001.1326 211 Duncko, R., Johnson, L., Merikangas, K., & Grillon, C. (2009). Working memory performance after acute exposure to the cold pressor stress in healthy volunteers. Neurobiology of Learning and Memory, 91(4), 377–381. https://doi.org/10.1016/j.nlm.2009.01.006 Dunn, A. J. (1988). Stress-related changes in cerebral catecholamine and indoleamine metabolism: Lack of effect of adrenalectomy and corticosterone. Journal of Neurochemistry, 51(2), 406–412. https://doi.org/10.1111/j.1471-4159.1988.tb01053.x Dunn, A. J., & Berridge, C. W. (1987). Corticotropin-releasing factor administration elicits a stress-like activation of cerebral catecholaminergic systems. Pharmacology, Biochemistry and Behavior, 27(4), 685–691. https://doi.org/10.1016/0091-3057(87)90195-X Dunn, A. J., & Berridge, C. W. (1990). Physiological and behavioral-responses to corticotropin- releasing factor administration - Is CRF a mediator of anxiety or stress responses. Brain Research Reviews, 15(2), 71–100. https://doi.org/10.1016/0165-0173(90)90012-D Elliott, R., Sahakian, B. J., Herrod, J. J., Robbins, T. W., & Paykel, E. S. (1997). Abnormal response to negative feedback in unipolar depression: Evidence for a diagnosis specific impairment. Journal of Neurology, Neurosurgery & Psychiatry, 63(1), 74–82. https://doi.org/10.1136/jnnp.63.1.74 Elzinga, B. M., Bakker, A., & Bremner, J. D. (2005). Stress-induced cortisol elevations are associated with impaired delayed, but not immediate recall. Psychiatry Research, 134(3), 211–223. https://doi.org/10.1016/j.psychres.2004.11.007 Epting, K. L. (1998). Sex-senistive taks in men and women: A search for performance fluctuations across the menstrual cycle. Behavioral Neuroscience, 112(6), 1304–1317. Erb, S., Petrovic, A., Yi, D., & Kayyali, H. (2006). Central injections of CRF reinstate cocaine seeking in rats after postinjection delays of up to 3 h: An influence of time and environmental context. Psychopharmacology, 187(1), 112–120. https://doi.org/10.1007/s00213-006-0392-5 Eshel, N., & Roiser, J. P. (2010). Reward and punishment processing in depression. Biological Psychiatry, 68(2), 118–124. https://doi.org/10.1016/j.biopsych.2010.01.027 Fields, S. A., Lange, K., Ramos, A., Thamotharan, S., & Rassu, F. (2014). The relationship between stress and delay discounting. Behavioural Pharmacology, 25, 434–444. https://doi.org/10.1097/FBP.0000000000000044 Filla, I., Bailey, M. R., Schipani, E., Winiger, V., Mezias, C., Balsam, P. D., & Simpson, E. H. (2018). Striatal dopamine D2 receptors regulate effort but not value-based decision making and alter the dopaminergic encoding of cost. Neuropsychopharmacology, 43(11), 2180–2189. https://doi.org/10.1038/s41386-018-0159-9 Fiorillo, C. D., Newsome, W. T., & Schultz, W. (2008). The temporal precision of reward prediction in dopamine neurons. Nature Neuroscience, 11(8), 966–973. https://doi.org/10.1038/nn.2159 212 Floresco, S. B. (2015). The nucleus accumbens: An interface between cognition, emotion, and action. Annual Review of Psychology, 66(1), 25–52. https://doi.org/10.1146/annurev-psych-010213-115159 Floresco, S. B., Block, A. E., & Tse, M. T. L. (2008). Inactivation of the medial prefrontal cortex of the rat impairs strategy set-shifting, but not reversal learning, using a novel, automated procedure. Behavioural Brain Research, 190(1), 85–96. https://doi.org/10.1016/j.bbr.2008.02.008 Floresco, S. B., Ghods-Sharifi, S., Vexelman, C., & Magyar, O. (2006). Dissociable roles for the nucleus accumbens core and shell in regulating set shifting. Journal of Neuroscience, 26(9), 2449–2457. https://doi.org/10.1523/JNEUROSCI.4431-05.2006 Floresco, S. B., Montes, D. R., Tse, M. M. T., & van Holstein, M. (2018). Differential contributions of nucleus accumbens subregions to cue-guided risk / reward decision making and implementation of conditional rules. The Journal of Neuroscience, 38(8), 1901–1914. https://doi.org/10.1523/JNEUROSCI.3191-17.2018 Floresco, S. B., St. Onge, J. R., Ghods-Sharifi, S., & Winstanley, C. A. (2008). Cortico-limbic-striatal circuits subserving different forms of cost-benefit decision making. Cognitive, Affective and Behavioral Neuroscience, 8(4), 375–389. https://doi.org/10.3758/CABN.8.4.375 Floresco, S. B., Todd, C. L., & Grace, A. A. (2001). Glutamatergic afferents from the hippocampus to the nucleus accumbens regulate activity of ventral tegmental area dopamine neurons. The Journal of Neuroscience, 21(13), 4915–4922. https://doi.org/21/13/4915 [pii] Floresco, S. B., Tse, M. T. L., & Ghods-Sharifi, S. (2008). Dopaminergic and glutamatergic regulation of effort- and delay-based decision making. Neuropsychopharmacology, 33(8), 1966–1979. https://doi.org/10.1038/sj.npp.1301565 Floresco, S. B., West, A. R., Ash, B., Moorel, H., & Grace, A. A. (2003). Afferent modulation of dopamine neuron firing differentially regulates tonic and phasic dopamine transmission. Nature Neuroscience, 6(9), 968–973. https://doi.org/10.1038/nn1103 Floresco, S. B., Zhang, Y., & Enomoto, T. (2009). Neural circuits subserving behavioral flexibility and their relevance to schizophrenia. Behavioural Brain Research, 204(2), 396–409. https://doi.org/10.1016/j.bbr.2008.12.001 Foote, S. L., Bloom, F. E., & Aston Jones, G. (1983). Nucleus locus ceruleus: New evidence of anatomical and physiological specificity. Physiological Reviews, 63(3), 844–914. https://doi.org/10.1152/physrev.1983.63.3.844 Gabrys, R. L., Howell, J. W., Cebulski, S. F., Anisman, H., & Matheson, K. (2019). Acute stressor effects on cognitive flexibility: Mediating role of stressor appraisals and cortisol. Stress, 22(2), 182–189. https://doi.org/10.1080/10253890.2018.1494152 Galesi, F. L., Ayanwuyi, L. O., Mijares, M. G., Cippitelli, A., Cannella, N., Ciccocioppo, R., & 213 Ubaldi, M. (2016). Role of Hypothalamic-Pituitary-Adrenal axis and corticotropin-releasing factor stress system on cue-induced relapse to alcohol seeking. European Journal of Pharmacology, 788, 84–89. https://doi.org/10.1016/j.ejphar.2016.06.020 Gallopin, T., Geoffroy, H., Rossier, J., & Lambolez, B. (2006). Cortical sources of CRF, NKB, and CCK and their effects on pyramidal cells in the neocortex. Cerebral Cortex, 16(10), 1440–1452. https://doi.org/10.1093/cercor/bhj081 Gathmann, B., Schulte, F. P., Maderwald, S., Pawlikowski, M., Starcke, K., Schäfer, L. C., … Brand, M. (2014). Stress and decision making: neural correlates of the interaction between stress, executive functions, and decision making under risk. Experimental Brain Research, 232(3), 957–973. https://doi.org/10.1007/s00221-013-3808-6 Ghitza, U. E., Gray, S. M., Epstein, D. H., Rice, K. C., & Shaham, Y. (2006). The anxiogenic drug yohimbine reinstates palatable food seeking in a rat relapse model: a role of CRF1 receptors. Neuropsychopharmacology, 31(10), 2188–2196. https://doi.org/10.1038/sj.npp.1300964 Ghods-Sharifi, S., & Floresco, S. B. (2010). Differential effects on effort discounting induced by inactivations of the nucleus accumbens core or shell. Behavioral Neuroscience, 124(2), 179–191. https://doi.org/10.1037/a0018932 Ghods-Sharifi, S., Haluk, D. M., & Floresco, S. B. (2008). Differential effects of inactivation of the orbitofrontal cortex on strategy set-shifting and reversal learning. Neurobiology of Learning and Memory, 89(4), 567–573. https://doi.org/10.1016/j.nlm.2007.10.007 Giambra, L. M., & Quilter, R. E. (1989). Sex differences in sustained attention across the adult life span. Journal of Applied Psychology, 74(1), 91–95. https://doi.org/10.1037/0021-9010.74.1.91 Gibbs, R. B., & Johnson, D. A. (2008). Sex-specific effects of gonadectomy and hormone treatment on acquisition of a 12-arm radial maze task by Sprague Dawley rats. Endocrinology, 149(6), 3176–3183. https://doi.org/10.1210/en.2007-1645 Glowa, J. R., Barrett, J. E., Russell, J., & Gold, P. W. (1992). Effects of corticotropin releasing hormone on appetitive behaviors. Peptides, 13(3), 609–621. https://doi.org/10.1016/0196-9781(92)90097-M Gold, P. E., & Van Buskirk, R. B. (1975). Facilitation of time-dependent memory processes with posttrial epinephrine injections. Behavioral Biology, 13(2), 145–153. https://doi.org/10.1016/S0091-6773(75)91784-8 Gold, P. W. (2015). The organization of the stress system and its dysregulation in depressive illness. Molecular Psychiatry, 20(1), 32–47. https://doi.org/10.1038/mp.2014.163 Gorlyn, M., Keilp, J. G., Oquendo, M. A., Burke, A. K., & John Mann, J. (2013). Iowa Gambling Task performance in currently depressed suicide attempters. Psychiatry Research, 207(3), 150–157. https://doi.org/10.1016/j.psychres.2013.01.030 214 Govindarajan, A., Shankaranarayana Rao, B. S., Nair, D., Trinh, M., Mawjee, N., Tonegawa, S., & Chattarji, S. (2006). Transgenic brain-derived neurotrophic factor expression causes both anxiogenic and antidepressant effects. Proceedings of the National Academy of Sciences of the United States of America, 103(35), 13208–13213. https://doi.org/10.1073/pnas.0605180103 Grace, A. A. (2016). Dysregulation of the dopamine system in the pathophysiology of schizophrenia and depression. Nature Reviews Neuroscience, 17(8), 524–532. https://doi.org/10.1038/nrn.2016.57 Grace, A. A., & Bunney, B. S. (1983). Intracellular and extracellular electrophysiology of nigral dopaminergic neurons. 1. Identification and characterization. Neuroscience, 10(2), 301–315. https://doi.org/10.1016/0306-4522(83)90135-5 Grace, A. A., Floresco, S. B., Goto, Y., & Lodge, D. J. (2007). Regulation of firing of dopaminergic neurons and control of goal-directed behaviors. Trends in Neurosciences, 30(5), 220–227. https://doi.org/10.1016/j.tins.2007.03.003 Grafe, L. A., Cornfeld, A., Luz, S., Valentino, R., & Bhatnagar, S. (2017). Orexins mediate sex differences in the stress response and in cognitive flexibility. Biological Psychiatry, 81(8), 683–692. https://doi.org/10.1016/j.biopsych.2016.10.013 Grahek, I., Shenhav, A., Musslick, S., Krebs, R. M., & Koster, E. H. W. (2019). Neuroscience and biobehavioral reviews motivation and cognitive control in depression. Neuroscience and Biobehavioral Reviews, 102(December 2018), 371–381. https://doi.org/10.1016/j.neubiorev.2019.04.011 Grammatopoulos, D. K., Dai, Y., Randeva, H. S., Levine, M. A., Karteris, E., Easton, A. J., & Hillhouse, E. W. (1999). A novel spliced variant of the type 1 corticotropin-releasing hormone receptor with a deletion in the seventh transmembrane domain present in the human pregnant term myometrium and fetal membranes. Molecular Endocrinology, 13(12), 2189–2202. https://doi.org/10.1210/mend.13.12.0391 Grammatopoulos, D. K., Randeva, H. S., Levine, M. A., Kanellopoulou, K. A., & Hillhouse, E. W. (2001). Rat cerebral cortex corticotropin-releasing hormone receptors: Evidence for receptor coupling to multiple G-proteins. Journal of Neurochemistry, 76(2), 509–519. https://doi.org/10.1046/j.1471-4159.2001.00067.x Graybeal, C., Feyder, M., Schulman, E., Saksida, L. M., Bussey, T. J., Brigman, J. L., & Holmes, A. (2011). Paradoxical reversal learning enhancement by stress or prefrontal cortical damage: Rescue with BDNF. Nature Neuroscience, 14(12), 1507–1509. https://doi.org/10.1038/nn.2954 Grieder, T. E., Herman, M. A., Contet, C., Tan, L. A., Vargas-Perez, H., Cohen, A., … George, O. (2014). VTA CRF neurons mediate the aversive effects of nicotine withdrawal and promote intake escalation. Nature Neuroscience, 17(12), 1751–1758. https://doi.org/10.1038/nn.3872 215 Grissom, N. M., & Reyes, T. M. (2019). Let’s call the whole thing off: Evaluating gender and sex differences in executive function. Neuropsychopharmacology, 44(1), 86–96. https://doi.org/10.1038/s41386-018-0179-5 Gruner, P., & Pittenger, C. (2017). Cognitive inflexibility in obsessive-compulsive disorder. Neuroscience, 345, 243–255. https://doi.org/10.1016/j.neuroscience.2016.07.030 Habib, K. E., Gold, P. W., & Chrousos, G. P. (2001). Neuroendocrinology of stress. Neuroendocrinology, 30(3), 695–728. Haleem, D. J., & Parveen, T. (1994). Brain regional serotonin synthesis following adaptation to repeated restraint. NeuroReport. https://doi.org/10.1097/00001756-199409080-00025 Haluk, D. M., & Floresco, S. B. (2009). Ventral striatal dopamine modulation of different forms of behavioral flexibility. Neuropsychopharmacology, 34(8), 2041–2052. https://doi.org/10.1038/npp.2009.21 Han, G., Klimes-Dougan, B., Jepsen, S., Ballard, K., Nelson, M., Houri, A., … Cullen, K. (2012). Selective neurocognitive impairments in adolescents with major depressive disorder. Journal of Adolescence, 35(1), 11–20. https://doi.org/10.1016/j.adolescence.2011.06.009 Handa, R. J., Burgess, L. H., Kerr, J. E., & O’Keefe, J. A. (1994). Gonadal steroid hormone receptors and sex differences in the hypothalamo-pituitary-adrenal axis. Hormones and Behavior, 28, 464–476. https://doi.org/10.1006/hbeh.1994.1044 Heinrichs, S. C. (2003). Modulation of social learning in rats by brain corticotropin-releasing factor. Brain Research, 994(1), 107–114. https://doi.org/10.1016/j.brainres.2003.09.028 Heinrichs, S. C., Britton, K. T., & Koob, G. F. (1991). Both conditioned taste preference and aversion induced by corticotropin-releasing factor. Pharmacology, Biochemistry and Behavior, 40(4), 717–721. https://doi.org/10.1016/0091-3057(91)90075-D Heinrichs, S. C., Stenzel-Poore, M. P., Gold, L. H., Battenberg, E., Bloom, F. E., Koob, G. F., … Merlo Pich, E. (1996). Learning impairment in transgenic mice with central overexpression of corticotropin-releasing factor. Neuroscience, 74(2), 303–311. https://doi.org/10.1016/0306-4522(96)00140-6 Henriques, J. B., & Davidson, R. J. (2000). Decreased responsiveness to reward in depression. Cognition and Emotion, 14(5), 711–724. https://doi.org/10.1080/02699930050117684 Henriques, J. B., Glowacki, J. M., & Davidson, R. J. (1994). Reward fails to alter response bias in depression. Journal of Abnormal Psychology, 103(3), 460–466. Herman, J. P., Figueiredo, H., Mueller, N. K., Ulrich-Lai, Y., Ostrander, M. M., Choi, D. C., & Cullinan, W. E. (2003). Central mechanisms of stress integration: Hierarchical circuitry controlling hypothalamo-pituitary-adrenocortical responsiveness. Frontiers in Neuroendocrinology, 24(3), 151–180. https://doi.org/10.1016/j.yfrne.2003.07.001 216 Herman, J. P., McKlveen, J. M., Ghosal, S., Kopp, B., Wulsin, A., Makinson, R., … Myers, B. (2016). Regulation of the hypothalamic-pituitary- adrenocortical stress response. Comprehensive Physiology, 6(2), 603–621. https://doi.org/10.1002/cphy.c150015 Hillhouse, E. W., & Grammatopoulos, D. K. (2006). The molecular mechanisms underlying the regulation of the biological activity of corticotropin-releasing hormone receptors: Implications for physiology and pathophysiology. Endocrine Reviews, 27(3), 260–286. https://doi.org/10.1210/er.2005-0034 Hillhouse, E. W., Randeva, H., Ladds, G., & Grammatopoulos, D. (2002). Corticotropin-releasing hormone receptors. Biochemical Society Transactions, 30(4), 428–432. https://doi.org/10.1042/BST0300428 Hollon, N. G., Burgeno, L. M., & Phillips, P. E. M. (2015). Stress effects on the neural substrates of motivated behavior. Nature Neuroscience, 18(10), 1405–1412. https://doi.org/10.1038/nn.4114 Holly, E. N., Boyson, C. O., Montagud-Romero, S., Stein, D. J., Gobrogge, K. L., DeBold, J. F., & Miczek, K. A. (2016). Episodic social stress-escalated cocaine self-administration: Role of phasic and tonic corticotropin releasing factor in the anterior and posterior ventral tegmental area. The Journal of Neuroscience, 36(14), 4093–4105. https://doi.org/10.1523/JNEUROSCI.2232-15.2016 Holly, E. N., Debold, J. F., & Miczek, K. A. (2015). Increased mesocorticolimbic dopamine during acute and repeated social defeat stress: Modulation by corticotropin releasing factor receptors in the ventral tegmental area. Psychopharmacology, 232(24), 4469–4479. https://doi.org/10.1007/s00213-015-4082-z Holly, E. N., & Miczek, K. A. (2016). Ventral tegmental area dopamine revisited: Effects of acute and repeated stress. Psychopharmacology, 233(2), 163–186. https://doi.org/10.1007/s00213-015-4151-3 Holmes, A. J., & Wellman, C. L. (2009). Stress-induced prefrontal reorganization and executive dysfunction in rodents. Neuroscience and Biobehavioral Reviews, 33(6), 773–783. https://doi.org/10.1016/j.neubiorev.2008.11.005 Holmes, Avram J., & Pizzagalli, D. A. (2007). Task feedback effects on conflict monitoring and executive control: Relationship to subclinical measures of depression. Emotion, 7(1), 68–76. https://doi.org/10.1037/1528-3542.7.1.68 Howland, J. G., & Cazakoff, B. N. (2010). Effects of acute stress and GluN2B-containing NMDA receptor antagonism on object and object-place recognition memory. Neurobiology of Learning and Memory, 93(2), 261–267. https://doi.org/10.1016/j.nlm.2009.10.006 Hubbard, D. T., Nakashima, B. R., Lee, I., & Takahashi, L. K. (2007). Activation of basolateral amygdala corticotropin-releasing factor 1 receptors modulates the consolidation of contextual fear. Neuroscience, 150(4), 818–828. https://doi.org/10.1016/j.neuroscience.2007.10.001 217 Hupalo, S., & Berridge, C. W. (2016). Working memory impairing actions of corticotropin-releasing factor (CRF) neurotransmission in the prefrontal cortex. Neuropsychopharmacology, 41(11), 2733–2740. https://doi.org/10.1038/npp.2016.85 Hupalo, S., Martin, A. J., Green, R. K., Devilbiss, D. M., & Berridge, C. W. (2019). Prefrontal corticotropin-releasing factor (CRF) neurons act locally to modulate frontostriatal cognition and circuit function. The Journal of Neuroscience, 39(11), 2080–2090. https://doi.org/10.1523/JNEUROSCI.2701-18.2019 Hurtubise, J. L., & Howland, J. G. (2017). Effects of stress on behavioral flexibility in rodents. Neuroscience, 345, 176–192. https://doi.org/10.1016/j.neuroscience.2016.04.007 Ilonen, T., & Leinonen, K. (2000). Impaired Wisconsin Card Sorting Test performance in first-episode severe depression. Nordic Journal of Psychiatry, 54(4), 275–280. https://doi.org/10.1080/080394800448156 Imaki, T., Nahan, J. L., Rivier, C., Sawchenko, P. E., & Vale, W. (1991). Differential regulation of corticotropin-releasing factor mRNA in rat brain regions by glucocorticoids and stress. Journal of Neuroscience, 11(3), 585–599. Introini-Collison, I., Saghafi, D., Novack, G. D., & McGaugh, J. L. (1992). Memory-enhancing effects of post-training dipivefrin and epinephrine: Involvement of peripheral and central adrenergic receptors. Brain Research, 572(1–2), 81–86. https://doi.org/10.1016/0006-8993(92)90454-H Isogawa, K., Bush, D. E. A., & Ledoux, J. E. (2013). Contrasting effects of pretraining, posttraining, and pretesting infusions of corticotropin-releasing factor into the lateral amygdala: Attenuation of fear memory formation but facilitation of its expression. Biological Psychiatry, 73(4), 353–359. https://doi.org/10.1016/j.biopsych.2012.08.021 Itoga, C. A., Chen, Y., Fateri, C., Echeverry, P. A., Lai, J. M., Delgado, J., … Xu, X. (2019). New viral-genetic mapping uncovers an enrichment of corticotropin-releasing hormone-expressing neuronal inputs to the nucleus accumbens from stress-related brain regions. Journal of Comparative Neurology, 527(15), 2474–2487. https://doi.org/10.1002/cne.24676 Iwasaki-Sekino, A., Mano-Otagiri, A., Ohata, H., Yamauchi, N., & Shibasaki, T. (2009). Gender differences in corticotropin and corticosterone secretion and corticotropin-releasing factor mRNA expression in the paraventricular nucleus of the hypothalamus and the central nucleus of the amygdala in response to footshock stress or psychological. Psychoneuroendocrinology, 34(2), 226–237. https://doi.org/10.1016/j.psyneuen.2008.09.003 Jenni, N. L., Larkin, J. D., & Floresco, S. B. (2017). Prefrontal dopamine D1 and D2 receptors regulate dissociable aspects of decision making via distinct ventral striatal and amygdalar circuits. The Journal of Neuroscience, 37(26), 6200–6213. https://doi.org/10.1523/JNEUROSCI.0030-17.2017 Jentsch, J. D., & Taylor, J. R. (2003). Sex-related differences in spatial divided attention and 218 motor impulsivity in rats. Behavioral Neuroscience, 117(1), 76–83. https://doi.org/10.1037/0735-7044.117.1.76 Jett, J. D., Boley, A. M., Girotti, M., Shah, A., Lodge, D. J., & Morilak, D. A. (2015). Antidepressant-like cognitive and behavioral effects of acute ketamine administration associated with plasticity in the ventral hippocampus to medial prefrontal cortex pathway. Psychopharmacology, 232(17), 3123–3133. https://doi.org/10.1007/s00213-015-3957-3 Jett, J. D., Bulin, S. E., Hatherall, L. C., McCartney, C. M., & Morilak, D. A. (2017). Deficits in cognitive flexibility induced by chronic unpredictable stress are associated with impaired glutamate neurotransmission in the rat medial prefrontal cortex. Neuroscience, 346(3), 284–297. https://doi.org/10.1016/j.neuroscience.2017.01.017 Joëls, M., & Baram, T. Z. (2009). The neuro-symphony of stress. Nature Reviews Neuroscience, 10(6), 459–466. https://doi.org/10.1038/nrn2632 Joëls, M., Pu, Z., Wiegert, O., Oitzl, M. S., & Krugers, H. J. (2006). Learning under stress: How does it work? Trends in Cognitive Sciences, 10(4), 152–158. https://doi.org/10.1016/j.tics.2006.02.002 Jollant, F., Richard-Devantoy, S., Ding, Y., Turecki, G., Bechara, A., & Near, J. (2016). Prefrontal inositol levels and implicit decision-making in healthy individuals and depressed patients. European Neuropsychopharmacology, 26(8), 1255–1263. https://doi.org/10.1016/j.euroneuro.2016.06.005 Jones, D. N. C., Kortekaas, R., Slade, P. D., Middlemiss, D. N., & Hagan, J. J. (1998). The behavioural effects of corticotropin releasing factor-related peptides in rats. Psychopharmacology, 138(2), 124–132. https://doi.org/10.1007/s002130050654 Kalivas, P. W., & Duffy, P. (1995). Selective activation of dopamine transmission in the shell of the nuceus accumbens by stress. Brain Research, 675(1–2), 325–328. Kalivas, P. W., Duffy, P., & Latimer, L. G. (1987). Neurochemical and behavioral effects of corticotropin-releasing factor in the ventral tegmental area of the rat. The Journal of Pharmacology and Experimental Therapeutics, 242(3), 757–763. Kelly, E. A., & Fudge, J. L. (2018). The neuroanatomic complexity of the CRF and DA systems and their interface: What we still don’t know. Neuroscience and Biobehavioral Reviews, 90, 247–259. https://doi.org/10.1016/j.neubiorev.2018.04.014 Kim, H. W., Kang, J. I., Namkoong, K., Jhung, K., Ha, R. Y., & Kim, S. J. (2015). Further evidence of a dissociation between decision-making under ambiguity and decision-making under risk in obsessive-compulsive disorder. Journal of Affective Disorders, 176, 118–124. https://doi.org/10.1016/j.jad.2015.01.060 Kim, J. J., & Diamond, D. M. (2002). The stressed hippocampus, synaptic plasticity and lost memories. Nature Reviews Neuroscience, 3(6), 453–462. https://doi.org/10.1038/nrn849 219 Kimura, K., Izawa, S., Sugaya, N., Ogawa, N., Yamada, K. C., Shirotsuki, K., … Hasegawa, T. (2013). The biological effects of acute psychosocial stress on delay discounting. Psychoneuroendocrinology, 38(10), 2300–2308. https://doi.org/10.1016/j.psyneuen.2013.04.019 Kinner, V. L., Wolf, O. T., & Merz, C. J. (2016). Cortisol alters reward processing in the human brain. Hormones and Behavior, 84, 75–83. https://doi.org/10.1016/j.yhbeh.2016.05.005 Kirschbaum, C., Kudelka, B., Gaab, J., Schommer, N. C., & Hellhammer, D. (1999). Impact of gender, menstrual cycle phase, and oral contraceptives on the activity of the hypothalamus-pituitary-adrenal axis. Psychosomatic Medicine, 61, 154–162. https://doi.org/0033-3174/99/6102-0154 Kirschbaum, C., Wolf, O., May, M., Wippich, W., & Hellhammer, D. (1996). Stress- and treatment-induced elevations of cortisol levels associated with impaired declarative memory in healthy adults. Life Sciences, 58(17), 1475–1483. Klanker, M., Feenstra, M., & Denys, D. (2013). Dopaminergic control of cognitive flexibility in humans and animals. Frontiers in Neuroscience, 7, 1–24. https://doi.org/10.3389/fnins.2013.00201 Kohn, H. (1954). The effect of variations of intensity of experimentally induced stress situations upon certain aspects of perception and performance. The Journal of Genetic Psychology, 85, 289–304. Koob, G. F. (2008). Brain stress systems in the amygdala and addiction. Brain Research, 1293, 61–75. https://doi.org/10.1016/j.brainres.2009.03.038 Koob, G. F. (2010). The role of CRF and CRF-related peptides in the dark side of addiction. Brain Research, 1314, 3–14. https://doi.org/10.1016/j.brainres.2009.11.008 Koob, G. F., & Kreek, M. J. (2007). Stress dysregulation of drug reward pathways and the transition to drug dependence. American Journal of Psychiatry, (4), 1149–1159. Koot, S., Baars, A., Hesseling, P., Van Den Bos, R., & Joëls, M. (2013). Time-dependent effects of corticosterone on reward-based decision-making in a rodent model of the Iowa Gambling Task. Neuropharmacology, 70, 306–315. https://doi.org/10.1016/j.neuropharm.2013.02.008 Korotkova, T. M., Brown, R. E., Sergeeva, O. A., Ponomarenko, A. A., & Haas, H. L. (2006). Effects of arousal- and feeding-related neuropeptides on dopaminergic and GABAergic neurons in the ventral tegmental area of the rat. European Journal of Neuroscience, 23(10), 2677–2685. https://doi.org/10.1111/j.1460-9568.2006.04792.x Kreek, M. J., & Koob, G. (1998). Drug dependence: Stress and dysregulation of brain reward pathways. Drug and Alcohol Dependence, 51, 23–47. Retrieved from http://ovidsp.ovid.com/ovidweb.cgi?T=JS&PAGE=reference&D=emed4&NEWS=N&AN=1998256313 220 Krishnan, V., Han, M. H., Graham, D. L., Berton, O., Renthal, W., Russo, S. J., … Nestler, E. J. (2007). Molecular adaptations underlying susceptibility and resistance to social defeat in brain reward regions. Cell, 131(2), 391–404. https://doi.org/10.1016/j.cell.2007.09.018 Kubota, Y., Shigematsu, N., Karube, F., Sekigawa, A., Kato, S., Yamaguchi, N., … Kawaguchi, Y. (2011). Selective coexpression of multiple chemical markers defines discrete populations of neocortical gabaergic neurons. Cerebral Cortex, 21(8), 1803–1817. https://doi.org/10.1093/cercor/bhq252 Kuhlmann, S., Piel, M., & Wolf, O. T. (2005). Impaired memory retrieval after psychosocial stress in healthy young men. Journal of Neuroscience, 25(11), 2977–2982. https://doi.org/10.1523/JNEUROSCI.5139-04.2005 Kumar, P., Berghorst, L. H., Nickerson, L. D., Dutra, S. J., Goer, F. K., Greve, D. N., & Pizzagalli, D. A. (2014). Differential effects of acute stress on anticipatory and consummatory phases of reward processing. Neuroscience, 266, 1–12. https://doi.org/10.1016/j.neuroscience.2014.01.058 Lapiz-Bluhm, M. D. S., Soto-Piña, A. E., Hensler, J. G., & Morilak, D. A. (2009). Chronic intermittent cold stress and serotonin depletion induce deficits of reversal learning in an attentional set-shifting test in rats. Psychopharmacology, 202(1–3), 329–341. https://doi.org/10.1007/s00213-008-1224-6 Lapiz, M. D. S., & Morilak, D. A. (2006). Noradrenergic modulation of cognitive function in rat medial prefrontal cortex as measured by attentional set shifting capability. Neuroscience, 137(3), 1039–1049. https://doi.org/10.1016/j.neuroscience.2005.09.031 Laredo, S. A., Steinman, M. Q., Robles, C. F., Ferrer, E., Ragen, B. J., & Trainor, B. C. (2015). Effects of defeat stress on behavioral flexibility in males and females: Modulation by the mu-opioid receptor. European Journal of Neuroscience, 41(4), 434–441. https://doi.org/10.1111/ejn.12824 Larkin, J. D., Jenni, N. L., & Floresco, S. B. (2016). Modulation of risk/reward decision making by dopaminergic transmission within the basolateral amygdala. Psychopharmacology, 233(1), 121–136. https://doi.org/10.1007/s00213-015-4094-8 Lee, E. H. Y., & Sung, Y. J. (1989). Differential influences of corticotropin-releasing factor on memory retention of aversive learning and appetitive learning in rats. Behavioral and Neural Biology, 52(3), 285–294. https://doi.org/10.1016/S0163-1047(89)90412-3 Lee, J., & Park, S. (2005). Working memory impairments in schizophrenia: A meta-analysis. Journal of Abnormal Psychology. Special Issue: Toward a Dimensionally Based Taxonomy of Psychopathology, 114(4), 599–611. https://doi.org/10.1037/0021-843X.114.4.599 Lee, R. S. C., Hermens, D. F., Porter, M. A., & Redoblado-Hodge, M. A. (2012). A meta-analysis of cognitive deficits in first-episode Major Depressive Disorder. Journal of Affective Disorders, 140(2), 113–124. https://doi.org/10.1016/j.jad.2011.10.023 221 Lemke, M. R., Puhl, P., Koethe, N., & Winkler, T. (1999). Psychomotor retardation and anhedonia in depression. Acta Psychiatrica Scandinavica, 99(4), 252–256. Lemmens, S. G., Rutters, F., Born, J. M., & Westerterp-Plantenga, M. S. (2011). Stress augments food “wanting” and energy intake in visceral overweight subjects in the absence of hunger. Physiology and Behavior, 103(2), 157–163. https://doi.org/10.1016/j.physbeh.2011.01.009 Lemos, J. C., Wanat, M. J., Smith, J. S., Reyes, B. A. S., Hollon, N. G., Van Bockstaele, E. J., … Phillips, P. E. M. (2012). Severe stress switches CRF action in the nucleus accumbens from appetitive to aversive. Nature, 490(7420), 402–406. https://doi.org/10.1038/nature11436 Liang, K. C., & Lee, E. H. Y. (1988). Intra-amygdala injections of corticotropin releasing factor facilitate inhibitory avoidance learning and reduce exploratory behavior in rats. Psychopharmacology, 96(2), 232–236. https://doi.org/10.1007/BF00177566 Lighthall, N. R., Mather, M., & Gorlick, M. A. (2009). Acute stress increases sex differences in risk seeking in the Balloon Analogue Risk Task. PLoS ONE, 4(7). https://doi.org/10.1371/journal.pone.0006002 Lim, M. M., Liu, Y., Ryabinin, A. E., Bai, Y., Wang, Z., & Young, L. J. (2007). CRF receptors in the nucleus accumbens modulate partner preference in prairie voles. Hormones and Behavior, 51(4), 508–515. https://doi.org/10.1016/j.yhbeh.2007.01.006 Liston, C., McEwen, B. S., & Casey, B. J. (2009). Psychosocial stress reversibly disrupts prefrontal processing and attentional control. Proceedings of the National Academy of Sciences, 106(3), 912–917. https://doi.org/10.1073/pnas.0807041106 Liu, R.-J., Ota, K. T., Dutheil, S., Duman, R. S., & Aghajanian, G. K. (2015). Ketamine strengthens CRF-activated amygdala inputs to basal dendrites in mPFC layer V pyramidal cells in the prelimbic but not infralimbic subregion, a key suppressor of stress responses. Neuropsychopharmacology, 40(9), 2066–2075. https://doi.org/10.1038/npp.2015.70 Liu, W. H., Wang, L. zhi, Shang, H. rui, Shen, Y., Li, Z., Cheung, E. F. C., & Chan, R. C. K. (2014). The influence of anhedonia on feedback negativity in major depressive disorder. Neuropsychologia, 53(1), 213–220. https://doi.org/10.1016/j.neuropsychologia.2013.11.023 Liu, X. (2015). Enhanced motivation for food reward induced by stress and attenuation by corticotrophin-releasing factor receptor antagonism in rats: Implications for overeating and obesity. Psychopharmacology, 232(12), 2049–2060. https://doi.org/10.1007/s00213-014-3838-1 Livezey, G. T., Miller, J. M., & Vogel, W. H. (1985). Plasma norepinephrine, epinephrine and corticosterone stress responses to restraint in individual male and female rats, and their correlations. Neuroscience Letters, 62(1), 51–56. https://doi.org/10.1016/0304-3940(85)90283-6 Lodge, D. J., & Grace, A. A. (2005). Acute and chronic corticotropin-releasing factor 1 receptor blockade inhibits cocaine-induced dopamine release: correlation with dopamine neuron 222 activity. The Journal of Pharmacology and Experimental Therapeutics, 314(1), 201–206. https://doi.org/10.1124/jpet.105.084913.of Lodge, D. J., & Grace, A. A. (2006a). The hippocampus modulates dopamine neuron responsivity by regulating the intensity of phasic neuron activation. Neuropsychopharmacology, 31(7), 1356–1361. https://doi.org/10.1038/sj.npp.1300963 Lodge, D. J., & Grace, A. A. (2006b). The laterodorsal tegmentum is essential for burst firing of ventral tegmental area dopamine neurons. Proceedings of the National Academy of Sciences, 103(13), 5167–5172. https://doi.org/10.1073/pnas.0510715103 Lovenberg, T. W., Liaw, C. W., Grigoriadis, D. E., Clevenger, W., Chalmers, D. T., De Souza, E. B., & Oltersdorf, T. (1995). Cloning and characterization of a functionally distinct corticotropin-releasing factor receptor subtype from rat brain. Proceedings of the National Academy of Sciences of the United States of America, 92(3), 836–840. https://doi.org/10.1073/pnas.92.3.836 Lowry, C. A., Rodda, J. E., Lightman, S. L., & Ingram, C. D. (2000). Corticotropin-releasing factor increases in vitro firing rates of serotonergic neurons in the rat dorsal raphe nucleus: Evidence for activation of a topographically organized mesolimbocortical serotonergic system. The Journal of Neuroscience, 20(20), 7728–7736. https://doi.org/20/20/7728 [pii] Luethi, M. (2009). Stress effects on working memory, explicit memory, and implicit memory for neutral and emotional stimuli in healthy men. Frontiers in Behavioral Neuroscience, 2, 1–9. https://doi.org/10.3389/neuro.08.005.2008 Ma, Y., Chen, K., Wei, C., & Lee, E. (1999). Corticotropin-releasing factor enhances brain-derived neurotrophic factor gene expression to facilitate memory retention in rats. Chinese Journal of Physiology, 42(2), 73–81. Maki, P. M., Mordecai, K. L., Rubin, L. H., Sundermann, E., Savarese, A., Eatough, E., & Drogos, L. (2015). Menstrual cycle effects on cortisol responsivity and emotional retrieval following a psychosocial stressor. Hormones and Behavior, 74, 201–208. https://doi.org/10.1016/j.yhbeh.2015.06.023 Mamalaki, E., Kvetnansky, R., Brady, L. S., Gold, P. W., & Herkenham, M. (1992). Repeated immobilization stress alters tyrosine hydroxylase, corticotropin‐releasing hormone and corticosteroid receptor messenger ribonucleic acid levels in rat brain. Journal of Neuroendocrinology, 4(6), 689–699. https://doi.org/10.1111/j.1365-2826.1992.tb00220.x Mantsch, J. R., Baker, D. A., Francis, D. M., Katz, E. S., Hoks, M. A., & Serge, J. P. (2008). Stressor- and corticotropin releasing factor-induced reinstatement and active stress-related behavioral responses are augmented following long-access cocaine self-administration by rats. Psychopharmacology, 195(4), 591–603. https://doi.org/10.1007/s00213-007-0950-5 Mantsch, J. R., Baker, D. A., Funk, D., Lê, A. D., & Shaham, Y. (2016). Stress-induced reinstatement of drug seeking: 20 years of progress. Neuropsychopharmacology, 41(1), 335–356. https://doi.org/10.1038/npp.2015.142 223 Marinelli, M., Aouizerate, B., Barrot, M., Le Moal, M., & Piazza, P. V. (1998). Dopamine-dependent responses to morphine depend on glucocorticoid receptors. Proceedings of the National Academy of Sciences, 95(13), 7742–7747. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/9636221%0Ahttp://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=PMC22744 Marinelli, M., & Piazza, P. V. (2002). Interaction between glucocorticoid hormones, stress and psychostimulant drugs. European Journal of Neuroscience, 16(3), 387–394. https://doi.org/10.1046/j.1460-9568.2002.02089.x Maroun, M., & Akirav, I. (2008). Arousal and stress effects on consolidation and reconsolidation of recognition memory. Neuropsychopharmacology, 33(2), 394–405. https://doi.org/10.1038/sj.npp.1301401 Martins, J., Kastin, A., & Banks, W. (1996). Unidirectional specific and modulated brain to blood transport of corticotropin-releasing hormone. Neuroendocrinology, 63(4), 338–348. https://doi.org/10.1159/000126974 Mather, M., & Lighthall, N. R. (2012). Risk and reward are processed differently in decisions made under stress. Current Directions in Psychological Science, 21(1), 36–41. https://doi.org/10.1177/0963721411429452 Matsuzaki, I., Takamatsu, Y., & Moroji, T. (1989). The effects of intracerebroventricularly injected corticotropin-releasing factor (CRF) on the central nervous system: Behavioural and biochemical studies. Neuropeptides, 13(3), 147–155. https://doi.org/10.1016/0143-4179(89)90085-1 McAlonan, K., & Brown, V. J. (2003). Orbital prefrontal cortex mediates reversal learning and not attentional set shifting in the rat. Behavioural Brain Research, 146(1–2), 97–103. https://doi.org/10.1016/j.bbr.2003.09.019 McEwen, B. S., & Milner, T. A. (2007). Hippocampal formation: Shedding light on the influence of sex and stress on the brain. Brain Research Reviews, 55(2 SPEC. ISS.), 343–355. https://doi.org/10.1016/j.brainresrev.2007.02.006 McFarland, B. R., & Klein, D. N. (2009). Emotional reactivity in depression: Diminished responsiveness to anticipated reward but not to anticipated punishment or to nonreward or avoidance. Depression and Anxiety, 26(2), 117–122. https://doi.org/10.1002/da.20513 McGovern, A. R., Alexopoulos, G. S., Yuen, G. S., Morimoto, S. S., & Gunning-Dixon, F. M. (2014). Reward-related decision making in older adults: Relationship to clinical presentation of depression. International Journal of Geriatric Psychiatry, 29(11), 1125–1131. https://doi.org/10.1002/gps.4200 Meador-Woodruff, J. H., Mansour, A., Bunzow, J. R., Van Tol, H. H., Watson Jr., S. J., & Civelli, O. (1989). Distribution of D2 dopamine receptor mRNA in rat brain. Proceedings of the National Academy of Sciences, 86(19), 7625–7628. https://doi.org/http://www.ncbi.nlm.nih.gov/pubmed/10723008 224 Merchenthaler, I. (1984). Corticotropin releasing factor (CRF)-like immunoreactivity in the rat central nervous system. Extrahypothalamic distribution. Peptides, 5, 53–69. https://doi.org/10.1016/0196-9781(84)90265-1 Millan, M., Samara, A., Wynn, P., Catt, K., & Aguilera, G. (1987). Receptors and actions of corticotropin-releasing hormone in the primate pituitary gland. The Journal of Clinical Endocrinology and Metabolism, 64(5), 1036–1041. https://doi.org/10.1210/jcem-64-5-1036 Mitra, R., Jadhav, S., McEwen, B. S., Vyas, A., & Chattarji, S. (2005). Stress duration modulates the spatiotemporal patterns of spine formation in the basolateral amygdala. Proceedings of the National Academy of Sciences of the United States of America, 102(26), 9371–9376. https://doi.org/10.1073/pnas.0504011102 Mizoguchi, K., Yuzurihara, M., Ishige, A., Sasaki, H., Chui, D. H., & Tabira, T. (2000). Chronic stress induces impairment of spatial working memory because of prefrontal dopaminergic dysfunction. The Journal of Neuroscience, 20(4), 1568–1574. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/10662846 Moffett, M. C., & Goeders, N. E. (2007). CP-154,526, a CRF type-1 receptor antagonist, attenuates the cue-and methamphetamine-induced reinstatement of extinguished methamphetamine-seeking behavior in rats. Psychopharmacology, 190(2), 171–180. https://doi.org/10.1007/s00213-006-0625-7 Moniz, M., De Jesus, S. N., Gonçalves, E., Pacheco, A., & Viseu, J. (2016). Decision-making in adult unipolar depressed patients and healthy subjects: Significant differences in Net Score and in non-traditional alternative measures. Neuropsychological Trends, 19(1), 7–15. https://doi.org/10.7358/neur-2016-019-moni Montes, D. R., Stopper, C. M., & Floresco, S. B. (2015). Noradrenergic modulation of risk/reward decision making. Psychopharmacology, 232(15), 2681–2696. https://doi.org/10.1007/s00213-015-3904-3 Morales, M., & Margolis, E. B. (2017). Ventral tegmental area: Cellular heterogeneity, connectivity and behaviour. Nature Reviews Neuroscience, 18(2), 73–85. https://doi.org/10.1038/nrn.2016.165 Morgado, P., Marques, F., Ribeiro, B., Leite-Almeida, H., Pêgo, J. M., Rodrigues, A. J., … Cerqueira, J. J. (2015). Stress induced risk-aversion is reverted by D2/D3 agonist in the rat. European Neuropsychopharmacology, 25(10), 1744–1752. https://doi.org/10.1016/j.euroneuro.2015.07.003 Morilak, D. A., Barrera, G., Echevarria, D. J., Garcia, A. S., Hernandez, A., Ma, S., & Petre, C. O. (2005). Role of brain norepinephrine in the behavioral response to stress. Progress in Neuro-Psychopharmacology and Biological Psychiatry, 29(8), 1214–1224. https://doi.org/10.1016/j.pnpbp.2005.08.007 Müller, M. B., Zimmermann, S., Sillaber, I., Hagemeyer, T. P., Deussing, J. M., Timpl, P., … Wurst, W. (2003). Limbic corticotropin-releasing hormone receptor 1 mediates anxiety- 225 related behavior and hormonal adaptation to stress. Nature Neuroscience, 6(10), 1100–1107. https://doi.org/10.1038/nn1123 Murphy, F. C., Rubinsztein, J. S., Michael, A., Rogers, R. D., Robbins, T. W., Paykel, E. S., & Sahakian, B. J. (2001). Decision-making cognition in mania and depression. Psychological Medecine, 31(4), 679–693. Murphy, F., Michael, A., Robbins, T., & Sahakian, B. (2003). Neuropsychological impairment in patients with major depressive disorder: The effects of feedback on task performance. Psychological Medicine, 33, 455–467. Must, A., Szabó, Z., Bódi, N., Szász, A., Janka, Z., & Kéri, S. (2006). Sensitivity to reward and punishment and the prefrontal cortex in major depression. Journal of Affective Disorders, 90(2–3), 209–215. https://doi.org/10.1016/j.jad.2005.12.005 Nelson, B. D., Shankman, S. A., & Proudfit, G. H. (2014). Intolerance of uncertainty mediates reduced reward anticipation in major depressive disorder. Journal of Affective Disorders, 158, 108–113. https://doi.org/10.1016/j.jad.2014.02.014 Nemeroff, C. B., Widerlöv, E., Bissette, G., Walléus, H., Karlsson, I., Eklund, K., … Vale, W. (1984). Elevated concentrations of CSF corticotropin-releasing factor-like immunoreactivitity in depressed patients. Science, 226(4680), 1342–1344. Nieoullon, A. (2002). Dopamine and the regulation of cognition and attention. Progress in Neurobiology, 67, 53–83. Retrieved from http://www.sciencedirect.com/science/article/pii/s0301008202000114%5Cnpapers2://publication/uuid/19BE4726-ED71-4ED6-8EE3-EF0EF142F2CB Nobrega, J. N., Hedayatmofidi, P. S., & Lobo, D. S. (2016). Strong interactions between learned helplessness and risky decision-making in a rat gambling model. Scientific Reports, 6, 4–12. https://doi.org/10.1038/srep37304 Nowend, K. L., Arizzi, M., Carlson, B. B., & Salamone, J. D. (2001). D1 or D2 antagonism in nucleus accumbens core or dorsomedial shell suppresses lever pressing for food but leads to compensatory increases in chow consumption. Pharmacology Biochemistry and Behavior, 69(3–4), 373–382. https://doi.org/10.1016/S0091-3057(01)00524-X Nyberg, L. (2018). Cognitive control in the prefrontal cortex: A central or distributed executive? Scandinavian Journal of Psychology, 59(1), 62–65. https://doi.org/10.1111/sjop.12409 Ochedalski, T., Subburaju, S., Wynn, P. C., & Aguilera, G. (2007). Interaction between oestrogen and oxytocin on hypothalamic-pituitary-adrenal axis activity. Journal of Neuroendocrinology, 19(3), 189–197. https://doi.org/10.1111/j.1365-2826.2006.01525.x Ohmura, Y., Yamaguchi, T., Futami, Y., Togashi, H., Izumi, T., Matsumoto, M., … Yoshioka, M. (2009). Corticotropin releasing factor enhances attentional function as assessed by the five-choice serial reaction time task in rats. Behavioural Brain Research, 198(2), 429–433. https://doi.org/10.1016/j.bbr.2008.11.025 226 Olausson, P., Kiraly, D. D., Gourley, S. L., & Taylor, J. R. (2013). Persistent effects of prior chronic exposure to corticosterone on reward-related learning and motivation in rodents. Psychopharmacology, 225(3), 569–577. https://doi.org/10.1007/s00213-012-2844-4. Oliver, G., Wardle, J., & Gibson, E. L. (2000). Stress and food choice: A laboratory study. Psychosomatic Medicine, 62(6), 853–865. https://doi.org/10.1097/00006842-200011000-00016 Olver, J. S., Pinney, M., Maruff, P., & Norman, T. R. (2015). Impairments of spatial working memory and attention following acute psychosocial stress. Stress and Health, 31(2), 115–123. https://doi.org/10.1002/smi.2533 Orsini, C. A., & Setlow, B. (2017). Sex differences in animal models of decision making. Journal of Neuroscience Research, 95(1–2), 260–269. https://doi.org/10.1002/jnr.23810 Orsini, C. A., Willis, M. L., Gilbert, R. J., Bizon, J. L., & Setlow, B. (2016). Sex differences in a rat model of risky decision making. Behavioral Neuroscience, 130(1), 50–61. https://doi.org/10.1037/bne0000111 Overman, W. H., & Pierce, A. (2013). Iowa Gambling Task with non-clinical participants: Effects of using real + virtual cards and additional trials. Frontiers in Psychology, 4, 1–15. https://doi.org/10.3389/fpsyg.2013.00935 Papaleo, F., Erickson, L., Liua, G., Chena, J., & Weinberger, D. R. (2012). Effects of sex and COMT genotype on environmentally modulated cognitive control in mice. Proceedings of the National Academy of Sciences of the United States of America, 109(49), 20160–20165. https://doi.org/10.1073/pnas.1214397109 Papp, M., Willner, P., & Muscat, R. (1991). An animal model of anhedonia: Attenuation of sucrose consumption and place preference conditioning by chronic unpredictable mild stress. Psychopharmacology, 104(2), 255–259. https://doi.org/10.1007/BF02244188 Patton, M. H., Bizup, B. T., & Grace, A. A. (2013). The infralimbic cortex bidirectionally modulates mesolimbic dopamine neuron activity via distinct neural pathways. Journal of Neuroscience, 33(43), 16865–16873. https://doi.org/10.1523/JNEUROSCI.2449-13.2013 Paxinos, G., & Watson, C. (2013). The rat brain in stereotaxic coordinates (7th ed.). New York. Peciña, S., Schulkin, J., & Berridge, K. C. (2006). Nucleus accumbens corticotropin-releasing factor increases cue-triggered motivation for sucrose reward: Paradoxical positive incentive effects in stress? BMC Biology, 4, 8. https://doi.org/10.1186/1741-7007-4-8 Petzold, A., Plessow, F., Goschke, T., & Kirschbaum, C. (2010). Stress reduces use of negative feedback in a feedback-based learning task. Behavioral Neuroscience, 124(2), 248–255. https://doi.org/10.1037/a0018930 Pizzagalli, D. A., Holmes, A. J., Dillon, D. G., Goetz, E. L., Birk, J. L., Bogdan, R., … Fava, M. (2009). Reduced caudate and nucleus accumbens response to rewards in unmedicated 227 subjects with major depressive disorder. American Journal of Psychiatry, 166, 702–710. https://doi.org/10.1176/appi.ajp.2008.08081201.Reduced Pizzagalli, D. A., Jahn, A. L., & O’Shea, J. P. (2005). Toward an objective characterization of an anhedonic phenotype: A signal-detection approach. Biological Psychiatry, 57(4), 319–327. https://doi.org/10.1016/j.biopsych.2004.11.026 Plager, J., Schmidt, K., & Staubitz, W. (1964). Increased unbound cortisol in the plasma of estrogen-treated subjects. The Journal of Clinical Investigation, 43(6), 1066–1072. https://doi.org/10.1172/JCI104990 Pletzer, B., Harris, T. A., & Ortner, T. (2017). Sex and menstrual cycle influences on three aspects of attention. Physiology and Behavior, 179, 384–390. https://doi.org/10.1016/j.physbeh.2017.07.012 Porcelli, A. J., Lewis, A. H., & Delgado, M. R. (2012). Acute stress influences neural circuits of reward processing. Frontiers in Neuroscience, 6, 1–9. https://doi.org/10.3389/fnins.2012.00157 Preston, S. D., Buchanan, T. W., Stansfield, R. B., & Bechara, A. (2007). Effects of anticipatory stress on decision making in a gambling task. Behavioral Neuroscience, 121(2), 257–263. https://doi.org/10.1037/0735-7044.121.2.257 Puglisi-Allegra, S., Imperato, A., Angelucci, L., & Cabib, S. (1991). Acute stress induces time-dependent responses in dopamine mesolimbic system. Brain Research, 554(1–2), 217–222. https://doi.org/10.1016/0006-8993(91)90192-X Puig, M. V., Antzoulatos, E. G., & Miller, E. K. (2014). Prefrontal dopamine in associative learning and memory. Neuroscience, 282, 217–229. https://doi.org/10.1016/j.neuroscience.2014.09.026 Pulcu, E., Trotter, P. D., Thomas, E. J., McFarquhar, M., Juhasz, G., Sahakian, B. J., … Elliott, R. (2014). Temporal discounting in major depressive disorder. Psychological Medicine, 44(9), 1825–1834. https://doi.org/10.1017/S0033291713002584 Putman, P., Antypa, N., Crysovergi, P., & Van Der Does, W. A. J. (2010). Exogenous cortisol acutely influences motivated decision making in healthy young men. Psychopharmacology, 208(2), 257–263. https://doi.org/10.1007/s00213-009-1725-y Radulovic, J., Rü, A., Liepold, T., & Spiess, J. (1999). Modulation of learning and anxiety by corticotropin-releasing factor (CRF) and stress: Differential roles of CRF Receptors 1 and 2. Journal of Neuroscience, 19(12), 5016–5025. Rahman, Q., Abrahams, S., & Jussab, F. (2005). Sex differences in a human analogue of the Radial Arm Maze: The “17-Box Maze Test.” Brain and Cognition, 58(3), 312–317. https://doi.org/10.1016/j.bandc.2005.03.001 Ravnkilde, B., Videbech, P., Clemmensen, K., Egander, A., Rasmussen, N. A., & Rosenberg, R. 228 (2002). Cognitive deficits in major depression. Scandinavian Journal of Psychology, 43(3), 239–251. https://doi.org/10.1111/1467-9450.00292 Redgrave, P., Prescott, T. J., & Gurney, K. (1999). Is the short latency dopamine burst too short to signal reinforcement error? Trends in Neuroscience, 22(4), 146–151. https://doi.org/10.1016/S0166-2236(98)01373-3 Reppermund, S., Ising, M., Lucae, S., & Zihl, J. (2009). Cognitive impairment in unipolar depression is persistent and non-specific: Further evidence for the final common pathway disorder hypothesis. Psychological Medicine, 39(4), 603–614. https://doi.org/10.1017/S003329170800411X Reul, J. M. H. M., & de Kloet, E. R. (1985). Two receptor systems for corticosterone in rat brain: Microdistribution and differential occupation. Endocrinology, 117(6), 2505–2511. https://doi.org/https://doi-org.ezproxy.library.ubc.ca/10.1210/endo-117-6-2505 Revusky, S., & Reilly, S. (1989). Attenuation of conditioned taste aversions by external stressors. Pharmacology, Biochemistry and Behavior, 33(1), 219–226. https://doi.org/10.1016/0091-3057(89)90453-X Riegel, A. C., & Williams, J. T. (2008). CRF facilitates calcium release from intracellular stores in midbrain dopamine neurons. Neuron, 57(4), 559–570. https://doi.org/10.1016/j.neuron.2007.12.029 Rincón-Cortés, M., & Grace, A. A. (2017). Sex-dependent effects of stress on immobility behavior and VTA dopamine neuron activity: Modulation by ketamine. International Journal of Neuropsychopharmacology, 20(10), 823–832. https://doi.org/10.1093/ijnp/pyx048 Rodaros, D., Caruana, D. A., Amir, S., & Stewart, J. (2007). Corticotropin-releasing factor projections from limbic forebrain and paraventricular nucleus of the hypothalamus to the region of the ventral tegmental area. Neuroscience, 150(1), 8–13. https://doi.org/10.1016/j.neuroscience.2007.09.043 Roozendaal, B. (2004). The basolateral amygdala interacts with the medial prefrontal cortex in regulating glucocorticoid effects on working memory impairment. Journal of Neuroscience, 24(6), 1385–1392. https://doi.org/10.1523/jneurosci.4664-03.2004 Roozendaal, B., Brunson, K. L., Holloway, B. L., McGaugh, J. L., & Baram, T. Z. (2002). Involvement of stress-released corticotropin-releasing hormone in the basolateral amygdala in regulating memory consolidation. Proceedings of the National Academy of Sciences of the United States of America, 99(21), 13908–13913. https://doi.org/10.1073/pnas.212504599 Roozendaal, B., Griffith, Q. K., Buranday, J., De Quervain, D. J. F., & McGaugh, J. L. (2003). The hippocampus mediates glucocorticoid-induced impairment of spatial memory retrieval: Dependence on the basolateral amygdala. Proceedings of the National Academy of Sciences of the United States of America, 100(3), 1328–1333. 229 https://doi.org/10.1073/pnas.0337480100 Roozendaal, B., McEwen, B. S., & Chattarji, S. (2009). Stress, memory and the amygdala. Nature Reviews Neuroscience, 10(6), 423–433. https://doi.org/10.1038/nrn2651 Roozendaal, B., & McGaugh, J. L. (1996). Amygdaloid nuclei lesions differentially affect glucocorticoid-induced memory enhancement in an inhibitory avoidance task. Neurobiology of Learning and Memory, 65(1), 1–8. https://doi.org/10.1006/nlme.1996.0001 Roozendaal, B., Okuda, S., Van Der Zee, E. A., & McGaugh, J. L. (2006). Glucocorticoid enhancement of memory requires arousal-induced noradrenergic activation in the basolateral amygdala. Proceedings of the National Academy of Sciences of the United States of America, 103(17), 6741–6746. https://doi.org/10.1073/pnas.0601874103 Rubinsztein, J. S., Michael, A., Paykel, E. S., & Sahakian, B. J. (2000). Cognitive impairment in remission in bipolar affective disorder. Psychological Medicine, 30(5), 1025–1036. https://doi.org/10.1017/S0033291799002664 Rubinsztein, J. S., Michael, A., Underwood, B. R., Tempest, M., & Sahakain, B. J. (2006). Impaired cognition and decision-making in bipolar depression but no “affective bias” evident. Psychological Medicine, 36(5), 629–639. https://doi.org/10.1017/S0033291705006689 Rutters, F., Nieuwenhuizen, A. G., Lemmens, S. G. T., Born, J. M., & Westerterp-Plantenga, M. S. (2009). Acute stress-related changes in eating in the absence of hunger. Obesity, 17(1), 72–77. https://doi.org/10.1038/oby.2008.493 Salamone, J., Cousins, M., McCullough, L., Carriero, D., & Berkowitz, R. (1994). Nucleus accumbens dopamine release increases during instrumental lever pressiing for food but not free food consumption. Pharmacological Biochemical Behavior, 49(1), 25–31. Salamone, J. D., & Correa, M. (2012). The mysterious motivational functions of mesolimbic dopamine. Neuron, 76(3), 470–485. https://doi.org/10.1016/j.neuron.2012.10.021 Salamone, J. D., Correa, M., Nunes, E. J., Randall, P. A., & Pardo, M. (2012). The behavioral pharmacology of effort-related choice behavior: Dopamine, adenosine and beyond. Journal of the Experimental Analysis of Behavior, 97(1), 125–146. https://doi.org/10.1901/jeab.2012.97-125 Salamone, J. D., Cousins, M. S., & Bucher, S. (1994). Anhedonia or anergia effects of haloperidol and nucleus accumbens dopamine depletion on instrumental response selection in a T-maze cost benefit procedure. Behavioural Brain Research, 65, 221–229. Salamone, J. D., Koychev, I., Correa, M., & Mcguire, P. (2015). Neurobiological basis of motivational deficits in psychopathology. European Neuropsychopharmacology, 25(8), 1225–1238. https://doi.org/10.1016/j.euroneuro.2014.08.014 Salamone, J. D., Steinpreis, R., McCullough, L., Smith, P., Grebel, D., & Mahan, K. (1991). 230 Haloperidol and nucleus accumbens dopamine depletion suppresses lever pressing for food but increase free food consumption in a novel food-choice procedure. Psychopharmacology, 104, 515–521. Salvatore, M., Wiersielis, K. R., Luz, S., Waxler, D. E., Bhatnagar, S., & Bangasser, D. A. (2018). Sex differences in circuits activated by corticotropin releasing factor in rats. Hormones and Behavior, 97, 145–153. https://doi.org/10.1016/j.yhbeh.2017.10.004 Sänger, J., Bechtold, L., Schoofs, D., Blaszkewicz, M., & Wascher, E. (2014). The influence of acute stress on attention mechanisms and its electrophysiological correlates. Frontiers in Behavioral Neuroscience, 8, 1–13. https://doi.org/10.3389/fnbeh.2014.00353 Sapolsky, R. M. (2003). Stress and plasticity in the limbic system. Neurochemical Research, 28(11), 1735–1742. Retrieved from http://sci-hub.bz/10.1023/A:1026021307833 Sautel, F., Griffon, N., Lévesque, D., Pilon, C., Schwartz, J. C., & Sokoloff, P. (1995). A functional test identifies dopamine agonists selective for D3 versus D2 receptors. Neuroreport. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/7756621 Schoofs, D., Pabst, S., Brand, M., & Wolf, O. T. (2013). Working memory is differentially affected by stress in men and women. Behavioural Brain Research, 241(1), 144–153. https://doi.org/10.1016/j.bbr.2012.12.004 Schoofs, D., Preuß, D., & Wolf, O. T. (2008). Psychosocial stress induces working memory impairments in an n-back paradigm. Psychoneuroendocrinology, 33(5), 643–653. https://doi.org/10.1016/j.psyneuen.2008.02.004 Schoofs, D., & Wolf, O. T. (2009). Stress and memory retrieval in women: No strong impairing effect during the luteal phase. Behavioral Neuroscience, 123(3), 547–554. https://doi.org/10.1037/a0015625 Schrijvers, D., de Bruijn, E. R. A., Maas, Y., De Grave, C., Sabbe, B. G. C., & Hulstijn, W. (2008). Action monitoring in major depressive disorder with psychomotor retardation. Cortex, 44(5), 569–579. https://doi.org/10.1016/j.cortex.2007.08.014 Schwabe, L., Bohringer, A., Chatterjee, M., & Schachinger, H. (2008). Effects of pre-learning stress on memory for neutral, positive and negative words: Different roles of cortisol and autonomic arousal. Neurobiology of Learning and Memory, 90(1), 44–53. https://doi.org/10.1016/j.nlm.2008.02.002 Schwabe, L., & Wolf, O. T. (2010). Stress impairs the reconsolidation of autobiographical memories. Neurobiology of Learning and Memory, 94(2), 153–157. https://doi.org/10.1016/j.nlm.2010.05.001 Selye, H. (1936). A syndrome produced by diverse nocuous agents. Nature, 1936(1), 32. https://doi.org/10.1176/jnp.10.2.230a Serfling, G., Buades-Rotger, M., Harbeck, B., Krämer, U. M., & Brabant, G. (2019). The 231 corticosteroid prednisolone increases amygdala and insula reactivity to food approach signals in healthy young men. Psychoneuroendocrinology, 99, 154–165. https://doi.org/10.1016/j.psyneuen.2018.09.007 Seymoure, P., Dou, H., & Juraska, J. M. (1996). Sex differences in radial maze performance: Influence of rearing environment and room cues. Psychobiology, 24(1), 33–37. Shafiei, N., Gray, M., Viau, V., & Floresco, S. B. (2012). Acute stress induces selective alterations in cost/benefit decision-making. Neuropsychopharmacology, 37(10), 2194–2209. https://doi.org/10.1038/npp.2012.69 Shaham, Y., & Stewart, J. (1995). Stress reinstates heroin-seeking in drug-free animals: An effect mimicking heroin, not withdrawal. Psychopharmacology, 119(3), 334–341. https://doi.org/10.1007/BF02246300 Shaham, Yavin, Erb, S., Leung, S., Buczek, Y., & Stewart, J. (1998). CP-154,526, a selective, non-peptide antagonist of the corticotropin-releasing factor1 receptor attenuates stress-induced relapse to drug seeking in cocaine- and heroin-trained rats. Psychopharmacology, 137(2), 184–190. https://doi.org/10.1007/s002130050608 Shaham, Yavin, Funk, D., Erb, S., Brown, T. J., Walker, C. D., & Stewart, J. (1997). Corticotropin-releasing factor, but not corticosterone, is involved in stress-induced relapse to heroin-seeking in rats. Journal of Neuroscience, 17(7), 2605–2614. Shalev, U., Erb, S., & Shaham, Y. (2010). Role of CRF and other neuropeptides in stress-induced reinstatement of drug seeking. Brain Research, 1314, 15–28. https://doi.org/10.1016/j.brainres.2009.07.028 Shansky, R. M., Rubinow, K., Brennan, A., & Arnsten, A. F. T. (2006). The effects of sex and hormonal status on restraint-stress-induced working memory impairment. Behavioral and Brain Functions, 2, 4–9. https://doi.org/10.1186/1744-9081-2-8 Shields, G. S., Bonner, J. C., & Moons, W. G. (2015). Does cortisol influence core executive functions? A meta-analysis of acute cortisol administration effects on working memory, inhibition, and set-shifting. Psychoneuroendocrinology, 58, 91–103. https://doi.org/10.1016/j.psyneuen.2015.04.017 Shields, G. S., Sazma, M. A., & Yonelinas, A. P. (2016). The effects of acute stress on core executive functions: A meta-analysis and comparison with cortisol. Neuroscience and Biobehavioral Reviews, 68, 651–668. https://doi.org/10.1016/j.neubiorev.2016.06.038 Shields, G. S., Trainor, B. C., Lam, J. C. W., & Yonelinas, A. P. (2016). Acute stress impairs cognitive flexibility in men, not women. Stress, 19(5), 542–546. https://doi.org/10.1080/10253890.2016.1192603 Sidlauskaite, J., González-Madruga, K., Smaragdi, A., Riccelli, R., Puzzo, I., Batchelor, M., … Fairchild, G. (2018). Sex differences in risk-based decision making in adolescents with conduct disorder. European Child and Adolescent Psychiatry, 27(9), 1133–1142. 232 https://doi.org/10.1007/s00787-017-1024-9 Simonovic, B., Stupple, E. J. N., Gale, M., & Sheffield, D. (2017). Stress and risky decision Making: Cognitive reflection, emotional learning or both. Journal of Behavioral Decision Making, 30(2), 658–665. https://doi.org/10.1002/bdm.1980 Simpson, E. H., Winiger, V., Biezonski, D. K., Haq, I., Kandel, E. R., & Kellendonk, C. (2014). Selective overexpression of dopamine D3 receptors in the striatum disrupts motivation but not cognition. Biological Psychiatry, 76(10), 823–831. https://doi.org/10.1016/j.biopsych.2013.11.023 Singh, V. B., Hao-Phan, T., Corley, K. C., & Boadle-Biber, M. C. (1992). Increase in cortical and midbrain tryptophan hydroxylase activity by intracerebroventricular administration of corticotropin releasing factor: block by adrenalectomy, by RU 38486 and by bilateral lesions to the central nucleus of the amygdala. Neurochemistry International, 20(1), 81–92. https://doi.org/10.1016/0197-0186(92)90129-F Skosnik, P. D., Chatterton, R. T., Swisher, T., & Park, S. (2000). Modulation of attentional inhibition by norepinephrine and cortisol after psychological stress. International Journal of Psychophysiology, 36(1), 59–68. https://doi.org/10.1016/S0167-8760(99)00100-2 Smeets, T., Giesbrecht, T., Jelicic, M., & Merckelbach, H. (2007). Context-dependent enhancement of declarative memory performance following acute psychosocial stress. Biological Psychology, 76(1–2), 116–123. https://doi.org/10.1016/j.biopsycho.2007.07.001 Smeets, Tom, Otgaar, H., Candel, I., & Wolf, O. T. (2008). True or false? Memory is differentially affected by stress-induced cortisol elevations and sympathetic activity at consolidation and retrieval. Psychoneuroendocrinology, 33(10), 1378–1386. https://doi.org/10.1016/j.psyneuen.2008.07.009 Smoski, M. J., Lynch, T. R., Rosenthal, M. Z., Cheavens, J. S., Chapman, A. L., & Krishnan, R. R. (2008). Decision-making and risk aversion among depressive adults. Journal of Behavior Therapy and Experimental Psychiatry, 39(4), 567–576. https://doi.org/10.1016/j.jbtep.2008.01.004 Snyder, K. P., Hill-Smith, T. E., Lucki, I., & Valentino, R. J. (2015). Corticotropin-releasing factor in the rat dorsal raphe nucleus promotes different forms of behavioral flexibility depending on social stress history. Neuropsychopharmacology, 40(11), 2517–2525. https://doi.org/10.1038/npp.2015.98 Snyder, K., Wang, W.-W., Han, R., McFadden, K., & Valentino, R. J. (2011). Corticotropin-releasing factor in the norepinephrine nucleus, locus coeruleus, facilitates behavioral flexibility. Neuropsychopharmacology, 37(2), 520–530. https://doi.org/10.1038/npp.2011.218 Sobin, C., & Sackeim, H. A. (1997). Psychomotor symptoms of depression. American Journal of Psychiatry, 154(1), 4–17. https://doi.org/10.1176/ajp.154.1.4 233 Sokoloff, P., Giros, B., Martres, M. P., Bouthenet, M. L., & Schwartz, J. C. (1990). Molecular cloning and characterization of a novel dopamine receptor (D3) as a target for neuroleptics. Nature, 347(6289), 146–151. https://doi.org/10.1038/347146a0 Sokolowski, J. D., & Salamone, J. D. (1998). The role of accumbens dopamine in lever pressing and response allocation: Effects of 6-OHDA injected into core and dorsomedial shell, 59(3), 557–566. Spierling, S. R., & Zorrilla, E. P. (2017). Don’t stress about CRF: assessing the translational failures of CRF1antagonists. Psychopharmacology, 234(9–10), 1467–1481. https://doi.org/10.1007/s00213-017-4556-2 St. Onge, J. R., Chiu, Y. C., & Floresco, S. B. (2010). Differential effects of dopaminergic manipulations on risky choice. Psychopharmacology, 211(2), 209–221. https://doi.org/10.1007/s00213-010-1883-y St. Onge, J. R., & Floresco, S. B. (2010). Prefrontal cortical contribution to risk-based decision making. Cerebral Cortex, 20(8), 1816–1828. https://doi.org/10.1093/cercor/bhp250 St Onge, J. R., Abhari, H., & Floresco, S. B. (2011). Dissociable contributions by prefrontal D1 and D2 receptors to risk-based decision making. Journal of Neuroscience, 31(23), 8625–8633. https://doi.org/10.1523/JNEUROSCI.1020-11.2011 St Onge, J. R., Ahn, S., Phillips, A. G., & Floresco, S. B. (2012). Dynamic fluctuations in dopamine efflux in the prefrontal cortex and nucleus accumbens during risk-based decision making. The Journal of Neuroscience, 32(47), 16880–16891. https://doi.org/32/47/16880 [pii]\r10.1523/JNEUROSCI.3807-12.2012 St Onge, J. R., & Floresco, S. B. (2009). Dopaminergic modulation of risk-based decision making. Neuropsychopharmacology, 34(3), 681–697. https://doi.org/10.1038/npp.2008.121 Stackman, R. W., Blasberg, M., Langan, J., & Clark, A. S. (1997). Stability of spatial working memory across the estrous cycle of Long-Evans rats. Neurobiology of Learning and Memory, 171(67), 167–171. Stahl, S. M. (2002). The psychopharmacology of energy and fatigue. The Journal of Clinical Psychiatry, 63(1), 7–8. Starcke, K., Wolf, O. T., Markowitsch, H. J., & Brand, M. (2008). Anticipatory stress influences decision making under explicit risk conditions. Behavioral Neuroscience, 122(6), 1352–1360. https://doi.org/10.1037/a0013281 Stopper, C. M., & Floresco, S. B. (2011). Contributions of the nucleus accumbens and its subregions to different aspects of risk-based decision making. Cognitive, Affective & Behavioral Neuroscience, 11(1), 97–112. https://doi.org/10.3758/s13415-010-0015-9 Stopper, C. M., & Floresco, S. B. (2015). Dopaminergic circuitry and risk/reward decision making: Implications for schizophrenia. Schizophrenia Bulletin, 41(1), 9–14. 234 https://doi.org/10.1093/schbul/sbu165 Stopper, C. M., Khayambashi, S., & Floresco, S. B. (2013). Receptor-specific modulation of risk-based decision making by nucleus accumbens dopamine. Neuropsychopharmacology, 38(5), 715–728. https://doi.org/10.1038/npp.2012.240 Swanson, L., Sawchenko, P., Rivier, J., & Vale, W. (1983). The organization of ovine corticotropin-releasing factor immunoreactive cells and fibers in the rat brain: An immunohistochemical study. Neuroendocrinology, 36(3), 165–186. Tagliaferro, P., & Morales, M. (2008). Synapses between corticotropin-releasing factor-containing axon terminals and dopaminergic neurons in the ventral tegmental area are predominantly glutamatergic. The Journal of Comparative Neurology, 506, 616–626. https://doi.org/10.1002/cne.21576 Tan, L., Vaughan, J., Perrin, M., Rivier, J., & Sawchenko, P. (2017). Distribution of corticotropin-releasing factor (CRF) receptor binding in the mouse brain using a new, high-affinity radioligand, [125 I]-PD-Sauvagine. The Journal of Comparative Neurology, 525(18), 3840–3864. https://doi.org/10.1002/cne.24307 Taylor Tavares, J. V., Clark, L., Cannon, D. M., Erickson, K., Drevets, W. C., & Sahakian, B. J. (2007). Distinct profiles of neurocognitive function in unmedicated unipolar depression and bipolar II depression. Biological Psychiatry, 62(8), 917–924. https://doi.org/10.1016/j.biopsych.2007.05.034 Taylor Tavares, J. V., Clark, L., Furey, M. L., Williams, G. B., Sahakian, B. J., & Drevets, W. C. (2008). Neural basis of abnormal response to negative feedback in unmedicated mood disorders. NeuroImage, 42(3), 1118–1126. https://doi.org/10.1016/j.neuroimage.2008.05.049 Terry, K. K., Chatman, L. A., Foley, G. L., Kadyszewski, E., Fleeman, T. L., Hurtt, M. E., & Chapin, R. E. (2005). Effects of feed restriction on fertility in female rats. Developmental and Reproductive Toxicology, 74(5), 431–441. https://doi.org/10.1002/bdrb.20060 Thai, C. A., Zhang, Y., & Howland, J. G. (2013). Effects of acute restraint stress on set-shifting and reversal learning in male rats. Cognitive, Affective, & Behavioral Neuroscience, 13(1), 164–173. https://doi.org/10.3758/s13415-012-0124-8 Treadway, M. T., Bossaller, N., Shelton, R. C., & Zald, D. H. (2012). Effort-based decision-making in major depressive disorder: A translational model of motivational anhedonia. The Journal of Abnormal Psychology, 121(3), 553–558. https://doi.org/10.1037/a0028813.Effort-Based Treadway, M. T., Buckholtz, J. W., Schwartzman, A. N., Lambert, W. E., & Zald, D. H. (2009). Worth the “EEfRT”? The effort expenditure for rewards task as an objective measure of motivation and anhedonia. PLoS ONE, 4(8), 1–9. https://doi.org/10.1371/journal.pone.0006598 235 Treadway, M. T., Buckholtz, J. W., Zald, D. H., Volz, K. G., & Reichardt, W. (2013). Perceived stress predicts altered reward and loss feedback processing in medial prefrontal cortex. Frontiers in Human Neuroscience, 7, 1–10. https://doi.org/10.3389/fnhum.2013.00180 Treadway, M. T., Peterman, J. S., Zald, D. H., & Park, S. (2015). Impaired effort allocation in patients with schizophrenia. Schizophrenia Research, 161(2–3), 382–385. https://doi.org/10.1016/j.schres.2014.11.024 Tye, K. M., Mirzabekov, J. J., Warden, M. R., Ferenczi, E. A., Tsai, H., Finkelstein, J., … Witten, I. B. (2013). Dopamine neurons modulate neural encoding and expression of depression-related behaviour. Nature, 493(7433), 537–541. https://doi.org/10.1038/nature11740 Tylee, A., Gastpar, M., Lépine, J. P., & Mendlewicz, J. (1999). Identification of depressed patient types in the community and their treatment needs: Findings from the DEPRES II (Depression Research in European Society II) survey. International Clinical Psychopharmacology. https://doi.org/10.1097/00004850-199905002-00002 Uban, K. A., Rummel, J., Floresco, S. B., & Galea, L. (2012). Estradiol modulates effort-based decision making in female rats. Neuropsychopharmacology, 37(2), 390–401. https://doi.org/10.1038/npp.2011.176 Ubl, B., Kuehner, C., Kirsch, P., Ruttorf, M., Diener, C., & Flor, H. (2014). Altered neural reward and loss processing and prediction error signalling in depression. Social Cognitive and Affective Neuroscience, 10(8), 1102–1112. https://doi.org/10.1093/scan/nsu158 Uhart, M., Chong, R. Y., Oswald, L., Lin, P. I., & Wand, G. S. (2006). Gender differences in hypothalamic-pituitary-adrenal (HPA) axis reactivity. Psychoneuroendocrinology, 31(5), 642–652. https://doi.org/10.1016/j.psyneuen.2006.02.003 Ulrich-Lai, Y. M., & Herman, J. P. (2009). Neural regulation of endocrine and autonomic stress responses. Nature Reviews Neuroscience, 10(6), 397–409. https://doi.org/10.1038/nrn2647 Ungless, M. A., & Grace, A. A. (2012). Are you or aren’t you? Challenges associated with physiologically identifying dopamine neurons. Trends in Neurosciences, 35(7), 422–430. https://doi.org/10.1016/j.tins.2012.02.003 Ungless, M. A., Magill, P. J., & Bolam, J. P. (2004). Uniform inhibition of dopamine neurons in the ventral tegmental area by aversive stimuli. Science, 303(5666), 2040–2042. Ungless, M. A., Singh, V., Crowder, T. L., Yaka, R., Ron, D., & Bonci, A. (2003). Corticotropin-releasing factor requires CRF binding protein to potentiate NMDA receptors via CRF receptor 2 in dopamine neurons. Neuron, 39(3), 401–407. https://doi.org/10.1016/S0896-6273(03)00461-6 Uribe-Mariño, A., Gassen, N. C., Wiesbeck, M. F., Balsevich, G., Santarelli, S., Solfrank, B., … Schmidt, M. V. (2016). Prefrontal cortex corticotropin-releasing factor receptor 1 conveys acute stress-induced executive dysfunction. Biological Psychiatry, 80(10), 743–753. 236 https://doi.org/10.1016/j.biopsych.2016.03.2106 Vale, W., Spiess, J., Rivier, C., & Rivier, J. (1981). Characterization of a 41-residue ovine hypothalamic peptide that stimulates secretion of corticotropin and beta-endorphin. Science, 213(4514), 1394–1397. Valenti, O., Lodge, D. J., & Grace, A. A. (2011). Aversive stimuli alter ventral tegmental area dopamine neuron activity via a common action in the ventral hippocampus. Journal of Neuroscience, 31(11), 4280–4289. https://doi.org/10.1523/jneurosci.5310-10.2011 Van’t Veer, A., Yano, J. M., Carroll, F. I., Cohen, B. M., & Carlezon, W. A. (2012). Corticotropin-releasing factor (CRF)-induced disruption of attention in rats is blocked by the κ-opioid receptor antagonist JDTic. Neuropsychopharmacology, 37, 2809–2816. https://doi.org/10.1038/npp.2012.151 van den Bos, R., Homberg, J., & de Visser, L. (2013). A critical review of sex differences in decision-making tasks: Focus on the Iowa Gambling Task. Behavioural Brain Research, 238(1), 95–108. https://doi.org/10.1016/j.bbr.2012.10.002 van den Bos, R., Jolles, J., Van der Knaap, L., Baars, A., & De Visser, L. (2012). Male and female Wistar rats differ in decision-making performance in a rodent version of the Iowa Gambling Task. Behavioural Brain Research, 234(2), 375–379. https://doi.org/10.1016/j.bbr.2012.07.015 van den Bos, R. Van Den, Harteveld, M., & Stoop, H. (2009). Stress and decision-making in humans : Performance is related to cortisol reactivity , albeit differently in men and women. Psychoneuroendocrinology, 34, 1449–1458. https://doi.org/10.1016/j.psyneuen.2009.04.016 Van Der Plas, E. A. A., Crone, E. A., Van Den Wildenberg, W. P. M., Tranel, D., & Bechara, A. (2009). Executive control deficits in substance-dependent individuals: A comparison of alcohol, cocaine, and methamphetamine and of men and women. Journal of Clinical and Experimental Neuropsychology, 31(6), 706–719. https://doi.org/10.1080/13803390802484797 Van Pett, K, Viau, V., Bittencourt, J. C., Chan, R. K. W., Li, H. Y., Arias, C., … Sawchenko, P. E. (2000). Distribution of mRNAs encoding CRF receptors in brain and pituitary of rat and mouse. Journal of Comparative Neurology, 428(2), 191–212. https://doi.org/10.1002/1096-9861(20001211)428:2<191::aid-cne1>3.0.co;2-u Vedhara, K., Hyde, J., Gilchrist, I. D., Tytherleigh, M., & Plummer, S. (2000). Acute stress, memory, attention and cortisol. Psychoneuroendocrinology, 25(6), 535–549. https://doi.org/10.1016/S0306-4530(00)00008-1 Voyer, D., Voyer, S. D., & Saint-Aubin, J. (2017). Sex differences in visual-spatial working memory: A meta-analysis. Psychonomic Bulletin and Review, 24(2), 307–334. https://doi.org/10.3758/s13423-016-1085-7 Vyas, A., Jadhav, S., & Chattarji, S. (2006). Prolonged behavioral stress enhances synaptic 237 connectivity in the basolateral amygdala. Neuroscience, 143(2), 387–393. https://doi.org/10.1016/j.neuroscience.2006.08.003 Wanat, M. J., Bonci, A., & Phillips, P. E. (2013). CRF acts in the midbrain to attenuate accumbens dopamine release to rewards but not their predictors. Nature Neuroscience, 16(4), 383–385. https://doi.org/10.1038/nn.3335 Wanat, M. J., Hopf, F. W., Stuber, G. D., Phillips, P. E. M., & Bonci, A. (2008). Corticotropin-releasing factor increases mouse ventral tegmental area dopamine neuron firing through a protein kinase C-dependent enhancement of Ih. The Journal of Physiology, 586(8), 2157–2170. https://doi.org/10.1113/jphysiol.2007.150078 Wang, B., Shaham, Y., Zitzman, D., Azari, S., Wise, R. A., & You, Z.-B. (2005). Cocaine experience establishes control of midbrain glutamate and dopamine by corticotropin-releasing factor: A role in stress-induced relapse to drug seeking. Journal of Neuroscience, 25(22), 5389–5396. https://doi.org/10.1523/JNEUROSCI.0955-05.2005 Wang, X. Y., Zhao, M., Ghitza, U. E., Li, Y. Q., & Lu, L. (2008). Stress impairs reconsolidation of drug memory via glucocorticoid receptors in the basolateral amygdala. Journal of Neuroscience, 28(21), 5602–5610. https://doi.org/10.1523/JNEUROSCI.0750-08.2008 Weafer, J., De Arcangelis, J., & de Wit, H. (2015). Sex differences in behavioral impulsivity in at-risk and non-risk drinkers. Frontiers in Psychiatry, 6, 1–7. https://doi.org/10.3389/fpsyt.2015.00072 Wellman, C. L. (2001). Dendritic reorganization in pyramidal neurons in medial prefrontal cortex after chronic corticosterone administration. Journal of Neurobiology, 49(3), 245–253. https://doi.org/10.1002/neu.1079 Wemm, S. E., & Wulfert, E. (2017). Effects of acute stress on decision making. Applied Psychophysiology Biofeedback, 42(1), 1–12. https://doi.org/10.1007/s10484-016-9347-8 West, R. K., Maynard, M. E., & Leasure, J. L. (2018). Binge ethanol effects on prefrontal cortex neurons, spatial working memory and task-induced neuronal activation in male and female rats. Physiology and Behavior, 188, 79–85. https://doi.org/10.1016/j.physbeh.2018.01.027 Wiersielis, K. R., Wicks, B., Simko, H., Cohen, S. R., Khantsis, S., Baksh, N., … Bangasser, D. A. (2016). Sex differences in corticotropin releasing factor-evoked behavior and activated networks. Psychoneuroendocrinology, 73, 204–216. https://doi.org/10.1016/j.psyneuen.2016.07.007 Williams, C. L., Buchta, W. C., & Riegel, A. C. (2014). CRF-R2 and the heterosynaptic regulation of VTA glutamate during reinstatement of cocaine seeking. The Journal of Neuroscience, 34(31), 10402–10414. https://doi.org/10.1523/JNEUROSCI.0911-13.2014 Winstanley, C. A., & Floresco, S. B. (2016). Deciphering decision making: Variation in animal models of effort- and uncertainty-based choice reveals distinct neural circuitries underlying core cognitive processes. Journal of Neuroscience, 36(48), 12069–12079. 238 https://doi.org/10.1523/JNEUROSCI.1713-16.2016 Winstanley, C. A., Zeeb, F. D., Bedard, A., Fu, K., Lai, B., Steele, C., & Wong, A. C. (2010). Dopaminergic modulation of the orbitofrontal cortex affects attention , motivation and impulsive responding in rats performing the five-choice serial reaction time task. Behavioural Brain Research, 210(2), 263–272. https://doi.org/10.1016/j.bbr.2010.02.044 Wise, R. A. (2004). Dopamine, learning and motivation. Nature Reviews Neuroscience, 5(6), 483–494. https://doi.org/10.1038/nrn1406 Wise, R. A., & Morales, M. (2010). A ventral tegmental CRF-glutamate-dopamine interaction in addiction. Brain Research, 1314, 38–43. https://doi.org/10.1016/j.brainres.2009.09.101 Wright, K. D., Lebell, M., & Carleton, R. N. (2016). Intolerance of uncertainty, anxiety sensitivity, health anxiety, and anxiety disorder symptoms in youth. Journal of Anxiety Disorders, 41, 35–42. https://doi.org/10.1016/j.janxdis.2016.04.011 Yuen, E. Y., Liu, W., Karatsoreos, I. N., Feng, J., McEwen, B. S., & Yan, Z. (2009). Acute stress enhances glutamatergic transmission in prefrontal cortex and facilitates working memory. Proceedings of the National Academy of Sciences, 106(33), 14075–14079. https://doi.org/10.1073/pnas.0906791106 Zapata, A., & Shippenberg, T. S. (2002). D3 receptor ligands modulate extracellular dopamine clearance in the nucleus accumbens. Journal of Neurochemistry, 81(5), 1035–1042. https://doi.org/10.1046/j.1471-4159.2002.00893.x Zapata, Agustin, Witkin, J. M., & Shippenberg, T. S. (2001). Selective D3 receptor agonist effects of (+)-PD 128907 on dialysate dopamine at low doses. Neuropharmacology, 41(3), 351–359. https://doi.org/10.1016/S0028-3908(01)00069-7 Zeeb, F. D., Robbins, T. W., & Winstanley, C. A. (2009). Serotonergic and dopaminergic modulation of gambling behavior as assessed using a novel rat gambling task. Neuropsychopharmacology, 34(10), 2329–2343. https://doi.org/10.1038/npp.2009.62 Zeidan, M. A., Igoe, S. A., Linnman, C., Vitalo, A., Levine, J. B., Klibanski, A., … Milad, M. R. (2011). Estradiol modulates medial prefrontal cortex and amygdala activity during fear extinction in women and female rats. Biological Psychiatry, 70(10), 920–927. https://doi.org/10.1016/j.biopsych.2011.05.016 Zhang, M., Balmadrid, C., & Kelley, A. E. (2003). Nucleus accumbens opioid, GABAergic, and dopaminergic modulation of palatable food motivation: Contrasting effects revealed by a progressive ratio study in the rat. Behavioral Neuroscience, 117(2), 202–211. https://doi.org/10.1037/0735-7044.117.2.202 Zhang, W. N., Chang, S. H., Guo, L. Y., Zhang, K. L., & Wang, J. (2013). The neural correlates of reward-related processing in major depressive disorder: A meta-analysis of functional magnetic resonance imaging studies. Journal of Affective Disorders, 151(2), 531–539. https://doi.org/10.1016/j.jad.2013.06.039 239 Zhao, L. Y., Zhang, X. L., Shi, J., Epstein, D. H., & Lu, L. (2009). Psychosocial stress after reactivation of drug-related memory impairs later recall in abstinent heroin addicts. Psychopharmacology, 203(3), 599–608. https://doi.org/10.1007/s00213-008-1406-2 Zimmerman, E. C., & Grace, A. A. (2016). The nucleus reuniens of the midline thalamus gates prefrontal-hippocampal modulation of ventral tegmental area dopamine neuron activity. Journal of Neuroscience, 36(34), 8977–8984. https://doi.org/10.1523/JNEUROSCI.1402-16.2016 Zorrilla, E. P., Heilig, M., de Wit, H., & Shaham, Y. (2013). Behavioral, biological, and chemical perspectives on targeting CRF1 receptor antagonists to treat alcoholism. Drug and Alcohol Dependence, 128(3), 175–186. https://doi.org/10.1016/j.drugalcdep.2012.12.017 Zou, Y. M., Ni, K., Wang, Y. Y., Yu, E. Q., Lui, S. S. Y., Zhou, F. C., … Chan, R. C. K. (2019). Effort–cost computation in a transdiagnostic psychiatric sample: Differences among patients with schizophrenia, bipolar disorder, and major depressive disorder. PsyCh Journal. https://doi.org/10.1002/pchj.316
Thesis/Dissertation
eng
Psychology
Vancouver : University of British Columbia Library
Attribution-NonCommercial-NoDerivatives 4.0 International
Graduate
The role of corticotropin-releasing and acute stress in modulating cognitive flexibility and decision making : physiological and behavioural interactions with mesolimbic dopamine
Text