UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

The contribution of prefrontal-subcortical circuitry to risk-based decision making St. Onge, Jennifer Rose 2011

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata

Download

Media
24-ubc_2012_spring_stonge_jennifer.pdf [ 5.32MB ]
Metadata
JSON: 24-1.0072501.json
JSON-LD: 24-1.0072501-ld.json
RDF/XML (Pretty): 24-1.0072501-rdf.xml
RDF/JSON: 24-1.0072501-rdf.json
Turtle: 24-1.0072501-turtle.txt
N-Triples: 24-1.0072501-rdf-ntriples.txt
Original Record: 24-1.0072501-source.json
Full Text
24-1.0072501-fulltext.txt
Citation
24-1.0072501.ris

Full Text

THE CONTRIBUTION OF PREFRONTAL-SUBCORTICAL CIRCUITRY TO RISK-BASED DECISION MAKING  by JENNIFER ROSE ST. ONGE B. A. Hons., The University of Regina, 2006 M. A., The University of British Columbia, 2008  A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY  in  THE FACUTLY OF GRADUATE STUDIES (Psychology)  THE UNIVERSITY OF BRITISH COLUMBIA (Vancouver)  December 2011  © Jennifer Rose St. Onge, 2011  ABSTRACT The ability to make decisions about different risks and rewards appears to recruit a neural circuit that includes the prefrontal cortex (PFC), the amygdala and the ventral striatum. The present thesis used a combination of behavioural, statistical, anatomical, and pharmacological techniques to elucidate the nature of risk-based decision making and the underlying neural circuits and neuromodulatory systems that contribute to this form of behaviour. Chapter 2 examined probabilistic discounting as a model of risk-based decision making. Statistical modeling revealed substantial individual variability in discounting of large, probabilistic rewards in well-trained animals which develops over the course of training. These discounting patterns were not influenced by luckiness in receiving reward early in training. Rather, patterns of risky vs. safe choices were influenced by both 1) recent luck in forced choice outcomes early in training and 2) outcomes of free choice trials in the animal’s recent reinforcement history. Chapter 3 revealed that the prelimbic region of the rat medial PFC makes a selective contribution to probabilistic discounting by keeping track of changes in reward probability in order to update value representations, whereas the insular and dorsal anterior cingulate subregions have no influence on risky choice. While it makes no contribution to choice outcome, the OFC region aids decision latency. Chapter 4 describes a series of asymmetrical disconnections which revealed that separate neural circuits mediate different aspects of risk-based decision making. Amygdala projections to the nucleus accumbens bias behaviour towards large, risky reward options whereas top-down projections from the medial PFC to the amygdala regulate this bias and promote adjustments in choice towards smaller, but potentially more valuable, options. Chapter 5 revealed that the contribution of medial PFC activity to probabilistic discounting is further modulated by a fine  ii  balance of D1 and D2 receptor activity. The general discussion in Chapter 6 integrates these findings into a broader perspective on the neural basis of decision making about probabilistic rewards, while focusing on the PFC and its interactions with other neural systems to guide decision making.  iii  PREFACE •  Sections describing the amygdala and risk-based decision making in Chapter 1 were published: Ghods-Sharifi, S., [St.Onge J. R.] & Floresco, S. B. (2009). Fundamental contribution by the basolateral amygdala to different forms of decision making. J Neurosci. 29: 5251-5259. o Together with S. Ghods-Sharifi, we conducted all the testing, analyzed the data and each wrote half of the manuscript (co-first authors).  •  A version of Chapter 2 has been published. [St.Onge, J. R.], Floresco, S. B. (2010). Prefrontal cortical contribution to risk-based decision making. Cerebral Cortex. 20:18161828. o I conducted all the testing, analyzed the data, and wrote most of the manuscript.  •  A version of Chapter 4 has been submitted for publication. [St.Onge, J. R.], Stopper, C. M., Zahm, S.D. and Floresco, S. B. (2011). PNAS. o I conducted the majority of testing except the anatomical procedures (S.D. Zahm) and the section on PFC-NAc disconnections (C. Stopper). I also analyzed the data and wrote most of the manuscript.  •  A version of Chapter 5 has been published. [St.Onge, J. R.], Abhari, H., and Floresco, S. B. (2011) Dissociable contributions by prefrontal D1 and D2 receptors to risk-based decision making. J Neurosci. 31: 8625-8633 o I directly supervised (H. Abhari) and assisted with all the testing, analyzed the data, and wrote most of the manuscript.  iv  TABLE OF CONTENTS ABSTRACT .................................................................................................................................................. ii PREFACE .................................................................................................................................................... iv TABLE OF CONTENTS .............................................................................................................................. v LIST OF TABLES...................................................................................................................................... vii LIST OF FIGURES ..................................................................................................................................... ix ACKNOWLEDGEMENTS......................................................................................................................... xi CHAPTER 1: GENERAL INTRODUCTION............................................................................................1 Cost/Benefit Decision Making about Certain Versus Risky Rewards ........................................................................ 3 Neurobiological Correlates of Risk-Based Decision Making ...................................................................................... 5 The Prefrontal Cortex .................................................................................................................................................. 5 Prefrontal-Subcortical Circuitry ................................................................................................................................. 10 Rodent Models of Cost/Benefit Decision Making................................................................................................... 15 Dopaminergic Modulation of Prefrontal-Subcortical Circuitry ............................................................................... 18 Summary and Objectives ....................................................................................................................................... 20  CHAPTER 2: PROBABILISTIC DISCOUNTING AS A MODEL OF RISK-BASED DECISION MAKING ..................................................................................................................................................... 23 Introduction........................................................................................................................................................... 23 Cost/Benefit Decision Making: Discounting Models ................................................................................................. 25 Rodent Models of Temporal and Probabilistic Discounting ...................................................................................... 26 Current Research: Probabilistic Discounting in Rats ................................................................................................. 29 Method.................................................................................................................................................................. 34 Animals ..................................................................................................................................................................... 34 Apparatus ................................................................................................................................................................. 34 Data Collection .......................................................................................................................................................... 35 Analysis 1: How are probabilistic discounting rates characterized and how do they change over training? ........... 35 Analysis 2: What is the effect of being “lucky” during the first 5 days of training on the slope of the probabilistic discounting curve during the last 5 days of training (i.e., when stable choice occurs)? ........................................... 36 Analysis 3: What is the effect of being “lucky” in the forced choice trials of a given probability block on the probability of choosing the Large/Risky lever on the first free choice trial? ............................................................. 37  v  Analysis 4: What is the effect of previous choice outcomes on the probability of choosing the Large/Risky lever both early and late in training and how far back in the past (in terms of number of choice trials) does this influence occur? ........................................................................................................................................................ 37 Results ................................................................................................................................................................... 38 Analysis 1: How are probabilistic discounting rates characterized and how do they change over training? ........... 38 Analysis 2: What is the effect of being “lucky” during the first 5 days of training on the slope of the probabilistic discounting curve during the last 5 days of training (i.e., when stable choice occurs)? ........................................... 42 Analysis 3: What is the effect of being “lucky” in the forced choice trials of a given probability block on the probability of choosing the Large/Risky lever on the first free choice trial? ............................................................. 42 Analysis 4: What is the effect of previous choice outcomes on the probability of choosing the Large/Risky lever both early and late in training and how far back in the past (#trials) does this influence occur? ............................ 45 Discussion .............................................................................................................................................................. 48 Individual Variation in Probabilistic Discounting ...................................................................................................... 48 Recent Forced and Free-Choice Outcomes Influence Future Choice of Certain vs. Uncertain Options ..................... 51 Early Task Experiences do not Influence Individual Learned Discounting Patterns .................................................. 53  CHAPTER 3: PREFRONTAL CORTICAL CONTRIBUTION TO RISK-BASED DECISION MAKING ..................................................................................................................................................... 55 Introduction........................................................................................................................................................... 55 Experiment 1: The Effects of Inactivations of the Medial PFC, OFC, Anterior Cingulate and Insular Cortex on Probabilistic Discounting ....................................................................................................................................... 57 Method ..................................................................................................................................................................... 57 Results ....................................................................................................................................................................... 61 Experiment 2: Effects of Medial PFC Inactivation on Performance of a Within-Session Reversal ........................... 66 Method ..................................................................................................................................................................... 67 Results ....................................................................................................................................................................... 68 Experiment 3: Effects of Inactivation of the Medial PFC on Probabilistic Discounting with Fixed Probabilities ...... 69 Method ..................................................................................................................................................................... 70 Results ....................................................................................................................................................................... 71 Discussion .............................................................................................................................................................. 73 Involvement of the Rat Medial PFC in Probabilistic discounting ............................................................................... 73 The Role of Other PFC Regions in Cost/Benefit Decision Making .............................................................................. 77  CHAPTER 4: SEPARATE NEURAL CIRCUITS BIAS CHOICE TOWARDS RISKY OR CERTAIN OPTIONS ................................................................................................................................................... 81 Introduction........................................................................................................................................................... 81 Method.................................................................................................................................................................. 83 Reward magnitude discrimination task .................................................................................................................... 83 Surgical, Microinfusion and Procedures .................................................................................................................... 84 Disconnection Design and Testing Procedures.......................................................................................................... 85 Histology ................................................................................................................................................................... 86 Data Analysis ............................................................................................................................................................ 87  vi  Win-Stay/Lose-Shift Analyses ................................................................................................................................... 87 Results ................................................................................................................................................................... 88 Effect of Functional Disconnection of the BLA – NAc Pathway on Probabilistic Discounting ................................ 88 Effect of Functional Disconnection of the Medial PFC – BLA Pathway on Probabilistic Discounting ................... 92 Effect of Functional Disconnection of the Ascending vs. Descending BLA-Medial PFC Pathways on Probabilistic Discounting ............................................................................................................................................................... 96 Discussion ............................................................................................................................................................ 102 Subcortical, amygdala-striatal circuitry drives choice towards larger, riskier rewards. ....................................... 102 Top-down control of amygdala-NAc circuit by the medial PFC............................................................................. 104  CHAPTER 5: THE CONTRIBUTION OF PREFRONTAL DOPAMINE TO RISK-BASED DECISION MAKING ............................................................................................................................. 108 Introduction......................................................................................................................................................... 108 Method................................................................................................................................................................ 110 Microinfusion Protocol ............................................................................................................................................ 110 Data Analysis .......................................................................................................................................................... 111 Results ................................................................................................................................................................. 111 D1 and D2 Receptor Antagonism and Probabilistic Discounting .............................................................................. 112 D1 and D2 Receptor Stimulation and Probabilistic Discounting ............................................................................... 115 Win-stay/Lose Shift Analysis ................................................................................................................................... 116 Reward Magnitude Discrimination ......................................................................................................................... 119 Discussion ............................................................................................................................................................ 121 Effects of D1/D2 receptor blockade.......................................................................................................................... 121 Effects of D1/D2 Receptor Stimulation ..................................................................................................................... 123 Dissociable Contributions of PFC D1 and D2 Receptors to Risk-Based Decision Making .......................................... 125  CHAPTER 6: GENERAL DISCUSSION ............................................................................................. 127 Cost/Benefit Decision Making about Certain vs. Probabilistic Rewards Recruits Specific Neural Circuits ............. 128 Regional Specialization for Different Types of Decision Making in the Rat PFC .................................................... 129 A Broader Role for the Medial PFC in Exploiting and Exploring Rewards ............................................................. 133 The PFC Exerts Executive Control Over Valuation Signals..................................................................................... 136 Mesocortical Dopamine May Signal Changes in Reward Obtained Over Time ..................................................... 139 Convergence of Choice Information in the NAc Biases Overt Behaviour .............................................................. 142 Summary and Conclusions ................................................................................................................................... 144  REFERENCES......................................................................................................................................... 147  vii  LIST OF TABLES Table 1. Descriptive statistics for slope of discounting curve and early luckiness (n=194). ..38 Table 2. Descriptive statistics for percentage choice of Large/Risky lever (n=194). ................41 Table 3. Linear regression results for predicting discount slope from early luckiness. ...........42 Table 4. Estimates of the effect of forced-choice luckiness on probability of choosing the Large/Risky lever. .......................................................................................................................................43 Table 5. Estimated probability of choosing the Large/Risky lever based on forced-choice luckiness in each probability block. .....................................................................................................45 Table 6. Regression coefficient estimates of the effect of lagged trial outcomes on choosing the Large/Risky lever. ...............................................................................................................................46 Table 7. Locomotion, trial omission, and response latency data obtained following saline treatment, ipsilateral inactivation, or functional disconnection of the various pathways. ............................................................................................................................................................................93 Table 8. Locomotion, trial omission, and response latency data obtained following saline or drug infusions into the medial PFC. .................................................................................................. 114 Table 9. Win stay/lose shift ratios for rats performing the probabilistic discounting task following infusion of saline and the highest or most effective dose of D1 or D2 antagonist or agonists. .................................................................................................................................................. 118  viii  LIST OF FIGURES Figure 1. Schematic of the probabilistic discounting task.....................................................................30 Figure 2. Histograms displaying proportion of rats that chose the Large/Risky lever during a set percentage of trials in each probability block. ..........................................................................40 Figure 3. Individual probabilistic discounting curves. ...........................................................................40 Figure 4. Rodent learning of the probabilistic discounting task.........................................................41 Figure 5. Estimates of probability of choosing the Large/Risky lever following different reward outcomes (0, 1, 4 pellets) at Lag 1 and Lag 8. ...................................................................47 Figure 6. Effects of inactivation of different PFC regions on probabilistic discounting. ...........62 Figure 7. Effects of medial PFC inactivation on probabilistic discounting with ascending probabilities...................................................................................................................................................65 Figure 8. Medial PFC inactivation does not affect performance of a within-session reversal. ............................................................................................................................................................................69 Figure 9. Inactivation of the medial PFC does not affect risky choice using fixed probabilities associated with larger reward. ...............................................................................................................72 Figure 10. Effects of disconnection of the BLA-NAc pathway on probabilistic discounting and reward magnitude discrimination. .......................................................................................................90 Figure 11. Effect of disconnection of the medial PFC-BLA pathway on probabilistic discounting.....................................................................................................................................................95 Figure 12. The ascending and descending axonal pathways between the medial PFC and BLA travel through separate routes through the brain. ........................................................................97 Figure 13. Effect of disconnecting ascending and descending pathways between the medial PFC and BLA on probabilistic discounting. .......................................................................................99 ix  Figure 14. Histology for medial PFC DA infusions. ............................................................................... 112 Figure 15. Effects of DA receptor manipulations in the medial PFC on probabilistic discounting.................................................................................................................................................. 113 Figure 16. Effects of PFC DA receptor manipulations on win-stay (grey bars) and lose-shift (white bars) tendencies. ........................................................................................................................ 117 Figure 17. Effects of DA receptor modulation in the medial PFC on reward magnitude discrimination............................................................................................................................................ 120 Figure 18. The ascending and descending axonal pathways between the medial PFC and BLA travel through separate routes through the brain. ..................................................................... 179  x  ACKNOWLEDGEMENTS First and foremost, I would like to thank my supervisor: Dr. Stan Floresco. Over the past 5 years, we have developed both an effective working relationship and friendship that have been vital to my success in this program. An endless fountain of knowledge in the area, an incomparable commitment to being available to, and understanding of, his students, and a positive, joyful attitude that rivals none, Stan is all of the above and more. I am extremely grateful for his choice to respect the path that I have taken and the vast support I continue to receive. Alas, there is not much else to say about Stan except, he is “Groovy baby, yeah”. I would also like to thank my committee members, Drs. Anthony Phillips and Alan Kingstone for their on-going support, cooperation, and useful comments and suggestions on my research, as well as my funding source, the Natural Sciences and Engineering Research Council of Canada. The following people at UBC were also all an integral part of my successful degree program in one form or another – thanks to all of you! Ying Zhang, Hamed Abhari, Yu Chi Chu (Pat), Titus Yip, Gina Yuan Chun Chang, Emily Green, Sky Liang, Colin Stopper, Naghmeh Shafiei, Fiona Zeeb, Kristina Uban, Kitty So, Soyon Ahn, Gemma Floresco, Lucille Hoover, Candace Marshall, Nina Di Pietro, Andrea Blair, Anna Cantor, Chris Lapish, Kelly Butts, Mark Schaller.  Most importantly: my family, my friends, and my Andy. Your love, support, and mere existence, helped shape the person that I am today, gave me the confidence to go after life, and continues to bring me happiness each and every day.  xi  CHAPTER 1: GENERAL INTRODUCTION How we make decisions about such as things as what to eat, what to buy, and who to marry, has been a perplexing question that has plagued, economists, ecologists, and psychologists for centuries, and more recently, neuroscientists and computer scientists. The origins of modern decision making theory can be traced back to mathematicians in the 17th century who argued that people make choices in a rational manner by selecting options that will maximize expected utility (von Neumann and Morgenstern, 1944; Bernoulli, 1954). However, there are numerous examples of people exhibiting behaviours inconsistent with normative economic choice theory. For example, people are often biased to choose certain options even if the expected value of that reward is lower than other probabilistic options (i.e.; loss-aversion; Kahneman and Tversky, 2000). Also, people do not evaluate the outcomes of decisions in terms of the overall states of wealth to which they lead, but as gains (or losses) relative to a neutral reference point (Tversky and Kahneman, 1974). These discrepancies pose a problem for using rational choice theory to explain behaviour. In contrast, behavioural ecologists argue that these non-optimal patterns of behaviour are a result of “core knowledge mechanisms” that evolved to maximize fitness but lead to systematic errors in current environmental conditions (Kinzler and Spelke, 2007; Santos and Hughes, 2009). For example, the disproportionately high value placed on short-term rewards may explain why humans have difficulty acting with long-term interest in mind, such as breaking diets, forgoing retirement savings, and postponing preventative health behaviours (Haynes, 2009). Therefore, when trying to understand irrational choice, it is important to consider the evolutionary constraints under which different behaviours evolved and what the ultimate goal of the behaviour is.  1  The use of animal models of decision making provides an opportunity to explain decision making behaviour as it is, not as it should be. Although there are advantages for technical reasons, animal models of choice also have utility in their ability to examine the evolutionary roots of decision making in different species by linking choice anomalies to specific neural substrates. In a similar manner to humans, rats, honeybees, and various birds show a variety of violations of economic theory, such as reflection, certainty, endowment, and framing effects, reference dependence, loss aversion, as well as reversals of risk attitudes (Kacelnik and Bateson, 1996; Marsh and Kacelnik, 2002; Chen et al., 2006). A wide range of animals, including monkeys, pigeons, rats, mice, and even insects, also show time-inconsistent intertemporal choices, including placing greater value on immediate outcomes (Rachlin and Green, 1972; Ainslie, 1974; Isles et al., 2003; Kalenscher and Pennartz, 2008; Kim et al., 2008; Louie and Glimcher, 2010). These similarities in economic violations between humans and non-human species suggest that we may share some of the neural circuitry that guides decision making behaviour due to similar natural selection processes. Animal models enable one to manipulate both cognitive processes and neural substrates in order to examine the effects on decision making behaviour. Moreover, while economic theories focus on the final outcome of a decision, animal models based on psychological principles can identify the underlying cognitive processes that produce these decisions and map these processes onto neural systems in the brain. In doing so, we may be able to identify both oneto-one mapping of neural systems to choice processes, as well as important cross-species differences that may supply answers as to why we make the decisions we do. Today, a multidisciplinary approach that utilizes knowledge from social, biological, and mathematical sciences may have a better chance at fully capturing how and why we make decisions to obtain rewards or avoid losses. Furthermore, understanding how our brain deals with  2  probabilities, computes value, and regulates behaviour in response to choice outcomes will help us better understand the underlying cognitive dysfunction experienced by those with brain damage, schizophrenia, pathological gambling, and other forms of addiction, who exhibit extremely poor decision making. Therefore, the goal of this thesis is to illuminate the neural circuitry underlying the process of decision making, specifically in situations where rewards are uncertain.  Cost/Benefit Decision Making about Certain Versus Risky Rewards Interest in how we process decisions where there is uncertainty about the likelihood of obtaining rewards has been present for hundreds of years (Glimcher, 2004). The recent worldwide economic recession has compounded this interest in understanding how individuals make decisions about probabilistic options and why these decisions go awry. In natural and artificial environments, rewards occur with variability in time, location, probability, and size. These characteristics affect the subjective value associated with different rewards. Rewards that are rare may become less valuable even if they are quite large. Organisms must calculate the probability and magnitude of rewards occurring in their environment in order to determine their value and decide whether to obtain them. Each course of action towards a goal can be perceived as having both benefits (more calories, money) and costs (greater chance of being eaten, possibility of debt) associated with it. As an animal forages for food, exploring a new environment is risky in that it might yield greater amounts of food or it might provide nothing, whereas exploiting a current environment may continue to provide food, but with decreasing amounts. When food sources are greatly depleted, it may be advantageous for the animal to seek new, yet risky food sources.  3  Today, humans also make choices between exploiting more reliable investment opportunities and exploring risky, but potentially more successful ones. Risk, in its objective form, is defined as the variance of the probability distribution of outcomes (Levy and Markowitz, 1979). In other words, risk denotes how much a decision maker in uncertain situations risks to gain or lose relative to the known mean possible outcome (expected value of the known probability distribution). Whereas probability is maximal at 100%, risk peaks at a probability of 0.5 of occurrence. Risk can also be considered subjective in that people vary in the extent to which they prefer certain over probabilistic rewards with the same expected value (Luce, 2000). An individual’s sensitivity to differences in payoff variances is thought to reflect their specific risk attitude (Markowitz, 1952). For example, a risk-neutral person will attach the same certainty equivalent to both 40% win/60% loss gambles and 10/90 gambles whereas a riskaversive decision maker will be affected by the increase of risk from the 40/60 to the 10/90 gamble. A variety of other animals, including insects, fish, birds, and mammals also show variability in their risk preferences during foraging (Kacelnik and Bateson, 1996). Regardless of an individual attitude, when choosing a probabilistic option over a certain option, an organism is accepting the risk associated with this more variable option that no reward will be delivered. As there are no sensory receptors that detect probability, the brain needs to derive probability from the frequency of past events. Therefore, calculating reward probability is a fundamental aspect of risk assessment. Distortions of probability calculations (e.g. overestimation) are one way in which preference for risky options may be enhanced. Moreover, being able to dynamically integrate the expected magnitude of a reward with the probability of it being available is important to maintain updated value representations of different choice options (Walton et al., 2007); a process vital to both the foraging animal and the stock market junkie.  4  Neurobiological Correlates of Risk-Based Decision Making The development of neuroscience as a field of its own, as well as the great advances in technology over the last 30 years has propelled investigations into the underlying neural basis of decision making. The explosion of clinical case studies in the 1990’s describing individuals with brain damage, stroke, and addiction to drugs or gambling that were experiencing severe difficulties with real-life decision making drew the world’s attention to the importance of understanding what is happening in the brains of these individuals. These patients experience tremendous personal, occupational, and financial consequences as a result of their impaired decision making. Although neuropsychological testing has been able to show that various types of neurological inflictions can impair decision making on standardized tests, it has been difficult to determine what particular aspects of decision making are specifically impaired in these individuals (e.g. estimations of reward size, probability, subjective value, etc.). A combination of clinical neuropsychological examinations and neuroimaging of both healthy and impaired individuals have implicated numerous brain regions in decision making about risks and rewards.  The Prefrontal Cortex The prefrontal cortex region of the frontal lobes (PFC) has long been thought to mediate higher cognitive functions, such as working memory, attention, cognitive flexibility and decision making through somewhat specialized but also interconnected subregions (Owen et al., 1990; Stuss and Alexander, 2000). Recent literature shows that the ventromedial/orbitofrontal, insular, dorsolateral and anterior cingulate regions of the PFC are particularly important for risk-based decision making. Individuals that sustain damage to the PFC via car accidents or strokes experience substantial changes in decision making behaviour and are often characterized as disadvantageous and sometimes risky (Eslinger and Damasio, 1985; Shallice et al., 1989). These  5  clinical observations sparked the interest of Damasio and colleagues who developed a novel “Iowa gambling task” (IGT) to simulate real-life decisions in terms of uncertainty, reward and punishment (Bechara et al., 1994). Since then, a variety of cost/benefit decision making tasks have been developed to examine decision making in brain-damaged individuals as well as in healthy controls using neuroimaging tools. The following discussion highlights the variety of different subregions of the PFC that are recruited in humans under different decision making conditions. Ventromedial PFC/Orbitofrontal Cortex Initially, Damasio and colleagues showed that patients with damage to the ventromedial (vmPFC) regions of the PFC (encompassing the orbitofrontal cortex (OFC) and ventral aspects of the anterior cingulate cortex) display impaired decision making, making more high-risk choices on the IGT (Bechara et al., 1994). However, the literature following these initial findings has been mixed. In some cases, patients with vmPFC damage performing other gambling tasks (e.g.; Cambridge Gambling Task) are accurate at choosing the advantageous option in order to gain reward, but show significantly increased betting behavior with regards to those decisions (Mavaddat et al., 2000; Clark et al., 2008). In other experiments, patients with more discrete lesions to the OFC were either unimpaired on two different versions of this same task (Manes et al., 2002; Clark et al., 2003) or impaired at the quality of decision making but showed decreased betting behavior (Rogers et al., 1999a). In healthy subjects, choice of large reward/high-risk options compared to low-reward/lowrisk outcomes activates the OFC during performance of different cost/benefit decision making tasks (Rogers et al., 1999b; Ernst et al., 2004; Fukui et al., 2005). In a number of different contexts, activity of the vmPFC, and the OFC in particular, is thought to be related to the value of different stimuli and decision options (O’Doherty et al., 2003; Padoa-Schioppa and Assad, 2006;  6  Kable and Glimcher, 2007; Plassmann et al., 2007; Hare et al., 2008). Therefore, the role for the vmPFC and OFC in these cost/benefit tasks may be to encode reinforcement expectancies (incentive information), which is then used to guide decision making (Schoenbaum and Setlow, 2001; Knutson et al., 2005). Indeed, these regions strongly respond to increasing reward magnitude (Arana et al., 2003; Rogers et al., 2004; Blair et al., 2006; Yacubian et al., 2006; Knutson and Bossaerts, 2007; Marsh et al., 2007; Tom et al., 2007), which would be expected if this area was signaling reinforcement expectancies. Insular Cortex Damage to the insular cortex, particularly the agranular anterior portion in humans, is also associated with alterations in risk-related judgments that involve monetary reward (Bar-On et al. 2003; Clark et al. 2008). These patients are able to make advantageous decisions (i.e., they correctly choose options yielding the most reward), but do not adjust their betting according to gain/loss probability, suggesting that they are less “risk sensitive” than healthy controls. Neuroimaging studies have shown that performance on the IGT activates the insula compared to a control task (Ernst et al., 2002). This region typically activates more during monetary losses or punishments than gains on reversal learning tasks (O’Doherty et al., 2003; Cohen et al., 2008) and activation of the insula is often associated with anticipating risk and risk aversion (Critchley et al., 2001; Huettel et al., 2005; Knutson and Bossaerts, 2007; Preschoff et al., 2008), which suggests that this region may be particularly important for processing the impact of potential losses that could lead to adjustments in behavior. Indeed, increased insular activation occurs when a subject switches from one stimulus to the other following a loss compared to when they stay with the same choice (O’Doherty et al., 2003; Paulus et al., 2003; Cohen et al., 2008) or prior to safe choices compared to risky ones (Kuhnen and Knutson, 2005). Therefore, the anterior insula may  7  contribute to decision making situations that involve monetary punishments, perhaps to facilitate behavioural shifts to more advantageous options. Dorsolateral PFC The initial studies using the IGT to examine decision making did not find impaired performance for subjects with dorsolateral PFC (DLPFC) or dorsomedial PFC (DMPFC) damage as compared to those with vmPFC damage (Bechara et al., 1998). From other groups using the IGT, the evidence has been inconsistent, with some studies showing increased risky choice in these patient groups (Manes et al., 2002; Fellows and Farah, 2005), whereas others suggest only large frontal lesions will produce deficits on decision making tasks (Clark et al., 2003). DLPFC and DMPFC patients are often also impaired on various measures of executive function, which may have affect their performance on the IGT (Manes et al., 2002). Using measures of risk-based decision making where the probabilities associated with the different response options are made explicit to the subject, there has been some evidence of decreased risk taking in DLPFC patients when betting accumulated money on response options, although the effect was small (Rogers et al., 1999a). Although the evidence from lesion studies suggests an involvement of DLPFC in riskbased decision making, the precise impairments associated with damage to this region remain unclear. DLPFC activity in healthy individuals is also increased during performance of the IGT (Ernst et al., 2002) and the Game of Dice Task (Labudda et al., 2008) although activation does not always correlate with performance. In a study looking at choices among possible gains only, subjects’ preferences for ambiguous options compared to certain ones significantly activated the DLPFC, and this activity was again specific to the decision phase of the task (Huettel et al., 2006). In a more medial region of the dorsal prefrontal cortex, activation has been linked to uncertainty  8  about predicting the probability of events and was negatively correlated with reward probability (Volz et al., 2003; 2004). It is possible that the DLPFC is one of multiple structures involved in the integration of information about probabilities and incentives of different choice options (Labudda et al., 2008). Medial/Anterior Cingulate Greater activation in the anterior cingulate region of the PFC (ACC) has been associated with a variety of choice conditions, including when subjects make risky relative to safe choices on the IGT (Lawrence et al., 2009) as well as when choosing low probability rewards compared to high probability rewards (Paulus and Frank, 2006; Smith et al,. 2009), which may reflect the role of the ACC in conflict detection because low probability rewards may be viewed as highly conflicting. Depending on the type of task used, increased cingulate cortex activation has been linked to both increasing potential or received gain (Knutson et al., 2003; Rogers et al., 2004; Tom et al., 2007) as well as to loss or negative feedback (Bush et al., 2002; Blair et al., 2006; Marsh et al., 2007; Cohen et al., 2008), although the specific foci of these activations often occur in different subregions (e.g. middle posterior, dorsal vs. rostral ACC), suggesting possible functional dissociations between cingulate subregions. The ACC may also respond to negative feedback by detecting aversive outcomes to adjust behavior to get more reward, similar to the anterior insula. Medial PFC regions, including the ACC, are consistently activated following presentation of negative feedback (Holroyd and Coles, 2002; Rushworth et al., 2004; Brown and Braver, 2005), especially when freely choosing a new rule (Walton et al., 2004) or when changes in reward contingencies require a modified behavioral response (Bush et al., 2002; Cohen et al., 2008). Activity in the ACC has also been shown to predict the amount of adjustment following punishments (Wrase et al., 2007).  9  However, it is difficult to use neuroimaging to parse out whether activations of the ACC are necessary for decision making behaviour or if they are merely the result of its role in executive attention, supervisory attentional control, conflict monitoring, and response selection (Elliott and Dolan, 1998; Ridderinkhof et al., 2004; Rushworth et al., 2004; Walton et al., 2004; Pochon et al., 2008). There are few studies with populations of patients with selective damage to Area 32 of the anterior cingulate. One study recruited patients with vmPFC lesions in which some had damage that included Area 32, and these patients displayed impaired decision making on the Cambridge Gamble Task (Clark et al., 2008). However, the limited sample makes it difficult to determine whether this region makes a necessary contribution to risk-based decision making.  Prefrontal-Subcortical Circuitry Different regions of the PFC have been anatomically identified in the primate and rodent brain and are intricately linked to two subcortical structures that are also implicated in risk-based decision making: the amygdala and the nucleus accumbens (Groenewegen et al., 1990). Generally speaking, this cortical-subcortical circuit is thought to be involved in the expression of motor behaviour driven by motivationally and emotionally relevant stimuli (Robbins et al., 1989; McDonald, 1991a,b; McDonald et al., 1996). These regions are discussed in greater detail below. The Amygdala The amygdala is has traditionally been associated with processing emotional (particularly fear-related) stimuli, as well as both Pavlovian and instrumental conditioning in animals (Paton et al., 2006; Belova et al., 2007, 2008; Salzman et al., 2007; Seymour and Dolan, 2008). In contrast to studies examining the effects of PFC damage in humans, consistent impairments in decision making in real-life social settings (Tranel and Hyman, 1990) and on different laboratory decision making tasks (Bechara et al., 1999; Brand et al., 2007; Weller et al., 2007), have been observed in  10  patients with damage to the amygdala. Typically, these individuals are more likely to choose options associated with larger rewards, but also greater chances of reward omission or loss, suggesting a possible impairment in determining the long-term value of different choice options. In line with this, patients with amygdala damage do not develop anticipatory skin conductance responses during anticipation of losses in the IGT as do controls, suggesting that information about the value of different outcomes may not be signaled appropriately in these individuals (Bechara et al., 1999). Moreover, subjects with focal bilateral amygdala damage show a dramatic reduction in loss aversion compared to matched controls (De Martino et al., 2010), which is consistent with the amgydala’s role in response inhibition to fear reducing stimuli (Delgado et al., 2008). Neuroimaging studies of normal human subjects have also implicated the amygdala in different types of decision making. When faced with decisions about outcomes that vary in magnitude, greater amygdala activity is seen when an individual wins a large payout versus not winning that same payout (Blair et al., 2006; Marsh et al., 2007; Smith et al., 2009). The amygdala is also activated simply by viewing high incentive restaurant menus compared to low incentive menus (Arana et al., 2003). However, other studies have also shown amygdala activity to be involved in processing negative stimuli, choices and outcomes (Breiter et al., 1996; O’Doherty et al., 2001; Yacubian et al., 2006). It has been proposed that separate populations of neurons in the amygdala may encode information about reward vs. punishment (Paton et al., 2006; Cohen et al., 2008). Indeed, a recent study using single unit recordings of amygdala neurons in human patients with epilepsy showed that amygdala neural firing was linearly related to the value of different food items and similar proportions of neurons exhibited positive and negative associations (Jenison et al., 2011). Combination of these two types of information could convey a  11  “value” signal that might be used by other cortical and subcortical regions to dictate the best course of action in a given context. The PFC shares reciprocal glutamatergic projections with the amygdala, particularly with the basolateral (BLA) region in the rat (Sesack et al., 1989; McDonald, 1991a; McDonald et al., 1996). Excitatory inputs from the amygdala to medial PFC pyramidal neurons arise predominately from the caudal parts of the BLA (McDonald, 1991a; Shinonaga et al., 1994; McDonald et al., 1996). These ipsilateral projections from the BLA to the frontal cortex exhibit a topographical arrangement, such that medial areas of the BLA project to the medial areas of the PFC (i.e.; prelimbic cortex; Sripanidkulchai et al., 1984; McDonald, 1987), whereas projections to more lateral PFC areas (e.g. agranular insular cortex) arise primarily from the ventrolateral and rostral areas of the BLA (Krettek and Price, 1977; McDonald, 1991a). The prelimbic cortex, in particular, tends to receive substantially more BLA projections than other areas of the medial PFC (Krettek and Price, 1977). Although these monosynaptic projections are glutamatergic, they synapse on both principle pyramidal cells and GABAergic interneurons in the PFC. Therefore, in vivo in the anesthetized rat, stimulation of BLA inputs to the PFC often produces sub-threshold EPSPs followed by inhibitory responses (Dilgen and O’Donnell, 2006) so that the net effect of these inputs on spike firing of projection neurons is typically inhibitory. Prefrontal inputs to the amygdala are also glutamatergic and preferentially target the same regions that projections from the BLA to the PFC originate in (Sripanidkulchai et al., 1984; McDonald, 1987; Sesack et al., 1989; McDonald, 1991a; McDonald et al., 1996). Again, although the monosynaptic pathway between the medial PFC and BLA is glutamatergic (Rosenkranz and Grace, 2001; Sotres-Bayon et al., 2004), the output of PFC stimulation tends to inhibit amygdala activity (Likhtik et al.,  12  2005), likely through medial PFC projections to inhibitory interneurons in the amygdala (Sesack et al., 1989; Freedman et al., 2000; Vertes et al., 2004; Berretta et al., 2005). The Nucleus Accumbens of the Ventral Striatum Imaging studies have been particularly useful in identifying certain subcortical structures in the mediation of decision making that are typically not damaged in human patients, such as the ventral striatum. The nucleus accumbens (NAc) region of the ventral striatum has substantial afferents from cortical, subcortical, thalamic and other inputs, as well as efferents to the pallidum and motor effector sites, and thus, has been termed a “limbic-motor interface” that guides motivationally relevant behaviour in a goal-directed fashion (Mogenson et al., 1980). In the neuroimaging literature, the NAc has consistently been associated with increasing anticipated (Knutson et al., 2001) or experienced (Ballard and Knutson, 2009) reward. Moreover, decreased activation is sometimes observed following losses (Delgado et al., 2000), suggesting a role for this region in signaling reward value. However, activation of the NAc has also been shown to be correlated with making risky choices (Kuhnen and Knutson, 2005; Tom et al., 2007). In one study (Kuhnen and Knutson, 2005), subjects chose between two risky “stock” options that could lead to either large rewards or losses, or a safe “bond” that always yielded a smaller reward. NAc activation preceded risky choices as well as risk-seeking mistakes. However, given that choosing risky vs. safe options is often intermingled with choices with high expected value, it can be difficult to ascertain whether the primary role of the NAc is to encode expected value of choice options or to promote risky choice, independent of expected value. Preuschoff and colleagues (2006) observed that activation of the NAc showed both a rapid response associated with expected value (peaked when the cue indicated a 100% chance of winning) and a sustained response associated with uncertainty (peaked when the cue indicated a 50% chance of winning) in a manner  13  similar to the physiological activity of midbrain DA neurons in response to cued rewards (Fiorillo et al., 2003), suggesting that the NAc may be one terminal region that receives important information about reward value and uncertainty from the midbrain. Currently, there does not appear to be a solid consensus among human imaging literature on the precise role of the NAc in decision making processes. Both the medial PFC and BLA send direct glutamatergic projections to the NAc (Krettek and Price, 1977; McDonald, 1991b; Brog et al., 1993), which then sends projections to the ventral pallidum (Mogenson et al., 1980). BLA to NAc inputs are also topographically organized, such that the medial to lateral coordinate in the striatum corresponds to the medial to lateral coordinate from the BLA inputs (McDonald, 1991b). Moreover, the rostral parts of the BLA project more to the core subregion of the NAc and caudal regions project more to shell subregion (Shinonaga et al., 1994). Therefore, through these connections, the medial PFC, amygdala and NAc are ideally positioned to form an integrated circuit to gather and update information about the magnitude and probabilities of rewards associated with different choice options and use this information to bias an animal towards actions associated with obtaining the greatest amount of reward in a given environmental context. Once a particular course of action has been determined, this transformation of cognitive information into the appropriate behavioral output is likely mediated by connections linking the NAc with pallidal and mesencephalic motor effector sites (Zahm and Heimer, 1990). However, although the different components of this circuit have been implicated in risk-based decision making, it remains to be clarified whether these regions function independently or cooperatively to guide choice behaviour.  14  Rodent Models of Cost/Benefit Decision Making Over the last two decades, there was been great interest in developing animal models of cost/benefit decision making that provide greater experimental control when examining the underlying neural circuitry that mediates this form of behaviour. In these models, animals typically choose between small rewards that have little to no cost associated with them, and larger rewards that are associated with greater costs (e.g. delays, effort, and uncertainty). A number of different animal models have now been developed to examine different aspects of decision making (see Chapter 2). Over the last decade, these rodent models have been instrumental in elucidating the precise neural circuits that mediate different components of cost-benefit decision making. Support for the idea that the amygdala may convey some component of value stems from rodent models of decision making showing that lesions or inactivations of the BLA region of the amygdala alters both delay and effort-based choice in a similar manner, reducing animals’ preference to either work harder or wait longer for larger rewards (Salamone et al., 1991; Cardinal et al., 2001; Winstanley et al., 2004, Ghods-Sharifi et al., 2009). Importantly, these manipulations do not impair discrimination between rewards of different magnitudes when the effort or delay costs associated with both rewards are equal. Zeeb and Winstanley (2011) examined the effect of either pre or post-training BLA lesions on performance of a rodent analogue of the IGT in humans. Lesions made prior to training slowed acquisition of choosing the advantageous options, whereas lesions made post-training reduced choice of the most advantageous pellet option and increased choice of the least favorable reward option, suggesting that BLA lesions may interfere with estimations of the long-term value of different response options. Similarly, we recently examined the effects of reversible inactivations of the BLA on probabilistic discounting, in which we observed a decreased preference for the large, yet probabilistic option following disruption of  15  the BLA, particularly when reward delivery was most uncertain (Ghods-Sharifi et al., 2009). Again, inactivation of the BLA did not disrupt the ability to choose the larger reward over the smaller reward when they were both delivered with 100% probability in a simple reward discrimination task, further supporting the notion that the BLA is not required to make simple “big” versus “small” reward judgments. Thus, disruptions of BLA activity similarly reduce preference for larger, more costly rewards across multiple types of decision making without disrupting reward magnitude discrimination. Notably, these findings are somewhat discrepant with those of studies with patients with amygdala damage, who make more risky choices on decision making tasks as well in real-life social settings (Tranel and Hyman, 1990; Bechara et al., 1999; Brand et al., 2004; Weller et al., 2007). However, upon closer inspection, the risky patterns of choice observed in human amygdala patients ultimately yields less overall reward over the course of a session; similarly, animals with disruptions of the amygdala either opt for smaller rewards that come with a lesser cost and/or fail to make any choice (Ghods-Sharifi et al., 2009) as well as larger rewards that also deliver larger timeout punishments (Zeeb and Winstanley, 2011), which also lead to less overall reward over repeated trials. Thus, these findings support the idea that disruptions of the amygdala generally interferes with processes that bias choice behavior towards options leading to greater long term payoffs, which, depending on the task at hand, would induce impulsive, lazy, impaired or risky patterns of choice (Ghods-Sharifi et al., 2009; Winstanley et al., 2004; Zeeb and Winstanley, 2011). Findings from animal models of decision making also help clarify the specific functions mediated by the NAc in these processes. Similarly to the BLA, lesions or inactivations of the NAc reduce animals’ preference to either work harder or wait longer for larger rewards (Salamone et  16  al. 1991; Cardinal et al., 2001). Cardinal and Howes (2005) found that lesions of the lateral core region of the NAc caused a pronounced risk-averse pattern of choice on a probabilistic discounting task, with rats preferring the smaller but certain reward, particularly when the probability of receiving the large reward was high. More recently, Stopper and Floresco (2011) examined the role of the core and shell subregions of the NAc, in a similar probabilistic discounting paradigm. Reversible inactivations of the entire NAc, as well as more discrete inactivation of the shell subregion, produced a similar decrease in choice of the large, risky option as observed by Cardinal and Howes (2005), primarily during the first three probability blocks where the value of the large reward lever was higher than the small reward lever. Similar disruptions of NAc activity slightly reduced preference for large over small rewards in a reward magnitude discrimination task and also decreased choice in a fixed risk task where the probability of the large reward was fixed at 40% for the entire session. These findings support the notion based on human neuroimaging that the NAc contributes to estimations of reward size and expected value during risk-based decision making. Dissociable regions of the rat PFC also appear to mediate different forms of these cost/benefit judgments. The OFC forms a critical component of the circuitry underlying delaybased decision making (Winstanley et al., 2004; Rudebeck et al., 2006), whereas the rat dorsal anterior cingulate has been implicated in decisions with effort costs associated with larger rewards (Rudebeck et al., 2006; Floresco and Ghods-Sharifi, 2007). Interestingly, there have been relatively few studies on the involvement of different regions of the rat PFC in decision making with probabilistic/risky outcomes. Mobini and colleagues (2002) reported that large lesions of the OFC (encompassing both lateral and medial PFC regions) increased preference for a small, but certain food reward, relative to a larger reward delivered in a probabilistic manner. Another study  17  used a procedure modeled after the IGT, where rats learned the contingencies of decision alternatives in one session and adjusted choice based on previous outcomes (Pais-Vieira et al., 2007). Rats with OFC lesions developed a preference for the high risk/large reward over the low risk/small reward option, even though the expected value of the two options were equal. In both of these studies, lesions were induced prior to training on the task, indicating that the OFC may play a role in the initial learning of reward magnitude and probability contingencies. In contrast, the contribution of the rat OFC to judgments after animals have learned the relative value of different response options is unknown. Moreover, to our knowledge, there have been no studies on the contribution of other subregions of the rodent medial PFC or insular cortex to risk-based decision making.  Dopaminergic Modulation of Prefrontal-Subcortical Circuitry All nodes of this complex prefrontal-subcortical circuit receive dense dopaminergic input from the ventral tegmental area (VTA; Beckstead et al., 1979). As discussed above, based on both human and rodent literature, different regions of the PFC seem to play important roles in our ability to choose among certain and uncertain rewards . The majority of DA inputs from the VTA innervate the prelimbic and infralimbic subregions of the PFC (Sesack et al., 1995). DA receptors in these regions are slow-acting and typically modulate other receptor systems and/or ion channels (Lachowicz and Sibley, 1997). Generally speaking, activating DA receptors does not produce large postsynaptic currents that can be measured electrophysiologically (Yang and Seamans, 1996). In the PFC, D1 and D2 receptors are localized on both pyramidal neurons and fast-spiking interneurons in both the primate (Goldman-Rakic et al., 1989; Verney et al., 1990; Sesack et al., 1995; Gorelova et al., 2002) and rodent (Vincent et al., 1993). Therefore, DA has the potential to  18  exert both direct and indirect effects on the excitability of PFC pyramidal neurons (Yang and Seamans, 1996; Gorelova et al., 2002; Seamans and Yang, 2004). This anatomical arrangement of DA inputs to the PFC suggests that perturbations in DA transmission may hamper decision making processes related to probabilistic outcomes. Indeed, aberrations of the mesocorticolimbic dopamine DA system have been linked to profound deficits in decision making associated with certain psychiatric diseases. Patients with schizophrenia (Hutton et al., 2002), Parkinson’s disease (Pagonabarraga et al., 2007), and stimulant abusers (Rogers et al., 1999a) all display abnormal choice behaviour when challenged with a number of decision making tasks. However, many of these studies used medicated patients, making it difficult to parse out whether these impairments are due to disruptions in endogenous DA transmission, direct effects of the drugs, or both. This may explain why increased choice of risky options compared to more safe options has been observed in patients afflicted with disorders thought to be associated with either abnormal increases (e.g.; schizophrenia) or decreases (e.g.; Parkinson’s) in DA transmission. Moreover, healthy individuals with acute catecholamine depletion continued to pursue a probabilistic option when potential losses were large compared to when they were small suggesting that potential decreases in DA activity may disrupt the ability to adjust to negative outcomes (Scarna et al., 2005). However this particular treatment alters all catecholamines so the results cannot be necessarily attributed to the DA system. Although a number of decision making paradigms have reported activation of the midbrain during decision making (Tom et al., 2007; Rao et al., 2008), imaging data often cannot differentiate between activation of the substantia nigra vs. VTA or between activation of DA or GABA neurons in the midbrain, again limiting the conclusions that can be made about the DA system during decision making in humans. Moreover, these individuals may be experiencing alterations in DA  19  transmission in any of a number of DA terminal regions so it is currently unclear whether the DA in the PFC is mediating these decision making impairments in humans.  Summary and Objectives Converging evidence from both human and animal literature suggests that alterations in different nodes of mesocorticolimbic circuitry may impair cognitive operations associated with making cost/benefit evaluations about certain vs. probabilistic rewards. However, the specific loci and laterality of damage in human studies varies considerably and the correlational nature of imaging does not allow us to determine which brain regions make necessary contributions to this form of decision making. Thus, important questions about the specific functions of these circuits are left unanswered. The process by which different organisms make decisions throughout their daily life is an enormous research domain. This dissertation attempts to understand the process by which rats choose between certain and probabilistic food rewards and reveal the underlying neural circuits that may be contributing to this form of behaviour. By comparing the findings from both sources of research, it is hoped that the current findings may provide some insight into how similar processes take place in the human brain. However, that there are other forms of decision making that may rely on different cognitive processes and recruit different neural circuits than those involved in choosing among probabilistic gains (Estle et al., 2006). For example, there is now an increasing literature on the neural basis of selective punishment in decision making (Simon et al., 2009), which will provide insight into the similarities and differences between these systems. Moreover, although humans exhibit similar decision making about food and monetary reward (Estle et al., 2007) and different organisms have similar preferences for foods that vary either quantitatively or qualitatively (Kalcenik and Bateson, 1996), it is possible that choices involving  20  other forms of rewards (e.g. access to mating, cars, jobs) may involve different neural substrates. Therefore, the scope of this dissertation and its implications is restricted to understanding choices among certain and probabilistic gains. The term “risk-based decision making” used in this thesis is defined as choosing among options where there is variance in the probability of receiving (not receiving) potential reward. This dissertation attempts to address the following research questions via four specific aims: 1. Examine probabilistic discounting as a rodent model of risk-based decision making. The task will first be described thoroughly. Thereafter, using statistical modeling, the development of discount rates over time and the influence of different outcomes (e.g.; winning or losing following choice of the larger, risky option) at different time points on future choice will be assessed in a large sample of rats. This analysis will provide key information about how rats learn this task and what potential inputs may drive individual differences in decision making behaviour. 2. Identify the specific contribution of different subregions of the PFC to risk-based decision making. This will be accomplished using a series of reversible inactivations of different PFC subregions prior to performance of the probabilistic discounting task. These data will help identify which particular brain regions make necessary contributions to risk-based decision making and what these specific functions are. 3. Identify the specific neural circuits connecting the PFC, amygdala, and nucleus accumbens that may mediate risk-based decision making. These experiments will adopt a functional disconnection approach consisting of unilateral inactivations of one structure in combination with unilateral inactivation of the other structure in the contralateral hemisphere. These data will reveal whether cortical-subcortical communication drives  21  discounting of probabilistic rewards and whether these interactions occur in a top-down or bottom-up manner. 4. Clarify whether DA inputs to the PFC modulate risk-based decision making. Local administration of selective agonists and antagonists of D1 and D2 receptors to PFC regions identified in Chapter 2 will determine whether DA activity in the PFC biases choice behaviour as a result of D1 or D2 activation or blockade. The experimental data in this thesis will be presented in the following four chapters. Each chapter will have a short introduction, which will provide the rationale for each experiment/analysis. In each chapter, there will also be a discussion section which deals with specific aspects of the data presented in the chapter. The general discussion will not deal with specific aspects of the data but will attempt to integrate the findings from each chapter into a broader view of how this mesocorticolimbic circuit is involved in decisions making and executive functioning.  22  CHAPTER 2: PROBABILISTIC DISCOUNTING AS A MODEL OF RISK-BASED DECISION MAKING  Introduction There has been growing interest across numerous academic fields into how human and nonhuman animals make decisions between different options that vary in size, time, effort, and probability of occurrence. A multitude of paradigms have tried to both describe decision making behaviour and make predictions based on different sources of information. There are advantages and disadvantages to these different approaches in their ability to accurately explain decision making behaviour. For example, in traditional normative economic models of decision making, such as expected utility theory, people are expected to make either correct or incorrect (rational/irrational) decisions that maximize the expected value of rewards (Friedman and Savage, 1942; von Neumann and Morgenstern, 1944). Decision makers have to conform to specific assumptions in order to be regarded as rational. Similar energy rate-maximizing theories exist regarding animal foraging (Stephens and Krebs, 1986), which predict that animals would be insensitive to the risk associated with different foraging options as long as the total amount gained in the least amount of time was maximized. In isolation, we would expect animals to choose food over no food, gain over loss, no pain over pain, etc. However, the majority of animal species evolved in unpredictable, often treacherous, environments, where each action selected could result in success (food, water, sex) or failure (starvation, pain, death) on any given day. While normative decision making theory can predict basic foraging behaviour with some success (Stephens and Krebs, 1986), there are numerous examples of behaviour in human and non-human animals that do not conform to these predictions (Rachlin and Green, 1972; Ainslie, 1974; Kacelnik and  23  Bateson, 1996; Marsh and Kacelnik, 2002; Isles et al., 2003; Chen et al., 2006; Kalenscher and Pennartz, 2008; Kim et al., 2008; Louie and Glimcher, 2010). All humans and non-human animals must struggle with decision making problems that they need to solve in order to successfully survive and reproduce. Paul Glimcher (2004) argues that all behaviour of an animal can be considered a form of decision making. The most basic actions related to foraging, sex, and survival are neither predetermined nor completely random, but can be predicted to some degree using Bayesian probability theory based on information about the animal’s past and present environment. Although we would like to fully predict what an animal is going to do in a given situation, behavior is probabilistic, not deterministic. Indeed, research in game theory indicates that the optimal strategy for competitors is to adopt some randomization into their behaviour (von Neumann and Morgenstern, 1944; Maynard Smith, 1982), perhaps because through the process of natural selection, we developed optimal brains, not optimal solutions (Glimcher, 2004). Rather than designating brain systems for specific environmental problems (e.g.; finding food in winter), we have brains that can adapt and solve new problems that arise. Unlike the laboratory, where conditions are nearly perfectly controlled by the researcher, the natural environment for a foraging animal is much more uncertain, and both positive and negative outcomes can be a source of information about the value of different actions. Therefore, what may be important is not whether animals choose available reward on every occasion, but how they determine which actions are more profitable, on average, than other actions (Gallistel and Gibbon 2000; Rushworth et al., 2008). Glimcher argues that we should “employ probabilistically-based approaches wherever possible to understand how the brain takes information from the outside world and uses that information in concert with stored representations of the structure of the world to achieve defined computational goals” (Glimcher,  24  2004, p.321). Rather than focusing on correct versus incorrect decisions, making estimations about the probability of a given action based on previous knowledge and experience may provide a more realistic insight into how our brains solve these problems.  Cost/Benefit Decision Making: Discounting Models When faced with a choice between a small and a large reward, human and non-human animals generally choose the larger, more valuable reward the majority of time after exposure to both options. However, when a cost is added to the large reward, the value of that reward decreases and animals shift to choosing other options that are smaller, but less costly. Animals are thought to monitor the changing costs, assess the characteristics of different choice options, and generally prefer the one with the current higher subjective value. This type of decision making is termed “discounting”. In temporal and probability discounting models in humans, people are typically asked for their preferences for different hypothetical amounts of money that vary in magnitude, delay, probability, and gain or loss (Holt et al., 2003; Green and Myerson, 2004; Estle et al., 2006). For example, people choose between two amounts, one that is 100% certain and the other is probabilistic. A number of probabilities are used (e.g. between .95 and .05; Estle et al., 2006) presented in decreasing order. The first choice at each probability is between an amount to be received with a certain probability (e.g. 100 for sure or $200 with a 95% chance). For each of the subsequent trials, the amount of the certain gain is adjusted based on the participants’ previous choice, while the probabilistic amount remains constant. In some studies, if the participant chose the certain amount, then on the next trial this amount was decreased; if the participant chose the probabilistic amount, then the certain amount was increased on the next trial. Subjective value is calculated as the amount halfway between the largest certain amount that was preferred to the  25  probabilistic amount and the smallest certain amount that was preferred to the probabilistic amount (i.e., an “indifference point”). It is well established that these indifference points are best characterized by a hyperbolic, rather than an exponential, mathematical function (Rachlin et al., 1991; Green et al., 1999). These models have been somewhat successful in examining how drug exposure or various addictions alter indifference points in delay discounting models as a measure of impulsive behaviour (Vuchinich and Simpson, 1998; Bickel et al., 1999; Madden et al., 1999) as well as relating measures of obesity to discount rates (Rasmussen et al., 2010). Although initial quantitative studies (Mazur, 1989) suggested that temporal and probability discounting processes are similar, increasing evidence from behavioural, psychopharmacological and lesion studies point to dissociable cognitive and neural mechanisms underlying these forms of decision making (Mobini et al., 2000; Holt et al., 2003; Green and Myerson, 2004; Winstanley et al., 2004; Adriani and Laviola, 2006; van Gaalen et al., 2006; St.Onge and Floresco, 2009; Simon et al., 2009; St.Onge and Floresco, 2010).  Rodent Models of Temporal and Probabilistic Discounting The above discounting models, as well as other types of cost/benefit decision making tasks used in humans (e.g. the IGT; Bechara et al., 1994), stimulated interest in developing animal models of cost/benefit decision making to study the neurobiological basis of decision making, for which there are now many different types (Salamone et al., 1991, Evenden and Ryan, 1999; Kaminski and Ator, 2001; Mobini et al., 2002; Winstanley et al., 2004; Cardinal and Howes, 2005; Pais-Vieira et al., 2007; Floresco et al., 2008b; Zeeb and Winstanley, 2011). These models examine different questions related to decision making and there are advantages and disadvantages to each model; however a full discussion is beyond the scope of this thesis. The predominant decision making model currently used in animal research is discounting.  26  In these studies, rats choose between smaller rewards associated with a nominal cost, and larger, yet more costly rewards (Floresco et al., 2008b). For example, “delay discounting” tasks are used as a measure of impulsive decision making, where response costs are varied by imposing a delay before delivery of a larger reward vs. acquiring an immediate, smaller reward (Evenden and Ryan, 1999; Cardinal et al., 2000; Winstanley et al., 2004). Alternatively, in “effort-based” decision making, animals choose between a small reward obtainable after a nominal amount of physical effort, or obtaining a larger reward after considerably more work (e.g.; climbing a barrier, pressing a lever multiple times) (Salamone et al., 1991; Floresco et al., 2008a). Many of the initial quantitative models in rodents used similar discounting paradigms as those in humans studies except that they utilized between session (day) shifts in reinforcer characteristics (e.g. magnitude, delay or probability) in order to identify the indifference points because of the limitations of food as a reinforcer (i.e.; animals become sated with too many trials in one day). Subjective value in these models of both temporal and probability discounting is represented by a multiplicative combination of hyperbolic discounting functions, incorporating the organisms’ sensitivity to reinforcer size, delay, and odds against reinforcement (Ho et al., 1999). However, although individual differences in indifference points could be detected, initial models that involved combining separate discounting parameters for size and response costs made it difficult to interpret what aspect of decision making was specifically altered by different brain manipulations. Therefore, future models were altered so that a change in slope indicated an alteration in processing reinforcer magnitude, whereas an alteration in the intercept but not the slope is explained by changes in processing delay only (Kheramin et al., 2002). Similarly, when decisions involve both delay and uncertainty, changes in sensitivity to reinforcer uncertainty  27  should influence both the slope and the intercept, whereas changes in delay should only affect the intercept (Kheramin et al., 2003). However, between-session shifts in reinforcer costs like that used in the above models are not amenable to examining the neural circuitry that may be mediating this type of decision making. In addition to the long training regimens required to expose animals to sufficient variation in delays, effort, or probabilities, it is difficult to assess the effects of brief disruptions in brain function on choice behaviour in order to determine what brain circuits are necessary for decision making under relatively familiar conditions. In these studies, animals must undergo brain lesions either before training, which places the emphasis on how reinforcer contingencies are learned as opposed to how choices are made, or after training, which is confounded by compensatory changes and new learning over time. For example, Kheramin and colleagues (2003) lesioned the rat OFC prior to training and observed increases in the rate of both delay and probability discounting. Furthermore, this type of model was used to show that destruction of the ascending serotonergic pathways promotes preference for small, immediate rewards over large, delayed ones, but has no effect on probability discounting (Mobini et al., 2000). However, these studies could not determine whether these choice biases were the result of compromised input or output decision information or altered learning of reinforcer contingencies, which may be separate processes. Therefore, discounting models that incorporate within-session shifts in response costs may permit the examination of temporary disruptions of brain activity on choice behaviour once stable preferences have been developed in order to determine what brain circuits and neurochemical systems make necessary contributions to decision making. Moreover, an alternative design to permanent lesions is to use transient inactivations of brain activity in the same animals, which provide a within-subject control and more statistical power.  28  Current Research: Probabilistic Discounting in Rats Five years ago our laboratory began using a probabilistic discounting task originally developed by Cardinal and Howes’ (2005), with slightly modified procedures, in order to study the neural circuits mediating decision making under risk. The methodology of this paradigm is fully described below. Lever Pressing Training Initially, animals are trained to learn to press levers for food. On the day prior to their first exposure to the chambers, rats are given approximately 25 sugar reward pellets in their home cage. On the first day of training, 2-3 pellets are delivered into the food cup and crushed pellets were placed on a lever before the animal was placed in the chamber. Rats are first trained under a fixedratio 1 schedule to a criterion of 60 presses in 30 min, first for one lever, and then repeated for the other lever (counterbalanced left/right between subjects). Rats are then trained on a simplified version of the probabilistic discounting task. These 90 trial sessions begin with the levers retracted and the operant chamber in darkness. Every 40 s, a trial is initiated with the illumination of the houselight and the insertion of one of the two levers into the chamber. If the rat fails to respond on the lever within 10 s, the lever is retracted, the chamber darkened and the trial scored as an omission. If the rat responds within 10 s, the lever is retracted and a single pellet delivered with 50% probability. This procedure is used to familiarize the rats with the probabilistic nature of the full task. In every pair of trials, the left or right lever is presented once, and the order within the pair of trials is random. Rats are trained for approximately 5-6 days to a criterion of 80 or more successful trials (i.e.; ≤ 10 omissions).  29  Probabilistic Discounting Task Rats receive daily sessions consisting of 72 trials, separated into 4 blocks of 18 trials. The entire session takes 48 minutes to complete, and animals are trained 6-7 days per week. A session begins in darkness with both levers retracted (the intertrial state). A trial begins every 40 s with the illumination of the houselight and, 3 s later, insertion of one or both levers into the chamber (the format of a single trial is shown in Figure 1).  Figure 1. Schematic of the probabilistic discounting task. A Cost/benefit contingencies associated with responding on either lever. B Format of the sequence of forced and free choice trials within each probability block of a training session. C Format of a single free-choice trial.  One lever is designated the Large/Risky lever, the other the Small/Certain lever, which remains consistent throughout training (counterbalanced left/right). If the rat does not respond by pressing a lever within 10 s of lever presentation, the chamber is reset to the intertrial state until the next trial (omission). When a lever is chosen, both levers retract. Choice of the Small/Certain lever always delivers one pellet with 100% probability; choice of the Large/Risky lever delivers 4 pellets but with a particular probability (see below). When food is delivered, the houselight  30  remains on for another 4 s after a response is made, after which the chamber reverts back to the intertrial state. Multiple pellets are delivered 0.5 s apart. The 4 blocks are comprised of 8 forcedchoice trials where only one lever is presented (4 trials for each lever, randomized in pairs) permitting animals to learn the amount of food associated with each lever press and the respective probability of receiving reinforcement over each block. This is followed by 10 free-choice trials, where both levers are presented and the animal chooses either the Small/Certain or the Large/Risky lever. In the original version of the task, the probability of obtaining 4 pellets after pressing the Large/Risky lever decreases across trial blocks: Using these probabilities, selection of the Large/Risky lever would be advantageous in the 100 and 50% probability blocks, and disadvantageous in the 12.5% block, whereas rats could obtain an equivalent number of food pellets after responding on either lever during the 25% block. Therefore, in the probabilistic trial blocks of this task, selection of the larger reward option carried with it an inherent “risk” of not obtaining any reward on a given trial. Rats are trained on the task until as a group, they (1) choose the Large/Risky lever during the 100% probability block on at least 80% of successful trials, and (2) demonstrate stable baseline levels of choice for 3 consecutive days. In brief, if the effect of Trial Block is significant at the p<.05 level but there is no main effect of Training Day or Training Day X Trial Bock interaction (at p>.10 level), animals are judged to have achieved stable baseline levels of choice behavior. Although as a group we train rats until they show a discounting pattern of choice, we typically observe substantial individual variability in choice of the Large/Risky vs. Small/Certain options, similar to other rodent models of discounting (Adriani et al., 2009). Until now, we have not characterized this variability in a formal manner.  31  Over the past five years, our lab has collected choice data from over 200 rats performing this probabilistic discounting task. The primary measure of interest has been the percentage choice of the Large/Risky lever in each of the probability blocks. Using this measure, we have been able to identify different neural substrates that make necessary contributions to choice behaviour (see Chapters 3-5). However, we have also been interested in formally examining the development of discount rates in this paradigm and determining potential parameters that may influence preferences for large, probabilistic vs. small, certain rewards. Because we used a within-session paradigm to ideally suit our neurobiological experiments, the quantitative models discussed above would not be appropriate to examine these data; thus, we prepared a novel model to examine how choice outcomes affect future choices on this task. First, some issues need to be addressed. Animals trained on this task have to learn about the magnitude and probabilities of receiving the two rewards through trial and error in order to determine the option with the highest value in each block. We do not utilize lights or any other external cue to signal what the probability of the large reward is. However, each probability block begins with 8 forced choice trials in which only one lever extends, facilitating learning of both the magnitude and probability of the reinforcers. Therefore, rats can use the first forced choice trial as an indicator that the probability has changed, but they are still required to select the risky lever and learn by the outcomes specifically what that probability is. By just observing the data in these experiments over the years, we have found that generally it takes the majority of rats 10 days of training before showing even subtle discounting of the large reward and up to 20 days before the behaviour of the group begins to stabilize. In some cases, the discounting curve of a group of animals can be consistent for upwards to 6 months (St.Onge and Floresco, 2009), similar to probabilistic discounting in humans (Peters and Buchel, 2009). One interesting question is how these  32  discounting patterns develop across training days and whether early experiences in choice outcomes can shape what the final stable discounting curves look like. Moreover, to what extent do previous choice outcomes influence preferences for safe or probabilistic options and how far back in time do these outcomes have an effect? Another important characteristic of our task is that for each session and trial block, the probability of receiving the large reward is drawn from a set probability distribution. In other words, the probability of receiving reward after choosing the Large/Risky lever is independent of previous outcomes; thus, each time the rat selects the risky lever in the 50% block, for example, there is a 50/50 chance of receiving 4 pellets on that trial. Across the 10 free choice trials, the experienced probability throughout the whole block is rarely the exact probability (e.g. 50%), but approximates it across many days of training. For example, one day a rat may choose the risky lever 8 times in the 50% block and receive 4 pellets on 6 of those trials, resulting in an experienced probability of 75%. The opposite result could also occur, such that a rat receives the 4 pellets on only 2 of the 8 trials that he picked the risky lever, resulting in an experienced probability of 25%. Across many training days, the actual probability experienced by the rat will approximate the set value. Therefore, on a given day, a rat may be considered “lucky” if it receives more reward than the expected value for that block. Alternatively, the rat may be deemed “unlucky” if he receives less reward than the expected value. Given that rats may incorporate information from the outcomes of each trial across a block in order to determine the expected value of each lever in the four blocks, we were interested in whether experiencing greater or less than expected rewards following choice of the risky lever would predict if the rat preferred that lever in the future. Therefore, in this chapter the following questions are examined:  33  1) How are probabilistic discounting rates characterized and how do they change over training? 2) What is the effect of being “lucky” during the first 5 days of training on the slope of the probabilistic discounting curve during the last 5 days of training (i.e., when stable choice occurs)? 3) What is the effect of being “lucky” in the forced choice trials of a given probability block on the probability of choosing the Large/Risky lever on the first free choice trial? 4) What is the effect of previous choice outcomes on the probability of choosing the Large/Risky lever both early and late in training and how far back in the past (in terms of number of choice trials) does this influence occur?  Method Animals All animals in all experiments were male Long Evans rats (Charles River Laboratories, Montreal, Canada) weighing 275-300 g at the beginning of behavioural training. Upon arrival, rats were given one week to acclimatize to the colony and food restricted to 85-90% of their free-feeding weight for an additional one week before behavioural training. Rats were given ad-libitum access to water for the duration of the experiment. Feeding occurred in the rats’ home cages at the end of the experimental day and body weights were monitored daily to ensure a steady weight loss during food restriction and maintenance or weight gain for the rest of the experiment. All testing was in accordance with the Canadian Council of Animal Care and the Animal Care Committee of the University of British Columbia.  Apparatus Behavioral testing for all experiments described here was conducted in 12 operant chambers (30.5 x 24 x 21cm; Med-Associates, St. Albans, VT., USA) enclosed in sound-attenuating boxes. The  34  boxes were equipped with a fan to provide ventilation and to mask extraneous noise. Each chamber was fitted with two retractable levers, one located on each side of a central food receptacle where food reinforcement (45 mg; Bioserv, Frenchtown, NJ) was delivered via a pellet dispenser. The chambers were illuminated by a single 100-mA house light located in the topcenter of the wall opposite the levers. Four infrared photobeams were mounted on the sides of each chamber. Locomotor activity was indexed by the number of photobeam breaks that occurred during a session. All experimental data were recorded by an IBM personal computer connected to the chambers via an interface.  Data Collection Data were compiled from a series of experiments conducted in the lab over the last five years. All rats were obtained and trained at a similar age in an identical manner. Only training data collected prior to any surgical or pharmacological manipulations were used. In total, the data from 194 animals were used in the analyses. The data were summarized and prepared for statistical analysis using Stata Analysis software (Stata Corp., 2009). Due to complications associated with proportional data, logistic regression models were used to assess the impact of previous choice outcomes on the probability of choosing the risky lever (binary variable). Statistical significance was determined via comparisons of the magnitude of the regression coefficients compared to zero.  Analysis 1: How are probabilistic discounting rates characterized and how do they change over training? Percentage choice of the Large/Risky lever in each probability block was averaged across the last 5 days of training for all rats (n=194) and displayed in histograms. Individual discounting curves for all animals using percentage choice of the Large/Risky lever were plotted. Descriptive statistics were calculated for the estimated slopes of the discounting curves using linear  35  regression. We also generated diagnostic plots to visually display the evolution of the rat’s discounting behavior over time as the rats learned the probabilistic discounting task.  Analysis 2: What is the effect of being “lucky” during the first 5 days of training on the slope of the probabilistic discounting curve during the last 5 days of training (i.e., when stable choice occurs)? In this model we tested the effect of luckiness in early training of the task on a rat’s discounting behavior in the last five days of training. Luckiness was defined as how many more/less pellets a rat received than the expected value of a given probability block based on its choices (e.g. the maximum expected value of pellets in the 50% block is 20 if the rat chose the risky lever all 10 times). We quantified a continuous measure of a rat’s luckiness over a block of trials as the sum of pellets received by a rat through that trial block minus the sum of the expected value of that rat’s choices through the block of trials. The discounting behaviour as a dependent measure was defined as the average slope of a rat’s lever choices over the four probability blocks across the last 5 days of training. For a given rat the relationship between a rat’s choices over a full day of trials and the probability blocks was expressed in a regression model as: P = γΧ + α + ε  (1)  where P is the percentage of choices of the Large/Risky lever made in a given probability block and X is an index of the probability block. The parameter α is an intercept term, ε is a normally distributed random error term with expected mean of zero and γ is the slope of a rat’s discounting curve over the probability blocks. We tested the effect of the sum of luckiness (Z) in the first five days of training on fitted values of the slope of this discounting curve (γ^) in the last five days of training, which are estimated using Equation 1. The effect of luckiness was estimated using the linear regression equation: γ^ = βZ + α + ε  36  Analysis 3: What is the effect of being “lucky” in the forced choice trials of a given probability block on the probability of choosing the Large/Risky lever on the first free choice trial? In this model, we tested the effect of luckiness in forced choice trials of a given probability block on the probability of choosing the Large/Risky lever on the first free choice trial. We only used data from the last 5 days of training (i.e.; the point in training where rats showed stable and prominent discounting). For these analyses we expressed the dependent variable as a binary outcome variable as equal to one if a rat chooses the Large/Risky lever and zero otherwise. This is expressed as: Y=  1 0  if large reward lever chosen otherwise  (2)  We then tested two specifications of the general logistic regression model: Y = f(X, Z)  (3)  where X represents the sum of a rat’s luckiness in the preceding block of forced choice trials and Z is a matrix of dummy variables identifying the probability block that a given trial belongs to. In total we tested two specifications of this model; a simple model with one continuous regressor and dummies identifying the four probability blocks (Model 1), as well as a second model which includes the interaction of this continuous estimate of luckiness with the four probability blocks (Model 2). Since the data we used in this model were collected across many individuals over time we used a random-effects logistic estimator fit using maximum-likelihood (panel logit model).  Analysis 4: What is the effect of previous choice outcomes on the probability of choosing the Large/Risky lever both early and late in training and how far back in the past (in terms of number of choice trials) does this influence occur? The model used in the fourth stage of analysis follows a similar methodology to Analysis 3. In this analysis, we tested the effect of lagged outcomes from the previous nine choices on the rat’s probability of choosing of the Large/Risky lever. We only used data from the free choice trials as  37  the forced choice trial data (where no choice is made) would cause errors in the analysis. It is not possible to use the pellets received data in this statistical model without also using the choice data. This would lead to strange predictions in the model as it could predict a non-zero probability of picking a lever that is unavailable to the rat in the forced choice trial. Using a panel logit model as first described in the Equation 3 we tested the following model: Y = L1 (P) + … + L9 (P) + ε where Y is described as in Equation 2 and L1 (P) to L9 (P) are lag operators for the first through ninth lags of P, the number of pellets received in a given trial (either 0, 1 or 4 pellets). We tested this model using data on two separate days-the first and last training days of a given experimentto investigate whether recent choice outcomes differentially affect decision making at the beginning and end of training.  Results Analysis 1: How are probabilistic discounting rates characterized and how do they change over training? Throughout our population of 194 rats, we found that the probabilistic discounting slope at the end of training (averaged across the last 5 days of training) had a mean of -0.11 over the probability blocks with standard deviation of 0.08 (Table 1). As expected, the negative slope indicates that as the probability associated with the Large/Risky lever decreases across each block of trials, choice of the Large/Risky lever also declines. Table 1. Descriptive statistics for slope of discounting curve and early luckiness (n=194). Slope of Discount Curve Mean Standard Deviation  Lower quartile (25%)  Early Luckiness  -0.11  -2.88  0.08  17.94  -0.17  -16.50  38  Median (50%)  -0.09  -1.75  Upper quartile (75%)  -0.05  8.50  The histograms in Figure 2 show the frequency of rats in the sample that chose the Large/Risky lever a given percentage in each of the four probability blocks. This figure highlights how little variation there is in risky choice during the 100% probability block compared to a great deal of variability in choice in the three other probabilistic trial blocks (particularly in the 12.5 and 25% blocks). Figure 3 plots the individual discounting curves of all 194 rats based on the average percentage choice of the Large/Risky lever across the last 5 days of training. This figure shows the substantial variation in risky choice across all animals. The grand mean choice of the Large/Risky lever for all animals in each probability block is shown in the thick black line in the figure and the descriptive data are presented in Table 2. Figure 4A shows that as the rats learn the probabilistic discounting task, the slope of their discounting curve changes from a slight positive or null slope to a negative slope, highlighting the progression from random choice behaviour early in training to discounting of the large reward as probabilities decrease after repeated training. We also examined the difference in slope from one day to the next day across the entire training period. Early in training, there is greater variation in change of slope from one day to the next among all the rats (Figure 4B).  39  50% Probability Block  Frequency (number of rats)  100% Probability Block  120  200  100 150 80 60  100  40 50 20 0  0 100 - 80% 60 - 80% 40-60%  20-40%  100 - 80% 60 - 80%  0-20%  Frequency (number of rats)  25% Probability Block  40-60%  20-40%  0-20%  12.5% Probability Block  80  60 50  60  40 40  30 20  20 10 0  0 100 - 80% 60 - 80% 40-60%  20-40%  Percentage choice  0-20%  100 - 80% 60 - 80%  40-60%  20-40%  0-20%  Percentage choice  Figure 2. Histograms displaying proportion of rats that chose the Large/Risky lever during a set percentage of trials in each probability block.  Figure 3. Individual probabilistic discounting curves. Lines represent percentage choice of the Large/Risky lever in each of the four probability blocks for all rats (n=194).  40  Table 2. Descriptive statistics for percentage choice of Large/Risky lever (n=194). Probability Block  Grand Mean  Std Error  100%  89.35  1.04  50%  79.02  1.34  25%  65.48  1.72  12.5%  55.37  1.85  As training progresses, that variation decreases, suggesting that rats become more stable in their choice preferences for the Small/Certain or Large/Risky levers over time. The data from these three graphs suggest that as the rats become familiar with the nature the probabilistic discounting task, they understand that selecting the probabilistic lever, especially in the fourth probability block, will have lowered expected payoff (pellets) and consistently choose this lever less as the probability of reward declines.  Figure 4. Rodent learning of the probabilistic discounting task. A Changes in slope of individual probabilistic discounting curves over the course of training days. B Change in slope of probabilistic discounting curves between two consecutive days throughout training.  41  Analysis 2: What is the effect of being “lucky” during the first 5 days of training on the slope of the probabilistic discounting curve during the last 5 days of training (i.e., when stable choice occurs)? Using a simple linear regression analysis, we found that although there was a positive correlation between luckiness early in the training of rats and the slope of their discount curve at the end or training (once stable preferences emerge), the magnitude of R2 indicates that little variation in the discount slope can be explained by early luckiness (R2 = 0.09; Table 3). Therefore, early luckiness has little influence on a rat’s discounting curve at the end of training. For example, a rat experiencing early luckiness in the upper quartile of the range (greater than 9 pellets above expected value) would only have a discount slope that is 10% higher than a rat who receives rewards equal to the expected value of the task. Therefore, two rats that experienced the same amount of luckiness early in training could have very different discounting slopes at the end of training.  Table 3. Linear regression results for predicting discount slope from early luckiness. Early luckiness (Coefficient)  0.001 (0.0003)***  Constant  -0.109 (0.006)***  Observations (N)  194  R-squared  0.09  Standard errors in parentheses, *** p<.01  Analysis 3: What is the effect of being “lucky” in the forced choice trials of a given probability block on the probability of choosing the Large/Risky lever on the first free choice trial? Using the two panel logit models described in the methods section, we tested the influence of luckiness in the forced choice trials on the probability of picking the Large/Risky lever on the first  42  free choice trial of the block using data from the last 5 days of training. We found that the effect of luckiness was dependent upon the trial block as indicated by the statistical significance of coefficients for dummy variables controlling for probability block (Rows 2-4; Table 4). Notably, the coefficients for the dummy variables (Probability Block) were negative as rats displayed a decreased probability of choosing the Large/Risky lever in the last three trials blocks compared to the first (100%), which is unsurprising given the general form of the discounting curve exemplified in Figures 2 and 3.  Table 4. Estimates of the effect of forced-choice luckiness on probability of choosing the Large/Risky lever.  Forced-choice luckiness (all blocks)  Model 1 0.027 (0.004)***  Model 2 0.159 (0.042)***  Probability block (50%)  -0.942 (0.046)***  -0.95 (0.046)***  Probability block (25%)  -1.785 (0.044)***  -1.793 (0.044)***  Probability block (12.5%)  -2.271 (0.044)***  -2.28 (0.044)***  Prob. Block (50%) * Luckiness  -0.123 (0.042)***  Prob. Block (25%) * Luckiness  -0.141 (0.042)***  Prob. Block (12.5%) * Luckiness  -0.136(0.043)***  Constant  2.7 (0.106)***  2.708 (0.106)***  Observations  38,312  38,312  Log-likelihood  -17282  -17275  Number of rats  194  194  Standard errors in parentheses. *** p<.01, ** p<.05, * p<.10  We also found that the addition of variables accounting for the interaction of luckiness with the probability blocks were significant predictors of choosing the Large/Risky lever. These  43  interaction terms, which allow the slope of the regression curve to vary by probability block, also improve the fit of the model slightly, as evidenced by higher log-likelihood in Model 2, justifying their inclusion in the model. Focusing on Model 2, which is a better fit to the data than Model 1, we found that luckiness in the forced choice trials had a statistically significant and positive effect on the probability of choosing the Large/Risky lever. The coefficient cannot be interpreted directly but transformed to a probability using the following equation: (6) where  is the estimated probability of choosing the Large/Risky lever, e is the mathematical  constant and  are coefficients presented in Table 4. Using Equation 6, we found that estimates  from Model 2 suggest, for example, that a rat making a decision in the 50% probability block who was rewarded each time he pressed the Large/Risky lever (a luckiness of 8 pellets over the expected value) in the forced choice trials will have a probability of choosing the Large/Risky lever on the next free choice trial of .89: = 1 / (1 + e^-(2.708 + 0.159*8 + -0.95 + -0.123*8 ) ) = 0.89  whereas a rat in a similar situation who receives the expected value of pellets during the forced choice trials (a luckiness of zero) will have a probability of choosing the Large/Risky lever of .85: = 1 (1 + e^-(2.708 + 0.159*0 + -0.95 + -0.123*0) ) = 0.85  To more clearly demonstrate how luckiness in the forced choice blocks can influence free choice performance, we used the equations above to generate the estimated probabilities of picking the Large/Risky lever in each of the four probability blocks under these different conditions. Specifically, we estimated free choice performance under conditions that were 1) the least lucky (i.e.; the larger reward was not delivered at all during the forced choice trials), 2) expected “luckiness”, 3) when they are most lucky (i.e; the larger reward as always delivered  44  during force choice trials). As displayed in Table 5, most or least lucky during forced choice trials can considerably influence changes in risky choice, (as much as 10% in the 25% block) compared to behaviour when the expected number of pellets were obtained. Therefore, the results of this analysis suggest that receiving greater or less than expected rewards in the forced choice trials has a significant impact on the probability of choosing the risky lever in upcoming free choice trials and this impact differs for each probability block. Table 5. Estimated probability of choosing the Large/Risky lever based on forced-choice luckiness in each probability block. Probability Block  Least Lucky  100% .94  50% .77  25% .65  12.5% .51  Expected Luckiness  .94  .85  .71  .61  Most Lucky  .94  .89  .76  .68  Analysis 4: What is the effect of previous choice outcomes on the probability of choosing the Large/Risky lever both early and late in training and how far back in the past (#trials) does this influence occur? The results from the final analysis revealed that a rat’s decision to select the Large/Risky option is greatly influenced by reward outcomes (receiving 0, 1 or 4 pellets) experienced during preceding trials. The results showed that on the last day of training, rats were significantly influenced by recent outcomes of the task as indicated by the significant coefficients in the right column of Table 6. Larger absolute values of the coefficients indicate greater influence of the outcome on the probability of selecting the Large/Risky lever. As displayed in the right column, the magnitude of the coefficients was highest for the first lag (Lag 1; i.e; the outcome immediately preceding a particular choice) and decreased with each subsequent lag (with the exception of Lag 6), suggesting that the most recent outcomes had the greatest influence on choice. However, the  45  analysis revealed that other recent outcomes (2-5 trials back) also exerted a significant impact on choice of the Large/Risky lever (p<.01); outcomes occurring as far back as 7-8 trials prior still achieved statistical significance at the at the .05 level. However, it should be noted that the magnitude of the coefficients at 7-8 lags were close to 0, indicating that receipt of 1 or 4 pellets would not result in more than a 1% change in probability of choosing the risky lever. Table 6. Regression coefficient estimates of the effect of lagged trial outcomes on choosing the Large/Risky lever. First day of training  Lag of pellets received  Last day of training  1  0.0201 (0.0212)  0.276 (0.0211)***  2  0.0184 (0.0209)  0.143 (0.0210)***  3  0.0059 (0.0206)  0.117(0.0211)***  4  0.0116 (0.0204)  0.0882 (0.0209)***  5  -0.0051 (0.0201)  0.0642 (0.0208)***  6  0.0327 (0.0202)  0.0143 (0.0206)  7  0.067 (0.0200)***  0.0476 (0.0205)**  8  -0.004 (0.0197)  0.0440 (0.0202)**  9  0.0205 (0.0195)  0.0351 (0.0197)*  Constant  0.472 (0.097)***  -0.032 (0.107)  Observations  5,982  7,541  Number of rats  194  194  Log-likelihood  -3517  -3548  Standard errors in parentheses *** p<.01, ** p<.05, * p<.10  In contrast, the effect of recent outcomes on future choice was not observed early in training. A similar analysis conducted on data taken from the first day of training showed that the only lag of trial outcomes that was statistically significant was the 7th lag, which is likely to be a spurious correlation based on the substantial power of the analysis. Moreover, the magnitude of  46  the coefficients were much smaller on the first day of training compared to those observed after extended training and showed no overall pattern between different lags. We used a similar equation to Equation 6 to estimate the probability of choosing the Large/Risky lever following each potential outcome of 0, 1, and 4 pellets that could occur in Lag 1 (one previous trial) and Lag 8 after extended training (Figure 5). We used a random sequence of rewards in Lags 2-7 for the comparison (4, 1, 0, 1, 0, 4, 1). This comparison highlights the result of the differential impact of these different lags. Thus, obtaining 0, 1, or 4 pellets on the 8th trial preceding a particular choice resulted in similar probabilities of choosing the Large/Risky lever on the next trial (80-83%), despite the accumulation of numerous influences of more recent rewards. In comparison, the most recent trial outcome influenced the likelihood of choosing the risky lever by as much as 25% (from 49 to 74%) if the rat received 4 pellets compared to 0 pellets. Collectively, the results of this analysis suggests that early in training, choice behaviour appears to be more random, with rats appearing to largely ignoring recent outcomes of the task in their decision making. In contrast, once rats become familiar with the task parameters, outcomes of preceding trials exert a significant impact on subsequent choice behavior and can influence the direction of choice up to 8 trials in the future.  Figure 5. Estimates of probability of choosing the Large/Risky lever following different reward outcomes (0, 1, 4 pellets) at Lag 1 and Lag 8.  47  Discussion The purpose of this chapter was to characterize the decision making behaviour exhibited by animals performing a probabilistic discounting task. In our studies described in subsequent sections (see Chapter 3-5), animals trained on this task showed considerable variation in their discounting of the large reward. Therefore, it was of interest to examine how these discount rates develop over time and what potential factors may influence animals to prefer large, risky or small, certain rewards in a much larger sample. First, we showed that in a large population of animals, substantial individual variation in probability discounting rates exists and these discounting patterns develop slowly across the 20-30 days of training. Although early in training individual rats sometimes experience greater or worse “luck” in receiving reward following choice of the Large/Risky lever, these outcomes do not substantially impact the final discounting rates exhibited by animals at the end of training. However, receiving greater than expected rewards in the forced choice trials of a block (particularly in the 25 and 12.5% blocks) significantly increases the probability of choosing the Large/Risky lever on the next free choice trial. Lastly, Analysis 4 revealed that at the beginning of training, rats choose fairly randomly and are not influenced by recent choice outcomes. In contrast, at the end of training, rats’ probability of choosing the Large/Risky lever is affected by outcomes that occur up to 8 trials in the past, with the most recent outcome having the strongest influence. Collectively, these findings suggest that rats’ preferences for certain or probabilistic rewards may be explained, in part, by experienced outcomes in recent choices, as well as recent luckiness during the forced choice trials.  Individual Variation in Probabilistic Discounting Our first analysis revealed that in a sample of 194 rats, there was substantial variability in discounting of the large reward across probability blocks, with some rats showing very limited 48  discounting, with others showing much steeper rates of discounting. These findings support a growing literature highlighting individual variability in decision making behaviour in both human and non-human animals, including probabilistic discounting (Kacelnik and Bateson, 1996; Adriani and Laviola, 2006; Adriani et al., 2009; Peters and Buchel, 2009). For example, people vary in the extent to which they prefer certain over probabilistic rewards with the same expected value (Luce, 2000) and the extent to which they subjectively weight probabilities of reward (Kahneman and Tversky, 1979). Moreover, it is well established that even when facing two options with the same expected value, the introduction of risk to one option alters the subjective value and leads some individuals to be risk-preferring and some to be risk-averse (while others risk-neutral) depending on their individual risk attitude (Markowitz, 1952; Christopoulos et al., 2009). Animal foragers from many different species also exhibit individual attitudes towards risk (Kacelnik and Bateson, 1996). Although many behavioural ecologists assume that maximizing fitness would require animals to use foraging strategies that maximize, in the long run, the net energy gain per time unit (Stephens and Krebs, 1986), the data here show that few rats actually perform this strategy to the extent predicted by foraging theory. Rats appear to be “satisficers” rather than “maximizers" (Kalenscher and Pennartz, 2008). Notably, the reduction in choice of the large reward option exhibited by these rats follows a similar pattern to the decreasing subjective value associated with increasing odds against receiving probabilistic rewards. In particular, both humans and animals show a substantial drop in subjective value of a larger reward magnitude as soon as it becomes uncertain compared to certain, as well as a flattening of the discount rate as rewards become very unlikely (Peters and Buchel, 2009). However, the restricted number of four discrete probabilities  49  used in the current study (compared to a large range in human discounting tasks) do not permit us to compare the discount functions of these two species beyond a descriptive level. The precise reasons why both human and non-human species do not conform to normative expected utility theory are still under debate. As discussed in the introduction, volatile environments are typically associated with probabilistic, rather than deterministic, behaviour, particularly when other animals that may be competitors are present (Glimcher, 2004). Thus, we may not expect rats performing this task to conform to deterministic behaviour when reward outcomes are probabilistic and their innate bias is to alternate between foraging options. Secondly, risk preferences, as measured by the variance in amount of food delivered probabilistically in different foraging locations, have been shown in other species to be influenced by the net energy budget of an animal (Kacelnik and Bateson, 1996). Animals with positive net energy budgets (receiving an average amount of food sufficient to maintain body weights) tend to be risk-averse, whereas animals with negative net energy budgets tend to be more risk prone. Food restriction protocols in our experiments exert differential effects on individual rats’ body weights, which may contribute to the risk-averse or risk-prone patterns of behaviour we observe. Indeed, we have observed that feeding rats to satiety tends to decrease choice of the Large/Risky lever when it is more advantageous (St.Onge and Floresco, 2009). Moreover, there is evidence to suggest that decision making biases may be influenced by genetic factors affecting brain development since these biases are observed across a wide range of cultures (Kahneman and Tversky, 1979; Sharp and Salter, 1997), are difficult to eradicate even with training (McNeil et al., 1982) and are observed in other non-human species, such as monkeys (Santos and Hughes, 2009). Furthermore, we showed in the present study that experienced “luckiness” can influence future risky choice, suggesting that individual choice experiences may bias animals to be more risky or conservative.  50  Collectively, these findings suggest that genetic predispositions, current net energy rates, and recent reward experiences may all contribute to individual differences in the discounting of large, probabilistic rewards.  Recent Forced and Free-Choice Outcomes Influence Future Choice of Certain vs. Uncertain Options Upon observation that rats show great variability in preferences for large, risky and small, certain rewards, we wanted to examine some of the influences that might contribute to these choice biases. In a separate analysis, we examined the influence that reinforcement history has over subsequent choices. We used a lag of 9 trials to see how far back in time this information might be used. Moreover, we analyzed data from two time points: on the first day of training and the last day of training. The results revealed that during initial training, trial outcomes up to 9 choices in the past do not seem to impact the probability of choosing the risky or the certain option. Rats appear to be making their choices somewhat randomly on the first day. In contrast, at the end of training, information about rewarded or non-rewarded outcomes does influence the likelihood of choosing the Large/Risky lever. The greatest influence occurred in the first lag. Receiving 4 pellets on the previous trial could increase the probability of choosing the risky lever by 25% compared to receiving 0 pellets. The decreasing magnitude of the subsequent coefficients as the lags increased indicates that the farther away the choice outcome occurred relative to the current choice, the weaker the influence it had. By the 8th lag, although the outcome still had a statistically significant impact on future choice, the effect was quite small. These data suggest that as rats become familiar with the magnitude and probabilities associated with the two levers over training, their behaviour becomes much more systematic and less random. Moreover, these data suggest that information from recent outcomes weighs much  51  heavier on future choice than outcomes farther back in time. Although some groups have shown carry-over of choice patterns from one day to the next (Slezak and Anderson, 2009), our findings suggest that this may represent a more generalized choice strategy (i.e.; I should choose the Large/Risky lever less as the session proceeds) rather than direct impact of choice outcomes from the end of one day to the next since only information from 8 previous trials has a significant impact on future choice. We also used a logistic regression model to examine whether experienced “luckiness” in forced choice trial outcomes influences the likelihood of choosing the Large/Risky lever on the next free choice trial, particularly in the 25 and 12.5% blocks. The analysis revealed that additional pellets received over the expected value significantly increased the probability of choosing the Large/Risky lever on the first free choice trial in the same block, whereas receiving fewer than expected pellets decreased choice of this option. However, we were limited in our ability to examine the effect of forced choice “luckiness” on any free choice trials occurring after the first trial because Analysis 4 showed that recent trial outcomes also influence the probability of choosing the Large/Risky lever, making it difficult to isolate the impact of the forced choice vs. recent free choice outcomes on subsequent free choice trials in the same block. Therefore, these data suggest that free choice behaviour (at least for the most immediate subsequent choice) appears to be sensitive to the amount of reward obtained during forced choice trials. These findings are in line with a vast literature on reinforcement learning. In a dynamic decision making environment, an animal should strive to maximize its ability to predict future outcomes associated with its actions by efficiently integrating reward prediction errors (i.e, the difference between the expected and actual outcomes) into its current beliefs about the environment (Sutton and Barto, 1998). Individual experiences with “luckiness” in forced choice trials may explain why some rats  52  prefer the risky/safe lever in certain probability blocks even though the long-term value associated with these options may be different than those “lucky” experiences. These findings support recent data from computational modeling in humans showing that we do not follow a predetermined strategy when making decisions (Symmonds et al., 2010). Although we may have an “average plan” for what choices might be made based on predictions of the future outcomes, it is advantageous to consider alternative actions in case the environment is somewhat unpredictable and the original strategy ceases to be optimal.  Early Task Experiences do not Influence Individual Learned Discounting Patterns In contrast to recent forced choice outcomes, experienced “luckiness” in the first five days of training had little effect on the development of probabilistic discounting curves as measured in the final five days of training. Although luckiness in the forced choice trials during these initial five days was included in the data, it was combined with data from all choices during the session, perhaps being washed out by the other trials. In Analysis 4, we showed that only information from the previous 8 trials had a significant impact on the probability of choosing the Large/Risky lever on the next trial. Therefore, it is not surprising that information about luckiness that is summed across an entire session may be too cumbersome for a rat to keep track of in order to bias its decision. Our data suggest that rats primarily use information from forced choice outcomes of the same block, as well as very recent free choice trials, in order to guide choice, but overall patterns of luckiness across many trials is not well-incorporated into their decisions. We can only speculate at this time whether this may be the result of limited capacity for attention or memory processing. Summary Collectively, these data are in keeping with Behrens and colleagues (2007) who argue that a decision maker need not store unique representations of each and every choice outcome in order  53  to make value-based decisions. The decision maker maintains an on-going estimate of the value of each option and the degree to which it is revised through prediction errors depends on the volatility of the environment. If the environment is volatile and fast changing, then historical outcomes have less bearing on what would be the best option to choose and more recent experience would provide more useful information. Because our task is programmed such that the probability of receiving reward after choosing the Large/Risky lever is independent of previous outcomes, the environment could be considered somewhat volatile in that, each day, the rat has a different experienced probability in each block and must approximate the set probability over many days of training. This may explain why outcomes during recent free and forced choice trials of the same probability block significantly predict choice of the Large/Risky lever. These findings have important implications for understanding the speed at which learning cognitive tasks like probabilistic discounting occurs. For example, when the reinforcement history is long and includes many experiences in the past, learning is considered slow, whereas a short reinforcement history entails fast learning because each new outcome has an impact on the next choice (Dickinson and Mackintosh, 1978; Pearce and Hall, 1980; Kakade and Dayan, 2002; Courville et al., 2006). Overall, our findings suggest that the probabilistic discounting task is a useful model for assessing decision making about probabilistic and certain rewards and is sensitive to detecting individual differences in decision making. Future research using more advanced statistical modeling could examine the impact of trial-by-trial outcomes on learning in this paradigm in order to be able to predict a rat’s choice given its previous history.  54  CHAPTER 3: PREFRONTAL CORTICAL CONTRIBUTION TO RISKBASED DECISION MAKING  Introduction Decision making entailing cost/benefit evaluations about risks and rewards recruits numerous regions of the prefrontal cortex (PFC). As described in Chapter 1, patients with damage to the ventromedial and orbital regions of the PFC make risky, disadvantageous choices compared to control subjects (Bechara et al., 1994; Damasio, 1994; Bechara et al., 1999). Subsequent research, using a variety of different tasks, patient populations, as well as neuroimaging in healthy subjects, has implicated multiple interconnected regions of the PFC in mediating these forms of decisions. These include the DLPFC (Ernst et al., 2002; Manes et al., 2002; Brand et al., 2004; Fellows and Farah, 2005; Labudda et al., 2008), the medial and lateral regions of the OFC (Rogers et al., 1999a,b; Manes et al., 2002; Clark et al., 2003; Ernst et al., 2004; Fukui et al., 2005; Clark et al., 2008), the dorsal (Brodmann’s Area 24) and ventral (Area 32) regions of the ACC (Ernst et al., 2002; Labudda et al., 2008; Lawrence et al., 2009), and the insular cortex (Ernst et al., 2002; Bar-On et al., 2003; Clark et al., 2008; Smith et al., 2009). Although multiple frontal lobe regions have been implicated in facilitating these types of decisions, the specific contribution that different PFC regions make to decision making under uncertainty remains unclear. For example, impaired decision making on the IGT displayed by patients with vmPFC damage may reflect a failure to learn the win-lose contingencies associated with the different choice options, a preference for high risk options regardless of potential loss, or deficits in strategy acquisition and maintenance (Clark et al., 2004). Moreover, initial choices of the “high-risk decks” typically yield disproportionately large gains, whereas subsequent choices  55  yield even larger losses. However, if the order of gains and losses is randomized, OFC patients perform similar to controls (Fellows and Farrah, 2005). Thus, some have argued that OFC lesions specifically impair the ability to shift to choosing the low-risk decks after initial exposure to large rewards associated with high risk decks (i.e.; reversal learning), rather than inducing increased risky decision making. On the Cambridge gambling task, where subjects are explicitly informed of the odds associated with a choice, OFC patients sometimes show normal decision making and in some instances, actually display risk-averse tendencies (Rogers et al., 1999a; Manes et al., 2002). Moreover, results obtained with these lesioned patients are further complicated by differences in lesion size. Some OFC lesions were unilateral and may not have been sufficient to disrupt performance (Manes et al., 2002). Widespread damage caused by larger lesions of the PFC (Clark et al., 2003), frontal variant frontotemporal dementia (Rahman et al., 1999), or aneurysmal subarachnoid hemorrhage of the anterior communicating artery (Mavaddat et al., 2000), which may incorporate other PFC subregions seem to be necessary in order to produce increased risky decision making on some of these gambling tasks. These findings highlight the substantial inconsistencies in the literature on the effects of PFC lesions on decision making. As noted above, fMRI and PET imaging studies in normal subjects have also implicated different frontal lobe regions in the mediation of probabilistic decisions, including the OFC, DLPFC, insular, and medial/cingulate PFC. However, the correlational nature of imaging does not allow us to determine which regions of the PFC make necessary contributions to decision making under uncertainty. Moreover, the complexity and variety of tasks used in imaging studies and human lesion studies makes it difficult to tease apart which specific sub-processes are mediated by activity in which regions, given that each task may recruit a different constellation of cognitive processes that guide choice behaviour (e.g.; working  56  memory, reward-related learning, ambiguity/conflict resolution). While gambling tasks like the IGT may be sensitive to the behavioural problems patients with frontal lobe damage exhibit in their everyday lives, they cannot isolate the different elements of decision making that may be impaired, such as impulsivity or risky choice. Therefore, the inherent limitations of human lesion and imaging studies suggest that important information about which subregions of the PFC contribute to choice behaviour may be achieved through the use of animal models of risk-based decision making. As discussed in the previous two chapters, one component of decision making in humans that can be assessed in rodents is the evaluation of certain costs associated with different actions relative to the potential reward that may be obtained by those actions. Initial studies suggest that there may be a role for the OFC in risk-based decision making (Mobini et al., 2002; Pais-Vieira et al., 2007; Zeeb and Winstanley, 2011) but no other subregions have been examined in this context. Given the paucity of research using selective disruptions of PFC subregions on decision making, the purpose of the current study was to systematically evaluate the contribution of four subregions of the rat prefrontal cortex (OFC, insula, dorsal anterior cingulate, and medial PFC) to risk-based decision making using the probabilistic discounting paradigm described in Chapter 2.  Experiment 1: The Effects of Inactivations of the Medial PFC, OFC, Anterior Cingulate and Insular Cortex on Probabilistic Discounting Method A detailed description of the animals, apparatus and training on the probabilistic discounting task can be found in Chapter 2.  57  Probabilistic Discounting Ascending Version: A priori we determined that if inactivation of a particular PFC region resulted in an increased preference for the Large/Risky lever, a separate group of animals would be trained on an ascending probability version of the probabilistic discounting task. The procedure was identical to the descending version except that the probabilities associated with the Large/Risky lever increased with each successive block of trials (12.5%, 25%, 50%. 100%)/ The purpose of this task was to determine if an increased preference for the Large/Risky lever following PFC inactivation was due specifically to a general increase in risky choice, or to alterations in other cognitive processes that facilitate probabilistic discounting. Training Procedure, Surgery and Microinfusion Protocol. Rats were trained on the probabilistic discounting task until as a group, they (1) chose the Large/Risky lever during the 100% probability trial block on at least 80% of successful trials, and (2) demonstrated stable baseline levels of choice. Brain inactivations were conducted once a group of rats displayed stable patterns of choice for 3 consecutive days, assessed using the procedure described in Chapter 2. After the stability criterion was achieved, rats were provided food ad libitum and 2 days later, were subjected to surgery. Rats were anesthetized with 100 mg/kg ketamine hydrochloride and 7 mg/kg xylazine and implanted with bilateral 23 gauge stainless steel guide cannulae into one of the following regions: the medial PFC (prelimbic cortex; anteroposterior [AP] = +3.4 mm; medial-lateral [ML] = ± 0.7 mm from bregma; and dorsoventral [DV] = -2.8 mm from dura); lateral OFC ([AP] = +3.9 mm; [ML] = ± 2.6 mm from bregma; [DV] = -2.9 mm from dura); anterior cingulate ([AP] = +2.0 mm; [ML] = ± 0.7 mm from bregma; and [DV] = -1.0 mm from dura); and agranular insular cortex ([AP] = +2.7 mm; [ML] = ± 3.8 mm from bregma; and [DV] = -3.8 mm from dura). For all surgical preparations, the mouthbar was set to -3.3 mm (flat skull). Thirty-gauge obdurators flush  58  with the end of guide cannulae remained in place until the infusions were made. Rats were given at least 7 days to recover from surgery before testing. During this recovery period, animals were handled at least 5 min each day, and food restricted to 85% of their free-feeding weight. Body weights were continued to be monitored daily to ensure a steady weight loss during this recovery period. Rats were subsequently retrained on the task for at least five days, and until, as a group, they displayed stable levels of choice behaviour. For three days before the first microinfusion test day, obdurators were removed and a mock infusion procedure was conducted. Stainless steel injectors were placed in the guide cannulae for 2 min, but no infusion was administered. This procedure habituated rats to the routine of infusions in order to reduce stress on subsequent test days. The day after displaying stable discounting, the group received its first microinfusion test day. A within-subjects design was used for all experiments. Inactivation of PFC regions was achieved by infusion of a drug solution containing the GABAA agonist muscimol (Sigma-Aldrich Canada, Oakville, Ontario, Canada) and the GABAB agonist baclofen (Sigma-Aldrich). Both drugs were dissolved separately in physiological saline at a concentration of 500 ng/µl, and then combined in equal volumes, with the final concentration of each compound in solution being 250 ng/µl. Intracranial microinfusions used a volume of 0.5 µl, so that the final dose of both baclofen and muscimol was 125 ng per side. These doses were chosen because we have found them to be effective at altering probabilistic discounting when infused in other brain regions (Ghods-Sharifi et al., 2009). In addition, infusions of comparable doses of baclofen and muscimol in the OFC (Takahashi et al., 2009) or muscimol alone into the anterior cingulate (Ragozzino and Rozman, 2007) have been reported to disrupt cognition. Note that these doses are substantially higher than doses used in previous experiments that have also reported behavioral effects (e.g. 10ng; Corrigall  59  et al., 2001). Infusions of GABA agonists or saline were administered bilaterally into one of the PFC regions via 30 gauge injection cannulae that protruded 0.8 mm past the end of the guide cannulae, at a rate of 0.5 µl/75s by a microsyringe pump. Injection cannulae were left in place for an additional 1 min to allow for diffusion. Each rat remained in its home cage for another 10 min period before behavioral testing. On the first infusion test day, half of the rats in each group received saline infusions and the other half received muscimol/baclofen. The next day, they received a baseline training day (no infusion). If, for any individual rat, choice of the Large/Risky lever deviated by more than 15% from its pre-infusion baseline, it received an additional day of training prior to the second infusion test. On the following day, rats received a second counterbalanced infusion of either saline or muscimol/baclofen. Histology After completion of all behavioral testing, rats were sacrificed in a carbon dioxide chamber. Brains were removed and fixed in a 4% formalin solution. The brains were frozen and sliced in 50 µm sections before being mounted and stained with Cresyl Violet. Placements were verified with reference to the neuroanatomical atlas of Paxinos and Watson (1998). The locations of acceptable infusions for rats used in Experiment 1 are presented in the right panels of Figure 6. Data Analysis A detailed description of general data analysis procedures is outlined in Chapter 2. The choice data were analyzed using 2-way within subjects analyses of variance (ANOVAs), with Treatment and Trial Block as the within subjects factors. The main effect of block for the choice data was significant in all experiments (p<.05) indicating that rats discounted choice of the Large/Risky lever as the probability of the large reward changed across the four blocks. This effect will not be mentioned further. Locomotor activity (photobeam breaks) and the number of trial omissions were  60  analyzed with one-way within subjects ANOVAs. Response latencies were analyzed either using 2-way or one-way within subjects ANOVAs.  Results Medial PFC (Prelimbic) Initially, 16 rats were trained for this experiment. Two animals died during surgery and the data from another rat were eliminated due to inaccurate placement. The remaining rats (n = 13) required an average of 35 days of training on the probabilistic discounting task prior to receiving counterbalanced infusions of saline or muscimol/baclofen into the medial PFC. Analysis of choice behavior following bilateral infusions of muscimol/baclofen or saline into the medial PFC revealed a significant main effect of treatment (F(1, 12) = 6.20, p<.05; Figure 6A, left panel). Medial PFC inactivation caused a significant increase in the proportion of choices directed towards the Large/Risky lever relative to saline infusions. Interestingly, inactivation of the medial PFC also significantly increased response latencies, but only in the 25 and 12.5% probability blocks, where the relative long-term value of the Large/Risky option was comparable or less than that of the Small/Certain option (treatment x block interaction: F(3, 36) = 3.85, p<.05, Dunnett’s, p<.05; Figure 6A, middle panel). However, this effect was only apparent during free-choice trials, as analysis of the response latency data collected during forced-choice trials revealed no differences between treatment conditions (F(1, 12) = 1.89, n.s.). There were no differences in the number of trial omissions or locomotor activity between saline and muscimol/baclofen treatments (all F’s < 0.89, n.s.). Thus, under these conditions, inactivation of the medial PFC reduced probabilistic discounting, leading to an apparent increase in risky choice.  61  Figure 6. Effects of inactivation of different PFC regions on probabilistic discounting. The panels of each figure plots percentage choice of the Large/Risky lever (left) and response latencies (middle) as a function of the trial block (x-axis) following infusions of saline (white circles) or muscimol/baclofen (inactivation, black squares) into each of the four PFC subregions: A Medial PFC; B OFC; C dorsal anterior cingulate; and D insular cortex. Symbols represent mean + SEM. Black stars denote a significant (p<.05) main effect of treatment. White stars denote significant (p<.05) differences between treatments during a specific block. Right panels are schematics of coronal sections of the rat brain showing the location of acceptable infusion placements through the rostral-caudal extent of each of the four PFC regions.  62  OFC A total of 16 rats were initially trained for this experiment. Five of these animals had placements that extended ventral to the OFC and their data were excluded from the analyses. The remaining rats (n = 11) required an average of 28 days of training on the probabilistic discounting task prior to receiving counterbalanced infusions of saline or muscimol/baclofen into the OFC. Analysis of the choice data revealed no main effect of treatment (F(1,10) = 0.38, n.s) or treatment x block interaction (F(3, 30) = 0.43, n.s.; Figure 2B, left panel). Although bilateral inactivation of the OFC had no effect on choice behavior, this manipulation did alter response latencies, as indicated by a significant treatment x block interaction (F(3, 30) = 4.26, p<.05; Dunnett’s, p<.05; Figure 6B, middle panel). Simple main effects analyses revealed that OFC inactivation did not affect response latencies during the 100 and 50% probability blocks, but significantly increased deliberation times in the latter two blocks, relative to saline infusions. As was observed from the medial PFC group, OFC inactivation did not affect response latencies during forced-choice trials (F(1, 10) = 3.48, n.s.). There was also a slight decrease in locomotion following OFC inactivation (1580 +/- 229) relative to saline treatment (1712 +/- 253), but this difference failed to reach significance (F(1, 10) = 4.27, p=.07). There was no effect of OFC inactivation on trial omissions (F(1,10) = 1.79, n.s.). Thus, inactivation of the OFC did not affect risky choice under conditions where rats were familiar with the relative risk/reward contingencies associated with different response options and changes in these contingencies that occurred over the course of a session. Anterior Cingulate Thirteen rats were trained for this experiment. The data from two rats were excluded due to inaccurate placements and another rat died during surgery. The remaining 10 rats took an average of 33 days of training prior to receiving counterbalanced infusions of saline or muscimol/baclofen  63  into the anterior cingulate. Inactivation of this region caused no discernible change in choice of the Large/Risky lever relative to saline treatment. This observation was confirmed by the lack of a significant main effect of treatment (F(1,9) = 0.00, n.s.), and treatment X block interaction (F(3, 27) = 0.31, n.s; Figure 6C, left panel). Likewise, response latencies during free-choice trials did not differ between treatment conditions (all F’s < 1.0, n.s.; Figure 6C, middle panel). However, during forced-choice trials, rats displayed slightly longer response latencies following anterior cingulate inactivation (1.3 +/- 0.1 s), compared to saline infusions (1.0 +/ -0.1 s; F(1, 9) = 11.26, p<.01). Locomotion, and trial omissions were not altered significantly by anterior cingulate inactivations (all F’s < 1.0, n.s.). Insular Cortex Thirteen rats were trained for this experiment, but data for six rats were excluded due to inaccurate placements. The remaining 7 rats took an average of 28 days of training on the probabilistic discounting task prior to surgery and receiving counterbalanced infusions of saline or muscimol/baclofen into the agranular insular cortex. Again, we observed no change in choice behavior following muscimol/baclofen infusions, as indicated by the lack of a significant main effect of treatment (F(1,12) = 0.16, n.s), or treatment x block interaction (F(3, 18) = 1.67, n.s; Figure 6D, left panel). Insular inactivation did not significantly affect response latencies during either free-choice (Figure 6D, middle panel) or forced-choice trials, nor did it alter locomotion or trial omissions (all F’s < 2.51, n.s.). Medial PFC Inactivation (Ascending Probabilities) Medial PFC inactivation increased selection of the Large/Risky lever when the probabilities of obtaining the larger reward decreased over the course of a probabilistic discounting session. In a subsequent experiment, a separate group of 12 rats were trained for 25 days on a variant of this  64  task, where the probability of obtaining the large reward increased over the session, prior to receiving counterbalanced infusions of saline or muscimol/baclofen into the medial PFC. In contrast to the effects on choice behavior reported above, inactivation of the medial PFC induced the opposite effect on probabilistic discounting under these conditions, with rats displaying a decreased preference for the Large/Risky lever over the 4 trial blocks. Analysis of these data revealed a significant main effect of treatment (F(1, 11) = 12.28, p<.05; Figure 7A). Rats chose the Large/Risky lever less often following medial PFC inactivations relative to saline infusions. There was no significant effect of inactivation on trial omissions or locomotion (F’s < 2.5, n.s.). However, there was a significant increase in average response latencies during free choice trials across all the blocks after medial PFC inactivation (F(1, 11) = 8.82, p<.05; Figure 7B). Again, this effect was selective to trials where rats were required to choose between the two levers, as response latencies during forced-choice trials did not differ between treatments (F(1, 11) = 2.87, n.s.).  Figure 7. Effects of medial PFC inactivation on probabilistic discounting with ascending probabilities. A Percentage of choice of the Large/Risky lever during free-choice trials following infusions of saline or muscimol/baclofen into the medial PFC for all 12 rats used in this experiment. Inactivation of the medial PFC decreased risky choice. Inset displays data from a subset of rats (n=9) that demonstrated comparable levels of choice of the Large/Risky lever after both saline and inactivation treatments during the initial, 12.5% probability block (i.e.; matching for performance). B Response latencies displayed as a function of trial block. Black stars denote a significant (p<.05) main effect of treatment. C Location of acceptable infusion placements for rats used in this experiment.  65  Inspection of Figure 7A reveals that the decreased preference for the Large/Risky lever after inactivation of the medial PFC was apparent during the first trial block (12.5%) and persisted over the remainder of the session. However, the decreased choice of the Large/Risky lever during the first block was primarily attributable to three rats. We conducted a supplemental analysis on the data from the 9 remaining rats that displayed a comparable preference for the Large/Risky lever during the first trial block (i.e.; matching for performance) after inactivation and saline treatments. In this instance, the analysis of these data again revealed a significant overall decrease in choice of the Large/Risky lever (F(1,8) = 7.39, p<.05; Figure 7A, inset). Thus, in this subset of rats, even though medial PFC inactivation did not reduce preference for the Large/Risky lever during the first block, these animals were still slower to adjust their choice behavior as the probability of obtaining the larger reward increased over subsequent blocks. This suggests that the medial PFC may facilitate adjustments in preference for the Large/Risky option in response to increases in the probability of obtaining the larger reward.  Experiment 2: Effects of Medial PFC Inactivation on Performance of a Within-Session Reversal Inactivation of the medial PFC (but not other frontal lobe regions) induced differential effects on probabilistic discounting depending on the manner in which the probabilities of obtaining the larger reward changed over a session. When the odds of the Large/Risky reward were initially 100% and then decreased, medial PFC inactivation increased risky choice, whereas when the probabilities were initially low and subsequently increased, similar inactivations reduced preference for the Large/Risky lever. Both variants of this task require a shift in the direction of responding, whereby rats choose the Large/Risky lever less or more often as the probabilities decrease or increase over the session, respectively. Thus, one potential interpretation of the effects  66  of medial PFC inactivation on probabilistic discounting may be that these manipulations impaired behavioral flexibility, such that rats were perseverating on the lever that they preferred during the first block (Large/Risky or Small/Certain). Notably, similar inactivations of the medial PFC do not impair performance of standard or probabilistic reversal tasks, although they do impair shifting between discrimination strategies (Ragozzino et al., 1999; Block et al., 2007; Ragozzino, 2007; Floresco et al., 2008c). Nevertheless, we conducted a subsequent experiment to determine whether alterations in choice induced by medial PFC inactivations were due to perturbations in response flexibility. Rats were trained on a modified within-session reversal task that was similar to the probabilistic discounting procedure in a number of respects.  Method Within-Session Reversal Task Rats underwent initial lever pressing training in a manner identical to that described in Experiment 1. They then received daily training sessions consisting of 56 trials, separated into 2 blocks of 28 trials. The 2 blocks were each comprised of 8 forced-choice trials where only one lever was presented (4 trials for each lever, randomized in pairs) permitting animals to learn the amount of food associated with each lever. This was followed by 20 free-choice trials, where both levers were presented (i.e.; 40 free-choice trials in total, as in the probabilistic discounting task). Each 38 min session began in darkness with both levers retracted, and trials began every 40 s with the illumination of the houselight and insertion of one or both levers into the chamber 3 s later. One lever was designated the Large Reward (LR) lever, the other the Small Reward (SR) lever (counterbalanced left/right). When a lever was chosen, both levers retracted. During the initial block, a press on the LR lever delivered 4 pellets, whereas the SR lever delivered 1 pellet, both with 100% probability, similar to the first block of the probabilistic discounting task. However,  67  during the second, reversal block, selection of the LR resulted in the immediate retraction of both levers, extinguishing of the houselight, and no pellets were delivered. During this block, responses on the SR lever continued to deliver 1 pellet. All other aspects of the task were identical to those described in Experiment 1. Rats were trained with these same contingencies until as a group, they chose the LR lever on at least 80% of successful trials during the first block, less than 20% of successful trials during the reversal block, and the group displayed stable patterns of choice for 3 consecutive days. Subsequently, they were subjected to surgery, and then retrained on the task for at least 5 days prior to receiving intracranial microinfusions. Surgery, Microinfusions, Histology Rats were implanted with bilateral guide cannulae into the medial PFC as described in Experiment 1. All other procedures were identical to those described in Experiment 1.  Results Eight animals were trained for 18 days prior to receiving counterbalanced infusions of muscimol/baclofen into the medial PFC. One rat had two infusions located in one hemisphere of the PFC, and its data were excluded from the analyses. Figure 8A displays choice data for the remaining rats (n = 7). After saline infusions, rats showed a strong preference for the LR lever during the initial block when it delivered 4 pellets, compared to the SR lever associated with 1 pellet. During the reversal block, rats shifted their choice away from the LR lever that now delivered 0 pellets, instead choosing the SR lever on the vast majority of free-choice trials, which continued to deliver 1 pellet. This pattern of choice was not altered by inactivation of the medial PFC. Analysis of the choice data revealed a significant main effect of block (F(1, 6) = 103.16, p<.001), but no significant main effect of treatment (F(1, 6) = 0.84, n.s.) or treatment x block interaction (F(1, 6) = 2.75, n.s.; Figure 8A). Interestingly, in this experiment, there were no effects  68  of medial PFC inactivations on response latencies (F(1 6) = 0.01, n.s.). Likewise, locomotion and trial omissions did not differ between treatment conditions (all F’s < 0.9, n.s.). The location of acceptable medial PFC infusion placements in this experiment are displayed in Figure 8B. Thus, medial PFC inactivation did not impair the ability to perform a within-session reversal in response to changes in reinforcement contingencies. This suggests that alterations in probabilistic discounting induced by medial PFC inactivation cannot be easily attributed to a general impairment in response flexibility.  Figure 8. Medial PFC inactivation does not affect performance of a within-session reversal. A Data are plotted as percentage choice of the Large Reward lever over two blocks of 20 free-choice trials (initial and reversal). The Large Reward lever was associated with a 4-pellet reward during the initial block, but did not deliver any reward (0 pellets) during the reversal block. Responding on the Small Reward lever delivered 1 pellet during both blocks. Rats were able to shift responding away from the Large Reward lever during the reversal block after both saline and muscimol/baclofen treatment. B Location of acceptable infusion placements for rats used in this experiment.  Experiment 3: Effects of Inactivation of the Medial PFC on Probabilistic Discounting with Fixed Probabilities Another potential explanation for the alterations in probabilistic discounting observed in Experiment 1 may be that medial PFC inactivation disrupted monitoring of changes in the  69  probability of obtaining the larger reward that occurred within a session. Alternatively, this manipulation may have induced a more fundamental disruption in calculating the relative value of the Large/Risky option within each trial block. To assess this latter hypothesis, a separate group of rats were trained on a simplified version of the task, in which the probability of obtaining the larger reward remained constant throughout a session. Initially, the probability was fixed at 40%, making the relative long-term value of the Large/Risky option (4 pellets @ 40%) higher than the Small/Certain option (1 pellet @ 100%). Rats were then retrained and tested under conditions where the probability of obtaining 4 pellets was set at 10%. In this instance, selection of the Small/Certain option would yield a greater amount of reward in the long-term.  Method Probabilistic Discounting with Fixed Probabilities Eight rats underwent initial lever pressing training in a manner identical to that described in Experiment 1. They were then trained on the modified probabilistic discounting procedure, consisting of 40 trials, broken down into 20 forced-choice, followed by 20 free-choice trials. Each session took 26 min to complete. All other aspects of the task were identical to the probabilistic discounting task used in Experiment 1. On each trial, selection of the Large/Risky lever delivered 4 pellets with a 40% probability, whereas pressing the Small/Certain lever delivered 1 pellet with 100% probability. The 40% probability was chosen to maximize the possibility of detecting either increases or decreases in the preference for the Large/Risky lever, given our observation in Experiment 1 that rats tend to select the Large/Risky lever on ~ 90% of trials when the probability of obtaining 4 pellets was higher (i.e., 50%; e.g.; see Figure 6A). Rats were trained 6-7 days a week until, as a group, they chose the Large/Risky lever on at least 60% of successful trials and  70  demonstrated stable performance for 3 consecutive days. They were then subjected to surgery, retraining, and their first sequence of intracranial microinfusions. Following the first set of microinfusion test days, rats were retrained on the same task for another 11 days, but with the probability of obtaining the larger reward now set to 10%. This probability was chosen because it biases rats to use a risk-aversive strategy, but was large enough to ensure that over 20 forced-choice trials, rats will experience at least some reward so that the probability would not be inferred to be 0%. By the end of this training period, rats as a group chose the Large/Risky lever on less than 40% of the free-choice trials and demonstrated stable performance for three consecutive days. They then received a second series of counterbalanced infusions into medial PFC. Surgery, Microinfusions, Histology Rats were implanted with bilateral guide cannulae into the medial PFC as described in Experiment 1. All other procedures were identical to those described in Experiment 1.  Results Rats were trained for 24 days prior to receiving the first sequence of counterbalanced infusions of muscimol/baclofen or saline into the medial PFC. Of the initial 8 rats, one died after surgery, and the infusion placement for another was outside the boundary of the prelimbic cortex. Its data were excluded from the analyses. Following saline infusions, the remaining 6 rats displayed a slight preference for the Large/Risky option when the probability of obtaining 4 pellets was 40%, selecting this lever on ~ 60% of free-choice trials (Figure 9A, left panel). Importantly, inactivation of the medial PFC did not alter this preference (F(1,5) = 0.40, n.s.; Figure 9A left panel). There were also no differences between treatment conditions in terms of locomotion, omissions, or response latencies (all F’s < 2.0, n.s.).  71  Figure 9. Inactivation of the medial PFC does not affect risky choice using fixed probabilities associated with larger reward. A Bars represent percentage choice of the Large/Risky lever following saline (white) or muscimol/baclofen (black) infusions into the medial PFC. The left panel displays data from infusion tests days when the Large/Risky reward probability was fixed at 40%. The right panel displays data from infusions test days following 11 days of re-training with the Large/Risky probability at 10%. The inset diagrams the significant reduction in preference for the Large/Risky lever averaged across the last three days of training prior to the first (40%) and second infusion tests (10%). Star denotes a significant (p<.05) difference between reward probability conditions. B Location of acceptable infusion placements for rats used in this experiment.  Rats were subsequently retrained for another 11 days with the probability of obtaining the Large/Risky reward set at 10%. By then end of this retraining period, rats were selecting the Large/Risky lever less often, compared to their performance when the probability was set at 40% (Figure 9, inset). Analysis of the choice data obtained from the last three baseline days prior to the first (40% chance of the Large/Risky reward) and second (10% chance) infusion tests days confirmed that rats significantly discounted the Large/Risky lever as the probability of reward was reduced (F(1,5) =7.82, p<.05). Again, inactivation of the medial PFC failed to alter choice  72  behavior (F(1, 5) = 3.91, n.s.; Figure 9A right panel), locomotion, omissions, or response latencies relative to saline infusions (all F’s<1.9, n.s.). The location of acceptable medial PFC infusion placements in this experiment are displayed in Figure 9B. Thus, inactivation of the medial PFC does not alter choice between small/certain and larger/uncertain rewards when the relative value of larger, probabilistic rewards remains constant during a session.  Discussion The major finding of this study is that inactivation of the prelimbic region of the medial PFC induced dramatic effects on risky choice assessed with a probabilistic discounting task. However, the specific effect of medial PFC inactivation was dependent on the manner in which the reward probabilities changed over a session. When the probabilities associated with the Large/Risky option were initially high (100%) and subsequently decreased, medial PFC inactivations increased risky choice. Yet, when the 4-pellet option was initially disadvantageous (12.5%) and then increased in value, similar inactivations had the opposite effect. Subsequent experiments indicated that these effects were not easily attributable to general disruptions in response flexibility or to evaluating the relative value of larger/uncertain versus smaller/certain rewards. In contrast, inactivation of the OFC did not affect probabilistic discounting, but did increase response latencies during the latter portion of the session. Similar inactivations of the anterior cingulate or insular cortex did not reliably alter choice behavior. Collectively, these findings indicate that the medial prelimbic PFC plays a critical role in guiding choice between certain and uncertain rewards of different magnitudes in response to changes in reward probability.  Involvement of the Rat Medial PFC in Probabilistic discounting  73  At first glance, the increase in risky choice induced by medial PFC inactivation in Experiment 1 may have indicated that this region normally biases choice towards small, certain rewards. However, similar inactivations had the opposite effect when the probabilities of obtaining the Large/Risky reward were initially low and then increased, suggesting that this relatively simple explanation is insufficient to account for our findings. Alternatively, it is possible that these treatments may have induced a general impairment in response flexibility, given that in both experiments, rats persisted in selecting the lever they displayed a bias for during the first probability block. It is notable that following medial PFC inactivations, rats were able to adjust choice of the Large/Risky lever in response to changes in reward probability, albeit at a slower rate than that displayed after saline treatments. Nevertheless, we conducted a subsequent experiment where rats were required to perform a within-session reversal, patterned after the probabilistic discounting task. Responding on one of two levers initially delivered a large 4-pellet (LR) or smaller 1-pellet (SR) reward. In the latter part of the session, the LR lever no longer delivered food, but the SR lever continued to deliver 1 pellet. If medial PFC inactivations induced a general tendency to perseverate towards the lever they initially displayed a bias for at the beginning of a session, we would have expected to observe an impaired ability to shift choice between levers after the reversal. However, this effect was not observed, arguing against the notion that the results of Experiment 1 were attributable to impairments in response flexibility. These results are in keeping with numerous other studies showing that inactivation of the prelimbic region of the medial PFC does not affect reversal learning (Ragozzino et al., 1999; Birrell and Brown, 2000; Floresco et al., 2008c). The prelimbic PFC has been implicated in guiding goal directed behavior in response to changes in the value of rewards linked to particular actions (Balleine and Dickinson, 1998;  74  Killcross and Coutureau, 2003). Experiment 3 examined whether medial PFC inactivations interfered with choice behavior when the relative long-term value of larger, probabilistic rewards remained constant throughout a session. Here, the probability of obtaining 4 pellets was fixed so that over 20 free-choice trials, this option either had a greater (40% = ~32 pellets) or lesser (10% = ~8 pellets) long-term value than the certain, 1 pellet option (20 pellets). Following saline infusions, rats displayed an appropriate bias towards the Large/Risky or Small/Certain lever when the odds of obtaining the larger reward were 40% or 10%, respectively. Importantly, inactivation of medial PFC did not alter choice behavior under either condition. This suggests that the medial PFC is not critical for basic estimations of reward probabilities when animals must evaluate the relative long-term value associated with smaller, certain versus larger, uncertain rewards. Moreover, this lack of effect on choice indicates that the decreased choice of the Large/Risky option following medial PFC inactivations observed in Experiment 1 (i.e.; probabilistic discounting with ascending probabilities) cannot be attributed to a general reduction in preference for larger, probabilistic rewards. Rather, it appears that this region plays a more specialized role in biasing choice in response to changes in the probabilities of obtaining larger, uncertain rewards. Further insight into the contribution of the medial PFC to risk-based decision making comes from a discussion of how rats may identify changes in reward probabilities to facilitate probabilistic discounting. Over repeated training, animals learn that changes in the probabilities associated with the Large/Risky reward are signaled by intermittent blocks of forced-choice trials that precede each set of 10 free-choice trials. Thus, rats must remember the outcomes of previous trials and use this information to update their representation of the relative value associated with the Large/Risky lever as a session proceeds. Rats may also use internal temporal cues to estimate the relative value of the Large/Risky option, as they are trained that the likelihood of obtaining 4  75  pellets early in a session is substantially different from that later in a session (Catania, 1970). There is some suggestion that damage to the medial PFC impairs aspects of time perception (Dietrich and Allen, 1998; Thorpe et al., 2002) and induces insensitivity to within-session shifts in delays in a delay discounting task (Cardinal et al., 2001). Thus, in this context, the medial PFC may subserve an “updating function”, whereby mnemonic and temporal information used to identify changes in reward probabilities is integrated to adjust the direction of behavior to maximize reward. Following inactivation of the medial PFC, rats were impaired at using withinsession cues to update their choice according to the reward contingencies of the current trial block. Therefore, dysfunction in this region may limit the ability to use internally-generated information to update representations of expected reward probabilities based on recently experienced events. The rat medial PFC displays anatomical connectivity homologous to the ventral portion of the anterior cingulate (Area 32) in primates (Uylings and van Eden, 1990) and recent studies suggest that this region plays an important role in mediating risk-based decisions in humans. General increases in activity in this region are observed when healthy subjects perform common cost-benefit decision making tasks (Ernst et al., 2002; Labudda et al., 2008; Lawrence et al., 2009), particularly when they make risky relative to safe choices on the Iowa gambling task (Lawrence et al., 2009). Similarly, using a task where the “risk” associated with the different choices was “not winning” a given reward (as in the task used in the present study), Smith and colleagues (2009) found greater activation in the ventral anterior cingulate (Area 32) during risky versus safe choices, as well as during low-probability versus high-probability choices. Moreover, patients with ventromedial PFC lesions that include damage to Area 32 show impaired decision making on the Cambridge Gamble Task (Clark et al., 2008). Yet, the rat medial PFC has also been proposed to share functional homology to the dorsolateral PFC of primates (Brown and  76  Bowman, 2002; Uylings et al., 2003). For example, the medial and lateral PFC have been implicated in working memory in rats (Seamans et al., 1995; Floresco et al., 1999), and humans (Owen et al., 1990; Stuss et al., 2000), respectively. In humans, dorsolateral PFC activity has been linked to changes in contexts associated with different gains and losses of reward, particularly when outcomes are the opposite of expectations (Akitsuki et al., 2003). Activity in this region is also increased during performance of the Iowa gambling task (Ernst et al., 2002), particularly in the decision phase (Labudda et al., 2008). Likewise, damage to the dorsolateral PFC has been associated with impairments in different forms of risk-based decision making (Manes et al., 2002; Brand et al., 2004; Fellows and Farah, 2005). These findings, viewed in light of the present data, suggest that with respect to risk-based decision making, the medial PFC of the rat may share functions that are similar to those mediated by both the ventral anterior cingulate cortex and dorsolateral PFC in humans.  The Role of Other PFC Regions in Cost/Benefit Decision Making Inactivation of the OFC did not affect probabilistic discounting, although these manipulations dramatically increased response latencies in a manner similar to that induced by medial PFC inactivation, whereby deliberation times were increased only in the last two trial blocks. This effect on response latencies was only apparent during free-choice trials when rats had to choose one of the two response options. Notably, it is during these blocks that the relative long-term value of the Large/Risky option switched from having a greater value (100%, 50%) to roughly equal (25%) or lesser (12.5%) value than the Small/Certain option. This suggests that although the OFC may not be necessary for making advantageous choices involving risks and rewards under these conditions, neural activity in this region may aid speed of processing about the cost/benefit contingencies associated with each decision following shifts in the relative long-term  77  values of probabilistic rewards. Consistent with these findings, patients with OFC damage also display increased deliberation times when performing certain risk-based decision making tasks (Rogers et al., 1999a; Manes et al., 2002; Clark et al., 2008). Our finding that inactivation of the OFC did not alter probabilistic discounting was somewhat surprising, considering previous studies with both rodents and humans that have implicated this region in cost/benefit decision making tasks about risks and rewards (Rogers et al., 1999a,b; Manes et al., 2002; Clark et al., 2003; Ernst et al., 2004; Fukui et al., 2005; Clark et al., 2008). Lesions of the rat OFC induced prior to training led to a reduced preference for larger probabilistic rewards (two pellets) relative to a smaller certain reward (one pellet; Mobini et al., 2002). In that study, the larger reward probability remained constant over a daily session, and was decreased systematically every 20-25 days. The “risk-averse” tendencies displayed by OFC lesioned rats were most prominent when the relative long-term value of the larger, probabilistic option (32% or 20%) was lesser than that of the small certain option. This suggests that OFC lesions may impair learning about the relative value of larger, probabilistic rewards. More recently, Zeeb and Winstanley (2011) examined the effect of either pre or post-training OFC lesions on performance of a rodent gambling task modeled after the Iowa gambling task in humans. Lesions made prior to training slowed acquisition of choosing the advantageous options, whereas lesions made post-training had no effect on choice compared to sham animals. This notion is also supported by findings from human neuroimaging studies, where lateral OFC activity is associated with learning risk/reward contingencies of the Iowa Gambling task over a session (Lawrence et al., 2009). In contrast, rats in the present study were trained for 3-4 weeks until they demonstrated stable patterns of probabilistic discounting, implying that they had learned to adjust their choice behavior in response to within-session changes in reward probabilities. Thus, the  78  lateral aspects of the OFC may facilitate the initial learning of risk/reward contingencies, but neural activity in this region may not be required to guide decision making once these contingencies have been acquired. It is notable that patients with OFC damage in one study displayed impaired decision making on a task where subjects were explicitly informed of reward probabilities prior to making a choice (Clark et al., 2008), similar to the task used in the present study, where rats were familiarized with changes in reward probability over many training days. However, in that study, the patient group that showed impairments included those with damage to the medial portions of the OFC. In fact, a lesion control group that included individuals with lateral OFC damage performed similar to healthy controls. Thus, it may be that the medial, rather than the lateral OFC, plays a more prominent role in facilitating risk-based decisions in situations where subjects have prior knowledge about reward probabilities. Damage to the insular cortex in humans has been associated with alterations in risk-related judgments (Bar-On et al., 2003; Clark et al., 2008). These patients are able to make advantageous decisions (i.e., they correctly choose options yielding the most reward), but do not adjust their betting according to gain/loss probability, suggesting that they are less “risk sensitive” than healthy controls. In a similar vein, imaging studies in healthy controls suggest that the insular cortex may have a specific role in using negative outcomes (i.e.; punishment) to bias choice behavior to maximize potential reward (O’Doherty et al., 2003; Cohen et al., 2008; Preuschoff et al., 2008). In contrast, we did not observe significant alterations in probabilistic discounting following insular cortex inactivation. Note that the task used in the present study measured whether rats were able to choose the most advantageous option based on changing reward probabilities. However, our procedures did not employ any explicit punishments per se; the “risk”  79  was a lost opportunity to obtain a reward, which may explain the lack of effect on probabilistic discounting following inactivation of this region. Thus, it would be of particular interest to investigate the role of the rat insular cortex in guiding decision making when animals evaluate potential aversive consequences associated with different options. Inactivation of the dorsal anterior cingulate cortex also did not affect probabilistic discounting, even though infusions of GABA agonists into the adjacent prelimbic cortex did affect choice and response latencies. These two regions of the medial PFC receive dissociable patterns of innervation from the mediodorsal thalamus (Conde et al., 1990) and also exhibit different afferent and efferent connectivity with other cortical and subcortical structures (Sesack et al., 1989; McDonald, 1991a; Heidbreder and Groenewegen, 2003). The present findings suggest that despite their close in proximity, these two cortical regions have dissociable functions. This notion is in keeping with numerous studies demonstrating that lesions or inactivation of the prelimbic versus anterior cingulate cortex induce differential effects on cognitive, executive and decision making functions (Seamans et al., 1995; Bussey et al., 1997; Walton et al., 2003). In summary, we have shown that the prelimbic region of the medial PFC makes a selective contribution to risk-based decision making by monitoring changes in reward probability over time in order to make adjustments in choice to more valuable options. Although the OFC does not contribute to choice, it helps guide response latency. In contrast, the dorsal anterior cingulate and insular cortex are not required for probabilistic discounting. These findings help elucidate the specific neural underpinnings of decision making about risks and rewards.  80  CHAPTER 4: SEPARATE NEURAL CIRCUITS BIAS CHOICE TOWARDS RISKY OR CERTAIN OPTIONS  Introduction The previous chapters of this dissertation described findings showing that the medial PFC, the amygdala, and the NAc form important contributions to risk-based decision making. Anatomically, these regions are intricately linked through a series of direct glutamatergic projections that would create an ideal system for integrating information about rewards with different magnitudes and risks associated with them in order to guide decision making (Sripanidkulchai et al., 1984; McDonald, 1987; Sesack et al., 1989; Groenewegen et al., 1990; McDonald, 1991a,b; Brog et al., 1993; McDonald et al., 1996). In particular, a role of medial and dorsal regions of the PFC in cognitive control has been proposed (Botvinick et al., 2001; Knoch et al., 2006; Figner et al., 2010), suggesting that this cortical region could transfer complex decision information top-down to subcortical regions recruited for more fundamental behavioural processes. However, it is currently unclear whether these regions work independently or in conjunction with each other to produce complex behavioral outputs when making decisions about certain vs. uncertain rewards. Functional imaging studies have provided indirect evidence that these regions do interact to facilitate decision making about probabilistic rewards. For example, functional connectivity between the human ACC and the NAc has been observed when subjects are choosing high-risk compared to low-risk gambles (Cohen and Ranganath, 2005) and between the ACC and amygdala when anticipating reward outcomes (Marsh et al., 2007). Individuals with genetic polymorphisms that alter amygdala reactivity and reduce PFC regulatory control are more susceptible to the framing effect, where afflicted individuals were more likely to be risk-averse in the context of  81  gains and risk-seeking in the context of losses (Roiser et al., 2009). Moreover, these individuals lacked the ACC-amygdala connectivity associated with resistance to the framing effect observed in controls, suggesting that interactions between these regions are an important mediator of decisions about probabilistic gains and losses. According to some authors, the amygdala may communicate expectancies of reward magnitude or value to the PFC, whereby it is used to guide behavioral choice (O’Doherty et al., 2003). In support of this idea, patients with bilateral lesions of the amygdala do not show a linear relationship between activity in the medial PFC and expected value as controls do when performing a probabilistic reversal learning task (Hampton et al., 2007). They also experience significantly less activation in the ACC when switching strategies, leading to a specific impairment in sensitivity to how reward values guide choice behaviour. However, these findings were in patients who experienced a developmental lesion of the amygdala prior to learning the task; therefore it is unclear whether amygdala-PFC connectivity is important for decision making performance after task contingencies have been learned. Alternatively, damage to the amygdala may impair the ability to use internally generated information from the medial PFC to update representations of expected reward probabilities based on recently experienced events. Thus, the conclusions of studies using patients with long-term brain lesions are limited since they cannot delineate the direction of functional connectivity between the PFC and amygdala, nor rule out the potential for other cortical-limbic-striatal systems to be recruited in compensation for the injury. Given that it is difficult to directly test whether anatomical connectivity between structures is necessary for task performance in humans, rodent models of decision making provide a unique opportunity to directly disconnect specific pathways. A recent study has identified functional connectivity between the OFC and NAc when performing a delay discounting task  82  (Bezzina et al., 2008). Using unilateral excitotoxic lesions in each region in opposite hemispheres, they showed that animals with a functional disconnection between the OFC and NAc core became more impulsive compared to sham-lesioned animals after extended training. On the other hand, effort-based decision making is mediated by circuitry involving projections between the rat dorsal anterior cingulate and the BLA (Floresco and Ghods-Sharifi, 2007). Using an asymmetrical inactivation procedure, these authors showed that a functional disconnection between the BLA and dorsal anterior cingulate significantly decreased preference for a large reward that was associated with more effort. In summary, accumulating evidence suggests that a similar corticolimbic-striatal circuit may drive decision making about probabilistic outcomes. Therefore, the purpose of the current study was to determine the routes of serial information transfer between the medial PFC, BLA and NAc that regulate our ability to make decisions about different risks and rewards in a continuously changing environment.  Method A detailed description of the animals, apparatus, and training on the probabilistic discounting task can be found in Chapter 2 and surgical and histological procedures in Chapter 3.  Reward magnitude discrimination task As we have done previously (Ghods-Sharifi et al., 2009; Stopper and Floresco, 2011), we determined a priori that if a particular treatment specifically decreased preference for the Large/Risky lever on the probabilistic discounting task, separate groups of animals would be trained and tested on a reward magnitude discrimination task to determine if this effect was due to an impairment in discriminating between reward magnitudes associated with the two levers. In these experiments, rats were trained to press retractable levers as in the probabilistic discounting  83  task, after which they were trained on the discrimination task. Here, rats chose between one lever that delivered one pellet and another that delivered four pellets. Both the small and large rewards were delivered immediately after a single response with 100% probability. A session consisted of four blocks of trials, with each block consisting of 2 forced-choice followed by 10 free-choice trials.  Surgical, Microinfusion and Procedures Once a group of rats displayed stable choice behavior, they were subjected to surgery. Rats were anesthetized with ketamine and xylazine (100/7 mg/kg) and implanted with two sets of bilateral 23 gauge stainless steel guide cannula, using standard stereotaxic techniques. Four separate combinations of placements were used; 1) BLA (AP= -3.1 mm; ML=± 5.2 mm from bregma; and DV=-6.5 mm from dura) and NAc (AP=+1.5 mm; ML=±1.4 mm; DV=-5.9 mm); 2) BLA and medial PFC (AP =+3.4 mm; ML= ± 0.7 mm; DV=-2.8); 3) medial PFC and ventrolateral amygalofugal pathway (ascending BLA medial PFC pathway; AP =-0.5 mm; ML=±5.0 mm; DV=-5.3) and 4) BLA and ventromedial internal capsule (descending PFC BLA pathway; AP=1.5 mm; ML=± 2.4 mm; DV=-6.3 mm). In this last preparation, we also performed a 1 mm transection of the corpus callosum on the opposite side of the brain that would receive inactivation of the internal capsule through which medial PFC inputs to the BLA transverse. Following a 7 d recovery from surgery, rats were retrained for at least five days, and until, as a group, they displayed stable levels of choice behavior. On the following day, a group received its first of three microinfusion test days. Reversible inactivation of brain nuclei (medial PFC, BLA, NAc) was achieved by infusion a combination of GABA agonists baclofen and muscimol using procedures described previously (125 ng each in 0.5 µl, delivered over 75 s; Ghods-Sharifi et al., 2009; St.Onge and Floresco, 2010; Stopper and Floresco, 2011). Inactivation of axonal  84  pathways (ventrolateral amygdalofugal pathway, ventromedial internal capsule) was induced by infusion of the local anesthetic bupivacaine (0.38 ug/0.5 µl), which prevents impulse traffic in the axonal fibers that reside in these pathways.  Disconnection Design and Testing Procedures The logic underlying the use of disconnection inactivations to identify components of a functional neural circuit is based on the assumption that information is transferred serially from one structure to an efferent region on both sides of the brain in parallel and that dysfunction will result from blockade of neural activity at the origin of a pathway in one hemisphere and the termination of the efferent pathway in the contralateral hemisphere. For example, if performance on a task is dependent on a serial connection linking the BLA to the NAc, then unilateral inactivation of the BLA would prevent the NAc in the ipsilateral hemisphere from gaining access to the information needed to perform the task. In the other hemisphere, information would be relayed from the BLA; however, suppression of neural activity within the NAc on this side of the brain would prevent it from processing incoming signals from the BLA. Thus, after this asymmetric disconnection, NAc neural activity that contributes to decision making processes would be compromised on both sides of the brain. The disconnection design rests on two further assumptions. First, it assumes that ipsilateral inactivation of one or both structures in a circuit should not have as a disruptive effect on behavior, because the intact structures in the hemisphere should be able to at least partially compensate for the unilateral disruption in function. In the design, this assumption is controlled for by using ipsilateral inactivation of both structures in a circuit. Second, in order to be maximally effective, contralateral connections between two brain regions should be minimized. With respect to the pathways explored in the present study, projections from the BLA to both the  85  NAc and PFC are primarily ipsilateral (McDonald et al., 1987; McDonald, 1991a,b). However, the PFC sends both ipsi- and contralateral descending projections to the BLA. As described above, to accommodate for this, in our experiment where we selectively disconnected descending PFC inputs to the BLA (using asymmetrical internal capsule/BLA inactivations), our surgical procedures included a transection of the corpus callosum in a region just caudal to the PFC, which is the region where axons from the ipsilateral PFC cross over and descend towards the contralateral BLA. Therefore, on separate test days, rats received three counterbalanced infusions: 1) a saline infusion into both structures contralaterally (saline), 2) drug infusions in two regions in the same hemisphere (ipsilateral inactivation) and 3) drug infusions in two regions in opposite hemispheres (functional disconnection). The ipsilateral inactivation was used as a within-subjects control to determine whether “mass action” effects may occur when we simultaneously inactivate two brain regions. The order of treatments and the hemispheres that received ipsilateral/contralateral infusions were counterbalanced across animals. Following the first infusion test day, rats received 1-2 baseline training days (no infusion) until an individual rat’s choice of the Large/Risky lever deviated by <15% from its pre-infusion baseline. On the following day, a rat received a second counterbalanced infusion, followed by another training day, and lastly, the final infusion.  Histology After completion of behavioral testing, rats were euthanized in a carbon dioxide chamber. Brains were removed and fixed in a 4% formalin solution. The brains were frozen and sliced in 50 µm sections before being mounted and stained with Cresyl Violet. Placements were verified with reference to a neuroanatomical atlas (Paxinos and Watson, 1998). Data from rats whose placements were outside the borders of the medial PFC, NAc, BLA, or the targeted axon fiber  86  bundles along the ventrolateral edge of the corpus callosum and the ventromedial internal capsule were removed from the analysis.  Data Analysis A detailed description of general data analysis procedures is outlined in Chapter 2. In brief, choice data were analyzed using either one or two-way within-subjects ANOVAs, with Treatment and Trial Block as within-subjects factors. Multiple comparisons were made with Dunnett’s or Newman-Keuls tests as appropriate. Locomotor activity (i.e. photobeam breaks), response latency, and the number of trial omissions were analyzed with one-way repeated-measures ANOVAs.  Win-Stay/Lose-Shift Analyses If we observed a significant effect on risky choice on the standard probabilistic discounting task, we conducted a supplementary analysis to obtain further insight into how these treatments affected patterns of choice and resulting alterations in discounting. Specifically, we conducted a choice-by-choice analysis to identify whether changes in behavior were due to alterations in the likelihood of choosing the risky lever after obtaining the larger reward (win-stay performance) or alterations in negative feedback sensitivity (lose-shift performance; Cardinal and Howes, 2005; Stopper and Floresco, 2011). Animals’ choices during the task were analyzed according to the outcome of each preceding free-choice trial (reward or non-reward) and expressed as a ratio. The proportion of win-stay trials was calculated from the number of times the rat chose the Large/Risky lever after choosing the risky option on the preceding trial and obtaining the large reward (a win), divided by the total number of free-choice trials where the rat obtained the larger reward. Conversely, lose-shift performance was calculated from the number of times rats shifted choice to the Small/Certain lever after choosing the risky option on the preceding trial and were  87  not rewarded (a loss), divided by the total number of free-choice trials resulting in a loss. This analysis was conducted for all trials across the four blocks. We could not conduct a block-by block analysis of these data because there were many instances where rats did not obtain the large reward at all during the latter blocks. Changes in win-stay performance were used as an index of the impact that obtaining the large, risky reward had on subsequent choice behavior, whereas changes in lose-shift performance served as an index of negative feedback sensitivity.  Results Effect of Functional Disconnection of the BLA – NAc Pathway on Probabilistic Discounting Disruption of neural activity in both the NAc and BLA reduces preference for large, risky options (Ghods-Sharifi et al., 2009; Stopper and Floresco, 2011). Therefore, our first aim was to assess whether these two regions form a functional neural circuit that normally biases choice towards larger, risky rewards. Initially, 16 rats were trained for this experiment and required an average of 32 days of training before showing stable choice behaviour and proceeding to microinfusion test days. Two animals died during surgery and the data from three others were eliminated due to inaccurate placements, resulting in a final n of 11 rats in this group. Functional disconnection of this circuit significantly decreased risky choice compared to saline (n=11; Figure 10A). Ipsilateral inactivation also reduced preference for the large, risky option, with this effect driven primarily by 4 of the 11 rats tested. These impressions were confirmed by analysis of the choice data, which showed that these treatments decreased choice of the Large/Risky lever compared to saline treatment (F(1,10)=11.46, p=.01; Figure 10A). Furthermore, a direct comparison of the effect of BLA-NAc disconnection to saline also revealed a significant decrease in risky choice (F(1,10)=6.60, p=.03), whereas a similar analysis on the effects of ipsilateral inactivation did not yield a statistically reliable effect (F(1,10)=3.25, p=.10). Moreover, when we analyzed data from  88  a subset of rats that showed comparable patterns of choice after ipsilateral inactivation and saline treatments (n=7), we again observed that asymmetrical inactivation of the BLA/NAc significantly reduced preference for the Large/Risky option (F(2,12)=4.25, p=.04; Figure 10A, inset). This latter finding confirms that the effects of BLA-NAc disconnections on risky choice were specifically due to a disruption of information transfer between these two regions. This decrease in risky choice did not appear to be selectively attributable to alterations in reward or negative feedback sensitivity. A choice-by-choice analysis of win-stay performance (i.e; choosing the risky option after obtaining the larger reward on the previous trial) showed that although both ipsilateral inactivation and functional disconnection tended to decrease win-stay performance compared to saline treatment, this effect was not statistically significant (F(2,18)=1.79, n.s.; Figure 10D). Similarly, inactivation treatments tended to increase lose-shift tendencies (i.e.; selecting the safe option after a risky choice and loss), but the effect failed to reach significance (F(2,20)=2.39, n.s.; Figure 10D). Functional disconnection and ipsilateral inactivation of the BLA-NAc pathway reduced preference for the larger, uncertain reward during certain trial blocks of the discounting task. To assess whether these effects were attributable to a general disruption in discriminating between rewards of different magnitudes, we conducted another experiment, wherein a separate group of rats was trained to choose between two levers that delivered either one or four pellets, both with 100% probability. Initially, 8 rats were used for this experiment and required 9 days of training before showing stable choice of the large reward lever. One animal died following surgery, resulting in a final n of 7 rats in this group. Under these conditions, ipsilateral inactivations  89  Figure 10. Effects of disconnection of the BLA-NAc pathway on probabilistic discounting and reward magnitude discrimination. A Percentage choice for the Large/Risky lever during free-choice trials following infusions of saline into the BLA and NAc, GABA agonists into the BLA and ipsilateral NAc (ipsilateral inactivation) or GABA agonists into the BLA and contralateral NAc (functional disconnection). Symbols represent mean + SEM. Data are plotted as a function of the Large/Risky lever probability by block (x axis). Black star denotes the average choice from ipsilateral inactivation and functional disconnection is significantly different from saline (p<.05). Disconnection of the BLA-NAc pathway decreased risky choice. The inset panel shows data from a subset of rats who did not show a decrease in risky choice after ipsilateral inactivation (n=7), yet still showed a decrease in risky choice following functional disconnection. B Percentage choice of the Large Reward lever during free-choice trials on the reward magnitude discrimination task following saline infusions, ipsilateral inactivation or functional disconnection of the BLA and NAc. Disrupting communication in this pathway had no effect on preference for larger versus smaller rewards. C Schematic of coronal sections of the rat brain showing the range of acceptable location of infusions on disconnection test days through the rostral-caudal extent of the NAc and BLA for rats in the discounting (circles) and reward magnitude discrimination experiments (squares). Numbers beside each plate correspond to mm from bregma. Note that the panel represents the disconnection procedure for clarity; the hemisphere of infusion was counterbalanced across animals. D Win-stay/loseshift ratios. Win stay values are displayed as the proportion of choices on the Large/Risky lever following a rewarded risky choice on the preceding trial. Lose-shift values are displayed as the proportion of choices on the Small/Certain lever following unrewarded risky choice on the preceding trial. Although disconnection of the BLA and NAc caused an overall decrease in risky choice, this effect did not appear to be selectively attributable changes in either reward or negative feedback sensitivity.  90  91  or functional disconnection of the BLA-NAc pathway did not alter preference for the larger reward (F(2,12)=0.28, n.s.; Figure 10B). Moreover, inactivation treatments did not affect response latencies, locomotion or trial omissions (all Fs<2.43, all p>.12, Table 7), indicating that the effects on decision making were not easily attributable to disruptions in motivational or motor processes or a reduction in general preference for larger rewards. These data show that neural activity within this subcortical amygdalar-ventral striatal pathway plays a key role in biasing choice behavior towards options yielding larger rewards that may be risky, but may also be more profitable. Effect of Functional Disconnection of the Medial PFC – BLA Pathway on Probabilistic Discounting The medial prelimbic PFC (homologous to Brodmann’s area 32 of the anterior cingulate) also makes a major contribution to risk-based decision making. In rats, PFC inactivation increases risky choice when the odds of obtaining a larger reward are initially favorable and subsequently decrease, suggesting that this region updates and modifies choice biases when reward probabilities change (St.Onge and Floresco, 2010; Chapter 3). Notably, this effect is opposite to that caused by inactivation of BLA (Ghods-Sharifi et al., 2009), a region that shares reciprocal connections with the PFC (Sesack et al., 1989; McDonald, 1991a; McDonald et al., 1996). Despite the seemingly contrasting roles these regions play in guiding decision making, we were particularly interested in assessing whether the PFC and BLA form a functional neural circuit that can influence the direction of risky choice.  92  Table 7. Locomotion, trial omission, and response latency data obtained following saline treatment, ipsilateral inactivation, or functional disconnection of the various pathways. Locomotion counts are measured in photobeam breaks. Values are displayed as Mean (SEM). Pathway  Saline  Ipsilateral  Functional  Inactivation  Disconnection  BLA-NAc (Discounting) Locomotion  1745(202)  1517 (146)  1428 (151)  Trial Omissions  0.27 (0.19)  5.54 (2.12)  3.45 (2.10)  Mean Response Latency (s)  0.75 (0.13)  1.33 (0.30)  1.14 (0.24)  Locomotion  1342 (151)  1227 (204)  1195 (138)  Trial Omissions  0 (0)  0.14 (0.14)  0 (0)  Mean Response Latency (s)  0.83 (0.12)  0.85 (0.09)  0.68 (0.07)  Locomotion  1513 (143)  1418 (126)  1418 (162)  Trial Omissions  0.18 (0.12)  1.09 (0.55)  2.27 (0.89)  Mean Response Latency (s)  0.55 (0.04)  0.89 (0.15)  0.86 (0.11)  Locomotion  1736 (222)  1781 (191)  1693 (175)  Trial Omissions  2.9 (2.35)  0.5 (0.31)  2.5 (1.97)  Mean Response Latency (s)  0.81 (0.08)  0.76 (0.13)  0.72 (0.10)  Locomotion  1525 (165)  1362 (165)  1518 (178)  Trial Omissions  0 (0)  2.55 (2.15)  2.27 (1.23)  Mean Response Latency (s)  0.49 (0.04)  0.83 (0.28)  0.65 (0.11)  BLA-NAc (Reward Magnitude Discrimination)  Medial PFC-BLA Overall (Discounting)  BLA-Medial PFC Ascending (Discounting)  Medial PFC-BLA Descending (Discounting)  93  Initially, 16 rats were trained for this experiment and required an average of 31 days of training before showing stable choice behaviour and proceeding to microinfusion test days. One animal died following surgery, one rat had a compromised test day, and three animals had inaccurate placements, resulting in a final n of 11 rats in this group. We observed that asymmetrical disconnection of the BLA-PFC pathway significantly increased risky choice (Figure 11A), an effect similar to bilateral PFC inactivation (St.Onge and Floresco, 2010; Chapter 3), but opposite to that observed after BLA inactivation (Ghods-Sharifi et al., 2009) or BLA-NAc disconnections (present study). Analysis of all the choice data revealed a significant Treatment X Block interaction (F(6,60)=2.24, p=.05). Simple main effects analyses further revealed that disconnection of the medial PFC-BLA pathway increased choice of the Large/Risky lever in last two trial blocks relative to saline and relative to ipsilateral inactivation in the 12.5% block (Newman-Keuls, p<.05; Figure 11A). In contrast, ipsilateral inactivation did not affect choice relative to saline. Locomotion was not affected by inactivation treatments (F(2,20)=0.32, n.s.), although these treatments did cause a slight increase in response latencies (F(2,20)=3.16, p=.06) and trial omissions (F(2,20)=2.75, p=.08; Table 7).  94  Figure 11. Effect of disconnection of the medial PFC-BLA pathway on probabilistic discounting. All conventions are identical to Fig.10. A Disconnection of the medial PFC-BLA pathway increased risky choice. White stars (saline vs. functional disconnection, p<.05) and asterisk (ipsilateral inactivation vs. functional disconnection, p<.05) denote significant difference between treatments at the particular trial block. B Acceptable location of infusions through the rostral-caudal extent of the medial PFC and BLA for all rats. C Win-stay/lose-shift data. Disconnection of the medial PFC-BLA pathway decreased negative feedback sensitivity and reduced the tendency to select the Small/Certain option after a non-rewarded risky choice.  95  A supplemental analysis of win-stay and lose-shift tendencies provided additional insight into the specific processes affected by disconnection of the PFC-BLA pathway. Win-stay performance was unaffected by these treatments (F(1,10)=0.04, n.s.; Figure 11C), suggesting that increased risky choice was not attributable to an enhanced tendency to play risky after obtaining the larger reward on a preceding trial. However, BLA-PFC disconnection did reduce lose-shift tendencies relative to saline (Figure 11C). Thus, under control conditions, when animals selected the Large/Risky option and were not rewarded, they shifted to the Small/Certain option on ~35% of subsequent trials. However, following PFC-BLA disconnection, rats were much less likely to choose conservatively after not being rewarded for a risky choice. Analysis of these data yielded an effect that approached statistical significance (F(1,10)=4.51, p=.059). From these data, we infer that BLA-medial PFC circuitry enables adjustments in decision making when reward probabilities change, primarily by mitigating choice behavior in response to negative feedback (i.e.; reward omissions). Disruption of communication in this pathway made animals less sensitive to “losses” and increased risky choice.  Effect of Functional Disconnection of the Ascending vs. Descending BLA-Medial PFC Pathways on Probabilistic Discounting The findings described above confirm that serial information transfer between the PFC and BLA is of critical importance to adjusting decision making biases in response to changes in reward probability. However, what remained to be resolved was the directionality of the communication between these two regions. As the BLA and PFC share reciprocal connections (Sesack et al., 1989; McDonald, 1991a; McDonald et al., 1996), disconnection of these regions would be expected to disrupt both bottom-up transfer of reward value information from BLA to PFC and top-down signals from PFC to BLA.  96  Anatomical studies have suggested that ascending and descending axonal projections connecting BLA and medial PFC may course through the brain via anatomically-distinct pathways (Krettek and Price, 1977; McDonald et al., 1996; Orozco-Cabal et al., 2006; Figure 12).  Figure 12. The ascending and descending axonal pathways between the medial PFC and BLA travel through separate routes through the brain.  Ascending projections from BLA to PFC mainly course through the external capsule and adjacent deep layer of the ventrolateral cortex and then turn medialward and upward around and through the rostral pole of the corpus striatum toward the PFC (Orozco-Cabal et al., 2006). In contrast, descending projections from PFC to BLA take a distinctly different route, running through the ventromedial internal capsule before diverging lateralward through the sublenticular forebrain to  97  terminate in the BLA (McDonald et al., 1996; S. Zahm, personal communication, Figure 18). We exploited the dissociable routes of these axonal pathways to selectively disrupt bottom-up and topdown communication between the BLA and PFC. Specifically, by unilaterally targeting ventrolateral amydalofugal axons, combined with contralateral inactivation of the medial PFC, we would selectively disrupt information transfer through the ascending BLA PFC pathway. Conversely, asymmetrical inactivation of ventromedial internal capsule and BLA would selectively disrupt information transfer through the descending PFC BLA pathway. Initially, 15 rats were trained for the ascending pathway group experiment and required an average of 37 days of training before showing stable choice behaviour and proceeding to microinfusion test days. Five animals had inaccurate placements in the amygdalofugal pathway (mainly in ventrolateral or medial striatum), resulting in a final n of 10 rats in this group. We observed that disconnection of ascending inputs from the BLA to the medial PFC had no effect on risky choice (main effect of Treatment (F(2,18)=0.66, p=n.s.; Treatment X Block interaction (F(6,54)=1.97, n.s.; Figure 13A). No other performance measures (response latencies, locomotion, omissions) were affected by inactivation treatments (all Fs<1, all p>.40; Table 7). Initially, 16 rats were trained for the descending pathway experiment and required an average of 36 days of training before showing stable choice behaviour and proceeding to microinfusion test days. One animal died after surgery and four animals had inaccurate placements in the internal capsule, resulting in a final n of 11 rats in this group.  98  Figure 13. Effect of disconnecting ascending and descending pathways between the medial PFC and BLA on probabilistic discounting. All conventions same as Fig.10. A Ascending BLA to PFC Pathway. Left panel: Percentage choice for the Large/Risky lever following infusions of saline into the medial PFC and ventrolateral amygdalofugal pathway, GABA agonists in the medial PFC and bupivacaine into the ipsilateral amygdalofugal pathway (ipsilateral inactivation) or asymmetrical unilateral inactivations of both regions (functional disconnection). Disconnection of the ascending pathway had no effect on choice. Right panel: Acceptable location of infusions through the rostral-caudal extent of the medial PFC and ventrolateral edge of the corpus callosum for all rats. B Descending PFC to BLA Pathway. Left panel: Percentage choice for the Large/Risky lever following saline infusions into the BLA and ventromedial internal capsule, GABA agonists into the BLA and bupivacaine into the ipsilateral ventromedial internal capsule (ipsilateral inactivation) or GABA agonists into the BLA and bupivacaine into the ventromedial internal capsule (functional disconnection). Disconnection of the descending pathway increased risky choice. Black star denotes significantly greater risky choice following disconnection vs. saline (p<.05). Right panel: Acceptable location of infusions through the rostral-caudal extent of the BLA and ventromedial internal capsule. C Win-stay and lose-shift data for saline and functional disconnection tests. Black star denotes a significant difference from saline, p<.05). The increase in risky choice induced by disconnection of the descending pathway was attributable primarily to a reduced sensitivity to reward omissions, indexed by a decrease in lose-shift performance.  99  100  In stark contrast to the ascending pathway, disruption of descending inputs from the medial PFC to the BLA significantly increased risky choice compared to saline (Figure 13B), similar to the effect induced by the overall medial-PFC disconnection. In this experiment, alterations in risky choice induced by ipsilateral inactivation relative to saline were highly variable across animals, even though these treatments had no overall effect on choice behavior. This variability limited our ability to detect statistically-significant effects of Treatment (F(2,20)=2.01, p=.16) or a Treatment X Block interaction (F(6,60)=0.90, p=.50) in the overall ANOVA. Nevertheless, targeted comparisons confirmed that asymmetrical inactivations significantly increased risky choice relative to saline (F(1,10)=6.80, p=.025) whereas ipsilateral inactivations did not (F(1,10)=0.02, n.s.). Furthermore, as was observed following the full BLA-PFC disconnection, the increase in risky choice was associated with reduced sensitivity to negative feedback. Asymmetrical inactivations substantially reduced the tendency to choose the safe option on trials following a non-rewarded risky choice (F(1,10)=6.23, p=.03; Figure 13C). However, win-stay tendencies following functional disconnection did not differ from saline treatments (F(1,10)=0.98, n.s.; Figure 13C). No other performance measures were affected by inactivation treatments (all Fs<2.30, all p>.12; Table 7). Collectively, these data show that regulation of decision making by prefrontal-amygdalar circuitry is achieved primarily by top-down control of the BLA by the medial PFC. When risky options become less profitable over time, a descending flow of information from the PFC mitigates the bias towards larger rewards driven by BLA-NAc circuitry.  101  Discussion All things being equal, our natural inclination is to go for larger versus smaller payoffs. However, if these larger rewards are associated with some degree of risk, we need to integrate information about the magnitude and probability associated with different options to help choose which one is more valuable in the long-term. These decisions become substantially more complex in dynamic environments where the likelihood of obtaining rewards is changing. Allocation of resources (as may occur when managing financial investments) requires one to monitor changes in the rate of return of riskier options (e.g.; stocks). If they become too low, it may be more advantageous to reallocate resources to options that yield smaller but more certain returns (e.g.; savings accounts). Ongoing competition between risky and conservative tendencies shape our choice patterns and cause us to lean more towards one option or another as payouts change over time. The present data suggest that separate neural circuits incorporating amygdalar, ventral striatal and prefrontal regions make dissociable contributions to these opposing biases and help us regulate whether we continue pursuing large, risky options or choose more conservative, reliable ones. Subcortical, amygdala-striatal circuitry drives choice towards larger, riskier rewards. Excitatory projections from the BLA modulate NAc activity (Floresco et al., 2001a; Brog et al., 2003) and have been proposed to influence the direction of ongoing behavior towards rewardrelated stimuli (Everitt et al., 1999; Setlow et al., 2002; Ambroggi et al., 2008). Here we show that disconnection of these two regions induced a profound shift in bias away from larger, uncertain rewards, primarily when this option would yield a greater or equal amount of reward than the small/certain option (i.e.; the first three trial blocks). Importantly, BLA-NAc disconnections did not affect choice on a reward magnitude discrimination task where animals simply chose between  102  a small and large reward, both delivered with 100% certainty. This indicates that the typical preference for larger over smaller rewards is not dependent on serial communication within this circuit. Rather, this circuit makes a more selective contribution to decision making when the subjective value of objectively larger rewards is diminished by uncertainty. This is in keeping with a growing literature showing that the BLA and NAc are critical for enabling organisms to endure a variety of costs (e.g.; delays, effort) that diminish subjective reward value in the pursuit of objectively larger rewards (Winstanley et al., 2004; Floresco et al., 2008b; Ghods-Sharifi et al., 2009; Stopper and Floresco, 2011). The present findings suggest that activity in this circuit helps an organism overcome the aversive aspects of reward uncertainty, driving behavior towards options that may not always be rewarded, but may prove more objectively profitable in the longterm. Neurophysiological and functional imaging studies have shown increased activation of the amygdala and NAc that correlates with selection of larger magnitude or more valuable rewards, suggesting that changes in neural activity within this circuitry encodes the relative value of different options (Ernst et al., 2005; Blair et al., 2006; Preuschoff et al., 2006; Marsh et al., 2007; Smith et al., 2009). For example, when subjects choose between a “safe” bond option yielding smaller, certain gains or “risky” stock options yielding potentially greater rewards or losses (similar to the task used here), riskier choices were preceded by increased NAc activation (Kuhnen and Knutson, 2005). Our data expand on these findings, suggesting that increased NAc activity associated with choice of larger, riskier rewards is likely driven by excitatory inputs from the BLA. Furthermore, they confirm that the direction of risky choice is causally-linked to increased activation of ventral striatal circuitry, which may influence volitional behavior through descending projections to motor effector sites (Zahm and Heimer, 1990).  103  Top-down control of amygdala-NAc circuit by the medial PFC. In stark contrast to the effects of BLA-NAc disconnection, disrupting communication between the BLA and PFC increased choice of the large, risky option. What is particularly striking about this result is that although it closely resembles the effect of bilateral inactivation of the medial PFC under similar task conditions (St.Onge and Floresco, 2010; Chapter 3), it is opposite to the effect induced by BLA inactivation (Ghods-Sharifi et al., 2009). Thus, if these two regions were acting independently, a parsimonious expectation would be that simultaneous disruption of activity in the BLA and PFC might cause no net change in choice, with one inactivation effectively cancelling out the effect of the other. The fact that disruption of BLA-PFC circuitry induced a “prefrontallike” profile confirms that communication between frontal and temporal lobe regions plays a critical role in tracking decision outcomes to estimate changes in reward probabilities over time and facilitate adjustments in risky versus certain choice biases. In addition, these data highlight the utility of employing a circuit analysis approach to understanding the neural basis of risk-based decision making, rather than separately assessing the contribution of different brain regions in isolation. We next sought to identify whether the effects of disconnecting these two structures was primarily due to a disruption of bottom-up (BLA-to-PFC) or top-down (PFC-to-BLA) signaling (or both). Guided by the findings from our neuroanatomical experiments, we again employed an asymmetrical disconnection approach, specifically targeting axonal pathways in the ascending BLA–PFC and descending PFC-BLA pathways. Disconnection of the ascending pathway did not produce a reliable change in choice behavior. Thus, even though the BLA may send information about reward value to the PFC, impeding the flow of information in this pathway is not sufficient to alter overt decision making about probabilistic rewards.  104  On the other hand, disconnection of the descending PFC inputs to the BLA did increase choice of the Large/Risky option in an identical manner to what was induced by full disconnection of this circuitry. This effect was attributable to a decreased tendency to select the certain option following a non-rewarded risky choice, indicative of a reduction in negative feedback sensitivity. This key finding suggests that PFC inputs to the BLA play a critical role in influencing decision biases, primarily by providing information about recent reward omissions that, in turn, influence the direction of subsequent choices. Disruption of PFC input to the BLA rendered animals unable to use information about recent losses to adapt behavior, thereby permitting subcortical circuitry to persist in biasing choice towards the risky option despite its decrease in value over time. The idea that the PFC exerts top-down supervisory control over subcortical regions in not new, although this conclusion has been derived primarily from indirect evidence from lesion and functional imaging studies (Mayberg et al., 2002; McClure et al., 2004; Ochsner et al., 2004; Etkin et al., 2006; Wager et al., 2008; Schiller and Delgado, 2010). However, to our knowledge, this is the first use of a combined neuroanatomical/disconnection approach to dissect the directionality of communication within this cortical-subcortical circuitry and directly demonstrate top-down control of the amygdala by the PFC. Medial regions of the human PFC, including the ACC, have long been associated with cognitive control and error monitoring and detection (Botvinick et al., 2001). With respect to decision making, activity in the anterior cingulate region of the PFC has been linked to monitoring increasing gain (Rogers et al., 2004) as well as negative feedback or changes in reward contingencies that require modifications in behavior (Marsh et al., 2007). Moreover, a recent study revealed that progressive increases in firing of ACC neurons were associated with disengagement from options providing diminishing returns in favor of exploring novel sources of  105  reward (Hayden et al., 2011a). Accordingly, lesions of this region of the PFC in monkeys impairs the ability to integrate signals related to the outcomes of the animals’ previous choices in order to switch from choosing one option to other options that eventually become more profitable over time (Kennerley et al., 2006). Thus, rather than simply monitoring recent single outcomes associated with different actions, the ACC may form and/or maintain representations of differently valued actions that are updated over time through repeated experiences of gain and loss in order to determine which options are relatively more profitable. This updating function may in turn influence whether choice biases promote exploitation of currently profitable situations or lean more towards exploration of new ones when rewards become less frequent (Daw et al., 2006; Rushworth et al., 2008). ACC activation is also observed during low probability choice situations which is predictive of reduced risk seeking, suggesting that this region helps to determine the degree of risk-seeking or risk-aversion in situations with uncertain outcomes (Paulus and Frank, 2006). Collectively, these findings suggest that across species, the anterior cingulate region of the medial PFC is important for keeping track of previous actions and their outcomes, and using this information to shape decision-making biases to guide subsequent behavior. When viewed in light of the above-mentioned observations, the present data provide novel insight into the dynamic interactions between prefrontal, amygdalar and striatal systems that shape decision biases. In situations where reward probabilities are volatile, subcortical circuits linking the amygdala and ventral striatum provide an intuitive bias towards options that may yield objectively larger rewards. In comparison, the medial PFC plays a more supervisory role, monitoring the frequency of rewarded and non-rewarded actions over time. As reward omissions accumulate, repeated “losses” signal that risky options are becoming less profitable, and this information may be used by the PFC to switch preference to more certain rewards. Alterations in  106  decision biases are executed via descending pathways from PFC to BLA, which tempers the preference that BLA-NAc circuitry exerts over the direction of choice, potentially via inhibitory feed-forward transmission within this circuitry (Rosenkranz and Grace, 2002). This later notion is in keeping with studies in humans, where transient disruption of PFC activity by transcranial magnetic stimulation increases choice of small, immediate rewards over larger delayed rewards (Figner et al., 2010) or risky over safe options (Knoch et al., 2006), without altering valuation processes, supporting the idea that the PFC can override decision making biases mediated by other brain systems. In conclusion, we show here that separate, yet interconnected, neural circuits mediate different components of cost/benefit decision making related to risks and rewards. A subcortical amgygdala-ventral striatal circuit provides a visceral bias that helps us stick to our guns and pursue options that may be risky but also provide good long-term benefit. In contrast, the medial PFC keeps track of actions and outcomes over time, and can temper urges for riskier rewards as they become less profitable, permitting a shift in preference for smaller yet more, reliable options. The dynamic competition between these circuits and their interplay with other brain systems shape our decision biases and underlies the struggles we face when evaluating choices that vary in terms of potential gains we can obtain.  107  CHAPTER 5: THE CONTRIBUTION OF PREFRONTAL DOPAMINE TO RISK-BASED DECISION MAKING  Introduction As described in Chapter 1 and 3, patients with lesions to DA terminal regions, such as the medial PFC, OFC or amygdala make risky or disadvantageous choices on various measures of decision making. However, it is unclear whether these effects occur as a result of damaged presynaptic DA inputs, cell body destruction in the target region leading to disrupted output, or both. Impaired decision making has also been observed in patients with specifically DA-related disorders, such as schizophrenia (Hutton et al., 2002), Parkinson’s disease (Pagonabarraga et al., 2007) and stimulant addiction (Rogers et al., 1999a). In keeping with these findings, healthy individuals with temporarily reduced DA levels from consuming a DA- depleting protein beverage are also impaired on the IGT (Sevy et al., 2006). However, the precise DA terminal regions that may be mediating these decision making impairments remains to be elucidated. Given the lack of understanding in individuals with disorders of the DA system, there has been a recent surge of experimental research into the role of the DA system in different forms of cost/benefit decision making in rodents. Systemic blockade of D1 or D2 receptors reduces the preference to wait longer or work harder to obtain a larger reward, whereas increasing DA transmission exerts differential effects on effort- or delay-based decision making, increasing or decreasing preference for larger rewards that come with a greater cost (Cousins et al., 1994; Cardinal et al., 2000; Denk et al., 2005; van Gaalen et al., 2006; Floresco et al., 2008a; Bardgett et al., 2009). Similarly, when rats choose between small, certain and large, yet risky rewards on a probabilistic discounting task, systemic administration of D1 or D2 antagonists reduce preference  108  for large, risky options (St.Onge and Floresco, 2009; St.Onge et al., 2010). Conversely, D1 or D2 agonists bias choice towards large, risky options. One potential candidate that may be mediating these alterations in choice is the PFC. DA modulates multiple cognitive functions mediated by different regions of the PFC, such as behavioral flexibility, working memory and attentional processes (Williams and GoldmanRakic, 1995; Granon et al., 2000; Chudasama and Robbins, 2004; Floresco et al., 2006), often in an “inverted U” shaped curve, where too little or too much DA activity impairs certain executive functions. However, there have been comparatively few studies investigating the contribution of PFC DA transmission to different forms of cost/benefit decision making. Reducing DA activity in the dorsal anterior cingulate alters effort-based decisions (Schweimer et al., 2005; Schweimer and Hauber, 2006), whereas blockade or stimulation of medial PFC D1 receptors reduces preference for larger, delayed rewards (Loos et al., 2010). 6-OHDA lesions of the OFC decrease impulsive choice, (Kheramin et al., 2004) and DA metabolism increases within the OFC during performance of a delay-discounting task that was specific to choice of delayed vs. immediate rewards (Winstanley et al., 2006). Increased DA release was also observed in the medial PFC in this study, but in this case, release occurred regardless of whether animals were performing the delaydiscounting task or in one of the yoked groups where reward was delivered with forced or no choice at all. Therefore, the DA signal in the medial PFC during delay discounting could contribute to the evaluation of response-outcome contingencies, a process in which the prelimbic PFC is known to be involved. Notably, there have been no studies investigating the contribution of different PFC DA receptors to decision making about risky rewards. Chapter 3 identified the prelimbic medial PFC as a critical region in the mediation of probabilistic discounting, whereas activity in other  109  subregions, (dorsal anterior cingulate, OFC, insular) do not appear to contribute to this behavior. Given the critical role that mesocortical DA plays in other forms of cognition (Floresco and Magyar, 2006), the present study investigated the contribution of medial PFC D1/D2 receptor activity to risk-based decision making using a probabilistic discounting task.  Method The animals, apparatus, and behavioural training on the probabilistic discounting task used in this chapter were identical to that described in Chapter 2. Surgical and histological procedures were identical to that described in Chapter 3. The behavioural training on the reward magnitude task was identical to that described in Chapter 4.  Microinfusion Protocol Following recovery from surgery, rats were subsequently retrained on either the probabilistic discounting or reward magnitude discrimination task for at least five days, and until, as a group, they displayed stable levels of choice behavior. Mock infusions were conducted in a similar manner to the experiments in Chapter 3 and 4. A within-subjects design was used for all experiments. The following drugs were used: the D1 antagonist R (+)-SCH23390 hydrochloride (1.0µg, 0.1µg; Sigma-Aldrich), the D2 antagonist eticlopride hydrochloride (1.0µg, 0.1µg; Sigma-Aldrich), the D1 receptor agonist SKF81297 (0.4µg, 0.1µg; Tocris Biosciences), the D2 agonist quinpirole (10µg, 1µg; SigmaAldrich). All drugs were dissolved in physiological 0.9% saline, sonicated until dissolved, and protected from light. The selected doses have all been well documented by both our group and others to be behaviorally active when given intracerebrally (Seamans et al., 1998; Ragozzino, 2002; Chudasama and Robbins, 2004; Floresco and Magyar, 2006; Floresco et al., 2006; Haluk  110  and Floresco, 2009; Loos et al., 2010). Infusions were conducted in a similar manner to that described in Chapter 3. Four separate groups of rats were used to test the effects of each of the four compounds (D1 antagonist, D2 antagonist, D1 agonist, D2 agonist). The order of treatments (saline, low dose, high dose) was counterbalanced across rats within a particular treatment group. Following the first infusion test day, rats received a baseline training day (no infusion). If, for any individual rat, choice of the Large/Risky lever on this day deviated by more than 15% from its pre-infusion baseline, the rat received an additional day of training prior to the second infusion test. On the next day, rats received a second counterbalanced infusion, followed by another baseline day, and finally the last infusion.  Data Analysis A detailed description of general data analysis procedures is outlined in Chapter 2. In brief, the choice data for each drug group were analyzed using 2-way within subjects ANOVAs with Treatment (saline, low dose, high dose) and Trial Block (100, 50, 25, 12.5%) as the within subjects factors. Response latencies, locomotor activity (photobeam breaks) and the number of trial omissions were analyzed with one-way ANOVAs.  Results Four groups of animals were initially trained in separate experiments and allocated to one of the four drug groups. The first two groups of 16 each, designated for D1 and D2 antagonist experiments, required an average of 28 days of training prior to reaching stable choice performance and receiving counterbalanced microinfusion tests. The second two groups of 14 and 14 rats for the D1 and D2 agonists required an average of 34 days of training prior to reaching  111  stable choice performance. Response latency, locomotor, and trial omission data obtained on test days for all four groups are presented in Table 8. Locations for acceptable placements in the medial PFC are displayed in Figure 14.  Figure 14. Histology for medial PFC DA infusions.  D1 and D2 Receptor Antagonism and Probabilistic Discounting D1 Blockade. Initially, 16 rats were trained for this experiment. One animal died during surgery and the data from three others were eliminated due to inaccurate placements, resulting in a final n=12. Analysis of the choice data revealed that intra-PFC infusions of the D1 antagonist SCH23390 resulted in a significant main effect of Treatment (F(2,22) = 3.26, p=.05) but no Treatment X Block interaction (F(6,66) = 0.92, n.s.). The high dose of SCH23390 (1µg) significantly decreased preference for the Large/Risky lever in the latter three blocks (p<.05; Figure 15A), whereas the low dose (0.1µg) produced no reliable change in choice behavior. D1 blockade had no effect on response latencies (F(2, 22) = 0.18, n.s.), trial omissions (F(2, 22) = 0.54, n.s.) or locomotor counts (F(2, 22) = 1.66, n.s.).  112  Figure 15. Effects of DA receptor manipulations in the medial PFC on probabilistic discounting. Data are plotted in terms of percentage choice of the Large/Risky lever during free choice trials by probability block (x axis). Symbols represent mean + SEM. Grey stars denote a significant main effect (saline vs. high dose, p<.05). Black stars denote a significant difference (p<.05) between treatment conditions during a particular probability block main effect. A Infusions of the 1.0µg dose of D1 antagonist SCH23390 accelerated probabilistic discounting, reducing risky choice. B In contrast, infusions of the 1.0µg dose of the D2 antagonist eticlopride retarded discounting and increased risky choice. C The D1 agonist SKF81297 induced a slight, non-significant increase in risky choice. D Infusions of the 10µg dose of the D2 agonist quinpirole abolished discounting, decreasing risky choice during the initial block and increasing choice during the final block.  113  Table 8. Locomotion, trial omission, and response latency data obtained following saline or drug infusions into the medial PFC. Locomotion counts are measured in photobeam breaks. Values are displayed as mean (SEM). * denotes p<.05 between vehicle and treatment. Saline  Low Dose  High Dose  1517 (187) 0.50 (0.19) 0.78 (0.11)  1423 (184) 0.50 (0.23) 0.75 (0.12)  1267 (105) 0.25 (0.13) 0.80 (0.08)  1924 (130) 0.17 (0.11) 0.54 (0.06)  1909 (134) 0 (0) 0.50 (0.04)  1998 (151) 0.25 (0.13) 0.50 (0.05)  Locomotion Trial Omissions Mean Response Latency (s)  1601 (252) 0.25 (0.18) 0.72 (0.05)  1682 (212) 0.25 (0.13)  1801 (276) 0.33 (0.33)  0.62 (0.10)  0.62 (0.07)  D2 (Quinpirole) Locomotion Trial Omissions Mean Response Latency (s)  1512 (181) 0.25 (0.25) 0.60 (0.07)  1475 (181) 0.75 (0.31) 0.66 (0.10)  1340 (148) 0.42 (0.26) 0.75 (0.11)*  DA Antagonists D1 (SCH23390) Locomotion Trial Omissions Mean Response Latency (s) D2 (Eticlopride) Locomotion Trial Omissions Mean Response Latency (s) DA Agonists D1 (SKF81297)  D2 Blockade. Initially, 16 rats were trained for this experiment. One animal died during surgery and the data from three others were eliminated due to inaccurate placements, resulting in a final n=12. Analysis of the choice data also revealed a significant main effect of Treatment (F(2,22) = 3.76, p<.05) but no Treatment X Block interaction (F(6,66) = 0.84, n.s.). However, in contrast to the effects of D1 receptor blockade, the high dose of eticlopride (1µg) significantly increased preference for the Large/Risky lever across all blocks (p<.05; Figure 15B), with the low dose (0.1 µg) producing a slight, but non-significant increase in choice. Eticlopride had no effect on response latencies (F(2, 22) = 0.63, n.s.), trial omissions (F(2, 22) = 1.45, n.s.) or locomotor counts (F(2, 22) = 0.99, n.s.). Thus, blockade of D1 or D2 receptors in the medial PFC had qualitatively opposite effects on probabilistic discounting. Reducing D1 receptor activity  114  increased discounting of larger, uncertain rewards, whereas D2 receptor antagonism reduced discounting, reflected as apparent decreases and increases in risky choice, respectively.  D1 and D2 Receptor Stimulation and Probabilistic Discounting D1 Stimulation. Initially, 14 rats were trained for this experiment. One animal died during surgery and the data from one rat were excluded because his baseline choice data were two standard deviations below the mean of the rest of the group, resulting in a final n=12. Following administration of the D1 agonist SKF81297 into the medial PFC, rats tended to show an effect opposite to that induced by the D1 antagonist, displaying a moderate increase in preference for the Large/Risky lever, with this effect being numerically greater after treatment with the lower, 0.1 µg dose. Despite this tendency, analysis of the choice data did not reveal a significant effect of Treatment (F(2,22) = 2.05, n.s.) or Treatment X Block interaction (F(6,66) = 0.10, n.s.; Figure 14C), although a direct comparison between the low dose and saline treatment conditions did show a trend towards statistical significance (p=.086). The D1 agonist also had no effect on response latencies (F(2, 22) = 0.67, n.s.), trial omissions (F(2, 22) = 0.06, n.s.) or locomotor counts (F(2, 22) = 0.36, n.s.). D2 Stimulation. Again, 14 rats were trained for this experiment. The data from one rat were excluded because his baseline choice data showed no prominent discounting after the 34 days of training, while the data pertaining to another rat were eliminated due to an inaccurate placement, resulting in a final n=12 in this group. Treatment with the D2 agonist quinpirole induced an effect on choice that was unique when compared to that induced by either DA receptor antagonist or the D1 agonist. Analysis of the choice data revealed no significant main effect of Treatment (F(2,22) = 0.05, n.s.) but there was a significant Treatment X Block interaction (F(6,66) = 2.33, p<.05, Dunnett’s p<.05). Simple main effects analyses further showed that,  115  whereas the low dose (1µg) of quinpirole had no effect on choice, the high dose (10µg) produced a pronounced “flattening” of the discounting curve. Specifically, this dose significantly (p<.05) decreased choice of the Large/Risky lever in the initial 100% block, but significantly increased risky choice during the last block (12.5%) relative to saline infusions (Figure 15D). Moreover, following infusions of either saline or the 1.0 µg dose of quinpirole, rats showed significant discounting of the Large/Risky option as the odds of obtaining the larger reward decreased over a session (p<.005). In contrast, the proportion of choice of this option did not significantly change across the four blocks after treatment with 10µg quinpirole (p>.25). Quinpirole had no effect on trial omissions (F(2, 22) = 0.84, n.s.) or locomotor counts (F(2, 22) = 1.72, n.s.), although the high dose significantly increased choice latencies across the four blocks (F(2, 22) = 3.54, p<.05 and Dunnett’s, p<.05; Table 9).  Win-stay/Lose Shift Analysis Infusions of selective D1 or D2 receptor agonists or antagonist into the medial PFC each induced distinct effects on decision making. To obtain further insight into how these treatments affected patterns of choice and resulting alterations in discounting, we conducted a supplementary analysis of the proportions of win-stay and lose-shift trials following each treatment using an identical procedure to that used in Chapter 4. Given that each of the four compounds induced distinct effects on choice behavior, we were particularly interested in directly comparing the effects of each compound relative to saline treatment. For this analysis, we used data obtained following treatment with the most effective doses of each drug and corresponding vehicle injections (for SKF81297, we used data obtained after treatment with the lower, 0.1µg dose). Analysis of winstay and lose-shift trials revealed a significant four way interaction of Trial Type (win-stay vs. lose-shift) X Treatment (saline vs. drug) X Receptor (D1 vs. D2) X Drug Type (Antagonist vs.  116  Agonist) (F(1,44) = 11.92, p<.05; Figure 16 and Table 10). As was observed with analysis of overall choice behavior, this four-way interaction was driven by the fact that each drug induced a distinct effect on win-stay/lose-shift tendencies. With respect to win-stay performance, under control conditions, rats displayed a strong tendency (between 80-90%) to select the risky lever after selecting this lever on the preceding trial and receiving reward, as we have observed previously (Stopper and Floresco, 2011). Conversely, animals tended to shift to the Small/Certain lever following a “loss” after choosing the Large/Risky lever on ~ 25-30% of these trials under control conditions.  Figure 16. Effects of PFC DA receptor manipulations on win-stay (grey bars) and lose-shift (white bars) tendencies. For clarity and comparative purposes, the data are presented here in terms of a difference score between the ratios obtained on drug vs. saline treatments (positive values indicate an increased ratio, negative values a decrease after drug treatment relative to control infusions). Raw data used in the overall analysis from which these values were obtained are presented in Table 2. Win-stay ratios index the proportion of trials rats chose the Large/Risky lever after receiving the larger reward on the previous trial. Lose-shift ratios index the proportion of trials rats shifted choice to the Small/Certain lever following unrewarded choice of the Large/Risky lever. Stars denote a significant difference from saline at the .05 level, n.s. – not significant.  117  Table 9. Win stay/lose shift ratios for rats performing the probabilistic discounting task following infusion of saline and the highest or most effective dose of D1 or D2 antagonist or agonists. Values are displayed as Mean (SEM). * denotes p<.05 between saline and drug treatment.  DA antagonists D1(SCH23390) 1µg D2(Eticlopride) 1µg DA agonists D1(SKF81297) 0.1µg D2(Quinpirole) 10µg  Win Stay ratio  Lose Shift ratio  (P: Risky choice after risky win) Saline Drug 0.91 (0.02) 0.91 (0.02) 0.81 (0.05) 0.89 (0.03)*  (P: Certain choice after risky loss) Saline Drug 0.26 (0.04) 0.35 (0.07)* 0.28 (0.04) 0.22 (0.05)  0.89 (0.03) 0.90 (0.03)  0.25 (0.05) 0.31 (0.06)  0.96 (0.02)* 0.80 (0.03)*  0.15 (0.04)* 0.26 (0.03)  Simple main effects analysis of the four-way interaction revealed that the D1 antagonist SCH23390 did not affect win-stay performance but did significantly increase lose-shift tendencies (Dunnett’s, p<.05), suggesting that the decrease in risky choice induced by these treatments may be attributable in part to increased sensitivity to negative feedback (i.e; reward omission). In contrast, D2 blockade with eticlopride (1µg) significantly increased the probability of choosing the risky option following a “win” (p<.05), while causing a non-significant decrease in lose-shift tendencies. Thus, the increase in risky choice induced by D2 blockade appears to be attributable primarily to an enhanced impact of obtaining a large reward on subsequent choice. The D1 agonist SKF81297 (0.1µg) significantly increased win-stay performance compared to saline (p<.05), but also had the opposite effect of SCH23390, reducing the tendency to shift after a loss from the Large/Risky lever (p<.05). In contrast, quinpirole (10µg) had the opposite effect of the D1 agonist on win-stay tendencies, significantly decreasing the probability of choosing the Large/Risky lever after a “win” (p<.05), suggesting a reduced sensitivity to receipt of larger, yet uncertain rewards. This treatment had no significant effect on lose-stay ratios. These findings indicate that D1 vs. D2 receptor modulation induces differential changes in choice performance that appear to be characterized by distinct changes in either the impact of obtaining the larger reward or negative feedback sensitivity.  118  Reward Magnitude Discrimination Blockade of D1 receptors or stimulation of D2 receptors reduced preference for the larger, uncertain reward during certain trial blocks of the discounting task. To assess whether these effects were attributable to a general disruption in discriminating between rewards of different magnitudes, we conducted another experiment, wherein two separate groups of rats were trained on a simpler reward magnitude discrimination task (see Chapter 4). Rats chose between two levers that delivered either one or four pellets, both with 100% probability. Fifteen rats were trained for 11 days on this task before receiving counterbalanced microinfusions of the high dose of SCH23390 (1µg) or quinpirole (10µg) and saline. The data for one animal were removed due to an inaccurate placement, leaving a final n of 6 in the SCH23390 group and 8 in the quinpirole group. D1 Blockade. Following saline infusions, rats displayed a very strong bias towards the larger reward, selecting this option on nearly 100% of the trials (Figure 17A). Following infusions of SCH23390 (1µg), there was no change in preference towards the four-pellet option, (F(1,5) = 1.72, n.s.). In contrast to choice, we did see a slight increase in response latencies following D1 blockade (saline = 0.81+/- 0.1s, SCH23390 = 0.98 +/-0.1s; F(1,5) = 7.18, p<.05). Locomotor activity (F(1,5) = 4.86, n.s.) and trial omissions (F(1,5) = 1.0, n.s.) were unaffected by SCH23390. Thus, even though infusions of this dose of SCH23390 reduced choice of the larger reward option during the probabilistic discounting task, this effect does not appear to be attributable to a general reduction in the subjective value of larger rewards.  119  Figure 17. Effects of DA receptor modulation in the medial PFC on reward magnitude discrimination. Rats were trained to choose between two levers that delivered either a 4 or 1-pellet reward immediately after a single press with 100% probability. A D1 blockade (SCH23390 1µg) did not significantly disrupt the preference for the larger 4-pellet reward during free-choice trials relative to saline treatment. B D2 receptor stimulation (quinpirole 10µg) also did not alter preference for the large reward.  D2 Receptor Stimulation. A similar profile of choice was observed for the rats receiving the high dose (10µg) of quinpirole into the medial PFC. Again, rats selected the four-pellet option on almost all of the free choice trials after saline infusions. This preference was not altered by stimulation of D2 receptors (F(1,6) = 0.53, n.s.; Figure 17B). Quinpirole also had no significant effect on latencies, locomotion, or omissions (all F’s <1.76, n.s.). Note that similar treatments did reduce choice of the larger reward on the probabilistic discounting task during the first, 100% probability block (Figure 15D). A possible explanation for this difference is that, unlike rats trained on the reward magnitude discrimination, those trained on the discounting task had learned that the relative utility of the Large/Risky option decreases over a session. Thus, their  120  representation of the relative value of the large reward option would be expected to be more labile than rats trained on the simpler task and therefore, more susceptible to disruption. Collectively, the results of this experiment show that even though blockade of D1 receptors and stimulation of D2 receptors substantially alters choices between small, certain and large, probabilistic rewards, these effects do not appear to be attributable to more fundamental impairments in the ability to discriminate between larger and smaller rewards.  Discussion Here we report that D1 and D2 receptors in the medial PFC exert a critical influence over choices between probabilistic versus certain rewards. Furthermore, decreasing or increasing activity of each of these receptors produced differing, and sometimes opposite, changes in choice, suggesting that they each exert distinct, yet complementary modulatory control over these decision making processes.  Effects of D1/D2 receptor blockade To our knowledge, this is the first demonstration that blockade of D1 or D2 receptor in the medial PFC induces opposing effects on behavior. Previous studies of this kind have revealed either that D1, but not D2 antagonism, disrupts functions such as attention or working memory (Williams and Goldman-Rakic, 1995; Seamans et al., 1998; Granon et al., 2000) or that both receptors act cooperatively to facilitate set-shifting or bias behavior away from conditioned punishers (Ragozzino, 2002; Floresco and Magyar, 2006). Our findings that SCH23390 and eticlopride induced opposite effects on choice suggest that normal decision making is dependent on a critical balance of frontal lobe D1 and D2 receptor activity, and that altering this balance induces dissociable changes in choice of certain/uncertain rewards.  121  PFC D1 blockade dose-dependently decreased preference for the Large/Risky option, most prominently during the last three probability blocks. SCH23390 increased probabilistic discounting, resembling the effects of this compound when administered systemically (St.Onge and Floresco, 2009). Interestingly, reducing DA transmission in human subjects via tyrosine depletion also leads to more conservative and poorer quality decision making on the Cambridge Gambling Task (McLean et al., 2004). Our results suggest that these effects may be mediated in part by reduced prefrontal D1 activation. Choice-by-choice analysis further revealed that this reduced preference for the risky option was linked to an increased tendency to choose the Small/Certain option following a non-rewarded risky choice, suggesting that the effects on decision making may be the result of increased sensitivity to negative feedback. In a similar vein, blockade of D1 receptors in the prelimbic or dorsal anterior cingulate reduces preference for larger rewards when they are either delayed (Loos et al., 2010) or associated with a greater effort cost (Schweimer and Hauber, 2006). Collectively, these findings suggest that PFC D1 signaling exerts a profound influence on cost/benefit evaluations, facilitating the ability to overcome costs that may be associated with larger rewards in an effort to maximize long-term gains. In stark contrast, PFC D2 receptor blockade increased preference for the Large/Risky option, slowing the shift in choice bias as reward probabilities decreased over a session. Notably, this effect resembles that induced by PFC inactivation under similar task conditions (St.Onge and Floresco, 2010; Chapter 3). However, we do not believe this reflects a general increase in “risky” behavior per se. Rather, our previous findings led us to conclude that the medial PFC plays a critical role in monitoring changes in reward probabilities to adjust behavior accordingly. The present results expand on this, revealing that D2 receptors make an essential contribution to PFC regulation of this aspect of decision making. This apparent increase in risky choice was driven  122  more prominently by an increased tendency to select the risky option after obtaining a large reward on the preceding trial. Thus, rather than integrating information about the likelihood of obtaining the larger reward across multiple trials, D2 blockade caused receipt of the larger reward to exert a greater, and more immediate impact on the direction of subsequent choice. This is in keeping with a recent study in humans, where D2 antagonism increased both choice of options associated with higher reward probabilities and corresponding changes in vmPFC activity (Jocham et al., 2011). Collectively, these findings show that PFC D1 and D2 receptors form distinct, yet complementary contributions to decision making. D1 receptor activity promotes choice of larger, yet uncertain or more costly rewards, whereas D2 receptors mitigate the immediate impact that larger, probabilistic rewards exert over choice bias, facilitating the ability to adjust behavior over the long-term when the likelihood of obtaining these rewards change.  Effects of D1/D2 Receptor Stimulation Intra-PFC infusions of D1 receptor agonist SKF81297, within dose ranges that have been shown to exert differential effects on other forms of cognition (attention, working memory), did not significantly alter risky choice, although these treatments slightly increased preference for the Large/Risky lever, most prominently with the low dose. Interpretation of this null effect should be approached with caution, as these non-monotonic dose/response effects suggest that SKF81297 may have an effective dose range that is narrower than it may be for other cognitive functions. Moreover, the 0.1µg dose did significantly alter choice patterns, increasing win-stay performance and decreasing lose-shift tendencies, where rats were more likely to choose the Large/Risky lever following both rewards and reward omissions. Nevertheless, the fact that increasing doses of SKF81297 did not significantly alter choice indicates that supranormal stimulation of PFC D1 receptors does not substantially interfere with decision making about risks and rewards. In  123  contrast, similar treatments decrease choice of larger, delayed rewards (Loos et al., 2010), providing further support that different types of cost/benefit decision making can be dissociated pharmacologically. The D2 agonist quinpirole induced a true “impairment” in decision making, markedly flattening the discounting curve, with rats displaying no discernible discounting upon changes in reward probabilities. Choice of the four-pellet option was reduced in the 100% block (when it was most advantageous), but increased in the 12.5% block (when it is least advantageous). Following D2 stimulation, the overall proportion of Large/Risky choices did not change relative to saline (~73%), but animals were completely insensitive to changes in these probabilities. Thus, excessive D2 receptor activation severely interfered with the ability to adjust choice, apparently causing rats to employ a simpler alternation strategy across blocks while maintaining a bias towards the Large/Risky lever. This finding, in combination with the effects of eticlopride, suggests that the relative levels of D2 (rather than D1) receptor tone in the medial PFC has a critical impact on this aspect of decision making and either increasing or decreasing this activity can interfere with performance. The disadvantageous choice pattern produced by quinpirole bears a striking resemblance to that induced by reducing motivation for food through long term free-feeding (St.Onge and Floresco, 2009). These complementary findings make it tempting to speculate that they may be related phenomena. Indeed, changes in medial PFC DA efflux have been proposed to reflect a generalized food reward or incentive motivational signal (Ahn and Phillips, 1999; Winstanley et al., 2006). Thus, changes in the amount of reward obtained over time may be signaled to the PFC by corresponding fluctuations in mesocortical DA levels that, via actions on D2 receptors, may be used to detect changes in the amount of reward obtained over time and facilitate alterations in  124  choice bias. It follows that flooding D2 receptors may disrupt this dynamic signal, which could ultimately produce more static patterns of choice.  Dissociable Contributions of PFC D1 and D2 Receptors to Risk-Based Decision Making The question remains as to why blockade of D1 or D2 receptors should exert opposing effects on risky choice, given that endogenous DA activates both receptors? Contemporary theory on how these receptors differentially affect PFC neural network activity may provide insight into this issue (Durstewitz et al., 2000; Seamans and Yang, 2004). D1 receptors have been proposed to decrease the influence of weak inputs, stabilizing network activity so that a single representation dominates PFC output. Conversely, D2 activity attenuates inhibitory influences, allowing PFC neural ensembles to process multiple stimuli/representations, placing theses networks in a more labile state that may permit changes in representations. During different phases of the probabilistic discounting task used here, animals at some points must either maintain (within a probability block) or modify (across blocks) their representation of the relative value of the Large/Risky option. Thus, the opposing effects of D1/D2 antagonism described here may reflect differential contributions of these receptors during distinct phases of the task. D1 activity may stabilize the representation of the relative long-term value of the risky option within a particular block, maintaining choice bias even when a risky choice leads to reward omission (keeping the “eye on the prize”). Blocking these receptors would make animals more sensitive to reward omissions (i.e; increasing lose-shift tendencies), and reduce risky choice. Conversely, as the Large/Risky option yields fewer rewards across blocks, D2 receptors (possibly on a different neuronal population) may facilitate modifications in value representations. As such, reducing their activity would disrupt the updating of these representations and corresponding changes in choice bias. This model may also partially account  125  for the effects of increasing D1 and D2 receptor activity, which would be expected to lead to either more persistent choice of the Large/Risky option or induce a “hyperflexible” state, respectively. Thus, our findings suggest that PFC DA tone makes a critical and complex contribution to risk/reward judgments. By striking a fine balance between D1/D2 receptor activity, mesocortical DA may help refine cost/benefit decisions between options of varying magnitude and uncertainty, promoting either exploitation of current favorable circumstances or exploration of more profitable ones when conditions change.  126  CHAPTER 6: GENERAL DISCUSSION The present thesis used a combination of behavioural, statistical, anatomical, and pharmacological techniques to elucidate the nature of probabilistic discounting in rats and the underlying neural circuits and neuromodulatory systems that contribute to this behaviour. Chapter 2 described probabilistic discounting as a model of risk-based decision making in rats. Statistical modeling was used to examine the development of probabilistic discounting rates over time and whether the experience of greater or less than expected reward from choice outcomes at different times would influence an individual rats’ likelihood of making a probabilistic versus certain choice in the future. The results of the analysis indicated that there is substantial individual variability in discounting of large, probabilistic rewards in well-trained animals that develops over the course of training, but these discounting patterns are not influenced by early luckiness in obtaining reward. Moreover, the probability of choosing larger, yet risky options is governed by recent luck in forced choice outcomes early in training and by outcomes of free choice trials in the animal’s recent reinforcement history. Chapter 3 showed that the prelimbic region of the rat medial PFC makes a selective contribution to probabilistic discounting by keeping track of changes in reward probability in order to update value representations, whereas the insular and dorsal anterior cingulate subregions have no influence on choice. Although it does not contribute to choice, activity in the OFC region influences decision latency. Through a series of asymmetrical disconnections, the data in Chapter 4 revealed that separate neural circuits mediate different aspects of probabilistic discounting. Amygdala projections to the nucleus accumbens bias choice towards large, risky reward options whereas top-down projections from the medial PFC to the amygdala regulate this bias and adjust choice towards smaller, but potentially more valuable, options. Chapter 5 showed that the contribution of medial PFC activity to probabilistic  127  discounting is further modulated by a fine balance of D1 and D2 receptor activity. The purpose of this final chapter is to integrate the findings of this dissertation into the larger body of literature on the neural basis of decision making about probabilistic rewards, with a particular focus on the broader functions of the PFC and how it interacts with other neural systems to guide decision making.  Cost/Benefit Decision Making about Certain vs. Probabilistic Rewards Recruits Specific Neural Circuits When deciding amongst different rewarding options, neural activity in a host of brain circuits is likely to be activated depending on the specific choice situation an organism is in. While this complexity is interesting in and of itself, it has posed difficulty in elucidating which parts of the brain are directly responsible for guiding choice in different contexts. Moreover, in situations that involve repeated decisions, additional systems are likely to be recruited in order to integrate choice information across time. By pulling together research from human clinical populations, neuroimaging, and animal models of decision making, we are beginning to unravel the specific neural underpinnings of the mechanisms that guide choice in situations of uncertain rewards. We are routinely faced with choices requiring us to evaluate the relative risks and rewards associated with different options, choosing between potentially more profitable, but uncertain outcomes, and safer, yet more modest, rewards. For example, managing an investment portfolio can cause one to be torn between the allure of risky stock investments that may lead to large payoffs and the nagging conservative urge to stick with the security of savings accounts. Effective allocation of our resources requires monitoring of the current state of the economy in order to decide whether to continue pursuing larger, yet risky, returns or be content with small, reliable  128  payouts. In order to make an informed decision in these situations, an individual must learn the magnitude and probability of achieving reward in order to calculate the value associated with the different options. A similar process occurs while animals are learning and performing a probabilistic discounting task. Evidence from previous literature, as well as that presented in this thesis, suggests that learning reward contingencies associated with different choice options may engage neural activity in the OFC, the BLA, with modulatory input from the DA system. In contrast, once these associations are learned, input from the OFC may not be required except to facilitate decision speed. When deciding amongst options where their relative values are being changed frequently, medial areas of the PFC appear to monitor those changes, facilitating adjustments in choice when certain options may have less long-term value. Moreover, under most choice situations, the potential for reward will also activate the DA system, facilitating stabilization of strategies that are successful in obtaining reward and promoting exploration of other options when strategies fail. Although the neural circuits that may be mediating other types of decision making, such as those involving explicit punishments or emotionally-laden stimuli, continue to be investigated, we can conclude that this mesocorticolimbic circuit involving the medial PFC, BLA, NAc and DA system specifically mediates cost/benefit decisions about certain vs. probabilistic rewards. Furthermore, our findings, as well as others, suggest that the PFC plays a substantially complex and important role in this type of decision making.  Regional Specialization for Different Types of Decision Making in the Rat PFC Our finding in Chapter 3 that the prelimbic region of the medial PFC contributes to riskbased decision making complements a broad literature associating different regions of the PFC  129  with complex cognitive functions. By using relatively discrete reversible inactivations, we were able to target different subregions of the PFC and assess their specific contribution to this aspect of decision making. In contrast to human brain lesion studies, the effects of temporary disruption of these regions could be examined without the confounds of lesion spread across different regions that can occur after permanent brain damage and repeated behavioral testing. We also discovered that, despite suggestions from lesion and imaging literature of the potential importance of the lateral OFC in risk-based decision making, discrete disruptions of this area had no effect on probabilistic choice per se, but this region may influence the speed at which individuals make these decisions once initial valuations have been calculated. Moreover, inactivation of the dorsal anterior cingulate and the agranular insular cortex had no effect on choice outcome or latency in this paradigm. It is interesting to compare the findings of Chapter 3 to previous studies on the dissociable contribution of rat PFC subregions to other forms of cost/benefit decision making, such as delay or effort discounting. The OFC appears to play a role in delay discounting, guiding choice between larger, delayed versus smaller, immediate rewards, whereas lesions of the dorsal anterior cingulate do not interfere with this form of decision making (Cardinal et al., 2001; Mobini et al., 2002; Winstanley et al., 2004; Rudebeck et al., 2006). Conversely, the dorsal anterior cingulate, but not the OFC, is a component of the circuitry underlying effort-related decisions, but not delay discounting (Walton et al., 2003; Rudebeck et al., 2006; Floresco and Ghods-Sharifi, 2007; but see Schweimer and Hauber, 2005). Notably, medial PFC lesions do not significantly alter cost/benefit judgments involving either delay- or effort- related costs (Cardinal et al., 2001; Walton et al., 2003). Our findings that the prelimbic PFC (but not the OFC or dorsal anterior cingulate) contributes to guiding choice between certain and probabilistic rewards further supports the notion  130  that different response costs (e.g.; delays, effort, uncertainty) are processed by separate frontal lobe regions (Rudebeck et al., 2006; Floresco et al., 2008b). It is important to highlight that with delay or effort discounting procedures, animals typically obtain some reward after each choice, receiving immediate feedback about the costs associated with larger rewards. Imposition of these costs may reduce the perceived value of larger rewards, biasing choice towards smaller but more easily obtainable rewards. On the other hand, probabilistic discounting requires continuous monitoring of rewarded and non-rewarded outcomes during both the forced and free choice trials, as well as integration of this information over many trials to estimate the relative probability of obtaining larger rewards. In turn, this information may be incorporated into computations about the relative long-term value of small/certain versus large/uncertain rewards under conditions where reward probability changes. The selective involvement of the medial PFC in this form of decision making may be linked to its role in the temporal organization of behavior (Fuster, 2000), whereby internallygenerated information is used to monitor changes in reward probabilities and update value representations that, in turn, may facilitate efficient decision making. The PFC is well-designed for this type of function. In general, the PFC receives substantial input from all other cortical areas, thalamic nuclei and many subcortical areas, such as the hippocampus, basal ganglia, amygdala, hypothalamus, and brain stem nuclei. In particular, the medial areas of the PFC receive strong input from the lateral orbital cortex, the lateral amygdala, the mediodorsal thalamus, subiculum of the hippocampus and the ventral tegmental area. Moreover, the pyramidal cells in the PFC are more spinous than other cortical pyramidal cells, increasing the amount of excitatory input they can receive (Elston, 2000). Therefore, the medial PFC is ideally suited to receive and  131  process information about environmental stimuli and recent action outcomes in order to regulate future behavioural output. The fact that only disruptions of medial PFC activity affected choice behaviour on this task supports the notion that different subregions of the PFC are more specialized for either certain types of processing or specific types of information. However, given that the OFC contributed to choice latency, it is also possible that PFC neurons in multiple areas were tracking salient features of the environment during task performance but only information in the medial PFC and OFC came to dominate frontal control of overt behaviour in terms of choice and latency, respectively. Indeed, findings from neural recordings in non-human animals suggests that PFC neurons are highly flexible and have the potential to be driven by different kinds of input depending on current internal and external conditions (Duncan, 2001). In this view, the PFC acts as a “workspace” where relevant information is attended to, irrelevant information filtered out, and important information is tracked by PFC neurons and linked up with sensory information. For example, Lapish and colleagues (2008) showed that networks of neurons in the rat dorsal anterior cingulate moved through different discernible population states as the rat entered different cognitively defined task stages while performing a delayed response task on a radial maze. PFC networks tracked or monitored responses and events through transitions among cell assemblies. The activity was also functionally relevant as the tracking was inconsistent when rats made many memory errors, but was comparatively cohesive when behavioural performance was good. Moreover, preliminary findings using neural recordings from rats performing the probabilistic discounting task suggest that medial PFC neurons also exhibit different population activity states for risky versus safe lever choices, as well as during different probability blocks (Di Pietro et al., 2010). While these findings highlight the flexibility of neurons within one subregion, they do not  132  necessarily suggest that neurons across the entirety of the PFC could exhibit the same function. Moreover, there is increasing evidence that the functions of the dorsolateral/anterior cingulate regions and the orbitofrontal regions, in particular, can be dissociated (Duncan, 2001). In sum, our findings suggest that although neurons in different PFC subregions could be recruited for task performance under certain conditions, the medial PFC in the rat specifically contains the population of cells that are exquisitely suited for keeping track of changes in reward probability in order to update value representations and corresponding changes in choice biases.  A Broader Role for the Medial PFC in Exploiting and Exploring Rewards Increasing evidence suggests that medial areas of the PFC across multiple species may be key neural substrates involved in the balance between exploitation and exploration of potentially rewarding stimuli (Walton et al., 2004). In natural settings, animals are usually faced with a few different foraging areas that yield food on different schedules and choose to apportion their time to each of the locations as it sees fit. When the animal selects each foraging area, it collects information to update its representation of the value of each location based on the frequency and magnitude of food it can obtain there. Once the animal has exploited the resources of one area, it should explore new areas, even if they might not have as abundant food as the previous environment. For example, a rat may choose between foraging in a dumpster near a restaurant or in an inhabited house. The house may provide fewer opportunities for large quantities of food, but if the rat is undetected, it may have a reliable source of calories with few predators to watch out for. In contrast, the dumpster may have the potential for larger amounts of food, but there is also a greater likelihood of other dangerous animals lurking. Also, some days the restaurant may be closed or throws no food away, potentially making the probability of actually getting food  133  associated with the dumpster lower than the house. Regardless of the rats’ initial choice, it should continue to occasionally sample from each location in order to update its value representation in case the average food rate changes (e.g. if the restaurant closes). This alternation process is likely to be instinctual and may explain why during decision making tasks that involve preferences for different options, rats often select less valuable options (e.g.; see Chapter 2). Matching tasks are designed to tap into this dynamic foraging situation whereby two different actions lead to reward with independent probabilities (Herrnstein, 1997). If a reward is associated with a particular action but that action is not performed, then the “unharvested” reward is still associated with that same action on the next trial. Under these conditions, the ideal strategy to maximize reward is to switch between the more and less profitable actions because the probability of the reward being associated with the less profitable action gradually increases on every consecutive trial that the action is not chosen to the point where it will then exceed the average probability of reward associated with the first profitable action. Therefore, they should allocate their choice of the two actions so that the average rate of reward associated with each action is equated. Similar to probabilistic discounting, a key cognitive process needed to perform matching tasks is the ability to form an estimate of the average reward rate associated with each option over time which will lead to an internal representation of the "value" of possible actions. In macaques performing a computerized matching paradigm where they chose to perform one of two joystick movements to obtain reward delivered with two independent probabilities, lesions to the ACC sulcus caused the monkeys to no longer use more than the most recent outcome to guide each choice and switch to the more profitable option (Kennerley et al., 2006). In another study, monkeys performed a probabilistic choice task where some outcomes were surprisingly larger or smaller than expected based on learned probabilities. In this paradigm,  134  ACC neurons fired more following these surprising rewards (or lack of) compared to regular trials (Hayden et al., 2011a). Furthermore, adjustments in choice were greater following these unexpected outcomes as well. These findings suggest that ACC neurons are sensitive to whether choice outcomes deviate from what is expected and therefore, may be involved in monitoring and detecting situations where choice would be required to change. Moreover, these neurons fired less during trials where the probabilities associated with reward were hidden from the monkey, suggesting that under conditions where information is not useful to learning, the ACC is not active. The same group also found that neurons in the same ACC region responded with greater activation the longer the monkey made choices associated with one depleting reward patch” up to a certain threshold, after which the monkey switched to choosing from the other “patch” that had replenished reward (Hayden et al., 2011b), suggesting that the ACC was signalling the relative value of leaving a diminishing resource patch for an alternate one. Increasing evidence suggests that the ACC in humans may have a similar function to that described in primates and rodents. In situations where human subjects are presented with new information that must be incorporated into existing value representations, the ACC is active when detecting the salience of the information as an indicator of how volatile the environment is (Behrens et al., 2007). Greater volatility, or uncertainty in the likelihood of reward associated with choosing different options, is associated with greater learning rates (the degree to which the representation is updated) (Dayan et al., 2000). In the Behrens et al. (2007) study, Bayesian modeling revealed that individuals with greater association between volatility and ACC activity in the monitoring phase gave more weight to the most recent piece of information and learned the matching paradigm faster. Therefore, the ACC appears to be particularly important for detecting when reward estimates need to be updated. These findings are consistent with recent  135  neuroimaging studies showing increased ACC activation during choices that ran counter to an individuals’ preferred choice strategy (Roiser et al., 2009). Collectively, these findings suggest that the ACC plays a key role in switching between actions associated with differential profitability of reward. In Chapter 3, our data showed that the medial PFC in rats (homologous to Area 32 of the human/primate ACC) is also involved in keeping track of changes in reward probability in order to adjust choice towards the more profitable option (either the risky or certain lever). Moreover, in Chapter 4, the increase in risky choice induced by the overall medial PFC-BLA disconnection, and the selective descending pathway disconnection, was primarily characterized by a decrease in the tendency to shift following a loss after choosing the risky lever across the entire session, indicating that animals were less sensitive to the effects of loss on subsequent choice. It appears that these animals were not able to integrate recent choice information into their estimates of the long-term value of the Large/Risky versus Small/Certain levers and thus, had difficulty adjusting choice towards the more profitable option. Thus, the medial regions of the PFC may play a fundamental role in regulating adjustments in action selection in different types of decision making situations where rewards are uncertain.  The PFC Exerts Executive Control Over Valuation Signals The top-down control of the PFC over the BLA highlighted in Chapter 4 further refines this complex role of the PFC in decision making. The idea that areas of the PFC may exert control over more fundamental behaviours mediated by evolutionarily-older areas is not novel. A healthy literature has implicated medial/ventral and lateral PFC regions in the regulation of emotional responses and appraisals thought to be mediated by the amygdala in both humans (Etkin et al., 2006; Wager et al., 2008) and non-human animals (Sotres-Bayon et al., 2004; Quirk and Beer,  136  2006; Milad and Rauch, 2007). ACC–amygdala coupling is also observed during pain regulation (Mayberg et al., 2002). The development of technology that transiently disrupts cortical activity in humans through transcranial magnetic stimulation or transcranial direct current stimulation has allowed for greater experimental control when assessing the neural pathways mediating different types of cognition and behaviour. For example, transient suppression of the DLPFC in humans increases choice of small, immediate rewards over larger delayed rewards (Figner et al., 2010), as well as risky over safe options (Knoch et al., 2006), whereas activation of DLPFC decreases risky choice (Fecteau et al., 2007). However, the same process does not affect a valuation task, suggesting that the DLPFC tempers reward valuations computed somewhere else in the brain that normally biases behaviour towards delayed or risky rewards with larger magnitudes (Figner et al., 2010). As discussed in Chapter 3, the prelimbic cortex of the medial PFC is considered homologous to Area 32 of the ACC and shares similar functions to the DLPFC in primates and humans. The findings presented in this thesis suggest that the prelimbic region of the rat PFC is also acting to “temper” the valuation process transmitted by the BLA to the NAc in a similar manner to that observed in humans. By monitoring current task conditions, the medial PFC can govern when to reduce the impact the BLA-NAc valuation signal has on overt behaviour. With the task used here, at the beginning of a session the best option is the large reward lever with high probability, but the value of this option changes to the point where it is no longer ideal. A form of cognitive control is needed when multiple reward factors (i.e.; both magnitude and probability) need to be integrated and the initial appeal of a large magnitude reward down-regulated, in order to focus on the longterm benefit of choosing a smaller reward. The medial PFC appears to be acting in this role while rats perform the probabilistic discounting task. More generally, top-down control by the PFC may  137  be necessary when selecting actions that have not yet been strongly associated with reward in order to explore other options (Daw et al., 2006). These findings have important implications for a debate in the literature regarding the process of reward discounting. One model proposes that independent neural systems represent tempting but costly rewards (e.g. small, immediate or large, risky rewards) and all rewards regardless of cost (McClure et al., 2004). Cognitive control is required to inhibit automatic responses for these costly rewards and elicits activation in DLPFC, inferior frontal gyrus and posterior parietal cortex (McClure et al., 2004). An alternative perspective is that information concerning reward magnitude and cost is integrated and computed in one system concerning valuation, which then drives behaviour towards the more valued option (Kable and Glimcher, 2007). In this model, brain regions representing reward magnitude (e.g. NAc, mesial prefrontal cortex, posterior cingulate cortex; Knutson et al., 2001; Knutson and Cooper, 2005) show reduced activation to increasing costs (e.g. delays) implying diminishing subjective value. Increasing lines of evidence support the former model. Using fMRI and a delay discounting task that isolated neural responses to reward magnitude and reward delay, Ballard and Knutson (2009) found that dissociable regions responded to magnitude (NAc, mesial PFC, posterior cingulate) and delay (DLPFC, posterior parietal) and the interaction of magnitude and delay revealed activation in the inferior frontal gyrus. As expected, individuals who exhibited greater impulsivity on the task showed less activation to reward magnitude in the NAc. Importantly, increasing reward delay negatively correlated with DLPFC activation and did so more steeply for impulsive individuals, a finding consistent with previous literature (McClure et al., 2004; 2007). Therefore, more impulsive people were both less sensitive to larger magnitudes and more sensitive to longer delays. These findings are consistent with the two system model, where choosing to forego  138  immediate gratification for greater long-term value depends on cognitive inhibition of other neural systems. In a similar vein, Hare and colleagues (2009) had dieters engage in real decisions about food consumption. Activity in ventromedial PFC was correlated with the value of the food regardless of whether the subject exerted self-control over choosing to consume the food. In contrast, DLPFC activation increased when subjects exercised self-control and avoided foods that they liked but were unhealthy; the activation was correlated with the ventromedial PFC signal. Thus, the DLPFC integrated information about health into the valuation of food choices in the vmPFC and doing so, exerted control over the choices made by subjects by modulating the valuation signal. Collectively, these findings suggest that while some regions encode the value of different options (e.g.; vmPFC, OFC, amygdala), the PFC may exert cognitive control over these valuations in order to maximize gain in situations of uncertainty.  Mesocortical Dopamine May Signal Changes in Reward Obtained Over Time If the medial PFC is primarily engaged in monitoring task contingencies in order to facilitate adjustments in choice biases, one question that remains is what input may signal changes in reward value to the PFC that, in turn, may be used to modify the valuation signal in the amygdala. The PFC receives substantial glutamatergic, GABA-ergic and modulatory input from different regions and systems. The results from Chapter 5 suggest that dopaminergic input from the VTA is another key modulator of medial PFC activity during decision making about risks and rewards, and is mediated by a fine balance between activity of D1 and D2 receptors. Intriguingly, blockade of these receptors in the PFC induced opposite changes in behaviour, with D1 antagonism increasing discounting of the risky option, whereas D2 receptor blockade had the opposite effect. Note that rats performing the probabilistic discounting task must both maintain (within a probability block) and modify (across  139  blocks) their representation of the relative value of the Large/Risky option during the different phases of the task. The contribution of DA signaling at these different receptor sites may be to bias PFC networks to either stabilize the current representation of the relative long-term value of the risky option through D1 activation when probabilities of reward are high (e.g. 100 and 50% blocks) or facilitate modifications in value representations of this option through D2 activation when probabilities are decreasing. In support of this notion, the DRD2 gene has been linked with exploitative learning as a function of positive and negative decision outcomes (Frank et al., 2009). These findings emphasize the importance of a “balance” of DA receptor activation in PFC, where too little or too much DA activity impairs certain executive functions (Williams and Goldman-Rakic, 1995; Granon et al., 2000; Chudasama and Robbins, 2004; Floresco and Magyar., 2006). As discussed in Chapter 5, the pattern of choice produced by D2 stimulation in the medial PFC looks very similar to that induced by reducing motivation for food through long term freefeeding (St.Onge and Floresco, 2009). Indeed, increases in DA efflux in the medial PFC are present during simply the consumption of food or fluid in food deprived animals (Cenci et al., 1992; Bassareo and Di Chiara, 1997; Taber and Fibiger, 1997; Phillips et al., 2004). This increase can last up to 30 minutes, even with small amounts of food (Feenstra and Botterblom, 1996). DA release in the medial PFC could be a particularly important signal about recent food consumption, information that can then guide future behaviors that rely on motivation. Winstanley and colleagues (2006) measured DA levels in the medial PFC while animals were performing a similarly-designed delay discounting task where rats chose between a small, immediate reward and a large, delayed reward. Increased DA release was observed while animals performed the task, but it tracked the amount of food the rats were getting based on their choice of the large reward as DA was also increased in a yoked group where reward was delivered with no choice at  140  all. Therefore, changes in medial PFC DA efflux have been proposed to reflect a generalized food reward or incentive motivational signal (Ahn and Phillips, 1999; Winstanley et al., 2006). Preliminary data from our laboratory suggest that this theory may apply to probabilistic discounting as well. DA efflux in the medial PFC exhibits a similar profile in both animals trained on the probabilistic discounting task and on a Food Only control task where rats are yoked to receive the same amount of reward as a rat on the probabilistic discounting task, but without making choices (St.Onge et al., 2011). In both cases, DA levels decreased across the session when animals were receiving less food. Thus, changes in the amount of reward obtained over time may be signaled to the PFC by corresponding fluctuations in mesocortical DA levels. This signal has no useful implications for the Food Only task, but provides key information to animals performing the probabilistic discounting task. Animals learn how to perform the task by keeping track of recent outcomes associated with choosing the Large/Risky or Small/Certain levers, particularly during the forced choice trials, which are meant to signal when the probability of the large reward has changed. When the animal chooses the Large/Risky lever, it either gets rewarded or it does not. As the probability of obtaining the 4 pellets increases or decreases, the rat must recognize these outcomes and adjust choice either towards or away from the Small/Certain lever in order to maximize the amount of sugar pellets obtained. Given that we provide no external cues that specifically signal a “win” or a “loss” following choice of the risky lever, the rat must use a “food signal” as a means of keeping track of changes in reward obtained over time. Fluctuations in mesocortical DA may represent part of this signal. As discussed in Chapter 2, recent rewarding outcomes play an important role in biasing choice towards the risky lever. We also showed that receiving greater than expected rewards during forced choice trials significantly increases the probability of making a risky choice. Given that unexpected  141  rewards result in an increase in phasic DA bursts (Schultz et al., 1997), it is possible that additional subsecond DA events may further influence choice biases above and beyond the tonic DA signals already present. Current technical limitations in measuring subsecond DA responses in the PFC make it difficult to assess this hypothesis. Nevertheless, the above findings suggest that DA efflux in the medial PFC, perhaps via actions on D2 receptors, may be used to detect changes in the amount of reward obtained over time and facilitate alterations in choice bias, in part by modulating BLA-NAc circuitry. Although speculative, one hypothesis is that D1 activation on GABA interneurons would increase inhibition of adjoining pyramidal neurons, reducing the influence that the PFC exerts over a BLA-NAc circuit that biases choice towards the large reward option, even when some choices of the risky lever lead to non-rewards (Seamans and Yang, 2004). In contrast, D2 receptor activation on pyramidal neurons could reduce GABAergic inhibition, leading to disinhibition of a population of pyramidal neurons projecting to the BLA. This, in turn, may facilitate inhibitory influences of the PFC on the BLA-NAc circuit, promoting shifts in choice to the smaller, but more valuable option when probabilities associated with the large reward are low. However, whether activation of D1 vs. D2 receptors on separate populations of neurons in the medial PFC is mediating these different activity states remains to be tested.  Convergence of Choice Information in the NAc Biases Overt Behaviour The neural processes related to choice discussed in the above sections are eventually formulated into overt behaviour (i.e., pressing a lever). Our findings suggest that the amygdala’s excitatory input to the NAc provides one mechanism by which this may occur. Inactivation of the BLA disrupts both reward-seeking approach behaviour as well as phasic activation of NAc neurons, suggesting that the BLA provides a crucial excitatory signal to the NAc that drives  142  reward-seeking behaviour (Ambroggi et al., 2008; Jones et al., 2010). The NAc then projects to pallidal and mesencephalic motor effector sites whereby it can directly influence motor behaviour (Zahm and Heimer, 1990). There is considerable convergence and overlap of different inputs in the NAc. Projections from different regions of the frontal lobes overlap with those from the amygdala and hippocampus. In some case, inputs from two or more of these regions converge on the same individual neuron (Callaway et al., 1991; O'Donnell and Grace, 1995; Mulder et al., 1998; Floresco et al., 2001b; French and Totterdell, 2003). This anatomical arrangement places this striatal region in an ideal position to regulate the cortical and limbic influences over behaviour, either by producing the motor plans delivered via descending pathways, or by gating the influence that information processed by the limbic system exerts over behaviour (O'Donnell and Grace, 1995; Goto and O'Donnell, 2002). More specifically, the NAc biases the direction of behaviour based on the relative strength of different inputs at a given time (Floresco, 2007). There is a vast literature implicating the NAc in representing reward magnitude in both humans (Delgado et al., 2000; Knutson et al., 2001; Ernst et al., 2005; Blair et al., 2006; Marsh et al., 2007; Preuschoff et al., 2006; Ballard and Knutson, 2009; Smith et al., 2009) and rodents (Cardinal and Howes, 2005; Stopper and Floresco, 2011), which may partially reflect the influence of amygdala inputs. However, as different costs and benefits are included in calculations of long-term value in a dynamic environment (such as probabilistic discounting), cortical regulation of amygdala value estimates may shift the bias of the NAc towards options that may not necessarily be associated with the largest magnitude but may have greater potential value in the future. This would fit with proposed role for the NAc in guiding behaviour when there is uncertainty about which stimuli in the environment are associated with reward, particularly under  143  conditions where the incentive value of these stimuli changes (Floresco, 2007; Nicola, 2007). It remains to be examined whether the NAc receives input solely from the amygdala to guide motor output relevant for decision making or if it also receives input from either different regions of the PFC (e.g. OFC) or the hippocampus. Another interesting line of research would be to examine the influence of local DA inputs in the NAc near the BLA projection fibers. Phasic DA release in the NAc has been associated with subjective value (Day et al., 2010; Gan et al., 2010; Wanat et al., 2010). Therefore, it would be interesting to test whether both BLA input and DA activation in the NAc bias choice in a particular way by inducing a unilateral inactivation of the BLA on one side and a contralateral DA antagonist in the NAc. The results may indicate whether DA released in the NAc facilitates the BLA input on behaviour by decreasing the influence of other weak inputs and further promoting choice of the larger reward option.  Summary and Conclusions The goal of this chapter was to integrate the findings from the previous four chapters into the broader literature on the functions of this mesocorticolimbic circuit. In doing so, I have shown that medial areas of the PFC across multiple species are intricately involved in monitoring decision making behaviour and adjusting behaviour in responses to changes in the environment . In addition, the PFC exerts top-down executive regulation of other fundamental systems involved in reward seeking. Finally, the mesocorticolimbic DA system may provide a key signal to the PFC about changes in food reward over time that is required to facilitate modifications in choice behaviour. One of the key strengths of the approach in this thesis is the focus on neural circuits, rather than limiting the examination to the function of an individual brain region. Our brain is an enormously complex system that simultaneously receives and processes information for behavioural output from all different sensory systems. While it is important to understand the  144  function of individual regions or neurochemical systems, a comprehensive understanding of behaviour entails examining how these systems to interact, which can look very different than those functions observed in isolation. Previous research on risk-based decision making in human and non-human animals has produced a long list of neural substrates that form potential candidates for the neural circuitry underlying this form of decision making. This dissertation not only refined the understanding of the specific components involved, but also how these systems interact with each other in order to produce complex behaviours. The use of a rodent model of decision making restricts the generalization of these findings to decision making situations involving choices among certain vs. probabilistic gains, where risk is defined as the potential for not obtaining reward. How the brain processes explicit monetary losses or potential punishments (e.g. risk of death by predator) is still unclear and research in this area will provide crucial information about how different organisms make decisions in more complex situations (Estle et al., 2006; Simon et al., 2009). Future research into choices involving other forms of rewards (e.g. access to mating, cars, jobs) will also help determine whether the same or different neural circuits are recruited for processing different forms of decision making. Although the current findings support numerous other reports of similarities between human and non-human animals in both the pattern of choice behaviour and the neural circuitry activated during probabilistic decision making, it remains possible that different species arrive at similar patterns of choice through different neural mechanisms. For example, some brain regions, such as the ventral striatum (Pennartz et al., 1994; Mezey and Csillag, 2002; Izawa et al., 2003) may be more conserved than others, such as the PFC. Therefore, whether this precise neural circuit identified in rats is entirely consistent with that mediating decision making in humans remains to be confirmed. However, it is exactly these inconsistencies between species that can be  145  highly informative and shed light on the function of the underlying systems regulating decision making in different contexts. For example, although some fundamental systems may regulate basic valuation processes, different regions of the PFC may be recruited depending on what type of information the organism is dealing with (e.g. whether a predator is involved). Therefore, it is imperative that comparative cross-species and interdisciplinary research in decision making continues so that a more comprehensive understanding of choice behaviour is achieved. Understanding both the cognitive processes and neural circuitry that mediate risk-based decision making has important implications for multiple clinical phenomena. Firstly, deciphering the specific processes governed by different brain regions (e.g. PFC, amygdala) not only explains, but can also help predict, the cognitive symptoms experienced by individuals with traumatic brain injury to these areas. In that regard, treatments that are specific to the targeted areas should have a much better success rate than more generalized treatments. Moreover, addiction to drugs of abuse, money, and food are characterized, in part, by altered subjective valuation of rewards (Reynolds, 2006; Madden et al., 2009). A better understanding of the variables that influence preferences for certain vs. probabilistic rewards (e.g. experienced luck), as well as the neural substrates associated with these variables, may contribute to a more comprehensive understanding of the associated problem behaviors. In addition, the current findings are likely to pave the way for more research on the cellular underpinnings of these processes and how they are altered by drugs (e.g. those that affect DA activity), as well as identify the plausibility of non-invasive treatments for decision making impairments, such as transcranial magnetic stimulation.  146  REFERENCES Adriani W, Boyer F, Gioiosa L, Macri S, Dreyer JL, Laviola G (2009). Increased impulsive behavior and risk proneness following lentivirus-mediated dopamine transporter overexpression in rats’ nucleus accumbens. Neuroscience 159:47-58. Adriani W, Laviola G (2006) Delay aversion but preference for large and rare rewards in two choice: implications for the measurement of self-control parameters. BMC Neurosci 7:52. Ahn S, Phillips AG (1999) Dopaminergic correlates of sensory-specific satiety in the medial prefrontal cortex and nucleus accumbens of the rat. J Neurosci 19: RC29. Ainslie GW (1974) Impulse control in pigeons. J Exp Anal Behav 21: 485-489. Akitsuki Y, Sugiura M, Watanabe J, Yamashita K, Sassa Y, Awata S, Matsuoka H, Maeda Y, Matsue Y, Fukuda H, Kawashima R (2003) Context-dependent cortical activation in response to financial reward and penalty: An event-related fMRI study. Neuroimage 19:1674-1685. Ambroggi F, Ishikawa A, Fields HL, Nicola SM (2008) Basolateral amygdala neurons facilitate reward-seeking behavior by exciting nucleus accumbens neurons. Neuron 59: 648-661. Arana FS, Parkinson JA, Hinton E, Holland AJ, Owen AM, Roberts AC (2003) Dissociable contributions of the human amygdala and orbitofrontal cortex to incentive motivation and goal selection. J Neurosci 23: 9632-9638. Aston-Jones G, Cohen JD (2005) An integrative theory of locus coeruleus-norepinephrine function: adaptive gain and optimal performance. Annu Rev Neurosci 28: 403-450. Ballard K, Knutson B (2009) Dissociable neural representations of future reward magnitude and delay during temporal discounting. Neuroimage 45:143-150.  147  Balleine BW, Dickinson A (1998) Goal-directed instrumental action: Contingency and incentive learning and their cortical substrates. Neuropharmacology 37:407-419. Bardgett ME, Depenbrock M, Downs N, Points M, Green L (2009) Dopamine modulates effortbased decision-making in rats. Behav Neurosci 123: 242-251. Bar-On R, Tranel D, Denburg NL, Bechara A (2003) Exploring the neurological substrate of emotional and social intelligence. Brain 126:1790-1800. Bassareo V, Di Chiara G (1997) Differential influence of associative and nonassociative learning mechanisms on the responsiveness of prefrontal and accumbal dopamine transmission to food stimuli in rats fed ad libitum. J Neurosci 17: 851–861. Bechara A, Damasio AR, Damasio H, Anderson SW (1994) Insensitivity to future consequences following damage to human prefrontal cortex. Cognition 50:7-15. Bechara A, Damasio H, Tranel D, Anderson SW (1998) Dissociation of working memory from decision making within the human prefrontal cortex. J Neurosci 18:428-437. Bechara A, Damasio H, Damasio AR, Lee GP (1999) Different contributions of the human amygdala and ventromedial prefrontal cortex to decision-making. J Neurosci 19:54735481. Beckstead RM, Domesick VB, Nauta WJ (1979) Efferent connections of the substantia nigra and ventral tegmental area in the rat. Brain Res 175:191-217. Behrens TE, Woolrich MW, Walton ME, Rushworth MF (2007) Learning the value of information in an uncertain world. Nat Neurosci 10:1214-1221. Belova MA, Paton JJ, Morrison SE, Salzman CD (2007) Expectation modulates neural responses to pleasant and aversive stimuli in primate amygdala. Neuron 55:970-984.  148  Belova MA, Paton JJ, Salzman CD (2008) Moment-to-moment tracking of state value in the amygdala. J Neurosci 28:10023-10030. Bernoulli D (1954) Exposition of a new theory on the measurement of risk. Econometrica 22: 23– 36. Berretta S, Pantazopoulos H, Caldera M, Pantazopoulos P, Pare D (2005) Infralimbic cortex activation increases c-Fos expression in intercalated neurons of the amygdala. Neuroscience 132:943-953. Bezzina G, Body S, Cheung TH, Hampson CL, Bradshaw CM, Szabadi E, Anderson IM, Deakin JF (2008) Effect of disconnecting the orbital prefrontal cortex from the nucleus accumbens core on inter-temporal choice behaviour: a quantitative analysis. Behav Brain Res 191:272-279. Bickel WK, Odum AL, Madden GJ (1999) Impulsivity and cigarette smoking: Delay discounting in current, never, and ex-smokers. Psychopharmacology 146: 447-454. Birrell JM and Brown VJ (2000) Medial frontal cortex mediates perceptual attentional set shifting in the rat. J Neurosci 20:4320-4324. Blair K, Marsh AA, Morton J, Vythilingam M, Jones M, Mondillo K, Pine DC, Drevets WC, Blair JR (2006) Choosing the lesser of two evils, the better of two goods: Specifying the roles of ventromedial prefrontal cortex and dorsal anterior cingulate in object choice. J Neurosci 26:11379-11386. Block AE, Dhanji H, Thompson-Tardif SF, Floresco SB (2007) Thalamic-prefrontal corticalventral striatal circuitry mediates dissociable components of strategy set shifting. Cereb Cortex 17:1625-1636.  149  Botvinick MM, Braver TS, Barch DM, Carter CS, Cohen JD (2001) Conflict monitoring and cognitive control. Psychol Rev 108: 624-652. Brand M, Kalbe E, Kracht LW, Riebel U, Munch J, Kessler J, Markowitsch HJ (2004) Organic and psychogenic factors leading to executive dysfunctions in a patient suffering from surgery of a colloid cyst of the foramen of monro. Neurocase 10:420-425. Brand M, Grabenhorst F, Starcke K, Vandekerckhove MM, Markowitsch HJ (2007) Role of the amygdala in decisions under ambiguity and decisions under risk: evidence from patients with Urbach-Wiethe disease. Neuropsychologia 45:1305-1317. Breiter HC, Etcoff NL, Whalen PJ, Kennedy WA, Rauch SL, Ruckner RL, Strauss MM, Hyman SE, Rosen BR (1996) Response and habituation of the human amygdala during visual processing of facial expression. Neuron 17:875-887. Brog JS, Salyapongse A, Deutch A, Zahm DS (1993) The patterns of afferent innervation of the core and shell in the “accumbens” part of the rat ventral striatum: immunohistochemical detection of retrogradely transported fluoro-gold. J Comp Neurol 338: 255-278. Brown VJ, Bowman EM (2002) Rodent models of prefrontal cortical function. Trends Neurosci 25:340-343. Brown JW, Braver TS (2005) Learned predictions of error likelihood in the anterior cingulate cortex. Science 307:1118-1121. Bush G, Vogt BA, Holmes J, Dale AM, Greve D, Jenike MA, Rosen BR (2002) Dorsal anterior cingulate cortex: A role in reward-based decision making. Proc Natl Acad Sci U S A. 99:523-528.  150  Bussey TJ Everitt BJ, Robbins TW (1997) Dissociable effects of cingulate and medial frontal cortex lesions on stimulus-reward learning using a novel Pavlovian autoshaping procedure for the rat: Implications for the neurobiology of emotion. Behav Neurosci 111:908-919. Callaway CW, Hakan RL, Henriksen SJ (1991) Distribution of amygdala input to the nucleus accumbens septi: an electrophysiological investigation. J Neural Trans Gen Sect 83: 215225. Cardinal RN, Robbins TW, Everitt BJ (2000) The effects of d-amphetamine, chlordiazepoxide, alpha-flupenthixol and behavioural manipulations on choice of signalled and unsignalled delayed reinforcement in rats. Psychopharmacology (Berl) 152:362-375. Cardinal RN, Pennicott DR, Sugathapala CL, Robbins TW, Everitt BJ (2001) Impulsive choice induced in rats by lesions of the nucleus accumbens core. Science 292:2499-2501. Cardinal RN, Howes NJ (2005) Effects of lesions of the nucleus accumbens core on choice between small certain rewards and large uncertain rewards in rats. BMC Neurosci 6:9. Catania AC. Reinforcement schedules and psychophysical judgment: a study of some temporal properties of behavior In: W.N. Schoenfeld, Editors, The theory of reinforcement schedules, Appleton (Century/Crofts), New York (1970), pp. 1–42. Cenci MA, Kalen P, Mandel RJ, Bjorklund A (1992) Regional differences in the regulation of dopamine and noradrenaline release in medial frontal cortex, nucleus accumbens and caudate-putamen: a microdialysis study in the rat. Brain Res 581:217-228. Chen MK, Lakshminarayanan V, Santos LR (2006) How basic are behavioral biases? Evidence from capuchin monkey trading behavior. Journal of Political Economy 114:517-537.  151  Christopoulos GI, Tobler PN, Bossaerts P, Dolan RJ, Schultz W (2009) Neural correlates of value, risk, and risk aversion contributing to decision making under risk. J Neurosci 29:1257412583. Chudasama Y, Robbins TW (2004) Dopaminergic modulation of visual attention and working memory in the rodent prefrontal cortex. Neuropsychopharmacology 29:1628-1636. Clark L, Manes F, Antoun N, Sahakian BJ, Robbins TW (2003) The contributions of lesion laterality and lesion volume to decision-making impairment following frontal lobe damage. Neuropsychologia 41:1474-1483. Clark L, Cools R, Robbins TW (2004) The neuropsychology of ventral prefrontal cortex: Decision-making and reversal learning. Brain Cogn 55:41-53. Clark L, Bechara A, Damasio H, Aitken MR, Sahakian BJ, Robbins TW (2008) Differential effects of insular and ventromedial prefrontal cortex lesions on risky decision-making. Brain 131:1311-1322. Cohen MX, Ranganath C (2005) Behavioral and neural predictors of upcoming decisions. Cogn Affect Behav Neurosci 5: 117-126. Cohen MX, Elger CE, Weber B (2008) Amygdala tractography predicts functional connectivity and learning during feedback-guided decision-making. Neuroimage 39:1396-1407. Conde F, Audinat E, Maire-Lepoivre E, Crepel F (1990) Afferent connections of the medial frontal cortex of the rat. A study using retrograde transport of fluorescent dyes. I. thalamic afferents. Brain Res Bull 24:341-354. Corrigall WA, Coen KM, Zhang J, Adamson KL (2001) GABA mechanisms in the pedunculopontine tegmental nucleus influence particular aspects of nicotine selfadministration selectively in the rat. Psychopharmacology 158:190-197.  152  Courville AC, Daw ND, Touretzky DS (2006) Bayesian theories of conditioning in a changing world. Trends Cogn Sci 10:294-300. Cousins MS, Wei W, Salamone JD (1994) Pharmacological characterization of performance on a concurrent lever pressing/feeding choice procedure: effects of dopamine antagonist, cholinomimetic, sedative and stimulant drugs. Psychopharmacology 116:529-537. Critchley HD, Mathias CJ, Dolan RJ (2001) Neural activity in the human brain relating to uncertainty and arousal during anticipation. Neuron 29:537-545. Damasio AR (1994) Descartes’ error: emotion, reason and the human brain. New York: Putnam. Daw ND, O’Doherty JP, Dayan P, Seymour B, Dolan RJ (2006) Cortical substrates for exploratory decisions in humans. Nature 441: 876-879. Day JJ, Jones JL, Wightman RM, Carelli RM (2010) Phasic nucleus accumbens dopamine release encodes effort- and delay-related costs. Biol Psychiatry 68:306-309. Dayan P, Kakade S, Montague PR (2000) Learning and selective attention. Nat Neurosci 3 Suppl: 1218-1223. De Martino B, Camerer CF, Adolphs R (2010) Amygdala damage eliminates monetary loss aversion. Proc Natl Acad USA 107:3788-3792. Delgado MR, Nystrom LE, Fissell C, Noll DC, Fiez JA (2000) Tracking the hemodynamic responses to reward and punishment in the striatum. J Neurophysiol 84:3072-3077. Delgado MR, Nearing KI, Ledoux JE, Phelps EA (2008) Neural circuitry underlying the regulation of conditioned fear and its relation to extinction. Neuron 59:829-838. Denk F, Walton ME, Jennings KA, Sharp T, Rushworth MF, Bannerman DM (2005) Differential involvement of serotonin and dopamine systems in cost-benefit decisions about delay or effort. Psychopharmacology (Berl) 179:587-596.  153  Di Pietro NC, Hyman JM, Caracheo F, Floresco SB, Seamans JK (2010) Network representations of risky decision making in the medial prefrontal cortex are adaptive and vary base don the relative degree of risk. Soc Neurosci Abstr 805.4 Dickinson A, Mackintosh NJ (1978) Classical conditioning in animals. Annu Rev Psychol 29:587-612. Dietrich A, Allen JD (1998) Functional dissociation of the prefrontal cortex and the hippocampus in timing behavior. Behav Neurosci 112:1043-1047. Dilgen JE, O'Donnell P (2006) Basolateral amygdala projections to the medial prefrontal cortex: an inhibitory pathway? Soc Neurosci Abstr 32:730–7. Duncan J (2001) An adaptive coding model of neural function in prefrontal cortex. Nat Rev Neurosci 2:820-829. Durstewitz D, Seamans JK, Sejnowski TJ (2000) Neurocomputational models of working memory. Nat Neurosci Suppl: 1184-1191. Elliott R, Dolan RJ (1998) Activation of different anterior cingulate foci in association with hypothesis testing and response selection. Neuroimage 8:17-29. Elston GN (2000) Pyramidal cells of the frontal lobe: all the more spinous to think with. J Neurosci 20:RC95. Ernst M, Bolla K, Mouratidis M, Contoreggi C, Matochik JA, Kurian V, Cadet JL, Kimes AS, London ED (2002) Decision-making in a risk-taking task: A PET study. Neuropsychopharmacology 26:682-691. Ernst M, Nelson EE, McClure EB, Monk CS, Munson S, Eshel N, Zarahn E, Leibenluft E, Zametkin A, Towbin K, et al. (2004) Choice selection and reward anticipation: An fMRI study. Neuropsychologia 42:1585-1597.  154  Ernst M, Nelson EE, Jazbec S, McClure EB, Monk CS, Leibenluft E, Blair J, Pine DS (2005) Amygdala and nucleus accumbens in responses to receipt and omission of gains in adults and adolescents. Neuroimage 25:1279-1291. Eslinger PJ, Damasio AR (1985) Severe disturbance of higher cognition after bilateral frontal lobe ablation: patient EVR. Neurology 35:1731-1741. Estle SJ, Green L, Myerson J, Holt DD (2006) Differential effects of amount on temporal and probability discounting of gains and losses. Mem Cognit 34:914-928. Estle SJ, Green L, Myerson J, Holt DD (2007) Discounting of monetary and directly consumable rewards. Psychol Sci. 18:58-63. Etkin A, Egner T, Peraza DM, Kandel ER, Hirsch J (2006) Resolving emotional conflict: a role for the rostral anterior cingulate cortex in modulating activity in the amygdala. Neuron 51, 871-882. Evenden JL, Ryan CN (1996). The pharmacology of impulsive behaviour in rats: the effects of drugs on response choice with varying delays of reinforcement. Psychopharmacology 128:161-170. Everitt BJ, Parkinson JA, Olmstead MC, Arroyo M, Robledo P, Robbins TW (1999) Associative processes in addiction and reward. The role of amygdala-ventral striatal subsystems. Ann NY Acad Sci 29: 412-438. Fecteau S, Knoch D, Fregni F, Sultani N, Boggio P, Pascual-Leone A (2007) Diminishing risktaking behavior by modulating activity in the prefrontal cortex: a direct current stimulation study. J Neurosci 27:12500-12505.  155  Feenstra MG, Botterblom MH (1996) Rapid sampling of extracellular dopamine in the rat prefrontal cortex during food consumption, handling, and exposure to novelty. Brain Res 742:17-24. Fellows LK, Farah MJ (2005) Different underlying impairments in decision-making following ventromedial and dorsolateral frontal lobe damage in humans. Cereb Cortex 15:58-63. Fiorillo CD, Tobler PN, Schultz W (2003) Discrete coding of reward probability and uncertainty by dopamine neurons. Science 299:1898-1902. Figner B, Knoch D, Johnson EJ, Krosch AR, Lisanby SH, Fehr E, Weber EU (2010) Lateral prefrontal cortex and self-control in intertemporal choice. Nat Neurosci 13:538-539. Floresco SB (2007) Dopaminergic regulation of limbic-striatal interplay. J Psychiatry Neurosci 32: 400-411. Floresco SB, Magyar O (2006) Mesocortical dopamine modulation of executive functions: beyond working memory. Psychopharmacology 188:567-585. Floresco SB and Ghods-Sharifi S (2007) Amygdala-prefrontal cortical circuitry regulates effortbased decision making. Cereb Cortex 17:251-260. Floresco SB, Braaksma DN, Phillips AG (1999) Thalamic-cortical-striatal circuitry subserves working memory during delayed responding on a radial arm maze. J Neurosci 19:1106111071. Floresco SB, Blaha CD, Yang CR, Phillips AG (2001a) Dopamine D1 and NMDA receptors mediate potentiation of basolateral amygdala-evoked firing of nucleus accumbens neurons. J Neurosci 21:6370-6376.  156  Floresco SB, Blaha CD, Yang CR, Phillips AG (2001b). Modulation of hippocampal and amygdalar-evoked activity of nucleus accumbens neurons by dopamine: cellular mechanisms of input selection. J Neurosci 21:2851-2860. Floresco SB, Magyar O, Ghods-Sharifi S, Vexelman C, Tse MT (2006) Multiple dopamine receptor subtypes in the medial prefrontal cortex of the rat regulate set-shifting. Neuropsychopharmacology 31:297-309. Floresco SB, Tse MT, Ghods-Sharifi S (2008a) Dopaminergic and glutamatergic regulation of effort- and delay-based decision making. Neuropsychopharmacology 33:1966-1979. Floresco SB, St.Onge JR, Ghods-Sharifi S, and Winstanley CA (2008b) Cortico-limbic-striatal circuits subserving different forms of cost-benefit decision making. Cogn Affect Behav Neurosci 8:375-389. Floresco SB, Block AE, Tse MT (2008c) Inactivation of the medial prefrontal cortex of the rat impairs strategy set-shifting, but not reversal learning, using a novel, automated procedure. Behav Brain Res 190:85-96. Frank MJ, Doll BB, Oas-Terpstra J, Moreno F (2009) Prefrontal and striatal dopaminergic genes predict individual differences in exploration and exploitation. Nat Neurosci 12:1062-1068. Friedman M, Savage LJ (1948) The utility analysis of choices involving risk. J Polit Econ 56: 279–304. Freedman LJ, Insel TR, Smith Y (2000) Subcortical projections of area 25 (subgenual cortex) of the macaque monkey. J Comp Neurol 421:172-188. French SJ, Totterdell S (2003) Individual nucleus accumbens-projection neurons receive both basolateral amygdala and ventral subicular afferents in rats. Neuroscience 119: 19-31.  157  Fukui H, Murai T, Fukuyama H, Hayashi T, Hanakawa T (2005) Functional activity related to risk anticipation during performance of the iowa gambling task. Neuroimage 24:253-259. Fuster JM (2000) Executive frontal functions. Exp Brain Res 133:66-70. Gallistel CR, Gibbon J (2000) Time, rate, and conditioning. Psychol Rev 107:289-344. Gan JO, Walton ME, Phillips PE (2010) Dissociable cost and benefit encoding of future rewards by mesolimbic dopamine. Nat Neurosci 13:25-27. Ghods-Sharifi S, St. Onge JR, Floresco SB (2009) Fundamental contribution by the basolateral amygdala to different forms of decision making. J Neurosci 29:5251-5259. Glimcher PW (2004) Decisions, Uncertainty and the Brain: The Science of Neuroeconomics. Cambridge, MA: MIT Press. Goldman-Rakic PS, Leranth C, Williams SM, Mons N, Geffard M (1989) Dopamine synaptic complex with pyramidal neurons in primate cerebral cortex. Proc Natl Acad Sci USA 86:9015-9019. Gorelova N, Seamans JK, Yang CR (2002) Mechanisms of dopamine activation of fast-spiking interneurons that exert inhibition in rat prefrontal cortex. J Neurophysiol 88:3150-3166. Goto Y, O'Donnell P (2002) Timing-dependent limbic-motor synaptic integration in the nucleus accumbens. Proc Natl Acad Sci 99: 13189-13193. Granon S, Passetti F, Thomas KL, Dalley JW, Everitt BJ, Robbins TW (2000) Enhanced and impaired attentional performance after infusion of D1 dopaminergic receptor agents into rat prefrontal cortex. J Neurosci 20:1208-1215. Green L, Myerson J (2004) A discounting framework for choice with delayed and probabilistic rewards. Psychol Bull 130:769-792.  158  Green L, Myerson J, Ostaszewski P (1999) Amount of reward has opposite effects on the discounting of delayed and probabilistic outcomes. J Exp Psychol Learn Mem Cogn 25:418-427. Groenewegen HJ, Berendse HW, Wolters JG, Lohman AH (1990) The anatomical relationship of the prefrontal cortex with the striatopallidal system, the thalamus, and the amygdala: evidence for a parallel organization. Prog Brain Res 85:95-116. Haluk DM, Floresco SB (2009) Ventral striatal dopamine modulation of different forms of behavioral flexibility. Neuropsychopharmacology 34:2041-2052. Hampton AN, Adolphs R, Tyszka MJ, O’Doherty JP (2007) Contributions of the amygdala to reward expectancy and choice signals in human prefrontal cortex. Neuron 55:545-555. Hare TA, O’Doherty J, Camerer CF, Schultz W, Rangel A (2008) Dissociating the role of the orbitofrontal cortex and the striatum in the computation of goal values and prediction errors. J Neurosci 28:5623-5630. Hare TA, Camerer CF, Rangel A (2009) Self-control in decision-making involves modulation of the vmPFC valuation system. Science 324:646-648. Hayden BY, Pearson JM, Platt ML (2011a) Neuronal basis of sequential foraging decisions in a patchy environment. Nat Neurosci 14:933-939. Hayden BY, Heilbronner SR, Pearson JM, Platt ML (2011b) Surprise signals in anterior cingulate cortex: neuronal encoding of unsigned reward prediction errors driving adjustment in behavior. J Neurosci 31:4178-4187. Haynes L (2009) Delaying Gratification. London: Parliamentary Office of Science and Technology of the United Kingdom, 1-4.  159  Heidbreder CA, Groenewegen HJ (2003) The medial prefrontal cortex in the rat: Evidence for a dorso-ventral distinction based upon functional and anatomical characteristics. Neurosci Biobehav Rev 27:555-579. Herrnstein RJ (1997) The matching law: papers in psychology and economics. Cambridge, MA: Harvard University Press. Ho MY, Mobini S, Chiang TJ, Bradshaw CM, Szabadi E (1999) Theory and method in the quantitative analysis of “impulsive choice” behaviour: implications for psychopharmacology. Psychopharmacology (Berl) 146:362-372. Holroyd CB, Coles MG (2002) The neural basis of human error processing: reinforcement learning, dopamine and the error-related negativity. Psychol Rev 109:679-709. Holt DD, Green L, Myerson J (2003) Is discounting impulsive? Evidence from temporal and probability discounting in gambling and non-gambling college students. Behav Processes 64:355-367. Huettel SA, Song AW, McCarthy G (2005) Decisions under uncertainty: probabilistic context influences activation of prefrontal and parietal cortices. J Neurosci 25:3304-3311. Huettel SA, Stowe CJ, Gordon EM, Warner BT, Platt ML (2006) Neural signatures of economic preferences for risk and ambiguity. Neuron 49:765-775. Hutton SB, Murphy FC, Joyce EM, Rogers RD, Cuthbert I, Barnes TR, McKenna PJ, Sahakian BJ, Robbins TW (2002) Decision making deficits in patients with first-episode and chronic schizophrenia. Schizophr Res 55:249-257. Isles AR, Humby T, Wilkinson LS (2003) Measuring impulsivity in mice using a novel operant delayed reinforcement task: effects of behavioural manipulations and d-amphetamine. Psychopharmacology (Berl) 170:376-382.  160  Izawa E, Zachar G, Yanagihara S, Matsushima T (2003) Localized lesion of caudal part of lobus parolfactorius caused impulsive choice in the domestic chick: evolutionary conserved function of ventral striatum. J Neurosci 23:1894-1902. Jenison RL, Rangel A, Oya H, Kawasaki H, Howard MA (2011) Value encoding in single neurons in the human amygdala during decision making. J Neurosci 31:331-338. Jocham G, Klein TA, Ullsperger M (2011) Dopamine-mediated reinforcement learning signals in the striatum and ventromedial prefrontal cortex underlie value-based choices. J Neurosci 31:1606-1613. Jones JL, Day JJ, Aragona BJ, Wheeler RA, Wightman RM, Carelli RM (2010) Basolateral amygdala modulates terminal dopamine release in the nucleus accumbens and conditioned responding. Biol Psychiatry 67:737-744. Kable JW, Glimcher PW (2007) The neural correlates of subjective value during intertemporal choice. Nat Neurosci 10:1625-1633. Kahneman D, Tversky A (1979) Prospect theory: An analysis of decision under risk. Econometrica 47: 263-292. Kahneman and Tversky (2000) Choices, Values, and Frames. New York: Cambridge University Press. Kacelnik A, Bateson M (1996) Risky theories – the effects of variance on foraging decisions. American Zoologist 36:402-434. Kakade S, Dayan P (2002) Acquisition and extinction in autoshaping. Psychol Rev 109:533-544. Kalenscher T, Pennartz CMA (2008) Is a bird in the hand worth two in the future? The neuroeconomics of intertemporal decision-making. Progress in Neurobiology 84:284-315.  161  Kaminski BJ, Ator NA (2001) Behavioral and pharmacological variables affecting risky choice in rats. J Exp Anal Behav 75:275-297. Kennerley SW, Walton ME, Behrens TE, Buckley MJ, Rushworth MF (2006) Optimal decision making and the anterior cingulate cortex. Nat Neurosci 9: 940–947. Kheramin S, Body S, Mobini S, Ho MY, Velazquez-Martinez DN, Bradshaw CM, Szabadi E, Deakin JF, Anderson IM (2002) Psychopharmacology (Berl) 165:9-17. Kheramin S, Body S, Ho MY, Velazquez-Martinez DN, Bradshaw CM, Szabadi E, Deakin JF, Anderson IM (2003) Role of the orbital prefrontal cortex in choice between delayed and uncertain reinforcers: a quantitative analysis. Behav Processes 64:239-250. Kheramin S, Body S, Ho MY, Velazquez-Martinez DN, Bradshaw CM, Szabadi E, Deakin JF, Anderson IM (2004) Effects of orbital prefrontal dopamine depletion on inter-temporal choice: a quantitative analysis. Psychopharmacology (Berl) 175:206-214. Killcross S, Coutureau E (2003) Coordination of actions and habits in the medial prefrontal cortex of rats. Cereb Cortex 13:400-408. Kim S, Hwang J, Lee D (2008) Prefrontal coding of temporally discounted values during intertemporal choice. Neuron 59: 161-172. Kinzler KD, Spelke ES (2007) Core systems in human cognition. Prog Brain Res 164:257-264. Knoch D, Gianotti LR, Pascual-Leone A, Treyer V, Regard M, Hohmann M, Brugger P (2006) Disruption of right prefrontal cortex by low-frequency repetitive transcranial magnetic stimulation induces risk-taking behavior. J Neurosci 26:6469-6472. Knutson B, Cooper JC (2005) Functional magnetic resonance imaging of reward prediction. Curr Opin Neurol 18:411-417.  162  Knutson B, Bossaerts P (2007) Neural antecedents of financial decisions. J Neurosci 27:81748177. Knutson B, Adams CM, Fong GW, Hommer D (2001) Anticipation of increasing monetary reward selectively recruits nucleus accumbens. J Neurosci 21:RC159. Knutson B, Fong GW, Bennett SM, Adams CM, Hommer D (2003) A region of mesial prefrontal cortex tracks monetarily rewarding outcomes: characterization with rapid event-related fMRI. Neuroimage 18:263-272. Knutson B, Taylor J, Kaufman M, Peterson R, Glover G (2005) Distributed neural representation of expected value. J Neurosci 25:4806-4812. Krettek JE, Price JL (1977) Projections from the amygdaloid complex to the cerebral cortex and thalamus in the rat and cat. J Comp Neurol 172: 687-722. Kuhnen CM, Knutson B (2005) The neural basis of financial risk taking. Neuron 47:763-770. Labudda K, Woermann FG, Mertens M, Pohlmann-Eden B, Markowitsch HJ, Brand M (2008) Neural correlates of decision making with explicit information about probabilities and incentives in elderly healthy subjects. Exp Brain Res 187:641-650. Lachowicz JE, Sibley DR (1997) Molecular characteristics of mammalian dopamine receptors. Pharmacol Toxicol 81:105-113. Lapish CC, Durstewitz D, Chandler LJ, Seamans JK (2008) Successful choice behavior is associated with distinct and coherent network states in anterior cingulate cortex. Proc Natl Acad Sci USA 105:11963-11968, Lawrence NS, Jollant F, O’Daly O, Zelaya F, Phillips ML (2009) Distinct roles of prefrontal cortical subregions in the Iowa Gambling Task. Cerebral Cortex 19:1134-1143.  163  Levy H, Markowitz HM (1979) Approximating expected utility by a function of mean and variance. American Economic Review 69:308-317. Likhtik E, Pelletier JG, Paz R, Pare D (2005) Prefrontal control of the amygdala. J Neurosci 25:7429-7437. Loos M, Pattij T, Janssen MC, Counotte DS, Schoffelmeer AN, Smit AB, Spijker S, van Gaalen MM (2010) Dopamine receptor D1/D5 gene expression in the medial prefrontal cortex predicts impulsive choice in rats. Cereb Cortex 20:1064-1070. Louie L, Glimcher PW (2010) Separating value from choice: delay discounting activity in the lateral intraparietal area. J Neurosci 30:5498-5507. Luce R (2000) Utility of gains and losses: measurement-theoretical and experimental approaches. Mahwah, NJ: Erlbaum. Madden GJ, Bickel WK, Jacobs EA (1999) Discounting of delayed rewards in opioid-dependent outpatients: Exponential or hyperbolic discounting functions? Experimental and Clinical Psychopharmacology 7: 284-293. Manes F, Sahakian B, Clark L, Rogers R, Antoun N, Aitken M, Robbins T (2002) Decisionmaking processes following damage to the prefrontal cortex. Brain 125:624-639. Markowitz HM (1952) Portfolio selection. Journal of Finance 7:77-91. Marsh B, Kacelnik A (2002) Framing effects and risky decisions in starlings. Proc Natl Acad Sci USA 99:3352-3355. Marsh AA, Blair KS, Vythilingam M, Busis S, Blair RJR (2007) Response options and expectations of reward in decision-making: The differential roles of dorsal and rostral anterior cingulate cortex. Neuroimage 35:979-988.  164  Mavaddat N, Kirkpatrick PJ, Rogers RD, Sahakian BJ (2000) Deficits in decision-making in patients with aneurysms of the anterior communicating artery. Brain 123:2109-2117. Mayberg HS, Silva JA, Brannan SK, Tekell JL, Mahurin RK, McGinnis S, Jerabek PA (2002) The functional neuroanatomy of the placebo effect. Am J Psychiatry 159:728-737. Maynard Smith J (1982) Evolution and the Theory of Games. Cambridge, MA: University Press. Mazur JE (1989) Theories of probabilistic reinforcement. J Exp Anal Behav 51:87-99. McClure SM, Laibson DI, Loewenstein G, Cohen JD (2004) Separate neural systems value immediate and delayed monetary rewards. Science 306:503-507. McClure SM, Ericson KM, Laibson DI, Loewenstein G, Cohen JD (2007) Time discounting for primary rewards. J Neurosci 27:5796-5804. McDonald AJ (1987) Organization of amygdaloid projections to the mediodorsal thalamus and prefrontal cortex: A fluorescence retrograde transport study in the rat. J Comp Neurol 262: 46-58. McDonald AJ (1991a) Organization of amygdaloid projections to the prefrontal cortex and associated striatum in the rat. Neuroscience 44:1-14. McDonald AJ (1991b) Topographical organization of amygdaloid projections to the caudatoputamen, nucleus accumbens, and related striatal-like areas of the brain. Neuroscience 44:15-33. McDonald AJ, Mascagni F, Guo L (1996) Projections of the medial and lateral prefrontal cortices to the amygdala: a Phaseolus vulgaris leucoagglutinin study in the rat. Neuroscience 71:55-75.  165  McLean A, Rubinsztein JS, Robbins TW, Sahakian BJ (2004) The effects of tyrosine depletion in normal healthy volunteers: implications for unipolar depression. Psychopharmacology 171:286-297. McNeil BJ, Pauker SG, Sox HC, Tversky A (1982) On the elicitation of preferences for alternative therapies. New England Journal of Medicine 306: 1259–1262. Mezey S, Csillag A (2002) Selective striatal connections of midbrain dopaminergic nuclei in the chick (Gallus domesticus). Cell Tissue Res 308:35-46. Milad MR, Rauch SL (2007) The role of the orbitofrontal cortex in anxiety disorders. Ann NY Acad Sci 1121:546-561. Mobini S, Chiang TJ, Ho MY, Bradshaw CM, Szabadi E (2000) Effects of central 5hydroxytryptamine depletion on sensitivity to delayed and probabilistic reinforcement. Psychopharmacology (Berl) 152: 390-397. Mobini S, Body S, Ho MY, Bradshaw CM, Szabadi E, Deakin JF, Anderson IM (2002) Effects of lesions of the orbitofrontal cortex on sensitivity to delayed and probabilistic reinforcement. Psychopharmacology (Berl) 160:290-298. Mogenson GJ, Jones DL, Yim CY (1980) From motivation to action: functional interface between the limbic system and the motor system. Prog Neurobiol 14:69-97. Mulder AB, Hodenpijl MG, Lopes da Silva FH (1998) Electrophysiology of the hippocampal and amygdaloid projections to the nucleus accumbens of the rat: convergence, segregation, and interaction of inputs. J Neurosci 18: 5095-5102. Nicola SM (2007) The nucleus accumbens as a part of a basal ganglia action selection circuit. Psychopharmacology 191: 521-550.  166  Ochsner KN, Ray RD, Cooper JC, Robertson ER, Chopra S, Gabrieli JD, Gross JJ (2005) For better or for worse: neural systems supporting the cognitive down- and up-regulation of negative emotion. Neuroimage 23:483-499. O’Doherty J, Rolls ET, Francis S, Bowtell R, McGlone F (2001) Representation of pleasant and aversive taste in the human brain. J Neurophysiol 85:1315-1321. O'Doherty J, Critchley H, Deichmann R, Dolan RJ (2003) Dissociating valence of outcome from behavioral control in human orbital and ventral prefrontal cortices. J Neurosci 23:79317939. O'Donnell P, Grace AA (1995) Synaptic interactions among excitatory afferents to nucleus accumbens neurons: hippocampal gating of prefrontal cortical input. J Neurosci 15: 36223639. Orozco-Cabal L, Pollandt S, Liu J, Vergara L, Shinnick-Gallagher P, Gallagher JP (2006) A novel rat medial prefrontal cortical slice preparation to investigate synaptic transmission from amygdala to layer V prelimbic pyramidal neurons. J Neurosci Methods 151: 148-158. Owen AM, Downes JJ, Sahakian BJ, Polkey CE, Robbins TW (1990) Planning and spatial working memory following frontal lobe lesions in man. Neuropsychologia 28:1021-1034. Padoa-Schioppa C, Assad JA (2006) Neurons in the orbitofrontal cortex encode economic value. Nature 441:223-226. Pais-Vieira M, Lima D, Galhardo V (2007) Orbitofrontal cortex lesions disrupt risk assessment in a novel serial decision-making task for rats. Neuroscience 145:225-231. Pagonabarraga J, García-Sánchez C, Llebaria G, Pascual-Sedano B, Gironell A, Kulisevsky J (2007) Controlled study of decision-making and cognitive impairment in Parkinson's disease. Mov Disord 22:1430-1435.  167  Paton JJ, Belova MA, Morrison SE, Salzman CD (2006) The primate amygdala represents the positive and negative value of visual stimuli during learning. Nature 439:865-870. Paulus MP, Rogalsky C, Simmons A, Feinstein JS, Stein MB (2003) Increased activation in the right insula during risk-taking decision making is related to harm avoidance and neuroticism. Neuroimage 19:1439-1448. Paulus MP, Frank LR (2006) Anterior cingulate activity modulates nonlinear decision weight function of uncertain prospects. Neuroimage 30:668-677. Paxinos G and Watson C (1998) The Rat Brain in Stereotaxic Coordinates, 4th Edition, San Diego: Academic Press. Pearce JM, Hall G (1980) A model for Pavlovian learning: variations in the effectiveness of conditioned but not of unconditioned stimuli. Psychol Rev 87:532-552. Pennartz CM, Groenewegen HJ, Lopes da Silva FH (1994) The nucleus accumbens as a complex of functionally distinct neuronal ensembles: an integration of behavioural, electrophysiological and anatomical data. Prog Neurobiol 42:719-761, Peters J, Buchel C (2009) Overlapping and distinct neural systems code for subjective value during intertemporal and risky decision making. J Neurosci 29:15727-15234. Phillips AG, Ahn S, Floresco SB (2004) Magnitude of dopamine release in medial prefrontal cortex predicts accuracy of memory on a delayed response task. J Neurosci 24:547-553 Plassmann H, O’Doherty J, Rangel A (2007) Orbitofrontal cortex encodes wilingness to pay in everyday economic transactions. J Neurosci 27:9984-9988. Pochon JB, Riis J, Sanfey AG, Nystrom LE, Cohen JD (2008) Functional imaging of decision conflict. J Neurosci 28 :3468-3473.  168  Preuschoff K, Bossaerts P, Quartz SR (2006) Neural differentiation of expected reward and risk in human subcortical structures. Neuron 51:381-390. Preuschoff K, Quartz SR, Bossaerts P (2008) Human insula activation reflects risk prediction errors as well as risk. J Neurosci 28:2745-2752. Quirk GJ, Beer JS (2006) Prefrontal involvement in the regulation of emotion: convergence of rat and human studies. Curr Opin Neurobiol 16:723-727. Rachlin H, Green L (1972) Commitment, choice and self-control. J Exp Anal Behav 17:15-22. Rachlin H, Raineri A, Cross D (1991) Subjective probability and delay. J Exp Anal Behav 55:233-244. Ragozzino ME (2002) The effects of dopamine D(1) receptor blockade in the prelimbicinfralimbic areas on behavioral flexibility. Learn Mem 9:18-28. Ragozzino ME, Detrick S, Kesner RP (1999) Involvement of the prelimbic-infralimbic areas of the rodent prefrontal cortex in behavioral flexibility for place and response learning. J Neurosci. 19:4585-4594. Ragozzino ME (2007) The contribution of the medial prefrontal cortex, orbitofrontal cortex, and dorsomedial striatum to behavioral flexibility. Ann N Y Acad Sci 1121:355-375. Ragozzino ME, Rozman S (2007) The effect of rat anterior cingulate inactivation on cognitive flexibility. Behav Neurosci 121:698-706. Rahman S, Sahakian BJ, Hodges JR, Rogers RD, Robbins TW (1999) Specific cognitive deficits in mild frontal variant frontotemporal dementia. Brain 122:1469-1493. Rasmussen EB, Lawyer SR and Reilly WR (2010) Percent body fat is related to delay and probability discounting for food in humans. Behav Processes 83:23-30.  169  Rao H, Korczykowski M, Pluta J, Hoang A, Detre JA (2008) Neural correlates of voluntary and involuntary risk taking in the human brain: an fMRI study of the Balloon Analog Risk Task (BART). Neuroimage 42:902-910. Reynolds B (2006) A review of delay-discounting research with humans: relations to drug use and gambling. Behav Pharmacol 17:651-667. Ridderinkhof KR, van den Wildenberg WP, Segalowitz SJ, Carter CS (2004) Neurocognitive mechanisms of cognitive control: the role of prefrontal cortex in action selection, response inhibition, performance monitoring, and reward-based learning. Brain Cogn 56:129-140. Robbins TW, Cador M, Taylor JR, Everitt BJ (1989) Limbic-striatal interactions in reward-related processes. Neurosci Biobehav Rev 13:155-162. Rogers RD, Everitt BJ, Baldacchino A, Blackshaw AJ, Swainson R, Wynne K, Baker NB, Hunter J, Carthy T, Booker E, et al. (1999a). Dissociable deficits in the decision-making cognition of chronic amphetamine abusers, opiate abusers, patients with focal damage to prefrontal cortex, and tryptophan-depleted normal volunteers: Evidence for monoaminergic mechanisms. Neuropsychopharmacology 20:322-339. Rogers RD, Owen AM, Middleton HC, Williams EJ, Pickard JD, Sahakian BJ, Robbins TW (1999b) Choosing between small, likely rewards and large, unlikely rewards activates inferior and orbital prefrontal cortex. J Neurosci 19:9029-9038. Rogers RD, Ramnani N, Mackay C, Wilson JL, Jezzard P, Carter CS, Smith SM (2004) Distinct portions of anterior cingulate cortex and medial prefrontal cortex are activated by reward processing in separable phases of decision-making cognition. Biol Psychiatry 55:594-602.  170  Roiser JP, de Martino B, Tan GC, Kumaran D, Seymour B, Wood NW, Dolan RJ (2009) A genetically mediated bias in decision making driven by failure of amygdala control. J Neurosci 29:5985-5981. Rosenkranz JA, Grace AA (2001) Dopamine attenuates prefrontal cortical suppression of sensory inputs to the basolateral amygdala of rats. J Neurosci 21:4090-4103. Rosenkranz JA, Grace AA (2002) Cellular mechanisms of infralimbic and prelimbic prefrontal cortical inhibition and dopaminergic modulation of basolateral amygdala neurons in vivo. J Neurosci 22:324-337. Rudebeck PH, Walton ME, Smyth AN, Bannerman DM, Rushworth MF (2006) Separate neural pathways process different decision costs. Nat Neurosci 9:1161-1168. Rushworth MF, Walton ME, Kennerley SW, Bannerman DM (2004) Action sets and decisions in the medial frontal cortex. Trends Cogn Sci 8:410-417. Rushworth, M (2008) Intention, choice, and the medial frontal cortex. Ann NY Acad Sci 1124: 181-207. Salamone JD, Steinpreis RE, McCullough LD, Smith P, Grebel D, Mahan K (1991) Haloperidol and nucleus accumbens dopamine depletion suppress lever pressing for food but increase free food consumption in a novel food choice procedure. Psychopharmacology (Berl) 104:515-521. Salzman CD, Paton JJ, Belova MA, Morrison SE (2007) Flexible neural representations of value in the primate brain. Ann N Y Acad Sci 1121:336-354. Santos LR, Hughes KD (2009) Economic cognition in humans and animals: the search for core mechanisms. Curr Opin Neurobiol 19:63-66.  171  Scarna A, McTavish SF, Cowen PJ, Goodwin GM, Rogers RD (2005) The effects of a branched chain amino acid mixture supplemented with tryptophan on biochemical indices of neurotransmitter function and decision-making. Psychopharmacology (Berl) 179:761-768. Schoenbaum G, Setlow B (2001) Integrating orbitofrontal cortex into prefrontal theory: common processing themes across species and subdivisions. Learn Mem 8:134-147. Schiller D, Delgado MR (2010) Overlapping neural systems mediating extinction, reversal and regulation of fear. Trends Cogn Sci 14:268-276. Schweimer J, Saft S, Hauber W (2005) Involvement of catecholamine neurotransmission in the rat anterior cingulate in effort-related decision making. Behav Neurosci 119: 1687-1692. Schweimer J, Hauber W (2006) Dopamine D1 receptors in the anterior cingulate cortex regulate effort-based decision making. Learn Mem 13:777-782. Schultz W, Dayan P, Montague PR (1997) A neural substrate of prediction and reward. Science 275:1593-1599. Seamans JK, Floresco SB, Phillips AG (1995) Functional differences between the prelimbic and anterior cingulate regions of the prefrontal cortex. Behav Neurosci 109: 1063-1073. Seamans JK, Floresco SB, Phillips AG (1998) D1 receptor modulation of hippocampal-prefrontal cortical circuits integrating spatial memory with executive functions in the rat. J Neurosci 18:1613-1621. Seamans JK, Yang CR (2004) The principal features and mechanisms of dopamine modulation in the prefrontal cortex. Progress in Neurobiology 74: 1-57. Sesack SR, Deutch AY, Roth RH, Bunney BS (1989) Topographical organization of the efferent projections of the medial prefrontal cortex in the rat: an anterograde tract-tracing study with Phaseolus vulgaris leucoagglutinin. J Comp Neurol 290:213-242.  172  Sesack SR, Bressler CN, Lewis DA (1995) Ultrastructural associations between dopamine terminals and local circuit neurons in the monkey prefrontal cortex: a study of calretininimmunoreactive cells. Neurosci Lett 200:9-12. Setlow B, Holland PC, Gallagher M (2002) Disconnection of the basolateral amygdala complex and nucleus accumbens impairs appetitive pavlovian second-order conditioned responses. Behav Neurosci 116: 267-275. Seymour B, Dolan R (2008) Emotion, decision making, and the amygdala. Neuron 58:662-671. Sevy S, Hassoun Y, Bechara A, Yechiam E, Napolitano B, Burdick K, Delman H, Malhotra A (2006) Emotion-based decision-making in healthy subjects: short-term effects of reducing dopamine levels. Psychopharmacology (Berl) 188:228-235. Shallice T, Burgess PW, Schon F, Baxter DM (1989) The origins of utilization behaviour. Brain 112:1587-1598. Sharp D, Salter S (1997) Prospect escalation and sunk costs: a test of the international generalizability of agency and prospect theories. Journal of International Business Studies 28: 101-122. Shinonaga Y, Takada M, Mizuno N (1994) Topographic organization of collateral projections from the basolateral amygdaloid nucleus to both the prefrontal cortex and nucleus accumbens in the rat. Neuroscience 58:389-397. Simon NW, Gilbert RJ, Mayse JD, Bizon JL, Setlow B (2009) Balancing risk and reward: a rat model of risky decision making. Neuropsychopharmacology 34:2208-2217. Slezak JM, Anderson KG (2009) Effects of variable training, signaled and unsignaled delays, and d-amphetamine on delay-discounting functions. Behav Pharmacol 20:424-436.  173  Smith BW, Mitchell DG, Hardin MG, Jazbec S, Fridberg D, Blair RJ, Ernst M (2009). Neural substrates of reward magnitude, probability, and risk during a wheel of fortune decisionmaking task. Neuroimage 44:600-609. Sotres-Bayon F, Bush DE, LeDoux JE (2004) Emotional perseveration: an update on prefrontalamygdala interactions in fear extinction. Learn Mem 11:525-535. Sripanidkulchai K, Sripanidkulchai B, Wyss JM (1984) The cortical projection of the basolateral amygdaloid nucleus in the rat: a retrograde fluorescent dye study. J Comp Neurol 229:419431. StataCorp. 2009. Stata Statistical Software: Release 11. College Station, TX: StataCorp LP. St. Onge JR and Floresco SB (2009) Dopaminergic modulation of risk-based decision making. Neuropsychopharmacology 34:681-697. St. Onge JR, Chiu YC, Floresco SB (2010) Differential effects of dopaminergic manipulations on risky choice. Psychopharmacology (Berl) 211:209-221. St.Onge JR, Ahn S, So K, Phillips AG, Floresco SB (2011) Dynamic and dissociable fluctuations in prefrontal and ventral striatal dopamine efflux during risk-based decision making. Soc Neurosci Abstr 511.11. Stephens DW, Krebs JR (1986) Foraging Theory. Princeton University Press: Princeton New Jersey. Stopper CM, Floresco SB (2011) Contributions of the nucleus accumbens and its subregions to different aspects of risk-based decision making. Cogn Affect Behav Neurosci 11:97-112. Stuss DT and Alexander MP (2000) Executive functions and the frontal lobes: A conceptual view. Psychol Res 63:289-298.  174  Sutton RS, Barto AG (1998) Reinforcement Learning: an Introduction. Cambridge, MA: MIT Press. Symmonds M, Bossaerts P, Dolan RJ (2010) A behavioral and neural evaluation of prospective decision-making under risk. J Neurosci 30:14380-14389. Taber MT, Fibiger HC (1997) Activation of the mesocortical dopamine system by feeding: lack of a selective response to stress. Neuroscience 77:295-298. Takahashi YK, Roesch MR, Stalnaker TA, Haney RZ, Calu DJ, Taylor AR, Burke KA, Schoenbaum G (2009) The orbitofrontal cortex and ventral tegmental area are necessary for learning from unexpected outcomes. Neuron 62:269-280. Tom SM, Fox CR, Trepel C, Poldrack RA (2007) The neural basis of loss aversion in decisionmaking under risk. Science 315:515-518. Thorpe C, Floresco S, Carr J, Wilkie D (2002) Alterations in time-place learning induced by lesions to the rat medial prefrontal cortex. Behav Processes 59:87-100. Tranel D, Hyman BT (1990) Neuropsychological correlates of bilateral amygdala damage. Arch Neurol 47:349-355. Tversky A, Kahneman D (1974) Judgment under uncertainty: heuristics and biases. Science 185:1124-1131. Uylings HB, van Eden CG (1990) Qualitative and quantitative comparison of the prefrontal cortex in rat and in primates, including humans. Prog Brain Res 85:31-62. Uylings HB, Groenewegen HJ, Kolb B (2003) Do rats have a prefrontal cortex? Behav Brain Res 146:3-17.  175  van Gaalen MM, van Koten R, Schoffelmeer AN, Vanderschuren LJ (2006) Critical involvement of dopaminergic neurotransmission in impulsive decision making. Biol Psychiatry 60:6673. Verney C, Alvarez C, Geffard M, Berger B (1990) Ultrastructural double-labeling study of dopamine terminals and GABA-Containing neurons in rat anteromedial cerebral cortex. Eur J Neurosci 2:960-972. Vertes RP (2004) Differential projections of the infralimbic and prelimbic cortex in the rat. Synapse 51:32-58. Vincent SL, Khan Y, Benes FM (1993) Cellular distribution of dopamine D1 and D2 receptors in rat medial prefrontal cortex. J Neurosci 13:2551-2564. Volz KG, Schubotz RI, von Cramon DY (2003) Predicting events of varying probability: uncertainty investigated by fMRI. Neuroimage 19:271-280. Volz KG, Schubotz RI, von Cramon DY (2004) Why am I unsure? Internal and external attributions of uncertainty dissociated by fMRI. Neuroimage 21:848-857. von Neumann J, Morgenstern O (1944) Theory of Games and Economic Behavior. Princeton, NJ: University Press. Vuchinich RE, Simpson CA (1998) Hyperbolic temporal discounting in social drinkers and problem drinkers. Experimental and Clinical Psychopharmacology 6: 292-305. Wager TD, Davidson ML, Hughes BL, Lindquist MA, Ochsner KN (2008) Prefrontal-subcortical pathways mediating successful emotion regulation. Neuron 59: 1037-1050. Walton ME, Bannerman DM, Alterescu K, Rushworth MF (2003) Functional specialization within medial frontal cortex of the anterior cingulate for evaluating effort-related decisions. J Neurosci 23:6475-6479.  176  Walton ME, Devlin JT, Rushworth MF (2004) Interactions between decision making and performance monitoring within prefrontal cortex. Nat Neurosci 7:1259-1265. Walton ME, Rudebeck PH, Bannerman DM, Rushworth MFS (2007) Calculating the cost of acting in frontal cortex. Ann NY Acad Sci 1104: 340-356. Wanat MJ, Kuhnen CM, Phillips PE (2010) Delays conferred by escalating costs modulate dopamine release to rewards but not their predictors. J Neurosci 30:12020-12027. Weller JA, Levin IP, Shiv B, Bechara A. (2007) Neural correlates of adaptive decision making for risky gains and losses. Psychol Sci 18:958-964. Williams GV, Goldman-Rakic (1995) Modulation of memory fields by dopamine D1 receptors in prefrontal cortex. Nature 376: 572-575. Winstanley CA, Theobald DE, Cardinal RN, Robbins TW (2004) Contrasting roles of basolateral amygdala and orbitofrontal cortex in impulsive choice. J Neurosci 24:4718-4722. Winstanley CA, Theobald DE, Dalley JW, Cardinal RN, Robbins TW (2006) Double dissociation between serotonergic and dopaminergic modulation of medial prefrontal and orbitofrontal cortex of impulsive choice. Cereb Cortex 16:106-114. Wrase J, Kahnt T, Schlagenhauf F, Beck A, Cohen MX, Knutson B, Heinz A (2007) Different neural systems adjust motor behavior in response to reward and punishment. Neuroimage 36:1253-1262. Yacubian J, Glascher J, Schroeder K, Sommer T, Braus DF, Buchel C (2006) Dissociable systems for gain- and loss-related value predictions and errors of prediction in the human brain. J Neurosci 26:9530-9537.  177  Yang CR, Seamans JK (1996) Dopamine D1 receptor actions in layers V-VI rat prefrontal cortex neurons in vitro: modulation of dendritic-somatic signal integration. J Neurosci 16:19221935. Zahm DS, Heimer L (1990) Two transpallidal pathways originating in the nucleus accumbens. J Comp Neurol 302:437-46 Zeeb FD, Winstanley CA (2011) Lesions of the basolateral amygdala and orbitofrontal cortex differentially affect acquisition and performance of a rodent gambling task. J Neurosci 31:2197-2204.  178  Figure 18. The ascending and descending axonal pathways between the medial PFC and BLA travel through separate routes through the brain.  179  

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
http://iiif.library.ubc.ca/presentation/dsp.24.1-0072501/manifest

Comment

Related Items