Arts, Faculty of
Psychology, Department of
DSpace
Hales, C., Clark, L., & Winstanley, C. (2023). Computational approaches to modeling gambling behaviour: Opportunities for understanding disordered gambling. Neuroscience & Biobehavioral Reviews, 147.
Djavad Mowafaghian Centre for Brain Health
Hales, C. A.
Clark, L.
Winstanley, C. A.
Computational modeling has become an important tool in neuroscience and psychiatry research to provide insight into the cognitive processes underlying normal and pathological behavior. There are two modeling frameworks, reinforcement learning (RL) and drift diffusion modeling (DDM), that are well developed in cognitive science, and have begun to be applied to Gambling Disorder. RL models focus on explaining how an agent uses reward to learn about the environment and make decisions based on outcomes. The DDM is a binary choice framework that breaks down decision making into psychologically meaningful components based on choice reaction time analyses. Both approaches have begun to yield insight into aspects of cognition that are important for, but not unique to, gambling, and thus relevant to the development of Gambling Disorder. However, these approaches also oversimplify or neglect various aspects of decision making seen in real-world gambling behavior. Gambling Disorder presents an opportunity for ‘bespoke’ modeling approaches to consider these neglected components. In this review, we discuss studies that have used RL and DDM frameworks to investigate some of the key cognitive components in gambling and Gambling Disorder. We also include an overview of Bayesian models, a methodology that could be useful for more tailored modeling approaches. We highlight areas in which computational modeling could enable progression in the investigation of the cognitive mechanisms relevant to gambling.
1 Article Title: Computational approaches to modeling gambling behaviour: opportunities for understanding disordered gambling Authors and Affiliations: C. A. Hales1,2, L. Clark1,2, C.A. Winstanley1,2 1. Djavad Mowafaghian Centre for Brain Health, University of British Columbia, Vancouver, BritishColumbia, Canada.2. Department of Psychology, University of British Columbia, Vancouver, British Columbia, Canada.Funding and acknowledgements: This work was supported by a Discovery Grant awarded to CAW from the Natural Sciences and Engineering Council of Canada (NSERC; RGPIN-2017-05006) and Canadian Institutes of Health Research operating grant to CAW (PJT-162312). CAH was supported by a Michael Smith Health Research BC Trainee Award (RT-2020-0564). The Centre for Gambling Research at the University of British Columbia (LC) is supported by the Province of British Columbia government and the British Columbia Lottery Corporation (BCLC; a Canadian Crown Corporation). LC holds a Discovery Award from the Natural Sciences and Engineering Research Council of Canada (RGPIN‐2017‐04069). This review covers experimental work which took place at a University of British Columbia campus situated on the traditional, ancestral, and unceded land of the xʷməθkʷəyə̓m (Musqueam), sə̓lílwətaʔɬSelilwitulh (Tsleil-Waututh) and Sḵwxw̱ú7mesh (Squamish) Peoples. We acknowledge and are grateful for their stewardship of this land for thousands of years. Conflicts of interest: LC is the Director of the Centre for Gambling Research at UBC, which is supported by funding from the Province of British Columbia and the British Columbia Lottery Corporation (BCLC), a Canadian Crown Corporation. The Province of BC government and the BCLC had no role in the preparation of this manuscript, and impose no constraints on publishing. LC has received a speaker/travel honorarium from the National Association for Gambling Studies (Australia) and the International Center for Responsible Gaming (US), and has received fees for academic services from the International Center for Responsible Gaming (US), GambleAware (UK) and Gambling Research Exchange Ontario (Canada). He has not received any further direct or indirect payments from the gambling industry or groups substantially funded by gambling. He has received royalties from Cambridge Cognition Ltd. relating to neurocognitive testing. CAW has received research funding and fees for academic services from the International Center for Responsible Gaming (US). CAH reports no disclosures. behaviour: Opportunities for understanding disordered gambling. Neuroscience & Biobehavioral Reviews, 147, 105083, which has been published in final form atThis is the postprint peer reviewed version of the following article: Hales, C., Clark, L., & Winstanley, C. (2023). Computational approaches to modeling gamblinghttps://doi.org/10.1016/j.neubiorev.2023.105083 © 2023. This manuscript version is made available under the CC-BY-NC-ND 4.0 license https://creativecommons.org/licenses/by-nc-nd/4.0/ 2 Abstract Computational modeling has become an important tool in neuroscience and psychiatry research to provide insight into the cognitive processes underlying normal and pathological behavior. There are two modeling frameworks, reinforcement learning (RL) and drift diffusion modeling (DDM), that are well-developed in cognitive science, and have begun to be applied to Gambling Disorder. RL models focus on explaining how an agent uses reward to learn about the environment and make decisions based on outcomes. The DDM is a binary choice framework that breaks down decision making into psychologically meaningful components based on choice reaction time analyses. Both approaches have begun to yield insight into aspects of cognition that are important for, but not unique to, gambling, and thus relevant to the development of Gambling Disorder. However, these approaches also oversimplify or neglect various aspects of decision making seen in real-world gambling behavior. Gambling Disorder presents an opportunity for ‘bespoke’ modeling approaches to consider these neglected components. In this review, we discuss studies that have used RL and DDM frameworks to investigate some of the key cognitive components in gambling and Gambling Disorder. We also include an overview of Bayesian models, a methodology that could be useful for more tailored modeling approaches. We highlight areas in which computational modeling could enable progression in the investigation of the cognitive mechanisms relevant to gambling. 3 Introduction Gambling decisions entail risking an item of value on the uncertain prospect of a larger reward, and real-world gambling includes three components: a stake, a prize, and the element of chance (Reber 2012). The cognitive processes involved in these elements, and interplay between them - cost-benefit analysis, reward processing and risk - are not specific to gambling, but feature in many aspects of real-life decision making, and are disrupted in a variety of mental health disorders. Many participate in gambling recreationally, but for a subset of people gambling becomes pathological, whereby they engage in compulsive, excessive and maladaptive gambling behaviors. In DSM-5 (APA, 2013), Gambling Disorder is classified under the category “Substance-Related and Addictive Disorders”, signifying the overlap in the underlying neurobiology and neurocognitive mechanisms between drug addictions and Gambling Disorder, the prototypical “behavioral addiction” (Clark et al., 2013). Diagnoses of Gambling Disorder are also highly co-morbid with other mental health disorders, such as anxiety and depression (Petry et al., 2005), as well as substance addictions and other behavioral addictions (Petry et al., 2005; 2014; Ford & Håkansson, 2020). The cognitive constructs thought to be important in gambling (discussed below) are many and complex. This complexity of behavior, not only in gambling, but across all mental health disorders is one of the key current challenges in neuroscience research today. The application of computational models to behavioral data from tasks investigating these processes has begun to provide some insight into how different cognitive processes are relevant to gambling behavior and, by extension, Gambling Disorder. This will be focus of this review. Decision making has been described as the cumulation of a set of cognitive processes that enable an individual to choose the most optimal course of action following reasoned consideration of existing alternatives (Bechara, 2005). Many decision making components are shared between gambling-specific decisions and other types of real-world decisions, including risk taking and the processing of uncertainty, ambiguity and cost-benefit trade-offs (see Clark, 2010; Kovács et al., 2017 for reviews). The 4 mechanisms underlying this span brain functions relating to multiple domains and linked to many neurochemical systems and brain areas. A comprehensive overview of the neurotransmitters and neuroanatomy that support risk-based decision making is outside the scope of this article, but there are clear roles of mesolimbic dopamine in reward processing and reward-based learning, serotonin in the prefrontal cortex for modulating flexibility in decision making, and noradrenaline for regulating strategic decisions like exploration versus exploitation (Doya, 2008; Rogers, 2011). Many of these circuits converge in the prefrontal cortex as a control centre for numerous facets of decision making (see Clark et al., 2004; Aston-Jones & Cohen, 2005; and Rushworth et al., 2011 for detailed reviews). Impulsivity - a predisposition for actions that are poorly conceived or carried out too rapidly, prematurely or inappropriately given the situation – is also closely linked to disordered gambling, and forms a psychopathological part of many mental health disorders (e.g. substance addictions, attention-deficit hyperactivity disorder (ADHD) and personality disorder; Perry & Carroll, 2008; Sharma & Couture, 2014; Lieb et al., 2004). Impulsivity is heterogenous, comprising motor impulsivity (or inhibitory control), temporal impulsivity (or delay discounting), and reflection/decision impulsivity (Evenden, 1999; Caswell et al., 2015). Impulsivity and decision making are closely linked, with impulsivity commonly being associated with maladaptive real-world choices (Franken et al., 2008; Wiehler & Peters, 2015), and decision making being central to both temporal and reflection facets of impulsivity (Evenden, 1999; Caswell et al., 2015). In a recent meta-analysis, Gambling Disorder was associated with impairments in motor impulsivity, attentional inhibition, as well as decision making impulsivity and discounting (Ioannidis et al., 2019). Other inter-related cognitive components that are pertinent to understanding Gambling Disorder include compulsivity (i.e. the repetition of actions despite negative consequences; van Timmeren et al., 2018), cognitive control (which includes processes such as response inhibition, conflict monitoring, and cognitive flexibility; Moccia et al., 2017) and abnormal neural responses to cues (van 5 Holst et al., 2010). Although multi-faceted and complex in themselves, these constructs are also conceptually linked to impulsivity and decision making. For example, compulsivity impacts habitual decision making pathways (Voon et al., 2015), and was still considered a subcomponent of impulsivity around the turn of the century (Moeller et al., 2001). Similarly, cue-based learning directly impacts decision making and impulsivity in human problem gamblers by modulating striatal value representations (Miedl et al., 2014). Computational modeling of the cognitive components implicated in Gambling Disorder Approaches that allow researchers to probe the neurobiological mechanisms underlying complex cognitive constructs can provide insight into not only a specific disorder of interest, but benefit much more broadly the neuroscience, psychology and psychiatry community as a whole. Computational modeling formalizes the cognitive processes carried out by the brain and enables researchers to characterize how changes in cognitive components associated with mental health disorders compare to normal brain function and neural processes (Adams et al., 2016). Most commonly, models are applied to behavioral data, with model parameters mapping to dissociable cognitive elements that contribute to measured behavior. Modeling approaches have been applied in decision making research for many years, helping to infer the psychological constructs important in health and disease. Reinforcement learning (RL), a framework that describes how agents learn about and interact with their environment (Sutton & Barto, 1998), has been extensively used to study how learning about rewarding and punishing outcomes is altered in mental health (see Maia & Frank, 2011; Ahn et al., 2016; Huys et al., 2021; Groman et al., 2022 for reviews). Drift diffusion models (DDMs; Ratcliff, 1978) have been widely used in cognitive psychology to map the cognitive processes that underlie decision making between two choices to 6 psychologically meaningful parameters (see Voss et al., 2013; Forstmann et al., 2016 for reviews). The central idea behind DDM is that evidence accumulates over time in favor of one option or another, with a decision being made once an evidence threshold is reached. Although traditionally applied to perceptual decisions where evidence is perceptual information from the stimuli, more recent work has demonstrated the utility of diffusion models in value-based decision making (Dutilh & Rieskamp, 2016; Tajima et al., 2016). Here, the evidence can be in the form of internal factors, such as a person's desires, learned value representations, potential rewards, and beliefs about consequences of different choices. Bayesian models are, in their simplest form, models that use probability to represent uncertainty. In cognitive science, Bayesian frameworks have been developed to test models of brain function, and to elucidate how the brain might implement Bayesian inference (see Rescorla, 2021 for a recent review). The principles of Bayes theorem and Bayesian inference can be used to build and design “bespoke” cognitive models to measure combined influence of multiple cognitive components, such as decision making and impulsivity, or to probe constructs not typically captured by other forms of well-established models (Colas et al., 2010). Bayesian approaches can also be used in combination with RL and DDM models for parameter estimation (Wiecki et al., 2013; Pedersen & Frank, 2020), as a method of model comparison using Bayes factors (e.g. Steingroever et al., 2016), and as a statistical method to analyse the significance of model results (Annis & Palmeri, 2018). Here, we will synthesise the main impacts of recent RL and DDM modeling approaches in gambling research, including studies where the two frameworks have been combined and/or utilised Bayesian methods. We will also highlight some recent tailored Bayesian approaches that are beginning to be used to study cognitive constructs more specific to Gambling Disorder. Finally, we will discuss the considerable untapped potential that modeling approaches based on animal models have to provide insight into Gambling Disorder. 7 Reinforcement learning models Addictive behaviors are commonly modeled using RL algorithms, which use information about the expected value of future rewards to select actions and make decisions. This expected value is compared to actual experience during learning, resulting in a reward prediction error, which guides updating of the model to improve future choices. Figure 1 depicts a typical RL model, along with a summary of findings from studies using RL models in gambling to date; those studies are discussed in more detail in the main text. Empirical studies have provided strong evidence that dopamine encodes reward prediction errors (Schultz et al., 1997; Keiflin & Janak, 2015). Activity of midbrain dopamine neurons closely matches the value-prediction error signal used in standard ‘temporal difference’ RL models to calculate the value function. Simulations of this model provide a good explanation for choice behavior in Gambling Disorder (Redish et al., 2007). More specifically, Redish and colleagues hypothesized that the updating representations of wins and losses are split into separate states during learning. This provides a mechanistic explanation for a common bias experienced in people with Gambling Disorder: that the expected value from early wins (a risk factor for Gambling Disorder) is not counteracted by lengthy sequences of losses (Redish et al., 2007). RL models have been extensively applied to data from the Iowa Gambling Task (IGT; see Steingroever et al., 2014; 2016 for recent summaries). The IGT (Bechara et al., 1994) involves the participant making a series of choices between card decks with unknown different probabilities of reward and loss. Net score (the number of choices from advantageous decks minus those from disadvantageous decks) is the most commonly reported behavioral measure, and participants with Gambling Disorder display performance deficits on the IGT due to increased selection of disadvantageous decks (Brevers et al., 2013). RL modeling has shown that net score is best conceptualized as a composite of several distinct processes, including learning rate, reward and loss 8 Figure 1 – Schematic summarizing the RL model parameters associated with aspects of learning and decision making relevant to gambling and Gambling Disorder A schematic of RL models showing parameters in the model (dashed grey outlines) and the associations that have been found with cognitive processes or experimental manipulations (linked rounded solid boxes) in the limited number of studies conducted in relation to Gambling Disorder. In RL learning models, an agent selects actions based on the expected value of future rewards that causes a reinforcing outcome from the environment. The agent perceives states and updates their representations of expected values based on the current state through learning from the received reinforcers. In model-free learning, the agent purely uses past choices and outcomes to guide learning and updating, whereas in model-based learning the agent builds an internal model to evaluate the prospective reward from future outcomes. To date, there has been a mix of findings in relation to what extent gamblers rely on model-free versus model-based learning. Other studies have begun to find associations between RL parameters and other cognitive processes, such as impulsivity, that impact learning and decision-making. 9 magnitudes, motivation and response consistency (e.g. Yechiam et al., 2008; Steingroever et al., 2016). Modeling enables researchers to probe which of these cognitive factors may be driving poorer performance, information that cannot be gathered from standard analysis of behavioral measures in terms of net score. This was demonstrated by Yechiam et al. (2008), in which the application of a simple version of the Expectancy-Valency RL model (Busemeyer & Stout, 2002) revealed that the cognitive profiles underlying poor IGT performance were different across six groups of criminals despite all displaying a similar reduction in net score. In a more recent study, a Bayesian hierarchical implementation of a popular RL model (the Prospect Valence Learning-Delta model), was combined with Bayes factors for model selection and Bayesian statistics to investigate individual differences in IGT performance (Steingroever et al., 2018). In this case, Bayesian modeling revealed both decision-style groups relied on similar cognitive processes in the IGT, despite people self-reporting reliance on intuitive versus deliberative decision making styles (Steingroever et al., 2018). Despite the lack of difference here, this study exemplifies the opportunity that combining Bayesian approaches with RL models provides for investigating individual differences in decision making mechanisms. This type of approach could be employed in the future to compare whether deficits in IGT performance in other populations (for example patients with Gambling Disorder compared to substance addictions or other behavioral addictions) is driven by similar or different cognitive factors. A key tenet of theories of addictive behaviors is the balance between habitual and goal-oriented mechanisms for learning and decision making, with it being hypothesised that control shifts from goal-directed to habitual systems in addictions (Everitt & Robbins, 2005; 2016). These two processes have been operationalised computationally in terms of model-free and model-based RL algorithms (Daw et al., 2005). Model-free RL uses retrospective evaluation of past choices and outcomes using reward prediction errors, whereas model-based RL uses an internal model of the environment to carry out prospective evaluation of future outcomes. The contributions of model-free and model-based learning 10 can be dissociated using the two-step task (Daw et al., 2011), a sequential decision making task in which the participant’s choice on the first stage probabilistically leads to one of two second stages, each of which has an associated probability of reward. This structure creates distinct performance profiles for model-free or model-based learning, and enables computational approaches to be used to directly test the hypothesised shift from goal-directed to habitual learning. So far, results studies in Gambling Disorder have been mixed. Wyckmans et al. (2019) found patients with Gambling Disorder rely less on model-based RL mechanisms on a two-step task compared to controls, particularly after an unrewarded trial. Reduced model-based learning on the two-step task was also found in participants with Gambling Disorder compared to healthy controls, when tested in both gambling-related and neutral virtual reality environments (Bruder et al., 2021). Reduced model-based learning was also confirmed in a study investigating the effect of acute stress on habitual versus flexible decision making in gamblers (Wyckmans et al., 2022). People with gambling problems and healthy controls performed the two-step task, half after a control manipulation, and half after an acute stress induction using the Socially Evaluated Cold Pressor Task (Schwabe et al., 2008). In the participants who did not undergo the stress manipulation, those with gambling problems exhibited less model-based learning than the controls. Under stress, healthy controls with increased cortisol levels (indicative a heightened stress response) showed a shift from model-based towards more model-free learning, but contrary to the authors’ hypothesis, this was not observed in the gamblers. The authors suggested the lack of a stress effect in the group with gambling problems may have been partly due to difference between groups in working memory, but potentially also due to differential adaptation to stress, or abnormal interoception of the somatic experience of stress, some evidence for which was provided by their modeling approach (Wyckmans et al., 2022). These studies suggest there may be some evidence for a shift from goal-directed towards more habitual learning and decision making in Gambling Disorder that is independent of the neurological effects of drugs of abuse in substance addictions. 11 However, in a similar study in which patients with Gambling Disorder and healthy controls carried out the two-step task after the same acute stress induction as above whilst in an fMRI scanner, it was found that the stress manipulation failed to alter amount of model-based or model-free learning on the task in either controls or gamblers (van Timmeren et al., 2022). Furthermore, there was no difference between groups at baseline (without the stress manipulation), at least as measured by the weighting parameter representing relative levels of model-free and model-based influence (van Timmeren et al., 2022), as had been seen previously (Wyckmans et al., 2019; 2022; Bruder et al., 2021). However, there were differences in choice randomness (RL model β parameter), such that patients with Gambling Disorder were less likely to repeat a choice following reward than healthy controls at baseline (i.e. gamblers displayed increased choice randomness that could be interpreted as indicative of greater model-free learning; van Timmeren et al., 2022). There was also an association of choice randomness with stress, driven by the acute stress manipulation reducing likelihood of repeating a rewarded choice in controls only. Although weaker evidence, the direction of effects match those seen by Wyckmans et al. (2022). Another study found no difference between healthy controls and patients with Gambling Disorder in the amount of model-free and model-based learning on the two-step task (Voon et al., 2020), whilst in another virtual reality study, RL modeling of the two-step task revealed that gamblers showed increased model-based and reduced model-free learning in the gambling context compared to the neutral context (Wagner et al., 2022). Furthermore, extent of model-free learning correlated with gambling symptom severity, such that increased severity was associated with a greater decrease in MF learning (Wagner et al., 2022). This result supports the idea of enhanced goal-directed control in gambling environments, perhaps through better utilization of task transition structure, but is in contrast to the majority of studies suggesting the opposite pattern, or no effect. 12 These opposing results may be due to subtle differences in task structure (for example whether outcomes were based on fluctuating reward probabilities or fluctuating reward magnitudes), or the clinical characteristics of the gamblers tested (e.g. symptom severity, treatment-seeking status, or preferred forms of gambling such as slot machines vs skill-based forms). It may be that this heterogeneity of findings (summarized in Figure 1) represents discrepancies between real-world gambling decisions that are inevitably more complex than those required on the relatively simple two-step task, and that we need more sophisticated tasks with correspondingly more sophisticated computational approaches to clearly and fully model these cognitive constructs. Clearly, more studies are warranted in order to conclusively elucidate how model-free and model-based learning contribute to gambling behaviors. Other RL approaches in Gambling Disorder include combining a RL action-selection model with Bayesian principles to investigate links between learning, decision making and impulsivity (Lim et al., 2015). 87 experienced (but non-disordered) gamblers recruited from the community performed a two-armed bandit task that simulated the volatile reward contingencies in a foraging environment, and performance was linked with two sets of established risk factors for gambling: trait impulsivity, and gambling-related cognitive distortions (Lim et al., 2015). The latter was measured with the widely-used Gambling Related Cognitions Scale (GRCS), which includes erroneous beliefs about randomness (e.g. the gambler’s fallacy), and the ability to predict or influence future wins (i.e. the illusion of control). In the model, each individual choice was regressed against an optimally estimated probability for that choice modeled by an ideal Bayesian learner, reward magnitude, and the winning or losing magnitude of the previous choice to capture the use of win-stay / lose-shift strategies. The model revealed that increased impulsivity and the GRCS predictive control scale were both associated with suboptimal learning and decision making on the task. Specifically, smaller learning rates, inaccurate weighting of probability outcomes, and reduced use of the optimal probability estimates for action selection were associated 13 with impulsivity, whilst predictive control (i.e. the belief that it is possible to predict winning outcomes) was linked to poor integration of previous reward and win/loss information to guide action selection (Lim et al., 2015). As further discussed in the Bayesian model section, this study highlights how utilization of more tailored Bayesian methods (in this case alongside well established RL modeling) allows researchers to probe how multiple cognitive constructs thought to be important in gambling are linked. Being able to link specific aspects of learning and decision making (such as learning rate or probability weighting) with other related cognitive constructs (such as impulsivity and predictive control) is a major benefit of computational modeling over and above traditional analyses of behavioral measures alone. The drift diffusion model (DDM) A major advantage of the DDM is its ability to simultaneously predict both choices and response times on a task. This means the wealth of information that is contained within reaction time data can be exploited to more comprehensively interrogate behavioral data. Key parameters include the decision starting point (a priori bias for one or other decision), the distance between the two thresholds that represent the two choices (boundary), and the rate of information accumulation (drift rate, modeled as a noisy random walk). Secondary parameters include the non-decision time, and measures of across-trial variability (described in more detail by Ratcliff & Tuerlinckx, 2002). Figure 2 shows a schematic representation of the DDM, as well as an overview of how these parameters relate to decision-making behavior in gambling. The DDM can account for optimal decision making in two-choice scenarios (Bogacz et al., 2006), whilst a variant of the DDM, Decision Field Theory, can account for choice patterns in multi-attribute settings such as choices among lotteries (Busemeyer & Johnson, 2004). With recent studies demonstrating that the DDM can also account for data from value-based choice tasks with high 14 Figure 2 – Schematic summarizing the alterations in Drift Diffusion Model parameters associated with aspects of decision making relevant to gambling and Gambling Disorder A schematic of the Drift Diffusion Model (DDM) showing parameters in the model (dashed grey outlines) and the associations that have been found with cognitive processes or experimental manipulations (linked rounded solid boxes) in the limited number of studies conducted in relation to Gambling Disorder. The y-axis (threshold separation) represents the distance between the two decision boundaries (labelled with dashed arrows). The evidence accumulation path (the steepness of which is the drift rate - represented by the grey, downwards pointing arrow) starts from the decision starting point (represented by the dotted line), located somewhere between the two decision boundaries. Once the evidence accumulation path (an example of which is represented by the black squiggly line) reaches one of the two boundaries, a decision for that corresponding choice is made. The x-axis represents time. The total reaction time that is measured in the behavioral task comprises the decision time (time taken for evidence accumulation to a boundary) and the non-decision time (time taken for other processes such as sensory and motor processing). In most cases, at least one study has found similar, overlapping results from different tasks or participant populations. This highlights the utility of modeling approaches to link together findings across cognitive domains. However, the number of studies using DDM in Gambling Disorder per se is very limited, and so it remains to see if further evidence supporting these associations is found with further research. 15 accuracy (e.g. Milosavljevic et al., 2010), this modeling framework has begun to be applied in relation to Gambling Disorder. For gambling relevant value-based decision, such as whether to take a risky gamble, it is assumed that the drift rate reflects a relative preference measure that combines weighting and integration of choice attributes, while the threshold is linked to maximisation of reward rate (Milosavljevic et al., 2010). Clay et al. (2017) applied the DDM to a modified Columbia Card gambling task. On each trial of this task, 8 cards were presented from a 32-card deck. Each card had either a positive or negative value. Participants chose whether or not to accept a gamble when allowed to turn over a fixed number of the 8 cards (either 1, 3 or 5), giving the task the two-choice format required for the DDM (Clay et al., 2017). Participants had to make repeated decisions in either a “winning” or “losing” environment, determined by whether the deck contained more or fewer positive value cards, and use the outcomes of their past choices to learn the underlying risk distribution (Clay et al., 2017). The DDM analysis revealed that the choice between accepting and rejecting a gamble was more difficult for more loss-averse individuals, as indicated by a smaller drift rate parameter, rather than loss aversion being due to a pre-existing bias shifting responding away from losses, which would have been captured by the starting point parameter (Clay et al., 2017). Another study used a gambling task that required the participant to accept or reject each bet in a series of ‘mixed’ gambles (Zhao et al. 2020). In their first experiment, each choice carried the possibility of either a positive or negative monetary outcome of equal probability. DDM was used to show that participants were generally loss averse, and this was explained by both the drift rate (the utility of the gamble, i.e. the difference in value between accepting and rejecting a gamble) and also the starting point (representing a priori bias) parameters (Zhao et al., 2020). In their second experiment, they were able to demonstrate that a priori bias can be altered by assigning participants to either a high payoff or low payoff condition: in the low payoff condition starting points were reduced, indicating that participants formed prior expectations towards rejecting gambles. They were also able to demonstrate a 16 dissociation between a priori bias (starting point) and drift rate mechanisms by looking at the effect of shared gambles where payoff structure was shared across the high and low conditions (Zhao et al., 2020). Combined, these two studies highlight some of the boundary conditions in task design for eliciting and measuring loss aversion in the lab, for example unequal versus equal probabilities of gains/losses or different payoff structures. They may also be informative when considering individual differences in problematic engagement with different types of gambling (e.g. poker vs electronic gaming machines) given that the way a gamble is presented can alter the relationship between different model parameters and cognitive processes. Insight into whether loss aversion is being driven by the same underlying cognitive processes in different populations of gamblers would not be possible without the use of modeling. Recently, the DDM has been combined with RL models to create a unified framework that enables investigation into both learning and decisional processes (Pedersen et al., 2017; Fontanesi et al., 2019; Miletić et al., 2020). In the model, the DDM is implemented as the choice rule (replacing the softmax rule) within a RL model framework, and fit to data using Hierarchical Bayesian modeling. Using this approach, two behavioral effects related to value-based decision making – how value magnitude and difficulty affect choice – can be explained by a combination of both RL model parameters (learning rate), and DDM drift rate and threshold parameters (Fontanesi et al., 2019). In relation to gambling, Peters and colleagues (2020) used the DDM as the choice rule in a risky choice model to investigate the effect of a pharmacological dopamine challenge on risky decision making in patients with Gambling Disorder. When comparing the clinical group with healthy controls in the placebo condition, gamblers showed less response caution (explained by smaller distance between thresholds) and faster non-decision times, indicative of motor impulsivity (Peters et al., 2020). When given tolcapone to elevate frontal dopamine, gamblers displayed increased risky decision making, an effect that was explained by a 17 reduced maximum drift rate along with increased value dependency of the drift rate. This suggests an important role for the subjective valuation of risky options (Peters et al., 2020). RL and DDM have also been used with virtual reality technology. Healthy controls and participants reporting frequent or pathological gambling performed a temporal discounting task and the two-step task (discussed above) in a neutral and a gambling-related environment (Bruder et al., 2021). Hierarchical Bayesian modeling was used to fit hyperbolic discounting models to data from the temporal discounting task with both softmax and DDM choice rules. As has been previously reported (Wiehler & Peters, 2015), gamblers exhibited increased temporal discounting, but this effect was not modulated by context. DDM revealed that the boundary separation parameter was reduced in the gambling-related context compared to the neutral context in gamblers only. This matches findings by Peters et al. (2020) and suggests that gamblers prioritise rapid responding in gambling environments. Using the same study design but only testing regular gamblers, a second study found that gamblers displayed steeper discounting curves in gambling-related environments (Wagner et al., 2022), in contrast to the lack of effect seen in the gambling environment by Bruder et al. (2021). DDM showed that non-decision times were reduced in the gambling context, as also seen by Peters et al. (2020), but that maximal drift rates were increased, reflecting increased value sensitivity in the gambling-related environment (Wagner et al., 2022). Reduced boundaries and non-decision times in participants with Gambling Disorder were also found when a combined RL and DDM model was applied to a simple RL task performed in fMRI scanner (Wiehler & Peters, 2020). Gamblers made fewer optimal responses in the task, and this was explained by reduced learning rates, a more rapidly collapsing boundary and shorter non-decision times (Wiehler & Peters, 2020). Reduced learning rates were partly linked to weaker value representations in the ventromedial prefrontal cortex (vmPFC), which in turn was proportional to symptoms of Gambling Disorder in an exploratory analysis (Wiehler & Peters, 2020). Damage to the vmPFC has previously been linked with poor performance on the IGT (e.g. Bechara et al., 18 1994), and is hypothesized to be a key node in pathophysiology that characterizes Gambling Disorder (Tanabe et al., 2007; Li et al., 2019). The vmPFC has also been linked with drift rate and value representations in a study measuring risky decision making in non-gamblers. Liu et al. (2022) used the DDM to evaluate performance of a two-stage sequential risk-taking task, in which participants had to choose between opening or not opening a box with potential reward or punishment. Information processing was different across the two stages, as suggested by neural decoding of electroencephalography and magnetoencephalography data. The initial valuation stage was associated with peak decoding accuracy in the vmPFC, which was also predicted by the drift rate, whilst the superior frontal gyrus was engaged in the final selection stage, and this activity was linked to non-decision time (Liu et al., 2022). The importance of the vmPFC in cognitive processes represented by diffusion model parameters relevant to gambling-related decision making was also demonstrated in a study measuring temporal and probability discounting in patients with ventromedial prefrontal cortex lesions (Peters & D'Esposito, 2020). Lesion patients had increased non-decision times in both tasks, whilst in the probability discounting task, patients had reduced threshold separation parameters, reflecting reduced response caution during risky choice, as well as diminished discounting over probabilities (Peters & D'Esposito, 2020). Although these two studies do not constitute direct evidence in the context of Gambling Disorder, they indicate that value representation (drift rate), response caution (boundary) and motor inhibition (non-decision time) are critical components in gambling decisions and thus important targets for future research in Gambling Disorder. Across these studies, the DDM has so far consistently identified disinhibition as a core component involved in gambling, but co-occurring with other alterations in decision making parameters, and with specific deficits perhaps linked to different types of decision making tasks. However, there is also much heterogeneity in individuals with Gambling Disorder with respect to severity of gambling problems, types of gambling engaged in, co-morbidity with other mental health disorder and socio- 19 economic and demographic characteristics (as highlighted in a recent special issue on the Pathways Model of disordered gambling; Clark et al., 2022). Discrepancies in modeling results not only from DDM studies, but also as highlighted in the RL section above, could be in part due to this sort of heterogeneity. This could be seen both as a challenge that needs to be overcome, but can also be framed as an advantage. Modeling approaches can find commonalities across results from tasks measuring different cognitive constructs (e.g. decision making, impulsivity), different experimental paradigms (e.g. behavioral tasks, neural recordings) and heterogeneous study populations, which can provide insight into processes relevant to Gambling Disorder, despite not all being directly carried out in populations with Gambling Disorder. However, the number of studies utilizing this type of approach is still sparse (Figure 2). With more data, it may be possible to use techniques such as clustering to stratify heterogenous populations (such as those with Gambling Disorder) through derived model parameters that represent distinct cognitive processes. This type of phenotyping could have important therapeutic implications, but its potential is yet to be realised in psychiatry applications. Further studies are warranted, not only to add to our understanding, but also to test whether results seen to date replicate or generalize. “Bespoke” Bayesian models and other approaches relevant to Gambling Disorder Standard RL, DDM, and more recently combined RL/DDM approaches are beginning to contribute towards our understanding of how learning and decision making are altered in Gambling Disorder. However, these approaches have limitations regarding the tasks or types of questions they are able to answer. RL models necessitate a learning component, whilst DDM models are limited to two-choice decisions. The combination of the two has leveraged the strength of the DDM in incorporating the rich amount of information contained in reaction times alongside the RL model advantage of 20 utilizing trial-by-trial information, but still requires specific conditions to be met for successful model-fitting. Often real-world gambling does not fit these constraints, and is influenced by a range of other cognitive constructs, as discussed in the introduction. For example, slot machines and other electronic gaming machines do not have a learning component, and often involve making choices between more than two options. It is also unclear how cognitive constructs such as impulsivity, compulsivity and conflict monitoring fit into model frameworks that predominantly measure learning and decision making. “Bespoke” Bayesian models and customisation of standard RL models provide methods to probe cognitive processing where multiple constructs may be contributing, task design does not allow for use of standard RL or DDM approaches, or previous results or hypotheses suggest more complex factors are involved than those usually incorporated into model frameworks. To our knowledge, there are not yet any examples of “bespoke” Bayesian modeling approaches that have been used with data from patients with Gambling Disorder, but there are a couple of studies exemplifying the use of tailored or customised computational approaches. Here, we highlight three studies using Bayesian methodologies whose results are relevant for gambling behaviors, and that demonstrate how this type of approach could be utilized in the future to directly study Gambling Disorder, and conclude with two examples of “bespoke” approaches that have been developed to answer questions about cognitive constructs with specific relevance to Gambling Disorder. In a proof-of concept study, Paliwal et al. (2014) showed that Bayesian modeling could be successfully applied to gambling decisions in a slot machine task with high ecological validity. Their task had features found in real-world slot machine play and that have been associated with problematic engagement in gambling: ‘near misses’ (Clark et al., 2009) and ‘losses disguised as wins’ (Dixon et al., 2010). Game play also included two levels of bet, the option to ‘double or nothing’ after jackpot wins, the ability to switch between four different machines, as well as players being able to ‘leave’ and ‘re-enter’ the casino. They applied Bayesian models that used trial-by-trial gambling outcomes to predict 21 when healthy volunteers would change their behavioral responses (indicative of ‘belief updating’), and correlated the best fitting model parameters with questionnaire measures of impulsivity. The two model parameters that predicted impulsivity scores were linked to uncertainty about how beliefs were updated throughout the task. The first, uncertainty about the chance of winning, correlated positively with impulsivity scores such that more impulsive players tended to perceive high levels of uncertainty, and this further correlated with increased variability on task behavioral measures. The second, uncertainty in the players precision estimate, which can be interpreted as an inverse measure of confidence, showed the opposite association. More impulsive participants were less confident in their precision, expecting greater volatility in their chances of winning. Together, these resulted in more erratic, riskier choices and suggest that impulsive gamblers might also be more sensitive to new information and changes (either genuine or illusory) in game structure (Paliwal et al., 2014). This study provides insight into how impulsivity and decision making mechanisms interact in gambling behavior, providing a novel advance over more traditional data analysis methods. The Balloon Analogue Risk Task (BART; Lejuez et al., 2002) is a cognitive test used to measure decision making and impulsivity with respect to an individual’s propensity for risk taking. Using a Bayesian updating model that learned risk probabilities and included a learning bias parameter that corresponds to how much beliefs shift across successive rounds of the task, Kim & Seo (2021) examined BART data from ADHD patients. They showed that overhasty updating of beliefs during risk learning on the task was related to questionnaire impulsivity measures, and specifically, that the learning bias model parameter suggested temporal impulsivity was driving the bias towards immediate versus delayed rewards (Kim & Seo, 2021). A similar approach has been used with the Cambridge Gambling Task (Rogers et al., 1999), a neurocognitive task commonly used to measure risky decision making and impulsivity in which the effect of learning or sequential influences are removed. Hierarchical Bayesian modeling and Bayesian model comparison were used to fit a set of cognitive models with differing 22 assumptions about choice behavior, and find which best fit data from control and participants with history of substance addiction (Romeu et al., 2020). The best fitting model had smaller values for a parameter representing a specific measure of impulsivity (reduced discounting of delayed bets) in the healthy control group, and also indicated differences in risk-seeking behavior, not just between the two groups overall, but also between participants who used a single substance compared to polysubstance cases (Romeu et al., 2020). Although these studies did not include patients with Gambling Disorder in their test group, the methodology could be readily applied to datasets from Gambling Disorder patients. Another strength of this type of approach is that model fits and parameter values could be used to compare similarities and differences in the cognitive components involved in impulsivity and decision making across different clinical conditions. Taken together, these studies highlight how Bayesian methodologies could be used to investigate cognitive constructs that are both unique to gambling (such as the near-miss effect and losses disguised as wins), but also link back to the broader constructs of decision making and impulsivity. A neglected aspect of decision making in relation to gambling behaviour is the processing of regret and counterfactual thinking. Various gambling products – most notably lottery games – leverage our tendencies to imagine alternative realities and missed opportunities. Extensive work in behavioural economics and decision neuroscience has operationalized this tendency with choice tasks in which, following the presentation of the (conventional) obtained outcome, the participant then sees the non-obtained outcome that they would have received, had they chosen the alternative gamble. Emotional ratings can capture regret as an intensification of negative affect when the non-obtained outcome is better than the received outcome (Mellers et al., 1997). In a case-control study of people with Gambling Disorder and healthy controls Wu et al. (2021) applied a computational choice model that included parameters corresponding to sensitivity to expected value, risk variance, and anticipated regret, to choices and affect ratings. The group with Gambling Disorder showed both reduced sensitivity to the 23 mathematical expected value of the gamble options, and increased preference for high risk (i.e. high outcome variance) gambles, as might be expected for this population. They further showed increased sensitivity to the anticipation of regret (i.e. avoiding options with a high potential to elicit regret). These preliminary results suggest that counterfactual thinking and regret may be under-studied influences in gambling behavior (Wu et al., 2021). As a final example, information seeking is another neglected component of goal-directed decision making that may be altered in Gambling Disorder (Wiehler et al., 2021) and other addictions (Morris et al., 2016), and is critical to the balance between exploration and exploitation in behavioural ecology (Addicott et al., 2017). In groups with Gambling Disorder and healthy controls, a computational model (customized from an RL formulation) assessed whether choices were driven by reward-seeking, novelty-seeking, or general information-seeking, (Dezza et al., 2021). Despite similar overall earnings on the task, the model revealed distinct information-seeking profiles, with gamblers displaying more general information-seeking and reduced novelty-seeking, with the opposite pattern in controls. Furthermore, these model parameters predicted the severity of gambling behavior (Dezza et al., 2021). These studies demonstrate the value of tailored modeling approaches in providing a more nuanced and in-depth understanding of cognition across a breadth of constructs relevant to Gambling Disorder. Although use of the same type of model can provide continuity across tasks and clinical disorders, a variety of frameworks and models is required to fully research the complexity of the processes involved in Gambling Disorder (and other mental health conditions). Modeling as a translational tool The modeling approaches discussed above have focused on human studies in which computational models have been applied to data from behavioral tasks to investigate the cognitive 24 processes implicated in mental health disorders, with a focus on Gambling Disorder. Although it is arguable whether psychiatric disorders can be fully replicated in animals, it is clear that animal models can be, and have been, successfully used to investigate some of the underlying cognitive and neural mechanisms. There are many human clinical tasks that are used to measure psychological constructs, such as decision making and impulsivity, that have direct counterparts for use in laboratory animals, including many of the tasks discussed above. For example, there are animal versions of delay and probability discounting tasks (Evenden et al., 1999; Cardinal & Howes, 2005); decision making tasks such as the rat Gambling Task (rGT; Zeeb et al., 2009) and other paradigms loosely based on the IGT (Rivalan et al., 2009; Simon et al., 2009; van den Bos et al., 2014); and versions of two-step task (Miller et al., 2017; Hasz & Redish, 2018; Groman et al., 2019). An important design decision in these studies is how to represent costs or losses in an animal model of gambling. The rGT and operant versions of the two-step task (Miller et al., 2017; Groman et al., 2019) use discrete, clearly signaled timeouts as negative outcomes, during which the animal is unable to gain further reward. In the maze-based version of the two-step task (Hasz & Reddish, 2018) varying delay to food delivery was used as a time cost during a limited length session. These are more tangible punishments/costs than merely the lack of a rewarding outcome, which is widely used in other procedures such as discounting tasks. Animal studies benefit from the ability of being able to directly manipulate brain function and neurochemistry more easily and specifically than in human studies. However, to date there has been less use of computational approaches in animal studies investigating the cognitive components involved in Gambling Disorder. As in humans, RL models have been used to investigate how choice behaviour on two-step tasks is dependent on model-free versus model-based strategies (Miller et al., 2017; Hasz & Redish, 2018; Groman et al., 2019). Results from these studies have been mixed. In an operant version of the two-step task that did not incorporate an actual choice at the second stage, Miller et al. (2017) found that rats mostly rely on model-based learning to complete the task, unlike in humans where a mix of model- 25 based and model-free strategies usually provides the best fit to the data (e.g. Daw et al., 2011). However, on a maze-based two-step task, rodent choices were best modeled by a combination of model-based and model-free learning (Hasz & Redish, 2018), a finding matched when an operant two-step task that did have two choice phases was used (Groman et al., 2019). As highlighted as a limitation in human studies, differences in task design may well contribute to the difference in findings, and is a challenge that should be addressed to improve generalizability of results across studies. Hasz & Redish (2018) also note that RL models were difficult to fit to rodent data, with some parameter values, particularly learning rates, being much lower than expected. A key obstacle in broadening the scope of computational modeling to rodent behavioral studies is ensuring that models used fit acceptably, and capture important features of the data. Despite this, these studies in rodents were able to demonstrate links between brain areas and neurotransmitters that have been implicated in Gambling Disorder and control of model-based learning in humans. Model-based learning was impaired by temporary inactivation of the OFC (Miller et al., 2017) and associated with dopamine levels in the OFC and ventral striatum (Groman et al., 2019). Further studies building upon these findings and guided by knowledge from human tasks conducted in patients with Gambling Disorder will be critical to continue to develop our understanding of how learning and decision making contribute to gambling behaviors. In the rGT, RL modeling revealed that risk preference was driven by relative insensitivity to the cost of the punishments, as opposed to increased sensitivity to larger rewards (Langdon et al., 2019). In another rodent model of the IGT, poor decision making was linked to high scores on risk-taking, reward-seeking, motor-impulsivity and behavioral flexibility in individual rats, and an RL model was used to confirm that these behavioral traits resulted in poor representation of the values and costs of rewards and punishments, which led to inefficient learning about choice outcomes (Rivalan et al., 2013). These results reflect similar findings in humans, linking impulsivity and altered responses to cues predicting positive and negative outcomes to poor performance on the IGT in gamblers (Brevers et al., 2013). 26 Perhaps because the use of computational approaches in pre-clinical studies is still uncommon, the potential utility of using model parameters to more directly compare and contrast performance between rodents and humans on behavioral tasks is mostly yet to be investigated. In many animal models and behavioral tasks there is debate as to how closely performance in non-human animals recapitulates that in humans. Using modeling approaches to identify whether the same or different cognitive constructs (e.g. changes in learning rates in RL models, or changes in specific DDM parameters) underlie behavior measured across species could provide a way to overcome some of the challenges present in translating findings from animal research to the clinic. This was highlighted earlier in the DDM section, where similar models were used across different tasks and test populations, but common changes in parameters were found. Increased application of modeling approaches in animal studies has untapped potential at a more basic level for providing further insight into the cognitive processes that drive different behaviors measured in tasks. For example, modeling could be useful in probing the cognitive processes that differ during decisions about gains versus losses, as opposed to gains versus no-gains. Furthermore, some of the rodent tasks highlighted above include other behavioral measures, such as premature responding (a measure of impulsivity), that have so far not yet been incorporated into model analyses. As with human studies, expansion of the modeling approaches used with rodent data through the application of Bayesian methodologies to incorporate more cognitive components, or alternative strategies such as use of heuristics or the influence of Pavlovian biases, could improve model fits and aid comparison with results from human tasks. Maladaptive Pavlovian learning about stimulus-outcome associations have been posited to play a key role in addictions (Meyer et al., 2016), but studies investigating how Pavlovian biases may interact with instrumental learning about choice outcomes, the neurobiology underlying this, and studies in Gambling Disorder are scarce (Dayan & Berridge, 2014). Modeling approaches provide an ideal method for investigating this, and in combination with animal studies has 27 the added advantage of allowing investigation into the neural circuitry involved in how these decisions are implemented in the brain. Conclusions It is encouraging that in recent years there has been an increase in the use of computational modeling approaches across multiple frameworks investigating the cognitive mechanisms relevant to gambling behaviour and Gambling Disorder. Across studies, there is at least some convergence building around the cognitive processes that are associated with model parameters, and in turn with gambling behavior. Expansions of these methodologies, application to more types of behavioral tasks, and increased cross-talk in the field to link together results from different modeling approaches will be key going forwards, to ensure that the significance of these findings is realized – not just with relevance to Gambling Disorder, but also more broadly across other mental health disorders. A key challenge will be to utilize this information not only to provide insight into the complexity of the cognitive processes underlying gambling behavior, but also to help to better characterize and understand the heterogeneity in Gambling Disorder in terms of clinical characteristics, preferred forms of gambling, demographic and psychosocial variables, and motivations to gamble. To date in psychiatry, computational modeling in psychiatry has yet to directly influence clinical practice, either in terms of treatment options or as a phenotyping tool. There is also potential for computational psychiatry approaches to be exploited to identify the cognitive processes driving maladaptive learning and decision making in an individual, and then used to develop personalized interventions tailored to impact that individual's specific cognitive profile. Progress in these directions will likely require the development of more complex, sophisticated modeling approaches that incorporate features such as variation across time and contexts, alongside the behavioral and neural measures commonly used currently (Hitchcock et al., 2022). 28 It is clear that much that could be gained from applying these types of approaches to preclinical data in the future, as this is an area where the use of computational methods is currently lacking. Along with the direct manipulation of neural circuits and neurotransmitter systems that is possible in preclinical studies, a key advantage of computational models may be to aid with the translation of preclinical findings to useful therapeutics that can be applied in clinical settings through direct comparison of model parameters across species. A shift towards the use of computational models to probe the cognitive mechanisms that are involved in healthy behaviors and those that are disrupted across multiple mental health disorders will allow new insight into some of the current complex questions in the fields of psychiatry, psychology and neuroscience. 29 References Adams, R. A., Huys, Q. J., & Roiser, J. P. (2016). Computational Psychiatry: towards a mathematically informed understanding of mental illness. Journal of Neurology, Neurosurgery, and Psychiatry, 87(1), 53–63. https://doi.org/10.1136/jnnp-2015-310737 Addicott, M. A., Pearson, J. M., Sweitzer, M. M., Barack, D. L., & Platt, M. L. (2017). A Primer on Foraging and the Explore/Exploit Trade-Off for Psychiatry Research. Neuropsychopharmacology, 42(10), 1931–1939. https://doi.org/10.1038/npp.2017.108 Ahn, W. Y., Dai, J., Vassileva, J., Busemeyer, J. R., & Stout, J. C. (2016). Computational modeling for addiction medicine: From cognitive models to clinical applications. Progress in Brain Research, 224, 53–65. https://doi.org/10.1016/bs.pbr.2015.07.032 Annis, J., & Palmeri, T. J. (2018). Bayesian statistical approaches to evaluating cognitive models. Wiley interdisciplinary reviews. Cognitive Science, 9(2), 10.1002/wcs.1458. https://doi.org/10.1002/wcs.1458 APA (2013) Diagnostic and Statistical Manual 5, 5th ed. American Psychiatric Press, Washington, D.C. Aston-Jones, G., & Cohen, J. D. (2005). Adaptive gain and the role of the locus coeruleus-norepinephrine system in optimal performance. The Journal of Comparative Neurology, 493(1), 99–110. https://doi.org/10.1002/cne.20723 Bechara A. (2005). Decision making, impulse control and loss of willpower to resist drugs: a neurocognitive perspective. Nature Neuroscience, 8(11), 1458–1463. https://doi.org/10.1038/nn1584 Bechara, A., Damasio, A. R., Damasio, H., & Anderson, S. W. (1994). Insensitivity to future consequences following damage to human prefrontal cortex. Cognition, 50(1-3), 7–15. https://doi.org/10.1016/0010-0277(94)90018-3 Bechara, A., Damasio, A. R., Damasio, H., & Anderson, S. W. (1994). Insensitivity to future consequences following damage to human prefrontal cortex. Cognition, 50(1-3), 7–15. https://doi.org/10.1016/0010-0277(94)90018-3 Bogacz, R., Brown, E., Moehlis, J., Holmes, P., & Cohen, J. D. (2006). The physics of optimal decision making: a formal analysis of models of performance in two-alternative forced-choice tasks. Psychological Review, 113(4), 700–765. https://doi.org/10.1037/0033-295X.113.4.700 Brevers, D., Bechara, A., Cleeremans, A., & Noël, X. (2013). Iowa Gambling Task (IGT): twenty years after - Gambling Disorder and IGT. Frontiers in Psychology, 4, 665. https://doi.org/10.3389/fpsyg.2013.00665 Bruder, L. R., Scharer, L., & Peters, J. (2021). Reliability assessment of temporal discounting measures in virtual reality environments. Scientific Reports, 11(1), 7015. https://doi.org/10.1038/s41598-021-86388-8 Busemeyer, J. R., & Johnson, J. G. (2004). Computational models of decision making. Cambridge, MA: Blackwell. Busemeyer, J. R., & Stout, J. C. (2002). A contribution of cognitive decision models to clinical assessment: decomposing performance on the Bechara gambling task. Psychological Assessment, 14(3), 253–262. https://doi.org/10.1037//1040-3590.14.3.253 30 Cardinal, R. N., & Howes, N. J. (2005). Effects of lesions of the nucleus accumbens core on choice between small certain rewards and large uncertain rewards in rats. BMC Neuroscience, 6, 37. https://doi.org/10.1186/1471-2202-6-37 Caswell, A. J., Bond, R., Duka, T., & Morgan, M. J. (2015). Further evidence of the heterogeneous nature of impulsivity. Personality and Individual Differences, 76, 68–74. https://doi.org/10.1016/j.paid.2014.11.059 Clark L. (2010). Decision-making during gambling: an integration of cognitive and psychobiological approaches. Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences, 365(1538), 319–330. https://doi.org/10.1098/rstb.2009.0147 Clark, L., Averbeck, B., Payer, D., Sescousse, G., Winstanley, C. A., & Xue, G. (2013). Pathological choice: the neuroscience of gambling and gambling addiction. The Journal of Neuroscience, 33(45), 17617–17623. https://doi.org/10.1523/JNEUROSCI.3231-13.2013 Clark, L., Billieux, J., & Ledgerwood, D.M. (2022) Introduction to the Special Issue “20 years of the Pathways Model: understanding disordered gambling and other behavioural addictions”, International Gambling Studies, 22:2, 183–187, https://doi.org/10.1080/14459795.2022.2108484 Clark, L., Cools, R., & Robbins, T. W. (2004). The neuropsychology of ventral prefrontal cortex: decision-making and reversal learning. Brain and Cognition, 55(1), 41–53. https://doi.org/10.1016/S0278-2626(03)00284-7 Clark, L., Lawrence, A. J., Astley-Jones, F., & Gray, N. (2009). Gambling near-misses enhance motivation to gamble and recruit win-related brain circuitry. Neuron, 61(3), 481–490. https://doi.org/10.1016/j.neuron.2008.12.031 Clay, S. N., Clithero, J. A., Harris, A. M., & Reed, C. L. (2017). Loss Aversion Reflects Information Accumulation, Not Bias: A Drift-Diffusion Model Study. Frontiers in Psychology, 8, 1708. https://doi.org/10.3389/fpsyg.2017.01708 Colas, F., Diard, J., & Bessière, P. (2010). Common Bayesian models for common cognitive issues. Acta Biotheoretica, 58(2-3), 191–216. https://doi.org/10.1007/s10441-010-9101-1 Daw, N. D., Gershman, S. J., Seymour, B., Dayan, P., & Dolan, R. J. (2011). Model-based influences on humans' choices and striatal prediction errors. Neuron, 69(6), 1204–1215. https://doi.org/10.1016/j.neuron.2011.02.027 Daw, N. D., Niv, Y., & Dayan, P. (2005). Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nature Neuroscience, 8(12), 1704–1711. https://doi.org/10.1038/nn1560 Dayan, P., & Berridge, K. C. (2014). Model-based and model-free Pavlovian reward learning: revaluation, revision, and revelation. Cognitive, Affective & Behavioral Neuroscience, 14(2), 473–492. https://doi.org/10.3758/s13415-014-0277-8 Dezza, I. C., Noel, X., Cleeremans, A., & Yu, A. J. (2021). Distinct motivations to seek out information in healthy individuals and problem gamblers. Translational Psychiatry, 11(1), 408. https://doi.org/10.1038/s41398-021-01523-3 31 Dixon, M. J., Harrigan, K. A., Snadhu, R., Collins, K., & Fugelsang, J. A. (2010). Losses disguised as wins in modern multi-line video slot machines. Addiction, 105, 1819-1824. https://doi.org/10.1111/j.1360-0443.2010.03050.x Doya K. (2008). Modulators of decision making. Nature Neuroscience, 11(4), 410–416. https://doi.org/10.1038/nn2077 Dutilh, G., & Rieskamp, J. (2016). Comparing perceptual and preferential decision making. Psychonomic Bulletin & Review, 23(3), 723–737. https://doi.org/10.3758/s13423-015-0941-1 Evenden J. L. (1999). Varieties of impulsivity. Psychopharmacology, 146(4), 348–361. https://doi.org/10.1007/pl00005481 Everitt, B. J., & Robbins, T. W. (2005). Neural systems of reinforcement for drug addiction: from actions to habits to compulsion. Nature Neuroscience, 8(11), 1481–1489. https://doi.org/10.1038/nn1579 Everitt, B. J., & Robbins, T. W. (2016). Drug Addiction: Updating Actions to Habits to Compulsions Ten Years On. Annual review of psychology, 67, 23–50. https://doi.org/10.1146/annurev-psych-122414-033457 Fontanesi, L., Gluth, S., Spektor, M. S., & Rieskamp, J. (2019). A reinforcement learning diffusion decision model for value-based decisions. Psychonomic Bulletin & Review, 26(4), 1099–1121. https://doi.org/10.3758/s13423-018-1554-2 Ford, M., & Håkansson, A. (2020). Problem gambling, associations with comorbid health conditions, substance use, and behavioural addictions: Opportunities for pathways to treatment. PloS One, 15(1), e0227644. https://doi.org/10.1371/journal.pone.0227644 Forstmann, B. U., Ratcliff, R., & Wagenmakers, E. J. (2016). Sequential Sampling Models in Cognitive Neuroscience: Advantages, Applications, and Extensions. Annual Review of Psychology, 67, 641–666. https://doi.org/10.1146/annurev-psych-122414-033645 Franken, I. H., van Strien, J. W., Nijs, I., & Muris, P. (2008). Impulsivity is associated with behavioral decision-making deficits. Psychiatry Research, 158(2), 155–163. https://doi.org/10.1016/j.psychres.2007.06.002 Groman, S. M., Massi, B., Mathias, S. R., Curry, D. W., Lee, D., & Taylor, J. R. (2019). Neurochemical and Behavioral Dissections of Decision-Making in a Rodent Multistage Task. The Journal of Neuroscience, 39(2), 295–306. https://doi.org/10.1523/JNEUROSCI.2219-18.2018 Groman, S. M., Thompson, S. L., Lee, D., & Taylor, J. R. (2022). Reinforcement learning detuned in addiction: integrative and translational approaches. Trends in Neurosciences, 45(2), 96–105. https://doi.org/10.1016/j.tins.2021.11.007 Hasz, B. M., & Redish, A. D. (2018). Deliberation and Procedural Automation on a Two-Step Task for Rats. Frontiers in Integrative Neuroscience, 12, 30. https://doi.org/10.3389/fnint.2018.00030 Hitchcock, P. F., Fried, E. I., & Frank, M. J. (2022). Computational Psychiatry Needs Time and Context. Annual Review of Psychology, 73, 243–270. https://doi.org/10.1146/annurev-psych-021621-124910 32 Huys, Q., Browning, M., Paulus, M. P., & Frank, M. J. (2021). Advances in the computational understanding of mental illness. Neuropsychopharmacology, 46(1), 3–19. https://doi.org/10.1038/s41386-020-0746-4 Ioannidis, K., Hook, R., Wickham, K., Grant, J. E., & Chamberlain, S. R. (2019). Impulsivity in Gambling Disorder and problem gambling: a meta-analysis. Neuropsychopharmacology, 44(8), 1354–1361. https://doi.org/10.1038/s41386-019-0393-9 Keiflin, R., & Janak, P. H. (2015). Dopamine Prediction Errors in Reward Learning and Addiction: From Theory to Neural Circuitry. Neuron, 88(2), 247–263. https://doi.org/10.1016/j.neuron.2015.08.037 Kim, M., & Seo, J. (2021). Impulsivity is related to overhasty risk learning in attention-deficit/hyperactivity disorder: A computational psychiatric approach. Journal of Psychiatric Research, 143, 84–90. https://doi.org/10.1016/j.jpsychires.2021.07.044 Kovács, I., Richman, M. J., Janka, Z., Maraz, A., & Andó, B. (2017). Decision making measured by the Iowa Gambling Task in alcohol use disorder and Gambling Disorder: a systematic review and meta-analysis. Drug and Alcohol Dependence, 181, 152–161. https://doi.org/10.1016/j.drugalcdep.2017.09.023 Langdon, A. J., Hathaway, B. A., Zorowitz, S., Harris, C., & Winstanley, C. A. (2019). Relative insensitivity to time-out punishments induced by win-paired cues in a rat gambling task. Psychopharmacology, 236(8), 2543–2556. https://doi.org/10.1007/s00213-019-05308-x Lejuez, C. W., Read, J. P., Kahler, C. W., Richards, J. B., Ramsey, S. E., Stuart, G. L., Strong, D. R., & Brown, R. A. (2002). Evaluation of a behavioral measure of risk taking: the Balloon Analogue Risk Task (BART). Journal of Experimental Psychology. Applied, 8(2), 75–84. https://doi.org/10.1037//1076-898x.8.2.75 Li, Y., Wang, Z., Boileau, I., Dreher, J. C., Gelskov, S., Genauck, A., Joutsa, J., Kaasinen, V., Perales, J. C., Romanczuk-Seiferth, N., Ruiz de Lara, C. M., Siebner, H. R., van Holst, R. J., van Timmeren, T., & Sescousse, G. (2019). Altered orbitofrontal sulcogyral patterns in Gambling Disorder: a multicenter study. Translational Psychiatry, 9(1), 186. https://doi.org/10.1038/s41398-019-0520-8 Lieb, K., Zanarini, M. C., Schmahl, C., Linehan, M. M., & Bohus, M. (2004). Borderline personality disorder. Lancet (London, England), 364(9432), 453–461. https://doi.org/10.1016/S0140-6736(04)16770-6 Lim, M. S., Jocham, G., Hunt, L. T., Behrens, T. E., & Rogers, R. D. (2015). Impulsivity and predictive control are associated with suboptimal action-selection and action-value learning in regular gamblers. International Gambling Studies, 15(3), 489–505. https://doi.org/10.1080/14459795.2015.1078835 Liu, Z., Liu, S., Li, S., Li, L., Zheng, L., Weng, X., Guo, X., Lu, Y., Men, W., Gao, J., & You, X. (2022). Dissociating Value-Based Neurocomputation from Subsequent Selection-Related Activations in Human Decision-Making. Cerebral Cortex, bhab471. Advance online publication. https://doi.org/10.1093/cercor/bhab471 Maia, T. V., & Frank, M. J. (2011). From reinforcement learning models to psychiatric and neurological disorders. Nature Neuroscience, 14(2), 154–162. https://doi.org/10.1038/nn.2723 33 Mellers, B. A., Schwartz, A., Ho, K., & Ritov, I. (1997). Decision Affect Theory: Emotional Reactions to the Outcomes of Risky Options. Psychological Science, 8(6), 423–429. https://doi.org/10.1111/j.1467-9280.1997.tb00455.x Meyer, P. J., King, C. P., & Ferrario, C. R. (2016). Motivational Processes Underlying Substance Abuse Disorder. Current Topics in Behavioral Neurosciences, 27, 473–506. https://doi.org/10.1007/7854_2015_391 Miedl, S. F., Büchel, C., & Peters, J. (2014). Cue-induced craving increases impulsivity via changes in striatal value signals in problem gamblers. The Journal of Neuroscience, 34(13), 4750–4755. https://doi.org/10.1523/JNEUROSCI.5020-13.2014 Miletić, S., Boag, R. J., & Forstmann, B. U. (2020). Mutual benefits: Combining reinforcement learning with sequential sampling models. Neuropsychologia, 136, 107261. https://doi.org/10.1016/j.neuropsychologia.2019.107261 Miller, K. J., Botvinick, M. M., & Brody, C. D. (2017). Dorsal hippocampus contributes to model-based planning. Nature Neuroscience, 20(9), 1269–1276. https://doi.org/10.1038/nn.4613 Milosavljevic, M., Malmaud, J., Huth, A., Koch, C., & Rangel, A. (2010). The drift diffusion model can account for the accuracy and reaction time of value-based choices under high and low time pressure. Judgment and Decision Making, 5(6), 437–449. https://doi.org/10.1017/S1930297500001285 Moccia, L., Pettorruso, M., De Crescenzo, F., De Risio, L., di Nuzzo, L., Martinotti, G., Bifone, A., Janiri, L., & Di Nicola, M. (2017). Neural correlates of cognitive control in Gambling Disorder: a systematic review of fMRI studies. Neuroscience and Biobehavioral Reviews, 78, 104–116. https://doi.org/10.1016/j.neubiorev.2017.04.025 Moeller, F. G., Barratt, E. S., Dougherty, D. M., Schmitz, J. M., & Swann, A. C. (2001). Psychiatric aspects of impulsivity. The American Journal of Psychiatry, 158(11), 1783–1793. https://doi.org/10.1176/appi.ajp.158.11.1783 Morris, L. S., Baek, K., Kundu, P., Harrison, N. A., Frank, M. J., & Voon, V. (2016). Biases in the Explore-Exploit Tradeoff in Addictions: The Role of Avoidance of Uncertainty. Neuropsychopharmacology, 41(4), 940–948. https://doi.org/10.1038/npp.2015.208 Paliwal, S., Petzschner, F. H., Schmitz, A. K., Tittgemeyer, M., & Stephan, K. E. (2014). A model-based analysis of impulsivity using a slot-machine gambling paradigm. Frontiers in Human Neuroscience, 8, 428. https://doi.org/10.3389/fnhum.2014.00428 Pedersen, M. L., & Frank, M. J. (2020). Simultaneous Hierarchical Bayesian Parameter Estimation for Reinforcement Learning and Drift Diffusion Models: a Tutorial and Links to Neural Data. Computational Brain & Behavior, 3(4), 458–471. https://doi.org/10.1007/s42113-020-00084-w Pedersen, M. L., Frank, M. J., & Biele, G. (2017). The drift diffusion model as the choice rule in reinforcement learning. Psychonomic Bulletin & Review, 24(4), 1234–1251. https://doi.org/10.3758/s13423-016-1199-y 34 Perry, J. L., & Carroll, M. E. (2008). The role of impulsive behavior in drug abuse. Psychopharmacology, 200(1), 1–26. https://doi.org/10.1007/s00213-008-1173-0 Peters, J., & D'Esposito, M. (2020). The drift diffusion model as the choice rule in inter-temporal and risky choice: A case study in medial orbitofrontal cortex lesion patients and controls. PLoS Computational Biology, 16(4), e1007615. https://doi.org/10.1371/journal.pcbi.1007615 Peters, J., Vega, T., Weinstein, D., Mitchell, J., & Kayser, A. (2020). Dopamine and Risky Decision-Making in Gambling Disorder. eNeuro, 7(3), ENEURO.0461-19.2020. https://doi.org/10.1523/ENEURO.0461-19.2020 Petry, N. M., Blanco, C., Auriacombe, M., Borges, G., Bucholz, K., Crowley, T. J., Grant, B. F., Hasin, D. S., & O'Brien, C. (2014). An overview of and rationale for changes proposed for pathological gambling in DSM-5. Journal of Gambling Studies, 30(2), 493–502. https://doi.org/10.1007/s10899-013-9370-0 Petry, N. M., Stinson, F. S., & Grant, B. F. (2005). Comorbidity of DSM-IV pathological gambling and other psychiatric disorders: results from the National Epidemiologic Survey on Alcohol and Related Conditions. The Journal of Clinical Psychiatry, 66(5), 564–574. https://doi.org/10.4088/jcp.v66n0504 Ratcliff, R. (1978). A theory of memory retrieval. Psychological Review 85:59-108. Ratcliff, R., & Tuerlinckx, F. (2002). Estimating parameters of the diffusion model: approaches to dealing with contaminant reaction times and parameter variability. Psychonomic Bulletin & Review, 9(3), 438–481. https://doi.org/10.3758/bf03196302 Reber, A.S. (2012). The EVF Model: A Novel Framework for Understanding Gambling and, by Extension, Poker. UNLV Gaming Research & Review Journal, 16, 59-76. Redish, A. D., Jensen, S., Johnson, A., & Kurth-Nelson, Z. (2007). Reconciling reinforcement learning models with behavioral extinction and renewal: implications for addiction, relapse, and problem gambling. Psychological Review, 114(3), 784–805. https://doi.org/10.1037/0033-295X.114.3.784 Rescorla M. (2021). Bayesian modeling of the mind: From norms to neurons. Wiley interdisciplinary reviews. Cognitive Science, 12(1), e1540. https://doi.org/10.1002/wcs.1540 Rivalan, M., Ahmed, S. H., & Dellu-Hagedorn, F. (2009). Risk-prone individuals prefer the wrong options on a rat version of the Iowa Gambling Task. Biological Psychiatry, 66(8), 743–749. https://doi.org/10.1016/j.biopsych.2009.04.008 Rivalan, M., Valton, V., Seriès, P., Marchand, A. R., & Dellu-Hagedorn, F. (2013). Elucidating poor decision-making in a rat gambling task. PloS One, 8(12), e82052. https://doi.org/10.1371/journal.pone.0082052 Rogers R. D. (2011). The roles of dopamine and serotonin in decision making: evidence from pharmacological experiments in humans. Neuropsychopharmacology, 36(1), 114–132. https://doi.org/10.1038/npp.2010.165 Rogers, R. D., Everitt, B. J., Baldacchino, A., Blackshaw, A. J., Swainson, R., Wynne, K., Baker, N. B., Hunter, J., Carthy, T., Booker, E., London, M., Deakin, J. F., Sahakian, B. J., & Robbins, T. W. (1999). Dissociable deficits in the decision-making cognition of chronic amphetamine abusers, opiate abusers, 35 patients with focal damage to prefrontal cortex, and tryptophan-depleted normal volunteers: evidence for monoaminergic mechanisms. Neuropsychopharmacology, 20(4), 322–339. https://doi.org/10.1016/S0893-133X(98)00091-8 Romeu, R. J., Haines, N., Ahn, W. Y., Busemeyer, J. R., & Vassileva, J. (2020). A computational model of the Cambridge gambling task with applications to substance use disorders. Drug and Alcohol Dependence, 206, 107711. https://doi.org/10.1016/j.drugalcdep.2019.107711 Rushworth, M. F., Noonan, M. P., Boorman, E. D., Walton, M. E., & Behrens, T. E. (2011). Frontal cortex and reward-guided learning and decision-making. Neuron, 70(6), 1054–1069. https://doi.org/10.1016/j.neuron.2011.05.014 Schultz, W., Dayan, P., & Montague, P. R. (1997). A neural substrate of prediction and reward. Science, 411 275(5306), 1593-1599. Schwabe, L., Haddad, L., & Schachinger, H. (2008). HPA axis activation by a socially evaluated cold-pressor test. Psychoneuroendocrinology, 33(6), 890–895. https://doi.org/10.1016/j.psyneuen.2008.03.001 Sharma, A., & Couture, J. (2014). A review of the pathophysiology, etiology, and treatment of attention-deficit hyperactivity disorder (ADHD). The Annals of Pharmacotherapy, 48(2), 209–225. https://doi.org/10.1177/1060028013510699 Simon, N. W., Gilbert, R. J., Mayse, J. D., Bizon, J. L., & Setlow, B. (2009). Balancing risk and reward: a rat model of risky decision making. Neuropsychopharmacology, 34(10), 2208–2217. https://doi.org/10.1038/npp.2009.48 Steingroever, H., Pachur, T., Šmíra, M., & Lee, M. D. (2018). Bayesian techniques for analyzing group differences in the Iowa Gambling Task: A case study of intuitive and deliberate decision-makers. Psychonomic Bulletin & Review, 25(3), 951–970. https://doi.org/10.3758/s13423-017-1331-7 Steingroever, H., Wetzels, R., & Wagenmakers, E.-J. (2014). Absolute performance of reinforcement-learning models for the Iowa Gambling Task. Decision, 1(3), 161–183. https://doi.org/10.1037/dec0000005 Steingroever, H., Wetzels, R., & Wagenmakers, E.-J. (2016). Bayes factors for reinforcement-learning models of the Iowa gambling task. Decision, 3(2), 115–131. https://doi.org/10.1037/dec0000040 Sutton, R. S., & Barto, A. G. (1998). Reinforcement Learning: An Introduction. Cambridge, MA: MIT Press. Tajima, S., Drugowitsch, J., & Pouget, A. (2016). Optimal policy for value-based decision-making. Nature Communications, 7, 12400. https://doi.org/10.1038/ncomms12400 Tanabe, J., Thompson, L., Claus, E., Dalwani, M., Hutchison, K. and Banich, M.T. (2007), Prefrontal cortex activity is reduced in gambling and nongambling substance users during decision-making. Human Brain Mapping, 28, 1276-1286. https://doi.org/10.1002/hbm.20344 van den Bos, R., Koot, S., & de Visser, L. (2014). A rodent version of the Iowa Gambling Task: 7 years of progress. Frontiers in Psychology, 5, 203. https://doi.org/10.3389/fpsyg.2014.00203 36 van Holst, R. J., van den Brink, W., Veltman, D. J., & Goudriaan, A. E. (2010). Why gamblers fail to win: a review of cognitive and neuroimaging findings in pathological gambling. Neuroscience and Biobehavioral Reviews, 34(1), 87–107. https://doi.org/10.1016/j.neubiorev.2009.07.007 van Timmeren, T., Daams, J. G., van Holst, R. J., & Goudriaan, A. E. (2018). Compulsivity-related neurocognitive performance deficits in Gambling Disorder: A systematic review and meta-analysis. Neuroscience and Biobehavioral Reviews, 84, 204–217. https://doi.org/10.1016/j.neubiorev.2017.11.022 van Timmeren, T., Piray, P., Goudriaan, A., van Holst, R. (2022) Goal-directed and habitual decision making under stress in Gambling Disorder: an fMRI study. PsyArXiv. https://doi.org/10.31234/osf.io/pcund Voon, V., Derbyshire, K., Rück, C., Irvine, M. A., Worbe, Y., Enander, J., Schreiber, L. R., Gillan, C., Fineberg, N. A., Sahakian, B. J., Robbins, T. W., Harrison, N. A., Wood, J., Daw, N. D., Dayan, P., Grant, J. E., & Bullmore, E. T. (2015). Disorders of compulsivity: a common bias towards learning habits. Molecular Psychiatry, 20(3), 345–352. https://doi.org/10.1038/mp.2014.44 Voon, V., Joutsa, J., Majuri, J., Baek, K., Nord, C. L., Arponen, E., Forsback, S., & Kaasinen, V. (2020). The neurochemical substrates of habitual and goal-directed control. Translational Psychiatry, 10(1), 84. https://doi.org/10.1038/s41398-020-0762-5 Voss, A., Nagler, M., & Lerche, V. (2013). Diffusion models in experimental psychology: a practical introduction. Experimental Psychology, 60(6), 385–402. https://doi.org/10.1027/1618-3169/a000218 Wagner, B., Mathar, D., & Peters, J. (2022). Gambling Environment Exposure Increases Temporal Discounting but Improves Model-Based Control in Regular Slot-Machine Gamblers. Computational Psychiatry, 6(1), 142–165. http://doi.org/10.5334/cpsy.84 Wiecki, T. V., Sofer, I., & Frank, M. J. (2013). HDDM: Hierarchical Bayesian estimation of the Drift-Diffusion Model in Python. Frontiers in Neuroinformatics, 7, 14. https://doi.org/10.3389/fninf.2013.00014 Wiehler, A., & Peters, J. (2015). Reward-based decision making in pathological gambling: the roles of risk and delay. Neuroscience Research, 90, 3–14. https://doi.org/10.1016/j.neures.2014.09.008 Wiehler, A., & Peters, J. (2015). Reward-based decision making in pathological gambling: the roles of risk and delay. Neuroscience Research, 90, 3–14. https://doi.org/10.1016/j.neures.2014.09.008 Wiehler, A., & Peters, J. (2020). Diffusion modeling reveals reinforcement learning impairments in Gambling Disorder that are linked to attenuated ventromedial prefrontal cortex value representations. BioRxiv, 2020.06.03.131359. https://doi.org/10.1101/2020.06.03.131359 Wiehler, A., Chakroun, K., & Peters, J. (2021). Attenuated Directed Exploration during Reinforcement Learning in Gambling Disorder. The Journal of Neuroscience, 41(11), 2512–2522. https://doi.org/10.1523/JNEUROSCI.1607-20.2021 Wu, Y., Kennedy, D., Goshko, C. B., & Clark, L. (2021). "Should've known better": Counterfactual processing in disordered gambling. Addictive Behaviors, 112, 106622. https://doi.org/10.1016/j.addbeh.2020.106622 37 Wyckmans, F., Banerjee, N., Saeremans, M., Otto, R., Kornreich, C., Vanderijst, L., Gruson, D., Carbone, V., Bechara, A., Buchanan, T., & Noël, X. (2022). The modulation of acute stress on model-free and model-based reinforcement learning in Gambling Disorder. Journal of Behavioral Addictions, 11(3), 831–844. https://doi.org/10.1556/2006.2022.00059 Wyckmans, F., Otto, A. R., Sebold, M., Daw, N., Bechara, A., Saeremans, M., Kornreich, C., Chatard, A., Jaafari, N., & Noël, X. (2019). Reduced model-based decision-making in Gambling Disorder. Scientific Reports, 9(1), 19625. https://doi.org/10.1038/s41598-019-56161-z Yechiam, E., Kanz, J. E., Bechara, A., Stout, J. C., Busemeyer, J. R., Altmaier, E. M., & Paulsen, J. S. (2008). Neurocognitive deficits related to poor decision making in people behind bars. Psychonomic bulletin & review, 15(1), 44–51. https://doi.org/10.3758/pbr.15.1.44 Zeeb, F. D., Robbins, T. W., & Winstanley, C. A. (2009). Serotonergic and dopaminergic modulation of gambling behavior as assessed using a novel rat gambling task. Neuropsychopharmacology, 34(10), 2329–2343. https://doi.org/10.1038/npp.2009.62 Zhao, W. J., Walasek, L., & Bhatia, S. (2020). Psychological mechanisms of loss aversion: A drift-diffusion decomposition. Cognitive Psychology, 123, 101331. https://doi.org/10.1016/j.cogpsych.2020.101331
Article
Postprint
eng
Reviewed
Vancouver : University of British Columbia Library
Attribution-NonCommercial-NoDerivatives 4.0 International
Faculty
Postdoctoral
Computational approaches to modeling gambling behaviour : opportunities for understanding disordered gambling
Text