Algorithmic learning in games
department or school: Arts, Faculty of
department or school: Vancouver School of Economics
data provider: DSpace
degree campus: UBCV
creator: Possnig, Clemens
issued: 2023-09-07T23:09:38Z
issued: 2023
related degree: Doctor of Philosophy - PhD
degree grantor: University of British Columbia
description: This dissertation studies algorithmic agents that interact repeatedly in strategic settings. Chapter 2 provides asymptotic results for a family of reinforcement learning algorithms known as ‘actor-critic learners’. Each such algorithmic agent simultaneously estimates what is called a ‘critic’, such as a value function, and updates its policy, which is referred to as the ‘actor’. The critic is used to indicate directions of improvement for the actor. I establish sufficient conditions for the consistency of each agent’s parametric critic estimator, which enables them to adapt and find optimal responses despite the non-stationarity inherent to multi-agent settings. The conditions depend on the environment, number of observations used in the critic estimation, and policy stepsize. Chapter 3 presents an analytical characterization of the long run policies learned by algorithmic agents in the multi-agent setting. The algorithms studied here form a superset of the family considered in chapter 2. These algorithms update policies, which are maps from observed states to actions. I show that the long run policies correspond to equilibria that are stable points of a tractable differential equation. In chapter 4, I consider algorithmic agents playing a repeated Cournot game of quantity competition. In this situation, learning the stage game Nash equilibrium serves as noncollusive benchmark. I give necessary and sufficient conditions for this Nash equilibrium not to be learned. These conditions are requirements on the state variables of the algorithms, and on the stage game. When algorithms determine actions based only on the past period’s price, the Nash equilibrium can be learned. However, agents may condition their actions on richer types of state variables beyond the past period’s price. In that case, I give sufficient conditions such that the policies converge to a collusive equilibrium with positive probability, while never converging to the Nash equilibrium.
aggregated CHO: https://circle.library.ubc.ca/rest/handle/2429/85877
note: Algorithmic Learning in GamesbyClemens PossnigB.A., Karl Franzens University, 2014M.Sc., Institute for Advanced Studies, Vienna, 2016A THESIS SUBMITTED IN PARTIAL FULFILLMENT OFTHE REQUIREMENTS FOR THE DEGREE OFDOCTOR OF PHILOSOPHYinThe Faculty of Graduate and Postdoctoral Studies(Economics)THE UNIVERSITY OF BRITISH COLUMBIA(Vancouver)September 2023© Clemens Possnig, 2023The following individuals certify that they have read, and recommend to the Faculty of Grad-uate and Postdoctoral Studies for acceptance, the dissertation titled:Algorithmic Learning in Gamessubmitted by Clemens Possnig in partial fulfillment of the requirements forthe degree of Doctor of Philosophyin EconomicsExamining Committee:Hao Li, Professor, Economics, UBCCo-SupervisorVitor Farinha Luz, Associate Professor, Economics, UBCCo-SupervisorChristopher Thomas Ryan, Associate Professor, Operations and Logistics Division, UBCUniversity ExaminerKevin Song, Professor, Economics, UBCUniversity ExaminerAdditional Supervisory Committee Members:Michael Peters, Professor, Economics, UBCSupervisory Committee MemberiiAbstractThis dissertation studies algorithmic agents that interact repeatedly in strategic settings.Chapter 2 provides asymptotic results for a family of reinforcement learning algorithmsknown as ‘actor-critic learners’. Each such algorithmic agent simultaneously estimates what iscalled a ‘critic’, such as a value function, and updates its policy, which is referred to as the‘actor’. The critic is used to indicate directions of improvement for the actor. I establish suf-ficient conditions for the consistency of each agent’s parametric critic estimator, which enablesthem to adapt and find optimal responses despite the non-stationarity inherent to multi-agentsettings. The conditions depend on the environment, number of observations used in the criticestimation, and policy stepsize.Chapter 3 presents an analytical characterization of the long run policies learned by al-gorithmic agents in the multi-ag…
has type: Thesis/Dissertation
date issued: 2023-11
is shown at: 10.14288/1.0435830
language: eng
degree discipline: Economics
provider: Vancouver : University of British Columbia Library
publisher: University of British Columbia
rights: Attribution-NonCommercial-NoDerivatives 4.0 International
rights URI: http://creativecommons.org/licenses/by-nc-nd/4.0/
scholar level: Graduate
contributor: Li, Hao
contributor: Farinha Luz, Vitor
type: Text
identifier URI: http://hdl.handle.net/2429/85877