Arts, Faculty of
Vancouver School of Economics
DSpace
UBCV
Ebrahimi Kahou, Mahdi
Doctor of Philosophy - PhD
University of British Columbia
The first chapter: I propose a new method for solving high-dimensional dynamic programming problems and recursive competitive equilibria with a large (but finite) number of heterogeneous agents using deep learning. The curse of dimensionality is alleviated thanks to three techniques: (1) exploiting symmetry in the approximate law of motion and the policy function; (2) constructing a concentration of measure to calculate high-dimensional expectations using a single Monte Carlo draw from the distribution of idiosyncratic shocks; and (3) designing and training deep learning architectures that exploit symmetry and concentration of measure. As an application, I find a global solution of a multi-firm version of the classic Lucan and Prescott (1971) model of investment under uncertainty. First, I compare the solution against a linear-quadratic Gaussian version for benchmarking. Next, I solve the nonlinear version where no accurate or closed-form solution exists. Finally, I describe how this approach applies to a large class of models in economics. The second chapter: in the long run, we are all dead. Nonetheless, even when investigating short-run dynamics, models require boundary conditions on long-run, forward-looking behavior (e.g., transversality condition). In this chapter, in sequential setups, I show how deep learning approximations can automatically fulfill these conditions despite not directly calculating the steady state and balanced growth path. The main implication is that one can solve for transition dynamics with forward-looking agents, confident that long-run boundary conditions will implicitly discipline the short-run decisions, even converging towards the correct equilibria in cases with steady-state multiplicity. While this chapter analyzes benchmark models such as the neoclassical growth model, the results suggest deep learning may allow us to calculate accurate transition dynamics with high-dimensional state spaces, and without directly solving for long-run behavior. The third chapter: the sequential models studied in the previous chapter can be very useful to study deterministic setups and one-time shocks to economic variables. In this chapter I focus on the recursive setups. I consider the recursive version of the neoclassical growth model, which can be extended to study investment decisions under uncertainty. I show how deep learning approximations automatically fulfill these boundary conditions.
Essays on Theory and Computation inEconomicsbyMahdi Ebrahimi KahouB.Sc., Sharif University of Technology, 2010M.Sc., University of Calgary, 2013A THESIS SUBMITTED IN PARTIAL FULFILLMENT OFTHE REQUIREMENTS FOR THE DEGREE OFDOCTOR OF PHILOSOPHYinThe Faculty of Graduate and Postdoctoral Studies(Economics)THE UNIVERSITY OF BRITISH COLUMBIA(Vancouver)July 2023© Mahdi Ebrahimi Kahou, 2023The following individuals certify that they have read, and recommend to the Faculty of Graduateand Postdoctoral Studies for acceptance, the dissertation titled:Essays on Theory and Computation in Economicssubmitted by Mahdi Ebrahimi Kahou in partial fulfillment of the requirements forthe degree of Doctor of Philosophyin EconomicsExamining Committee:Jesse Perla, Associate Professor, Vancouver School of Economics, UBCSupervisorKyungchul (Kevin) Song, Professor, Vancouver School of Economics, UBCSupervisory Committee MemberPaul T. Schrimpf, Associate Professor, Vancouver School of Economics, UBCUniversity ExaminerDaniel McDonald, Associate Professor, Department of Statistics, UBCUniversity ExaminerAdditional Supervisory Committee Member:Hiroyuki Kasahara, Professor, Vancouver School of Economics, UBCSupervisory Committee MemberVadim Marmer, Professor, Vancouver School of Economics, UBCSupervisory Committee MemberiiAbstractThe first chapter: I propose a new method for solving high-dimensional dynamic programmingproblems and recursive competitive equilibria with a large (but finite) number of heterogeneousagents using deep learning. The curse of dimensionality is alleviated thanks to three techniques:(1) exploiting symmetry in the approximate law of motion and the policy function; (2) constructinga concentration of measure to calculate high-dimensional expectations using a single Monte Carlodraw from the distribution of idiosyncratic shocks; and (3) designing and training deep learningarchitectures that exploit symmetry and concentration of measure. As an application, I find aglobal solution of a multi-firm version of the classic Lucas and Prescott (1971) model of investmentunder uncertainty. First, I compare the solution against a linear-quadratic Gaussian version forbenchmarking. Next, I solve the nonlinear version where no accurate or closed-form solution exists.Finally, I describe how this approach applies to a large class of models in economics.The second chapter: in the long run, we are all dead. Nonetheless, even when investigatingshort-run dynamics, models require boundary conditions on long-run, forward-looking behavior(e.g., transversality condition). In this chapter, in sequential setups, I show how deep learningapproximations can automatically fulfill these conditions despite not directly calculating the steadystate and balanced growth path. The main implication is that one can solve for transition dynamicswith forward-looking agents, confident that long-run boundary conditions will implicitly disciplinethe short-run decisions, even converging towards the correct equilibria in cases with steady-statemultiplicity. While this chapter analyzes benchmark models such as the neoclassical growth model,the results suggest deep learning may allow us to calculate accurate transition dynamics with high-dimensional state spaces, and without directly solving for long-run behavior.The third chapter: the sequential models studied in the previous chapter can be very useful tostudy deterministic setups and one-time shocks to economic variables. In this chapter I focus onthe recursive setups. I consider the recursive version of the neoclassical growth model, which can beextended to study investment decisions under uncertainty. I show how deep learning approximationsautomatically fulfill these boundary conditions.iiiLay SummaryThis thesis aims to offer solutions to tackle high-dimensional dynamic models in economics, uti-lizing insights from economic theory. It is divided into three chapters. The first chapter utilizesa symmetry commonly found in many heterogenous agent models in economics, which is the ex-changeability of agents. I demonstrate how this symmetry can alleviate the two primary sourcesof the curse of dimensionality: interpolating functions in high dimensions with good generaliza-tion and the calculation of high-dimensional expectations. The last two chapters establish thatusing deep learning (guided by economic knowledge) to interpolate these functions automaticallysatisfy long-run boundary conditions required for optimality of the solutions. One can argue thatthese boundary conditions are the main impediment in increasing economic models’ dimensional-ity. While these chapters analyze benchmark models, the results suggest deep learning may let uscalculate accurate transition dynamics with high-dimensional state spaces.ivPrefaceChapter 2 is a joint work with Dr. Jesu´s Ferna´ndez-Villaverde (University of Pennsylvania), Dr.Jesse Perla (University of British Columbia), and Arnav Sood (Carnegie Mellon University). Chap-ter 3 and 4 is a joint work with Dr. Jesu´s Ferna´ndez-Villaverde (University of Pennsylvania), Se-bastia´n Go´mez-Cardona (University of British Columbia), Dr. Jesse Perla (University of BritishColumbia), and Jan Rosa (University of British Columbia).All chapters are original and unpublished works. In all of these chapters, I have been involvedthroughout all the stages: forming the research question, mathematical proofs, and implementationof the computer codes.vTable of ContentsAbstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iiiLay Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ivPreface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vTable of Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viList of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ixList of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xAcknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xviiDedication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .xviii1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 Symmetry in High-Dimensional Dynamical Problems in Economics . . . . . . . 52.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52.2 Permutation-invariant dynamic programming and concentration of measure . . . . . 102.2.1 The symmetric group of permutation matrices . . . . . . . . . . . . . . . . . 102.2.2 Permutation-invariant dynamic programming . . . . . . . . . . . . . . . . . . 112.2.3 Concentration of measure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142.3 An application: a model of investment . . . . . . . . . . . . . . . . . . . . . . . . . 162.4 A deep learning implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212.4.1 Architecture of the network . . . . . . . . . . . . . . . . . . . . . . . . . . . 212.4.2 Training the network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222.4.3 Case I: ν = 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242.4.4 Case II: ν > 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 292.4.5 Generalization: where is the transversality condition? . . . . . . . . . . . . . 312.5 What about concentration of measure? . . . . . . . . . . . . . . . . . . . . . . . . . 322.5.1 The linear-quadratic regulator formulation . . . . . . . . . . . . . . . . . . . 322.5.2 The solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 332.6 Extensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36vi2.6.1 Multiple productivity types . . . . . . . . . . . . . . . . . . . . . . . . . . . . 362.6.2 Complex idiosyncratic states . . . . . . . . . . . . . . . . . . . . . . . . . . . 362.6.3 Global solutions with transitions and aggregate shocks . . . . . . . . . . . . 372.7 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 383 Exploring Transversality and Stationarity with Deep Learning in Sequential Se-tups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 393.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 393.2 Linear asset pricing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 443.2.1 Interpolating solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 463.2.2 Minimum norm interpretation and no-bubble condition . . . . . . . . . . . . 473.2.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 493.3 Neoclassical growth model: sequential form . . . . . . . . . . . . . . . . . . . . . . . 543.3.1 Interpolating solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 553.3.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 573.3.3 Minimum norm interpretation and transversality condition . . . . . . . . . . 633.3.4 Neoclassical growth model with multiplicity of equilibria: sequential form . . 673.3.5 Connection to turnpike theory . . . . . . . . . . . . . . . . . . . . . . . . . . 703.4 Implicit bias of deep neural networks . . . . . . . . . . . . . . . . . . . . . . . . . . 703.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 724 Exploring Transversality and Stationarity with Deep Learning in Recursive Se-tups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 744.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 744.2 The neoclassical growth model: recursive form . . . . . . . . . . . . . . . . . . . . . 754.2.1 Interpolating solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 764.2.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 774.2.3 Minimum norm interpretation and transversality condition . . . . . . . . . . 824.3 Are Euler and value function residuals enough? . . . . . . . . . . . . . . . . . . . . 854.4 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 915 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94AppendicesA Appendix to Chapter 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100A.1 Mathematics definitions and background . . . . . . . . . . . . . . . . . . . . . . . . 100A.2 Concentration of measure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100viiA.3 Proofs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101A.3.1 Proof of proposition 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101A.3.2 Proof of proposition 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102A.3.3 Proof of proposition 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103A.4 LQ Formulations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104A.4.1 Proof of proposition 5: linear-quadratic Gaussian control . . . . . . . . . . . 104A.4.2 Proof of proposition 5: permutation invariance . . . . . . . . . . . . . . . . . 105A.4.3 Proof of proposition 5: concentration of measure . . . . . . . . . . . . . . . . 106A.4.4 Representation of the v function . . . . . . . . . . . . . . . . . . . . . . . . . 109B Appendix to Chapter 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110B.1 Sequential linear asset pricing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110B.1.1 Learning the growth rate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110B.1.2 Misspecification of growth in prices . . . . . . . . . . . . . . . . . . . . . . . 111B.2 Sequential neoclassical growth . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112B.2.1 Zero as a fixed point for capital . . . . . . . . . . . . . . . . . . . . . . . . . 112B.2.2 Solutions violating the transversality condition: initial capital above thesteady state . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113B.2.3 An alternative way of approximating the optimal solution . . . . . . . . . . . 114B.2.4 Being far from the steady state . . . . . . . . . . . . . . . . . . . . . . . . . 114B.2.5 Misspecification of growth in capital and consumption paths . . . . . . . . . 115B.2.6 Analysis of the approximate solution in the vicinity of the critical point . . . 117C Appendix to Chapter 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119C.1 Robustness of the solution to the initial condition of capital . . . . . . . . . . . . . 119viiiList of Tables2.1 Convergence of baseline network architectures in solving case I: ν = 1 . . . . . . . . 272.2 Convergence of different network architectures in solving case I: ν = 1 . . . . . . . . 292.3 Convergence of ϕ(ReLU), linear case (ν = 1) with few data points . . . . . . . . . . 31ixList of Figures2.1 Moments architecture; M is the number of moments. . . . . . . . . . . . . . . . . . . 222.2 ReLU/Deep Sets architecture. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222.3 Comparison between the LQ-regulator solution and the three deep learning archi-tectures for the case with ν = 1 and N = 128. . . . . . . . . . . . . . . . . . . . . . . 252.4 The policy errors for ν = 1 and N = 128 for ϕ(Identity), ϕ(Moments), and ϕ(ReLU).The dark blue curve shows the median along equilibrium paths. The shaded areasshow 10th-90th and 25th-75th percentiles. . . . . . . . . . . . . . . . . . . . . . . . . 272.5 Performance of the ϕ(ReLU) for different N . . . . . . . . . . . . . . . . . . . . . . . 282.6 The Euler residuals square for ν = 1.5 and N = 128 for ϕ(Moments) and ϕ(ReLU).The dark blue curve shows the median of the residuals along equilibrium paths. Theshaded areas show 10th-90th and 25th-75th percentiles. . . . . . . . . . . . . . . . . 302.7 The optimal policy u along the equilibrium paths for ν = [1.0, 1.3, 1.5] and N = 128.Each path shows the optimal policy for a single trajectory. . . . . . . . . . . . . . . . 302.8 The concentration of Euler residuals ε(X;u) and the optimal policy u(X ′). . . . . . 353.1 Comparison between the price based on the fundamentals and the price approxi-mated by a deep neural network for the sequential linear asset pricing model. In theleft panel, the solid curve shows the approximate price and the dashed curve showsthe price based on the fundamentals. The right panel shows the relative errors be-tween the approximate price path and the price based on the fundamentals. Thedashed vertical lines separate the interpolation from the extrapolation region. . . . . 503.2 Comparison between the contiguous grid and two sparse grids for the sequentiallinear asset pricing model. The first sparse grid is defined as Xtrain(Grid 1) ≡{0, 1, 2, 4, 6, 8, 12, 16, 20, 24, 29} and the second sparse grid is defined as Xtrain(Grid 2) ≡{0, 1, 4, 8, 12, 16, 20, 24, 29}. The top panels show the results for price paths for oneseed. The bottom panels show the median of the approximate price paths and therelative errors for the contiguous grid and Xtrain(Grid 2). The shaded regions showthe 10th and the 90th percentiles. The left panels provide a comparison between theapproximate price paths and the price based on the fundamentals. The right panelsshow the relative errors between the approximate price paths and the price basedon the fundamentals. The dashed vertical lines separate the interpolation from theextrapolation region. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51x3.3 Comparison between the price based on the fundamentals and the price approxi-mated by a deep neural network for the sequential linear asset pricing model withgrowing dividends (g = 0.02). The solid curve in the left panel shows the median ofthe approximate price paths and the dashed curve shows the price based on the fun-damentals. The solid curve in the right panel shows the median of the relative errorsbetween the approximate price paths and the price based on the fundamentals. Theshaded regions show the 10th and the 90th percentiles. The dashed vertical linesseparate the interpolation from the extrapolation region. . . . . . . . . . . . . . . . . 533.4 Comparison between the value function iteration and approximate solution using adeep neural network for the sequential neoclassical growth model. In the left panelthe black solid line shows the approximate capital path and the blue solid line showsthe consumption path. The dashed curves show the capital and consumption pathsobtained by the value function iteration method. The right panel shows the relativeerrors for both capital and consumption paths. The dashed vertical lines separatethe interpolation from the extrapolation region. . . . . . . . . . . . . . . . . . . . . 583.5 Comparison between the value function iteration and approximate solution using adeep neural network for the sequential neoclassical growth model. The first sparsegrid is defined as Xtrain(Grid 1) ≡ {0, 1, 2, 4, 6, 8, 12, 16, 20, 24, 29} and the secondsparse grid is defined as Xtrain(Grid 2) ≡ {0, 1, 4, 8, 12, 16, 20, 24, 29}. The top panelsshow the approximate capital paths and the relative errors between the approximatecapital path and the capital path obtained by the value function iteration method,for Xtrain(Grid 1), Xtrain(Grid 2), and the contiguous grid. The bottom panels showthe median of the approximate capital paths and the median of relative errors forXtrain(Grid 2), and the contiguous grid. The shaded regions show the 10th and the90th percentiles. The dashed curves, denoted by k(t), show the capital and pathobtained by the value function iteration method. The dashed vertical line separatesthe interpolation from the extrapolation region. . . . . . . . . . . . . . . . . . . . . . 603.6 Comparison between the value function iteration and approximate solution using adeep neural network for the sequential neoclassical growth model with a mediumtime horizon (i.e., Xtrain = {0, 1, . . . , 9}). The solid curves in the left panels show themedian of the approximate capital and consumption paths. The dashed curves showthe capital and consumption paths obtained by the value function iteration method.The solid curves in the right panels show the median of the relative errors betweenthe approximate solutions and solutions obtained by the value function iterationmethod. The shaded regions show the 10th and the 90th percentiles. The dashedvertical lines separate the interpolation from the extrapolation region. . . . . . . . . 61xi3.7 Comparison between the value function iteration and approximate solution using adeep neural network for the sequential neoclassical growth model with growth intotal factor productivity (i.e., g = 0.02). The solid curves in the left panels show themedian of the approximate capital and consumption paths. The dashed curves showthe capital and consumption paths obtained by the value function iteration method.The solid curves in the right panels show the median of the relative errors betweenthe approximate solutions and the solutions obtained by the value function iterationmethod. The shaded regions show the 10th and the 90th percentiles. The dashedvertical lines separate the interpolation from the extrapolation region. . . . . . . . . 643.8 Comparison between the optimal solution and the solutions that violate the transver-sality condition for k0 < k∗. The blue curves, denoted by k˜(t), c˜(t), and u′(c˜(t)),show a set of capital, consumption, and marginal utility of consumption paths thatviolate the transversality condition. The black curves denoted by k(t), c(t), andu′(c(t)) show the capital, consumption, and marginal utility of consumption pathsfor the optimal solution. The steady states for capital an consumption are denotedby k∗ and c∗. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 653.9 The approximate capital and consumption paths for the sequential neoclassical growthmodel with a convex-concave production function. The left panel shows the capitalpaths for four levels of initial capital k0 ∈ {0.5, 1.0, 3.0, 4.0} and the right panel showsthe corresponding consumption paths. The dashed horizontal lines show the steadystates of capital k∗1 and k∗2 and the corresponding steady states for consumption c∗1and c∗2. The dashed vertical line separates the interpolation from the extrapolationregion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 683.10 The approximate capital and consumption paths for the sequential neoclassical growthmodel with convex-concave production function. The grid for the initial conditionof capital belongs to [0.5, 1.75] and [2.75, 4]. The top panel shows the capital pathsand the bottom panel shows the consumption paths. The dashed horizontal linesshow the steady states of capital k∗1 and k∗2 and the corresponding steady states forconsumption c∗1 and c∗2. The dashed vertical line separates the interpolation fromthe extrapolation region. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69xii4.1 Comparison between the value function iteration and approximate solution using adeep neural network for the recursive neoclassical growth model. The left panels showthe median of the approximate capital and consumption paths, denoted by kˆ(t) andcˆ(t). The dashed curves, denoted by k(t) and c(t), show the capital and consumptionpaths obtained by the value function iteration method. The right panels show themedian of the relative errors between the approximate paths and the paths obtainedby the value function iteration method. The shaded regions show the 10th and the90th percentiles. The dashed curves in the right panels show the relative errorsfor the parts of the approximate capital paths that lie outside of the interpolationregions. The gray rectangle in the top-right panel shows the interpolation region. . . 794.2 Comparison between the value function iteration and approximate solution using adeep neural network for the recursive neoclassical growth model using a local gridaround the initial condition for capital Xtrain = {0.8, . . . , 1.5}. The left panels showthe median of the approximate capital and consumption paths, denoted by kˆ(t) andcˆ(t). The dashed curves, denoted by k(t) and c(t), show the capital and consumptionpaths obtained by the value function iteration method. The right panels show themedian of the relative errors between the approximate paths and the paths obtainedby the value function iteration method. The shaded regions show the 10th and the90th percentiles. The dashed curves in the right panels show the relative errors forthe parts of the approximate capital path that lie outside of the interpolation regions.The gray rectangle in the top-right panel shows the interpolation region. . . . . . . . 814.3 Comparison between the value function iteration and approximate solution using adeep neural network for the recursive neoclassical growth model with growth in totalfactor productivity (i.e., g = 0.02). I use Xtrain = [0.8, 3.5] × [0.8, 1.8]. The leftpanels show the median of the approximate capital and consumption paths, denotedby kˆ(t) and cˆ(t). The dashed curves, denoted by k(t) and c(t), show the capitaland consumption paths obtained by the value function iteration method. The rightpanels show the median of the relative errors between the approximate paths andthe paths obtained by the value function iteration method. The shaded regions showthe 10th and the 90th percentiles. The dashed curves in the right panels show therelative errors for the parts of the approximate capital path that lie outside of theinterpolation regions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 834.4 Comparison between the optimal solution, denoted by k′(k), and a solution thatviolate the transversality condition, denoted by k˜′(k). The solution that violates thetransversality equation intersects with the 45 degree line at k˜max ≈ 30. . . . . . . . 84xiii4.5 Comparison between approximating the policy function for capital k′(k) versus theconsumption function c(k) with a deep neural network. The left panels show themedian of Euler residuals and policy functions for capital when k′(k; θ) is approx-imated with a deep neural network. The right panels show the median of Eulerresiduals squared and policy functions for capital when c(k) is approximated witha deep neural network. The solid curves show the medians and the shaded regionsshow the 10th and the 90th percentiles over 100 different seeds. . . . . . . . . . . . 874.6 The effect of the L2 explicit regularization on the solutions when the consumptionfunction c(k) is approximated with a deep neural network. The left panels show theresults without regularization. The right panels show the results with L2 regulariza-tion with penalization coefficient λ = 10−9. The solid curves in the top panels showthe median of the Euler residuals squared. The solid curves in the bottom panelsshow the median of the approximate policy functions for capital, i.e., k′ (k; c(·θ)).The shaded regions show the 10th and the 90th percentiles. . . . . . . . . . . . . . . 894.7 Comparison between the accurate and approximate solution using a deep neuralnetwork for the sequential linear asset pricing model for small tN . The dashedvertical lines separate the interpolation from the extrapolation region. In the leftpanel the solid curve shows the median of th approximate prices. The dashed curveshows the price based on the fundamentals. The solid curve in the right panel showsthe median of relative errors between the approximate price and the price based onthe fundamentals. The shaded regions show the 10th and the 90th percentiles of therelative errors. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 904.8 Analyzing the approximate solutions of the sequential linear asset pricing modelfor small tN. In this experiment Xtrain = {0, 1, . . . , 9}. The dashed vertical linesseparate the interpolation from the extrapolation region. In the left panel the blacksolid curve, denoted by pˆ(t), shows the median of the approximate prices. The dashedcurve, denoted by pf (t), shows the price based on the fundamentals. The blue solidcurve shows the median of pˆ(t)− ζˆβ−t. The right panel shows the difference betweenthe approximate prices and the price based on the fundamentals. The shaded regionsshow the 10th and the 90th percentiles. . . . . . . . . . . . . . . . . . . . . . . . . . 91B.1 The distribution of the approximated growth rate of the dividends in the sequentiallinear asset pricing model. The dashed vertical line shows the actual value of g. . . . 110xivB.2 Comparison between the price based on the fundamentals and the price approxi-mated by a deep neural network for the sequential linear asset pricing model withgrowing dividends (g = 0.02) in the presence of functional misspecification of thegrowth. The solid curve in the left panel shows the median of the approximateprice paths, the dashed curve shows the price based on the fundamentals. The solidcurve in the right panel shows the median of the relative errors. The shaded re-gions show the 10th and the 90th percentiles. The dashed vertical lines separate theinterpolation from the extrapolation region. . . . . . . . . . . . . . . . . . . . . . . . 111B.3 Comparison between the value function iteration and approximate solution using adeep neural network for the sequential neoclassical growth model with small levelsof initial capital k0 = 0.01. The solid curve, in the left panel, shows the approximatecapital path and the dashed curve shows the solution obtained by the value functioniteration method. The right panel shows the relative errors for the capital paths.The dashed vertical lines separate the interpolation from the extrapolation region. . 112B.4 Comparison between the optimal solution and the solutions that violate the transver-sality condition for k0 > k∗. The blue curves denoted by k˜(t), c˜(t), and u′(c˜(t)) showa set of capital, consumption, and marginal utility of consumption paths that vio-late the transversality condition. The black curves denoted by k(t), c(t), and u′(c(t))show the capital, consumption, and marginal utility of consumption paths for theoptimal solution. The steady states for capital an consumption are denoted by k∗and c∗. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113B.5 Comparison between the value function iteration and approximate solution using adeep neural network for the sequential neoclassical growth model with a short timehorizon (i.e., Xtrain = {0, 1, . . . , 4}). The solid curves in the left panels show themedian of the approximate capital and consumption paths. The dashed curves showthe capital and consumption paths obtained by the value function iteration method.The solid curves in the right panels show the median of the relative errors betweenthe approximate solutions and solutions obtained by the value function iteration.The shaded regions show the 10th and the 90th percentiles. The dashed verticallines separate the interpolation from the extrapolation region. . . . . . . . . . . . . 115xvB.6 Comparison between the value function iteration and approximate solution using adeep neural network for the sequential neoclassical growth model with growth intotal factor productivity in the presence of functional misspecification of the growth.The solid curves in the left panels show the median of the approximate capital andconsumption paths. The dashed curves show the capital and consumption pathsobtained by the value function iteration method. The solid curves in the rightpanels show the median of the relative errors between the approximate solutionsand solutions obtained by the value function iteration method. The shaded regionsshow the 10th and the 90th percentiles. The dashed vertical lines separate theinterpolation from the extrapolation region. . . . . . . . . . . . . . . . . . . . . . . . 116B.7 The approximate capital and consumption paths for the sequential neoclassical growthmodel with convex-concave production function. The grid for the initial conditionof capital belongs to [0.5, 4]. The top panel shows the capital paths and the bot-tom panel shows the consumption paths. The red trajectories show the capital andconsumption paths in the vicinity of the critical point. The dashed horizontal linesshow the steady states of capital k∗1 and k∗2 and their corresponding steady statesfor consumption c∗1 and c∗2. The dashed vertical line separates the interpolation fromthe extrapolation region. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118C.1 Comparison between the value function iteration and approximate solution using adeep neural network for the recursive neoclassical growth model for a wide range ofinitial levels of capital. The left panels show the approximate capital and consump-tion paths from three initial levels conditions outside of Xtrain, k0 ∈ {0.5, 3.25, 4.0}.The dashed curves, denoted by k(t) and c(t), show the capital and consumptionpaths obtained by the value function iteration method. The right panels show thefull range of the relative errors for capital and consumption paths for 70 initial con-ditions of capital in [0.4, 4]. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120C.2 Comparison between the value function iteration and approximate solution using adeep neural network for the recursive neoclassical growth model with growth in totalfactor productivity (i.e., g = 0.02) for a wide range of initial levels of capital. Theleft panels show the approximate capital and consumption paths from three initiallevels conditions k0 ∈ {0.5, 3.25, 4.0}. The dashed curves, denoted by k(t) and c(t),show the capital and consumption paths obtained by the value function iterationmethod. The right panels show the full range of the relative errors for capital andconsumption paths for 70 initial conditions of capital in [0.4, 4]. . . . . . . . . . . . . 121xviAcknowledgementsCompleting a PhD is a challenging journey, and I could not have made it to the end without thesupport and guidance of numerous individuals. I would like to express my heartfelt gratitude toall those who have helped me in this endeavor.First and foremost, I would like to thank my supervisor, Dr. Perla, for his mentorship, patience,and support throughout my PhD journey. His guidance and insights have been invaluable, and Iam grateful for the time and effort he invested in my research.I would also like to extend my appreciation to my committee members, Dr. Kasahara and Dr.Marmer, for their support, constructive feedback, and insightful comments that have shaped myresearch and contributed to its quality.Furthermore, I would like to thank Dr. Ferna´ndez-Villaverde for his invaluable guidance andmentorship. His support and advice on navigating the job market process have been crucial in se-curing my current position. Moreover, working with him has been an excellent learning experience,and I am grateful for his contributions as a co-author.I would also like to thank Dr. Song for being an intellectual mentor and providing me withvaluable insights and feedback on my research. His feedback has been instrumental in improvingthe quality of my work.I would like to express my gratitude to my fellow PhD students and friends, Sina MoghadasKhorasani. Allen Peters, Amirhossein Tavakoli, Navid Siami, Federico Ricca, Pooya Rezaeinia,Sejin Ahn, Sebastia´n Go´mez-Cardona, Jan Rosa, Anubhav Jha, Max Norton, Joshua Catalano,Ali Karimirad, Joao Rodrigues, and Lejvi Dalani. Their support, camaraderie, and intellectual ex-change have been an integral part of my PhD journey. I am grateful for the friendly and supportiveenvironment they have created.I would also like to thank our Graduate Programs Managers, Devin Clemens and MaureenChin, and Karol Pasciano, for their assistance throughout the PhD program and the job marketprocess. Their support and guidance have been instrumental in navigating the various challengesI faced.I would like to express my gratitude to my parents and my sisters for their unwavering supportand love throughout my life. Their encouragement and belief in me have been a source of strengthand motivation.Finally, I would like to thank my family, Kerri and Ailey, for their emotional support, patience,and unconditional love. Their unwavering support and encouragement have been a constant sourceof motivation and inspiration.xviiDedicationTo my family and the representative Canadian taxpayer.xviiiChapter 1IntroductionTraditionally, most models in macroeconomics (and other fields) have dealt with either a repre-sentative agent (the canonical real business cycle and New Keynesian models) or a continuum ofagents (the Aiyagari, 1994 and Krusell and Smith, 1998 models). Some models take an intermediateposition with two (or a few) agents, for example, models of financial frictions (with a borrower anda lender) or models of international business cycles (with a home country and a foreign country).Recently, models that incorporate a finite but large number of agents have garnered significantattention, largely attributable to the proliferation of microdata. For instance, models for analyz-ing industry dynamics characterized by many firms or in examining complex scenarios involvingmultiple countries. In fact, one could argue that the adoption of models utilizing a continuumof agents in macroeconomic or international trade research is primarily a pragmatic abstraction.For instance, Canada is comprised of only ten provinces and three territories, the European Unionconsists of twenty-seven countries, as opposed to an infinite number of entities.Unfortunately, dealing with multi-agent (or high-dimensional state space) models poses a sig-nificant challenge, as the curse of dimensionality (Bellman, 1958, p. ix) swiftly emerges. Once thenumber of agents surpasses a few, solving these models or relating them to empirical data becomesnearly impossible. According to Powell (2007), dynamic programming suffers from two sources ofthe curse of dimensionality. Firstly, the size of the state space becomes enormous. Secondly, calcu-lating highly dimensional conditional expectations proves arduous. To illustrate, consider a basiceconomy with 266 agents, each of whom can make a binary decision (e.g., to enter a market or not).In this economy, if an agent decides based on the other agents’ choices, the agent must evaluate2266 potential scenarios, a number exceeding the estimated number of protons in the universe.The primary aim of this thesis is to propose solutions for addressing high-dimensional dynamicmodels in economics, drawing inspiration from economic theory. Furthermore, this objective isachieved by incorporating theoretical insights from modern machine learning methods. The thesisis divided into three distinct chapters.Chapter one proposes a new method for solving high-dimensional dynamic programming prob-lems and recursive competitive equilibria with a large (but finite) number of heterogeneous agents.The key to this approach is to exploit symmetry in dynamic programming to build deep neuralnetworks that are remarkably efficient, generalize well across the relevant parts of the state space,and are easy to train. Moreover, I use concentration measure, something akin to the law of largenumbers, to calculate high-dimensional expectations.Many heterogeneous agent models in macroeconomics exhibit an obvious but latent symmetry,1in which the agents are exchangeable. For example, in general equilibrium models, this symmetryis enforced by the Walrasian auctioneer, who collects all the excess supply of each agent in theeconomy, aggregates them, and sells them to the demand side of the economy. Through this aggre-gation process, the agents’ indices in the economy are removed. Consequently, the outcome of theeconomy remains unchanged if one exchanges (or permutes) the agents. Therefore, any functionrepresenting the solution to the problem above belongs to the family of symmetric functions. Thesolution is a function on a set rather than on a vector space (Macdonald, 1998) and it is invariantunder the permutations of its inputs.To formalize the idea of symmetry, I introduce the notion of permutation-invariant dynamicprogramming and describe how one can use powerful theorems, proved by Zaheer et al. (2017) andWagstaff et al. (2019), to represent functions invariant to permutations. These theorems can leadto a reduction in the dimensionality of the state space, setting up a suitable geometry for deeplearning approximations. With this theoretical foundation, deep neural networks are designed toaccurately capture the symmetry and approximate the unknown functions. I establish that thisdesign of neural networks leads to outstanding generalization power, i.e., solutions that performwell on points in the space that are not present in the training data.To address the challenge of high-dimensional expectations, concentration of measure, a conceptsimilar to the law of large numbers, is employed. Concentration of measure asserts that, undermild conditions, a function dependent on many independent variables is effectively constant, hencevery close to its mean. To tackle expectations regarding agents’ idiosyncratic shocks, concentra-tion of measure is utilized to prove that a single draw of such shocks is sufficient to achieve asatisfactory approximation. In order to calculate expectations with respect to the aggregate shock,low-dimensional methods like the Gaussian quadrature method are used. In fact, in models with acontinuum of heterogeneous agents, economists already use an extreme form of this result: usually,it is assumed that a law of large numbers holds in the economy regarding the idiosyncratic shocks.This chapter sheds light on why this feature is natural and pervasive.To illustrate these concepts I use a multi-firm variation of the classic Lucas and Prescott (1971)model of investment under uncertainty. There are two reasons for choosing this model. First,this model generates one of the simplest nontrivial recursive competitive equilibria. For instance,Ljungqvist and Sargent (2018, ch. 7) use this model to introduce recursive competitive equilibriato graduate students in economics. Second, by modifying just one parameter in the inverse demandfunction, the model can exhibit either a known and exact linear-quadratic (LQ) solution or be fullynonlinear. Thus, the Lucas and Prescott (1971) model is an ideal testing ground for this chapter:it is both simple and engaging, yet it encompasses scenarios with known and unknown solutions.First, I solve the case when the inverse demand function is linear both with the LQ approachand the deep neural network. By benchmarking the solution from the deep neural network againstthe exact LQ solution, I demonstrate that the deep neural network solution is highly accurate evenwhen there are hundreds of firms across a large section of the state space far outside of the tra-jectories used for training. Then, I solve the case when the inverse demand function is nonlinear.2The method handles the nonlinear case as easily as the LQ case. In fact, I only need to changeone parameter value in the code that solves the model: everything else is exactly as it was for thelinear case. While, in this nonlinear case, there is no exact solution to benchmark the deep neuralnetworks, I still get a highly accurate solution according to the Euler residuals.One of the significant results obtained in the first chapter is that fully flexible deep learning-based approximations often (with high probability) find stationary solutions. For example, thelinear-quadratic (LQ) case studied in this chapter is not a well-posed problem as it has two solu-tions. While one solution is stationary and non-explosive, the other leads to explosive solutionswhere the firms producing indefinitely without any constraints. The second solution violates thetransversality condition. It is important to note that the method introduced in this chapter doesnot have any explicit mechanism to exclude such solutions. Therefore, it must be that the deeplearning-based algorithms have an implicit bias toward the stationary solution. Chapter 3 and 4provide a detailed understanding of this “implicit bias” and its application in dynamic problems ineconomics, in both sequential and recursive models.Generally the solution of dynamic equilibrium models are accompanied with economic condi-tions that rule out explosive solutions. For instance, the transversality condition in neoclassicalgrowth rules out solutions with divergent marginal utility of consumption, or non-bubble condi-tions in linear asset pricing models rules out prices that diverge faster than a specific rate. Theseconditions impose a certain structure on the long-run behavior of forward-looking agents. Mathe-matically speaking, these conditions are variations of “boundary conditions” that are motivated bythe forward-looking behavior of economic agents. How these conditions appear and their concreteform varies from model to model. Still, they are always present, whether one is interested in solvingthe deterministic or stochastic version of the model or formulating the problem in a sequential orrecursive structure. Without these conditions, dynamic optimization problems are not well-posedand have a multiplicity of solutions, many of which imply sub-optimal descriptions of economicbehavior.One could argue, these forward-looking boundary conditions are one of the main impedimentsin increasing the model’s dimensionality in economics (e.g., raising the number of state variables).Without these forward-looking boundary conditions, one could compute large models written ina sequential setup by solving the associated initial value problem or simulating backward-lookingrecursive stochastic difference equations.These boundary conditions require obtaining accurate solutions over a big range of points inthe state-space. For example, in recursive setups it is required to have an accurate solution forarbitrary values of the state, even though given the initial conditions, of the economy might nevervisit those points. Equivalently in sequential setups, it is required to have an accurate long-runsolutions, even though one is interested in short and medium-run path of the economy.Therefore, the question is: if one is interested in the path of the economy given some initialcondition, is it possible to have accurate short- and medium-run solution without calculating long-run properties of the economy such as steady state, balanced growth path (BGP), or stationary3distribution? Chapter 3, and 4 show that the deep learning-based solutions can achieve this goal.This is due to their implicit bias toward a specific class of interpolating solutions.It has been shown that deep neural networks and their optimizers have a built-in “Occam’s ra-zor” and are biased toward “simple” functions. Simple here means functions with small derivatives(with respect to their inputs). For instance, see Ma and Ying (2021). Since I want to understandthis phenomenon and its applications in economic models, I use benchmark models in economics asan application. More specifically, I focus on the neoclassical growth model and linear asset pricing.These models are well understood and allow me to provide concrete theoretical investigations.Chapter 3 focuses on the sequential version of these models. I establish that in standard ver-sions of these models deep learning can provide accurate short- and medium-run solutions that aredisciplined by the long-run boundary conditions. This chapter also provides a theoretical insighton why and when deep learning solutions exhibit this behavior. Intuitively, solutions that violatethese conditions are explosive with big derivatives. Therefore, they are automatically ruled out bythe implicit bias of deep learning solutions. I also show that this is also true in the presence ofbalanced growth path. Additionally, I establish one can attain short- and medium-run solutionsin case with multiplicity of steady states. At a very fundamental level, this results should be fa-miliar to economists. I provide a close connection between this implicit bias and turnpike theory(McKenzie, 1976).Chapter 4 focuses on recursive models. To be more specific, I consider the recursive versionof the neoclassical growth model. I establish that solving the Euler equation with deep learninginterpolation yields solutions that satisfy the transversality condition. I also establish that thisresult is robust to the choice of the grid for capital, even when the grid is far from the steady state.This chapter also provides a theoretical insight on why and when deep learning solutions exhibitthis behavior. I also provide examples to show how lack of understanding of the implicit bias ofdeep neural networks and economic theory can lead to interpolating solutions that are sub-optimaland the violation of the transversality condition.4Chapter 2Symmetry in High-DimensionalDynamical Problems in Economics2.1 IntroductionI propose a new method for solving high-dimensional dynamic programming problems and recursivecompetitive equilibria with a large (but finite) number of heterogeneous agents. The key to thisapproach is to exploit symmetry in dynamic programming and concentration of measure to builddeep neural networks that are remarkably efficient, generalize well across the state space, and areeasy to train.Traditionally, most models in macroeconomics (and other fields) have dealt with either a rep-resentative agent (the canonical real business cycle and New Keynesian models) or a continuum ofagents (the Aiyagari and Krusell-Smith models). Some models take an intermediate position withtwo (or a few) agents, for example, models of financial frictions (with a borrower and a lender) ormodels of international business cycles (with a home country and a foreign country).More recently, models with many finite agents are becoming very popular, in large part dueto the increasing availability of microdata. For instance, models of industry dynamics with manyfirms, models with many locations, or models with many households. In fact, one can argue thatmodels with a continuum of agents in macroeconomics or international trade are a convenient ab-straction: the U.S. economy had around 128.45 million households in 2020, not an infinite numberof them.Unfortunately, dealing with multi-agent models is challenging, as one quickly encounters thecurse of dimensionality (Bellman, 1958, p. ix). As soon as there are more than a few agents, itbecomes nearly impossible to solve these models or take them to the data. As noted by Powell(2007), there are two sources for the curse of dimensionality in dynamic programming. First, thecardinality of the state space is enormous. Second, it is difficult to compute highly dimensionalconditional expectations.To illustrate these two sources, consider the classic multi-region business cycles model of Backuset al. (1992). In this model, each location (i.e., a country or a region) is characterized by seven statevariables. Consider a version of this model applied to study the 50 U.S. states. The representativehousehold in California (or any other state) must keep track of its own seven states and the sevenstate variables in each of the other 49 locations of the model, for a total of 350 state variables.Using just a few points in the grid of each state variable (if one relies on value function iteration as5a solution method) or employing a few Chebyshev polynomials in each dimension of the problem(if a projection method is used) will require an amount of computer memory beyond feasibility.Even the most aggressive sparse grid methods a` la Brumm and Scheidegger (2017) cannot handle350 state variables.However, not only must the representative household in California keep track of 350 state vari-ables, but also must they compute the conditional expectation of its continuation value functionover these 350 states. This computation requires either large sums (if the states are finite) or com-plex highly dimensional integrals (if the states are continuous). Both tasks become prohibitivelyexpensive as soon as the number of state variables grows above single digits.One could try to tackle this problem using perturbation methods (as Backus et al., 1992, do),but that approach will fail as soon as one incorporates complex features into the model, such asoccasionally binding constraints, or if one is interested in non-ergodic properties of the model (e.g.,models without a steady state or with large transitional dynamics).This example might seem discouraging. However, it also suggests intriguing possibilities. Forexample, does the representative household in California need to keep track of the states of everysingle other region in the model? Consider a version of the model with only one state variable:a productivity shock, which can be high or low. Following the idea of “exchangeability” of Pakesand McGuire (2001), under an appropriate symmetry assumption, one only needs to keep trackof how many regions have a high productivity shock in this period. The indices (e.g., whetherCalifornia or Texas is the region with high productivity today) are irrelevant. The cardinality ofthe new state space becomes much smaller: instead of being 250 (all the combinations of high/lowproductivity across the 50 regions), the cardinality is 51 (do we have zero, one, ..., or 50 regionswith high productivity?).Any function representing the solution to the problem mentioned above belongs to the family ofsymmetric functions, meaning that the solution is a function defined on a set rather than a vectorspace (Macdonald, 1998). To formalize this idea, I introduce the notion of permutation-invariantdynamic programming and prove that both the optimal policy and the value function are sym-metric functions, meaning they are invariant under permutations. I use theorems established byZaheer et al. (2017) and Wagstaff et al. (2019) for representing functions that remain invariant topermutations. These theorems provide a transformation of the state space into a new latent space,setting up a suitable geometry for deep learning approximations.This result is very general, it holds in each case where one can find symmetry in the problem.Since one can include heterogeneity among the agents (e.g., the representative household in Califor-nia is more risk-averse than the representative household in Vermont) in the definition of the states,there will be plenty of cases where symmetry holds.1 In fact, most heterogeneous agent models ineconomics have a latent but obvious symmetry imposed by the presence of a Walrasian auctioneer.1Although there are also cases where symmetry does not hold because one cannot easily track heterogeneity as anextra state variable, for example, if the representative household in California has a constant relative risk aversion(CRRA) utility function and the representative household in Vermont has a constant absolute risk aversion (CARA)utility function. Keeping a whole utility function as a state variable is cumbersome.6In general equilibrium, the auctioneer collects all the excess supply of each agent, aggregates them,and sells them. This aggregation removes the indices of agents in the economy and the solution ofthe model is invariant under all the permutations of other agents’ states.2 For symmetry to hold,I only require that the agents behave the same when they happen to have the same state values.In other words, if Texas has the same capital, population, and other state variables as California,it would behave as California does.Also, the functional representation theorem provided in this section sheds light on why themethod in Krusell and Smith (1998) works so well so often. In many models, the moments ofthe distribution of agents perform the dimensionality reduction required by this chapter’s theoremeither exactly or nearly exactly. In other words, the spirit of my method is to take the “big K,little k” approach and generalize it to a “many big K, one little k” where each “k” can accuratelyforecast functions of the evolution of the other “K” that are invariant to relabeling and permuta-tion. My approach has two clear advantages. First, I can build from theorems that tell us whichrepresentation functions are needed, instead of relying on the intuition of the researcher. Second,and I will argue momentarily, the representation theorem is easily implementable with deep neuralnetworks.3But my argument goes even further. Suppose there are many agents and there is an underlyingsymmetry assumption. Will I not have something that resembles a law of large numbers whencomputing the conditional expectation of the continuation value function? Intuitively, suppose thecontinuation value function does not depend too much on the states of any given agent. In thatcase, the expected continuation value will become essentially constant for all draws of (indepen-dent) idiosyncratic shocks. Hence, I can calculate the expectation in the continuation utility witha single Monte Carlo draw from the distribution of idiosyncratic shocks (or with a few draws incase there are not many agents).I prove this possibility by relying on recent developments in high-dimensional probability theoryand concentration of measure (Barvinok, 2005, and Vershynin, 2018). Loosely speaking, concentra-tion of measure states that, under some conditions, a function that depends on lots of independentrandom variables is essentially constant (when conditioned on variables with a disproportionateimpact on the function.). Applying this idea to the continuation value function, if there are manyagents and their idiosyncratic shocks are independent, each draw from the distribution of idiosyn-cratic shocks will deliver, approximately, the same result.In fact, in models with a continuum of heterogeneous agents, economists already use an extremeform of this result. Usually, it is assumed that a law of large numbers holds in the economy regard-2The symmetry imposed by the auctioneer is often used implicitly in models with heterogeneous agents. In manypapers, the problem is manually transformed so that agents only need to forecast the auctioneer’s prices, therebysimplifying the state space, even though the forecast of the prices themselves becomes difficult with aggregate shocks.This transformation is at the heart of Krusell and Smith (1998) and the recent literature exploiting functionalderivatives with perturbations such as Bilal (2021).3By emphasizing the connection to Krusell and Smith (1998), my method also offers a path forward for modelswith heterogeneous agents. Instead of relying on a continuum approximation, I can have economies with tens ofthousands of agents and track the distribution of agents with the same granularity as in the microdata.7ing the idiosyncratic shocks. Similarly, the use of random simulations to calculate high-dimensionalintegrals has a long history in many applied fields, such as in Rust (1997) and Keane and Wolpin(1994). This chapter provides a mathematical explanation why this feature is natural and perva-sive.4The importance of this result is that, instead of having to integrate over the idiosyncratic shocksof each agent and the aggregate shocks to solve the dynamic programming problem of agent i, Ionly need to integrate over the idiosyncratic shocks of agent i and the aggregate shocks, a muchmore manageable task. In other words, while normally people think of a high-dimensional statespace as making integrals more difficult, a large number of independent shocks make expectationseasier to calculate.With all these results, I define a general class of recursive competitive equilibrium models withheterogeneous agents and establish that the value function and optimal policy of each agent areinvariant under permutation of other agents in the economy. Thus, I can take advantage of the rep-resentation theorems for permutation-invariant functions and concentration of measure to design adeep neural network architecture that solves, in an incredibly efficient way, a permutation-invariantdynamic programming problem. I also choose training points for the deep neural network inspiredby the symmetry of the problem. The strategy is justified by the observation that neural networksare universal approximators, i.e., they can get ϵ-close to any continuous function (Cybenko, 1989,and Hornik, 1991). Also, neural networks are flexible in their design, and there are efficient software(e.g., Pytorch and TensorFlow) and hardware (e.g., GPUs) platforms for training them.5I illustrate my arguments with a multi-firm variation of the classic Lucas and Prescott (1971)model of investment under uncertainty. I choose this model for two reasons. First, this modelgenerates one of the simplest nontrivial recursive competitive equilibria. For instance, Ljungqvistand Sargent (2018, ch. 7) use this model to introduce recursive competitive equilibria to graduatestudents in economics. Second, by changing just one parameter in the inverse demand function,the model can either have a known and exact linear-quadratic (LQ) solution or be fully nonlinear.This makes the Lucas and Prescott (1971) model the perfect testbed for the method. It is simpleand interesting, yet it nests cases with known and unknown solutions.I solve first the case when the inverse demand function is linear both with the LQ approach andthe deep neural network. By benchmarking the solution from the deep neural network against theexact LQ solution, I demonstrate that deep neural network solution is highly accurate even whenthere are hundreds of firms across a large section of the state space far outside of the trajectoriesused for training. Then, I solve the case when the inverse demand function is nonlinear. The4As I will explain in section 2.3, drawing from the distribution of idiosyncratic shocks, even just once, is the rightthing to do. Instead, setting the shocks to zero (as a naive intuition would suggest if the idiosyncratic shocks areGaussian) would miss the typical set of the distribution, which in high dimensions does not include the mode ofthe distribution (i.e., zero). My argument of drawing once from the distribution of idiosyncratic shocks, thanks toconcentration of measure, is different from Judd et al. (2017), who propose precomputing the integrals required inBellman or Euler equations.5It is noteworthy that the combination of my proposed method with the recently introduced hardware designs,specialized in solving dynamic economic problems, as presented by Peri (2020) and Cheela et al. (2022), has thepotential to yield highly efficient and reliable solutions for heterogeneous agent models.8method handles the nonlinear case as easily as the LQ case. In fact, I only need to change oneparameter value in the deep learning code that solves the model. Everything else is exactly as itwas for the linear case. While, in this nonlinear case, I do not have an exact solution to benchmarkthe deep neural network performance, I still get a highly accurate solution according to the Eulerresiduals.Furthermore, I can solve the global solution when the inverse demand function is linear by onlyfitting it to few data points (ranging from 2 to 10), regardless of the number of firms. Given thatthe closed-form model with linearity only requires two parameters, this should not be a surprise.The interesting point is that I can do so even in cases where the guessed solution is heavily over-parameterized (e.g., if I pretend not to know the exact functional form of the solution and, instead,I try to fit the approximate solution with 199.6K parameters). An even more exciting result isthat this solution is found without directly applying a transversality condition. Despite possiblytens of thousands of parameters, my approach finds the “right” (i.e., stationary) equilibrium, whichgeneralizes well outside of the small number of points used in the approximation. 6This example also highlights the importance of deep learning for my approach. In the literatureon randomized algorithms in IO and labor (e.g., Rust, 1997, and Keane and Wolpin, 1994) amongmany others), functions tend to be approximated in relatively low dimensions. The need for thisgoes beyond computational tractability, it becomes essential for good generalization and extrap-olation performance without overfitting. For many of the same reasons, Kalouptsidi (2017) usesLASSO to select the coefficients of approximating functions, tinkering with the classic bias-variancetradeoff to achieve less overfitting and better generalization. Counterintuitively, by radically in-creasing the number of parameters, I manage to both perfectly fit the data and achieve outstandinggeneralization due to the “double-descent” phenomena.7To frame this chapter within the literature, notice that symmetry has been applied in eco-nomics before. For instance, Hartford et al. (2016) use permutation-equivariant symmetry in orderto reduce dimensionality to study the best responses in a normal form game. Mertens and Judd(2018) exploit the symmetry of the DSGE models with incomplete markets and a finite but arbi-trarily large number of heterogeneous agents to propose a perturbation approach that overcomesthe curse of dimensionality. Symmetry is also a key component of anonymous games (Jovanovic andRosenthal, 1988) and in models of Markov perfect equilibria (Pakes and McGuire, 2001), where theidea of symmetry is known as the “exchangeability assumption” (Aguirregabiria and Nevo, 2013).With respect to this literature, I show that symmetry leads to powerful functional representationtheorems and efficient deep neural networks. Notice, as well, that symmetry is implicitly usedwhen economists solve models with a mean-field approximation whenever a continuum of agents isassumed.6Under certain conditions, it has been shown (either empirically or theoretically) that highly over-parameterizedfunctional approximations (when the number of parameters is much greater than the data points), utilizing stochasticoptimization algorithms, tend to generalize better (see Belkin et al., 2019, and Advani et al., 2020). I investigate thisphenomenon and its applications in economics in chapters 3 and 4.7Code to implement the methods can be found at https://github.com/HighDimensionalEconLab/symmetry_dynamic_programming.9There is a literature on Markov decision processes (MDPs) that uses symmetry to reduce di-mensionality, but it has mainly focused on discovering symmetries in MDPs rather than exploitinga priori knowledge of symmetry, as I do; see Ravindran and Barto (2001) and Narayanamurthy andRavindran (2008).Using neural networks as a family of approximating functions to solve a stochastic dynamicproblem in economics can be traced back to Duffy and McNelis (2001). Due to recent substantialhardware and software advances in training deep neural networks, this method has become popularin solving dynamic models in economics. See, among others, Maliar et al. (2021), Ferna´ndez-Villaverde et al. (2019), Duarte (2018), Valaitis and Villa (2021), Kase et al. (2022), Azinovic andZˇemlicˇka (2023), Chen et al. (2021) and Azinovic et al. (2022).Finally, recent developments in the machine learning literature have exploited symmetry tosignificantly alleviate the curse of dimensionality in regression and classification problems. For in-stance, see Bloem-Reddy and Teh (2020), Zaheer et al. (2017), and Yarotsky (2018). My choice offunctional representation is also closely linked to the literature of symmetric functions; see Prasad(2018) and Zabrocki (2015).The rest of this chapter is organized as follows. Section 2.2 formally introduces the ideas ofpermutation-invariant dynamic programming and concentration of measure. Section 2.3 presentsthe application. Section 2.4 introduces the deep learning implementation to solve both the casewhere the inverse demand function is linear and when it is not. Section 2.5 further discusses the roleof concentration of measure. Section 2.6 explores the full generality of these techniques. Section2.7 concludes.2.2 Permutation-invariant dynamic programming andconcentration of measureThis section introduces permutation-invariant dynamic programming and concentration of measure.Permutation invariance provides a structure in the solution of dynamic programming problems thatI can exploit, even when the number of states is large. Concentration of measure helps deal with thecomplex conditional expectations that appear in highly-dimensional dynamic programs. In section2.3, I will present a multi-firm model of investment under uncertainty where I can apply these twoideas.Before introducing these ideas, I briefly discuss the concept of a permutation matrix and theassociated symmetric group of permutation matrices under matrix multiplication.2.2.1 The symmetric group of permutation matricesI denote a column vector of length N of 1s or 0s as 1N and 0N respectively. Similarly, I writematrices of size M ×N of 0s or 1s as 0MN and 1MN respectively, and an identity matrix of size Nas IN . Notice that 1NN = 1N1⊤N , a result that will be helpful momentarily.A permutation matrix is a square matrix with a single 1 in each row and column and zeros10everywhere else. These matrices are called “permutation” because, when they premultiply (post-multiply) a conformable matrix A, they permute the rows (columns) of A.Let SN be the set of all N ! permutation matrices of size N ×N . For example,S2 ={[1 00 1],[0 11 0]}.The first element of S2 is the identity matrix, which leaves rows and columns of a multiplied matrixA unchanged, while the second element swaps the rows or columns of A. The set SN forms agroup under matrix multiplication that is isomorphic to the symmetric group on N letters (i.e., theindeces of agents in the model). See section A.1 for further notation.2.2.2 Permutation-invariant dynamic programmingI want to study a class of dynamic programming problems where the aggregate state vector Xcan be permuted. Using recursive notation, I start by defining a general dynamic programmingproblem for environments with many agents.Definition 1 (A “big X, little x” dynamic programming problem). Consider the dynamic pro-gramming problem:v(x,X) = maxu{r(x, u,X)+ βE[v(x′, X ′)]}(2.1)s.t. x′ = g(x, u) + σw + ηω (2.2)X ′ = G(X) +ΩW + ηω1N . (2.3)Here, x is the individual state of the agent, X is a vector stacking the individual states of all ofthe N agents in the economy, u is the control, w is random innovation to the individual state ofthe agent, which gets stacked in W ∼ N (0N , IN ) where, without loss of generality w = W1, andω ∼ N (0, 1) is a random aggregate innovation to all the individual states. Also, v : RN+1 → R isthe value function, r : RN+2 → R is the return function, g : R2 → R is the deterministic componentof the law of motion for x, G : RN → RN is the deterministic component of the law of motion forX, and E [·] is the expectation operator with respect to W and ω.To simplify derivations, from now on, I focus on the case where r(·), g(·), and G(·) are differ-entiable.Definition 2 (Permutation-invariant dynamic programming). A “big X, little x” dynamic pro-gramming problem is a permutation-invariant dynamic programming problem if, for all (x,X) ∈RN+1 and all permutations π ∈ SN , the reward function r is permutation invariant:r(x, u, πX) = r(x, u,X), (2.4)11the deterministic component of the law of motion for X is permutation equivariant:G(πX) = πG(X), (2.5)and the covariance matrix of the idiosyncratic shocks satisfiesπΩ = Ωπ. (2.6)The previous definition tells us that we are interested in dynamic programming problems wherewe care, for instance, about the distribution of capital among the agents in the economy, but notwhether the agent with high capital is agent 12 or agent 27. If agent 12 swaps their capital withagent 27 (and other relevant states such as her preference shock or income level), the equilibriumdynamics of the economy will remain unchanged.The next proposition shows a basic property of permutation-invariant dynamic programming.Proposition 1 (Permutation invariance of the optimal solution). The optimal solution of a permutation-invariant dynamic programming problem is permutation invariant. That is, for all π ∈ SN :u(x, πX) = u(x,X) (2.7)andv(x, πX) = v(x,X). (2.8)Proof. Section A.3.1. Q.E.D.Building on the previous example, proposition 1 tells us that, if the equilibrium dynamics of theeconomy do not change if agent 12 swaps their capital with agent 27, the policy and value functionsof agent 1 will not change either. This straightforward property allows me to write the solution ofthe problem using a result regarding the representation of permutation-invariant functions.Proposition 2 (Representation of permutation-invariant functions). Let f : RN+1 → R be acontinuous permutation-invariant function under SN , i.e., for all (x,X) ∈ RN+1 and all π ∈ SN :f(x, πX) = f(x,X).Then, there exist L ≤ N and continuous functions ρ : RL+1 → R and ϕ : R→ RL such that:f(x,X) = ρ(x,1NN∑i=1ϕ(Xi)), (2.9)where Xi is the ith element of X.Proof. The proof can be found in Wagstaff et al. (2019). Q.E.D.12This result is powerful, if one can find functions ρ and ϕ with a small L, they can representhighly dimensional functions –such as the policy and value functions of models with high N– veryefficiently. For instance, consider the case of one extreme permutation-invariant function, the meanfunction f(X) = 1N∑Ni=1Xi (for clarity, in this paragraph and the next only, I will drop the nui-sance x variable, since symmetry does not apply to it). Hence, I have the trivial representationϕ(Xi) = Xi and ρ(y) = y, where only L = 1 is required.In comparison, other extreme permutation-invariant functions are harder to represent withproposition 2. Consider the max function, f(X) = max{X1, . . . XN}. While there are several waysto represent the max function, all of them require L = N .8Proposition 2 is an instance of a general class of functional representation theorems. Withsome minor changes in the proof of proposition 2, one can reformulate equation (2.9) as f(x,X) =ρ (x,max{ϕ(X1), . . . , ϕ(XN )}). In that case, the representation of the max function becomes trivial,with ρ(y) = y, ϕ(Xi) = Xi, and L = 1 (of course, this representation would have difficulty repre-senting the mean function, which would now require L = N). Many other variations of equation(2.9) are possible. One can use the economics knowledge of the problem to choose a convenientpermutation-invariant reduction in the representation and implement such an approximation incode using what machine learning researchers call a “pooling layer.”Implementing the functional representation: in practice, one does not know ρ, ϕ, or Lfor most problems. But they will be able to train neural networks to approximate ρ, ϕ, and L.Here I highlight two points. First, even when L is high, an L̂ (the guess for L in the architectureof a network) such that L̂≪ L can be good enough in terms of numerical accuracy.Second, neural networks are very good at fitting cases within a high-dimensional parameterspace even when a low-dimensional manifold is sufficient. Thus, if one errs by specifying an L≪ L̂(but still L̂≪ N), the performance of the neural networks is unlikely to deteriorate. To show this,in table 2.2, I will solve a model where it is known L = 1 would be sufficient. Nonetheless, theperformance and accuracy of the approximation are nearly identical if one deliberately ignore thatknowledge and set L̂ to 16 and beyond.9Functional representations and the Krusell-Smith method: one can link proposition2 with Krusell and Smith (1998). L̂ corresponds to the dimensions that an agent uses to predictthe law of motion of the distribution of agents in the economy. For example, if only the first twomoments are required to keep track of the distribution, an approximation with L̂ = 2 here wouldbe sufficient. Nonetheless, researchers have discovered that, even when the distribution is infinite-8For instance, let ϕ(Xi) calculate the first N powers of Xi to ensure that the equations are not collinear. Then,ρ(y) solves the system of N equations to reverse-engineer the original X ∈ RN from y ∈ RN before applying the maxitself.9The optimization methods used in deep learning –such as reverse-mode auto-differentiation for gradients– caneffectively deal with high dimensions if the approximation has been transformed to have an easier geometry, as Ido with the representation theorem in this chapter. It is much easier for these optimization algorithms to ignoresmoments than to ignore grid points if I were implementing a projection.13dimensional, a low L̂ is often sufficient to accurately approximate a distribution.When does the symmetry break? This question can be answered by carefully analyzingdefinition 2. The first point where symmetry breaks is when the condition described in equation(2.4) is not satisfied. In such cases, the economy is not invariant to the relabeling of agents,indicating a lack of exchangeability. The second point where symmetry breaks is when the conditiondescribed in equation (2.5) is not satisfied. This condition is met as long as the agents follow asymmetric strategy (equilibrium function). However, when agents follow asymmetric strategies, thesymmetry breaks down. For example, consider an economy with two agents where each follows adifferent equilibrium function. If one were to exchange the agents’ states in this economy, the stateof the economy in the next period would not be a permutation of the state of the original economyin the next period. Although the reward function is exchangeable, the asymmetry in equilibriumchoice renders the agents nonexchangeable.2.2.3 Concentration of measureI complete this section by introducing the idea of concentration of measure. First, I need to have anappropriate notion of boundedness and how it applies to gradients of functions of Gaussian randomvariables.Definition 3 (Bounded functions in N). Let:H(M) ≡ {y ∈ RN : |yi| ≤M ∀i = 1, ..., N}be an N -dimensional hypercube in RN . A function f : RN → R is bounded in N if for every Mthere exists KM such thatsupy∈H(M)|f(y)| < KM ,where KM is a constant that does not depend on N , but may depend on M .A simple example of a symmetric function fulfilling definition 3 is the mean, f(X) = 1N∑Ni=1Xisince supy∈H(M) |f(y)| < M for all N . The main purpose of this definition is to reject functionsthat explode in the number of states, such as the f(X) =∑Ni=1Xi since supy∈H(M) |f(y)| = NM .Definition 4 (Expected gradient bounded in N). Let f : RN → R be a bounded function in N andz ∼ N (0N , IN ) be a normalized Gaussian random vector. The function f has its expected gradientbounded in N if there exists a C such that:E[∥∇f(z)∥2] ≤ CN,where C does not depend on N .Loosely speaking, definition 4 tells us that a function has an expected gradient bounded in N14when its value does not change too fast when its Gaussian random arguments vary by a smallamount.The next proposition delivers the intuitive result that, for a large N , a function f(·) with anexpected gradient bounded in N is essentially a constant. That is, the measure of its values is“concentrating”.Proposition 3 (Concentration of measure when expected gradients are bounded in N). Supposethe random variable z ∼ N (0N , Σ), has a spectral radius, ρ(Σ), independent of N . If f : RN → Ris a function with expected gradient bounded in N and z1 is a single random draw from z, then:P(∣∣f(z1)− E [f(z)] ∣∣ ≥ ϵ) ≤ ρ(Σ)Cϵ21N.Proof. Section A.3.2. Q.E.D.It is worth mentioning that when z is an uncorrelated Gaussian distribution with variance σ2,the variance-covariance matrix can be written as Σ = σ2IN . In this case the spectral radius of Σis equal to σ2 and does not depend on N .10If this is the case, then I can evaluate E [f(z)] with a single Monte Carlo draw z1 fromN (0N , Σ):E [f(z)] ≈ f(z1).It is correct to approximate E [f(z)] by f(z1) than by f(0N ), that is, f(·) evaluated at the meanof the distribution. An N -dimensional Gaussian random variable concentrates on a hypersphereof radius√N , rather than at the origin. This hypersphere is the typical set relevant to computeE [f(z)] when f(·) is not linear. Also, the distance between the typical set and the origin increaseswith N grows.11 Even when N is not sufficiently large for a single Monte Carlo draw to be accurateenough, concentration of measure tells that us one can still get high numerical accuracy when theyevaluate an expectation by drawing a small number of{z1, . . . zM}: E [f(z)] ≈ 1M∑Mi=1 f(zi).There is an important difference between the functional representation in proposition 2 andthe concentration measure, which provides an upper bound on the error of the evaluation of theexpectation. Also, the results of proposition 3 rely on ensuring that definition 4 holds, and doesnot require any symmetry. However, I conjecture that permutation invariance and definition 3 aresufficient for proposition 3 to hold (i.e., definition 4 is implied). Loosely, functions that are invari-ant to permutations either have a concentration of measure or are unbounded in N . In particular,functions like f(X) =∑Ni=1Xi do not fulfill definition 3, but are of limited use since they divergeas N increases.1210There are concentration inequalities that require the function f(z) to be Lipschitz continuous. In this case theGaussian assumption can be relaxed and replaced by sub-Gaussian distributions. The concentration of measure usedin this chapter does not require the Lipschitz continuity assumption.11See Vershynin (2018) and Ferna´ndez-Villaverde and Guerro´n-Quintana (2021) for details.12To understand the intuition, consider that the key criterion for a concentration of measure is that a single randomvariable cannot affect the result a disproportionate amount relative to the others. Permutation invariance impliesthat individual variables affect the function in a symmetric way, and the same applies to their gradients (i.e., if f(X)is permutation invariant, then ∇f(X) is permutation equivariant). If the function does not explode as N increases(i.e., definition 3), then symmetry ensures that all of the elements of ∇f(z) are of the same scale relative to N ,ensuring that definition 4 is fulfilled.15This helps explain part of the Krusell and Smith (1998) intuition where “agents” do not need tocalculate any integrals, since they can simply use the approximate law of motion of the simulationitself (after conditioning on an aggregate shock, as I do here) to form expectations of the distribu-tional variables. Similarly, continuous-time mean-field games (Kaplan et al., 2018) implicitly usethe exchangeability of all agents with identical states to define permutation-invariant functions,and this leads to a concentration of measure for all payoff relevant values.The purpose of concentration of measure: the concentration of measure introduced inproposition 3 justifies the strategy utilized in this chapter for calculating conditional expectationsby using one (or a few) Monte Carlo draws of the idiosyncratic shocks. This proposition establishesthat, under the stated conditions, the number of Monte Carlo draws required to obtain a goodapproximation of conditional expectations decreases as N (i.e., number of agents) increases. Thereare other concentration inequalities in the literature that yield the same result, depending on theproperties of the function of interest. For instance, McDiarmid’s inequality requires f(z) to havebounded differences, or some require the function to be Lipschitz continuous (see Vershynin, 2018 fora comprehensive introduction to concentration inequalities). Loosely speaking, these assumptionson the underlying function such as the expected gradient being bounded in N (as defined in 4),bounded differences, or Lipschitz continuity, exclude functions that depend too much on theirinputs. Depending on the concentration inequality used, different rates than 1N may be obtained.However, they all imply that as N increases, the number of Monte Carlo draws required to achievea good approximation for calculating expectations decreases.2.3 An application: a model of investmentTo illustrate how permutation-invariant dynamic programming and concentration of measure op-erate in an economic application, I propose an extension of the classic model of investment underuncertainty by Lucas and Prescott (1971). I will consider an industry with a large number offirms and follow the recursive formulation in Prescott and Mehra (1980). In contrast to Lucasand Prescott (1971), I will not require that all firms have symmetric states, just that they havesymmetric policy functions.More concretely, I study an industry consisting of N ≥ 1 price-taking firms, each producinga single good using capital under constant returns to scale. By picking appropriate units, I canconsider that a firm i produces output x with x units of capital. I stack the production (or capital)of the whole industry in the vector X ≡ [X1, . . . XN ]⊤.The capital of the firm depreciates at a rate δ ∈ [0, 1], is increased by investment u, and is shiftedby an i.i.d. idiosyncratic shock w ∼ N (0, 1) and a single i.i.d. aggregate shock, ω ∼ N (0, 1), com-mon to all firms (adding persistence for the shocks is trivial but complicates notations). Thus, thelaw of motion for capital is:x′ = (1− δ)x+ u+ σw + ηω.16Although this law of motion allows for capital to be negative, I will consider only the cases wherex > 0 (for instance, because investment is sufficiently large on average in relation to the standarddeviation of the idiosyncratic and aggregate shocks and, therefore, x < 0 only occurs with a smallprobability). Due to adjustment frictions, investing u has a cost, in valuation terms, c(u) = γ2u2.I assume that the inverse demand function for the industry is, for some ν ≥ 1:p(X) = α0 − α1 1NN∑i=1Xνi . (2.10)When ν = 1, this inverse demand function means that the model can be represented as a linear-quadratic (LQ) dynamic programming problem as described in Ljungqvist and Sargent (2018,Section 7.2). When ν > 1, I will have a fully nonlinear model.Since I will be looking at competitive equilibria where a firm is a price taker, the firm does notconsider the impact of its individual decisions on p(X). Also, all firms use their cash flow to financetheir investment, u. Hence, the period profits of firm x are:profits = p(X)x− γ2u2.The dynamic problem of the firm is to choose u to maximize the expected present discounted valueof the firm at discount rate β. The corresponding Bellman equation of the firm takes the evolutionof X as given and determined by the equilibrium policy of all firms, including itself.First, I write such a Bellman equation considering X as a vector, as it is the most commonpractice in economics.Bellman equation with vector X: the firm of interest with capital x takes a symmetricpolicy uˆ(·, X) as given for all other firms in X. Without loss of generality, I assume that the firmis the first in the vector, i.e., x = X1. With this assumption, I can write the recursive problem ofthe firm taking the exogenous policy uˆ(·, X) as:v(x,X) = maxu{p(X)x− γ2u2 + βE[v(x′, X ′)]}(2.11)s.t. x′ = (1− δ)x+ u+ σw + ηω (2.12)X ′i = (1− δ)Xi + uˆ(Xi, X) + σWi + ηω, for i ∈ {2, ..., N} (2.13)X ′1 = (1− δ)X1 + uˆ(X1, X) + σw + ηω. (2.14)Four points deserve further explanation. First, the expectation in equation (2.11) is taken overthe i.i.d. idiosyncratic shocks {w,W2, . . .WN} and the aggregate shock ω. Second, equation (2.12)is the law of motion of capital of the firm given its control u. Third, equation (2.13) is the forecastthe firm makes over the evolution of the distribution of capital of all firms given the exogenous pol-icy. Fourth, equation (2.14) takes care of the contribution of the firm’s own policy to the aggregatestate X. When σ > 0, I can let the firm consider that w = W1. When N is sufficiently large, this17becomes irrelevant for the optimal u and I could remove the special case of that correlated shock,combining equation (2.14) into equation (2.13). When N = 1, this formulation nests the textbook“big-K, little-k” example derived in Ljungqvist and Sargent (2018), Section 7.2.Next, I show that it is easy to rewrite the previous problem considering X as a set. While bothformulations look similar cosmetically, the conditions for treating the aggregate distribution as aset rather than a vector are essential to my method.Bellman equation with set X: to use the results, I need to show that the investmentmodel described in equations (2.11) to (2.14) satisfies all the conditions of a permutation-invariantdynamic programming problem in definition 2.First, to show that the payoff is permutation invariant according to equation (2.4), it is sufficientfor the p(X) function in equation (2.10) to be permutation invariant. This is easily shown sincethe function is an affine transformation of a power mean.Second, the law of motion must be permutation equivariant according to equation (2.5). SectionA.4.1 shows this formally, but the intuition is that, conditional on a permutation-invariant uˆ(·, X),I can apply a permutation π to reorder any of the indices in equations (2.13) and (2.14), i.e.,X ′i = (1 − δ)Xi + uˆ(Xi, X) to X ′j = (1 − δ)Xj + uˆ(Xj , πX) since uˆ(Xj , πX) = uˆ(Xj , X). Thepermutation equivariance is a direct consequence of symmetric equilibrium.Finally, to show that the covariance matrix fulfills condition (2.6), from equations (2.13) and(2.14), notice that the idiosyncratic shocks of other agents are embodied in the identity matrixtimes σ. Therefore, for all permutations π ∈ SN , πσIN = σIN .Hence, by proposition 1, the solution to the investment model (u, v) is permutation invariantand can be treated as a function on sets rather than a vector space (i.e., the ordering of the inputsdoes not matter). If I create a tuple of the states and the corresponding shocks as (X,W ), the setequivalent of equations (2.13) and (2.14) becomes:X ′ = {(1− δ)xˆ+ uˆ(xˆ, X) + σwˆ + ηω for all (xˆ, wˆ) ∈ (X,W ) \ (x,w)}∪ {(1− δ)x+ uˆ(x,X) + σw + ηω} . (2.15)As before, the special case of the x in expression (2.15) is to ensure the firm does not take itschoice into account when forecasting X ′, but can still allow for the correlation of shocks when cal-culating expectations. While this is important for correctness with very small N –and in particularnesting the N = 1 case– it has almost no quantitative impact as N increases. For that reason, Iwill drop the special case in further calculations in this section, effectively decoupling x from theset X. Since the forecast X ′ is only used for expectations, this is equivalent to assuming that thefirm does not take into account that its contribution to X is correlated with its own shock whencalculating expectations of X ′.18Using this simplification, I can rewrite equations (2.11) to (2.14) as:v(x,X) = maxu{p(X)x− γ2u2 + βE[v(x′, X ′)]}(2.16)s.t. x′ = (1− δ)x+ u+ σw + ηω (2.17)X ′ = {(1− δ)xˆ+ uˆ(xˆ, X) + σwˆ + ηω for all (xˆ, wˆ) ∈ (X,W )} . (2.18)The expectation in equation (2.16) is taken over w ∈ R, ω ∈ R, and W ∈ RN , all independentnormal shocks. This reformulation allows me to solve for the equilibrium with functions on setsrather than vectors and to use the representation in proposition 2.Since the problem fulfills the requirements of definition 2 for any permutation-invariant uˆ(·, ·),proposition 1 tells us that we have a permutation-invariant optimal solution. Since in the symmetricequilibrium every agent solves the same policy, the uˆ(x,X) is also permutation invariant, whichis needed to ensure that the law of motion, equation (2.5), is equivariant. Thus, I can define asymmetric competitive equilibrium.Definition 5. An equilibrium is a v(x,X) and uˆ(x,X) such that:• Given uˆ(x,X), v(x,X) is the value function solving equation (2.16) for each agent and u(x,X)is the optimal policy function.• The optimal policy is symmetric, i.e., u(x,X) = uˆ(x,X).In this chapter, I focus on problems for which the optimal solution is unique and can be foundvia an Euler equation, though my techniques are not specific to that class of models.Proposition 4. For a permutation-invariant p(X), the Euler equation takes the form:γu(x,X) = βE[P (X ′) + γ(1− δ)u(x′, X ′)] (2.19)X ′ = {(1− δ)xˆ+ u(xˆ, X) + σwˆ + ηω, for (xˆ, wˆ) ∈ (X,W )} . (2.20)Proof. See section A.3.3. Q.E.D.The economic interpretation of the Euler equation (2.19) is standard. The firm weighs themarginal cost of an additional unit of investment, γu(x,X), against the benefit of additional profitsfrom that marginal increase in output at a price p(X ′) plus the value of investment after deprecia-tion. That result is, however, incidental.Dependence on x: as evident in the Euler equation, described in equation (2.19), u(·, ·) doesnot depend on x. This is due to the payoff structure chosen in this application. However, this isincidental. The payoff function, r(x,X, u) = p(X)x − γ2u2, is a linear function of x. Therefore, inthis case, one can omit x as an input and rewrite the Euler equation and the laws of motion as:19γu(X) = βE[P (X ′) + γ(1− δ)u(X ′)] (2.21)X ′ = {(1− δ)xˆ+ u(X) + σwˆ + ηω, for (xˆ, wˆ) ∈ (X,W )} . (2.22)In the general case where u is a function of x, one can approximate E [u(x′, X ′)] conditional onthe first agent’s idiosyncratic shock and the aggregate shock, using one draw (or a few) of theidiosyncratic shocks of the other agents. Subsequently, using Gaussian quadrature method one cancalculate the expectation with respect to the first agent’s idiosyncratic shock and the aggregateshock.The logic of the procedure stated above to calculate E [u(x′, X ′)] can be extended. The reasonfor conditioning on the first agent’s idiosyncratic shock and the aggregate shock was due to thesignificant dependence of the optimal policy function u on those variables. To be more specific, asthe number of agents (N) increases, the influence of those variables remains significant and doesnot diminish. In general, if there are variables whose influence does not diminish as N increases,one can approximate the conditional expectation, conditioned on those variable, by using one (ora few) Monte Carlo draws, and use other low-dimensional methods such as Gaussian quadraturemethod to calculate the expectation with respect to those variables.Dimensionality of the problem: while seemingly simple, the dimensionality of the Eulerequation (2.19) is nontrivial. If N = 1, 000, then u(x,X) is a policy function on a 1, 001 dimen-sional state-space. Furthermore, even given a u(x,X) function, the expectation is taken over 1, 001dimensions. It is not feasible to solve this problem with standard methods.However, I can use the results of the previous sections to solve this problem. First, consider theexpectation in the simple case of η = 0 but σ > 0. While considering 1, 000 individual forecasts ofelements of X ′ from equation (2.20) is computationally intractable, a firm may have an accurateforecast of functions of the distribution of X ′. In fact, the firm does not care about the individualsin X ′ unless its u(·, X) depends a great deal on a single element of X.In the case where firms do not care about the individual elements of X ′, a forecast of the dis-tribution is good enough. Furthermore, when η = 0 and N is large, the distribution evolves almostdeterministically. Similarly, even when there are aggregate shocks, an agent may have an accurateforecast of functions of the distribution conditional on the realization of an aggregate shock. Theextreme case of this result appears in models a` la Aiyagari-Krusell-Smith. With a continuum ofagents, one only cares about the distribution, not the position of each agent on it.Next, consider the structure of u(·, X). Since the price function was invariant to permutations,one knows that the policy function will be invariant as well. Hence, even if the firm cannot accu-rately predict an individual xˆ ∈ X without large amounts of computations, only needs to predictthe general distribution of X ′, rather than each element in it.If u(·) and p(·) are functions of a large number of random xˆ′ ∈ X ′, but do not depend too much20on any individual one of them (as defined by the notion of bounded gradients, definition 4), thenthe expectation conditional on the aggregate shock may be close to a deterministic function. Then,p(X ′) and u(·, X ′) are fully predictable. Thus, for a large enough N , I can exploit proposition 3and use a single Monte Carlo draw of individual shocks to obtain an accurate approximation.In the case of a linear p(·), I could rely on certainty equivalence and normalize wˆ ∈W to havezero mean. However, when p(·) is not linear, I draw one random W . As emphasized in Subsection2.2.3, high-dimensional expectations of a function f(·) are not well approximated by the functionevaluated at its mean, but are well approximated with random draws from the distribution itself.2.4 A deep learning implementationThe goal is to find a policy function, u(X), that satisfies proposition 4 (I will drop the dependenceof u on x when I want to highlight that I am searching for the policy function of all firms). WhileI am interested in a global solution, I will not require that solutions be accurate for all X ∈ X(the domain of u). I just want the u(·) to generalize well on paths {X1t , X2t , . . . XJt }Tt=0 that arestarting from some distributional initial conditions{X10 , X20 , . . . XJ0}of interest. Fortunately, aswill be shown later, the solutions I find are quite often accurate in a much larger set of points.The strategy is to exploit the representation in proposition 2:u(x,X) = ρ(x,1NN∑i=1ϕ(Xi)),since it encodes the symmetry built into the approximate functions. I approximate ρ, ϕ, and Lusing a deep learning architecture F(θ), where θ is the vector of parameters (coefficients) in thenetwork. As a result, I get a parameterized policy function u(x,X; θ) that is as close as possible tou(x,X) under some appropriate metric. This strategy is much more efficient than implementing ablind neural network and resembles similar applications in other environments.132.4.1 Architecture of the networkI specify three different network architectures to deal with ϕ. First, I approximate ϕ as a function ofa first moment ofX a` la Krusell-Smith, but allowing this moment to enter in a fully nonlinear way asin Ferna´ndez-Villaverde et al. (2019). I call this approximation ϕ(Identity). Second, I approximateϕ as in the first architecture, but using four moments. I call this approximation ϕ(Moments).Third, I approximate ϕ with a flexible ReLU network, with one layer with 128 nodes.14 I call this13For instance, Hartford et al. (2016) use permutation-equivariant symmetry to reduce dimensionality in the ap-proximation of best responses in a normal form game.14A rectified linear unit (ReLU) is an activation function f(x; θ1, θ2) = θ1max (0, θ2x, ), where the max is thepoint-wise maximum, and θ1 and θ2 are weights found in the training procedure.21approximation, interchangeably, ϕ(ReLU) or Deep Sets. Across all of their layers, the baselineϕ(Identity), ϕ(Moments), and ϕ(ReLU) have 49.4K, 50.3K, and 199.6K parameters respectively,regardless of N .Figure 2.1: Moments architecture; M is the number of moments.In these three architectures, ρ is represented by a neural network with four layers. Each layerof ρ in ϕ(Identity) and ϕ(Moments) has 128 nodes. Each layer of ρ in ϕ(ReLU) has 256 nodes.Figures 2.1 and 2.2 illustrate these three architectures.Figure 2.2: ReLU/Deep Sets architecture.2.4.2 Training the networkAs mentioned before, due to the payoff structure (i.e., p(X)x− γ2u2) and its linearity in x, u doesnot depend on x, therefore I replace u(x,X) with u(X).I train the network by minimizing the Euler residuals defined when I plug u(X; θ) into equation22(2.21) instead of u(X), and using equation (2.22) for the law of motion of X ′:ε(X; θ) ≡ γu(X; θ)− βE [P (X ′) + γ(1− δ)u(X ′; θ)] . (2.23)More concretely, I evaluate ε(X; θ) using simulations from a single initial condition and only16 trajectories from t = 0 to 63 (including both aggregate and idiosyncratic shocks). I call theevaluated Euler error, in each simulation j and period t, εj,t(X; θ). I use the concentration ofmeasure described in section 2.2.3 to calculate the integral over theW shocks in equation (2.21). Todo so, though, I must condition on the aggregate ω shock to ensure that definition 4 holds. Finally,I use Gaussian quadrature method to calculate the expectation with respect to the aggregate shock.Finally, I minimize the sum of square Euler errors:minθ1J × (T + 1)J∑j=1T∑t=0(εj,t(X; θ))2 (2.24)using ADAM, a stochastic gradient descent with adaptive moment estimation, although any othersuitable optimization algorithm could be applicable.The pseudocode below summarizes the steps.Algorithm Network training1: Given by network architecture F(θ) for u(X), define the Euler residuals:ε(X; θ) ≡ γu(X; θ)− βE [P (X ′) + γ(1− δ)u(X ′; θ)]2: Pick {Xj(0), ..., Xj(T )} for j = 1, .., J trajectories given some initial point of interest.3: Evaluate εj,t(X; θ) for some or all the points above.4: Solve:minθ1J × (T + 1)J∑j=1T∑t=0(εj,t(X; θ))2I highlight three additional points. First, I found in preliminary testing (specially for the DeepSets architecture) that it is not important for the performance of the training that the transitionbe very close to any stationary solution (i.e., this does not require a hidden transversality condi-tion or some form of backward induction from a steady state) or that a small or large number oftrajectories are used. In fact, in section 2.4.5, I solve a version with only a few of those data points.Second, there are different possibilities regarding the choice of points to evaluate the Eulererrors. The points can be some collocation nodes, simulated points a` la Rust, or many others.The main divergence from collocation-style approaches is that, when I minimize the Euler errors,with relatively few points in X and ignore stationarity in both the simulation and the solution.My method succeeds because, as I will discuss later, the approximations it finds for u(·) i) gen-eralize and extrapolate well outside of the evaluated points, this is due to use of symmetry andthe representation theorem ; and (ii) provide stable solutions that converge to the ergodic sets23and do not violate transversality. Neither of those conditions would hold, in general, for classicalcollocation-style solutions.Third, a surprising benefit of a high-dimensional approximation is the “double-descent” phe-nomenon in machine learning (see Belkin et al., 2019, and Advani et al., 2020). In classical statistics,the bias-variance trade-off tells us that the generalization error of a function approximation becomeslarge when the number of free parameters (capacity) approaches the number of data points (at thelimit, when the number of free parameters is equal to the number of data points, one is just inter-polating). For that reason, one typically wants to choose a smaller number of free parameters thanthe number of data points. Furthermore, it is known that when one is exactly interpolating, theapproximation can be especially very poor exactly at the edges (e.g., Runge’s phenomenon), whichalso makes extrapolation fail in general. The double-descent phenomenon, which is shown withmany machine learning algorithms, is that if one further increases the number of parameters farabove this interpolation threshold, then the generalization error begins to decrease again. Often,the cure to over-fitting is to add more parameters. I explore this topic in section 2.4.5 and the nexttwo chapters, where I consider the lack of treatment of transversality conditions and stationarityin more detail.2.4.3 Case I: ν = 1I implement the deep learning architectures by fixing the economic model parameters at conven-tional levels: σ = 0.005, η = 0.001, α0 = 1, α1 = 1, β = 0.95, and γ = 90. To match a commonfeature of the data, I make the standard deviation of the idiosyncratic shock much larger than thestandard deviation of the aggregate shock. Also, in this way, concentration of measure phases inmore slowly, better showing its power to approximate expectations even in challenging parameter-izations.In this subsection, I set ν = 1, the last parameter of the model. I pick this parameterizationbecause, with it, the investment model becomes an LQ dynamic programming problem that I cansolve using classical optimal control as described in Anderson et al. (1996). This provides an exactsolution (up to machine precision) with the form:u(X) = H0 +1NH1N∑i=1xi,where H0 and H1 are scalars. In this case, regardless of the number of firms N , the problem is twodimensional (i.e., there are two unknowns H0 and H1). See section A.4.4 for the functional repre-sentation of v(x,X). I can use this exact solution to benchmark the deep learning approximations.Notice how in terms of proposition 2,u(X) = H0 +1NH1N∑i=1xi = ρ(1NN∑i=1ϕ(Xi)),24where ϕ : R → R1 (i.e., L = 1) is the identity function ϕ(X) =[X]and ρ : R1 → R such thatρ(y) = H0+H1y for some undetermined H0 and H1. However, I do not supply this information tothe deep learning architectures. I want to gauge whether this information can be “learned” by thedeep neural networks .Figure 2.3 plots investment decision u(Xt) along the equilibrium path of an economy with 128firms and aggregate shocks over a simulation of 63 periods for four solutions. The blue dashed tra-jectory provides the LQ solution, the orange trajectory depicts the solution obtained by ϕ(Identity),the green trajectory depicts the solution obtained by ϕ(Moments), and the red trajectory depictsthe solution obtained by ϕ(ReLU)/Deep Sets. All four paths are almost identical. This shows thatthe deep learning solutions are incredibly accurate even when, as in this figure, the initial conditionsare such that u(X0) (slightly below 0.024) is far from its mean value in the ergodic distribution ofthe economy (around 0.034).0 10 20 30 40 50 60Time(t)0.02500.02750.03000.0325u(Xt) with φ(Identity), φ(Moments) and φ(ReLU) : Equilibrium Pathu(Xt), LQu(Xt), φ(Identity)u(Xt), φ(Moments)u(Xt), φ(ReLU)43 440.03420.0343Figure 2.3: Comparison between the LQ-regulator solution and the three deep learning architecturesfor the case with ν = 1 and N = 128.There are three sources that the method proposed in this chapter might fail in finding thebenchmark solution:1. Violation of transversality: in the linear case (i.e., ν = 1) the Euler equation leads to aquadratic equation with two roots. One of these roots yields an explosive solution. In thiscase firms produce large amounts of capital leading to negative prices. It is worth mentioningthat in theory this solutions is an interpolating one and exactly solves equation (2.23). Inthe method proposed in this chapter there is no explicit device to rule out this solution.Therefore, it is expected that half of the approximate solutions (depending on initializationof the parameters of the neural networks) will converge to the explosive root.2. Early stopping failure: algorithms that use deep learning are inherently stochastic proce-25dures.15 Given this stochasticity some solutions might not converge in a reasonable amountof time. In the optimization process I stop the procedure early if the mean squared of theEuler residuals of the validation data reaches a certain level (i.e, 10−6). Early stopping failurehappens if this criterion is not met within a certain amount of optimization steps.3. Overfitting: when the approximating function has high complexity and is very flexibleoverfitting might occur. For instance ϕ(Moments)and ϕ(ReLU) are potentially prone tooverfitting. In the linear case the first moment of X is enough to obtain an accurate solution.However, ϕ(Moments)utilizes four moments, and ϕ(ReLU) uses four (L = 4) fully flexibleneural networks. The criterion for overfitting failure is when the mean squared Euler residualsof the test data (i.e., trajectories simulated by the approximate solution) exceed a certainthreshold. In this chapter 10−4 is chosen as this threshold.Table 2.1 reports the success rate for three baseline network architecture ϕ(Identity), ϕ(Moments),and ϕ(ReLU) (Deep Sets) for 100 number of trials. In each trial I use a different random initial-ization of the parameters for these architectures. Table 2.1 also provides three categories of failureof convergence.Because there are two possible solutions, one explosive and one non-explosive, one can expectto find the explosive solution (which violates the transversality condition) with a probability of50%. The results indicate that this is true for ϕ(Identity). However, it becomes less prevalentfor ϕ(Moments). Surprisingly, the probability of finding the explosive solution withϕ(ReLU) isextremely low (i.e., 3%). Additionally, ϕ(ReLU) that is most susceptible to overfitting shows nosign of ovefitting. This phenomena can be explained by the implicit bias of deep neural networksand their optimizers. Section 2.4.5 and chapters 3 and 4 provide a deeper understanding this phe-nomena.In order to test the accuracy of the approximate solutions, I solve the optimization problemdescribed in (2.24) for 100 random initialization (i.e., seeds) of the parameters θ. I collect thesolutions that have converged and generate 32 different trajectories for each of them. Figure 2.4shows the median of policy errors, ϵrel defined asϵrel(X) ≡∣∣u(X)− uref(X)uref(X)∣∣, (2.25)where uref is the LQ-regulator solution. The policy error can be interpreted as the absolute valueof the relative error between the approximate and closed-form solution. The shaded areas show10th-90th and 25th-75th percentiles. The left panel shows the results for ϕ(Identity), the middlepanel shows the results for ϕ(Moments), and the right panel shows the results for ϕ(ReLU). Whileall architectures are highly accurate, ϕ(ReLU) architecture is especially stable. This result can beinterpreted as the function generalizing very consistently because of —rather than in spite of— the15This stochasticity stems from two main sources. First, the initial values of the parameters are chosen randomly.Second, the optimization algorithms, such as stochastic gradient descent, use a random “batch” of data in each stepof the optimization.26extra parameters of this architecture.Table 2.1: Convergence of baseline network architectures in solving case I: ν = 1Success(%)Violation of transversality(%)Early stopping failure(%)Overfitting(%)Group DescriptionIdentity Baseline 48% 50% 2% 0%Moments Baseline 59% 37% 2% 2%Deep Sets Baseline 97% 3% 0% 0%0 20 40 60Time(t)10−410−3Policy errors (²rel) with φ(Identity)Median10th and 90th percentiles25th and 75th percentiles0 20 40 60Time(t)10−410−3Policy errors (²rel) with φ(Moments)Median10th and 90th percentiles25th and 75th percentiles0 20 40 60Time(t)10−410−3Policy errors (²rel) with φ(ReLU)Median10th and 90th percentiles25th and 75th percentilesFigure 2.4: The policy errors for ν = 1 and N = 128 for ϕ(Identity), ϕ(Moments), and ϕ(ReLU).The dark blue curve shows the median along equilibrium paths. The shaded areas show 10th-90thand 25th-75th percentiles.While these figures show that the solution is very accurate, I need to gauge whether the per-formance is practical, and whether the curse of dimensionality has been sidestepped. Figure 2.5shows the performance for ϕ(ReLU), the more densely parameterized architecture, as a function ofN (number of agents). The left panel shows the computation time in seconds.16 The right panelshows the policy error (i.e., ϵrel) on the test data. The reported numbers are the median values of100 trials with different seeds. The shaded areas show 10th-90th and 25th-75th percentiles. Thesolid blue lines show the median.The algorithm finds the global solution of the model with high accuracy in less than a minute.Furthermore, this computation is not a systematic function of N , but of order O(1) for reason-able N . The variation in solution time is largely due to the use of random initial conditions in a16The time is calculated from the beginning of the fitting process until the mean squared Euler error reaches1 × 10−6, which is chosen to be on the same order of magnitude as the error coming from the concentration ofmeasure approximation of the integral in Figure 2.8 when N =128.27high-dimensional space (the 199.6K parameters).17102 103 104 105N20406080100120Computation time (seconds)Median10th and 90th percentiles25th and 75th percentiles102 103 104 105N10−410−3Policy errors (²rel)Median10th and 90th percentiles25th and 75th percentilesFigure 2.5: Performance of the ϕ(ReLU) for different N .Finally, table 2.2 shows the results for different variations in the approximations, ϕ(Identity),ϕ(ReLU), and ϕ(Moments). The columns are as follows: Success shows the percentage of solutionsthat converged (over 100 random seeds). Time represents the median of the computation timemeasured in seconds. Parameters represents the number of parameters (θ) in functional approxi-mation, in thousands. Train MSE reports the median of mean squared error of the Euler residualsfor the simulated trajectories used for training. Val MSE displays the median of the mean squarederror on the validation data, that is, trajectories used to check the quality of the generalizationduring the training process, but where the data are not used to fit the model directly. Test MSEdocuments the median of the mean squared error of the Euler residuals on simulated data afterthe training is complete, and which internally uses the approximation itself during the simulation.Policy error (ϵrel) is the median of absolute value of the relative error of the approximate solutionand the accurate solution (uref) on the test data. The results in table 2.2 are the median among100 trials each with different seeds.17It is worth mentioning theses results are produced using GPU. For mid-range values of N (< 250) CPU yieldsfaster convergence with the same levels accuracy. See the Time column in table 2.2.28Table 2.2: Convergence of different network architectures in solving case I: ν = 1Success(%)Time(s)Parameters(Thousands, K)Train MSE(ε)Val MSE(ε)Test MSE(ε)Policy Error(ϵrel)Group DescriptionIdentity Baseline 48% 28 49.4 6.7e-07 4.9e-07 5.0e-07 0.10%MomentsBaseline: Moments (1,2,3,4) 59% 36 50.3 9.0e-07 8.7e-07 1.2e-06 0.13%Moments (1,2) 61% 35 50.0 1.1e-06 8.8e-07 1.0e-06 0.12%Very Shallow (1 layer) 0% NaN 0.8 NaN NaN NaN NaNDeep SetsBaseline: L= 4 97% 17 199.6 1.3e-06 3.6e-07 3.6e-07 0.09%L = 2 93% 17 201.1 1.3e-06 4.0e-07 4.3e-07 0.09%L = 16 93% 14 204.2 1.5e-06 3.5e-07 3.5e-07 0.10%Deep (ϕ : 2 layers, ρ : 4 layers) 100% 25 215.9 2.0e-06 3.8e-07 3.7e-07 0.10%Shallow (ϕ : 1 layer, ρ : 2 layers) 1% 16 68.0 1.6e-07 3.3e-07 3.5e-07 0.10%The results in table 2.1 show that regardless of the variations in the architecture of Deep Sets,one can robustly obtain a very accurate solution. Making this architecture deeper can enhance thesuccess ratio to 100%. Additionally, making this architecture shallow can drastically reduce thesuccess ratio, and the same is also true for the Moments architecture.2.4.4 Case II: ν > 1Now, I move to the case where ν > 1, i.e., the inverse demand function is nonlinear, while all theother economic parameter values are the same as before. Nonetheless, I can find the solution tothis nonlinear case using exactly the same algorithm, code, and functional approximations as insubsection 2.4.3, I just change, in the code, the value of ν and drop the identity architecture since,by construction, it cannot encode the nonlinearity in the problem. As before, I fit the model andcheck its generalization with a set of simulated trajectories.However, in this situation, the LQ solution no longer holds, and I do not have an exact solutionagainst which to benchmark the deep learning solution. Instead, in figure 2.6 I show the resultof median Euler residuals for converged solutions over 100 seeds and 32 simulated trajectories foreach seed, with the corresponding 10th-90th and 25th-75th percentile bandwidth. The economyuses ν = 1.5 and N = 128. The left panel shows the results for ϕ(Moments) and the right panel theresults for ϕ(ReLU). As in figure 2.4, the mean squared error is fairly low. In the non-linear case,where the first moment of X is not enough and and the exact solution is not known, one can seethe full benefits of using ϕ(ReLU)/Deep Sets. The Deep Sets architecture systematically provideslower Euler residuals. The main conclusion is that one indeed finds a highly accurate solution, asin the linear case. It is worth mentioning that similar to the linear case, the Deep Sets architecturehas substantially higher success ratio than ϕ(Moments), 99% versus 49%. Therefore, in both linearand non-linear cases it is preferable to use the Deep Sets architecture.As stated in section 2.4.3, minimizing the Euler residuals is not enough and one has to ensurethat the solution is non-explosive. I investigate the qualitative behavior of the approximate optimal290 20 40 60Time(t)10−810−710−610−5Euler residuals squared (ε2) with φ(Moments)Median10th and 90th percentiles25th and 75th percentiles0 20 40 60Time(t)10−810−710−610−5Euler residuals squared (ε2) with φ(ReLU)Median10th and 90th percentiles25th and 75th percentilesFigure 2.6: The Euler residuals square for ν = 1.5 and N = 128 for ϕ(Moments) and ϕ(ReLU).The dark blue curve shows the median of the residuals along equilibrium paths. The shaded areasshow 10th-90th and 25th-75th percentiles.0 10 20 30 40 50 60Time(t)0.0250.0300.035u(Xt) with φ(ReLU): Equilibrium Pathν = 1.5ν = 1.3ν = 1.0Figure 2.7: The optimal policy u along the equilibrium paths for ν = [1.0, 1.3, 1.5] and N = 128.Each path shows the optimal policy for a single trajectory.solution, u(·), as ν approaches one, which coincides with the linear case, where an accurate solutionexists. Figure 2.7 depicts the optimal policy computed with ϕ(ReLU) and N = 128 along equi-librium paths with aggregate shocks and initial conditions far away from the mean of the ergodicdistribution for ν = [1.0, 1.3, 1.5].Figure 2.7 shows how the optimal policy for a trajectory smoothly approaches the known ana-lytical solution as ν approaches one, establishing the continuity (as function of ν) and stationarityof the deep learning approximation.30Table 2.3: Convergence of ϕ(ReLU), linear case (ν = 1) with few data pointsSuccess(%)Train MSE(ε)Test MSE(ε)Policy Error(ϵrel)Number of data points2 25% 1.4e-13 2.2e-05 0.25%3 32% 6.8e-11 8.1e-06 0.15%4 46% 2.5e-09 1.3e-05 0.14%5 47% 6.4e-09 1.8e-05 0.21%2.4.5 Generalization: where is the transversality condition?Intriguingly, to get the approximation, I neither applied a transversality condition nor solved astationary problem. While I fitted the data to a large time period T where the equilibrium wasgetting close to some ergodic distribution, I did not use that behavior as a special case. However,in general, I can have many candidate solutions that satisfy the first-order conditions of a dynamicmodel if I do not constrain the long-run behavior of these solutions in one form or another. Infact, that is what I did to get the LQ solution, the classic control solutions implicitly applied atransversality condition. And, yet, the neural networks (specially Deep Sets) consistently pickedthe solution that generalizes best.18This question can be recast in terms of generalization and regularization. First, why would thealgorithm choose the “right” solution to the functional equation in the absence of explicit handlingof the long-run behavior? Second, if the algorithm chooses a particular solution (rather than arandom one), is it the desired one (as measured by its ability to generalize)?In terms of answering the first question, one might suspect that this is because I am simulatingwith a great deal of data, but this is not the case. To show this, I can conduct an even moreextreme example by fitting ϕ(ReLU) baseline which has 199.6K parameters using only few randomdata points in R128 for the linear demand function.19 Recall that, in section 2.4.3, I showed thatthe exact solution only has two parameters (i.e., H0 and H1). Therefore, the neural network isheavily over-parameterized (2≪ 199600).The results are shown in table 2.3. In this experiment, I use ϕ(ReLU) baseline. The modelfits using random “batches” of data of size 2, where the randomization in the stochastic gradientdescent is an essential part of regularizing the solutions.For each experiment I randomly draw few points (e.g., 2 or 3) from simulated trajectories and fitthe model over 100 seeds and for each seed generate 32 trajectories with the approximate functionfor the testing procedure.Success ratio shows the percentage of experiments that converged (i.e., successful).20 Train MSE18With shorter simulations, the generalization error for very large t > T worsens because there is an insufficientnumbers of data points in a domain of interest; it is not a consequence of whether the system is ergodic.19In this case, the solution is consistent with a transversality condition. In other words, imposing a stationarysolution boundary condition only needs 2 points to interpolate.20In the linear case the closed-form solution is known, therefore I use ϵrel, defined in equation (2.25), as the31(ε) documents the median of the mean squared error of the Euler residuals on the few training datapoints. Test MSE (ε) displays the median of the mean squared error of the Euler residuals on thetest trajectories. Policy errors shows the median of policy errors (absolute value of the relativeerrors) on the test trajectories.It is evident that in this case I do not need many data points to obtain a very accurate approxi-mation. Despite being extremely over-parameterized, ϕ(ReLU) can achieve accurate generalizationbeyond few data points used in the training procedure.The results show that with five data points one can obtain a solution that satisfies the “right”long-run behavior. Additionally, this solution generalizes very well on the test data.While the theoretical understanding of this over-fitting phenomenon is still an active area ofresearch within computer science, it is robustly found in many functional approximation problems.In application after application, using first-order stochastic optimization methods with highly over-parameterized models tends to robustly find solutions that fulfill a minimum norm solution in thespace of approximating functions. Recent work on this issue includes Arora et al. (2018), Allen-Zhuet al. (2019), and Nakkiran et al. (2019).For the specific problem of this chapter, the minimum norm solution seems to coincide withthe most generalizable solution and it seems to automatically fulfill the boundary-value conditions(e.g., a transversality condition or a boundary at a terminal period) that is often needed to applydirectly when approximating with lower-dimensional models.Chapter 3 and 4 of this thesis study this phenomenon and its implications for dynamic prob-lems in economics. I show this result is not specific to the problem studied in this chapter andthe minimum norm solution coincides with solutions that satisfy the optimal log-run behavior in awide range of dynamic problems in economics.2.5 What about concentration of measure?In section 2.4 I establish how symmetry allows me to design an efficient deep learning architecture.This section aims to provide a clear understanding of the concentration of measure.To investigate this point, I return to the case where ν = 1 and propose a modified LQ regulatorwhere I use Monte Carlo to evaluate the required expectations instead of relying on certaintyequivalence. By comparing the exact solution that I get from the classic LQ regulator with theapproximated solution from the modified LQ regulator, I can assess the importance of concentrationof measure and explicitly demonstrate permutation invariance.2.5.1 The linear-quadratic regulator formulationI start by working with a vector (notice that I am not applying proposition 4 here, now it is justto simplify notation). As before, without loss of generality, assume that the x in equation (2.17) isthe first element of X. Denote the state of all firms and a constant term as x⃗ ≡[1 x X⊤]⊤ ≡convergence criteria. I define an experiment as successful if ϵrel of the test data is less than 0.01.32[1 x x X2 . . . XN]⊤ ∈ RN+2 and the vector of all Gaussian shocks as w⃗ ≡ [ω W⊤]⊤ =[ω w W⊤-1]⊤=[ω w W2 . . . WN]⊤ ∼ N (0N+1, IN+1). The set of idiosyncratic shocks ofall of the other firms is denoted W-1 for convenience.Next, define π ∈ S to be a permutation represented by an (N+2)×(N+2) permutation matrixwhere the lower-right N ×N block is in SN , i.e., holding the first and second entries fixed. Thatis, I am interested in permutations of the formπ = 1 0 0⊤N0 1 0⊤N0N 0N πN , (2.26)where πN ∈ SN as defined in section 2.2.1.Defining the following matrices:A ≡ 1 0 0⊤N0 1− δ 0⊤NH01N 0N (1− δ)IN + H1N 1N1⊤N (2.27)B ≡[0 1 0⊤N]⊤(2.28)C ≡ 0 0 0⊤N−1η σ 0⊤N−1η1N σ1N σIN−1 (2.29)R ≡ 0α02 0⊤Nα02 0 − α12N 1⊤N0N − α12N 1N 0N0⊤N (2.30)Q ≡[γ2], (2.31)I can write the problem as an LQ optimal regulator as derived in section A.4.1.2.5.2 The solutionI solve the LQ formulation using the following proposition.Proposition 5 (Monte Carlo LQ formulation). Taking H0 and H1 as given, each firm solves:v(x⃗) = maxu{−x⃗⊤Rx⃗− u⊤Qu+ βE [v(x⃗ ′)]} (2.32)s.t. x⃗ ′ = Ax⃗+Bu+ Cw⃗ (2.33)33With solutions characterized by a P , d, and F :v(x⃗) = −x⃗⊤Px⃗− d (2.34)u(x⃗) = −Fx⃗. (2.35)A symmetric recursive competitive equilibrium is defined as H0 and H1 such that:F = −[H0 0H1NH1N . . .H1N]= −[H0 0H1N 1⊤N]. (2.36)Furthermore, both the control u(·) and value function v(·):1. are invariant to permutations π ∈ S as defined by section 2.26;2. have expected gradients bounded in N , according to section 4:∇W-1u(x⃗ ′) =σH1N1N−1; (2.37)3. have a concentration of measure according to proposition 3 when calculating with a MonteCarlo draw, x⃗ ′1, rather than certainty equivalence, which leads to —for some Cv independentof N :P(∣∣u(x⃗ ′1)− E [u(x⃗ ′) ∣∣ w,ω, x⃗] ∣∣ ≥ ϵ) ≤ σ2H21ϵ21N − 1 (2.38)P(∣∣v(x⃗ ′1)− E [v(x⃗ ′) ∣∣ w,ω, x⃗] ∣∣ ≥ ϵ) ≤ σ2C2vϵ21N − 1; (2.39)Also, when using a Monte Carlo draw instead of certainty equivalence, the Euler residual ε(X;u)is a Gaussian random variable:ε(X;u) ∼ N(0,N − 1N2β2σ2[α1 − γ(1− δ)H1]2). (2.40)Similarly, the error in forecasting the policy in the next period is Gaussian random variable as well:u(x⃗ ′1)− E [u(x⃗ ′) ∣∣ w,ω, x⃗] ∼ N(0, N − 1N2H21σ2). (2.41)Proof. See section A.4.1 for the proof using classic LQ-Gaussian control, section A.4.2 for permu-tation invariance, and section A.4.3 for concentration of measure.As a demonstration of some of the symmetry properties, I sketch out the proof for u(·) inproposition 5, where I take the functional form of F as given.1. Since F as defined in equation (2.36) is identical for all but the first two elements, it alignswith the block structure of the permutations. Hence, for any of the π in equation 2.26,Fπ = F , which ensures that u(·) is symmetric in all changes to the X vector.342. The gradient of an affine function is constant. Hence, for any w⃗, the policy next period isu(Ax⃗+Bu+Cw⃗). Given the block structure of F and C, I differentiate with respect to W-1in the w⃗ to find:∇W-1u(Ax⃗+Bu+ Cw⃗) =σH1N1N−1. (2.42)Therefore, the square of the norm of the gradient of the policy can be bounded as:E[∥∇W-1u(x⃗ ′)∥2] =(N∑i=2σH1N)2≤ σ2H21N − 1 . (2.43)3. SinceW-1 ∼ N (0N−1, IN−1), the spectral radius of IN−1 is 1 and the concentration of measurefollows from equation (2.37) and proposition 3.Q.E.D.In terms of solving for an equilibrium in proposition 5, I guess an H0 and H1, solve the LQproblem as a matrix Ricatti equation, and then find the fixed point of H0 and H1. In comparisonwith classic LQ control, I do not use certainty equivalence. The variance of the shock only entersinto the d of the solution equation (2.34). The d can be decomposed into a component related tothe ω aggregate shock and another for the W associated with the finite number of firms. As Ngoes to infinity, the component of d associated with the idiosyncratic shocks goes to 0. That is,conditioning on the aggregate shock, the firm can get closer and closer to the deterministic forecastof the distribution of X ′.100 101 102 103 104N10−210−310−410−5Std. Dev. of ε(X; u)100 101 102 103 104N10−210−310−410−5Std. Dev. of u(X ′) ErrorsFigure 2.8: The concentration of Euler residuals ε(X;u) and the optimal policy u(X ′).To illustrate how quickly and strongly concentration of measure kicks in, I use the same param-eter values as in section 2.4.3. The left panel of figure 2.8 plots the standard deviation of the Euler35residuals ε(X;u) induced by the concentration of measure approximation (this standard deviationcomes directly from equation 2.40). For 10, 000 firms, the standard deviation of the Euler errorsis 2.4 × 10−4. In the right panel of figure 2.8, I plot the standard deviation of the optimal policyu(X ′) (equal to H1σ√N−1N /√N , as shown in equation 2.41) for the linear aggregate prices giventhe same Monte Carlo approximation and find that the standard deviation of the policy error is2.5×10−6 for N = 10, 000 and 2.2×10−5 for the baseline case in many of the previous experimentsof N = 128.While the LQ algorithm I just outlined works in delivering accurate solutions, as N becomeslarge, it is too slow, as it requires computations of order O(N3). In comparison, as I argued beforethat the neural networks only require computations of order O(1) for a reasonable N .2.6 ExtensionsIn this section, I consider how these general techniques can be used for a much larger class of models.Most broadly, these tools are useful for solving any high-dimensional functional equations with somedegree of symmetry, especially when these equations contain high-dimensional expectations. Thefunctional equation could come from static problems, Bellman equations, or Euler equations, andcan be solved by various approximation techniques.212.6.1 Multiple productivity typesI consider now the case when firms have profitsj = ζjp(X)x1−µ1−µ − γ2u2 for some finite set of pro-ductivities, e.g., ζ1, ζ2. Then, rather than having permutation invariance based on the entire X, Icould group them such that the policy becomes u(x,X1, X2), where X1 contains all firms with ζ1,etc. That is, the elements of X could be symmetric in blocks, and this could be implemented inthe network with a representation such as:u(x,X1, X2) ≈ ρ(x,1N1N1∑i=1ϕ1(X1i ),1N2N2∑i=1ϕ2(X2i )).If there were reasons in the problem structure to think that the ϕ1 and ϕ2 could be proportionalto each other, then the representation might even be:u(x,X1, X2) ≈ ρ(x,1N1N1∑i=1ϕ(X1i ),1N2N2∑i=1ϕ(X2i )).2.6.2 Complex idiosyncratic statesNext, I consider the case where firms also have a stochastic productivity y, such that the firm isidentified by the tuple z ≡ (x, y) and the aggregate state is a collection of tuples Z ≡ {(Xi, Yi)}Ni=1.21For example, one could use reinforcement learning (also known as approximate dynamic programming or neuro-dynamic programming). See van der Pol et al. (2020) for an example applying symmetry to reinforcement learning.36I can apply the techniques introduced in this chapter by considering that it is the symmetry withrespect to the z tuple of states, rather than permutation within that tuple, that is required. I canthen implement this symmetry in an approximation as:u(x, y, Z) ≈ ρ(x, y,1NN∑i=1ϕ(Xi, Yi)),for some ϕ : R2 → RL and (x, y) the idiosyncratic state of interest.22 Hence my techniques areeasily applied to models with exchangeable “agents” containing a finite number of states.The approach can even be extended to cases where the collection of underlying states is itselfexchangeable. For example, consider if, in addition to its x, each firm has up to M customers ofvarying sizes, yj , where those customers are otherwise exchangeable in the firm’s payoffs.23 Thenone can denote the customers themselves as a set, and use z ≡ (x,{yj}Mj=1) as an individual firm’sstate. If Yi ≡{yji}Mj=1, the aggregate distribution is Z ≡ {Xi, Yi}.A neural network imposing this symmetry might beu(x, Y, Z) ≈ ρ1x, 1MM∑j=1ϕ1(yj),1NN∑i=1ρ2Xi, 1MM∑j=1ϕ2(Yji ) ,where the training process would fit parameters for ρ1, ϕ1, ρ2, and ϕ2, and the ρ2 function itselfcould have a latent dimension greater than one. This last example demonstrates why deep learning,in the sense of many layers of highly parameterized functions, is so important for dealing with sym-metry. Without modern (and easily available) machine learning libraries, it would be impracticalto fit this nested approximation.Given the flexibility of working with states that are themselves complicated structures, a naturalapplication of these methods is to solve dynamic models with networks—expanding the computa-tional feasibility of solving extensions of models surveyed in Carvalho and Tahbaz-Salehi (2019).In particular, it holds the promise of making multi-period dynamic programming in models suchas Oberfield (2018) possible by enabling network connections as a persistent state variable andenabling firms to form expectations of the network’s evolution.2.6.3 Global solutions with transitions and aggregate shocksIn many applications, researchers use a continuum of agents. Unfortunately, as soon as one addsseveral aggregate shocks to them, these models are difficult to tackle. Since the distributional statebecomes a stochastic partial differential equation (or its discrete-time equivalent), agents need tosolve optimal control problems with an infinite-dimensional state, which in turn depends on theirdecisions. The forward equation for the evolution is difficult enough to simulate, even as an initial22In a model with a continuum of agents, this would simply lead to a two-dimensional distributional state, and themodeler would typically write payoffs as expectations using that distribution.23See Hartford et al. (2018) for methods to represent permutation equivariance with interactions between sets.37value problem, but becomes intractable when it is part of an infinite-dimensional optimal controlproblem. The continuum approximation is then the source of the numerical intractability ratherthan the solution to it, hinting that leaving things discrete might help.As a path forward, the method proposed in this chapter offers the possibility of simulatingeconomies with tens of thousands of agents even when the model is far away from its ergodic meanor when the economy is hit by a large shock. Then, one can either have a fine granular grouping ofagents that matches the microdata (i.e., the same number of agents or groups in the model as in themicrodata) or approximate a continuum of agents distribution with histograms as in Mertens andJudd (2018). In comparison, in perturbation solutions such as Kaplan et al. (2018) in continuoustime and Den Haan (2010) in discrete time, one requires appropriate stationarity properties for theperturbation to work accurately.This insight might be particularly fruitful when solving growth models with heterogeneousagents, which often have steady states and balanced growth paths that are extremely sensitive toparameters, multiplicity of equilibria, and hysteresis (i.e., they may not even be ergodic). By usinga finite number of agents—similar to the models discussed in Buera and Lucas (2018)—modelswith endogeneity of both innovation and diffusion as in Benhabib et al. (2021) without ergodicitycould be solved from arbitrary initial conditions.2.7 ConclusionIn this chapter, I have shown how to exploit symmetry in dynamic programming problems andthe solution of recursive competitive equilibria with a large (but finite) number of heterogeneousagents using deep learning. Symmetry, together with concentration of measure, allowed me toalleviate the curse of dimensionality. The method is easy to implement using standard, open-source software libraries and fast to run on a standard laptop. The results open the door tomany applications of interest, including macroeconomics, finance, international finance, industrialorganization, international trade, spatial economies, and other fields.38Chapter 3Exploring Transversality andStationarity with Deep Learning inSequential Setups3.1 IntroductionBut this long run is a misleading guide to current affairs. In the long run we are alldead—J.M. Keynes, A Tract on Monetary Reform (1923), p. 65.Computing the solution of dynamic equilibrium models usually requires ensuring that some eco-nomic assumption that rules out an explosive solution (e.g., transversality or no-bubble conditions)holds. These assumptions are variations of “boundary conditions” in ordinary and partial differ-ential equations but motivated by the forward-looking behavior of economic agents. How theseconditions appear and their concrete form varies from model to model. Still, they are alwayspresent, whether one is solving the deterministic or stochastic version of the model or formulatingthe problem in a sequential or recursive structure. Without these conditions, dynamic optimizationproblems are not well-posed and have a multiplicity of solutions, many of which imply unplausibledescriptions of economic behavior.Unfortunately, these forward-looking boundary conditions are a fundamental limitation in in-creasing the model’s dimensionality (e.g., raising the number of state variables). Without forward-looking boundary conditions, one could compute large models written in a sequential setup bysolving the associated initial value problem or simulating backward-looking recursive stochastic dif-ference equations. In fact, researchers in natural sciences and engineering routinely solve stochasticdifferential equations with initial values in the millions of equations.Why are these boundary conditions an impediment in increasing dimensionality? Because theyforce us to solve the model over a wide range of possible values of the state variables. For example,conditions for recursive formulations manifest as requiring accurate solutions for arbitrary valuesof the state variables, even though one may only care about the solution from a single initial con-dition.24But the fact that one might only care about the dynamics of the economy given some initialconditions suggests an alternative: can one avoid calculating the steady state, balanced growth path24In recursive formulations, one needs to compute the value and policy functions for all values of the state variables,regardless of whether or not the equilibrium dynamics of the economy will travel through those values.39(BGP), or stationary distribution of the model, which are never reached, and still have accurateshort/medium-run dynamics disciplined by the boundary conditions? If the answer is yes, one canbypass the most complicated part of solving a dynamic model, which opens the door to the solutionof vastly more complicated macroeconomic models with transitional dynamics, endogenous growth,aggregate shocks, and agent distributions.The contribution of this chapter is to show how, why, and when deep learning solutions tomacroeconomic models let us achieve this trade-off in sequential setups. More precisely, I documentnumerically and provide theoretical explanations of how the minimum-norm solutions delivered bydeep learning automatically fulfill the long-run boundary conditions for forward-looking agents re-quired in many dynamic models. In other words, deep learning allows us to emphasize accurateshort- and medium-run dynamics and worry less about precision in the long run, while still ensuringthat agent decisions are disciplined asymptotically.Since I want to understand when and why this happens, I investigate this trade-off with stan-dard benchmark models in macroeconomics, the linear asset pricing and the neoclassical growthmodels, rather than investigating complicated models where the state-of-the-art algorithms areheuristic. Both the linear asset pricing and the neoclassical growth models are well understood,can be analyzed theoretically, and have benchmark numerical solutions. Furthermore, I study caseswith multiple steady states in a variation of the neoclassical growth model. Even in the presence ofthis multiplicity in long-run boundaries, deep learning will impose the correct short-run behaviorwithout requiring any information about multiplicity. Finally, I show how the theoretical under-standing of why these techniques work helps in designing deep learning architectures that solve forthe short-term dynamics in the presence of a long-run BGP, where the economic variables follow anon-stationary trajectory.The “inductive bias” of deep learning: collocation/interpolation-style solution methodsfor dynamic equilibrium models, the canonical global solution methods in macroeconomics, followa straightforward template. First, the researcher chooses a parametric class of approximating func-tions on some compact region of the state space and a corresponding set of points one wishes to fitthe approximation exactly (e.g., Chebyshev polynomials for the investment policy and Chebyshevcollocation nodes as the grid points). Next, the researcher solves a system of equations for all theequations of the model (e.g., optimality conditions, budget and resource constraints, etc.), usuallyorganized as residuals, at the grid points to find the coefficients for the approximating function.When there are more coefficients than grid points, the problem is underdetermined. Still, one cannumerically minimize the sum of the squared residuals by interpolating (i.e., achieve zero residualsat all grid points). In those cases, many combinations of parameters yield zero residuals and inter-polate the grid points.25The methods that I employ in this section follow the same approach, but rather than ap-proximating with polynomials, I use flexible “deep learning”-style functional forms (e.g., neural25In the case of sequential formulations, the grid points could be points in time.40networks) .26 The first defining feature of deep learning is massive over-parameterization, wherethe number of coefficients of the class of approximations is often orders of magnitude more thanthe number of grid points.The second defining feature of deep learning, especially important in higher dimensions, is thatthe functional approximations are organized into nested layers (i.e., “deep”) rather than the flatstructure of polynomials or splines. Given the models in this section are low-dimensional, the de-tails of the approximation class are less important (unlike in Ebrahimi Kahou et al., 2021, where acore contribution is a design that enforces symmetry and enables high-dimensional expectations).While “neural networks” is a broad term encompassing everything from linear functions to compli-cated functions used in computer vision, experience shows they usually perform better with morestructure, nested layers of function composition, and many parameters.27 Deep nesting allows forexponential growth in the representations of the inputs to the network while having only a lineargrowth in computational cost.While the algorithms used in collocation and deep learning might seem narrowly focused onfitting the function at the grid points, this is just a means to an end. The goal is not interpo-lation at the grid points (which are usually a measure-zero subset of the state space) but rathergeneralization performance of the function across much larger regions of the state space (e.g., someexpected value of residuals over the domain).To understand why deep learning generalizes well, one must look beyond the structure of theapproximation class (i.e., the nested layers of activation functions). Instead, the key behind goodgeneralization performance lies in the optimizers that minimize residuals. Computer science re-searchers have shown that an essential ingredient responsible for the empirical success of deeplearning is its use of gradient-based optimization algorithms.28 These include a variety of methodsfamiliar to economists (e.g., LBFGS and gradient descent) as well as algorithms essential whenthere are a huge number of grid points (e.g., stochastic gradient descent and ADAM).Why does combining a massive number of parameters and a class of gradient-based optimiza-tion lead to good generalization performance rather than overfitting and multiplicity? Because, inthis situation, theory and practice show that there is an implicit bias towards simple and smoothinterpolating solutions (see Belkin, 2021, for details and a discussion of how an appropriate sense of26In this chapter, I do not need to radically change the algorithm from the familiar collocation template becauseI am analyzing low-dimensional models. Consequently, I can choose grid points without concerns about whetherrelevant regions of the state space are covered. High-dimensional models require algorithms to find appropriate gridpoints for the fitting process. See Ebrahimi Kahou et al. (2021) for one variation that uses symmetry to simplify thenumber of required grid points in high dimensions.27An example for approximating f : R→ R would be a collection of coefficients θ ≡ {b1,W1, b2,W2, b3,W3, b4} andfˆ(x; θ) = tanh(tanh(tanh(W1 · x + b1) ·W2 + b2) ·W3 + b3) + b4 where tanh(·) is applied pointwise, b1 ∈ RN ,W1 ∈RN×1, b2 ∈ RN ,W2 ∈ RN×N ,W3 ∈ R1×N , b3 ∈ R, and b4 ∈ R. The number of nested functions (i.e., layers), thesizes of the vectors within those (i.e., nodes), and the particular nonlinear functions like tanh (i.e., what they callactivation functions) are all flexible. Many variations exist that exploit problem structure (e.g., one may want toexponentiate the last layer to ensure it is always positive).28The role of deep-learning frameworks, such as JAX or PyTorch, is to provide a flexible environment for specifyingdifferent approximations and compute gradients to aid optimization algorithms that minimize residuals (see Childerset al., 2022, for a broader discussion of automatic differentiation for economists).41functional smoothness means the “simplest” function). Furthermore, in many cases, the algorithmwill reliably converge to a unique solution.29 In other words, deep learning solutions have a built-inOccam’s Razor: among many interpolating solutions the simple and smooth one is chosen.Formally, deep learning tends to choose the interpolating solution with the minimum norm,where the norm is such that the smoothest functions subject to interpolating the grids have thesmallest norms.30 I will explore these concepts more rigorously in section 3.4.31The inductive bias is the core of the argument, and it appears even if there is a massive degreeof multiplicity in the possible functions that could interpolate the grid points. If one solves theproblem with completely different initial conditions for the coefficients (parameters of the neuralnetwork), the approximating function will end up often numerically indistinguishable, albeit withdifferent coefficients. That is, deep learning usually converges to roughly the same approximationdespite the risks of over-fitting and multiplicity. Importantly, this approximation leads to goodgeneralization in the models I study. Since the inductive bias of deep learning is frequently alignedwith the functions that generalize best for models in macroeconomics (which also tend to be simpleand smooth), I can beget accurate solutions across large regions of a state space with very few gridpoints.At a fundamental level, nothing in the previous paragraphs should be mysterious to economists,since there is a deep connection between the inductive bias and a classic literature in economics:turnpike theory (McKenzie, 1976). I will argue now that the inductive bias gives a solution thatsatisfies a stronger turnpike result, an “on-ramp” result.Turnpikes and on-ramps: turnpike theorems teach us that models with finite but long hori-zons usually have the same properties as infinite-horizon models. Consider the problem of drivingfrom the Vancouver School of Economics at the University of British Columbia to the departmentof economics at the University of Pennsylvania in Philadelphia. One can think about this trip asconsisting of three segments: the leg from the origin to the on-ramp of the turnpike, the travel alongthe turnpike itself (where one would spend almost the entire drive because it is the lowest-cost pathbetween the cities), and the choice of the off-ramp and travel to the destination. A critical insightof turnpike theory is that if one is primarily interested in the short- and medium-run dynamics oftravel starting at Vancouver, one can ignore the details of what to do when arrived in Philadelphia.Instead, first focus on finding what the solution would look like if one drove on the turnpike forever(i.e., calculate the stationary solution and policies for the infinite-horizon problem); next, solve thetransition dynamics to get from the University of British Columbia to the turnpike (i.e., the transi-29There is a continuum of possible coefficients that achieve the same interpolating solution up to numerical accuracy.Uniqueness here refers to the approximating functions using those coefficients, not the coefficients themselves.30This phenomenon also occurs with OLS whenever the model is underdetermined (see Liang and Rakhlin, 2020).The intuition in the case of OLS is that these methods choose among the interpolating solutions those with the mini-mum L2 norm, which can be interpreted as finding the equivalent solution to a ridge regression with an asymptoticallyvanishing regularization penalty.31See Belkin et al. (2019) for a readable survey of this implicit bias, “double-descent” phenomenon and how itextends the classic variance-bias tradeoff in statistics.42tional dynamics using the turnpike as a boundary condition), and ignore the destination dynamicscompletely. Hence, the problem is simplified by avoiding the need to work through the details ofwhich off-ramp and local roads to take in Philadelphia by solving an infinite-horizon problem whereone drives on the turnpike forever. While it is often proven with deterministic, representative agentmodels, the basic intuition of turnpike theory is robust: the present discounted value of optimalbehavior at the end of the problem is trivially small when the time horizon is sufficiently long and,thus, “getting it right at the end” is close to irrelevant.But while turnpike theory simplifies models by avoiding solving for the dynamics from the off-ramp to the destination, it still requires calculating the steady state, BGP, or ergodic distributionof the infinite-horizon problem to find the dynamics local to the origin. Even after characterizingthese fragile steady state, BGP, or ergodic distribution, one needs to use it as a boundary conditionto solve the transitional dynamics of the first leg of the journey, from the origin to the on-ramp.Returning to idea of this chapter, the main contribution is to take the spirit of the turnpiketheory one step further. Rather than just ignoring the dynamics local to the destination (i.e., theoff-ramp), deep learning allows us to avoid characterizing the travel or the location of the turnpikeitself and solely focus on the dynamics starting at the origin (i.e., solve models anticipating theon-ramp will correctly lead to the turnpike). Loosely speaking, this is an “on-ramp result” insteadof a “turnpike” one.Deep learning makes this “on-ramp result” possible because the location of the turnpike istightly connected with it being the stable solution to the agents’ optimization problems. As Iargued above, deep learning solutions have an inductive bias toward choosing simple and smoothsolutions, which are precisely the non-explosive solutions that satisfy the boundary conditionsdisciplining forward-looking behavior. Hence, the solutions from deep learning are consistent byconstruction with the features of the turnpike. Deep learning methods also pick which turnpiketo go to in the case of multiplicity because, if there are two turnpikes it could go to, the roads tothe “closest” turnpike are also the shortest. Deep learning finds the appropriate roads to a BGPturnpike if one sets up the approximation to consider exponential scaling: the smoothest path isthe one with the correct BGP rate and the smoothest detrended trajectories. The wrong BGPwould lead to explosive trajectories that are eliminated by the inductive bias.Literature. While there are many uses of machine learning in economics, such as in creat-ing new data sources from text and images (e.g., Shen et al., 2021, and Gentzkow et al., 2019)and econometrics (e.g., Athey and Imbens, 2019), in this chapter, I narrowly focus on solutionsto standard macroeconomic models in the most traditional sense and without any behavioral ap-proximations (e.g., not using bounded rationality assumptions such as Krusell and Smith, 1998,approximations or and model-free reinforcement learning such as in Calvano et al., 2020).Within macroeconomics and finance, there is an active literature solving dynamic models withmachine learning methods that can be traced back to Duffy and McNelis (2001). Some more recentpapers that solve the full model rather than a behavioral approximation of it are Azinovic et al.43(2022), Ebrahimi Kahou et al. (2021), Duarte (2018), and some of the examples in Maliar et al.(2021). Other papers make trade-offs between solving the full model and making behavioral as-sumptions, such as Ferna´ndez-Villaverde et al. (2019), which uses deep learning with a Krusell andSmith (1998)-style behavioral assumption, and some of the examples in Maliar et al. (2021) whichuse a large number of discrete agents to approximate the evolution equation of the distribution.None of these papers directly impose transversality conditions, nor do they calculate the stationarydistribution analytically. Since transversality conditions are necessary for these models to be well-posed, it is unclear which interpolating solution each algorithm would converge to. In particular,many solutions can have Euler or Bellman residuals approximately zero and do not fulfill long-runforward-looking behavior and stability. I provide examples of this phenomenon in the next chapter(i.e., section 4.3).This chapter and the next provide insights on which interpolating solution those papers mightconverge towards and how to design the deep learning approximations to ensure the correct solutionis attained. The results can help explain why deep learning interpolation finds the correct solutiondespite not directly imposing transversality in high-dimensional models, for instance Ebrahimi Ka-hou et al. (2021). This chapter solely focuses on sequential setups. The next chapter is dedicatedto recursive models.The rest of the chapter is organized as follows: section 3.2 provides an introductory example withthe canonical linear asset pricing, showing both numerically and theoretically why deep learningmethods converge to the solution fulfilling the no-bubble condition even when ill-posed. Section 3.3continues these examples by analyzing the sequential formulation of the neoclassical growth model,where forward-looking behavior requires a transversality condition to be well-specified. There, Ishow how deep learning leads to solutions that automatically fulfill transversality despite not di-rectly solving for the steady state, even in cases with steady-state multiplicity and non-stationarity.Section 3.4 elaborates on the theoretical connections to the deep learning literature and section 3.5offers some final remarks.3.2 Linear asset pricingI start the investigation with a linear asset pricing model in sequential form. This classic modelillustrates the connection between the implicit bias of over-parameterized functions and forward-looking boundary conditions.Consider an asset that pays a dividend stream with a linear form:y(t+ 1) = c+ (1 + g)y(t)y0 given.where c ≥ 0, g ≥ −1 and t = 0, . . . ,∞.Although this problem is discrete in nature, I use a function notation (e.g., y(t) instead of yt).In order to solve sequential models of this sort I embed the discrete solution within a continuous44function. For instance, I use deep neural networks to represent p : R+ → R instead of p : N →R. However, I still evaluate p(t) at integer values. This approach is desirable for a couple ofreasons. First, from a practical perspective, deep neural networks are defined on a continuousdomain. Second, it allows me to use sparse grids, which can be used to assess the interpolation andextrapolation performance of the solutions.I rewrite this problem as a linear space model:x(t+ 1) = Ax(t)y(t) = Gx(t),where:x(t) ≡[1 y(t)]⊤A ≡[1 0c 1 + g]G ≡[0 1].The price pf (t), based on the fundamentals, is the present discounted value of the dividendstream:pf (t) ≡∞∑j=0βjy(t+ j) = G(I− βA)−1x(t), (3.1)where β is the discount factor, and I is a two-dimensional identity matrix. Here, I assume β ∈ (0, 1)and β(1 + g) < 1. The second assumption ensures the convergence of the infinite sum in equation(3.1). I can also write the evolution of the price in a recursive way:p(t) = Gx(t) + βp(t+ 1) (3.2)x(t+ 1) = Ax(t) (3.3)x0 given. (3.4)Equation (3.2) states that the price of an asset that pays a stream of dividends {y(t + j)}∞j=0 isequal to the dividend the agent receives at time t plus the discounted price of owning the asset attime t + 1. Although for a given initial condition x0, the price based on the fundamentals pf (t)satisfies equations (3.2)-(3.4), this system of equations do not form a well-posed problem and thereare infinitely many solutions of the form:p(t) = pf (t) + ζβ−t. (3.5)See Brunnermeier (2017) for a derivation of these solutions and their interpretation. The initialcondition x0, uniquely determines pf (t). However, ζ is not uniquely determined. In order to have45a well-posed problem, a no-bubble condition is required to eliminate any solution that grows fasteror at the same rate as β−t. With this condition, the well-posed problem becomes:p(t) = Gx(t) + βp(t+ 1) (3.6)x(t+ 1) = Ax(t) (3.7)0 = limT→∞βT p(T ) (3.8)x0 given, (3.9)where equation (3.8) is the no-bubble condition.323.2.1 Interpolating solutionIn this setup, prices are a function of time. Hence, their domain is defined as X ≡ R+. Asa generalization of the collocation approach one can pick a space of parametric functions H(Θ),where Θ ≡ {θ1, . . . , θM} represents the set of parameters. For instance, in a standard collocationmethod H(Θ) can be the space of Chebyshev polynomials of degree M . The interpolating functionp(·; θ) : X → R belongs to H(Θ). In order to represent the domain X , one can pick a finite setof grid points Xtrain ⊂ X . For instance, in a standard Chebyshev collocation, Xtrain is the set ofChebyshev nodes. Generally, for sequential models such as this, the grid is a set of points in time:Xtrain ≡ {t1, . . . , tN}.The no-bubble condition is a boundary condition at infinity. One can approximate infinity bychoosing a very large point in time, denoted by T . Usually, T is much larger than the largest pointin Xtrain (i.e. T ≫ tN ). Given a space of parametric functions H(Θ), a grid Xtrain, and a very largepoint in time T , one can find the interpolating price function by solving:minθ∈Θ[1|Xtrain|∑t∈Xtrain(p(t; θ)−Gx(t)− βp(t+ 1; θ))2 + (βT p(T ; θ))2︸ ︷︷ ︸Is this necessary?], (3.10)where x(t) is evaluated by the law of motion for x:x(t) = Atx0 for t ∈ Xtrain. (3.11)The first term in the minimization represents the residuals of the recursive formulation for theprices (i.e., equation 3.6) and the second term is a finite approximation of the no-bubble condition(i.e., equation 3.8).In a standard collocation method, where the number of parameters is equal to the number ofgrid points, this problem can be solved exactly as a system of equations with a boundary condi-32Sometimes the no-bubble condition is referred to as the transversality condition, see Brunnermeier (2017). How-ever, to avoid confusion I reserve that term for the neoclassical growth model.46tion at T . Since the interpolating solution depends on T , the solutions of minimization problemdescribed in (3.10) defines a sequence of solutions indexed by T .Choice of Xtrain: in high-dimensional cases the choice of Xtrain is very crucial, however in thischapter I focus on low-dimensional cases. Although this choice is not very crucial for this study,I provide the results for different grids. In this chapter I am also interested in the generalizationpower of the solutions. Therefore, I also evaluate the solutions outside of Xtrain. For instance, insequential models I am interested in the behavior of the solutions for t > tN .Is the no-bubble condition necessary? As discussed before, without the no-bubble condi-tion the linear asset pricing model has infinitely many solutions of the form expressed in equation(3.5). Each solution corresponds to a different value for ζ. Therefore, in the absence of the sec-ond term (finite approximation of the no-bubble condition), the optimization problem described in(3.10), has infinitely many solutions that achieve the zero of the objective function.In the next section I establish how using over-parameterized functions, where the number ofparameters is larger than the number of grid points, yields convergence to the solution based onthe fundamentals without worrying about the no-bubble condition.Therefore, using over-parameterized functions I can drop the approximate no-bubble conditionand solve:minθ∈Θ1|Xtrain|∑t∈Xtrain(p(t; θ)−Gx(t)− βp(t+ 1; θ))2 , (3.12)to obtain an interpolating solution that satisfies the no-bubble condition.3.2.2 Minimum norm interpretation and no-bubble conditionIn this section I use a space of over-parameterized functions (i.e., deep neural networks) to solve theminimization problem described in (3.12). An over-parameterized function p(t; θ) is characterizedby M parameters {θ1, . . . , θM} where M is larger then the number of grid points, i.e., M > N . Inpractice the number of parameters is much larger than the number of grid points. Since M ≫ N ,over-parameterized functions and their corresponding optimization algorithms can provide exactinterpolating solutions. For these solutions the objective function attains a zero value.Since the number of parameters is much larger than the number of grid points, there are toomany degrees of freedom. Therefore, there are many arrangements of the parameters that canachieve an exact interpolation. It has been observed and in some cases proved that the optimiza-tion algorithms used in over-parameterized interpolation problems have an implicit bias toward aspecific class of functions. Recently, it has been proposed that this property can be interpreted asminimizing a new objective function on the space of parametric functions H(Θ) subject to exactinterpolation at all the points on the grid. I call this minimum norm interpretation of the solutionsof an over-parameterized interpolation. Section 3.4 provides a comprehensive background and dis-47cussion on this robust phenomena.The minimum norm interpretation of the solutions of the minimization problem described in(3.10) for the liming case of T →∞ can be written as:minp(·;θ)∈H(Θ)∥p(·; θ)∥S (3.13)s.t. p(t; θ)−Gx(t; θ)− βp(t+ 1; θ) = 0 for t ∈ Xtrain (3.14)0 = limT→∞βT p(T ; θ) (3.15)where H(Θ) is a space of over-parameterized functions, Xtrain is a set of points in time (i.e.,Xtrain = {t1, . . . , tN}), and x(t) is evaluated by the exogenous law of motion x(t+1) = Ax(t), giventhe initial condition x0.What is the this new objective function? The new objective function ∥·∥S is a semi-norm33from a space functions (for this case H(Θ)) to real numbers satisfying the following assumption.Assumption 1. Let Ψ1 and Ψ2 be two differentiable function from a compact space X in R to Rsuch that: ∫X∣∣∣∣dΨ1ds∣∣∣∣2 ds > ∫X∣∣∣∣dΨ2ds∣∣∣∣2 dsthen:∥Ψ1∥S > ∥Ψ2∥S .Moreover, since ∥ · ∥S is a semi-norm, it satisfies the triangle inequality:∥Ψ1 + Ψ2∥S ≤ ∥Ψ1∥S + ∥Ψ2∥S .This is an assumption on the implicit bias of the over-parameterized interpolation and theiroptimization algorithms. Intuitively, this assumption states that among two interpolating solutionson a domain X the bias is toward the one with smaller derivatives. In other words, the bias istoward the smoother one. Through the rest of this and the next chapter I assume assumption 1holds. See 3.4 for a discussion of the validity of this assumption and recent advances in the statisticsand optimization theory regarding this assumption.As shown in equation (3.5) any solution that violates the no-bubble condition is associated witha non-zero ζ. In the following proposition I establish that minimizing ∥ · ∥S automatically makes33For a given space of function H, a semi-norm ∥ · ∥S : H → R is a mapping that for all functions ψ1 and ψ2 in Hsatisfies1. Triangle inequality: ∥ψ1 + ψ2∥S ≤ ∥ψ1∥S + ∥ψ2∥S , and2. Absolute homogeneity: ∥γψ1∥S = |γ|∥ψ1∥S for all γ ∈ R.48the no-bubble condition redundant. Intuitively, due to the explosive term β−t (0 < β < 1), all thesolutions that contain a bubble term (ζ > 0) have bigger derivatives than the fundamental pricepf (t). More formally for any compact interval of the form [0, T ]:∫ T0∣∣∣∣dpdt∣∣∣∣2 dt > ∫ T0∣∣∣∣dpfdt∣∣∣∣2 dtI only focus on prices that are positive for all t, this excludes solutions with ζ < 0.Proposition 6. Let P be the set of all the solutions of equations (3.6) and (3.7) for a given x0:P ≡ {p(t) : p(t) = pf (t) + ζβ−t, ∀ζ ∈ R+},where pf (t) is defined by equation (3.1). Let ∥ · ∥S be a norm that satisfies assumption 1, then:argminp∈P∥p∥S = pf .In other words the minimum of ∥p∥S is attained when ζ = 0.Proof. Since ζ ≥ 0, then by the triangle inequality and assumption 1:∥pf∥S ≤ ∥p∥S ≤ ∥pf∥S + ζ∥β−t∥S .Q.E.D.This proposition establishes that the price based on the fundamentals has the lowest semi-norm.34 Therefore, over-parameterized interpolating solutions have a bias toward pf (t) on Xtrainand automatically satisfy the no-bubble condition. Therefore, the optimization problem describedin (3.12) is enough to find the price based on the fundamentals without imposing any explicitregularity regarding the long-run behavior.3.2.3 ResultsFigure 3.1 shows the result of the minimization problem described in (3.12) for β = 0.9, c = 0.01,g = −0.1, and y0 = 0.08. In this experiment Xtrain = {0, 1, 2, . . . , 29} and Xtest = {0, 1, 2, ..., 49}.I approximate the price p(t; θ) using a deep neural network with four hidden layers, each with128 nodes. Each hidden layer uses Tanh and the output layer uses Sofplus as activation functions.The dashed vertical lines separate the interpolation from the extrapolation region. To be morespecific, the interpolation region is defined as Xtrain and the extrapolation as the points in Xtestthat are not in Xtrain. The left panel shows the price paths. The price based on the fundamentals,34What is required in this proof is triangle inequality and an innocuous assumption that if p(t) > pf (t) for all t ∈ Xthen∥p∥S > ∥pf∥S .Therefore, assumption 1 is strong for this proof and can be relaxed.49denoted by pf (t), is calculated using equation (3.1), and pˆ(t) shows the approximate path (orapproximate solution). More specifically, pˆ(t) ≡ p(t; θ∗), whereθ∗ ≡ argminθ∈Θ∑t∈Xtrain(p(t; θ)−Gx(t)− βp(t+ 1; θ))2 .The right panel shows the relative error between the approximate price and price based on thefundamentals, defined as:εp(t) ≡ pˆ(t)− pf (t)pf (t)for t ∈ Xtest. (3.16)The results of this experiment are multi-fold. First, I can achieve an accurate solution based on0 10 20 30 40 50Time(t)0.850.900.951.001.05Prices: p(t)pf(t)p(t)0 10 20 30 40 50Time(t)0.00250.00200.00150.00100.00050.0000Relative errors: p(t)0.9850.990Figure 3.1: Comparison between the price based on the fundamentals and the price approximatedby a deep neural network for the sequential linear asset pricing model. In the left panel, the solidcurve shows the approximate price and the dashed curve shows the price based on the fundamentals.The right panel shows the relative errors between the approximate price path and the price basedon the fundamentals. The dashed vertical lines separate the interpolation from the extrapolationregion.the fundamentals and eliminate bubble solutions simply by relying on the implicit bias of deepneural networks. Second, within the grid points Xtrain, the short- and medium-run dynamics areaccurate and it is not impaired by the long-run errors (at most −0.05% relative error). Third, theextrapolation errors are very small which can be explained by the smoothness imposed by deepneural networks and the choice of Xtrain. Here, I used a large time horizon in Xtrain, i.e., tN = 29.The prices in the last few points of the grid are very close to their asymptotic value. Therefore,the deep neural network learns that for large values of t the price is almost a constant and staysvery close to that value even outside of Xtrain.50Contiguous vs. Sparse Grid: there is a common belief that deep neural networks areonly suitable for high-dimensional environments with an abundance of data (grid points). In thisexperiment, I show that they can be very useful when the grid contains few points.0 10 20 30 40 50Time(t)0.850.900.951.001.05Prices: p(t)pf(t)p(t): Contiguousp(t): train(Grid 1)p(t): train(Grid 2)0 10 20 30 40 500.00300.00250.00200.00150.00100.00050.00000.0005p(t): Contiguousp(t): train(Grid 1)p(t): train(Grid 2)0 10 20 30 40 50Time(t)0.850.900.951.001.05Prices: p(t)pf(t)p(t): Contiguousp(t): train(Grid 2)0 10 20 30 40 50Time(t)0.0100.0080.0060.0040.0020.0000.0020.004Relative errors: p(t)0.9850.9900.9850.990Figure 3.2: Comparison between the contiguous grid and two sparse grids for the se-quential linear asset pricing model. The first sparse grid is defined as Xtrain(Grid 1) ≡{0, 1, 2, 4, 6, 8, 12, 16, 20, 24, 29} and the second sparse grid is defined as Xtrain(Grid 2) ≡{0, 1, 4, 8, 12, 16, 20, 24, 29}. The top panels show the results for price paths for one seed. Thebottom panels show the median of the approximate price paths and the relative errors for thecontiguous grid and Xtrain(Grid 2). The shaded regions show the 10th and the 90th percentiles.The left panels provide a comparison between the approximate price paths and the price based onthe fundamentals. The right panels show the relative errors between the approximate price pathsand the price based on the fundamentals. The dashed vertical lines separate the interpolation fromthe extrapolation region.Figure 3.2 shows the results of previous experiment with two different sparse grids for Xtrain.51The first grid contains 11 points and is defined as Xtrain(Grid 1) ≡ {0, 1, 2, 4, 6, 8, 12, 16, 20, 24, 29}.The second grid contains 9 points and is defined as Xtrain(Grid 2) ≡ {0, 1, 4, 8, 12, 16, 20, 24, 29}.The contiguous grid is the same as the previous experiment, Xtrain = {0, 1, 2, . . . , 29} denoted byContiguous. The top-left panel shows the price paths for these grids. pˆ(t) : Contiguous showsthe approximate price path for Xtrain, pˆ(t) : Xtrain(Grid 1) shows the approximate price pathfor Xtrain(Grid 1), and pˆ(t) : Xtrain(Grid 2) shows the approximate price path for Xtrain(Grid 2).The top-right panel shows the relative errors between the approximate price paths and the pricebased on the fundamentals. εp(t) : Contiguous shows the relative errors for the contiguous grid,εp(t) : Xtrain(Grid 1) shows the relative errors for Xtrain(Grid 1), and εp(t) : Xtrain(Grid 2) showsthe relative errors for Xtrain(Grid 2). In the top panels I use only one random initialization (onerandom seed) of the deep neural network to generate the results.I solve the minimization problem described in (3.12) for 100 times. Each time with a differentrandom initialization of the parameters of the deep neural network. I report the median, the 10th,and the 90th percentiles of the results.The solid curves in the bottom-left panel show the median of price paths for the contiguous gridand Xtrain(Grid 2), the shaded regions show the 10th and the 90th percentiles of the approximateprice paths. The solid curves in the bottom-right panel show the median of the relative errors forthe contiguous grid and Xtrain(Grid 2), the shaded regions show the 10th and the 90th percentilesof the relative errors. The dashed vertical lines separate the interpolation from the extrapolationregion.These results show that the contiguous grid outperforms both sparse grids. However, the resultsfor both sparse grids are very accurate (at most relative error of −0.35 %). As expected, the mostsparse grid has higher relative errors. It is worth mentioning that convergence to the price basedon the fundamentals is not sensitive to the grid size. Therefore, it can be a promising avenue formethods of adaptive sample selection in high-dimensional problems.These results also demonstrate the robustness of deep neural networks to random initialization,for both contiguous and sparse grids. This confirms the implicit bias in the deep neural networks,which consistently favors prices based on fundamentals despite variations in parameter initialization.Positive growth in dividends (g > 0) : in the last two experiments I assumed that thedividends follow a stationary process. In this experiment I show that exploiting a priori economicknowledge combined with the implicit bias, the method can deal with cases where no stationarysolution exists. In this case I know the price based on the fundamentals grows exponentially, thiseconomic knowledge can be flexibly implemented in the design of the function space H(Θ). In thecase of g > 0 and c = 0, the fundamental price grows with the rate g. In this experiment I use thisinformation to construct an approximating function of the form:pˆ(t; θ) = eϕtNN(t; θ1)where θ ≡ {ϕ, θ1}, NN(·; θ1) is a deep neural network, and ϕ is a single parameter that needs to52be found in the optimization process.Again for this experiment, I solve the minimization problem described in (3.12) for 100 times.Each time with a different random initialization of the parameters of the deep neural network. Ireport the median, the 10th, and the 90th percentiles of the results. Figure 3.3 shows the results for0 10 20 30 40 50Time(t)1.001.251.501.752.002.252.502.75Price: p(t)p(t)pf(t)0 10 20 30 40 50Time(t)0.010.000.010.020.030.04Relative error: p(t)1.4501.475Figure 3.3: Comparison between the price based on the fundamentals and the price approximatedby a deep neural network for the sequential linear asset pricing model with growing dividends(g = 0.02). The solid curve in the left panel shows the median of the approximate price paths andthe dashed curve shows the price based on the fundamentals. The solid curve in the right panelshows the median of the relative errors between the approximate price paths and the price basedon the fundamentals. The shaded regions show the 10th and the 90th percentiles. The dashedvertical lines separate the interpolation from the extrapolation region.sequential linear asset pricing model with growing dividends (i.e., c = 0, g = 0.02). I use the samedeep neural network utilized in the first experiment. The only difference is the additional exponen-tial term. In the left panel, the solid curve, denoted by pˆ(t), shows the median of the approximateprice paths over 100 seeds, the dashed curve, denoted by pf (t), shows the solution based on thefundamentals. The shaded region shows the 10th and the 90th percentiles of the approximate pricepaths. The right panel shows the median of the relative errors between the approximate pricepaths and the price based on the fundamentals. The shaded region shows the 10th and the 90thpercentiles of the the relative errors. The dashed vertical lines separate the interpolation from theextrapolation region.These results show that the long-run errors do not impair the short- and medium-run accuracyeven in the presence of growing dividends. It is worth noting that if interpolation is the primaryobjective (left of the dashed lines), the same result can be achieved without including the exponen-tial term in the approximating function. However, when extrapolation (right of the dashed lines)is the objective, incorporating economic knowledge can significantly enhance generalization power(resulting in less than 0.5% relative error after 20 periods). Additionally, the algorithm accurately53learns the growth rate. See figure B.1 and the discussion in B.1.1 for a detailed treatment of theapproximate growth rate.In this case, I used the correct information about the functional form of the growth in the pricepath. However, in more complicated dividend processes, when the growth in prices is unknown, itis possible to misspecify the functional form of the growth. Figure B.2 in B.1.2 confirms that evenin the presence of functional misspecification, long-run errors do not impair the accuracy of short-and medium-run dynamics.3.3 Neoclassical growth model: sequential formAnother form of forward-looking boundary condition at infinity that appears frequently in dynamiceconomic models is the transversality condition. In these models the transversality condition is anecessary condition for optimality (see Ekeland and Scheinkman, 1986, and Kamihigashi, 2005).In this section I focus on the sequential version of the neoclassical growth model.Setup: consider an agent maximizing discounted utility of consumption∑∞t=0 βtu(c(t)). Theagent has access to a production technology z1−αf(k), where z is the total factor productivity andk is the capital. Capital depreciates at rate δ ∈ [0, 1]. The agent’s optimization problem can bewritten as:maxc(t),k(t+1)∞∑t=0βtu(c(t))(3.17)s.t. k(t+ 1) = z(t)1−αf(k(t))+ (1− δ)k(t)− c(t) (3.18)z(t+ 1) = (1 + g)z(t) (3.19)k(t) ≥ 0 (3.20)0 = limT→∞βTu′(c(T ))k(T + 1) (3.21)k0, z0 given . (3.22)The problem described above is the standard neoclassical growth problem. Here I focus on theconstant relative risk aversion utility functions u(c) = c1−σ1−σ , and production function of the formf(k) = kα. The first order conditions for this problem can be written as:u′(c(t))= βu′(c(t+ 1))[z(t+ 1)1−αf ′(k(t+ 1))+ (1− δ)] (3.23)k(t+ 1) = f(k(t))+ (1− δ), k(t)− c(t) (3.24)where the first equation is the Euler equation and the second one is the feasibility condition. Equa-tions (3.23) and (3.24) do not form a well-posed problem. There are infinitely many solutions54{c(t), k(t)}∞t=0 that satisfy these equations. The transversality condition, described in equation(3.21), uniquely determines the optimal solution.Saddle path nature of the optimal solution: the optimal solution of this problem is asaddle path and the steady state of the optimal solution is a saddle point. The saddle path natureof the optimal solution makes the optimal path {k(t), c(t)} very sensitive to the choice of the initiallevel consumption (i.e. c(0)). Therefore, a small error in the choice of consumption at time zerocan lead to a solution that violate the transversality condition.It is also worth noting that if an economy starts with zero capital (i.e. k = 0), it stays at zerolevel of capital forever, therefore, k = 0 is a fixed point of this economy. However, it is a repulsivefixed point. Therefore, for an economy that starts with any non-zero level of capital there is nooptimal path that takes the economy to k = 0. See the discussion and figure B.3 in section B.2.1for analysis of the solution method for very small levels for initial conditions of capital.3.3.1 Interpolating solutionIn this setup, capital functions (also consumption functions) are a function of time. Therefore,their domain is defined as X ≡ R+. As a generalization of the collocation approach one can pick aspace of functions H(Θ), where Θ ≡ {θ1, . . . , θM} represents the set of parameters. For instance,in a standard collocation method H(Θ) can be the space of Chebyshev polynomials of degree M .The interpolating function k(·; θ) : X → R belongs to H(Θ). In order to represent the domain X ,one can pick a finite set of grid points Xtrain ⊂ X. Since this is a sequential model, similar to thelinear asset pricing model:Xtrain ≡ {t1, . . . , tN}.The Euler equation (i.e., equation 3.23), the feasibility condition (i.e., equation 3.24), the law ofmotion for total factor productivity (i.e., equation 3.19), and the transversality condition (i.e.,equation 3.21) combined with the initial conditions for capital and productivity form a systemof equations. However, the transversality condition is a boundary condition at infinity. One canapproximate infinity by choosing a very large point in time, denoted by T . Usually, T is muchlarger that the largest point in Xtrain (i.e., T ≫ tN ).For a given capital function k(t; θ) ∈ H(Θ) I define the consumption function c(t; k(·; θ)) throughthe feasibility condition:c(t; k(·; θ)) ≡ f(k(t; θ))+ (1− δ)k(t; θ)− k(t+ 1; θ). (3.25)55Given a space of parametric functions H(Θ), a grid Xtrain, and a very large point in time T , onecan find the interpolating capital function by solving the following optimization problem:minθ∈Θ[1|Xtrain|∑t∈Xtrain(u′(c(t; k(·; θ)))u′(c(t+ 1; k(·; θ))) − β[z(t+ 1)1−αf ′(k(t+ 1; θ))+ (1− δ)])2+(k(0; θ)− k0)2 +(βTu′(c(T ; k(·; θ)))k(T + 1; θ))2︸ ︷︷ ︸Is this necessary?](3.26)where z(t) is evaluated by the law of motion for zz(t) = (1 + g)tz0 for t ∈ Xtrain. (3.27)The first term in the optimization problem represents the Euler residuals and the second term isthe residual for the initial condition. The second term ensures that the capital function at time zeromatches the initial level of capital. The third term is a finite approximation of the transversalitycondition. The non-negativity of capital, k(t; θ) ≥ 0, can be built into H(Θ).In a standard collocation method, where the number of parameters is equal to the number ofgrid points, this problem can be solved exactly as a system of equations with a boundary conditionat T . It is worth noting that the interpolating solution depend on T . Therefore, the solutions ofthe minimization problem described in (3.26) defines a sequence of solutions indexed by T .Here, I only provide a framework for interpolation of the capital function. It is also possibleto interpolate both capital and consumption functions simultaneously. See section B.2.3 for moredetails.The approach I introduce for solving sequential models in this chapter has some similaritieswith parametric path methods. These methods approximate the equilibrium paths with a familyof parametric functions, see Judd (2002). In these methods the convergence to a solution that doesnot violate the transversality condition is baked into the approximating functions as an a priorieconomic knowledge. For instance, by design the approximating function approaches the desiredsteady state as time goes to infinity. Therefore, these methods require calculating the steady statefor finding the approximate solution. However, the proposed method in this chapter does not re-quire any knowledge of the steady state.35Choice of Xtrain: similar to the case of linear asset pricing model, the choice of Xtrain is notvery crucial. However, I provide the results for different grids. I also evaluate the approximatecapital function outside of Xtrain to study the generalization performance of the solutions.35Judd (2002) uses Laguerre polynomials instead of the deep neural networks. The author uses an approximatingfunction with two terms. The first term is the parametric form to deal with the short- and medium-run transition.The second term, by design, becomes the dominant term (with an exponential rate) in the long-run and approachesthe steady state.56Is the transversality condition necessary? Without the transversality condition the neo-classical growth problem has infinitely many solutions. Equations (3.19), (3.23), and (3.24) form athree-dimensional dynamical system with two initial conditions k0 and z0.36 Therefore, in the ab-sence of the third term (i.e., finite approximation of the transversality condition) the minimizationproblem described in (3.26) has infinitely many solutions that minimize the objective function.In section 3.3.3 I establish how using over-parameterized functions (when M ≫ N) and theiroptimization processes yield convergence to a solution that does not violate the transversality condi-tion. Intuitively, the solutions that violate the transversality condition have bigger derivatives thanthe optimal solution. Therefore, using over-parameterized functions I can drop the transversalitycondition and solve:minθ∈Θ[1|Xtrain|∑t∈Xtrain(u′(c(t; k(·; θ)))u′(c(t+ 1; k(·; θ))) − β [z(t+ 1)1−αf ′(k(t+ 1; θ))+ (1− δ)])2+ (k(0; θ)− k0)2],(3.28)to obtain an interpolating solution that does not violate the transversality condition.373.3.2 ResultsFigure 3.4 shows the result of the minimization problem described in (3.28) for β = 0.9, α =0.33, δ = 0.1, g = 0, k0 = 0.4, and z0 = 1. In this experiment Xtrain = {0, 1, 2, . . . , 29} andXtest = {0, 1, 2, ..., 49}. I approximate the capital path k(t; θ) using a deep neural network withfour hidden layers, each with 128 nodes. Each hidden layer uses Tanh and the output layer usesSoftplus as activation functions. The dashed vertical lines separate the interpolation from theextrapolation region. The left panel shows the capital and consumption paths. The benchmarksolution is calculated using value function iteration method and are denoted by k(t) and c(t).38The approximate solutions (i.e., the approximate capital and consumption paths) are denoted by36Given the initial level of capital k0, factor productivity z0, and an arbitrary c0 for the initial level of consumptionone can generate a set of solutions {k˜(t), c˜(t), z(t)} by iterating forward the Euler equation, feasibility condition, andthe law of motion for total factor productivity. By construction this set of solutions is a solution for this system ofequations. Therefore, there are infinitely many solutions each characterized by a different c0.37In practice one can solveminθ∈Θ[1|Xtrain|∑t∈Xtrainλ1(u′(c(t; k(·; θ)))u′(c(t+ 1; k(·; θ))) − β [z(t+ 1)1−αf ′(k(t+ 1; θ))+ (1− δ)])2+ λ2 (k(0; θ)− k0)2],where λ1 and λ2 are a pair of positive parameters that determine the importance of each residuals in the optimizationprocess. In my experiments, the results are not sensitive to the choice of these parameters so one can choose equallyweighted residuals (i.e., λ1 = λ2 =12).38The value function iteration is also an approximation method, here I assume that value function iteration methodyields more accurate results than my method.570 10 20 30 40 50Time(t)0.250.500.751.001.251.501.752.002.25Capital and Consumption: k(t) and c(t)k(t)k(t)c(t)c(t)0 10 20 30 40 50Time(t)0.00050.00000.00050.00100.00150.00200.00250.0030Relative errors: k(t) and c(t)1.9251.9501.975Figure 3.4: Comparison between the value function iteration and approximate solution using adeep neural network for the sequential neoclassical growth model. In the left panel the black solidline shows the approximate capital path and the blue solid line shows the consumption path. Thedashed curves show the capital and consumption paths obtained by the value function iterationmethod. The right panel shows the relative errors for both capital and consumption paths. Thedashed vertical lines separate the interpolation from the extrapolation region.kˆ(t), and cˆ(t). More specifically, kˆ(t) ≡ k(t; θ∗), whereθ∗ ≡ argminθ∈Θ[1|Xtrain|∑t∈Xtrain(u′(c(t; k(·; θ)))u′(c(t+ 1; k(·; θ))) − β [z(t+ 1)1−αf ′(k(t+ 1; θ))+ (1− δ)])2+(k(0; θ)− k0)2],and cˆ(t) is calculated using the consumption function defined in equation (3.25) for the capitalfunction k(t; θ∗), i.e., cˆ(t) ≡ c (t; k(·; θ∗)). The right panel shows the relative errors between theapproximate solutions and the solutions obtained by the value function iteration method definedas:εc(t) ≡ cˆ(t)− c(t)c(t)for t ∈ Xtest (3.29)εk(t) ≡ kˆ(t)− k(t)k(t)for t ∈ Xtest. (3.30)The results of this experiment are multi-fold. First, I achieve an accurate solutions that do notviolate the transversality condition without imposing any long-run constraint. Second, the short-and medium-run dynamics are accurate and they are not impaired by the long-run errors (at most−0.05% relative error for capital). Third, the extrapolation errors are very small which can be58explained by the smoothness imposed by deep neural networks and the choice of Xtrain. Here, Iused a large time horizon in Xtrain, i.e., tN = 29. The capital path in the last few points of the gridare very close to the steady state. Therefore, the deep neural network learns that for large valuesof t the capital is almost a constant and stays very close to that value even outside of Xtrain.Contiguous vs. Sparse Grid: figure 3.5 shows the result of the previous experiment for twodifferent sparse grid for Xtrain. The first grid contains 11 points and is defined as Xtrain(Grid 1) ≡{0, 1, 2, 4, 6, 8, 12, 16, 20, 24, 29}. The second grid contains 9 points and is defined as Xtrain(Grid 2) ≡{0, 1, 4, 8, 12, 16, 20, 24, 29}. The contiguous grid is the same as the previous experiment Xtrain ={0, 1, 2, . . . , 29}, denoted by Contiguous. The dashed vertical lines separate the interpolation fromthe extrapolation region. The top-left panel shows the approximate capital paths for these grids,denoted by kˆ(t), versus the capital path generated by value function iteration method, denoted byk(t). The top-right panel shows the relative errors between the approximate capital paths and thecapital path generated by the value function iteration method. In both top panels I only use onerandom initialization (one random seed) of the deep neural network to generate the results.I solve the minimization problem described in (3.28) for 100 times. Each time with a differentrandom initialization of the parameters of the deep neural network. I report the median, the 10th,and the 90th percentiles of the results.The bottom-left panel shows the capital paths obtained by value function iteration method,denoted by k(t) and the median of the approximate capital paths for the contiguous grid andXtrain(Grid 2). The shaded regions show the 10th and the 90th percentiles of the approximate cap-ital paths. The bottom-right panel shows the median of relative errors between the approximatecapital paths and the capital path obtained by value function iteration method for the contiguousgrid and Xtrain(Grid 2). The shaded regions show the 10th and the 90th percentiles of the relativeerrors.These results show that the contiguous grid outperforms both sparse grids. However, theresults for both sparse grids are very accurate (at most 0.01 % in relative errors). As expected themost sparse grid has higher relative errors. It is worth mentioning that convergence to the optimalsolution is not sensitive to the grid size. Therefore, it can be a promising avenue for adaptiveoptimization and sample selection in high-dimensional sequential problems.These results also demonstrate the robustness of deep neural networks to random initialization,for both contiguous and sparse grids. This confirms the implicit bias in the deep neural networks,which consistently favors the optimal capital path despite variations in parameter initialization.Far from the steady state: in the previous experiment I established that one can achieveaccurate short- and medium-run accuracy on sparse grids. However, in both grids I used a largetime horizon (i.e., tN = 29). After 29 periods the capital and consumption paths are very closeto their steady states. In those cases the neural networks learn that for large t the capital path isvery flat which explains the generalization power of the approximate solution. In this experiment590 10 20 30 40 50Time(t)0.51.01.52.02.5Capital: k(t)k(t)k(t): Contiguousk(t): train(Grid 1)k(t): train(Grid 2)0 10 20 30 40 50Time(t)0.0010.0000.0010.0020.003Relative error: k(t)0 10 20 30 40 50Time(t)0.51.01.52.02.5Capital: k(t)k(t)k(t): Contiguousk(t): train(Grid 2)0 10 20 30 40 50Time(t)0.0100.0050.0000.0050.0100.0150.020Relative errors: k(t)p(t): Contiguousp(t): train(Grid 2)1.952.001.952.00Figure 3.5: Comparison between the value function iteration and approximate solution using adeep neural network for the sequential neoclassical growth model. The first sparse grid is de-fined as Xtrain(Grid 1) ≡ {0, 1, 2, 4, 6, 8, 12, 16, 20, 24, 29} and the second sparse grid is defined asXtrain(Grid 2) ≡ {0, 1, 4, 8, 12, 16, 20, 24, 29}. The top panels show the approximate capital pathsand the relative errors between the approximate capital path and the capital path obtained bythe value function iteration method, for Xtrain(Grid 1), Xtrain(Grid 2), and the contiguous grid.The bottom panels show the median of the approximate capital paths and the median of relativeerrors for Xtrain(Grid 2), and the contiguous grid. The shaded regions show the 10th and the 90thpercentiles. The dashed curves, denoted by k(t), show the capital and path obtained by the valuefunction iteration method. The dashed vertical line separates the interpolation from the extrapo-lation region.I investigate whether the approximate solution has accurate short run dynamics when using amedium time horizon for Xtrain (e.g., tN ≈ 10).Again for this experiment, I solve the minimization problem described in (3.28) for 100 times.60Each time with a different random initialization of the parameters of the deep neural network. Ireport the median, the 10th, and the 90th percentiles of the results.0 5 10 15Time(t)0.81.01.21.41.61.82.0Capital: k(t)k(t)k(t)0 5 10 15Time(t)0.0250.0200.0150.0100.0050.000Relative error: k(t)0 5 10 15Time(t)0.50.60.70.80.91.01.1Consumption: c(t)c(t)c(t)0 5 10 15Time(t)0.0030.0020.0010.0000.0010.0020.003Relative error: c(t)1.801.851.001.02Figure 3.6: Comparison between the value function iteration and approximate solution using adeep neural network for the sequential neoclassical growth model with a medium time horizon (i.e.,Xtrain = {0, 1, . . . , 9}). The solid curves in the left panels show the median of the approximate cap-ital and consumption paths. The dashed curves show the capital and consumption paths obtainedby the value function iteration method. The solid curves in the right panels show the median ofthe relative errors between the approximate solutions and solutions obtained by the value functioniteration method. The shaded regions show the 10th and the 90th percentiles. The dashed verticallines separate the interpolation from the extrapolation region.Figure 3.6 shows the results for the sequential neoclassical growth model with Xtrain = {0, 1, . . . , 9}.I used the same parameters and deep neural network as the previous experiments. The dashed ver-tical lines separate the interpolation from the extrapolation region. The solid curve in the top-left61panel shows the median of the approximate capital paths, the dashed curve shows the capital pathobtained by the value function iteration method, and the shaded region shows the 10th and the90th percentiles of the approximate capital paths. The solid curve in the top-right panel shows themedian of the relative errors between the approximate capital paths and the capital path obtainedby the value function iteration method, the shaded region shows the 10th and the 90th percentilesof the relative errors. The solid curve in the bottom-left panel shows the median of the approx-imate consumption paths, the dashed curve shows the consumption path obtained by the valuefunction iteration method, and the shaded region shows the 10th and the 90th percentiles of theapproximate consumption paths. The solid curve in the bottom-right panel shows the median ofrelative errors between the approximate consumption paths and the consumption path obtained bythe value function iteration method, the shaded region shows the 10th and the 90th percentiles ofthe relative errors.These results show that I obtain solutions that do not violate the transversality condition evenwhen I use medium time horizons in Xtrain. Moreover, the long-run errors do not impair the ac-curacy of short run dynamics (less than 0.5% after 5 periods). Therefore, I get accurate solutionswhen the capital and consumption paths are far from the steady state. This result is robust tousing very short time horizons in Xtrain (i.e., tN = 4). See figure B.5 and the discussion in sectionB.2.4 for more details.Balanced growth path (g > 0): in the previous experiments I assumed that the total factorproductivity is constant and stationary. Here I show that exploiting a priori economic knowledgecombined with the implicit bias of deep neural networks, the method can deal with models whereno stationary solution exists. Since the production function is homogeneous of degree one, in thelong-run the capital path has a growth rate of g. This economic knowledge can be implementedflexibly in the design of the function space H(Θ). In this experiment I construct an approximatingfunction of the following form:kˆ(t; θ) = eϕtNN(t; θ1),where θ ≡ {ϕ, θ1}, NN(·; θ1) is a deep neural network, and ϕ is a single parameter that needs tobe found in the optimization process.Again for this experiment, I solve the minimization problem described in (3.28) for 100 times.Each time with a different random initialization of the parameters of the deep neural network. Ireport the median, the 10th, and the 90th percentiles of the results.Figure 3.7 shows the results for sequential neoclassical growth for non-stationary total factorproductivity (i.e., g = 0.02). I use the same deep neural network as before for NN(·; θ1) and usethe original grid, Xtrain = {0, 1, 2, . . . , 29}. The only difference is the additional exponential term.The solid curve in the top-left panel shows the median of the approximate capital paths and thedashed curve shows the capital path obtained by the value function iteration method. The shadedregion shows the 10th and the 90th percentiles of the approximate capital paths. The solid curve in62the top-right panel shows the median of the relative errors between the approximate capital pathsand the capital path obtained by the value function iteration method. The shaded region shows the10th and the 90th percentiles of the relative errors. The solid curve in the bottom-left panel showsthe median of the approximate consumption paths, the dashed curve shows the consumption pathobtained by the value function iteration method. In this case the 10th and the 90th percentilesfor the relative errors are so close to each other that they are not visible in the plot. The solidcurve in the bottom-right panel shows the median of the relative errors between the approximateconsumption paths and the consumption path obtained by the value function iteration method, theshaded region shows the 10th and the 90th percentiles of the relative errors.These results show that the long-run errors do not impair the short- and medium-run accuracyeven in the presence of balanced growth path. It is worth noting that if interpolation is theprimary objective (left of the dashed lines), the same result can be achieved without including theexponential term in the approximating function. However, when extrapolation (right of the dashedlines) is the objective, incorporating economic knowledge can significantly enhance generalizationpower (less than 0.03% relative errors in capital after 49 periods).Here I used the correct information about the functional form of the growth in capital andconsumption paths. However, in cases where the production function is not homogeneous of degreeone or the total factor productivity follows a non-linear process, it is possible to misspecify thefunctional form of the growth. Figure B.6 in section B.2.5 confirms that even in the presence offunctional misspecification long-run errors do not impair the accuracy of the short- and medium-run dynamics. Moreover, the results shows that the solutions generalize well in the extrapolationregion.3.3.3 Minimum norm interpretation and transversality conditionIn this section I provide the connection between minimum norm interpretation and the transver-sality condition. I focus on the case of zero total factor productivity growth (i.e., g = 0 and z = 1).The solutions that satisfy the Euler equation and feasibility condition (i.e., equations 3.23-3.24) and violate the transversality condition can be characterized by their asymptotic behavior.The common feature between these solutions is that the consumption goes to zero over time. Let{c˜(t), k˜(t)}∞t=0 be one of those solutions, then:limt→∞ c˜(t) = 0limt→∞ k˜(t) = k˜max,where k˜max solvesδk˜max = f(k˜max).Since the production function f is increasing and strictly concave, k˜max exists and is unique.Moreover, it is the stable fixed point for a dynamical system described by k(t+1) = f(k(t))+(1−630 10 20 30 40 50Time(t)1234Capital: k(t)k(t)k(t)0 10 20 30 40 50Time(t)0.0050.0000.0050.0100.0150.020Relative error: k(t)0 10 20 30 40 50Time(t)0.51.01.52.02.5Consumption: c(t)c(t)c(t)0 10 20 30 40 50Time(t)0.00100.00050.00000.00050.00100.00150.0020Relative error: c(t)4.34.42.502.552.60Figure 3.7: Comparison between the value function iteration and approximate solution using a deepneural network for the sequential neoclassical growth model with growth in total factor productivity(i.e., g = 0.02). The solid curves in the left panels show the median of the approximate capitaland consumption paths. The dashed curves show the capital and consumption paths obtained bythe value function iteration method. The solid curves in the right panels show the median of therelative errors between the approximate solutions and the solutions obtained by the value functioniteration method. The shaded regions show the 10th and the 90th percentiles. The dashed verticallines separate the interpolation from the extrapolation region.δ)k(t). For production functions of the form f(k) = kα:k˜max = δ1α−1 .64Comparing k˜max with the steady state of the capital k∗ ≡ (β−1+δ−1α ) 1α−1 , one can establish that:k˜max > k∗.The blue curves in figure 3.8 show a set of paths for capital, consumption and marginal utility ofconsumption (shadow prices), denoted by k˜(t), c˜(t), and u′(c˜(t)), that satisfy the Euler equationand feasibility condition, and violate the transversality condition.39 The black curves, denoted byk(t), c(t), and u′(c(t)), show the optimal paths for capital, consumption and marginal utility ofconsumption. These paths satisfy the Euler equation, feasibility condition and the transversalitycondition. The steady states for capital and consumption in the optimal solution are denoted byk∗ and c∗. It is important to note that for this set of parameters k˜maxk∗ ≈ 15. The minimum norm0 50 100Time(t)051015202530Capitalk(t)k(t)k *kmax0 50 100Time(t)0.00.20.40.60.81.0Consumptionc(t)c(t)c *0 50 100Time(t)101103105107Marginal utility of consumptionu′(c(t))u′(c(t))u′(c * )Figure 3.8: Comparison between the optimal solution and the solutions that violate the transver-sality condition for k0 < k∗. The blue curves, denoted by k˜(t), c˜(t), and u′(c˜(t)), show a set ofcapital, consumption, and marginal utility of consumption paths that violate the transversalitycondition. The black curves denoted by k(t), c(t), and u′(c(t)) show the capital, consumption, andmarginal utility of consumption paths for the optimal solution. The steady states for capital anconsumption are denoted by k∗ and c∗.interpretation of the solutions of the minimization problem described in (3.26) for the limiting case39The parameters I use are k0 = 0.4, β = 0.9, α = 0.33, and δ = 0.1.65of T →∞ can be written asmink(·;θ)∈H(Θ)∥k(·; θ)∥Ss.t. u′(c(t; k(·; θ))) = βu′(c(t+ 1; k(·; θ)))[f ′(k(t+ 1; θ))+ 1− δ] for t ∈ Xtraink(0) = k00 = limT→∞βTu′(c(T ; k(·; θ)))k(T + 1),where H(Θ) is a space of over-parameterized functions, Xtrain is a set of points in time (i.e.,Xtrain = {t1, . . . , tN}), the semi-norm ∥ · ∥S satisfies assumption 1, and consumption functionc((t; k(·; θ)) is defined in equation (3.25).As evident in figure 3.8, for the case of k0 < k∗, any capital sequence that starts from k0and violates the transversality condition converges to k˜max. Both capital paths k˜(t) and k(t)increase monotonically. Starting from k0, k˜(t) must have bigger derivatives than k(t) to reachk˜max. Therefore, the solutions that violate the transversality condition have bigger gradients. Moreformally, let k˜(t) be a capital path violating the transversality condition and k(t) be the optimalsolution, then in a compact space of the form [0, T ]:∫ T0∣∣∣∣∣dk˜dt∣∣∣∣∣2dt >∫ T0∣∣∣∣dkdt∣∣∣∣2 dt.Therefore, by assumption 1:∥k∥S < ∥k˜∥S .The solutions that violate the transversality condition have bigger semi-norms. In other words,the interpolating solutions of a space of over-parameterized functions has a bias toward k(t) andautomatically satisfy the transversality condition. Therefore, the optimization described in (3.28) isenough to find the optimal solution without imposing any explicit regularity regarding the long-runbehavior.It is important to note that when the transversality condition is violated the marginal utilityof consumption (shadow price) grows boundlessly (order of 107 after 100 periods). Alternatively,one can use an over-parameterized function to approximate the shadow price. Since the optimalsolution has bounded shadow prices, it is easy to establish that40:∥u′(c)∥S < ∥u′(c˜)∥S .40For the case of shadow prices assumption 1 can be relaxed. Because for all the semi-norms (and norms) I amaware of, an unbounded function has a bigger semi-norm than a bounded function.66Since the difference between the derivatives of marginal utility of consumption for the solutionsthat violate the transversality condition and the optimal solution is much larger than the differencebetween the derivatives of their corresponding capital paths, it is easier for the implicit bias ofover-parameterized functions to detect the optimal solution. Therefore, in practice approximatingthe marginal utility yields faster convergence to the optimal solution.In this argument I focus on cases where the initial value of capital is below the steady state.For cases where the initial condition is above the steady state the same argument can be made, seefigure B.4 and the discussion in section B.2.2.3.3.4 Neoclassical growth model with multiplicity of equilibria: sequentialformThe optimal solution of the sequential neoclassical growth model discussed and studied in section3.3 has a steady state characterized by (k∗, c∗), while the solutions that violate the transversalitycondition approach a fixed point at (k, c) = (k˜max, 0). Given that for regular parameters k˜max isextremely larger that k∗, the semi-norm of the capital path that converges to k∗ is substantiallysmaller than those that violate the transversality condition. Therefore, it is easy for the implicitbias to distinguish the difference between these two solutions and choose the optimal solution path.However, a natural question that arise is: can the implicit bias of over-parameterized functionsdistinguish the optimal path in the presence of multiplicity of steady states? Especially, when thesteady states are fairly close to each other.In this section I address this question by studying the sequential neoclassical growth problemwith a convex-concave production function of the form:f(k) = amax{kα, b1kα − b2}, (3.31)where a, b1, b2 a are positive constants and b1 > 1.41 This production function is continuous,however, has a kink at(b2b1−1) 1α . I embed this production function in the sequential setup ofneoclassical growth described in section 3.3. Specifically, I solve the optimization illustrated in(3.28) with this convex-concave production function. In this experiment I focus on constant totalfactor productivity (i.e., z0 = 1 and g = 0).42This problem has two steady states for capital:k∗1 =(β−1 + δ − 1aα) 1α−1k∗2 =(β−1 + δ − 1ab1α) 1α−1.41See Skiba (1978) for a comprehensive and detailed treatment of this problem in a continuous time setup.42In this experiment I use Adam optimizer instead of LBFGS. The Adam optimizer is more stable than LBFGS.Due to Adam’s longer process of optimization (more optimization steps), the solution is exposed to more implicitregularization, which is very desirable for this problem.67with the corresponding steady state for consumption c∗1 and c∗2.Figure 3.9 shows the results for β = 0.9, α = 0.33, δ = 0.1, g = 0, z0 = 1, a = 0.5, b1 = 3,b2 = 2.5, and k0 = {0.5, 1.0, 3.0, 4.0}. In this experiment Xtrain = {0, 1, 2, . . . , 29} and Xtest ={0, 1, 2, ..., 49}. I approximate the capital path using a deep neural network with four hidden layers,each with 128 nodes. Hidden layers use Tanh and the output layer uses Softplus as activationfunctions. The left panel shows the capital paths and the right panel shows the correspondingconsumption paths. The dashed vertical lines separate the interpolation from the extrapolationregion.These results show that even in the presence of multiplicity of steady states, the implicit bias0 10 20 30 40 50Time(t)0.51.01.52.02.53.03.54.0Capital: k(t)k(t) : k0=0.50k(t) : k0=1.00k(t) : k0=3.00k(t) : k0=4.00k *1k *20 10 20 30 40 50Time(t)0.30.40.50.60.70.8Consumption: c(t)c(t) : k0=0.50c(t) : k0=1.00c(t) : k0=3.00c(t) : k0=4.00c *1c *2Figure 3.9: The approximate capital and consumption paths for the sequential neoclassical growthmodel with a convex-concave production function. The left panel shows the capital paths forfour levels of initial capital k0 ∈ {0.5, 1.0, 3.0, 4.0} and the right panel shows the correspondingconsumption paths. The dashed horizontal lines show the steady states of capital k∗1 and k∗2 andthe corresponding steady states for consumption c∗1 and c∗2. The dashed vertical line separates theinterpolation from the extrapolation region.chooses the right paths for capital and consumption in the vicinity of the steady states. It is worthnoting that when the initial capital is above k∗2 or below k∗1 the optimal solution has the lowest semi-norm and the implicit bias of deep neural networks can accurately represent the optimal solution.Figure 3.10 shows the capital and consumption paths for a grid of initial conditions k0 ∈[0.5, 1.75] and k0 ∈ [2.75, 4]. The top panel shows the capital paths and the bottom panel showsthe consumption paths. The dashed horizontal lines show the steady states of capital k∗1 and k∗2and the corresponding steady states for consumption c∗1 and c∗2. The dashed vertical lines separatethe interpolation from the extrapolation region.680 10 20 30 40 50Time(t)0.51.01.52.02.53.03.54.0Capital: k(t)k *1k *20 10 20 30 40 50Time(t)0.30.40.50.60.70.8Consumption: c(t)c *1c *2Figure 3.10: The approximate capital and consumption paths for the sequential neoclassical growthmodel with convex-concave production function. The grid for the initial condition of capital belongsto [0.5, 1.75] and [2.75, 4]. The top panel shows the capital paths and the bottom panel shows theconsumption paths. The dashed horizontal lines show the steady states of capital k∗1 and k∗2 andthe corresponding steady states for consumption c∗1 and c∗2. The dashed vertical line separates theinterpolation from the extrapolation region.These results show that solutions are robust to variations in the initial condition for capital inthe vicinity of the steady states.It is worth noting that the proposed method has two basins of attraction in the space of initiallevels of capital. There exists a critical point, where above that point all the capital paths convergeto the higher steady state of capital (i.e. k∗2) and below that point all the capital paths convergeto the lower steady state of capital (i.e. k∗1). I do not expect the solution method to find a reliable69solution when the initial condition for capital is very close to that critical point.43See the discussion in section B.2.6 and figure B.7 for an analysis of the approximate solutionsvery close to the critical point.3.3.5 Connection to turnpike theoryAs discussed in the introduction, classic turnpike theorems provides conditions under which thesolution of a finite-horizon dynamic problem is close to the infinite-horizon version of the problem.In particular, they emphasize how the dynamics local to a “destination” are not relevant if mostof the time will be spent local to a stationary solution (i.e., the turnpike). In this work, I take thisfurther. As shown in the results, I can entirely focus on dynamics local to the origin, without evendirectly characterizing the steady state.Deep learning solutions and the turnpike: in this work I do not specify any terminalcondition for the capital at the last point of the grid (i.e., tN ) and let the problem to be ill-posedand rely on the implicit bias of the deep neural networks to find the approximate path of thefinite-horizon problem. As shown in figure 3.5 the approximate capital path fully stays close to thepath of infinite-horizon problem and does not deviate in the interpolation region. Moreover, in theextrapolation region it stays on the turnpike (at most 0.03% relative error for capital). As evidentin figure 3.6, these results are not sensitive to the time horizon used in Xtrain. The approximatepath for capital stays very close to the infinite-horizon path even when smaller time horizons areused (i.e., tN = 9). As shown in figure 3.7 these results hold for non-stationary neoclassical growthmodel regardless of the functional specification of the growth. Therefore, deep learning solutionshave a strong tendency toward the turnpikes (infinite-horizon paths) because of their implicit biastoward interpolating solutions with small derivatives.3.4 Implicit bias of deep neural networksFor a given space of over-parameterized functions H(Θ), a residual ℓ(·, ·), and a grid Xtrain theinterpolation problem can be written asminΨ(·;θ)∈H(Θ)1|Xtrain|∑x∈Xtrainℓ(Ψ(·; θ), x)2. (3.32)Since in over-parameterized interpolation problems, such as interpolation with deep neuralnetworks, the number of parameters is larger than the number of grid points there are many inter-polating solutions that can achieve 1|Xtrain|∑x∈Xtrain ℓ(Ψ(·; θ), x)2 = 0.43See Benjamin Moll’s lecture notes for a global treatment of this problem in a continuous time setup athttps://benjaminmoll.com/wp-content/uploads/2020/06/skiba.pdf.70Discussion regarding the validity of assumption 1: over-parameterized function approx-imation, specially deep learning, has shown an incredible generalization power, see Zhang et al.(2021). Given that they typically have many more parameters than data points, they are capableof achieving exact interpolation. One might expect this exact interpolation leads to over-fittingand hence poor generalization power. However, in many empirical settings it has been shown thatis not the case, to name a few see Belkin et al. (2019), Neyshabur et al. (2014).One approach to avoid over-fitting in over-parameterized problems is to use explicit regulariza-tion. The two most common methods are L1 and L2 regularization. L1 regularization correspondsto penalizing the sum of the absolute values of the parameters in the approximating function (sim-ilar to Lasso regression). L2 regularization corresponds to penalizing the sum of the squared ofparameters (similar to ridge regression). Surprisingly, it has been shown that, even without anyexplicit regularization, they can still achieve good generalization.These two observations have led researchers to believe that the optimization methods used inover-parameterized interpolation posses an intrinsic implicit bias toward a specific class of solutions,for example see Neyshabur et al. (2017), Arora et al. (2019), and Smith et al. (2021). Understandingthe nature of this implicit bias is still an active field of research in computer science, optimizationtheory, and statistics. However, it is mostly believed that the implicit bias is caused by the firstorder optimization methods such as gradient descent and stochastic gradient descent, which arewidely used in over-parameterized interpolation problems. It has been established that the firstorder optimization methods favor flat minima in the space of parameters. In this context the flat-ness/sharpness of the minima is characterized by the trace of the Hessian matrix of the objectivefunction.For instance, in regression problems, Damian et al. (2021) show that the stochastic gradientdescent minimizes∥Ψ(·; θ)∥S ≡ −1ηtr[log(I − η21|Xtrain|∑x∈Xtrain∇2θℓ(Ψ(·; θ), x)2)],where I is the M ×M identity matrix, η is the size of the optimization step (learning rate), and∇2θℓ(Ψ(·; θ), x)2 is the Hessian of the objective function. In the interpolation regime, where zeroof the objective function (i.e., the objective function defined in 3.32) is reached, there is a tightconnection between the trace of the Hessian and the gradient of the interpolating function withrespect to its parameters. Blanc et al. (2020) show that the stochastic gradient descent minimizes∥Ψ(·; θ)∥S ≡ 1|Xtrain|∑x∈Xtrain∥∇θΨ(x; θ)∥22,which is the L2 norm of the gradient of the interpolating function with respect to the parameters.Convergence to a flat minima in the space of parameters does not provide enough informationabout the properties of the function with respect to its inputs such as smoothness and small deriva-tives. Due to the multiplicative structure of deep neural networks, Ma and Ying (2021) observe71that there is a tight connection between the flatness of a minima and the norm of the gradientof the function with respect to its inputs. More specifically, in an interpolation regime, stochasticgradient descent as an optimization algorithm implicitly minimizes the Sobolev semi-norm of theapproximating function. In other words deep neural networks along with their optimizers havea tendency to find approximating functions that have small derivatives (i.e., not explosive in thecontext of this chapter).Here I provide the mathematical definition of Sobolev 1−2 semi-norm for a univariate function.Definition 6 (Sobolev 1-2 seminorm). Let Ψ be a univariate function from a compact space inX ⊂ R to R, Sobolev 1-2 semi-norm is defied as:∥Ψ∥1,2,X ≡(∫X∣∣∣∣dΨdx∣∣∣∣2 dx) 12 .Therefore, among two differentiable interpolating functions Ψ1 and Ψ2 such that:(∫X∣∣∣∣dΨ1dx∣∣∣∣2 dx) 12 > (∫X∣∣∣∣dΨ2dx∣∣∣∣2 dx) 12 ,there is an implicit bias toward Ψ2.It is worth mentioning that Maennel et al. (2018) study the case of a one hidden layer neu-ral network with ReLU activation function (i.e., ReLU(x)= max{0, x}) with low-dimensional datapoints. They show when the initial parameters are set to have small values, the gradient descentconverges to a data dependent (as opposed to network size dependence) linear interpolation whichis an indication of an implicit bias toward low Sobolev semi-norms.Remark: all the theoretical and empirical results in the literature regarding the implicit bias ofover-parameterized interpolation focus on regression and classification problems. The interpolationproblems I solve in theoretical equilibrium models have a different nature and the objective functions(such as Bellman or Euler residuals) are slightly different from regression problems. However, asverified in the result sections of this work, I strongly believe the same results can be proved for thesort of problems I considered in this chapter.3.5 ConclusionDynamic models of forward-looking agents in economics require conditions for long-run behavior(e.g, transversality and no-bubble condition). These conditions rule out explosive solutions. Inthis chapter, I show, in sequential setups, that how deep learning approximations automaticallysatisfy these conditions without explicitly imposing them through calculating the steady state evenin models with steady-state multiplicity. What makes deep learning approximations special is theirinherent implicit bias toward functions with small derivatives that match the class of solutions we72are interested in dynamic models. These results suggest that deep learning may let us calculateaccurate transition dynamics with high-dimensional state spaces without directly solving for theboundary conditions at infinity.73Chapter 4Exploring Transversality andStationarity with Deep Learning inRecursive Setups4.1 IntroductionIn the previous chapter I established, in sequential models, deep learning chooses the solutionsthat do not violate the boundary conditions that are imposed by forward-looking behavior. Morespecifically, I showed that in the linear asset pricing model, the implicit bias of solutions based ondeep learning is toward interpolating solutions that do not violate the no-bubble condition. Also,in the sequential formulation of the neoclassical growth model this bias is towards solutions thatsatisfy the transversality condition.The sequential models outlined in the previous chapter can be very useful to study deterministicsetups and one-time shocks to economic variables (MIT shocks). However, they are not suitable fordealing with shocks that happen in every period. In this Chapter I focus on the recursive setups.To be more specific, I consider the recursive version of the neoclassical growth model, which canbe extended to study investment decision under uncertainty.In this chapter I establish that solving the Euler equation with deep learning interpolation yieldssolutions that satisfy the transversality condition. This solution is robust to random initializationof the parameters of the neural networks. In other words, the implicit bias causes convergence tothe same function, regardless of the initial values of the parameters. In recursive setups it is re-quired to have an accurate solution over the whole state space, regardless of whether the economicvariables are going to visit that region of space. In this chapter I investigate whether deep learningcan yield accurate short- and medium-run solutions when the grid for capital is local to the initialcondition for capital. This is done by choosing a grid for capital that contains the initial levelof capital and does not include the steady state. I show that, in this case, deep neural networkprovides an accurate short- and medium-run solutions.I also show that with appropriate transformations of the state-space deep learning provides anaccurate solution in the presence of balanced growth path. Moreover, this transformation thatpartially stems from economic a priori knowledge leads to outstanding generalization.As stated before, these boundary conditions, such as transversality condition, are necessaryconditions to obtain the optimal solution. Therefore, it is possible to have solutions that achieve74accurate interpolation on the Euler equation (equivalently the Bellman equation), but violate theseboundary conditions. In this chapter I provide two examples of this phenomenon and show howsolutions of this kind can be avoided by combining economic knowledge and theoretical understand-ing of the implicit bias of deep learning methods.The rest of this chapter is organized as follows: section 4.2 introduces the recursive form of theneoclassical growth model, provides a theoretical insight on the connection between the minimumnorm interpolation and the solutions that satisfy the transversality condition, and presents theresults. Section 4.3 presents how transversality conditions may be violated even with low Euler orBellman residuals and provides a theoretical discussion on why this happens. Section 4.4 providessome final remarks.4.2 The neoclassical growth model: recursive formSetup: the Bellman equation for the neoclassical growth model, introduced in section 3.3, can bewritten asv(k, z) =maxc,k′{u(c) + βv(k′, z′)}(4.1)s.t. k′ = z1−αf(k) + (1− δ)k − c (4.2)z′ = (1 + g)z (4.3)k′ ≥ 0, (4.4)where k is the capital, c is the consumption, and z is the total factor productivity. Capital depre-ciates with rate δ ∈ [0, 1], the agent discounts the future with β ∈ (0, 1). I focus on the constantrelative risk aversion utility functions u(c) = c1−σ1−σ , and production function of the form f(k) = kα.The Euler equation for the recursive form of the neoclassical growth model can be written as:u′(c) = βu(c′)[z′1−αf ′(k′) + 1− δ], (4.5)where k′ and c′, z′ are the capital, consumption and total factor productivity next period. Since thisis a recursive model, k′ and c are functions of capital k, and total factor productivity z, therefore,the domain of interest is X ≡ R2+. The objects of interest are policy function for capital k′(k, z) andconsumption function c(k, z) such that they satisfy the Euler equation and the feasibility condition.Moreover, the optimality of the solution requires all the capital and consumption paths to satisfythe transversality condition for all initial conditions in X :0 = limT→∞βTu′(cT (k0, z0))k′T (k0, z0) for all (k0, z0) ∈ X , (4.6)75where k′T (k0, z0) and cT (k0, z0) are generated by iterating forward the optimal policy function forcapital k′(k, z), the consumption function c(k, z), and the law of motion for total factor productivityup to the T th period for a given k0 and z0.4.2.1 Interpolating solutionAs a generalization of the collocation approach one can pick a space of parametric functions H(Θ),where Θ ≡ {θ1, . . . , θM} represents the parameters. For instance, in a standard collocation methodH(Θ) can be the space of two-dimensional Chebyshev polynomials with M parameters.44 Theinterpolating capital function k(·, ·; θ) : X → R belongs to H(Θ). In order to represent the domainX , one can pick a finite set of grid points Xtrain ⊂ X. Since this is a two-dimensional recursivemodelXtrain ≡ {k1, . . . , kNk} × {z1, . . . , zNz}. (4.7)This is a Cartesian product of sets of points in the capital and total factor productivity space. TheEuler equation (i.e., equation 4.5), the feasibility condition (i.e, equation 4.2), the law of motionfor the total factor productivity (i.e., equation 4.3), and the transversality condition (i.e., equation4.6) form a system of equations. However, the transversality condition is a boundary condition atinfinity. One can approximate infinity by choosing a very large point in time, denoted by T . Inthe recursive case, T should be large enough such that the optimal capital path is very close to thesteady state at time T .For a given policy function for capital k′(k, z; θ) ∈ H(Θ) I define a consumption functionc(k, z; k′(·; θ)) through the feasibility condition:c(k, z; k′(·; θ)) ≡ z1−αf(k) + (1− δ)k − k′(k, z; θ). (4.8)Given a space of parametric functions H(Θ), a grid Xtrain, and a very large point in time T , onecan find the interpolating policy function for capital by solving the following optimization problem:minθ∈Θ[1|Xtrain|∑(k,z)∈Xtrain(u′(c(k, z; k′(·; θ)))u′(c(k′(k, z; θ), (1 + g)z; k′(·; θ))) − β[((1 + g)z)1−αf ′(k′(k, z; θ))+ (1− δ)])2+1|Xtrain|∑(k0,z0)∈Xtrain(βTu′(cT(k0, z0; k′(·; θ)))k′T (k0, z0; θ))2︸ ︷︷ ︸Is this necessary?]. (4.9)The first term in the optimization problem is the Euler residuals and the second term is a finite ap-proximation of the transversality condition. The non-negativity of k′(k, z; θ) can be built intoH(Θ).44Since this is a two-dimensional problem, one should adjust the Chebyshev polynomials and Chebyshev nodes.76Choice of Xtrain: similar to the previous cases the choice of Xtrain is not very crucial. However,I provide results for different grids. I also evaluate the interpolating policy function for values ofcapital outside of Xtrain to study the generalization performance of the solutions.In standard collocation methods, where the number of parameters is equal to the number ofgrid points, this problem can be solved exactly as a system of equations with a boundary conditionat T . It is worth noting that the interpolating solution depend on T . Therefore, the minimizationproblem described in (4.9) defines a sequence of solutions indexed by T .Is the transversality condition necessary? Without the transversality condition the re-cursive version of the neoclassical growth has more than one solution. Therefore, in the absence ofthe second term (finite approximation of the transversality condition) the minimization problemdescribed in (4.9) has more than one solutions that minimize the objective function. As noted byFerna´ndez-Villaverde et al. (2016), without explicitly imposing the transversality condition, it isnecessary to verify that solutions satisfy the transversality condition. This verification process isdone after solving the minimization problem by generating capital and consumption paths fromsome initial conditions.In section 4.2.3, I establish how using over-parameterized functions (M ≫ Nk ×Nz) and theiroptimization processes yield an interpolating solution that does not violate the transversality con-dition. Intuitively, the policy functions for capital that leads to violation of the transversalitycondition have bigger derivatives than the optimal policy function for capital. With this result, Ido not need to be worried about the transversality condition. Therefore, using over-parameterizedfunctions I can drop the transversality condition and solve:minθ∈Θ1|Xtrain|∑(k,z)∈Xtrain(u′(c(k, z; k′(·; θ)))u′(c(k′(k, z; θ), (1 + g)z; k′(·; θ))) − β[((1 + g)z)1−αf ′(k′(k, z; θ))+ (1− δ)])2,(4.10)to obtain an interpolating solution that does not violate the transversality condition.4.2.2 ResultsGenerating capital and consumption paths: in the recursive version of the neoclassicalgrowth, the interpolating solution is a policy function for capital k′(k, z; θ∗), whereθ∗ ≡ argminθ∈Θ1|Xtrain|∑(k,z)∈Xtrain(u′(c(k, z; k′(·; θ)))u′(c(k′(k, z; θ), (1 + g)z; k′(·; θ))) − β[((1 + g)z)1−αf ′(k′(k, z; θ))+ (1− δ)])2.Given k′(k, z; θ∗), I construct the consumption function c (k, z; k′(·; θ∗)) defined by equation (4.8).In order to generate the capital path up to period t, I apply the interpolating policy function forcapital and the law of motion for total factor productivity t times from given initial conditions k0and z0. Similarly, the consumption path is constructed by applying the consumption function t77times given the capital and total factor productivity level.This procedure provides a pair of approximate capital and consumption paths. In this section,I solve the minimization problem described in (4.10) for 100 times. Each time with a differentrandom initialization (random seed) of the parameters of the deep neural network, then I generatea pair of capital and consumption paths. This procedure provides 100 pairs of approximate capitaland consumption paths. I report the median, the 10th and the 90th percentiles of these paths.I also report the median, the 10th, and the 90th percentiles of the relative errors between theseapproximate paths a the paths generated by the value function iteration method.Figure 4.1 shows the result of the minimization problem described in (4.10) for β = 0.9, α = 0.33,δ = 0.1, g = 0, k0 = 0.4, and z0 = 1. In this experiment I use a grid with 16 points between k1 = 0.8and kNk = 2.5. I use a deep neural network with four hidden layers, each with 128 nodes. Eachhidden layer uses Tanh and the output layer uses Softplus as activation functions. The solid curvein the top-left panel, denoted by kˆ(t), shows the median of the approximate capital paths. Thedashed curve, denoted by k(t), shows the capital path obtained by the value function iterationmethod. The solid curve is accompanied with the 10th and the 90th percentiles of the approximatecapital paths. However, they are so close to each other that they are not visible in the plot. Thegray rectangle shows the interpolation region, i.e., [0.8, 2.5]. The top-right panel shows the medianof the relative errors between the approximate capital paths and the capital path obtained by thevalue function iteration method. The shaded region shows the 10th and the 90th percentiles of therelative errors. The dashed part of the curve shows the median of relative errors in the extrapola-tion region (i.e., the parts of the approximate capital path outside of [0.8, 2.5]) and the solid partshows the median of relative errors in the interpolation region (i.e., the parts of the approximatecapital path inside of [0.8, 2.5]). The solid curve, denoted by cˆ(t), in the bottom-left panel showsthe median of the approximate consumption paths. The dashed curve shows the consumption pathobtained by the value function iteration method. The bottom-right panel shows the median of therelative errors between the approximate consumption paths and the consumption path obtained bythe value function iteration method. The shaded region shows the 10th and the 90th percentiles ofthe relative errors. The dashed part of the curve shows the median of relative errors for consump-tion where the corresponding levels of capital are in the extrapolation region and the solid partshows the median of relative errors for consumption where the corresponding levels of capital arein the interpolation region.The results of this experiment are multi-fold. First, the implicit bias of deep neural networksyields a solution that automatically satisfies the transversality condition. Second, the solutionsare accurate within Xtrain. Third, given the initial level of capital, k0 = 0.4, lies outside ofXtrain = [0.8, 2.5], the generalization error are small (at most 5% for capital).As stated in the original problem, the transversality condition should hold for all the initiallevel of capital, figure C.1 in section C.1 shows the results for different initial conditions of capitalboth inside Xtrain and outside Xtrain.780 10 20 30 40 50Time(t)0.51.01.52.02.5Capital: k(t)k(t)k(t)Xtrain0 10 20 30 40 50Time(t)0.000.010.020.030.040.05Relative error: k(t)ExtrapolationInterpolation0 10 20 30 40 50Time(t)0.40.50.60.70.80.91.01.1Consumption: c(t)c(t)c(t)0 10 20 30 40 50Time(t)0.060.040.020.000.02Relative error: c(t)ExtrapolationInterpolation1.701.750.960.981.00Figure 4.1: Comparison between the value function iteration and approximate solution using adeep neural network for the recursive neoclassical growth model. The left panels show the medianof the approximate capital and consumption paths, denoted by kˆ(t) and cˆ(t). The dashed curves,denoted by k(t) and c(t), show the capital and consumption paths obtained by the value functioniteration method. The right panels show the median of the relative errors between the approximatepaths and the paths obtained by the value function iteration method. The shaded regions show the10th and the 90th percentiles. The dashed curves in the right panels show the relative errors forthe parts of the approximate capital paths that lie outside of the interpolation regions. The grayrectangle in the top-right panel shows the interpolation region.Far from the steady state: in the last experiment, the steady state for capital k∗ is includedin the interpolation region (i.e., k∗ ∈ [0.8, 2.5]). In this experiment I investigate whether I canachieve accurate short-run dynamics by using a local grid around the initial condition for capital.Specifically, I use a grid of the form Xtrain = {k1, . . . , kNk} with kNk < k∗.79Fig 4.2 shows the results for the recursive neoclassical growth using a local grid around theinitial level of capital k0 = 0.9. In this experiment I use a grid with 16 points between k1 = 0.8and kNk = 1.5. I use the same parameters and deep neural network as the previous experiment.The solid curve in the top-left panel, denoted by kˆ(t), shows the median of the approximate capitalpaths. The dashed curve, denoted by k(t), shows the capital path obtained by the value functioniteration method. The shaded region shows the 10th and the 90th percentiles of the approximatecapital paths. The gray rectangle shows the interpolation region(i.e., [0.8, 1.5]). The top-right panelshows the median of the relative errors between the approximate capital paths and the capital pathobtained by the value function iteration method. The shaded region shows the 10th and the 90thpercentiles of the relative errors. The dashed part of the curve shows the median of relative errorsin the extrapolation region (i.e., the parts of the approximate capital path outside of [0.8, 1.5])and the solid part shows the median of relative errors in the interpolation region (i.e., the parts ofthe approximate capital path inside of [0.8, 1.5]). The solid curve, denoted by cˆ(t), in the bottom-left panel shows the median of the approximate consumption paths. The dashed curve shows theconsumption path obtained by the value function iteration method. The shaded region shows the10th and the 90th percentiles of the approximate consumption paths. The bottom-right panel showsthe median of the relative errors between the approximate consumption paths and the consumptionpath obtained by the value function iteration method. The shaded region shows the 10th and the90th percentiles of the relative errors. The dashed part of the curve shows the median of relativeerrors for consumption where the corresponding levels of capital are in the extrapolation region andthe solid part shows the median of relative errors for consumption where the corresponding levelsof capital are in the interpolation region.These results show that one can achieve accurate short run dynamics when the grid for capitalis locally defined around the initial capital and is far from the steady state. For the first few periodsthe relative error for capital is less then 1%. Moreover, the relative errors stay bounded and do notgrow excessively even after 20 periods.Balanced growth path (g > 0): in this experiment I focus on the positive growth in totalfactor productivity (i.e. g > 0). I partially use an a priori economic intuition that the solutionis homogeneous of degree one in z. I design the space of functions H(Θ) such that it containsfunctions of the form:kˆ′(k, z; θ) = zNN(kz, z; θ),where NN(·, ·; θ) is a deep neural network. I say partially, because the correctly specified functionalform of a homogeneous of degree one function is zNN(kz ; θ).Figure 4.3 shows the results for recursive neoclassical growth for non-stationary total factorproductivity (i.e., g = 0.02). I use 16 points in [0.8, 3.5] for capital, and 8 points in [0.8, 1.8] fortotal factor productivity. The deep neural network NN(·, ·; θ) is the same as the first experiment.The only difference is the linear term z.The solid curve in the top-left panel, denoted by kˆ(t), shows the median of the approximate800 5 10 15Time(t)0.60.81.01.21.41.61.82.0Capital: k(t)k(t)k(t)Xtrain0 5 10 15Time(t)0.140.120.100.080.060.040.020.00Relative error: k(t)ExtrapolationInterpolation0 5 10 15Time(t)0.50.60.70.80.91.01.1Consumption: c(t)c(t)c(t)0 5 10 15Time(t)0.020.010.000.01Relative error: c(t)ExtrapolationInterpolation1.501.550.900.92Figure 4.2: Comparison between the value function iteration and approximate solution using adeep neural network for the recursive neoclassical growth model using a local grid around the initialcondition for capital Xtrain = {0.8, . . . , 1.5}. The left panels show the median of the approximatecapital and consumption paths, denoted by kˆ(t) and cˆ(t). The dashed curves, denoted by k(t) andc(t), show the capital and consumption paths obtained by the value function iteration method.The right panels show the median of the relative errors between the approximate paths and thepaths obtained by the value function iteration method. The shaded regions show the 10th and the90th percentiles. The dashed curves in the right panels show the relative errors for the parts of theapproximate capital path that lie outside of the interpolation regions. The gray rectangle in thetop-right panel shows the interpolation region.capital paths. The dashed curve, denoted by k(t), shows the capital path obtained by the valuefunction iteration method. The shaded region shows the 10th and the 90th percentiles of theapproximate capital paths. The top-right panel shows the median of the relative errors betweenthe approximate capital paths and the capital path obtained by the value function iteration method.81The shaded region shows the 10th and the 90th percentiles of the relative errors. The dashed partof the curve shows the median of relative errors in the extrapolation region (i.e., the parts ofthe approximate capital path outside of [0.8, 3.5]) and the solid part shows the median of relativeerrors in the interpolation region (i.e., the parts of the approximate capital path inside of [0.8, 3.5]).The solid curve, denoted by cˆ(t), in the bottom-left panel shows the median of the approximateconsumption paths. The dashed curve shows the consumption path obtained by the value functioniteration method. The solid curve is accompanied with the 10th and the 90th percentiles of theapproximate consumption paths. However, they are so close to each other that they are not visible inthe plot. The bottom-right panel shows the median of the relative errors between the approximateconsumption paths and the consumption path obtained by the value function iteration method.The shaded region shows the 10th and the 90th percentiles of the relative errors. The dashed partof the curve shows the median of relative errors for consumption where the corresponding levels ofcapital are in the extrapolation region and the solid part shows the median of relative errors forconsumption where the corresponding levels of capital are in the interpolation region.The results of this experiment are multi-fold. First, the implicit bias of deep neural networksyields a solution that automatically satisfies the transversality condition. Second, the solutions areaccurate within Xtrain (less than 1% relative error for capital in short- and medium-run). Third,the solution generalizes well (less than 5% relative errors after 40 periods).As stated in the original problem, the transversality condition should hold for all the initialconditions in X , figure C.2 in section C.1 shows the results for different initial conditions of capitalboth inside and outside of the interpolation region.4.2.3 Minimum norm interpretation and transversality conditionIn this section I provide the connection between minimum norm interpretation and the transver-sality condition for the case of recursive neoclassical growth. I focus on the case of zero total factorproductivity growth (i.e., g = 0 and z0 = 1).As discussed in section 3.3.3 the solutions that violate the transversality condition are associatedwith a sequence of consumption that approaches zero and a sequence of capital that approachesk˜max. Therefore, any policy function for capital k˜′(k) that solves the Euler equation and feasibilitycondition, and violates the transversality condition, must have a fixed point at k˜max. Formally:k˜′(k˜max)= k˜max.The optimal policy function for capital k′(k) has a fixed point at k∗ =(β−1+δ−1α) 1α−1 . Formally:k′ (k∗) = k∗.Given the fact that k˜max ≫ k∗, k˜′(·) must intersect with 45 degree line way further to the right ofk∗.820 10 20 30 40 50Time(t)1234Capital: k(t)k(t)k(t)0 10 20 30 40 50Time(t)0.100.080.060.040.020.00Relative error: k(t)ExtrapolationInterpolation0 10 20 30 40 50Time(t)0.51.01.52.02.5Consumption: c(t)c(t)c(t)0 10 20 30 40 50Time(t)0.01250.01000.00750.00500.00250.00000.00250.0050Relative error: c(t)ExtrapolationInterpolation3.33.41.901.95Figure 4.3: Comparison between the value function iteration and approximate solution using a deepneural network for the recursive neoclassical growth model with growth in total factor productivity(i.e., g = 0.02). I use Xtrain = [0.8, 3.5] × [0.8, 1.8]. The left panels show the median of theapproximate capital and consumption paths, denoted by kˆ(t) and cˆ(t). The dashed curves, denotedby k(t) and c(t), show the capital and consumption paths obtained by the value function iterationmethod. The right panels show the median of the relative errors between the approximate pathsand the paths obtained by the value function iteration method. The shaded regions show the 10thand the 90th percentiles. The dashed curves in the right panels show the relative errors for theparts of the approximate capital path that lie outside of the interpolation regions.In figure 4.4 the blue curve, denoted by k˜′(k), shows a policy function for capital that violates thetransversality condition and the black curve, denoted by k′(k), shows the optimal policy functionfor capital. The dashed line shows the 45 degree line. k˜′(k) intersects with the 45 degree line at83k˜max ≈ 30. 45The minimum norm interpretation of the solutions of the minimization problem described in(4.9) for the limiting case of T →∞ can be written as:mink′(·;θ)∈H(Θ)∥k′(·; θ)∥S (4.11)s.t. u′(c(k; k′(·; θ))) = βu′(c(k′(k); k′(·; θ)))[f ′(k′(k; θ))+ (1− δ)] for k ∈ Xtrain(4.12)k′(k; θ) ≥ 0 for k ∈ Xtrain (4.13)0 = limT→∞βTu′(cT (k0))k′T (k0) for all k0 ∈ X . (4.14)Since the total factor productivity is constant and is always equal to one, I replace k′(k, z; θ) byk′(k; θ) and Xtrain is defined as:Xtrain ≡ {k1, . . . , kNk}.H(Θ) is a space of over-parameterized functions and ∥ · ∥S satisfies assumption 1. As evident in2 4 6 8 10Capital(k)246810 k′(k): Violating TVCk ′(k)45 degree lineFigure 4.4: Comparison between the optimal solution, denoted by k′(k), and a solution that violatethe transversality condition, denoted by k˜′(k). The solution that violates the transversality equationintersects with the 45 degree line at k˜max ≈ 30.Figure 4.4, k˜′(k) has bigger derivatives than k′(k). More specifically, for an arbitrary compact space45In this figure the solutions that violate the transversality condition (i.e., k˜′(k)) are generated by transformingthe sequences of capitals that solve the sequential problem and violate the transversality condition.84of the form [kmin, kmax]:∫ kmaxkmin∣∣∣∣∣dk˜′dk∣∣∣∣∣2dk >∫ kmaxkmin∣∣∣∣dk′dk∣∣∣∣2 dk. (4.15)Therefore, by assumption 1∥k′∥S < ∥k˜′∥S . (4.16)Since the solutions that violate the transversality condition have bigger semi-norms, the over-parameterized interpolation eliminates those solutions. Therefore, the optimization problem de-scribed in (4.10) is enough to find the optimal policy function for capital k′(k, z; θ) without im-posing any explicit regularity regarding the long-run behavior, such a finite approximation of thetransversality condition.4.3 Are Euler and value function residuals enough?In dynamic models where there is multiplicity of solutions and some boundary conditions at infinityis required, finding the root of the functional operators such as Euler or value function residuals isnot enough. Due to multiplicity, very low levels of residuals can be misleading and the interpolatingsolutions might represent non-optimal ones.Minimizing the Euler residuals is not enough. In the recursive neoclassical growth model, Iwant to have a solution that minimizes the Euler residuals and satisfy the transversality condition.Equivalently, the optimal policy function for capital k′(k) has to minimize the Euler residuals andhave a fixed point at the steady state k∗. As illustrated in section 4.2.3, when I approximatethe policy function for capital k′(k) with a deep neural network, the implicit bias automaticallychooses the optimal solution. This happens because the policy functions for capital that violatethe transversality condition have bigger derivatives.Due to the trade off between the consumption function c(k) and the policy function for capitalk′(k) through the feasibility condition, if k′(k) has a bigger semi-norm then c(k) has smaller semi-norm. Therefore, if one decides to approximate the consumption function instead of the policyfunction for capital they get solutions that violate the transversality condition. Equivalently, thecorresponding policy function for capital has a fixed point at k˜max instead of the steady state k∗. Ifocus on the case of constant total factor productivity (i.e., g = 0 and z0 = 1) and in the notationdrop the total factory productivity as an input. The optimization I solve for approximating theconsumption function can be written as:minθ∈Θ1|Xtrain|∑k∈Xtrain(u′(c(k; θ))u′(c(k′(k; c(·; θ)); θ)) − β[f ′(k′(k; c(·; θ)))+ (1− δ)])2 , (4.17)85where for a given consumption function c(k; θ) ∈ H(Θ), the policy function for capital is defined ask′(k; c(·; θ)) ≡ f(k) + (1− δ)k − c(k; θ), (4.18)The policy function for capital is defined exactly and not approximated. The consumption functionc : R → R+ is approximated by a deep neural network. Similar to the previous cases the grid forcapital is defined as Xtrain ≡ {k1, . . . , kNk}.Figure 4.5 shows the comparison between approximating the policy function for capital k′(k)versus approximating the consumption function c(k) with a deep neural network. The economicparameters are the same as before. The solid curve in top-left panel shows the median of the Eulerresiduals squared over 100 different initialization of the parameters of the deep neural network(seeds) when k′(k) is approximated with a deep neural network.46 In this case, the Euler residualis defined as:εkE(k; θ) ≡u′(c(k; k′(·; θ)))u′(c(k′(k; θ); k′(·; θ))) − β [f ′(k′(k; θ))+ (1− δ)] ,where the superscript k denotes the Euler residuals when the policy function for capital is approx-imated with a deep neural network. The shaded region around the solid curve shows the 10th andthe 90 percentiles of the Euler residuals squared. The bottom-left panel shows the median of theapproximate policy functions for capital, when k′(k) is approximated with a deep neural network.The figure also contains the 10th and the 90 percentiles of the approximate policy functions forcapital. However, they are so close to each other that they are not visible in the plot. In thiscase the approximate policy function for capital has a fixed point at k∗ ≈ 2.0. The solid curve inthe top-right panel shows the median of the Euler residuals squared over 100 seeds when c(k) isapproximated with a deep neural network. In this case, the Euler residual is defined as:εcE(k; θ) ≡u′(c(k; θ))u′(c(k′(k; c(·; θ)); θ)) − β [f ′(k′(k; c(·; θ)))+ (1− δ)] ,where the superscript c denotes the Euler residuals when the consumption function is approximatedwith a deep neural network. The shaded region shows the 10th and the 90 percentiles of the Eulerresiduals squared. The bottom-right panel shows the median of the approximate policy functionsfor capital, when c(k) is approximated with a deep neural network. The shaded region showsthe 10th and the 90 percentiles of the approximate policy functions for capital. In this case theapproximate policy function for capital has a fixed point at k˜max ≈ 30.These results show that when the policy function for capital is approximated with a deep neuralnetwork, k′(k) has a fixed point at k∗ ≈ 2.0 and it does not violate the transversality condition(the bottom-left panel). However, when the consumption function is approximated with a deepneural network, the policy function for capital has a fixed point at k˜max ≈ 30 and it violates thetransversality condition (the bottom-right panel). As shown in the bottom-right panel, this result46I solve the minimization problem described in (4.17) for 100 times, each time with a different random initializationof the parameters of the deep neural networks. I report the median, 10th, and 90th percentiles of these solutions.861.0 1.5 2.0 2.5capital(k)10 1010 910 810 710 6Euler residuals2: NN= k ′(k)1.0 1.5 2.0 2.5capital(k)10 1010 910 810 710 6Euler residuals2: NN= c(k)1.0 1.5 2.0 2.5capital(k)1.01.52.02.53.03.5k ′(k): NN= k ′(k)k ′(k)45 degree line1.0 1.5 2.0 2.5capital(k)1.01.52.02.53.03.5k ′(k): NN= c(k)k ′(k)45 degree lineFigure 4.5: Comparison between approximating the policy function for capital k′(k) versus theconsumption function c(k) with a deep neural network. The left panels show the median of Eulerresiduals and policy functions for capital when k′(k; θ) is approximated with a deep neural network.The right panels show the median of Euler residuals squared and policy functions for capital whenc(k) is approximated with a deep neural network. The solid curves show the medians and theshaded regions show the 10th and the 90th percentiles over 100 different seeds.is robust to changing different random initialization of the parameters of the deep neural network.Most importantly, the Euler residuals for the solutions that violate the transversality condition (thetop-right panel) are systematically and substantially lower than the solution that does not violatethe transversality condition. Therefore, having low Euler residuals is not sufficient for convergenceto the optimal solution and can be very misleading.87Can explicit regularization solve the problem? As shown in figure 4.5 approximating theconsumption function c(k) instead of policy function for capital leads to solutions that violate thetransversality condition. One natural question that arises is whether explicit regularization of theparameters of the deep neural network can fix this problem. One of the most common methodsof explicit regularization is L2 regularization. This regularization is achieved by penalizing theL2 norm of the parameters of the neural networks (i.e.,∑θi∈Θ θ2i ).47 Therefore, the minimizationtakes the following form:minθ∈Θ[1|Xtrain|∑k∈Xtrain(u′(c(k; θ))u′(c(k′(k; c(·; θ)); θ)) − β [f ′(k′(k; c(·; θ)))+ (1− δ)])2+ λ∑θi∈Θθ2i],where λ is the penalization coefficient. I solve this minimization, with and without (i.e., λ = 0)the explicit regularization, for 100 times. Each time with a different random initialization of theparameters of the deep neural network. I report the median, the 10th, and 90th percentiles of theresults.Figure 4.6 shows the results for approximating the consumption function with and withoutregularization. The left panels show the results without regularization. The right panels show theresults with L2 regularization with penalization coefficient λ = 10−9. The solid curve in the toppanels shows the median of the Euler residuals squared defined in equation (4.3), and the shadedregions show the 10th and the 90th percentiles of the Euler residuals squared. The solid curvesin the bottom panels shows the median of the policy functions for capital, i.e., k′ (k; c(·; θ)) andthe shaded regions show the 10th and the 90th percentiles of the approximate policy functions forcapital.These results show that explicit regularization cannot lead to solutions that do not violate thetransversality condition. However, this regularization reduces the variation of the Euler residualsand policy functions for capital.Minimizing the Bellman residuals is not enough. Similarly, for the sequential models suchas linear asset pricing low residual errors can be misleading. As noted before the solutions can bewritten as:p(t) = pf (t) + ζβ−t.From the theoretical perspective, given that the dividends are positive, prices should be non-negative for every t. Therefore, negative values of ζ are not allowed and as shown in proposition6 the price based on the fundamentals has the lowest semi-norm for all non-negative values of ζ.However, when Xtrain = {t1, . . . , tN} contains small values, negative values of ζ can leak into theinterpolating solutions. It is important to note that they are still interpolating solutions and solvethe optimization problem accurately. This is because when tN is small the solutions with negativeζ have smaller semi-norms.47This is also called weight decay, and it is very easy to implement in PyTorch and TensorFlow.881.0 1.5 2.0 2.5capital(k)10 1010 910 810 7Euler residuals2: NN= c(k), without regularization1.0 1.5 2.0 2.5capital(k)10 1010 910 810 7Euler residuals2: NN= c(k), with regularization1.0 1.5 2.0 2.5capital(k)1.01.52.02.53.03.5k ′(k): NN= c(k), without regularizationk ′(k)45 degree line1.0 1.5 2.0 2.5capital(k)1.01.52.02.53.03.5k ′(k): NN= c(k), with regularizationk ′(k)45 degree lineFigure 4.6: The effect of the L2 explicit regularization on the solutions when the consumptionfunction c(k) is approximated with a deep neural network. The left panels show the results withoutregularization. The right panels show the results with L2 regularization with penalization coefficientλ = 10−9. The solid curves in the top panels show the median of the Euler residuals squared. Thesolid curves in the bottom panels show the median of the approximate policy functions for capital,i.e., k′ (k; c(·θ)). The shaded regions show the 10th and the 90th percentiles.I solve the minimization problem described in (3.12) for 100 times. Each time with a differentrandom initialization of the parameters of the deep neural network. I report the median, the 10th,and the 90th percentiles of the results.Figure 4.7 shows the results for the sequential asset pricing model for Xtrain = {0, 1, . . . , 14}.The parameters and the deep neural network is the same as before. The dashed curve in the leftpanel shows the price based on the fundamentals and the solid curve shows the median of theapproximate prices. The shaded region shows the 10th and the 90th percentiles of the approximate890 5 10 15 20Time(t)0.70.80.91.0Prices: p(t)p(t)pf(t)0 5 10 15 20Time(t)0.040.030.020.01Relative error: p(t)Figure 4.7: Comparison between the accurate and approximate solution using a deep neural networkfor the sequential linear asset pricing model for small tN . The dashed vertical lines separate theinterpolation from the extrapolation region. In the left panel the solid curve shows the median ofth approximate prices. The dashed curve shows the price based on the fundamentals. The solidcurve in the right panel shows the median of relative errors between the approximate price and theprice based on the fundamentals. The shaded regions show the 10th and the 90th percentiles ofthe relative errors.prices. The solid curve in the right panel shows the median of the relative errors. The shadedregion shows the 10th and the 90th percentiles of the relative errors.These results show the leakage of negative value of ζ into the interpolating solutions. Since ζis negative and non-negativity of p(·; θ) is built into H(Θ), the approximate solutions are alwaysbounded between 0 and pf (t).Figure 4.8 is the result of the same experiment with Xtrain = {0, 1, . . . , 9}. The black solid curve,denoted by pˆ(t), shows the median of the approximate price paths. The shaded region shows the10th and the 90th percentiles of the approximate price paths. The dashed curve, denoted by pf (t)shows the price based on the fundamentals. The solid curve in the right panel shows the median ofthe relative errors between the approximate prices and the price based on the fundamentals. Theshaded region shows the 10th and the 90th percentiles of the relative errors.Since the approximate prices solve the interpolation problem accurately, then each of them canbe written as:pˆ(t) = pf (t) + ζˆβ−t.By setting t = 0 one can find the corresponding ζˆ as:ζˆ = pˆ(0)− pf (0). (4.19)The solid blue line shows the median of pˆ(t) − ζˆβ−t. If each solution solves the interpolation900 5 10 15Time(t)0.70.80.91.0Prices: p(t)p(t)pf(t)p(t) t0 5 10 15Time(t)0.080.070.060.050.040.030.020.01Relative error: p(t)Figure 4.8: Analyzing the approximate solutions of the sequential linear asset pricing model forsmall tN. In this experiment Xtrain = {0, 1, . . . , 9}. The dashed vertical lines separate the inter-polation from the extrapolation region. In the left panel the black solid curve, denoted by pˆ(t),shows the median of the approximate prices. The dashed curve, denoted by pf (t), shows the pricebased on the fundamentals. The blue solid curve shows the median of pˆ(t)− ζˆβ−t. The right panelshows the difference between the approximate prices and the price based on the fundamentals. Theshaded regions show the 10th and the 90th percentiles.problem accurately, then pˆ(t) − ζˆβ−t must be very close to pf (t). The blue shaded region showsthe 10th and the 90th percentiles for pˆ(t)− ζˆβ−t.These results confirm that when tN is small (in this example tN = 9) the approximate solutionaccurately solves the interpolation problem. This is evident from observing that pf (t) ≈ pˆ(t)− ζˆβ−tvery accurately in Xtrain. The 10th and the 90th percentiles for pˆ(t)− ζˆβ−t are very close to eachother. This happens because the variations in the approximate prices are caused by very smallvariations in ζˆ. Moreover these results confirm that the leakage of negative values of ζ gets moresevere for smaller values of tN . However, the approximate solutions are still bounded between 0and pf (t). Therefore, the bias is always downward.4.4 ConclusionIn this chapter I extend the results of the previous chapter to recursive setups. In the case ofrecursive formulation of the neoclassical growth model, I establish that the implicit bias of deeplearning finds solutions that satisfy the transversality conditions. An accurate short- and medium-run solution is obtained even when the grid for capital is local to the initial condition and does notinclude the steady state. Moreover, utilizing economic a priori knowledge to transform the state-space one can achieve accurate solutions with outstanding generalization power. I also provide twoexamples, one in the context of recursive setups and one in sequential setups, to show minimizing91the Euler and Bellman residuals are not enough to obtain the optimal solutions. Moreover, I showhow economic knowledge combined with understanding of the implicit bias of deep learning helpus avoiding these solutions.92Chapter 5ConclusionThis thesis consists of three chapters providing methods for solving high-dimensional dynamic mod-els in economics.In chapter 2, I show how to exploit symmetry in dynamic programming problems and the so-lution of recursive competitive equilibria with a large (but finite) number of heterogeneous agentsusing deep learning. Symmetry, together with concentration of measure, allows me to alleviate thecurse of dimensionality. The method is easy to implement using standard, open-source softwarelibraries and fast to run on a standard laptop. The results open the door to many applications ofinterest, including macroeconomics, finance, international finance, industrial organization, interna-tional trade, spatial economies, and other fields.Dynamic models of forward-looking agents in economics require conditions for long-run behav-ior (e.g, transversality and no-bubble condition). These conditions rule out explosive solutions.In chapter 3, I show, in sequential setups, how deep learning approximations automatically sat-isfy these conditions without explicitly imposing them through calculating the steady state evenin models with steady-state multiplicity. What makes deep learning approximations special is itsinherent implicit bias toward functions with small derivatives that match the class of solutions weare interested in dynamic models. These results suggest that deep learning may let us calculateaccurate transition dynamics with high-dimensional state spaces without directly solving for theboundary conditions at infinity.In chapter 4, I extend the results of chapter 3 to recursive setups. In the case of recursive for-mulation of the neoclassical growth model, I establish that the implicit bias of deep learning findssolutions that satisfy the transversality conditions. An accurate short- and medium-run solutionis obtained even when the grid for capital is local to the initial condition and does not include thesteady state. Moreover, utilizing economic a priori knowledge to transform the state-space one canachieve accurate solutions with outstanding generalization power. I also provide two examples, onein the context of recursive setups and one in sequential setups, to show minimizing the Euler andBellman residuals are not enough to obtain the optimal solutions. Moreover, I show how economicknowledge combined with understanding of the implicit bias of deep learning help us avoiding thesesolutions.93BibliographyAdvani, M. S., A. M. Saxe, and H. Sompolinsky (2020). High-dimensional dynamics of generaliza-tion error in neural networks. Neural Networks 132, 428–446.Aguirregabiria, V. and A. Nevo (2013). Recent developments in empirical IO: Dynamic demandand dynamic games. In D. Acemoglu, M. Arellano, and E. Dekel (Eds.), Advances in Economicsand Econometrics: Tenth World Congress, Volume 3, pp. 53–122. Cambridge University Press.Aiyagari, S. R. (1994, 08). Uninsured Idiosyncratic Risk and Aggregate Saving. The QuarterlyJournal of Economics 109 (3), 659–684.Allen-Zhu, Z., Y. Li, and Z. Song (2019). A convergence theory for deep learning via over-parameterization. In International Conference on Machine Learning, pp. 242–252. PMLR.Anderson, E. W., E. R. McGrattan, L. P. Hansen, and T. J. Sargent (1996). Mechanics of formingand estimating dynamic linear economies. Handbook of Computational Economics 1, 171–252.Arora, S., N. Cohen, W. Hu, and Y. Luo (2019). Implicit regularization in deep matrix factorization.Advances in Neural Information Processing Systems 32.Arora, S., R. Ge, B. Neyshabur, and Y. Zhang (2018). Stronger generalization bounds for deep netsvia a compression approach. In J. Dy and A. Krause (Eds.), Proceedings of the 35th InternationalConference on Machine Learning, Volume 80, pp. 254–263.Athey, S. and G. W. Imbens (2019). Machine learning methods that economists should know about.Annual Review of Economics 11 (1), 685–725.Azinovic, M., L. Gaegauf, and S. Scheidegger (2022). Deep equilibrium nets. International Eco-nomic Review .Azinovic, M. and J. Zˇemlicˇka (2023). Economics-inspired neural networks with stabilizing homo-topies. Technical report, arXiv. org.Backus, D. K., P. J. Kehoe, and F. E. Kydland (1992). International Real Business Cycles. Journalof Political Economy 100 (4), 745–775.Barvinok, A. (2005). Math 710: Measure concentration. Lecture notes.Belkin, M. (2021). Fit without fear: remarkable mathematical phenomena of deep learning throughthe prism of interpolation. Acta Numerica 30, 203–248.94Belkin, M., D. Hsu, S. Ma, and S. Mandal (2019). Reconciling modern machine-learning practiceand the classical bias–variance trade-off. Proceedings of the National Academy of Sciences of theUnited States of America 116 (32), 15849–15854.Bellman, R. (1958). Dynamic Programming. Princeton University Press.Benhabib, J., J. Perla, and C. Tonetti (2021). Reconciling models of diffusion and innovation: Atheory of the productivity distribution and technology frontier. Econometrica (Forthcoming).Bilal, A. (2021). Solving Heterogeneous Agent Models with the Master Equation. Technical report,University of Chicago.Blanc, G., N. Gupta, G. Valiant, and P. Valiant (2020). Implicit regularization for deep neuralnetworks driven by an ornstein-uhlenbeck like process. Proceedings of Machine Learning Researchvol 125, 1–31.Bloem-Reddy, B. and Y. W. Teh (2020). Probabilistic symmetries and invariant neural networks.Journal of Machine Learning Research 21, 90–1.Boucheron, S., G. Lugosi, and P. Massart (2013). Concentration Inequalities: A NonasymptoticTheory of Independence. Oxford University Press.Brumm, J. and S. Scheidegger (2017). Using adaptive sparse grids to solve high-dimensional dy-namic models. Econometrica 85 (5), 1575–1612.Brunnermeier, M. K. (2017). Bubbles, pp. 1–8. London: Palgrave Macmillan UK.Buera, F. J. and R. E. Lucas (2018). Idea flows and economic growth. Annual Review of Eco-nomics 10 (1), 315–345.Calvano, E., G. Calzolari, V. Denicolo, and S. Pastorello (2020). Artificial intelligence, algorithmicpricing, and collusion. American Economic Review 110 (10), 3267–97.Carvalho, V. M. and A. Tahbaz-Salehi (2019). Production networks: A primer. Annual Review ofEconomics 11 (1), 635–663.Cheela, B., A. DeHon, J. Ferna´ndez-Villaverde, and A. Peri (2022). Programming fpgas for eco-nomics: An introduction to electrical engineering economics. Technical report, National Bureauof Economic Research.Chen, H., A. Didisheim, and S. Scheidegger (2021). Deep surrogates for finance: With an applicationto option pricing. Available at SSRN 3782722 .Childers, D., J. Ferna´ndez-Villaverde, J. Perla, C. Rackauckas, and P. Wu (2022). Differentiablestate space models and hamiltonian monte carlo estimation. NBER Working Paper .95Cybenko, G. (1989). Approximation by superpositions of a sigmoidal function. Mathematics ofControl, Signals and Systems 2 (4), 303–314.Damian, A., T. Ma, and J. D. Lee (2021). Label noise sgd provably prefers flat global minimizers.Advances in Neural Information Processing Systems 34, 27449–27461.Den Haan, W. J. (2010). Comparison of solutions to the incomplete markets model with aggregateuncertainty. Journal of Economic Dynamics and Control 34 (1), 4–27.Duarte, V. (2018). Machine learning for continuous-time finance. Technical report, Gies College ofBusiness.Duffy, J. and P. D. McNelis (2001). Approximating and simulating the stochastic growth model:Parameterized expectations, neural networks, and the genetic algorithm. Journal of EconomicDynamics and Control 25 (9), 1273–1303.Ebrahimi Kahou, M., J. Ferna´ndez-Villaverde, J. Perla, and A. Sood (2021). Exploiting symmetryin high-dimensional dynamic programming. Working Paper 28981, National Bureau of EconomicResearch.Ekeland, I. and J. A. Scheinkman (1986). Transversality conditions for some infinite horizon discretetime optimization problems. Mathematics of operations research 11 (2), 216–229.Ferna´ndez-Villaverde, J. and P. A. Guerro´n-Quintana (2021). Estimating dsge models: Recentadvances and future challenges. Annual Review of Economics 13 (1), 229–252.Ferna´ndez-Villaverde, J., J. F. Rubio-Ramı´rez, and F. Schorfheide (2016). Solution and estimationmethods for dsge models. In Handbook of macroeconomics, Volume 2, pp. 527–724. Elsevier.Ferna´ndez-Villaverde, J., S. Hurtado, and G. Nun˜o (2019). Financial frictions and the wealthdistribution. Working Paper 26302, National Bureau of Economic Research.Gentzkow, M., B. Kelly, and M. Taddy (2019). Text as data. Journal of Economic Literature 57 (3),535–74.Hartford, J., D. R. Graham, K. Leyton-Brown, and S. Ravanbakhsh (2018). Deep models ofinteractions across sets. In J. Dy and A. Krause (Eds.), Proceedings of the 35th InternationalConference on Machine Learning, Volume 80 of Proceedings of Machine Learning Research, pp.1909–1918. PMLR.Hartford, J. S., J. R. Wright, and K. Leyton-Brown (2016). Deep learning for predicting humanstrategic behavior. In Advances in Neural Information Processing Systems, pp. 2424–2432.Hornik, K. (1991). Approximation capabilities of multilayer feedforward networks. Neural Net-works 4 (2), 251–257.96Jovanovic, B. and R. W. Rosenthal (1988). Anonymous sequential games. Journal of MathematicalEconomics 17 (1), 77–87.Judd, K. L. (2002). The parametric path method: an alternative to fair–taylor and l–b–j for solvingperfect foresight models. Journal of Economic Dynamics and Control 26 (9-10), 1557–1583.Judd, K. L., L. Maliar, S. Maliar, and I. Tsener (2017). How to solve dynamic stochastic modelscomputing expectations just once. Quantitative Economics 8 (3), 851–893.Kalouptsidi, M. (2017). Detection and Impact of Industrial Subsidies: The Case of Chinese Ship-building. Review of Economic Studies 85 (2), 1111–1158.Kamihigashi, T. (2005). Necessity of the transversality condition for stochastic models withbounded or crra utility. Journal of Economic Dynamics and Control 29 (8), 1313–1329.Kaplan, G., B. Moll, and G. L. Violante (2018). Monetary policy according to hank. AmericanEconomic Review 108 (3), 697–743.Kase, H., L. Melosi, and M. Rottner (2022). Estimating nonlinear heterogeneous agents modelswith neural networks.Keane, M. P. and K. I. Wolpin (1994). The solution and estimation of discrete choice dynamic pro-gramming models by simulation and interpolation: Monte carlo evidence. Review of Economicsand Statistics 76 (4), 648–672.Krusell, P. and A. A. Smith, Jr (1998). Income and wealth heterogeneity in the macroeconomy.Journal of political Economy 106 (5), 867–896.Liang, T. and A. Rakhlin (2020). Just interpolate: Kernel “ridgeless” regression can generalize.The Annals of Statistics 48 (3).Ljungqvist, L. and T. J. Sargent (2018). Recursive Macroeconomic Theory (4 ed.). MIT Press.Lucas, R. E. and E. C. Prescott (1971). Investment under uncertainty. Econometrica, 659–681.Ma, C. and L. Ying (2021). The sobolev regularization effect of stochastic gradient descent. arXivpreprint arXiv:2105.13462 .Macdonald, I. G. (1998). Symmetric Functions and Hall Polynomials. Oxford University Press.Maennel, H., O. Bousquet, and S. Gelly (2018). Gradient descent quantizes relu network features.arXiv preprint arXiv:1803.08367 .Maliar, L., S. Maliar, and P. Winant (2021). Deep learning for solving dynamic economic models.Journal of Monetary Economics 122, 76–101.McKenzie, L. W. (1976). Turnpike theory. Econometrica, 841–865.97Mertens, T. M. and K. L. Judd (2018). Solving an incomplete markets model with a large cross-section of agents. Journal of Economic Dynamics and Control 91, 349–368.Nakkiran, P., G. Kaplun, Y. Bansal, T. Yang, B. Barak, and I. Sutskever (2019). Deep doubledescent: Where bigger models and more data hurt. arXiv preprint arXiv:1912.02292 .Narayanamurthy, S. M. and B. Ravindran (2008). On the hardness of finding symmetries in markovdecision processes. In Proceedings of the 25th International Conference on Machine Learning,pp. 688–695.Neyshabur, B., R. Tomioka, R. Salakhutdinov, and N. Srebro (2017). Geometry of optimizationand implicit regularization in deep learning. arXiv preprint arXiv:1705.03071 .Neyshabur, B., R. Tomioka, and N. Srebro (2014). In search of the real inductive bias: On the roleof implicit regularization in deep learning. arXiv preprint arXiv:1412.6614 .Oberfield, E. (2018). A theory of input-output architecture. Econometrica 86 (2), 559–589.Pakes, A. and P. McGuire (2001). Stochastic algorithms, symmetric markov perfect equilibrium,and the ‘curse’ of dimensionality. Econometrica 69 (5), 1261–1281.Peri, A. (2020). A hardware approach to value function iteration. Journal of Economic Dynamicsand Control 114, 103894.Powell, W. B. (2007). Approximate Dynamic Programming: Solving the Curses of Dimensionality,Volume 703. John Wiley & Sons.Prasad, A. (2018). An introduction to schur polynomials. arXiv preprint arXiv:1802.06073 .Prescott, E. C. and R. Mehra (1980). Recursive competitive equilibrium: The case of homogeneoushouseholds. Econometrica 48 (6), 1365–1379.Ravindran, B. and A. G. Barto (2001). Symmetries and Model Minimization in Markov DecisionProcesses. University of Massachusetts.Rust, J. (1997). Using randomization to break the curse of dimensionality. Econometrica 65 (3),487–516.Shen, Z., R. Zhang, M. Dell, B. C. G. Lee, J. Carlson, and W. Li (2021). Layoutparser: Aunified toolkit for deep learning based document image analysis. In International Conference onDocument Analysis and Recognition, pp. 131–146. Springer.Skiba, A. K. (1978). Optimal growth with a convex-concave production function. Econometrica,527–539.Smith, S. L., B. Dherin, D. G. Barrett, and S. De (2021). On the origin of implicit regularizationin stochastic gradient descent. arXiv preprint arXiv:2101.12176 .98Valaitis, V. and A. T. Villa (2021). A machine learning projection method for macro-finance models.Available at SSRN 3209934 .van der Pol, E., D. E. Worrall, H. van Hoof, F. A. Oliehoek, and M. Welling (2020). Mdp homomor-phic networks: Group symmetries in reinforcement learning. arXiv preprint arXiv:2006.16908 .Vershynin, R. (2018). High-Dimensional Probability: An Introduction with Applications in DataScience, Volume 47. Cambridge university press.Wagstaff, E., F. B. Fuchs, M. Engelcke, I. Posner, and M. Osborne (2019). On the limitations ofrepresenting functions on sets. arXiv preprint arXiv:1901.09006 .Yarotsky, D. (2018). Universal approximations of invariant maps by neural networks. arXiv preprintarXiv:1804.10306 .Zabrocki, M. (2015). Introduction to symmetric functions. unpublished work http://garsia. math.yorku. ca/ghana03/mainfile. pdf .Zaheer, M., S. Kottur, S. Ravanbakhsh, B. Poczos, R. R. Salakhutdinov, and A. J. Smola (2017).Deep sets. In Advances in Neural Information Processing Systems, pp. 3391–3401.Zhang, C., S. Bengio, M. Hardt, B. Recht, and O. Vinyals (2021). Understanding deep learning(still) requires rethinking generalization. Communications of the ACM 64 (3), 107–115.99Appendix AAppendix to Chapter 2A.1 Mathematics definitions and backgroundDefinition 7 (The permutation group and the permutation operator). Define:• A permutation group is a group G whose elements are permutations of a given set I andwhose group operation is the composition of permutations in G.• The symmetric group of a set I is the group of all permutations of set I.Definition 8 (Permutation invariance). A function f : RN → R is permutation invariant if, forall permutations π ∈ SN :f(πX) = f(X).Definition 9 (Permutation equivariance). A function f : RN → RN is permutation equivariant if,for all permutations π ∈ SN :f(πX) = πf(X).Proposition 7. Let X be a compact subset of RN and f : X ⊆ RN → R be a continuous permuta-tion invariant function under SN , i.e., for all X ∈ RN and all π ∈ SN :f(πX) = f(X).Then, there exist L ≤ N , and continuous functions ϱ : RL → R and φ : R→ RL, such thatf (X) = ϱ(1NN∑i=1φ(Xi)),where Xi is the ith element of X.Proof. See Wagstaff et al. (2019) and Zaheer et al. (2017). Q.E.D.A.2 Concentration of measureProposition 8 (Gaussian Poincare´ inequality). Suppose f : RN → R is a continuously differen-tiable function and z ∼ N (0N , IN ) is a standard Gaussian random vector. Then:Var[f(z)] ≤ E [∥∇f(z)∥2] ,100Moreover, if z ∼ N (0N , Σ) is a Gaussian random vector with mean zero and covariance matrix Σ,then:Var[f(z)] ≤ E[∇f⊤(z)Σ∇f(z)].Proof. See Boucheron et al. (2013), Theorem 3.20. Q.E.D.A.3 ProofsA.3.1 Proof of proposition 1Proof. Let π ∈ SN be a permutation matrix and Y ≡ πX. The Bellman equation can be writtenas:v(x, Y ) = maxu{r(x, u, Y ) + βE[v(x′, Y ′)]}(A.1)s.t. x′ = g(x, u) + σw + ηω (A.2)Y ′ = G(Y ) +ΩπW + ηωπ1N . (A.3)Assuming ∂g(x,u)∂u ̸= 0, the Euler equation can be written asru(x, u(x, Y ), Y)gu(x, u(x, Y )) + βE [rx(x′, u(x′, Y ′), Y ′)− ru(x′, u(x′, Y ′), Y ′)gx(x′, u(x′, Y ′))gu(x′, u(x′, Y ′))] = 0,where ru ≡ ∂r∂u , rx ≡ ∂r∂x , gx ≡ ∂g∂x , and gu ≡ ∂g∂u . Since r is permutation invariant with respect toX:ru(x, u(x, Y ), Y)= ru(x, u(x, Y ), X)rx(x, u(x, Y ), Y)= rx(x, u(x, Y ), X).Using the fact that G(Y ) = πG(X) and Ωπ = πΩ:Y ′ = π(G(X) +ΩW + ηω1N)= πX ′. (A.4)Defining u(x, πX) = u ◦ π(x,X) ≡ uπ(x,X), the Euler equation becomes:ru(x, uπ(x,X), X)gu(x, uπ(x,X)) + βE [rx(x′, uπ(x′, X ′), X ′)− ru(x′, uπ(x′, X ′), X ′)gx(x′, uπ(x′, X ′))gu(x′, uπ(x′, X ′))] = 0,where:x′ = g(x, u) + σw + ηω (A.5)X ′ = G(X) +ΩW + ηω1N . (A.6)101Since the solution of the Euler equation is unique, then:uπ(x,X) = u(x, πX) = u(x,X).Given the permutation invariant optimal policy, u, and reward function, r, the Bellman equationcan be written as:v(x, Y ) = r(x, u(x,X), X)+ βE[v(x′, Y ′)],where x′ and Y ′ are described by equations (A.2) and (A.3). Using equation (A.4) and definingv(x, πX) = v ◦ π(x,X) ≡ vπ(x,X), the Bellman equation can be written as:vπ(x,X) = r(x, u(x,X), X)+ βE[vπ(x′, X ′)],where x′ and X ′ are described by equations (A.5) and (A.6). By the contraction mapping theorem,the solution of the Bellman equation is unique. Therefore:vπ(x,X) = v(x, πX) = v(x,X).Q.E.D.A.3.2 Proof of proposition 3Proof. By the Chebyshev inequality:P(∣∣f(z1)− E [f(z)] ∣∣ ≥ ϵ) ≤ Var[f(z)]ϵ2.By the Gaussian Poincare´ inequality (proposition 8):Var[f(z)] ≤ E [∇f(z)⊤Σ∇f(z)] .Therefore:P(∣∣f(z1)− E [f(z)] ∣∣ ≥ ϵ) ≤ 1ϵ2E[∇f(z)⊤Σ∇f(z)].Since a covariance matrix is symmetric and positive semi-definite, there exists an orthogonal matrixU (i.e. U⊤ = U−1) and a diagonal matrix Λ such that:Σ = UΛU⊤,102where Λ is the diagonal matrix of the eigenvalues of Σ. Hence:∇f(z)⊤Σ∇f(z) =N∑i=1λi[U⊤∇f(z)]2i≤ maxi=1,..,N{λi}∥∇f(z)∥2 = ρ(Σ)∥∇f(z)∥2,where ρ(Σ) ≡ maxi=1,..,N{λi} is the spectral radius of Σ. Therefore,P(∣∣f(z1)− E [f(z)] ∣∣ ≥ ϵ) ≤ ρ(Σ)Cϵ21N.Q.E.D.A.3.3 Proof of proposition 4Proof. Since the price function p(X) is permutation invariant and uˆ is differentiable, the agentproblem can be written as:v(x,X) = maxu{p(X)x− γ2u2 + βE[v(x′, X ′)]}X ′ = {(1− δ)xˆ+ uˆ(xˆ, X) + σwˆ + ηω for all (xˆ, wˆ) ∈ (X,W )} .The first-order condition can be written as:−γu+ βE[∂v(x′, X ′)∂x′]= 0. (A.7)Given that the other agents’ law of motion is independent of the agent of interest’s choice, theenvelope condition is:∂v(x′, X ′)∂x′= P (X ′) + γ(1− δ)u′, (A.8)where u′ is the optimal policy in the next period. Combining the first-order condition (A.7) andthe envelope condition (A.8) yields:−γu(x,X) + βE [P (X ′) + γ(1− δ)u(x′, X ′)] = 0.In a symmetric equilibrium, all the agents use the same policy function, i.e., uˆ(., .) = u(., .), andthe laws of motion can be rewritten as:x′ = (1− δ)x+ u+ σw + ηωX ′ = {(1− δ)xˆ+ u(xˆ, X) + σwˆ + ηω for all (xˆ, wˆ) ∈ (X,W )} .Q.E.D.103A.4 LQ FormulationsThis section solves the special case of the LQ model. The purpose is to demonstrate the symmetryand concentration of measure when the closed-form solution of the problem is known.A.4.1 Proof of proposition 5: linear-quadratic Gaussian controlWith an LQ Gaussian model, the policy will be affine transformation of X . Therefore, uˆ for otherfirms can be written as:uˆ(Xi, X) = H0 +H2Xi +H1NN∑i=1Xi i = 1, ..., N,for some undetermined H0, H1, and H2. As shown in proposition 4, in a permutation invariantsymmetric equilibrium, u(xˆ, X) does not depend on xˆ and, hence, H2 = 0. The symmetric con-stants, which I arbitrarily define as H1N for all of the X vector, are the only ones that can fulfillpermutation invariance with respect to the π permutations defined in equation (2.26). The law ofmotion for x⃗ is then:x⃗ ′ = Ax⃗+Bu+ Cw⃗where A, B, and C are defined in equations (2.27), (2.28), and (2.29). The reward given policy uof the firm of interest at state x⃗ can be written as:−x⃗⊤Rx⃗− u⊤Qu,where R and Q are defined in equations (2.30) and (2.31).For a given pair of (H0, H1), the solution of this LQ problem has the form:v⃗(x⃗) = −x⃗⊤Px⃗− du⃗(x⃗) = −Fx⃗,where P solves the following discounted algebraic matrix Riccati equation:P = R+ βA⊤PA− β2A⊤PB(Q+ βB⊤PB)−1B⊤PA, (A.9)d can be found via:d =β1− β trace(PCC⊤), (A.10)104and F satisfies:F = β(Q+ βB⊤PB)−1B⊤PA. (A.11)Ljungqvist and Sargent (2018, ch. 5) derive equations (A.9), (A.10), and (A.11).A.4.2 Proof of proposition 5: permutation invarianceNow, I illustrate the permutation invariance of the optimal policy and value function of the LQmodel described in proposition 5. Define y⃗ ≡ πx⃗. The value function associated with y⃗ in equation(2.34) can be written as:v(y⃗) = −y⃗⊤P y⃗ − d = −x⃗⊤(π⊤Pπ)x⃗− d,where P solves the discounted algebraic Riccati equation:P = R+ βA⊤PA− β2A⊤PB(Q+ βB⊤PB)−1B⊤PA. (A.12)Sandwiching both sides of equation (A.12) by π⊤ and π and using the fact that R is permutationinvariant (i.e., π⊤Rπ = R) yields:π⊤Pπ = R+ βπ⊤A⊤PAπ − β2π⊤A⊤PB(Q+ βB⊤PB)−1B⊤PAπ.Since the deterministic part of the laws of motion A is permutation equivariant (i.e., Aπ = πA)and the vector B remains unchanged under a permutation (i.e., πB = B), the above equation canbe rewritten as:π⊤Pπ = R+ βA⊤(π⊤Pπ)A− β2A⊤(π⊤Pπ)B[Q+ βB⊤(π⊤Pπ)B]−1B⊤(π⊤Pπ)A.Notice that π⊤Pπ solves the same discounted algebraic Riccati equation (A.12) as P . Since thesolution of equation (A.12) is unique:π⊤Pπ = P.The constant term d can be written asd = β(1− β)−1trace(PπCC⊤π⊤)= β(1− β)−1trace(π⊤PπCC⊤)= β(1− β)−1trace(PCC⊤).105As shown above, d remains unchanged under permutation, therefore:v(y⃗) = v(πx⃗) = v(x⃗).The optimal policy u⃗(y⃗) can be written asu(y⃗) = −β(Q+ βB⊤PB)−1B⊤PAy⃗ = −β(Q+ βB⊤PB)−1B⊤(π⊤Pπ)Ax⃗.Since (π⊤Pπ) = P , then:u(y⃗) = u(πx⃗) = u(x⃗).Structure of the P quadratic form: when solving for proposition 5, one can further find thatthe permutation invariant matrix P is of the formP ≡ p1 p2p3N 1⊤Np2 0p4N 1⊤Np3N 1Np4N 1Np5N21N1⊤N (A.13)for some p1, . . . p5. The lower block of this matrix isp5N21N1⊤N (the N ×N matrix of 1s is then theouter-product 1N1⊤N .) This shows the symmetry with respect to the i = 2, . . . N states in x⃗ due tothe block structure similar to that in equation (2.26), and the magnitude of their interaction (i.e.,the lower block decreases at rate N2 rather than N).A.4.3 Proof of proposition 5: concentration of measureThe optimal policy for x⃗ ′ can be written as:u(x⃗ ′) = −Fx⃗ ′ = −F (Ax⃗+Bu+ Cw⃗).The optimal policy defined in proposition 5 for a given H0 and H1 is u(x⃗) = H0 +H1N∑Ni=1Xi =H0 + H1E [x], and, hence, a bounded function in N according to definition 3. For any finite H1(including the equilibrium value), the gradient of u(x⃗ ′) with respect to the idiosyncratic shocks ofall other firms W-1 ≡[W2 . . . WN]becomes:∇W-1u(x⃗ ′) =σH1N1N−1.Therefore, the square of the norm of the gradient of the policy can be bounded by:E[∥∇W-1u(x⃗ ′)∥2] = σ2H21(N − 1N2 ) ≤ σ2H21N − 1 .106Using proposition 3 and conditioning on w, ω, and x⃗:P(∣∣u(x⃗ ′1)− E [u(x⃗ ′) ∣∣ w,ω, x⃗] ∣∣ ≥ ϵ) ≤ σ2H21ϵ21N − 1 . (A.14)The value function can be written as:v(x⃗ ′) = −x⃗ ′⊤Px⃗ ′ − d. (A.15)Defining X ′-1 ≡[X ′2 . . . X ′N]⊤and given the symmetry in equations (A.13) and (2.36), using thechain rule and additive structure of the idiosyncratic shocks, the gradient of the value function is:∇W-1v(x⃗ ′) = −2σ[p3 + p4x′N+p5x′N2]1N−1 − 2p5σN21N−11⊤N−1X′-1,and then:∥∇W-1v(x⃗ ′)∥2 =[2σ(p3 + p4x′ +p5x′N)]2N − 1N2+ 8σ2p5(p3 + p4x′ +p5x′N)1⊤N−1X′-1N − 1N3+(2σp5)2X ′-1⊤1N−11⊤N−1X′-1N − 1N4.Since conditional on (w,ω, x⃗), x′ is not a random variable and X ′-1 is a normal random vector withconditional mean E [X ′-1 | w,ω, x⃗] = X-1 +(u(X) + ηω)1N−1 one obtains:E[∥∇W-1v(x⃗ ′)∥2 ∣∣ w,ω, x⃗] =(N − 1)2N2 [2σ(p3 + p4x′ + p5x′N )]2 1N − 1+8(N − 1)3N3σ2p5[p3 + p4x′ +p5x′N](1⊤N−1X-1N − 1 + u(X) + ηω) 1N − 1+N − 1N4(2σp5)2E[X ′-1⊤1N−11⊤N−1X′-1∣∣∣ w,ω, x⃗]Considering X ′-1 is a normal random vector with conditional covariance cov (X ′-1 | w,ω, x⃗) =σ2IN−148:N − 1N4E[X ′-1⊤1N−11⊤N−1X′-1∣∣∣ w,ω, x⃗] = [(N − 1)3N4σ2 +(N − 1)4N4(1⊤N−1X-1N − 1 + u(X) + ηω)2] 1N − 1Since1⊤N−1X-1N−1 and u(X) are bounded functions in N (definition 3), there exists K1 such that:∣∣1⊤N−1X-1N − 1 + u(X) + ηω∣∣ ≤ K1.Note that |p5| and∣∣p3 + p4x′ + p5x′N ∣∣ are both bounded, and (N−1)3N4 σ2 ≤ σ2 (N−1)4N4 . Therefore, by48If z⃗ is a random vector with mean m⃗ and covariance matrix Σ, then, for any matrix Λ, E[z⃗⊤Λz⃗]= trace(ΛΣ) +m⃗⊤Λm⃗.107the Cauchy-Schwarz inequality, there exists a positive Cv such thatE[∥∇W-1v(x⃗ ′)∥2 ∣∣ w,ω, x⃗] ≤ σ2C2vN − 1 .Hence, by proposition 3:P(∣∣v(x⃗ ′1)− E [v(x⃗ ′) ∣∣ w,ω, x⃗] ∣∣ ≥ ϵ) ≤ σ2C2vϵ21N − 1 . (A.16)The inequalities in (A.14) and (A.16) show that even with a single draw, one can calculate theexpectation as N increases. A high-dimensional state space is a help rather than a hindrance whencalculating expectations (as long as there is sufficient independence in the states) because of theconcentration of measure phenomenon. Inequality (2.38) shows that, with a single draw ofW-1, oneknows the precise distribution of the deviation from the expectation, and with a large enough N , itconverges to 0. One could take multiple draws if required. However, since Monte Carlo integrationconverges very slowly in general, the gains in extra accuracy will be minor even if one increases thenumber of draws substantially.The Euler residuals corresponding to a Monte Carlo draw of W is defined as:ε(X,u) =γu(X)− βEw,ω[P (X ′) + γ(1− δ)u(X ′)]where Ew,ω [·] corresponds to expectation over w and ω. Substituting for a linear price and linearpolicy u(X ′) = H0 + H1N X1 +H1N∑Ni=2X′iε(X,u) =γu(X)− β[α0 − α1NN∑i=1(Xi + u(X))]− βγ(1− δ)[H0 + H1NN∑i=1(Xi + u(X))]+βσ[α1 − γ(1− δ)H1] 1NN∑i=2WiFrom the Euler equation (2.21), the first three terms sum up to zero, therefore:ε(X,u) = βσ[α1 − γ(1− δ)H1] 1NN∑i=2Wi.Utilizing the fact that Wi ∼ N (0, 1), the Euler residual is a Gaussian random variable:ε(X,u) ∼ N(0,N − 1N2β2σ2[α1 − γ(1− δ)H1]2).For a linear policy, the error in forecasting the policy in the next period is:u(x⃗ ′1)− E [u(x⃗ ′) ∣∣ w,ω, x⃗] = H1σNN∑i=2Wi,108therefore, it is a Gaussian random variable:u(x⃗ ′1)− E [u(x⃗ ′) ∣∣ w,ω, x⃗] ∼ N(0, N − 1N2H21σ2).A.4.4 Representation of the v functionIf it is known that the LQ problem satisfies certainty equivalence and the value function is quadraticand permutation invariant, v can be represented as:v(x,X) = −(p1 + d+ 2p2x+ 2p3∑Ni=1XiN+ 2p4x∑Ni=1XiN+ p5(∑Ni=1XiN)2)= ρv(x,1NN∑i=1ϕv(Xi))In other words, ϕv : R→ R (i.e., L = 1) is the identity function ϕv(X) =[X], and ρv : R×R→ Rsuch that:ρv(x, y) = −(p1 + d+ 2p2x+ 2p3y + 2p4xy + p5y2).109Appendix BAppendix to Chapter 3B.1 Sequential linear asset pricingB.1.1 Learning the growth rate0.0195 0.0200 0.0205 0.0210 0.0215 0.0220g0.000.050.100.150.200.250.300.350.40g approximation: gFigure B.1: The distribution of the approximated growth rate of the dividends in the sequentiallinear asset pricing model. The dashed vertical line shows the actual value of g.Figure B.1 shows the distribution for the approximated growth rate for the sequential linearasset pricing model. The approximated growth rate is defined as:gˆ ≡ eϕ − 1,This figure shows the approximate growth rate for 100 seeds, the red vertical line shows the truegrowth rate, g = 0.02. The approximation is slightly biased to the right. However, given theoptimization problem is extremely non-convex, the accuracy of approximation is impressive. Thisresults shows that the algorithm can separate the growth path from the stationary solution.110B.1.2 Misspecification of growth in pricesIn this experiment I use a linear function to approximate the solution of the sequential linear assetpricing model with growing dividendspˆ(t; θ) = tNN(t; θ1) + ϕ,where θ ≡ {ϕ, θ1}, NN(·; θ1) is a deep neural network, and ϕ is a parameter that needs to befound in the optimization process. Figure B.2 shows the results for sequential linear asset pricing0 10 20 30 40 50Time(t)0.751.001.251.501.752.002.252.502.75Prices: p(t)p(t)pf(t)0 10 20 30 40 50Time(t)0.160.140.120.100.080.060.040.020.00Relative error: p(t)1.1751.200Figure B.2: Comparison between the price based on the fundamentals and the price approximatedby a deep neural network for the sequential linear asset pricing model with growing dividends(g = 0.02) in the presence of functional misspecification of the growth. The solid curve in the leftpanel shows the median of the approximate price paths, the dashed curve shows the price basedon the fundamentals. The solid curve in the right panel shows the median of the relative errors.The shaded regions show the 10th and the 90th percentiles. The dashed vertical lines separate theinterpolation from the extrapolation region.model with growing dividends (i.e., c = 0, g = 0.02) and a misspecification of the functional formof the growth. I use the same neural network utilized in the correctly specified form. The onlydifference is the linear term. In the left panel, the solid curve, denoted by pˆ(t), shows the medianof the approximate price paths, the dashed curve, denoted by pf (t) shows the solution based on thefundamentals. The shaded region shows the 10th and the 90th percentiles of the approximate pricepaths. The right panel shows the median of the relative errors. The shaded region shows the 10thand the 90th percentiles of the relative errors. The dashed vertical lines separate the interpolationfrom the extrapolation region.These results show that even in the presence of misspecification the long-run errors do not impairthe accuracy of short- and medium-run dynamics (at most 2% relative error after 10 periods).However, the misspecification can reduce the generalization power of the solution.111B.2 Sequential neoclassical growthB.2.1 Zero as a fixed point for capitalAs discussed before k = 0 is a repulsive fixed point for the neoclassical growth model. In experimentI investigate whether the approximate solutions utilizing deep neural networks pick a capital paththat converges to the repulsive fixed point due to numerical errors. Figure B.3 shows the results0 10 20 30 40 50Time(t)10 410 310 210 1100Capital: k(t)k(t)k(t)0 10 20 30 40 50Time(t)0.0040.0020.0000.0020.0040.006Relative error: k(t)1.752.002.25Figure B.3: Comparison between the value function iteration and approximate solution using adeep neural network for the sequential neoclassical growth model with small levels of initial capitalk0 = 0.01. The solid curve, in the left panel, shows the approximate capital path and the dashedcurve shows the solution obtained by the value function iteration method. The right panel showsthe relative errors for the capital paths. The dashed vertical lines separate the interpolation fromthe extrapolation region.for the sequential neoclassical growth model with small level of initial capital. In this experimentI use the same parameters and deep neural network as the sequential neoclassical growth model,except the initial condition for capital. I use k0 = 10−2. In the left panel the solid curve shows theapproximate capital path and the dashed curve shows the solution obtained by the value functioniteration method. The right panel shows the relative errors between the approximate capital pathand the solution obtained by the value function iteration method. The dashed vertical lines separatethe interpolation from the extrapolation region.This result shows that even for small levels of initial capital the solutions do not converge tok = 0. Therefore, the solutions can detect that k0 is a repulsive fixed point. Moreover, the short-and medium-run solutions are accurate and are not impaired by the long-run errors (at most 0.6%).112B.2.2 Solutions violating the transversality condition: initial capital above thesteady stateThe blue curves in figure B.4 show a set of paths for capital, consumption and marginal utility ofconsumption (shadow prices), denoted by k˜(t), c˜(t), and u′(c˜(t)), that satisfy the Euler equationand feasibility condition, and violate the transversality condition.49 The black curves, denoted byk(t), c(t), and u′(c(t)), show the optimal paths for capital, consumption and marginal utility ofconsumption. These paths satisfy the Euler equation, feasibility condition and the transversalitycondition. The steady states for capital and consumption in the optimal solution are denoted by k∗and c∗. As evident in figure B.4 the capital path that violate the transversality condition has higher0 50 100Time(t)51015202530Capitalk(t)k(t)k *kmax0 50 100Time(t)0.00.51.01.52.02.53.0Consumptionc(t)c(t)c *0 50 100Time(t)100101102103104105106Marginal utility of consumptionu′(c(t))u′(c(t))u′(c * )Figure B.4: Comparison between the optimal solution and the solutions that violate the transver-sality condition for k0 > k∗. The blue curves denoted by k˜(t), c˜(t), and u′(c˜(t)) show a set ofcapital, consumption, and marginal utility of consumption paths that violate the transversalitycondition. The black curves denoted by k(t), c(t), and u′(c(t)) show the capital, consumption, andmarginal utility of consumption paths for the optimal solution. The steady states for capital anconsumption are denoted by k∗ and c∗.derivatives over its domain. More formally, let k˜(t) be a capital path violating the transversalitycondition and k(t) be the optimal solution, then in a compact space of the form [0, T ]:∫ T0∣∣∣∣∣dk˜dt∣∣∣∣∣2dt >∫ T′∣∣∣∣dkdt∣∣∣∣2 dt.Therefore, by assumption 1:∥k∥S < ∥k˜∥S .49The parameters I use are k0 = 10, β = 0.9, α = 0.33, and δ = 0.1.113B.2.3 An alternative way of approximating the optimal solutionAnother approach to solve the sequential neoclassical growth model is to simultaneously approx-imate consumption and capital functions. In this case I pick a space of over-parameterized func-tions H(Θ), and a grid Xtrain = {t1, . . . , tN} and find the the capital and consumption function[k(·; θ), c(·; θ)] ∈ H(Θ) by solving the following optimization problemminθ∈Θ[1|Xtrain|∑t∈Xtrain(β[z(t+ 1)1−αf ′(k(t+ 1; θ)) + 1− δ]− u′(c(t; θ))u′(c(t+ 1; θ)))2+1|Xtrain|∑t∈Xtrain(z(t)1−αf(k(t; θ)) + (1− δ)k(t; θ)− c(t; θ)− k(t+ 1; θ))2 + (k(0; θ)− k0)2 ],where z(t) is evaluated by the law of motion for zz(t) = (1 + g)tz0 for t ∈ Xtrain.Non-negativity of capital and consumption can be built into H(Θ). Here I omit the results becausethey are identical to the results illustrated in section 3.3.2.B.2.4 Being far from the steady stateFigure B.5 shows the results for the sequential neoclassical growth model with Xtrain = {0, 1, . . . , 4}.The dashed vertical lines separate the interpolation region from the extrapolation region. The solidcurve in the top-left panel shows the median of the approximate capital paths, the dashed curveshows the capital path obtained by the value function iteration method, and the shaded regionsshows the 10th and the 90th percentiles of the approximate capital paths. The solid curve in thetop-right panel shows the median of the relative errors between the approximate capital pathsand the capital path obtained by the value function iteration method, the shaded region showsthe 10th and the 90th percentiles of the relative errors. The solid curve in the bottom-left panelshows the median of the approximate consumption paths, the dashed curve shows the consumptionpath obtained by the value function iteration method, and the shaded region shows the 10th andthe 90th percentiles of the approximate consumption paths. The solid curve in the bottom-rightpanel shows the median of the relative errors between the approximate consumption paths and theconsumption path obtained by the value function iteration method, the shaded region shows the10th and the 90th percentiles of the relative errors.Theses results show that the approximate solutions are robust to using short time horizons inXtrain. The long-run errors do not impair the accuracy of short-run dynamics (less than 1% relativeerrors in capital in the first three periods).1140 5 10 15Time(t)0.81.01.21.41.61.82.0Capital: k(t)k(t)k(t)0 5 10 15Time(t)0.1750.1500.1250.1000.0750.0500.0250.000Relative error: k(t)0 5 10 15Time(t)0.50.60.70.80.91.01.1Consumption: c(t)c(t)c(t)0 5 10 15Time(t)0.030.020.010.000.010.02Relative error: c(t)1.5251.5501.5750.920.94Figure B.5: Comparison between the value function iteration and approximate solution using a deepneural network for the sequential neoclassical growth model with a short time horizon (i.e., Xtrain ={0, 1, . . . , 4}). The solid curves in the left panels show the median of the approximate capital andconsumption paths. The dashed curves show the capital and consumption paths obtained by thevalue function iteration method. The solid curves in the right panels show the median of the relativeerrors between the approximate solutions and solutions obtained by the value function iteration.The shaded regions show the 10th and the 90th percentiles. The dashed vertical lines separate theinterpolation from the extrapolation region.B.2.5 Misspecification of growth in capital and consumption pathsIn this experiment I use a different functional form to approximate the solution of the sequentialneoclassical growth model with growing total factor productivity:kˆ(t; θ) = tNN(t; θ1) + ϕ,115where θ ≡ {ϕ, θ1}, NN(·; θ1) is a deep neural network, and ϕ is a parameter that needs to be foundin the optimization process.0 10 20 30 40 50Time(t)1234Capital: k(t)k(t)k(t)0 10 20 30 40 50Time(t)0.050.040.030.020.010.00Relative error: k(t)0 10 20 30 40 50Time(t)0.51.01.52.02.5Consumption: c(t)c(t)c(t)0 10 20 30 40 50Time(t)0.0030.0020.0010.0000.001Relative error: c(t)1.851.901.1251.150Figure B.6: Comparison between the value function iteration and approximate solution using a deepneural network for the sequential neoclassical growth model with growth in total factor productivityin the presence of functional misspecification of the growth. The solid curves in the left panels showthe median of the approximate capital and consumption paths. The dashed curves show the capitaland consumption paths obtained by the value function iteration method. The solid curves in theright panels show the median of the relative errors between the approximate solutions and solutionsobtained by the value function iteration method. The shaded regions show the 10th and the 90thpercentiles. The dashed vertical lines separate the interpolation from the extrapolation region.Figure B.6 shows the results for the sequential neoclassical growth model with non-stationarytotal factor productivity (i.e., g = 0.02) in the presence of functional misspecification of the growth.I use the same deep neural network as before for NN(·; θ1). The only difference is the linear term.116The solid curve in the top-left panel shows the median of the approximate capital paths and thedashed curve shows the capital paths obtained by the value function iteration method. The shadedregion shows the 10th and the 90th percentiles of the approximate capital paths. The solid curve inthe top-right panel shows the median of the relative errors between the approximate capital pathsand the capital path obtained by the value function iteration method. The shaded region showsthe 10th and the 90th percentiles of the relative errors. The solid curve in the bottom-left panelshows the median of the approximate consumption paths, the dashed curve shows the consumptionpath obtained by the value function iteration method, and the shaded region shows the 10th andthe 90th percentiles of the approximate consumption paths. The solid curve in the bottom-rightpanel shows the median of the relative errors between the approximate consumption paths and theconsumption path obtained by the value function iteration method. The shaded region shows the10th and the 90th percentiles of the relative errors.These results show that the long-run errors do not impair the accuracy of the short- andmedium-run dynamics even in the presence of functional misspecification of growth in total factorproductivity. Moreover, the approximate solutions generalize well in the extrapolation region (lessthan 5% relative errors after 50 periods).B.2.6 Analysis of the approximate solution in the vicinity of the critical pointFigure B.7 shows the capital and consumption paths for a grid of initial conditions k0 ∈ [0.5, 4] forthe sequential neoclassical growth model with the convex-concave production function. The toppanel shows the capital paths and the bottom panel shows the consumption paths. The dashedhorizontal lines show the steady states of capital k∗1 and k∗2 and the corresponding steady states forconsumption c∗1 and c∗2. The red trajectories show the approximate solutions near the critical point.The non-monotonicity of the red consumption paths (or equivalently the inflection points in redcapital paths) indicates that these approximate solutions are not correct. However, the algorithmdetects there is a critical point in the space of initial levels of capital around k ≈ 2.5. Due tothe discontinuity in the derivative of the production function it is cumbersome to confirm whetheralgorithm finds the right critical point and it is beyond the scope of this work.1170 10 20 30 40 50Time(t)0.51.01.52.02.53.03.54.0Capital: k(t)k *1k *20 10 20 30 40 50Time(t)0.30.40.50.60.70.8Consumption: c(t)c *1c *2Figure B.7: The approximate capital and consumption paths for the sequential neoclassical growthmodel with convex-concave production function. The grid for the initial condition of capital belongsto [0.5, 4]. The top panel shows the capital paths and the bottom panel shows the consumptionpaths. The red trajectories show the capital and consumption paths in the vicinity of the criticalpoint. The dashed horizontal lines show the steady states of capital k∗1 and k∗2 and their correspond-ing steady states for consumption c∗1 and c∗2. The dashed vertical line separates the interpolationfrom the extrapolation region.118Appendix CAppendix to Chapter 4C.1 Robustness of the solution to the initial condition of capitalAs stated in the recursive formulation of the neoclassical growth problem, the optimality of thesolution requires that all the capital and consumption paths generated by the policy function tosatisfy the transversality condition.Figure C.1 shows the approximate capital and consumption paths and the relative errors for arange of initial conditions of capital (for the case of g = 0 and z0 = 1). The solid curves in thetop-left panel, denoted by kˆ(t), show the approximate capital paths from three initial conditions forcapital outside of the interpolation region, k0 ∈ {0.5, 3.25, 4.0}. The dashed curves, denoted by k(t),show the capital paths obtained by the value function iteration method. The gray rectangle showsthe interpolation region. The top-right panel shows the full range of the relative errors between theapproximate capital paths and the capital paths obtained by the value function iteration method for70 initial levels of capital in [0.4, 4]. The solid curves, denoted by cˆ(t), in the bottom-left panel showsthe approximate consumption paths. The dashed curves, denoted by c(t), show the consumptionpaths obtained by the value function iteration method. The bottom-right panel shows the fullrange of the relative errors between the approximate consumption paths and the consumptionpaths obtained via the value function iteration method for 70 initial levels of capital in [0.4, 4].These results show that the policy function for capital satisfies the transversality condition inthe interpolation and the extrapolation regions.Figure C.2 shows the result of the recursive neoclassical growth problem for a range of initialconditions of capital in the presence of non-stationary total factor productivity (i.e., g = 0.02).The solid curves in the top-left panel, denoted by kˆ(t), show the approximate capital paths fromthree initial conditions for capital k0 ∈ {0.5, 3.25, 4.0} and z0 = 1. The dashed curves, denoted byk(t), show the capital paths obtained by the value function iteration method. The top-right panelshows the full range of the relative errors between approximate capital paths and the capital pathobtained by the value function iteration method for 70 initial levels of capital k0 ∈ [0.4, 4]. The solidcurves in the bottom-left panel, denoted by cˆ(t), show the approximate consumption paths fromthree initial conditions for capital k0 ∈ {0.5, 3.25, 4.0} and z0 = 1. The bottom-right panel showsthe full range of the relative errors between the approximate consumption paths and consumptionpath obtained by the value function iteration method for 70 initial levels of capital k0 ∈ [0.4, 4].These results show that even in the presence of non-stationary, the approximate policy function1190 10 20 30 40 50Time(t)0.51.01.52.02.53.03.54.0Capital: k(t)k(t)k(t)Xtrain0 10 20 30 40 50Time(t)0.000.010.020.030.04Relative error: k(t)0 10 20 30 40 50Time(t)0.40.60.81.01.21.41.6Consumption: c(t)c(t)c(c)0 10 20 30 40 50Time(t)0.060.050.040.030.020.010.000.010.02Relative error: c(t)2.12.21.1001.125Figure C.1: Comparison between the value function iteration and approximate solution using adeep neural network for the recursive neoclassical growth model for a wide range of initial levelsof capital. The left panels show the approximate capital and consumption paths from three initiallevels conditions outside of Xtrain, k0 ∈ {0.5, 3.25, 4.0}. The dashed curves, denoted by k(t) andc(t), show the capital and consumption paths obtained by the value function iteration method.The right panels show the full range of the relative errors for capital and consumption paths for 70initial conditions of capital in [0.4, 4].for capital satisfies the transversality condition inside and outside of the interpolation regions.1200 10 20 30 40 50Time(t)1234Capital: k(t)k(t)k(t)0 10 20 30 40 50Time(t)0.050.040.030.020.010.00Relative error: k(t)0 10 20 30 40 50Time(t)0.51.01.52.02.5Consumption: c(t)c(t)c(c)0 10 20 30 40 50Time(t)0.0020.0000.0020.0040.006Relative error: c(t)2.22.31.251.30Figure C.2: Comparison between the value function iteration and approximate solution using a deepneural network for the recursive neoclassical growth model with growth in total factor productivity(i.e., g = 0.02) for a wide range of initial levels of capital. The left panels show the approximatecapital and consumption paths from three initial levels conditions k0 ∈ {0.5, 3.25, 4.0}. The dashedcurves, denoted by k(t) and c(t), show the capital and consumption paths obtained by the valuefunction iteration method. The right panels show the full range of the relative errors for capitaland consumption paths for 70 initial conditions of capital in [0.4, 4].121
Thesis/Dissertation
eng
Economics
Vancouver : University of British Columbia Library
Attribution-NonCommercial-NoDerivatives 4.0 International
Graduate
Perla, Jesse
Essays on theory and computation in economics
Text