Reinforcement learning in complex environments with locally trained naïve agents
department or school: Applied Science, Faculty of
department or school: Engineering, School of (Okanagan)
data provider: DSpace
degree campus: UBCO
creator: Gupta, Kashish
issued: 2021-11-12T23:44:11Z
issued: 2021
related degree: Doctor of Philosophy - PhD
degree grantor: University of British Columbia
description: Reinforcement learning has long been advertised as the one with the capability to intelligently mimic and understand human learning and behavior. While the upshot of the field's advances is not underrated, its applicability and extension to large, complex and highly dynamic environments remain inefficient, inaccurate or unsolved. As the complexity of objectives increase, trend in reinforcement learning research is to tackle it with more computational power and training samples. The inspiration for the proposed methodology is derived from human learning, where increasingly complex objectives are learned with limited computational power by re-purposing previously learned skills. An intuitive and elegant concept is presented which exploits abstract symmetries present in training environments to learn a naïve agent and bypass some of the core challenges of reinforcement learning training. The naïve agent, trained in a local environment, can then be used in numerous ways to improve training in high-dimensional state-space environments. The proposed solution is incorporated with heuristic-based planning, learning from demonstration and state-space abstraction methods to present the efficacy and ease of adaptability of the proposed concept on a range of domains. The proposed method provides a structured approach to the training process and improves the trained agent's generalization capabilities and training sample efficiency. While the presented concept benefits from additional information about the state-space structure, it is notably different from traditional reinforcement learning approaches which augments the training process with expensive domain-specific knowledge to improve sample efficiency. Experiments and analyses are presented for challenging navigation and control environments, solved with augmented off-policy family of reinforcement learning methods.
aggregated CHO: https://circle.library.ubc.ca/rest/handle/2429/80187
note: Reinforcement Learning in ComplexEnvironments With Locally TrainedNa¨ıve AgentsbyKashish GuptaB.Tech., Vellore Institute of Technology, Chennai, India, 2016A THESIS SUBMITTED IN PARTIAL FULFILLMENT OFTHE REQUIREMENTS FOR THE DEGREE OFDOCTOR OF PHILOSOPHYinTHE COLLEGE OF GRADUATE STUDIES(Electrical Engineering)THE UNIVERSITY OF BRITISH COLUMBIA(Okanagan)November 2021© Kashish Gupta, 2021The following individuals certify that they have read, and recommendto the College of Graduate Studies for acceptance, a thesis/dissertation en-titled:Reinforcement Learning in Complex Environments With Lo-cally Trained Na¨ıve Agentssubmitted by Kashish Gupta in partial fulfilment of the requirements ofthe degree of Doctor of PhilosophyDr. Homayoun Najjaran, School of EngineeringSupervisorDr. Abbas Milani, School of EngineeringSupervisory Committee MemberDr. Chen Feng, School of EngineeringSupervisory Committee MemberDr. Solomon Tesfamariam, School of EngineeringUniversity ExaminerDr. William Melek, University of WaterlooExternal ExamineriiAbstractReinforcement learning has long been advertised as the one with thecapability to intelligently mimic and understand human learning and be-havior. While the upshot of the field’s advances is not underrated, its appli-cability and extension to large, complex and highly dynamic environmentsremain inefficient, inaccurate or unsolved. As the complexity of objectivesincrease, trend in reinforcement learning research is to tackle it with morecomputational power and training samples. The inspiration for the proposedmethodology is derived from human learning, where increasingly complexobjectives are learned with limited computational power by re-purposingpreviously learned skills. An intuitive and elegant concept is presentedwhich exploits abstract symmetries present in training environments to learna na¨ıve agent and bypass some of the core challenges of reinforcement learn-ing training. The na¨ıve agent, trained in a local environment, can then beu…
has type: Thesis/Dissertation
date issued: 2022-02
is shown at: 10.14288/1.0403370
language: eng
degree discipline: Electrical Engineering
provider: Vancouver : University of British Columbia Library
publisher: University of British Columbia
rights: Attribution-NonCommercial-NoDerivatives 4.0 International
rights URI: http://creativecommons.org/licenses/by-nc-nd/4.0/
scholar level: Graduate
contributor: Najjaran, Homayoun
type: Text
identifier URI: http://hdl.handle.net/2429/80187