- Library Home /
- Search Collections /
- Open Collections /
- Browse Collections /
- UBC Theses and Dissertations /
- Accelerating reinforcement learning through imitation
Open Collections
UBC Theses and Dissertations
UBC Theses and Dissertations
Accelerating reinforcement learning through imitation Price, Robert Roy
Abstract
Imitation can be viewed as a means of enhancing learning in multiagent environments. It augments an agent's ability to learn useful behaviours by making intelligent use of the knowledge implicit in behaviors demonstrated by cooperative teachers or other more experienced agents. Using reinforcement learning theory, we construct a new, formal framework for imitation that permits agents to combine prior knowledge, learned knowledge and knowledge extracted from observations of other agents. This framework, which we call implicit imitation, uses observations of other agents to provide an observer agent with information about its action capabilities in unexperienced situations. Efficient algorithms are derived from this framework for agents with both similar and dissimilar action capabilities and a series of experiments demonstrate that implicit imitation can dramatically accelerate reinforcement learning in certain cases. Further experiments demonstrate that the framework can handle transfer between agents with different reward structures, learning from multiple mentors, and selecting relevant portions of examples. A Bayesian version of implicit imitation is then derived and several experiments are used to illustrate how it smoothly integrates model extraction with prior knowledge and optimal exploration. Initial work is also presented to illustrate how extensions to implicit imitation can be derived for continuous domains, partially observable environments and multi-agent Markov games. The derivation of algorithms and experimental results are supplemented with an analysis of relevant domain features for imitating agents and some performance metrics for predicting imitation gains are suggested.
Item Metadata
Title |
Accelerating reinforcement learning through imitation
|
Creator | |
Publisher |
University of British Columbia
|
Date Issued |
2002
|
Description |
Imitation can be viewed as a means of enhancing learning in multiagent environments.
It augments an agent's ability to learn useful behaviours by making intelligent use
of the knowledge implicit in behaviors demonstrated by cooperative teachers or other more
experienced agents. Using reinforcement learning theory, we construct a new, formal framework
for imitation that permits agents to combine prior knowledge, learned knowledge and
knowledge extracted from observations of other agents. This framework, which we call implicit
imitation, uses observations of other agents to provide an observer agent with information
about its action capabilities in unexperienced situations. Efficient algorithms are derived
from this framework for agents with both similar and dissimilar action capabilities and
a series of experiments demonstrate that implicit imitation can dramatically accelerate reinforcement
learning in certain cases. Further experiments demonstrate that the framework
can handle transfer between agents with different reward structures, learning from multiple
mentors, and selecting relevant portions of examples. A Bayesian version of implicit
imitation is then derived and several experiments are used to illustrate how it smoothly integrates
model extraction with prior knowledge and optimal exploration. Initial work is also
presented to illustrate how extensions to implicit imitation can be derived for continuous domains,
partially observable environments and multi-agent Markov games. The derivation of
algorithms and experimental results are supplemented with an analysis of relevant domain
features for imitating agents and some performance metrics for predicting imitation gains
are suggested.
|
Extent |
8414399 bytes
|
Genre | |
Type | |
File Format |
application/pdf
|
Language |
eng
|
Date Available |
2009-11-13
|
Provider |
Vancouver : University of British Columbia Library
|
Rights |
For non-commercial purposes only, such as research, private study and education. Additional conditions apply, see Terms of Use https://open.library.ubc.ca/terms_of_use.
|
DOI |
10.14288/1.0051625
|
URI | |
Degree | |
Program | |
Affiliation | |
Degree Grantor |
University of British Columbia
|
Graduation Date |
2002-11
|
Campus | |
Scholarly Level |
Graduate
|
Aggregated Source Repository |
DSpace
|
Item Media
Item Citations and Data
Rights
For non-commercial purposes only, such as research, private study and education. Additional conditions apply, see Terms of Use https://open.library.ubc.ca/terms_of_use.