Non UBC
DSpace
Grand Clement, Julien
2019-07-18T08:17:57Z
2019-01-18T09:08
Markov decision processes (MDPs) are a common approach to model dynamic optimization problems in many applications. However, in most real world problems, the model parameters that are estimated from noisy observations are uncertain, and the optimal policy for the nominal parameter values might be highly sensitive to even small perturbations in the parameters leading to significantly suboptimal outcomes.
We consider a robust approach where the uncertainty in probability transitions is modeled as an adversarial selection from
an uncertainty set. Most prior work considers the case where uncertainty on parameters related to different states is unrelated and the
adversary is allowed to select worst possible realization for each state unrelated to others, potentially leading to highly conservative solutions. On the other hand, the case of general uncertainty sets is known to be intractable. We consider a factor model for probability transitions where the transition probability is a linear function of a factor matrix that is uncertain and belongs to a factor matrix uncertainty
set. This a significantly less conservative approach to modeling uncertainty in probability transitions while allowing to model dependence between probability transitions across different states. We show that under a certain rectangularity assumption, we can efficiently compute the optimal robust policy under the factor matrix uncertainty model. We also present a computational study to demonstrate the usefulness
of our approach.
https://circle.library.ubc.ca/rest/handle/2429/71046?expand=metadata
41.0
video/mp4
Author affiliation: Columbia University
Banff (Alta.)
10.14288/1.0379915
eng
Unreviewed
Vancouver : University of British Columbia Library
Banff International Research Station for Mathematical Innovation and Discovery
Attribution-NonCommercial-NoDerivatives 4.0 International
http://creativecommons.org/licenses/by-nc-nd/4.0/
Graduate
BIRS Workshop Lecture Videos (Banff, Alta)
Mathematics
Operations research, mathematical programming
Computer science
Applied computer science
Robust Markov Decision Processes: Beyond Rectangularity
Moving Image
http://hdl.handle.net/2429/71046