Open Collections

Dershan, Ram

University of British Columbia

With the advent of Industry 4.0, we see a push for manufacturing processes to be more intelligent. One of the main components of a manufacturing process are the robots, which also must be intelligent and more autonomous. One method of achieving intelligent robots is the use of reinforcement learning (RL). However, the need for large data samples for learning forces us to use simulations to train RL agents for robotic tasks. While simulations help alleviate the data burden, they can negatively impact the robustness of RL agents. Since simulations are not necessarily accurate representations of the real-world, the agents trained in these environments are sensitive to modelling errors and input disturbances, resulting in failure when transferred to the real-world. Many solutions exist to improve the sim-to-real transfer of agents to make them more robust. These solutions include system identification, domain adaptation, and domain randomization. While these methods succeed in creating more robust agents, they are highly extensive, heuristic, and require expert knowledge to implement them. In this work, we propose a method based on the principles of domain randomization that produces robust agents with simpler implementation. We start by proposing a novel simulation platform that incorporates robot dynamics into an existing industrial simulation tool, CIROS, to allow high-fidelity simulations of manufacturing processes. During the testing of this platform, we notice some intrinsic stochasticity in the robot trajectories caused by its real-time nature. We investigate this intrinsic stochasticity further and find that it is comparable to the stochasticity of a real-robot. Based on this finding, we propose using the intrinsic stochasticity as a simpler alternative to domain randomization for robust RL. We validate this claim by training and evaluating the robustness of an agent trained on the intrinsic stochasticity. We found that our method produces a significant level of robustness and can indeed be a viable alternative that is easier to implement than existing solutions.

Text

2022-08-12

Attribution-NonCommercial-NoDerivatives 4.0 International

http://hdl.handle.net/2429/82345

Mechanical Engineering

University of British Columbia

UBCO

http://creativecommons.org/licenses/by-nc-nd/4.0/