Reinforcement learning is essentially a simulation-based approach in obtaining an approximate solution to an optimal control/Markov decision problem. Due to the popularity of deep learning, there has been a growing interest in using deep neural networks to solve RL problems. This has led to a big success in RL, especially in playing various Atari games. However, existing methods are often purely algorithmic. When applied to complex physical systems, they often suffer from issues like poor sampling complexity, bad local minima, and lack of physical interpretations, etc.

Our research aims to address these issues by incorporating analytical insights into design of reinforcement learning algorithms. Our insights are either based on well-established engineering principles (such as existing robot control structure) or based on structural properties of optimal policies, both of which lead to new network structures for more effective learning