Abstract: Receding horizon optimal control problems compute the solution at each time step to operate the system on a near-optimal path. However, in many practical cases, the boundary conditions, such as external inputs, constraint equations, or the objective function, vary only marginally from one time step to the next. In this case, recomputing the optimal solution at each time represents a significant burden for real-time applications.
In this talk, we will discuss a recently developed approach to solve perturbed constrained dynamic programs that significantly improves the computational burden when the dynamical equation, the objective function, and/or the constraints are perturbed slightly. The method hinges on determining closed-form expressions for first-order perturbations in the optimal strategy and the Lagrange multipliers of the perturbed constrained dynamic programming problem. We demonstrate through numerical simulations that our algorithms significantly outperforms existing methods.
The extension of perturbation theory to MDPs is discussed next. By viewing transition kernel and policy as linear operators over function spaces, we devise a novel way to look at a reinforcement learning problem. This new viewpoint leads to a new class of PPO algorithms for reinforcement learning.
Bio: Abhishek Gupta is an Associate Professor and Graduate Studies Chair with the Electrical and Computer Engineering Department, The Ohio State University, Columbus, OH, USA. He received the B.Tech. in Aerospace Engineering from the Indian Institute of Technology, Bombay, Mumbai, India, in 2009, the M.S. degree in Applied Mathematics from the University of Illinois at Urbana-Champaign (UIUC), Champaign, IL, USA, in 2012, and the M.S. and Ph.D. degrees in Aerospace Engineering from UIUC, in 2014. His research interests include stochastic control theory, probability theory, and reinforcement learning with applications to autonomous vehicles, cybersecurity, and transportation systems. He is the recipient of Lumley Research Award from The Ohio State University in 2019.
Faculty Host: Vaibhav Srivastava
The seminar is also available via Zoom:
Zoom Meeting ID: 959 8794 6122
Passcode: eceseminar