Learning finite-horizon optimal control with unknown control-affine dynamics

This paper introduces a method for learning finite-horizon optimal control for systems with control-affine dynamics without using the model of the system dynamics. We approximate the time- and state-dependent optimal control policy using model-free relearning of simple linear control policies. Assuming persistent excitation, we prove the convergence and optimality of the proposed learning method and demonstrate its use through a numerical example.