Stochastic Gradient Variational Inference of Unmodeled Dynamics in State-Space With Applications

Industrial processes rarely admit accurate state-space models in practice, leading to deteriorated estimation accuracy of Kalman filters due to the existence of unmodeled dynamics. Previous coordinate ascent variational inference method highly depends on the ideal mean-field and conjugacy conditions, typically assuming that the estimated unmodeled dynamics follow a simple, independent Gaussian distribution within the conjugate exponential family. To relax these limitations, this article designs a state-dependent, richer, yet nonconjugate model for the unmodeled dynamics, and proposes a universal optimization framework for its variational inference. Specifically, we explicitly parameterize the unmodeled dynamics using a Bayesian linear basis function model with probabilistic weights, where potential dependence on states can be captured by diverse nonlinear features. The conjugate priors of weights are no longer available in such a complex model structure. Thus, the stochastic gradient variational inference technique is introduced to maximize the evidence lower bound via gradient ascent methods, whereas the common coordinate ascent paradigm fails to achieve an analytical solution. Experiments exhibit significant improvement in mismodeling compensation and state estimation compared to previous methods with Gaussian approximation.