Online reinforcement learning for condition-based group maintenance using factored Markov decision processes

Jianyu Xu, Bin Liu*, Xiujie Zhao, Xiao Lin Wang

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

Abstract

We investigate a condition-based group maintenance problem for multi-component systems, where the degradation process of a specific component is affected only by its neighbouring ones, leading to a special type of stochastic dependence among components. We formulate the maintenance problem into a factored Markov decision process taking advantage of this dependence property, and develop a factored value iteration algorithm to efficiently approximate the optimal policy. Through both theoretical analyses and numerical experiments, we show that the algorithm can significantly reduce computational burden and improve efficiency in solving the optimization problem. Moreover, since model parameters are unknown a priori in most practical scenarios, we further develop an online reinforcement learning algorithm to simultaneously learn the model parameters and determine an optimal maintenance action upon each inspection. A novel feature of this online learning algorithm is that it is capable of learning both transition probabilities and system structure indicating the stochastic dependence among components. We discuss the error bound and sample complexity of the developed learning algorithm theoretically, and test its performance through numerical experiments. The results reveal that our algorithm can effectively learn the model parameters and approximate the optimal maintenance policy.

Original languageEnglish
Pages (from-to)176-190
Number of pages15
JournalEuropean Journal of Operational Research
Volume315
Issue number1
DOIs
Publication statusPublished - 16 May 2024

Keywords

  • Condition-based group maintenance
  • Factored Markov decision process
  • Factored value iteration
  • Maintenance
  • Online reinforcement learning

Fingerprint

Dive into the research topics of 'Online reinforcement learning for condition-based group maintenance using factored Markov decision processes'. Together they form a unique fingerprint.

Cite this