An Assessment of Multistage Reward Function Design for Deep Reinforcement Learning-Based Microgrid Energy Management

Hui Hwang Goh*, Yifeng Huang, Chee Shen Lim, Dongdong Zhang, Hui Liu, Wei Dai, Tonni Agustiono Kurniawan, Saifur Rahman

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

21 Citations (Scopus)

Abstract

Reinforcement learning based energy management strategy has been an active research subject in the past few years. Different from the baseline reward function (BRF), the work proposes and investigates a multi-stage reward mechanism (MSRM) that scores the agent's step and final performance during training and returns it to the agent in real time as a reward. MSRM will also improve the agent's training through expert intervention which aims to prevent the agent from being trapped in sub-optimal strategies. The energy management performance considered by MSRM-based algorithm includes the energy balance, economic cost, and reliability. The reward function is assessed in conjunction with two deep reinforcement learning algorithms: double deep Q-learning network (DDQN) and policy gradient (PG). Upon benchmarking with BRF, the numerical simulation shows that MSRM tends to improve the convergence characteristic, reduce the explained variance, and reduce the tendency of the agent being trapped in suboptimal strategies. In addition, the methods have been assessed with MPC-based energy management strategies in terms of relative cost, self-balancing rate, and computational time. The assessment concludes that, in the given context, PG-MSRM has the best overall performance.

Original languageEnglish
Pages (from-to)4300-4311
Number of pages12
JournalIEEE Transactions on Smart Grid
Volume13
Issue number6
DOIs
Publication statusPublished - 1 Nov 2022

Keywords

  • Microgrid energy management
  • deep reinforcement learning
  • optimal scheduling
  • reward function

Fingerprint

Dive into the research topics of 'An Assessment of Multistage Reward Function Design for Deep Reinforcement Learning-Based Microgrid Energy Management'. Together they form a unique fingerprint.

Cite this