A framework of hierarchical deep Q-network for portfolio management

Yuan Gao, Ziming Gao, Yi Hu, Sifan Song, Zhengyong Jiang, Jionglong Su*

*Corresponding author for this work

Research output: Chapter in Book or Report/Conference proceedingConference Proceedingpeer-review

10 Citations (Scopus)

Abstract

Reinforcement Learning algorithms and Neural Networks have diverse applications in many domains, e.g., stock market prediction, facial recognition and automatic machine translation. The concept of modeling the portfolio management through a reinforcement learning formulation is novel, and the Deep Q-Network has been successfully applied to portfolio management recently. However, the model does not take into account of commission fee for transaction. This paper introduces a framework, based on the hierarchical Deep Q-Network, that addresses the issue of zero commission fee by reducing the number of assets assigned to each Deep Q-Network and dividing the total portfolio value into smaller parts. Furthermore, this framework is flexible enough to handle an arbitrary number of assets. In our experiments, the time series of four stocks for three different time periods are used to assess the efficacy of our model. It is found that our hierarchical Deep Q-Network based strategy outperforms ten other strategies, including nine traditional strategies and one reinforcement learning strategy, in profitability as measured by the Cumulative Rate of Return. Moreover, the Sharpe ratio and Max Drawdown metrics both demonstrate that the risk of policy associated with hierarchical Deep Q-Network is the lowest among all ten strategies.

Original languageEnglish
Title of host publicationICAART 2021 - Proceedings of the 13th International Conference on Agents and Artificial Intelligence
EditorsAna Paula Rocha, Luc Steels, Jaap van den Herik
PublisherSciTePress
Pages132-140
Number of pages9
ISBN (Electronic)9789897584848
Publication statusPublished - 2021
Event13th International Conference on Agents and Artificial Intelligence, ICAART 2021 - Virtual, Online
Duration: 4 Feb 20216 Feb 2021

Publication series

NameICAART 2021 - Proceedings of the 13th International Conference on Agents and Artificial Intelligence
Volume2

Conference

Conference13th International Conference on Agents and Artificial Intelligence, ICAART 2021
CityVirtual, Online
Period4/02/216/02/21

Keywords

  • Convolutional neural network
  • Hierarchical reinforcement learning
  • Portfolio management
  • Q-Learning

Fingerprint

Dive into the research topics of 'A framework of hierarchical deep Q-network for portfolio management'. Together they form a unique fingerprint.

Cite this