NFVdeep: Adaptive online service function chain deployment with deep reinforcement learning

Yikai Xiao, Qixia Zhang, Fangming Liu, Jia Wang, Miao Zhao, Zhongxing Zhang, Jiaxing Zhang

Research output: Chapter in Book or Report/Conference proceedingConference Proceedingpeer-review

118 Citations (Scopus)

Abstract

With the evolution of network function virtualization (NFV), diverse network services can be ?exibly o?ered as service function chains (SFCs) consisted of di?erent virtual network functions (VNFs). However, network state and tra?c typically exhibit unpredictable variations due to stochastically arriving requests with di?erent quality of service (QoS) requirements. Thus, an adaptive online SFC deployment approach is needed to handle the real-time network variations and various service requests. In this paper, we ?rstly introduce a Markov decision process (MDP) model to capture the dynamic network state transitions. In order to jointly minimize the operation cost of NFV providers and maximize the total throughput of requests, we propose NFVdeep, an adaptive, online, deep reinforcement learning approach to automatically deploy SFCs for requests with di?erent QoS requirements. Speci?cally, we use a serialization-and-backtracking method to e?ectively deal with large discrete action space. We also adopt a policy gradient based method to improve the training e?ciency and convergence to optimality. Extensive experimental results demonstrate that NFVdeep converges fast in the training process and responds rapidly to arriving requests especially in large, frequently transferred network state space. Consequently, NFVdeep surpasses the state-of-the-art methods by 32.59% higher accepted throughput and 33.29% lower operation cost on average.

Original languageEnglish
Title of host publicationProceedings of the International Symposium on Quality of Service, IWQoS 2019
PublisherAssociation for Computing Machinery, Inc
ISBN (Electronic)9781450367783
DOIs
Publication statusPublished - 24 Jun 2019
Externally publishedYes
Event2019 International Symposium on Quality of Service, IWQoS 2019 - Phoenix, United States
Duration: 24 Jun 201925 Jun 2019

Publication series

NameProceedings of the International Symposium on Quality of Service, IWQoS 2019

Conference

Conference2019 International Symposium on Quality of Service, IWQoS 2019
Country/TerritoryUnited States
CityPhoenix
Period24/06/1925/06/19

Keywords

  • Deep Reinforcement Learning
  • Network Function Virtualization (NFV)
  • QoS-Aware Resource Management
  • Service Function Chain

Fingerprint

Dive into the research topics of 'NFVdeep: Adaptive online service function chain deployment with deep reinforcement learning'. Together they form a unique fingerprint.

Cite this