Two-Phase Deep Reinforcement Learning of Dynamic Resource Allocation and Client Selection for Hierarchical Federated Learning

Xiaojing Chen; Zhenyuan Li; Wei Ni; Xin Wang; Shunqing Zhang; Shugong Xu; Qingqi Pei

doi:10.1109/ICCC55456.2022.9880724

Two-Phase Deep Reinforcement Learning of Dynamic Resource Allocation and Client Selection for Hierarchical Federated Learning

Xiaojing Chen
, Zhenyuan Li
, Wei Ni
, Xin Wang
, Shunqing Zhang
, Shugong Xu
, Qingqi Pei

Research output: Chapter in Book or Report/Conference proceeding › Conference Proceeding › peer-review

11 Citations (Scopus)

Abstract

This paper presents a new two-phase Deep Deter-ministic Policy Gradient (DDPG) framework, referred to as 'TP-DDPG', to balance online the learning delay and model accuracy of a federated learning (FL) process in an energy harvesting hierarchical FL (HFL) system. The key idea is to design a DDPG-based approach to learn the selection of participating clients, the CPU configuration, and the transmission powers of the clients, while the other decisions are efficiently optimized by a new straggler-aware client association and bandwidth allocation algorithm. The algorithm evaluates the reward of the DDPG, and substantially improves its convergence rate and stability. Experimental results demonstrate that the proposed TP-DDPG can substantially reduce the training time while achieving a higher test accuracy over the existing benchmarks.

Original language	English
Title of host publication	2022 IEEE/CIC International Conference on Communications in China, ICCC 2022
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	518-523
Number of pages	6
ISBN (Electronic)	9781665484800
DOIs	https://doi.org/10.1109/ICCC55456.2022.9880724
Publication status	Published - 2022
Externally published	Yes
Event	2022 IEEE/CIC International Conference on Communications in China, ICCC 2022 - Sanshui, Foshan, China Duration: 11 Aug 2022 → 13 Aug 2022

Publication series

Name	2022 IEEE/CIC International Conference on Communications in China, ICCC 2022

Conference

Conference	2022 IEEE/CIC International Conference on Communications in China, ICCC 2022
Country/Territory	China
City	Sanshui, Foshan
Period	11/08/22 → 13/08/22

Keywords

DDPG
Hierarchical federated learning
client scheduling
resource allocation

Access to Document

10.1109/ICCC55456.2022.9880724

Cite this

Chen, X., Li, Z., Ni, W., Wang, X., Zhang, S., Xu, S., & Pei, Q. (2022). Two-Phase Deep Reinforcement Learning of Dynamic Resource Allocation and Client Selection for Hierarchical Federated Learning. In 2022 IEEE/CIC International Conference on Communications in China, ICCC 2022 (pp. 518-523). (2022 IEEE/CIC International Conference on Communications in China, ICCC 2022). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICCC55456.2022.9880724

Chen, Xiaojing ; Li, Zhenyuan ; Ni, Wei et al. / Two-Phase Deep Reinforcement Learning of Dynamic Resource Allocation and Client Selection for Hierarchical Federated Learning. 2022 IEEE/CIC International Conference on Communications in China, ICCC 2022. Institute of Electrical and Electronics Engineers Inc., 2022. pp. 518-523 (2022 IEEE/CIC International Conference on Communications in China, ICCC 2022).

@inproceedings{1a262017f10344c19ef3b3677486ed44,

title = "Two-Phase Deep Reinforcement Learning of Dynamic Resource Allocation and Client Selection for Hierarchical Federated Learning",

abstract = "This paper presents a new two-phase Deep Deter-ministic Policy Gradient (DDPG) framework, referred to as 'TP-DDPG', to balance online the learning delay and model accuracy of a federated learning (FL) process in an energy harvesting hierarchical FL (HFL) system. The key idea is to design a DDPG-based approach to learn the selection of participating clients, the CPU configuration, and the transmission powers of the clients, while the other decisions are efficiently optimized by a new straggler-aware client association and bandwidth allocation algorithm. The algorithm evaluates the reward of the DDPG, and substantially improves its convergence rate and stability. Experimental results demonstrate that the proposed TP-DDPG can substantially reduce the training time while achieving a higher test accuracy over the existing benchmarks.",

keywords = "DDPG, Hierarchical federated learning, client scheduling, resource allocation",

author = "Xiaojing Chen and Zhenyuan Li and Wei Ni and Xin Wang and Shunqing Zhang and Shugong Xu and Qingqi Pei",

note = "Publisher Copyright: {\textcopyright} 2022 IEEE.; 2022 IEEE/CIC International Conference on Communications in China, ICCC 2022 ; Conference date: 11-08-2022 Through 13-08-2022",

year = "2022",

doi = "10.1109/ICCC55456.2022.9880724",

language = "English",

series = "2022 IEEE/CIC International Conference on Communications in China, ICCC 2022",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "518--523",

booktitle = "2022 IEEE/CIC International Conference on Communications in China, ICCC 2022",

}

Chen, X, Li, Z, Ni, W, Wang, X, Zhang, S, Xu, S & Pei, Q 2022, Two-Phase Deep Reinforcement Learning of Dynamic Resource Allocation and Client Selection for Hierarchical Federated Learning. in 2022 IEEE/CIC International Conference on Communications in China, ICCC 2022. 2022 IEEE/CIC International Conference on Communications in China, ICCC 2022, Institute of Electrical and Electronics Engineers Inc., pp. 518-523, 2022 IEEE/CIC International Conference on Communications in China, ICCC 2022, Sanshui, Foshan, China, 11/08/22. https://doi.org/10.1109/ICCC55456.2022.9880724

Two-Phase Deep Reinforcement Learning of Dynamic Resource Allocation and Client Selection for Hierarchical Federated Learning. / Chen, Xiaojing; Li, Zhenyuan; Ni, Wei et al.
2022 IEEE/CIC International Conference on Communications in China, ICCC 2022. Institute of Electrical and Electronics Engineers Inc., 2022. p. 518-523 (2022 IEEE/CIC International Conference on Communications in China, ICCC 2022).

Research output: Chapter in Book or Report/Conference proceeding › Conference Proceeding › peer-review

TY - GEN

T1 - Two-Phase Deep Reinforcement Learning of Dynamic Resource Allocation and Client Selection for Hierarchical Federated Learning

AU - Chen, Xiaojing

AU - Li, Zhenyuan

AU - Ni, Wei

AU - Wang, Xin

AU - Zhang, Shunqing

AU - Xu, Shugong

AU - Pei, Qingqi

PY - 2022

Y1 - 2022

N2 - This paper presents a new two-phase Deep Deter-ministic Policy Gradient (DDPG) framework, referred to as 'TP-DDPG', to balance online the learning delay and model accuracy of a federated learning (FL) process in an energy harvesting hierarchical FL (HFL) system. The key idea is to design a DDPG-based approach to learn the selection of participating clients, the CPU configuration, and the transmission powers of the clients, while the other decisions are efficiently optimized by a new straggler-aware client association and bandwidth allocation algorithm. The algorithm evaluates the reward of the DDPG, and substantially improves its convergence rate and stability. Experimental results demonstrate that the proposed TP-DDPG can substantially reduce the training time while achieving a higher test accuracy over the existing benchmarks.

AB - This paper presents a new two-phase Deep Deter-ministic Policy Gradient (DDPG) framework, referred to as 'TP-DDPG', to balance online the learning delay and model accuracy of a federated learning (FL) process in an energy harvesting hierarchical FL (HFL) system. The key idea is to design a DDPG-based approach to learn the selection of participating clients, the CPU configuration, and the transmission powers of the clients, while the other decisions are efficiently optimized by a new straggler-aware client association and bandwidth allocation algorithm. The algorithm evaluates the reward of the DDPG, and substantially improves its convergence rate and stability. Experimental results demonstrate that the proposed TP-DDPG can substantially reduce the training time while achieving a higher test accuracy over the existing benchmarks.

KW - DDPG

KW - Hierarchical federated learning

KW - client scheduling

KW - resource allocation

UR - https://www.scopus.com/pages/publications/85139481204

U2 - 10.1109/ICCC55456.2022.9880724

DO - 10.1109/ICCC55456.2022.9880724

M3 - Conference Proceeding

AN - SCOPUS:85139481204

T3 - 2022 IEEE/CIC International Conference on Communications in China, ICCC 2022

SP - 518

EP - 523

BT - 2022 IEEE/CIC International Conference on Communications in China, ICCC 2022

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 2022 IEEE/CIC International Conference on Communications in China, ICCC 2022

Y2 - 11 August 2022 through 13 August 2022

ER -

Chen X, Li Z, Ni W, Wang X, Zhang S, Xu S et al. Two-Phase Deep Reinforcement Learning of Dynamic Resource Allocation and Client Selection for Hierarchical Federated Learning. In 2022 IEEE/CIC International Conference on Communications in China, ICCC 2022. Institute of Electrical and Electronics Engineers Inc. 2022. p. 518-523. (2022 IEEE/CIC International Conference on Communications in China, ICCC 2022). doi: 10.1109/ICCC55456.2022.9880724

Two-Phase Deep Reinforcement Learning of Dynamic Resource Allocation and Client Selection for Hierarchical Federated Learning

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this