Two-Phase Deep Reinforcement Learning of Dynamic Resource Allocation and Client Selection for Hierarchical Federated Learning

Xiaojing Chen; Zhenyuan Li; Wei Ni; Xin Wang; Shunqing Zhang; Shugong Xu; Qingqi Pei

doi:10.1109/ICCC55456.2022.9880724

Two-Phase Deep Reinforcement Learning of Dynamic Resource Allocation and Client Selection for Hierarchical Federated Learning

Xiaojing Chen, Zhenyuan Li, Wei Ni, Xin Wang, Shunqing Zhang, Shugong Xu, Qingqi Pei

Research output: Chapter in Book or Report/Conference proceeding › Conference Proceeding › peer-review

9 Citations (Scopus)

Abstract

This paper presents a new two-phase Deep Deter-ministic Policy Gradient (DDPG) framework, referred to as 'TP-DDPG', to balance online the learning delay and model accuracy of a federated learning (FL) process in an energy harvesting hierarchical FL (HFL) system. The key idea is to design a DDPG-based approach to learn the selection of participating clients, the CPU configuration, and the transmission powers of the clients, while the other decisions are efficiently optimized by a new straggler-aware client association and bandwidth allocation algorithm. The algorithm evaluates the reward of the DDPG, and substantially improves its convergence rate and stability. Experimental results demonstrate that the proposed TP-DDPG can substantially reduce the training time while achieving a higher test accuracy over the existing benchmarks.

Original language	English
Title of host publication	2022 IEEE/CIC International Conference on Communications in China, ICCC 2022
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	518-523
Number of pages	6
ISBN (Electronic)	9781665484800
DOIs	https://doi.org/10.1109/ICCC55456.2022.9880724
Publication status	Published - 2022
Externally published	Yes
Event	2022 IEEE/CIC International Conference on Communications in China, ICCC 2022 - Sanshui, Foshan, China Duration: 11 Aug 2022 → 13 Aug 2022

Publication series

Name	2022 IEEE/CIC International Conference on Communications in China, ICCC 2022

Conference

Conference	2022 IEEE/CIC International Conference on Communications in China, ICCC 2022
Country/Territory	China
City	Sanshui, Foshan
Period	11/08/22 → 13/08/22

Keywords

client scheduling
DDPG
Hierarchical federated learning
resource allocation

Access to Document

10.1109/ICCC55456.2022.9880724

Cite this

Chen, X., Li, Z., Ni, W., Wang, X., Zhang, S., Xu, S., & Pei, Q. (2022). Two-Phase Deep Reinforcement Learning of Dynamic Resource Allocation and Client Selection for Hierarchical Federated Learning. In 2022 IEEE/CIC International Conference on Communications in China, ICCC 2022 (pp. 518-523). (2022 IEEE/CIC International Conference on Communications in China, ICCC 2022). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICCC55456.2022.9880724

Chen, Xiaojing ; Li, Zhenyuan ; Ni, Wei et al. / Two-Phase Deep Reinforcement Learning of Dynamic Resource Allocation and Client Selection for Hierarchical Federated Learning. 2022 IEEE/CIC International Conference on Communications in China, ICCC 2022. Institute of Electrical and Electronics Engineers Inc., 2022. pp. 518-523 (2022 IEEE/CIC International Conference on Communications in China, ICCC 2022).

@inproceedings{1a262017f10344c19ef3b3677486ed44,

title = "Two-Phase Deep Reinforcement Learning of Dynamic Resource Allocation and Client Selection for Hierarchical Federated Learning",

abstract = "This paper presents a new two-phase Deep Deter-ministic Policy Gradient (DDPG) framework, referred to as 'TP-DDPG', to balance online the learning delay and model accuracy of a federated learning (FL) process in an energy harvesting hierarchical FL (HFL) system. The key idea is to design a DDPG-based approach to learn the selection of participating clients, the CPU configuration, and the transmission powers of the clients, while the other decisions are efficiently optimized by a new straggler-aware client association and bandwidth allocation algorithm. The algorithm evaluates the reward of the DDPG, and substantially improves its convergence rate and stability. Experimental results demonstrate that the proposed TP-DDPG can substantially reduce the training time while achieving a higher test accuracy over the existing benchmarks.",

keywords = "client scheduling, DDPG, Hierarchical federated learning, resource allocation",

author = "Xiaojing Chen and Zhenyuan Li and Wei Ni and Xin Wang and Shunqing Zhang and Shugong Xu and Qingqi Pei",

note = "Publisher Copyright: {\textcopyright} 2022 IEEE.; 2022 IEEE/CIC International Conference on Communications in China, ICCC 2022 ; Conference date: 11-08-2022 Through 13-08-2022",

year = "2022",

doi = "10.1109/ICCC55456.2022.9880724",

language = "English",

series = "2022 IEEE/CIC International Conference on Communications in China, ICCC 2022",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "518--523",

booktitle = "2022 IEEE/CIC International Conference on Communications in China, ICCC 2022",

}

Chen, X, Li, Z, Ni, W, Wang, X, Zhang, S, Xu, S & Pei, Q 2022, Two-Phase Deep Reinforcement Learning of Dynamic Resource Allocation and Client Selection for Hierarchical Federated Learning. in 2022 IEEE/CIC International Conference on Communications in China, ICCC 2022. 2022 IEEE/CIC International Conference on Communications in China, ICCC 2022, Institute of Electrical and Electronics Engineers Inc., pp. 518-523, 2022 IEEE/CIC International Conference on Communications in China, ICCC 2022, Sanshui, Foshan, China, 11/08/22. https://doi.org/10.1109/ICCC55456.2022.9880724

Two-Phase Deep Reinforcement Learning of Dynamic Resource Allocation and Client Selection for Hierarchical Federated Learning. / Chen, Xiaojing; Li, Zhenyuan; Ni, Wei et al.
2022 IEEE/CIC International Conference on Communications in China, ICCC 2022. Institute of Electrical and Electronics Engineers Inc., 2022. p. 518-523 (2022 IEEE/CIC International Conference on Communications in China, ICCC 2022).

Research output: Chapter in Book or Report/Conference proceeding › Conference Proceeding › peer-review

TY - GEN

T1 - Two-Phase Deep Reinforcement Learning of Dynamic Resource Allocation and Client Selection for Hierarchical Federated Learning

AU - Chen, Xiaojing

AU - Li, Zhenyuan

AU - Ni, Wei

AU - Wang, Xin

AU - Zhang, Shunqing

AU - Xu, Shugong

AU - Pei, Qingqi

PY - 2022

Y1 - 2022

N2 - This paper presents a new two-phase Deep Deter-ministic Policy Gradient (DDPG) framework, referred to as 'TP-DDPG', to balance online the learning delay and model accuracy of a federated learning (FL) process in an energy harvesting hierarchical FL (HFL) system. The key idea is to design a DDPG-based approach to learn the selection of participating clients, the CPU configuration, and the transmission powers of the clients, while the other decisions are efficiently optimized by a new straggler-aware client association and bandwidth allocation algorithm. The algorithm evaluates the reward of the DDPG, and substantially improves its convergence rate and stability. Experimental results demonstrate that the proposed TP-DDPG can substantially reduce the training time while achieving a higher test accuracy over the existing benchmarks.

AB - This paper presents a new two-phase Deep Deter-ministic Policy Gradient (DDPG) framework, referred to as 'TP-DDPG', to balance online the learning delay and model accuracy of a federated learning (FL) process in an energy harvesting hierarchical FL (HFL) system. The key idea is to design a DDPG-based approach to learn the selection of participating clients, the CPU configuration, and the transmission powers of the clients, while the other decisions are efficiently optimized by a new straggler-aware client association and bandwidth allocation algorithm. The algorithm evaluates the reward of the DDPG, and substantially improves its convergence rate and stability. Experimental results demonstrate that the proposed TP-DDPG can substantially reduce the training time while achieving a higher test accuracy over the existing benchmarks.

KW - client scheduling

KW - DDPG

KW - Hierarchical federated learning

KW - resource allocation

UR - http://www.scopus.com/inward/record.url?scp=85139481204&partnerID=8YFLogxK

U2 - 10.1109/ICCC55456.2022.9880724

DO - 10.1109/ICCC55456.2022.9880724

M3 - Conference Proceeding

AN - SCOPUS:85139481204

T3 - 2022 IEEE/CIC International Conference on Communications in China, ICCC 2022

SP - 518

EP - 523

BT - 2022 IEEE/CIC International Conference on Communications in China, ICCC 2022

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 2022 IEEE/CIC International Conference on Communications in China, ICCC 2022

Y2 - 11 August 2022 through 13 August 2022

ER -

Chen X, Li Z, Ni W, Wang X, Zhang S, Xu S et al. Two-Phase Deep Reinforcement Learning of Dynamic Resource Allocation and Client Selection for Hierarchical Federated Learning. In 2022 IEEE/CIC International Conference on Communications in China, ICCC 2022. Institute of Electrical and Electronics Engineers Inc. 2022. p. 518-523. (2022 IEEE/CIC International Conference on Communications in China, ICCC 2022). doi: 10.1109/ICCC55456.2022.9880724

Two-Phase Deep Reinforcement Learning of Dynamic Resource Allocation and Client Selection for Hierarchical Federated Learning

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this