TY - JOUR
T1 - Large-Scale Traffic Signal Control Using Constrained Network Partition and Adaptive Deep Reinforcement Learning
AU - Gu, Hankang
AU - Wang, Shangbo
AU - Ma, Xiaoguang
AU - Jia, Dongyao
AU - Mao, Guoqiang
AU - Lim, Eng Gee
AU - Wong, Cheuk Pong Ryan
N1 - Publisher Copyright:
© 2000-2011 IEEE.
PY - 2024
Y1 - 2024
N2 - Multi-agent Deep Reinforcement Learning (MADRL) based traffic signal control lbecomes a popular research topic in recent years. To alleviate the scalability issue of completely centralized reinforcement learning (RL) techniques and the non-stationarity issue of completely decentralized RL techniques on large-scale traffic networks, some literature utilizes a regional control approach where the whole network is firstly partitioned into multiple disjoint regions, followed by applying the centralized RL approach to each region. However, the existing partitioning rules either have no constraints on the topology of regions or require the same topology for all regions. Meanwhile, no existing regional control approach explores the performance of optimal joint action in an exponentially growing regional action space when intersections are controlled by 4-phase traffic signals (EW, EWL, NS, NSL). In this paper, we propose a novel RL training framework named RegionLight to tackle the above limitations. Specifically, the topology of regions is firstly constrained to a star network which comprises one center and an arbitrary number of leaves. Next, the network partitioning problem is modeled as an optimization problem to minimize the number of regions. Then, an Adaptive Branching Dueling Q-Network (ABDQ) model is proposed to decompose the regional control task into several joint signal control sub-tasks corresponding to particular intersections. Subsequently, these sub-tasks maximize the regional benefits cooperatively. Finally, the global control strategy for the whole network is obtained by concatenating the optimal joint actions of all regions. Experimental results demonstrate the superiority of our proposed framework over all baselines under both real and synthetic scenarios in all evaluation metrics.
AB - Multi-agent Deep Reinforcement Learning (MADRL) based traffic signal control lbecomes a popular research topic in recent years. To alleviate the scalability issue of completely centralized reinforcement learning (RL) techniques and the non-stationarity issue of completely decentralized RL techniques on large-scale traffic networks, some literature utilizes a regional control approach where the whole network is firstly partitioned into multiple disjoint regions, followed by applying the centralized RL approach to each region. However, the existing partitioning rules either have no constraints on the topology of regions or require the same topology for all regions. Meanwhile, no existing regional control approach explores the performance of optimal joint action in an exponentially growing regional action space when intersections are controlled by 4-phase traffic signals (EW, EWL, NS, NSL). In this paper, we propose a novel RL training framework named RegionLight to tackle the above limitations. Specifically, the topology of regions is firstly constrained to a star network which comprises one center and an arbitrary number of leaves. Next, the network partitioning problem is modeled as an optimization problem to minimize the number of regions. Then, an Adaptive Branching Dueling Q-Network (ABDQ) model is proposed to decompose the regional control task into several joint signal control sub-tasks corresponding to particular intersections. Subsequently, these sub-tasks maximize the regional benefits cooperatively. Finally, the global control strategy for the whole network is obtained by concatenating the optimal joint actions of all regions. Experimental results demonstrate the superiority of our proposed framework over all baselines under both real and synthetic scenarios in all evaluation metrics.
KW - Adaptive traffic signal control
KW - multi-agent deep reinforcement learning
KW - regional control
UR - http://www.scopus.com/inward/record.url?scp=85189622827&partnerID=8YFLogxK
U2 - 10.1109/TITS.2024.3352446
DO - 10.1109/TITS.2024.3352446
M3 - Article
AN - SCOPUS:85189622827
SN - 1524-9050
VL - 25
SP - 7619
EP - 7632
JO - IEEE Transactions on Intelligent Transportation Systems
JF - IEEE Transactions on Intelligent Transportation Systems
IS - 7
ER -