Lane Change Decision-making through Deep Reinforcement Learning with Rule-based Constraints

Junjie Wang; Qichao Zhang; Dongbin Zhao; Yaran Chen

doi:10.1109/IJCNN.2019.8852110

Lane Change Decision-making through Deep Reinforcement Learning with Rule-based Constraints

Junjie Wang
, Qichao Zhang
, Dongbin Zhao
, Yaran Chen

CAS - Institute of Automation
University of Chinese Academy of Sciences

Research output: Chapter in Book or Report/Conference proceeding › Conference Proceeding › peer-review

167 Citations (Scopus)

Abstract

Autonomous driving decision-making is a great challenge due to the complexity and uncertainty of the traffic environment. Combined with the rule-based constraints, a Deep Q-Network (DQN) based method is applied for autonomous driving lane change decision-making task in this study. Through the combination of high-level lateral decision-making and low-level rule-based trajectory modification, a safe and efficient lane change behavior can be achieved. With the setting of our state representation and reward function, the trained agent is able to take appropriate actions in a real-world-like simulator. The generated policy is evaluated on the simulator for 10 times, and the results demonstrate that the proposed rule-based DQN method outperforms the rule-based approach and the DQN method.

Original language	English
Title of host publication	2019 International Joint Conference on Neural Networks, IJCNN 2019
Publisher	Institute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)	9781728119854
DOIs	https://doi.org/10.1109/IJCNN.2019.8852110
Publication status	Published - Jul 2019
Event	2019 International Joint Conference on Neural Networks, IJCNN 2019 - Budapest, Hungary Duration: 14 Jul 2019 → 19 Jul 2019

Publication series

Name	Proceedings of the International Joint Conference on Neural Networks
Volume	2019-July

Conference

Conference	2019 International Joint Conference on Neural Networks, IJCNN 2019
Country/Territory	Hungary
City	Budapest
Period	14/07/19 → 19/07/19

Access to Document

10.1109/IJCNN.2019.8852110

Cite this

Wang, J., Zhang, Q., Zhao, D., & Chen, Y. (2019). Lane Change Decision-making through Deep Reinforcement Learning with Rule-based Constraints. In 2019 International Joint Conference on Neural Networks, IJCNN 2019 Article 8852110 (Proceedings of the International Joint Conference on Neural Networks; Vol. 2019-July). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/IJCNN.2019.8852110

@inproceedings{2419662a4c66420889fdbe77a71a9489,

title = "Lane Change Decision-making through Deep Reinforcement Learning with Rule-based Constraints",

abstract = "Autonomous driving decision-making is a great challenge due to the complexity and uncertainty of the traffic environment. Combined with the rule-based constraints, a Deep Q-Network (DQN) based method is applied for autonomous driving lane change decision-making task in this study. Through the combination of high-level lateral decision-making and low-level rule-based trajectory modification, a safe and efficient lane change behavior can be achieved. With the setting of our state representation and reward function, the trained agent is able to take appropriate actions in a real-world-like simulator. The generated policy is evaluated on the simulator for 10 times, and the results demonstrate that the proposed rule-based DQN method outperforms the rule-based approach and the DQN method.",

author = "Junjie Wang and Qichao Zhang and Dongbin Zhao and Yaran Chen",

note = "Publisher Copyright: {\textcopyright} 2019 IEEE.; 2019 International Joint Conference on Neural Networks, IJCNN 2019 ; Conference date: 14-07-2019 Through 19-07-2019",

year = "2019",

month = jul,

doi = "10.1109/IJCNN.2019.8852110",

language = "English",

series = "Proceedings of the International Joint Conference on Neural Networks",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

booktitle = "2019 International Joint Conference on Neural Networks, IJCNN 2019",

}

Wang, J, Zhang, Q, Zhao, D & Chen, Y 2019, Lane Change Decision-making through Deep Reinforcement Learning with Rule-based Constraints. in 2019 International Joint Conference on Neural Networks, IJCNN 2019., 8852110, Proceedings of the International Joint Conference on Neural Networks, vol. 2019-July, Institute of Electrical and Electronics Engineers Inc., 2019 International Joint Conference on Neural Networks, IJCNN 2019, Budapest, Hungary, 14/07/19. https://doi.org/10.1109/IJCNN.2019.8852110

Lane Change Decision-making through Deep Reinforcement Learning with Rule-based Constraints. / Wang, Junjie; Zhang, Qichao; Zhao, Dongbin et al.
2019 International Joint Conference on Neural Networks, IJCNN 2019. Institute of Electrical and Electronics Engineers Inc., 2019. 8852110 (Proceedings of the International Joint Conference on Neural Networks; Vol. 2019-July).

Research output: Chapter in Book or Report/Conference proceeding › Conference Proceeding › peer-review

TY - GEN

T1 - Lane Change Decision-making through Deep Reinforcement Learning with Rule-based Constraints

AU - Wang, Junjie

AU - Zhang, Qichao

AU - Zhao, Dongbin

AU - Chen, Yaran

PY - 2019/7

Y1 - 2019/7

N2 - Autonomous driving decision-making is a great challenge due to the complexity and uncertainty of the traffic environment. Combined with the rule-based constraints, a Deep Q-Network (DQN) based method is applied for autonomous driving lane change decision-making task in this study. Through the combination of high-level lateral decision-making and low-level rule-based trajectory modification, a safe and efficient lane change behavior can be achieved. With the setting of our state representation and reward function, the trained agent is able to take appropriate actions in a real-world-like simulator. The generated policy is evaluated on the simulator for 10 times, and the results demonstrate that the proposed rule-based DQN method outperforms the rule-based approach and the DQN method.

AB - Autonomous driving decision-making is a great challenge due to the complexity and uncertainty of the traffic environment. Combined with the rule-based constraints, a Deep Q-Network (DQN) based method is applied for autonomous driving lane change decision-making task in this study. Through the combination of high-level lateral decision-making and low-level rule-based trajectory modification, a safe and efficient lane change behavior can be achieved. With the setting of our state representation and reward function, the trained agent is able to take appropriate actions in a real-world-like simulator. The generated policy is evaluated on the simulator for 10 times, and the results demonstrate that the proposed rule-based DQN method outperforms the rule-based approach and the DQN method.

UR - https://www.scopus.com/pages/publications/85073238905

U2 - 10.1109/IJCNN.2019.8852110

DO - 10.1109/IJCNN.2019.8852110

M3 - Conference Proceeding

AN - SCOPUS:85073238905

T3 - Proceedings of the International Joint Conference on Neural Networks

BT - 2019 International Joint Conference on Neural Networks, IJCNN 2019

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 2019 International Joint Conference on Neural Networks, IJCNN 2019

Y2 - 14 July 2019 through 19 July 2019

ER -

Wang J, Zhang Q, Zhao D, Chen Y. Lane Change Decision-making through Deep Reinforcement Learning with Rule-based Constraints. In 2019 International Joint Conference on Neural Networks, IJCNN 2019. Institute of Electrical and Electronics Engineers Inc. 2019. 8852110. (Proceedings of the International Joint Conference on Neural Networks). doi: 10.1109/IJCNN.2019.8852110

Lane Change Decision-making through Deep Reinforcement Learning with Rule-based Constraints

Abstract

Publication series

Conference

Access to Document

Other files and links

Cite this