File(s) under permanent embargo
Urban Traffic Signal Control with Reinforcement Learning from Demonstration Data
conference contribution
posted on 2023-02-21, 22:15 authored by M Wang, L Wu, Jianxin LiJianxin Li, D Wu, C MaReinforcement learning has been applied to various decision-making tasks and has achieved high profile successes. More and more studies have proposed to use reinforcement learning (RL) for traffic signal control to improve transportation efficiency. However, these methods suffer from a major exploration problem, and their performance is particularly poor. And even fail to quickly converge during the initial stage when interacting with the environment. To overcome this problem, we propose an RL model for traffic signal control based on demonstration data, which provides prior expert knowledge before RL model training. The demonstrations are collected from the classic method self-organizing traffic light (SOTL). It not only serves as expert knowledge but also explores and improves the entire decision-making system. Specifically, we use small demonstration data sets to pre-train the Ape-X Deep Q-learning Network (DQ N) for traffic signal control. When training a RL model from scratch, we often need a lot of data and time to learn a better initialization. Our approach is dedicated to making the RL algorithm converge quickly and accelerating the pace of learning. Extensive experiments on three urban datasets confirm that our method performs better with faster convergence and least travel time than the current RL-based methods by an average of 23.9%, 23.8%, 11.6%
History
Volume
2022-JulyLocation
ITALY, PaduaPublisher DOI
Start date
2022-07-18End date
2022-07-23ISSN
2161-4393ISBN-13
9781728186719Language
EnglishPublication classification
E1 Full written paper - refereedTitle of proceedings
Proceedings of the International Joint Conference on Neural NetworksEvent
IEEE International Conference on Fuzzy Systems (FUZZ-IEEE) / IEEE World Congress on Computational Intelligence (IEEE WCCI) / International Joint Conference on Neural Networks (IJCNN) / IEEE Congress on Evolutionary Computation (IEEE CEC)Publisher
IEEESeries
IEEE International Joint Conference on Neural Networks (IJCNN)Usage metrics
Categories
No categories selectedKeywords
Science & TechnologyTechnologyLife Sciences & BiomedicineComputer Science, Artificial IntelligenceComputer Science, Hardware & ArchitectureEngineering, Electrical & ElectronicNeurosciencesComputer ScienceEngineeringNeurosciences & Neurologytraffic signal controlreinforcement learningdemonstrations learningApe-X DQNtransportation efficiencyBasic Behavioral and Social ScienceBehavioral and Social Science
Licence
Exports
RefWorks
BibTeX
Ref. manager
Endnote
DataCite
NLM
DC