Applying Hierarchal Clusters on Deep Reinforcement Learning Controlled Traffic Network

El-Mahalawy, Ahmed; Shouman, Ahmed; El-Sayed, Ayman; Taher, Fady

doi:10.21608/mjeer.2021.146284

Applying Hierarchal Clusters on Deep Reinforcement Learning Controlled Traffic Network

Document Type : Original Article

Authors

Dept. of Computer Science and Engineering Faculty of Electectronic engineering Minufiya University

10.21608/mjeer.2021.146284

Abstract

Traffic congestions is a crucial problem affecting
cities around the globe and they are only getting worse as the
number of vehicles tends to increase significantly. Traffic signal
controllers are considered as the most important mechanism to
control traffic, specifically at intersections, the field of Machine
Learning introduces advanced techniques which can be applied
to provide more flexibility and adaptiveness to traffic control
techniques. Efficient traffic controllers can be designed using a
reinforcement learning (RL) approach but major problems of
following RL approach are, exponential growth in the state and
action spaces and the need for coordination. We use real traffic
data of 65 intersection of the city of Ottawa to build our
simulations and show that, clustering the network using
hierarchal techniques has a great potential in reducing the stateaction
pair significantly and enhance overall traffic
performance.

Keywords

References

REFERENCES
[1] Kuyer, Lior, Shimon Whiteson, Bram Bakker, and Nikos Vlassis. "Multiagent reinforcement learning for urban traffic control using coordination graphs." In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, PP. 656-671, Year 2008.
[2] Labadie, John W. "Advances in Water Resources Systems Engineering In Applications of Machine Learning." Modern Water Resources Engineering. Humana Press, PP. 467-523. Year 2014
[3] El-Tantawy, Samah, Baher Abdulhai, and Hossam Abdelgawad. "Multiagent reinforcement learning for integrated network of adaptive traffic signal controllers (MARLIN-ATSC): methodology and large-scale application on downtown toronto." In: Intelligent Transportation Systems, IEEE Transactions Vol.14 No.3, PP. 1140-1150, Year 2013.
[4] Behrisch, Michael, et al. "SUMO–Simulation of Urban MObility." The Third International Conference on Advances in System Simulation (SIMUL 2011), Barcelona, Spain. 2011.
[5] Carbonell, Jaime G., Ryszard S. Michalski, and Tom M. Mitchell. "An overview of machine learning." In Machine learning, pp. 3-23. Morgan Kaufmann, Year, 1983.
[6] Sutton, Richard S., and Andrew G. Barto. Introduction to reinforcement learning. Vol. 2, No. 4. Cambridge: MIT press, 1998.
[7] Watkins, Christopher JCH, and Peter Dayan. "Q-learning." Machine learning 8, no. 3-4 (1992): 279-292.
[8]
[9] Messer, Carroll J., and Ramanan V. Nageswara. Improved traffic signal coordination strategies for actuated control. No. SWUTC/96/465110-1. In Southwest Region University Transportation Center, Center for Transportation Research, University of Texas, Year, 1996.
[10] Mousavi, Seyed Sajad, Michael Schukat, and Enda Howley. "Traffic light control using deep policy-gradient and value-function-based reinforcement learning." IET Intelligent Transport Systems Vol.11, No. 7 PP.417-423, Year 2017.
[11] Lin, Yilun, Xingyuan Dai, Li Li, and Fei-Yue Wang. "An efficient deep reinforcement learning model for urban traffic control." arXiv preprint arXiv: 1808.01876, Year 2018.
[12] Van der Pol, Elise, and Frans A. Oliehoek. "Coordinated deep reinforcement learners for traffic light control." Proceedings of Learning, Inference and Control of Multi-Agent Systems (at NIPS 2016) Year 2016.
[13] Genders, Wade, and Saiedeh Razavi. "Using a deep reinforcement learning agent for traffic signal control." arXiv preprint arXiv: 1611.01142 Year 2016.
[14] F.Taher et al Comparing different techniques for controlling traffic signals In International Journal on Power Engineering and Energy (IJPEE), Vol.7, No.3, PP. Year 2016
[15] Jain, Anil K., and Richard C. Dubes. Algorithms for clustering data. Vol. 6. PP. Englewood Cliffs, NJ: Prentice hall, 1988.
[16] Chollet F. keras. GitHub repository. https://github. com/fchollet/keras>. Accessed on. 2015;25:2017.
[17] Chen, Hong-Yunn. "TensorFlow–A system for large-scale machine learning." In OSDI, Vol. 16, PP. 265-283, Year. 2016.
[18] Wu, Xiaoping, Shuai Deng, Xiaohong Du, and Jing Ma. "Green-wave traffic theory optimization and analysis. In “World Journal of Engineering and Technology Vol.2, No. 03 Year 2014.