With the development of the economy, urban traffic congestion has become increasingly serious, causing a series of problems such as safety and environmental pollution. To address the challenge of urban traffic congestion problems, traffic signal configuration optimization based on time-varying traffic states and real-time performance is an important direction with practical engineering significance. In this study, we design a traffic signal controller that learns online without requiring a traffic model. A model-free action-dependent adaptive dynamic programming (ADP) that employs two neural networks (an action network and a critic network) to approximate the Hamilton–Jacobi–Bellman equation is employed to provide self-learning optimization ability with varying traffic states. The proposed controller employs a neuro-fuzzy system that functions as an action network to generate control decisions utilizing expertise and mitigates the stochastic exploration inefficiency inherent in conventional ADP. ADP provides reinforcement signals that indicate a reward or punishment for the neuro-fuzzy system to guide it in adjusting its parameters. Subsequently, actions associated with lower cumulative delay are reinforced. The proposed traffic signal controller can reduce the blindness of learning by using experience and the inaccuracy of the traffic model. An artificial bee colony algorithm was employed to train the ADP to meet the real-time requirement for traffic signal control. The controller can adapt to fluctuating traffic states by training continuously, and achieves a smaller average delay in the long run. The simulation results demonstrate that proposed controller achieves a reduced delay through supervised learning with an accelerated training speed.

1. [1] H. Wei, G. Zheng, V. Gayah, and Z. Li, “Recent advances in reinforcement learning for traffic signal control: A survey of models and evaluation,” ACM SIGKDD Explorations Newsletter, Vol.22, No.2, pp. 12-18, 2021. https://doi.org/10.1145/3447556.3447565
2. [2] B. Xu, Y. Wang, Z. Wang, H. Jia, and Z. Lu, “Hierarchically and cooperatively learning traffic signal control,” Proc. of the AAAI Conf. on Artificial Intelligence, Vol.35, No.1, pp. 669-677, 2021. https://doi.org/10.1609/aaai.v35i1.16147
3. [3] Y. Tang, X. Dai, and Y. Lv, “Large language model-assisted arterial traffic signal control,” IEEE J. of Radio Frequency Identification, Vol.8, pp. 322-326, 2024. https://doi.org/10.1109/jrfid.2024.3384289
4. [4] W. Wei and M. Wang, “Research on traffic signal control based on intelligence techniques,” Proc. of the 2003 IEEE Int. Conf. on Intelligent Transportation Systems, Vol.1, pp. 892-896, 2003. https://doi.org/10.1109/itsc.2003.1252078
5. [5] L. Huang and X. Qu, “Improving traffic signal control operations using proximal policy optimization,” IET Intelligent Transport Systems, Vol.17, No.3, pp. 592-605, 2023. https://doi.org/10.1049/itr2.12286
6. [6] C.-Y. Jiang, X.-M. Hu, and W.-N. Chen, “An urban traffic signal control system based on traffic flow prediction,” 2021 13th Int. Conf. on Advanced Computational Intelligence, pp. 259-265, 2021. https://doi.org/10.1109/icaci52617.2021.9435905
7. [7] F. C. Wang, “Design and implementation of intelligent traffic signal light control system based on LoRa technology,” Applied and Computational Engineering, Vol.117, pp. 107-113, 2025. https://doi.org/10.54254/2755-2721/2025.20314
8. [8] Y. Zhang et al., “An occupancy-based adaptive signal control for a congested signalized intersection in the low CV penetration environment,” J. of Advanced Transportation, Vol.2022, Article No.4745879, 2022. https://doi.org/10.1155/2022/4745879
9. [9] W. Wei, Y. Zhang, J. B. Mbede, Z. Zhang, and J. Song, “Traffic signal control using fuzzy logic and MOGA,” 2001 IEEE Int. Conf. on Systems, Man and Cybernetics, e-Systems and e-Man for Cybernetics in Cyberspace, Vol.2, pp. 1335-1340, 2001. https://doi.org/10.1109/ICSMC.2001.973106
10. [10] Y. Wu, “Enhancing urban traffic flow through fuzzy logic-based signal light control optimization,” Int. J. of e-Collaboration, Vol.20, No.1, 2024. https://doi.org/10.4018/ijec.358746
11. [11] B. A. Khan and N. S. Lai, “An advanced fuzzy logic based traffic controller,” Int. J. of Innovation in the Digital Economy, Vol.5, No.1, pp. 31-40, 2014. https://doi.org/10.4018/ijide.2014010103
12. [12] T. Meepokgit and S. Wisayataksin, “Traffic signal control with state-optimizing deep reinforcement learning and fuzzy logic,” Applied Sciences, Vol.14, No.17, Article No.7908, 2024. https://doi.org/10.3390/app14177908
13. [13] E. Bingham, “Reinforcement learning in neurofuzzy traffic signal control,” European J. of Operational Research, Vol.131, No.2, pp. 232-241, 2001. https://doi.org/10.1016/s0377-2217(00)00123-5
14. [14] M. C. Choy, D. Srinivasan, and R. L. Cheu, “Cooperative, hybrid agent architecture for real-time traffic signal control,” IEEE Trans. on Systems, Man, and Cybernetics – Part A: Systems and Humans, Vol.33, No.5, pp. 597-607, 2003. https://doi.org/10.1109/tsmca.2003.817394
15. [15] Ö. N. Kenger and E. Özceylan, “Fuzzy min–max neural networks: A bibliometric and social network analysis,” Neural Computing and Applications, Vol.35, No.7, pp. 5081-5111, 2023. https://doi.org/10.1007/s00521-023-08267-9
16. [16] C. Wang, X. Lv, M. Shao, Y. Qian, and Y. Zhang, “A novel fuzzy hierarchical fusion attention convolution neural network for medical image super-resolution reconstruction,” Information Sciences, Vol.622, pp. 424-436, 2023. https://doi.org/10.1016/j.ins.2022.11.140
17. [17] J. Dai et al., “A fuzzy adaptive zeroing neural network model with event-triggered control for time-varying matrix inversion,” IEEE Trans. on Fuzzy Systems, Vol.31, No.11, pp. 3974-3983, 2023. https://doi.org/10.1109/tfuzz.2023.3272653
18. [18] H. Zhao, C. Dong, J. Cao, and Q. Chen, “A survey on deep reinforcement learning approaches for traffic signal control,” Engineering Applications of Artificial Intelligence, Vol.133, Part A, Article No.108100, 2024. https://doi.org/10.1016/j.engappai.2024.108100
19. [19] X. Du et al., “FELight: Fairness-aware traffic signal control via sample-efficient reinforcement learning,” IEEE Trans. on Knowledge and Data Engineering, Vol.36, No.9, pp. 4678-4692, 2024. https://doi.org/10.1109/TKDE.2024.3376745
20. [20] H. Jiang et al., “A general scenario-agnostic reinforcement learning for traffic signal control,” IEEE Trans. on Intelligent Transportation Systems, Vol.25, No.9, pp. 11330-11344, 2024. https://doi.org/10.1109/tits.2024.3377106
21. [21] L. Kuang, J. Zheng, K. Li, and H. Gao, “Intelligent traffic signal control based on reinforcement learning with state reduction for smart cities,” ACM Trans. on Internet Technology, Vol.21, No.4, Article No.102, 2021. https://doi.org/10.1145/3418682
22. [22] M. Raeis and A. Leon-Garcia, “A deep reinforcement learning approach for fair traffic signal control,” arXiv:2107.10146, 2021. https://doi.org/10.48550/arXiv.2107.10146
23. [23] Z. Zhao, K. Wang, Y. Wang, and X. Liang, “Enhancing traffic signal control with composite deep intelligence,” Expert Systems with Applications, Vol.244, Article No.123020, 2024. https://doi.org/10.1016/j.eswa.2023.123020
24. [24] A. R. M. Jamil, K. K. Ganguly, and N. Nower, “Adaptive traffic signal control system using composite reward architecture based deep reinforcement learning,” IET Intelligent Transport Systems, Vol.14, No.14, pp. 2030-2041, 2020. https://doi.org/10.1049/iet-its.2020.0443
25. [25] H. Kamal, W. Yánez, S. Hassan, and D. Sobhy, “Digital-twin-based deep reinforcement learning approach for adaptive traffic signal control,” IEEE Internet of Things J., Vol.11, No.12, pp. 21946-21953, 2024. https://doi.org/10.1109/jiot.2024.3377600
26. [26] P. J. Werbos, “Approximate dynamic programming for real-time control and neural modeling,” D. A. White and D. A. Sofge (Eds.), “Handbook of Intelligent Control: Neural, Fuzzy, and Adaptive Approaches,” pp. 493-525, Van Nostrand Reinhold, 1992.
27. [27] D. Wang, N. Gao, D. Liu, J. Li, and F. L. Lewis, “Recent progress in reinforcement learning and adaptive dynamic programming for advanced control applications,” IEEE/CAA J. of Automatica Sinica, Vol.11, No.1, pp. 18-36, 2024. https://doi.org/10.1109/JAS.2023.123843
28. [28] C. Cai, “An approximate dynamic programming strategy for responsive traffic signal control,” 2007 IEEE Int. Symp. on Approximate Dynamic Programming and Reinforcement Learning, pp. 303-310, 2007. https://doi.org/10.1109/ADPRL.2007.368203
29. [29] T. Li, D. Zhao, and J. Yi, “Application of ADP to intersection signal control,” Proc. of the 4th Int. Symp. on Neutral Networks, Part 1, pp. 374-379, 2007. https://doi.org/10.1007/978-3-540-72383-7_44
30. [30] D. Karaboga, “An idea based on honey bee swarm for numerical optimization,” Technical Report-TR 06, Erciyes University, 2005.