Abstract
This paper proposes a deep reinforcement learning (DRL) approach that dynamically determines the dispatching of bus services at the starting bus stop for a high-frequency bus service line. Most previous studies focus on planning bus timetables in advance based on expected future passenger demand. They often ignore real-time data and are therefore not competent at handling unexpected passenger demand fluctuations. To address this issue, we propose a Spatial-Temporal data driven Dynamic Holding (STDH) approach in this paper to dispatch bus on the fly at any decision granularity, e.g., every minute. Both spatial and temporal information regarding bus fleet and passengers are captured in a newly designed state matrix. STDH further employs a Deep Q-Network (DQN) based learning system to optimize timetabling decisions dynamically. Our DQN features the use of a newly designed self-attention network architecture to facilitate effective processing of spatial-temporal data, enabling DRL to make desirable bus dispatching decisions in accordance with real-time passenger flow. Experiments have been conducted using real-world data collected in Xiamen China. Our experiments show that STDH can effectively learn a control policy to dynamically dispatch bus services in a high-frequency urban line.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Ai, G., Zuo, X., Wu, B., et al.: Deep reinforcement learning based dynamic optimization of bus timetable. arXiv preprint arXiv:2107.07066 (2021)
Alesiani, F., Gkiotsalitis, K.: Reinforcement learning-based bus holding for high-frequency services. In: 21st International Conference on Intelligent Transportation Systems. ITSC, pp. 3162–3168 (2018)
Asgharzadeh, M.A., Shafahi, Y.: Real-time bus-holding control strategy to reduce passenger waiting time. J. Transp. Res. Board 2647(1), 9–16 (2017)
Bartholdi, J.J., Eisenstein, D.D.: A self-coördinating bus route to resist bus bunching. Transp. Res. Part B Methodol. 46(4), 481–491 (2012)
Chen, W., Zhou, K., Chen, C.: Real-time bus holding control on a transit corridor based on multi-agent reinforcement learning. In: 2016 IEEE 19th International Conference on Intelligent Transportation Systems (ITSC), pp. 100–106 (2016)
Esfeh, M.A., Wirasinghe, S.C., Saidi, S., Kattan, L.: Waiting time and headway modelling for urban transit systems - a critical review and proposed approach. Transp. Rev. 41(2), 141–163 (2021)
Fu, L., Yang, X.: Design and implementation of bus-holding control strategies with real-time information. Transp. Res. Rec. 1791(1), 6–12 (2002)
Hasselt, H.V., Guez, A., Silver, D.: Deep reinforcement learning with double q-learning. In: Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, AAAI 2016, pp. 2094–2100. AAAI Press (2016)
Haydari, A., Yilmaz, Y.: Deep reinforcement learning for intelligent transportation systems: a survey. IEEE Trans. Intell. Transp. Syst. PP, 1–22 (2020)
Hickman, M.D.: An analytic stochastic model for the transit vehicle holding problem. Transp. Sci. 35(3), 215–237 (2001)
Ibarra-Rojas, O., Delgado, F., Giesen, R., Muñoz, J.: Planning, operation, and control of bus transport systems: a literature review. Transp. Res. Part B Methodol. 77, 38–75 (2015)
Li, Z., Hensher, D.A.: Performance contributors of bus rapid transit systems: an ordered choice approach. Econ. Anal. Policy 67, 154–161 (2020)
Mnih, V., et al.: Playing atari with deep reinforcement learning (2013)
Ning, L., Li, Y., Zhou, M., Song, H., Dong, H.: A deep reinforcement learning approach to high-speed train timetable rescheduling under disturbances. In: 2019 IEEE Intelligent Transportation Systems Conference (ITSC), pp. 3469–3474 (2019)
Silver, D., et al.: Mastering the game of go without human knowledge. Nature 550(7676), 354–359 (2017)
Tunstall, L., von Werra, L., Wolf, T.: Natural Language Processing with Transformers, Revised Edition. O’Reilly Media. California (2022)
Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems (2017)
Wang, J., Sun, L.: Dynamic holding control to avoid bus bunching: a multi-agent deep reinforcement learning framework. Transp. Res. Part C Emerg. Technol. 116, 102661 (2020)
Xuan, Y., Argote, J., Daganzo, C.F.: Dynamic bus holding strategies for schedule reliability: optimal linear control and performance analysis. Transp. Res. Part B Methodol. 45(10), 1831–1845 (2011)
Yau, K.L.A., Qadir, J., Khoo, H.L., Ling, M.H., Komisarczuk, P.: A survey on reinforcement learning models and algorithms for traffic signal control. ACM Comput. Surv. 50(3), 1–38 (2017)
Zhao, J., Dessouky, M.M., Bukkapatnam, S.T.S.: Optimal slack time for schedule-based transit operations. Transp. Sci. 40, 529–539 (2006)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Zhao, Y., Chen, G., Ma, H., Zuo, X., Ai, G. (2022). Dynamic Bus Holding Control Using Spatial-Temporal Data – A Deep Reinforcement Learning Approach. In: Aziz, H., Corrêa, D., French, T. (eds) AI 2022: Advances in Artificial Intelligence. AI 2022. Lecture Notes in Computer Science(), vol 13728. Springer, Cham. https://doi.org/10.1007/978-3-031-22695-3_46
Download citation
DOI: https://doi.org/10.1007/978-3-031-22695-3_46
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-22694-6
Online ISBN: 978-3-031-22695-3
eBook Packages: Computer ScienceComputer Science (R0)