Lateral and Longitudinal Driving Behavior Prediction Based on Improved Deep Belief Network
<p>Proposed driving behavior prediction system.</p> "> Figure 2
<p>The typical DBN driving behavior prediction architecture.</p> "> Figure 3
<p>Improved MSR-DBN prediction model.</p> "> Figure 4
<p>Schematic diagram for the RBM and training process for the pre-training.</p> "> Figure 5
<p>Data acquisition route.</p> "> Figure 6
<p>Prediction errors for surrounding vehicles based on different methods.</p> "> Figure 7
<p>Prediction results of the front wheel angle based on different methods.</p> "> Figure 8
<p>Prediction results of the speed based on different methods.</p> "> Figure 9
<p>Prediction results on highD dataset. (<b>a</b>) Prediction result of lateral speed. (<b>b</b>) Prediction result of longitudinal speed.</p> ">
Abstract
:1. Introduction
- Developing a general prediction system, which allows us to consider real-world data including states of surrounding vehicles and the ego vehicle and the driver’s control inputs simultaneously to predict the driving behavior in an end-to-end way;
- Proposing a systematic testing method to obtain optimal parameters of the prediction model;
- Proposing an MSR-DBN prediction model with a multi-target sigmoid regression layer to realize coupled optimization for lateral and longitudinal behavior prediction.
2. Driving Behavior Prediction Model
2.1. MSR-DBN Prediction Model
2.2. Training Procedure
- Initialization—Initialize the weights and biases and preprocess training and testing data;
- Pre-training—Train the shared layers and independent layers for the feature extraction to obtain the model parameters based on two sub-networks (i.e., Step 1 in Figure 3);
- Fine tuning—Obtain the initial prediction based on a multi-target sigmoid regression layer, and utilize a back propagation to fine tune the final prediction model (i.e., Step 2 in Figure 3);
- Prediction—Predict driving behavior and output for the ego vehicle.
2.2.1. Pre-Training Procedure
2.2.2. Fine-Tuning Procedure
3. Data Collection and Data Processing
3.1. Data Collection
3.1.1. Data Acquisition
3.1.2. Data Preprocessing
3.2. Performance Evaluation
3.3. Model Input Selection
3.3.1. State Prediction of Surrounding Vehicles
3.3.2. Different Historical Data
4. Results
4.1. Experiment of DBN Structure
4.1.1. Learning Rates
4.1.2. Hidden Layers
4.1.3. Hidden Nodes
4.2. Structure of MSR-DBN
4.3. Result Analysis and Comparison
4.4. Generalization Performance Analysis
5. Discussion
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Olabiyi, O.; Martinson, E.; Chintalapudi, V.; Guo, R. Driver Action Prediction Using Deep (Bidirectional) Recurrent Neural Network. arXiv 2017, arXiv:1706.02257. [Google Scholar]
- Sun, L.; Yin, Y. Discovering themes and trends in transportation research using topic modeling. Transp. Res. Part C Emerg. Technol. 2017, 77, 49–66. [Google Scholar] [CrossRef] [Green Version]
- Meyer, J.; Becker, H.; Bösch, P.M.; Axhausen, K.W. Autonomous vehicles: The next jump in accessibilities? Res. Transp. Econ. 2017, 62, 80–91. [Google Scholar] [CrossRef] [Green Version]
- Lu, C.; Wang, H.; Lv, C.; Gong, J.; Xi, J.; Cao, D. Learning Driver-specific Behaviour for Overtaking: A Combined Learning Framework. IEEE Trans. Veh. Technol. 2018, 67, 6788–6802. [Google Scholar] [CrossRef]
- Schwehr, J.; Willert, V. Driver’s gaze prediction in dynamic automotive scenes. In Proceedings of the 2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC), Yokohama, Japan, 16–19 October 2017; pp. 1–8. [Google Scholar]
- Lee, D.; Kwon, Y.P.; McMains, S.; Hedrick, J.K. Convolution neural network-based lane change intention prediction of surrounding vehicles for ACC. In Proceedings of the 2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC), Yokohama, Japan, 16–19 October 2017; pp. 1–6. [Google Scholar]
- Li, X.; Wang, W.; Roetting, M. Estimating Driver’s Lane-Change Intent Considering Driving Style and Contextual Traffic. IEEE Trans. Intell. Transp. Syst. 2018, 20, 3258–3271. [Google Scholar] [CrossRef]
- Wang, W.; Xi, J.; Zhao, D. Learning and inferring a driver’s braking action in car-following scenarios. IEEE Trans. Veh. Technol. 2018, 67, 3887–3899. [Google Scholar] [CrossRef] [Green Version]
- Bocklisch, F.; Bocklisch, S.F.; Beggiato, M.; Krems, J.F. Adaptive fuzzy pattern classification for the online detection of driver lane change intention. Neurocomputing 2017, 262, 148–158. [Google Scholar] [CrossRef]
- Liu, Q.; Xu, S.; Lu, C.; Yao, H.; Chen, H. Early Recognition of Driving Intention for Lane Change Based on Recurrent Hidden Semi-Markov Model. IEEE Trans. Veh. Technol. 2020, 69, 10545–10557. [Google Scholar] [CrossRef]
- Maye, J.; Triebel, R.; Spinello, L.; Siegwart, R. Bayesian on-line learning of driving behaviors. In Proceedings of the Robotics and Automation (ICRA), 2011 IEEE International Conference, Shanghai, China, 9–13 May 2011; pp. 4341–4346. [Google Scholar]
- Sun, L.; Chen, X.; He, Z.; Miranda-Moreno, L.F. Pattern Discovery and Anomaly Detection of Individual Travel Behavior Using License Plate Recognition Data. In Proceedings of the Transportation Research Board 98th Annual Meeting, Washington, DC, USA, 13–17 January 2019. [Google Scholar]
- Dou, Y.; Yan, F.; Feng, D. Lane changing prediction at highway lane drops using support vector machine and artificial neural network classifiers. In Proceedings of the 2016 IEEE International Conference on Advanced Intelligent Mechatronics (AIM), Banff, AB, Canada, 12–15 July 2016; pp. 901–906. [Google Scholar]
- Wang, W.; Zhao, D.; Han, W.; Xi, J. A learning-based approach for lane departure warning systems with a personalized driver model. IEEE Trans. Veh. Technol. 2018, 67, 9145–9157. [Google Scholar] [CrossRef] [Green Version]
- Kumagai, T.; Akamatsu, M. Prediction of human driving behavior using dynamic Bayesian networks. IEICE Trans. Inf. Syst. 2006, 89, 857–860. [Google Scholar] [CrossRef]
- Li, J.; Dai, B.; Li, X.; Xu, X.; Liu, D. A dynamic bayesian network for vehicle maneuver prediction in highway driving scenarios: Framework and verification. Electronics 2019, 8, 40. [Google Scholar] [CrossRef] [Green Version]
- Yuan, W.; Li, Z.; Wang, C. Lane-change prediction method for adaptive cruise control system with hidden Markov model. Adv. Mech. Eng. 2018, 10. [Google Scholar] [CrossRef] [Green Version]
- Peng, J.; Guo, Y.; Fu, R.; Yuan, W.; Wang, C. Multi-parameter prediction of drivers’ lane-changing behaviour with neural network model. Appl. Ergon. 2015, 50, 207–217. [Google Scholar] [CrossRef] [PubMed]
- Chen, X.; Chi, R.; Wang, J.; Pang, C.; Xiao, Q. Driver safe speed model based on BP neural network for rural curved roads. In Proceedings of the International Conference on Computer and Computing Technologies in Agriculture, Beijing, China, 29–31 October 2011; pp. 92–102. [Google Scholar]
- Leonhardt, V.; Wanielik, G. Neural network for lane change prediction assessing driving situation, driver behavior and vehicle movement. In Proceedings of the Intelligent Transportation Systems (ITSC), 2017 IEEE 20th International Conference, Yokohama, Japan, 16–19 October 2017; pp. 1–6. [Google Scholar]
- Yang, S.; Wang, W.; Liu, C.; Deng, W. Scene Understanding in Deep Learning-Based End-to-End Controllers for Autonomous Vehicles. IEEE Trans. Syst. Man, Cybern. Syst. 2019, 49, 53–63. [Google Scholar] [CrossRef]
- Wang, L.; Liu, X.m.; Zhong, X.m. A car-following model based on artificial neural networks in urban expressway sections. In Proceedings of the 85th TRB Annual Meeting, Transportation Research Board, National Research Council, Washington, DC, USA, 22–26 January 2006. [Google Scholar]
- Dogan, Ü.; Edelbrunner, J.; Iossifidis, I. Autonomous driving: A comparison of machine learning techniques by means of the prediction of lane change behavior. In Proceedings of the 2011 IEEE International Conference on Robotics and Biomimetics, Karon Beach, Thailand, 7–11 December 2011; pp. 1837–1843. [Google Scholar]
- Lv, Y.; Duan, Y.; Kang, W.; Li, Z.; Wang, F. Traffic flow prediction with big data: A deep learning approach. IEEE Trans. Intell. Transp. Syst. 2015, 16, 865–873. [Google Scholar] [CrossRef]
- Polson, N.G.; Sokolov, V.O. Deep learning for short-term traffic flow prediction. Transp. Res. Part C Emerg. Technol. 2017, 79, 1–17. [Google Scholar] [CrossRef] [Green Version]
- Lemieux, J.; Ma, Y. Vehicle speed prediction using deep learning. In Proceedings of the Vehicle Power and Propulsion Conference (VPPC), Montreal, QC, Canada, 19–22 October 2015; pp. 1–5. [Google Scholar]
- Patel, S.; Griffin, B.; Kusano, K.; Corso, J.J. Predicting Future Lane Changes of Other Highway Vehicles using RNN-based Deep Models. arXiv 2018, arXiv:1801.04340. [Google Scholar]
- Ciregan, D.; Meier, U.; Schmidhuber, J. Multi-column deep neural networks for image classification. In Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition, Providence, RI, USA, 16–21 June 2012; pp. 3642–3649. [Google Scholar]
- Krizhevsky, A.; Sutskever, I.; Hinton, G.E. Imagenet classification with deep convolutional neural networks. Adv. Neural Inf. Process. Syst. 2012, 25, 1097–1105. [Google Scholar] [CrossRef]
- Li, S.Z.; Yu, B.; Wu, W.; Su, S.Z.; Ji, R.R. Feature learning based on SAE–PCA network for human gesture recognition in RGBD images. Neurocomputing 2015, 151, 565–573. [Google Scholar] [CrossRef]
- Liu, H.L.; Taniguchi, T.; Tanaka, Y.; Takenaka, K.; Bando, T. Visualization of Driving Behavior Based on Hidden Feature Extraction by Using Deep Learning. IEEE Trans. Intell. Transp. Syst. 2017, 18, 2477–2489. [Google Scholar] [CrossRef]
- Chi, L.; Mu, Y. Deep Steering: Learning End-to-End Driving Model from Spatial and Temporal Visual Cues. arXiv 2017, arXiv:1708.03798. [Google Scholar]
- Li, L.; Ota, K.; Dong, M. Human-Like Driving: Empirical Decision-Making System for Autonomous Vehicles. IEEE Trans. Veh. Technol. 2018, 67, 6814–6823. [Google Scholar] [CrossRef] [Green Version]
- Mänttäri, J.; Folkesson, J.; Ward, E. Learning to Predict Lane Changes in Highway Scenarios Using Dynamic Filters On a Generic Traffic Representation. In Proceedings of the 2018 IEEE Intelligent Vehicles Symposium (IV), Changshu, China, 26–30 June 2018; pp. 1385–1392. [Google Scholar] [CrossRef]
- You, C.W.; Montes-de Oca, M.; Bao, T.J.; Lane, N.D.; Lu, H.; Cardone, G.; Torresani, L.; Campbell, A.T. CarSafe: A driver safety app that detects dangerous driving behavior using dual-cameras on smartphones. In Proceedings of the 2012 ACM Conference on Ubiquitous Computing, Pittsburgh, PA, USA, 5–8 September 2012; pp. 671–672. [Google Scholar]
- Romera, E.; Bergasa, L.M.; Arroyo, R. A real-time multi-scale vehicle detection and tracking approach for smartphones. In Proceedings of the Intelligent Transportation Systems (ITSC), 2015 IEEE 18th International Conference, Gran Canaria, Spain, 1–15 September 2015; pp. 1298–1303. [Google Scholar]
- Kaplan, S.; Guvensan, M.A.; Yavuz, A.G.; Karalurt, Y. Driver behavior analysis for safe driving: A survey. IEEE Trans. Intell. Transp. Syst. 2015, 16, 3017–3032. [Google Scholar] [CrossRef]
- Sharma, A.; Kumar, D. Non-image Data Classification with Convolutional Neural Networks. arXiv 2020, arXiv:2007.03218. [Google Scholar]
- He, Z.; Hu, J.; Park, B.B.; Levin, M.W. Vehicle Sensor Data-Based Transportation Research: Modeling, Analysis, and Management. J. Intell. Transp. Syst. 2019, 23, 99–102. [Google Scholar] [CrossRef]
- Su, S.; Muelling, K.; Dolan, J.; Palanisamy, P.; Mudalige, P. Learning Vehicle Surrounding-aware Lane-changing Behavior from Observed Trajectories. In Proceedings of the 2018 IEEE Intelligent Vehicles Symposium (IV), Changshu, China, 26–30 June 2018; pp. 1412–1417. [Google Scholar] [CrossRef]
- Huang, W.; Song, G.; Hong, H.; Xie, K. Deep Architecture for Traffic Flow Prediction: Deep Belief Networks With Multitask Learning. IEEE Trans. Intell. Transp. Syst. 2014, 15, 2191–2201. [Google Scholar] [CrossRef]
- Lee, H.; Pham, P.; Largman, Y.; Ng, A.Y. Unsupervised feature learning for audio classification using convolutional deep belief networks. Adv. Neural Inf. Process. Syst. 2009, 22, 1096–1104. [Google Scholar]
- Bengio, Y. Learning deep architectures for AI. Found. Trends® Mach. Learn. 2009, 2, 1–127. [Google Scholar] [CrossRef]
- Huang, H.B.; Huang, X.R.; Li, R.X.; Lim, T.C.; Ding, W.P. Sound quality prediction of vehicle interior noise using deep belief networks. Appl. Acoust. 2016, 113, 149–161. [Google Scholar] [CrossRef] [Green Version]
- Hinton, G.E.; Osindero, S.; Teh, Y.W. A Fast Learning Algorithm for Deep Belief Nets. Neural Comput. 2014, 18, 1527–1554. [Google Scholar] [CrossRef]
- Le Roux, N.; Bengio, Y. Representational power of restricted Boltzmann machines and deep belief networks. Neural Comput. 2008, 20, 1631–1649. [Google Scholar] [CrossRef]
- Kim, J.; Kang, U.; Lee, Y. Statistics and Deep Belief Network-Based Cardiovascular Risk Prediction. Healthc. Inf. Res. 2017, 23, 169–175. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- LeCun, Y.; Bengio, Y.; Hinton, G. Deep learning. Nature 2015, 521, 436. [Google Scholar] [CrossRef] [PubMed]
- Mao, Y.; Shen, J.; Gui, X. A Study on Deep Belief Net for Branch Prediction. IEEE Access 2017, 6, 10779–10786. [Google Scholar] [CrossRef]
- Wang, L.; Xia, X.; Cao, J.; Liu, X. Modeling and predicting remanufacturing time of equipment using deep belief networks. Clust. Comput. 2017, 22, 2677–2688. [Google Scholar] [CrossRef]
- Lv, Q.; Dou, Y.; Niu, X.; Xu, J.; Xia, F. Remote sensing image classification based on DBN model. J. Comput. Res. Dev. 2014, 51, 1911–1918. [Google Scholar]
- Shen, F.; Chao, J.; Zhao, J. Forecasting exchange rate using deep belief networks and conjugate gradient method. Neurocomputing 2015, 167, 243–253. [Google Scholar] [CrossRef]
- Duan, Y.; Lv, Y.; Liu, Y.; Wang, F. An efficient realization of deep learning for traffic data imputation. Transp. Res. Part C Emerg. Technol. 2016, 72, 168–181. [Google Scholar] [CrossRef]
- Huang, H.B.; Li, R.X.; Yang, M.L.; Lim, T.C.; Ding, W.P. Evaluation of vehicle interior sound quality using a continuous restricted Boltzmann machine-based DBN. Mech. Syst. Signal Process. 2017, 84, 245–267. [Google Scholar] [CrossRef] [Green Version]
- Zhao, C.; Gong, J.; Lu, C.; Xiong, G.; Mei, W. Speed and steering angle prediction for intelligent vehicles based on deep belief network. In Proceedings of the Intelligent Transportation Systems (ITSC), 2017 IEEE 20th International Conference, Yokohama, Japan, 16–19 October 2017; pp. 301–306. [Google Scholar]
- Huang, L.; Xiang, L.Y. Method for Meteorological Early Warning of Precipitation-Induced Landslides Based on Deep Neural Network. Neural Process. Lett. 2018, 48, 1243–1260. [Google Scholar] [CrossRef]
- Zhang, X.L.; Wu, J. Deep Belief Networks Based Voice Activity Detection. IEEE Trans. Audio Speech Lang. Process. 2013, 21, 697–710. [Google Scholar] [CrossRef]
- Krajewski, R.; Bock, J.; Kloeker, L.; Eckstein, L. The highD Dataset: A Drone Dataset of Naturalistic Vehicle Trajectories on German Highways for Validation of Highly Automated Driving Systems. In Proceedings of the 2018 21st International Conference on Intelligent Transportation Systems (ITSC), Maui, HI, USA, 4–7 November 2018; pp. 2118–2125. [Google Scholar] [CrossRef] [Green Version]
Name | Parameter | Units |
---|---|---|
power unit | combustion engine with automatic transmission | - |
travel distance | 48 | km |
travel time | 0.95 | h |
average velocity | 50.27 | km/h |
collection time | 3 to 3.57 (off-peak) | pm |
Type | Surrounding Vehicles∖Errors | ||||||
---|---|---|---|---|---|---|---|
Case 00 | Left front vehicle | 0.1793 | 0.1223 | 2.6915 | 2.2376 | 1.3976 | 0.8935 |
Front vehicle | 0.2865 | 0.2139 | 2.3622 | 1.6879 | 1.2073 | 0.8054 | |
Right front vehicle | 0.2875 | 0.2293 | 3.5747 | 1.9832 | 1.5416 | 1.1863 | |
Case 01 | Left front vehicle | 0.4094 | 0.3794 | 3.9323 | 2.5273 | 0.9838 | 0.7563 |
Front vehicle | 0.1396 | 0.1036 | 3.6175 | 3.0658 | 0.5568 | 0.4278 | |
Right front vehicle | 0.4806 | 0.4685 | 2.5734 | 2.0658 | 0.8483 | 0.6464 | |
Case 02 | Left front vehicle | 0.2568 | 0.1527 | 3.6497 | 2.5632 | 1.5037 | 1.0070 |
Front vehicle | 0.2391 | 0.1758 | 9.7533 | 6.8699 | 1.3156 | 0.9809 | |
Right front vehicle | 0.2899 | 0.2175 | 9.8397 | 5.5563 | 1.8377 | 1.2637 | |
Case 03 | Left front vehicle | 0.1814 | 0.1068 | 6.2141 | 4.4135 | 1.8260 | 1.6235 |
Front vehicle | 0.1909 | 0.1352 | 5.1570 | 3.1565 | 2.0950 | 1.7989 | |
Right front vehicle | 0.1901 | 0.1217 | 3.8158 | 2.2171 | 1.5692 | 1.3391 | |
Case 04 | Left front vehicle | 0.2912 | 0.1947 | 11.3866 | 9.5590 | 2.0985 | 1.4397 |
Front vehicle | 0.2587 | 0.2040 | 9.0340 | 8.0025 | 1.5723 | 1.2682 | |
Right front vehicle | 0.2613 | 0.1882 | 5.4806 | 3.4477 | 2.8110 | 2.4085 |
Type∖Errors | |||||
---|---|---|---|---|---|
Case 0 | 0.1935 | 0.1874 | 0.3055 | 0.2364 | 0.0172 |
Case 1 | 0.0946 | 0.0514 | 0.4668 | 0.3920 | 0.0654 |
Case 2 | 0.0323 | 0.0212 | 0.6665 | 0.5054 | 0.0317 |
Case 3 | 0.0617 | 0.0516 | 0.5971 | 0.4851 | 0.0395 |
Case 4 | 0.0279 | 0.0228 | 1.4551 | 1.2334 | 0.0968 |
Learning Rate | |||||
---|---|---|---|---|---|
0.1 | 1.0406 | 1.0118 | 15.1516 | 15.0381 | 2.5666 |
0.3 | 0.1029 | 0.0547 | 0.5516 | 0.4731 | 0.0814 |
0.5 | 0.0939 | 0.0493 | 0.5019 | 0.4335 | 0.0741 |
0.7 | 0.0946 | 0.0514 | 0.4668 | 0.3920 | 0.0654 |
0.9 | 0.0942 | 0.0494 | 0.4166 | 0.3381 | 0.0571 |
Number of RBM | |||||
---|---|---|---|---|---|
1 | 0.0946 | 0.0514 | 0.4668 | 0.3920 | 0.0654 |
2 | 0.0900 | 0.0586 | 0.5763 | 0.4576 | 0.0878 |
3 | 0.1431 | 0.0811 | 0.3858 | 0.2965 | 0.0531 |
4 | 0.1171 | 0.0906 | 1.8569 | 1.5595 | 0.2867 |
5 | 0.1866 | 0.1310 | 1.5647 | 1.2909 | 0.2387 |
6 | 0.1427 | 0.1044 | 1.7684 | 1.4769 | 0.2716 |
Average | 0.1290 | 0.0862 | 1.1032 | 0.9122 | 0.1672 |
Hidden Nodes | |||||
---|---|---|---|---|---|
32 | 0.1160 | 0.0766 | 1.8241 | 1.5283 | 0.2829 |
50 | 0.1279 | 0.0768 | 1.6475 | 1.3799 | 0.2543 |
64 | 0.1212 | 0.0717 | 1.0640 | 0.8687 | 0.1592 |
100 | 0.1431 | 0.0811 | 0.3858 | 0.2965 | 0.0531 |
128 | 0.2090 | 0.1555 | 1.2237 | 0.9832 | 0.1830 |
150 | 0.1426 | 0.0778 | 0.5932 | 0.4585 | 0.0778 |
200 | 0.1751 | 0.1260 | 1.5396 | 1.2659 | 0.2316 |
256 | 0.1614 | 0.1093 | 1.1938 | 0.9601 | 0.1720 |
Average | 0.1495 | 0.0968 | 1.1840 | 0.9676 | 0.1767 |
Methods∖Errors | |||||
---|---|---|---|---|---|
SVR | 0.1256 | 0.0813 | 0.9142 | 0.8447 | 0.1541 |
BP | 0.1277 | 0.0860 | 0.5799 | 0.3684 | 0.0695 |
RBF | 0.1806 | 0.1009 | 1.0403 | 0.6894 | 0.1090 |
0.1431 | 0.3858 | 0.0811 | 0.2965 | 0.0531 | |
MSR-DBN | 0.1165 | 0.0593 | 0.2067 | 0.1626 | 0.0259 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Yang, L.; Zhao, C.; Lu, C.; Wei, L.; Gong, J. Lateral and Longitudinal Driving Behavior Prediction Based on Improved Deep Belief Network. Sensors 2021, 21, 8498. https://doi.org/10.3390/s21248498
Yang L, Zhao C, Lu C, Wei L, Gong J. Lateral and Longitudinal Driving Behavior Prediction Based on Improved Deep Belief Network. Sensors. 2021; 21(24):8498. https://doi.org/10.3390/s21248498
Chicago/Turabian StyleYang, Lei, Chunqing Zhao, Chao Lu, Lianzhen Wei, and Jianwei Gong. 2021. "Lateral and Longitudinal Driving Behavior Prediction Based on Improved Deep Belief Network" Sensors 21, no. 24: 8498. https://doi.org/10.3390/s21248498