Real-Time Vehicle Make and Model Recognition with the Residual SqueezeNet Architecture
<p>Some cases that cause failure of the license plate recognition-based MMR systems.</p> "> Figure 2
<p>Flowchart of the typical MMR system.</p> "> Figure 3
<p>Illustration of one convolutional layer.</p> "> Figure 4
<p>Micro-architectural view: Convolution filters organization in the Fire modules.</p> "> Figure 5
<p>Framework for the proposed car detection and clustering method.</p> "> Figure 6
<p>Examples of clustered images.</p> "> Figure 7
<p>Illustration of SqueezeNet-based MMR system. (<b>a</b>) SqueezeNet architecture. (<b>b</b>) Our proposed Residual SqueezeNet architecture.</p> "> Figure 8
<p>Proposed residual Squeeze Net architecture with simple bypass connections.</p> "> Figure 9
<p>(<b>a</b>) Learning rate decay is visualized over training epochs. Here, a step function decay is used, and the learning rate is divided by 10 after one-third and two-thirds of training. (<b>b</b>) Training loss, validation loss and accuracy are plotted over training epochs.</p> "> Figure 10
<p>Number of parameters of SqueezeNet with respect to Fire modules.</p> ">
Abstract
:1. Introduction
2. Background
2.1. General Architecture of Vehicle MMR System
2.2. Convolutional Neural Networks
2.3. SqueezeNet
3. Vehicle Data Clustering and Labelling
3.1. Dataset Clustering
- (1)
- The vehicles are first detected based on frame difference and symmetrical filter. The frame difference method is applied to images by shifting one image with moderate pixels to generate another image. The difference between these two images is used to detect the vehicle by a symmetrical filter, which makes use of the symmetrical structure of the vehicles.
- (2)
- Then, the discriminative and simple features are extracted based on deep learning before using the K-means algorithm. To use deep learning for feature extraction, we introduce a third-party dataset, which may have a lower number of images and car models. The third-party dataset named as “compcars” [26] is labelled and used for training the deep network. The trained model is used to extract features of the unlabeled vehicles. However, the features are still high-dimension for adopting the K-means algorithm. Thus, we use principal component analysis (PCA) to reduce the dimensions.
- (3)
- Finally, the K-means algorithm is used to cluster the data and assign the group to each data. The manual correction of the wrongly clustered data is performed to get a new dataset. This dataset is added to the third-party dataset to train a more powerful deep network. In this sense, the labelling is performed iteratively. For every iteration, we introduce 100,000 images as incremental data.
3.2. Dataset Labelling
4. The Proposed MMR System
4.1. Residual SqueezeNet Architecture
4.2. Hardware and Software
5. Performance Evaluation
5.1. Dataset
5.2. Performance of SqueezeNet and Residual SqueezeNet
5.3. Generalization of New Data and Processing
5.4. Model Size and Number of Parameters
5.5. Comparison of the Proposed Method with State-Of-The-Art Methods
5.6. Performance of SqueezeNet with Respect to Fire Modules
6. Conclusions
Author Contributions
Funding
Conflicts of Interest
References
- Lim, S. Intelligent transport systems in Korea. IJEI Int. J. Eng. Ind. 2012, 3, 58–64. [Google Scholar]
- He, H.; Shao, Z.; Tan, J. Recognition of car makes and models from a single traffic-camera image. IEEE Trans. Intell. Transp. Syst. 2015, 16, 3182–3192. [Google Scholar] [CrossRef]
- Chen, L.; Hsieh, J.; Yan, Y.; Chen, D. Vehicle make and model recognition using sparse representation and symmetrical SURFs. Pattern Recognit. 2015, 48, 1979–1998. [Google Scholar] [CrossRef]
- Llorca, D.; Colás, D.; Daza, I.; Parra, I.; Sotelo, M. Vehicle model recognition using geometry and appearance of car emblems from rear view images. In Proceedings of the 2014 IEEE 17th International Conference on Intelligent Transportation Systems (ITSC), Qingdao, China, 8–11 October 2014; pp. 3094–3099. [Google Scholar]
- Fraz, M.; Edirisinghe, E.A.; Sarfraz, M.S. Mid-level-representation based lexicon for vehicle make and model recognition. In Proceedings of the 2014 22nd International Conference on Pattern Recognition (ICPR), Stockholm, Sweden, 24–28 August 2014; pp. 393–398. [Google Scholar]
- AbdelMaseeh, M.; Badreldin, I.; Abdelkader, M.F.; El Saban, M. Car Make and Model recognition combining global and local cues. In Proceedings of the 2012 21st International Conference on Pattern Recognition (ICPR), Tsukuba, Japan, 11–15 November 2012; pp. 910–913. [Google Scholar]
- Jang, D.; Turk, M. Car-Rec: A real time car recognition system. In Proceedings of the 2011 IEEE Workshop on Applications of Computer Vision (WACV), Kona, HI, USA, 5–7 January 2011; pp. 599–605. [Google Scholar]
- Baran, R.; Rusc, T.; Fornalski, P. A smart camera for the surveillance of vehicles in intelligent transportation systems. Multimed. Tools Appl. 2016, 75, 10471–10493. [Google Scholar] [CrossRef]
- Santos, D.; Correia, P.L. Car recognition based on back lights and rear view features. In Proceedings of the 10th Workshop on Image Analysis for Multimedia Interactive Services, London, UK, 6–8 May 2009; pp. 137–140. [Google Scholar]
- Ren, Y.; Lan, S. Vehicle make and model recognition based on convolutional neural networks. In Proceedings of the 2016 7th IEEE International Conference on Software Engineering and Service Science (ICSESS), Beijing, China, 26–28 August 2016; pp. 692–695. [Google Scholar]
- Dehghan, A.; Masood, S.Z.; Shu, G.; Ortiz, E.G. View Independent Vehicle Make, Model and Color Recognition Using Convolutional Neural Network. arXiv, 2017; arXiv:1702.01721. [Google Scholar]
- Huang, K.; Zhang, B. Fine-grained vehicle recognition by deep Convolutional Neural Network. In Proceedings of the International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI), Datong, China, 15–17 October 2016; pp. 465–470. [Google Scholar]
- Gao, Y.; Lee, H.J. Local tiled deep networks for recognition of vehicle make and model. Sensors 2016, 16, 226. [Google Scholar] [CrossRef] [PubMed]
- Al-Smadi, M.; Abdulrahim, K.; Salam, R.A. Traffic surveillance: A review of vision based vehicle detection, recognition and tracking. Int. J. Appl. Eng. Res. 2016, 11, 713–726. [Google Scholar]
- Hsieh, J.W.; Chen, L.; Chen, D. Symmetrical SURF and its applications to vehicle detection and vehicle make and model recognition. IEEE Trans. Int. Transp. Syst. 2014, 15, 6–20. [Google Scholar] [CrossRef]
- Ma, X.; Grimson, W.E.L. Edge-based rich representation for vehicle classification. In Proceedings of the Tenth IEEE International Conference on Computer Vision (ICCV’05), Beijing, China, 17–21 October 2005; pp. 1185–1192. [Google Scholar]
- Aarathi, K.S.; Abraham, A. Vehicle color recognition using deep learning for hazy images. In Proceedings of the 2017 International Conference on Inventive Communication and Computational Technologies (ICICCT), Coimbatore, India, 10–11 March 2017; pp. 335–339. [Google Scholar]
- Biglari, M.; Soleimani, A.; Hassanpour, H. A Cascaded Part-Based System for Fine-Grained Vehicle Classification. IEEE Trans. Intell. Transp. Syst. 2018, 19, 273–283. [Google Scholar] [CrossRef]
- Girshick, R. Fast R-CNN. In Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV’15), Santiago, Chile, 7–13 December 2015; pp. 1440–1448. [Google Scholar]
- Ren, S.; He, K.; Girshick, R.; Sun, J. Faster R-CNN: Towards real-time object detection with region proposal networks. In Proceedings of the 28th International Conference on Neural Information Processing Systems (NIPS’15), Montreal, QC, Canada, 7–12 December 2015; pp. 91–99. [Google Scholar]
- Redmon, J.; Divvala, S.; Girshick, R.; Farhadi, A. You Only Look Once: Unified, Real-Time Object Detection. In Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 27–30 June 2016. [Google Scholar]
- Hinton, G.E.; Salakhutdinov, R.R. Reducing the Dimensionality of Data with Neural Networks. Science 2006, 313, 504–507. [Google Scholar] [CrossRef] [PubMed]
- Alex, K.; Sutskever, I.; Hinton, G.E. ImageNet classification with deep convolutional neural networks. In Proceedings of the Advances in Neural Information Processing Systems, Lake Tahoe, NV, USA, 3–6 December 2012. [Google Scholar]
- Szegedy, C.; Liu, W.; Jia, Y. Going Deeper with Convolutions. Available online: http://arxiv.org/abs/1409.4842 (accessed on 17 September 2014).
- Iandola, F.N.; Han, S.; Moskewicz, M.W.; Ashraf, K.; Dally, W.J.; Keutzer, K. SqueezeNet: AlexNet-level accuracy with 50× fewer parameters and <0.5 MB model size. arXiv, 2016; arXiv:1602.07360. [Google Scholar]
- Yang, L.; Luo, P.; Loy, C.C.; Tang, X. A large-scale car dataset for fine-grained categorization and verification. In Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Boston, MA, USA, 7–12 June 2015; pp. 3973–3981. [Google Scholar]
- Pearce, G.; Pears, N. Automatic make and model recognition from frontal images of cars. In Proceedings of the 2011 8th IEEE International Conference on Advanced Video and Signal-Based Surveillance (AVSS), Klagenfurt, Austria, 30 August–2 September 2011; pp. 373–378. [Google Scholar]
- Psyllos, A.; Anagnostopoulos, C.N.; Kayafas, E. Vehicle model recognition from frontal view image measurements. Comput. Stand. Interfaces 2011, 33, 142–151. [Google Scholar] [CrossRef]
- Psyllos, A.; Anagnostopoulos, C.N.; Kayafas, E. SIFT-based measurements for vehicle model recognition. In Proceedings of the XIX IMEKO World Congress Fundamental and Applied Metrology, Lisbon, Portugal, 6−11 September 2009. [Google Scholar]
- Siddiqui, A.J.; Mammeri, A.; Boukerche, A. Real-time vehicle make and model recognition based on a bag of surf features. IEEE Trans. Intell. Transp. Syst. 2016, 17, 3205–3219. [Google Scholar] [CrossRef]
- Fang, J.; Zhou, Y.; Yu, Y.; Du, S. Fine-grained vehicle model recognition using a coarse-to-fine convolutional neural network architecture. IEEE Trans. Intell. Transp. Syst. 2017, 18, 1782–1792. [Google Scholar] [CrossRef]
ID | Make | Model | Type | ID | Make | Model | Type |
---|---|---|---|---|---|---|---|
1 | Hyundai | Grand-Starex | MiniBus | 6 | Kia | Grand-Carnival | SumoCar |
2 | Hyundai | Starex-2006-Model | MiniBus | 7 | Volkswagen | New-CC | Sedan |
3 | Chevrolet | 25 Tons-Cargo-Truck | Truck | 8 | Samsung | QM3 | Sedan |
4 | Kia | Sorento | SumoCar | 9 | Hyundai | Porter-2 | MiniTruck |
5 | Hyundai | The-Luxury-Grandeur | Sedan | 10 | Hyundai | Avante-Hybrid | Sedan |
Architecture | No. of Classes | No. of Training Samples | No. of Test Samples | Rank-1 Accuracy | Rank-5 Accuracy | Loss |
---|---|---|---|---|---|---|
SqueezeNet | 766 | 233,280 | 58,322 | 94.23% | 99.38% | 0.0516 |
Proposed Residual SqueezeNet | 766 | 233,280 | 58,322 | 96.33% | 99.52% | 0.0397 |
Architecture | Correctly Recognized Vehicles (out of 112) | Per Vehicle Recognition Time (ms) |
---|---|---|
SqueezeNet | 111 | 108.81 |
Proposed Residual SqueezeNet | 111 | 109.25 |
AlexNet [23] | GoogLeNet [23] | SqueezeNet | Proposed Residual SqueezeNet | |
---|---|---|---|---|
No. of Parameters | 59,983,292 | 6,752,430 | 1,118,974 | 1,118,974 |
Model Size (MB) | 229.0 | 49.4 | 4.4 | 4.4 |
Methods | Classes | No. of Samples | Rank-1 Accuracy (%) | Recognition Time (ms) |
---|---|---|---|---|
He et al. [2] | 30 | 1196 | 92.47 | 500.0 |
Llorca et al. [4] | 52 | 1342 | 94.00 | - |
Pearce et al. [27] | 74 | 262 | 96.00 | 432.5 |
Psyllos et al. [28] | 11 | 110 | 85.00 | 363.8 |
Psyllos et al. [29] | 10 | 400 | 92.00 | 913.0 |
Siddiqui et al. [30] | 29 | 6601 | 94.84 | 137.9 |
Fang et al. [31] | 281 | 44,481 | 98.63 | - |
Proposed Method | 766 | 291,602 | 96.33 | 109.5 |
Parameter | AlexNet [23] | GoogleNet [23] | SqueezeNet | Proposed Residual SqueezeNet |
---|---|---|---|---|
Rank-1 Accuracy (%) | 93.57 | 94.31 | 94.23 | 96.33 |
Rank-5 Accuracy (%) | 99.02 | 99.46 | 99.38 | 99.52 |
Loss | 0.0835 | 0.1259 | 0.0516 | 0.0397 |
Parameters (K) | 59,983 | 6752 | 1118 | 1118 |
Size (KB) | 229,000 | 49,446 | 4379 | 4379 |
Classification Time (ms) | 294.71 | 531.40 | 108.81 | 109.54 |
No. of Fire Modules | Training Loss | Rank-1 Accuracy (%) | Rank-5 Accuracy (%) |
---|---|---|---|
10 | 0.2044 | 93.38 | 99.17 |
9 | 0.1725 | 93.56 | 99.18 |
8 | 0.0516 | 94.23 | 99.38 |
6 | 0.1765 | 93.58 | 99.14 |
4 | 0.1367 | 93.61 | 99.19 |
© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Lee, H.J.; Ullah, I.; Wan, W.; Gao, Y.; Fang, Z. Real-Time Vehicle Make and Model Recognition with the Residual SqueezeNet Architecture. Sensors 2019, 19, 982. https://doi.org/10.3390/s19050982
Lee HJ, Ullah I, Wan W, Gao Y, Fang Z. Real-Time Vehicle Make and Model Recognition with the Residual SqueezeNet Architecture. Sensors. 2019; 19(5):982. https://doi.org/10.3390/s19050982
Chicago/Turabian StyleLee, Hyo Jong, Ihsan Ullah, Weiguo Wan, Yongbin Gao, and Zhijun Fang. 2019. "Real-Time Vehicle Make and Model Recognition with the Residual SqueezeNet Architecture" Sensors 19, no. 5: 982. https://doi.org/10.3390/s19050982