Decision-Based Fusion for Vehicle Matching
<p>Sample images from the PRIMAVERA dataset.</p> "> Figure 2
<p>Diagram of overall vehicle matching algorithm.</p> "> Figure 3
<p>Two vehicle images and their detected wheels.</p> "> Figure 4
<p>Performance of whole-vehicle matching neural network during training.</p> "> Figure 5
<p>Performance of the vehicle matching network on the validation set with different threshold values.</p> "> Figure 6
<p>Comparison of training set performance during training between wheel-locking and non-wheel-locking preprocessing approaches.</p> "> Figure 7
<p>Comparison of validation set performance during training between wheel-locking and non-wheel-locking preprocessing approaches.</p> "> Figure 8
<p>Performance of wheel matching network.</p> "> Figure 9
<p>Performance of decision fusion by averaging.</p> "> Figure 10
<p>ROC curves comparing the performances of whole-vehicle-only matching, wheels-only matching, and averaging-based decision fusion of the two matching approaches.</p> "> Figure 11
<p>Comparison of the baseline, vehicle, and wheel-network-matching accuracies.</p> "> Figure 12
<p>Comparison of the baseline, vehicle, and wheel-network-matching true positive rates.</p> "> Figure 13
<p>Comparison of the baseline, vehicle, and wheel-network-matching true negative rates.</p> "> Figure 14
<p>Vehicle similarity score = 0.958; wheel similarity scores = 0.01, 0.001.</p> "> Figure 15
<p>Vehicle similarity score = 0.64; wheel similarity scores = 0.02, 0.01.</p> "> Figure 16
<p>Vehicle similarity score = 0.91; wheel similarity scores = 0.01, 0.03.</p> "> Figure 17
<p>Decision fusion network.</p> "> Figure 18
<p>Comparison of baseline, majority vote, soft vote, and fusion network matching accuracy.</p> "> Figure 19
<p>Baseline, majority vote, soft vote, and fusion network ROC curves.</p> ">
Abstract
:1. Introduction
2. Related Work
3. Dataset Description
3.1. Data Collection
3.2. Data Quantity
4. Network Structure
4.1. Problem Formulation
4.2. Vehicle Matching
4.3. Wheel Detection
4.4. Wheel Matching
5. Experimental Analysis
5.1. Vehicle Matching Results
5.2. Wheel Locking
5.3. Wheel Matching Results
5.4. Decision Fusion Network and Results
6. Conclusions and Future Work
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Wang, H.; Sun, S.; Zhou, L.; Guo, L.; Min, X.; Li, C. Local feature-aware siamese matching model for vehicle re-identification. Appl. Sci. 2020, 10, 2474. [Google Scholar] [CrossRef] [Green Version]
- Shen, Y.; Xiao, T.; Li, H.; Yi, S.; Wang, X. Learning deep neural networks for vehicle re-id with visual-spatio-temporal path proposals. In Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy, 22–29 October 2017; pp. 1900–1909. [Google Scholar]
- Wang, Z.; Tang, L.; Liu, X.; Yao, Z.; Yi, S.; Shao, J.; Yan, J.; Wang, S.; Li, H.; Wang, X. Orientation invariant feature embedding and spatial temporal regularization for vehicle re-identification. In Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy, 22–29 October 2017; pp. 379–387. [Google Scholar]
- He, B.; Li, J.; Zhao, Y.; Tian, Y. Part-regularized near-duplicate vehicle re-identification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA, 16–20 June 2019; pp. 3997–4005. [Google Scholar]
- de Oliveira, I.O.; Fonseca, K.V.O.; Minetto, R. A two-stream siamese neural network for vehicle re-identification by using non-overlapping cameras. In Proceedings of the 2019 IEEE International Conference on Image Processing (ICIP), Taipei, Taiwan, 22–25 September 2019; pp. 669–673. [Google Scholar]
- Wei, X.-S.; Zhang, C.L.; Liu, L.; Shen, C.; Wu, J. Coarse-to-fine: A RNN-based hierarchical attention model for vehicle re-identification. In Asian Conference on Computer Vision; Springer: Cham, Switzeland, 2018; pp. 575–591. [Google Scholar]
- Wang, H.; Skau, E.; Krim, H.; Cervone, G. Fusing heterogeneous data: A case for remote sensing and social media. IEEE Trans. Geosci. Remote Sens. 2018, 56, 6956–6968. [Google Scholar] [CrossRef]
- Ghanem, S.; Panahi, A.; Krim, H.; Kerekes, R.A.; Mattingly, J. Information subspace-based fusion for vehicle classification. In Proceedings of the 2018 26th European Signal Processing Conference (EUSIPCO), Rome, Italy, 3–7 September 2018; pp. 1612–1616. [Google Scholar]
- Ghanem, S.; Roheda, S.; Krim, H. Latent Code-Based Fusion: A Volterra Neural Network Approach. arXiv 2021, arXiv:2104.04829. [Google Scholar]
- Xu, L.; Krzyzak, A.; Suen, C.Y. Methods of combining multiple classifiers and their applications to handwriting recognition. IEEE Trans. Syst. Man Cybern. 1992, 22, 418–435. [Google Scholar] [CrossRef] [Green Version]
- Hellwich, O.; Wiedemann, C. Object extraction from high-resolution multisensor image data. In Proceedings of the Third International Conference Fusion of Earth Data, Sophia Antipolis, France, 26–28 January 2000; Volume 115. [Google Scholar]
- Hall, D.L.; Llinas, J. An introduction to multisensor data fusion. Proc. IEEE 1997, 85, 6–23. [Google Scholar] [CrossRef] [Green Version]
- Khaleghi, B.; Khamis, A.; Karray, F.O.; Razavi, N.S. Multisensor data fusion: A review of the state-of-the-art. Inf. Fusion 2013, 14, 28–44. [Google Scholar] [CrossRef]
- Lippman, D. Math in Society; David Lippman: Washington, DC, USA, 2017. [Google Scholar]
- Dempster, A.P. A generalization of Bayesian inference. J. R. Stat. Soc. Ser. B (Methodol.) 1968, 30, 205–232. [Google Scholar] [CrossRef]
- Shafer, G. A Mathematical Theory of Evidence; Princeton University Press: Princeton, NJ, USA, 1976. [Google Scholar]
- Ben Atitallah, S.; Driss, M.; Boulila, W.; Koubaa, A.; Ghézala, H.B. Fusion of convolutional neural networks based on Dempster–Shafer theory for automatic pneumonia detection from chest X-ray images. Int. J. Imaging Syst. Technol. 2021, 32, 658–672. [Google Scholar] [CrossRef]
- Buede, D.M.; Girardi, P. A target identification comparison of Bayesian and Dempster-Shafer multisensor fusion. IEEE Trans. Syst. Man Cybern.-Part A Syst. Hum. 1997, 27, 569–577. [Google Scholar] [CrossRef]
- Hou, T.; Wang, S.; Qin, H. Vehicle matching and recognition under large variations of pose and illumination. In Proceedings of the 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, Miami, FL, USA, 20–25 September 2009; pp. 24–29. [Google Scholar]
- Jelača, V.; Pižurica, A.; Niño-Castañeda, J.O.; Frias-Velazquez, A.; Philips, W. Vehicle matching in smart camera networks using image projection profiles at multiple instances. Image Vis. Comput. 2013, 31, 673–685. [Google Scholar] [CrossRef]
- Zeng, N.; Crisman, J.D. Vehicle matching using color. In Proceedings of the Conference on Intelligent Transportation Systems, Boston, MA, USA, 12 November 1997; pp. 206–211. [Google Scholar]
- Kerekes, R.A. Profile Images and Annotations for Vehicle Reidentification Algorithms (PRIMAVERA). Available online: http://doi.ccs.ornl.gov/ui/doi/367 (accessed on 1 January 2022). [CrossRef]
- Chicco, D. Siamese neural networks: An overview. In Artificial Neural Networks; Springer, 2021; pp. 73–94. [Google Scholar]
- Bromley, J.; Bentz, J.W.; Bottou, L.; Guyon, I.; LeCun, Y.; Moore, C.; Säckinger, E.; Shah, R. Signature verification using a siamese time delay neural network. Int. J. Pattern Recognit. Artif. Intell. 1993, 7, 669–688. [Google Scholar] [CrossRef] [Green Version]
- Tzutalin LabelImg. Free Software: MIT License. 2015. Available online: http://github.com/tzutalin/labelImg (accessed on 1 July 2021).
- Sandler, M.; Howard, A.; Zhu, M.; Zhmoginov, A.; Chen, L. Mobilenetv2: Inverted residuals and linear bottlenecks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–23 June 2018; pp. 4510–4520. [Google Scholar]
- Lin, T.-Y.; Maire, M.; Belongie, S.; Hays, J.; Perona, P.; Ramanan, D.; Dollár, P.; Zitnick, C.L. Microsoft coco: Common objects in context. In Proceedings of the European Conference on Computer Vision, Zurich, Switzerland, 6–12 September 2014; Springer: Cham, Switzerland, 2014; pp. 740–755. [Google Scholar]
- Kingma, D.P.; Ba, J. Adam: A Method for stochastic optimization. arXiv 2014, arXiv:1412.6980. [Google Scholar]
Layer | Structure | Activation |
---|---|---|
Layer 1 | 4 × 3 × 3 | Relu |
Layer 2 | 8 × 3 × 3 | Relu |
Layer 3 | 16 × 3 × 3 | Relu |
Layer 4 | 32 × 3 × 3 | Relu |
Layer 5 | 32 × 3 × 3 | Relu |
Layer 6 | 32 × 3 × 3 | Relu |
Layer 7 | 32 × 3 × 3 | Relu |
Layer | Structure | Activation |
---|---|---|
Layer 1 | 64 × 3 × 3 | Relu |
Layer 2 | 64 × 3 × 3 | Relu |
Layer 3 | 64 × 2 × 2 | Relu |
Layer 4 | 64 × 1 × 1 | Relu |
Layer 5 | 32 × 1 × 1 | Relu |
Layer 6 | 1 × 1 × 1 | Relu |
Matching Accuracy | |
---|---|
Training | 95.45% |
Validation | 95.20% |
Layer | Structure | Activation |
---|---|---|
Layer 1 | 64 × 10 × 10 | Relu |
Layer 2 | 128 × 7 × 7 | Relu |
Layer 3 | 128 × 4 × 4 | Relu |
Layer 4 | 256 × 4 × 4 | Relu |
Layer 5 | 4096 × 1 | Sigmoid |
Matching Accuracy | |
---|---|
Training | 96.95% |
Validation | 93.21% |
Matching Accuracy | |
---|---|
Vehicle Matching Score | 95.5% |
Wheel Matching Scores | 93.21% |
Average | 97.63% |
Layer | Structure | Activation |
---|---|---|
Layer 1 | 100 | Relu |
Layer 2 | 70 | Relu |
Layer 3 | 20 | Relu |
Layer 4 | 1 | Sigmoid |
Fusion Network | Soft Voting | Baseline | Majority Voting | Vehicle Score | Wheel Scores | |
---|---|---|---|---|---|---|
Accuracy | 97.77 ± 0.56% | 97.28 ± 0.62% | 96.31 ± 0.71% | 95.68 ± 0.77% | 95.46 ± 0.79% | 92.93 ± 0.97% |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Ghanem, S.; Kerekes, R.A.; Tokola, R. Decision-Based Fusion for Vehicle Matching. Sensors 2022, 22, 2803. https://doi.org/10.3390/s22072803
Ghanem S, Kerekes RA, Tokola R. Decision-Based Fusion for Vehicle Matching. Sensors. 2022; 22(7):2803. https://doi.org/10.3390/s22072803
Chicago/Turabian StyleGhanem, Sally, Ryan A. Kerekes, and Ryan Tokola. 2022. "Decision-Based Fusion for Vehicle Matching" Sensors 22, no. 7: 2803. https://doi.org/10.3390/s22072803