The Development of a Prototype Solution for Detecting Wear and Tear in Pedestrian Crossings
<p>Illustration of the concepts of detection and classification.</p> "> Figure 2
<p>Illustration of the architecture of a CNN.</p> "> Figure 3
<p>Comparison of threshold methods.</p> "> Figure 4
<p>Use of Adaptive Threshold to determine degradation status: (<b>a</b>) Outcome for a crosswalk exhibiting signs of wear; (<b>b</b>) Outcome for a crosswalk without signs of wear; (<b>c</b>) Outcome for a crosswalk exhibiting a considerable degree of wear.</p> "> Figure 5
<p>Example of a question asked on the form.</p> "> Figure 6
<p>Example of the results obtained in the questionnaire.</p> "> Figure 7
<p>Example of how the increase in pixels can influence the results.</p> "> Figure 8
<p>Illustration from CNN CSPDarknet53-tiny. Adapted from [<a href="#B9-applsci-14-06462" class="html-bibr">9</a>].</p> "> Figure 9
<p>Architecture of the YOLOv4-tiny model. Adapted from [<a href="#B11-applsci-14-06462" class="html-bibr">11</a>].</p> "> Figure 10
<p>Example of a captured image.</p> "> Figure 11
<p>Illustration of objects captured by a 90° camera vs. a 160° camera.</p> "> Figure 12
<p>Diagram of how the first approach works.</p> "> Figure 13
<p>Examples of Adaptive Threshold detection: (<b>a</b>) Bounding boxes of three crosswalks (1)–(3) detected by the YOLOv4-tiny model; (<b>b</b>) Result of using the Adaptive Threshold method to determine the degradation state of the three crosswalks (1)–(3).</p> "> Figure 14
<p>Examples of Adaptive Threshold detection in scenes without shadows: (<b>a</b>) Example 1: detection (1), identification (2), and classification (3) process; (<b>b</b>) Example 2: detection (1), identification (2), and classification (3) process.</p> "> Figure 15
<p>Comparison between the two approaches in two different examples: Example (<b>a</b>): (1) and (2) shows the detection using the first approach considered—YOLOv4-tiny and Adaptive Threshold. (3) shows the result obtained under the same conditions but using the second approach—YOLOv4-tiny with three classes. Example (<b>b</b>): (1) and (2) shows the detection using the first approach considered—YOLOv4-tiny and Adaptive Threshold. (3) shows the result obtained under the same conditions but using the second approach—YOLOv4-tiny with three classes.</p> "> Figure 16
<p>The architecture of the prototype was developed.</p> "> Figure 17
<p>Schematic of the electrical circuit of the various hardware components.</p> "> Figure 18
<p>Hardware components of the prototype developed.</p> "> Figure 19
<p>Use case diagram.</p> "> Figure 20
<p>Flowchart of how the Raspberry Pi works.</p> "> Figure 21
<p>Structure required in documents.</p> "> Figure 22
<p>Endpoints on the API.</p> "> Figure 23
<p>Example of a response from the Geoapify API.</p> "> Figure 24
<p>Sequence diagram between API and Raspberry PI.</p> "> Figure 25
<p>Illustration of the mechanism for identifying the distance between coordinates.</p> "> Figure 26
<p>Sequence diagram of communication between the API and the CrosSafe application.</p> "> Figure 27
<p>Mockup Home page.</p> "> Figure 28
<p>Mockup Dashboard page.</p> "> Figure 29
<p>Crosswalk counting component by state of degradation.</p> "> Figure 30
<p>Interactive map in the application.</p> "> Figure 31
<p>Table with additional information on crosswalks detected using the pagination method.</p> "> Figure 32
<p>Popup for confirmation of crosswalk repair.</p> "> Figure 33
<p>Example of a success message following confirmation of a crosswalk repair.</p> "> Figure 34
<p>An example of a crosswalk classified as severely worn.</p> "> Figure 35
<p>Example of the number of documents received by Firestore.</p> "> Figure 36
<p>Illustration of the route taken.</p> "> Figure 37
<p>Detection and classification time required.</p> "> Figure 38
<p>An image containing a crosswalk slightly diagonally.</p> "> Figure 39
<p>Visualization of the data transmitted by the Raspberry PI in the CrosSafe web application.</p> ">
Abstract
:1. Introduction
2. Classification of the State of Degradation of Pedestrian Crossings with Computer Vision Techniques
2.1. Adaptive Threshold Classification
- Global Thresholding: This applies a single threshold value to the entire image.
- Otsu’s Thresholding: This automatically determines the optimal threshold value based on the image histogram.
- Adaptive Mean Thresholding: This calculates a threshold value for each region of the image based on the average of the intensity values of the pixels in that region.
- Adaptive Gaussian Thresholding: This is like Adaptive Mean Thresholding but calculates the threshold value using a weighted average based on the Gaussian distribution.
Threshold Values
2.2. Classification with YOLOv4-Tiny
Dataset
2.3. Performance Evaluation
2.3.1. Benchmark Scenario
2.3.2. Performance Metrics
2.3.3. Results and Discussion
3. Prototype
3.1. Architecture
3.2. Hardware Components
- Raspberry PI (U1)
- GPS NEO-6M (U2)
- Wide-angle 1080 p UVC-Compliant USB Camera Module with Metal Case (U3)
3.3. Software Components
- Register New Crosswalk Detection, which has as a precondition the current GPS location, the execution of the YOLOv4-tiny model, and an internet connection. As an output, this detection will be saved by the API, if there is internet access, otherwise it will be saved locally.
- Send Data to API, initially requires a crosswalk to be detected, and then verification of the existence of locally stored data to be sent. As an output, the information is stored in the database via the endpoint provided by the API.
- Receive Data from Prototype, for this requirement to work, a Hypertext Transfer Protocol (HTTP) request of the POST type must be made by the Raspberry PI. The output will be the information stored in the database.
- Send Stored Data requires an HTTP GET request. All the detections stored in the database will be returned.
- View Detections requires the user to be on the web application’s dashboard page. The output will show all the crosswalks registered to date and their state of disrepair.
3.3.1. Internet of Things Device
3.3.2. Application Programming Interface and Database (Back-End)
3.3.3. Web Application (Front-End)
4. Testing and Validation
5. Conclusions
Author Contributions
Funding
Data Availability Statement
Conflicts of Interest
References
- Técnica, C. Relatório Novembro 2023; Barcarena: Lisboa, Portugal, 2024. [Google Scholar]
- Song, Z.; Chen, Q.; Huang, Z.; Hua, Y.; Yan, S. Contextualizing Object Detection and Classification. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Colorado Springs, CO, USA, 20–25 June 2011; pp. 1585–1592. [Google Scholar] [CrossRef]
- IBM. What Is Computer Vision? Available online: https://www.ibm.com/topics/computer-vision (accessed on 8 December 2023).
- Sarker, I.H. Deep Learning: A Comprehensive Overview on Techniques, Taxonomy, Applications and Research Directions. SN Comput. Sci. 2021, 2, 420. [Google Scholar] [CrossRef] [PubMed]
- AWS. What Is Deep Learning? Deep Learning Explained. Available online: https://aws.amazon.com/what-is/deep-learning/ (accessed on 12 January 2024).
- Huo, Y.-K.; Wei, G.; Zhang, Y.-D.; Wu, L.-N. An Adaptive Threshold for the Canny Operator of Edge Detection. In Proceedings of the 2010 International Conference on Image Analysis and Signal Processing, Xiamen, China, 9–11 April 2010. [Google Scholar]
- Roy, P.; Dutta, S.; Dey, N.; Dey, G.; Chakraborty, S.; Ray, R. Adaptive Thresholding: A Comparative Study. In Proceedings of the 2014 International Conference on Control, Instrumentation, Communication and Computational Technologies, ICCICCT 2014, Kanyakumari, India, 10–11 July 2014; pp. 1182–1186. [Google Scholar] [CrossRef]
- Rosa, G.J.M.; Afonso, J.M.S.; Gaspar, P.D.; Soares, V.N.G.J.; Caldeira, J.M.L.P. Detecting Wear and Tear in Pedestrian Crossings Using Computer Vision Techniques: Approaches, Challenges, and Opportunities. Information 2024, 15, 169. [Google Scholar] [CrossRef]
- Xu, P.; Li, Q.; Zhang, B.; Wu, F.; Zhao, K.; Du, X.; Yang, C.; Zhong, R. On-Board Real-Time Ship Detection in HISEA-1 SAR Images Based on CFAR and Lightweight Deep Learning. Remote Sens. 2021, 13, 1995. [Google Scholar] [CrossRef]
- Misra, D. Mish: A Self Regularized Non-Monotonic Activation Function. In Proceedings of the 31st British Machine Vision (BMVC 2020), Virtual, 7–10 September 2020. [Google Scholar]
- Bochkovskiy, A.; Wang, C.-Y.; Liao, H.-Y.M. YOLOv4: Optimal Speed and Accuracy of Object Detection. arXiv 2020, arXiv:2004.10934v1. [Google Scholar]
- Roboflow. Crosswalks Dataset > Overview. Available online: https://universe.roboflow.com/projeto-5fy5m/crosswalks-zbjgg (accessed on 5 December 2023).
- Roboflow. Project Overview. Available online: https://universe.roboflow.com/passadeiras/crosswalks-one-class (accessed on 18 April 2024).
- Roboflow. Crosswalks Dataset > Overview. Available online: https://universe.roboflow.com/passadeiras/crosswalks-uo9bq (accessed on 18 April 2024).
- Google Colab. Available online: https://colab.research.google.com/ (accessed on 20 April 2024).
- GitHub-AlexeyAB/Darknet: YOLOv4/Scaled-YOLOv4/YOLO-Neural Networks for Object Detection (Windows and Linux Version of Darknet). Available online: https://github.com/AlexeyAB/darknet (accessed on 22 April 2024).
- Firebase|Google’s Mobile and Web App Development Platform. Available online: https://firebase.google.com/?hl=pt (accessed on 2 April 2024).
- Render. Cloud Application Hosting for Developers|Render. Available online: https://render.com/ (accessed on 2 April 2024).
- JSON. Available online: https://www.json.org/json-en.html (accessed on 7 May 2024).
- AWS. O Que é Uma Chave de API?—Explicação Sobre Chaves e Tokens de API. Available online: https://aws.amazon.com/pt/what-is/api-key/ (accessed on 21 April 2024).
- Chuantao, Y.; Zhang, X.; Hui, C.; Jingyuan, W.; Daven, C.; Bertrand, D. A Literature Survey on Smart Cities. Sci. China Inf. Sci. 2015, 58, 100102. [Google Scholar] [CrossRef]
- Raspberrypi. Buy a Raspberry Pi 5–Raspberry Pi. Available online: https://www.raspberrypi.com/products/raspberry-pi-5/ (accessed on 16 May 2024).
- Botnroll. Módulo Gps Neo-6m-Uart C/ Antena Cerâmica. Available online: https://www.botnroll.com/pt/gps-gnss/4548-m-dulo-gps-neo-6m-uart-c-antena-cer-mica.html (accessed on 16 May 2024).
- Arducam. Arducam 1080P Low Light WDR USB Camera Module with Metal Case. Available online: https://www.arducam.com/product/b020201-arducam-1080p-low-light-wdr-usb-camera-module-with-metal-case-2mp-1-2-8-cmos-imx291-160-degree-ultra-wide-angle-mini-uvc-webcam-board-with-microphones/ (accessed on 16 May 2024).
- EasyEDA-PCB Design & Simulação de Circuitos Online. Available online: https://easyeda.com/pt (accessed on 4 April 2024).
- Maguire, M.; Bevan, N. User Requirements Analysis; Springer: Boston, MA, USA, 2002; pp. 133–148. [Google Scholar] [CrossRef]
- Jama Software. Functional Requirements Examples and Templates. Available online: https://www.jamasoftware.com/requirements-management-guide/writing-requirements/functional-requirements-examples-and-templates (accessed on 16 April 2024).
- Altexsoft. Nonfunctional Requirements: Examples, Types and Approaches. Available online: https://www.altexsoft.com/blog/non-functional-requirements/ (accessed on 16 April 2024).
- Booch, G.; Rumbaugh, J.; Jacobson, I. The Unified Modeling Language for Object-Oriented Development Documentation Set Version 0.9a Addendum; Rational Software Corporation: San Jose, CA, USA, 1996. [Google Scholar]
- UML. Use Case Diagram Tutorial|Lucidchart. Available online: https://www.lucidchart.com/pages/uml-use-case-diagram (accessed on 16 April 2024).
- OpenCV.Open Computer Vision Library. Available online: https://opencv.org/ (accessed on 4 April 2024).
- GitHub. Arunponnusamy/Cvlib: A Simple, High Level, Easy to Use, Open Source Computer Vision Library for Python. Available online: https://github.com/arunponnusamy/cvlib (accessed on 4 April 2024).
- GitHub. Knio/Pynmea2: Python Library for Parsing the NMEA 0183 Protocol (GPS). Available online: https://github.com/Knio/pynmea2 (accessed on 8 May 2024).
- Available online: https://nodejs.org/en (accessed on 2 April 2024).
- Fastify. Fast and Low Overhead Web Framework, for Node.Js. Available online: https://fastify.dev/ (accessed on 2 April 2024).
- Firestore. Firebase. Available online: https://firebase.google.com/docs/firestore (accessed on 16 May 2024).
- Prisma. Simplify Working and Interacting with Databases. Available online: https://www.prisma.io/ (accessed on 21 April 2024).
- Microsoft Learn. Web API Design Best Practices-Azure Architecture Center. Available online: https://learn.microsoft.com/en-us/azure/architecture/best-practices/api-design (accessed on 21 April 2024).
- Bloch, J. How to Design a Good API and Why It Matters. In Proceedings of the OOPSLA ′06: Companion to the 21st ACM SIGPLAN Symposium on Object-Oriented Programming Systems, Languages, and Applications, Portland, OR, USA, 22–26 October 2006. [Google Scholar]
- Postman. Postman API Platform Sign Up for Free. Available online: https://www.postman.com/ (accessed on 18 April 2024).
- Geoapify. Reverse Geocoding API Lat/Lon to Address. Available online: https://www.geoapify.com/reverse-geocoding-api (accessed on 18 April 2024).
- Render Docs. Web Services. Available online: https://docs.render.com/web-services (accessed on 8 April 2024).
- React. Available online: https://react.dev/ (accessed on 2 April 2024).
- Typescrip. TypeScript: JavaScript with Syntax for Types. Available online: https://www.typescriptlang.org/ (accessed on 21 April 2024).
- Vite. Next Generation Frontend Tooling. Available online: https://vitejs.dev/ (accessed on 2 April 2024).
- Tailwind CSS. Available online: https://tailwindcss.com/ (accessed on 31 March 2024).
- Shadcn/Ui. Available online: https://ui.shadcn.com/ (accessed on 21 April 2024).
- Leaflet. A JavaScript Library for Interactive Maps. Available online: https://leafletjs.com/ (accessed on 18 April 2024).
- Logrocket. Rendering Large Lists in React: 5 Methods with Examples-LogRocket Blog. Available online: https://blog.logrocket.com/render-large-lists-react-5-methods-examples/ (accessed on 19 April 2024).
Intervals | State of Degradation |
---|---|
<30 | No wear |
[30, 50] | Moderate wear |
>50 | Severe wear |
Class | State of Degradation |
---|---|
No wear | 555 |
Moderate wear | 413 |
Severe wear | 471 |
Model | Images | Input Size | mAP (%) |
---|---|---|---|
YOLOv4-tiny [8] | 642 | 608 × 608 | 87 |
YOLOv4-tiny * | 1182 | 608 × 608 | 90 |
Class | mAP (%) | Overall (%) |
---|---|---|
No wear | 82.13 | 71.21 |
Moderate wear | 54.72 | |
Severe wear | 76.96 |
Hardware | Watts | Volts | Amps |
---|---|---|---|
Raspberry PI 5 8 GB with cooler | 27 | 5 | 6 |
Raspberry PI 5 8 GB without cooler | 27 | 5 | 5 |
Class | Real Classification | YOLOv4-Tiny |
---|---|---|
No wear | 9 | 9 |
Moderate wear | 3 | 2 |
Severe wear | 5 | 4 |
Total | 17 | 15 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Rosa, G.J.M.; Afonso, J.M.S.; Gaspar, P.D.; Soares, V.N.G.J.; Caldeira, J.M.L.P. The Development of a Prototype Solution for Detecting Wear and Tear in Pedestrian Crossings. Appl. Sci. 2024, 14, 6462. https://doi.org/10.3390/app14156462
Rosa GJM, Afonso JMS, Gaspar PD, Soares VNGJ, Caldeira JMLP. The Development of a Prototype Solution for Detecting Wear and Tear in Pedestrian Crossings. Applied Sciences. 2024; 14(15):6462. https://doi.org/10.3390/app14156462
Chicago/Turabian StyleRosa, Gonçalo J. M., João M. S. Afonso, Pedro D. Gaspar, Vasco N. G. J. Soares, and João M. L. P. Caldeira. 2024. "The Development of a Prototype Solution for Detecting Wear and Tear in Pedestrian Crossings" Applied Sciences 14, no. 15: 6462. https://doi.org/10.3390/app14156462