CN114418950A - Road disease detection method, device, equipment and storage medium - Google Patents

Abstract

The invention belongs to the technical field of artificial intelligence and discloses a road disease detection method, a device, equipment and a storage medium. The method comprises the following steps: acquiring an image to be detected; segmenting the image to be detected to obtain a road image; inputting the road image into a target detection model to obtain a detection result; and judging whether the road corresponding to the image to be detected has diseases or not according to the detection result. By the mode, the detection result of the road is obtained by training the image input value to be detected in the target detection model, the detection result contains the diseases existing in the road, the detection efficiency of the road diseases is improved, and road managers can repair the road diseases according to the detection result, so that the safety of the road is improved.

Description

Road disease detection method, device, equipment and storage medium

technical field

The invention relates to the technical field of artificial intelligence, and in particular, to a road disease detection method, device, equipment and storage medium.

Background technique

With the rapid growth of new expressways, many of the original expressways have been damaged and damaged to varying degrees. Moreover, the lack of timely and reasonable road maintenance for these diseases and damages has led to a sharp decline in the road conditions of some expressways, which has had a considerable impact on the comfort, economy and safety of vehicles. Therefore, in recent years, highway pavement maintenance management and related issues have received more and more attention.

At the same time, due to the untimely maintenance of roads, accidents such as loss of manhole covers and serious road damage caused by overloading of heavy vehicles have occurred frequently in various places, resulting in serious loss of people's lives and properties.

SUMMARY OF THE INVENTION

The main purpose of the present invention is to provide a road disease detection method, device, equipment and storage medium, which aims to solve the technical problem of how to improve the detection efficiency of road diseases in the prior art.

In order to achieve the above object, the present invention provides a road disease detection method, the method comprises the following steps:

Obtain the image to be detected;

segmenting the to-be-detected image to obtain a road image;

Inputting the road image into a target detection model to obtain a detection result;

According to the detection result, it is determined whether the road corresponding to the to-be-detected image has a disease.

Optionally, before the step of inputting the road image into the target detection model and obtaining the detection result, the method further includes:

Get the initial training dataset;

Obtain a target training data set according to the initial training data set;

An initial detection model is trained according to the target training data set to obtain a target detection model.

Optionally, the step of obtaining a target training data set according to the initial training data set includes:

Determine an initial training image according to the initial training data set;

Obtain a training image pair according to the initial training image;

A target training data set is obtained according to the training image pair.

Optionally, the step of obtaining the target training data set according to the training image pair includes:

Inputting the training image pair to a preset training model to obtain a pseudo-label image pair;

determining a confidence level for the pair of pseudo-label images;

Screen the pseudo-label image pair according to the confidence to obtain a target training image;

A target training data set is obtained according to the target training image.

Optionally, the step of acquiring the image to be detected includes:

Get the video to be detected;

Obtain a frame image set according to the video to be detected;

Determine adjacent frame images according to the frame image set;

The adjacent frame images are fused into an image to be detected.

Optionally, the step of segmenting the to-be-detected image to obtain a road image includes:

converting the to-be-detected image into a grayscale image;

performing noise reduction processing on the grayscale image to obtain a noise reduction image;

determining the gradient value of each pixel in the denoised image, and obtaining a road edge image according to the gradient value;

A road image is determined from the road edge image.

Optionally, after the step of judging whether the road corresponding to the to-be-detected image has a disease according to the detection result, the method further includes:

When there is a disease on the road corresponding to the image to be detected, obtain the location information and the disease type corresponding to the road;

generating a road repair plan according to the disease type;

A road disease report is generated based on the location information, the disease type, and the road repair plan.

In addition, in order to achieve the above purpose, the present invention also provides a road disease detection device, the road disease detection device includes:

an acquisition module for acquiring the image to be detected;

a segmentation module, configured to segment and segment the to-be-detected image to obtain a road image;

a detection module for inputting the road image into the target detection model to obtain a detection result;

A judgment module, configured to judge whether the road corresponding to the to-be-detected image has a disease according to the detection result.

In addition, in order to achieve the above object, the present invention also provides a road disease detection device, the road disease detection device includes: a memory, a processor, and a road disease detection device stored in the memory and running on the processor A program configured to implement the steps of the road disease detection method as described above.

In addition, in order to achieve the above object, the present invention also provides a storage medium, on which a road disease detection program is stored, and when the road disease detection program is executed by a processor, the above-mentioned road disease detection method is realized. step.

The present invention obtains a road image by acquiring an image to be detected; dividing the image to be detected to obtain a road image; inputting the road image into a target detection model to obtain a detection result; and judging whether the road corresponding to the image to be detected is determined according to the detection result Diseases exist. Through the above method, in the target detection model trained by the input value of the image to be detected, the target detection model calculates and analyzes the image to be detected, thereby obtaining the detection result of the road. The detection result includes the diseases existing on the road. Road diseases can be repaired according to the detection results, thereby improving road safety.

Description of drawings

1 is a schematic structural diagram of a road disease detection device of a hardware operating environment involved in an embodiment of the present invention;

2 is a schematic flowchart of the first embodiment of the road disease detection method according to the present invention;

3 is a schematic flowchart of a second embodiment of a road disease detection method according to the present invention;

FIG. 4 is a structural block diagram of a first embodiment of a road disease detection device according to the present invention.

The realization, functional characteristics and advantages of the present invention will be further described with reference to the accompanying drawings in conjunction with the embodiments.

Detailed ways

It should be understood that the specific embodiments described herein are only used to explain the present invention, but not to limit the present invention.

Referring to FIG. 1 , FIG. 1 is a schematic structural diagram of a road disease detection device in a hardware operating environment involved in an embodiment of the present invention.

As shown in FIG. 1 , the road disease detection device may include: a processor 1001 , such as a central processing unit (Central Processing Unit, CPU), a communication bus 1002 , a user interface 1003 , a network interface 1004 , and a memory 1005 . Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. Optionally, the network interface 1004 may include a standard wired interface and a wireless interface (eg, a Wireless-Fidelity (Wi-Fi) interface). The memory 1005 may be a high-speed random access memory (Random Access Memory, RAM) memory, or may be a stable non-volatile memory (Non-Volatile Memory, NVM), such as a disk memory. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001 .

Those skilled in the art can understand that the structure shown in FIG. 1 does not constitute a limitation on the road disease detection device, and may include more or less components than the one shown, or combine some components, or arrange different components.

As shown in FIG. 1 , the memory 1005 as a storage medium may include an operating system, a network communication module, a user interface module and a road disease detection program.

In the road disease detection device shown in FIG. 1, the network interface 1004 is mainly used for data communication with the network server; the user interface 1003 is mainly used for data interaction with the user; the processor 1001, the memory in the road disease detection device of the present invention 1005 may be set in a road disease detection device, the road disease detection device invokes the road disease detection program stored in the memory 1005 through the processor 1001, and executes the road disease detection method provided by the embodiment of the present invention.

An embodiment of the present invention provides a road disease detection method. Referring to FIG. 2 , FIG. 2 is a schematic flowchart of a first embodiment of a road disease detection method of the present invention.

In this embodiment, the road disease detection method includes the following steps:

Step S10: Acquire an image to be detected.

It should be noted that the execution subject of this embodiment is a terminal device, such as a computer and other devices that can process data and run application programs.

In a specific implementation, the images to be detected include images that need to be detected for road diseases. The image to be detected is collected by a camera set on a movable device, and the movable device can be a vehicle or a drone.

Further, in order to improve the efficiency of the terminal equipment during detection, step S10 includes: acquiring a video to be detected; obtaining a frame image set according to the to-be-detected video; determining adjacent frame images according to the frame image set; Frame images are fused into images to be detected.

It can be understood that when the camera collects road images, it usually shoots videos of the road according to the driving direction on the road, but the video is composed of multiple frames of images. Therefore, multiple consecutive images will contain the same road image. Needing to detect every frame can lead to a waste of resources.

In the specific implementation, the video to be detected is firstly decoded into frames, decomposed into several frames of continuous images, that is, frame image sets, and then the frame images containing the same road image are fused. Usually, adjacent frame images will contain more repetitions. For example, every 10 adjacent frames is a group of frame images that need to be fused, then 10 frames of images are fused to obtain an image to be detected.

It should be noted that the essence of multi-image fusion is to splicing multiple images into one image. When splicing images, the adjacent frame images are first preprocessed, and the collected images are corrected by the geometric deformation correction method. , which can make the images of the same scene in the overlapping area of the images have the same shape and consistent relative spatial position. When the camera collects road images, it can shoot perpendicular to the ground and maintain the same height. Therefore, the road’s angle of view in the image is consistent, but it may be caused by the existence of potholes on the ground or the existence of speed bumps. Slight changes require image correction.

After the image is corrected, take the intermediate image of the adjacent frame image group as the reference image, and find the same feature points as the reference image in the remaining frame images. The feature points include: closed area, contour and edge area, corner points, lines, etc. On the road, the road line is usually used as the feature point for matching. After the matched feature points are determined, multiple frame images are spliced into an image to be detected according to the feature points.

Step S20: segment the to-be-detected image to obtain a road image.

It should be noted that, in order to capture the road completely, the images collected by the camera usually contain some images outside the road. In order to avoid the influence of the images outside the road on the detection results, it needs to be segmented at this time, and only the images containing the road are left. image, the road image.

In this embodiment, the road image is segmented by detecting the edge of the road. First, the image is transformed from the spatial domain to the frequency domain through Fourier transform, and the high-frequency part is determined according to the frequency of the frequency domain. Usually, the high-frequency part corresponds to the image in the image. On the urban asphalt road, the lane line can be used as the edge of the road for detection, and the color difference between the cement floor and the surrounding environment is usually large, so it can also achieve a good segmentation effect.

Further, step S20 further includes: converting the image to be detected into a grayscale image; performing noise reduction processing on the grayscale image to obtain a noise reduction image; determining the gradient value of each pixel in the noise reduction image, and obtaining a road edge image according to the gradient value; determining a road image according to the road edge image.

In the specific implementation, it is also possible to grayscale the image to be detected, convert it into a grayscale image, and then denoise the grayscale image. The grayscale image can be denoised by means of wavelet noise reduction. The gradient value of each pixel in the resulting grayscale image is taken as a road edge with a change in the gradient value greater than a preset threshold, thereby obtaining a road edge image, and segmenting the image to be detected according to the edge lines in the road edge image to obtain a road image.

Step S30: Input the road image into a target detection model to obtain a detection result.

It should be noted that the target detection model is a trained model, and the target detection model can be YOLOv5. Compared with the YOLOv4 model, YOLOv5 is smaller, has faster operation speed, and can process road images more efficiently.

In a specific implementation, after the road image is input into the target detection model, the detection result can be obtained, and the detection result includes whether there is a disease on the road, and if there is a disease, the type of the disease is marked.

Step S40: Determine whether the road corresponding to the to-be-detected image has a disease according to the detection result.

It can be understood that the types of diseases include transverse cracks, longitudinal cracks, tortoise cracks and potholes, etc. When the above-mentioned diseases are detected on the road, the diseased part is marked and the type of the disease is displayed. Road managers can determine the disease information of the road according to the detection results.

Further, after step S40, it also includes: when there is a disease on the road corresponding to the image to be detected, acquiring the location information and the disease type corresponding to the road; generating a road repair plan according to the disease type; according to the location information , the disease type, and the road repair plan to generate a road disease report.

It should be noted that when the road is collected, the geographic location of the road will be collected at the same time, so as to form a road-location mapping relationship. When a disease is detected on the road, the corresponding location information can be searched according to the mapping relationship, so as to facilitate the road. Managers repair the road. The road information includes the location area where the disease exists.

Similarly, after the disease type of the road is detected, a corresponding road repair plan is generated according to the disease type. For example, when the disease type is a crack, the repair plan is to remove the sand deposited in the crack on the road section with crack disease. , Use tape to separate the area to be repaired, fill the repair area with aggregate and composite modified emulsion evenly, and wait for two hours to complete the repair.

In the specific implementation, the road disease report is generated according to the location information, the disease type and the road repair plan, the road disease report is sent to the road management personnel, and the disease report is uploaded to the cloud for backup.

In this embodiment, a road image is obtained by acquiring an image to be detected; dividing the image to be detected to obtain a road image; inputting the road image into a target detection model to obtain a detection result; and judging the road corresponding to the image to be detected according to the detection result Whether there is disease. In the above method, in the target detection model trained with the input value of the image to be detected, the target detection model calculates and analyzes the image to be detected, thereby obtaining the detection result of the road. Road diseases can be repaired according to the detection results, thereby improving road safety.

Referring to FIG. 3 , FIG. 3 is a schematic flowchart of a second embodiment of a road disease detection method according to the present invention.

Based on the above-mentioned first embodiment, before the step S30, the road disease detection method of this embodiment further includes:

Step S21: Obtain an initial training data set.

It should be noted that the initial training data set includes images of various types of roads such as urban streets, rural roads, and highways in different weather and environmental conditions. Since roads in different countries may be different, when detecting road diseases in different regions, it is necessary to use the initial training dataset of local roads.

Step S22: Obtain a target training data set according to the initial training data set.

Further, in order to make the model training based on richer semantic information, step S22 includes: determining an initial training image according to the initial training data set; obtaining a training image pair according to the initial training image; obtaining a training image pair according to the training image pair target training dataset.

In the specific implementation, the initial training image is obtained from the initial training data set, and the initial training image is the road image in the initial training data set, and the road part in the image is segmented based on the above road segmentation method, and then the segmented image is segmented. Image enhancement, for example, mirroring and flipping the segmented image to form a training image pair, where the training image pair includes the segmented initial training image and the corresponding image-enhanced image.

Further, the step of obtaining the target training data set according to the training image pair includes: inputting the training image pair into a preset training model to obtain a pseudo-label image pair; determining the confidence level of the pseudo-label image pair ; Screen the pseudo-label image pair according to the confidence level to obtain a target training image; and obtain a target training data set according to the target training image.

It should be noted that the preset training model is used to screen out images with greater uncertainty in the training images, so as to prevent the waste of resources caused by training the initial detection model.

In this embodiment, the value range of the pseudo-label is set to [0, 1], and the calculation process of the training image pair by the preset training model is as follows:

公式1；

Formula 1;

、

为伪标签阈值，其中

。Among them, _gi is the pseudo-label value of one image in the training image pair, _pi is the probability that it can be used as a training image,

,

is the pseudo-label threshold, where

.

时，g_i为1，当p_i小于等于

时，g_i为0。It should be noted that when _pi is greater than or equal to

When _gi is 1, when _pi is less than or equal to

, g _i is 0.

Among them, the loss function of the preset training model is:

公式2；

formula 2;

为训练图像对中任一图像的伪标签，

为预设训练模型的原始输出概率。where s is the number of pseudo-labels,

is the pseudo-label for any image in the training image pair,

The original output probabilities of the trained model for the preset.

It is understandable that the pseudo-label values of training images that are not 1 or 0 need to be determined by the loss function. And determine the confidence based on the pseudo-label value:

公式3；

formula 3;

Among them, β is the confidence level, and α is a preset value, usually 0.5.

It can be understood that the training images with the confidence greater than the confidence threshold are used as the target training images, so as to screen out the training images that can be used as the training model.

Further, this embodiment also collects a batch of road data production and labeling software that conforms to local actual conditions and is more targeted, and labels the training images through the labeling software. The types of road defects to be labelled are: transverse cracks, longitudinal Cracks and potholes. A total of more than 6,000 multi-label valid datasets are annotated. After the standard is completed, the target training data set is obtained.

Step S23: Train an initial detection model according to the target training data set to obtain a target detection model.

In a specific implementation, after training the initial detection model through the target training data set, a target detection model for detecting road diseases can be obtained.

In this embodiment, a target detection model is obtained by acquiring an initial training data set; obtaining a target training data set according to the initial training data set; and training an initial detection model according to the target training data set. Through the above method, the model training is based on richer semantic information, which has better robustness and improves the accuracy of target detection.

In addition, an embodiment of the present invention also provides a storage medium, where a road disease detection program is stored thereon, and when the road disease detection program is executed by a processor, the steps of the road disease detection method as described above are implemented.

Since the storage medium adopts all the technical solutions of the above-mentioned embodiments, it has at least all the beneficial effects brought by the technical solutions of the above-mentioned embodiments, which will not be repeated here.

Referring to FIG. 4 , FIG. 4 is a structural block diagram of a first embodiment of a road disease detection apparatus according to the present invention.

As shown in FIG. 4 , the road disease detection device proposed by the embodiment of the present invention includes:

The acquiring module 10 is used for acquiring the image to be detected.

The segmentation module 20 is configured to segment and segment the to-be-detected image to obtain a road image.

The detection module 30 is used for inputting the road image into the target detection model to obtain the detection result.

The judging module 40 is used for judging whether the road corresponding to the to-be-detected image has a disease according to the detection result.

It should be understood that the above are only examples, and do not constitute any limitation to the technical solutions of the present invention. In specific applications, those skilled in the art can make settings as required, which is not limited by the present invention.

In this embodiment, a road image is obtained by acquiring an image to be detected; segmenting the image to be detected; inputting the road image into a target detection model to obtain a detection result; Whether there is a disease on the road. In the above method, in the target detection model trained with the input value of the image to be detected, the target detection model calculates and analyzes the image to be detected, thereby obtaining the detection result of the road. Road diseases can be repaired according to the detection results, thereby improving road safety.

In one embodiment, the detection module 30 is further configured to obtain an initial training data set;

Obtain a target training data set according to the initial training data set;

An initial detection model is trained according to the target training data set to obtain a target detection model.

In one embodiment, the detection module 30 is further configured to determine an initial training image according to the initial training data set;

Obtain a training image pair according to the initial training image;

A target training data set is obtained according to the training image pair.

In one embodiment, the detection module 30 is further configured to input the training image pair to a training model to obtain a pseudo-label image pair;

determining a confidence level for the pair of pseudo-label images;

Screen the pseudo-label image pair according to the confidence to obtain a target training image;

A target training data set is obtained according to the target training image.

In one embodiment, the obtaining module 10 is further configured to obtain the video to be detected;

Obtain a frame image set according to the video to be detected;

Determine adjacent frame images according to the frame image set;

The adjacent frame images are fused into an image to be detected.

In one embodiment, the segmentation module 20 is further configured to convert the to-be-detected image into a grayscale image;

performing noise reduction processing on the grayscale image to obtain a noise reduction image;

determining the gradient value of each pixel in the denoised image, and obtaining a road edge image according to the gradient value;

A road image is determined from the road edge image.

In one embodiment, the judging module 40 is further configured to obtain the location information and the type of the disease corresponding to the road when there is a disease on the road corresponding to the image to be detected;

generating a road repair plan according to the disease type;

A road disease report is generated based on the location information, the disease type, and the road repair plan.

It should be noted that the above-described workflow is only illustrative, and does not limit the protection scope of the present invention. In practical applications, those skilled in the art can select some or all of them to implement according to actual needs. The purpose of the solution in this embodiment is not limited here.

In addition, for technical details that are not described in detail in this embodiment, reference may be made to the road disease detection method provided by any embodiment of the present invention, which will not be repeated here.

Furthermore, it should be noted that, herein, the terms "comprising", "comprising" or any other variation thereof are intended to encompass non-exclusive inclusion, such that a process, method, article or system comprising a series of elements includes not only those elements, but also other elements not expressly listed or inherent to such a process, method, article or system. Without further limitation, an element qualified by the phrase "comprising a..." does not preclude the presence of additional identical elements in the process, method, article or system that includes the element.

The above-mentioned serial numbers of the embodiments of the present invention are only for description, and do not represent the advantages or disadvantages of the embodiments.

From the description of the above embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus a necessary general hardware platform, and of course hardware can also be used, but in many cases the former is better implementation. Based on this understanding, the technical solutions of the present invention can be embodied in the form of software products in essence or the parts that make contributions to the prior art. The computer software products are stored in a storage medium (such as a read-only memory (Read Only Memory , ROM)/RAM, magnetic disk, optical disk), including several instructions to make a terminal device (which may be a mobile phone, a computer, a server, or a network device, etc.) execute the methods described in the various embodiments of the present invention.

The above are only preferred embodiments of the present invention, and are not intended to limit the scope of the present invention. Any equivalent structure or equivalent process transformation made by using the contents of the description and drawings of the present invention, or directly or indirectly applied in other related technical fields , are similarly included in the scope of patent protection of the present invention.

Claims (10)

1. A road disease detection method is characterized by comprising the following steps:

acquiring an image to be detected;

segmenting the image to be detected to obtain a road image;

inputting the road image into a target detection model to obtain a detection result;

and judging whether the road corresponding to the image to be detected has diseases or not according to the detection result.

2. The method of claim 1, wherein the step of inputting the road image into an object detection model to obtain a detection result further comprises:

acquiring an initial training data set;

obtaining a target training data set according to the initial training data set;

and training an initial detection model according to the target training data set to obtain a target detection model.

3. The method of claim 2, wherein the step of deriving a target training data set from the initial training data set comprises:

determining an initial training image according to the initial training data set;

obtaining a training image pair according to the initial training image;

and obtaining a target training data set according to the training image pair.

4. The method of claim 3, wherein the step of deriving a target training data set from the training image pair comprises:

inputting the training image pair to a preset training model to obtain a pseudo label image pair;

determining a confidence level of the pseudo-tag image pair;

screening the pseudo label image pair according to the confidence coefficient to obtain a target training image;

and obtaining a target training data set according to the target training image.

5. The method of claim 1, wherein the step of acquiring an image to be detected comprises:

acquiring a video to be detected;

obtaining a frame image set according to the video to be detected;

determining adjacent frame images according to the frame image set;

and fusing the adjacent frame images into an image to be detected.

6. The method of claim 1, wherein the step of segmenting the image to be detected to obtain a road image comprises:

converting the image to be detected into a gray image;

carrying out noise reduction processing on the gray level image to obtain a noise reduction image;

determining the gradient value of each pixel point in the noise-reduced image, and obtaining a road edge image according to the gradient value;

and determining a road image according to the road edge image.

7. The method according to any one of claims 1 to 6, wherein after the step of determining whether the road corresponding to the image to be detected has a disease according to the detection result, the method further comprises:

when the road corresponding to the image to be detected has a disease, acquiring position information and a disease type corresponding to the road;

generating a road repairing scheme according to the disease type;

and generating a road disease report according to the position information, the disease type and the road repairing scheme.

8. A road disease detection device, characterized in that, road disease detection device includes:

the acquisition module is used for acquiring an image to be detected;

the segmentation module is used for segmenting and segmenting the image to be detected to obtain a road image;

the detection module is used for inputting the road image into a target detection model to obtain a detection result;

and the judging module is used for judging whether the road corresponding to the image to be detected has diseases or not according to the detection result.

9. A road disease detection apparatus, characterized in that the apparatus comprises: a memory, a processor and a road damage detection program stored on the memory and operable on the processor, the road damage detection program being configured to implement the road damage detection method according to any one of claims 1 to 7.

10. A storage medium having stored thereon a road damage detection program which, when executed by a processor, implements the road damage detection method according to any one of claims 1 to 7.

Images