CN114155375B - Method, device, electronic equipment and storage medium for detecting airport pavement defects - Google Patents
Method, device, electronic equipment and storage medium for detecting airport pavement defects Download PDFInfo
- Publication number
- CN114155375B CN114155375B CN202111143345.9A CN202111143345A CN114155375B CN 114155375 B CN114155375 B CN 114155375B CN 202111143345 A CN202111143345 A CN 202111143345A CN 114155375 B CN114155375 B CN 114155375B
- Authority
- CN
- China
- Prior art keywords
- disease
- semantic
- image
- airport pavement
- feature
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 42
- 230000007547 defect Effects 0.000 title abstract 12
- 230000011218 segmentation Effects 0.000 claims abstract description 65
- 238000001514 detection method Methods 0.000 claims abstract description 58
- 238000012549 training Methods 0.000 claims abstract description 49
- 238000000605 extraction Methods 0.000 claims abstract description 19
- 201000010099 disease Diseases 0.000 claims description 201
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 claims description 201
- 238000010586 diagram Methods 0.000 claims description 81
- 230000006870 function Effects 0.000 claims description 24
- 239000011159 matrix material Substances 0.000 claims description 24
- 238000012545 processing Methods 0.000 claims description 15
- 230000007246 mechanism Effects 0.000 claims description 14
- 238000004590 computer program Methods 0.000 claims description 12
- 230000009466 transformation Effects 0.000 claims description 8
- 238000004891 communication Methods 0.000 description 6
- 230000008439 repair process Effects 0.000 description 5
- 230000008569 process Effects 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 238000004422 calculation algorithm Methods 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000008030 elimination Effects 0.000 description 1
- 238000003379 elimination reaction Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000011176 pooling Methods 0.000 description 1
- 230000035040 seed growth Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/2163—Partitioning the feature space
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
- G06T7/0008—Industrial image inspection checking presence/absence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/12—Edge-based segmentation
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
- G01N2021/8887—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges based on image processing techniques
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Chemical & Material Sciences (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Artificial Intelligence (AREA)
- Health & Medical Sciences (AREA)
- Evolutionary Computation (AREA)
- Analytical Chemistry (AREA)
- Biochemistry (AREA)
- General Health & Medical Sciences (AREA)
- Immunology (AREA)
- Pathology (AREA)
- Quality & Reliability (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
Abstract
本发明公开了机场道面病害检测的方法、装置、电子设备及存储介质，包括：获取待检测机场道面病害图像；将待检测机场道面病害图像输入至训练好的病害检测模型，得到待检测机场道面病害图像中不同类型病害对应区域的提取结果；其中，训练好的病害检测模型为利用不同机场道面病害图像以及不同机场道面病害图像对应的语义分割真值图以及语义边缘真值图进行训练后得到。本发明通过病害检测模型实现了对机场道面图像中多种病害的像素级分割，提高了分割的精度。
The present invention discloses a method, device, electronic device and storage medium for detecting airport pavement defects, including: obtaining an airport pavement defect image to be detected; inputting the airport pavement defect image to be detected into a trained defect detection model to obtain extraction results of regions corresponding to different types of defects in the airport pavement defect image to be detected; wherein the trained defect detection model is obtained after training using different airport pavement defect images and semantic segmentation truth maps and semantic edge truth maps corresponding to different airport pavement defect images. The present invention realizes pixel-level segmentation of multiple defects in airport pavement images through the defect detection model, thereby improving the accuracy of segmentation.
Description
Technical Field
The invention relates to the technical field of computers, in particular to a method and a device for detecting airport pavement diseases, electronic equipment and a storage medium.
Background
Airport pavement safety detection is a serious problem of civil aviation operation safety, and the detection of airport pavement diseases is still mainly performed manually at present.
In the existing disease detection method, a secondary threshold segmentation algorithm is provided based on morphology, so that segmentation of cracks in images containing runway marks is realized, and the cracks are detected on multiple scales by using a seed growth algorithm, and the segmentation results of the cracks are obtained by matching and fusing crack areas on different scales. Most of the methods depend on parameters of manual design, have poor generalization capability, cannot process complicated and changeable airport pavement pictures, and have great detection precision influenced by environment. With the rise of deep learning, more and more disease detection methods based on deep learning are proposed. For example, the full convolution network is applied to fracture segmentation, and the fracture prediction of the fracture with different scales is performed by training the network by inputting multiple types of fracture pictures. However, this method only realizes disease detection based on a rectangular frame, but cannot realize finer pixel-level division. In addition, the shape of different types of diseases cannot be reflected, and the position information still has larger deviation.
In view of the foregoing, there is a need for a method for detecting airport pavement diseases, which is capable of solving the above-mentioned problems in the prior art.
Disclosure of Invention
In view of the above problems with the prior art, the present invention provides a method, apparatus, electronic device, and storage medium for detecting airport pavement diseases.
In a first aspect, the present invention provides a method of airport pavement disease detection comprising:
Acquiring an airport pavement disease image to be detected;
Inputting the airport pavement disease image to be detected into a trained disease detection model to obtain extraction results of different types of disease corresponding areas in the airport pavement disease image to be detected;
the trained disease detection model is obtained by training different airport pavement disease images and semantic segmentation truth diagrams and semantic edge truth diagrams corresponding to the different airport pavement disease images.
Further, before inputting the airport pavement disease image to be detected into a trained disease detection model to obtain an extraction result of a disease corresponding region in the airport pavement disease image to be detected, the method further comprises the steps of:
Obtaining a disease training image, a semantic segmentation truth value diagram and a semantic edge truth value diagram;
performing wavelet transformation on the disease training image to obtain a low-frequency characteristic image;
Extracting image features of the disease training image by adopting a residual error network to obtain a semantic edge feature image;
coding the disease training image based on the U-Net network according to the low-frequency feature image and the semantic edge feature image to obtain a first feature image;
Decoding the first feature map according to the low-frequency feature map and the semantic edge feature map to obtain a semantic segmentation prediction map;
determining a first loss function according to the semantic segmentation truth diagram and the semantic segmentation prediction diagram;
And updating parameters of the disease detection model according to the first loss function to obtain a trained disease detection model.
Further, the obtaining the semantic segmentation truth-value diagram and the semantic edge truth-value diagram includes:
Extracting the outline of the disease in the disease training image to obtain image annotation information;
Determining a semantic segmentation truth value diagram according to the image annotation information;
and extracting semantic edges of the disease area based on the semantic segmentation truth-value diagram to obtain a semantic edge truth-value diagram.
Further, after the image feature extraction is performed on the disease training image by adopting a residual network to obtain a semantic edge feature image, the method further comprises the following steps:
obtaining a channel domain weight matrix of the semantic edge feature map by adopting an attention mechanism;
obtaining a semantic edge prediction graph according to the semantic edge feature graph and the channel domain weight matrix;
Determining a second loss function according to the semantic edge truth diagram and the semantic edge prediction diagram;
and updating parameters of the disease detection model according to the second loss function.
Further, after the obtaining the first feature map, before the decoding the first feature map according to the low-frequency feature map and the semantic edge feature map, the method further includes:
adopting an attention mechanism to obtain a spatial domain weight matrix of the first feature map;
and determining a second feature map according to the first feature map and the spatial domain weight matrix.
Further, the types of the diseases in the image of the airport pavement diseases to be detected are cracks, repairs, patches, ground lights and plate seams.
In a second aspect, the present invention provides an apparatus for detecting an airport pavement disease, comprising:
The acquisition module is used for acquiring an airport pavement disease image to be detected;
The processing module is used for inputting the airport pavement disease images to be detected into a trained disease detection model to obtain extraction results of different types of disease corresponding areas in the airport pavement disease images to be detected, wherein the trained disease detection model is obtained after training by using different airport pavement disease images and semantic segmentation truth diagrams and semantic edge truth diagrams corresponding to the different airport pavement disease images.
Further, the processing module is further configured to:
before inputting the airport pavement disease image to be detected into a trained disease detection model to obtain an extraction result of a disease corresponding region in the airport pavement disease image to be detected, obtaining a disease training image, a semantic segmentation truth-value diagram and a semantic edge truth-value diagram;
performing wavelet transformation on the disease training image to obtain a low-frequency characteristic image;
Extracting image features of the disease training image by adopting a residual error network to obtain a semantic edge feature image;
coding the disease training image based on the U-Net network according to the low-frequency feature image and the semantic edge feature image to obtain a first feature image;
Decoding the first feature map according to the low-frequency feature map and the semantic edge feature map to obtain a semantic segmentation prediction map;
determining a first loss function according to the semantic segmentation truth diagram and the semantic segmentation prediction diagram;
And updating parameters of the disease detection model according to the first loss function to obtain a trained disease detection model.
Further, the processing module is specifically configured to:
Extracting the outline of the disease in the disease training image to obtain image annotation information;
Determining a semantic segmentation truth value diagram according to the image annotation information;
and extracting semantic edges of the disease area based on the semantic segmentation truth-value diagram to obtain a semantic edge truth-value diagram.
Further, the processing module is further configured to:
obtaining a channel domain weight matrix of the semantic edge feature map by adopting an attention mechanism;
obtaining a semantic edge prediction graph according to the semantic edge feature graph and the channel domain weight matrix;
Determining a second loss function according to the semantic edge truth diagram and the semantic edge prediction diagram;
and updating parameters of the disease detection model according to the second loss function.
Further, the processing module is further configured to:
After the first feature map is obtained, before the first feature map is decoded according to the low-frequency feature map and the semantic edge feature map, a spatial domain weight matrix of the first feature map is obtained by adopting an attention mechanism;
and determining a second feature map according to the first feature map and the spatial domain weight matrix.
Further, the processing module is specifically configured to:
The types of diseases in the airport pavement disease image to be detected are cracks, repairs, patches, floor lamps and plate seams.
In a third aspect, the present invention also provides an electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the method of airfield pavement disease detection according to the first aspect when executing the computer program.
In a fourth aspect, the present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which when executed by a processor implements the method of airport pavement disease detection according to the first aspect.
According to the technical scheme, the method, the device, the electronic equipment and the storage medium for detecting the airport pavement diseases, provided by the invention, realize pixel-level segmentation of various diseases in an airport pavement image through the disease detection model, and improve the segmentation precision.
Drawings
In order to more clearly illustrate the embodiments of the invention or the technical solutions of the prior art, the drawings that are necessary for the description of the embodiments or the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the invention and that other drawings can be obtained from these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a system frame of a method of airport pavement disease detection provided by the present invention;
FIG. 2 is a flow chart of a method for detecting airport pavement diseases provided by the invention;
FIG. 3 is a flow chart of a method for detecting airport pavement diseases provided by the invention;
FIG. 4 is a schematic diagram of wavelet transformation provided by the present invention;
FIG. 5 is a schematic diagram of a residual network provided by the present invention;
FIG. 6 is a schematic diagram of a residual network provided by the present invention;
FIG. 7 is a schematic diagram of a U-Net network provided by the present invention;
FIG. 8 is a schematic diagram of a U-Net network provided by the present invention;
FIG. 9 is a schematic diagram of an apparatus for detecting airport pavement diseases provided by the present invention;
fig. 10 is a schematic structural diagram of an electronic device provided by the present invention.
Detailed Description
The following describes the embodiments of the present invention further with reference to the accompanying drawings. The following examples are only for more clearly illustrating the technical aspects of the present invention, and are not intended to limit the scope of the present invention.
The method for detecting the airport pavement diseases provided by the embodiment of the invention can be applied to a system architecture shown in fig. 1, wherein the system architecture comprises an image sensor 100 and a disease detection model 200.
Specifically, the image sensor 100 is used to acquire an image of an airport pavement disease to be detected.
The disease detection model 200 is used for obtaining an extraction result of a disease corresponding region in the airport pavement disease image to be detected after inputting the airport pavement disease image to be detected.
The trained disease detection model is obtained by training different airport pavement disease images and semantic segmentation truth diagrams and semantic edge truth diagrams corresponding to the different airport pavement disease images.
It should be noted that fig. 1 is only an example of a system architecture according to an embodiment of the present invention, and the present invention is not limited thereto in particular.
Based on the system architecture illustrated above, fig. 2 is a schematic flow diagram corresponding to a method for detecting an airport pavement disease according to an embodiment of the present invention, as shown in fig. 2, where the method includes:
step 201, an airport pavement image to be detected is acquired.
Specifically, an image of an airport pavement disease in a real environment is captured by an image sensor such as a visible light camera.
Step 202, inputting the airport pavement disease image to be detected into a trained disease detection model to obtain an extraction result of a disease corresponding region in the airport pavement disease image to be detected.
The trained disease detection model is obtained by training different airport pavement disease images and semantic segmentation truth diagrams and semantic edge truth diagrams corresponding to the different airport pavement disease images.
According to the scheme, pixel-level segmentation of various diseases in the airport pavement image is realized through the disease detection model, and the segmentation accuracy is improved.
Before step 202, the flow of the steps of the embodiment of the present invention is shown in fig. 3, which specifically includes the following steps:
step 301, obtaining a disease training image, a semantic segmentation truth-value diagram and a semantic edge truth-value diagram.
Specifically, extracting the outline of the disease in the disease training image to obtain image annotation information;
Determining a semantic segmentation truth value diagram according to the image annotation information;
And extracting semantic edges of the disease area based on the semantic segmentation truth value graph to obtain a semantic edge truth value graph.
In one possible implementation, the disease training image is manually marked by LabelMe to obtain a semantic segmentation truth-value diagram of the airport pavement disease image, and then the corresponding semantic edge truth-value diagram is extracted from the semantic segmentation truth-value diagram by Matlab.
In the embodiment of the invention, the types of diseases in the images of the airport pavement diseases to be detected are cracks, repairs, patches, ground lights and plate seams.
The scheme considers the common typical categories in the two airfield pavement of the ground lamp and the slab joint to better detect the airfield pavement besides the three diseases of crack, repair and patch, and provides more detailed information.
And 302, performing wavelet transformation on the disease training image to obtain a low-frequency characteristic image.
In one possible implementation, the disease training image is haar wavelet transformed.
Specifically, as shown in fig. 4, four-stage haar wavelet transform is performed on an input disease training image, and the low-frequency part generated by each stage of wavelet transform is used as effective frequency domain information and used as input of the next stage of wavelet transform, thereby obtaining LL1, LL2, LL3 and LL4.
According to the scheme, the noise of the disease training image under the complex background is reduced by extracting the multi-scale low-frequency information.
And 303, extracting image features of the disease training image by adopting a residual error network to obtain a semantic edge feature image.
In one possible implementation, the residual network ResNet is used to perform image feature extraction on the disease training image.
Further, a attention mechanism is adopted to obtain a channel domain weight matrix of the semantic edge feature map;
Obtaining a semantic edge prediction graph according to the semantic edge feature graph and the channel domain weight matrix;
determining a second loss function according to the semantic edge truth diagram and the semantic edge prediction diagram;
And updating parameters of the disease detection model according to the second loss function.
As shown in fig. 5, the outputs of the five parts of the residual network are feature maps c1, c2, c3, c4, c5 of different scales, respectively.
Specifically, the output result of the first branch is obtained by performing a convolution operation on c 5. And deconvoluting the four feature maps c1, c2, c3 and c5 to be the same size by the second branch, splicing the feature maps together, and finally outputting a rough semantic edge prediction result through convolution operation.
In one possible implementation, the semantic edge prediction graph is compared to a semantic edge truth graph to calculate weighted cross entropy loss and back-propagated.
In the embodiment of the present invention, a specific calculation formula of the second loss function is as follows:
Wherein M represents the number of categories, w c represents the weight of category c, y ic is a sign function, N is the number of samples, if the true category of sample i is equal to c, 1 is taken, otherwise 0;p ic represents the prediction probability that the observed sample i belongs to category c.
The scheme can accelerate the convergence of the network and reduce the influence of noise contained in c1, c2, c3 and c4 on the prediction branch corresponding to c 5.
The scheme relieves the problems of tiny semantic edges, small quantity and extremely unbalanced duty ratio of foreground and background in the data set.
Further, as shown in fig. 6, since the collected image disease edge has a small proportion and generally only occupies one pixel, in order to improve the prediction effect of the semantic edge, in the embodiment of the invention, a channel domain attention mechanism is added after the c5 predicts the branch, that is, a channel domain weight matrix of the feature map c5 is obtained through the global pooling layer and the full connection layer, and the output after the c5 is multiplied by the weight matrix is convolved to obtain the semantic edge segmentation result.
According to the scheme, based on the residual network, the bottom features of the residual network are fused, so that the semantic segmentation result is less influenced by noise of the bottom features. The addition of the attention mechanism enables the network to better capture the tiny edge characteristics, so that more accurate prediction results are obtained.
And step 304, coding the disease training image based on the U-Net network according to the low-frequency feature map and the semantic edge feature map to obtain a first feature map.
As shown in fig. 7, the coding part is composed of four convolution blocks, where c1, c2, c3, and c5 are semantic edge feature maps, and LL1, LL2, LL3, and LL4 are low-frequency feature maps.
It should be noted that each convolution operation halves the feature map edge length.
For example, the input picture size is 512×512, and the feature size after four convolution operations is 32×32.
And step 305, decoding the first feature map according to the low-frequency feature map and the semantic edge feature map to obtain a semantic segmentation prediction map.
Further, the decoding portion includes four deconvolution layers, each deconvolution operation doubles the edge length of the feature map, for example, the input feature map size is 64×64, and the feature map size after four deconvolution operations is 512×512.
Based on the image, the disease training image is subjected to a U-Net network to obtain pixel-level semantic segmentation prediction graphs with the same size.
Step 306, determining a first penalty function according to the semantic segmentation truth-value graph and the semantic segmentation prediction graph.
In one possible implementation, a weighted cross entropy loss is calculated.
And step 307, updating parameters of the disease detection model according to the first loss function to obtain a trained disease detection model.
According to the scheme, the wavelet transformation is adopted to extract the image low-frequency information, and the image low-frequency information is fused through the U-Net network, so that noise elimination is facilitated, and the precision of semantic segmentation is improved. The multi-scale frequency domain information features, the semantic edge features and the image encoder extraction features are fused based on the U-Net network, so that high-precision pixel level segmentation of various diseases in the airport pavement image is realized, and the precision of semantic segmentation results and the generalization performance of the model are improved.
Further, before step 305, the embodiment of the present invention obtains a spatial domain weight matrix of the first feature map by using an attention mechanism;
And determining a second feature map according to the first feature map and the spatial domain weight matrix.
Specifically, as shown in fig. 8, the coding part is immediately followed by a spatial domain attention mechanism, and a spatial domain weight matrix of the first feature map is obtained by using convolution operation, and the weighted feature map, namely the second feature map, is output after the first feature map is multiplied by the weight matrix.
According to the scheme, the attention mechanism is added between the encoder and the decoder to weight the fusion characteristics, so that the accuracy of the semantic segmentation result is improved.
Based on the same inventive concept, fig. 9 illustrates an apparatus for detecting an airfield pavement disease, which may be a flow of a method for detecting an airfield pavement disease according to an embodiment of the present invention.
The device comprises:
An acquisition module 901, configured to acquire an image of a disease of an airport pavement to be detected;
the processing module 902 is configured to input the to-be-detected airport pavement disease image to a trained disease detection model to obtain an extraction result of different types of disease corresponding areas in the to-be-detected airport pavement disease image, where the trained disease detection model is obtained by training with different airport pavement disease images and semantic segmentation truth diagrams and semantic edge truth diagrams corresponding to the different airport pavement disease images.
Further, the processing module 902 is further configured to:
before inputting the airport pavement disease image to be detected into a trained disease detection model to obtain an extraction result of a disease corresponding region in the airport pavement disease image to be detected, obtaining a disease training image, a semantic segmentation truth-value diagram and a semantic edge truth-value diagram;
performing wavelet transformation on the disease training image to obtain a low-frequency characteristic image;
Extracting image features of the disease training image by adopting a residual error network to obtain a semantic edge feature image;
coding the disease training image based on the U-Net network according to the low-frequency feature image and the semantic edge feature image to obtain a first feature image;
Decoding the first feature map according to the low-frequency feature map and the semantic edge feature map to obtain a semantic segmentation prediction map;
determining a first loss function according to the semantic segmentation truth diagram and the semantic segmentation prediction diagram;
And updating parameters of the disease detection model according to the first loss function to obtain a trained disease detection model.
Further, the processing module 902 is specifically configured to:
Extracting the outline of the disease in the disease training image to obtain image annotation information;
Determining a semantic segmentation truth value diagram according to the image annotation information;
and extracting semantic edges of the disease area based on the semantic segmentation truth-value diagram to obtain a semantic edge truth-value diagram.
Further, the processing module 902 is further configured to:
After the residual error network is adopted to extract image features of the disease training image to obtain a semantic edge feature image, a attention mechanism is adopted to obtain a channel domain weight matrix of the semantic edge feature image;
obtaining a semantic edge prediction graph according to the semantic edge feature graph and the channel domain weight matrix;
Determining a second loss function according to the semantic edge truth diagram and the semantic edge prediction diagram;
and updating parameters of the disease detection model according to the second loss function.
Further, the processing module 902 is further configured to:
After the first feature map is obtained, before the first feature map is decoded according to the low-frequency feature map and the semantic edge feature map, a spatial domain weight matrix of the first feature map is obtained by adopting an attention mechanism;
and determining a second feature map according to the first feature map and the spatial domain weight matrix.
Further, the processing module 902 is specifically configured to:
The types of diseases in the airport pavement disease image to be detected are cracks, repairs, patches, floor lamps and plate seams.
Based on the same inventive concept, a further embodiment of the invention provides an electronic device, see fig. 10, comprising in particular a processor 1001, a memory 1002, a communication interface 1003 and a communication bus 1004;
The processor 1001, the memory 1002 and the communication interface 1003 complete communication with each other through the communication bus 1004, wherein the communication interface 1003 is used for realizing information transmission between all devices;
The processor 1001 is configured to invoke a computer program in the memory 1002, and when the processor executes the computer program, all steps of the method for detecting an airport pavement disease are implemented, for example, when the processor executes the computer program, the steps are implemented to obtain an airport pavement disease image to be detected, input the airport pavement disease image to a trained disease detection model to obtain an extraction result of a region corresponding to the disease in the airport pavement disease image to be detected, where the trained disease detection model is obtained after training by using different airport pavement disease images and semantic segmentation truth diagrams and semantic edge truth diagrams corresponding to the different airport pavement disease images.
Based on the same inventive concept, a further embodiment of the present invention provides a non-transitory computer readable storage medium, on which a computer program is stored, which when executed by a processor implements all the steps of the above-mentioned method for detecting airfield pavement diseases, for example, the processor implements the steps of obtaining an airfield pavement disease image to be detected, inputting the airfield pavement disease image to a trained disease detection model to obtain an extraction result of a disease corresponding region in the airfield pavement disease image to be detected, wherein the trained disease detection model is obtained after training by using different airfield pavement disease images and semantic segmentation truth diagrams and semantic edge truth diagrams corresponding to the different airfield pavement disease images.
Further, the logic instructions in the memory described above may be implemented in the form of software functional units and stored in a computer-readable storage medium when sold or used as a stand-alone product. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in the form of a software product stored in a storage medium, comprising instructions for causing a computer device (which may be a personal computer, means for detecting an airport pavement disease, or network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. The storage medium includes a U disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, an optical disk, or other various media capable of storing program codes.
The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules can be selected according to actual needs to achieve the purpose of the embodiment of the invention. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
From the above description of the embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus necessary general hardware platforms, or of course may be implemented by means of hardware. Based on this understanding, the above technical solution may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a computer readable storage medium, such as ROM/RAM, a magnetic disk, an optical disk, etc., comprising several instructions for causing a computer device (which may be a personal computer, an apparatus for detecting airport pavement diseases, or a network device, etc.) to perform the method for detecting airport pavement diseases according to the embodiments or some parts of the embodiments.
Furthermore, in the present disclosure, such as "first," "second," and the like, are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include at least one such feature. In the description of the present invention, the meaning of "plurality" means at least two, for example, two, three, etc., unless specifically defined otherwise.
Moreover, in the present invention, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises an element.
Furthermore, in the description herein, reference to the terms "one embodiment," "some embodiments," "examples," "specific examples," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, schematic representations of the above terms are not necessarily directed to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, the different embodiments or examples described in this specification and the features of the different embodiments or examples may be combined and combined by those skilled in the art without contradiction.
It should be noted that the above-mentioned embodiments are merely for illustrating the technical solution of the present invention, and not for limiting the same, and although the present invention has been described in detail with reference to the above-mentioned embodiments, it should be understood by those skilled in the art that the technical solution described in the above-mentioned embodiments may be modified or some technical features may be equivalently replaced, and these modifications or substitutions do not make the essence of the corresponding technical solution deviate from the spirit and scope of the technical solution of the embodiments of the present invention.
Claims (9)
1. A method of airport pavement disease detection comprising:
Acquiring an airport pavement disease image to be detected;
The method comprises the steps of obtaining a disease training image, a semantic segmentation truth value chart and a semantic edge truth value chart, carrying out wavelet transformation on the disease training image to obtain a low-frequency feature chart, carrying out image feature extraction on the disease training image by adopting a residual error network to obtain a semantic edge feature chart, encoding the disease training image according to the low-frequency feature chart and the semantic edge feature chart on the basis of a U-Net network to obtain a first feature chart, decoding the first feature chart according to the low-frequency feature chart and the semantic edge feature chart to obtain a semantic segmentation prediction chart, determining a first loss function according to the semantic segmentation truth value chart and the semantic segmentation prediction chart, updating parameters of a disease detection model according to the first loss function to obtain a trained disease detection model;
Inputting the airport pavement disease image to be detected into a trained disease detection model to obtain extraction results of different types of disease corresponding areas in the airport pavement disease image to be detected;
the trained disease detection model is obtained by training different airport pavement disease images and semantic segmentation truth diagrams and semantic edge truth diagrams corresponding to the different airport pavement disease images.
2. The method of claim 1, wherein the obtaining a semantic segmentation truth-chart and a semantic edge truth-chart comprises:
Extracting the outline of the disease in the disease training image to obtain image annotation information;
Determining a semantic segmentation truth value diagram according to the image annotation information;
and extracting semantic edges of the disease area based on the semantic segmentation truth-value diagram to obtain a semantic edge truth-value diagram.
3. The method for detecting the airport pavement diseases according to claim 1, wherein after the image feature extraction is performed on the disease training image by adopting a residual network, the method further comprises:
obtaining a channel domain weight matrix of the semantic edge feature map by adopting an attention mechanism;
obtaining a semantic edge prediction graph according to the semantic edge feature graph and the channel domain weight matrix;
Determining a second loss function according to the semantic edge truth diagram and the semantic edge prediction diagram;
and updating parameters of the disease detection model according to the second loss function.
4. The method of airport pavement disease detection of claim 1, wherein after said obtaining a first signature, said decoding said first signature from said low frequency signature and said semantic edge signature further comprises:
adopting an attention mechanism to obtain a spatial domain weight matrix of the first feature map;
and determining a second feature map according to the first feature map and the spatial domain weight matrix.
5. The method for detecting the airport pavement diseases according to claim 1, wherein the types of the diseases in the airport pavement disease image to be detected are cracks, patches, floor lamps and slab joints.
6. An apparatus for detecting an airport pavement disease, comprising:
The system comprises an acquisition module, a processing module and a processing module, wherein the acquisition module is used for acquiring an airport pavement disease image to be detected, acquiring a disease training image, a semantic segmentation truth value diagram and a semantic edge truth value diagram, performing wavelet transformation on the disease training image to obtain a low-frequency feature diagram, performing image feature extraction on the disease training image by adopting a residual error network to obtain a semantic edge feature diagram, encoding the disease training image according to the low-frequency feature diagram and the semantic edge feature diagram based on a U-Net network to obtain a first feature diagram, decoding the first feature diagram according to the low-frequency feature diagram and the semantic edge feature diagram to obtain a semantic segmentation prediction diagram, determining a first loss function according to the semantic segmentation truth value diagram and the semantic segmentation prediction diagram, updating parameters of a disease detection model according to the first loss function, and obtaining a trained disease detection model;
The processing module is used for inputting the airport pavement disease images to be detected into a trained disease detection model to obtain extraction results of different types of disease corresponding areas in the airport pavement disease images to be detected, wherein the trained disease detection model is obtained after training by using different airport pavement disease images and semantic segmentation truth diagrams and semantic edge truth diagrams corresponding to the different airport pavement disease images.
7. A computer program product comprising a computer program which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 5.
8. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the steps of the method according to any one of claims 1 to 5 when the program is executed.
9. A non-transitory computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111143345.9A CN114155375B (en) | 2021-09-28 | 2021-09-28 | Method, device, electronic equipment and storage medium for detecting airport pavement defects |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111143345.9A CN114155375B (en) | 2021-09-28 | 2021-09-28 | Method, device, electronic equipment and storage medium for detecting airport pavement defects |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114155375A CN114155375A (en) | 2022-03-08 |
| CN114155375B true CN114155375B (en) | 2025-03-28 |
Family
ID=80462249
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111143345.9A Active CN114155375B (en) | 2021-09-28 | 2021-09-28 | Method, device, electronic equipment and storage medium for detecting airport pavement defects |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114155375B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114579876B (en) * | 2022-03-10 | 2025-02-25 | 中国科学院空天信息创新研究院 | False information detection method, device, equipment and medium |
| CN114594103B (en) * | 2022-04-12 | 2023-05-16 | 四川大学 | Automatic detection and report generation method and system for surface defects of nuclear industrial equipment |
| CN114463187B (en) * | 2022-04-14 | 2022-06-17 | 合肥高维数据技术有限公司 | Image semantic segmentation method and system based on aggregation edge features |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP6454109B2 (en) * | 2014-09-10 | 2019-01-16 | 雄章 石川 | Road surface state management device and road surface state management program |
| CN110210407A (en) * | 2019-06-04 | 2019-09-06 | 武汉科技大学 | A salient target detection method in foggy images |
| CN110956183B (en) * | 2019-11-04 | 2023-05-02 | 东南大学 | A method for extracting the crack shape of asphalt pavement |
| CN112634192B (en) * | 2020-09-22 | 2023-10-13 | 广东工业大学 | A cascaded U-N Net brain tumor segmentation method combined with wavelet transform |
-
2021
- 2021-09-28 CN CN202111143345.9A patent/CN114155375B/en active Active
Non-Patent Citations (1)
| Title |
|---|
| A Three-Stream Boundary-Aware Network for Fine-Crained pavement Disease Segmentation;Yujia Zhang et al.;2021 IEEE Winter Conference on Application of Computer Vision》;20210103;正文第1、2.1、2.4、3.1节，附图1 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114155375A (en) | 2022-03-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114155375B (en) | Method, device, electronic equipment and storage medium for detecting airport pavement defects | |
| CN109583345B (en) | Road recognition method, device, computer device and computer readable storage medium | |
| CN106530266B (en) | A kind of infrared and visible light image fusion method based on region rarefaction representation | |
| CN112232391A (en) | Dam crack detection method based on U-net network and SC-SAM attention mechanism | |
| Jiang et al. | Focal-test-based spatial decision tree learning | |
| Liu et al. | Semantic segmentation and photogrammetry of crowdsourced images to monitor historic facades | |
| CN110852316A (en) | Image tampering detection and positioning method adopting convolution network with dense structure | |
| US11410300B2 (en) | Defect inspection device, defect inspection method, and storage medium | |
| CN116524189B (en) | A semantic segmentation method for high-resolution remote sensing images based on codec-indexed edge representation | |
| CN111797920B (en) | Remote sensing extraction method and system for depth network impervious surface with gate control feature fusion | |
| CN110532914A (en) | Building analyte detection method based on fine-feature study | |
| CN112785480B (en) | Image stitching forgery detection method based on frequency domain transformation and residual feedback module | |
| CN112070159A (en) | A Multi-scale Sliding Window Similar Image Detection Method | |
| CN115457044A (en) | Pavement crack segmentation method based on class activation mapping | |
| CN117095173A (en) | Colorectal cancer H & E staining pathological image semantic segmentation method and system | |
| CN114882367B (en) | A method for detecting and evaluating airport pavement defects | |
| CN114782949B (en) | Traffic scene semantic segmentation method for boundary guide context aggregation | |
| Zhang et al. | Unsupervised object-based spectral unmixing for subpixel mapping | |
| Jin et al. | Registration of UAV images using improved structural shape similarity based on mathematical morphology and phase congruency | |
| CN116309612B (en) | Semiconductor silicon wafer detection method, device and medium based on frequency decoupling supervision | |
| CN117237859B (en) | Visibility detection method in foggy days on highways at night based on low light enhancement | |
| CN117470137A (en) | Land planning and design land area measurement method and system | |
| CN113887499B (en) | A dune image recognition model and its creation method and dune image recognition method | |
| CN114494184B (en) | A PCB defect detection method based on improved fully convolutional neural network | |
| Zhang et al. | GFU-Net: A deep learning approach for automatic metal crack detection |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |