CN114882367B - A method for detecting and evaluating airport pavement defects - Google Patents

Abstract

The present invention relates to an airport pavement defect detection and state assessment method, comprising the following steps: S1: determine the target airport pavement range, obtain the target airport pavement video, and obtain the video frame image; S2: input all video frame images into the depth estimation network, and obtain the depth map corresponding to all video frame images; S3: splice all video frame images into a full-size pavement RGB image, and splice the depth map into a full-size pavement depth map; S4: process the full-size pavement RGB image and the full-size pavement depth map into a local RGB image and a local grayscale image, respectively, and input them into a semantic segmentation network to obtain a segmentation result mask map; S5: splice the segmentation result mask map into a full-size defect mask map, and combine it with the full-size pavement depth map to judge the severity of different defects of the target airport pavement, and then evaluate the pavement quality status according to the road surface condition index. Compared with the prior art, the invention can achieve efficient and accurate evaluation of the airport pavement condition.

Description

A method for detecting and evaluating airport pavement defects

Technical Field

The invention relates to the technical field of computer image segmentation, and in particular to an airport pavement defect detection and state assessment method.

Background Art

Due to the large amount of passenger and cargo transportation, airport pavement inevitably suffers from various diseases. The pavement condition is calculated according to the severity of the disease, and the calculated Pavement Condition Index (PCI) is used to remind and assist the pavement inspection to ensure the safe takeoff and landing of aircraft.

The pavement condition index (PCI) is an indicator for evaluating the degree of road damage. It combines the quantitative conditions of the type of pavement defects, the degree of damage, and the scope or density of damage. The pavement damage grade assessment criteria are shown in Table 1.

Table 1 Pavement structure damage level assessment standard

Pavement damage level Difference Second-rate middle good excellent PCI Scope [0,40) [40,55) [55,70) [70,85) [85,100)

Traditional airport pavement defect detection requires on-site measurement by professionals, which has the following problems:

(1) Since the airport pavement carries a large number of takeoffs and landings, the pavement appearance contrast is relatively low, which increases the difficulty of detection and limits the detection efficiency;

(2) Since the aircraft's takeoff and landing activities need to be suspended during manual inspection, the inspection cost is further increased.

In recent years, deep learning has developed rapidly, and using deep learning-based methods to replace manual inspection has become one of the trends in airport pavement defect detection. The current mainstream method is to collect 2D image data and detect airport pavement defects through semantic segmentation methods based on neural networks. However, this method can only use surface features to distinguish different defects. When it comes to defects with small differences in texture features but large differences in depth features (such as cracks and scratches), the detection accuracy is poor.

Therefore, a method of integrating 2D and 3D features of pavement is needed to improve the efficiency and accuracy of airport pavement defect detection. However, the simultaneous acquisition of RGB images and depth images is prone to errors, and the simultaneous annotation of two modal data requires high labor costs.

Summary of the invention

The purpose of the present invention is to provide a method for detecting and evaluating the defects of the above-mentioned prior art, which can realize efficient and accurate evaluation of the condition of the airport pavement.

The purpose of the present invention can be achieved by the following technical solutions:

The present invention provides an airport pavement defect detection and status assessment method, comprising the following steps:

S1: Determine the target airport pavement range, obtain the target airport pavement video, and extract the video frame to obtain the video frame image;

S2: Input all the video frame images obtained in S1 into the depth estimation network in sequence to obtain the depth maps corresponding to all the video frame images;

S3: stitch all the video frame images obtained by S1 into a full-size road surface RGB image, and stitch the depth images obtained by S2 into a full-size road surface depth image;

S4: Process the full-size road surface RGB image and the full-size road surface depth image obtained in S3 into a local RGB image and a local grayscale image respectively, and input them into the semantic segmentation network to obtain a segmentation result mask image;

S5: The segmentation result mask map obtained in S4 is spliced into a full-size defect mask map, and combined with the full-size pavement depth map obtained in S3, the severity of different defects of the target airport pavement is determined, and then the pavement quality condition is evaluated according to the pavement condition index.

Preferably, the depth estimation network in S2 adopts a monocular unsupervised depth estimation network based on PackNet, and the monocular unsupervised depth estimation network includes a PackNet network branch for generating 2D image depth information and a Pose Convnet network branch for generating pose information between adjacent video frame images.

Preferably, S2 comprises the following steps:

S2.1: Input the video frame image I d at time _d into the PackNet network branch to generate the initial depth map of I _d , and input the video frame images I _s and I _{d at time s and time d} into the Pose ConvNet network branch to generate the pose information between the video frame images at time s and time d;

S2.2: Obtain the reconstructed image data at time d based on the initial depth map of I _d obtained in S2.1, the pose information between the video frame images at time s and time d, and the video frame image I _s at time s

S2.3: Reconstructed image data at time d obtained according to S2.2 The Loss function of the PackNet network branch is calculated based on the video frame image I _d at time d, and the Loss function of the Pose ConvNet network branch is obtained according to the translation vector t _d->s , the instantaneous speed v of the monocular camera at time d, and the time difference Δt _d->s between time s and time d, so that the depth estimation network updates and iterates itself to generate a depth map corresponding to the video frame image at time d;

S2.4: Repeat S2.1 to S2.3 to obtain depth maps corresponding to all video frame images.

Preferably, S3 comprises the following steps:

S3.1: extract feature points of two adjacent video frame images, namely, the first video frame image and the second video frame image, respectively, generate feature descriptors, and perform fast approximate nearest neighbor matching to obtain matching points corresponding to the two adjacent video frame images;

S3.2: Eliminate the matching points with large errors in S3.1 and obtain multiple sets of reliable matching point pairs;

S3.3: Decompose the two video frame images in S3.1 to obtain the homography matrix H ₁₂ of the two video frame images;

S3.4: According to the homography matrix H ₁₂ , the second video frame image is converted to the pixel plane of the first video frame image for image stitching to obtain a stitched video frame image, and the depth map corresponding to the second video frame image is converted to the pixel plane of the depth map corresponding to the first video frame image for image stitching to obtain a stitched depth map;

S3.5: Perform the operations of S3.1 to S3.4 on the depth maps corresponding to all the video frame images obtained by S1 and all the video frame images obtained by S2, so as to obtain a full-size road surface RGB map and a full-size road surface depth map.

Preferably, the semantic segmentation network in S4 adopts a U-Net semantic segmentation network with feature layer fusion.

Preferably, the semantic segmentation network in S4 includes an encoder and a decoder, the encoder includes an RGB encoder and a depth encoder, the RGB encoder and the depth encoder each include a plurality of encoding layers, each of the encoding layers includes a convolutional layer, a pooling layer and an activation layer, the decoding layer includes a plurality of decoding layers, each of the decoding layers includes a convolutional layer, an upsampling layer and an activation layer.

Preferably, the RGB encoder and the depth encoder each include four encoding layers, and the decoder includes three decoding layers.

Preferably, the S4 comprises the following steps:

S4.1: Process the full-size road surface RGB image and the full-size road surface depth image obtained in S3 into multiple local RGB images and local grayscale images respectively;

S4.2: Convolve the local RGB image and the local grayscale image respectively to obtain a first RGB feature image and a first depth feature image;

S4.3: Input the first RGB feature map into the RGB encoder to obtain a second RGB feature map group, and input the first depth feature map into the depth encoder to obtain a second depth feature map group;

S4.4: splicing the feature maps of the same size in the two feature map groups obtained in S4.3 in channel dimension, and obtaining the corresponding spliced feature maps in sequence;

S4.5: The smallest concatenated feature map in S4.4 is used as the input of the decoder, and is combined with the remaining concatenated feature maps in S4.4, and is decoded and output by the decoder to obtain a third feature map;

S4.6: Perform a convolution operation on the third feature map obtained in S4.4 to obtain mask maps of segmentation results of different types of defects.

Preferably, S5 is specifically:

The segmentation result mask map obtained in S4 is spliced into a full-size defect mask map, and the defect areas of different types are obtained according to the full-size defect mask map. According to the full-size pavement depth map obtained in S3, the depth and length of the full-size defect mask map are calculated, and then the damage severity of different defects of the target airport pavement is judged according to the pavement condition index.

Preferably, the calculation formula of the road condition index PCI is specifically:

In the formula, _a0 and _a1 are the pavement material coefficients, and DR is the pavement damage rate (%).

Compared with the prior art, the present invention has the following advantages:

1. The present invention uses a depth estimation network to mine 3D depth information in 2D video frame images, solving the problem of high dependence on synchronous multi-modal data acquisition technology in the prior art, and further solving the problem of high labor costs required for simultaneous labeling of multiple modal data in the prior art;

2. The present invention uses an improved U-Net semantic segmentation network to fuse the 2D texture data and 3D depth data of the road surface at the feature layer, solving the problem of reduced accuracy caused by the lack of 3D road surface information in the prior art;

3. The present invention realizes efficient and accurate evaluation of airport pavement conditions based on semantic segmentation masks and estimated depth values in combination with national standards for PCI measurement.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a schematic flow chart of a method for detecting and assessing airport pavement defects provided by this embodiment;

FIG. 2 is a schematic diagram of the structure of the collection device in the embodiment shown in FIG. 1 .

FIG3 is a schematic diagram of the network structure of a monocular unsupervised depth estimation network based on PackNet in the embodiment shown in FIG1 .

FIG4 is a schematic diagram of the network structure of the semantic segmentation network in the embodiment shown in FIG1 .

FIG. 5 is a flow chart of a splicing algorithm based on SIFT descriptors in the embodiment shown in FIG. 1 .

FIG. 6 is a flow chart of evaluating road surface defect conditions based on the segmentation result mask in the embodiment shown in FIG. 1 .

Description of the markings in the figure:

1. Collection vehicle, 2. Fixed bracket, 3. Monocular camera, 4. Target airport pavement, 5. Single shot of the pavement.

DETAILED DESCRIPTION

The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

Example

This embodiment provides an airport pavement defect detection and status assessment method, comprising the following steps:

S1: Determine the target airport pavement range, obtain the target airport pavement video through the acquisition device, and extract the video frame to obtain the video frame image;

As an optional implementation, a rectangle uniquely determined by four vertices is used to describe the range of the target airport runway surface.

As an optional embodiment, referring to FIG2 , the acquisition device includes a acquisition vehicle 1, a fixed bracket 2 and a monocular camera 3, one end of the fixed bracket 2 is fixed to the acquisition vehicle 1, and the other end of the fixed bracket 2 is installed with the monocular camera 3, and the lens plane of the monocular camera 3 is parallel to the target airport runway surface 4.

As an optional implementation, the acquisition device is operated to pass over the target airport pavement 4 at a speed not exceeding 45 km/h, and video is shot at the same time. After the shooting is completed, video frames are extracted at a standard of 20 frames per second to obtain video frame images. The range of the video frame image is equal to the size of the pavement 5 that can be shot at a single time.

Specifically, the purpose of extracting video frames is to make adjacent video frames have overlapping areas to facilitate subsequent depth estimation.

S2: Input all the video frame images obtained in S1 into the depth estimation network in sequence to generate the depth maps corresponding to all the video frame images;

As an optional implementation, the depth estimation network in S2 adopts a monocular unsupervised depth estimation network based on PackNet. Referring to FIG3 , the network includes a PackNet network branch for generating 2D image depth information and a Pose Convnet network branch for generating pose information between adjacent video frame images.

Among them, the PackNet network includes an encoding-decoding structure, using packing and unpacking to replace traditional upsampling and downsampling operations. In the encoding part, residual blocks are used to replace ordinary convolutional layers, and packing is used between residual blocks to implement downsampling operations. After each downsampling operation, the size of the feature map is reduced by one time; in the decoding part, unpacking is used between convolutional blocks to implement upsampling operations. After each upsampling operation, the size of the feature map will be doubled, and finally a depth information map with the same length and width as the original image is generated.

S2.1: Input the video frame image I d at time _d into the PackNet network branch to generate the initial depth map of I _d , and input the video frame images I _s and I _{d at time s and time d} into the Pose ConvNet network branch to generate the pose information between the video frame images at time s and time d.

Specifically, the pose information includes the rotation matrix R and the translation vector t.

S2.2: Obtain the reconstructed image data at time d based on the initial depth map of I _d obtained in S2.1, the pose information between the video frame images at time s and time d, and the video frame image I _s at time s

S2.3: Reconstructed image data at time d obtained according to S2.2 The Loss function of the PackNet network branch is calculated based on the video frame image I _d at time d, and the Loss function of the Pose ConvNet network branch is obtained according to the translation vector t _d->s , the instantaneous velocity v of the monocular camera at time d, and the time difference Δt _d->s between time s and time d, so that the depth estimation network updates and iterates itself to generate a depth map corresponding to the video frame image at time d.

Specifically, t _d->s is the translation vector t between the video frame image I _s at time s and the video frame image I _d at time d.

Specifically, the formula describing the Loss function of the PackNet network branch is:

In the formula, is the video frame image I _d at time d and the reconstructed image data at time d The structural similarity between them, a is a hyperparameter, with a value between 0 and 1. is the video frame image I _d at time d and the reconstructed image data at time d The modulus of the difference between is the Loss function of the PackNet network branch.

Specifically, the formula describing the Loss function of the Pose ConvNet network branch is:

L _v (t _d-＞s ,v)＝||t _d-＞s ||-|v||Δt _d-＞s |

Wherein, ||t _d->s || is the modulus of the translation vector between the video frame image I _d at time d and the video frame image I _{s at time s} , |v| is the modulus of the instantaneous velocity of the monocular camera at time d, and |Δt _d->s | is the modulus of the time difference between time d and time s.

S2.4: Repeat S2.1 to S2.3 to obtain depth maps corresponding to all video frame images.

S3: stitch all the video frame images obtained by S1 into a full-size road surface RGB image, and stitch the depth images obtained by S2 into a full-size road surface depth image;

As an optional implementation, a stitching algorithm based on SIFT descriptor is used to stitch all video frame images obtained by S1 into a full-size road surface RGB image. The purpose of this step is to remove the overlapping areas between different video frames, avoid differences in segmentation results of the same area in different pictures during the semantic segmentation process, and reduce the subsequent calculation amount.

Referring to FIG5 , taking two adjacent video frame images RGB ₁ and RGB ₂ and their corresponding depth maps Depth1 and Depth2 as an example, S3 includes the following steps:

S3.1: Extract SIFT feature points of RGB ₁ and RGB ₂ of video frame images respectively, generate SIFT feature descriptors, and perform fast approximate nearest neighbor matching to obtain matching points corresponding to two adjacent video frame images;

S3.2: Use the RANSAC algorithm to remove the matching points with large errors in S3.1 and obtain multiple sets of reliable matching point pairs;

S3.3: Decompose the video frame images RGB ₁ and RGB ₂ using SVD to obtain the homography matrix H ₁₂ of the video frame images RGB ₁ and RGB ₂ ;

S3.4: According to the homography matrix H ₁₂ , the video frame image RGB ₂ is converted to the pixel plane where RGB ₁ is located for image stitching to obtain the stitched video frame image RGB ₁₂ , and Depth2 is converted to the pixel plane where Depth1 is located for image stitching to obtain the stitched depth map Depth ₁₂ .

S3.5: Perform the operations of S3.1 to S3.4 on the depth maps corresponding to all the video frame images obtained by S1 and all the video frame images obtained by S2, so as to obtain a full-size road surface RGB map and a full-size road surface depth map.

S4: Segment the full-size road surface RGB image and the full-size road surface depth image obtained in S3 into a local RGB image and a local grayscale image respectively, and input them into a semantic segmentation network to obtain a segmentation result mask image;

As an optional implementation, the semantic segmentation network adopts a U-Net semantic segmentation network with feature layer fusion, as shown in Figure 4, the network includes an encoder and a decoder, the input of the network is the processed RGB image and the corresponding depth image, and the output of the network is the segmentation result mask image.

Specifically, the encoder includes an RGB encoder and a depth encoder. The RGB encoder and the depth encoder each include multiple encoding layers. Each encoding layer includes a convolution layer, a pooling layer, and an activation layer. After each convolution layer of an encoding layer, the length and width of the feature map are halved and the number of channels is doubled. The decoder includes multiple decoding layers. Each decoding layer includes a convolution layer, an upsampling layer, and an activation layer. After each convolution layer of a decoding layer, the length and width of the feature map are doubled and the number of channels is halved.

As an optional implementation, the RGB encoder and the depth encoder each include four encoding layers, and the decoder includes three decoding layers.

Preferably, the full-size road surface RGB image and the full-size road surface depth image are processed into a local RGB image of 1024*512*3 and a local grayscale image of 1024*512*1 respectively, and S4 specifically includes the following steps:

S4.1: Process the full-size road surface RGB image and the full-size road surface depth image obtained in S3 into multiple 1024*512*3 local RGB images and 1024*512*1 local grayscale images respectively;

S4.2: Perform 1*1 convolution on the processed local RGB image and local grayscale image respectively to obtain the first RGB feature map and the first depth feature map, both of which have a shape of 1024*512*16;

S4.3: Input the first RGB feature map into the RGB encoder to obtain a second RGB feature map group, and input the first depth feature map into the depth encoder to obtain a second depth feature map group;

Specifically, when the first RGB feature map is input into the RGB encoder, a second RGB feature map is obtained after each encoding layer. Since there are four encoding layers, there are four second RGB feature maps in the second RGB feature map group; correspondingly, there are also four second depth feature maps in the second depth feature map group.

And the sizes of the feature maps of the second RGB feature map group and the second depth feature map group are equal. The sizes of the feature maps in the two feature map groups from input to output are: 512*256*32, 256*128*64, 128*64*128, 64*32*256;

S4.4: Concatenate the feature maps of the same size in the two feature map groups obtained in S4.3 in terms of channel dimension to obtain four concatenated feature maps, the shapes of which are 512*256*64, 256*128*128, 128*64*256, and 64*32*512 respectively;

S4.5: The smallest concatenated feature map in S4.4 is used as the input of the decoder, and is combined with the remaining concatenated feature maps in S4.4, and is decoded and output by the decoder to obtain a third feature map.

Specifically, a spliced feature map with a size of 64*32*512 is input into the decoder. At the first decoding layer, the spliced feature map with a size of 64*32*512 and the spliced feature map with a size of 128*64*256 are processed by the decoding layer into a feature map with a size of 128*64*512, and the feature map with a size of 128*64*512 is combined with the feature map with a size of 256*128*128 and processed by the second decoding layer into a feature map with a size of 256*128*256. The feature map with a size of 256*128*256 is combined with the feature map with a size of 512*256*64 and processed by the third decoding layer to finally obtain a third feature map with a size of 512*256*128.

S4.6: Perform two convolution operations on the third feature map to obtain a mask map of different types of defect segmentation results with a size of 1024*512*1.

To ensure the fusion of features of different modalities, the RGB image and its corresponding depth image need to be input into different encoders, called RGB encoder and depth encoder, respectively, and the feature maps of the two branches at different stages need to be spliced in the channel dimension.

In order to ensure the fusion of features of different scales, the size of the feature map output by encoding must be consistent with the size of the feature map output by decoding, and the two feature maps must be spliced in the channel dimension.

S5: The segmentation result mask map obtained in S4 is spliced into a full-size defect mask map, and combined with the full-size pavement depth map obtained in S3, the severity of different defects of the target airport pavement is determined, and then the pavement quality condition is evaluated according to the pavement condition index.

Specifically, referring to FIG6 , the defect areas of different types are obtained according to the full-size defect mask map, and the depth and length of the full-size defect mask map are calculated according to the full-size pavement depth map obtained in S3, so as to judge the damage severity of different defects of the target airport pavement.

The calculation formula of the road condition index PCI is as follows:

Wherein, _a0 and _a1 are the pavement material coefficients. When the target airport pavement is cement pavement, _a0 = 10.66, _a1 = 0.461; when the target airport pavement is asphalt pavement, _a0 = 15, _a1 = 0.412; DR is the pavement damage rate (%), that is, the percentage of the sum of the reduced damaged areas of different types of defects to the total pavement area. The calculation formula for the pavement damage rate is:

Where i ₀ is the total number of defect categories. When the target airport pavement is a cement pavement, i ₀ = 15; when the target airport pavement is an asphalt pavement, i ₀ = 16. A is the total area of the target pavement (square meters);

A _i is the area of the i-th defect (square meters);

w _i is the weight of the i-th defect. Its value and the basis for judging the degree of damage on cement pavement and asphalt pavement are shown in Table 2 and Table 3 respectively:

Table 2 Cement pavement defect weights

Table 3 Asphalt pavement defect weights

The preferred specific embodiments of the present invention are described in detail above. It should be understood that a person skilled in the art can make many modifications and changes based on the concept of the present invention without creative work. Therefore, any technical solution that can be obtained by a person skilled in the art through logical analysis, reasoning or limited experiments based on the concept of the present invention on the basis of the prior art should be within the scope of protection determined by the claims.

Claims (4)

1. An airport pavement defect detection and state assessment method is characterized by comprising the following steps:

s1: determining a target airfield pavement range, acquiring a target airfield pavement video, extracting a video frame, and acquiring a video frame image;

s2: sequentially inputting all the video frame images acquired in the step S1 into a depth estimation network to acquire depth maps corresponding to all the video frame images;

S3: all video frame images acquired in the step S1 are spliced into full-size road surface RGB images, and the depth images acquired in the step S2 are spliced into full-size road surface depth images;

S4: processing the full-size road surface RGB image and the full-size road surface depth image obtained in the step S3 into a partial RGB image and a partial gray image respectively, inputting a semantic segmentation network, and obtaining a segmentation result mask image;

S5: splicing the segmentation result mask map obtained in the step S4 into a full-size defect mask map, judging the severity of different defects of a target airport pavement by combining the full-size pavement depth map obtained in the step S3, and evaluating the pavement quality according to pavement condition indexes;

The depth estimation network in S2 adopts a PackNet-based monocular non-supervision depth estimation network, wherein the monocular non-supervision depth estimation network comprises PackNet network branches used for generating 2D image depth information and Pose Convnet network branches used for generating pose information of adjacent video frame images;

Wherein PackNet the network comprises an encoding-decoding structure, and uses packing and unpacking to replace the traditional up-sampling and down-sampling operations; in the coding part, a residual block is adopted to replace a common convolution layer, the residual blocks use packing to realize downsampling operation, and the size of a feature map is doubled after downsampling operation every time; in the decoding part, the up-sampling operation is realized by adopting unpacking among the convolution blocks, the size of the feature map is doubled after each up-sampling operation, and finally, a depth information map with the same length and width as the original map is generated;

S2.1: inputting a video frame image I _d at the d moment into a PackNet network branch to generate an initial depth map of I _d, inputting video frame images I _s and I _d at the s moment and the d moment into a Pose ConvNet network branch to generate pose information between the video frame images at the s moment and the d moment;

Specifically, the pose information includes a rotation matrix R and a translation vector t;

S2.2: acquiring reconstruction slice data at the time d according to the initial depth map of I _d acquired at the time S2.1, pose information between video frame images at the time S and the time d and the video frame image I _s at the time S

S2.3: reconstructed slice data from time d acquired in S2.2And calculating PackNet a Loss function of the network branch by the video frame image I _d at the d moment, and acquiring Pose ConvNet the Loss function of the network branch according to the translation vector t _d-＞s, the instantaneous speed v of the monocular camera at the d moment and the time difference deltat _d-＞s between the s moment and the d moment, so that the depth estimation network automatically updates and iterates to generate a depth map corresponding to the video frame image at the d moment;

Specifically, t _d-＞s is a translation vector between the video frame image I _s at time s and the video frame image I _d at time d of the translation vector t;

specifically, the formula describing the Loss function of PackNet network branches is specifically:

In the method, in the process of the invention, Reconstructed slice data for d-temporal video frame image I _d and d-temporalStructural similarity between the two, a is an over parameter, the value is between 0 and 1,Reconstructed slice data for d-temporal video frame image I _d and d-temporalThe modulus of the difference between them,A Loss function for PackNet network branches;

Specifically, the formula describing the Loss function of Pose ConvNet network branches is specifically:

L_v(t_d-＞s,v)＝||t_d-＞s||-|v||Δt_d-＞s|

Wherein, |t _d-＞s | is a module of a translation vector between the d-moment video frame image I _d and the s-moment video frame image I _s, |v| is a module of an instantaneous speed of the monocular camera at the d-moment, and |delta t _d-＞s | is a module of a time difference between the d-moment and the s-moment;

s2.4: repeating the steps S2.1-S2.3 to obtain depth maps corresponding to all video frame images;

s4, the semantic segmentation network adopts a U-Net semantic segmentation network fused by a feature layer, the network comprises an encoder and a decoder, the input of the network is a processed RGB image and a corresponding depth image, and the output of the network is a segmentation result mask image;

specifically, the encoder comprises an RGB encoder and a depth encoder, wherein the RGB encoder and the depth encoder comprise a plurality of encoding layers, each encoding layer comprises a convolution layer, a pooling layer and an activating layer, and the length and width of a feature map are halved and the number of channels is doubled when each convolution layer passes through one encoding layer; the decoder comprises a plurality of decoding layers, each decoding layer comprises a convolution layer, an up-sampling layer and an activation layer, and the length and width of the feature map are doubled and the channel number is halved through the convolution layer of each decoding layer;

the RGB encoder and the depth encoder each comprise four encoding layers, and the decoder comprises three decoding layers;

processing the full-size road surface RGB map and the full-size road surface depth map into a local RGB map of 1024 x 512 x 3 and a local gray map of 1024 x 512 x 1, respectively, wherein S4 specifically comprises the following steps:

S4.1: processing the full-size road surface RGB image and the full-size road surface depth image obtained in the step S3 into a plurality of 1024 x 512 x 3 partial RGB images and 1024 x 512 x 1 partial gray images respectively;

s4.2: carrying out 1*1 convolutions on the processed local RGB image and the local gray image respectively to obtain a first RGB feature image and a first depth feature image with the two shapes of 1024 x 512 x 16;

S4.3: inputting the first RGB feature map into an RGB encoder to obtain a second RGB feature map set, and inputting the first depth feature map into a depth encoder to obtain a second depth feature map set;

Specifically, when the first RGB feature map is input into an RGB encoder, a second RGB feature map is obtained after each pass through one coding layer, and four coding layers are arranged, so that four second RGB feature maps are arranged in a second RGB feature map group; correspondingly, four second depth feature images are also arranged in the second depth feature image group;

And the sizes of the feature images of the second RGB feature image group and the second depth feature image group are correspondingly equal; the sizes of the feature graphs in the two feature graph groups from input to output are as follows: 512 x 256 x 32, 256 x 128 x 64, 128 x 64 x 128, 64 x 32 x 256;

S4.4: the feature images with the same size in the two feature image groups obtained in the step S4.3 are subjected to channel dimension stitching to obtain 4 stitching feature images, wherein the shapes of the stitching feature images are 512 x 256 x 64, 256 x 128, 128 x 64 x 256, 64 x 32 x 512 respectively;

S4.5: taking the spliced characteristic diagram with the smallest size in the S4.4 as the input of a decoder, and combining the rest spliced characteristic diagrams in the S4.4, decoding and outputting by the decoder to obtain a third characteristic diagram;

Specifically, a concatenated feature map with a size of 64×32×512 is input to a decoder, when a first layer decodes a layer, a concatenated feature map with a size of 64×32×512 and a concatenated feature map with a size of 128×64×256 are processed by the decoding layer to form a feature map with a size of 128×64×512, and a feature map with a size of 128×64×512 combined with a feature map with a size of 256×128×128 is processed by a second layer decoding layer to form a feature map with a size of 256×128×256, and a feature map with a size of 256×128×256 combined with a feature map with a size of 512×256×64 is processed by a third layer decoding layer to finally obtain a third feature map with a size of 512×128×128.

S4.6: performing convolution operation on the third feature map twice to obtain different types of defect segmentation result mask maps with the size of 1024 x 512 x 1;

In order to ensure feature fusion of different modes, respectively inputting an RGB image and a corresponding depth image into different encoders, namely an RGB encoder and a depth encoder, and splicing the feature images of the two branches at different stages to perform channel dimension;

In order to ensure feature fusion of different scales, the sizes of the feature images output by encoding and decoding are consistent, and the two feature images are spliced in channel dimension.

2. The method for detecting and evaluating the defects of the airport pavement according to claim 1, wherein said S3 comprises the steps of:

S3.1: respectively extracting feature points of two adjacent video frame images, namely a first video frame image and a second video frame image, generating feature descriptors, and carrying out quick approximate nearest matching to obtain matching points corresponding to the two adjacent video frame images;

S3.2: removing the matching points with larger errors in the S3.1 to obtain a plurality of groups of reliable matching point pairs;

s3.3: decomposing two video frame images in the S3.1 to obtain homography matrixes H ₁₂ of the two video frame images;

S3.4: according to the homography matrix H ₁₂, converting the second video frame image into the pixel plane of the first video frame image for image stitching, obtaining a stitched video frame image, converting the depth image corresponding to the second video frame image into the pixel plane of the depth image corresponding to the first video frame image for image stitching, and obtaining a stitched depth image;

s3.5: and (3) performing operations of S3.1-S3.4 on all the video frame images acquired in the S1 and all the depth maps corresponding to all the video frame images acquired in the S2, so as to acquire a full-size road surface RGB map and a full-size road surface depth map.

3. The method for detecting and evaluating the defect of the airport pavement according to claim 1, wherein S5 is specifically:

And (3) splicing the segmentation result mask map obtained in the step (S4) into a full-size defect mask map, obtaining different types of defect areas according to the full-size defect mask map, calculating the depth and the length of the full-size defect mask map according to the full-size pavement depth map obtained in the step (S3), and judging the damage severity of different defects of the target airport pavement according to the pavement condition index.

4. The method for detecting and evaluating the defect of an airport pavement according to claim 3, wherein the calculation formula of the pavement condition index PCI is specifically as follows:

Where a ₀ and a ₁ are road surface material coefficients and DR is a road surface breakage rate.