CN118015068B - A road surface structure depth prediction method, device, terminal equipment and medium - Google Patents

Abstract

The present application is applicable to the field of road engineering technology, and provides a method, device, terminal equipment and medium for predicting the depth of pavement structure, which constructs a depth map by collecting RGB image data; calculates the average depth and depth variance between pixels and adjacent pixels, and based on them, corrects the depth value of the pixel to obtain a corrected depth map; calculates the relative concave area ratio; constructs an image pyramid; based on a Gaussian local adaptive threshold, binarizes the image of each scale, and upsamples the binary image to obtain an adjusted binary image; fuses the images of each scale in the image pyramid, and performs a bitwise OR operation on the adjusted binary image and the fused binary image to obtain a final binary image; calculates the maximum bone particle diameter ratio; predicts the pavement structure depth according to the relative concave area ratio, the maximum bone particle diameter ratio and the pre-trained GBT model. The present application can improve the accuracy of pavement structure depth prediction and reduce complexity.

Description

A road surface structure depth prediction method, device, terminal equipment and medium

Technical Field

The present application belongs to the field of road engineering technology, and in particular, relates to a method, device, terminal equipment and medium for predicting the depth of a pavement structure.

Background technique

Pavement structure depth prediction is an important issue in the field of road engineering technology and can provide critical information for road maintenance and safety management.

At present, the commonly used pavement structural depth prediction technologies include laser-based methods, digital image technology methods, volumetric methods (sand laying methods), etc. Among them, the laser-based method refers to the use of laser scanners or linear lasers to illuminate the road surface vertically, and then measure the time of reflected laser to calculate the height change of the road surface; the digital image technology method refers to the use of digital image technology to measure and evaluate the structural depth of the road surface; the volumetric method refers to determining the pavement structural depth by laying a layer of material (usually sand) on the road surface and measuring the volume of the material.

Although the above methods can realize the prediction of pavement structure depth, they have defects such as time-consuming and labor-intensive, low accuracy, equipment, and inability to provide visual information in real time. In practical application, they still have certain limitations.

Summary of the invention

The present application provides a pavement structure depth prediction method, device, terminal equipment and medium, which can solve the problems of low accuracy and complexity of traditional pavement structure depth prediction methods.

In a first aspect, the present application provides a method for predicting pavement structure depth, comprising:

Collect RGB image data of the road surface, and construct a depth map based on the RGB image data;

For each pixel in the depth map, the average depth and depth variance between the pixel and adjacent pixels within a preset radius are calculated, and the depth value of the pixel is corrected based on the average depth and the depth variance to obtain a corrected depth map;

Calculate the relative concave area ratio of the corrected depth map; the relative concave area ratio is used to characterize the roughness of the road surface;

Downsample the RGB image data and construct an image pyramid; the image pyramid includes RGB images of multiple scales;

Based on the Gaussian local adaptive threshold, the image of each scale is binarized to obtain a binary image, and the binary image is upsampled to obtain an adjusted binary image;

The images of each scale in the image pyramid are fused to obtain a fused binary image, and the adjusted binary image and the fused binary image are bitwise ORed to obtain a final binary image;

Draw a circumscribed circle for all aggregates in the final binary image, and calculate the ratio of the diameter of the largest circumscribed circle to the image width to obtain the maximum bone particle diameter ratio; the maximum bone particle diameter ratio is used to describe the ratio of the maximum particle diameter of the aggregate used on the road surface to the width of the entire depth map. Aggregates include crushed stone, gravel and sand.

The pavement structure depth is predicted based on the relative concave area ratio, the maximum bone particle diameter ratio, and the pre-trained GBT model.

Optionally, the depth value of the pixel is corrected based on the average depth and depth variance, including:

By calculating the formula , get the denoised depth value ;in, represents the initial depth value of the pixel, represents the noise variance, represents the depth variance, represents the average depth;

By calculating the formula , get the corrected depth value ;in, , All represent fitting coefficients.

Optionally, the relative concave area ratio can be calculated as follows:

in, represents the relative concave area ratio, Indicates the number of pixels relative to the concave part, Indicates the horizontal pixel size, Indicates the vertical pixel size, Indicates horizontal pixels, , Indicates vertical pixels, .

Optionally, based on a Gaussian local adaptive threshold, binarize the image at each scale to obtain a binary image, and upsample the binary image to obtain an adjusted binary image, including:

For each pixel in the image of each scale, the formula is calculated

Get the binary image after the scaled image is binarized ;in, represents the Gaussian local adaptive threshold, Represents a constant used to control the offset of the Gaussian local adaptive threshold relative to the local variance. represents the local mean, represents the local standard deviation;

By upsampling, the binary image is adjusted to the same size as the corrected depth map to obtain an adjusted binary image.

Alternatively, the expression for the maximum bone particle diameter ratio is ;in, represents the diameter of the largest circumscribed circle, Indicates the image width.

In a second aspect, the present application provides a road surface structure depth prediction device, comprising:

An image acquisition module is used to collect RGB image data of the road surface and construct a depth map based on the RGB image data;

A depth correction module is used to calculate the average depth and depth variance between each pixel and adjacent pixels within a preset radius for each pixel in the depth map, and correct the depth value of the pixel based on the average depth and depth variance to obtain a corrected depth map;

A concave area ratio calculation module is used to calculate the relative concave area ratio of the corrected depth map; the relative concave area ratio is used to characterize the roughness of the road surface;

Image pyramid module, used to downsample RGB image data and construct image pyramid; image pyramid includes RGB images of multiple scales;

A binary image adjustment module, used for binarizing the image of each scale based on a Gaussian local adaptive threshold to obtain a binary image, and upsampling the binary image to obtain an adjusted binary image;

An image fusion module is used to fuse images of different scales in the image pyramid to obtain a fused binary image, and perform a bitwise OR operation on the adjusted binary image and the fused binary image to obtain a final binary image;

The maximum bone particle diameter ratio calculation module is used to make a circumscribed circle for all aggregates in the final binary image, and calculate the ratio of the diameter of the maximum circumscribed circle to the image width to obtain the maximum bone particle diameter ratio; the maximum bone particle diameter ratio is used to describe the ratio of the maximum particle diameter of the aggregate used on the road surface to the width of the entire depth map, and the aggregates include crushed stone, gravel and sand and gravel;

The depth prediction module is used to predict the pavement structure depth based on the relative concave area ratio, the maximum bone particle diameter ratio and the pre-trained GBT model.

In a third aspect, the present application provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned pavement structure depth prediction method when executing the computer program.

In a fourth aspect, the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned pavement structure depth prediction method is implemented.

The above solution of the present application has the following beneficial effects:

The pavement structure depth prediction method provided in the present application corrects the depth value of the pixel by averaging the depth and the depth variance, which can reduce the negative impact caused by image noise, thereby helping to improve the accuracy of the pavement structure depth prediction; the relative concave area ratio and the maximum bone particle diameter ratio are proposed to characterize the pavement texture from different dimensions, which simplifies the complex calculations in the traditional method, and is highly interpretable and more visual. At the same time, the relative concave area ratio and the maximum bone particle diameter ratio are combined with the GBT model to accurately predict the pavement structure depth.

Other beneficial effects of the present application will be described in detail in the subsequent specific implementation section.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

FIG1 is a flow chart of a method for predicting road surface structure depth provided by an embodiment of the present application;

FIG2 is a comparison diagram of the prediction effects of the pavement structure depth prediction method and the sand spreading method in one embodiment of the present application;

FIG3 is a diagram showing the error comparison between the pavement structure depth prediction method and the sand spreading method in one embodiment of the present application;

FIG4 is a schematic diagram of the structure of a road surface structure depth prediction device provided in an embodiment of the present application;

FIG5 is a schematic diagram of the structure of a terminal device provided in an embodiment of the present application.

Detailed ways

In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and/or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and/or combinations thereof.

It should also be understood that the term “and/or” used in the specification and appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

In response to the problems of low accuracy and complexity of traditional pavement structure depth prediction methods, the present application provides a pavement structure depth prediction method, device, terminal equipment and medium. The method corrects the depth value of the pixel by averaging the depth and depth variance, which can reduce the negative impact caused by image noise, thereby helping to improve the accuracy of pavement structure depth prediction; the relative concave area ratio and the maximum bone particle diameter ratio are proposed to characterize the pavement texture from different dimensions, simplifying the complex calculations in the traditional method, and having strong interpretability and more visualization. At the same time, the relative concave area ratio and the maximum bone particle diameter ratio are combined with the GBT model to accurately predict the pavement structure depth.

The pavement structure depth prediction method provided in this application is exemplified below.

As shown in FIG1 , the pavement structure depth prediction method provided in the present application includes the following steps:

Step 11: collect RGB image data of the road surface, and construct a depth map based on the RGB image data.

Exemplarily, in an embodiment of the present application, the RGB image data includes 1 reference image and 12 source images taken from multiple angles. The reference image refers to an image taken by a camera with an optical axis perpendicular to the road surface, and the source image refers to an image taken by a camera with an optical axis not perpendicular to the road surface. The shooting angles corresponding to different source images are different. At the same time, for the sake of accuracy, the RGB image in the embodiment of the present application is taken at a height of about 15 cm to 20 cm from the road surface to be measured. Considering that the construction of the depth map depends on the feature matching between a series of images, the shooting angle between the source images is set to 30° to 40°, and the focus of the depth map construction is to recover the depth information from the reference image to extract the road surface texture features. Therefore, the overlap between the reference image and the source image is set to be no less than 70%.

It should be noted that, in the specific implementation, in order to ensure the consistency of the size of the reference image, a steel hollow square calibration plate can be used, and its inner length is set to 10 centimeters (cm); in addition, considering the variability of lighting conditions during outdoor collection, the inner edge of the calibration plate should be aligned with the camera imaging area to control the shooting height and achieve consistent image pixel size, and try to shoot at the same time of the day to ensure uniform lighting conditions.

The following is an exemplary description of the process of constructing a depth map based on RGB image data.

Exemplarily, the structured beam method (sfM, Structure from Motion, a 3D technology for recovering scenes and camera poses from a set of images taken from different perspectives, commonly used open source software such as colmap) can be used to obtain camera pose information from RGB (Red Green Blue) image data. After obtaining a series of images (reference images and source images) and camera parameters, the Patchmatchnet model is used to construct the corresponding depth map of the series of images, and the depth map corresponding to the reference image is used as a test image for subsequent prediction effects.

Since the reference image represents relative depth values in the local camera coordinate system, without a precisely fixed shooting distance and angle, the depth map constructed from the same measurement point may have different depth ranges. Therefore, in order to make the depth values comparable and interpretable, they need to be mapped to a uniform range of [0, 1].

Specifically, by calculating the formula

Get the normalized depth value ;in, Represents the depth value before normalization, Indicates the maximum depth value, Indicates the minimum depth value.

Step 12, for each pixel in the depth map, calculate the average depth and depth variance between the pixel and adjacent pixels within a preset radius, and correct the depth value of the pixel based on the average depth and depth variance to obtain a corrected depth map.

It should be noted that, considering the influence of image noise on the prediction effect, before executing step 12, it is necessary to use a bilateral filter to process the depth map obtained in step 11, then fill its edges, and finally perform adaptive local noise reduction.

Among them, based on the average depth and depth variance, the depth value of the pixel is corrected, including:

By calculating the formula , get the denoised depth value ;in, represents the initial depth value of the pixel, represents the noise variance, represents the depth variance, represents the average depth, where Yes Depth value The depth value on the plane after depth fitting.

In addition, due to the road slope and the relative inclination between the camera and the plane, if no correction is performed, the incorrect relative depth information extracted from the reference view will affect the accuracy of the road performance evaluation. Therefore, in an embodiment of the present application, a random sample consensus (RANSAC) algorithm is used to obtain a third-order polynomial of the fitting surface, as follows:

in, are fitting coefficients, Denotes the surface fitting value. Subsequently, the tilt effect is eliminated by subtracting the corresponding surface fitting value from the RGB image data.

Step 13, calculating the relative concave area ratio of the corrected depth map.

The relative concave area ratio is used to characterize the roughness of the road surface. The road surface is composed of many tiny undulations, depressions and protrusions. The relative concave area is used to quantify the proportion of the concave part in the road surface structure and is an important parameter used to describe the roughness of the road surface.

Specifically, the calculation expression of the relative concave area ratio is as follows:

in, represents the relative concave area ratio, Indicates the number of pixels relative to the concave part, Indicates the horizontal pixel size, Indicates the vertical pixel size, Indicates horizontal pixels, , Indicates vertical pixels, .

Step 14: downsample the RGB image data and construct an image pyramid.

The above image pyramid includes RGB images of multiple scales.

Step 15: binarize the image of each scale based on the Gaussian local adaptive threshold to obtain a binary image, and upsample the binary image to obtain an adjusted binary image.

Among them, based on the Gaussian local adaptive threshold, the image of each scale is binarized to obtain a binary image, and the binary image is upsampled to obtain an adjusted binary image, including:

Step 15.1, for each pixel in the image of each scale, calculate the formula

Get the binary image after the scaled image is binarized .

in, represents the Gaussian local adaptive threshold, Represents a constant used to control the offset of the Gaussian local adaptive threshold relative to the local variance. represents the local mean, represents the local standard deviation.

Step 15.2, by upsampling, the binary image is adjusted to the same size as the corrected depth map to obtain an adjusted binary image.

Step 16, fuse the images of each scale in the image pyramid to obtain a fused binary image, and perform a bitwise OR operation on the adjusted binary image and the fused binary image to obtain a final binary image.

It should be noted that after executing step 16, in order to improve the accuracy of the prediction, in the embodiment of the present application, the adjusted binary image is subjected to hole filling and adhesion segmentation, which are respectively described below.

The hole filling operation is implemented using a variation of the flood fill algorithm. The algorithm is essentially a region growing process that starts with a single foreground pixel and expands to include all connected foreground pixels belonging to the same object. Exemplarily, white void regions within the aggregate are filled with black to facilitate subsequent analysis and identification.

For adhesion segmentation, an adjustable watershed algorithm can be used to segment adhesion particles. The algorithm is based on the watershed principle, but can be fine-tuned by adjusting parameters to adapt to different surface conditions to control the level of segmentation details. The watershed algorithm treats the image as a terrain, where higher intensity gradients correspond to higher peaks and lower intensity gradients correspond to lower valleys. For example, water flows along a path with decreasing gradients, eventually forming segmented areas.

Step 17, draw a circumscribed circle for all aggregates in the final binary image, and calculate the ratio of the diameter of the largest circumscribed circle to the image width to obtain the maximum bone particle diameter ratio.

The maximum bone particle size ratio is used to describe the maximum bone particle size ratio of the pavement. It represents the ratio of the maximum particle size of the aggregate (crushed stone, gravel, sand and gravel, etc.) used on the pavement to the width of the entire depth map (100 mm).

Specifically, the expression of the maximum bone particle diameter ratio is: ;in, represents the diameter of the largest circumscribed circle, Indicates the image width.

Step 18, predicting the pavement structure depth according to the relative concave area ratio, the maximum bone particle diameter ratio and the pre-trained GBT model.

Specifically, ① Data collection and segmentation:

Collect a dataset containing the target variable MTD and features P and D. Divide the dataset into a training set and a validation set. 70% of the data is used to train the model and 30% of the data is used to evaluate the model performance.

②Model training:

The training set is used to train the GBT model. The model will be iterated repeatedly, and each iteration will train a new decision tree to reduce the prediction error.

③Model tuning:

Based on the model performance, the model parameters are tuned to improve the accuracy of predictions. The best parameters are selected using cross-validation techniques.

④Model evaluation:

The validation set is used to evaluate the performance of the model. Evaluation indicators include mean square error (MSE), determination coefficient (R-squared), and mean absolute error (MAE).

⑤Prediction target value:

Once the model is trained and performs well, the values of features P and D can be input into the model to predict the target value. The GBT model will generate the final prediction result by combining the predictions of multiple decision trees.

In order to verify the effectiveness of the pavement structure depth prediction method provided by the present application, in one embodiment of the present application, the above steps are performed in sequence to predict the pavement structure depth of 40 sets of test data. These predicted values are then compared with the reference values measured using the sand spreading method, and the prediction effect comparison is shown in Figure 2. At the same time, the absolute error and relative error between the two are also compared, as shown in Figure 3.

As can be seen from Figures 2 and 3, the pavement structure depth prediction method provided in this application is more accurate, and most of the absolute errors are within 0.15 millimeters (mm), and the relative errors generally do not exceed 16%, which meets the needs of practical applications.

The pavement structure depth prediction device provided in this application is exemplarily described below.

As shown in FIG4 , the pavement structure depth prediction device 400 includes:

The image acquisition module 401 is used to acquire RGB image data of the road surface and construct a depth map based on the RGB image data;

The depth correction module 402 is used to calculate the average depth and depth variance between each pixel and adjacent pixels within a preset radius for each pixel in the depth map, and correct the depth value of the pixel based on the average depth and the depth variance to obtain a corrected depth map;

The concave area ratio calculation module 403 is used to calculate the relative concave area ratio of the modified depth map; the relative concave area ratio is used to characterize the roughness of the road surface;

An image pyramid module 404 is used to downsample the RGB image data to construct an image pyramid; the image pyramid includes RGB images of multiple scales;

A binary image adjustment module 405 is used to binarize the image of each scale based on a Gaussian local adaptive threshold to obtain a binary image, and upsample the binary image to obtain an adjusted binary image;

The image fusion module 406 is used to fuse the images of each scale in the image pyramid to obtain a fused binary image, and perform a bitwise OR operation on the adjusted binary image and the fused binary image to obtain a final binary image;

The maximum bone particle diameter ratio calculation module 407 is used to make a circumscribed circle for all aggregates in the final binary image, and calculate the ratio of the diameter of the maximum circumscribed circle to the image width to obtain the maximum bone particle diameter ratio; the maximum bone particle diameter ratio is used to describe the ratio of the maximum particle diameter of the aggregate used on the road surface to the width of the entire depth map, and the aggregate includes crushed stone, gravel and sand and gravel;

The depth prediction module 408 is used to predict the road surface structure depth according to the relative concave area ratio, the maximum bone particle diameter ratio and the pre-trained GBT model.

It should be noted that the information interaction, execution process, etc. between the above-mentioned devices/units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

As shown in Figure 5, an embodiment of the present application provides a terminal device. As shown in Figure 5, the terminal device D10 of this embodiment includes: at least one processor D100 (only one processor is shown in Figure 5), a memory D101, and a computer program D102 stored in the memory D101 and executable on the at least one processor D100. When the processor D100 executes the computer program D102, the steps in any of the above-mentioned method embodiments are implemented.

Specifically, when the processor D100 executes the computer program D102, it collects RGB image data of the road surface, and constructs a depth map based on the RGB image data, calculates the average depth and depth variance between the pixel and the adjacent pixels within a preset radius for each pixel in the depth map, and corrects the depth value of the pixel based on the average depth and depth variance to obtain a corrected depth map, calculates the relative concave area ratio of the corrected depth map, downsamples the RGB image data, constructs an image pyramid, binarizes the image of each scale based on a Gaussian local adaptive threshold to obtain a binary image, upsamples the binary image to obtain an adjusted binary image, fuses the images of each scale in the image pyramid to obtain a fused binary image, and performs a bitwise OR operation on the adjusted binary image and the fused binary image to obtain a final binary image, draws a circumscribed circle for all aggregates in the final binary image, and calculates the ratio of the diameter of the maximum circumscribed circle to the image width to obtain the maximum bone particle diameter ratio, and predicts the road surface structural depth based on the relative concave area ratio, the maximum bone particle diameter ratio, and the pre-trained GBT model. Among them, the depth value of the pixel is corrected by averaging the depth and depth variance, which can reduce the negative impact caused by image noise, thereby helping to improve the accuracy of pavement structure depth prediction; the relative concave area ratio and the maximum bone particle diameter ratio are proposed to characterize the pavement texture from different dimensions, simplifying the complex calculations in the traditional method, and having strong interpretability and more visualization. At the same time, combining the relative concave area ratio and the maximum bone particle diameter ratio with the GBT model can accurately predict the pavement structure depth.

The processor D100 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

In some embodiments, the memory D101 may be an internal storage unit of the terminal device D10, such as a hard disk or memory of the terminal device D10. In other embodiments, the memory D101 may also be an external storage device of the terminal device D10, such as a plug-in hard disk, a smart memory card (SMC, SmartMedia Card), a secure digital (SD, Secure Digital) card, a flash card (Flash Card), etc. equipped on the terminal device D10. Further, the memory D101 may also include both an internal storage unit of the terminal device D10 and an external storage device. The memory D101 is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as the program code of the computer program, etc. The memory D101 may also be used to temporarily store data that has been output or is to be output.

An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

An embodiment of the present application provides a computer program product. When the computer program product is run on a terminal device, the terminal device can implement the steps in the above-mentioned method embodiments when executing the computer program product.

If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium can at least include: any entity or device that can carry the computer program code to the pavement structure depth prediction device/terminal device, recording medium, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

In the embodiments provided in the present application, it should be understood that the disclosed devices/network equipment and methods can be implemented in other ways. For example, the device/network equipment embodiments described above are merely schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

The pavement structure depth prediction method provided in this application has the following advantages:

① The existing methods for estimating the depth of road structures based on image processing usually start from an 8-bit grayscale image. This solution can obtain a 32-bit depth map. The image data has high storage accuracy and can achieve higher-precision depth prediction.

② An adaptive local bilateral depth map filtering algorithm is proposed, which performs better in preserving the original data features than traditional algorithms;

③ A road surface tilt correction method based on surface fitting is proposed, which is more practical than the method based on plane fitting;

④ According to the data format of the depth map, the pavement texture features, namely the relative depression area ratio P and the maximum aggregate size ratio D, are proposed to jointly represent the pavement texture from multiple dimensions;

⑤ Use the nonlinear regressor gradient boosted tree (GBT) model to handle the complex relationship between features, showing better regression results and stability.

The above is a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles described in the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims (7)

1. A pavement structure depth prediction method, comprising:

Collecting RGB image data of a road surface, and constructing a depth map according to the RGB image data; the constructing a depth map according to the RGB image data includes: acquiring camera parameters of the RGB image data based on a structured light beam method, and constructing a depth map of the RGB image data by using a PATCHMATCHNET model;

calculating the average depth and the depth variance between the pixel and the adjacent pixel in the preset radius range for each pixel in the depth map, and correcting the depth value of the pixel based on the average depth and the depth variance to obtain a corrected depth map;

calculating the relative concave area proportion of the corrected depth map; the relative concave area proportion is used for representing the roughness of the road surface;

downsampling the RGB image data to construct an image pyramid; the image pyramid comprises RGB images of multiple scales;

Binarizing each scale image based on a Gaussian local self-adaptive threshold to obtain a binary image, and upsampling the binary image to obtain an adjusted binary image;

Fusing the images of all scales in the image pyramid to obtain a fused binary image, and performing bit-wise OR operation on the adjusted binary image and the fused binary image to obtain a final binary image;

Making a circumcircle for all aggregates in the final binary image, and calculating the ratio of the diameter of the maximum circumcircle to the image width to obtain the maximum bone particle diameter ratio; the maximum bone particle size ratio is used for describing the ratio of the maximum particle size of aggregate used on the pavement to the width of the whole depth map, and the aggregate comprises broken stone, cobble and sand stone;

And predicting the pavement construction depth according to the relative concave area proportion, the maximum bone particle size ratio and the pre-trained GBT model.

2. The pavement structure depth prediction method according to claim 1, wherein the correcting the depth value of the pixel based on the average depth and the depth variance includes:

By calculation formula Obtaining the depth value after denoising; Wherein,Representing the initial depth value of the pixel,Representing the variance of the noise and,Representing the variance of the depth of the image,Representing the average depth;

By calculation formula Obtaining a corrected depth value; Wherein,Representing a surface fit value, wherein,，All of which represent the fitting coefficients,Representing pixel coordinates.

3. The pavement construction depth prediction method according to claim 2, wherein the calculation expression of the relative concave area ratio is as follows:

wherein, Representing the relative concave area ratio of the said areas,The number of pixels of the opposite concave portion is indicated,Representing the size of the horizontal pixels,Representing the size of the vertical pixels,Represent the firstA number of horizontal pixels are used for the display,，Represent the firstA number of vertical pixels are used for the display,。

4. The method according to claim 1, wherein binarizing the image of each scale based on the gaussian local adaptive threshold to obtain a binary image, upsampling the binary image to obtain an adjusted binary image, and comprising:

For each pixel in the image of each scale, respectively, through a calculation formula

Obtaining a gaussian local threshold for each of said scaled images; Wherein,A representation constant for controlling the offset of said gaussian local adaptation threshold with respect to the local variance,The local average value is represented as such,Representing the local standard deviation;

based on Obtaining the binary image;

and (3) through up-sampling, the binary image is adjusted to be the same as the corrected depth map in size, and the adjusted binary image is obtained.

5. A pavement structure depth prediction apparatus, comprising:

The image acquisition module is used for acquiring RGB image data of the road surface and constructing a depth map according to the RGB image data; the constructing a depth map according to the RGB image data includes: acquiring camera parameters of the RGB image data based on a structured light beam method, and constructing a depth map of the RGB image data by using a PATCHMATCHNET model;

The depth correction module is used for calculating the average depth and the depth variance between the pixel and the adjacent pixel within the preset radius range for each pixel in the depth map respectively, and correcting the depth value of the pixel based on the average depth and the depth variance to obtain a corrected depth map;

the concave area proportion calculation module is used for calculating the relative concave area proportion of the corrected depth map; the relative concave area proportion is used for representing the roughness of the road surface;

The image pyramid module is used for downsampling the RGB image data to construct an image pyramid; the image pyramid comprises RGB images of multiple scales;

The binarization image adjustment module is used for binarizing each scale image based on Gaussian local self-adaptive threshold values to obtain binary images, and upsampling the binary images to obtain adjusted binary images;

The image fusion module is used for fusing the images of all scales in the image pyramid to obtain a fused binary image, and carrying out bit-wise OR operation on the adjusted binary image and the fused binary image to obtain a final binary image;

the maximum bone particle diameter ratio calculation module is used for making circumscribed circles for all aggregates in the final binary image, and calculating the ratio of the diameter of the maximum circumscribed circle to the image width to obtain the maximum bone particle diameter ratio; the maximum bone particle size ratio is used for describing the ratio of the maximum particle size of aggregate used on the pavement to the width of the whole depth map, and the aggregate comprises broken stone, cobble and sand stone;

and the depth prediction module is used for predicting the pavement construction depth according to the relative concave area proportion, the maximum bone particle size ratio and the pre-trained GBT model.

6. A terminal device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the road construction depth prediction method according to any one of claims 1 to 4 when executing the computer program.

7. A computer-readable storage medium storing a computer program, wherein the computer program when executed by a processor implements the pavement construction depth prediction method according to any one of claims 1 to 4.