CN119048711A

CN119048711A - Self-adaptive restoration method and system for three-dimensional reconstruction point cloud cavity of side slope surface

Info

Publication number: CN119048711A
Application number: CN202411099234.6A
Authority: CN
Inventors: 叶子贤; 单宝华
Original assignee: Harbin Institute of Technology Shenzhen
Current assignee: Harbin Institute of Technology Shenzhen
Priority date: 2024-08-12
Filing date: 2024-08-12
Publication date: 2024-11-29

Abstract

The present invention belongs to the field of image processing and repair technology, and specifically relates to a method and system for adaptively repairing holes in a three-dimensional reconstruction point cloud of a slope surface. Step 1: Acquire a pair of stereo images of a slope; Step 2: Based on the stereo image pair acquired in step 1, the image pair is adaptively enhanced using a three-histogram enhancement method based on an S membership function; Step 3: Select uniformly distributed reference points on the left image enhanced in step 2, and construct a sub-area with the reference points as the center; Step 4: Acquire the right image coordinates of the matching points corresponding to the reference points in the enhanced right image; Step 5: Acquire the right image coordinates of the matching points after hole repair; Step 6: Calculate the three-dimensional coordinates of the reference points, and obtain the three-dimensional data of the slope surface obtained through adaptive repair. The present invention is used to solve the problem of data holes in the three-dimensionally reconstructed point cloud of the slope surface caused by the weather randomness of the slope deformation monitoring environment and the lighting randomness of the slope monitoring scene, thereby causing the discontinuity problem of the calculation of the three-dimensional deformation data of the slope.

Description

Self-adaptive restoration method and system for three-dimensional reconstruction point cloud cavity of side slope surface

Technical Field

The invention belongs to the technical field of image processing and repairing, and particularly relates to a self-adaptive repairing method and system for a three-dimensional reconstruction point cloud cavity of a side slope surface.

Background

When the binocular stereoscopic vision-based three-dimensional reconstruction monitoring method is used in a natural environment, three-dimensional reconstruction data have certain data loss, and three-dimensional information data loss is caused because three problems of uncertainty of a field environment and surface texture characteristics of a side slope can influence a three-dimensional matching process of three-dimensional reconstruction calculation.

The slope monitors the illumination randomness of the scene. Because the illumination condition of the field environment can change along with time, when the sun moves to a certain angle, the illumination distribution on the side slope can be uneven, the local area can be blocked by shadows, and the local area can be overexposed due to direct sunlight, so that the effective information quantity in the photo, which can be subjected to subsequent analysis, is reduced, and the illumination change schematic diagram is shown in fig. 1.

The slope monitors the climate randomness of the environment. In a field environment, mist in the morning and rainwater in an indefinite period can cause that a monitoring area with a rear depth of field in a picture acquired by monitoring equipment is shielded by the mist and a camera monitoring picture is influenced by raindrop noise, and the two weather conditions can have significant influence on subsequent calculation and influence of slope mist on image quality, as shown in fig. 2.

As one of core technologies in image processing, the image enhancement technology is often affected by factors such as limitations of an image capturing apparatus and variable illumination conditions due to the influence of geometric characteristics of a photographic subject, which often results in low quality of an acquired image. When the quality of the actually obtained image is poor, such as low contrast, unclear details, uneven overall brightness, etc., difficulties are brought to subsequent image analysis and processing. The main purpose of image enhancement is to expand the differences between the features of different objects in the image, inhibit the uninteresting features, improve the image quality and enrich the information content, so that the image is more suitable for the visual characteristics of people and machine recognition analysis, thereby facilitating the further processing and analysis of the image.

Disclosure of Invention

The invention provides a self-adaptive restoration method for a three-dimensional reconstruction point cloud cavity of a side slope surface, which is used for enhancing the adaptability of a stereoscopic vision system to changeable monitoring environments and solving the problem of three-dimensional data missing in the three-dimensional reconstruction monitoring process of the side slope surface.

The invention provides a slope surface three-dimensional reconstruction point cloud hole self-adaptive repair system, which is used for solving the problem that data holes exist in the three-dimensional reconstruction slope surface point cloud caused by weather randomness of a slope deformation monitoring environment and illumination randomness of a slope monitoring scene, so that the calculation of three-dimensional deformation data of a slope is discontinuous.

The invention is realized by the following technical scheme:

the self-adaptive repair method for the three-dimensional reconstruction point cloud cavity of the side slope surface comprises the following steps,

Step 1, acquiring a side slope stereo image pair, wherein the side slope stereo image pair is left and right camera images of a side slope at the same moment, which are acquired by a stereo vision system consisting of two cameras with the same model specification, and acquiring internal and external parameters of the stereo vision system by a checkerboard calibration method;

step 2, based on the stereo image pair obtained in the step 1, adopting a three-histogram enhancement method based on an S membership function to adaptively enhance the stereo image pair;

step 3, selecting uniformly distributed reference points on the left image enhanced in the step 2, and constructing a matching sub-region with the size of (2m+1) x (2m+1) by taking the reference points as the center;

Step 4, taking the matching sub-region divided on the enhanced left image in the step 3 as a reference sub-region, and carrying out three-dimensional matching in the right image after image enhancement to obtain the right image coordinates of the matching point corresponding to the reference point in the right image after enhancement;

Step 5, bilateral filtering is carried out on the hole points in the right image coordinates of the reference points obtained by the three-dimensional matching in the step 4, and the right image coordinates of the matching points after hole repair are obtained;

And 6, according to the stereoscopic vision mathematical model, combining the left image coordinate of the reference point with the right image coordinate of the matched point after cavity repair and the internal and external parameters of the stereoscopic vision system, calculating to obtain the three-dimensional coordinate of the reference point, and obtaining the three-dimensional data of the side slope surface obtained through self-adaptive repair.

Further, the step 2 specifically comprises the following steps,

Step 2.1, stretching the whole acquired image, stretching the histogram range of the image to the full gray range, and obtaining a stretched histogram image I;

Step 2.2, partitioning the histogram image stretched in the step 2.1 into four mutually non-overlapping sub-blocks I _i, I epsilon {1,2,3,4};

Step 2.3, performing self-adaptive clipping on the histograms of the sub-blocks after the block division in the step 2.2;

Step 2.4, taking the mean square error of each sub-image block I _i as a limit for dividing different brightness areas, segmenting the histogram of each sub-image block I _i, and further dividing the sub-image block into sub-images I _ij with three brightness ranges;

step 2.5, carrying out equalization treatment on each sub-histogram in the step 2.4 based on the histogram cut in the step 2.3;

and 2.6, processing the image by using bilinear interpolation on each sub-histogram processed in the step 2.5 to obtain a final image enhancement result.

Further, in the step 2.1, the gray value of the pixel point at the (i, j) position before the stretching transformation in the image is defined as G (i, j), the gray value of the pixel point after the stretching transformation is defined as G (i, j), the minimum gray value in the image is defined as G _min, and the maximum gray value in the image is defined as G _max, and then the histogram stretching transformation can be expressed as:

further, the step 2.3 is specifically that the histogram clipping formula based on the threshold is as follows:

Where N represents the total number of pixels of a sub-block, β represents the clipping value of the histogram, L _max represents the maximum gray level in the sub-block, γ represents the gray average of the image sub-block, Representing the gray mean square error of the sub-block, k _max being the maximum slope, and CL representing the clipping value of the sub-block histogram finally obtained;

S-type membership function is introduced to replace beta fixed magnitude, and a function relation between a clipping coefficient and a gray level value is established, wherein the S (x) expression is as follows:

Wherein, x represents different gray levels, the value range is 0 to 255, and the value range of S (x) is 0 to 1;

Obtaining different histogram clipping values under different pixel duty ratio gray levels to realize self-adaptive clipping of images, wherein the clipping value expression of each level of modified gray values is as follows:

Wherein n _i is the number of pixels of the ith gray level, n _max、n_min is the number of pixels of the gray level with the largest and smallest number of pixels in the image sub-block, x _i is the S membership function parameter corresponding to the ith gray level, all n _i with the gray level between n _max and n _min in the histogram are mapped between intervals [0,255], and a clipping parameter value cl _i which changes with the change of the number of pixels in different gray levels is obtained;

Substituting formula (5) into the clipping threshold expression (2) to obtain a histogram clipping threshold CL _i under different number distributions, the expression being as follows:

And carrying out histogram clipping on the four sub-graph block histograms according to the clipping threshold value calculated based on the S-shaped membership function, uniformly distributing pixels exceeding the threshold value to other gray levels, and obtaining a histogram distribution H _s (i) after the histogram clipping.

Further, in step 2.4, if the processed image is an image shot by a single camera, components in different brightness ranges in the image are separated, and a mean square error calculation formula is as follows:

wherein G _mean is the gray average value of the image, n _i is the number of pixels with the gray level number of the original image being I, I _sd is the mean square value of the image, and the calculated upper and lower segmentation points U, B of the histogram of the input image are:

U=L₀+I_sd (10)

B=L_max-I_sd (11)

Where L ₀、L_max is the minimum and maximum gray level, respectively, where L ₀、L_max is 0 and 255, respectively, since the histogram has been stretched to the full gray range in the first step.

Further, if the image processed in the step 2.4 is a stereo image pair, selecting the sub-histogram segmentation point positions of the left and right images at the same time, considering the average value of the segmentation points calculated by selecting the left and right images, and calculating the expression as follows:

Further, the step 2.4 is specifically that the first segment of sub-histogram interval is [0,U ], the second segment of sub-histogram gray level interval is [ u+1, B ], and the third segment of sub-histogram gray level interval is [ b+1,255];

Histogram equalization is performed on each sub-histogram, and probability density distribution functions PDF ₁(i)、PDF₂(i)、PDF₃ (i) of the three sub-histograms are respectively calculated as follows:

wherein N ₁、N₂、N₃ is the total number of pixels in the three sub-histograms, and the cumulative probability distribution function CDF ₁(I)、CDF₂(I)、CDF₃ (I) is calculated according to the probability distribution of each sub-histogram, and the calculation formula is as follows:

And further obtaining the histogram equalization mapping relation of each sub-histogram as follows:

I_S1＝U×CDF₁(I) (20)

I_S2＝(U+1)+(B-U+1)×CDF₂(I) (21)

I_S3＝(B+1)+(255-B+1)×CDF₃(I) (22)

I _S1、I_S2、I_S3 is three enhanced image components of a certain image sub-block of the input image I, and is overlapped to obtain I _S, namely the image sub-block processed by the three histogram enhancement method based on the S membership function, and after four sub-blocks divided by the original image are adaptively enhanced by steps 2.3 to 2.5, the edges of the sub-blocks are processed by bilinear interpolation, so that the image block effect is reduced.

Furthermore, the step 5 adopts a self-adaptive three-dimensional point cloud hole repairing algorithm based on bilateral filtering to realize filling of the three-dimensional reconstruction data hole of the side slope surface, specifically,

Step 5.1, traversing the right image coordinates of the reference points obtained by the stereo matching in the step 4, and determining effective hole points to be filled;

step 5.2, determining the size of each effective hole point filter to be filled;

Step 5.3, bilateral filtering calculation is carried out on the effective hole point to be filled in the right image coordinate, and a right image coordinate value of the reference point after hole filling is obtained;

further, in step 5.1, the right image pixel coordinate of the three-dimensional matching reference point is traversed, all the hole points in the right image pixel coordinate are scanned by using a window with the size of m×m, the number of the effective points in the neighborhood of the hole points is calculated to occupy the ratio Y, the threshold value R of the effective point occupancy ratio is manually determined, and the rule for effectively filling the hole points is as follows:

step 5.1.1, judging that Y > R in the neighborhood of the current hole point is effective to fill the hole point;

Step 5.1.2, namely, the effective point occupation ratio in the neighborhood of the current hole point is 0.1< Y < R, the side length of the neighborhood is enlarged by 2, at the moment, the neighborhood is changed to (m+2) x (m+2), a new effective point occupation ratio Y ₁ is calculated, if Y ₁ is more than 0.1, the process is continued, the neighborhood is enlarged, the new effective point occupation ratio Y _k is calculated, until the point is judged to be an ineffective hole point to be filled when Y _k＜Y_k-1 < R, or the point is judged to be an effective hole point to be filled when Y _k is more than R;

In step 5.2, the size of the scanning window initially set is m, and the size of the decision window of part of the hole points may be larger than m, and specifically, the size of the filtering window of each hole point is determined according to the size of the hole point decision window when the condition Y > R is finally satisfied in step 5.1.

A self-adaptive repair system for a three-dimensional reconstruction point cloud cavity of a side slope surface, which uses the self-adaptive repair method for the three-dimensional reconstruction point cloud cavity of the side slope surface, the repair system comprises,

The image acquisition module is used for acquiring an original image of three-dimensional reconstruction data of the side slope surface, wherein the image comprises a photo image shot by a single camera and a binocular vision image;

the image enhancement module is used for enhancing the image by adopting a three-histogram enhancement method based on an S membership function based on an image pair acquired by the stereoscopic vision system;

And the data hole filling module is used for filling the three-dimensional reconstruction data hole of the side slope surface based on bilateral filtering to the enhanced image, so as to finish the self-adaptive repair of the three-dimensional reconstruction data hole of the side slope surface.

The beneficial effects of the invention are as follows:

Aiming at the problems of light change and rain and fog noise in natural environment, the invention optimizes based on the existing limiting contrast histogram equalization algorithm (CLAHE), adopts an adaptive change S membership function on a determined histogram clipping threshold value, and performs adaptive clipping according to different gray level distribution characteristics in the histogram. In the histogram equalization process, the original image is subjected to histogram segmentation, three sub-histograms representing different brightness areas are differentiated, and corresponding image parts are enhanced based on the sub-histograms, so that targeted enhancement of the different brightness areas is realized. The method is subjected to an image enhancement test, and test results show that the method has obvious improvement in brightness change and simulated rain and fog scenes compared with the traditional HE, AHE, CLAHE algorithm, and the quality and gray information quantity of images are improved while the image details are ensured.

According to the method, a three-dimensional data restoration method based on bilateral filtering is adopted aiming at the problem that a texture missing area exists on the surface of a monitored object to cause a data hole to exist in a three-dimensional reconstruction result, and the method has obvious advantages in subjective evaluation of three-dimensional data restoration and objective evaluation indexes of point cloud quality, so that the effectiveness of the method for three-dimensional data hole restoration is verified.

Drawings

FIG. 1 is a schematic illustration of illumination changes during monitoring.

Fig. 2 is an image of the effect of side slope mist during monitoring.

Fig. 3 is a flow chart of the method of the present invention.

Fig. 4 is a flow chart of a three histogram enhancement algorithm based on an S membership function of the present invention.

Fig. 5 is a flowchart of a three-histogram enhancement algorithm based on an S membership function for left and right image pairs of the present invention.

Fig. 6 is an exemplary diagram of image enhancement after side slope blocking in a three histogram enhancement algorithm based on an S membership function in the present invention.

Fig. 7 is an image enhancement experimental result diagram in an embodiment of the present invention, where (a 1) is a conventional image, (a 2) is an HE algorithm image enhancement experimental result diagram, (a 3) is an AHE algorithm image enhancement experimental result diagram, (a 4) is a CLAHE algorithm image enhancement experimental result diagram, and (a 5) is an algorithm image enhancement experimental result diagram of the present invention; (b 1) is a rain and fog image 1, (b 2) is a HE algorithm rain and fog image 1 image enhancement experimental result diagram, (b 3) is a AHE algorithm rain and fog image 1 image enhancement experimental result diagram, (b 4) is a CLAHE algorithm rain and fog image 1 image enhancement experimental result diagram, (b 5) is a CLAHE algorithm rain and fog image 1 image enhancement experimental result diagram, (c 1) is a rain and fog image 2, (c 2) is a HE algorithm rain and fog image 2 image enhancement experimental result diagram, (c 3) is a AHE algorithm rain and fog image 2 image enhancement experimental result diagram, (c 4) is a CLAHE algorithm rain and fog image 2 image enhancement experimental result diagram, (c 5) is a CLAHE algorithm rain and fog image 2 image enhancement experimental result diagram, (d 1) is a rain and fog image 3 image enhancement experimental result diagram, (d 3) is a HEE algorithm rain and fog image 3 image enhancement experimental result diagram, (d 4) is a CLAHE algorithm rain and fog image 3 image enhancement experimental result diagram, (d 5) is a HE algorithm rain and fog image 3 image enhancement experimental result diagram, (d 1) is a HE algorithm illumination result diagram 1) is an abnormal illumination result diagram, (e3) The method comprises the steps of (1) obtaining an AHE algorithm abnormal illumination 1 image enhancement experimental result graph, (e 4) obtaining a CLAHE algorithm abnormal illumination 1 image enhancement experimental result graph, (e 5) obtaining an AHE algorithm abnormal illumination 1 image enhancement experimental result graph, (f 1) obtaining an abnormal illumination 2, (f 2) obtaining an HE algorithm abnormal illumination 2 image enhancement experimental result graph, (f 3) obtaining an AHE algorithm abnormal illumination 2 image enhancement experimental result graph, (f 4) obtaining a CLAHE algorithm abnormal illumination 2 image enhancement experimental result graph, (f 5) obtaining an inventive algorithm abnormal illumination 2 image enhancement experimental result graph, (g 1) obtaining an abnormal illumination 3, (g 2) obtaining an HE algorithm abnormal illumination 3 image enhancement experimental result graph, (g 3) obtaining an AHE algorithm abnormal illumination 3 image enhancement experimental result graph, (g 4) obtaining a CLAHE algorithm abnormal illumination 3 image enhancement experimental result graph, and (g 5) obtaining an inventive algorithm abnormal illumination 3 image enhancement experimental result graph.

Fig. 8 is a schematic diagram of a three-dimensional reconstruction point cloud cavity of a monitored area according to the present invention.

Fig. 9 is a diagram of the effect of repairing the three-dimensional reconstruction point cloud cavity.

Fig. 10 is a graph of point cloud hole repair experiments according to the present invention, wherein (a 1) is a5×5 median filter point cloud hole repair experiment result graph, (a 2) is a5×5 gaussian filter point cloud hole repair experiment result graph, (a 3) is a5×5 bilateral filter point cloud hole repair experiment result graph, (b 1) is a 7×7 median filter point cloud hole repair experiment result graph, (b 2) is a 7×7 gaussian filter point cloud hole repair experiment result graph, (b 3) is a 7×7 bilateral filter point cloud hole repair experiment result graph, (c 1) is a 9×9 median filter point cloud hole repair experiment result graph, (c 2) is a 9×9 bilateral filter point cloud hole repair experiment result graph, and (c 3) is a 9×9 bilateral filter point cloud hole repair experiment result graph.

Fig. 11 is a perspective view system layout diagram of a slope deformation monitoring test.

FIG. 12 is a schematic view of a slope deformation monitoring test area.

Fig. 13 is a three-dimensional reconstruction data result graph of a slope deformation monitoring test, in which (a) is a slope surface deformation field monitoring result when t=0s, (b) is a slope surface deformation field monitoring result when t=1s, (c) is a slope surface deformation field monitoring result when t=2s, (d) is a slope surface deformation field monitoring result when t=3s, (e) is a slope surface deformation field monitoring result when t=4s, (f) is a slope surface deformation field monitoring result when t=5s, (g) is a slope surface deformation field monitoring result when t=6s, and (h) is a slope surface deformation field monitoring result when t=7s.

Detailed Description

In the following description, for purposes of explanation and not limitation, specific details are set forth such as the particular system architecture, techniques, etc., in order to provide a thorough understanding of the embodiments of the present application. It will be apparent, however, to one skilled in the art that the present application may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.

It should be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

It is also to be understood that the terminology used in the description of the application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in this specification and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.

The following description of the embodiments of the present application will be made more fully with reference to the accompanying drawings, in which 1-13 are shown, it being apparent that the embodiments described are only some, but not all embodiments of the application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.

In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present application, but the present application may be practiced in other ways other than those described herein, and persons skilled in the art will readily appreciate that the present application is not limited to the specific embodiments disclosed below.

Embodiment one

The embodiment provides a self-adaptive restoration method for a three-dimensional reconstruction point cloud cavity of a side slope surface, which specifically comprises the following steps,

Step 1, acquiring a slope stereo image pair, and acquiring internal and external parameters of a stereo vision system by adopting a checkerboard calibration method. The side slope stereo image pair is images of left and right cameras of the side slope at the same moment, which are acquired by a stereo vision system consisting of two cameras with the same model specification;

Step 2, based on the stereo image pair obtained in the step 1, adopting a three-histogram enhancement method based on an S membership function to adaptively enhance the image pair;

step 3, selecting uniformly distributed reference points on the left image enhanced in the step 3, and constructing a matching sub-region with the size of (2m+1) x (2m+1) by taking the reference points as the center;

Step 6, according to the stereoscopic vision mathematical model, combining the left image coordinate of the reference point, the right image coordinate after cavity repair and the internal and external parameters of the stereoscopic vision system, calculating to obtain the three-dimensional coordinate of the reference point, and obtaining the three-dimensional data of the side slope obtained through self-adaptive repair;

further, step 1.1, calibrating the stereoscopic vision system by using a Zhang Zhengyou calibration method and utilizing a checkerboard calibration plate;

The method comprises the steps of 1.2, vertically placing a checkerboard calibration plate with proper size at the bottom of a slope to be monitored, wherein the checkerboard calibration plate is parallel to the lower edge of the slope, defining a geodetic coordinate system O _g-X_gY_gZ_g, taking the position of a left lower corner point of the checkerboard calibration plate as an origin of the geodetic coordinate system, being parallel to the ground, taking the direction of a right lower corner point of the checkerboard calibration plate as the positive direction of an X _g axis of the geodetic coordinate system, taking the direction of a left lower corner point of the checkerboard calibration plate as the positive direction of a Z _g axis of the geodetic coordinate system, and taking the direction of a left upper corner point of the checkerboard calibration plate as the positive direction of a Y _g axis of the geodetic coordinate system, being perpendicular to the plane of the checkerboard calibration plate;

The three-dimensional vision system obtains left and right images of the checkerboard calibration plate, obtains pixel coordinates of each corner of the checkerboard calibration plate in the left image and the right image according to a Harris corner recognition algorithm, and obtains three-dimensional coordinates of each corner of the checkerboard calibration plate under a left camera coordinate system by combining a three-dimensional vision mathematical model;

Taking the left lower corner point as the origin (0, 0) of a geodetic coordinate system, determining coordinate values of X _g axis and Z _g axis of other corner points under the geodetic coordinate system according to physical dimensions of the checkerboard, setting a Y _g axis of the geodetic coordinate system based on the checkerboard calibration plate to be perpendicular to the plane of the checkerboard calibration plate, determining three-dimensional coordinates of each corner point of the checkerboard calibration plate under the geodetic coordinate system according to the conditions, wherein Y _g coordinate values of each corner point are 0;

And 1.5, establishing a transformation matrix formula of a left camera coordinate system and a geodetic coordinate system according to the left image pixel coordinates of each corner point of the checkerboard calibration plate and the three-dimensional coordinates under the geodetic coordinate system in combination with the linear projection model of the camera.

Further, in the step 1.1, the internal parameter matrix a _l and a _r of the left and right cameras and the external parameter matrix M _lr between the left and right cameras are obtained by calibration, and the internal and external parameters of the stereoscopic vision system are expressed by the following formula:

Wherein the method comprises the steps of And (3) withIs the principal point image coordinates of the left and right cameras,And (3) withThe external parameter matrix M _lr consists of a rotation matrix R _lr and a translation vector T _lr, R _i is a rotation component, and T _x,t_y,t_z is a translation component in the three-axis direction.

Further, the specific formula of the step 1.5 is as follows,

Wherein z _li is the z _l axis coordinate value of the ith checkerboard corner in the left camera coordinate system, M _li is the rotation component and translation component elements in the transformation matrix of the left camera coordinate system and the geodetic coordinate system, (u _i,v_i, 1) is the homogeneous coordinate of the left image pixel coordinate of the ith checkerboard corner, (X _gi,Y_gi,Z_gi, 1) is the homogeneous coordinate of the three-dimensional coordinate of the ith checkerboard corner in the geodetic coordinate system, and the transformation matrix M _l2g of the left camera coordinate system and the geodetic coordinate system is obtained by solving the unknown parameter M _li in the transformation matrix by a direct linear transformation method according to the linear equation set constructed by the formula (40).

Furthermore, the existing histogram equalization method cannot solve the self-adaptive enhancement effect of the image for a period of time sequence, and in addition, the characteristic of spatial distribution of different local areas in the image is considered to locally enhance, but independent consideration of different brightness areas in the histogram is lacked, and for a scene with uneven brightness distribution in the image, the existing algorithm is difficult to conduct independent processing on pixel sets with different brightness of the image.

The S membership function is used for adaptively cutting the histogram to avoid that a predefined cutting value does not have universality under different illumination conditions, the cut histogram can improve the increasing trend of the cumulative distribution function, limit extra noise caused by excessive contrast enhancement, the pixel point sets of different brightness areas are enhanced in a targeted manner by adopting a segmentation processing sub-histogram mode, the three segmented sub-histograms respectively represent darker scenes, normal illumination areas and brighter scenes, and the segmentation of the histogram can effectively enhance the pertinence of the areas with abnormal local exposure.

As shown in fig. 2 and 3, the step 2 specifically includes the following steps,

Step 2.1, stretching the whole acquired image, stretching the histogram range of the image to the full gray range, and improving the contrast of the whole image to obtain a stretched histogram image I;

Further, in the step 2.1, the gray value of the pixel point at the (i, j) position before the stretching transformation in the image is defined as G (i, j), the gray value of the pixel point after the stretching transformation is defined as G (i, j), the minimum gray value in the image is defined as G _min, and the maximum gray value is defined as G _max, and then the histogram stretching transformation can be expressed as:

Step 2.2, partitioning the histogram image stretched in the step 2.1 into four mutually non-overlapped sub-blocks I _i, I epsilon {1,2,3,4}, so as to realize enhancement of local positions of the image;

step 2.3, clipping the histogram after the block in the step 2.2, and limiting the noise amplification problem caused by contrast enhancement after histogram equalization;

because the threshold value of the histogram clipping in the CLAHE needs to be manually determined, and a single limiting value is insufficient for scenes with different illumination conditions and the self-adaptive adjustment capability of the image, the threshold value beta is functionalized, and the clipping threshold value of the histogram is dynamically adjusted according to the histogram characteristic of the image, so that the robustness and the self-adaptation of an image enhancement algorithm to scenes with larger brightness change are improved.

The contrast limiting method provided by the invention can realize the self-adaptive enhancement of different images, and introduce an S-shaped membership function to replace a beta fixed magnitude value, thereby realizing the self-adaptive threshold clipping of a histogram and establishing the functional relation between clipping coefficients and gray level values, wherein the S (x) expression is as follows:

The invention relates to a method for adaptively clipping images, which comprises the steps of (a) obtaining a gray level distribution characteristic of a histogram, wherein x represents different gray levels, the value range is 0 to 255, the value range of S (x) is 0 to 1, 256 different clipping values are obtained by calculating clipping values of different gray levels so as to realize adaptive clipping of the images, but certain limitation exists in the method, as S (x) does not directly consider the distribution characteristic of the histogram, only different clipping values of different definitions of the gray levels are considered, the invention refers to an S membership function and modifies the S membership function, introduces the distribution characteristic of the gray level histogram, takes the number of pixels under different gray levels as an independent variable, obtains different histogram clipping values under different pixel duty gray levels, and the clipping value expression of each level of the modified gray values is as follows:

Wherein n _i is the number of pixels of the ith gray level, n _max、n_min is the number of pixels of the gray level with the largest and smallest number of pixels in the image sub-block, x _i is the S membership function parameter corresponding to the ith gray level, all n _i with the gray level between n _max and n _min in the histogram are mapped between intervals [0,255], a clipping parameter value cl _i which changes with the change of the number of pixels in different gray levels can be obtained, a clipping threshold which is higher for gray levels with higher duty ratios of different gray levels can be realized, and a clipping threshold which is lower for gray levels with lower duty ratios of pixels can be realized;

Substituting equation (5) into (2) clipping threshold expression, histogram clipping threshold CL _i under different number distributions can be obtained, which expression is as follows:

Step 2.4, taking the mean square error of the image as a limit for dividing different brightness areas, segmenting the histogram, and dividing the original image into three subgraphs;

Further, in step 2.4, if the processed image is a photo image shot by a single camera, the method uses the mean square error of the image as a boundary for dividing different brightness regions, segments the histogram, divides the original image into three sub-images, and separates components in different brightness regions in the image so as to achieve the enhancement effect in each local brightness region, wherein the mean square error calculation formula is as follows:

U=L₀+I_sd (10)

B=L_max-I_sd (11)

Or further, when the image processed in the step 2.4 is a binocular vision image, because the image is enhanced by the method, in order to enable the three-dimensional reconstruction under the stereoscopic vision model of the image to have more feature information capable of being matched, in order to enable the subsequent binocular vision image to obtain a synchronous enhancement effect, for selecting the positions of sub-histogram segmentation points of the left and right images at the same moment, taking the average value of the segmentation points calculated by the left and right images into consideration, and the calculation expression is as follows:

furthermore, the step 2.4 is that the first segment of sub-histogram interval is [0,U ], the gray level value of the pixel in the interval is smaller and represents a darker area in the image, the second segment of sub-histogram interval is [ U+1, B ], the pixel in the interval range occupies the largest amount and the brightness is more uniform, the third segment of sub-histogram interval is [ B+1,255] and represents the area with the highest brightness in the image;

I_S1＝U×CDF₁(I) (20)

I_S2＝(U+1)+(B-U+1)×CDF₂(I) (21)

I_S3＝(B+1)+(255-B+1)×CDF₃(I) (22)

I _S1、I_S2、I_S3 is respectively three enhanced image components of an input image I, and is overlapped to obtain I _S, namely an image sub-block processed by a three-histogram enhancement method based on an S membership function, after four sub-blocks divided by an original image are adaptively enhanced by steps 2.3 to 2.5, edges of the sub-block are processed by bilinear interpolation, and image block effects are reduced.

And 2.6, processing the image by bilinear interpolation on each sub-histogram processed in the step 2.5 to obtain a final enhanced result.

Image enhancement is performed on the example image by adopting the steps from 2.1 to 2.6, and the processing flow of the image is shown in fig. 6.

The image enhancement contrast experiment includes an image quality evaluation index, specifically,

The evaluation indexes of the image quality are mainly classified into a subjective evaluation method and an index evaluation method. In the subjective evaluation method, the subjective factors can cause poor consistency of observation and evaluation results by observing images through human eyes, and in addition, the difference of the human eyes on pixel levels is difficult to observe, so that accurate judgment on micro details can not be performed. The index evaluation method is based on a plurality of quantifiable indexes in the image gray level distribution characteristics, can quantitatively analyze the quality of the image, and has better reference value. The method uses the image Information Entropy (IE), the image two-dimensional information entropy (2 DIE), the Tenengard evaluation function and the modified Laplace function to quantitatively compare the image before and after enhancement, and is used for objectively evaluating the effectiveness of the algorithm for solving the problem of image enhancement of the local exposure abnormal region.

1. The information entropy reflects the chaotic degree of the random event and can also be used as a measure of the system complexity of the random event, the gray information distribution in the same image has randomness, the image information entropy also reflects the information quantity contained in the image, and the larger the information entropy is, the more random the content contained in the image is represented, and the larger the information quantity is contained.

Where P _i is the proportion of pixels of gray level i to the total pixels.

2. The two-dimensional information entropy can more consider the spatial characteristics of the image gray information distribution, so the invention adopts the two-dimensional information entropy as the quantitative index of the image information quantity, and the expression of the two-dimensional information entropy is as follows:

where f (i, j) is a (i, j) binary group, the number of occurrences in the image, i represents the gray value of the center pixel in the sliding window, and j represents the average of the gray values of the pixels in the sliding window other than the center pixel.

The tenengrad function is a commonly used image sharpness evaluation index, the basic principle of which is that the higher the image sharpness, the higher the edge sharpness. The teningrad evaluation function uses a sobel operator to extract gradient values in the horizontal direction and the vertical direction of the image respectively, and for the intuitiveness of a result, the gradient values in the x direction and the y direction are subjected to square summation in the result, and the concrete expression is as follows:

Wherein G _x is the gradient in the horizontal direction of the image pixels, G _y is the gradient in the vertical direction of the image pixels, I is the image to be evaluated, and w and h are the width and height of the image respectively.

The LaPM Gaussian correction function is an objective evaluation index of image quality, is widely applied to the field of image processing for evaluating the focusing sharpness of an image, corrects a Laplace operator and is derived from a Laplace second derivative filter, the filter highlights a region with rapid intensity change, and edge information of the image is effectively captured. The expression is as follows:

L_x＝I*M_h (30)

L_y＝I*M_v (31)

Wherein L _x is the gradient of the image pixel in the horizontal direction, L _y is the gradient of the image pixel in the vertical direction, I is the image to be evaluated, N is the total number of pixels of the image, M _h、M_v is the Laplacian convolution kernels of the image in the horizontal and vertical directions, and M _h＝M_v ^T.

The comparison experiment result is that, in particular,

The image enhancement algorithm provided by the invention is subjected to experiments, and is compared with the image enhancement effect of HE, AHE, CLAHE algorithm, and illumination abnormal condition simulation and rain mist state simulation are performed on the slope sandy soil scene simulated in a laboratory. According to the invention, the influence of different illumination conditions possibly occurring in the slope monitoring process is simulated by adopting the mobile light source, the noise interference suffered by the camera during raining and fogging is simulated by adopting the watering can, 3 pictures are respectively acquired by the two low-quality pictures, and the comprehensive comparison is carried out by combining one picture obtained during conventional acquisition (no abnormal illumination and no raining and foggy conditions are applied). As shown in FIG. 7, the result diagram enhanced according to different algorithms can be intuitively seen, when the simulated rain and fog image and the scene with darker overall brightness are processed by the histogram equalization algorithm, the contrast of the image in the dark part and the bright part is strong, and the details in the dark part of the image are not effectively enhanced. The AHE and CLAHE algorithms can effectively improve details of the image for the scene with abnormal illumination and overexposure of the whole brightness, and effectively strengthen the details of the whole image for the rain and fog image, but because the whole contrast of the image is larger, the noise of the image is correspondingly enhanced, and more irrelevant gray information is added. The algorithm provided by the invention effectively improves the details of the image for both the rain and fog scene and the illumination abnormal scene, and the enhancement effect for different image scenes is stable due to the adoption of the self-adaptive clipping of the histogram, and the noise is obviously inhibited, and the implementation flow of the image enhancement algorithm is shown in the figure 4 or the figure 5.

The objective evaluation indexes of the image enhancement effect are shown in table 1, and the indexes of the algorithm in the rain and fog scene are higher than the indexes of the image enhancement result by other algorithms, are consistent with the enhancement subjective effect of the image, and the enhancement effect of the algorithm in the image enhancement results of the image illumination anomaly 2 and the illumination anomaly 3 is slightly lower than the enhancement effect of the CLAHE enhanced image, but in general, the algorithm has good adaptability to the rain and fog environment and the illumination anomaly environment, and has good enhancement effect to different scenes.

Table 1 image enhancement experimental evaluation index

Further, the step 4 specifically includes the following steps,

Step 4.1, selecting a slope monitoring area on the initial left image, selecting s pixels with the step length, performing grid division on the slope monitoring area of the initial left image to obtain R multiplied by C grid points which are uniformly distributed, wherein the determination of R and C is specifically that,

Wherein fl0or represents a downward rounding function, height is an image height value of the slope monitoring area, and width is an image width value of the slope monitoring area.

The grid points obtained by dividing the monitoring area on the initial left image according to the fixed step length s are used as time sequence fixed image points, and the pixel positions of the time sequence fixed image points on the left image at other moments of the left image sequence are always fixed;

Step 4.2, grid dividing the slope monitoring area of the left image at other moments after image enhancement by the same step length s in the monitoring process to obtain the left image coordinates of all time sequence fixed image points Wherein i is 1,2, R, j is 1,2, C, k is the k-th frame of image during monitoring;

Respectively constructing reference subregions with the window size of (2m+1) pixel x (2m+1) pixel in the respective neighborhood range, adopting zero-mean normalized cross-correlation function (ZNCC) to carry out three-dimensional matching on all time sequence fixed image points on the left image on the right image, and using inverse combined Gaussian Newton algorithm (IC-GN) to carry out iterative calculation to obtain sub-pixel coordinates of matching points of corresponding time sequence fixed image points on the right image

Further, the data structure in the stereo matching process is analyzed, so that a solution for how to repair the data is found fundamentally. The bilateral filter can be used for repairing images with obvious boundaries and retaining edge characteristic values of the images, and has stronger edge characteristics due to the fact that the bilateral filter can be used for repairing image coordinate matrixes, and the bilateral filter based on the formula (34) is used for filling two-dimensional cavity points of the three-dimensional matching results, so that cavity three-dimensional point values in three-dimensional point clouds are repaired.

The step 5 adopts a self-adaptive three-dimensional point cloud cavity repairing algorithm based on bilateral filtering to realize filling of the three-dimensional deformation data cavity of the side slope surface, specifically,

and 5.3, carrying out bilateral filtering calculation on the effective hole point to be filled in the right image coordinate to obtain a right image coordinate value of the reference point after filling the hole, as shown in fig. 9.

Further, step 5.1 specifically includes traversing pixel coordinates of an image of the three-dimensional deformation field of the side slope, scanning all points in a matrix with a window of m×m, calculating a number of effective points in the cavity points to occupy a ratio Y, determining an effective point occupying ratio threshold R, and filling the cavity points effectively according to the following rules:

Step 5.1.2, the effective point ratio in the neighborhood of the current hole point is 0.1< Y < R, the neighborhood side length is enlarged by 2, at the moment, the neighborhood is (m+2) multiplied by (m+2), the new effective point ratio Y ₁ is calculated, if Y ₁ is more than 0.1, the process is continued, the new effective point ratio Y _k is calculated by the enlarged neighborhood until the point is judged to be an ineffective hole point to be filled when Y _k＜Y_k-1 < R, or the point is judged to be an effective hole point to be filled when Y _k > R;

In step 5.2, specifically, in order to determine the effective hole point in step 5.1.1, the size of the initially set neighborhood is m, and the determined neighborhood of part of hole points may be greater than m, and when specifically performing calculation, the size of the filtering neighborhood window of each hole point determines the filtering size of each effective hole point to be filled according to the window when step 5.1.1 finally satisfies the condition of Y > R.

Step 5.3 is specifically to perform bilateral filtering calculation on the hole points to be filled according to formula (34),

Wherein:

In the above formula, I _b (p) is the intensity value of the central pixel of the filtering window after bilateral filtering, I (p) and I (q) are the intensity values of the central pixel of the filtering window before filtering and the intensity values of other pixels respectively, p and q are position coordinates, sigma _s is the standard deviation of a spatial gaussian kernel, the influence of spatial proximity on the weight coefficient of the filter is controlled, sigma _r is the standard deviation of the value range gaussian kernel, the influence of the similarity of pixel values on the weight coefficient of the filter is controlled, W _p is a normalization factor, the weight sum of the filter is ensured to be 1, and S is the neighborhood region range of the bilateral filter.

The three-dimensional point cloud cavity repairing test specifically comprises that,

According to the binocular image pair three-dimensional reconstruction repair experiment, a concrete surface defect test block picture is selected for experiment, the point cloud repair method based on bilateral filtering is adopted for point cloud repair, the result is compared with the three-dimensional point cloud repair method adopting mean filtering and Gaussian filtering, and the effectiveness of the algorithm is evaluated from the visual three-dimensional point cloud repair effect and the three objective evaluation indexes. The physical dimensions of the concrete test block used were 100mm×100mm. The binocular stereo vision system has a left and right camera resolution of 2064pixel x 1544pixel. And setting parameters of three-dimensional reconstruction, wherein the step length of the set matching points is 15 pixels, the window size of the selected matching sub-region is 31×31 pixels, and dividing to obtain 101×135 three-dimensional points in total. In order to perform omnibearing comparison with filling effects of other filtering modes, filling and repairing are performed on the three-dimensional point cloud by adopting an average value, gaussian filtering and bilateral filtering with initial window sizes of 5,7 and 9, and a visualized result after filling the point cloud is shown in fig. 10.

As can be seen from the result of fig. 8, compared with the three-dimensional point cloud restoration result based on bilateral filtering, the median filtering is smoother than the gaussian filtering, and the continuity of the restored effect is better. Meanwhile, the filtering effect under different windows can be obviously distinguished, and the result when the initial value of the filtering window is 5 multiplied by 5 is obviously better than the result when the initial value of the window is 9 multiplied by 9, which proves that the oversized window is unfavorable for recovering the three-dimensional point at the position of the filtering center. In order to further objectively compare the relation between the window size and the point cloud recovery quality, objective evaluation is performed by using a point cloud surface quality evaluation index, the results of objective evaluation indexes of filters with different window sizes and cavity recovery effects of different filters are shown in the following table, S in the table represents a local smoothness evaluation index, sigma represents a local standard deviation of three-dimensional coordinates, D represents an average distance of a fitting plane, different columns represent the range of a local evaluation area, and (5×5) represents index calculation of three-dimensional point clouds in a 5×5 range in a cavity point neighborhood range.

Table 2 point cloud repair local smoothness evaluation

TABLE 3 evaluation of local standard deviation of three-dimensional coordinates for point cloud repair

Table 4 point cloud repair fit plane average distance evaluation

As can be seen from the three objective evaluation index calculation results in table 2, table 3 and table 4, the three-dimensional point cloud repair result is the repair result with highest quality in the local areas with different sizes, which are evaluated based on the bilateral filtering repair result, in the three-dimensional coordinate local standard deviation and the fitting plane average distance, and in the local smoothness evaluation index, when the evaluated area is large enough, the three-dimensional point cloud local repair result based on the bilateral filtering is better. In addition, in the table, three evaluation indexes have small change of the size of the filtering window in the point cloud restoration result based on bilateral filtering, and the quality evaluation result gradually becomes worse along with the enlargement of the filtering window in the restoration result based on average filtering, so that the self-adaptive effect of bilateral filtering on the window size is good, the robustness is high, and the point cloud restoration method has higher practicability in the smoothness of point cloud restoration.

Second embodiment

The embodiment provides a self-adaptive repair system for a three-dimensional reconstruction point cloud cavity of a side slope surface, which uses the self-adaptive repair method for the three-dimensional reconstruction point cloud cavity of the side slope surface according to the embodiment, the repair system comprises,

The image acquisition module is used for acquiring a side slope stereo image pair, wherein the side slope stereo image pair is images of left and right cameras of a side slope at the same moment, which are acquired by a stereo vision system consisting of two cameras with the same model and specification, and internal and external parameters of the stereo vision system are acquired by a checkerboard calibration method;

The data hole filling module is used for selecting uniformly distributed reference points on the enhanced left image, and constructing a matching sub-region with the size of (2m+1) x (2m+1) by taking the reference points as the center;

The method comprises the steps that a matching subarea divided on an enhanced left image is used as a reference subarea, stereo matching is carried out on a right image after image enhancement, and right image coordinates of a matching point corresponding to a reference point in the right image after enhancement are obtained;

bilateral filtering is carried out on the hole points in the right image coordinates of the reference points obtained through stereo matching, and the right image coordinates of the matching points after hole repair are obtained;

According to the stereoscopic vision mathematical model, the left image coordinates of the reference point, the right image coordinates after cavity repair and the internal and external parameters of the stereoscopic vision system are combined, the three-dimensional coordinates of the reference point are obtained through calculation, and the three-dimensional data of the side slope obtained through self-adaptive repair are obtained.

Embodiment III

The embodiment is a three-dimensional deformation monitoring test of the side slope surface, and the test aims to verify the effectiveness of the method, and is specifically prepared by the following steps:

The experimental study was performed using a stereoscopic slope slip deformation monitoring system consisting of two ME2P-1840-21U3M cameras (resolution 4504 pixel. Times.4096 pixel) and THINKPAD P moving graphics workstation, with the experimental arrangement shown in FIG. 11. The base distance between the left camera and the right camera of the stereoscopic vision system is 190mm, the left camera and the right camera select 25mm focal length lenses, the height of the cameras is 1.5m, and the nearest end of the stereoscopic vision system to the side slope is 7.2m. During the test, a geodetic coordinate system is established based on the checkerboard calibration plate, and a 7X7 checkerboard calibration plate is selected, wherein the size of the checkerboard is 150mm multiplied by 150mm, and the thickness is 10mm. When the test is carried out, the checkerboard is vertically arranged at the junction of the underground ditch and the ground and is vertical to the ground, the identification point O _g at the left lower corner of the checkerboard is taken as an original point, the direction of the vertical checkerboard calibration plate is a Y _g axis, the vertical upward direction is a Z _g axis, and the horizontal direction is an X _g axis.

In the test, the stereo vision monitoring system continuously acquires the side slope surface images, the total of 81 pairs of stereo image pairs is acquired, 8 pairs of representative image pairs are selected for three-dimensional reconstruction, continuous three-dimensional reconstruction calculation is carried out on the side slope surface deformation based on a geodetic coordinate system, and as shown in fig. 12, the size of a three-dimensional reconstruction area in the side slope monitoring area of the stereo vision system is 3.7mX3.0m. The step length of the monitoring points divided by the three-dimensional deformation monitoring of the side slope is set to be 35pixel, and the size of the stereo matching window is set to be 75pixel multiplied by 75pixel. The method provided by the invention is used for monitoring image enhancement and three-dimensional data restoration, and the three-dimensional monitoring result of the side slope is shown in fig. 13. According to the test result, the slope three-dimensional deformation monitoring data enhancement and self-adaptive restoration method provided by the invention can effectively fill the data hole during slope monitoring, and improve the integrity of the slope three-dimensional deformation monitoring data.

Claims

1. A method for adaptively repairing holes in a point cloud of a three-dimensionally reconstructed slope surface, characterized in that the method specifically comprises the following steps:

Step 1: Obtain a pair of stereo images of the slope, wherein the pair of stereo images of the slope is a left and right camera image of the slope at the same time obtained by a stereo vision system composed of two cameras of the same model and specification, and the internal and external parameters of the stereo vision system are obtained by using a chessboard calibration method;

Step 2: Based on the stereo image pair obtained in step 1, the stereo image pair is adaptively enhanced using a three-histogram enhancement method based on an S membership function;

Step 3: Select uniformly distributed reference points on the left image enhanced in step 2, and construct a matching sub-region of size (2m+1)×(2m+1) with the reference points as the center;

Step 4: Using the matching sub-region divided on the enhanced left image in step 3 as the reference sub-region, stereo matching is performed in the enhanced right image to obtain the right image coordinates of the matching points corresponding to the reference points in the enhanced right image;

Step 5: Perform bilateral filtering on the hole points in the right image coordinates of the reference points obtained by stereo matching in step 4 to obtain the right image coordinates of the matching points after hole repair;

Step 6: According to the stereo vision mathematical model, the three-dimensional coordinates of the reference point are calculated by combining the left image coordinates of the reference point with the right image coordinates of the matching point after hole repair and the internal and external parameters of the stereo vision system to obtain the three-dimensional data of the slope surface obtained by adaptive repair.

2. According to the method for adaptively repairing holes in point cloud of three-dimensional reconstruction of slope surface according to claim 1, it is characterized in that step 2 specifically comprises the following steps:

Step 2.1: Perform stretching processing on the entire acquired image, stretching the histogram range of the image to the full grayscale range, and obtaining a stretched histogram image I;

Step 2.2: Divide the histogram image I stretched in step 2.1 into four non-overlapping sub-blocks I _i , i∈{1,2,3,4};

Step 2.3: Adaptively crop the histogram of each sub-block after the block division in step 2.2;

Step 2.4: Using the mean square error of each sub-image block I _i as the boundary for dividing different brightness areas, segment the histogram of each sub-image block I _i , and further divide the sub-image block into sub-images I _ij of three brightness ranges;

Step 2.5: Based on the histogram clipped in step 2.3, each sub-histogram in step 2.4 is equalized;

Step 2.6: Use bilinear interpolation to process the images of each sub-histogram processed in step 2.5 to obtain the final image enhancement result.

3. According to claim 2, a method for adaptively repairing holes in point cloud of three-dimensional reconstruction of slope surface, characterized in that step 2.1 is specifically to define the gray value of the pixel at the position (i, j) in the image before stretching transformation as g(i, j), and the gray value of the point after stretching transformation as G(i, j), define the minimum gray value in the image as g _min , and the maximum gray value as g _max , then the histogram stretching transformation can be expressed as:

4. According to claim 3, a method for adaptively repairing holes in point cloud of three-dimensional reconstruction of slope surface, characterized in that, in step 2.3, a threshold-based histogram clipping formula is as follows:

Where N represents the total number of pixels in a sub-block, β represents the cropping value of the histogram, L _max represents the maximum grayscale level in the sub-block, and γ represents the grayscale mean of the image sub-block. represents the grayscale mean square error of the sub-block, _kmax is the maximum slope, and CL represents the final clipping value of the histogram of the sub-block;

The S-type membership function is introduced to replace the fixed value of β, and the functional relationship between the clipping coefficient and the grayscale value is established; the expression of S(x) is as follows:

Where x represents different gray levels, ranging from 0 to 255, and S(x) ranges from 0 to 1. By calculating the cropping values of different gray levels, 256 different cropping values are obtained to achieve adaptive cropping of the image.

Obtain different histogram clipping values at different pixel proportion gray levels to achieve adaptive clipping of the image; the modified clipping value expressions for each gray level are as follows:

Wherein, _ni is the number of pixels of the i-th gray level, _nmax and _nmin are the number of pixels of the gray level with the largest and smallest number of pixels in the image sub-block, and _xi is the S membership function parameter corresponding to the i-th gray level. All _ni with gray levels between _nmax and _nmin in the histogram are mapped to the interval [0,255] to obtain the cropping parameter value cl _i that changes with the number of pixels in different gray levels;

Substituting equation (5) into the clipping threshold expression (2), we can obtain the histogram clipping threshold CL _i under different quantity distributions, which is expressed as follows:

The histograms of the four sub-blocks are clipped according to the clipping threshold calculated based on the S-type membership function, and the pixels exceeding the threshold are evenly distributed to other gray levels to obtain the histogram distribution H _s (i) after histogram clipping.

5. According to the method for adaptively repairing holes in point cloud of three-dimensional reconstruction of slope surface in claim 4, it is characterized in that the step 2.4 is specifically that if the processed image is an image taken by a single camera, the components of different brightness ranges in the image are separated, and the mean square error calculation formula is as follows:

Among them, G _mean is the grayscale mean of the image, _ni is the number of pixels with grayscale level i in the original image, and I _sd is the mean square error of the image; the upper and lower segmentation points U and B of the histogram of the input image are calculated as:

U＝L ₀ +I _sd (10)

B＝ _Lmax - _Isd (11)

Among them, L ₀ and L _max are the minimum and maximum grayscale levels respectively. Since the histogram has been stretched to the full grayscale range in the first step, L ₀ and L _max are 0 and 255 respectively.

6. According to claim 5, a method for adaptively repairing holes in point cloud of three-dimensional reconstruction of slope surface is characterized in that, if the image processed in step 2.4 is a stereo image pair, then for the selection of the subhistogram segmentation point positions of the left and right images at the same time, the mean of the segmentation points calculated by selecting the left and right images is considered, and the calculation expression is as follows:

7. According to claim 6, a method for adaptively repairing holes in point cloud of three-dimensional reconstruction of slope surface, characterized in that, in step 2.4, the first sub-histogram interval is [0, U]; the second sub-histogram grayscale interval is [U+1, B]; the third sub-histogram grayscale interval is [B+1, 255];

Perform histogram equalization on each subhistogram, and calculate the probability density distribution functions PDF ₁ (i), PDF ₂ (i), and PDF ₃ (i) of the three subhistograms as follows:

Wherein, N ₁ , N ₂ , and N ₃ are the total number of pixels in the three sub-histograms, respectively. The cumulative probability distribution functions CDF ₁ (I), CDF ₂ (I), and CDF ₃ (I) are calculated according to the probability distribution of each sub-histogram. The calculation formula is as follows:

Then the histogram equalization mapping relationship of each sub-histogram is obtained as follows:

I _S1 =U×CDF ₁ (I) (20)

I _S2 =(U+1)+(B-U+1)×CDF ₂ (I) (21)

I _S3 =(B+1)+(255-B+1)×CDF ₃ (I) (22)

Among them, I _S1 , I _S2 , and I _S3 are three image components of a certain image sub-block of the input image I after enhancement. After superposition, I _S is obtained, which is the image sub-block processed by the three-histogram enhancement method based on the S membership function. After the four sub-blocks divided by the original image are adaptively enhanced from step 2.3 to step 2.5, bilinear interpolation is used to process the edges of the sub-blocks to reduce the image block effect.

8. According to the method for adaptively repairing holes in point cloud of three-dimensional reconstruction of slope surface in claim 1, it is characterized in that the step 5 adopts an adaptive three-dimensional point cloud hole repair algorithm based on bilateral filtering to achieve the filling of holes in the three-dimensional reconstruction data of the slope surface, specifically,

Step 5.1: Traverse the right image coordinates of the reference points obtained by stereo matching in step 4 to determine the valid hole points to be filled;

Step 5.2: Determine the size of each effective hole point filter to be filled;

Step 5.3: Perform bilateral filtering calculation on the valid hole points to be filled in the right image coordinates to obtain the right image coordinate values of the reference points after hole filling.

9. According to claim 8, a method for adaptively repairing holes in point cloud of three-dimensional reconstruction of slope surface, characterized in that step 5.1 specifically comprises traversing the right image pixel coordinates of the stereo matching reference point, scanning all the hole points in the right image pixel coordinates with a window of size m×m, and calculating the proportion Y of the number of valid points in the neighborhood of the hole point, and artificially determining the threshold value R of the proportion of valid points, and the rules for the effective hole points to be filled are as follows:

Step 5.1.1: If Y>R in the neighborhood of the current hole point, it is determined to be a valid hole point to be filled;

Step 5.1.2: The effective point ratio in the neighborhood of the current hole point is 0.1<Y<R, and the neighborhood side length is expanded by 2. At this time, the neighborhood becomes (m+2)×(m+2), and the new effective point ratio Y ₁ is calculated; if Y ₁ >0.1, continue the above process, expand the neighborhood and calculate the new effective point ratio Y _k ; until Y _k <Y _k-1 <R, the point is determined to be an ineffective hole point to be filled, or when Y _k >R, it is determined to be an effective hole point to be filled;

Specifically, the scanning window size is initially set to m, and the determination window of some hole points may be larger than m. The filter window size of each hole point is determined according to the hole point determination window size when the Y>R condition is finally satisfied in step 5.1.

10. A slope surface 3D reconstruction point cloud hole adaptive repair system, characterized in that the repair system uses the slope surface 3D reconstruction point cloud hole adaptive repair method according to any one of claims 1 to 9, and the repair system comprises:

An image acquisition module is used to acquire the original image of the three-dimensional reconstruction data of the slope surface, wherein the image includes a photo image taken by a single camera and a binocular vision image;

The image enhancement module uses a three-histogram enhancement method based on the S membership function to enhance the image based on the image pair obtained by the stereo vision system;

The data hole filling module fills the data holes of the three-dimensional reconstruction of the slope surface based on bilateral filtering on the enhanced image, and completes the adaptive repair of the data holes of the three-dimensional reconstruction of the slope surface.