CN113421236A - Building wall surface water leakage apparent development condition prediction method based on deep learning - Google Patents

Abstract

The invention discloses a building wall surface water leakage apparent development condition prediction method based on deep learning, belonging to the field of deep learning and comprising the following steps: acquiring a water leakage video of a building wall surface, and carrying out distortion correction on all frame images in the water leakage video; extracting shape information of the leakage water from all frame images through video semantic segmentation and binarization technology based on a deep learning network, and establishing a time sequence sample library for predicting apparent development conditions of the leakage water; inputting the time sequence sample base into a Transformer network to obtain a time sequence prediction model of the relation between time and the apparent development condition of the leakage water; and inputting the video of the building wall leakage water to be predicted into the time sequence prediction model to obtain the apparent development condition of the building wall leakage water at a certain time in the future, wherein the apparent development condition of the leakage water comprises the shape information of the leakage water. According to the method, the semantic segmentation model and the time sequence prediction model based on deep learning are established, the future leakage water shape is predicted according to the existing leakage water video, and the defects of the prior art are overcome.

Description

Prediction method of apparent development of building wall seepage water based on deep learning

technical field

The invention relates to the technical field of deep learning, in particular to a method for predicting the apparent development condition of seepage water on a building wall based on deep learning.

Background technique

In recent years, with the continuous improvement of my country's economic level and the gradual enhancement of comprehensive national strength, my country's infrastructure construction, which is an important fulcrum for driving the economy, has shown a booming trend, thus promoting the continuous growth of the real estate development industry, and the number of large communities and high-rise buildings. And the volume continues to expand. However, due to the complex environment in the construction, maintenance and operation stages of housing construction, quality problems often occur during the use process. Among them, water leakage is one of the most common diseases of buildings.

With the advancement of technology, the traditional method of manually evaluating wall leakage is gradually being replaced by new methods due to the disadvantages of high subjectivity of results, high requirements for professional experience of inspectors, and low accuracy and efficiency. alternative.

With the rapid development of computer technology, deep learning methods have gradually replaced traditional image processing techniques and become the mainstream methods for image classification, object detection and instance segmentation. Therefore, they are widely used in the civil engineering industry and have achieved many results. The detection of the apparent development of seepage water based on deep learning has also been applied in the construction industry, which has the advantage of being more accurate and convenient than manual work. In addition, for shape detection, various fields have also made great progress. However, the research on the development prediction of seepage water containing time information is still in the blank stage.

In 2010, Liu Lei published the article "Application of Machine Learning in Candy Shape Detection" in the journal "Software Engineer". The feature vector method was used to describe the shape of candy as knowledge, and together with the corresponding response, it was used to train decision trees for training data. Classify acceptable candies into the correct shape classification and separate from non-qualified candies. This paper realizes the preliminary judgment of the shape, but because the decision tree has fewer levels, only about 60% success rate, it is not widely applicable.

Huang Hongwei and Li Qingtong published "Image Recognition of Leakage Diseases in Shield Tunnels Based on Deep Learning" published in "Chinese Journal of Rock Mechanics and Engineering" in 2017, and proposed the image recognition of water leakage diseases in shield tunnels based on fully convolutional networks. The convolution operation and pooling operation are used to automatically extract and learn the advanced image features of tunnel diseases. Compared with the traditional image recognition method, it can effectively avoid the influence of interfering objects such as segment joints, bolt holes, pipelines, brackets, etc., especially It has superior robustness in overcoming the occlusion of pipelines, but this method can only identify and locate the leaking water area, cannot obtain other information such as the area of leaking water, and cannot evaluate the degree of disease of the leaking water.

The Chinese invention patent with the publication number CN109359130A applied by China Communications Highway Planning and Design Institute Co., Ltd. in 2018 discloses a bridge disease grading and grading maintenance method and system, which fills the gap of disease grading for different bridge components. The information and disease information of each component and its components of the bridge structure are stored in the database, the influencing factors and weights of the disease are determined to obtain the disease grade score, and then the severity of the disease is judged according to the input disease information and disease characteristics. The maintenance decision is more scientific, but the classification of diseases is relatively rough. In the example, only two kinds of cracking diseases and cracking diseases are listed. The subsequent research can further refine the diseases.

In 2020, Yu Guolong, Zhao Yong, Wu Lian, and Cui Zhongwei published in the journal "Computer Engineering and Application" the article "The Improvement of QPSO Algorithm and Its Application in DBN Parameter Optimization", which will optimize the DBN network with the improved QPSO algorithm ( LQ_DBN) is applied in egg yolk shape detection. The method proposed in this paper has the advantages of higher accuracy and less hardware requirements than the predictions made by existing models.

The 2020 master's thesis of South China University of Technology "Time Series Forecasting and Classification Based on Deep Learning" conducts research on time series forecasting and classification problems by establishing deep learning models, and successfully proves the classification and generalization capabilities of the two deep learning models. The method proposed in this paper effectively improves the time-consuming and labor-intensive problems caused by artificially designing features in traditional machine learning.

The Chinese invention patent with publication number CN109615653A discloses a method for detecting and identifying leakage water area based on deep learning and field of view projection model, which proposes an accurate detection and identification of leakage water area based on deep learning and field of view projection model method. The invention does not need manual participation in the measurement of the leakage water area of the subway tunnel, improves the work efficiency, and can perform surface projection transformation on the detected leakage water area to obtain a more accurate leakage water area. However, it is only suitable for detection of relatively regular shapes, such as planes, elliptical cylinders, etc., and the scope of application is severely limited.

The Chinese invention patent with publication number CN111899288A discloses a method for detecting and identifying tunnel water leakage areas based on the fusion of infrared and visible light images, which proposes a method for detecting and identifying tunnel water leakage areas based on fusion of infrared and visible light images. The invention can effectively solve the interference problem of poor lighting conditions in the tunnel and abnormal epoxy resin on the detection of water leakage, and has the advantages of high precision and high adaptability. But the requirements for equipment are higher.

At present, scholars at home and abroad have carried out a lot of research on image shape judgment methods based on computer vision. However, in the field of construction engineering, the research on water leakage identification and detection mainly focuses on judging the size of the leakage area by calculating Its severity, time series prediction based on various neural network structures is temporarily unexplored.

Summarizing the existing evaluation methods, for the evaluation of the apparent development of seepage water, the current evaluation methods are mainly based on the study of the shape and area, and cannot complete the prediction of the future development trend.

SUMMARY OF THE INVENTION

Aiming at the problem that the existing technology cannot predict the apparent development of the seepage water on the building wall, the purpose of the present invention is to provide a method for predicting the apparent development of the seepage water on the building wall based on deep learning.

For achieving the above object, the technical scheme of the present invention is:

A method for predicting the apparent development of seepage water in building walls based on deep learning includes the following steps:

Acquiring a video of water leakage on the building wall, and performing distortion correction on all frame images in the water leakage video;

The shape information of the seepage water is extracted from all the frame images through the video semantic segmentation and binarization technology based on the deep learning network, so as to establish a time series sample library for the prediction of the apparent development status of the seepage water;

The time series sample library is input into the Transformer network to obtain the time series prediction model of the relationship between time and the apparent development of seepage water;

Input the water leakage video of the building wall to be predicted into the time series prediction model, and obtain the apparent development status of the building wall leakage water at a certain time in the future, and the apparent development status of the seepage water includes the shape information of the leakage water.

Further, the method also includes,

After obtaining the apparent development status of the seepage water on the building wall at a certain time in the future, the stage of the seepage water is determined according to the shape information of the seepage water and the preset judgment conditions; wherein, the stage of the seepage water is It includes the occurrence stage of seepage water, the development stage of seepage water, the expansion stage of seepage water and the forming stage of seepage water.

Preferably, the steps of obtaining a water leakage video on the building wall and performing distortion correction on all frame images in the water leakage video are:

The camera is calibrated by the computer vision toolkit HALCON calibration method, and the internal parameters of the camera are obtained;

Keep the internal parameters of the camera unchanged, shoot the water leakage video on the front of the building wall, and measure the shooting distance and shooting inclination;

Then, a division distortion model is used to perform distortion correction on all frame images in the water leakage video.

Preferably, the computer vision toolkit HALCON calibration method is used to calibrate the camera, and the steps of obtaining the internal parameters of the camera are:

Select the calibration plate made of ceramic material;

When sampling, keep the light source in front of the calibration plate, and the light source is located in the opposite direction of the camera, and ensure that all four corners of the calibration plate are within the field of view of the camera;

Collect more than 12 calibration plate images;

The HALCON calibration program is used to process the collected calibration plate images to obtain the internal parameters of the camera.

Preferably, the steps of performing distortion correction on all frame images in the water leakage video by using a division distortion model are:

The division distortion model is used to correct the distortion of all frame images. The radial distortion correction formula is as follows:

分别表示畸变后的投影点

在成像平面坐标系下的行坐标和列坐标；径向畸变参数κ表示径向畸变量级,如果κ为负,畸变为桶型畸变,如果κ为正,畸变为枕型畸变；u、v分别表示畸变校正后的投影点

在成像平面坐标系下的行坐标和列坐标。in,

respectively represent the distorted projection points

The row and column coordinates in the imaging plane coordinate system; the radial distortion parameter κ represents the magnitude of radial distortion, if κ is negative, the distortion is barrel distortion, if κ is positive, the distortion is pincushion distortion; u, v Respectively represent the projection points after distortion correction

Row and column coordinates in the imaging plane coordinate system.

Preferably, the shape information of the seepage water is extracted from all the frame images through video semantic segmentation and binarization technology based on a deep learning network, so as to establish a time series sample library for predicting the apparent development of seepage water The steps are:

Select part of frame images from all frame images at preset intervals, and use the labelme image annotation tool to manually annotate the part of the frame images to obtain manual annotation samples;

According to the manual labeling samples, use the spatial displacement convolution block to predict future frames and future labels, and simultaneously propagate the future frames and future labels, obtain synthetic samples, and form a leaky water database;

Train the Deeplabv+3 model with the water leakage database to obtain the semantic segmentation model of the water leakage shape;

Semantically segment all frame images through the water leakage shape semantic segmentation model, and extract the water leakage images;

Converting the water leakage images extracted from all frame images into grayscale images;

Obtain the shape information of the black part in the grayscale image of each frame, form the corresponding relationship between the time and the shape characteristics of the seepage water, and obtain the time series sample library for the prediction of the apparent development status of the seepage water.

Preferably, the spatial displacement convolution block predicts objects in the future frame according to the current frame of the video and the past frame, and the future frame and the future label can be propagated forward or backward by the original frame when propagating the future frame; wherein, the future frame , the coordinates F _t+1 (x, y) and M _t+1 (x, y) of any point in the future label are determined by the following formulas:

Ft ₊₁ (x,y)=K(x,y)D _Ft (x+u,y+v);

_Mt+1 (x,y)=K(x,y)D _Mt (x+u,y+v);

(u,v)=C(I _1:t );

where (x, y) is the coordinate of any pixel in a certain frame, C is the 3D CNN used to predict the motion vector (u, v) based on the input frames F ₁ to F _t , K(x, y) ∈ R ^N×N is the N×N two-dimensional weight kernel predicted by C at (x, y), D _Ft (x+u, y+v), D _Mt (x+u, y+v) are F respectively _1. An N×N two-dimensional kernel centered at (x+u, y+v) in M ₁ .

Preferably, the step of converting the water leakage images extracted from all frame images into grayscale images is as follows:

The gray value of the pixel block in the water leakage image is determined according to the following formula,

其中，D_i为彩色图像中第i个像素点转换后的灰度值；

Among them, D _i is the converted gray value of the ith pixel in the color image;

A corresponding grayscale image is generated according to the grayscale value corresponding to each pixel block.

Preferably, the steps of obtaining the shape information of the black part in the grayscale image of each frame, forming the relationship between the time and the shape characteristics of the seepage water, and obtaining the time series sample library for predicting the apparent development status of the seepage water are:

Determine the maximum distance in the horizontal direction of the black part in the grayscale image of each frame, and connect the left and right endpoints at the maximum distance to be the baseline segment;

Taking the midpoint of the baseline segment as the emission point of the central ray, and taking the emission point as the origin, a plurality of rays are made at a preset interval angle, and the intersection point of the ray and the boundary contour of the black part is determined;

After connecting the intersection points in sequence, the shape information of the black part is obtained, and then the shape information of the black part is corresponding to each frame of image, so as to obtain a time series sample library for predicting the apparent development of seepage water.

Preferably, the step of determining the stage of water leakage according to the shape information of the water leakage and a preset judgment condition includes:

Determine the maximum distance in the horizontal direction of the leakage area, and connect the left and right endpoints at the maximum distance as the baseline segment;

Taking the midpoint of the baseline segment as the emission point of the central ray, and taking the emission point as the origin, multiple rays are made at a preset interval angle, and the intersection point of the ray and the boundary contour of the leakage area is determined;

determining the distance from the emission point to the intersection point in all rays, and determining the ratio of the maximum value of the distance to the baseline segment;

If the ratio is equal to 1, it is the water leakage occurrence stage, if it is greater than 1 but less than or equal to 1.5, it is the leakage water development stage, if it is greater than 1.5 but less than or equal to 2, it is the leakage water expansion stage, and if it is greater than 2 but less than or equal to 3, it is Leaky water forming stage.

The beneficial effect of the present invention is that the shape information of the seepage water is extracted from the video image by using the video semantic segmentation and binarization technology of the deep learning network, and then a time series sample library for predicting the apparent development status of the seepage water is established. setting, so that after the time series sample library is input into the Transformer network, the time series prediction model of the relationship between time and the apparent development of seepage water can be obtained, and then the model can be used to conduct the seepage water area on the building wall. Prediction, so as to obtain the prediction of the shape of the water leakage on the building wall at a certain time in the future.

Description of drawings

1 is a flow chart of a method according to Embodiment 1 of the present invention;

Fig. 2 is the process schematic diagram of carrying out semantic segmentation to each frame image in the leaking water video in the present invention and obtaining the leaking water image and converting it into a grayscale image;

Fig. 3 is the development schematic diagram of water seepage shape of building wall in the present invention;

Fig. 4 is the method flow chart of the second embodiment of the present invention;

FIG. 5 is a schematic diagram of the development of the shape of the water leakage on the building wall in the present invention.

Detailed ways

The specific embodiments of the present invention will be further described below with reference to the accompanying drawings. It should be noted here that the descriptions of these embodiments are used to help the understanding of the present invention, but do not constitute a limitation of the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

It should be noted that, in the description of the present invention, the orientation or positional relationship indicated by the terms "upper", "lower", "left", "right", "front", "rear", etc. The description of the structure of the invention is only for the convenience of describing the invention, rather than indicating or implying that the referred device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention.

For the "first" and "second" in this technical solution, it is only the appellation and distinction of the same or similar structures, or the corresponding structures with similar functions, not the arrangement of the importance of these structures, nor the ordering or comparison. size, or otherwise.

In addition, unless otherwise expressly specified and limited, the terms "installation" and "connection" should be understood in a broad sense. For example, the connection may be a fixed connection, a detachable connection, or an integral connection; it may be a mechanical connection, or a connection. It can be an electrical connection; it can be a direct connection, or an indirect connection through an intermediate medium, or the internal connection of the two structures. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to the general idea of the present invention and the specific circumstances of the context of the present solution.

Example 1

As shown in FIG. 1 , the method for predicting the apparent development condition of seepage water on a building wall based on deep learning includes steps S1 , S2 , S3 , S4 and S5 .

In step S1, a video of water leakage on the building wall is obtained.

In this embodiment, while keeping the internal parameters of the camera unchanged, the required water leakage video is shot from the front of the building wall by the camera, and the shooting distance and shooting inclination of the camera are measured at the same time.

Among them, the internal parameters of the camera are obtained by calibrating the camera through the computer vision toolkit HALCON calibration method. The specific steps are:

1. Select a calibration plate of suitable size and made of ceramic material; 2. Keep the light source in front of the calibration plate, and ensure that the light source is in the opposite direction of the camera, and ensure that the four corners of the calibration plate are all within the camera's field of view; 3. Collect more than 12 calibration plate images; 4. Use HALCON calibration program to process the collected calibration plate images to obtain the internal parameters of the camera.

Step S2, perform distortion correction on all frame images in the water leakage video.

In this embodiment, the water leakage video is disassembled in advance to obtain all frame images in the video, and then the division distortion model is used to perform distortion correction on all frame images in the water leakage video obtained in the above step S1. The division distortion model is used to correct the distortion of all frame images. The radial distortion correction formula is as follows:

分别表示畸变后的投影点

在成像平面坐标系下的行坐标和列坐标；径向畸变参数κ表示径向畸变量级,如果κ为负,畸变为桶型畸变,如果κ为正,畸变为枕型畸变；u、v分别表示畸变校正后的投影点

在成像平面坐标系下的行坐标和列坐标。formula,

respectively represent the distorted projection points

The row and column coordinates in the imaging plane coordinate system; the radial distortion parameter κ represents the magnitude of radial distortion, if κ is negative, the distortion is barrel distortion, if κ is positive, the distortion is pincushion distortion; u, v Respectively represent the projection points after distortion correction

Row and column coordinates in the imaging plane coordinate system.

In step S3, the shape information of the seepage water is extracted from all the frame images through the video semantic segmentation and binarization technology based on the deep learning network, so as to establish a time series sample library for predicting the apparent development state of the seepage water.

In this embodiment, the specific steps for establishing a time series sample library for predicting the apparent developmental state of seepage water are as follows:

1. Select some frame images from all the frame images of the leaking video at preset intervals, and use the labelme image annotation tool to manually label the selected frame images to obtain manually labeled samples.

2. According to the above-obtained manually labeled samples, use spatial displacement convolution blocks to predict future frames and future labels, and simultaneously propagate future frames and future labels to obtain synthetic samples to form a leaky water database.

Among them, the spatial displacement convolution block is used to predict objects in future frames according to the current frame of a video and its past frames, and the future frames and future labels can be propagated forward or backward by the original frame. Among them, the coordinates F _t+1 (x, y) and M _t+1 (x, y) of any point in the future frame and future label are determined by the following formulas:

Ft ₊₁ (x,y)=K(x,y)D _Ft (x+u,y+v);

_Mt+1 (x,y)=K(x,y)D _Mt (x+u,y+v);

(u,v)=C(I _1:t );

In the formula, (x, y) is the coordinate of any pixel in a certain frame, C is the 3D CNN used to predict the motion vector (u, v) based on the input frame F ₁ to F _t , K(x, y) ∈R ^N×N is the N×N two-dimensional weight kernel predicted by C at (x, y), D _Ft (x+u, y+v), D _Mt (x+u, y+v) are respectively An N×N two-dimensional kernel centered on (x+u, y+v) in F ₁ and M ₁ .

3. Train the Deeplabv+3 model with the leaked water database obtained above to obtain a semantic segmentation model of the leaked water shape.

4. Semantically segment all frame images in the leaking water video through the above-mentioned leaky water shape semantic segmentation model, so as to extract the leaking water image that only contains the leaking water area.

5. Convert the water leakage images extracted from all frame images into grayscale images.

In this embodiment, the gray value of the pixel block in the water leakage image is determined according to the following formula:

其中，D_i为彩色图像中第i个像素点转换后的灰度值；

Among them, D _i is the converted gray value of the ith pixel in the color image;

After that, a corresponding grayscale image can be generated according to the grayscale value corresponding to each pixel block.

As shown in Figure 2, it shows that when the image contains sundries such as doors and windows, the water leakage image containing only the water leakage area is extracted and converted into a grayscale image through the semantic segmentation model of the water leakage shape. the process of.

6. Obtain the shape information of the black part in the grayscale image of each frame, and since the grayscale images are arranged in time series, the corresponding relationship between time and the shape characteristics of the seepage water can be formed, so as to obtain the apparent development status of the seepage water. Predicted time series sample library.

In this embodiment, the specific steps of obtaining the shape information of the black part in each frame of grayscale image and forming the corresponding relationship between time and the shape feature of the leaking water are:

1. Determine the maximum distance in the horizontal direction of the black part in the grayscale image of each frame, and connect the left and right endpoints at the maximum distance as the baseline segment.

2. Take the midpoint of the baseline segment as the emission point of the central ray, and take the emission point as the origin, make multiple rays at a preset interval angle (for example, 1°), and determine the intersection of all rays with the boundary contour of the black part.

3. After connecting the above-mentioned intersection points in sequence, the shape information of the black part is obtained, and then the obtained shape information of the black part is corresponding to each frame image, thereby obtaining a time series sample library for predicting the apparent development of seepage water.

As shown in Table 1, a perfect circular water leakage area with a diameter of 1 cm appeared on the building wall on the 1st day; as shown in Table 2, it is the development of the above-mentioned leakage area on the 10th day, approximately forming the upper half It is a semicircle of 1 cm, and the lower part is a semi-ellipse with a short semi-axis of 1 cm and a long semi-axis of 1.5 cm; as shown in Table 3, it is the continuous development of the above-mentioned leakage area on the 30th day, and its approximate upper half is 1 cm. The lower half is a semi-ellipse with a short semi-axis of 1 cm and a long semi-axis of 2 cm; as shown in Table 4, it is the continuous development of the above-mentioned leakage area on the 60th day, which approximately forms a semi-circle with an upper half of 1 cm. The lower part is a semi-ellipse with a minor semi-axis of 1 cm and a major semi-axis of 3 cm.

Table 1: Ray Azimuth and Distance from Launch Point to Intersection on Day 1

Table 2: Ray Azimuth and Distance from Launch Point to Intersection on Day 10

Table 3: Ray azimuth and distance from launch point to intersection on day 30

Table 4: Ray Azimuth and Distance from Launch Point to Intersection on Day 60

In addition, the graph shown in Figure 3 is obtained after the data in the above Tables 1-4 are graphed. It can be seen that the apparent development of the seepage water on the building wall has a certain trend, and its upper half basically remains a semicircle. The lower part is an ellipse whose major axis increases with time.

In step S4, the time series sample library obtained above is input into the Transformer network, and a time series prediction model of the relationship between time and the apparent development status of the seepage water can be obtained.

That is, take the time series sample library established in the above table 1-4 as the input variable of the time series prediction model, take the shape parameter of seepage water at a certain time (the time in the table) as the output variable of the time series prediction model, and compare the output variable with the actual measurement in the above table. The values of , are compared and verified, so that the Transformer time series prediction model can be established.

Step S5, input the video of the water leakage on the building wall to be predicted into the time series prediction model, and obtain the apparent development status of the leakage water on the building wall at a certain time in the future, wherein the apparent development status of the leakage water includes the shape information of the leakage water .

According to the establishment process of the above-mentioned time series sample library and time series prediction model, after inputting the water leakage video of the building wall to be predicted and a certain time in the future into the Transformer time series prediction model, the Transformer time series prediction model will output a series of predictions. The prediction point is the outer contour point of the seepage area at a certain moment in the future, and the outer contour point refers to the intersection point in the above time series prediction library, and the average point of the outer contour output by the model is determined by the emission point of the ray. , the azimuth angle of the ray, the distance from the emission point to the intersection point are limited by three parameters, where the emission point of the ray is the midpoint of the maximum width in the horizontal direction of the leakage area, and the azimuth angle of the ray is the angle between the ray and the horizontal direction (The azimuth angle difference between adjacent rays satisfies the above-mentioned preset interval angle), the distance from the emission point to the intersection point is the distance between each prediction point and the emission point of the ray, and the representative seepage output of the Transformer time series prediction model is used. After the predicted points of the outer contour of the water leakage area are connected in sequence, the water leakage shape of the building wall at a certain moment in the future can be obtained.

Embodiment 2

The difference from the first embodiment is that: in this embodiment, as shown in FIG. 4 , the above method further includes step S6.

Step S6, after obtaining the apparent development status of the water seepage on the building wall at a certain time in the future, then determine the water seepage stage according to the shape information of the seepage water and the preset judgment conditions; wherein, the seepage water stage includes: Seepage water occurrence stage, seepage water development stage, seepage water expansion stage and seepage water forming stage.

It can be understood that since the predicted point output by the Transformer time series prediction model contains a distance information, it is only necessary to compare the distance between the predicted point and the maximum horizontal width of the seepage shape (the length of the baseline segment where the ray emission point is located). The proportion of water leakage can be judged. As shown in Figure 5, with the continuous development of the seepage water, the lower half of the seepage water is elliptical and the long axis is continuously elongated. Therefore, as the azimuth angle of the ray increases, the distance between the ray and the seepage water area increases. The distance between the intersection of the outer contour boundary and the emission point of the ray also increases.

Among them, the size of the maximum width (baseline segment) in the horizontal direction of the leakage water shape will remain stable.

This embodiment is specifically set up. The ratio equal to 1 is the stage of water leakage. As shown in Table 1, a perfect circular water leakage area with a diameter of 1 cm appears on the wall; the ratio greater than 1 but less than or equal to 1.5 is the leakage In the development stage of water leakage, as shown in Table 2, the above-mentioned water leakage area develops into a semicircle with an upper half of 1 cm, and a lower half of a semi-ellipse with a short semi-axis of 1 cm and a long semi-axis of 1.5 cm; the ratio is greater than 1.5 However, if it is less than or equal to 2, it is the expansion stage of seepage water. As shown in Table 3, the above-mentioned seepage water area develops into a semicircle with an upper half of 1 cm, and a lower half of a semi-circle with a short semi-axis of 1 cm and a long semi-axis of 2 cm. Ellipse; if the ratio is greater than 2 but less than or equal to 3, it is the water leakage forming stage. As shown in Table 4, the above water leakage area develops into a semicircle with an upper half of 1 cm, and a lower half of a short semi-axis of 1 cm. A semi-ellipse with a major semi-axis of 3cm.

The embodiments of the present invention have been described in detail above with reference to the accompanying drawings, but the present invention is not limited to the described embodiments. For those skilled in the art, without departing from the principle and spirit of the present invention, various changes, modifications, substitutions and alterations to these embodiments still fall within the protection scope of the present invention.

Claims (10)

1. the method for predicting the apparent development status of the seepage water in building walls based on deep learning, is characterized in that: comprise the following steps, Acquiring a video of water leakage on the building wall, and performing distortion correction on all frame images in the water leakage video; The shape information of the seepage water is extracted from all the frame images through the video semantic segmentation and binarization technology based on the deep learning network, so as to establish a time series sample library for the prediction of the apparent development status of the seepage water; The time series sample library is input into the Transformer network to obtain the time series prediction model of the relationship between time and the apparent development of seepage water; Input the water leakage video of the building wall to be predicted into the time series prediction model, and obtain the apparent development status of the building wall leakage water at a certain time in the future, and the apparent development status of the seepage water includes the shape information of the leakage water. 2. The method according to claim 1, wherein the method further comprises: After obtaining the apparent development status of the seepage water on the building wall at a certain time in the future, the stage of the seepage water is determined according to the shape information of the seepage water and the preset judgment conditions; wherein, the stage of the seepage water is It includes the occurrence stage of seepage water, the development stage of seepage water, the expansion stage of seepage water and the forming stage of seepage water. 3. The method according to claim 1 or 2, characterized in that: the step of obtaining the water leakage video on the building wall, and performing distortion correction on all frame images in the water leakage video is: The camera is calibrated by the computer vision toolkit HALCON calibration method, and the internal parameters of the camera are obtained; Keep the internal parameters of the camera unchanged, shoot the water leakage video on the front of the building wall, and measure the shooting distance and shooting inclination; Then, a division distortion model is used to perform distortion correction on all frame images in the water leakage video. 4. method according to claim 3, is characterized in that: described adopting computer vision toolkit HALCON calibration method to demarcate camera, the step of acquiring camera internal parameter is: Select the calibration plate made of ceramic material; When sampling, keep the light source in front of the calibration plate, and the light source is located in the opposite direction of the camera, and ensure that all four corners of the calibration plate are within the field of view of the camera; Collect more than 12 calibration plate images; The HALCON calibration program is used to process the collected calibration plate images to obtain the internal parameters of the camera. 5. The method according to claim 3, wherein: the step of performing distortion correction on all frame images in the water leakage video using a division distortion model is: The division distortion model is used to correct the distortion of all frame images. The radial distortion correction formula is as follows:

in,

respectively represent the distorted projection points

The row and column coordinates in the imaging plane coordinate system; the radial distortion parameter κ represents the magnitude of radial distortion, if κ is negative, the distortion is barrel distortion, if κ is positive, the distortion is pincushion distortion; u, v Respectively represent the projection points after distortion correction

Row and column coordinates in the imaging plane coordinate system. 6. The method according to claim 1 or 2, wherein the shape information of the leaking water is extracted from all the frame images through video semantic segmentation and binarization technology based on a deep learning network, thereby The steps for establishing a time series sample library for predicting the apparent development of seepage water are: Select part of frame images from all frame images at preset intervals, and use the labelme image annotation tool to manually annotate the part of the frame images to obtain manual annotation samples; According to the manual labeling samples, use the spatial displacement convolution block to predict future frames and future labels, and simultaneously propagate the future frames and future labels, obtain synthetic samples, and form a leaky water database; Train the Deeplabv+3 model with the water leakage database to obtain the semantic segmentation model of the water leakage shape; Semantically segment all frame images through the water leakage shape semantic segmentation model, and extract the water leakage images; Converting the water leakage images extracted from all frame images into grayscale images; Obtain the shape information of the black part in the grayscale image of each frame, form the corresponding relationship between the time and the shape characteristics of the seepage water, and obtain the time series sample library for the prediction of the apparent development status of the seepage water. 7. The method according to claim 6, wherein the spatial displacement convolution block predicts objects in future frames according to the current frame and past frames of the video, and can be forwarded by the original frame when propagating future frames and future labels. Propagation or backward propagation; wherein, the coordinates F _t+1 (x, y) and M _t+1 (x, y) of any point in the future frame and the future label are determined by the following formulas: Ft ₊₁ (x,y)=K(x,y)D _Ft (x+u,y+v); _Mt+1 (x,y)=K(x,y)D _Mt (x+u,y+v); (u,v)=C(I _1:t ); where (x, y) is the coordinate of any pixel in a certain frame, C is the 3D CNN used to predict the motion vector (u, v) based on the input frames F ₁ to F _t , K(x, y) ∈ R ^N×N is the N×N two-dimensional weight kernel predicted by C at (x, y), D _Ft (x+u, y+v), D _Mt (x+u, y+v) are F respectively _1. An N×N two-dimensional kernel centered at (x+u, y+v) in M ₁ . 8. The method according to claim 6, wherein the step of converting the water leakage images extracted from all frame images into a grayscale image is: The gray value of the pixel block in the water leakage image is determined according to the following formula,

Among them, D _i is the converted gray value of the ith pixel in the color image; A corresponding grayscale image is generated according to the grayscale value corresponding to each pixel block. 9. The method according to claim 6, wherein: obtaining the shape information of the black part in the grayscale image of each frame, forming the relationship between the time and the shape feature of the seepage water, and obtaining the apparent development condition for the seepage water The steps of the predicted time series sample library are: Determine the maximum distance in the horizontal direction of the black part in the grayscale image of each frame, and connect the left and right endpoints at the maximum distance to be the baseline segment; Taking the midpoint of the baseline segment as the emission point of the central ray, and taking the emission point as the origin, a plurality of rays are made at a preset interval angle, and the intersection point of the ray and the boundary contour of the black part is determined; After connecting the intersection points in sequence, the shape information of the black part is obtained, and then the shape information of the black part is corresponding to each frame of image, so as to obtain a time series sample library for predicting the apparent development of seepage water. 10 . The method according to claim 2 , wherein the step of determining the stage of water leakage according to the shape information of the water leakage and a preset judgment condition comprises: 10 . Determine the maximum distance in the horizontal direction of the leakage area, and connect the left and right endpoints at the maximum distance as the baseline segment; Taking the midpoint of the baseline segment as the emission point of the central ray, and taking the emission point as the origin, multiple rays are made at a preset interval angle, and the intersection point of the ray and the boundary contour of the leakage area is determined; determining the distance from the emission point to the intersection point in all rays, and determining the ratio of the maximum value of the distance to the baseline segment; If the ratio is equal to 1, it is the water leakage occurrence stage, if it is greater than 1 but less than or equal to 1.5, it is the leakage water development stage, if it is greater than 1.5 but less than or equal to 2, it is the leakage water expansion stage, and if it is greater than 2 but less than or equal to 3, it is Leaky water forming stage.

Images