CN116188348A - Crack detection method, device and equipment - Google Patents

Abstract

A crack detection method, apparatus and device are disclosed to reduce crack detection costs. The method comprises the following steps: the camera acquires N frames of first images, wherein the N frames of first images are obtained by shooting different areas of a target object through a preset operation, and the preset operation is to rotate a lens of the camera and adjust the focal length of the lens; the camera sends the N frames of first images to the computing device; the computing equipment splices part of each image in the N frames of first images to obtain a second image; the camera detects a crack of the target object based on the second image. The camera collects N frames of first images and then sends the N frames of first images to the computing equipment, and the computing equipment splices parts of each image in the N frames of first images to obtain a second image. The computing device detects a crack of the target object based on the second image. Therefore, manual detection is not needed, manpower and material resources can be saved, and the crack detection cost is reduced.

Description

Crack detection method, device and equipment

technical field

The present application relates to the technical field of image processing, in particular to a crack detection method, device and equipment.

Background technique

Cracks in buildings such as roads or walls will affect the life of the building, and even threaten the personal safety of the people who use the building. Therefore, it is necessary to detect the cracks on the buildings, so as to repair the buildings with cracks and reduce the risk of further damages on the buildings.

Traditional crack detection mainly relies on manual inspection. In recent years, with the continuous addition of scientific and technological means, intelligent equipment such as road inspection vehicles and drones can also be used for crack detection.

However, whether it is manual inspection or crack detection with the help of intelligent equipment, it needs to invest manpower and material resources, and the detection cost is high.

Contents of the invention

The present application provides a crack detection method, device and equipment to reduce the cost of crack detection.

The first aspect provides a crack detection method. The method is performed by a camera and a computing device. The camera is connected to a computing device, the camera is used to collect the first image, and the computing device is used to extract cracks based on the first image. Specifically, the method includes: the camera collects N frames of first images. N is an integer greater than or equal to 2. The N frames of first images are obtained by the camera shooting different areas of the target object through preset operations. Wherein, the target object is, for example, a road, a wall, and the like. The preset operation is to rotate the camera lens and adjust the focal length of the lens. Different regions of the target object have different distances from the camera, and the distance from the region to the camera is proportional to the focal length of the lens. Specifically, the area of the target object is selected by rotating the lens of the camera, and the focal length of the lens of the camera is changed: for the area farther from the camera, the focal length of the lens is larger; for the area closer to the camera, the focal length of the lens is smaller . Therefore, target objects in different first images can be clearly presented. The camera collects N frames of first images and sends them to the computing device, and the computing device stitches parts of each of the N frames of first images to obtain a second image. The computing device detects cracks in the target object based on the second image. Therefore, manual detection is not required, manpower and material resources can be saved, and the cost of crack detection can be reduced.

With reference to the first aspect, in a first implementation manner of the first aspect of the present application, the second image is an image without a perspective effect. Due to the perspective phenomenon of the camera lens, the target object captured by the camera has the effect of being far smaller and closer, resulting in the loss of details of the distant target object, and the crack cannot be accurately detected. The first image of N frames is stitched to obtain the first image without perspective effect. With the second image, the details of distant target objects can also be clearly presented, and crack detection based on the second image will also be more accurate.

In combination with the first aspect or the first implementation of the first aspect, in the second implementation of the first aspect of the present application, the computing device stitches parts of each of the N frames of first images to obtain a second image The method includes: the computing device respectively intercepts M rows of pixels of the N frames of the first image, and M is an integer greater than or equal to 1. The computing device stitches the M rows of pixels of the N frames of the first image to obtain the second image. Therefore, the second image can be quickly spliced.

In combination with the first aspect or the first implementation of the first aspect, in the third implementation of the first aspect of the present application, the computing device stitches parts of each of the N frames of first images to obtain the second image The method includes: the computing device respectively intercepts partial pixels of each image in the N frames of first images, wherein the number of intercepted pixels of each image is different. The computing device splices some pixels of each of the N frames of first images to obtain a second image. Partial pixels of each first image correspond to a sub-region of the target object, and the second image includes a complete target object composed of N sub-regions. Since the camera poses and focal lengths corresponding to each first image are different, the number of pixels intercepted in each first image may be different.

In combination with the first aspect or the first implementation of the first aspect, in the fourth implementation of the first aspect of the present application, the computing device stitches parts of each of the N frames of first images to obtain the second image The method includes: the computing device intercepts P pixels of the N frames of the first image respectively, and P is an integer greater than or equal to 1. The computing device stitches the P pixels of the N frames of the first image to obtain the second image. By intercepting the same number of pixels from each first image, the second image can be quickly spliced.

In combination with the first aspect or any one of the first to fourth implementations of the first aspect, in the fifth implementation of the first aspect of the present application, the computing device detecting cracks of the target object based on the second image includes: computing device The second image is preprocessed to obtain J first image blocks, where the first image block includes a part of the target object, and J is an integer greater than or equal to 1. The computing device processes each first image block to obtain J second image blocks, where at least part of the second image blocks marked with cracks corresponds to target pixels. The computing device merges J second image blocks to obtain a third image. The computing device extracts the target pixels in the third image, resulting in cracks. By dividing the second image into blocks and processing the image blocks separately, the influence of the external environment on the extraction of cracks in the target object can be reduced, and the accuracy of crack detection can be improved.

In combination with the fifth implementation of the first aspect, in the sixth implementation of the first aspect of the present application, the computing device processes each first image block, and obtaining J second image blocks includes: using the computing device to The region growing algorithm classifies the pixels in the first image block to obtain K pixel sets, each pixel set includes at least one pixel, and K is an integer greater than or equal to 1. The computing device respectively acquires the average gray value of the K pixel sets, and the average gray value is an average value of the gray values of the pixels in the pixel set. The computing device determines that the average gray value corresponding to the pixel set with the largest number of pixels among the K pixel sets is the gray threshold. The calculation device sets the gray value of the pixels in the pixel set whose average gray value is less than the gray threshold value as the first gray value, and sets the gray value of the pixels in the pixel set whose average gray value is greater than or equal to the gray threshold value Set as the second gray value to obtain the second image block. Wherein, the first grayscale value may be 0, and the second grayscale value may be 255. Alternatively, the first grayscale value is 255, and the second grayscale value is 0. The pixels corresponding to the cracks in the image block are screened out, and then their gray value is assigned as the first gray value, thereby completing the binarization of the image block, that is, marking the pixels corresponding to the cracks.

In combination with the sixth implementation of the first aspect, in the seventh implementation of the first aspect of the present application, the computing device uses a region growing algorithm to classify the pixels in the image block to obtain K pixel sets including: computing device Convert image patches to 3D point clouds. The computing device obtains the neighborhood radius of each 3D point in the 3D point cloud. The neighborhood radius is the average of the Euclidean distances between the target 3D point in the 3D point cloud and the L neighborhood 3D points. The L neighborhood 3D points is a 3D point corresponding to L pixels adjacent to the pixel corresponding to the target 3D point, where L is an integer greater than or equal to 4. The computing device acquires a search radius of the three-dimensional point cloud, where the search radius is an average value of neighborhood radii of all three-dimensional points in the three-dimensional point cloud. The computing device classifies the three-dimensional points in the three-dimensional point cloud according to the search radius, and obtains K pixel sets corresponding to the K three-dimensional point sets. Each image block obtains a corresponding search radius according to the characteristics of its own 3D point cloud, so that the 3D points in the image block can be classified more accurately, and the 3D points corresponding to the cracks in the image block can be segmented out.

In combination with the seventh implementation of the first aspect, in the eighth implementation of the first aspect of the present application, the three-dimensional points in the three-dimensional point cloud are classified according to the search radius, and K corresponding to the K three-dimensional point sets are obtained. The pixel collection includes: determining an unclassified three-dimensional point in the three-dimensional point cloud as a seed point. Taking the seed point as the center, classify the 3D points located within the search radius of the seed point into the target 3D point set where the seed point is located. Determine a 3D point that is not used as a seed point in the target 3D point set as the seed point, and return to the step of dividing the 3D points within the search radius of the seed point into the target 3D point set corresponding to the seed point with the seed point as the center . Until there is no 3D point in the target 3D point set that is not used as a seed point, return to the step of determining an unclassified 3D point in the 3D point cloud as a seed point.

The second aspect provides a crack detection method. The method is applied to a camera, and the method includes: the camera collects N frames of first images. N is an integer greater than or equal to 2. The N frames of first images are obtained by the camera shooting different areas of the target object through preset operations. Wherein, the target object is, for example, a road, a wall, and the like. The default operation is to rotate the camera lens and adjust the focal length of the lens. Different regions of the target object have different distances from the camera, and the distance from the region to the camera is proportional to the focal length of the lens. Specifically, the area of the target object is selected by rotating the lens of the camera, and the focal length of the lens of the camera is changed: for the area farther from the camera, the focal length of the lens is larger; for the area closer to the camera, the focal length of the lens is smaller . Therefore, target objects in different first images can be clearly presented. The camera stitches parts of each of the N frames of first images to obtain a second image. The camera detects cracks in the target object based on the second image. Therefore, manual detection is not required, manpower and material resources can be saved, and the cost of crack detection can be reduced.

With reference to the second aspect, in the first implementation manner of the second aspect of the present application, the second image is an image without a perspective effect. Due to the perspective phenomenon of the camera lens, the target object captured by the camera has the effect of being far smaller and closer, resulting in the loss of details of the distant target object, and the crack cannot be accurately detected. The first image of N frames is stitched to obtain the first image without perspective effect. With the second image, the details of distant target objects can also be clearly presented, and crack detection based on the second image will also be more accurate.

In combination with the second aspect or the first implementation of the second aspect, in the second implementation of the second aspect of the present application, the camera stitches parts of each of the N frames of first images to obtain the second image including : The camera intercepts M rows of pixels of the first image of N frames respectively, and M is an integer greater than or equal to 1. The camera stitches the M rows of pixels of the N frames of the first image to obtain the second image. Therefore, the second image can be quickly spliced.

In combination with the second aspect or the first implementation of the second aspect, in the third implementation of the second aspect of the present application, the camera stitches parts of each of the N frames of first images to obtain the second image including : The camera intercepts some pixels of each image in the first N frames of images respectively, where the number of pixels intercepted by each image is different. The camera splices some pixels of each of the N frames of first images to obtain a second image. Partial pixels of each first image correspond to a sub-region of the target object, and the second image includes a complete target object composed of N sub-regions. Since the camera poses and focal lengths corresponding to each first image are different, the number of pixels intercepted in each first image may be different.

In combination with the second aspect or the first implementation manner of the second aspect, in the fourth implementation manner of the second aspect of the present application, the camera stitches parts of each image in the N frames of first images to obtain the second image including : The camera intercepts P pixels of the first image of N frames respectively, and P is an integer greater than or equal to 1. The camera stitches P pixels of the N frames of the first image to obtain the second image. By intercepting the same number of pixels from each first image, the second image can be quickly spliced.

In combination with the second aspect or any one of the first to fourth implementations of the second aspect, in the fifth implementation of the second aspect of the present application, the camera detecting cracks of the target object based on the second image includes: the camera detects the first The second image is preprocessed to obtain J first image blocks. The part of the target object included in the first image block, J is an integer greater than or equal to 1. Wherein, the preprocessing may be block, and may also be filtering and block. The camera processes each first image block to obtain J second image blocks, where target pixels corresponding to sub-cracks are marked. A sub-crack is a part of a crack. The camera combines J second image blocks to obtain a third image. The camera extracts the target pixels in the third image, resulting in cracks. By dividing the second image into blocks and processing the image blocks separately, the influence of the external environment on the extraction of cracks in the target object can be reduced, and the accuracy of crack detection can be improved.

In combination with the fifth implementation of the second aspect, in the sixth implementation of the first aspect of the present application, the target pixel is a pixel with the first gray value, and the camera performs The processing to obtain the J second image blocks includes: the camera uses a region growing algorithm to classify the pixels in the first image block to obtain K pixel sets. Wherein, each pixel set includes at least one pixel, and K is an integer greater than or equal to 1. The camera obtains the average gray value of the K pixel sets respectively, and the average gray value is the average value of the gray values of the pixels in the pixel set. The camera determines that the average gray value corresponding to the pixel set with the largest number of pixels among the K pixel sets is the gray threshold. The camera sets the gray value of the pixels in the pixel set whose average gray value is less than the gray threshold as the first gray value, and sets the gray value of the pixels in the pixel set whose average gray value is greater than or equal to the gray threshold as the first gray value. is the second gray value to obtain the second image block. Wherein, the first grayscale value may be 0, and the second grayscale value may be 255. Alternatively, the first grayscale value is 255, and the second grayscale value is 0. The pixels corresponding to the cracks in the image block are screened out, and then their gray value is assigned as the first gray value, thereby completing the binarization of the image block, that is, marking the pixels corresponding to the cracks.

In combination with the sixth implementation of the second aspect, in the seventh implementation of the second aspect of the present application, the camera uses the region growing algorithm to classify the pixels in the image block, and the K pixel sets obtained include: the camera divides the image Blocks are converted to 3D point clouds. The camera obtains the neighborhood radius of each 3D point in the 3D point cloud. The neighborhood radius is the average value of the Euclidean distance between the target 3D point in the 3D point cloud and the L neighborhood 3D points, and the L neighborhood 3D points are the 3D distances corresponding to the L pixels adjacent to the pixel corresponding to the target 3D point. point. L is an integer greater than or equal to 4. The camera obtains the search radius of the 3D point cloud, and the search radius is the average value of the neighborhood radii of all 3D points in the 3D point cloud. The camera classifies the 3D points in the 3D point cloud according to the search radius, and obtains K pixel sets corresponding to the K 3D point sets. Each image block obtains a corresponding search radius according to the characteristics of its own 3D point cloud, so that the 3D points in the image block can be classified more accurately, and the 3D points corresponding to the cracks in the image block can be segmented out.

In combination with the seventh implementation of the second aspect, in the eighth implementation of the second aspect of the present application, the three-dimensional points in the three-dimensional point cloud are classified according to the search radius, and K corresponding to the K three-dimensional point sets are obtained. The pixel collection includes: determining an unclassified three-dimensional point in the three-dimensional point cloud as a seed point. Taking the seed point as the center, classify the 3D points located within the search radius of the seed point into the target 3D point set where the seed point is located. Determine a 3D point that is not used as a seed point in the target 3D point set as the seed point, and return to the step of dividing the 3D points within the search radius of the seed point into the target 3D point set corresponding to the seed point with the seed point as the center . Until there is no 3D point in the target 3D point set that is not used as a seed point, return to the step of determining an unclassified 3D point in the 3D point cloud as a seed point.

A third aspect provides a crack detection device. The device includes a collection module, a splicing module and a detection module. Wherein, the collection module is used to collect N frames of first images. The N frames of first images are obtained by the camera shooting different areas of the target object through preset operations. The default operation is that the camera rotates the camera lens and adjusts the focal length of the lens, and different areas have different distances from the camera. N is an integer greater than or equal to 2. The splicing module is configured to splice parts of each image in the N frames of first images to obtain a second image. A detection module, configured to detect cracks of the target object based on the second image.

With reference to the third aspect, in the first implementation manner of the third aspect of the present application, the second image is an image without a perspective effect. Due to the perspective phenomenon of the camera lens, the target object captured by the camera has the effect of being far smaller and closer, resulting in the loss of details of the distant target object, and the crack cannot be accurately detected. The first image of N frames is stitched to obtain the first image without perspective effect. With the second image, the details of distant target objects can also be clearly presented, and crack detection based on the second image will also be more accurate.

In combination with the third aspect or the first implementation manner of the third aspect, in the second implementation manner of the third aspect of the present application, the splicing module is specifically used to: respectively intercept M rows of pixels of the N frames of the first image, and M is An integer greater than or equal to 1. The second image is obtained by splicing the M rows of pixels of the N frames of the first image.

In combination with the third aspect or the first implementation manner of the third aspect, in the third implementation manner of the third aspect of the present application, the splicing module is specifically configured to: respectively intercept some pixels of each image in the N frames of the first image , where the number of pixels intercepted by each image is different. Partial pixels of each image in the N frames of first images are spliced to obtain a second image.

In combination with the third aspect or the first implementation of the first aspect, in the fourth implementation of the third aspect of the present application, the splicing module is specifically used to: respectively intercept P pixels of the N frames of the first image, and P is An integer greater than or equal to 1. The P pixels of the N frames of the first image are spliced to obtain the second image.

In combination with the third aspect or any one of the first to fourth implementations of the third aspect, in the fifth implementation of the third aspect of the present application, the detection module is specifically configured to: preprocess the second image to obtain J first image blocks, where the first image block includes a part of the target object, and J is an integer greater than or equal to 1. Each first image block is processed to obtain J second image blocks, where at least part of the second image blocks marked with cracks correspond to target pixels. Merge J second image blocks to obtain a third image. Extract the target pixel in the third image to get the crack.

In combination with the fifth implementation of the third aspect, in the sixth implementation of the third aspect of the present application, the detection module is further configured to: use the region growing algorithm to classify the pixels in the first image block to obtain K pixel sets, each pixel set includes at least one pixel, and K is an integer greater than or equal to 1. The average gray value of the K pixel sets is obtained respectively, and the average gray value is an average value of the gray values of the pixels in the pixel set. Determine the average gray value corresponding to the pixel set with the largest number of pixels among the K pixel sets as the gray threshold. Set the gray value of the pixels in the pixel set whose average gray value is less than the gray threshold to the first gray value, and set the gray value of the pixels in the pixel set whose average gray value is greater than or equal to the gray threshold to be The second grayscale value is used to obtain the second image block.

With reference to the sixth implementation manner of the third aspect, in the seventh implementation manner of the third aspect of the present application, the detection module is further configured to: convert the image block into a three-dimensional point cloud. Obtain the neighborhood radius of each 3D point in the 3D point cloud. The neighborhood radius is the average of the Euclidean distances between the target 3D point in the 3D point cloud and the L neighborhood 3D points. The L neighborhood 3D points are the target The pixel corresponding to the 3D point corresponds to the 3D point corresponding to L pixels adjacent to each other, where L is an integer greater than or equal to 4. Get the search radius of the 3D point cloud, the search radius is the average of the neighborhood radii of all 3D points in the 3D point cloud. The 3D points in the 3D point cloud are classified according to the search radius, and K pixel sets corresponding to the K 3D point sets are obtained.

With reference to the seventh implementation manner of the third aspect, in the eighth implementation manner of the third aspect of the present application, the detection module is further configured to: determine an unclassified 3D point in the 3D point cloud as a seed point. Taking the seed point as the center, classify the 3D points located within the search radius of the seed point into the target 3D point set where the seed point is located. Determine a 3D point that is not used as a seed point in the target 3D point set as the seed point, and return to the step of dividing the 3D points within the search radius of the seed point into the target 3D point set corresponding to the seed point with the seed point as the center . Until there is no 3D point in the target 3D point set that is not used as a seed point, return to the step of determining an unclassified 3D point in the 3D point cloud as a seed point.

A fourth aspect provides a crack detection device. The crack detection device includes a processor and a memory, the processor is coupled to the memory, and the processor is configured to execute the crack detection method in the above first aspect or any implementation manner thereof based on instructions stored in the memory.

A fifth aspect provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and the computer program is executed by a processor to realize the crack detection method of the first aspect or any implementation thereof, or realize the crack of the second aspect or any implementation thereof Detection method.

Description of drawings

FIG. 1 is a schematic flow chart of the first embodiment of the crack detection method provided by the present application;

Fig. 2 is the schematic diagram that the camera provided by the present application shoots the target object;

Fig. 3 is a schematic diagram of the relationship between two adjacent frames of first images;

FIG. 4 is a schematic diagram of a second image provided by the present application;

Fig. 5 is a schematic flow chart of the second embodiment of the crack detection method provided by the present application;

FIG. 6 is a schematic structural view of an embodiment of a crack detection device provided by the present application;

FIG. 7 is a schematic structural diagram of an embodiment of a crack detection system provided by the present application;

FIG. 8 is a schematic structural diagram of an embodiment of a crack detection device provided in the present application.

Detailed ways

Embodiments of the present application provide a crack detection method, device and equipment, so as to reduce the cost of crack detection.

Embodiments of the present invention will be described below with reference to the drawings in the embodiments of the present invention. The terms used in the embodiments of the present invention are only used to explain specific examples of the present invention, and are not intended to limit the present invention. Those of ordinary skill in the art know that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

The terms "first", "second", "third", "fourth", etc. (if any) in the description and claims of the present invention and the above drawings are used to distinguish similar objects, and not necessarily Used to describe a specific sequence or sequence. It is to be understood that the data so used are interchangeable under appropriate circumstances such that the embodiments of the invention described herein can be practiced in sequences other than those illustrated or described herein. Furthermore, the terms "comprising" and "having", as well as any variations thereof, are intended to cover a non-exclusive inclusion, for example, a process, method, system, product or device comprising a sequence of steps or elements is not necessarily limited to the expressly listed instead, may include other steps or elements not explicitly listed or inherent to the process, method, product or apparatus.

Buildings such as roads, bridges, tunnels and walls are affected by factors such as improper construction technology, overloading and use, temperature changes, foundation settlement or material aging, which will cause cracks to form on the surface of the building. Cracks in building objects will reduce the bearing capacity of building objects and accelerate damage, thus affecting the comfort and safety of building objects. Therefore, cracks on building objects need to be detected and checked in time.

In view of the fact that the crack detection methods of building objects rely on manual work to a large extent, the efficiency is low and the cost is high. The present application provides the following embodiments to complete crack detection of building objects with low cost and high efficiency.

The embodiment of the crack detection method of the present application can be completed independently by the camera, or can be completed by the cooperation of the camera and the computing device. The two cases are described respectively below.

As shown in FIG. 1 , FIG. 1 is a schematic flowchart of a first embodiment of the crack detection method provided in the present application. The execution subject of this embodiment includes a camera and a computing device. The camera is connected to the computing device. The camera is responsible for collecting N frames of first images and sending the N frames of first images to the computing device. After receiving the N frames of first images, the computing device detects cracks of the target object based on the N frames of first images. The computing device is, for example, a device with computing power such as a cloud server, a computer, or a smart terminal.

Specifically, this embodiment includes the following steps:

S101: The camera collects N frames of the first image. The N frames of the first image are obtained by the camera shooting different areas of the target object through a preset operation. The preset operation is that the camera rotates the camera lens and adjusts the focal length of the lens. Different areas and The distance of the camera is different.

In this application, the camera is a pan-tilt camera with a pan-tilt control function and a zoom function, such as a dome camera, specifically a pan/tilt/zoom (PTZ) camera with omnidirectional movement, lens zoom, and zoom control. Of course, the camera can also be a bullet camera or a barrel camera, as long as it has the above-mentioned pan/tilt control and zoom functions, and the specific form of the camera is not limited in this application.

The camera includes a pan-tilt and a lens. The pan-tilt can rotate up and down and/or left and right, and the rotation of the pan-tilt can drive the lens to rotate. By rotating the pan-tilt, the camera rotates the lens of the camera so as to be able to shoot different areas of the target object.

Wherein, the target object may be a building object such as a road, a bridge, a wall or a tunnel within the visible range of the camera, or a part of the building object within the visible range. For example, as shown in FIG. 2 , FIG. 2 is a schematic diagram of a scene where a camera provided in the present application shoots a target object. In Fig. 2, the target object is a road as an example, and the camera is generally installed above the road by means of suspension or the like. Due to the limited focal length range of the camera lens and the perspective of the lens, the image captured by the camera may have problems such as unclearness and lack of detail in the area beyond the distance corresponding to the maximum focal length of the camera lens on the road, so the target object is That part of the road that can be clearly captured by the camera.

Before the camera shoots the target object, it needs to aim at at least a partial area of the target object. The camera can automatically identify the target object and then aim at the target object. For example, the camera recognizes the target object through the trained neural network, and determines the relative position of the target object relative to the camera, and then the camera rotates the gimbal so that the lens is aimed at the target object. Of course, the target object is manually selected, and the camera is controlled to rotate the gimbal so that the lens is aimed at the target object.

Different regions of the target object have different distances from the camera. It can be understood that there may be overlapping parts between different regions. The distance between the area of the target object and the camera may be the distance from the intersection point of the optical axis of the camera lens and the target object to the camera. In this embodiment, the camera shoots different areas of the target object with different focal lengths to obtain N frames of first images of the target object, and the distance between different areas and the camera is proportional to the focal length of the camera lens. That is, the farther the target object is from the camera, the larger the focal length is; the closer the target object is to the camera, the smaller the focal length is. Therefore, the region of the target object that is far away from the camera can also be presented clearly and in detail in the first image.

In this embodiment, the camera may shoot the target object in a linear scanning manner. Specifically, the camera rotates the lens along a fixed direction, so that the lens shoots different areas of the target object from far to near or from near to far, and adjusts the focal length while the lens is rotating or after rotating, based on different camera poses and N frames of first images are obtained by focal length shooting. The camera can be rotated in only one direction, such as only up or down, or only left or right, depending on the position and extension of the target object relative to the camera. For example, as shown in Figure 2, take the camera shooting the target object from near to far as an example, the road is located in front of the camera, and the camera changes the shooting angle of the lens by rotating the lens upward (that is, increasing the pitch angle of the lens) , so that the shooting range of the lens falls on the farther area of the target object, and at the same time, the lens increases the focal length, and then shoots until the shooting of the road is completed.

For the same target object, the larger the range of each rotation of the camera is, the smaller the number N of first images collected is, and the more pixels are used for splicing the second image in each subsequent first image, the more each first image The larger the area of the target object corresponding to the pixels used to stitch the second image, the greater the impact of the perspective effect; on the contrary, the smaller the rotation of the camera each time, the number N of the first images collected is The larger the , the fewer pixels are used to stitch the second image in each subsequent first image, and the smaller the area of the target object corresponding to the pixels used to stitch the second image in each first image, and it is also affected by the perspective effect. smaller. Therefore, in this embodiment, each rotation of the camera may be rotated in a small range, and correspondingly, the focal length may also be adjusted in a small range, so that the target object can be more accurately recorded in the N frames of the first image. specifically,

The first images of two adjacent frames refer to the first image a x captured based on a certain pose p _x of the camera and the focal length f _x when the camera rotates the lens in a fixed direction to capture the first _image , which is directly changed to the pose p _x The first image a _{x -1 obtained by taking the previous pose p x- 1} with the focal length f x and the previous focal length f _x-1 , or directly changing to the next pose based on the pose p _x and the focal length f _x The first image a x+1 obtained by shooting p _x+1 and the next focal length f _{x+ 1} . The first image a _x-1 is the previous frame image of the first image a _x , the first image a _x is the previous frame image of the first image a _x+1 , and the first image a _x+1 is the first image a The next frame image of _x , the first image a _x and the first image a _x-1 are two adjacent frames of the first image, the first image a _x and the first image a _x+1 are the two adjacent frames of the first image .

Due to the distortion of the camera lens, the closer to the edge of the first image, the more obvious the distortion, and the center of the first image is least affected by the lens distortion. Optionally, in order to further reduce the influence of lens distortion, each time the camera rotates, the area corresponding to the middle M rows/columns of the first image of the previous frame captured by the camera can be compared with the area corresponding to the pixel phase of the first image of the next frame. The area corresponding to the pixels in the middle M rows/columns is an adjacent area. Therefore, the influence of the edge distortion of the first image caused by the camera lens distortion is reduced, so as to ensure the integrity and accuracy of the subsequent second image stitched based on the N frames of the first images.

For example, as shown in FIG. 3 , FIG. 3 is a schematic diagram of the relationship between two adjacent frames of first images, and M=1 is taken as an example for illustration. The camera shoots the first area based on the first pitch angle and the first focal length to obtain the kth frame of the first image, then the camera rotates to the second pitch angle, and the focal length is adjusted to the second focal length (the second focal length is greater than the first focal length), based on the first After adjusting the second pitch angle and the second focal length, the k+1th frame of the first image is obtained. The first sub-area and the second sub-area are two adjacent sub-areas on the road. In the k-th frame of the first image, the first sub-area of the road is located in the middle row of the k-th frame of the first image, and the second sub-area is located in the upper row of the k-th frame of the first image. In the first image of the k+1th frame, the second subregion is located in the middle row of the first image in the k+1th frame, and the second subregion is located in the next row of the middle row of the first image in the k+1th frame.

Of course, each time the camera rotates, it can also be the area corresponding to the M row/column pixels at the bottom/top of the first frame of the previous frame captured by the camera, and the bottom/top of the first image of the next frame. The area corresponding to the M rows/columns of pixels is an adjacent area, which is not limited in the present application.

Wherein, M may be a positive integer such as 1, 3, 5, 8 or 10. The value of M can be determined according to the actual situation. For example, when the target object is longer or the detection efficiency is higher, the value of M can be appropriately increased, which is not limited in this application.

The N frames of first images may be images in the world coordinate system. Specifically, before the camera shoots the target object, the camera has been calibrated, that is, the internal parameter matrix and the external parameter matrix of the camera are known. When the camera captures each first image, it records the rotation amount of the lens that captures the corresponding first image relative to the previous posture and the focal length of the lens. The rotation amount is substituted into the external parameter matrix as an external parameter, and the focal length is substituted into the internal parameter matrix as an internal parameter, so that the first image in the world coordinate system can be obtained. Unify the photos taken by the camera based on different postures and focal lengths into one absolute coordinate system (ie, the world coordinate system), so as to facilitate subsequent processing based on the first image of N frames. The first image may also be a distortion-corrected image. Specifically, in the process of camera calibration, distortion parameters can also be obtained, and the distortion parameters can be substituted into the distortion correction formula, and the original image collected by the camera can be calculated using the distortion correction formula to output the first distortion-corrected image.

Generally, public places such as traffic roads and building clusters have been covered with such spherical cameras. This application uses these deployed spherical cameras to collect roads, walls and other building objects to reduce the material cost of input.

S102: The camera sends N frames of first images to the computing device.

After collecting N frames of first images, the camera sends N frames of first images to the computing device. Or the camera sends the first images to the computing device while collecting N frames of the first image, which can improve the efficiency of the computing device in acquiring the first images and reduce the storage requirements of the camera.

S103: The computing device stitches parts of each of the N frames of first images to obtain a second image.

Since the distortion of the camera lens will cause the edge area of the first image to be distorted, after the camera collects N frames of the first image, the computing device respectively intercepts some pixels of the N frames of the first image, and takes part of the pixels of each image in the N frames of the first image Stitching to obtain the second image. The area of the target object corresponding to the partial pixels intercepted in each first image is also adjacent to the area of the target object corresponding to the partial pixels intercepted in the adjacent first image. Therefore, it can be guaranteed that the target object in the second image obtained by splicing is a complete object.

In some implementations, some of the pixels may be M rows of pixels in the first image. It is sufficient that the M rows of pixels intercepted from each first image can ensure the integrity of the target object in the spliced second image, and there is no restriction on the position of the M rows of pixels from the first image.

In order to further improve the accuracy of the second image and improve the splicing efficiency of the second image, M rows of pixels in the middle of each first image may be intercepted and spliced into a second image. No matter whether the first image has undergone distortion correction or not, the impact of camera distortion on the middle M rows of pixels in the first image is minimal, so the second image obtained by mosaicing the middle M rows of pixels of each first image is more accurate. Moreover, it is more efficient to capture a fixed number of pixels from a fixed position of each first image, and to capture pixels.

In some other implementation manners, the second image may also be obtained by stitching pixels corresponding to the target object of the same area captured in each first image. For example, if the target object is a road with a width of 5 meters and a length of 100 meters, and the number of the first images is 200, then the pixels corresponding to the road of 5 meters*0.5 meters in each first image can be intercepted, and the pixels in the 200 first images can be intercepted. The second image is obtained by splicing pixels corresponding to the 5m*0.5m road. Since each first image is intercepted from each first image because each first image is collected based on different camera poses and focal lengths, even if it is a partial area of the target object with the same area, the number of pixels occupied in different first images may be different or not completely the same, therefore, the number of pixels intercepted from each first image may be different or not completely the same.

In some other implementation manners, the second image may also be obtained by cutting P pixels in each first image and stitching them together. It is enough that the P pixels intercepted from each first image can ensure the integrity of the target object in the spliced second image, and there is no restriction on the position of the P pixels from the first image. P is an integer greater than or equal to 1.

Since the first image is an image in the world coordinate system, the second image stitched based on the first image is also an image in the world coordinate system. Moreover, since different regions of the target object are clearly recorded in at least one frame of the first image as the focal length is adjusted, regardless of the distance from the camera, the second image is a frame of high-definition, no distortion/minimal distortion and Image without perspective.

As shown in FIG. 4, FIG. 4 is a schematic diagram of the second image provided by the present application. In Fig. 4, the target object is a road as an example. It can be seen that N frames of first images are obtained by shooting based on different poses and focal lengths, and the second image obtained by splicing partial pixels of each of the N frames of first images is an image without perspective effect and distortion.

S104: The computing device detects cracks of the target object based on the second image.

After stitching the N frames of first images into a second image with high definition, no/minimal distortion, and no perspective effect, the computing device can accurately detect whether there is a crack in the target object based on the second image.

Specifically, in order to improve the accuracy of crack detection and improve detection efficiency, the computing device may perform preprocessing on the second image.

Generally, there may be green belts or other building objects inside or around building objects such as roads and walls. These green belts and buildings other than the target object will be captured together in the first image, and will also be spliced into the second image. in the second image. If there are objects other than the target object in the image during crack detection, the detection efficiency will be reduced and the crack detection of the target object will be interfered. Therefore, only the area where the target object in the second image is located can be detected, thereby improving crack detection efficiency and detection accuracy. The preprocessing may include extracting the target object in the second image to obtain a target image containing only the target object. The target object in the second image can be extracted by the computing device through the trained neural network to identify and extract the target object, or it can be extracted from the second image based on the artificially selected region of interest (the region corresponding to the target object) . Of course, the preprocessing may also not include the processing of extracting the target object from the second image to obtain the target image. For the convenience of description, the preprocessing includes extracting the target image as an example for description below.

Optionally, due to differences in building materials, illumination changes, and aging degrees, the preprocessing may also include performing noise reduction processing on the target image to reduce the differences in material and illumination, as well as the brightness difference of the target object in the target image caused by wear and aging. error. Specifically, the noise reduction process is, for example, a median filter process, a Gaussian filter process, or the like.

The preprocessing may also include performing block processing on the target image to obtain J first image blocks. J may be an integer greater than or equal to 1. When J is equal to 1, the first image block is the target image. When J is greater than or equal to 2, the target image is divided into a plurality of first image blocks, which can further reduce the influence of uneven brightness of unmarked objects in the target image. The following takes J greater than or equal to 2 as an example for illustration.

Performing block processing on the target image may specifically be that the computing device divides the target image into blocks according to a preset size and the size of the target image. The preset size is, for example, 64 pixels*64 pixels, 128 pixels*64 pixels, 128 pixels*128 pixels, 128 pixels*256 pixels, 256 pixels*256 pixels, or 512 pixels*512 pixels. This application is not limited to this. Taking the preset size as 64 pixels*64 pixels and the target image size as 3200 pixels*6400 pixels as an example for illustration, the target image may be divided into 5000 first image blocks.

The computing device performs processing such as classification and binarization on each image block to obtain J second image blocks. When a sub-crack exists in a certain second image block, the target pixel of the sub-crack in the second image block will be marked. Wherein, since the cracks are relatively slender and the size of the second image block is small, the sub-cracks in the second image block are part of the cracks. The target pixel is, for example, a pixel whose grayscale value is a first grayscale value, and pixels other than the target pixel in the second image block have grayscale values that are a second grayscale value, and the first grayscale value is different from the second grayscale value. For example, the first grayscale value may be 0, and the second grayscale value may be 255. Alternatively, the first grayscale value is 255, and the second grayscale value is 0.

Specifically, the computing device uses a region growing algorithm to classify the pixels in the first image block to obtain K pixel sets. Each pixel set includes at least one pixel, and K is an integer greater than or equal to 1. The region growing algorithm can combine pixels with similar characteristics into a set. For example, when there are sub-cracks in a certain first image block, and there are target object surfaces that are not cracks, then some pixels in the first image block are similar to sub-cracks. Correspondingly, some pixels correspond to the surface of the target object, and the region growing algorithm can roughly classify these two types of pixels into at least two pixel sets, one of which represents sub-cracks, and one pixel set represents the surface of the target object.

The computing device converts the first image patch into a three-dimensional point cloud. The XY axis plane of the coordinate axis of the 3D point cloud is the pixel coordinate of the first image block, and the Z axis is the gray value of the pixel. For the convenience of calculation, the gray value of the pixel is normalized to [0,1], which is used as the Z-axis coordinate of the pixel. The computing device traverses the three-dimensional point cloud to obtain the neighborhood radius of each three-dimensional point in the three-dimensional point cloud. Among them, the neighborhood radius is the average value of the Euclidean distance between the target 3D point in the 3D point cloud and the L neighborhood 3D points, and the L neighborhood 3D points are the L pixels adjacent to the pixel corresponding to the target 3D point. 3D point. L is an integer greater than or equal to 4, specifically 4, 8, 10 or 12, etc. The computing device calculates the search radius of the three-dimensional point cloud according to the radius of the area of each three-position point, and the search radius is an average value of the neighborhood radii of all three-dimensional points in the three-dimensional point cloud. Each first image block determines its search radius according to its own characteristics, so that the three-dimensional points can be classified more accurately when the search radius is used to classify the three-dimensional point cloud.

The computing device classifies the three-dimensional points in the three-dimensional point cloud according to the search radius, and obtains K pixel sets corresponding to the K three-dimensional point sets. Specifically, an unclassified 3D point in the 3D point cloud is determined as a seed point. Taking the seed point as the center, classify the 3D points located within the search radius of the seed point into the target 3D point set where the seed point is located. Determine a 3D point that is not used as a seed point in the target 3D point set as a seed point, and use the seed point as the center to divide the 3D points within the search radius of the seed point into the target 3D point set corresponding to the seed point. Until there is no 3D point in the target 3D point set that is not used as a seed point, then a 3D point that has not been classified in the 3D point cloud is re-determined as a seed point, and the seed point and the 3D point grown based on the seed point are classified into a new target 3D point collection. The classification of the 3D point cloud is completed until there is no unclassified 3D point in the 3D point cloud.

After the K pixel sets are obtained by classification, the computing device obtains the average gray value of the K pixel sets respectively, and the average gray value is the average value of the gray values of the pixels in the pixel set. The computing device determines that the average gray value corresponding to the pixel set with the largest number of pixels among the K pixel sets is the gray threshold. Generally, cracks appearing on the surface of building objects only occupy a very small part of the entire building body surface, and the non-cracked surface area of building objects is much larger than the area of cracks. Then, in the first image block, the larger the area, the more the number of pixels, and the number of pixels corresponding to the surface of the target object is greater than the number of pixels corresponding to the crack. The pixel set with the largest number of pixels among the K pixel sets may be determined to be the set of pixels corresponding to the surface of the target object. Because the depth of the crack is deeper than the surface of the target object, and the shape is thin and narrow, less light enters it, so the crack is darker than the surface of the target object. Then in the first image block, the gray value of the pixel corresponding to the sub-crack is lower than the gray value of the pixel corresponding to the surface of the target object. According to this feature, the pixels corresponding to the cracks in the first image block can be distinguished from the pixels corresponding to the surface of the target object. Specifically, the computing device sets the grayscale value of the pixels in the pixel set whose average grayscale value is smaller than the grayscale threshold value as the first grayscale value, and sets the grayscale value of the pixels in the pixel set whose average grayscale value is greater than or equal to the grayscale threshold value The gray value is set as the second gray value to obtain a binarized second image block.

After obtaining the J second image blocks, the computing device merges the J second image blocks to obtain a third image. The third image is the target image after binarization processing. The computing device extracts target pixels in the third image to extract cracks in the third image.

In order to enable the sub-cracks in the third image block to be merged into continuous cracks, a closing operation may be performed on the third image before extracting the cracks in the third image. The closing operation can fuse the target pixels with subtle connections in the third image block, that is, convert some non-target pixels into target pixels, so that the sub-cracks distributed in different second image blocks can be connected to form a complete crack profile .

Optionally, in order to reduce the false detection rate of cracks, after extracting the cracks, the computing device further calculates the length of the cracks in the third image, and the unit is pixel. Then compare the cracks with the length threshold, the cracks below the length threshold are determined to be false detections and not reported, and the cracks greater than or equal to the length threshold are determined to be true cracks.

Optionally, after detecting the crack in the third image, the geographic location information of the crack in the real world may also be acquired. Specifically, the computing device acquires a reference image carrying geographic location information, the reference image is also an image in a world coordinate system, and pixels in the reference image have a mapping relationship with geographic location information in the real world. The reference image is, for example, a high-precision city map in a digital twin city system or a smart city system. Among them, the digital twin city refers to the construction of a digital twin city that matches the physical world, and reproduces the architectural geographical structure of the city as it is, including geographical data such as the height and coordinates of important facilities such as roads and buildings. Since the third image is obtained through binarization conversion based on the second image, the third image is also an image in the world coordinate system, and there is a one-to-one correspondence between the pixels in the third image and the pixels in the reference image. The reference image is used to obtain the target geographic location information of the target pixel to obtain the geographic location information of the crack. In this way, the staff can accurately know the location of the crack, and then can quickly confirm and carry out repair and maintenance.

In this embodiment, by using the cameras that have been deployed around the building to take pictures of the building and detect the cracks of the building based on the pictures, no manual inspection or additional material resources are required, and the cost of crack detection can be reduced. Moreover, the camera can be called at any time to shoot the target object, and the computing device can also perform detection at any time, so that the current status of the building object can be obtained in time, and cracks can also be found in time. Multiple cameras can be called at the same time to collect first images of cracks in multiple building objects, and then the computing device can perform splicing detection based on the first images, thereby improving detection efficiency.

Computing equipment to complete image stitching and crack detection of target objects can reduce the computing power requirements for cameras, so that more cameras can be used to measure cracks in building objects, reducing the cost of crack detection.

In the above embodiments, the camera and the computing device cooperate to complete the detection of cracks. In some other implementation manners, the camera can also independently complete the detection of cracks.

As shown in FIG. 5 , FIG. 5 is a schematic flowchart of a second embodiment of the crack detection method provided by the present application. This embodiment includes the following steps:

S501: The camera collects N frames of the first image. The N frames of the first image are obtained by the camera shooting different areas of the target object through a preset operation. The preset operation is that the camera rotates the camera lens and adjusts the focal length of the lens. Different areas and The distance of the camera is different.

This step is similar to S101, so it will not be repeated here.

S502: The camera stitches some pixels of each of the N frames of first images to obtain a second image.

This step is similar to S102, the difference is that the execution subject is a camera, but the specific splicing method is the same, so it will not be repeated here.

S503: The camera detects cracks of the target object based on the second image.

This step is similar to S103, except that the execution subject is a camera, but the specific detection method is the same, so it will not be repeated here.

The above first embodiment of the crack detection method is implemented by a crack detection device. As shown in FIG. 6 , FIG. 6 is a schematic structural diagram of an embodiment of a crack detection device provided in the present application. The crack detection device 600 includes a collection module 601 , a splicing module 602 and a detection module 603 .

The acquisition module 601 is configured to acquire N frames of the first image, and the N frames of the first image are obtained by the camera shooting different regions of the target object through a preset operation. The preset operation is to rotate the camera lens of the camera and adjust the focal length of the lens, Different regions have different distances from the camera, and N is an integer greater than or equal to 2.

A splicing module 602, configured to splice parts of each image in the N frames of first images to obtain a second image.

A detection module 603, configured to detect cracks of the target object based on the second image.

In some possible implementation manners, the second image is an image without a perspective effect. Due to the perspective phenomenon of the camera lens, the target object captured by the camera has the effect of being far smaller and closer, resulting in the loss of details of the distant target object, and the crack cannot be accurately detected. The first image of N frames is stitched to obtain the first image without perspective effect. With the second image, the details of distant target objects can also be clearly presented, and crack detection based on the second image will also be more accurate.

In some possible implementation manners, the splicing module 602 is specifically configured to: respectively intercept M rows of pixels of the N frames of the first image, where M is an integer greater than or equal to 1. The second image is obtained by splicing the M rows of pixels of the N frames of the first image.

In some possible implementation manners, the splicing module 602 is specifically configured to: respectively capture partial pixels of each image in the N frames of first images, wherein the number of pixels to be captured in each image is different. Partial pixels of each image in the N frames of first images are spliced to obtain a second image.

In some possible implementation manners, the splicing module 602 is specifically configured to: respectively intercept P pixels of the N frames of the first image, where P is an integer greater than or equal to 1. The P pixels of the N frames of the first image are spliced to obtain the second image.

In some possible implementation manners, the detection module 603 is specifically configured to: perform preprocessing on the second image to obtain J first image blocks. The part of the target object included in the first image block, J is an integer greater than or equal to 1. Wherein, the preprocessing may be block, and may also be filtering and block. The detection module 603 processes each first image block to obtain J second image blocks, in which target pixels corresponding to sub-cracks are marked. A sub-crack is a part of a crack. The camera combines J second image blocks to obtain a third image. The detection module 603 extracts target pixels in the third image to obtain cracks. By dividing the second image into blocks and processing the image blocks separately, the influence of the external environment on the extraction of cracks in the target object can be reduced, and the accuracy of crack detection can be improved.

In a possible implementation, the target pixel is a pixel with a gray value of the first gray value, and the detection module 603 is specifically configured to classify the pixels in the first image block using the region growing algorithm to obtain K pixel sets . Wherein, each pixel set includes at least one pixel, and K is an integer greater than or equal to 1. The detection module 603 respectively obtains the average gray value of the K pixel sets, and the average gray value is the average value of the gray values of the pixels in the pixel set. The detection module 603 determines that the average gray value corresponding to the pixel set with the largest number of pixels among the K pixel sets is the gray threshold. The detection module 603 sets the grayscale value of the pixels in the pixel set whose average grayscale value is less than the grayscale threshold value as the first grayscale value, and sets the grayscale value of the pixels in the pixel set whose average grayscale value is greater than or equal to the grayscale threshold value The value is set to the second grayscale value to obtain the second image block. Wherein, the first grayscale value may be 0, and the second grayscale value may be 266. Alternatively, the first grayscale value is 266, and the second grayscale value is 0. The pixels corresponding to the cracks in the image block are screened out, and then their gray value is assigned as the first gray value, thereby completing the binarization of the image block, that is, marking the pixels corresponding to the cracks.

In a possible implementation manner, the detection module 603 is further configured to convert the image block into a three-dimensional point cloud. The detection module 603 acquires the neighborhood radius of each 3D point in the 3D point cloud. The neighborhood radius is the average value of the Euclidean distance between the target 3D point in the 3D point cloud and the L neighborhood 3D points, and the L neighborhood 3D points are the 3D distances corresponding to the L pixels adjacent to the pixel corresponding to the target 3D point. point. L is an integer greater than or equal to 4. The detection module 603 acquires the search radius of the 3D point cloud, and the search radius is the average value of the neighborhood radii of all 3D points in the 3D point cloud. The detection module 603 classifies the 3D points in the 3D point cloud according to the search radius, and obtains K pixel sets corresponding to the K 3D point sets. Each image block obtains a corresponding search radius according to the characteristics of its own 3D point cloud, so that the 3D points in the image block can be classified more accurately, and the 3D points corresponding to the cracks in the image block can be segmented out.

In a possible implementation manner, the detection module 603 is specifically further configured to determine an unclassified three-dimensional point in the three-dimensional point cloud as a seed point. The detection module 603 takes the seed point as the center, and classifies the 3D points located within the search radius of the seed point into the target 3D point set where the seed point is located. The detection module 603 determines that a 3D point that is not used as a seed point in the set of target 3D points is the seed point, and returns to the center of the seed point, and the detection module 603 divides the 3D point located within the search radius of the seed point into the target corresponding to the seed point Steps for a collection of 3D points. Until there is no 3D point in the target 3D point set that is not used as a seed point, the detection module 603 returns to the step of determining an unclassified 3D point in the 3D point cloud as a seed point. Up to 3D points in the 3D point cloud are classified.

In a possible implementation manner, the pixels in the third image have a mapping relationship with the pixels in the reference image, and the pixels in the reference image carry geographic location information. The crack detection device 600 also includes an acquisition module 604 . The obtaining module 604 obtains the target geographic location information of the target pixel according to the reference image, so as to obtain the geographic location information of the fracture. After the crack is detected, the geographic location information of the crack is further obtained to locate the crack.

The crack detection device 600 may be a camera, specifically a camera with a pan/tilt and a zoom lens, such as a spherical camera. Alternatively, the acquisition module 601 belongs to the camera, and the stitching module 602 and the detection module 603 belong to the computing device.

The above second embodiment of the crack detection method is implemented by a crack detection system. As shown in FIG. 7 , FIG. 7 is a schematic structural diagram of an embodiment of a crack detection system provided by the present application. The crack detection system 700 includes a camera 701 and a computing device 702, and the camera 701 and the computing device 702 are connected.

The camera 701 is used to collect N frames of the first image. The N frames of the first image are obtained by the camera shooting different areas of the target object through a preset operation. The preset operation is that the camera rotates the camera lens and adjusts the focal length of the lens. The distance between the area and the camera is different, and N is an integer greater than or equal to 2.

The camera 701 is further configured to send N frames of the first image to the computing device 702 .

The computing device 702 is configured to stitch parts of each of the N frames of first images to obtain a second image.

The computing device 702 is further configured to detect cracks of the target object based on the second image.

In some possible implementation manners, the computing device 702 is specifically configured to: respectively intercept M rows of pixels of the N frames of the first image, where M is an integer greater than or equal to 1. M rows of pixels of N frames of the first image are spliced into a second image.

In some possible implementation manners, the computing device 702 is specifically configured to: perform preprocessing on the second image to obtain J first image blocks. The part of the target object included in the first image block, J is an integer greater than or equal to 1. Wherein, the preprocessing may be block, and may also be filtering and block. The computing device 702 processes each first image block to obtain J second image blocks, where target pixels corresponding to sub-cracks are marked. A sub-crack is a part of a crack. The camera combines J second image blocks to obtain a third image. The computing device 702 extracts the target pixels in the third image to obtain cracks. By dividing the second image into blocks and processing the image blocks separately, the influence of the external environment on the extraction of cracks in the target object can be reduced, and the accuracy of crack detection can be improved.

In a possible implementation, the target pixel is a pixel with a gray value of the first gray value, and the computing device 702 is specifically configured to classify the pixels in the first image block using a region growing algorithm to obtain K pixel sets . Wherein, each pixel set includes at least one pixel, and K is an integer greater than or equal to 1. The calculation device 702 respectively acquires the average gray value of the K pixel sets, and the average gray value is an average value of the gray values of the pixels in the pixel set. The computing device 702 determines the average gray value corresponding to the pixel set with the largest number of pixels among the K pixel sets to be the gray threshold. The calculation device 702 sets the grayscale value of the pixels in the pixel set whose average grayscale value is smaller than the grayscale threshold value as the first grayscale value, and sets the grayscale value of the pixels in the pixel set whose average grayscale value is greater than or equal to the grayscale threshold value The value is set to the second grayscale value to obtain the second image block. Wherein, the first grayscale value may be 0, and the second grayscale value may be 255. Alternatively, the first grayscale value is 255, and the second grayscale value is 0. The pixels corresponding to the cracks in the image block are screened out, and then their gray value is assigned as the first gray value, thereby completing the binarization of the image block, that is, marking the pixels corresponding to the cracks.

In a possible implementation manner, the computing device 702 is further configured to convert the image block into a three-dimensional point cloud. The computing device 702 obtains the neighborhood radius of each 3D point in the 3D point cloud. The neighborhood radius is the average value of the Euclidean distance between the target 3D point in the 3D point cloud and the L neighborhood 3D points, and the L neighborhood 3D points are the 3D distances corresponding to the L pixels adjacent to the pixel corresponding to the target 3D point. point. L is an integer greater than or equal to 4. The computing device 702 acquires a search radius of the three-dimensional point cloud, where the search radius is an average value of neighborhood radii of all three-dimensional points in the three-dimensional point cloud. The computing device 702 classifies the three-dimensional points in the three-dimensional point cloud according to the search radius, and obtains K pixel sets corresponding to the K three-dimensional point sets. Each image block obtains a corresponding search radius according to the characteristics of its own 3D point cloud, so that the 3D points in the image block can be classified more accurately, and the 3D points corresponding to the cracks in the image block can be segmented out.

In a possible implementation manner, the computing device 702 is specifically further configured to determine an unclassified three-dimensional point in the three-dimensional point cloud as a seed point. The computing device 702 centers on the seed point, and classifies the three-dimensional points located within the search radius of the seed point into the target three-dimensional point set where the seed point is located. The computing device 702 determines that a 3D point that is not used as a seed point in the target 3D point set is the seed point, returns the seed point as the center, and the computing device 702 divides the 3D point located within the search radius of the seed point into the corresponding 3D point of the seed point. Steps to target a set of 3D points. Until there is no 3D point in the target 3D point set that is not used as a seed point, the computing device 702 returns to the step of determining an unclassified 3D point in the 3D point cloud as a seed point. Up to 3D points in the 3D point cloud are classified.

In a possible implementation manner, the pixels in the third image have a mapping relationship with the pixels in the reference image, and the pixels in the reference image carry geographic location information. The computing device 702 also obtains the target geographic location information of the target pixel according to the reference image, so as to obtain the geographic location information of the fracture. After the crack is detected, the geographic location information of the crack is further obtained to locate the crack.

As shown in FIG. 8 , FIG. 8 is a schematic structural diagram of an embodiment of a crack detection device provided in the present application. The crack detection device 800 in this embodiment includes a processor 801 and a memory 802 . The processor 801 is coupled to the memory 802, and the processor 801 is configured to execute the first embodiment or the second embodiment of the above crack detection method based on instructions stored in the memory 802.

The embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a computer, the flow of the crack detection method in any one of the above method embodiments is implemented.

Those skilled in the art can clearly understand that for the convenience and brevity of the description, the specific working process of the above-described system, device and unit can refer to the corresponding process in the foregoing method embodiment, which will not be repeated here.

In the several embodiments provided in this application, it should be understood that the disclosed system, device and method can be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or May be integrated into another system, or some features may be ignored, or not implemented. In another point, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

If the integrated unit is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or part of the contribution to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium , including several instructions to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, read-only memory), random access memory (RAM, random access memory), magnetic disk or optical disk, and other media that can store program codes.

Claims (17)

1. A method of crack detection, the method comprising:

the method comprises the steps that a camera collects N frames of first images, wherein the N frames of first images are obtained by shooting different areas of a target object through a preset operation, the preset operation is to rotate a lens of the camera and adjust the focal length of the lens, the distances between the different areas and the camera are different, and N is an integer greater than or equal to 2;

The camera sending the N frames of first images to a computing device;

the computing equipment splices part of each image in the N frames of first images to obtain a second image;

the computing device detects a crack of the target object based on the second image.

2. The method of claim 1, wherein the computing device stitching portions of each of the N frames of first images to obtain a second image comprises:

the computing equipment intercepts M rows of pixels of the N frames of first images respectively, wherein M is an integer greater than or equal to 1;

the computing device stitch together M rows of pixels of the N frames of first images to obtain the second image.

3. The method of any of claims 1 or 2, wherein the computing device detecting a crack of the target object based on the second image comprises:

the computing equipment pre-processes the second image to obtain J first image blocks, wherein the first image blocks comprise parts of the target object, and J is an integer greater than or equal to 1;

the computing equipment processes each first image block to obtain J second image blocks, target pixels corresponding to sub-cracks are marked in the second image blocks, and the sub-cracks are part of the cracks;

The computing equipment merges the J second image blocks to obtain a third image;

the computing device extracts the target pixel in the third image, resulting in the crack.

4. A method according to claim 3, wherein the target pixel is a pixel having a gray value of a first gray value, and wherein the computing device processing each of the first image blocks to obtain J second image blocks comprises:

the computing equipment classifies pixels in the first image block by utilizing a region growing algorithm to obtain K pixel sets, wherein each pixel set comprises at least one pixel, and K is an integer greater than or equal to 1;

the computing equipment respectively acquires average gray values of the K pixel sets, wherein the average gray values are average values of gray values of pixels in the pixel sets;

the computing equipment determines the average gray value corresponding to the pixel set with the largest number of pixels in the K pixel sets as a gray threshold value;

the computing device sets the gray value of the pixel in the pixel set with the average gray value smaller than the gray threshold value as a first gray value, and sets the gray value of the pixel in the pixel set with the average gray value larger than or equal to the gray threshold value as a second gray value, so that the second image block is obtained.

5. The method of claim 4, wherein the computing device classifying pixels in the image block using the region growing algorithm to obtain K sets of pixels comprises:

the computing device converting the image block into a three-dimensional point cloud;

the computing equipment acquires a neighborhood radius of each three-dimensional point in the three-dimensional point cloud, wherein the neighborhood radius is an average value of Euclidean distances between a target three-dimensional point in the three-dimensional point cloud and L neighborhood three-dimensional points, the L neighborhood three-dimensional points are three-dimensional points corresponding to L pixels adjacent to the pixel corresponding to the target three-dimensional point, and L is an integer greater than or equal to 4;

the computing equipment acquires a search radius of the three-dimensional point cloud, wherein the search radius is an average value of the neighborhood radiuses of all three-dimensional points in the three-dimensional point cloud;

and classifying the three-dimensional points in the three-dimensional point cloud according to the search radius by the computing equipment to obtain K pixel sets corresponding to the K three-dimensional point sets.

6. A method of crack detection, the method comprising:

the method comprises the steps that a camera collects N frames of first images, wherein the N frames of first images are obtained by shooting different areas of a target object through a preset operation, the preset operation is to rotate a lens of the camera and adjust the focal length of the lens, the distances between the different areas and the camera are different, and N is an integer greater than or equal to 2;

The camera splices the part of each image in the N frames of first images to obtain a second image;

the camera detects a crack of the target object based on the second image.

7. The method of claim 6, wherein the stitching the portion of each of the N frames of first images into a second image by the camera comprises:

the camera intercepts M rows of pixels of the N frames of first images respectively, wherein M is an integer greater than or equal to 1;

and the camera splices M rows of pixels of the N frames of first images to obtain the second image.

8. The method of claim 6 or 7, wherein the camera detecting a crack of the target object based on the second image comprises:

the camera pre-processes the second image to obtain J first image blocks, wherein the first image blocks comprise parts of the target object, and J is an integer greater than or equal to 1;

the camera processes each first image block to obtain J second image blocks, target pixels corresponding to sub-cracks are marked in the second image blocks, and the sub-cracks are part of the cracks;

the camera merges the J second image blocks to obtain a third image;

And the camera extracts the target pixel in the third image to obtain the crack.

9. The method of claim 8, wherein the target pixel is a pixel having a gray value of a first gray value, and wherein the processing each of the first image blocks by the camera to obtain J second image blocks comprises:

the camera classifies pixels in the first image block by using a region growing algorithm to obtain K pixel sets, wherein each pixel set comprises at least one pixel, and K is an integer greater than or equal to 1;

the camera respectively acquires average gray values of the K pixel sets, wherein the average gray values are average values of gray values of pixels in the pixel sets;

the camera determines the average gray value corresponding to the pixel set with the largest number of pixels in the K pixel sets as a gray threshold value;

and the camera sets the gray value of the pixel in the pixel set with the average gray value smaller than the gray threshold value as the first gray value, and sets the gray value of the pixel in the pixel set with the average gray value larger than or equal to the gray threshold value as the second gray value, so as to obtain the second image block.

10. The method of claim 9, wherein the camera classifying pixels in the image block using the region growing algorithm to obtain K sets of pixels comprises:

the camera converts the first image block into a three-dimensional point cloud;

the camera acquires a neighborhood radius of each three-dimensional point in the three-dimensional point cloud, wherein the neighborhood radius is an average value of Euclidean distances between a target three-dimensional point in the three-dimensional point cloud and L neighborhood three-dimensional points, the L neighborhood three-dimensional points are three-dimensional points corresponding to L pixels adjacent to the pixel corresponding to the target three-dimensional point, and L is an integer greater than or equal to 4;

the camera acquires the searching radius of the three-dimensional point cloud, wherein the searching radius is the average value of the neighborhood radiuses of all three-dimensional points in the three-dimensional point cloud;

and classifying the three-dimensional points in the three-dimensional point cloud by the camera according to the search radius to obtain K pixel sets corresponding to the K three-dimensional point sets.

11. A crack detection device, the device comprising:

the acquisition module is used for acquiring N frames of first images, wherein the N frames of first images are obtained by shooting different areas of a target object through a preset operation, the preset operation is to rotate a lens of the camera and adjust the focal length of the lens, the distances between the different areas and the camera are different, and N is an integer greater than or equal to 2;

The splicing module is used for splicing the part of each image in the N frames of first images to obtain a second image;

and the detection module is used for detecting the crack of the target object based on the second image.

12. The apparatus of claim 11, wherein the stitching module is specifically configured to:

respectively intercepting M rows of pixels of the N frames of first images, wherein M is an integer greater than or equal to 1;

and splicing M rows of pixels of the N frames of first images to obtain the second image.

13. The apparatus according to claim 11 or 12, wherein the detection module is specifically configured to:

preprocessing the second image to obtain J first image blocks, wherein the first image blocks comprise parts of the target object, and J is an integer greater than or equal to 1;

processing each first image block to obtain J second image blocks, wherein target pixels corresponding to sub-cracks are marked in the second image blocks, and the sub-cracks are part of the cracks;

combining the J second image blocks to obtain a third image;

and extracting the target pixel in the third image to obtain the crack.

14. The apparatus of claim 13, wherein the detection module is further specifically configured to:

Classifying pixels in the first image block by using a region growing algorithm to obtain K pixel sets, wherein each pixel set comprises at least one pixel, and K is an integer greater than or equal to 1;

respectively obtaining average gray values of the K pixel sets, wherein the average gray values are average values of gray values of pixels in the pixel sets;

determining the average gray value corresponding to the pixel set with the largest number of pixels in the K pixel sets as a gray threshold value;

and setting the gray value of the pixel in the pixel set with the average gray value smaller than the gray threshold value as the first gray value, and setting the gray value of the pixel in the pixel set with the average gray value larger than or equal to the gray threshold value as a second gray value, so as to obtain the second image block.

15. The apparatus of claim 14, wherein the detection module is further specifically configured to:

converting the first image block into a three-dimensional point cloud;

obtaining a neighborhood radius of each three-dimensional point in the three-dimensional point cloud, wherein the neighborhood radius is an average value of Euclidean distances between a target three-dimensional point in the three-dimensional point cloud and L neighborhood three-dimensional points, the L neighborhood three-dimensional points are three-dimensional points corresponding to L adjacent pixels corresponding to the pixel corresponding to the target three-dimensional point, and L is an integer greater than or equal to 4;

Acquiring a search radius of the three-dimensional point cloud, wherein the search radius is an average value of the neighborhood radiuses of all three-dimensional points in the three-dimensional point cloud;

and classifying the three-dimensional points in the three-dimensional point cloud according to the search radius to obtain K pixel sets corresponding to the K three-dimensional point sets.

16. A crack detection device, characterized in that the device comprises a processor and a memory, the processor being coupled to the memory, the processor being configured to perform the crack detection method according to any of claims 1-5 based on instructions stored in the memory.

17. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program that is executed by a processor to implement the crack detection method of any one of claims 1 to 10.

Images