CN112561983A - Device and method for measuring and calculating surface weak texture and irregular stacking volume - Google Patents

Abstract

The invention relates to a real-time volume measuring and calculating technology of a material pile, and aims to provide a device and a method for measuring and calculating the volume of an irregular material pile with weak texture on the surface. The method comprises the following steps: projecting a laser grid on the surface of the material pile by using a laser group, shooting at intervals by using a binocular camera, and acquiring a left image and a right image of the surface of the material pile in real time; correcting and preprocessing the image, and then carrying out edge detection; traversing and searching the position of the laser grid on the edge image; calculating parallax by using the position difference, and further calculating the depth value of each pixel on the laser grid; and calculating the height value of each pixel point on the surface of the material pile, then establishing a material pile model under a three-dimensional coordinate system, and calculating the real-time volume value of the material pile based on an integral principle. The invention combines laser and binocular vision technology, and solves the problem that the binocular vision technology is difficult to measure the depth of the surface weak texture material pile; the shape of the laser emitted by the laser group is in a grid type, which is beneficial to improving the accuracy of three-dimensional modeling and material pile volume calculation.

Description

A device and method for measuring the volume of weakly textured and irregular piles on the surface

technical field

The invention relates to the technical field of real-time measurement and calculation of the volume of a stockpile, in particular to a device and method for measuring the volume of a pile with weak surface texture and irregularity.

Background technique

In the process of industrial production, it is often encountered that it is necessary to quickly obtain the volume of the stockpile, such as coal stockpiles, garbage stockpiles, grain stockpiles, etc. Since these stockpiles are relatively large in size and generally have irregular surfaces, Manual measurements are time-consuming, labor-intensive, and not accurate enough. With the continuous improvement of industrial automation, the requirements for stockpile measurement technology are also constantly improving. With the continuous development of camera measurement technology, it is widely used in real-time detection of stockpile volume. For example, in the Chinese patent application "A In the method for measuring the volume of irregular stockpiles based on binocular cameras" (CN110738618A), binocular vision technology is used to measure the volume of stockpiles, but binocular vision technology is difficult to measure stockpiles with weak surface textures, such as: crushed garbage materials, sand, grain and coal, etc.

In order to solve the above problems, there is an urgent need for a technology that can measure the weakly textured and irregular material piles on the surface.

SUMMARY OF THE INVENTION

The technical problem to be solved by the present invention is to overcome the deficiencies in the prior art, and to provide a device and method for measuring the volume of materials with weak surface texture and irregular stacking.

For solving the technical problem, the solution of the present invention is:

The invention provides a method for measuring the volume of weakly textured and irregular piles on the surface, comprising the following steps:

(1) Set up a binocular camera at a suitable position directly above the material stack, and perform a calibration operation on the binocular camera to obtain the calibration parameters;

(2) Set up lighting and a laser group consisting of several line lasers at a suitable position above the material pile, and use the laser group to project a laser grid on the surface of the material pile; use a binocular camera to shoot at a set time interval, real-time Collect left and right images of the pile surface;

(3) Correcting and preprocessing the left and right images of the stockpile surface obtained by the binocular camera, and then performing edge detection to obtain an edge image;

(4) Traverse the left and right images after edge detection to find the position of the laser grid on the two edge images; use the difference of the laser positions on the two edge images to calculate the parallax, and further calculate the depth value of each pixel on the laser grid ;

(5) According to the depth value of each pixel on the laser grid, calculate the height value of each pixel point on the surface of the pile; then establish the pile model under the three-dimensional coordinate system, and calculate the real-time volume value of the pile based on the integration principle.

In the present invention, the weakly textured and irregular stacking refers to any one of garbage stacking, straw stacking, grain stacking or coal stacking, and the surface of the stacking has uneven surface and unclear texture. or features that are not sufficiently obvious.

In the step (1) of the present invention, the calibration operation refers to calculating the calibration parameters of the binocular camera through the Zhang Zhengyou chessboard calibration method.

In step (3) of the present invention, the correction refers to using MATLAB software to transfer the calibration parameters to the rectifyStereoImages function to obtain the corrected left and right images of the stockpile surface; the preprocessing refers to performing grayscale on the corrected images Binarization and binarization are performed to obtain a binary image, followed by noise filtering and morphological processing to obtain a preprocessed image.

The step (4) of the present invention specifically includes:

(4.1) In the order from left to right and top to bottom, traverse each pixel of the left and right images after edge detection, find the position of the laser grid and record the position information;

(4.2) Using the abscissa difference between each pixel corresponding to the image and the laser grid, calculate the parallax d=|x _l -x _r |;

计算激光网格上每个像素点的深度信息Z_ij；(4.3) According to the depth formula

Calculate the depth information Z _ij of each pixel on the laser grid;

In the formula, f is the focal length of the binocular camera; B is the distance between the two central axes of the binocular camera (measured with a vernier caliper); d is the parallax calculated in step (4.2).

The step (5) of the present invention specifically includes the following calculation process:

(5.1) Subtract the depth information Z _ij of the pixel points of the laser grid from the height H of the binocular camera from the bottom of the hopper to obtain the height h _ij of each pixel point; establish a three-dimensional coordinate system according to the height information and position information of each pixel point The stockpile model below;

求取每个像素点对应的大小a_ij；(5.2) Using the resolution X×Y of the binocular camera, the viewing angle θ, and the depth information Z _ij of each pixel on the laser grid, and the relational expression

Find the size a _ij corresponding to each pixel;

(5.3) It is known that the ratio of the horizontal and vertical bars of the laser grid on the images of the left and right parts of the stack surface is n×m, and the total volume of the stack is calculated based on the integration principle of each pixel point.

The present invention further includes: recording the volume change of the stockpile within the set time interval, and performing a difference operation on the volume values at the previous and subsequent times to obtain the rate of change of the stockpile volume.

The present invention also provides a device for implementing the aforementioned method, comprising a binocular camera, a laser group, a lighting lamp and an upper-level industrial computer, wherein the binocular camera is connected to the upper-level industrial computer through a signal line; the laser group is appropriately arranged above the material stack The position is composed of several line lasers, which are used to emit laser grids and evenly project them on the surface of the stockpile; the binocular camera is composed of two industrial cameras with the same parameter structure that are connected horizontally in parallel, located at a suitable position just above the stockpile and Lens down.

In the present invention, an inverted L-shaped iron frame rod formed by connecting a vertical rod and a horizontal rod is also included, and the laser group, the lighting lamp and the binocular camera are fixedly installed on the horizontal rod.

In the present invention, the upper industrial computer is embedded with a left and right image preprocessing module, a stock pile height calculation module, and a stock pile volume calculation module; wherein,

The left and right image preprocessing module is used to perform correction preprocessing and edge detection on the left and right images of the stockpile surface obtained by the binocular camera;

The stack height calculation module is used to traverse the left and right images after edge detection to find the position of the laser grid on the two edge images; calculate the parallax by using the difference of the laser positions on the two edge images, and then calculate the laser grid through the parallax. The depth value of each pixel on the grid;

The stack volume calculation module is used to calculate the height value of each pixel point on the stack surface according to the depth value of each pixel on the laser grid; then establish a stack model under the three-dimensional coordinate system, and calculate the real-time volume of the stack based on the principle of integration value.

Compared with the prior art, the beneficial effects of the present invention are as follows:

1. The present invention uses a laser group composed of several line lasers, and combines the laser with the binocular vision technology to solve the problem that the binocular vision technology is difficult to measure the depth of the pile with weak surface texture;

2. The present invention is suitable for environments with various lighting conditions by using lighting lamps to supplement light;

3. In the present invention, the shape of the laser emitted by the laser group adopts a grid type, which is beneficial to improve the accuracy of three-dimensional modeling and volume calculation of the stockpile;

4. The present invention has a small amount of calculation and high speed, and can be applied to various automatic control scenarios.

Description of drawings

Fig. 1 is the schematic diagram of the real-time measurement and calculation process of stockpile volume;

Fig. 2 is the schematic diagram of the operation flow between computing modules in the present invention;

3 is a schematic diagram of the working structure of the device;

Reference numerals in the figure: 301 iron frame rod; 302 material pile; 303 laser group; 304 binocular camera; 305 lighting.

Detailed ways

First of all, it should be noted that the present invention relates to image processing technology, which is an application of computer technology in the field of image processing and industrial control. In the implementation process of the present invention, the application of multiple software function modules will be involved. The applicant believes that, after carefully reading the application documents, accurately understanding the realization principle of the present invention and the purpose of the invention, and in combination with the prior art, those skilled in the art can fully use the software programming skills they master to realize the present invention. The aforementioned software function modules include, but are not limited to: left and right image preprocessing modules, stock pile height calculation modules, stock pile volume calculation modules, etc. All mentioned in the application documents of the present invention belong to this category, and the applicant will not list them one by one.

In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described below with reference to the accompanying drawings and specific embodiments. Obviously, the above-mentioned embodiments are only the most basic embodiments of the present invention, rather than all the embodiments. . Other embodiments based on the present invention all belong to the protection scope of the present invention.

The device for calculating the volume of materials with weak surface texture and irregular stacking in this embodiment is provided with an iron frame rod 301, a laser group 303, a binocular camera 304, and a lighting lamp 305; the iron frame rod 301 is formed by connecting a horizontal rod and a vertical rod And it is set in an inverted L shape, and the laser group 303, the lighting lamp 305 and the binocular camera 304 are installed on the horizontal bar; The binocular camera 304 is composed of two industrial cameras with the same parameter structure, which are connected horizontally in parallel, and are connected to the upper industrial computer through a signal line. The laser group 303 is arranged at a suitable position above the material stack 302, which can emit laser grids and ensure that the laser grid can be uniformly projected on the surface of the material pile 302; the laser group 303 is composed of several line lasers, and commercially available conventional products can be selected, such as Yiyi. The YZ6506KS5-GD1035 model product of Zhenglas Laser.

The left and right image preprocessing modules, the stack height calculation module, and the stack volume calculation module are embedded in the upper IPC, which are all software function modules. All image storage, processing, and operations are completed in the upper IPC. Among them, the left and right image preprocessing module is used to perform grayscale and binarization processing on the left and right cameras obtained by the binocular camera, and perform median filtering processing and opening operation processing on the obtained binarized images in turn. Perform the edge detection of Canny operator; the stack height calculation module respectively traverses each pixel of the left and right images in the following order from left to right, and from top to bottom, to find the position of the laser grid and Record the position information; use the abscissa difference between each pixel corresponding to the laser grid of the left and right images to calculate the parallax; use the vernier caliper to measure the distance B between the central axes of the two industrial cameras, and calculate the camera focal length f and the two industrial cameras. The distance B of the central axis is substituted into the depth formula, and the depth information Z _ij of each pixel on the laser grid is obtained; the pile volume calculation module, the height H from the binocular camera to the bottom of the hopper is subtracted from the obtained laser line depth information Z _ij , that is, the height h _ij of each pixel point of the laser grid on the surface of the material pile is obtained, and the material pile model under the three-dimensional coordinate system is established according to the height information and position information of each pixel point on the laser grid; according to the resolution X of the binocular camera ×Y, the viewing angle θ, and the depth information Z _ij of each pixel on the laser grid, use the correlation formula to find the size a _ij corresponding to each pixel; The ratio of the horizontal and vertical bars of the grid is n×m, and the total volume of the stockpile is calculated based on the integration principle of each pixel point.

Specifically, the rapid and accurate method for measuring the volume of weakly textured surface and irregular piles in the present embodiment, the process of which is shown in Figure 1, including the following steps:

Step 101: After installing the binocular camera, perform a calibration operation on the binocular camera;

Step 102: Use the binocular camera to collect the left image and the right image of the surface of the stockpile;

Step 103: Based on the set image preprocessing method, correct and preprocess the left and right images to obtain corrected and preprocessed images;

Step 104: Perform edge detection on the preprocessed left and right images to obtain left and right images after edge detection;

Step 105: traverse the left and right images after edge detection, find the position of the laser grid on the image, calculate the parallax by using the difference between the laser positions of the left and right images, and calculate the depth value of each pixel on the laser grid through the parallax;

Step 106 : According to the depth value of each pixel on the laser grid, calculate the height value of each pixel point on the surface of the material pile, and establish a material pile model in a three-dimensional coordinate system; and then calculate the real-time volume value of the material pile based on the integration principle.

Compared with manual measurement, this method can measure the volume change speed and provide technical support for the coupled automatic control system. Specifically: record the volume change of the stockpile within the set time interval, and calculate the difference between the volume values before and after, to obtain the rate of change of the stockpile volume;

Below, each step in the embodiment of the present invention will be described in detail:

Step 101 : After installing the binocular camera, perform a calibration operation on the binocular camera. The specific calibration operation is to calculate the calibration parameters of the binocular camera through the Zhang Zhengyou chessboard calibration method.

Step 102: Use the binocular camera to acquire left and right images of the surface of the stockpile.

After the left and right images are collected, based on the set image preprocessing method, the left and right images are corrected and preprocessed to obtain corrected and preprocessed images, that is, step 103 .

Step 103: Based on the set image preprocessing method, correct and preprocess the left and right images by using the left and right image preprocessing module.

Use MATLAB software to transfer the calibration parameters to the rectifyStereoImages function to obtain the corrected left and right images; grayscale and binarize the corrected left and right images to obtain a binary image; then perform noise filtering and Morphological processing to obtain the left and right images after preprocessing.

Preferably, median filtering is used in the noise filtering process, which can better remove noise and keep the image outline better; the morphological process uses open operation, which removes impurities outside the stack outline and also The ability to fill voids within the stack contour provides a better image for subsequent stack height calculations.

After the preprocessed left and right images are obtained, edge detection is performed on the preprocessed left and right images to obtain the left and right images after edge detection, that is, step 104 .

Step 104: Perform edge detection on the preprocessed left and right images to obtain left and right images after edge detection.

Preferably, a Canny algorithm is used for edge detection to obtain left and right images after edge detection.

Step 105 : traverse the left and right images after edge detection, find the position of the laser grid on the image, calculate the parallax by using the difference of the laser positions of the left and right images, and calculate the depth value of each pixel on the laser grid through the parallax.

获得激光网格上每个像素点的深度信息Z_ij；Using the stack height calculation module, traverse each pixel of the left and right images in the following order from left to right and from top to bottom, find the position of the laser grid and record the position information; use the left and right images There is abscissa difference between each pixel corresponding to the laser grid, calculate the parallax d=|x _l -x _r |; use the vernier caliper to measure the distance B between the central axes of the two industrial cameras, and compare the camera focal length f with the two The distance B of the central axis of the industrial camera is substituted into the depth formula:

Obtain the depth information Z _ij of each pixel on the laser grid;

Step 106: Calculate the height value of each pixel point on the surface of the material pile according to the depth value of each pixel on the laser grid; then establish a material pile model in a three-dimensional coordinate system, and calculate the real-time volume value of the material pile based on the integration principle.

求取每个像素点对应的大小a_ij；已知料堆表面左右两部分的图像上激光网格横纵条数比为n×m，将每个像素点基于积分原理，计算料堆的总体积

Using the stack volume calculation module, the height H from the binocular camera to the bottom of the hopper is subtracted from the obtained laser line depth information Z _ij , that is, the height h _ij of each pixel point of the laser grid on the stack surface is obtained. The height information and position information of each pixel point establishes a stockpile model in a three-dimensional coordinate system; according to the resolution X×Y of the binocular camera, the viewing angle θ and the depth information Z _ij of each pixel point on the laser grid, and the relationship Mode

Find the size a _ij corresponding to each pixel point; it is known that the ratio of the horizontal and vertical bars of the laser grid on the images of the left and right parts of the material pile surface is n×m, and each pixel point is based on the integration principle to calculate the total amount of the material pile. volume

Obviously, those skilled in the art can make subsequent various applications, supplements, changes and modifications to the present invention without departing from the spirit and scope of the present invention. If various applications, supplements, modifications and variations based on the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these applications, supplements, modifications and variations.

Claims (10)

1. A method for measuring and calculating the volume of a weak texture and an irregular stacking material on a surface is characterized by comprising the following steps:

(1) arranging a binocular camera at a proper position right above the material pile, and calibrating the binocular camera to obtain calibration parameters;

(2) arranging an illuminating lamp and a laser group consisting of a plurality of line lasers at a proper position above the material pile, and projecting a laser grid on the surface of the material pile by using the laser group; shooting by using a binocular camera according to a set time interval, and acquiring a left image and a right image of the surface of the stockpile in real time;

(3) correcting and preprocessing a left image and a right image of the surface of the material pile acquired by a binocular camera, and then carrying out edge detection to obtain an edge image;

(4) traversing the left and right images after the edge detection, and searching the positions of the laser grids on the two edge images; calculating parallax by using the difference of laser positions on the two edge images, and further calculating the depth value of each pixel on the laser grid;

(5) calculating the height value of each pixel point on the surface of the material pile according to the depth value of each pixel on the laser grid; and then establishing a stock pile model under a three-dimensional coordinate system, and calculating a real-time volume value of the stock pile based on an integral principle.

2. The method of claim 1, wherein the weak texture and irregular windrow is: any one of garbage stacking, straw stacking, grain stacking or coal stacking, and the surface of the stacking has the characteristics of uneven surface, unclear texture or insufficiently obvious characteristics.

3. The method according to claim 1, wherein in step (1), the calibration operation is to calculate calibration parameters of the binocular camera by Zhang Yongyou chessboard calibration method.

4. The method according to claim 1, wherein in the step (3), the correcting means that MATLAB software is used to introduce calibration parameters into a recitfyStereoImages function to obtain corrected left and right images of the surface of the windrow; the preprocessing refers to performing graying and binarization processing on the corrected image to obtain a binary image, and then sequentially performing noise filtering processing and morphological processing to obtain a preprocessed image.

5. The method according to claim 1, characterized in that said step (4) comprises in particular:

(4.1) traversing each pixel of the left and right images after edge detection according to the sequence from left to right and from top to bottom, searching the position of a laser grid and recording position information;

(4.2) calculating a parallax d ═ x using a difference in the horizontal coordinate between each pixel corresponding to the image and the laser grid_l-x_r|；

(4.3) according to the depth formula

Calculating depth information Z of each pixel point on laser grid_ij；

In the formula, f is the focal length of the binocular camera; b is the distance between the two central axes of the binocular camera (measured with a vernier caliper); d is the parallax calculated in step (4.2).

6. The method according to claim 1, wherein the step (5) specifically comprises the following calculation process:

(5.1) subtracting the depth information Z of the laser grid pixel points from the height H of the binocular camera from the bottom of the hopper_ijObtaining the height h of each pixel point_ij(ii) a Establishing a material pile model under a three-dimensional coordinate system according to the height information and the position information of each pixel point;

(5.2) Using the resolution X Y of the binocular Camera, the viewing Angle θ and the depth information Z of each pixel on the laser grid_ijAnd a relational expression

Finding the corresponding size a of each pixel point_ij；

(5.3) knowing that the ratio of the horizontal to vertical strips of the laser grids on the images of the left part and the right part of the surface of the material pile is n multiplied by m, and enabling each pixel point to be based on the integral principleCalculating the total volume of the pile

7. The method of claim 1, further comprising: and recording the volume change condition of the stockpile within a set time interval, and performing difference operation on the volume values of the front moment and the rear moment to obtain the volume change rate of the stockpile.

8. An apparatus for implementing the method of claim 1, comprising a binocular camera; the binocular camera system is characterized by further comprising a laser group, a lighting lamp and an upper industrial personal computer, wherein the binocular camera is connected to the upper industrial personal computer through a signal line; the laser group is arranged at a proper position above the material pile, consists of a plurality of line lasers and is used for emitting laser grids and uniformly projecting the laser grids on the surface of the material pile; the binocular camera is formed by horizontally connecting two industrial cameras with completely the same parameter structure in parallel, is positioned at a proper position right above the material pile, and has a downward lens.

9. The device of claim 8, further comprising an inverted L-shaped iron frame formed by connecting vertical bars and cross bars, wherein the laser group, the illuminating lamp and the binocular camera are fixedly mounted on the cross bars.

10. The device according to claim 8, wherein a left and right image preprocessing module, a stock pile height calculating module and a stock pile volume calculating module are embedded in the upper industrial personal computer; wherein,

the left and right image preprocessing module is used for carrying out correction preprocessing and edge detection on a left image and a right image of the surface of the stockpile acquired by the binocular camera;

the material pile height calculating module is used for traversing the left image and the right image after the edge detection and searching the positions of the laser grids on the two edge images; calculating parallax by using the laser position difference on the two edge images, and then calculating the depth value of each pixel on the laser grid through the parallax;

the stock pile volume calculation module is used for calculating the height value of each pixel point on the surface of the stock pile according to the depth value of each pixel on the laser grid; and then establishing a stock pile model under a three-dimensional coordinate system, and calculating a real-time volume value of the stock pile based on an integral principle.

Images