CN104751458B - A kind of demarcation angular-point detection method based on 180 ° of rotation operators - Google Patents

Abstract

The invention discloses a calibration corner detection method based on a 180° rotation operator, comprising the following steps: (1) acquiring a calibration image I to be processed, (2) judging the degree of grayscale change Z1; (3) judging 180 ° Rotation coincidence degree Z2; (4) Extract the corner points of the image. In the process of camera calibration, the present invention designs a template based on a 180° rotation operator according to the rotational symmetry of the patterns around the checkerboard corner points on the three sides of the calibration block, so as to realize efficient detection of the corner points of the calibration block. Through the analysis of experimental data and results, this method has high accuracy and stability for corner point detection. It is not only suitable for general calibration plate images, but also suitable for calibration block images with multiple surfaces. For those with certain inclination and Distorted calibration images also have better results.

Description

A Calibration Corner Detection Method Based on 180° Rotation Operator

technical field

The invention relates to the field of image processing in surface mounting equipment, in particular to a method for quickly and accurately detecting corner points in camera calibration based on a 180° rotation operator.

Background technique

Camera calibration is an important part of machine vision and image processing, and its accuracy determines the performance of the entire 3D measurement system in many cases. In the series of equipment for precision electronics manufacturing, surface mount equipment is used to mount components on PCBs. Therefore, the importance of equipment system accuracy is self-evident. In general, the checkerboard image is used as the calibration image, and the corner points are the points where the brightness change of the image is very different from that of adjacent points. Corner detection is an important part of camera calibration. By extracting the corner position information on the image, the relationship between the image plane coordinates and the space coordinate system of the information of each point of the 3D object in the 3D measurement system can be determined, so as to obtain the internal and external parameters of the camera, and then determine its spatial position.

The way of camera calibration is to solve the model parameters of the camera from the image coordinates of the known feature points according to the camera model. Each pixel in the image is obtained by transmission projection, corresponding to a ray formed by the optical center and the scene point. How to determine the equation of this ray in the scene coordinate system is the problem to be solved by camera calibration. During the calibration process, it is necessary to determine the intrinsic and extrinsic parameters of the camera. In the 3D measurement system, according to the projective geometry and the camera pinhole model, the formula for deducing the transformation relationship between the scene world coordinate system and the pixel coordinate system is as follows.

Where (X _wi , Y _wi , Z _wi ,1) is the world coordinate of the i-th point in space, (u _i , v _i ,1) is the image coordinate of the i-th point; m _ij is the i-th row of the projection matrix M The jth column element.

By using the 3D coordinates of a large number of known corner points on the calibration block and their pixel coordinates on the camera, the projection matrix M can be solved. It can be seen from the above formula that the detection accuracy of corner points directly determines the calculation accuracy of camera parameters.

At present, there are two main methods of corner detection based on checkerboard images: one is the method of manual acquisition, and the position of the corners is obtained one by one by clicking the mouse; the other is the method of automatic detection, such as using Harris operator, Susan The operator etc. obtains the corner position of the image. Manually obtain the location information of the corner points, the existing problems are: firstly, manual intervention is required. It affects the speed of the camera calibration image processing process, and there is no way to achieve dynamic calibration; secondly, the detection accuracy of corner points is unstable, and to a certain extent, it is also related to the operator's experience and so on. In the automatic detection method, although Harris operator, Susan operator, etc. can be used to realize the automatic calibration process, but the accuracy is unstable, false corners, etc., especially in the tilted checkerboard image, which affects the entire Accuracy of camera calibration.

Contents of the invention

The main purpose of the present invention is to overcome the shortcomings and deficiencies of the prior art, and to provide a calibration corner detection method based on a 180° rotation operator. Compared with the existing corner detection methods, it has strong practicability, high precision and fast Fast feature.

In order to achieve the above object, the present invention adopts the following technical solutions:

A kind of calibration corner point detection method based on 180 ° rotation operator, comprises the following steps:

(1) Obtain the calibration image I to be processed: in the three-dimensional measurement system, the calibration block is placed in a fixed position, the brightness of the light source is adjusted, and the calibration image is obtained by shooting with a camera;

(2) Judging the degree of gray scale change Z1: scan each point I(i, j) on the image one by one, and establish a square rotation operator template W with a size of n×n centered on the pixel point I(i, j) , using the method of calculating the mean square error of the gray level to calculate the flatness of the image in the operator template, that is, the degree of gray change Z1;

(3) Judgment of the 180° rotation coincidence degree Z2: After judging the degree of gray change Z1 and extracting the area where the edge and corner points are located, for the qualified pixel point I(i, j), calculate the black and white in the rotation operator template W Grayscale ratio, perform necessary black-and-white grayscale exchange, generate a new template, and then calculate the coincidence function on this new template, that is, after rotating the rotation operator template W by 180°, the rotation template W0 is obtained, Calculate the mean value Z2 of the sum of the absolute values of the elements in the difference matrix between the two;

(4) Extract the corner points of the image: After the gray scale change degree Z1 and the rotation coincidence degree Z2 are judged by the rotation operator template, the connected domain of the area where each corner point is located is obtained, and the point with the highest degree of coincidence in the image is the point of the image corner.

Preferably, in step (1), before using the rotation operator to process, first superimpose a two-dimensional Gaussian filter with a template size of 5×5 and a width of 1 on the image to filter out impurity noise.

Preferably, the function g(i,j) of the two-dimensional Gaussian filter, in for template center σ is the width, that is, the degree of smoothness.

Preferably, in step (1), the three-dimensional measurement system includes a projector, a camera device and a calibration block.

Preferably, in step (2), the mean square error function Z ₁ (i, j) of calculating the degree of gray scale change Z1 is:

In the formula, is the average gray value of pixels in the template W, and n is the width of the window W.

Preferably, in step (3), it is judged whether the degree of gray scale change Z1 satisfies the preset threshold T1, the threshold T1 is selected as 0.1, and it is judged whether Z ₁ (i,j) satisfies Z ₁ (i,j)>T1, if it can Satisfied, it proves that the rotation operator template centered on the current pixel point I(i,j) is in an edge point or corner point rather than a flat area, and its gray level changes greatly, so continue to calculate its coincidence function value Z2 for processing ; If unsatisfactory, do not need to process, skip.

Preferably, in step (3), before calculating the 180° rotation coincidence degree Z2, judge the black-and-white grayscale ratio, the total area of the template is S=n×n, the area of the white part is S1, and the preset threshold e=(n×n- 1)/2, then the area ratio α=S1/S,

Among them, W′(i,j) is the rotation operator template after the black-white-grayscale swap.

Preferably, calculate the function Z ₂ (i, j) of the 180° rotation coincidence degree Z2, In the formula, n is the width of the window W, (x,y) is the moving point in the window, I(ix,j+y) is the rotation of the I(i+x,jy) point around I(i,j) Corresponding point after 180°.

Preferably, it is judged whether the rotation coincidence degree Z2 satisfies the preset threshold T2, the selected threshold T2 does not exceed 0.1, and it is judged whether Z ₂ (i,j) satisfies Z ₂ (i,j)>T2, if it can be satisfied, it is proved that the current pixel The symmetry of the rotation operator template centered at point I(i,j) is sufficient, and the coincidence degree is high. The pixel point I(i,j) is located in the corner area, and its position can be accurately extracted. Otherwise, no processing is required. jump over.

Preferably, from Z ₁ (i, j) obtained in step (2) and Z ₂ (i, j) obtained in step (3), the corner point is located in an approximately fuzzy area, and the connected domain is obtained by using eight neighbors , and then the superposition judgment algorithm screens out the point with the highest coincidence value Z2 in the connected domain, which is the corner point of the image.

Compared with the prior art, the present invention has the following advantages and beneficial effects:

(1) The present invention detects corner points through 180° rotation transformation, which speeds up the detection speed, has simple process and strong operability.

(2) The invention has invariance to the rotation transformation, can effectively remove the interference of false corner points and noise points, and improves the accuracy and stability of the algorithm.

(3) The present invention is applicable not only to general calibration plate images, but also to calibration block images with multiple surfaces, and it also has a good effect on calibration images with certain inclination and distortion.

(4) The present invention can meet the expected design requirements, and compared with other currently popular automatic detection algorithms, it has wider applicability, better accuracy and higher stability.

Description of drawings

Fig. 1 is the structural representation of detection system of the present invention;

Fig. 2 is the flowchart of detection method of the present invention;

Fig. 3 is the schematic diagram of rotation operator of the present invention;

Fig. 4 is a schematic diagram of the centrosymmetry of the rotation operator of the present invention.

detailed description

The present invention will be further described in detail below in conjunction with the embodiments and the accompanying drawings, but the embodiments of the present invention are not limited thereto.

As shown in FIG. 1 , the detection system of the calibration corner detection method based on the 180° rotation operator of the present invention includes a calibration block 4 placed on a platform 3 , the projector 2 can be used as a light source, and the camera 1 collects images. In the normal calibration process, it is usually necessary to collect more than or equal to 2 images of the calibration board, and the calibration board must not be placed on the same plane. In this way, the first is that the operation is cumbersome, and the second is that in the process of replacing the calibration plate, it will inevitably have a certain impact on the measurement accuracy. The calibration block used here is a cube composed of at least 3 such calibration boards. As shown in Figure 1. Using the calibration block only needs to take a picture once, and the algorithm is processed in the same picture, so it is more convenient and quicker.

As shown in Figure 2, it is an overall flowchart of a corner detection method of the present invention, and its specific operation steps are as follows:

(1) Obtain the calibration image I to be processed: In the three-dimensional measurement system shown in Figure 1, the calibration block is placed at a fixed position, the brightness of the light source is adjusted, and the calibration image I is obtained by shooting with a camera. Before using the rotation operator to process, first superimpose the image with a 5×5 two-dimensional Gaussian filter with a width of 1 to filter out impurity noise;

(2) Judging the degree of gray scale change Z1: scan each point I(i, j) on the image one by one, and establish a square rotation operator template W with a size of n×n centered on the pixel point I(i, j) . The flatness of the image in the operator template is calculated by calculating the mean square error of the gray level, that is, the gray level change degree Z1.

As shown in FIG. 3 , it is a schematic diagram of the principle of a rotation operator designed in the present invention. For each point I(i,j) of the image, the template centered on this point is called W. W can be circular or square. Here, a square template is selected. If the width of the template is n, then the number of pixels in the template is n×n. For calculation accuracy and detection accuracy, the width n of W should be smaller than the side length of a checkerboard. The value of n is an odd number, which ensures that the value of each point of the template in the calculation process is an integer.

When the template W is in the region a, the image is flat and the gray variance value is small, so there are no edge points and corner points. When template W is in area b, its coincidence degree after 180° rotation is high, which is an ideal corner point. Although the images at c and d are not flat, they also have certain symmetry, but their symmetry about the center point is low. In this way, we can accurately determine the position of the corner point according to the calculated coincidence degree after 180° rotation.

After that, it is necessary to effectively remove the flat area in the image, and extract the edge point and the area where the corner point is located. Because the mean square error value of the pixel grayscale is relatively small in the template where the pixel points in the flat area are located, the mean square error response value of the rotation operator is designed as the grayscale variance reflecting the intensity of the gray value change of the surrounding pixels. To remove flat areas, the formula expression, that is, to calculate the mean square error function Z ₁ (i,j) of the degree of gray change Z1 is as follows

In the formula, is the average gray value of pixels in the template W, and n is the width of the window W.

It is judged whether the degree of gray scale change Z1 satisfies a preset threshold T1. The threshold T1 is chosen to be approximately 0.1. Judging whether Z ₁ (i,j) satisfies Z ₁ (i,j)>T1, if so, it proves that the rotation operator template centered on the current pixel point I(i,j) is at the edge or corner position In the non-flat area, the gray level changes greatly, and the coincidence function value Z2 is continued to be calculated for processing. If it is not satisfied, there is no need to deal with it, this is a non-corner point.

(3) Judgment of the 180° rotation coincidence degree Z2: After judging the gray scale change degree Z1, before calculating the 180° rotation coincidence degree Z2, for the qualified pixel point I(i,j), calculate the black and white in the rotation operator template W Grayscale ratio, perform necessary black-and-white grayscale exchange, and generate a new template. This is because, in the inclined calibration block image, the above coincidence degree function is directly calculated, and the template with the same inclination will have a large difference in the calculation results for those with more white parts or more black parts.

Therefore, a judgment condition is added in the process of calculating the integration function: if the area of the white part accounts for more than half of the total area in the template of the rotation operator, then the black and white part is converted to gray value. On this new template, the coincidence function calculation is performed, that is, after the rotation operator template W is rotated by 180°, the rotation template W0 is obtained, and the mean value Z2 of the sum of the absolute values of each element in the difference matrix between the two is obtained .

The total area of the template is S=n×n, the area of the white part is S1, and the preset threshold e=(n×n−1)/2 is selected. Then the area ratio α=S1/S. Swap if the following conditions are met.

Among them, W′(i,j) is the rotation operator template after the black-white-grayscale swap.

As shown in FIG. 4 , it is a schematic diagram of the centrosymmetry of the rotation operator of the present invention. For a certain pixel A(i+x,jy) in the template, the symmetrical point about the central point I(i,j) is A1(i+x,jy). The coincidence degree response value of the rotation operator is defined as the average value of the absolute value of the gray square difference of each pair of pixels symmetrical about the center point I(i, j) in the template W, then the expression of the coincidence degree function Z ₂ (i ,j) as follows:

In the formula, n is the width of the window W, and (x, y) is the moving point in the window. I(i-x,j+y) is the corresponding point after point I(i+x,j-y) is rotated 180° around I(i,j). The calculated value Z2 of the coincidence function of the characteristic corner points on the calibration image is small; while boundary points and noise points, etc., due to asymmetry, the gray value of the pixel on one side is large, and the gray value of the pixel on the other side is small, so its Z2 is relatively large. The coincidence function calculation value Z2 of a certain point I(i,j) on the image is the reflection of the spatial symmetry of the pixel gray distribution within the small window centered on the pixel at this point.

It is judged whether the rotation coincidence degree Z2 satisfies the preset threshold T2. The threshold T2 is chosen not to exceed 0.1. Judging whether Z ₂ (i,j) satisfies Z ₂ (i,j)>T2, if it can be satisfied, it proves that the symmetry of the rotation operator template centered on the current pixel point I(i,j) is sufficient, and the coincidence degree is relatively high. High, the pixel point I(i,j) is in the position of the corner area, and its position can be accurately extracted. Otherwise, there is no need to deal with it, and it belongs to non-corner points.

(4) Extract the corners of the image: From the schematic diagram of the rotation operator in Figure 4, it can be seen that this is the rotation operator template W (an n×n square matrix) centered on the pixel point I(i,j), and the 180 After °rotation, the new rotation template W0 is obtained, and the two are subtracted again to obtain the mean value of the sum of the absolute values of each element in the difference matrix, that is, the coincidence degree function value Z2.

After judging the degree of gray change Z1 and the degree of rotation coincidence Z2 by the rotation operator template, after obtaining the connected domain of the region where each corner point is located, the corner points in each region are extracted respectively. Next, the eight-neighborhood calculation is performed on the entire image to obtain the connected domain of the area where each corner point is located. The point with the highest degree of inner coincidence is the corner point of the image. Let the number of pixels in the connected domain extracted in the previous step be l, when l<2 or l>8, the pixels in this connected domain are noise points or edge points, and remove them; if 1<l<9, filter The point with the highest coincidence value Z2 in the connected domain is the corner point of the image. So far, the corner point detection of the calibration image is completed.

The above-mentioned embodiment is a preferred embodiment of the present invention, but the embodiment of the present invention is not limited by the above-mentioned embodiment, and any other changes, modifications, substitutions, combinations, Simplifications should be equivalent replacement methods, and all are included in the protection scope of the present invention.

Claims (10)

1. a kind of calibration corner point detection method based on 180 ° of rotation operators, is characterized in that, comprises the steps: (1) Obtain the calibration image I to be processed: in the three-dimensional measurement system, the calibration block is placed in a fixed position, the brightness of the light source is adjusted, and the calibration image is obtained by shooting with a camera; (2) Judging the degree of gray scale change Z1: scan each point I(i, j) on the image one by one, and establish a square rotation operator template W with a size of n×n centered on the pixel point I(i, j) , using the method of calculating the mean square error of the gray level to calculate the flatness of the image in the template of the rotation operator, that is, the degree of gray level change Z1; (3) Judgment of the 180° rotation coincidence degree Z2: After judging the degree of gray change Z1 and extracting the area where the edge and corner points are located, for the qualified pixel point I(i, j), calculate the black and white in the rotation operator template W The grayscale ratio is used to perform necessary black-and-white grayscale swaps to generate a new template. On this new template, the coincidence function calculation is performed, that is, after the rotation operator template W is rotated by 180°, a new rotation operator is obtained. Sub-template W0, calculate the mean value Z2 of the sum of the absolute values of the elements in the difference matrix between the two; (4) Extract the corner points of the image: After the gray scale change degree Z1 and the rotation coincidence degree Z2 are judged by the rotation operator template W, the point with the highest coincidence degree in the connected domain of the area where each corner point is located is obtained, which is the corner of the image point. 2. the calibration corner point detection method based on 180 ° rotation operator according to claim 1, it is characterized in that, step (1) is before utilizing rotation operator processing, at first to image superposition template is 5 * 5, and width is 1 two-dimensional Gaussian filter to filter out impurity noise. 3. the calibration corner detection method based on 180 ° of rotation operators according to claim 2, is characterized in that, the function g (i, j) of described two-dimensional Gaussian filter, in for template center σ is the width, that is, the degree of smoothness. 4. The calibration corner detection method based on the 180° rotation operator according to claim 1, characterized in that, in the step (1), the three-dimensional measurement system includes a projector, a camera device and a calibration block. 5. the calibration corner point detection method based on 180 ° of rotation operators according to claim 1, is characterized in that, in step (2), calculates the mean square error function Z ₁ (i, j) of degree of change of gray level Z1, for: <mrow> <msub> <mi>Z</mi> <mn>1</mn> </msub> <mrow> <mo>(</mo> <mi>i</mi> <mo>,</mo> <mi>j</mi> <mo>)</mo> </mrow> <mo>=</mo> <mfrac> <mrow> <msub> <mo>&amp;Sigma;</mo> <mrow> <mo>(</mo> <mi>i</mi> <mo>,</mo> <mi>j</mi> <mo>)</mo> <mo>&amp;Element;</mo> <mi>W</mi> </mrow> </msub> <msup> <mrow> <mo>(</mo> <mi>I</mi> <mo>(</mo> <mrow> <mi>i</mi> <mo>,</mo> <mi>j</mi> </mrow> <mo>)</mo> <mo>-</mo> <mover> <mi>I</mi> <mo>&amp;OverBar;</mo> </mover> <mo>)</mo> </mrow> <mn>2</mn> </msup> </mrow> <mrow> <mi>n</mi> <mo>&amp;times;</mo> <mi>n</mi> <mo>-</mo> <mn>1</mn> </mrow> </mfrac> </mrow> In the formula, is the average gray value of pixels in the rotation operator template W, and n is the width of the rotation operator template W. 6. The calibration corner point detection method based on 180° rotation operator according to claim 5, characterized in that, in step (3), it is judged whether the degree of gray scale change Z1 satisfies the preset threshold T1, and the selection threshold T1 is 0.1 , to judge whether Z ₁ (i,j) satisfies Z ₁ (i,j)>T1, if so, it proves that the rotation operator template centered on the current pixel point I(i,j) is at an edge point or a corner point The position is not a flat area, and its gray level changes greatly. Continue to calculate its rotation coincidence degree Z2 for processing; if it is not satisfied, no processing is required, and it is skipped. 7. the calibration corner point detection method based on 180 ° of rotation operators according to claim 1, is characterized in that, in step (3), before calculating 180 ° of rotation coincidence degree Z2, judge black and white gray scale ratio, template total area S=n×n, the area of the white part is S1, select the preset threshold e=(n×n-1)/2, then the area ratio α=S1/S, <mfenced open = "{" close = ""> <mtable> <mtr> <mtd> <mrow> <mi>&amp;alpha;</mi> <mo>&amp;le;</mo> <mi>e</mi> <mo>,</mo> </mrow> </mtd> <mtd> <mrow> <msup> <mi>W</mi> <mo>&amp;prime;</mo> </msup> <mrow> <mo>(</mo> <mi>i</mi> <mo>,</mo> <mi>j</mi> <mo>)</mo> </mrow> <mo>=</mo> <mi>W</mi> <mrow> <mo>(</mo> <mi>i</mi> <mo>,</mo> <mi>j</mi> <mo>)</mo> </mrow> </mrow> </mtd> </mtr> <mtr> <mtd> <mrow> <mi>&amp;alpha;</mi> <mo>&gt;</mo> <mi>e</mi> <mo>,</mo> </mrow> </mtd> <mtd> <mrow> <msup> <mi>W</mi> <mo>&amp;prime;</mo> </msup> <mrow> <mo>(</mo> <mi>i</mi> <mo>,</mo> <mi>j</mi> <mo>)</mo> </mrow> <mo>=</mo> <mn>1</mn> <mo>-</mo> <mi>W</mi> <mrow> <mo>(</mo> <mi>i</mi> <mo>,</mo> <mi>j</mi> <mo>)</mo> </mrow> </mrow> </mtd> </mtr> </mtable> </mfenced> Among them, W′(i,j) is the rotation operator template after the black-white-grayscale swap. 8. the calibration corner detection method based on 180 ° of rotation operators according to claim 5, is characterized in that, calculates the function Z ₂ (i, j) of 180 ° of rotation coincidence degree Z2, In the formula, n is the width of the rotation operator template W, (x, y) is the moving point in the template, I(ix, j+y) is the I(i+x, jy) point with I(i, j) The corresponding point after the center is rotated 180°. 9. The calibration corner point detection method based on the 180° rotation operator according to claim 8, characterized in that, judging whether the rotation coincidence degree Z2 meets the preset threshold T2, the selection threshold T2 does not exceed 0.1, and the judgment Z ₂ (i ,j) Whether Z ₂ (i,j)>T2 is satisfied, if it can be satisfied, it proves that the symmetry of the rotation operator template W centered on the current pixel point I(i,j) is sufficient, the coincidence degree is high enough, and the pixel point I(i,j) is in the position of the corner area, and its position can be accurately extracted, otherwise, it does not need to be processed and skipped. 10. the calibration corner point detection method based on 180 ° rotation operator according to claim 8, is characterized in that, Z ₁ (i, j) gained by step (2) and Z ₂ ( i, j), get the approximate fuzzy area where the corner point is located, use the eight neighbors to obtain the connected domain, and then superimpose the judgment algorithm to select the point with the highest rotation coincidence degree Z2 in the connected domain, which is the corner point of the image.