CN103047943B - Based on the door skin geomery detection method of single projection coded structured light - Google Patents

Abstract

The invention discloses a method for detecting the shape and size of an outer panel of a car door based on single-projection coded structured light, aiming at breaking the rare situation that structured light visual detection technology is used for online detection of the shape and size of an outer panel of a car door. The steps are: 1. The structured light detection system calibration stage: calibrate the camera (2) and the projector (4), and establish the corresponding nonlinear relationship between the spatial position and the image coordinates; 2. The coding projection template design stage; 3 . Detection image decoding and recognition stage: use the projector (4) to project the coded projection template to the surface of the door outer panel, then use the camera (2) to shoot the image of the surface of the door outer panel, and send the captured image to the computer for image processing and Recognition; 4. Spatial point cloud coordinate calculation and error evaluation stage: 1) Calculate the three-dimensional space coordinates of the feature points of the projected image on the surface of the outer panel of the car door; 2) Compare the calculated detection point cloud with the point cloud model measured by the three-coordinate measuring machine for registration.

Description

Detection method of shape and size of car door outer panel based on single-projection coded structured light

technical field

The invention relates to a non-contact on-line detection method for the shape and size of a car door outer panel, more specifically, the invention relates to a method for detecting the shape and size of a car door outer panel based on single-projection coded structured light.

Background technique

Most of the outer panels of automobile doors are three-dimensional curved surfaces, which generally refer to automobile stamping parts and components welded by multiple stamping parts; the shape and size detection of the outer panel of the door is an important means to evaluate the manufacturing quality of the outer panel of the automobile door. Accuracy seriously affects the assembly, safety and visual aesthetics of the car. At present, the commonly used detection method is mainly to use the three-coordinate measuring machine to conduct sampling off-line detection. Although it can obtain high detection accuracy, it is easy to cause damage to the surface of the car door because it belongs to point-to-point contact detection, and it takes a long time. Therefore, 100% online detection cannot be realized. Since the 21st century, the development of vision-based three-dimensional perception technology has promoted a huge change in the detection methods of the manufacturing industry. The spatial outline size of a specific object can only be formed by a series of points, that is, the point cloud. Structured light vision technology has the advantages of non-contact, large range and high degree of automation. It can project a specific coding template to the surface of the detection object. Since the template is coded by a pseudo-random array, only one image needs to be projected, which is conducive to Carry out online size detection on the outer panel of the car door, and finally use the triangulation method to calculate the spatial coordinates of the feature points on the surface of the object, and obtain a three-dimensional point cloud describing the surface characteristics of the outer panel of the car door, which has the advantages of fast speed, high degree of automation and high precision Advantages, great potential for development.

The structured light 3D vision technology based on the spatial neighborhood coding strategy has achieved great development in China since the 21st century. At present, as far as the research on the structured light vision technology itself is concerned, many patents and papers have been published. For the application of structured light 3D vision technology in 3D reconstruction, please refer to the Chinese patent publication No. CN101697233A, the publication date is 20120606, the title of the invention is "a method for reconstructing the surface of 3D objects based on structured light" and the Ph.D. Thesis (SongZhan.UseofStructuredLightfor3DReconstruction,TheChineseUniversityofHongKong,August2008), this paper gives a detailed introduction and specific application process to several commonly used methods of structured light technology. Although there are more and more researches on structured light vision technology, the research on applying structured light vision detection technology to the online detection of the shape and size of the outer panel of the car door is very rare.

Contents of the invention

The technical problem to be solved by the present invention is to break the rare situation that the structured light visual detection technology is applied to the online detection of the shape and size of the outer panel of the car door, and provide a method for detecting the shape and size of the outer panel of the car door based on single-projection coded structured light.

In order to solve the above-mentioned technical problems, the present invention is realized by adopting the following technical scheme: the steps of the method for detecting the shape and size of the outer panel of the car door based on single-projection coded structured light are as follows:

1. Calibration stage of structured light detection system:

In order to ensure the smooth progress of the detection, it is necessary to calibrate the camera and projector in the structured light detection system, establish the corresponding nonlinear relationship between the spatial position and the image coordinates, and use this nonlinear relationship to calculate and detect the corresponding image feature points. space coordinates;

2. Code projection template design stage:

During the detection process, it is only necessary to project a coded pattern onto the surface of the outer panel of the car door. The coded pattern is a graphic array with a size of 40 rows and 45 columns consisting of 4 X-shaped primitive symbols in different directions. The arrangement order is determined by a pseudo-random M array, which has the following characteristics: the graphics in each 3×3 sub-array appear only once in the entire array template, and there are at least 3 different graphics in the corresponding positions in any two sub-arrays. The same; each X-type primitive symbol except the edge has been coded, and the size of its coded value is determined by the elements in its 3×3 neighborhood. The different angles between the axis and the base axis give different codewords, so the graphic array corresponds to a 40×45 digital array, which is called a pseudo-random M array;

3. Detection image decoding recognition stage:

Use the projector to project the coded projection template to the surface of the outer panel of the car door, and then use the camera to capture the image of the surface of the outer panel of the car door, and send the captured image to the computer for image processing and recognition;

4. Space point cloud coordinate calculation and error evaluation stage:

(1) Calculate the three-dimensional space coordinates of the projected image feature points on the surface of the outer panel of the car door;

(2) Register the calculated detection point cloud with the point cloud model measured by the three-coordinate measuring machine.

The steps in the calibration phase of the structured light detection system described in the technical proposal are as follows:

1) Calibrate the camera:

Use Zhang Zhengyou’s calibration method to calibrate, and use a projector to project the designed 12×16 black and white checkerboard target to the right side of the calibration plane, and then use the camera to shoot the entire calibration plane. The camera mark in the lower left corner of the 12 images obtained is Extract feature corners from the target image, extract 100 corners from each camera target image, and obtain 1200 feature corner image coordinates in total from 12 images. According to Zhang Zhengyou’s calibration method, the world coordinate system is set at the upper left corner of the checkerboard image At a corner point at , the coordinate value in the Z direction is taken as zero. Since the size of each checkerboard is precisely designed to be 30mm, the world coordinates of the corner points of the checkerboard feature in the X and Y directions can be accurately known. Therefore, the obtained plane After the image feature corner coordinates and the spatial coordinates of the corresponding points, the toolbox can be used to complete the calibration and calculate the internal and external parameters of the camera;

2) Calibrate the projector:

Extract the feature corners of the projected target in the camera image, and then calculate the space of the feature corners of the projected image on the calibration plane in the world coordinate system according to the obtained internal and external parameters of the camera and the light plane intersection method Coordinates, so far, the image coordinates and space coordinates of the projection target have been calculated, therefore, the calibration parameters of the projector are calculated using the same method as the camera calibration, and the internal and external parameters of the projector and the rotation between the projector and the camera are obtained translation matrix.

The steps in the coding projection template design phase described in the technical proposal are as follows:

1) Pseudo-random M array generation corresponding to the graphic array:

The characteristic of the pseudo-random M array is that the sub-array of a specific 3×3 window is unique in the entire array. The larger the Hamming distance between any two windows, the greater the difference between the two. The pseudo-random M The calculation method of array Hamming distance is as follows:

Among them: V _(ij) represents the 9-element vector of the 3×3 neighborhood, and H represents the number of times that different codewords appear in the corresponding position vectors between two different 3×3 neighborhoods, which is the Hamming distance. The larger the Hamming distance, Indicates that the greater the degree of difference between the windows, the pseudo-random M array constructed in the design stage of the encoding projection template, the minimum Hamming distance is set to 3, which means that the elements corresponding to any two sub-arrays of 3×3 windows have at least 3 The two are different, such a design is conducive to enhancing the stability of the sub-array and the adaptability to the degree of spatial variation of the surface of the detection object;

In the code projection template design stage, it is necessary to generate a pseudo-random M array with a digital primitive of {0,1,2,3}, a minimum Hamming distance equal to 3, and a size of 40×45; the circular filling method is used to complete the code projection template design To construct the required pseudo-random array, first generate a 3×3 subarray in the upper left corner of the array, and then substitute a 3×1 subarray along the right side to compare whether the Hamming distance between adjacent 3×3 windows meets the conditions. If it does not meet the requirements, proceed to the right until the boundary; then substitute the 1×3 sub-array along the bottom of the initial 3×3 array, the method is the same as above, until the boundary; and then proceed to the blank position of 4×4 Substituting a new code value to form a new 3×3 sub-array, comparing the Hamming distance with all the sub-arrays generated before, if the condition is met, continue, if not, re-substituting, if substituting all possible code words is not satisfied, then Restart the build process;

2) X-type primitive symbol design based on geometric features:

Geometric primitives for constructing coded projection templates often use squares or circles, but due to shadows or occlusions, circular or square patterns with geometric centers as feature corners are more likely to have geometric center offsets or ambiguous angles. point, resulting in inaccurate extraction of corner coordinates, and according to the corner characteristics of the checkerboard, a class of X-shaped geometric primitives is designed to have relatively stable corner characteristics in the case of occlusion or absence. The symbol is designed with a checkerboard image as the feature, symmetrical about the center of the feature corner, and composed of a triangle or square that is symmetrical to the center, according to the angle between the longer main inertial axis and the base axis {0°, 45°, 90 The difference of °,135°} corresponds to 4 codewords {0,1,2,3} respectively;

According to the correspondence between the above corresponding codewords and graphics, according to the obtained pseudo-random M array template, the X-type primitives are sequentially substituted into the graphics array template to generate a coded projection template with a size of 768×1024. A single X- The size of the type primitive symbol is 10×10pixel, and the interval between the X-type primitive symbols is set to 9pixel. The coding projection template uses black as the background and white as the background color of the X-type primitive graphics to generate a monochrome coding projection The template is beneficial to reduce the degree of reflection on the surface of the outer panel of the car door.

The steps in the detection image decoding and recognition stage described in the technical solution are as follows:

1) Image preprocessing is carried out on the image of the initial detection of the surface of the outer panel of the car door in the camera image;

2) Perform primitive segmentation and pattern recognition on the projected image on the surface of the door outer panel in the camera image;

3) Stereo matching is performed on the feature corner points of the projected image on the surface of the door outer panel and the feature corner points of the coded projection template in the camera image.

The image preprocessing described in the technical solution to the image of the initial detection of the surface of the outer panel of the car door in the camera image includes the following steps:

1) Take a mask operation on the target area image in the camera image, obtain the local area image on the surface of the door outer panel projected with the coded projection template, and eliminate the useless background area;

2) Gaussian filtering is performed on the obtained target area image to remove noise;

3) Calculate the average gray value of the image background in the target area, and subtract it from the original image gray value, remove the background, make the background gray uniform, avoid the gray gradient change in the background image, and affect the edge extraction result, Leading to the extraction of too many non-primitive symbol edges, causing interference to the segmentation mark;

4) In order to make the edge of the X-type primitive symbol in the image of the target area more prominent, the Laplacian 5-neighborhood edge enhancement method is used to strengthen the edge of the X-type primitive symbol in the image of the target area, which is beneficial to edge extraction.

Carrying out primitive segmentation and pattern recognition of the projected image on the surface of the outer panel of the car door in the camera image described in the technical solution includes the following steps:

1) X-type primitive symbol edge extraction and segmentation marking:

The canny operator is used to extract the edge of the local target area image, and the edges of all X-type primitive symbols and a large number of non-primitive symbols can be obtained. After edge extraction, the image is a binary image, and the intensity value of the edge part is 1, and the non-edge The intensity value is 0; when calculating the corner point and the long main inertial axis angle, the coordinate positions of the edges of all primitive symbols should be used. If there are many edges of non-primitive symbols, it will cause multiple A non-primitive symbol feature corner point and corner point extraction are not accurate, so it needs to be eliminated; and because of calculating the feature corner point coordinates of the X-type primitive symbol edge image and restoring the codeword of the X-type primitive symbol It is necessary to calculate the edge of each X-type primitive symbol separately, so that the different X-type primitive symbols remain relatively independent and do not affect each other during the calculation process, so it is also necessary to calculate each X-type primitive symbol Use the segmentation labeling algorithm to classify and label different X-type primitive symbols, and perform calculations according to the different labels, and the X-type primitive symbols with different labels cannot be operated at the same time;

2) Feature corner point extraction of the projected image on the surface of the outer panel of the door in the camera image:

a. Coarse positioning of feature corners based on centroid:

Using the formula for calculating the centroid coordinates, calculate the centroid coordinates separately using the static moment formula for the obtained edge coordinates and gray intensity value pairs of each X-type primitive symbol according to the marking order;

First, the static moment formula is:

${\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; p</span>}\mathrm{<span\; class="notranslate">\; q</span>}}<span\; class="notranslate">\; =</span>{<span\; class="notranslate">\; \&Integral;</span>}_{<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; \&infin;</span>}}^{\mathrm{<span\; class="notranslate">\; \&infin;</span>}}{<span\; class="notranslate">\; \&Integral;</span>}_{<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; \&infin;</span>}}^{\mathrm{<span\; class="notranslate">\; \&infin;</span>}}{\mathrm{<span\; class="notranslate">\; x</span>}}^{\mathrm{<span\; class="notranslate">\; p</span>}}{\mathrm{<span\; class="notranslate">\; the\; y</span>}}^{\mathrm{<span\; class="notranslate">\; q</span>}}\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span>\mathrm{<span\; class="notranslate">\; d</span>}\mathrm{<span\; class="notranslate">\; x</span>}\mathrm{<span\; class="notranslate">\; d</span>}\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 8</span>}<span\; class="notranslate">\; )</span>$

In the formula: x represents the coordinate value of the x-axis direction, y represents the coordinate value of the y-axis direction, and the values of p and q are determined according to the calculated static moments of different coordinate axes. When calculating the static moments of the x-axis, p is 0 and q is 1. When calculating the static moment of the y-axis, p is 1 and q is 0; introducing the static moment formula into image processing can be written as:

${\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; p</span>}\mathrm{<span\; class="notranslate">\; q</span>}}<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{<span\; class="notranslate">\; \&Sigma;</span>}}\underset{\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; l</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{<span\; class="notranslate">\; \&Sigma;</span>}}{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; p</span>}}{\mathrm{<span\; class="notranslate">\; j</span>}}^{\mathrm{<span\; class="notranslate">\; q</span>}}\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 9</span>}<span\; class="notranslate">\; )</span>$

Among them: the product of k and l represents the pixel size of the target image, f(i+1, j+1) represents the grayscale intensity of the pixel at the image coordinates (i, j), since the origin of the image coordinate system is placed at the upper left of the image On the pixel point, the image coordinate (0,0) corresponds to the pixel coordinate (1,1), so the coordinate size of each pixel point needs to be reduced by 1 relative to its pixel position. Here, the edge pixel points in the image are all 1, The remaining pixels are all 0;

Then the centroid coordinates are:

$\begin{array}{cc}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}<span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; 10</span>}}}{{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; 00</span>}}}<span\; class="notranslate">\; ,</span>& {\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}<span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; 01</span>}}}{{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; 00</span>}}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span>\end{array}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 10</span>}<span\; class="notranslate">\; )</span>$

x _c represents the centroid coordinates of the X-type primitive symbol in the x direction;

y _c represents the centroid coordinates of the X-type primitive symbol in the y direction;

When calculating the centroid of the X-type primitive symbol, proceed in order from small to large according to the labels 1-n. When calculating the centroid coordinates of the edge of the primitive symbol labeled a, where: 1≤a≤n, only the label is a The gray value of the edge coordinate pixel is 1, and the rest are all set to zero, so to avoid the influence of other edges on the calculation of the centroid coordinates, similarly perform n times of calculations in turn, and calculate all the X-type primitive symbol edges The centroid coordinates of the image;

b. Fine positioning of feature corners based on Harris

Since the corner point coarse positioning algorithm based on the centroid is based on the corner point coordinates obtained from the edge shape of the X-type primitive symbol, when the X-type primitive symbol is deformed, the obtained corner point coordinates are relative to its real corner point The coordinates will be shifted, so after the rough positioning of the corner points is completed, the results obtained by the rough positioning of the corner points need to be substituted into the original initial image. In the original image, the coordinates obtained by the rough positioning are taken as the center, and the 3× In the neighborhood of 3, the Harris corner point extraction algorithm is used to search according to the gray gradient to find the point with the largest gradient change, and the position coordinates of the point with the largest gray gradient gradient are taken as the fine positioning coordinates to obtain the fine positioning result of the characteristic corner points;

3) Recognize and restore the code word of the X-type primitive symbol in the projected image on the surface of the door outer panel in the camera image:

The codeword is restored by calculating the angle between the longer main inertia axis and the base axis, and the inertia moment integral formula is used to calculate the angle between the longer main inertia axis and the base axis. The inertia moment integral formula is as follows:

${\mathrm{<span\; class="notranslate">\; CM</span>}}_{\mathrm{<span\; class="notranslate">\; p</span>}\mathrm{<span\; class="notranslate">\; q</span>}}<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{<span\; class="notranslate">\; \&Sigma;</span>}}\underset{\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; l</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{<span\; class="notranslate">\; \&Sigma;</span>}}{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; p</span>}}{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; q</span>}}\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 11</span>}<span\; class="notranslate">\; )</span>$

The formula for the angle between the longer main inertial axis and the horizontal coordinate axis is as follows:

$\mathrm{<span\; class="notranslate">\; \&alpha;</span>}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; 2</span>}}\mathrm{<span\; class="notranslate">\; a</span>}\mathrm{<span\; class="notranslate">\; r</span>}\mathrm{<span\; class="notranslate">\; c</span>}\mathrm{<span\; class="notranslate">\; t</span>}\mathrm{<span\; class="notranslate">\; a</span>}\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; (</span>\frac{\mathrm{<span\; class="notranslate">\; 2</span>}{\mathrm{<span\; class="notranslate">\; CM</span>}}_{\mathrm{<span\; class="notranslate">\; 11</span>}}}{{\mathrm{<span\; class="notranslate">\; CM</span>}}_{\mathrm{<span\; class="notranslate">\; 20</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; CM</span>}}_{\mathrm{<span\; class="notranslate">\; 02</span>}}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 12</span>}<span\; class="notranslate">\; )</span>$

The direction of the small arrow on the X-type primitive symbol points to the direction of the longer main inertial axis, and the line between the centroids of two horizontally adjacent X-type primitives is the baseline, because in the projection process, due to There is no guarantee that the horizontal projection will keep the graphics array consistent with the horizontal direction, so there is a certain angle between the baseline and the horizontal direction, and the angle formula is:

$\mathrm{<span\; class="notranslate">\; \&beta;</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; a</span>}\mathrm{<span\; class="notranslate">\; r</span>}\mathrm{<span\; class="notranslate">\; c</span>}\mathrm{<span\; class="notranslate">\; t</span>}\mathrm{<span\; class="notranslate">\; a</span>}\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; (</span>\frac{{\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; no</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}}{{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; no</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 13</span>}<span\; class="notranslate">\; )</span>$

Among them: (x _n , y _n ) represents the characteristic corner point coordinates of the nth X-type primitive symbol, then finally the angle between the longer main inertial axis and the baseline can be obtained as:

Δ = α - β (14)

Set the difference range to ±10°, compare the difference between Δ and {0, 45, 90, 135}, if the difference between the two is within the set error range, then follow the {0 corresponding to the corresponding angle , 1, 2, 3} to restore the codeword, and compare them in turn to obtain the codeword matrix of the detected image.

The three-dimensional matching of the feature corner points of the projection image on the surface of the door outer panel in the camera image and the feature corner points of the coded projection template as described in the technical solution refers to:

After the target image is decoded, a codeword array is obtained, and each codeword represents not only an X-type primitive symbol, but also the characteristic corner point of the X-type primitive symbol, and also represents the characteristic angle The image coordinates of the point, in order to determine the specific position of the X-type primitive symbol in the target image in the projection template, that is, to determine the corresponding position of the X-type primitive symbol in the projection template in the captured image, or the camera image outside the door The corresponding corner positions in the projection template corresponding to the feature corners in the projection image on the surface of the board realize the one-to-one matching between the camera image and the projection template. According to the neighborhood-based spatial encoding strategy, each X-type primitive symbol The code value is composed of 8 neighborhoods of its upper, lower, left, and right sides. Therefore, in order to facilitate stereo matching, it is necessary to calculate the code value of each X-type primitive symbol according to the combination shown in the figure, so that the obtained code value is unique. ;

After obtaining the code value of each corresponding X-type primitive symbol, the circular search algorithm is used to search the original pseudo-random M array. The code value with the highest degree is used as the result of stereo matching, and the corresponding position of each X-type primitive symbol in the detection image in the original coding projection template is determined.

The steps in the space point cloud coordinate calculation and error assessment phase described in the technical proposal are as follows:

1) Calculate the three-dimensional space coordinates of the feature points of the projected image on the surface of the outer panel of the car door

After realizing the one-to-one matching of the feature corners of the camera image and the feature corners of the projection image, using the calibration results of the structured light detection system, the triangulation method is used to calculate the three-dimensional space coordinates of the feature points of the projection image on the surface of the outer panel of the door, where the depth direction The calculation formula of the space coordinate Z _R is:

${\mathrm{<span\; class="notranslate">\; Z</span>}}_{\mathrm{<span\; class="notranslate">\; R</span>}}<span\; class="notranslate">\; =</span>\frac{<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; L</span>}}}<span\; class="notranslate">\; |</span>{<span\; class="notranslate">\; |</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; (</span>{\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{\mathrm{<span\; class="notranslate">\; \&alpha;</span>}}}_{\mathrm{<span\; class="notranslate">\; R</span>}}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; T</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; <</span>{\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{\mathrm{<span\; class="notranslate">\; \&alpha;</span>}}}_{\mathrm{<span\; class="notranslate">\; R</span>}}<span\; class="notranslate">\; ,</span>{\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{\mathrm{<span\; class="notranslate">\; x</span>}}}_{\mathrm{<span\; class="notranslate">\; L</span>}}<span\; class="notranslate">\; ></span><span\; class="notranslate">\; <</span>{\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{\mathrm{<span\; class="notranslate">\; x</span>}}}_{\mathrm{<span\; class="notranslate">\; L</span>}}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; T</span>}<span\; class="notranslate">\; ></span>}{<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>{\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{\mathrm{<span\; class="notranslate">\; \&alpha;</span>}}}_{\mathrm{<span\; class="notranslate">\; R</span>}}<span\; class="notranslate">\; |</span>{<span\; class="notranslate">\; |</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; L</span>}}}<span\; class="notranslate">\; |</span>{<span\; class="notranslate">\; |</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; -</span>{<span\; class="notranslate">\; (</span>{\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{\mathrm{<span\; class="notranslate">\; \&alpha;</span>}}}_{\mathrm{<span\; class="notranslate">\; R</span>}}<span\; class="notranslate">\; ,</span>{\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{\mathrm{<span\; class="notranslate">\; x</span>}}}_{\mathrm{<span\; class="notranslate">\; L</span>}}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 15</span>}<span\; class="notranslate">\; )</span>$

In the formula: etc. represent the dot product operator, where and Represents the left and right image coordinate vectors that complete the stereo matching, R and T represent the rotation matrix and translation vector from the projector coordinate system to the camera coordinate system respectively; use the triangulation method space coordinate calculation formula, and use the structured light system to calibrate the results, Calculate the space coordinates of the feature points on the surface of the car door to obtain a schematic diagram of the detection point cloud;

2) Register the calculated detection point cloud with the point cloud model measured by the three-coordinate measuring machine:

Using the ICP-based registration method, the main steps are divided into two steps: coarse registration and fine registration. The coarse registration method uses Gaussian curvature and average curvature as registration features, and calculates the Gaussian curvature and the average curvature of each point in the detected point cloud. Average curvature, search for the closest point in the point cloud model, set the error conditions, make the detection point cloud and the point cloud model reach a relatively close state, and get the initial registration result; the detection point cloud and the initial matching The point cloud model uses the ICP algorithm for precise registration, which minimizes the shape deviation between the detected point cloud and the point cloud model. The minimum distance objective function used is:

${\mathrm{<span\; class="notranslate">\; \&Delta;</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}}{<span\; class="notranslate">\; \&Sigma;</span>}}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; T</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; \&CenterDot;</span><span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; p</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; no</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; |</span>{<span\; class="notranslate">\; |</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 16</span>}<span\; class="notranslate">\; )</span>$

In the above formula, {p _j (x _j , y _j , z _j )|=1, 2...k}=P _p is the detection point cloud obtained after initial registration, {n _j (x _j , y _j , z _j )|j=1, 2...k}=N _c is the point cloud model, and the final result of point cloud registration is obtained;

After the detection point cloud and the point cloud model achieve the best registration, in order to simplify the iterative process of image processing, the nearest point search algorithm is used to search for the closest point of the registered detection point cloud in the point cloud model, and calculate the distance between the two The distance, which represents the deviation;

In order to express the error of different areas in the detection point cloud more clearly, the error is represented by the color patch map. First, the color index level is set to 64 levels, and the relationship between the point cloud deviation data and the color index is established. The formula is as follows:

$\mathrm{<span\; class="notranslate">\; R</span>}\mathrm{<span\; class="notranslate">\; G</span>}\mathrm{<span\; class="notranslate">\; B</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; l</span>}\mathrm{<span\; class="notranslate">\; e</span>}\mathrm{<span\; class="notranslate">\; v</span>}\mathrm{<span\; class="notranslate">\; e</span>}\mathrm{<span\; class="notranslate">\; l</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; r</span>}\mathrm{<span\; class="notranslate">\; o</span>}\mathrm{<span\; class="notranslate">\; u</span>}\mathrm{<span\; class="notranslate">\; no</span>}\mathrm{<span\; class="notranslate">\; d</span>}<span\; class="notranslate">\; (</span>\frac{\mathrm{<span\; class="notranslate">\; \&Delta;</span>}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; \&Delta;</span>}}_{\mathrm{<span\; class="notranslate">\; m</span>}\mathrm{<span\; class="notranslate">\; i</span>}\mathrm{<span\; class="notranslate">\; no</span>}}}{{\mathrm{<span\; class="notranslate">\; \&Delta;</span>}}_{\mathrm{<span\; class="notranslate">\; max</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; \&Delta;</span>}}_{\mathrm{<span\; class="notranslate">\; min</span>}}}<span\; class="notranslate">\; *</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; L</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 17</span>}<span\; class="notranslate">\; )</span>$

In the formula: Δ represents the obtained deviation value, Δ _max represents the maximum deviation value, Δ _min represents the minimum deviation value, L represents the color index classification, and RGB_level can calculate the color index value of all points in the detected point cloud.

Compared with prior art, the beneficial effects of the present invention are:

1. At present, the three-coordinate measuring machine is mainly used for the detection of the shape and size of the outer surface of the door outer panel. Although the detection accuracy is high, the detection speed is slow, and 100% online detection cannot be achieved, only sampling detection is possible, and contact detection is easy. cause scratches to the surface. The method for detecting the shape and size of the door outer panel based on single-projection encoded structured light in the present invention is based on computer vision technology, and develops a non-contact detection method that can detect the shape and size of moving objects, which can realize automatic operation and detection. The speed is fast, and it can achieve the purpose of 100% online detection of the outer surface of the outer panel of the car door.

2. The method for detecting the shape and size of the door outer panel based on single-projection coded structured light of the present invention uses the structured light vision technology based on the spatial neighborhood coding strategy to detect moving objects, and designs an X-shaped geometric pattern as The projector encoding template of the primitive, when the surface of the detection object is occluded or the depth changes drastically, the projection primitive projected onto the surface of the detection object will be deformed with the change of the surface structure, and the X-type primitive can effectively improve the deformation caused by the primitive The offset phenomenon of the geometric feature point position ensures the accuracy of feature corner point extraction.

3. The structured light system calibration method adopted in the method for detecting the shape and size of the door outer panel based on single-projection coded structured light in the present invention is relatively simple and fast. Only one imaging is required to complete the calibration of the camera and projector, which is convenient for real-time construction and detection System; at the same time, the detection error is represented by the color patch map, which is conducive to the clarity of the detection results.

Description of drawings

Below in conjunction with accompanying drawing, the present invention will be further described:

Fig. 1 is a schematic diagram of the online detection of the surface shape and size of the door outer panel by the car door outer panel shape and size detection system based on single-projection coded structured light used in the present invention;

Figure 2-a is a schematic diagram of camera imaging in the method for detecting the shape and size of the outer panel of a car door based on single-projection coded structured light adopted in the present invention;

Fig. 2-b is a schematic diagram of the projected image of the projector in the method for detecting the shape and size of the outer panel of the car door based on single-projection coded structured light adopted in the present invention;

Figure 3-1 is a schematic diagram of light plane intersection (arbitrary plane) in the method for detecting the shape and size of the outer panel of a car door based on single-projection coded structured light according to the present invention;

Fig. 3-2 is a schematic diagram of the principle of spatial geometric transformation in the method for detecting the shape and size of the door outer panel based on single-projection coded structured light according to the present invention;

Fig. 4 is a schematic diagram of light plane intersection (calibration plane) in the method for detecting the shape and size of the outer panel of the car door based on single-projection coded structured light according to the present invention;

Fig. 5-a is a schematic diagram of a random 3x3 sub-array generated in the method for detecting the shape and size of a door outer panel based on single-projection coded structured light according to the present invention;

Fig. 5-b is a schematic diagram of sequentially adding 3x1 column vectors to the right in the method for detecting the shape and size of the outer panel of a car door based on single-projection coded structured light according to the present invention;

Figure 5-c is a schematic diagram of sequentially adding 1x3 row vectors downward in the method for detecting the shape and size of the outer panel of a car door based on single-projection coded structured light according to the present invention;

Fig. 5-d is a schematic diagram of a new 3x3 sub-array by sequentially adding a random primitive value in the method for detecting the shape and size of the outer panel of a car door based on single-projection coded structured light according to the present invention;

Fig. 6 is the partial array result of the pseudo-random M array corresponding to the code word arrangement rule in the coded projection template used to detect the surface shape and size of the door outer panel in the method for detecting the shape and size of the outer panel of the car door based on single-projection coded structured light according to the present invention schematic diagram;

7 is a schematic diagram of X-type primitive symbols and code word classification rules in the projection template used to detect the surface shape and size of the door outer panel in the method for detecting the shape and size of the outer panel of the car door based on single-projection coded structured light according to the present invention;

Fig. 8-a is a pseudo-random M-array coded projection template used for projecting a projector onto the surface of a car door outer panel to detect the shape and size in the method for detecting the shape and size of a car door outer panel based on single-projection coded structured light according to the present invention;

Figure 8-b is an enlarged schematic diagram of a local area of the rectangular box in Figure 8-a;

9 is a block diagram of the image preprocessing operation for the grayscale image obtained by shooting the projected image of the surface of the door outer panel by using a camera in the method for detecting the shape and size of the outer panel of the door based on single-projection coded structured light according to the present invention;

Fig. 10 is a schematic diagram of performing element segmentation marking and decoding recognition operations on the edge-enhanced image of the X-type primitive symbol on the surface of the door outer panel after preprocessing in the method for detecting the shape and size of the outer panel of the car door based on single-projection coded structured light according to the present invention flow chart;

11 is a schematic diagram of the X-type primitive symbol edge obtained by performing edge extraction on the preprocessed detected image in the method for detecting the shape and size of the outer panel of the car door based on single-projection coded structured light according to the present invention;

Fig. 12-a is the X-shaped primitive symbol edge image of the X-shaped primitive symbol edge image in the method for detecting the shape and size of the door outer panel based on single-projection coded structured light according to the present invention. type primitive symbol edge image;

Figure 12-b is an enlarged schematic diagram of a local area of the rectangular box in Figure 12-a;

Fig. 13 is the detection method for the shape and size of the door outer panel based on single-projection coded structured light according to the present invention. After the edge extraction is performed on the projection image of the door outer panel surface in the camera image, the rough positioning of the feature corners is performed to obtain the local area enlargement of the coarse positioning result. picture;

Fig. 14 shows the projection image of the surface of the door outer panel in the camera image in the method for detecting the shape and size of the outer panel of the car door based on single-projection coded structured light according to the present invention. Harris The enlarged map of the local area of the fine positioning result of the feature corner points obtained by the fine positioning of the feature corner points;

Fig. 15 shows the relationship between the X-type primitive symbol in the local area in the projection image of the surface of the door outer panel in the camera image in the method for detecting the shape and size of the outer panel of the car door based on single-projection coded structured light according to the present invention. Schematic diagram of the included angle;

Fig. 16 is a schematic diagram of the code value combination method for encoding the detected projected image of the surface of the door outer panel and the non-edge primitives in the projection template in the method for detecting the shape and size of the outer door panel based on single-projection encoded structured light according to the present invention;

Fig. 17 is a schematic diagram of a three-dimensional point cloud obtained by detecting the surface shape and size of the door outer panel using the coded structured light vision technology in the method for detecting the shape and size of the outer door panel based on single-projection encoded structured light according to the present invention;

18 is a schematic diagram of a point cloud CAD model obtained by using a three-coordinate measuring machine to detect the door outer panel in the method for detecting the shape and size of the outer panel of the car door based on single-projection coded structured light according to the present invention;

Fig. 19 is obtained by placing the detected three-dimensional point cloud and the point cloud CAD model detected by a three-coordinate measuring machine in the same world coordinate system in the method for detecting the shape and size of the door outer panel based on single-projection coded structured light according to the present invention Schematic diagram of the relative position of the detected point cloud and point cloud CAD model;

Fig. 20-a is a schematic diagram of the results of coarse registration of point clouds using an ICP-based registration algorithm in the method for detecting the shape and size of a door outer panel based on single-projection coded structured light according to the present invention;

Fig. 20-b is a schematic diagram of the results of precise registration of point clouds using an ICP-based registration algorithm in the method for detecting the shape and size of a door outer panel based on single-projection coded structured light according to the present invention;

Fig. 21 shows the method of calculating the distance between the closest points between two point clouds in the method for detecting the shape and size of the door outer panel based on single-projection coded structured light according to the present invention, indicating that the deviation between the detected point cloud and the point cloud CAD model is obtained The statistical distribution map of the deviation;

Fig. 22 is a flow chart of the method for detecting the shape and size of the door outer panel based on single-projection coded structured light according to the present invention;

In the figure: 1. Door outer panel assembly line, 2. Camera, 3. Information processing terminal equipment, 4. Projector,

detailed description

The present invention is described in detail below in conjunction with accompanying drawing:

Referring to Fig. 1, the figure is a simplified schematic diagram of a door outer panel shape and size detection system based on single-projection coded structured light designed by the present invention, wherein the one marked with 1 is the assembly line of the outer panel of the car door, the one marked with 2 is the camera, and the one marked with 3 It is an information processing terminal device, and the one marked with 4 is a projector. When the door outer panel assembly line 1 moves slowly along the track, the projector 4 projects the pattern onto the surface of the door outer panel, and then the camera 2 takes it and transmits it to the information processing The terminal device 3 performs processing, and finally obtains the detection result of the surface of the outer panel of the car door. By comparing the obtained detection result with the point cloud CAD model, it can be known whether the manufacturing accuracy of the surface of the outer panel of the car door meets the production requirements.

During detection, the camera 2 is located 1.5 meters directly in front of the outer panel of the car door. The camera 2 adopts a CMOS model DH-HV1302UM-T produced by Beijing Daheng Image Equipment Co., Ltd. with a resolution of 1280×1024 and a focal length f of 12.5- 75mm CCTV lens; Projector 4 is placed 1.5 meters directly in front of the outer panel of the car door, using a projector model of InfocusLP260, with a standard resolution of 800×600 and a maximum supported resolution of 1024×768. Referring to Figure 2, the calibration plane is a white-bottomed writing board with a length of 2 meters and a width of 1 meter. A 9×9 camera checkerboard target is glued to the lower left corner, and a space is left on the right side for the projection checkerboard target.

The entire detection process of the shape and size of the outer door panel is mainly divided into four stages: 1. The calibration stage of the shape and size detection system of the door outer panel based on single-projection coded structured light (structured light detection system for short); 2. The stage of coded projection template design; 3. Detection image decoding and recognition stage; 4. Spatial point cloud coordinate calculation and error evaluation stage.

1. Calibration stage of structured light detection system

In order to ensure the smooth progress of the detection, it is first necessary to calibrate the two components of the equipment used in this method, that is, the structured light detection system, namely the camera 2 and the projector 4, in order to establish the corresponding nonlinearity between the spatial position and the image coordinates. Using this nonlinear relationship, the spatial coordinates corresponding to the feature points of the detected image can be calculated.

The present invention uses a plane-based calibration method to calibrate the structured light detection system, and takes 12 images in total. The camera 2 uses a 9×9 black and white checkerboard image with a size of 270×270 mm in the lower left corner of the calibration plane as a target, and the projector targets It is a 12×16 black and white checkerboard image projected to the right side of the calibration plane by projector 4.

1) Calibrate the camera

Referring to Figure 2, use Zhang Zhengyou’s calibration method (ZHANGZY, AFlexibleNewTechniqueforCameraCalibration[R].MicrosoftCorporation, NSR-TR-98-71, 1998) for calibration, first paste a 9×9 camera checkerboard target on the lower left corner of the calibration plane and use Projector 4 projects the designed 12×16 checkerboard target to the right side of the calibration plane, and then uses camera 2 to photograph the entire calibration plane, as shown in Figure 2-a. The camera checkerboard target image in the lower left corner of the 12 captured images is subjected to feature corner extraction. Each camera checkerboard target image can extract 100 corner points, and a total of 1200 feature corner image coordinates are obtained from the 12 images. . According to Zhang Zhengyou's calibration method, the world coordinate system is set at a corner point at the upper left corner of the camera checkerboard target image, and the coordinate value in the Z direction is taken as zero. Since the size of each checkerboard is precisely designed to be 30mm, the checkerboard The world coordinates of the feature corners in the X and Y directions can be accurately known. Therefore, after obtaining the coordinates of the characteristic corner points of the checkerboard target image on the calibration plane and the spatial coordinates of the corresponding points, the toolbox can be used to complete the calibration and calculate the internal and external parameters of the camera 2 .

2) Calibrate the projector

Referring to Figure 3-1, first extract the feature corners of the projected target in the camera image, and then calculate the features of the projected image on the calibration plane according to the obtained internal and external parameters of camera 2 according to the light plane intersection method The spatial coordinates of the corner points in the world coordinate system. So far, the image coordinates and spatial coordinates of the projection template have been calculated. Therefore, the calibration parameters of the projector can be calculated by the same method as the camera calibration, and the internal and external dimensions of the projector 4 can be obtained. parameters and the rotation-translation matrix between projector4 and camera2.

Since it is necessary to use the light plane intersection method to calculate the spatial coordinates of the projected image feature corners on the calibration plane on the basis of camera 2 calibration, it is necessary to introduce the light plane intersection method in detail:

Referring to Figure 3-2, according to the principle of spatial geometric transformation, to define a plane in a three-dimensional space, it is only necessary to know any known point on the plane and a non-zero normal vector passing through the point. The third column vector n of the rotation matrix R can be regarded as the normal vector of the calibration plane π passing through the origin of the world coordinate system under the reference coordinate system of camera 2, and the translation matrix can be regarded as the origin p of the world coordinate system in the camera coordinate system The space coordinates below.

Therefore, the equation of any point r on the calibration plane π can be expressed as:

n×(r-p)=0(1)

Formula (1) is in the form of the quantitative product between two vectors, using The representation is a unit vector in each direction of the Cartesian coordinate system, then the normal vector n and the point vector r can be expressed as:

$\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; a</span>}\stackrel{<span\; class="notranslate">\; ^</span>}{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; b</span>}\stackrel{<span\; class="notranslate">\; ^</span>}{\mathrm{<span\; class="notranslate">\; the\; y</span>}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; c</span>}\stackrel{<span\; class="notranslate">\; ^</span>}{\mathrm{<span\; class="notranslate">\; z</span>}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; )</span>$

$\mathrm{<span\; class="notranslate">\; r</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; \&times;</span>\stackrel{<span\; class="notranslate">\; ^</span>}{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; \&times;</span>\stackrel{<span\; class="notranslate">\; ^</span>}{\mathrm{<span\; class="notranslate">\; the\; y</span>}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; z</span>}<span\; class="notranslate">\; \&times;</span>\stackrel{<span\; class="notranslate">\; ^</span>}{\mathrm{<span\; class="notranslate">\; z</span>}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 3</span>}<span\; class="notranslate">\; )</span>$

Let d=-n×p, since n and p are known, so d is also known, the equation of the calibration plane π can be expressed as:

ax+by+cz+d=0, (a, b, c∈R and not all equal to 0) (4)

Using the calibration result of camera 2, the image coordinates of the characteristic corner points of the projector target in the camera image are substituted into the reverse perspective projection transformation matrix, and the corresponding spatial point coordinates are calculated. The formula is as follows:

$\left[\begin{array}{c}\mathrm{<span\; class="notranslate">\; the\; s</span>}{\mathrm{<span\; class="notranslate">\; R</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}\\ {\mathrm{<span\; class="notranslate">\; R</span>}}_{\mathrm{<span\; class="notranslate">\; the\; y</span>}}\\ {\mathrm{<span\; class="notranslate">\; R</span>}}_{\mathrm{<span\; class="notranslate">\; z</span>}}\\ \mathrm{<span\; class="notranslate">\; the\; s</span>}\end{array}\right]<span\; class="notranslate">\; =</span>{<span\; class="notranslate">\; \&lsqb;</span>{\mathrm{<span\; class="notranslate">\; K</span>}}_{\mathrm{<span\; class="notranslate">\; int</span>}}{\mathrm{<span\; class="notranslate">\; K</span>}}_{\mathrm{<span\; class="notranslate">\; e</span>}\mathrm{<span\; class="notranslate">\; x</span>}\mathrm{<span\; class="notranslate">\; t</span>}}<span\; class="notranslate">\; \&rsqb;</span>}^{<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}\left[\begin{array}{c}\mathrm{<span\; class="notranslate">\; x</span>}\\ \mathrm{<span\; class="notranslate">\; the\; y</span>}\\ \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 5</span>}<span\; class="notranslate">\; )</span>$

s represents the scale factor in the above formula, K _int and K _ext represent the internal and external parameters of the camera respectively, and the size of s determines the specific position and depth of the calculated spatial point coordinates in space, as shown in Figure 3, when s At the same time, the position of the plane formed by the spatial point coordinates in space is also different. Therefore, in order to solve the spatial coordinates of the projected image feature points on the calibration plane, it is necessary to determine a scale factor s, so that the spatial points calculated by formula (5) are located on the calibration plane, as shown in Figure 4. After the calculation, the specific scale factor s In the case of , the spatial points are all located on the calibration plane, and the coordinates of the spatial points obtained at this time are the world coordinates of the feature corner points of the projected image on the calibration plane. Therefore, substituting the space coordinates (sR _x , sR _y , sR _z ) into the plane equation (4), the obtained formula is as follows:

a(sR _x )+b(sR _y )+c(sR _z )+d=0(6)

In the above formula, a, b, c, and d are all known numbers, and R _x , R _y , and R _z can be obtained by using the perspective projection transformation formula from the calibration result of the camera 2 . After obtaining the spatial coordinates of the feature corners of the projection image on the calibration plane, and then extracting the feature corners of the projection target projected by the projector, all the two-dimensional image coordinates and corresponding three-dimensional coordinates required for calibration can be obtained. The projector 4 can be calibrated by using the "reverse camera" calibration method.

According to the calibration results of camera 2 and projector, calculate the back-projection error. The average error of camera 2 calibration in the x direction is 0.17827pixel, and the average error of camera 2 calibration in the y direction is 0.16199pixel; the projector is calibrated in the x direction The average error is 0.38383pixel, and the average error of the projector calibration in the y direction is 0.4555pixel. The rotation-translation matrix between camera 2 and projector 4 is:

$\begin{array}{cc}\mathrm{<span\; class="notranslate">\; R</span>}<span\; class="notranslate">\; =</span>\left[\begin{array}{ccc}\mathrm{<span\; class="notranslate">\; 0.9711</span>}& \mathrm{<span\; class="notranslate">\; 0.0353</span>}& <span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 0.2362</span>}\\ <span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 0.0033</span>}& \mathrm{<span\; class="notranslate">\; 0.9909</span>}& \mathrm{<span\; class="notranslate">\; 0.1345</span>}\\ \mathrm{<span\; class="notranslate">\; 0.2388</span>}& <span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 0.1298</span>}& \mathrm{<span\; class="notranslate">\; 0.9624</span>}\end{array}\right]& \mathrm{<span\; class="notranslate">\; T</span>}<span\; class="notranslate">\; =</span>\left[\begin{array}{c}\mathrm{<span\; class="notranslate">\; 326.3736</span>}\\ <span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 413.8772</span>}\\ <span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 558.2592</span>}\end{array}\right]\end{array}$

2. Code projection template design stage

In order to meet 100% online detection and realize the detection of dynamic objects, the method for detecting the shape and size of the outer panel of the car door based on single-projection coded structured light in the present invention adopts the structured light vision technology based on the neighborhood coding strategy. During the detection process, only one coding pattern needs to be projected. The coding pattern is a graphics array with a size of 40 rows and 45 columns consisting of 4 X-shaped primitive symbols in different directions. The arrangement order of graphics in the graphics array is determined by a pseudo The random M array determines that the pattern array has the following characteristics: the pattern in each 3×3 subarray appears only once in the entire array template, and at least 3 patterns in corresponding positions in any two subarrays are different. Each X-shaped primitive symbol except the edge is coded, and its coded value size is jointly determined by the elements in its 3×3 neighborhood. The graphic array is given different codewords according to the angle between the longer main inertial axis and the base axis of the X-type primitive symbol, so the graphic array corresponds to a 40×45 digital array, which is called a pseudo-random M Array, the characteristics of the pseudo-random M array determine the characteristics of the graphic array, and the coding projection template designed by the present invention is beneficial to the detection of dynamic objects, and has better adaptability and stability.

1) Pseudo-random M array generation corresponding to the graphic array

The characteristic of the pseudo-random M array is that the sub-array of a specific window size (3×3) is unique in the entire array, and the greater the Hamming distance between any two windows, the greater the difference between them. The calculation method of the pseudo-random M array Hamming distance is as follows:

Among them: V _(ij) represents the 9-element vector of the 3×3 neighborhood, and H represents the number of times that different codewords appear in the corresponding position vectors between two different 3×3 neighborhoods, which is the Hamming distance. The larger the Hamming distance, Indicates the greater the degree of window dissimilarity. In the pseudo-random M array constructed by the present invention, the minimum Hamming distance is set to 3, which means that at least 3 elements corresponding to any two sub-arrays of 3×3 windows are different, and such a design is conducive to enhancing The stability of the sub-array and its adaptability to the degree of spatial variation of the surface of the detection object.

Referring to Fig. 5-a to Fig. 5-d and Fig. 6, the present invention needs to generate a pseudo-random M array whose digital primitives are {0, 1, 2, 3}, the minimum Hamming distance is equal to 3, and the size is 40×45; The construction of the pseudo-random array that meets the requirements of the present invention can be completed by using the circular filling method shown in the figure. First, a 3×3 subarray is generated in the upper left corner of the array, and then a 3×1 subarray is sequentially substituted along the right side, and the adjacent 3 subarrays are compared. Whether the Hamming distance between the ×3 windows meets the conditions, if not, re-substitute, if the conditions are met, proceed to the right until the boundary; then follow the bottom of the initial 3×3 array and substitute into the 1×3 sub-array, the method is the same as above, until to the boundary; secondly, substitute a new code value in the blank position of 4×4 to form a new 3×3 subarray, and compare the Hamming distance with all the subarrays generated before. Substituting, if all possible codewords are not satisfied, restart the construction process. Figure 6 shows the results of partial arrays with the minimum Hamming distance equal to 3 obtained by the present invention using the methods shown in Figure 5-a to Figure 5-d.

2) X-shaped primitive symbol design based on geometric features

Generally speaking, the geometric primitives for constructing coding projection templates often use squares or circles, but due to shadows or occlusions, circular or square patterns with geometric centers as feature corners are more likely to have geometric center deviations Or ambiguous corner points are generated, resulting in inaccurate extraction of corner point coordinates. However, according to the corner feature of the checkerboard, the present invention designs a class of X-shaped geometric primitives as shown in FIG. The X-shaped primitive symbol is characterized by a checkerboard image, is symmetrical about the center of the characteristic corners, and is composed of triangles or squares that are symmetrical to the center. According to the angles {0°, 45°, 90°, 135°} between the longer main inertial axis and the base axis, the four codewords {0, 1, 2, 3} are respectively corresponding.

According to the correspondence between the above-mentioned corresponding codewords and graphics, and according to the obtained pseudo-random M array template, the X-type primitive symbols are sequentially substituted into the graphics array template, and a size of 768×1024 is generated as shown in Figure 8 Encoding projection template, the size of a single X-type primitive symbol is 10×10pixel, and the interval between X-type primitive symbols is set to 9pixel. The coded projection template uses black as the background and white as the background color of the X-shaped primitive symbol to generate a single-color coded projection template, which is beneficial to reduce the degree of reflection on the surface of the outer panel of the car door.

3. Detection image decoding recognition stage

After completing the calibration of the structured light detection system and the design of the coded projection template, the detection can be started; first, the coded projection template is connected to the projector 4 through the computer, and the projector 4 is used to project the surface of the car door outer panel to the surface of the car door outer panel. Encode the projection template, and then use the camera 2 to shoot the surface image of the door outer panel from a suitable angle, and transmit the captured image to a personal computer for a series of image processing and recognition processes.

1) Perform image preprocessing on the initial detection image of the surface of the door outer panel in the camera image

Referring to Figure 9, since the image captured by camera 2 has many useless backgrounds, and the image is easily affected by the reflection and noise on the surface of the outer panel of the car door, it is necessary to eliminate the useless background to reduce the time and complexity of image processing, and adjust the reflection of the image And noise reduction processing is carried out to the noise, and the steps of image preprocessing are carried out to the initial detection image of the surface of the outer panel of the car door captured by the camera 2 as follows:

(1) Take a mask operation to the target area image in the camera image, obtain the local area image of the coded projection template designed for the projection of the outer panel surface of the car door in the camera image, and remove the useless background area;

(2) Gaussian filtering is performed on the obtained target area image to remove noise;

(3) Calculate the average gray value of the image background in the target area, and subtract it from the gray value of the original image to remove the background to make the background gray uniform, avoid the gray gradient change in the background image, and affect the edge extraction results , leading to the extraction of too many non-X-type primitive symbol edges, which will interfere with the segmentation mark;

(4) In order to make the edge of the X-type primitive symbol in the image of the target area more prominent, the Laplacian 5-neighborhood edge enhancement method is used to enhance the edge of the X-type primitive symbol in the image of the target area to facilitate edge extraction.

2) Carry out primitive segmentation and pattern recognition on the projected image of the surface of the door outer panel in the camera image

Referring to Fig. 10, since in the definition process of the X-type primitive symbol, what we adopt is to utilize the difference of the included angle between the long main inertial axis and the base axis of the X-type primitive symbol to classify and define the code words , so we need to use the same method to identify the X-type primitive symbols in the camera image. First, we need to extract the X-type primitive symbol edge from the preprocessed image of the target area, and then use the extracted edge point coordinates of the X-type primitive symbol to calculate the centroid coordinates of the X-type primitive symbol. The centroid coordinates are used as the result of rough extraction of corner points. On the basis of the centroid coordinates, the gray gradient in its neighborhood on the original image is calculated, and the gradient maximum point is taken as the feature point. Since the projected image and the horizontal direction cannot be kept completely horizontal during the projection process, when calculating the angle between the longer main inertial axis and the base axis, only the edge of the X-type primitive symbol can be calculated using the formula for the moment of inertia The angle between the longer main inertial axis and the horizontal axis, and then use the coordinates of the characteristic corner points between two adjacent X-type primitive symbols to calculate the angle between the base axis and the horizontal axis, and then the two By subtracting the included angles, the included angle between the longer main inertial axis and the base axis can be obtained. The specific steps are as follows.

(1) X-type primitive symbol edge extraction and segmentation marking

Since the image of the local target area with the coded projection template projected on the surface of the door outer panel has been obtained, and the image preprocessing has been performed, the edge extraction of the local target area image can be directly performed using the canny operator, and all X-type primitives can be obtained The edge of the symbol and a large number of non-primitive edges, the image after edge extraction is a binary image, the edge part intensity value is 1, and the non-edge intensity value is 0; due to the existence of a large number of non-primitive symbol edges, the corner calculation and When calculating the angle of the longer main inertial axis, since the coordinate positions of the edges of all primitive symbols are used, if there are more edges of non-primitive symbols, multiple non-primitive symbol feature corners and corners will appear. The point extraction is inaccurate, so it needs to be eliminated; and because the feature corner coordinates of the X-type primitive symbol edge image and the restoration of the codeword of the primitive symbol need to be calculated separately for each primitive symbol edge, making Different primitive symbols remain relatively independent and do not affect each other in the calculation process. Therefore, it is necessary to use a segmentation labeling algorithm for each primitive symbol to classify and label different primitive symbols, and perform calculations according to the different labels. Primitive symbols with different labels cannot be operated on at the same time.

Referring to Figure 11, the 8-neighborhood region growing algorithm is used to search the image from left to right and from top to bottom to ensure that all independently connected regions are marked and segmented so that they can be operated independently without affecting each other. Due to the existence of a large number of non-primitive symbol edges (noise points or overexposed areas), the number of marks obtained after preliminary marking is 1810, and the same marked non-primitive symbol edges and noise points need to be eliminated. Considering that the shape and size of the edge of the primitive symbol are basically the same, that is, the number of edge pixels is basically the same, but there are two extremes in the noise point and the overexposed area. The noise point is mostly fragmented single or a few pixels, while the strong light The area contains more edge pixels, so the average threshold method is used to calculate the number of pixels in all marked areas, and 2/3 and 4/3 of the average number are taken as the upper and lower limits to eliminate noise points or exposure areas. The influence of the X-type primitive symbol edge extraction results, while re-marking and segmenting the X-type primitive symbol edge regions that meet the requirements, the labels are 1-n, and the final marking results are 1713. After removing the non-primitive edge area, the result shown in Figure 12-a is obtained. Although some X-shaped primitive symbol areas are also mistakenly eliminated, the proportion is very small and does not affect the overall effect. Figure 12- b shows the enlarged view of the X-type primitive symbol edge point detection.

(2) Feature corner point extraction of the projected image on the surface of the outer panel of the door in the camera image

To extract the feature corners of the edge image of the X-type primitive symbol obtained after marking, it is necessary to separately calculate the coordinates of the feature corners for each X-type primitive symbol edge area, and the steps are mainly divided into two steps:

a. Coarse positioning of feature corners based on centroid

Using the centroid coordinates calculation formula, the obtained edge coordinates and gray intensity values (1 or 0) of each X-type primitive symbol are used to calculate the centroid coordinates separately according to the marking order using the static moment formula;

First, the static moment formula is:

${\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; p</span>}\mathrm{<span\; class="notranslate">\; q</span>}}<span\; class="notranslate">\; =</span>{<span\; class="notranslate">\; \&Integral;</span>}_{<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; \&infin;</span>}}^{\mathrm{<span\; class="notranslate">\; \&infin;</span>}}{<span\; class="notranslate">\; \&Integral;</span>}_{<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; \&infin;</span>}}^{\mathrm{<span\; class="notranslate">\; \&infin;</span>}}{\mathrm{<span\; class="notranslate">\; x</span>}}^{\mathrm{<span\; class="notranslate">\; p</span>}}{\mathrm{<span\; class="notranslate">\; the\; y</span>}}^{\mathrm{<span\; class="notranslate">\; q</span>}}\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span>\mathrm{<span\; class="notranslate">\; d</span>}\mathrm{<span\; class="notranslate">\; x</span>}\mathrm{<span\; class="notranslate">\; d</span>}\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 8</span>}<span\; class="notranslate">\; )</span>$

In the formula (7), x represents the coordinate value of the x-axis direction, y represents the coordinate value of the y-axis direction, and the values of p and q are determined according to the calculated static moments of different coordinate axes. When calculating the static moments of the x-axis, p is 0, q is 1, p is 1 and q is 0 when calculating the y-axis static moment. Introducing the static moment formula into image processing can be written as:

${\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; p</span>}\mathrm{<span\; class="notranslate">\; q</span>}}<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{<span\; class="notranslate">\; \&Sigma;</span>}}\underset{\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; l</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{<span\; class="notranslate">\; \&Sigma;</span>}}{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; p</span>}}{\mathrm{<span\; class="notranslate">\; j</span>}}^{\mathrm{<span\; class="notranslate">\; q</span>}}\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 9</span>}<span\; class="notranslate">\; )</span>$

The product of k and l in the above formula represents the pixel size of the target image, and f(i+1, j+1) represents the grayscale intensity of the pixel at the image coordinates (i, j), since the origin of the image coordinate system is placed at the upper left of the image On the pixel, the image coordinate (0,0) corresponds to the pixel coordinate (1,1), so the coordinate size of each pixel needs to be reduced by 1 relative to its pixel position. In the image here, the edge pixels are all 1, and the rest of the pixels are 0;

Then the centroid coordinates can be:

$\begin{array}{cc}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}<span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; 10</span>}}}{{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; 00</span>}}}<span\; class="notranslate">\; ,</span>& {\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}<span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; 01</span>}}}{{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; 00</span>}}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span>\end{array}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 10</span>}<span\; class="notranslate">\; )</span>$

x _c represents the centroid coordinates of the X-type primitive symbol in the x direction;

y _c represents the centroid coordinates of the X-type primitive symbol in the y direction;

When calculating the centroid of an X-type primitive symbol, proceed according to the labels 1-n from small to large, and when calculating the centroid coordinates of the edge of an X-type primitive symbol labeled a (1≤a≤n), only the label The gray value of the edge coordinate pixel of a is 1, and the rest are all set to zero, so the influence of other edges on the calculation of the centroid coordinates can be avoided. In the same way, n times of calculations can be performed sequentially to obtain all X-type The centroid coordinates of the primitive symbol edge image.

b. Fine positioning of feature corners based on Harris

Since the corner point coarse positioning algorithm based on the centroid is based on the corner point coordinates obtained from the edge shape of the X-type primitive symbol, when the X-type primitive symbol is deformed, the obtained corner point coordinates are relative to its real corner point The coordinates will be slightly shifted, so after the rough positioning of the corner points is completed, the results of the rough positioning of the corner points need to be substituted into the original initial image. In the original image, the coordinates obtained by the rough positioning are taken as the center, and the In the domain (3×3), the Harris corner point extraction algorithm is used to search according to the gray gradient to find the point with the largest gradient change, and the position coordinates of the point with the largest gray gradient gradient are taken as the fine positioning coordinates. The result of fine positioning of the characteristic corner points is shown in Figure 14 As shown, the coordinate values of the two are placed in the original initial image for comparison, and the result proves that the result obtained by fine positioning is more accurate.

(3) Recognize and restore the codewords of the X-type primitive symbols in the projected image on the surface of the outer panel of the car door in the camera image

Since in the design process, the X-type primitive symbol is used to classify the codewords by the angle between the longer main inertial axis and the base axis, so according to this correspondence, it is also necessary to calculate the longer main inertia The angle between the axis and the base axis restores the codeword. Generally speaking, the inertial moment integral formula is often used to calculate the angle between the longer main inertial axis and the base axis. The inertial moment integral formula is as follows:

${\mathrm{<span\; class="notranslate">\; cm</span>}}_{\mathrm{<span\; class="notranslate">\; p</span>}\mathrm{<span\; class="notranslate">\; q</span>}}<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{<span\; class="notranslate">\; \&Sigma;</span>}}\underset{\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; l</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{<span\; class="notranslate">\; \&Sigma;</span>}}{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; p</span>}}{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; q</span>}}\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 11</span>}<span\; class="notranslate">\; )</span>$

In the formula: i represents the coordinates of an edge point of a certain primitive area in the x direction, j represents the coordinates in the y direction, f represents the pixel intensity value (0 or 1), x _c and y _e represent the shape heart coordinates;

The formula for the angle between the longer main inertial axis and the horizontal coordinate axis is as follows:

$\mathrm{<span\; class="notranslate">\; \&alpha;</span>}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; 2</span>}}\mathrm{<span\; class="notranslate">\; a</span>}\mathrm{<span\; class="notranslate">\; r</span>}\mathrm{<span\; class="notranslate">\; c</span>}\mathrm{<span\; class="notranslate">\; t</span>}\mathrm{<span\; class="notranslate">\; a</span>}\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; (</span>\frac{\mathrm{<span\; class="notranslate">\; 2</span>}{\mathrm{<span\; class="notranslate">\; CM</span>}}_{\mathrm{<span\; class="notranslate">\; 11</span>}}}{{\mathrm{<span\; class="notranslate">\; CM</span>}}_{\mathrm{<span\; class="notranslate">\; 20</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; CM</span>}}_{\mathrm{<span\; class="notranslate">\; 02</span>}}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 12</span>}<span\; class="notranslate">\; )</span>$

Referring to Figure 15, the direction of the small arrow on the X-type primitive symbol points to the direction of the longer main inertial axis, and in the first row in the figure, from left to right, two adjacent X-type primitive symbols are connected by a line The connection line of the characteristic corners of is the baseline. Since the horizontal projection cannot be guaranteed during the projection process, the graphic array is consistent with the horizontal direction. Therefore, there is a certain angle between the baseline and the horizontal direction. The angle formula is:

$\mathrm{<span\; class="notranslate">\; \&beta;</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; a</span>}\mathrm{<span\; class="notranslate">\; r</span>}\mathrm{<span\; class="notranslate">\; c</span>}\mathrm{<span\; class="notranslate">\; t</span>}\mathrm{<span\; class="notranslate">\; a</span>}\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; (</span>\frac{{\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; no</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}}{{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; no</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 13</span>}<span\; class="notranslate">\; )</span>$

Where (x _n , y _n ) represents the characteristic corner point coordinates of the nth X-type primitive symbol, then finally the angle between the longer main inertial axis and the baseline can be obtained as:

Δ = α - β (14)

Set the difference range to ±10°, compare the difference between Δ and {0, 45, 90, 135}, if the difference between the two is within the set error range, then follow the {0 corresponding to the corresponding angle , 1, 2, 3} to restore the codeword, and compare them in turn to obtain the codeword matrix of the detected image.

3) Stereo matching of the feature corners of the projected image on the surface of the door outer panel and the feature corners of the coded projection template in the camera image

Referring to Figure 16, after the target image is decoded, a codeword array can be obtained, each codeword represents not only an X-type primitive symbol, but also a characteristic corner of the X-type primitive symbol, It also represents the image coordinates of the feature corners. In order to determine the specific position of the X-type primitive symbol in the target image in the projection template, that is to determine the corresponding position of the X-type primitive symbol in the projection template in the captured image, or the features in the projected image of the surface of the door outer panel in the camera image The corresponding corner positions of the corner points in the projection template realize the one-to-one matching between the camera image and the projection template. According to the neighborhood-based spatial coding strategy, the code value of each X-type primitive symbol is composed of its 8 neighborhoods. Therefore, in order to facilitate stereo matching, it is necessary to calculate each The code value of the X-type primitive symbol, and the code value obtained in this way is unique.

After obtaining the code value of each corresponding X-type primitive symbol, the circular search algorithm is used to search the original pseudo-random M array. The code value with the highest degree is used as the result of stereo matching, and the corresponding position of each X-type primitive symbol in the detection image in the original coding projection template is determined.

4. In the camera image, the space point cloud coordinate calculation and error evaluation stage corresponding to the feature corner points of the projected image on the surface of the outer panel of the door

1) Calculate the three-dimensional space coordinates of the feature points of the projected image on the surface of the outer panel of the car door

Referring to Figure 17, after one-to-one matching of the feature corners of the camera image and the feature corners of the projected image, the three-dimensional space coordinates of the feature points of the projected image on the surface of the door outer panel are calculated using the triangulation method using the calibration results of the structured light detection system , where the calculation formula of the spatial coordinate Z _R in the depth direction is:

${\mathrm{<span\; class="notranslate">\; Z</span>}}_{\mathrm{<span\; class="notranslate">\; R</span>}}<span\; class="notranslate">\; =</span>\frac{<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; L</span>}}}<span\; class="notranslate">\; |</span>{<span\; class="notranslate">\; |</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; (</span>{\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{\mathrm{<span\; class="notranslate">\; \&alpha;</span>}}}_{\mathrm{<span\; class="notranslate">\; R</span>}}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; T</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; <</span>{\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{\mathrm{<span\; class="notranslate">\; \&alpha;</span>}}}_{\mathrm{<span\; class="notranslate">\; R</span>}}<span\; class="notranslate">\; ,</span>{\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{\mathrm{<span\; class="notranslate">\; x</span>}}}_{\mathrm{<span\; class="notranslate">\; L</span>}}<span\; class="notranslate">\; ></span><span\; class="notranslate">\; <</span>{\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{\mathrm{<span\; class="notranslate">\; x</span>}}}_{\mathrm{<span\; class="notranslate">\; L</span>}}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; T</span>}<span\; class="notranslate">\; ></span>}{<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>{\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{\mathrm{<span\; class="notranslate">\; \&alpha;</span>}}}_{\mathrm{<span\; class="notranslate">\; R</span>}}<span\; class="notranslate">\; |</span>{<span\; class="notranslate">\; |</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; L</span>}}}<span\; class="notranslate">\; |</span>{<span\; class="notranslate">\; |</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; -</span>{<span\; class="notranslate">\; (</span>{\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{\mathrm{<span\; class="notranslate">\; \&alpha;</span>}}}_{\mathrm{<span\; class="notranslate">\; R</span>}}<span\; class="notranslate">\; ,</span>{\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{\mathrm{<span\; class="notranslate">\; x</span>}}}_{\mathrm{<span\; class="notranslate">\; L</span>}}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 15</span>}<span\; class="notranslate">\; )</span>$

In the formula etc. represent the dot product operator, where and Represents the left and right image coordinate vectors for stereo matching, and R and T represent the rotation matrix and translation vector from the projector coordinate system to the camera coordinate system, respectively. The space coordinate calculation formula of the triangulation method is used, and the calibration results of the structured light system are used to calculate the space coordinates of the feature points on the surface of the car door, and the detection point cloud diagram shown in the figure is obtained.

2) Register the calculated detection point cloud with the point cloud model measured by the three-coordinate measuring machine

Referring to Fig. 17 and Fig. 18, the point cloud calculated in the present invention is shown in Fig. 17, and the point cloud model of a door outer panel surface detected by a three-coordinate measuring machine is shown in Fig. 18. Since the coordinates of the calculated point cloud and the point cloud model obtained by the three-coordinate measuring machine are not uniform, the two do not overlap after being placed in the same coordinate system, as shown in Figure 19, so registration is required for accuracy evaluation . The present invention adopts a registration method based on ICP (IterativeClosestAlgorithm), and its main steps are divided into two steps: coarse registration and fine registration. The coarse registration method uses Gaussian curvature and average curvature as registration features to calculate and detect the For the Gaussian curvature and average curvature of each point, search for the closest point in the point cloud model, and set the error conditions so that the detection point cloud and the point cloud model are relatively close. The initial registration results are shown in Figure 20- a. The ICP algorithm is used for precise registration of the detected point cloud and the point cloud model after the initial matching, so that the shape deviation between the detected point cloud and the point cloud model is minimized, and the minimum distance objective function adopted is:

${\mathrm{<span\; class="notranslate">\; \&Delta;</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}}{<span\; class="notranslate">\; \&Sigma;</span>}}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; T</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; \&CenterDot;</span><span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; p</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; no</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; |</span>{<span\; class="notranslate">\; |</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 16</span>}<span\; class="notranslate">\; )</span>$

In the above formula, {p _j (x _j , y _j , z _j )|=1, 2...k}=P _p is the detection point cloud obtained after initial registration, {n _j (x _j , y _j , z _j )|j=1, 2...k}=N _c is the point cloud model, and the final result of point cloud registration is shown in Figure 20-b.

After achieving the best registration between the detected point cloud and the point cloud model, in order to simplify the iterative process of image processing, the nearest point search algorithm is used to search for the closest point of the registered detected point cloud in the point cloud model, and calculate the distance between the two The distance is used to represent the deviation, as shown in Figure 21 is the distribution map of the deviation value domain and the frequency diagram of the deviation value domain.

In order to express more clearly the size of errors in different regions of the detected point cloud, the color patch map is used to represent the errors in the present invention. First, the color index level is set to 64 levels, and the relationship between the point cloud deviation data and the color index is established. The relationship between them is as follows:

$\mathrm{<span\; class="notranslate">\; R</span>}\mathrm{<span\; class="notranslate">\; G</span>}\mathrm{<span\; class="notranslate">\; B</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; l</span>}\mathrm{<span\; class="notranslate">\; e</span>}\mathrm{<span\; class="notranslate">\; v</span>}\mathrm{<span\; class="notranslate">\; e</span>}\mathrm{<span\; class="notranslate">\; l</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; r</span>}\mathrm{<span\; class="notranslate">\; o</span>}\mathrm{<span\; class="notranslate">\; u</span>}\mathrm{<span\; class="notranslate">\; no</span>}\mathrm{<span\; class="notranslate">\; d</span>}<span\; class="notranslate">\; (</span>\frac{\mathrm{<span\; class="notranslate">\; \&Delta;</span>}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; \&Delta;</span>}}_{\mathrm{<span\; class="notranslate">\; m</span>}\mathrm{<span\; class="notranslate">\; i</span>}\mathrm{<span\; class="notranslate">\; no</span>}}}{{\mathrm{<span\; class="notranslate">\; \&Delta;</span>}}_{\mathrm{<span\; class="notranslate">\; max</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; \&Delta;</span>}}_{\mathrm{<span\; class="notranslate">\; min</span>}}}<span\; class="notranslate">\; *</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; L</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 17</span>}<span\; class="notranslate">\; )</span>$

In formula 16: Δ represents the obtained deviation value, Δ _max represents the maximum deviation value, Δ _min represents the minimum deviation value, L represents the color index classification, and RGB_level can calculate the color index value of all points in the detected point cloud.

Example

1. Calibration stage of structured light detection system

1. Camera calibration

Referring to Figure 2, use Zhang Zhengyou’s calibration method (ZHANGZY, AFlexibleNewTechniqueforCameraCalibration[R].MicrosoftCorporation, NSR-TR-98-71, 1998) for calibration, first paste a 9×9 camera checkerboard target on the lower left corner of the calibration plane and use The projector 4 projects the designed 12×16 checkerboard target to the right side of the calibration plane, and then uses the camera 2 to shoot the entire calibration plane, as shown in Figure 2-a. The camera checkerboard target image in the lower left corner of the 12 captured images is subjected to feature corner extraction. Each camera checkerboard target image can extract 100 corner points, and a total of 1200 feature corner image coordinates are obtained from the 12 images. . According to Zhang Zhengyou's calibration method, the world coordinate system is set at a corner point at the upper left corner of the camera checkerboard target image, and the coordinate value in the Z direction is taken as zero. Since the size of each checkerboard is precisely designed to be 30mm, the checkerboard The world coordinates of the feature corners in the X and Y directions can be accurately known. Therefore, after obtaining the coordinates of the characteristic corners of the checkerboard target image on the calibration plane and the spatial coordinates of the corresponding points, the toolbox can be used to complete the calibration, and the internal parameters _{Kc int} and nonlinear distortion parameters kc of the camera 2 are calculated as follows:

${\mathrm{<span\; class="notranslate">\; k</span>}}_{\mathrm{<span\; class="notranslate">\; int</span>}}<span\; class="notranslate">\; =</span>\left[\begin{array}{ccc}\mathrm{<span\; class="notranslate">\; 2691.6164</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 639.5</span>}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 2669.1156</span>}& \mathrm{<span\; class="notranslate">\; 511.5</span>}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]$

k _c = [-0.325021.561770.00075-0.008760.00000]

2) Projector calibration

Referring to Figure 3-1, first extract the feature corners of the projected target in the camera image, and then calculate the features of the projected image on the calibration plane according to the obtained internal and external parameters of camera 2 according to the light plane intersection method The spatial coordinates of the corner points in the world coordinate system. So far, the image coordinates and spatial coordinates of the projection template have been calculated. Therefore, the calibration parameters of the projector can be calculated by the same method as the camera calibration, and the internal parameters of the projector 4 can be obtained. And the rotation-translation matrix between projector4 and camera2. Wherein the internal parameter K _p-int and the nonlinear distortion parameter k _p of the projector 4 are as follows:

${\mathrm{<span\; class="notranslate">\; K</span>}}_{\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; int</span>}}<span\; class="notranslate">\; =</span>\left[\begin{array}{ccc}\mathrm{<span\; class="notranslate">\; 2429.6767</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 473.275</span>}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 2416.7672</span>}& \mathrm{<span\; class="notranslate">\; 762.576</span>}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]$

_kp = [0.092470.074290.01070-0.001770]

The rotation matrix R and translation matrix T between the projector 4 and the camera 2 are as follows:

$\begin{array}{cc}\mathrm{<span\; class="notranslate">\; R</span>}<span\; class="notranslate">\; =</span>\left[\begin{array}{ccc}\mathrm{<span\; class="notranslate">\; 0.9711</span>}& \mathrm{<span\; class="notranslate">\; 0.0353</span>}& <span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 0.2362</span>}\\ <span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 0.0033</span>}& \mathrm{<span\; class="notranslate">\; 0.9909</span>}& \mathrm{<span\; class="notranslate">\; 0.1345</span>}\\ \mathrm{<span\; class="notranslate">\; 0.2388</span>}& <span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 0.1298</span>}& \mathrm{<span\; class="notranslate">\; 0.9624</span>}\end{array}\right]& \mathrm{<span\; class="notranslate">\; T</span>}<span\; class="notranslate">\; =</span>\left[\begin{array}{c}\mathrm{<span\; class="notranslate">\; 326.3736</span>}\\ <span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 413.8772</span>}\\ <span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 558.2592</span>}\end{array}\right]\end{array}$

2. Code projection template design stage

1) Pseudo-random M array generation corresponding to the graphic array

In the pseudo-random M array constructed by the present invention, the minimum Hamming distance is set to 3, which means that at least 3 elements corresponding to any two sub-arrays of 3×3 windows are different, and such a design is conducive to enhancing The stability of the sub-array and its adaptability to the degree of spatial variation of the surface of the detection object.

Referring to Fig. 5-a to Fig. 5-d and Fig. 6, the present invention needs to generate a pseudo-random M array whose digital primitives are {0, 1, 2, 3}, the minimum Hamming distance is equal to 3, and the size is 40×45; The construction of the pseudo-random array that meets the requirements of the present invention can be completed by using the circular filling method shown in the figure. First, a 3×3 subarray is generated in the upper left corner of the array, and then a 3×1 subarray is sequentially substituted along the right side, and the adjacent 3 subarrays are compared. Whether the Hamming distance between the ×3 windows meets the conditions, if not, re-substitute, if the conditions are met, proceed to the right until the boundary; then follow the bottom of the initial 3×3 array and substitute into the 1×3 sub-array, the method is the same as above, until to the boundary; secondly, substitute a new code value in the blank position of 4×4 to form a new 3×3 subarray, and compare the Hamming distance with all the subarrays generated before. Substituting, if all possible codewords are not satisfied, restart the construction process. Figure 6 shows the results of partial arrays with the minimum Hamming distance equal to 3 obtained by the present invention using the methods shown in Figure 5-a to Figure 5-d.

2) X-shaped primitive symbol design based on geometric features

According to the corner features of the checkerboard, the present invention designs a class of X-shaped geometric primitives as shown in FIG. 7 to have relatively stable corner features in the case of occlusion or loss. The X-shaped primitive symbol is characterized by a checkerboard image, is symmetrical about the center of the characteristic corners, and is composed of triangles or squares that are symmetrical to the center. According to the angles {0°, 45°, 90°, 135°} between the longer main inertial axis and the base axis, the four codewords {0, 1, 2, 3} are respectively corresponding.

According to the correspondence between the above-mentioned corresponding codewords and graphics, and according to the obtained pseudo-random M array template, the X-type primitive symbols are sequentially substituted into the graphics array template, and a size of 768×1024 is generated as shown in Figure 8 Encoding projection template, the size of a single X-type primitive symbol is 10×10pixel, and the interval between X-type primitive symbols is set to 9pixel. The coded projection template uses black as the background and white as the background color of the X-shaped primitive symbol to generate a single-color coded projection template, which is beneficial to reduce the degree of reflection on the surface of the outer panel of the car door.

3. Detection image decoding and recognition stage

1) Perform image preprocessing on the initial detection image of the surface of the door outer panel in the camera image

(1) Take a mask operation to the target area image in the camera image, obtain the local area image of the coded projection template designed for the projection of the outer panel surface of the car door in the camera image, and remove the useless background area;

(2) Gaussian filtering is performed on the obtained target area image to remove noise;

(3) Calculate the average gray value of the image background in the target area, and subtract it from the gray value of the original image to remove the background to make the background gray uniform, avoid the gray gradient change in the background image, and affect the edge extraction results , leading to the extraction of too many non-X-type primitive symbol edges, which will interfere with the segmentation mark;

(4) In order to make the edge of the X-type primitive symbol in the image of the target area more prominent, the Laplacian 5-neighborhood edge enhancement method is used to enhance the edge of the X-type primitive symbol in the image of the target area to facilitate edge extraction.

2) Carry out primitive segmentation and pattern recognition on the projected image of the surface of the door outer panel in the camera image

Referring to Fig. 10, since in the definition process of the X-type primitive symbol, what we adopt is to utilize the difference of the included angle between the long main inertial axis and the base axis of the X-type primitive symbol to classify and define the code words , so we need to use the same method to identify the X-type primitive symbols in the camera image.

(1) X-type primitive symbol edge extraction and segmentation marking

The present invention directly uses the canny operator to extract the edge of the preprocessed local target area image to obtain the edges of all X-type primitive symbols and a large number of non-primitive edges. The image after edge extraction is a binary image, and the edge part The intensity value is 1, and the non-edge intensity value is 0; due to the existence of a large number of non-primitive symbol edges, there will be multiple non-primitive symbol feature corners and the corner point extraction is not accurate, so it needs to be eliminated; and because the calculation of X - The characteristic corner coordinates of the edge image of the primitive symbol and the restoration of the codeword of the primitive symbol need to calculate the edge of each primitive symbol separately, so that the different primitive symbols remain relatively independent. During the calculation process They do not affect each other, so it is necessary to use the segmentation labeling algorithm for each primitive symbol to classify and label different primitive symbols, and perform calculations according to the different labels, and primitive symbols with different labels cannot be operated at the same time.

Referring to Figure 11, the 8-neighborhood region growing algorithm is used to search the image from left to right and from top to bottom to ensure that all independently connected regions are marked and segmented so that they can be operated independently without affecting each other. Due to the existence of a large number of non-primitive symbol edges (noise points or overexposed areas), the number of marks obtained through preliminary marking in the present invention is 1810, and the same marked non-primitive symbol edges and noise points need to be eliminated. Considering that the shape and size of the edge of the primitive symbol are basically the same, that is, the number of edge pixels is basically the same, but there are two extremes in the noise point and the overexposed area. The noise point is mostly fragmented single or a few pixels, while the strong light The area contains more edge pixels, so the average threshold method is used to calculate the number of pixels in all marked areas, and 2/3 and 4/3 of the average number are taken as the upper and lower limits to eliminate noise points or exposure areas. The influence of the X-type primitive symbol edge extraction results, while re-marking and segmenting the X-type primitive symbol edge regions that meet the requirements, the labels are 1-n, and the final marking results are 1713. The results shown in Figure 12-a are obtained after removing non-primitive edge regions. Although some X-type primitive symbol regions are also mistakenly removed, the proportion is very small and does not affect the overall

Figure 12-b shows the enlarged image of X-type primitive symbol edge point detection.

(2) Feature corner point extraction of the projected image on the surface of the outer panel of the car door in the camera image. To extract the feature corner points from the edge image of the X-type primitive symbol after marking, it is necessary to separately perform feature corners on the edge area of each X-type primitive symbol The calculation of point coordinates is mainly divided into two steps:

a. Coarse positioning of feature corners based on centroid

Using the centroid coordinates calculation formula, the obtained edge coordinates and gray intensity values (1 or 0) of each X-type primitive symbol are used to calculate the centroid coordinates separately according to the marking order using the static moment formula;

b. Fine positioning of feature corners based on Harris

After the rough positioning of the corner points is completed, the result of the rough positioning of the corner points needs to be substituted into the original initial image. In the original image, the coordinates obtained by the rough positioning are taken as the center, and Harris is used in its neighborhood (3×3). The corner point extraction algorithm searches according to the gray gradient to find the point with the largest gradient change, and takes the position coordinates of the point with the largest gray gradient as the fine positioning coordinates. The result of fine positioning of the characteristic corner points is shown in Figure 14.

(3) Recognize and restore the codewords of the X-type primitive symbols in the projected image on the surface of the outer panel of the car door in the camera image

Since in the design process, the X-type primitive symbol is used to classify the codewords by the angle between the longer main inertial axis and the base axis, so according to this correspondence, it is also necessary to calculate the longer main inertia The angle between the axis and the base axis restores the codeword.

3) Stereo matching of the feature corners of the projected image on the surface of the door outer panel and the feature corners of the coded projection template in the camera image

Referring to Figure 16, after the target image is decoded, a codeword array can be obtained, each codeword represents not only an X-type primitive symbol, but also a characteristic corner of the X-type primitive symbol, It also represents the image coordinates of the feature corners. According to the neighborhood-based spatial coding strategy, the code value of each X-type primitive symbol is composed of its 8 neighborhoods. Therefore, in order to facilitate stereo matching, it is necessary to calculate each The code value of the X-type primitive symbol, and the code value obtained in this way is unique.

After obtaining the code value of each corresponding X-type primitive symbol, the circular search algorithm is used to search the original pseudo-random M array. The code value with the highest degree is used as the result of stereo matching, and the corresponding position of each X-type primitive symbol in the detection image in the original coding projection template is determined.

4. Space point cloud coordinate calculation and error evaluation stage corresponding to the feature corner points of the projected image on the surface of the door outer panel in the camera image

1) Calculate the three-dimensional space coordinates of the feature points of the projected image on the surface of the outer panel of the car door

Referring to Figure 17, after one-to-one matching of the feature corners of the camera image and the feature corners of the projected image, the three-dimensional space coordinates of the feature points of the projected image on the surface of the door outer panel are calculated using the triangulation method using the calibration results of the structured light detection system .

The code values and coordinates of the matching points in the left and right images are shown in the following table:

2) Register the calculated detection point cloud with the point cloud model measured by the three-coordinate measuring machine

Referring to Fig. 17 and Fig. 18, the point cloud calculated in the present invention is shown in Fig. 17, and the point cloud model of a door outer panel surface detected by a three-coordinate measuring machine is shown in Fig. 18. Since the coordinates of the calculated point cloud and the point cloud model obtained by the three-coordinate measuring machine are not uniform, the two do not overlap after being placed in the same coordinate system, as shown in Figure 19, so registration is required for accuracy evaluation . The present invention adopts a registration method based on ICP (IterativeClosestAlgorithm), and its main steps are divided into two steps: coarse registration and fine registration. The coarse registration method uses Gaussian curvature and average curvature as registration features to calculate and detect the For the Gaussian curvature and average curvature of each point, search for the closest point in the point cloud model, and set the error conditions so that the detection point cloud and the point cloud model are relatively close. The initial registration results are shown in Figure 20- a. The ICP algorithm is used for precise registration of the detected point cloud and the point cloud model after the initial matching, so that the shape deviation between the detected point cloud and the point cloud model is minimized, and the minimum distance objective function adopted is:

${\mathrm{<span\; class="notranslate">\; \&Delta;</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}}{<span\; class="notranslate">\; \&Sigma;</span>}}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; T</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; \&Center\; Dot;</span><span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; p</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; no</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; |</span>{<span\; class="notranslate">\; |</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 16</span>}<span\; class="notranslate">\; )</span>$

In the above formula, {p _j (x _j , y _j , z _j )|=1, 2...k}=P _p is the detection point obtained after the initial registration

{n _j (x _j , y _j , z _j )j=1, 2...k} N _c is the point cloud model, and the final result of point cloud registration is shown in Figure 20-b. After the detection point cloud and the point cloud model achieve the best registration, in order to simplify the iterative process of image processing, the nearest point search algorithm is used to search for the closest point of the registered detection point cloud in the point cloud model, and calculate the distance between the two The distance is used to represent the deviation, as shown in Figure 21 is the distribution map of the deviation value domain and the frequency diagram of the deviation value domain.

In order to express more clearly the size of errors in different regions of the detected point cloud, the color patch map is used to represent the errors in the present invention. First, the color index level is set to 64 levels, and the relationship between the point cloud deviation data and the color index is established. The relationship between them is as follows:

$\mathrm{<span\; class="notranslate">\; R</span>}\mathrm{<span\; class="notranslate">\; G</span>}\mathrm{<span\; class="notranslate">\; B</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; l</span>}\mathrm{<span\; class="notranslate">\; e</span>}\mathrm{<span\; class="notranslate">\; v</span>}\mathrm{<span\; class="notranslate">\; e</span>}\mathrm{<span\; class="notranslate">\; l</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; r</span>}\mathrm{<span\; class="notranslate">\; o</span>}\mathrm{<span\; class="notranslate">\; u</span>}\mathrm{<span\; class="notranslate">\; no</span>}\mathrm{<span\; class="notranslate">\; d</span>}<span\; class="notranslate">\; (</span>\frac{\mathrm{<span\; class="notranslate">\; \&Delta;</span>}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; \&Delta;</span>}}_{\mathrm{<span\; class="notranslate">\; m</span>}\mathrm{<span\; class="notranslate">\; i</span>}\mathrm{<span\; class="notranslate">\; no</span>}}}{{\mathrm{<span\; class="notranslate">\; \&Delta;</span>}}_{\mathrm{<span\; class="notranslate">\; max</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; \&Delta;</span>}}_{\mathrm{<span\; class="notranslate">\; min</span>}}}<span\; class="notranslate">\; *</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; L</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 17</span>}<span\; class="notranslate">\; )</span>$

In Formula 16: Δ represents the obtained deviation value, Δ _max represents the maximum deviation value, Δ _min = represents the minimum deviation value, L represents the color index classification, and RGB_level can calculate the color index value of all points in the detected point cloud.

The present invention calculates the deviation of the detected point cloud, and the average deviation of the point cloud is 1.6873mm. Because the present invention uses encoded structured light technology to detect the surface shape and size of the front door outer panel of Brilliance Automobile, if the experimental hardware level is further improved, the surface deviation can be controlled within 0.75mm. This method has very important engineering application value.

Claims (8)

1. based on a door skin geomery detection method for single projection coded structured light, it is characterized in that, the step of the described door skin geomery detection method based on single projection coded structured light is as follows:

1) structured light detection system calibration phase:

In order to ensure carrying out smoothly of detection, must demarcate the video camera (2) in structured light detection system and projector (4), set up the nonlinear relationship of the correspondence between locus and image coordinate, utilize this nonlinear relationship to calculate the volume coordinate of detected image Feature point correspondence;

2) the coding projection stencil design stage:

A width coding pattern only need be projected to door skin surface in testing process, this coding pattern is a size is the graphic array be made up of the X-type primitive symbol of 4 different directions that 40 row 45 arrange, in graphic array, figure puts in order by a pseudorandom M array decision, this graphic array has following characteristic: the figure in every 3 × 3 subarrays only occurs once in whole array mould plate, and in any two subarrays the figure of correspondence position have at least 3 different; Each X-type primitive symbol except edge have passed through coding, its encoded radio size is all determined jointly by the element in its 3 × 3 neighborhood, this graphic array imparts different code words respectively according to the longer principal axis of inertia of X-type primitive symbol is different from standard shaft angle, therefore graphic array correspond to again the digital array of 40 × 45, and this array is called pseudorandom M array;

3) detected image decoding cognitive phase:

Utilize projector (4) to sedan door outer plate surfaces projection coding projection template, recycling video camera (2) is taken door skin surface image, and is sent in computing machine by the image of shooting and carries out image procossing and identification;

4) spatial point cloud coordinate calculates and the error evaluation stage:

(1) three dimensional space coordinate of door skin surface projection images unique point is calculated;

(2) registration is carried out to the point cloud model that the check point cloud calculated and three coordinate measuring machine record.

2. according to the door skin geomery detection method based on single projection coded structured light according to claim 1, it is characterized in that, the step of described structured light detection system calibration phase is as follows:

1) video camera is demarcated:

The method utilizing Zhang Zhengyou to demarcate is demarcated, adopt 12 × 16 black and white chessboard case marker targets of projector (4) projection design to the right side demarcating plane, and then utilize video camera (2) to take whole demarcation plane, the video camera target image taking the lower left corner in the 12 width images obtained carries out Feature corner extraction, each video camera target image extracts 100 angle points, 12 width images obtain 1200 characteristic angle dot image coordinates altogether, according to Zhang Zhengyou scaling method, world coordinate system is set on an angle point of the most left upper of cross-hatch pattern picture, Z-direction coordinate figure is taken as zero, because each gridiron pattern size careful design is 30mm, therefore gridiron pattern feature angle point can accurately be learnt at the world coordinates of X and Y-direction, therefore after the volume coordinate obtaining plane picture characteristic angle point coordinate and corresponding point, tool box can be utilized to complete demarcation, calculate the internal and external parameter of video camera (2),

2) projector is demarcated:

Projection target in camera review is carried out to the extraction of feature angle point, and then according to the inside and outside parameter result of the video camera obtained (2), intersect method according to optical plane and calculate the volume coordinate be positioned under the feature angle point alive boundary coordinate system demarcating projected image in plane, so far, image coordinate and the volume coordinate of projection target all calculate, therefore, the method identical with camera calibration is adopted to calculate projector calibrating parameter, obtain the internal and external parameter of projector (4) and the rotation translation matrix between projector (4) and video camera (2).

3., according to the door skin geomery detection method based on single projection coded structured light according to claim 1, it is characterized in that, the step in described coding projection stencil design stage is as follows:

1) the pseudorandom M array of corresponding graphic array generates:

The feature of pseudorandom M array is, the subnumber group of specific 3 × 3 windows has uniqueness in whole array, and the Hamming distance between any two windows is larger, represents that difference is between the two larger, pseudorandom M array Hamming distance account form as shown in the formula:

Wherein: V _(ij)represent 9 element vector of 3 × 3 neighborhoods, H represents that between two difference 3 × 3 neighborhoods, the number of times of different code word appears in correspondence position vector, be Hamming distance, Hamming distance is larger, represent that the different degree of window is larger, the pseudorandom M array constructed in the coding projection stencil design stage, smallest hamming distance size is set as 3, namely represent that the element that the subarray of any two 3 × 3 windows is corresponding has at least 3 to be different, such being designed with is beneficial to the stability of enhancer array and the adaptability to detected object space surface intensity of variation;

Need in the coding projection stencil design stage to generate digital primitive for { 0,1,2,3}, smallest hamming distance equals 3, and size is the pseudorandom M array of 40 × 45; Employing cyclic pac king method completes the pseudorandom arrays structure meeting coding projection stencil design and require, first the subarray of 3 × 3 is generated in the array upper left corner, then the subarray of 3 × 1 is substituted into successively along right side, whether the Hamming distance size between more adjacent 3 × 3 windows is eligible, do not meet and again substitute into, eligible, carry out to the right until border successively; Then the below along initial 3 × 3 arrays substitutes into 1 × 3 subarray successively, and method is the same, until border; A new code value is substituted into successively and then at the blank position of 4 × 4, form new 3 × 3 subarrays, the all subarrays generated above compare Hamming distance size, eligible continuation, do not meet and again substitute into, as substitute into likely code word do not meet, then restart this building process;

2) based on the X-type primitive Design of Symbols of geometric properties:

The geometric primitive figure of structure coding projection template often adopts square or circular, but under there is shade or circumstance of occlusion, the skew of geometric center or the angle point that produces ambiguity is there is than being easier to using geometric center as the circle of feature angle point or square pattern, angular coordinate is caused to extract inaccurate, and according to tessellated Corner Feature, the geometric primitive designing a class X-type has more stable Corner Feature when blocking or lack, this X-type primitive symbol with cross-hatch pattern picture for characteristic Design, symmetrical about characteristic angle dot center, be made up of triangle in a center of symmetry or square, according to the angle angular dimension between longer principal axis of inertia and standard shaft { 0 °, 45 °, 90 °, 135 ° } difference respectively corresponding 4 code words { 0, 1, 2, 3},

According to the code word of above-mentioned correspondence and the corresponding relation of figure, according to the pseudorandom M array masterplate obtained, X-type primitive is updated in the middle of graphic array template successively, generate the coding projection template that size is 768 × 1024, single X-type primitive symbol size is 10 × 10pixel, the intersymbol interval of X-type primitive is set as 9pixel, this coding projection template take black as background, white is as the background color of X-type primitive figure, generate monochromatic coding projection template, be conducive to reducing door skin surface reflection degree.

4. according to the door skin geomery detection method based on single projection coded structured light according to claim 1, it is characterized in that, the step of described detected image decoding cognitive phase is as follows:

1) Image semantic classification is carried out to the image of the initial detecting on door skin surface in camera review;

2) primitive segmentation and pattern-recognition are carried out to the projected image on door skin surface in camera review;

3) Stereo matching is carried out to door skin surface projection images feature angle point in camera review and coding projection template characteristic angle point.

5. according to the door skin geomery detection method based on single projection coded structured light according to claim 4, it is characterized in that, described Image semantic classification is carried out to the image of the initial detecting on door skin surface in camera review comprise the steps:

1) masking operations is taked to the target area image in camera review, obtain the local area image that projection has the door skin surface of coding projection template, reject useless background area;

2) gaussian filtering is carried out, cancelling noise to the target area image obtained;

3) average gray value of target area image background is calculated, and subtract each other with original image gray scale, background is rejected, make background Histogram equalization, avoid shade of gray change in background image, affect the result of edge extracting, cause extracting too much non-primitive symbol edge, bring interference to dividing mark;

4) in order to make the X-type primitive symbol edge in target area image more outstanding, adopting X-type primitive symbol edge in Laplce 5 neighborhood Edge-enhancement strengthening target area image, being beneficial to edge extracting.

6. according to the door skin geomery detection method based on single projection coded structured light according to claim 4, it is characterized in that, described primitive segmentation is carried out to the projected image on door skin surface in camera review and pattern-recognition comprises the steps:

1) X-type primitive symbol edge extracting and dividing mark:

Adopt canny operator to carry out edge extracting to local target area image, can obtain the edge of whole X-type primitive symbol and a large amount of non-primitive symbol edge, after edge extracting, image is bianry image, and marginal portion intensity level is 1, and non-edge intensity level is 0; When carrying out angle point calculating and longer principal axis of inertia angle calculation, use the coordinate position at the edge of all primitive symbols, if there is the edge of more non-primitive symbol, can cause occurring that multiple non-primitive symbolic feature angle point and angle point grid are forbidden, therefore need to reject; And owing to calculating the characteristic angle point coordinate of X-type primitive symbol edge image and carrying out reduction needs to the code word of X-type primitive symbol and calculate each X-type primitive symbol edge separately, make to keep relatively independent between different X-type primitive symbols, be independent of each other in computation process, therefore also need to adopt dividing mark algorithm to carry out classification designator to different X-type primitive symbols to each X-type primitive symbol, calculate successively according to the difference of label, the X-type primitive symbol of different labels can not carry out computing simultaneously;

2) door skin surface projection images Feature corner extraction in camera review:

A. based on the feature angle point coarse positioning of the centre of form:

Adopt centre of form coordinate computing formula, to utilizing static moment formula according to flag sequence, centre of form coordinate is calculated separately to the edge coordinate of each X-type primitive symbol obtained and gray-scale intensity value;

First static moment formula is:

$M_{pq}={\&Integral;}_{-\mathrm{\&infin;}}^{\mathrm{\&infin;}}{\&Integral;}_{-\mathrm{\&infin;}}^{\mathrm{\&infin;}}{x}^p{y}^{q}f(x,y)dxdy---\left(8\right)$

In formula: x represents x-axis direction coordinate figure, y represents y-axis direction coordinate figure, and p, q value is determined according to the static moment of the different coordinate axis calculated, and when calculating the static moment of x-axis, p is 0, q is 1, and when calculating y-axis static moment, p is 1, q is 0; Static moment formula is incorporated in the middle of image procossing and can be write as:

$m_{pq}=\underset{i=0}{\overset{k-1}{\&Sigma;}}\underset{j=0}{\overset{l-1}{\&Sigma;}}{i}^p{j}^{q}f(i+1,j+1)---\left(9\right)$

Wherein: the product representation target image pixel size of k and l, f (i+1, j+1) gray-scale intensity of image coordinate (i, j) place pixel is represented, because image coordinate system initial point is placed on the upper left pixel of image, image coordinate (0,0) just in time respective pixel coordinate (1,1), therefore the coordinate size of each pixel needs to subtract 1 relative to its location of pixels, in image, edge pixel point is whole 1 herein, and rest of pixels point is all 0;

Then centre of form coordinate is:

$x_c=\frac{m_{10}}{m_{00}},y_c=\frac{m_{01}}{m_{00}}---\left(10\right)$

X _crepresent the centre of form coordinate of X-type primitive symbol in x direction;

Y _crepresent the centre of form coordinate of X-type primitive symbol in y direction;

When calculating the X-type primitive symbol centre of form, carry out successively from small to large according to label 1-n, when to label being the primitive symbol edge calculations centre of form coordinate time of a, wherein: 1≤a≤n, only having label to be the edge coordinate grey scale pixel value size of a is 1, and all the other are all set as zero, therefore avoids the impact calculated centre of form coordinate at other edges, in like manner carry out n time successively to calculate, calculate the centre of form coordinate of all X-type primitive symbol edge images;

B. based on the feature angle point fine positioning of Harris

The angular coordinate obtained based on the edge shape of X-type primitive symbol owing to adopting based on the angle point coarse positioning algorithm of the centre of form, when X-type primitive symbol deforms, the angular coordinate obtained can offset relative to its true angular coordinate, therefore after completing angle point coarse positioning, the result that angle point coarse positioning obtains also is needed to be updated in original initial image, extract by coarse positioning in original image centered by the coordinate obtained, in its neighborhood of 3 × 3, adopt Harris Robust Algorithm of Image Corner Extraction to search for according to shade of gray, find graded maximum point, get shade of gray maximum point position coordinates as fine positioning coordinate, the feature angle point fine positioning result obtained,

3) code word identification reduction is carried out to X-type primitive symbol in door skin surface projection images in camera review:

Adopt the angle calculated between longer principal axis of inertia and standard shaft to reduce to code word, adopt the angle between the moment of inertia integral formula longer principal axis of inertia of calculating and standard shaft, moment of inertia integral formula is as follows:

${\mathrm{CM}}_{pq}=\underset{i=0}{\overset{k-1}{\&Sigma;}}\underset{j=0}{\overset{l-1}{\&Sigma;}}{(i-x_c)}^p{(j-y_c)}^{q}f(i+1,j+1)---\left(11\right)$

Between longer principal axis of inertia and horizontal axis, angle formulae is as follows:

$\mathrm{\&alpha;}=\frac{1}{2}\arctan \left(\frac{2{\mathrm{CM}}_{11}}{{\mathrm{CM}}_{20}-{\mathrm{CM}}_{02}}\right)---\left(12\right)$

The direction of what small arrow direction on X-primitive symbol was pointed to is longer principal axis of inertia, and the line between adjacent two the X-type primitive centres of form of level is baseline, due in projection process, owing to cannot ensure that horizontal projection makes graphic array and horizontal direction be consistent, therefore there is a certain size angle between baseline and horizontal direction, its angle formulae is:

$\mathrm{\&beta;}=\arctan \left(\frac{y_n-y_{n-1}}{x_n-x_{n-1}}\right)---\left(13\right)$

Wherein: (x _n, y _n) what represent is the characteristic angle point coordinate of the n-th X-type primitive symbol, then can between longer principal axis of inertia and baseline angle be finally:

Δ＝α-β(14)

Difference range is set as ± 10 °, compare Δ and { 0,45,90, difference between 135}, if difference is between the two within the scope of specification error, then according to corresponding angle corresponding { 0,1,2,3} carries out code word reduction, compares successively, obtains the code word matrix of detected image.

7. according to the door skin geomery detection method based on single projection coded structured light according to claim 4, it is characterized in that, described Stereo matching is carried out to door skin surface projection images feature angle point in camera review and coding projection template characteristic angle point refer to:

After decoding effort is completed to target image, obtain a code word array, what each code word represented is not only an X-type primitive symbol, and represent the feature angle point of this X-type primitive symbol, the simultaneously image coordinate of also representative feature angle point, in order to determine the particular location of X-type primitive symbol in projection template in target image, namely determine to take the correspondence position of X-type primitive symbol in projection template in image, the corresponding corner location in projection template in other words in camera review in door skin surface projection images corresponding to feature angle point, realize the coupling one by one of camera review and projection template, according to the space encoding strategy based on neighborhood, the code value of each X-type primitive symbol by its up and down 8 neighborhoods form, therefore the carrying out of conveniently Stereo matching, need the code value calculating each X-type primitive symbol according to array mode shown in figure, the code value obtained like this has uniqueness,

After the code value obtaining each corresponding X-type primitive symbol, cyclic search algorithm is adopted to search for original pseudorandom M array again, circulation contrast is carried out according to detecting the code value of code value to original coding projection template obtained, get the result that the highest code value of similarity makes Stereo matching, determine the position that in detected image, each X-type primitive symbol is corresponding in original coding projection template.

8., according to the door skin geomery detection method based on single projection coded structured light according to claim 1, it is characterized in that, described spatial point cloud coordinate calculate and the step in error evaluation stage as follows:

1) three dimensional space coordinate of door skin surface projection images unique point is calculated

After the coupling one by one realizing camera review feature angle point and projected image feature angle point, utilize the calibration result of structured light detection system, triangulation is adopted to calculate the three dimensional space coordinate of door skin surface projection images unique point, the wherein volume coordinate Z of depth direction _rcomputing formula be:

$Z_R=\frac{\left|\right|\stackrel{\&OverBar;}{x_L}|{|}^2<{\stackrel{\&OverBar;}{\mathrm{\&alpha;}}}_R,T>-<{\stackrel{\&OverBar;}{\mathrm{\&alpha;}}}_R,{\stackrel{\&OverBar;}{x}}_L><{\stackrel{\&OverBar;}{x}}_L,T>}{\left|\right|{\stackrel{\&OverBar;}{\mathrm{\&alpha;}}}_R\left|{|}^2\right|\left|\stackrel{\&OverBar;}{x_L}\right|{|}^2-<{\stackrel{\&OverBar;}{\mathrm{\&alpha;}}}_R,{\stackrel{\&OverBar;}{x}}_L{>}^2}---\left(15\right)$

In formula: what represent is dot product operations symbol, wherein with what represent has been the left images coordinate vector of Stereo matching, and what R and T represented respectively is rotation matrix and the translation vector that projector coordinates is tied to camera coordinate system; Adopt triangulation spatial coordinates calculation formula, utilize structured-light system calibration result, car door surface characteristics space of points coordinate is calculated, obtains check point cloud schematic diagram;

2) registration is carried out to the point cloud model that the check point cloud calculated and three coordinate measuring machine record:

Adopt the method for registering based on ICP, its key step is divided into rough registration and smart registration two step, rough registration method is using Gaussian curvature and mean curvature as registration features, calculate Gaussian curvature and the mean curvature of each point in check point cloud, immediate point is with it searched in point cloud model, specification error condition, makes check point cloud and point cloud model reach state relatively, obtains initial registration result; Adopt ICP algorithm to carry out accuracy registration to the check point cloud after initial matching and point cloud model, make the form variations between check point cloud and point cloud model reach minimum, the minimum distance function of employing is:

${\mathrm{\&Delta;}}_i=\underset{j=1}{\overset{M}{\&Sigma;}}\left|\right|T_i\&CenterDot;\left(p_j\right)-n_j|{|}^2---\left(16\right)$

In above formula, { p _j(x _j, y _j, z _j) | j=1,2...k}P _pfor the check point cloud obtained after carrying out initial registration, { n _jx _j, y _j, z _j) | j=1,2...k}=N _cfor point cloud model, the point cloud registering net result obtained;

Make after check point cloud and point cloud model reach optimal registration, in order to simplified image process iterative process, to be searched for the closest approach of check point cloud in point cloud model after registration by closest approach searching algorithm, calculate distance between the two, represent deviation with this;

In order to indicate the error size of zones of different in check point cloud more clearly, utilizing color color spot figure to represent error, is first 64 ranks by color index grade setting, and the relational expression set up between some cloud deviation data and color index is as follows:

$RGB\_level=round\left(\frac{\mathrm{\&Delta;}-{\mathrm{\&Delta;}}_{\min }}{{\mathrm{\&Delta;}}_{\max }-{\mathrm{\&Delta;}}_{\min }}*\left(L-1\right)\right)---\left(17\right)$

In formula: Δ represents the deviate size obtained, Δ _maxrepresent maximum deflection difference value, Δ _minrepresent minimum deviation value, L represents color index classification, RGB_level to calculate in check point cloud color rope value size a little.