CN101033958A - Mechanical vision locating method - Google Patents

Abstract

The invention discloses a machine vision positioning method, which adopts two ordinary cameras to jointly complete the detection task, including a long-distance camera for taking a global image of a target object and a short-distance camera for taking a short-distance image of an interested part. First calibrate the long-distance camera, and determine the proportional relationship between the pixel distance on the imaging plane of the short-distance camera and the actual physical distance; use the long-distance camera to capture the image of the target object, and determine the image coordinates and actual coordinates of the center of the interested part; Use the manipulator to move the close-range camera to the center of the part of interest, acquire the image and determine its image coordinates and actual coordinates, and obtain the actual precise position, shape, size and other information of the part. The invention utilizes two ordinary cameras to obtain the detailed information of the local interested part of the target object with high precision, and realizes the high-precision requirement of the industrial application of machine vision at a very low cost.

Description

A method of machine vision positioning

technical field

The invention belongs to the application field of machine vision in industry, and specifically relates to a machine vision positioning method, which utilizes two ordinary cameras to obtain detailed information of local interest parts of a target object in a large scene with high precision, such as accurate line segment Length, specific location of solder joints, etc.

Background technique

In modern industrial automation production, more and more machine vision technology is applied, involving various inspection, measurement and processing, such as laser engraving, printed circuit board pin welding, laser drilling, trademark cutting, etc. What these applications have in common is the need for continuous mass production and high requirements for product quality. This kind of work is highly repetitive and is usually done by manual inspection.

If this process is completed simply by increasing the number of testing workers, it will inevitably increase the factory's huge labor and management costs. Moreover, because the human eye will be fatigued after doing the same thing for a long time, the probability of error detection will increase, so it is still A 100% inspection pass rate cannot be guaranteed. Therefore, there is great value in introducing machine vision technology into industrial production.

At present, research on the measurement or detection of planar objects basically uses a camera, such as the planar measurement method based on a single image in Chinese patent documents (its publication number is CN1427245, and the publication date is 2003.07.02) and the single-view method based on parallel lines Plane measurement method (its public number is CN1427246, and the public date is 2003.07.02), but because the camera needs to detect a larger area, when it is necessary to accurately observe some local interested parts of the entire area and real-time monitor these parts When performing necessary processing, such as more detailed detection and soldering of circuit board solder joints, a high-quality camera is required, otherwise it will not be possible to accurately analyze local parts of interest, which will inevitably increase the cost of the equipment.

Existing machine vision applications in the industry, either the detection accuracy is not high enough, or the cost of equipment is increased in order to achieve high accuracy. At present, the industry mainly uses a high-quality camera to detect products on the assembly line, so the whole set of equipment is relatively expensive. The invention can reduce the equipment cost of the camera part while ensuring the same high precision, and is especially suitable for application systems that need to use a manipulator to move on the measured target plane, such as laser engraving, laser cutting, mechanical welding and other systems.

Contents of the invention

The purpose of the present invention is to provide a machine vision positioning method, which can obtain high-precision information of the local interest position of the target object with low equipment cost.

The machine vision positioning method provided by the present invention uses two ordinary cameras to jointly complete the detection task, one of which is a camera with a fixed position, which is far away from the measured target plane, and is used to capture the global image of the target object, which is called long-distance Camera; the other is fixed on the manipulator, which is closer to the measured target plane and can move on the measured target plane to capture close-range images of the interested parts of the target object, called a close-range camera. Its steps include:

(1) Calibrate the long-distance camera, determine the corresponding coordinate transformation relationship between its imaging plane and the measured target plane, and determine the proportional relationship between the pixel distance on the imaging plane of the short-distance camera and the actual physical distance;

(2) Place the target object to be detected on the measured target plane, and take a global image containing the target object with a long-distance camera;

(3) Determine the part of interest on the target object on the captured global image, and record the image coordinates (u _i1 , v _i1 ) of the center of the part of interest, i=1,..., m, a total of m Part of interest, where m≥1;

(4) According to the corresponding coordinate transformation relationship between the long-distance camera imaging plane and the measured target plane obtained in step (1), convert the image coordinates of the center of the part of interest recorded in step (3) into actual coordinates (x _i1 , y _i1 ), i=1,..., m;

(5) Utilize the manipulator to move the close-range camera to the actual coordinates of the center of the part of interest, the center of the part of interest coincides with the center of the imaging plane of the close-range camera, and take a close-range image of the part;

(6) Determine the image coordinates of the interested part in the short-range image of the above-mentioned interested part, and use the proportional relationship between the pixel distance on the close-range camera imaging plane obtained in step (1) and the actual physical distance to convert the above sense The image coordinates of the close-range image of the part of interest are converted into actual coordinates, and the actual information of the part of interest is obtained.

The present invention uses the polynomial calibration method to calibrate the long-distance camera, which eliminates the distortion effects of the camera such as radial distortion and tangential distortion, and can accurately determine the corresponding coordinate transformation relationship between the imaging plane and the measured target plane; then use The manipulator moves the close-range camera to the center of the part of interest to further accurately determine the detailed information of the part of interest of the target object, and further process the obtained detailed information according to the needs of practical applications. Due to the low price of ordinary cameras, the cost of using two cameras is similar to that of using one camera, but the detection accuracy is greatly improved; and, compared with the use of high-precision and expensive single-camera machine vision systems, the cost is much lower.

Description of drawings

Fig. 1 is the flowchart of machine vision positioning method of the present invention;

Fig. 2 is the image of the plane calibration plate used by the long-distance camera of the present invention;

Fig. 3 is the template image used by the close-range camera of the present invention;

Fig. 4 is a diagram of the image feature extraction results of the calibration plate according to the embodiment of the present invention.

Detailed ways

The basic principle of the present invention is: use two cameras, wherein, the long-distance camera processes the target object as a whole, shoots the global image of the target object, and detects the interesting part on the captured global image (depending on the needs of the application) ); and then control the manipulator to move the close-range camera to these parts of interest, and take high-precision images for further detailed detection and analysis.

The technical solutions of the present invention will be further described in detail below in conjunction with the accompanying drawings and examples.

As shown in Figure 1, the machine vision positioning method of the present invention comprises the following steps:

(1) Calibrate the long-distance camera, determine the corresponding coordinate transformation relationship between its imaging plane and the measured target plane; and determine the proportional relationship between the pixel distance on the imaging plane of the short-distance camera and the actual physical distance;

(A) The process of calibrating the long-distance camera is:

(A1) Obtain the calibration board image: place the prepared calibration board image in the plane where the target object is located, try to make the calibration board fill the entire long-distance camera viewing area, and use the long-distance camera to take a calibration board image; calibration board Images similar to chess boards, circle arrays and other images with obvious features and known actual dimensions can be used;

(A2) Extracting feature points (such as corner points or circle centers) in the calibration plate image, and determining the image coordinates of all extracted feature points;

For some types of calibration boards, there may be some noise points or some background in the obtained calibration board images, which may cause the extracted corner points to be offset or non-calibration board corners, so not all feature points are useful. Useless feature points should be removed. For example, the image of a chess board should first eliminate useless feature points, but the circle array does not need it.

(A3) according to the image of the calibration plate, determine the corresponding actual coordinates of the feature points extracted in the step (A2);

(A4) Using the polynomial calibration method and the image coordinates and actual coordinates of the above-mentioned feature points to determine the corresponding coordinate transformation relationship between the imaging plane and the measured plane. Note that the image coordinates of the feature points are (u _i , v _i ), the actual coordinates are (xi _, y _i ), i=1, 2, . . . , m. The formula for the polynomial calibration method is:

x _i ＝a ₁ u _i ^k +a ₂ v _i ^k +a ₃ u _i ^k-1 v _i +a ₄ u _i v _i ^k-1 +a ₅ u _i ^k-1 +...+a _{(4 *k-4)} u _i +a _(4*k-3) v _i +a _(4*k-2)

y _i ＝b ₁ u _i ^k +b ₂ v _i ^k +b ₃ u _i ^k-1 v _i +b ₄ u _i v _i ^k-1 +b ₅ u _i ^k-1 +...+b _{(4 *k-4)} u _i +b _(4*k-3) v _i +b _(4*k-2)

Among them, a ₁ , a ₂ , ..., a _(4*k-2) and b ₁ , b ₂ , ..., b _(4*k-2) are undetermined parameters, and the least square method can be used to find Parameters to be determined.

(B) Determine the proportional relationship between the pixel distance on the imaging plane of the close-range camera and the actual physical distance according to the following steps:

(B1) Place a template containing one or several line segments of known length on the measured target plane;

(B2) Take one or several images containing the above-mentioned template in different positions;

(B3) extracting a line segment of known length on the template image, recording the pixel length between the two ends of the line segment, that is, the number of pixels forming this line segment;

(B4) Calculate the proportional relationship between the pixel distance and the actual physical distance.

(2) Place the target object to be detected on the measured target plane, and take a global image containing the target object with a long-distance camera;

(3) Determine the part of interest on the target object on the captured global image, and record the image coordinates (u _i1 , v _i1 ) of the center of the part of interest, i=1,..., m, a total of m Part of interest, where m≥1;

(4) According to the corresponding coordinate transformation relationship between the imaging plane of the long-distance camera obtained in step (1) and the measured target plane, the image coordinates of the center of the part of interest recorded in step (3) are converted into actual coordinates (x _i1 , y _i1 ), where i=1,...,m;

(5) Use the manipulator to move the close-range camera to the actual coordinates of the center of the interested part, the center of the interested part coincides with the center of the imaging plane of the close-range camera, and take a close-range image of the part; because the camera is far from the measured object The plane is very close, the viewing range is small, and the imaging is clear, so the precise details of the interested part can be obtained;

(6) Determine the image coordinates of the part of interest in the close-range image of the above-mentioned part of interest, according to the relative positional relationship between its image coordinates and the center of the imaging plane of the close-range camera, and use the imaging plane of the close-range camera obtained in step (1) The proportional relationship between the pixel distance on the image and the actual physical distance, the image coordinates of the close-range image of the above-mentioned interested part are converted into actual coordinates, and the actual information of the interested part is obtained, including position, size, shape and other information. Then compare the actual standard to determine whether there is any abnormality such as damage or fracture in this part. If there is, it can be further processed as needed.

Example of circuit board solder joint inspection:

(1) Calibrate the long-distance camera, determine the corresponding coordinate transformation relationship between its imaging plane and the measured target plane, and determine the proportional relationship between the pixel distance on the imaging plane of the short-distance camera and the actual physical distance. The specific steps are as follows:

(A) The process of calibrating the long-distance camera is:

(A1) Obtain the calibration board image: take the chess board image shown in Figure 2 as the calibration board, place the calibration board image on the plane where the target object is located, try to make the calibration board fill the entire long-distance camera viewing area, and use the long-distance The camera shoots an image of the calibration board, and the physical length of the side length of each grid of the calibration board is known;

(A2) extract the feature of calibration board image, determine the image coordinate of feature point: because what adopt among the present invention is chess board image, so can extract corner point as the feature point of calibration board. Corner extraction methods include SUSAN, Harris, etc. Here, the Harris corner extraction algorithm is used, and the results are shown in Figure 4. The center of the white cross in the figure is the corner point;

(A3) determining the actual coordinates of the feature points extracted in (A2);

According to the selected calibration plate image, the acquired image may have some noise points or some backgrounds that cause the extracted corner points to be offset or not the corner points of the calibration plate, so not all feature points are useful, need According to removing useless feature points and determining the actual coordinates of valid feature points, the steps are as follows:

(a) Select a coordinate system on the calibration plate image, determine the effective area of the coordinate origin O and the X, Y axis directions and the calibration plate;

(b) Segment the effective area, extract the edge, and extract the straight lines perpendicular to each other on the calibration plate: methods such as maximum entropy method and OTSU algorithm can be used to carry out binary segmentation of the effective area, and in the present invention, the OTSU algorithm is adopted. After segmentation, extract the contour of the image, and then use the Hough transform algorithm to extract each straight line;

(c) Sort the straight lines and find the intersection points of the perpendicular straight lines: first, divide all the straight lines into two types, the two types of lines are perpendicular to each other, and the lines of the same type are parallel to each other; then, according to the coordinate origin The distance to the straight line is used to judge the order of straight lines of the same type; finally, the actual coordinates of the intersection points of each perpendicular straight line are obtained;

(d) Compare the calculated intersection point with the corner point calculated in (A2), and then determine the actual coordinate information of each effective corner point: the nearest neighbor method is used here. For example, the actual coordinates of the intersection of two intersecting straight lines are (20, 30) (mm), and their image coordinates are (151, 124), if the Euclidean distance between the coordinates of the corner point and the image coordinates of the intersection is less than 3, and If the absolute value of the distance difference between the two components of the image coordinates is less than 2, then this corner point is considered to be a valid feature point, and its actual coordinates are (20, 30) (mm);

(A4) In the present invention, the coordinate conversion relationship between the imaging plane and the measured target plane is determined by a third-degree polynomial calibration method: the image coordinates of each feature point are (u _i , v _i ), and the actual coordinates are ( x _i , y _i ), i=1, 2, ... (because the 3rd degree polynomial calibration method contains 20 undetermined parameters, so at least 20 equations are needed to obtain it, so there must be: i>=10), then The following relationships exist:

x _i ＝a ₁ u _i ³ +a ₂ v _i ³ +a ₃ u _i ² v _i +a ₄ u _i v _i ² +a ₅ u _i ² +a ₆ v _i ² +a ₇ u _i v _i + a ₈ u _i +a ₉ v _i +a ₁₀

y _i ＝b ₁ u _i ³ +b ₂ v _i ³ +b ₃ u _i ² v _i +b ₄ u _i v _i ² +b ₅ u _i ² +b ₆ v _i ² +b ₇ u _i v _i + b ₈ u _i +b ₉ v _i +b ₁₀

Wherein, a ₁ , a ₂ , ..., a ₁₀ and b ₁ , b ₂ , ..., b ₁₀ are undetermined parameters.

The parameters a _j , b _j , j=1, . . . , 10 are calculated by the least square method.

(B) Determine the proportional relationship between the pixel distance on the imaging plane of the close-range camera and the actual distance according to the following steps:

(B1) Place the template image as shown in Figure 3 on the measured target plane; wherein, the physical distance between the two points a and b is known;

(B2) Take one or several (positions can be different) of the above-mentioned template images with a close-range camera;

(B3) Extract a line segment with a known length on the template, and record the pixel length of the line segment: use an automatic or human-computer interaction method to obtain the image coordinates of two points a and b, and obtain the pixel length D. The larger the span of the line segment, the more accurate the calculation result;

(B4) Calculate the proportional relationship between the pixel distance and the actual physical distance: for the extracted n line segments, calculate the proportional relationship between the pixel distance and the actual physical distance, and then calculate the average of the obtained n results value as the final result.

(2) Place the circuit board on the measured target plane, and take a global image including the circuit board with a long-distance camera;

(3) Use human interaction or some image processing methods, such as threshold segmentation, feature extraction, template matching and a series of methods to find out the approximate image coordinates of all solder joints on the circuit board on the global image (u _i1 , v _i1 ), i=1,..., m, wherein, m>=1, record it;

(4) According to the corresponding coordinate transformation relationship between the imaging plane of the long-distance camera obtained in step (1) and the measured target plane, the image coordinates of all solder joints marked in step (3) are converted into each solder joint using the following formula The actual coordinates of the point (x _i1 , y _i1 );

x _i1 ＝a ₁ u _i1 ³ +a ₂ v _i1 ³ +a ₃ u _i1 ² v _i1 +a ₄ u _i1 v _i1 ² +a ₅ u _i1 ² +a ₆ v _i1 ² +a ₇ u _i1 v _i1 + a ₈ u _i1 +a ₉ v _i1 +a ₁₀

y _i1 ＝b ₁ u _i1 ³ +b ₂ v _i1 ³ +b ₃ u _i1 ² v _i1 +b ₄ u _i1 v _i1 ² +b ₅ u _i1 ² +b ₆ v _i1 ² +b ₇ u _i1 v _i1 + b ₈ u _i1 +b ₉ v _i1 +b ₁₀

Wherein, a ₁ , a ₂ , ..., a ₁₀ and b ₁ , b ₂ , ..., b ₁₀ have been obtained in step (1).

(5) According to the actual coordinates of each solder joint, use the manipulator to move the close-range camera to the actual position (x _i1 , y _i1 ) of the solder joint, which is in the center of the imaging plane of the close-range camera, that is, the image coordinate of the center is (u ₂ , v ₂ ), the actual coordinates are (x _i1 , y _i1 ), take a close-up image of the solder joint;

(6) Determine the image coordinates (u _i2 , v _i2 ) of the solder joints in the close-range image of the above-mentioned solder joints, and obtain the relative positional relationship between the image coordinates and the center of the imaging plane of the close-range camera (u _i2 -u ₂ , v _i2 -v ₂ ), and according to the proportional relationship between the pixel distance on the imaging plane of the close-range camera obtained in step (1) and the actual physical distance, the above relative positional relationship is transformed into the relative relationship in the actual coordinates (x _i2 ′, y _i2 ′), and then, according to the actual coordinates (x _i1 , y _i1 ) of the center of the imaging plane of the close-range camera, the actual precise coordinates (x _i2 , y _i2 ) of the solder joint are obtained as (x _i1 +x _i2 ′, y _i1 +y _i2 ′), the laser welder can be controlled to move to the actual position of the welding spot for welding.

Claims (3)

1. A machine vision positioning method. This method uses two ordinary cameras to jointly complete the detection task. One of them is a camera with a fixed position, which is far away from the measured target plane and is used to capture the global image of the target object. It is called Long-distance camera; the other is fixed on the manipulator, which is closer to the measured target plane, and can move on the measured target plane to capture close-range images of parts of interest, which is called a close-range camera; the steps include: (1) Calibrate the long-distance camera, determine the corresponding coordinate transformation relationship between its imaging plane and the measured target plane, and determine the proportional relationship between the pixel distance on the imaging plane of the short-distance camera and the actual physical distance; (2) Place the target object to be detected on the measured target plane, and take a global image containing the target object with a long-distance camera; (3) Determine the part of interest on the target object on the captured global image, and record the image coordinates (u _i1 , v _i1 ) of the center of the part of interest, i=1,..., m, a total of m Part of interest, where m≥1; (4) According to the corresponding coordinate transformation relationship between the imaging plane of the long-distance camera obtained in step (1) and the measured target plane, the image coordinates of the center of the part of interest recorded in step (3) are converted into actual coordinates (x _i1 , y _i1 ), i=1, . . . , m; (5) Utilize the manipulator to move the close-range camera to the actual coordinates of the center of the part of interest, wherein the center position of the part of interest coincides with the center of the imaging plane of the close-range camera, and take a close-range image of the part; (6) Determine the image coordinates of the region of interest in the close-range image of the above-mentioned region of interest, according to the positional relationship between the image coordinates and the center of the imaging plane of the near-distance camera, use the position on the imaging plane of the near-distance camera obtained in step (1) The proportional relationship between the pixel distance and the actual physical distance transforms the image coordinates of the short-distance image of the above-mentioned interested part into actual coordinates to obtain the actual information of the interested part. 2. The machine vision positioning method according to claim 1, characterized in that: in step (1), the remote camera is calibrated according to the following steps; (A1) Place the image of the calibration plate in the plane where the target object is located, and take a picture of the calibration plate with a long-distance camera; (A2) Extract the feature points of the calibration plate image, and determine the image coordinates of all the extracted feature points; (A3) determine the actual coordinates of the feature points extracted in step (A2) according to the image of the calibration plate; (A4) Use the following formula and the image coordinates and actual coordinates of the above-mentioned feature points to determine the corresponding coordinate transformation relationship between the imaging plane and the measured plane, record the image coordinates of the feature points as (u _i , v _i ), and the actual coordinates is (x _i , y _i ), i=1, 2, ..., m; x _i ＝a ₁ u _i ^k +a ₂ v _i ^k +a ₃ u _i ^k-1 v _i +a ₄ u _i v _{i k-1} +a5u _i ^k-1 +...+a _{(4*k -4)} u _i +a _(4*k-3) v _i +a _(4*k-2) y _i ＝b ₁ u _i ^k +b ₂ v _i ^k +b ₃ u _i ^k-1 v _i +b ₄ u _i v _i ^k-1 +b ₅ u _i ^k-1 +...+b _{(4 *k-4)} u _i +b _(4*k-3) v _i +b _(4*k-2) Among them, a ₁ , a ₂ , ..., a _(4*k-2) and b ₁ , b ₂ , ..., b _(4*k-2) are undetermined parameters, and the least square method is used to obtain the undetermined parameter. 3. The machine vision positioning method according to claim 1 or 2, characterized in that: said step (1) determines the proportional relationship between the pixel distance on the imaging plane of the close-range camera and the actual physical distance according to the following process: (B1) Place a template containing one or several line segments of known length on the measured target plane; (B2) Take one or several images containing the above-mentioned template in different positions; (B3) extracting a line segment of known length on the template image, recording the pixel length between the two ends of the line segment, that is, the number of pixels forming this line segment; (B4) Calculate the proportional relationship between the pixel distance and the actual physical distance.

Images