CN109272544B - Structured light 3D measurement model and image processing method for all-position welds in pipelines - Google Patents

Abstract

The invention discloses a structured light three-dimensional measurement model and an image processing method of a pipeline all-position weld, comprising: S1. Constructing a mathematical model of the pipeline weld structured light three-dimensional measurement, and converting the coordinate values in the image coordinate system to the camera coordinate system Next; S2, image preprocessing: real-time acquisition of weld images, and preprocessing of the collected weld images; S3, light bar center extraction: use the improved Gaussian fitting method to extract the light bar center of the weld image; S4, Inflection point location: Use the algorithm combining the slope analysis method and the "closest principle" to locate the inflection point and extract the weld information. For the V-shaped weld or the inverted trapezoidal weld, the method of the invention can very accurately extract the center of the light bar near the inflection point under the interference of arc light and splash, and then obtain the position of the inflection point of the weld edge by the slope analysis method and the "nearest principle". Therefore, it can quickly and accurately obtain the three-dimensional topography of the weld under strong interference conditions, and then realize the welding seam tracking.

Description

Structured light 3D measurement model and image processing method for all-position welds in pipelines

technical field

The invention relates to the field of structured light three-dimensional measurement, in particular to the field of structured light three-dimensional measurement of weld seam information for automatic pipe welding at all positions, in particular to a structured light three-dimensional measurement model and image processing method for pipe full position welds.

Background technique

The rapid development of welding robots is almost synchronized with the development of typical joint robots. However, in various large-scale pipeline laying, large-scale spherical tank welding and other places, the joint robot has a lot of space limitations, and it is impossible to perform a whole circle of girth welding at one time, and it is difficult to carry out field construction and underwater operations. flexibility to be widely used in these occasions.

Achieving fast and effective welding seam tracking is the key to improving the welding quality of pipeline robots. There are many common welding seam tracking methods, such as mechanical, electromagnetic, arc sensing, and visual sensing. Among them, the visual sensing method based on structured light method is favored due to its high speed, high precision and rich information obtained. Since the position and posture of the vision sensor installed on the welding robot relative to the welding seam will inevitably change during the walking process of the welding robot on the pipeline, in order to carry out accurate welding seam tracking, it is necessary to realize the welding seam shape through the vision sensor. 3D measurement of appearance.

The use of vision sensors for welding seam tracking is easily affected by factors such as arc light, spatter, smoke and rust, scratches, marks and oxide skin on the workpiece during welding, especially V-shaped or inverted trapezoidal welds have multiple reflections on structured light. In addition, it is necessary to control the welding parameters of the welding machine and synchronize wire feeding and air supply during welding. Therefore, how to quickly and accurately obtain the three-dimensional shape of the weld under strong interference conditions is the key to real-time weld tracking. where.

SUMMARY OF THE INVENTION

The purpose of the present invention is to provide a three-dimensional measurement model of structured light and an image processing method for all-position welded seam of a pipeline in view of the existing problems in the prior art.

For achieving the above object, the technical scheme adopted in the present invention is:

Structured light three-dimensional measurement model and image processing method of pipeline all-position weld, including the following steps:

S1. Construct a mathematical model for 3D measurement of pipeline welds with structured light, and convert the coordinate values in the image coordinate system to the camera coordinate system;

S2. Image preprocessing: real-time acquisition of weld images during the welding process, and preprocessing of the collected weld images;

S3. Light strip center extraction: use the improved Gaussian fitting method to extract the light strip center of the weld image;

S4. Inflection point location: use the algorithm combining the slope analysis method and the "nearest principle" to locate the inflection point and extract the weld information.

Preferably, S1 further includes: constructing a mathematical model for three-dimensional measurement of pipeline welds with structured light, using the camera coordinate system O _C X _C Y _C Z _C as the measurement coordinate system, the O _C X _C axis is perpendicular to the walking direction of the robot, and O _C Y The _C axis is parallel to the walking direction of the robot, O _C Z _C is the optical axis of the camera lens, through the relationship between the coordinates of a point in the camera coordinate system and the ideal computer image coordinates of this point, the ideal computer image coordinates of this point and The relationship between the actual image coordinates of this point, and the equation of the light plane in the camera coordinate system, get the coordinates of any point on the image in the camera coordinate system.

Preferably, the step of collecting the image of the welding seam further includes: collecting the image of the welding seam by using a welding robot, the welding robot is provided with a laser, an industrial camera, and a welding gun, the welding gun is aimed at the welding seam, and the laser emits laser light to irradiate the welding seam Above, the industrial camera captures the weld image of the irradiation spot.

Preferably, the step of preprocessing the acquired weld image further includes: windowing, denoising, smoothing, erosion, dilation and Gaussian filtering.

Preferably, the improved Gaussian fitting method includes: according to the difference between the light bar centers of two adjacent columns of pixels not exceeding one pixel, the light bar center of the pixels in the previous column is used as the center of the data range of the Gaussian curve fitting of the pixels in the current column, The amount of data to be fitted is adaptively 1.5 times the number of pixels of the current column width of the light bar. According to the characteristics of low quality in the groove of the light bar image and high quality on the pipeline, the extraction of the center of the light bar is divided into two sections. , respectively, from both sides to the middle. In order to improve the processing speed, only the center of the two short light bars near the edge of the weld is extracted.

Preferably, S3 further comprises: utilize the slope analysis method to extract all possible inflection points, and the slope calculation formula of the center point of the i-th row of light bars on the graph is:

All possible inflection points are obtained by analyzing the slopes of all light bar center points.

Preferably, S3 further includes: the "closest principle" is to use the inflection point that is closest to the inflection point of the previous image and less than 3 pixels away from the inflection point of the previous image as the inflection point of the welding seam edge of the current image.

Preferably, the method further comprises: acquiring an image of the welding seam, extracting information of the welding seam, and calculating the "origin" before starting welding.

Preferably, the midpoint of the two inflection points of the edge of the weld found by the "nearest principle" is used as the center of the weld, the three-dimensional coordinates of the midpoint are obtained, and the coordinates are subtracted from the "origin" to perform vertical welding Weld tracking in seam direction and height direction.

Compared with the prior art, the beneficial effects of the present invention are: 1) According to the characteristics of the weld image, the present invention proposes a light bar center extraction algorithm based on Gaussian curve fitting in the curve fitting method. position, adaptively to the data to be fitted, and has the characteristics of fast processing speed, high precision and strong anti-interference ability; 2) the present invention aims at the problem that some redundant inflection points will appear when the image quality is not high in the existing slope analysis method, and proposes An inflection point locating algorithm that combines the slope analysis method and the "closest principle" is proposed. , Accurately locate the inflection point, and the impact on the welding seam tracking is very small when the image processing fails, and the impact can be eliminated in the next welding torch correction; 3) The method of the present invention is suitable for V-shaped welding seam or inverted trapezoidal welding seam, Under the interference of arc light and splash, the center of the light bar near the inflection point can be very accurately extracted, and then the position information of the inflection point of the weld edge can be obtained by the slope analysis method and the "closest principle", so as to achieve fast and accurate acquisition under strong interference conditions. Three-dimensional topography of the weld seam, and then realize the weld seam tracking.

Description of drawings

1 is a schematic flow diagram of a method of the present invention according to an embodiment;

Fig. 2 is the coordinate schematic diagram of mathematical model in the method of the present invention;

3 to 6 are schematic diagrams of the results of performing light bar center extraction and inflection point positioning on a weld image by the method of the present invention according to an embodiment;

In the picture: 1. Welding seam; 2. Welding gun; 3. Camera; 4. Laser.

Detailed ways

The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

The present invention provides a structured light three-dimensional measurement model and an image processing method for a pipeline all-position weld, comprising the following steps:

S1. Construct a mathematical model for 3D measurement of pipeline welds with structured light, and convert the coordinate values in the image coordinate system to the camera coordinate system;

S2. Image preprocessing: real-time acquisition of weld images during the welding process, and preprocessing of the collected weld images;

S3. Light strip center extraction: use the improved Gaussian fitting method to extract the light strip center of the weld image;

S4. Inflection point location: use the algorithm combining the slope analysis method and the "nearest principle" to locate the inflection point and extract the weld information.

1. Mathematical model for 3D measurement of pipeline welds

In the specific implementation, since the real-time tracking of the welding seam only needs to obtain the three-dimensional appearance of the welding seam in real time at the moment of image acquisition, it is not necessary to obtain the complete three-dimensional appearance of the entire welding seam. Based on this, in order to improve the measurement accuracy and simplify the operation, the three-dimensional measurement mathematical model of the present invention uses the camera coordinate system O _C X _C Y _C Z _C as the measurement coordinate system, the O _C X _C axis is perpendicular to the walking direction of the robot, and the O _C Y _C The axis is parallel to the walking direction of the robot, O _C Z _C is the optical axis of the camera lens, as shown in Figure 2.

The relationship between the coordinates (x _c , y _c , z _c ) of a point in the camera coordinate system and its ideal computer image coordinates (u, v) is shown in formula (1):

γ表示u轴和v轴的不垂直因子；(u₀，v₀)为计 算机图像坐标系原点。where R and T are the rotation matrix and translation matrix from the world coordinate system to the camera coordinate system respectively; matrix A is the internal parameter matrix of the camera;

γ represents the non-perpendicular factor of u-axis and v-axis; (u ₀ , v ₀ ) is the origin of the computer image coordinate system.

Considering the influence of distortion, the transformation relationship between ideal computer image coordinates (u, v) and actual image coordinates ( _ud , v _d ) is:

In the formula, x=(uu ₀ )/f _x , y=(vv ₀ )/f _y , r=x ² +y ² , k ₁ and k ₂ are the radial distortion coefficients of the lens, and p ₁ and p ₂ are Tangential distortion factor.

Simultaneous equations (1) and (2) and the equation aX _c +bY _c +cZ _c +d=0 of the light plane in the camera coordinate system can be calculated according to the coordinates (u _d , v _d ) of any point on the image Get its coordinates (x _c , y _c , z _c ) in the camera coordinate system.

2. Extraction of weld information

For the arc light, splash, smoke and other disturbances in the welding process, in order to realize the welding seam tracking, the welding seam information to be extracted is the position information of the two inflection points of the welding seam edge, which is divided into three steps: image preprocessing, light bar center extraction and inflection point positioning. As shown in Figure 1.

Specifically, the welding seam image is collected by a welding robot. The welding robot is provided with a laser, an industrial camera, and a welding gun. The welding gun is aimed at the welding seam. The laser emits laser light and irradiates the welding seam. The industrial camera collects the irradiation point seam image.

2.1 Image preprocessing

The collected weld image is preprocessed. The image preprocessing first selects the appropriate ROI area, reduces the size of the processed image by adding a window, and improves the image processing speed. Next, the denoising and smoothing of the image is performed. Considering that the subsequent extraction of the center of the structured light strip and the extraction of the inflection point of the weld image are required, the image is corroded and expanded, and then Gaussian filtering is performed to filter out a wavelength division. While reducing noise, it retains more characteristics of the overall grayscale distribution of the image.

2.2 Light strip center extraction

The extraction of the center of the light bar is an important step in the acquisition of weld information, and its processing accuracy and speed will directly affect the accuracy and speed of the final result. The extraction methods of the center of the light strip mainly include: geometric center method, threshold method, thinning method, extreme value method, gray barycenter method, direction template method, Hessian matrix method and curve fitting method. The geometric center method has a fast processing speed, but has high requirements for image quality; the threshold method has low accuracy and is easily disturbed by noise; the refinement method has a slow processing speed and poor real-time performance; the extreme value method is extremely fast, but is also susceptible to noise. Interference; the gray-scale centroid method has high accuracy and is suitable for occasions where the intensity distribution of light bars is relatively concentrated; the direction template method and the Hessian matrix method have a large amount of computation and a slow processing speed; the curve fitting method is more accurate than the geometric center method and the gray-scale centroid method, etc. The accuracy is high, but it is not suitable for the extraction of the center of the narrow-band light strip. According to the characteristics of the welding seam image, the invention designs a light strip center extraction algorithm based on Gaussian curve fitting in the curve fitting method. High precision and strong anti-interference ability.

Gaussian curve fitting is a fitting method that uses the Gaussian function as shown in the following formula to perform function approximation on the data point set with the least squares method.

In the formula: y _max , x _max , and S are the peak value, peak position and half-width information of the Gaussian curve, respectively.

When using the Gaussian curve fitting method to extract the center of the light bar, it is generally necessary to first roughly extract the light bar center from the light bar image to obtain the approximate position of the light bar center, and then take the position as the center to obtain the appropriate fitted data. Gaussian curve fitting is performed to obtain the exact center point of the light bar. In fact, because the weld image has many interference factors such as arc light, splash, and bevel reflection, it is difficult to guarantee the rough extraction accuracy of the center of the light bar, and if the peak of the Gaussian curve is not near the center of the fitted data range, the fitting result There will be a large error; and the rough extraction process will also affect the extraction speed of the center of the light bar; in addition, due to the slope of the groove and the characteristics of the Gaussian beam itself, the width of the light bar on both sides of the pipe part and the middle groove part on the weld image If the amount of fitted data is not suitable, it will also affect the accuracy of Gaussian fitting.

In view of the above problems, the present invention proposes an improved Gaussian fitting method for the extraction of the light bar center of the weld image: the algorithm is based on the difference between the light bar centers of two adjacent columns of pixels being no more than one pixel, and the light bar center of the pixels in the previous column As the center of the data range fitted by the Gaussian curve of this column of pixels, the amount of data to be fitted is adaptively 1.5 times the number of pixels of the width of the light bar in the column, and according to the low quality in the groove of the light bar image and the quality on the pipeline For the higher characteristic, the extraction of the center of the light bar is divided into two sections, which are extracted from both sides to the middle. In order to improve the processing speed, only the center of the light bar of two small segments near the edge of the weld is extracted. The specific extraction process is as follows:

First determine a suitable left starting point. Considering that the positions of the light bars of the two adjacent images are very similar, subtract 40 from the abscissa of the image coordinates of the left edge point of the weld in the previous picture as the _abscissa of the left starting point; , y _ol ) The vertical coordinate y _ol is used as the center of the fitted data range at the left starting point of the currently processed image; then the light bar width d of the pixels in this column is obtained by analyzing the threshold, and 1.5 times the light bar width is used as the fitted data The ordinates of these points (y _ol -3d/4)～(y _ol +3d/4) and their corresponding gray values g ₁ , g ₂ ,..., g _3d/2 are taken into The method is fitted to a Gaussian curve. Considering that the _ordinate value is large and the interval between two points is only 1, the coefficient of the fitting result may be too large or too small, which will affect the accuracy. Fitting, then the ordinate of the left starting point is:

In the formula: a ₁ and a ₂ are the linear and quadratic coefficients of the Gaussian curve fitting result.

Combined with the abscissa of the left starting point calculated earlier, the specific coordinates of the left starting point are obtained. The first image can be collected before the welding torch starts. Since there is no interference such as arc light and spatter, the image quality is very high, and the center of the light bar is easy to be extracted very accurately, which ensures the correctness of the subsequent starting point; and according to the processing The two inflection points of the obtained weld image are used to obtain the center of the weld, and its three-dimensional coordinates are used as the "origin" of the weld tracking.

Next, starting from the left starting point, the center of the light bar is extracted from the light bar center of the previous column of pixels as the center of the fitted data range, that is, the light bar center point of the previous column of pixels is used as the Gaussian curve fitting light bar center rough extraction result , the amount of data to be fitted is 1.5 times the width of this column of light bars, and then the center of each column of light bars is accurately calculated in turn. Considering the actual image size and improving the processing speed, the light bar center of 80 columns of pixels is continuously extracted as the extraction result of the left segment light bar center.

Then, the right starting point is obtained in a manner similar to calculating the left starting point. On the contrary, the center of the light bar is extracted from the right starting point forward, that is, the extraction of the center of the light bar of the pixels in the previous column is to extract the fitted data of the pixel light bar of the next column. The center is taken as the center, and the amount of data is still 1.5 times the width of the light bar in this column, and the light bar center of the 80-column pixels is sequentially calculated to obtain the extraction result of the right segment light bar center.

Finally, the center of the left segment light bar and the center of the right segment light bar are combined together as the final light bar center extraction result.

2.3 Inflection point positioning

Accurate inflection point location is the key to obtaining weld information. Common inflection point location methods include template matching method and slope analysis method. The template matching method is a method of searching for a known template in the collected weld image to locate the inflection point. This method has high extraction accuracy, but it needs to replace the template frequently for multi-layer multi-pass welding; slope analysis method It is a method of extracting the inflection point according to the change of the slope of the light strip on the pipeline. This method is fast and has strong applicability. It can also have an ideal inflection point extraction result during cover welding. There will be some extra inflection points. Aiming at the shortcomings of the slope analysis method, the present invention designs an inflection point positioning algorithm that combines the slope analysis method with the "nearest principle", which has the characteristics of fast processing speed, high precision, and not easy to be disturbed.

First, use the slope analysis method to extract all possible inflection points. The slope calculation formula of the center point of the i-th column light bar on the image is:

All possible inflection points are obtained by analyzing the slopes of all light bar center points. Next, according to the position of the light bar in each image, the difference is very small, that is, the position difference of the inflection point of the welding seam edge of each image is also very small. The inflection point with a distance of less than 3 pixels is regarded as the inflection point of the weld edge of this image. Use this "nearest principle" to find the two inflection points of the edge of the weld, take their midpoint as the center of the weld, and obtain its three-dimensional coordinates, and make a difference with the "origin" to perform the vertical and height direction of the weld. Seam tracking.

Using the "nearest principle" can not only locate the inflection point quickly and accurately, but also has little effect on the welding seam tracking when the image processing fails, which can be eliminated in the next time the welding torch is rectified.

3. Implementation

The algorithm extracts the center of the light bar and locates the inflection point for some images collected during the welding process, as shown in Figure 3 to Figure 6. It can be seen from the figure that whether it is a V-shaped weld or an inverted trapezoidal weld, the improved Gaussian curve fitting method can very accurately extract the center of the light bar near the inflection point under the interference of arc light and splash, and then use the slope analysis method. Use the "nearest principle" to get the position information of the inflection point of the weld edge.

According to the characteristics of the welding seam image, the invention proposes a light bar center extraction algorithm based on Gaussian curve fitting in the curve fitting method. According to the position of the light bar on the image, the data to be fitted is adaptively fitted, and the processing speed is fast and the precision is high. High and strong anti-interference ability.

Aiming at the problem that some redundant inflection points will appear in the existing slope analysis method when the image quality is not high, the present invention proposes an inflection point positioning algorithm combining the slope analysis method with the "closest principle". The algorithm has both high speed and applicability. It can also have the advantages of relatively ideal inflection point extraction results during cover welding, and at the same time can quickly and accurately locate the inflection point, and the impact on the welding seam tracking is also small when the image processing fails, which can be used in the next welding torch. The influence will be eliminated when the deviation is corrected.

The method of the invention can extract the center of the light strip near the inflection point very accurately under the interference of arc light and splash for the V-shaped welding seam or the inverted trapezoidal welding seam, and then obtain the inflection point of the edge of the welding seam by the slope analysis method and the "nearest principle". Position information, so as to quickly and accurately obtain the three-dimensional shape of the weld under strong interference conditions, and then realize the weld tracking.

Although embodiments of the present invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, and substitutions can be made in these embodiments without departing from the principle and spirit of the invention and modifications, the scope of the present invention is defined by the appended claims and their equivalents.

Claims (6)

1. Structured light three-dimensional measurement model and image processing method of pipeline all-position weld, characterized in that, comprising the following steps: S1, constructing a mathematical model for the three-dimensional measurement of the pipeline weld structured light, and converting the coordinate values in the image coordinate system to the camera coordinate system; S1 further includes: constructing a mathematical model for the three-dimensional measurement of the pipeline weld structured light, and converting the camera coordinate system O _C X _C Y _C Z _C is the measurement coordinate system, the O _C X _C axis is perpendicular to the walking direction of the robot, the O _C Y _C axis is parallel to the walking direction of the robot, and O _C Z _C is the optical axis of the camera lens. The relationship between the coordinates in the coordinate system and the ideal computer image coordinates of this point, the relationship between the ideal computer image coordinates of this point and the actual image coordinates of this point, and the equation of the light plane in the camera coordinate system, get the image The coordinates of the coordinates of any point on the camera in the camera coordinate system; S2. Image preprocessing: real-time acquisition of weld images during the welding process, and preprocessing of the collected weld images; S3. Extraction of the center of the light bar: use the improved Gaussian fitting method to extract the light bar center of the weld image; the improved Gaussian fitting method includes: according to the difference between the light bar centers of two adjacent columns of pixels not exceeding one pixel, The center of the light bar of a column of pixels is used as the center of the data range fitted by the Gaussian curve of the current column of pixels, and the amount of data to be fitted is adaptively 1.5 times the number of pixels of the width of the light bar in the current column. Due to the characteristics of low quality and high quality on the pipeline, the extraction of the center of the light strip is divided into two sections, which are extracted from both sides to the middle. In order to improve the processing speed, only the center of the light strip of two small sections of the light strip near the edge point of the weld is extracted; S4. Inflection point location: use the algorithm combining the slope analysis method and the "nearest principle" to locate the inflection point and extract the welding seam information; The inflection point of the pixel is taken as the inflection point of the seam edge of this image. 2 . The structured light three-dimensional measurement model and image processing method for all-position welds of pipelines according to claim 1 , wherein the step of collecting images of welds further comprises: collecting images of welds by using a welding robot, and the welding robots A laser, an industrial camera, and a welding gun are provided, the welding gun is aimed at the welding seam, the laser emits laser light to irradiate the welding seam, and the industrial camera collects the welding seam image of the irradiation point. 3. The structured light three-dimensional measurement model and image processing method of pipeline all-position welds according to claim 1, wherein the step of preprocessing the collected weld images further comprises: windowing, denoising, smoothing , erosion, dilation and Gaussian filtering. 4. The structured light three-dimensional measurement model and image processing method of pipeline all-position welds according to claim 1, characterized in that, S3 further comprises: utilizing slope analysis method to extract all possible inflection points, the i-th column of light on the graph The slope of the bar center point is calculated as:

All possible inflection points are obtained by analyzing the slopes of all light bar center points. 5. The structured light three-dimensional measurement model and image processing method for all-position welds of pipelines according to claim 1, wherein the method further comprises: collecting images of welds before starting welding, extracting information of welds, and Calculate the "origin". 6. The structured light three-dimensional measurement model and image processing method of the pipeline all-position weld according to any one of claims 1 and 5, characterized in that, two of the weld edges found by using the "nearest principle" The midpoint of the inflection point is used as the center of the weld, the three-dimensional coordinates of the midpoint are obtained, and the coordinates are subtracted from the "origin" to perform weld tracking in the vertical and height directions.

Images