CN106181162B - A kind of real-time weld joint tracking detection method based on machine vision - Google Patents

Abstract

The invention discloses a real-time welding seam tracking detection system based on machine vision, which includes: an industrial computer as the core control equipment of the system; an image acquisition system composed of a light source controller, an LED light source, an image acquisition card and a CCD camera unit; and a motion control unit composed of a robot controller and a welding robot; the present invention also includes a real-time weld seam tracking detection based on machine vision, including steps such as image acquisition → image processing → data processing → motion control; the present invention adopts The method of real-time online detection does not require manual redundant operations, and corrects deviations by comparing the expected weld trajectory generated by the groove image with the actual weld trajectory; and when the on-site welding environment is different from the manual teaching environment and the offline programming simulation environment In the case of certain differences, real-time weld seam tracking can be carried out to overcome various interferences on the welding site and ensure the accuracy and reliability of the welding effect.

Description

A real-time seam tracking detection method based on machine vision

technical field

The invention relates to a real-time welding seam tracking detection system and method based on machine vision, which belongs to the technical field of machine vision and mechanical automation.

Background technique

Welding has always played an extremely important role in industrial production, and with the development of robots and visual inspection processing technology, the automation level of welding technology is increasing day by day, and the welding seam tracking inspection method has also developed from manual inspection to computer intelligent inspection.

Most of the existing welding robots are of the on-site manual teaching type or offline programming type. In these two methods, the welding parameters and welding paths of the welding robot are planned before production, so that the welding robot can repeat the previous work with high precision in the working space. Plan your exercise.

The traditional seam tracking detection method compares the planned seam path with the actual seam path to correct errors and realize the seam tracking. Such a seam tracking method can ensure the high-precision operation and seam tracking of the welding robot in the planning area, but it can only track the planned path, and has poor adaptability to emergencies.

In the actual welding process, there are many interferences at the welding site, and the shape of the weldment will change with the change of the production environment, such as the high temperature generated by the molten pool, low assembly accuracy, and sudden collisions during the welding process, etc. It has a corresponding impact on welding production; when the on-site welding environment is different from the manual teaching environment and the off-line programming simulation environment, the effect of traditional seam tracking is not ideal, which restricts the further development of existing seam tracking technology. develop.

Contents of the invention

Aiming at the problems existing in the above-mentioned prior art, the present invention provides a real-time weld seam tracking detection system and method based on machine vision, which has high accuracy and adaptability, and can perform real-time weld seam tracking and overcome various problems in the welding site. Interference, so as to ensure the accuracy and reliability of the welding effect.

In order to achieve the above object, the technical solution adopted in the present invention is: a real-time weld seam tracking detection method based on machine vision, which includes:

Industrial computer as the core control equipment of the system;

Image acquisition unit composed of light source controller, LED light source, image acquisition card and CCD camera;

And a motion control unit composed of a robot controller and a welding robot;

The industrial computer is responsible for image and data processing and calculation; the light source controller controls the LED light source to move with the movement of the welding torch, so that the light source covers the weld seam and the groove to be welded, ensuring the lighting of the image acquisition area ;

Described CCD camera comprises CCD camera one, CCD camera two; CCD camera one, CCD camera two are respectively installed in the both sides of welding robot welding torch, moves with the movement of welding torch; CCD camera 2 is directly facing the welding position under the welding torch;

Also includes the following steps:

1) Image collection: Before welding starts, the light source controller controls the LED light source to irradiate the groove and weld in the way of forward lighting. The CCD camera first collects the image of the groove area to be welded. After time t, the CCD camera starts Collect the weld seam image of the welding part, and both initial images are transmitted to the industrial computer;

Wherein t is the delay time that the groove area is photographed by the CCD camera 1 to be photographed by the CCD camera 2 after the groove area is welded;

2) Image processing: through image filtering, image segmentation, morphological processing and edge detection processing methods to obtain locally enlarged edge images of grooves and weld areas, the specific process of image processing is as follows:

(1) Denoise the initial image by using the Gaussian filter method;

A digital image can be expressed as a two-dimensional array f(x, y), where x and y represent pixel coordinates, f(x, y) represents the grayscale of the image, where G(x, y) is a two-dimensional Gaussian function ;

f _s (x,y)=G(x,y)*f(x,y)

f _s (x, y) represents the digital image after Gaussian filtering;

(2) Use the maximum between-class variance method to segment the image:

Use a threshold to divide the entire data into two classes. If the variance between the two classes is the largest, then this threshold is the best threshold, where T is the gray value of the pixel in the image;

That is to choose T ^* which makes the largest σ ² (T) as the best segmentation threshold;

(3) Morphological processing of the image using the dilation-corrosion algorithm:

First use the structural elements to expand f(x, y), and then use the structural elements to perform corrosion operations on the results; using the expansion and corrosion algorithm can eliminate redundant holes in the target object, connect similar objects, and fill in the small interior on the contour line Concave sharp corners to smooth the boundaries of objects;

(4) Use the Canny edge detection algorithm to carry out edge detection on the image:

Compute the gradient magnitude and direction of the filtered image,

Gradient magnitude:

Gradient direction:

Then perform non-maximum suppression on the gradient amplitude, if the value of M(x, y) is smaller than one of any two neighborhoods of the pixel, then set g _N (x, y) = 0; otherwise, set g _N (x, y)=M(x, y); g _N (x, y) represents the image after non-maximum suppression;

Finally, a double-threshold algorithm is used to determine the edge point of the image: set an upper threshold T _H and a lower threshold T _L , and if the pixel in the image is greater than the upper threshold, that is, g _N (x,y)> _TH , it is considered to be the boundary , if it is less than the threshold lower bound, that is, g _N (x, y)<T _L , it is considered that it must not be a boundary, and those between the two are considered candidates;

3) Data processing: process the partially enlarged edge image of the groove area, judge the groove type according to the pattern recognition algorithm, and calculate the planned expected weld trajectory; process the weld image of the welding part, and calculate the weld trajectory actually generated by the equipment, After filtering, two smooth weld center trajectories are obtained, and the two weld trajectories are compared and the tracking correction amount is calculated;

4) Motion control: The industrial computer performs real-time deviation correction according to the deviation correction amount, controls the movement of the welding robot and further welding, so as to realize the precise tracking of the weld seam.

Compared with the existing seam tracking method, the method and system adopted by the present invention have the following advantages and effects:

1. The present invention adopts the method of real-time online detection, which does not require manual redundant operations, and the welding robot can automatically perform weld seam tracking and detection after welding starts;

2. The present invention uses the groove image to generate the expected weld trajectory, and has strong adaptability to the on-site welding environment. When the on-site welding environment is different from the manual teaching environment and the off-line programming simulation environment, real-time welding can be performed. Tracking and overcoming all kinds of interference on the welding site to ensure the accuracy and reliability of the welding effect.

Description of drawings

Fig. 1 is a schematic diagram of the overall principle of the solution of the present invention.

Fig. 2 is a schematic diagram of the system principle of the present invention.

Detailed ways

The present invention will be further described below in conjunction with accompanying drawing.

The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some, not all, embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

Figure 2 is a real-time weld tracking detection system based on machine vision, which includes:

Industrial computer as the core control equipment of the system;

Image acquisition unit composed of light source controller, LED light source, image acquisition card and CCD camera;

And a motion control unit composed of a robot controller and a welding robot;

The industrial computer is responsible for image and data processing and calculation; the light source controller controls the LED light source to move with the movement of the welding torch, so that the light source covers the weld seam and the groove to be welded, ensuring the lighting of the image acquisition area ;

Described CCD camera comprises CCD camera one, CCD camera two; CCD camera one, CCD camera two are respectively installed in the both sides of welding robot welding torch, moves with the movement of welding torch; The second CCD camera is facing the welding position under the welding torch.

As shown in Figure 1, it is a real-time seam tracking detection method based on machine vision, which adopts the above-mentioned system and includes the following steps:

1) Image collection: Before welding starts, the light source controller controls the LED light source to irradiate the groove and weld in the way of forward lighting. The CCD camera first collects the image of the groove area to be welded. After time t, the CCD camera starts Collect the weld image of the welding part, and transmit the data of the two initial images to the industrial computer;

Among them, t is the delay time when the groove area is photographed by CCD camera 1 and captured by CCD camera 2 after the region is welded;

2) Image processing: through image filtering, image segmentation, morphological processing and edge detection processing methods to obtain locally enlarged edge images of grooves and weld areas;

3) Data processing: process the partially enlarged edge image of the groove area, judge the groove type according to the pattern recognition algorithm, calculate the planned expected weld trajectory, process the weld image of the welding part, and calculate the weld trajectory actually generated by the equipment, After filtering, two smooth weld center trajectories are obtained, and the two weld trajectories are compared to calculate the tracking correction amount.

4) Motion control: The industrial computer performs real-time deviation correction according to the deviation correction amount, controls the movement of the welding robot and further welding, so as to realize the precise tracking of the weld seam.

In the above scheme, steps 1), 2), and 3) all detect and process two different areas: the groove area to be welded and the weld seam area.

The groove area to be welded takes the blunt edge at the groove as the application object, and extracts the morphological features of the blunt edge of the groove through the difference between the gray value of the blunt edge of the groove and the rest of the gray value, judges the type of the groove and calculates the expected welding The seam center trajectory; the welding seam area takes the weld seam as the application object, and calculates the actual weld seam center trajectory through the edge information of the weld seam.

In the above scheme, if the blunt edge information is not collected in step 1), that is, the welding object does not have a bevel, then in the data processing of step 3), the set of center points of the gaps on both sides of the object to be welded is the expected weld center track.

In the above scheme, t is the delay time when the groove area is photographed by CCD camera 1 and captured by CCD camera 2 after the groove area is welded. It is related to the welding process, welding speed, and computer processing speed of the welding equipment. Expected and actual weld trajectories are weld trajectories that match each other in the same area.

Among them, the image processing in step 2) is mainly divided into four parts: image filtering, image segmentation, morphological processing and edge detection. The specific process is as follows:

(1) Denoise the initial image by using the Gaussian filter method;

A digital image can be expressed as a two-dimensional array in the form f(x, y), where x and y represent pixel coordinates respectively, and f(x, y) represents the grayscale of the image;

f _s (x,y)=G(x,y)*f(x,y)

f _s (x, y) represents the digital image after Gaussian filtering;

(2) Use the maximum between-class variance method to segment the image:

Use a threshold to divide the entire data into two classes. If the variance between the two classes is the largest, then this threshold is the best threshold;

That is to choose T ^* which makes σ ² (T _b ) the largest as the optimal segmentation threshold;

(3) Morphological processing of the image using the dilation-corrosion algorithm:

First use the structural elements to perform expansion operation on f(x, y), and then use the structural elements to perform corrosion operation on the result. The dilation and erosion algorithm can eliminate redundant holes in the target object, connect similar objects, and fill in the small concave sharp corners on the contour line to smooth the boundary of the object;

(4) Use the Canny edge detection algorithm to carry out edge detection on the image:

Compute the gradient magnitude and direction of the filtered image,

Gradient magnitude:

Gradient direction:

Then perform non-maximum suppression on the gradient amplitude, if the value of M(x, y) is smaller than one of any two neighborhoods of the pixel, then set g _N (x, y) = 0; otherwise, set g _N (x, y) = M(x, y). g _N (x, y) represents the image after non-maximum suppression;

Finally, a double-threshold algorithm is used to determine the edge point of the image: set an upper threshold T _H and a lower threshold T _L , if the pixel in the image is greater than the upper threshold, it is considered to be the boundary (g _N (x, y) ≥ T _H ), if it is less than the lower threshold, it is considered not necessarily the boundary (g _N (x, y)≤T _L ), and those between the two are considered candidates.

In the above scheme, step 3) judges the groove type by pattern recognition algorithm, and then calculates the expected weld trajectory according to the welding characteristics of different grooves, combined with the groove image obtained in step 2); the expected weld trajectory and the actual weld trajectory Comparing after optimization and smoothing, the optimal calculation of deviation amount, deviation direction and deviation speed is carried out.

Compared with the existing seam tracking method, the method and system adopted by the present invention have the following advantages and effects:

1. The present invention adopts the method of real-time online detection, which does not require manual redundant operations, and the welding robot can automatically perform weld seam tracking and detection after welding starts;

2. The present invention uses the groove image to generate the expected weld trajectory, and has strong adaptability to the on-site welding environment. When the on-site welding environment is different from the manual teaching environment and the off-line programming simulation environment, real-time welding can be performed. Tracking and overcoming all kinds of interference on the welding site to ensure the accuracy and reliability of the welding effect.

It will be apparent to those skilled in the art that the invention is not limited to the details of the above-described exemplary embodiments, but that the invention can be embodied in other specific forms without departing from the spirit or essential characteristics of the invention. Accordingly, the embodiments should be regarded in all points of view as exemplary and not restrictive, the scope of the invention being defined by the appended claims rather than the foregoing description, and it is therefore intended that the scope of the invention be defined by the appended claims rather than by the foregoing description. All changes within the meaning and range of equivalents of the elements are embraced in the present invention. Any reference sign in a claim should not be construed as limiting the claim concerned.

The above is only a preferred embodiment of the present invention, and is not intended to limit the present invention. Any minor modifications, equivalent replacements and improvements made to the above embodiments according to the technical essence of the present invention shall be included in the technical aspects of the present invention. within the scope of protection of the program.

Claims (1)

1. a kind of real-time weld joint tracking detection method based on machine vision, which is characterized in that it includes：

Industrial personal computer as system core control device；

The image acquisition units being made of light source controller, LED light source, image pick-up card and CCD camera；

And the motion control unit being made of robot controller, welding robot；

The industrial personal computer is responsible for the processing and calculating of image, data；The light-source controller controls LED light source is with welding gun It moves and moves, so that light source is covered at weld seam and groove to be welded, ensure the daylighting degree of image acquisition region；

The CCD camera includes CCD camera one, CCD camera two；CCD camera one, CCD camera two are installed respectively In the both sides of welding robot welding gun, moved with the movement of welding gun；And CCD camera one is towards welding gun front end groove to be welded Place, two positive Butt welding gun of CCD camera lower section welding position；

It is further comprising the steps of：

1) Image Acquisition：Before welding starts, light-source controller controls LED light source is irradiated to groove and weldering in a manner of front illumination At seam, CCD camera acquires the image in groove region to be welded at the beginning, and after time t, CCD camera two starts to acquire weld part The weld image divided, two initial pictures are transferred in industrial personal computer；

When wherein t is that groove region takes the delay shot by CCD camera two after the welding of groove region by CCD camera one Between；

2) image procossing：Groove and weldering are obtained by image filtering, image segmentation, Morphological scale-space and edge detection process method The edge image of region partial enlargement is stitched, image procossing detailed process is as follows：

(1) gaussian filtering method is used to carry out denoising to initial pictures；

Digital picture can be expressed as the form f (x, y) of two-dimensional array, and x, y indicate that pixel point coordinates, f (x, y) indicate respectively The gray scale of image, wherein G (x, y) are two-dimensional Gaussian function；

f_s(x, y)=G (x, y) * f (x, y)

f_s(x, y) indicates the digital picture after gaussian filtering；

(2) maximum variance between clusters are used to carry out image segmentation to image：

Entire data are divided into two classes using a threshold value, if the variance between two classes is maximum, then this threshold value is just It is best threshold value, wherein T is the gray value of pixel in image；

I.e. selection makes maximum σ²(T) T^*As optimal segmenting threshold；

(3) dilation erosion algorithm is used to carry out Morphological scale-space to image：

Dilation operation first is carried out to f (x, y) with structural element, erosion operation then is carried out to result with structural element；Using swollen Swollen erosion algorithm can eliminate the extra hole of target object, connect similar object, while fill up tiny interior on contour line Recessed wedge angle is with the boundary of smooth object；

(4) Canny edge detection algorithms are used to carry out edge detection into image：

Gradient magnitude and the direction of filtering image are calculated,

Gradient magnitude：

Gradient direction：

Non-maxima suppression is carried out to gradient magnitude again, if the value of M (x, y) is less than one of any two neighborhood of pixel, Then enable g_N(x, y)=0；Otherwise, g is enabled_N(x, y)=M (x, y)；g_N(x, y) indicates to carry out the image after non-maximum suppression；

Finally use dual threashold value-based algorithm process decision chart as marginal point：Set a threshold value upper bound T_HWith threshold value lower bound T_L, the picture in image Vegetarian refreshments is if it is greater than the threshold value upper bound, that is, g_N(x,y)>T_HThen think necessarily boundary, is less than threshold value lower bound, that is, g_N(x,y)<T_LThen recognize For it is inevitable be not boundary, between the two be then considered candidate item；

3) data processing：The edge image for handling groove region partial enlargement, judges groove type according to algorithm for pattern recognition, counts Calculate planned expectation seam track；Handle the weld image of welding portion, the seam track that computing device actually generates, warp Two smooth Weld pipe mill tracks are obtained after filtering, and two seam tracks are compared and calculate tracking correction amount；

4) motion control：Industrial personal computer carries out real-time deviation correcting according to correction amount, controls the movement of welding robot and further weldering It connects, to realize the accurate tracking of weld seam.