CN104708239A - Large-scale marine pipeline initial point recognition and location method and system based on CCD - Google Patents

Abstract

The invention discloses a CCD-based method and system for identifying and locating the starting point of a large ocean pipeline. Step 1. Preliminary positioning of the weld seam position: CCD industrial digital camera collects weld seam pictures, and the main control computer calculates the real-time change of the pixel deviation between the center of the welding torch and the center line of the weld seam, so that the welding robot is initially positioned above the weld seam; Step 2, extracts the Pixel coordinates of the starting point: identify a welding spot in the positioning welding, the positioner drives the pipeline to make a rotary motion, collects images in real time, and performs image processing on the rectangular area of interest including the welding torch. If a welding spot is detected, the positioner stops rotating. Extract the pixel coordinates of the centroid of the solder joint; step 3, locate the starting point: the welding robot moves horizontally for a short distance along the Y direction of the base coordinate system, collects the picture of the weld seam, extracts the pixel coordinates of the centroid of the solder joint again, and uses stereo vision to calculate the starting point The three-dimensional coordinates of the starting point, the welding robot is positioned to the starting point.

Description

Method and system for identifying and locating the starting point of large ocean pipelines based on CCD

Technical field:

The invention relates to a CCD-based method and system for identifying and locating the starting point of a large ocean pipeline, which belongs to the technical field of automatic welding.

Background technique:

For the welding of large ocean pipelines, our country is still at the level of semi-automatic welding. In the actual welding process, due to the long length of the pipeline and the diameter of the pipeline can reach 1000mm or even larger, when working in the workshop, the positioning welding is carried out first, and the pipeline is temporarily fixed, and then the position of the welding torch is basically fixed, and the positioner The welding is completed by driving the pipeline to make a circular arc rotary motion. Since a welding point in the positioning welding is generally used as the starting point, after the pipe is clamped, the position of the welding point is not fixed each time. For rotary motion, when the operator observes that there are welding spots appearing on the top, the positioner is manually stopped, and the operator uses the teaching box or other methods to manually move the robot to the starting point, so the level of the operator is high and the labor intensity is high. Large, the positioning accuracy cannot be guaranteed, and more importantly, the welding efficiency is seriously reduced, which restricts the development of marine pipeline construction.

Due to the high demand of marine pipelines, using manual positioning of the starting point will greatly increase the time cost. Therefore, combining information sensing technology with existing welding robots to develop an automated pipeline welding system to realize automatic positioning of the starting point and improve the efficiency of the entire pipeline welding system is an inevitable trend of current automated welding technology, and it is also a technology The inevitable need for innovation.

At present, as a research hotspot in the field of welding, visual sensing technology has achieved some remarkable results, but it is mostly applied to some structural parts, and there are few researches and applications on the application of visual sensing to large-scale pipeline welding. . At present, in visible reports, the title is "Robot Initial Welding Position Recognition System and Method Based on Monocular Vision Sensing" (application number 201110224808.4) in order to enable the robot to accurately move to the starting point of the welding path, providing a The initial welding position recognition method of the robot based on the monocular vision sensor, but the weld seam of the workpiece to be welded in this method is horizontal, and the height of the vision sensor is fixed, which is not suitable for the precise positioning of the starting point of large-scale pipeline welding. The document "Chen Haiyong, Fang Zaojun, Xu De, et al. Vision-based start point recognition and positioning control of thin steel plate welding robot [(Journal) Robotics], 2013, Volume 35, Issue 1, 90-97" proposed a The visual control method of starting point recognition and welding torch precise centering of welding seam starting point automatically determines a circular region of interest containing welding torch feature points, but this application is aimed at thin steel plate welding and is not suitable for large pipe welding.

Therefore, it is necessary to improve the prior art to solve the deficiencies of the prior art.

Invention content:

Aiming at the deficiencies in the above existing technologies and production needs, a CCD-based method and system for identifying and locating the starting point of large-scale ocean pipelines is provided, which can automatically extract a welding spot of the pipeline weld in the positioning welding, and realize the location of the welding starting point. Accurate positioning and control of the robot to move to the starting point, thereby solving the technical problem of the robot automatically positioning to the welding starting point in large-scale pipeline welding.

The present invention adopts the following technical scheme: a CCD-based method for identifying and locating the starting point of a large ocean pipeline, which includes the following steps:

Step 1: Preliminary positioning of the weld position

(a) Keep the pipeline to be welded still, the welding robot moves to the preset position, and the weld seam image of the pipeline to be welded is collected by the CCD industrial digital camera, and the main control computer receives the picture through the USB cable and performs image processing for preliminary identification Weld seam, extract the center line of the weld seam, scan the middle row of the picture, get a point on the center line of the weld seam, calculate the pixel deviation between the center of the welding torch and the abscissa direction of this point, and feed it back to the welding robot;

(b) The welding robot moves to the center of the weld. During the movement, the CCD collects images in real time. The main control computer receives and extracts the center line of the weld in real time, scans the middle row of the picture, and obtains a point on the center line of the weld. Calculate the distance between the welding torch center and The real-time change of the pixel deviation in the abscissa direction of the point. When the pixel deviation is zero, the main control computer sends a stop signal to the welding robot, and the welding robot stops moving, so that the welding robot is initially positioned above the weld seam;

Step 2: Extraction of pixel coordinates of welding starting point

(a) The main control computer sends a start signal to the positioner, the positioner drives the pipeline to make a rotary motion, the CCD collects images in real time, and the main control computer receives and processes the image of the rectangular region of interest including the welding torch in real time. When the welding spot in the positioning welding of the pipeline appears in the area of interest, a stop signal is sent to the positioner, and the pipeline stops rotating;

(b) The main control computer carries out image processing to the rectangular region of interest comprising the solder joint, extracts the outer contour and the centroid of the solder joint, and the pixel coordinates of the centroid are just used as the pixel coordinates of the starting point of the pipeline;

Step 3: Positioning of welding starting point

(a) The main control computer sends a motion signal to the welding robot, and the welding robot moves horizontally for a short distance along the Y direction of the base coordinate system. The CCD industrial digital camera collects the image of the weld seam, and the main control computer receives the picture and is interested in the rectangle containing the welding spot Image processing is performed on the area, and the pixel coordinates of the centroid of the solder joint are extracted again by image processing;

(b) Use the general binocular vision to calculate the three-dimensional coordinates of the starting point in the camera coordinate system, and according to the transformation matrix between the tool coordinate system and the robot base coordinate system and the transformation matrix between the tool coordinate system and the camera coordinate system, Transform the obtained three-dimensional coordinates of the starting point in the camera coordinate system to the three-dimensional coordinates in the robot base coordinate system.

Further, the specific steps for calculating the pixel deviation in step 1(a) are:

(1) Calculate the pixel coordinates (u ,v);

(2) Perform image processing on the collected pictures, extract the edge of the weld and the center line of the weld, scan the middle row of the weld picture from the left, and obtain a point on the center line of the weld, denoted as (u _k , v _k ), calculate the pixel deviation Δu=uu _k from the center of the welding torch to the center line of the weld.

Further, the moving direction of the welding robot in step 1(b) is determined by the positive or negative of the pixel distance Δu, and the machine only moves horizontally along the Y direction of the base coordinate system.

Further, the transformation matrix between the tool coordinate system and the robot base coordinate system in step 3 (b) is obtained by the main control computer reading the pose conversion of the welding robot from the robot control cabinet, and the tool coordinate system The transformation matrix between and the camera coordinate system is obtained from the hand-eye calibration experiment.

The present invention adopts the following technical solutions: a CCD-based large-scale marine pipeline starting point identification and positioning system, which includes a positioner, a visual sensor, a welding robot, a robot control cabinet, a welding power supply and a main control computer; the visual sensor includes The CCD industrial digital camera rigidly fixed on the welding torch at the end of the welding robot, the industrial digital camera moves together with the welding robot; the positioner, the welding power supply are connected to the robot control cabinet, and the main control computer is connected to the robot control cabinet through Ethernet connection, the main control computer sends a signal to the robot control cabinet and then controls the movement of the positioner; the main control computer and the visual sensor are connected through a USB; the main control computer includes an image acquisition module, an image processing module and an information feedback and control module , the image acquisition module controls the CCD industrial digital camera through the USB line to collect the weld picture and sends it to the main control computer. Coordinate positioning has three image processing sub-modules. The information feedback and control module feeds back the results of image processing to the robot control cabinet, controls the rotary motion of the positioner and the motion of the robot, and finally controls the welding robot to move to the welding starting point.

The invention has the following beneficial effects: the invention provides a CCD-based method and system for identifying and locating the starting point of a large-scale ocean pipeline, which meets the current requirements of robot pipeline welding automation. Only need to add the visual sensor system, without affecting the flexibility and accessibility of the welding torch, use the main control computer as the control center, automatically collect the welding seam pictures, realize the automatic identification of the welding starting point, so that the welding robot can automatically locate the welding seam The starting point avoids the process of manual positioning, saves time and cost, reduces manual positioning errors, reduces the labor intensity of workers, improves the automation level and welding efficiency of pipeline welding, and ensures the positioning accuracy of the starting point. The automatic welding has laid a good foundation.

Description of drawings:

Fig. 1 is a working flow diagram of a specific embodiment of the present invention.

Figure 2 is a diagram of the motion process of the welding robot during the identification and positioning process.

Figure 3 is a schematic diagram of the poses of each coordinate system in the system during the welding process.

Fig. 4 is a schematic diagram of the CCD-based pipeline welding robot platform of the present invention.

Fig. 5 is a functional block diagram of the main control computer in the present invention.

in:

1-Positioner; 2-Pipeline; 3-Vision sensor; 4-Welding robot; 5-Robot control cabinet; 6-Welding power supply; 7-Main control computer.

Detailed ways:

In order to make the purpose, technical solutions and beneficial effects of the present invention clearer, the present invention will be further described in detail below in conjunction with specific embodiments and with reference to the accompanying drawings, but the implementation of the present invention is not limited thereto.

Please refer to shown in Fig. 1, the present invention is based on CCD large ocean pipeline welding start point identification and location method, and it comprises the following steps:

Step 1: Preliminary positioning of the weld position

(a) Keep the pipeline to be welded still, the welding robot moves to the preset position, and the weld seam image of the pipeline to be welded is collected by the CCD industrial digital camera, and the main control computer receives the picture through the USB line and performs image processing. Preliminary Identify the weld, extract the center line of the weld, scan the middle row of the picture, get a point on the center line of the weld, calculate the pixel deviation Δu between the center of the welding torch and the abscissa direction of this point, and feed it back to the welding robot;

(b) The welding robot moves to the center of the weld. During the movement, the CCD collects images in real time. The main control computer receives and extracts the center line of the weld in real time, scans the middle row of the picture, and obtains a point on the center line of the weld. Calculate the distance between the welding torch center and The real-time change of the pixel deviation Δu in the abscissa direction of the point. When the pixel deviation is zero, the main control computer sends a stop signal to the welding robot, and the welding robot stops moving, so that the welding robot is initially positioned above the weld. As shown in Figure 2, when the welding robot moves to position 1, Δu=0. The moving direction of the welding robot is determined by the positive or negative of the pixel distance Δu. The welding robot only moves horizontally along the Y direction of the base frame of the welding robot. The position of the base frame of the welding robot is shown in Figure 3.

Step 2: Extraction of pixel coordinates of welding starting point

(a) The main control computer sends a start signal to the positioner, the positioner drives the pipeline to make a rotary motion, the CCD collects images in real time, and the main control computer receives and processes the image of the rectangular region of interest including the welding torch in real time. When the welding spot in the positioning welding of the pipeline appears in the area of interest, a stop signal is sent to the positioner, and the pipeline stops rotating;

(b) The main control computer performs image processing on the rectangular region of interest including the welding spot, extracts the outer contour and centroid of the welding spot, and the pixel coordinates of the centroid are used as the pixel coordinates of the starting point of pipeline welding.

Step 3: Positioning of welding starting point

(a) The main control computer sends motion signals to the welding robot, and the welding robot moves horizontally for a short distance along the Y direction of the base coordinate system to position 2 in Figure 2. The CCD camera collects images of weld seams, and the main control computer receives the pictures and Perform image processing on the rectangular region of interest containing the solder joint, and use the image processing to extract the pixel coordinates of the solder joint centroid again;

(b) Use the general binocular vision to calculate the three-dimensional coordinates of the starting point in the camera coordinate system, and according to the transformation matrix between the tool coordinate system and the robot base coordinate system and the transformation matrix between the tool coordinate system and the camera coordinate system, Transform the obtained three-dimensional coordinates of the starting point in the camera coordinate system to the three-dimensional coordinates in the robot base coordinate system. The transformation matrix between the tool coordinate system and the robot base coordinate system can be calculated by the main control computer reading the pose of the robot end effector from the robot control cabinet, and the transformation matrix between the tool coordinate system and the camera coordinate system It can be obtained from the hand-eye calibration experiment.

The specific calculation steps of the pixel deviation Δu in step 1(a) are:

(1) According to the internal parameter matrix of the CCD industrial digital camera, the conversion matrix between the tool coordinate system and the base coordinate system of the welding robot, and the conversion matrix between the tool coordinate system and the camera coordinate system, use formula (1) and formula (2) ) Calculate the pixel coordinates (u, v) of the welding torch center in the picture;

$\left[\begin{array}{c}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}\\ {\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}\\ {\mathrm{<span\; class="notranslate">\; z</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}\\ \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]<span\; class="notranslate">\; =</span>\left[\begin{array}{cc}{\mathrm{<span\; class="notranslate">\; R</span>}}^{\mathrm{<span\; class="notranslate">\; h</span>}}& {\mathrm{<span\; class="notranslate">\; t</span>}}^{\mathrm{<span\; class="notranslate">\; h</span>}}\\ {\mathrm{<span\; class="notranslate">\; o</span>}}^{\mathrm{<span\; class="notranslate">\; T</span>}}& \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]\left[\begin{array}{cc}{\mathrm{<span\; class="notranslate">\; R</span>}}^{\mathrm{<span\; class="notranslate">\; B</span>}}& {\mathrm{<span\; class="notranslate">\; t</span>}}^{\mathrm{<span\; class="notranslate">\; B</span>}}\\ {\mathrm{<span\; class="notranslate">\; o</span>}}^{\mathrm{<span\; class="notranslate">\; T</span>}}& \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]\left[\begin{array}{c}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; W</span>}}\\ {\mathrm{<span\; class="notranslate">\; Y</span>}}_{\mathrm{<span\; class="notranslate">\; W</span>}}\\ {\mathrm{<span\; class="notranslate">\; Z</span>}}_{\mathrm{<span\; class="notranslate">\; W</span>}}\\ \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; HB</span>}\left[\begin{array}{c}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; W</span>}}\\ {\mathrm{<span\; class="notranslate">\; Y</span>}}_{\mathrm{<span\; class="notranslate">\; W</span>}}\\ {\mathrm{<span\; class="notranslate">\; Z</span>}}_{\mathrm{<span\; class="notranslate">\; W</span>}}\\ \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>$

$\left[\begin{array}{c}\mathrm{<span\; class="notranslate">\; u</span>}\\ \mathrm{<span\; class="notranslate">\; v</span>}\\ \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]<span\; class="notranslate">\; =</span>\left[\begin{array}{ccc}{\mathrm{<span\; class="notranslate">\; k</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}& \mathrm{<span\; class="notranslate">\; 0</span>}& {\mathrm{<span\; class="notranslate">\; u</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& {\mathrm{<span\; class="notranslate">\; k</span>}}_{\mathrm{<span\; class="notranslate">\; the\; y</span>}}& {\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]\left[\begin{array}{c}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}<span\; class="notranslate">\; /</span>{\mathrm{<span\; class="notranslate">\; z</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}\\ {\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}<span\; class="notranslate">\; /</span>{\mathrm{<span\; class="notranslate">\; z</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}\\ \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; m</span>}\left[\begin{array}{c}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}<span\; class="notranslate">\; /</span>{\mathrm{<span\; class="notranslate">\; z</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}\\ {\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}<span\; class="notranslate">\; /</span>{\mathrm{<span\; class="notranslate">\; z</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}\\ \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; )</span>$

Among them, (X _W , Y _W , Z _W ) are the three-dimensional coordinates of the center of the welding torch in the coordinate system of the robot base; (u, v) are the pixel coordinates of the center of the welding torch in the image; M is the internal parameter matrix of the CCD, determined by the camera Obtained from the calibration experiment; H is the hand-eye transformation matrix (the transformation matrix from the tool coordinate system to the camera coordinate system), which can be obtained from the hand-eye calibration experiment. The origin of the tool coordinate system is the center of the welding torch; B is the transformation from the robot base coordinate system to the tool coordinate system The matrix can be calculated by the main control computer from the tool coordinates read from the robot control cabinet; R ^H is a 3×3 rotation matrix, t ^H is a 3×1 translation matrix, O=(0,0,0) ^T . Figure 3 shows the relative position relationship between the tool coordinate system O _t -X _t Y _t Z _t , the camera coordinate system O _c -X _c Y _c Z _c and the welding robot base frame O-XYZ.

(2) Perform image processing on the collected pictures, extract the edge of the weld and the center line of the weld, scan the middle row of the weld picture from the left, and obtain the pixel coordinates of a point on the center line of the weld, denoted as (u _k ,v _k ), calculate the pixel deviation Δu=uu _k from the center of the welding torch to the center line of the weld.

The specific calculation steps for the positioning of the three-dimensional coordinates of the starting point in step 3(b) are as follows: set the pixel coordinates of the centroids extracted at position 1 and position 2 to be (u ₁ , v ₁ ) and (u ₂ , v ₂ ), respectively. At position 1, the three-dimensional coordinates of the center of mass in the camera coordinate system are (x ₁ , y ₁ , z ₁ ), and at position 2, the three-dimensional coordinates of the center of mass in the camera coordinate system are (x ₂ , y ₂ , z ₂ ) , assuming that the relative position relationship of the CCD industrial digital camera at position 1 and position 2 is M ₁₂ =[R|T], R is a 3×3 rotation matrix, and T is a 3×1 translation matrix. Assuming that the focal length of the CCD industrial digital camera is f, according to the pinhole camera model, there are equations (3) and (4),

${\mathrm{<span\; class="notranslate">\; z</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}\left[\begin{array}{c}{\mathrm{<span\; class="notranslate">\; u</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}\\ {\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}\\ \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]<span\; class="notranslate">\; =</span>\left[\begin{array}{ccc}\mathrm{<span\; class="notranslate">\; f</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; f</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]\left[\begin{array}{c}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}\\ {\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}\\ {\mathrm{<span\; class="notranslate">\; z</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}\end{array}\right]<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 3</span>}<span\; class="notranslate">\; )</span>$

${\mathrm{<span\; class="notranslate">\; z</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}\left[\begin{array}{c}{\mathrm{<span\; class="notranslate">\; u</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}\\ {\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}\\ \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]<span\; class="notranslate">\; =</span>\left[\begin{array}{ccc}\mathrm{<span\; class="notranslate">\; f</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; f</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]\left[\begin{array}{c}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}\\ {\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}\\ {\mathrm{<span\; class="notranslate">\; z</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}\end{array}\right]<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 3</span>}<span\; class="notranslate">\; )</span>$

Equation (5) is the conversion relationship of the three-dimensional coordinates of the center of mass in the camera coordinate system in position 1 and position 2,

$\left[\begin{array}{c}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}\\ {\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}\\ {\mathrm{<span\; class="notranslate">\; z</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}\end{array}\right]<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; 12</span>}}\left[\begin{array}{c}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}\\ {\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}\\ {\mathrm{<span\; class="notranslate">\; z</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}\end{array}\right]<span\; class="notranslate">\; =</span>\begin{array}{c}\left[\begin{array}{cccc}{\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}& {\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}& {\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; 3</span>}}& {\mathrm{<span\; class="notranslate">\; t</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}\\ {\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; 4</span>}}& {\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; 5</span>}}& {\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; 6</span>}}& {\mathrm{<span\; class="notranslate">\; t</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}\\ {\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; 7</span>}}& {\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; 8</span>}}& {\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; 9</span>}}& {\mathrm{<span\; class="notranslate">\; t</span>}}_{\mathrm{<span\; class="notranslate">\; 3</span>}}\end{array}\right]\end{array}\left[\begin{array}{c}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}\\ {\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}\\ {\mathrm{<span\; class="notranslate">\; z</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}\\ \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 5</span>}<span\; class="notranslate">\; )</span>$

Therefore, at position 1, the three-dimensional coordinates of the starting point in the camera coordinate system can be obtained:

$\left\{\begin{array}{c}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; z</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; u</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}}{\mathrm{<span\; class="notranslate">\; f</span>}}\\ {\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; z</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}}{\mathrm{<span\; class="notranslate">\; f</span>}}\\ \mathrm{<span\; class="notranslate">\; z</span>}\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; f</span>}{\mathrm{<span\; class="notranslate">\; t</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; u</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}{\mathrm{<span\; class="notranslate">\; t</span>}}_{\mathrm{<span\; class="notranslate">\; 3</span>}}<span\; class="notranslate">\; )</span>}{{\mathrm{<span\; class="notranslate">\; u</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; 7</span>}}{\mathrm{<span\; class="notranslate">\; u</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; 8</span>}}{\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; f</span>}{\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; 9</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; u</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}{\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; f</span>}{\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; 3</span>}}<span\; class="notranslate">\; )</span>}\end{array}\right.<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 6</span>}<span\; class="notranslate">\; )</span>$

Among them, the specific steps for calculating the relative position relationship _M12 of the CCD industrial digital camera at position 1 and position 2 are as follows: set the conversion relationship from the robot base coordinate system to the tool coordinate system at position 1 and position 2 as _B1 and B respectively ₂ , can be calculated from the tool coordinates read in the robot control cabinet, H is the hand-eye transformation matrix, so the following formula can be obtained:

$\begin{array}{c}\end{array}{{\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}}^{<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; h</span>}}^{<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}\left[\begin{array}{c}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}\\ {\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}\\ {\mathrm{<span\; class="notranslate">\; z</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}\\ \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]<span\; class="notranslate">\; =</span>{{\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}}^{<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; h</span>}}^{<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}\left[\begin{array}{c}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}\\ {\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}\\ {\mathrm{<span\; class="notranslate">\; z</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}\\ \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]<span\; class="notranslate">\; \&DoubleRightArrow;</span>\left[\begin{array}{c}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}\\ {\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}\\ {\mathrm{<span\; class="notranslate">\; z</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}\\ \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; h</span>}{\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}{{\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}}^{<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; h</span>}}^{<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}\left[\begin{array}{c}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}\\ {\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}\\ {\mathrm{<span\; class="notranslate">\; z</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}\\ \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]<span\; class="notranslate">\; =</span>\left[\begin{array}{c}{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; 12</span>}}\\ \mathrm{<span\; class="notranslate">\; I</span>}\end{array}\right]\left[\begin{array}{c}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}\\ {\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}\\ {\mathrm{<span\; class="notranslate">\; z</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}\\ \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 7</span>}<span\; class="notranslate">\; )</span>$

where I=(0,0,0,1) ^T . After obtaining the three-dimensional coordinates (x ₁ , y ₁ , z ₁ ) of the starting point in the camera coordinate system, it can be converted to the robot base coordinate system according to formula (8):

$\left[\begin{array}{c}\mathrm{<span\; class="notranslate">\; x</span>}\\ \mathrm{<span\; class="notranslate">\; Y</span>}\\ \mathrm{<span\; class="notranslate">\; Z</span>}\\ \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]<span\; class="notranslate">\; =</span>{{\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}}^{<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; h</span>}}^{<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}\left[\begin{array}{c}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}\\ {\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}\\ {\mathrm{<span\; class="notranslate">\; z</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}\\ \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 8</span>}<span\; class="notranslate">\; )</span>$

(X, Y, Z) are the three-dimensional coordinates of the starting point in the coordinate system of the robot base. The weld seam of the workpiece to be welded described in this specific embodiment is a V-shaped ocean pipeline weld seam where pipes are butt-jointed.

Please refer to shown in Fig. 4, the present invention is based on the large-scale ocean pipeline starting point identification and positioning system of CCD and comprises positioner 1, visual sensor 3, welding robot 4, robot control cabinet 5, welding power supply 6 and main control computer 7; Wherein The visual sensor 3 includes a CCD industrial digital camera rigidly fixed on the welding torch at the end of the welding robot 4, and the camera moves together with the welding robot 4; the positioner 1, the welding power supply 6 are connected with the robot control cabinet 5, and the main control computer 7 is connected via Ethernet Connected with the robot control cabinet 5, the main control computer 7 sends a signal to the robot control cabinet 5 to control the movement of the positioner 1; the main control computer 7 and the visual sensor 3 are connected through USB. As shown in FIG. 5 , the main control computer 7 includes an image acquisition module, an image processing module, and an information feedback and control module.

Image acquisition module: control the CCD to collect weld seam pictures through the USB cable and send them to the main control computer;

Image processing module: including three image processing sub-modules including preliminary positioning of the welding seam position, extraction of pixel coordinates of the starting point of welding, and positioning of the three-dimensional coordinates of the starting point of the welding seam;

Information feedback and control module: Feedback the results of image processing to the robot control cabinet, control the rotary motion of the positioner and the motion of the robot, and finally control the movement of the welding robot to the starting point of welding.

The above is only a preferred embodiment of the present invention, it should be pointed out that for those of ordinary skill in the art, some improvements can also be made without departing from the principle of the present invention, and these improvements should also be regarded as the invention. protected range.

Claims (5)

1. A CCD-based large-scale ocean pipeline starting point identification and positioning method, characterized in that: comprising the following steps Step 1: Preliminary positioning of the weld position (a) Keep the pipeline to be welded still, the welding robot moves to the preset position, and the weld seam image of the pipeline to be welded is collected by the CCD industrial digital camera, and the main control computer receives the picture through the USB cable and performs image processing for preliminary identification Weld seam, extract the center line of the weld seam, scan the middle row of the picture, get a point on the center line of the weld seam, calculate the pixel deviation between the center of the welding torch and the abscissa direction of this point, and feed it back to the welding robot; (b) The welding robot moves to the center of the weld. During the movement, the CCD collects images in real time. The main control computer receives and extracts the center line of the weld in real time, scans the middle row of the picture, and obtains a point on the center line of the weld. Calculate the distance between the welding torch center and The real-time change of the pixel deviation in the abscissa direction of the point. When the pixel deviation is zero, the main control computer sends a stop signal to the welding robot, and the welding robot stops moving, so that the welding robot is initially positioned above the weld seam; Step 2: Extraction of pixel coordinates of welding starting point (a) The main control computer sends a start signal to the positioner, the positioner drives the pipeline to make a rotary motion, the CCD collects images in real time, and the main control computer receives and processes the image of the rectangular region of interest including the welding torch in real time. When the welding spot in the positioning welding of the pipeline appears in the area of interest, a stop signal is sent to the positioner, and the pipeline stops rotating; (b) The main control computer carries out image processing to the rectangular region of interest comprising the solder joint, extracts the outer contour and the centroid of the solder joint, and the pixel coordinates of the centroid are just used as the pixel coordinates of the starting point of the pipeline; Step 3: Positioning of welding starting point (a) The main control computer sends a motion signal to the welding robot, and the welding robot moves horizontally for a short distance along the Y direction of the base coordinate system. The CCD industrial digital camera collects the image of the weld seam, and the main control computer receives the picture and is interested in the rectangle containing the welding spot Image processing is performed on the area, and the pixel coordinates of the centroid of the solder joint are extracted again by image processing; (b) Use the general binocular vision to calculate the three-dimensional coordinates of the starting point in the camera coordinate system, and according to the transformation matrix between the tool coordinate system and the robot base coordinate system and the transformation matrix between the tool coordinate system and the camera coordinate system, Transform the obtained three-dimensional coordinates of the starting point in the camera coordinate system to the three-dimensional coordinates in the robot base coordinate system. 2. the CCD-based large-scale ocean pipeline starting point identification and location method as claimed in claim 1, is characterized in that: the calculation concrete steps of pixel deviation in the step 1 (a) are: (1) Calculate the pixel coordinates (u ,v); (2) Perform image processing on the collected pictures, extract the edge of the weld and the center line of the weld, scan the middle row of the weld picture from the left, and obtain a point on the center line of the weld, denoted as (u _k , v _k ), calculate the pixel deviation Δu=uu _k from the center of the welding torch to the center line of the weld. 3. The CCD-based method for identifying and locating the starting point of a large ocean pipeline as claimed in claim 2, wherein the moving direction of the welding robot in step 1 (b) is determined by the positive or negative of the pixel distance Δu, and the machine only moves along The Y direction of the base frame moves horizontally. 4. the CCD-based large-scale ocean pipeline starting point identification and location method as claimed in claim 3, is characterized in that: the transformation matrix between the tool coordinate system in the described step 3 (b) and the robot base coordinate system is by The main control computer reads the pose conversion of the welding robot from the robot control cabinet, and the transformation matrix between the tool coordinate system and the camera coordinate system is obtained from the hand-eye calibration experiment. 5. A CCD-based identification and positioning system for the starting point of a large ocean pipeline, characterized in that it includes a positioner (1), a visual sensor (3), a welding robot (4), a robot control cabinet (5), and a welding power supply (6) and main control computer (7); Described vision sensor (3) comprises the CCD industrial digital camera that is rigidly fixed on welding robot (4) end welding torch, and industrial digital camera moves together with welding robot (4); Described The positioner (1), the welding power source (6) are connected with the robot control cabinet (5), the main control computer (7) is connected with the robot control cabinet (5) through Ethernet, and the main control computer (7) sends a signal Give robot control cabinet (5) and then control the motion of positioner (1); Described main control computer (7) and visual sensor (3) are connected by USB; Described main control computer (7) comprises image acquisition module, image processing module and information feedback and control module, the image acquisition module controls the CCD industrial digital camera to collect weld seam pictures through the USB line and sends them to the main control computer; The extraction of the three-dimensional coordinates of the weld starting point and the three image processing sub-modules, the information feedback and control module feeds back the results of image processing to the robot control cabinet, controls the rotary motion of the positioner and the motion of the robot, and finally controls the welding The robot moves to the starting point of welding.