Disclosure of Invention
In view of the above, the present invention is directed to a method, a control system and a device for calibrating robot vision based on binocular vision, the robot vision calibration method based on binocular vision can combine a vision system, image processing and a robot technology, adopts a robot to match the binocular vision system, the calibration is carried out by the serial communication mode and the Hough image processing, thus solving the problems of large labor consumption, large visual error, fussy repeated calibration, tool damage risk and the like of the traditional four-point calibration, compared with the traditional demonstrator calibration, the method for calibrating by using vision has the characteristics of high speed, high accuracy, no need of setting a reference point, avoidance of collision and the like, therefore, the efficiency is greatly improved, the time cost is reduced, the unit time output value of the assembly line is increased, and further great economic benefits can be brought to manufacturing enterprises.
In order to achieve the purpose, the technical scheme of the invention is realized as follows:
a robot vision calibration system based on binocular vision comprises the following steps:
s1, clamping the calibration ball at the tail end of the robot tool by using a clamp;
s2, presetting a central point at the center of the camera, shooting the image of the calibration ball by using the camera, and preprocessing the shot image to obtain the environmental information of the calibration ball;
s3, obtaining the position information of the calibration ball through the binocular vision technology, identifying and calculating the center of the calibration ball, and calculating the position coordinate value according to the center of the calibration ball;
s4, comparing the coordinate value of the calibration sphere center with the set point, and feeding back the difference to the controller;
and S5, the controller sends out a control instruction, the camera and the robot are communicated and interacted in a serial port communication mode, the position of the calibration ball at the tail end of the tool of the robot is moved until the center of the calibration ball coincides with the set point, and the coordinate point is recorded by using the demonstrator to finish calibration.
Further, in step S2, two cameras vertically installed in the robot arm motion space are used to capture an image of the calibration ball, and the captured image is preprocessed after the image is captured, so as to obtain the environmental information of the calibration ball.
Further, in step S2, the captured image is subjected to the gradation processing and the binarization processing.
Further, the step S3 further includes the following sub-steps:
s31, identifying and positioning the calibration ball at the tail end of the robot tool after image preprocessing, and acquiring the position information of the calibration ball;
s32, performing edge detection on the circle in the image by using Hough transform;
s33, coordinate transformation is carried out, the equation of the circle in the plane coordinate system is transformed into a parameter equation, namely, the xy coordinate system is transformed into an ab coordinate system, and the equation of the circle can be expressed as a formula (1);
(x-a)2+(y-b)2=r2 (1)
the three degrees of freedom are coordinates a and b of the circle center and a radius r respectively, which are a three-dimensional space, and r is taken as a preset value;
s34, each circle in the x and y coordinate system has countless points corresponding to the edge of the circle, and the circles in the a and b coordinate system correspond to a plurality of circles, which is obviously represented that the circles corresponding to the countless points on the edge of the circle in the x and y coordinate system in the a and b coordinate system intersect at one point, and the coordinate of the intersection point is the circle center (a, b);
s35, finding a plurality of circles with relatively large radius by finding a large number of critical circles, finding the common circle center of the critical circles, namely, the center of the sphere of the calibration ball at the tail end of the robot tool is calculated, and the position coordinate value of the robot tool is calculated according to the center of the sphere.
A robot vision calibration control system based on binocular vision comprises a communication system and a vision system;
the communication system comprises serial port communication, wherein a Robot Studio is used for compiling a RAPID program, and the RAPID program is used for enabling an upper computer and a lower computer to communicate and interact in a serial port communication mode, changing the position of a calibration ball at the tail end of a tool of the Robot, controlling the tail end of the tool of the Robot to clamp the calibration ball by using a clamp, acquiring the current position and sending position data;
the vision system comprises two cameras vertically installed in a motion space of a robot mechanical arm, a central point is preset in the center of a camera of the camera, the camera is used for shooting an image of the calibration ball, the shot image is preprocessed, the position information of the calibration ball is obtained through a binocular vision technology, the position relation between the center of the calibration ball and the set point is calculated, the robot is controlled to enable the center of the calibration ball to move to the position point to be coincident with the set point, a demonstrator is used for recording a coordinate point, and calibration is completed.
Further, the vision system also comprises an image processing module, wherein the image processing module comprises an image graying processing module and an image binarization processing module.
Further, the vision system identifies the calibration sphere based on the hough transform and calculates the coordinates of the sphere center.
A robot vision calibration device based on binocular vision comprises a camera, a controller, a serial port communication interface, a robot and a binocular vision system;
the camera is used for shooting a calibration ball image and capturing target data, and sending the calibration ball image and the target data to the controller;
the controller is used for receiving the calibration ball image and the target data and transmitting the calibration ball image and the target data to the robot through the serial port communication interface;
the serial port communication interface is used for controlling the robot to move through a program instruction by the controller so as to complete a calibration task;
the robot is used for receiving data sent by the serial port communication interface, completing different actions by analyzing the serial port data and feeding back path data to the controller;
the binocular vision system is used for preprocessing an image of the calibration ball, acquiring position information of the calibration ball, identifying and calculating the center of the calibration ball, calculating a position coordinate value of the calibration ball according to the center of the calibration ball, feeding back a difference value of the position coordinate value and a given coordinate value to the controller, guiding the calibration ball to move towards a set point at a certain direction, angle and speed until the center of the calibration ball coincides with the set point, and recording a coordinate point by using the demonstrator to finish calibration.
Further, the visual calibration device further comprises an image processing system, and the image processing system is used for carrying out graying processing and binarization processing on the calibration ball image.
Further, the cameras include two, and the two cameras are installed perpendicular to each other in a motion space of a robot arm of the robot.
Compared with the prior art, the binocular vision-based robot vision calibration method, the control system and the device have the following advantages that:
1. the calibration is carried out by combining a visual system and an image processing technology, and the method is a pioneering technology in the calibration industry.
2. An intelligent calibration scheme is explored, and a binocular vision system is used for automatic calibration, so that the participation degree of staff is reduced.
3. An image processing technology is mastered, and the defect of low human eye calibration precision of the traditional calibration method is overcome.
4. The storage function of the controller is utilized, different tool model data are stored, tools can be directly calibrated when being replaced at each time, the complex process of traditional calibration repeated calibration is avoided, and the burden of staff is reduced.
5. Compared with the traditional calibration system, the intelligent calibration system greatly reduces the risk of tool damage caused by staff misoperation.
6. An efficient and accurate calibration system is established, a new intelligent calibration way is developed, the calibration efficiency is improved for enterprises, and the calibration cost is reduced.
7. The point marking is taken as an entry point, and the digital and intelligent development of the marking technology in the manufacturing industry is promoted.
Additional features and advantages of the invention will be set forth in the detailed description which follows.
Detailed Description
The following detailed description of embodiments of the invention refers to the accompanying drawings. It should be understood that the detailed description and specific examples, while indicating the present invention, are given by way of illustration and explanation only, not limitation.
According to an aspect of the present invention, there is provided a binocular vision-based robot vision calibration method, as shown in fig. 1, the method including the steps of:
s1, clamping the calibration ball at the tail end of the robot tool by using a clamp;
s2, presetting a central point at the center of the camera, shooting the image of the calibration ball by using the camera, and preprocessing the shot image to obtain the environmental information of the calibration ball;
s3, obtaining the position information of the calibration ball through the binocular vision technology, identifying and calculating the center of the calibration ball, and calculating the position coordinate value according to the center of the calibration ball;
s4, comparing the coordinate value of the calibration sphere center with the set point, and feeding back the difference to the controller;
and S5, the controller sends out a control instruction, the camera and the robot are communicated and interacted in a serial port communication mode, the position of the calibration ball at the tail end of the tool of the robot is moved until the center of the calibration ball coincides with the set point, and the coordinate point is recorded by using the demonstrator to finish calibration.
In the embodiment of the invention, the robot vision calibration method based on binocular vision combines a vision system, image processing and a robot technology, adopts a robot to match the binocular vision system, performs communication interaction in a serial port communication mode, and performs calibration by means of Hough image processing, thereby solving the problems of large manpower consumption, large vision error, tedious repeated calibration, tool damage risk and the like in the traditional four-point calibration method.
The robot vision calibration method based on binocular vision is suitable for TCP calibration of any type of robot, and is particularly suitable for TCP calibration of an ABB robot. The ABB robot IRB120 is the smallest one of 6-axis industrial robots, the specification load is only 3kg, but the robot can complete the functions completed by basically all models of robots, and the robot uses a RAPID programming language, wherein RAPID is a simpler programming language, and different functions such as controlling the movement, input/output, communication and simulation of the robot can be completed according to different instructions. TCP is short for a Tool center Point (Tool center Point), the calibration of TCP is the determination of the position of the Tool center Point, and the Tool center Point is provided, so that the teaching is convenient in practical application. When the calibration worker teaches by using the tool coordinate, the calibration worker can move according to the defined coordinate direction, and can easily and accurately find the position point to be moved, so that the teaching difficulty is greatly reduced.
In order to acquire more comprehensive image information of the calibration ball at the end of the robot tool, in step S2, two cameras vertically installed in the motion space of the robot arm are used to capture an image of the calibration ball, and the captured image is preprocessed after the image is acquired, so as to obtain environment information of the calibration ball.
In order to facilitate subsequent calibration, in step S2, the captured image is subjected to a graying process and a binarization process. Firstly, preprocessing an image, converting a color image acquired by a camera into a gray image by a system, namely graying the image, wherein the gray image can still clearly reflect the contour and the feature texture of the whole color image on the basis of well reserving the contour and the feature texture in the color image; the grayscale image is then smoothed to eliminate or suppress noise in the image. Specifically, the graying processing is as follows: the process of converting a color image into an off-white image through special processing is called filtering and graying. This gray color is transformed from white, and by different values, the gray color also differs in degree. The binarization processing comprises the following steps: it is a process of calculating the threshold values of the gray data values of the pixel groups and nodes on the digital image. The resulting image is binarized, so called an image consisting of 0 and 1, i.e. black and white. Therefore, each shot element square is composed of 1 and 0, so that the computer can quickly identify the boundary, the boundary composed of various shapes such as a circle, a square and the like can be obtained through the boundary, and the required target point can be obtained through automatic adjustment of an algorithm.
In order to more quickly and conveniently identify and calculate the center of sphere of the calibration ball, the step S3 further includes the following sub-steps:
s31, identifying and positioning the calibration ball at the tail end of the robot tool after image preprocessing, and acquiring the position information of the calibration ball;
s32, performing edge detection on the circle in the image by using Hough transform;
s33, coordinate transformation is carried out, the equation of the circle in the plane coordinate system is transformed into a parameter equation, namely, the xy coordinate system is transformed into an ab coordinate system, and the equation of the circle can be expressed as a formula (1);
(x-a)2+(y-b)2=r2 (1)
the three degrees of freedom are coordinates a and b of the circle center and a radius r respectively, which are a three-dimensional space, and r is taken as a preset value;
s34, each circle in the x and y coordinate system has countless points corresponding to the edge of the circle, and the circles in the a and b coordinate system correspond to a plurality of circles, which is obviously represented that the circles corresponding to the countless points on the edge of the circle in the x and y coordinate system in the a and b coordinate system intersect at one point, and the coordinate of the intersection point is the circle center (a, b);
s35, finding a plurality of circles with relatively large radius by finding a large number of critical circles, finding the common circle center of the critical circles, namely, the center of the sphere of the calibration ball at the tail end of the robot tool is calculated, and the position coordinate value of the robot tool is calculated according to the center of the sphere.
In the above, firstly, the calibration ball of the target object is identified, and there are many methods for identifying the circle. The general equation of a circle can be expressed as formula (1), and there are three degrees of freedom, namely coordinates a and b of the center of the circle and a radius r, which is a three-dimensional space. Finally, a large number of critical circles are found, the common circle center of the critical circles is found, the obtained circle data are compared with preset coordinates, a difference value is found out, data are sent to a serial port, the position of a calibration ball is moved, and the calibration ball is drawn close to a set point.
According to another aspect of the present invention, there is provided a binocular vision based robot vision calibration control system, as shown in fig. 2 to 4, the control system includes a communication system and a vision system;
the communication system comprises serial port communication, wherein a Robot Studio is used for compiling a RAPID program, and the RAPID program is used for enabling an upper computer and a lower computer to communicate and interact in a serial port communication mode, changing the position of a calibration ball at the tail end of a tool of the Robot, controlling the tail end of the tool of the Robot to clamp the calibration ball by using a clamp, acquiring the current position and sending position data;
the vision system comprises two cameras vertically installed in a motion space of a robot mechanical arm, a central point is preset in the center of a camera of the camera, the camera is used for shooting an image of the calibration ball, the shot image is preprocessed, the position information of the calibration ball is obtained through a binocular vision technology, the position relation between the center of the calibration ball and the set point is calculated, the robot is controlled to enable the center of the calibration ball to move to the position point to be coincident with the set point, a demonstrator is used for recording a coordinate point, and calibration is completed.
In the embodiment of the invention, the communication system can enable the upper computer and the lower computer to communicate and interact in a serial port communication mode, so that the robot is controlled to drive the calibration ball to move; and acquiring an image of the calibration ball through the vision system, positioning the calibration ball, acquiring the position of the calibration ball, comparing the position with a set point, and guiding the calibration ball to move towards the set point at a certain direction, angle and speed until the calibration ball coincides with the set point to finish calibration. Therefore, dynamic data of the center of the calibration ball can be collected and displayed more conveniently and timely, and the position of the calibration ball is controlled, so that the camera can be used for controlling the moving center of the calibration ball only by knowing the position of the calibration ball in real time. The control system needs to give information such as the position of a calibration point in advance, and then completes the calibration process according to a taught motion mode by combining the data collected by the camera and serial port communication. The control system collects calibration ball information through the vision system so as to control the robot, has the characteristics of high accuracy, good real-time performance and the like compared with the traditional method, and can better adapt to environmental changes.
In order to more rapidly carry out positioning control on the calibration ball, the vision system further comprises an image processing module, and the image processing module comprises an image graying processing module and an image binarization processing module.
In order to calculate the sphere center of the calibration sphere more accurately and quickly, the vision system identifies the calibration sphere and calculates the coordinates of the sphere center of the calibration sphere based on Hough transform.
According to another aspect of the present invention, there is provided a robot vision calibration apparatus based on binocular vision, as shown in fig. 5, the robot vision calibration apparatus includes a camera, a controller, a serial communication interface, a robot, and a binocular vision system;
the camera is used for shooting a calibration ball image and capturing target data, and sending the calibration ball image and the target data to the controller;
the controller is used for receiving the calibration ball image and the target data and transmitting the calibration ball image and the target data to the robot through the serial port communication interface;
the serial port communication interface is used for controlling the robot to move through a program instruction by the controller so as to complete a calibration task;
the robot is used for receiving data sent by the serial port communication interface, completing different actions by analyzing the serial port data and feeding back path data to the controller;
the binocular vision system is used for preprocessing an image of the calibration ball, acquiring position information of the calibration ball, identifying and calculating the center of the calibration ball, calculating a position coordinate value of the calibration ball according to the center of the calibration ball, feeding back a difference value of the position coordinate value and a given coordinate value to the controller, guiding the calibration ball to move towards a set point at a certain direction, angle and speed until the center of the calibration ball coincides with the set point, and recording a coordinate point by using the demonstrator to finish calibration.
In the embodiment of the invention, the image of the calibration ball is collected by a camera, the image is preprocessed to obtain the position information of the calibration ball, the center of the calibration ball is identified and calculated, the position coordinate value of the calibration ball is calculated according to the center of the calibration ball, the difference value between the position coordinate value and a given coordinate is fed back to a controller, the controller controls a Robot Studio to write a RAPID program, the RAPID program is transmitted to the Robot through a serial communication interface, the Robot is controlled to move, the calibration ball is guided to move towards a set point in a certain direction, angle and speed until the center of the calibration ball coincides with the set point, a demonstrator is used for recording a coordinate point, and calibration is completed. Compared with the traditional demonstrator calibration, the method for calibrating by using the vision has the characteristics of high speed, high accuracy, no need of setting a reference point, avoidance of collision and the like, thereby greatly improving the efficiency, reducing the time cost, increasing the unit time output value of the production line and further bringing great economic benefit to manufacturing enterprises.
In the above, the controller is a PC, and is used for the upper computer to control the Robot Studio software to write RAPID program, and the Robot becomes flexible through the program, and can also be used as a display device, so that each action of the Robot and real-time data of the Robot can be seen in the PC.
In order to be able to acquire more comprehensive images, the cameras comprise two, and the two cameras are vertically arranged in the motion space of the mechanical arm of the robot.
Further, the visual calibration device further comprises an image processing system, and the image processing system is used for carrying out graying processing and binarization processing on the calibration ball image.
To facilitate understanding of the calibration process, the camera calibration process is utilized as described in detail below: a calibration ball is fixed at the tail end of a tool of the robot to be used as a reference object, so that the robot is convenient to identify and automatically regulate and control; peripheral equipment installation: two cameras forming an included angle of 90 degrees are installed in a motion space of the mechanical arm to construct a three-dimensional space; the whole calibration process is mainly divided into a right camera calibration part and a rear camera calibration part. In the calibration process, firstly, a camera is used to obtain image information in real time, and as shown in fig. 6 (a), it can be seen that the calibration ball is located in the fourth quadrant. And then the robot is controlled to move towards the positive direction x, and the robot moves to the center of the vertical line of the Z axis of the screen in a stepping mode by 1mm each time. After judging whether the center of the cross line is reached, if the center is reached, the direction of the Z axis is calibrated, as shown in (c) to (d) in fig. 6, the robot is moved to the positive direction of the Z axis by using the same method until the center point is reached, and the calibration of the right camera is finished; and the program opens the rear camera to start calibration, and completes calibration of the Y axis by the same method. The calibration process of the rear camera is as shown in fig. 6 (e) to (g), and the current position point is recorded after the calibration is completed.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.