CN112170825B - Long nozzle replacing method, equipment, terminal and medium based on visual servo - Google Patents

Abstract

The invention provides a long nozzle replacing method, equipment, a terminal and a medium based on visual servo.A visual positioning mark block is assembled on a tundish to obtain a current image comprising the visual positioning mark block and determine the current mark block posture of the visual positioning mark block; the invention also provides visual servo long nozzle replacing equipment, a terminal and a medium, which realize that the corresponding current attitude of the end effector is determined according to different positions of the tundish, and the robot is guided to reach the current attitude point of the end effector to complete the replacing work of the long nozzle, so that the automation of replacing the long nozzle is realized, the manual operation with high strength and high risk is replaced, the method is simple and easy to implement, and the practicability and the applicability are higher.

Description

A vision servo-based long nozzle replacement method, equipment, terminal and medium

technical field

The invention relates to the field of industrial automation, in particular to a long nozzle replacement method, equipment, terminal and medium based on visual servoing.

Background technique

In the tundish platform of the iron and steel metallurgy continuous casting process, the equipment connecting the ladle and the tundish is called a long nozzle, also known as a protective casing, which is installed under the ladle and connected to the runner of the sliding plate mechanism. The long nozzle has a certain service life and needs to be loaded and unloaded regularly during the continuous casting tundish operation.

In the related art, for the metallurgical continuous casting production site, the modern continuous casting machine adopts the long shroud manipulator to realize the electromechanical drive device to replace the long shroud instead of the manual replacement of the long shroud, but it cannot achieve complete automation and still requires on-site operation by workers. Due to the high temperature and high risk of the continuous casting tundish operation site, the labor risk of workers is relatively large.

SUMMARY OF THE INVENTION

In view of the above-mentioned shortcomings of the prior art, the purpose of the present invention is to provide a method, equipment, terminal and medium for replacing the shroud based on visual servoing, which is used to solve the problem that the replacement of the shroud in the metallurgical continuous casting production site still requires workers on site Operation, technical problems with high labor risk and high risk for workers.

In view of the above problems, the present invention provides a long nozzle replacement method based on visual servoing, including:

Assemble the visual positioning marker block on the tundish;

Obtain the current image of the visual positioning marker block and determine the current marker block posture of the visual positioning marker block;

Determine the current end effector attitude according to the current marker block attitude and a relative relationship, where the relative relationship includes a mapping between the marker block attitude and the end effector attitude;

The end effector is controlled to reach the current end effector attitude, and the shroud is installed and/or removed.

Optionally, the controlling the end effector to reach the current end effector attitude, and installing the long shroud includes:

the end effector acquires a long shroud;

controlling the end effector to reach the current end effector attitude;

Install the shroud.

Optionally, the disassembling the long shroud includes:

If the installation of the long shroud is completed, obtain the exit point of the end effector, and determine the disassembly attitude;

The end effector is controlled to reach the disassembly attitude, and the long shroud is disassembled.

Optionally, the relative relationship includes a relative attitude transformation matrix, and the relative attitude transformation matrix is determined in the following manner:

Obtain the initial image of the visual positioning marker block at the preset shooting point;

Determine the initial marker block pose of the visual positioning marker block according to the initial image;

The servo robot installs the long shroud, and obtains the initial end effector posture of the end effector when the installation of the long shroud is completed;

A relative pose transformation matrix is determined according to the initial marker block pose and the initial end effector pose.

Optionally, before obtaining the current image of the visual positioning marker block and determining the current marker block posture of the visual positioning marker block, the method further includes:

The reference attitude transformation matrix from the camera coordinate system to the robot coordinate system is determined according to the hand-eye calibration and the joint parameter feedback of the robot control system.

Optionally, the visual positioning marker block includes a central sub-marking block and at least two peripheral sub-marking blocks, and the peripheral sub-marking blocks are arranged around the central sub-marking block, and the determining the position of the visual positioning marker block. Current marker block poses include:

determining a region of interest in the current initial image, the region of interest including the imaging region of the visual positioning marker;

Obtaining pixel information of preset points in the contour of each sub-marker block in the region of interest, and relative position information between the preset points in the contour of each of the sub-marker blocks;

Determine the posture of the preset point in the outline of the center sub-marker block in the camera coordinate system according to the pixel information and the relative position information;

The current marker block pose is determined according to the reference pose transformation matrix and the pose of the preset point in the outline of the central sub-marker block in the camera coordinate system.

Optionally, it also includes at least one of the following:

The visual positioning mark block is fixedly installed on the sliding plate mechanism of the continuous casting tundish;

The current image is acquired at a preset shooting point by an image acquisition device, and the image acquisition device is assembled at the joint end of the robot;

The visual positioning marker block includes at least 3 different sub-marker blocks;

The visual positioning marker block includes at least two background backgrounds, and the current background color of the visual positioning marker block is determined according to the on-site background background.

The present invention also provides a vision servo-based long nozzle replacement device, comprising:

The assembly module is used to assemble the visual positioning marker block on the tundish;

A current marker block attitude obtaining module is used to obtain the current image of the visual positioning marker block and determine the current marker block attitude of the visual positioning marker block;

a determination module, configured to determine the current end effector attitude according to the current marker block attitude and a relative relationship, where the relative relationship includes a mapping between the marker block attitude and the end effector attitude;

The control module is used to control the end effector to reach the current end effector attitude, and to install and/or remove the shroud.

The present invention also provides a terminal, including a processor, a memory and a communication bus;

the communication bus is used to connect the processor and the memory;

The processor is configured to execute the computer program stored in the memory, so as to implement the vision servo-based shroud replacement method described in one or more of the above embodiments.

The present invention also provides a computer-readable storage medium on which a computer program is stored,

The computer program is used to make the computer execute the vision servo-based shroud replacement method according to any one of the above embodiments.

As described above, a visual servo-based long nozzle replacement method, device, terminal and medium provided by the present invention have the following beneficial effects:

By assembling the visual positioning marker block on the tundish, obtain the current image including the visual positioning marker block and determine the current marker block attitude of the visual positioning marker block, and determine the current end effector attitude according to the current marker block attitude and relative relationship, and the relative relationship Including the mapping between the posture of the marker block and the posture of the end effector, controlling the end effector to reach the current posture of the end effector, and installing and/or disassembling the shroud; realizes the need to calculate the replacement and installation of the shroud according to the different positions of the tundish The current installation end effector attitude, and guide the robot to reach the current installation end effector attitude point to complete the replacement and installation of the shroud, realize the automation of replacement and installation of the shroud, replace the high-intensity and high-risk manual operation, and the method is simple Easy to apply, high practicability and applicability.

Description of drawings

1 is a schematic flowchart of a method for replacing a long shroud based on visual servoing according to Embodiment 1 of the present invention;

2 is a schematic structural diagram of a visual positioning marker block according to Embodiment 1 of the present invention;

3 is a specific schematic flow chart of a method for replacing a long shroud based on visual servoing provided in Embodiment 2 of the present invention;

4 is a schematic structural diagram of a long shroud replacement device based on visual servoing provided in Embodiment 3 of the present invention;

FIG. 5 is a schematic structural diagram of a terminal according to Embodiment 4 of the present invention.

Detailed ways

The embodiments of the present invention are described below through specific specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments may be combined with each other under the condition of no conflict.

It should be noted that the drawings provided in the following embodiments are only used to illustrate the basic concept of the present invention in a schematic way, so the drawings only show the components related to the present invention rather than the number, shape and number of components in actual implementation. For dimension drawing, the type, quantity and proportion of each component can be changed at will in actual implementation, and the component layout may also be more complicated.

Example 1

Referring to FIG. 1, an embodiment of the present invention provides a method for replacing a long shroud based on visual servoing, including:

S101: Assemble the visual positioning marker block on the tundish.

In some embodiments, the visual positioning marker block is fixedly mounted on the slide mechanism of the continuous casting tundish.

In some embodiments, the visual positioning marker block includes at least three distinct sub-marker blocks.

In some embodiments, the visual positioning marker block includes at least two background colors, and the current background color background of the visual positioning marker block is determined according to the on-site background color background. Optionally, the outline shapes of the sub-marker blocks are the same, and the sizes of the sub-marker blocks are different.

Optionally, the size of each sub-marker block can be adjusted according to actual working conditions, which is not limited here, and the background color and background of each sub-marker block is adjusted according to the on-site environment, which is not limited here.

Optionally, the outline shapes of each sub-marker block are different, and the size of each sub-marker block is not limited here.

Optionally, the material of each sub-flag block is not limited here.

In some embodiments, referring to FIG. 2, FIG. 2 is a schematic structural diagram of a visual positioning marker block. The visual positioning marker block includes five sub-marker blocks with circular contours with different radii, and the sub-marker block with the smallest circle contour radius is The center sub-mark block A, and the remaining four sub-mark blocks are evenly distributed around the center sub-mark block A. Optionally, the preset point includes the center of each circle outline, and on the visual positioning marker block, the center of the circle outline of the center sub-marker block A is the coordinate origin to establish a marker block coordinate system, and the above-mentioned 4 points around the center sub-marker block A are distributed. The coordinates of the circle center of each sub-marker block in the marker block coordinate system can be directly designed in the marker block design stage. The coordinates in can also be obtained by measurement, and then the relative position information of the center of each circle outline can be obtained.

In some embodiments, the contrast between the background color background of each sub-marker block imaged by the image acquisition device and the scene background imaged by the image acquisition device is greater than the first preset threshold. Optionally, the greater the contrast between the background color and background of each sub-marker block in the image of the image acquisition device and the image of the scene background in the image of the image acquisition device, the more convenient the replacement of the shroud is and the smaller the amount of calculation.

S102: Acquire the current image of the visual positioning marker block and determine the current marker block pose of the visual positioning marker block.

In some embodiments, the current image is acquired at a preset shooting point by an image acquisition device, and the image acquisition device is assembled at the joint end of the robot.

In some embodiments, the image capture device is mounted on a joint end tool flange of the robot.

Optionally, the image acquisition device includes an industrial camera.

In some embodiments, the image acquisition device includes a monocular camera.

In some embodiments, the current marker block pose is generated based on a preset coordinate system. Optionally, the preset coordinate system includes a robot base coordinate system.

It should be known that the preset coordinate system can also be other coordinate systems designated by those skilled in the art, and the gestures including the visual positioning marker block, the end effector, etc. in their respective coordinate systems are converted into gestures based on the preset coordinate system. After that, the shroud replacement method of this embodiment can also be used.

In some embodiments, before acquiring the current image of the visual positioning marker block and determining the current marker block pose of the visual positioning marker block, the method further includes:

The reference attitude transformation matrix from the camera coordinate system to the robot coordinate system is determined according to the hand-eye calibration and the joint parameter feedback of the robot control system.

Optionally, the reference attitude transformation matrix includes the coordinate system transformation relationship from the camera coordinate system where the image acquisition device is located to the robot base coordinate system.

Optionally, if the image acquisition device that obtains the current image is set at the joint end of the robot, the reference attitude transformation matrix can be determined in the following way:

Determine the camera coordinate system where the image acquisition device is located through the hand-eye calibration operation, and the first attitude transformation matrix of the end tool coordinate system where the end tool of the robot is located;

Determine the second attitude transformation matrix from the end tool coordinate system to the robot base coordinate system through the real-time joint parameter feedback of the robot control system;

The reference attitude transformation matrix from the camera coordinate system to the robot base coordinate system is determined according to the first attitude transformation matrix and the second attitude transformation matrix.

It should be noted that the image acquisition device can also be installed in other positions of the robot, and the calibration operation is performed by the method of the related art, and then the preset reference attitude transformation matrix from the camera coordinate system to the robot base coordinate system is calculated.

In some embodiments, the image acquisition device can also be installed on other fixed structures other than the robot structure, the fixed structure can be fixed, and the fixed structure can also move with the movement of the robot joint end joint to realize the image acquisition device Relative rest with the robot end joint. At this time, those skilled in the art can also use related technologies to obtain the preset reference attitude transformation matrix.

Optionally, according to the posture of the visual positioning mark block in the camera coordinate system and the preset reference posture transformation matrix, the mark block posture of the visual positioning mark block relative to the robot base coordinate system can be obtained.

In some embodiments, the visual positioning marker block includes a central sub-marking block and at least two peripheral sub-marking blocks, the peripheral sub-marking blocks are arranged around the central sub-marking block, and determining the current marker block posture of the visual positioning marker block includes:

Determine the region of interest in the current image, and the region of interest includes the imaging region of the visual positioning marker block;

Obtain the pixel information of the preset points in the contour of each sub-marker block in the region of interest, and the relative position information between the preset points in the contour of each sub-marker block;

Determine the pose of the preset point in the outline of the center sub-marker block in the camera coordinate system according to the pixel information and relative position information;

The pose of the marker block is determined according to the transformation matrix of the reference pose and the pose of the preset point in the outline of the central sub-marker block in the camera coordinate system.

Optionally, the current marker block pose of the visual positioning marker block in the camera coordinate system includes the pose of the preset point in the outline of the center sub marker block under the camera coordinate system.

Optionally, the region of interest includes images of each sub-marker block, and the contour extraction module extracts the contour of each sub-marker block in the region of interest according to the contour shape of each sub-marker block, and obtains the outline of each submarker block. The pixel information of a preset point, and the relative position information between each preset point, based on the pixel information and relative position information, the monocular vision PnP algorithm can be used to calculate the three-dimensional attitude of the center sub-marker block in the camera coordinate system, It is regarded as the pose of the visual positioning marker block in the camera coordinate system.

In some embodiments, the visual positioning marker block includes at least four circular outline sub-marker blocks with different radii, wherein the sub-marker block with the smallest radius is used as the center sub-marker block, and the rest of the sub-marker blocks are evenly distributed around the center sub-marker block , the region of interest includes the image of each sub-marker block, the contour extraction module processes the region of interest, and obtains the pixel information of the center of each circle outline, and the marker block pose determination module obtains the pixel information of each circle outline and the center of each circle outline according to the obtained pixel information The relative relationship of coordinates in the coordinate system of the marker block is calculated by using the monocular vision 3D positioning algorithm to calculate the 3D pose of the circle center of the center sub-marker block in the camera coordinate system, and it is regarded as the visual positioning marker block in the camera coordinate system. The relative position relationship of the coordinates in the coordinate system of the marker block of the center of each circle contour can be calculated by pre-measurement, or after the design and processing of the visual positioning marker block is completed, the relative position relationship between the centers of each circle contour can be recorded. . Optionally, the radius of the circle outline can be adjusted according to the actual working conditions, so that the image acquisition device can clearly image the visual positioning marker block.

In some embodiments, the relative positional relationship of each preset point can be recorded by visual positioning marker block design and processing, and the relative positional relationship between each preset point can be recorded, or each preset point can be obtained by pre-measurement and calculation. The relative positional relationship between the points, the preset point is a certain point in the outline of the sub-marker block, which can be preset by those skilled in the art as required.

In some embodiments, obtaining a current image including a visual positioning marker and determining the current marker pose of the visual positioning marker includes:

According to the current image, the three-dimensional pose of the visual positioning marker block in the camera coordinate system is calculated by using a PnP algorithm including but not limited to monocular vision, and the three-dimensional pose is taken as the pose of the visual positioning marker block in the camera coordinate system.

S103: Determine the current posture of the end effector according to the posture of the current marker block and the relative relationship.

Optionally, the relative relationship includes a mapping between the pose of the marker block and the pose of the end effector.

In some embodiments, the relative relationship can be performed by enumerating the tundish at different positions, visually locating the marker block pose of the marker block, and, under the marker block pose, the end effector of the robot reaching the end of the shroud installation point. Based on the above-mentioned marker block posture and end effector posture, a basic table is constructed, and the basic table is used as a relative relationship. After the current marker block pose is determined, the corresponding end effector pose is searched based on the basic table.

In some embodiments, the relative relationship includes a relative pose transformation matrix.

In some embodiments, the relative attitude transformation matrix is generated according to the initial marker block attitude and the initial end effector attitude, wherein the initial marker block attitude is based on the successful installation of the shroud by the servo robot, and the images captured at the preset shooting points include visual positioning marks The initial image of the block is determined; the initial end-effector pose includes the servo robot's successful installation of the shroud, the robot's end-effector pose.

Optionally, the relative pose transformation matrix is determined by:

Obtain the initial image of the visual positioning marker block at the preset shooting point;

Determine the initial marker block pose of the visual positioning marker block according to the initial image;

Serve the robot to install the long shroud, and obtain the initial end effector posture of the end effector when the long shroud is installed;

The relative pose transformation matrix is determined according to the initial marker block pose and the initial end effector pose.

Optionally, since the visual positioning marker block is assembled on the steelmaking continuous casting tundish, the gesture of the marker block of the visual positioning marker block can also represent the attitude of the tundish. The replacement method can realize the matching of the corresponding current posture of the end effector according to the current position of the tundish, thereby realizing the automatic loading and unloading of the shroud.

Optionally, if the relative position of the visual positioning marker block and the water outlet of the tundish sliding plate mechanism is fixed, the above-mentioned relative attitude transformation matrix is fixed.

Optionally, if the relative position of the visual positioning marker block and the shroud of the tundish sliding plate mechanism changes after the servo robot successfully installs the long shroud, the relative attitude transformation matrix needs to be re-determined.

Optionally, when the servo robot successfully installs the shroud, the installation of the shroud can be completed by manual servo teaching.

Optionally, if the image acquisition device is fixedly installed at the end of the robot joint, the preset shooting point can be any point on the path passed by the end of the robot joint when the servo robot successfully installs the shroud.

Optionally, the current image is captured by the image capturing device at a preset capturing point.

Optionally, the shooting location of the current image can also be a location other than the preset shooting point, that is, the shooting location of the current image and the shooting location of the initial image can also be different. The transformation relationship between the poses of the images taken at the initial image shooting location, and finally transform the pose of the visual positioning marker block in the current image into the pose of the current marker block based on the robot base coordinate system.

It should be noted that, if the preset shooting point is predetermined, the execution sequence between step S102 and step S103 is not limited here. If no preset shooting point is set in advance, step S103 needs to be performed first, and then step S102 is performed according to the shooting position of the initial image in step S103 as the preset shooting point.

In some embodiments, each pose is normalized to be based on data in a preset coordinate system. The preset coordinate system includes, but is not limited to, the robot base coordinate system. At this time, the current marker block attitude, the relative attitude transformation matrix, and the current installation attitude are all attitudes in the robot base coordinate system. It should be known that the preset coordinate system can also be other coordinate systems set by those skilled in the art. , the visual servo-based shroud replacement method provided in this embodiment can be applied to realize the automatic replacement of the shroud.

S104: Control the end effector to reach the current end effector attitude, and install and/or remove the shroud.

In some embodiments, when the robot adjusts the end effector to reach the current end effector attitude, the end effector of the robot reaches a position where it can directly start the installation of the shroud, and the end effector can perform the installation of the shroud according to the preset installation method. Install.

In some embodiments, the preset installation method may be the installation method of the shroud when the servo robot successfully installs the shroud. For the same tundish, the long nozzle replacement and installation methods are all uniform, so even if the visual positioning mark block is moved, the preset installation method does not need to be reset. Similarly, the way of disassembling the shroud can also be determined by means including but not limited to parameter input, manual servo and so on.

In some embodiments, controlling the end effector to reach the current end effector attitude, installing the shroud includes:

The end effector obtains the long shroud;

Control the end effector to reach the current end effector attitude;

Install the long nozzle.

In some embodiments, removing the shroud includes:

If the installation of the shroud is completed, obtain the exit point of the end effector and determine the disassembly attitude;

The end effector is controlled to reach the disassembly attitude, and the long shroud is disassembled.

In some embodiments, after the shroud is successfully installed, the end effector has an exit installation point, and the attitude of the exit point is determined as the disassembly attitude that the end effector needs to reach when the shroud is disassembled.

Optionally, the disassembly attitude can be determined through the feedback of the joint parameters of the robot control system.

Optionally, when the shroud needs to be disassembled, the robot is guided to move to the above-mentioned disassembly posture and the disassembly operation of the shroud is completed.

Since the position of the shroud is not fixed, every time the robot successfully installs the shroud, the removal posture of the exit installation point (exit point) of the end effector must be re-determined. It is considered that the end effector needs to reach the end of the shroud when removing the shroud. disassembly attitude. When the shroud needs to be disassembled, guide the robot to move to the above disassembly posture and complete the disassembly operation of the shroud.

In some embodiments, after the installation or removal of the shroud is completed, the robot will return to the designated position and stand by.

In some embodiments, one robot may be responsible for the loading and unloading of one or more tundish long shrouds. In this case, the initial shooting positions for different tundishes may be the same preset shooting point or different preset shooting points. Those skilled in the art can set as required.

In some embodiments, when the robot is responsible for loading and unloading multiple tundish long nozzles, and the preset shooting points corresponding to each tundish are different, the visual positioning marker blocks set on each tundish may be the same or different. . When the robot is responsible for the loading and unloading of multiple tundish runners, when the preset shooting points corresponding to each tundish are the same, and the models of runners corresponding to each tundish are different, the visual positioning mark blocks set on each tundish are different. of. At this time, according to each visual positioning marker block, the model of the long shroud that needs to be installed and removed at present can be determined, and then the corresponding installation method or removal method is used for installation and removal.

In some embodiments, if the robot is responsible for the loading and unloading of multiple tundish runners, a corresponding identification mark is set for each tundish, and the disassembly posture corresponding to each tundish is recorded according to its identification mark, so as to facilitate the subsequent installation of the tundish. disassemble.

The embodiment of the present invention provides a long shroud replacement method based on visual servoing. By assembling the visual positioning mark block on the tundish, obtaining a current image including the visual positioning mark block and determining the current mark block posture of the visual positioning mark block, Determine the current end effector attitude according to the current marker block attitude and the relative relationship, the relative relationship includes the mapping between the marker block attitude and the end effector attitude, control the end effector to reach the current end effector attitude, and install and/or remove the shroud; It realizes the calculation of the current end effector posture required for the replacement of the shroud according to the different positions of the tundish, and guides the robot to reach the current end effector posture to complete the replacement of the shroud, and realizes the automation of the replacement of the shroud. High-intensity, high-risk manual operation, the method is simple and easy to implement, and has high practicability and applicability.

Optionally, by acquiring the current image including the visual positioning marker block and determining the current marker block attitude of the visual positioning marker block, the visual positioning marker block is assembled on the tundish, and the relative attitude transformation matrix is obtained, according to the current marker block attitude and relative attitude. The transformation matrix determines the current end effector attitude, the end effector reaches the current end effector attitude, and installs and/or removes the long shroud; realizes the determination of the corresponding current end effector attitude according to the different positions of the tundish, and guides the robot to reach the end effector. The current position of the end effector completes the replacement of the long shroud, realizes the automation of replacing the long shroud, and replaces the high-intensity and high-risk manual operation. The method is simple and easy to implement, and has high practicability and applicability.

Embodiment 2

Hereinafter, a specific embodiment will be used to illustrate the visual servo-based shroud replacement method provided by the present embodiment. Referring to FIG. 3 , the specific visual servo-based shroud replacement method includes:

S301: Determine the visual positioning mark block, and fix the visual positioning mark block on the sliding plate mechanism of the tundish.

In some embodiments, the outline of the visual locator block is a circular outline.

Optionally, the background color, background, shape and size of the visual positioning marker block and the radius of the circle outline can be determined according to the on-site environment. After the design and processing of the visual positioning marker block is completed, the relative positional relationship between the centers of the circle outlines is recorded.

In some embodiments, a schematic diagram of the structure of each sub-marker block on the visual positioning mark block can be seen in FIG. 2 , there are 5 sub-marker blocks with different radii on the visual positioning mark block, and the sub-marker block with the smallest radius is the central sub-marker block A, the remaining 4 sub-sign blocks are evenly distributed around the center sub-sign block A. At this time, the preset point in the outline of the sub-marker block is set as the center of the circle outline, and the relative position information between the preset points in the outline of each sub-marker block can be determined by the following method: The center of the marker block A is the origin of the coordinates to establish the marker block coordinate system; the coordinates of the center of the four circle outlines distributed around the center sub-marker block A in the marker block coordinate system can be directly designed in the marker block design stage or measured later. Obtained: Based on the coordinates of the center of the four circle outlines distributed around the central sub-marker block A, its relative position information is determined.

Optionally, the shape, size, background color and background of the visual positioning mark block are determined according to the production site environment of the continuous casting tundish based on the clear imaging of the photographing equipment and the obvious imaging characteristics of the visual positioning mark block.

S302: Obtain a relative attitude transformation matrix.

Optionally, the relative attitude transformation matrix includes the attitude transformation matrix between the initial visual positioning marker block attitude in the robot base coordinate system and the initial end effector attitude of the end effector of the robot when the shroud is successfully installed.

In some embodiments, the determination of the reference pose transformation matrix is as follows:

The industrial camera is fixedly installed at the end of the robot joint, and the first attitude transformation matrix between the camera coordinate system and the end tool coordinate is calculated through the hand-eye calibration operation;

Calculate the second attitude transformation matrix from the end tool coordinate system to the robot base coordinate system through the real-time joint parameter feedback of the robot control system;

The reference attitude transformation matrix from the camera coordinate system to the robot base coordinate system is determined according to the first attitude transformation matrix and the second attitude transformation matrix.

In some embodiments, the relative attitude transformation matrix is determined as follows:

The manual servo robot is operated to install the long nozzle, and the preset shooting point is set on the servo path as the shooting point of the industrial camera. The posture of the visual positioning mark block in the camera coordinate system, and according to the reference attitude transformation matrix, the posture of the visual positioning mark block in the camera coordinate system is transformed into the initial mark block posture of the visual positioning mark block in the robot base coordinate system;

When the servo teaching completes the installation of the long shroud, record the initial end effector posture of the end effector in the robot base coordinate system;

The relative pose transformation matrix is generated from the initial landmark block pose and the initial end effector pose.

Optionally, the visual positioning mark block includes at least 4 circular contours with different radii, the circular contour with the smallest radius is used as the central circle contour, and the rest of the circular contours are distributed and set around the central circle contour, and the initial mark block posture is obtained as follows:

Process the initial image captured by the industrial camera, and extract the image area of interest including the imaging area of each sub-marker block;

Extract the pixel information of the center of all circle outlines in the region of interest;

According to the obtained pixel information of the circle center and the relative position relationship between the circle centers, the monocular vision PnP algorithm is used to calculate the three-dimensional pose of the center circle contour center in the camera coordinate system, and it is regarded as the visual positioning mark block in the camera coordinate system. tie down posture;

According to the pose of the visual positioning marker block in the camera coordinate system and the transformation matrix of the reference attitude, the initial marker block pose is determined.

The relative positional relationship between the circle centers may be determined during the design and production of the visual positioning marker block, the relative positional relationship between the circle centers may also be calculated in advance through measurement, and the relative positional relationship between the circle centers may also be obtained through other related technologies The means are obtained, which is not limited here.

Optionally, because the relative position of the visual positioning marker block and the water outlet of the sliding plate mechanism is fixed, the above-mentioned relative attitude transformation matrix is fixed.

S303: Obtain the current marker block pose.

Optionally, the industrial camera captures a current image including the visual positioning marker block at a preset shooting point, determines the pose of the current visual positioning marker block in the camera coordinate system based on the current image based on methods such as the monocular vision PnP algorithm, and determines the pose of the current visual positioning marker block in the camera coordinate system according to the current image. The reference attitude transformation matrix converts the pose of the current visual positioning marker block in the camera coordinate system into the current marker block pose of the visual positioning marker block in the robot base coordinate system.

S304: Determine the current end effector attitude according to the current marker block attitude and the relative attitude transformation matrix, and install the shroud.

Optionally, when automatic loading and unloading of the shroud is required, the current end effector posture of the end effector is calculated to guide the end effector of the robot to move to the current end effector posture, thereby completing the installation of the shroud.

An exemplary installation process is as follows: the robot moves to a preset shooting point, the industrial camera collects the current image, and through image processing and attitude transformation, the current marker block pose of the current visual positioning marker block in the robot base coordinate system is calculated, and the steps are combined. The relative posture transformation matrix obtained in S302 is calculated to obtain the current end effector posture, and guide the robot to move to the current end effector posture and complete the installation of the long shroud.

S305: Obtain the exit point of the end effector, determine the disassembly posture, control the end effector to reach the disassembly posture, and disassemble the long shroud.

Optionally, after the installation of the shroud is completed, the end effector has an exit installation point, and the attitude of the exit point is recorded as the disassembly attitude, and the disassembly attitude is regarded as the disassembly attitude that the end effector needs to reach when the shroud is removed. When the shroud needs to be disassembled, guide the robot to move to the above disassembly posture and complete the disassembly operation of the shroud.

The embodiment of the present invention provides a specific method for replacing a long shroud based on visual servoing. The current mark block image of the visual positioning mark block is obtained through an industrial camera, wherein the industrial camera is installed at the joint end of the industrial robot, and the visual positioning mark block is fixedly installed On the slide mechanism of continuous casting tundish, the current position of the visual positioning marker block in the robot base coordinate system is obtained through image processing and coordinate transformation. The reached current end effector attitude guides the robot to reach the current end effector attitude and completes the installation of the shroud; realizes the automation of the installation of the shroud, replacing the high-intensity and high-risk manual operation, the method is simple, easy to implement, and practical and greater applicability. At the same time, based on the position of the visual positioning marker block in the current image, the current position of the tundish can be obtained, which is helpful for the robot to calculate the required current end effector posture for shroud replacement in real time according to the different positions of the tundish, and guide the robot to reach the position of the tundish. The current end effector attitude completes the replacement of the shroud.

Optionally, record the exit point of the end effector after the shroud is successfully installed, determine the disassembly posture, guide the robot to move to the above disassembly posture when the shroud needs to be disassembled, and complete the disassembly operation of the shroud, thus realizing the automation of disassembling the shroud. , replacing high-intensity and high-risk manual operations, the method is simple and easy to implement, and has high practicability and applicability.

Embodiment 3

Please refer to FIG. 4 , a long shroud replacement device 400 based on visual servoing, including:

The assembly module 401 is used for assembling the visual positioning marker block on the tundish;

The current marker block attitude obtaining module 402 is used to obtain the current image of the visual positioning marker block and determine the current marker block attitude of the visual positioning marker block;

A determination module 403, configured to determine the current end effector attitude according to the current marker block attitude and the relative relationship, and the relative relationship includes the mapping between the marker block attitude and the end effector attitude;

The control module 404 is used to control the end effector to reach the current end effector attitude, and to install and/or remove the shroud.

In this embodiment, the visual servo-based shroud replacement device is essentially provided with multiple modules to execute the visual servo-based shroud replacement method in the above-mentioned embodiment. For specific functions and technical effects, refer to the above-mentioned first embodiment. That's it, and will not be repeated here.

Embodiment 4

Referring to FIG. 5, an embodiment of the present invention further provides a terminal 500, including a processor 501, a memory 502, and a communication bus 503;

A communication bus 503 is used to connect 502 the processor 501 and the memory;

The processor 501 is configured to execute the computer program stored in the memory 502 to implement the method for replacing a shroud based on visual servoing as described in one or more of the above-mentioned first embodiment.

An embodiment of the present invention also provides a computer-readable storage medium, characterized in that a computer program is stored thereon,

The computer program is used to make the computer execute the method for replacing the shroud based on the visual servoing described in any one of the above-mentioned first embodiment.

Embodiments of the present application further provide a non-volatile readable storage medium, where one or more modules (programs) are stored in the storage medium, and when the one or more modules are applied to a device, the device can be executed by the device. Instructions for steps included in Embodiment 1 of the embodiments of the present application.

It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the above two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or a combination of any of the above. More specific examples of computer readable storage media may include, but are not limited to, electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), erasable Programmable read only memory (EPROM or flash memory), optical fiber, portable compact disk read only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In the present disclosure, however, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave with computer-readable program code embodied thereon. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device . Program code embodied on a computer readable medium may be transmitted using any suitable medium including, but not limited to, electrical wire, optical fiber cable, RF (radio frequency), etc., or any suitable combination of the foregoing.

The above-mentioned computer-readable medium may be included in the above-mentioned electronic device; or may exist alone without being assembled into the electronic device.

Computer program code for carrying out operations of the present disclosure may be written in one or more programming languages, including object-oriented programming languages—such as Java, Smalltalk, C++, but also conventional Procedural programming language - such as the "C" language or similar programming language. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (eg, using an Internet service provider through Internet connection).

The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code that contains one or more logical functions for implementing the specified functions executable instructions. It should also be noted that, in some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It is also noted that each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations, can be implemented in dedicated hardware-based systems that perform the specified functions or operations , or can be implemented in a combination of dedicated hardware and computer instructions.

The above-mentioned embodiments merely illustrate the principles and effects of the present invention, but are not intended to limit the present invention. Anyone skilled in the art can modify or change the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or changes made by those with ordinary knowledge in the technical field without departing from the spirit and technical idea disclosed in the present invention should still be covered by the claims of the present invention.

Claims (6)

1. A long nozzle replacing method based on visual servo is characterized by comprising the following steps:

assembling a visual positioning mark block on a tundish, wherein the visual positioning mark block is fixedly arranged on a continuous casting tundish slide plate mechanism and comprises a central sub-mark block and at least two peripheral sub-mark blocks, and the peripheral sub-mark blocks are arranged around the central sub-mark block;

determining a reference posture transformation matrix from a camera coordinate system to a robot coordinate system according to hand-eye calibration and joint parameter feedback of a robot control system;

acquiring a current image of the visual positioning mark block and determining a current mark block posture of the visual positioning mark block, wherein the determining of the current mark block posture of the visual positioning mark block comprises determining an interested region in the current image, the interested region comprises an imaging region of the visual positioning mark block, acquiring pixel information of preset points in the outline of each sub mark block in the interested region and relative position information between the preset points in the outline of each sub mark block, determining the posture of the preset point in the outline of a central sub mark block under a camera coordinate system according to the pixel information and the relative position information, and determining the current mark block posture according to the reference posture transformation matrix and the posture of the preset point in the outline of the central sub mark block under the camera coordinate system;

determining a current end effector posture according to the current mark block posture and a relative relation, wherein the relative relation comprises mapping between the mark block posture and the end effector posture, the relative relation comprises a relative posture transformation matrix, the relative posture transformation matrix is determined in the following mode, an initial image of a visual positioning mark block is obtained at a preset shooting point, the initial mark block posture of the visual positioning mark block is determined according to the initial image, a servo robot is taught to install a long nozzle, the initial end effector posture of the end effector is obtained when the installation of the long nozzle is finished, and the relative posture transformation matrix is determined according to the initial mark block posture and the initial end effector posture;

controlling the end effector to reach the current end effector posture, and installing and/or disassembling the long nozzle;

and the step of disassembling the long nozzle comprises the steps of obtaining the exit point of the end effector if the long nozzle is completely installed, determining a disassembling gesture, controlling the end effector to reach the disassembling gesture, and disassembling the long nozzle.

2. The method of claim 1, wherein the controlling the end effector to the current end effector position, and installing the long nozzle comprises:

the end effector acquires a long nozzle;

controlling the end effector to reach the current end effector pose;

and installing the long nozzle.

3. A long nozzle changing method based on visual servo according to any of claims 1 or 2, further comprising at least one of:

acquiring the current image at a preset shooting point through image acquisition equipment, wherein the image acquisition equipment is assembled at the tail end of a joint of the robot;

the visual locator block comprises at least 3 different sub-marker blocks;

the visual positioning mark block comprises at least two background colors, and the current background color of the visual positioning mark block is determined according to the background color of the scene.

4. A long nozzle replacing device based on visual servo is characterized by comprising:

the assembly module is used for assembling a visual positioning mark block on a tundish, the visual positioning mark block is fixedly arranged on a continuous casting tundish slide plate mechanism, the visual positioning mark block comprises a central sub-mark block and at least two peripheral sub-mark blocks, and the peripheral sub-mark blocks are arranged around the central sub-mark block;

a current mark block posture obtaining module, configured to determine a reference posture transformation matrix from a camera coordinate system to a robot coordinate system according to hand-eye calibration and joint parameter feedback of a robot control system, obtain a current image of the visual positioning mark block, and determine a current mark block posture of the visual positioning mark block, where determining the current mark block posture of the visual positioning mark block includes determining an area of interest in the current image, where the area of interest includes an imaging area of the visual positioning mark block, obtaining pixel information of preset points in a contour of each sub-mark block in the area of interest, and relative position information between the preset points in the contour of each sub-mark block, determining a posture of a preset point in a contour of a center sub-mark block in the camera coordinate system according to the pixel information and the relative position information, and determining a posture of the preset point in the contour of the center sub-mark block in the camera coordinate system according to the reference posture transformation matrix and the posture transformation matrix of the preset point in the contour of the center sub-mark block in the camera coordinate system according to the reference posture transformation matrix and the joint parameter feedback of the robot control system Determining the current mark block posture;

a determining module, configured to determine a current end effector pose according to the current marker block pose and a relative relationship, where the relative relationship includes mapping between the marker block pose and the end effector pose, the relative relationship includes a relative pose transformation matrix, the relative pose transformation matrix is determined by obtaining an initial image of a visual positioning marker block at a preset shooting point, determining an initial marker block pose of the visual positioning marker block according to the initial image, instructing a robot to install a long nozzle, obtaining an initial end effector pose of the end effector when the installation of the long nozzle is completed, and determining a relative pose transformation matrix according to the initial marker block pose and the initial end effector pose;

the control module is used for controlling the end effector to reach the current attitude of the end effector and installing and/or disassembling the long nozzle;

and the step of disassembling the long nozzle comprises the steps of obtaining the exit point of the end effector if the long nozzle is completely installed, determining a disassembling gesture, controlling the end effector to reach the disassembling gesture, and disassembling the long nozzle.

5. A terminal comprising a processor, a memory, and a communication bus;

the communication bus is used for connecting the processor and the memory;

the processor is configured to execute a computer program stored in the memory to implement the method for long nozzle replacement based on visual servoing according to any of claims 1-3.

6. A computer-readable storage medium, having stored thereon a computer program,

the computer program is for causing the computer to execute the long nozzle changing method based on visual servoing of any of claims 1-3.

Images