CN115284297A - Workpiece positioning method, robot and robot operation method - Google Patents

Abstract

The present application discloses a workpiece positioning method, computer equipment, computer-readable storage medium, robot and robot operation method. The solution constructs a virtual camera and a workpiece model, uses the virtual camera to photograph the workpiece model in the virtual scene, and obtains the workpiece model in the The virtual point cloud in the base coordinate system matches the virtual point cloud with the actual point cloud obtained by the workpiece image collected by the camera device, and determines the position of the workpiece based on the matching result, avoiding the direct matching of the actual point cloud and the workpiece model. The inaccurate matching improves the accuracy of real-time positioning of the workpiece, thereby improving the quality and efficiency of robot operations.

Description

Workpiece positioning method, robot and robot operation method

technical field

The present application relates to the field of robot technology, in particular to a workpiece positioning method, computer equipment, computer readable storage medium, robot and robot operation method.

Background technique

In the field of robot operations, obtaining the position of the workpiece by scanning the workpiece with a line laser, and obtaining the position of the workpiece by manual teaching and programming, etc., have problems such as long time-consuming and low efficiency. At the same time, since the point cloud data of the workpiece collected by line laser scanning will have many noise points, the accuracy is low, resulting in low quality and efficiency of the robot's work; and the manual teaching programming method introduces artificial uncontrollable factors, which often lead to ACCIDENT.

With the rapid development of robot technology, 3D vision technology, and the upgrading of industrial intelligent manufacturing, robot vision is more and more used in industrial production and service industries. Guiding robot operations through visual systems is the key to realizing intelligent robot operations. important means. Based on the three-dimensional vision to guide the robot operation, the efficiency is higher and the positioning of the workpiece is more accurate.

However, in the prior art, the workpiece point cloud data acquired by the camera device is directly matched with the virtual point cloud data corresponding to the workpiece model. The workpiece model contains thickness data, and the camera device can only collect the workpiece within the line of sight. The point cloud data directly matches the workpiece point cloud data with the virtual point cloud data in the workpiece model. It cannot ensure that the virtual point cloud data matched with the workpiece point cloud data is corresponding, and the matching error is large. As shown in Figure 1, mark a represents the contour of the workpiece point cloud, mark b represents the contour of the virtual point cloud obtained directly from the workpiece model, mark a and mark b are not located on the same plane, but have a certain distance; that is, The virtual point cloud data is not completely corresponding to the workpiece point cloud data, which will lead to inaccurate matching.

application content

In order to solve the problem of inaccurate workpiece positioning, the present application provides a workpiece positioning method, computer equipment, computer readable storage medium, robot and robot operation method.

According to an aspect of the embodiments of the present application, a workpiece positioning method is disclosed, which is used in a scene where a robot works on a workpiece, and the robot is provided with a camera device. The workpiece positioning method includes:

Acquiring an actual point cloud of the workpiece in the base coordinate system of the robot based on the image of the workpiece collected by the camera device;

Obtaining a workpiece model corresponding to the workpiece in the base coordinate system;

Constructing a virtual camera and a virtual scene, placing the workpiece model in the virtual scene, and making the workpiece model within the field of view of the virtual camera;

photographing the workpiece model by using the virtual camera, and obtaining a virtual point cloud of the workpiece model in the base coordinate system;

Matching the actual point cloud and the virtual point cloud to obtain the position of the workpiece.

In an exemplary embodiment, the camera device is arranged at the end of the robot; the actual point cloud of the workpiece in the base coordinate system of the robot is obtained based on the image of the workpiece collected by the camera device ,include:

Obtaining the workpiece image collected by the camera device and the pose matrix of the robot when the camera device collects the workpiece image;

According to the pose matrix and the hand-eye matrix of the robot, the coordinates of the point cloud in the workpiece image in the camera coordinate system are transformed into the base coordinate system to obtain the actual point cloud; the hand-eye matrix Indicates the transformation relationship between the camera coordinate system and the tool coordinate system of the robot.

In an exemplary embodiment, said constructing a virtual camera includes:

determining the position of the virtual camera in the virtual scene according to the pose matrix and the hand-eye matrix;

According to the parameters of the camera device, the parameters of the virtual camera are configured to construct the virtual camera; the parameters include field angle and pixels.

In an exemplary embodiment, configuring parameters of the virtual camera according to parameters of the camera device includes:

Configuring the angle of view of the virtual camera to be consistent with the angle of view of the camera device;

The pixels of the virtual camera are configured to coincide with the pixels of the camera device.

In an exemplary embodiment, the determining the position of the virtual camera in the virtual scene according to the pose matrix and the hand-eye matrix includes:

determining a position of the camera device relative to the robot based on the pose matrix and the hand-eye matrix;

According to the position of the camera device relative to the robot, the position of the virtual camera in the virtual scene is set, so that the position of the virtual camera corresponds to the position of the camera device.

In an exemplary embodiment, the acquiring the workpiece model of the workpiece in the base coordinate system includes:

acquiring image data from multiple angles of the workpiece;

Using three-dimensional modeling software, the image data of the multiple angles are integrated into a three-dimensional digital model;

Transforming the three-dimensional digital-analog into the base coordinate system to obtain the workpiece model.

In an exemplary embodiment, the matching the actual point cloud and the virtual point cloud to obtain the position of the workpiece includes:

An iterative nearest neighbor algorithm is used to register the actual point cloud and the virtual point cloud to obtain the position of the workpiece.

According to an aspect of the embodiment of the present application, a robot is disclosed, including:

a robot body, the robot body has a plurality of motion axes;

a camera device, the camera device is arranged on the robot body; and

A controller, the controller is connected with the robot body and the camera device, and is used to control the robot body and the camera device, and execute the steps of the workpiece positioning method as described above.

According to an aspect of the embodiment of the present application, a robot working method is disclosed, the robot is provided with an end tool and a camera device, and the working method includes:

controlling the camera device to collect images of workpieces;

Using the above-mentioned workpiece positioning method to perform workpiece positioning to obtain the position of the workpiece;

Controlling the end tool to move to the position of the workpiece for operation.

In an exemplary embodiment, the end tool is a welding gun, and the camera device is a three-dimensional camera.

According to one aspect of the embodiments of the present application, a computer device is disclosed, which is used in a scene where a robot works on a workpiece, and the robot is provided with a camera device. This computer equipment includes:

The first acquisition module is used to acquire the actual point cloud of the workpiece in the base coordinate system of the robot based on the image of the workpiece collected by the camera device;

The first building block is used to obtain the workpiece model corresponding to the workpiece in the base coordinate system;

The second building block is used to construct a virtual camera and a virtual scene, place the workpiece model in the virtual scene, and make the workpiece model within the field of view of the virtual camera;

A second acquiring module, configured to use the virtual camera to photograph the workpiece model, and acquire a virtual point cloud of the workpiece model in the base coordinate system;

A point cloud matching module, configured to match the actual point cloud and the virtual point cloud to obtain the position of the workpiece.

According to an aspect of the embodiment of the present application, a computer device is disclosed, including:

one or more processors;

The memory is used to store one or more programs, and when the one or more programs are executed by the one or more processors, the computer device implements the aforementioned workpiece positioning method.

According to an aspect of the embodiments of the present application, a computer-readable storage medium is disclosed, the computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by a processor of a computer, all The computer executes the aforementioned workpiece positioning method.

The technical solutions provided by the embodiments of the present application at least include the following beneficial effects:

The technical solution provided by this application constructs a virtual camera and a workpiece model, uses the virtual camera to shoot the workpiece model in the virtual scene, obtains the virtual point cloud of the workpiece model in the base coordinate system, and combines the virtual point cloud with the workpiece collected by the camera device The actual point cloud matching obtained from the image determines the position of the workpiece based on the matching result, avoiding the inaccurate matching caused by directly matching the actual point cloud with the workpiece model, improving the accuracy of real-time positioning of the workpiece, thereby improving the quality and efficiency.

It is to be understood that both the foregoing general description and the following detailed description are exemplary only and are not restrictive of the application.

Description of drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and together with the description serve to explain the principles of the application.

Fig. 1 is an interface diagram when a workpiece point cloud is matched with a workpiece model in the prior art.

Fig. 2 is a structural diagram of a robot shown in an exemplary embodiment.

Fig. 3 is a flow chart of a workpiece positioning method according to an exemplary embodiment.

FIG. 4 is a detailed flowchart of step S101 in the embodiment corresponding to FIG. 3 .

FIG. 5 is a detailed flowchart of step S102 in the embodiment corresponding to FIG. 3 .

FIG. 6 is a partially detailed flowchart of step S103 in the embodiment corresponding to FIG. 3 .

FIG. 7 is a detailed flowchart of step S1031 in the embodiment corresponding to FIG. 6 .

Fig. 8 is a schematic diagram of a virtual camera and a workpiece model according to an exemplary embodiment.

Fig. 9 is an interface diagram of matching the actual point cloud with the virtual point cloud according to an exemplary embodiment.

Fig. 10 is a flow chart of a robot working method according to an exemplary embodiment.

Fig. 11 is a block diagram showing a computer device for implementing the embodiment of the present application according to an exemplary embodiment.

Fig. 12 is a structural block diagram of a computer system for implementing a computer device according to an exemplary embodiment of the present application.

The reference signs are explained as follows:

100. Robot; 101. Robot body; 102. Camera device; 103. End tool; 200. Computer equipment; 201. First acquisition module; 202. First building module; 203. Second building module; 204. Second acquisition Module; 205, point cloud matching module; 300, computer system; 301, CPU; 302, ROM; 303, storage part; 304, RAM; 305, bus; 306, I/O interface; 307, input part; 308, output part; 309, communication part; 310, driver; 311, removable medium; 401, virtual camera; 402, workpiece model.

Detailed ways

Although the present application may readily take the form of embodiments, only some of them are shown in the drawings and will be described in detail in this specification, and it is to be understood that this specification should be considered as the It is an exemplary illustration of the principles of the application and is not intended to limit the application to what is described herein.

In addition, the terms "comprising", "comprising", "having" and any variations thereof mentioned in the description of the present application are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device comprising a series of steps or modules is not limited to the listed steps or modules, but optionally also includes other unlisted steps or modules, or optional Other steps or modules inherent to these processes, methods, products or devices are also included.

In addition, the terms "first" and "second" are used for descriptive purposes only, and cannot be interpreted as indicating or implying relative importance or implicitly specifying the quantity of indicated technical features. Thus, features defined as "first" and "second" may explicitly or implicitly include one or more features.

In the description of the present application, unless otherwise specified, the meaning of "plurality" refers to two or more.

It should be noted that, in the embodiments of the present application, words such as "exemplary" or "example" or "for example" are used as examples, illustrations or illustrations. Any embodiment or design solution described as "exemplary" or "example" or "for example" in the embodiments of the present application shall not be interpreted as being more preferred or more advantageous than other embodiments or design solutions. Rather, the use of words such as "exemplary" or "for example" or "such as" is intended to present related concepts in a concrete manner.

Exemplary embodiments will be described in detail below. The implementations described in the following exemplary embodiments do not represent all implementations consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with aspects of the present application, as detailed in the Summary of the Application.

Fig. 2 shows a structural diagram of a robot of an exemplary embodiment. As shown in Figure 2, the robot 100 includes a robot body 101, a camera device 102 and an end tool 103, the camera device 102 is arranged on the robot body 101, and the end tool 103 is arranged at the end of the robot body 101; the camera device 102 is driven by the robot body 101 Move with the end tool 103, so as to use the camera device 102 to take a close-range image of the workpiece, and then obtain the position of the workpiece by analyzing the image of the workpiece, and further drive the end tool 103 to move to the position of the workpiece for operation.

It can be understood that the robot 100 also includes a controller, the controller is connected with the robot body 101, the camera device 102 and the end tool 103 to control the movement of the robot body 101, and control the camera device 102 to take images of the workpiece, and the controller also analyzes The image of the workpiece is obtained to obtain the position of the workpiece, and then the movement of the robot body 101 can be controlled to drive the end tool 103 to move to the position of the workpiece for operation. It can be understood that the controller can be built in the robot body 101 , or can be arranged outside the robot body 101 .

The end tool 103 may be a welding torch, the workpiece is the object to be welded, and the robot 100 is a welding robot at this time. The end tool 103 may be a glue gun, the workpiece is the object to be glued, and the robot 100 is a glue-applying robot. The end tool 103 can also be a cutting tool, the workpiece is the object to be cut, and the robot 100 is a cutting robot at this time. Of course, the end tool 103 can also be other tools that can be arranged at the end of the robot 100 and driven by the robot 100 to perform operations, not limited to the aforementioned welding gun, glue gun, knife, etc.

The embodiments of the present application provide a workpiece positioning method, computer equipment, computer readable storage medium, robot and robot operation method, which can accurately locate the real-time position of the workpiece, thereby improving the quality and efficiency of the robot operation.

The workpiece positioning method, computer equipment, computer-readable storage medium, robot and robot operation method provided in the embodiments of the present application are specifically described through the following embodiments. First, the workpiece positioning method in the embodiments of the present application is described.

The embodiment of the present application firstly provides a workpiece positioning method, which is used in a scene where a robot performs three-dimensional positioning on a workpiece to work on the workpiece, and the robot is provided with a camera device. The workpiece positioning method includes:

Based on the image of the workpiece collected by the camera device, the actual point cloud of the workpiece in the base coordinate system of the robot is obtained;

Obtain the workpiece model corresponding to the workpiece in the base coordinate system;

Construct a virtual camera and a virtual scene, place the workpiece model in the virtual scene, and make the workpiece model within the field of view of the virtual camera;

Use a virtual camera to shoot the workpiece model, and obtain the virtual point cloud of the workpiece model in the base coordinate system;

Match the actual point cloud and the virtual point cloud to obtain the position of the workpiece.

The technical solution provided by this application constructs a virtual camera and a workpiece model, uses the virtual camera to shoot the workpiece model in the virtual scene, obtains the virtual point cloud of the workpiece model in the base coordinate system, and combines the virtual point cloud with the workpiece collected by the camera device The actual point cloud matching obtained by the image determines the position of the workpiece based on the matching result, avoiding the inaccurate matching caused by directly matching the actual point cloud with the workpiece model, and has strong anti-interference ability to the actual point cloud noise of the workpiece, which improves the real-time positioning of the workpiece accuracy, thereby improving the quality and efficiency of robot operations. At the same time, it also avoids the operation by teaching the robot, reduces the workload of the staff, and improves the operation quality and efficiency of the robot.

The implementation manners of the present application will be further described in detail below in conjunction with the accompanying drawings in the embodiments of the present specification.

Referring to FIG. 3 , the workpiece positioning method provided by an exemplary embodiment of the present application includes the following steps S101-S105.

S101. Acquire an actual point cloud of the workpiece in the base coordinate system of the robot based on the image of the workpiece collected by the camera device.

In an exemplary embodiment, a camera device is mounted at the end of the robot, as shown in FIG. 2 . In this exemplary embodiment, as shown in FIG. 4 , step S101 includes the following steps S1011-S1012.

S1011, acquiring the image of the workpiece collected by the camera device and the pose matrix of the robot when the image of the workpiece is collected by the camera device.

S1012. According to the pose matrix and the hand-eye matrix of the robot, the coordinates of the point cloud in the workpiece image in the camera coordinate system are transformed into the base coordinate system to obtain an actual point cloud.

Among them, the hand-eye matrix represents the transformation relationship between the camera coordinate system and the tool coordinate system of the robot; specifically, it includes the translation from the end of the robot to the camera device installed on the end plus the rotation from the end of the robot to the camera device installed on the end. As for how to obtain the hand-eye matrix is the prior art, it will not be repeated here.

Among them, the pose matrix represents the transformation relationship between the tool coordinate system of the robot and the base coordinate system of the robot. The tool coordinate system is used to define the center position of the end tool and the pose of the end tool.

In detail, in step S1012, based on the mapping relationship pcd1=transform(pcd0, toolPos*handEye), the coordinates of the point cloud in the workpiece image in the camera coordinate system are transformed into the base coordinate system to obtain the actual point cloud.

Among them, pcd1 represents the actual point cloud in the robot base coordinate system after transformation, pcd0 represents the actual point cloud in the camera coordinate system before transformation, transform represents the transformation function, toolPos represents the pose matrix of the current robot, and the pose matrix is a A 4*4 matrix, handEye represents the positional relationship matrix from the TCP at the end of the robot to the origin of the camera coordinate system, that is, the hand-eye matrix.

S102. Obtain a workpiece model corresponding to the workpiece in the base coordinate system.

In an exemplary embodiment, as shown in FIG. 5 , step S102 includes the following steps S1021-S1023.

S1021. Acquire image data from multiple angles of the workpiece.

S1022, using 3D modeling software to integrate image data from multiple angles into a 3D digital model.

Understandably, a 3D digital model is a product model made with 3D modeling software, such as UG, CATIA, etc. A three-dimensional digital model is a polygonal representation of an object, usually displayed by a computer or other video equipment.

S1023, converting the three-dimensional digital model from the world coordinate system to the base coordinate system to obtain the workpiece model.

Wherein, the workpiece model may be, for example, an STL image file or the like.

S103, constructing a virtual camera and a virtual scene, placing the workpiece model in the virtual scene, and making the workpiece model within the field of view of the virtual camera.

Specifically, software such as Blender can be used to construct a virtual camera.

In an exemplary embodiment, as shown in FIG. 6, in step S103, constructing a virtual camera includes the following steps S1031-S1032.

S1031. Determine the position of the virtual camera in the virtual scene according to the pose matrix and the hand-eye matrix of the robot.

In an exemplary embodiment, as shown in FIG. 7, step S1031 includes the following steps S10311-S10312.

S10311. Determine the position of the camera device relative to the robot according to the pose matrix and the hand-eye matrix.

In detail, in step S10311, the position of the camera device relative to the robot is determined based on the mapping relation camPos=toolPos*handEye.

Among them, camPos represents the position of the camera device relative to the robot, toolPos represents the pose matrix of the robot when the camera device captures the image of the workpiece, and handEye represents the hand-eye matrix.

S10312. Set the position of the virtual camera in the virtual scene according to the position of the camera device relative to the robot, so that the position of the virtual camera corresponds to the position of the camera device.

S1032. Configure the parameters of the virtual camera according to the parameters of the camera device to construct a virtual camera.

In detail, the aforementioned parameters include viewing angle and pixels. In step S1032, the angle of view of the virtual camera is configured to be consistent with the angle of view of the camera device, and the pixels of the virtual camera are configured to be consistent with those of the camera device.

By setting the position of the virtual camera in the virtual scene to correspond to the position of the camera device on the robot, and setting the parameters of the virtual camera to be consistent with the parameters of the camera device, it can be ensured that the virtual point cloud collected by the virtual camera is consistent with the The position of the actual point cloud collected by the camera device is corresponding to improve the accuracy of workpiece positioning.

In an exemplary embodiment, the field angle camView of the camera device is 0.87, and the resolution of the camera device is x=1280, y=1024; correspondingly, the field angle camView of the virtual camera is 0.87, and the resolution of the virtual camera is The rate size is x=1280, y=1024.

An exemplary embodiment of a virtual camera and a workpiece model is shown in FIG. 8 , wherein a mark 401 denotes a virtual camera, and a mark 402 denotes a workpiece model.

It can be understood that the aforementioned parameters may also include other parameters of the camera, for example, the pitch angle, azimuth angle, and roll angle of the camera.

S104, using a virtual camera to photograph the workpiece model, and obtaining a virtual point cloud of the workpiece model in the base coordinate system.

In the embodiment where the position of the virtual camera is set to correspond to the position of the camera device relative to the robot, and the field angle and other parameters are consistent with the camera device, what is obtained in step S104 is the workpiece model under the same viewing angle as the actual point cloud surface point cloud.

S105, matching the actual point cloud and the virtual point cloud to obtain the position of the workpiece.

In an exemplary embodiment, in step S105, an iterative nearest neighbor algorithm is used to register the actual point cloud and the virtual point cloud to obtain the position of the workpiece.

Understandably, using the iterative nearest neighbor algorithm, also known as ICP registration (Point CloudRegistration), refers to inputting two point clouds (source data and registration data), and outputting a transformation (compensation matrix) to make the source data and registration data The degree of coincidence of quasi-data is as high as possible. Transformations can be rigid or not, rigid transformations include rotations and translations. How to use the iterative nearest neighbor algorithm to match the virtual point cloud to the position of the actual point cloud is a prior art, and will not be repeated here.

Fig. 9 is an interface diagram of matching the actual point cloud with the virtual point cloud according to an exemplary embodiment. As shown in Figure 9, the virtual point cloud and the actual point cloud are attached on the same surface, and the virtual point cloud and the actual point cloud can be completely matched together. Compared with the prior art shown in Figure 1, the matching accuracy of this application is It is obviously improved, and the position of the workpiece obtained by matching is the real position of the workpiece in the robot base coordinate system.

Please refer to FIG. 2 again. In order to realize the workpiece positioning method provided by the embodiment of the present application, the embodiment of the present application provides a robot 100, which includes a robot body 101, a camera device 102, an end tool 103 and a controller (not shown in the figure). ). The robot body 101 has multiple motion axes, and the camera device 102 and the end tool 103 are arranged at the end of the robot body 101 . The controller is connected to the robot body 101, the camera device 102, and the end tool 103, and is used to control the robot body 101, the camera device 102, and the end tool 103, so that the robot 100 can execute the workpiece positioning method shown in any one of Fig. 3 to Fig. 7 all or part of the steps.

For example, the robot body 101 has six motion axes, that is, the robot 100 is a six-axis robot.

For example, the end tool 103 is a welding gun.

Fig. 10 shows a flow chart of a robot working method in an exemplary embodiment. As shown in FIG. 10 , the robot working method includes the following steps S201-S203.

S201. Control the camera device to collect images of workpieces.

S202, using the aforementioned workpiece positioning method to perform workpiece positioning to obtain the position of the workpiece.

S203, controlling the end tool to move to the position of the workpiece for operation.

In an exemplary embodiment, as shown in FIG. 2, the end tool 103 is a welding torch, and the workpiece is an object to be welded. In step S202, the position of the weld seam in the workpiece is obtained. In step S203, the end tool is controlled to move to the workpiece. The position of the middle weld seam, and use the end tool 103 to weld the weld seam of the workpiece.

Please refer to FIG. 11 next. FIG. 11 is a block diagram of a computer device 200 according to an exemplary embodiment. The computer device 200 can be applied to a robot to execute the workpiece positioning method shown in any one of FIGS. 2 to 7 all or part of the steps. As shown in FIG. 11 , the computer device 200 includes but not limited to: a first acquisition module 201 , a first construction module 202 , a second construction module 203 , a second acquisition module 204 and a point cloud matching module 205 .

Wherein, the first acquisition module 201 is configured to acquire the actual point cloud of the workpiece in the base coordinate system of the robot based on the image of the workpiece collected by the camera device.

The first construction module 202 is used to obtain the workpiece model corresponding to the workpiece in the base coordinate system.

The second construction module 203 is used to construct a virtual camera and a virtual scene, place the workpiece model in the virtual scene, and make the workpiece model within the field of view of the virtual camera.

The second acquiring module 204 is used for taking pictures of the workpiece model with a virtual camera, and acquiring a virtual point cloud of the workpiece model in the base coordinate system.

The point cloud matching module 205 is used to match the actual point cloud and the virtual point cloud to obtain the position of the workpiece.

For the implementation process of the functions and effects of each module in the above computer device 200, please refer to the implementation process of the corresponding steps in the above workpiece positioning method for details, and will not be repeated here.

The above-mentioned computer device 200 may be any terminal with an information processing function, such as a desktop computer, a notebook computer, and the like.

Fig. 12 schematically shows a structural block diagram of a computer system of a computer device for implementing the workpiece positioning method according to the embodiment of the present application.

It should be noted that the computer system 300 of the computer device shown in FIG. 12 is only an example, and should not limit the functions and scope of use of this embodiment of the present application.

As shown in Figure 12, the computer system 300 includes a central processing unit 301 (Central Processing Unit, CPU), which can be stored in a program in a read-only memory 302 (Read-Only Memory, ROM) or loaded from a storage part 303 to a random Various appropriate actions and processes are executed by accessing programs in the memory 304 (Random Access Memory, RAM). In the random access memory 304, various programs and data necessary for system operation are also stored. The CPU 301 , the read only memory 302 and the random access memory 304 are connected to each other through a bus 305 . An input/output interface 306 (Input/Output interface, ie, I/O interface) is also connected to the bus 305 .

The following components are connected to the input/output interface 306: an input section 307 including a keyboard, a mouse, etc.; an output section 308 including a cathode ray tube (Cathode Ray Tube, CRT), a liquid crystal display (Liquid Crystal Display, LCD), etc., and a speaker ; a storage section 303 including a hard disk or the like; and a communication section 309 including a network interface card such as a LAN card, a modem, or the like. The communication section 309 performs communication processing via a network such as the Internet. A driver 310 is also connected to the input/output interface 306 as needed. A removable medium 311 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. is mounted on the drive 310 as necessary so that a computer program read therefrom is installed into the storage section 303 as necessary.

In particular, according to the embodiments of the present application, the processes described in the respective method flowcharts can be implemented as computer software programs. For example, the embodiments of the present application include a computer program product, which includes a computer program carried on a computer-readable medium, where the computer program includes program codes for executing the methods shown in the flowcharts. In such an embodiment, the computer program may be downloaded and installed from a network via communication portion 309 and/or installed from removable media 311 . When this computer program is executed by the central processing unit 301, various functions defined in the system of the present application are performed.

It should be noted that the computer-readable medium shown in the embodiment of the present application may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, electrical connections with one or more wires, portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable Programmable Read-Only Memory (Erasable Programmable Read Only Memory, EPROM), flash memory, optical fiber, portable compact disk read-only memory (Compact Disc Read-Only Memory, CD-ROM), optical storage device, magnetic storage device, or any suitable The combination. In the present application, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, however, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code therein. Such propagated data signals may take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium other than a computer readable storage medium that can transmit, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

Those skilled in the art should be aware that, in the above one or more examples, the functions described in this application may be implemented by hardware, software, firmware or any combination thereof. When implemented in software, the functions may be stored on or transmitted over as one or more instructions or code on a computer-readable storage medium.

Through the description of the above embodiments, those skilled in the art can clearly understand that for the convenience and brevity of the description, only the division of the above-mentioned functional modules is used as an example for illustration. In practical applications, the above-mentioned functions can be allocated according to needs It is completed by different functional modules, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above.

In the several embodiments provided in this application, it should be understood that the disclosed devices and methods may be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of modules is only a logical function division, and there may be other division methods in actual implementation. For example a plurality of units or components may be combined or may be integrated into another device, or some features may be omitted, or not implemented.

It should be understood that the present application is not limited to the precise constructions which have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the application is limited only by the appended claims.

Claims (10)

1. A workpiece positioning method, which is used in the scene of a robot working on a workpiece, the robot is provided with a camera device, and it is characterized in that the method comprises: Acquiring an actual point cloud of the workpiece in the base coordinate system of the robot based on the image of the workpiece collected by the camera device; Obtaining a workpiece model corresponding to the workpiece in the base coordinate system; Constructing a virtual camera and a virtual scene, placing the workpiece model in the virtual scene, and making the workpiece model within the field of view of the virtual camera; photographing the workpiece model by using the virtual camera, and obtaining a virtual point cloud of the workpiece model in the base coordinate system; Matching the actual point cloud and the virtual point cloud to obtain the position of the workpiece. 2. The workpiece positioning method according to claim 1, wherein the camera device is arranged at the end of the robot; the workpiece image collected based on the camera device is obtained to obtain the workpiece at the end of the robot. The actual point cloud in the base coordinate system, including: Obtaining the workpiece image collected by the camera device and the pose matrix of the robot when the camera device collects the workpiece image; According to the pose matrix and the hand-eye matrix of the robot, the coordinates of the point cloud in the workpiece image in the camera coordinate system are transformed into the base coordinate system to obtain the actual point cloud; the hand-eye matrix Indicates the transformation relationship between the camera coordinate system and the tool coordinate system of the robot. 3. workpiece positioning method according to claim 2, is characterized in that, described construction virtual camera comprises: determining the position of the virtual camera in the virtual scene according to the pose matrix and the hand-eye matrix; According to the parameters of the camera device, the parameters of the virtual camera are configured to construct the virtual camera; the parameters include field angle and pixels. 4. The workpiece positioning method according to claim 3, wherein the parameter configuring the virtual camera according to the parameter of the camera device comprises: Configuring the angle of view of the virtual camera to be consistent with the angle of view of the camera device; The pixels of the virtual camera are configured to coincide with the pixels of the camera device. 5. The workpiece positioning method according to claim 4, wherein said determining the position of said virtual camera in said virtual scene according to said pose matrix and said hand-eye matrix comprises: determining a position of the camera device relative to the robot based on the pose matrix and the hand-eye matrix; According to the position of the camera device relative to the robot, the position of the virtual camera in the virtual scene is set, so that the position of the virtual camera corresponds to the position of the camera device. 6. The workpiece positioning method according to claim 1, wherein the acquiring the workpiece model of the workpiece in the base coordinate system comprises: acquiring image data from multiple angles of the workpiece; Using three-dimensional modeling software, the image data of the multiple angles are integrated into a three-dimensional digital model; Transforming the three-dimensional digital-analog into the base coordinate system to obtain the workpiece model. 7. The workpiece positioning method according to any one of claims 1 to 6, wherein said matching said actual point cloud and said virtual point cloud to obtain the position of said workpiece comprises: An iterative nearest neighbor algorithm is used to register the actual point cloud and the virtual point cloud to obtain the position of the workpiece. 8. A robot, characterized in that, comprising: a robot body, the robot body has a plurality of motion axes; a camera device, the camera device is arranged on the robot body; and a controller, the controller is connected with the robot body and the camera device, and is used to control the robot body and the camera device, and execute the workpiece positioning method according to any one of claims 1 to 7 step. 9. A robot operation method, the robot is provided with end tool and camera device, it is characterized in that, the operation method comprises: controlling the camera device to collect images of workpieces; Using the workpiece positioning method according to any one of claims 1 to 7 to perform workpiece positioning to obtain the position of the workpiece; Controlling the end tool to move to the position of the workpiece for operation. 10. The robot working method according to claim 9, wherein the end tool is a welding torch, and the camera device is a three-dimensional camera.

Images