CN118386238A

CN118386238A - Motion control method and device of actuating mechanism, mobile robot and medium

Info

Publication number: CN118386238A
Application number: CN202410557917.5A
Authority: CN
Inventors: 张凌楷; 王辉; 张敕; 赵炎
Original assignee: Hangzhou Hikrobot Co Ltd
Current assignee: Hangzhou Hikrobot Co Ltd
Priority date: 2024-05-07
Filing date: 2024-05-07
Publication date: 2024-07-26

Abstract

The embodiment of the application provides a motion control method and device of an actuating mechanism, a mobile robot and a medium. The method comprises the following steps: acquiring a target image of a target workpiece acquired by image acquisition equipment; determining a first current position relationship between the target workpiece and the image acquisition equipment based on the target image and equipment parameters of the image acquisition equipment; determining a motion strategy of an executing mechanism based on a first current position relation, a predetermined calibration position relation and a position relation between image acquisition equipment and the executing mechanism, wherein the calibration position relation comprises the first calibration position relation between the calibration tool and the image acquisition equipment when the executing mechanism executes a preset action on the calibration tool corresponding to a target workpiece; and controlling the mobile robot to move according to a movement strategy, and controlling the executing mechanism to execute a preset action on the target workpiece when the mobile robot reaches a movement end point. Thereby improving the operation precision of the actuating mechanism of the mobile robot.

Description

Motion control method and device of actuating mechanism, mobile robot and medium

Technical Field

The present application relates to the field of robots, and in particular, to a method and apparatus for controlling motion of an actuator, a mobile robot, and a medium.

Background

Along with the continuous development of mobile robot technology, mobile robots are widely used in various production links, and power-assisted enterprises improve the degree of production automation. In general, a mobile robot may include an actuator and an image capture device. The actuating mechanism is used for realizing specific functions such as load movement, rotation, decomposition or combination and the like and comprises a motor, a speed reducer, a transmission assembly and the like. Depending on the functions implemented, the actuators may be classified into horizontal transplanting type actuators, lifting type actuators, rotating type actuators, and the like. Image acquisition devices are used to acquire images including, but not limited to, code reading cameras, flat panel cameras, three-dimensional cameras, and the like. The position recognition and the state recognition of the feature objects can be realized by processing the image acquired by the image acquisition equipment.

The product quality is closely related to the operation precision of the mobile robot, and in the related art, the operation precision of the actuating mechanism is improved by improving the precision of a transmission structure, or the positioning precision of a feature is improved by improving the perception precision of image acquisition equipment, so that the operation precision of the mobile robot is improved.

However, in the related art, in the process of moving the position of the feature object determined according to the image capturing apparatus, since the movement accuracy is still limited, there is still an error in the action execution result of the actuator, and the overall operation accuracy of the mobile robot still needs to be improved.

Disclosure of Invention

The embodiment of the application aims to provide a motion control method and device of an actuating mechanism, a mobile robot and a medium, so as to improve the operation precision of the actuating mechanism of the mobile robot. The specific technical scheme is as follows:

in a first aspect, an embodiment of the present application provides a motion control method of an actuator, where the actuator is disposed on a mobile robot, and the mobile robot further includes an image capturing device, and the method includes:

Under the condition that the mobile robot moves to an execution area, acquiring a target image of a target workpiece acquired by the image acquisition equipment;

determining a first current position relationship between the target workpiece and the image acquisition equipment based on the target image and the equipment parameters of the image acquisition equipment calibrated in advance;

Determining a motion strategy of the execution mechanism based on the first current position relationship, a predetermined calibration position relationship and a position relationship between the image acquisition equipment and the execution mechanism, wherein the calibration position relationship comprises a first calibration position relationship between the calibration tool and the image acquisition equipment in a state that the execution mechanism calibrated in advance executes a preset action on a calibration tool corresponding to the target workpiece;

and controlling the mobile robot to move according to the movement strategy, and controlling the executing mechanism to execute the preset action on the target workpiece when the mobile robot reaches a movement end point.

Optionally, the step of determining the motion policy of the actuator based on the first current position relationship, a predetermined calibration position relationship, and a position relationship between the image capturing device and the actuator includes:

determining a motion parameter corresponding to the image acquisition equipment according to the difference between the first current position relation and a predetermined calibration position relation;

and determining a motion strategy of the executing mechanism based on the motion parameters corresponding to the image acquisition equipment and the position relation between the image acquisition equipment and the executing mechanism.

Optionally, the step of determining the motion policy of the executing mechanism based on the motion parameters corresponding to the image capturing device and the positional relationship between the image capturing device and the executing mechanism includes:

If the position relation between the image acquisition equipment and the executing mechanism is unchanged, taking the motion parameter corresponding to the image acquisition equipment as the motion parameter of the executing mechanism to obtain a motion strategy of the executing mechanism; or alternatively, the first and second heat exchangers may be,

And if the position relation between the image acquisition equipment and the executing mechanism is variable, and the calibration position relation further comprises a second calibration position relation between the executing mechanism and the image acquisition equipment in a state that the executing mechanism calibrated in advance executes the preset action on the calibration tool corresponding to the target workpiece, determining the motion strategy of the executing mechanism based on the motion parameters corresponding to the image acquisition equipment, the second position relation and the current position relation between the image acquisition equipment and the executing mechanism.

Optionally, the calibration tool has a target feature, and the target feature is preset based on an actual feature of the target workpiece;

Before the step of acquiring the target image of the target workpiece acquired by the image acquisition device, the method further includes:

controlling an actuating mechanism of the mobile robot to execute a preset action on a calibration tool corresponding to the target workpiece;

Acquiring a calibration image of the calibration tool currently acquired by the image acquisition equipment;

and determining the position relation between the calibration tool and the image acquisition equipment based on the position of the target feature in the calibration image and the equipment parameter of the image acquisition equipment calibrated in advance, and taking the position relation as a calibration position relation.

Optionally, the step of determining the first current positional relationship between the target workpiece and the image acquisition device based on the target image and the device parameters of the image acquisition device calibrated in advance includes:

and determining a first current position relation between the target workpiece and the image acquisition equipment based on the position of the actual feature included in the target image and the equipment parameters of the pre-calibrated image acquisition equipment.

Optionally, the step of controlling the mobile robot to move according to the movement strategy includes:

Determining a second current position relationship between the target workpiece and the image acquisition equipment based on the image of the target workpiece and the equipment parameters which are currently acquired by the image acquisition equipment in the process of moving the mobile robot according to the movement strategy;

calculating the current motion state of the executing mechanism based on the second current position relation and the current position relation between the image acquisition equipment and the executing mechanism;

If the difference between the current motion state of the executing mechanism and the target motion state meets a first preset precision condition, controlling the mobile robot to continue to move according to the motion strategy, wherein the target motion state is a motion state corresponding to the executing mechanism included in the motion strategy when the executing mechanism is at the current position;

And if the difference between the current motion state and the target motion state does not meet the first preset precision condition, controlling the mobile robot to stop moving, and reporting abnormal information.

Optionally, after the step of controlling the actuator to perform the preset action on the target workpiece, the method further includes:

Determining a third current position relationship between the target workpiece and the image acquisition equipment based on the image of the target workpiece and the equipment parameters, which are currently acquired by the image acquisition equipment;

And determining the precision of the execution mechanism for executing the preset action on the target workpiece according to the relation between the difference of the third position relation and the first calibration position relation and a second preset precision condition.

Optionally, the mobile robot further comprises a sensor for detecting a motor parameter of the mobile robot;

The step of controlling the mobile robot to move according to the motion strategy comprises the following steps:

Determining the current motion state of an actuating mechanism of the mobile robot based on the motor parameters of the mobile robot acquired by the sensor in the process of moving the mobile robot according to the motion strategy; if the difference between the current motion state of the executing mechanism and the target motion state meets a first preset precision condition, controlling the mobile robot to continue to move according to the motion strategy, wherein the target motion state is a motion state corresponding to the executing mechanism included in the motion strategy when the executing mechanism is at the current position; if the difference between the current motion state and the target motion state does not meet the first preset precision condition, controlling the mobile robot to stop moving, and reporting abnormal information; and/or

After the step of controlling the actuator to perform the preset action on the target workpiece, the method further includes:

determining a third current position relationship between the target workpiece and the image acquisition equipment based on the motor parameters of the mobile robot acquired by the sensor; and determining the precision of the execution mechanism for executing the preset action on the target workpiece according to the relation between the difference of the third position relation and the first calibration position relation and a second preset precision condition.

In a second aspect, an embodiment of the present application provides a motion control apparatus for an actuator, where the actuator is disposed on a mobile robot, and the mobile robot further includes an image capturing device, and the apparatus includes:

the image acquisition module is used for acquiring a target image of the target workpiece acquired by the image acquisition equipment under the condition that the mobile robot moves to an execution area;

the position relation determining module is used for determining a first current position relation between the target workpiece and the image acquisition equipment based on the target image and the equipment parameters of the image acquisition equipment calibrated in advance;

The motion strategy determining module is used for determining a motion strategy of the executing mechanism based on the first current position relationship, a predetermined calibration position relationship and a position relationship between the image acquisition equipment and the executing mechanism, wherein the calibration position relationship comprises a first calibration position relationship between the calibration tool and the image acquisition equipment when the executing mechanism calibrated in advance executes a preset action on a calibration tool corresponding to the target workpiece;

And the motion control module is used for controlling the mobile robot to move according to the motion strategy and controlling the executing mechanism to execute the preset action on the target workpiece when the mobile robot reaches a motion end point.

In a third aspect, an embodiment of the present application provides a mobile robot including an image acquisition device and a processor;

The image acquisition equipment is used for acquiring images;

the processor is configured to perform the method steps described in the first aspect.

In a fourth aspect, embodiments of the present application provide a computer-readable storage medium having stored therein a computer program which, when executed by a processor, implements the method steps of the first aspect described above.

In a fifth aspect, embodiments of the present application also provide a computer program product comprising instructions which, when run on a computer, cause the computer to perform the method steps of the first aspect described above.

The embodiment of the application has the beneficial effects that:

In the scheme provided by the embodiment of the application, the electronic equipment can acquire the target image of the target workpiece acquired by the image acquisition equipment under the condition that the mobile robot moves to the execution area; determining a first current position relationship between a target workpiece and image acquisition equipment based on a target image and equipment parameters of the image acquisition equipment calibrated in advance; determining a motion strategy of an execution mechanism based on a first current position relationship, a predetermined calibration position relationship and a position relationship between image acquisition equipment and the execution mechanism, wherein the calibration position relationship comprises a first calibration position relationship between the calibration tool and the image acquisition equipment when the calibration tool corresponding to a target workpiece is executed with a preset action by the predetermined calibration execution mechanism; and further controlling the mobile robot to move according to the movement strategy, and controlling the executing mechanism to execute preset actions on the target workpiece when the mobile robot reaches the movement end point.

The calibration position relation between the image acquisition equipment and the calibration tool is the position relation between the image acquisition equipment and the target workpiece under the condition that the execution mechanism executes the preset action on the target workpiece. According to the calibration position relation and the first current position relation between the image acquisition equipment and the target workpiece, the motion parameters required to be changed for the image acquisition equipment to move from the current motion state to the state corresponding to the image acquisition equipment corresponding to the first calibration position relation can be determined. Based on the position relation between the image acquisition equipment and the executing mechanism and the motion parameters required to be changed by the image acquisition equipment, the motion parameters required by the state of the executing mechanism corresponding to the first calibration position relation when the executing mechanism moves from the current state can be determined. And the motion parameters are used for controlling the motion of the actuating mechanism, so that the actuating mechanism can accurately move from the current state to the state of the actuating mechanism corresponding to the first calibration position relation, the motion precision of the actuating mechanism is improved, and the aim of 'reaching hands' is really achieved. And because the actuating mechanism moves with higher movement precision, when the actuating mechanism finishes the preset action, the position relation between the image acquisition equipment and the target workpiece meets the first calibration position relation, and the actuating mechanism can accurately perform the operation on the target workpiece, so that the operation precision of the mobile robot is improved. Of course, it is not necessary for any one product or method of practicing the application to achieve all of the advantages set forth above at the same time.

Drawings

In order to more clearly illustrate the embodiments of the application or the technical solutions in the prior art, the drawings used in the embodiments or the description of the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the application, and other embodiments may be obtained according to these drawings to those skilled in the art.

FIG. 1 is a flow chart of a method for controlling movement of an actuator according to an embodiment of the present application;

Fig. 2 is a schematic diagram of a mobile robot according to an embodiment of the present application;

FIG. 3 (a) is a flowchart of step S103 in the embodiment shown in FIG. 1;

FIG. 3 (b) is a schematic diagram of the first current position relationship and the calibration position relationship;

FIG. 4 is a schematic illustration of the positional relationship between the image capture device and the actuator;

FIG. 5 is another schematic illustration of the positional relationship between the image capture device and the actuator;

FIG. 6 is a flowchart of a calibration position relationship determination process according to an embodiment of the present application;

FIG. 7 is a flowchart of step S104 in the embodiment shown in FIG. 1;

FIG. 8 is a flowchart of an example of actuator control provided in an embodiment of the present application;

FIG. 9 is a schematic diagram of a motion control device of an actuator according to an embodiment of the present application;

fig. 10 is a schematic structural diagram of a mobile robot according to an embodiment of the present application.

Detailed Description

The following description of the embodiments of the present application will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present application, but not all embodiments. Based on the embodiments of the present application, all other embodiments obtained by the person skilled in the art based on the present application are included in the scope of protection of the present application.

In order to improve the motion accuracy of an actuator of a mobile robot, embodiments of the present application provide a motion control method and apparatus for an actuator, a mobile robot, a computer readable storage medium, and a computer program product, and the motion control method for an actuator provided by the embodiments of the present application is first described below.

The motion control method of the execution mechanism provided by the embodiment of the application can be applied to any electronic equipment which can control the execution mechanism of the mobile robot to move and has an image processing function, such as a processor, a controller and the like which are mounted on the mobile robot, can be communicated with the mobile robot and has the image processing function; a server or the like which can communicate with the mobile robot and has an image processing function is not particularly limited herein. For clarity of description, hereinafter, referred to as an electronic device.

The mobile robot may be equipped with an image capturing device for capturing images during the movement of the actuator of the mobile robot, where the image capturing device equipped with the mobile robot may be various types of image capturing devices, such as a code reading camera, a plane camera, and a three-dimensional camera, and is not particularly limited herein.

The actuator of the mobile robot may be a horizontal transplanting type actuator, for example: a cantilever; may be a lift-type actuator, for example: a manipulator; but also lifting type actuators such as: the clip arms and the like are not particularly limited herein.

As shown in fig. 1, a motion control method of an actuator, where the actuator is disposed on a mobile robot, and the mobile robot further includes an image acquisition device, the method includes:

s101: under the condition that the mobile robot moves to an execution area, acquiring a target image of a target workpiece acquired by the image acquisition equipment;

S102: determining a first current position relationship between the target workpiece and the image acquisition equipment based on the target image and the equipment parameters of the image acquisition equipment calibrated in advance;

S103: determining a motion strategy of the execution mechanism based on the first current position relationship, a predetermined calibration position relationship and a position relationship between the image acquisition equipment and the execution mechanism, wherein the calibration position relationship comprises a first calibration position relationship between the calibration tool and the image acquisition equipment in a state that the execution mechanism calibrated in advance executes a preset action on a calibration tool corresponding to the target workpiece;

s104: and controlling the mobile robot to move according to the movement strategy, and controlling the executing mechanism to execute the preset action on the target workpiece when the mobile robot reaches a movement end point.

The calibration position relation between the image acquisition equipment and the calibration tool is the position relation between the image acquisition equipment and the target workpiece under the condition that the execution mechanism executes the preset action on the target workpiece. According to the calibration position relation and the first current position relation between the image acquisition equipment and the target workpiece, the motion parameters required to be changed for the image acquisition equipment to move from the current motion state to the state corresponding to the image acquisition equipment corresponding to the first calibration position relation can be determined. Based on the position relation between the image acquisition equipment and the executing mechanism and the motion parameters required to be changed by the image acquisition equipment, the motion parameters required by the state of the executing mechanism corresponding to the first calibration position relation when the executing mechanism moves from the current state can be determined. And the motion parameters are used for controlling the motion of the actuating mechanism, so that the actuating mechanism can accurately move from the current state to the state of the actuating mechanism corresponding to the first calibration position relation, the motion precision of the actuating mechanism is improved, and the aim of 'reaching hands' is really achieved. And because the actuating mechanism moves with higher movement precision, when the actuating mechanism finishes the preset action, the position relation between the image acquisition equipment and the target workpiece meets the first calibration position relation, and the actuating mechanism can accurately perform the operation on the target workpiece, so that the operation precision of the mobile robot is improved.

When the mobile robot performs a production task, the accuracy of the operation of the mobile robot can be improved by improving the accuracy of the motion of the actuator of the mobile robot. In order to improve the motion precision of the executing mechanism, the executing mechanism can be controlled to move according to the position relation between the image acquisition equipment and the target workpiece. Based on this, in the above-described step S101, in the case where the mobile robot moves to the execution area, the electronic apparatus may acquire the target image of the target workpiece to be processed in the current production link acquired by the image acquisition apparatus.

Before the mobile robot performs the production task, the mobile robot may be first controlled to move to an execution area corresponding to the current production link. In the case where the mobile robot moves to the execution area, the image pickup device provided to the mobile robot can pick up the target image of the target workpiece placed in the execution area, so that the electronic device can pick up the target image picked up by the image pickup device. Wherein the target image is related to the type of image capturing device, e.g. when the image capturing device is a planar camera, the target image may be a two-dimensional image; where the image capturing device is a three-dimensional camera, the target image may be a three-dimensional image, as is reasonable, and the type of image capturing device and the type of target image are not specifically limited herein. The image acquisition equipment can be various types of image acquisition equipment, so that the scheme has high universal value in industrial production.

Further, in the step S102, the electronic device may determine the first current positional relationship between the target workpiece and the image capturing device based on the target image and the device parameters of the image capturing device calibrated in advance.

For every mobile robot, in order to reduce the influence of equipment structural errors and equipment installation errors of the image acquisition equipment on the motion precision of an actuating mechanism, the machining precision of the image acquisition equipment can be improved in a structural design stage of the image acquisition equipment in a finishing mode and the like, so that the equipment structural errors of the image acquisition equipment are reduced, the image acquisition sensing precision of the image acquisition equipment is improved, and in an installation stage of the image acquisition equipment, the equipment installation errors of the image acquisition equipment are reduced by improving the installation precision of the image acquisition equipment.

After the image acquisition device is arranged on the mobile robot, the device parameters of the image acquisition device can be calibrated, wherein the device parameters of the image acquisition device can comprise internal parameters and external parameters; the internal parameters are parameters for describing the internal properties of the image acquisition equipment, and comprise focal length, principal point (optical center) coordinates, distortion coefficients and the like; the external parameters are parameters describing the position and the posture of the image acquisition device in a world coordinate system, and comprise a rotation matrix, a translation vector and the like.

Thus, after the electronic device acquires the target image of the target workpiece, the first current position relationship between the target workpiece and the image acquisition device in the preset coordinate system can be determined based on the target image and the device parameters of the image acquisition device calibrated in advance. The preset coordinate system may be a robot coordinate system or a world coordinate system, which is not specifically limited herein. The first current positional relationship between the target workpiece and the image acquisition device may include data that may represent the positional relationship, such as an angle, a distance, a horizontal offset, and a vertical offset, which are not specifically limited herein.

As an embodiment, a correspondence between a pixel point in a target image corresponding to a device parameter of the image capturing device and a position relationship may be established in advance, so that after the target image of the target workpiece is acquired, the electronic device may determine, based on the target pixel point in the target image and the correspondence, a position relationship between the target pixel point and the image capturing device as a first current position relationship between the target workpiece and the image capturing device. The target pixel point may be a pixel point corresponding to an actual feature of the target workpiece, or may be a pixel point corresponding to a geometric center of the target workpiece, which is not specifically limited herein.

After determining the first current position relation between the target workpiece and the image acquisition device, the electronic device can perform motion control on the execution mechanism of the mobile robot according to the first current position relation. Further, in the step S103, the electronic device may determine a motion policy of the actuator based on the first current position relationship, the predetermined calibration position relationship, and the position relationship between the image capturing device and the actuator. The calibration position relation comprises a first calibration position relation between the calibration tool and the image acquisition equipment in a state that the calibration tool corresponding to the target workpiece is subjected to preset actions by the pre-calibrated execution mechanism.

Because there is an unavoidable installation error of the device when the image acquisition device is arranged on the mobile robot, the actual positional relationship between the workpiece and the image acquisition device is deviated from the ideal positional relationship in a state that the execution mechanism executes a preset action on the workpiece. Moreover, because of the difference of image acquisition sensing precision between different image acquisition devices, even under the same image acquisition condition, the position relationship between the image acquisition devices and the workpiece, which are reflected by the image acquisition devices for acquiring the image of the same workpiece, also has the difference. Therefore, after the image acquisition equipment is arranged on each mobile robot, the first calibration position relation between the calibration tool and the image acquisition equipment can be calibrated by using the calibration tool corresponding to the target workpiece to be processed in the current production link under the state that the execution mechanism executes the preset action on the calibration tool, so as to obtain the calibration position relation. The calibration positional relationship includes a first calibration positional relationship, where the first calibration positional relationship may include data that may represent the positional relationship, such as an angle, a distance, a horizontal offset, and a vertical offset, between the calibration tool and the image capturing device, and the data included in the first current positional relationship corresponds to the data included in the first calibration positional relationship.

The execution mechanism executes a preset action on the calibration tool, namely, the action execution process of the execution mechanism under the standard working condition; the first calibration position relation between the calibration fixture and the image acquisition equipment is the position relation between the calibration fixture and the image acquisition equipment when the execution mechanism finishes the action under the standard working condition.

When the calibration tool is used for calibrating the execution mechanism of the mobile robot to execute the preset action on the calibration tool, the first calibration position relation between the image acquisition equipment and the calibration tool of the mobile robot can be used for calibrating the tolerance degree of the mobile robot to the equipment structure error, the equipment installation error and the action execution error of the execution mechanism of the image acquisition equipment, so that the process of performing motion control on the execution mechanism of the mobile robot by using the corresponding first calibration position relation of the mobile robot is performed based on the tolerance degree of the mobile robot, and the motion precision of the execution mechanism can be improved.

For clarity of the line, a detailed description will be given below of a manner of determining the calibration positional relationship.

Under the condition that the executing mechanism executes the preset action on the target workpiece, the final position relation between the image acquisition equipment and the target workpiece meets the calibration position relation, so after the first current position relation between the image acquisition equipment and the target workpiece is determined, the electronic equipment can determine the motion parameters required to be changed for the image acquisition equipment corresponding to the first calibration position relation from the current state to the state corresponding to the image acquisition equipment corresponding to the first calibration position relation based on the first current position relation and the predetermined calibration position relation.

Because the image acquisition equipment and the executing mechanism are arranged on the mobile robot, a position relationship exists between the image acquisition equipment and the executing mechanism, the electronic equipment can determine the position relationship between the image acquisition equipment and the executing mechanism, and then according to the motion parameters which are required to be changed when the image acquisition equipment moves from the current state to the state corresponding to the image acquisition equipment corresponding to the first calibration position relationship and the position relationship between the image acquisition equipment and the executing mechanism, the motion parameters which are required to be changed when the executing mechanism moves from the current state to the state corresponding to the executing mechanism corresponding to the first calibration position relationship are determined.

The electronic device may then determine a motion strategy for the actuator based on the motion parameters that the actuator needs to change. The motion strategy of the actuator may include an angle of rotation of the actuator to be controlled, a distance of movement of the actuator to be controlled, and an action parameter corresponding to a preset action, corresponding to a first calibration positional relationship between the image acquisition device and the target workpiece.

As shown in fig. 2, the mobile robot is provided with an actuator 210 and an image acquisition device 220, and in the case that the pre-calibrated actuator 210 performs a preset action, a calibration positional relationship between the image acquisition device 220 and the calibration fixture 230 is shown in fig. 5. After determining the first current positional relationship between the image capturing device 220 and the target workpiece, the electronic device may determine, based on the predetermined calibration positional relationship and the positional relationship between the image capturing device 220 and the actuator 210, a motion parameter that needs to be changed when the image capturing device 220 moves from the current state to a state corresponding to the image capturing device 220 corresponding to the calibration positional relationship shown in fig. 5, and determine, based on the positional relationship between the actuator 210 and the image capturing device 220 shown in fig. 2 and the motion parameter that needs to be changed when the image capturing device 220 moves from the current state to a state corresponding to the actuator 210 corresponding to the first calibration positional relationship shown in fig. 5, and determine, based on the motion parameter, a motion policy of the actuator 210, where the motion policy includes controlling the actuator 210 to move and controlling the actuator to perform a preset motion.

After determining the motion strategy of the actuator, the electronic device may perform motion control on the actuator according to the control strategy. Further, in the step S104, the electronic device may control the mobile robot to move according to the movement strategy, and when the mobile robot reaches the movement end point, control the actuator to perform the preset action on the target workpiece.

Because the executing mechanism is arranged on the mobile robot, when the executing mechanism is static relative to the mobile robot, the moving robot moves to drive the executing mechanism to move, and the moving parameters of the executing mechanism and the moving robot are the same. Based on the motion strategy, after the motion strategy of the executing mechanism is determined, the electronic equipment can control the mobile robot to move according to the motion strategy, so that the executing mechanism can move according to the motion strategy under the driving of the mobile robot, and when the mobile robot reaches the motion end point, the executing mechanism can reach the executing position of the preset action. Then, the electronic device may control the executing mechanism to execute a preset action on the target workpiece, so that the executing mechanism performs a job on the target workpiece. Thus, under the condition that the executing mechanism executes the preset action on the target workpiece, the position relationship between the target workpiece and the image acquisition equipment meets the first calibration position relationship, so that the high-precision motion control of the executing mechanism is completed according to the motion strategy.

As an implementation manner, the electronic device may control the executing mechanism to execute the preset action on the target workpiece in the process of controlling the movement of the mobile robot according to the movement strategy.

As shown in fig. 3 (a), the step S103, that is, the step of determining the motion policy of the actuator based on the first current position relationship, the predetermined calibration position relationship, and the position relationship between the image capturing device and the actuator, may include:

s301: and determining the motion parameters corresponding to the image acquisition equipment according to the difference between the first current position relation and the predetermined calibration position relation.

Because the first calibration position relation between the image acquisition equipment and the calibration tool is calibrated in advance under the condition that the execution mechanism executes the preset action on the calibration tool, and the position relation between the image acquisition equipment and the target workpiece should meet the first calibration position relation under the condition that the execution mechanism executes the preset action on the target workpiece, after the first current position relation between the image acquisition equipment and the target workpiece is determined, the electronic equipment can determine the motion parameter which needs to be changed when the image acquisition equipment moves from the current motion state to the state corresponding to the image acquisition equipment corresponding to the first calibration position relation according to the difference between the first current position relation and the predetermined first calibration position relation, so as to obtain the motion parameter corresponding to the image acquisition equipment. After the motion of the image acquisition equipment is changed from the current state to the motion parameters, the image acquisition equipment is in a state corresponding to the image acquisition equipment corresponding to the first calibration position relation.

S302: and determining a motion strategy of the executing mechanism based on the motion parameters corresponding to the image acquisition equipment and the position relation between the image acquisition equipment and the executing mechanism.

Because the motion parameters corresponding to the image acquisition equipment characterize the motion parameters required to be changed when the image acquisition equipment moves from the current motion state to the state corresponding to the image acquisition equipment corresponding to the first calibration position relation, and for the image acquisition equipment and the executing mechanism which are arranged on the same mobile robot, the position relation exists between the image acquisition equipment and the executing mechanism, then the electronic equipment can determine the motion parameters required to be changed when the executing mechanism moves from the current motion state to the state corresponding to the executing mechanism corresponding to the first calibration position relation based on the motion parameters corresponding to the image acquisition equipment and the position relation between the image acquisition equipment and the executing mechanism, and determine the motion strategy of the executing mechanism based on the motion parameters required to be changed by the executing mechanism.

For example, as shown in fig. 3 (b), in a state that the calibration tool corresponding to the target workpiece is pre-calibrated by the executing mechanism, the calibration position relationship between the calibration tool and the image acquisition device is calibrated, and at this time, the position where the calibration tool is located in the calibration image 32 acquired by the image acquisition device is the first position 320. In the actual process of workpiece capturing and the like, according to the target image currently acquired by the image acquisition device, the first current position relationship between the image acquisition device and the target workpiece can be determined, as shown in fig. 3 (b), and the position of the target workpiece in the target image 31 is the second position 310. And further, according to the difference between the first current position relation and the calibration position relation, the motion parameter of the image acquisition equipment, which is required to be changed when the current motion state corresponding to the first current position relation moves to the state corresponding to the calibration position relation, can be determined.

According to the motion parameters corresponding to the image acquisition equipment and the position relation between the image acquisition equipment and the executing mechanism, the motion parameters required to be changed when the executing mechanism moves from the current motion state corresponding to the first current position relation to the state corresponding to the calibration position relation can be determined, and the motion strategy of the executing mechanism is determined according to the parameters required to move by the executing mechanism. Thus, after the control actuator moves according to the motion strategy, the position of the target workpiece in the image shot by the image acquisition device is the same as the position of the marking tool in the marking image 32, which is the first position 320.

It can be seen that, in this embodiment, the electronic device may first determine, according to a first current position relationship between the image capturing device and the target workpiece and a calibrated position relationship between the image capturing device and the target workpiece when the pre-calibrated executing mechanism executes the preset action, a motion parameter that needs to be changed when the image capturing device moves from the current state to a state where the image capturing device corresponding to the first calibrated position relationship is located, and then determine, according to a position relationship between the image capturing device and the executing mechanism and a motion parameter that needs to be changed when the image capturing device needs to be changed, a motion parameter that needs to be changed when the executing mechanism moves from the current state to a state where the executing mechanism corresponding to the first calibrated position relationship is located, and determine, based on the motion parameter, a motion policy of the executing mechanism. Therefore, the motion strategy is utilized to control the motion of the executing mechanism, and the executing mechanism can be controlled to move from the current state to the state corresponding to the executing mechanism corresponding to the first calibration position relation more accurately, so that the motion precision of the executing mechanism is improved.

Generally, according to the installation position of the image capturing apparatus and the category of the actuator, there are two cases of the positional relationship between the image capturing apparatus and the actuator:

In one case, the image acquisition device is fixedly arranged on the executing mechanism, and the position relationship between the image acquisition device and the executing mechanism is unchanged. Or the image acquisition equipment is not arranged on the executing mechanism, but the position relation between the image acquisition equipment and the mobile robot is unchanged, and the position relation between the executing mechanism and the mobile robot is unchanged, so that the position relation between the image acquisition equipment and the executing mechanism is also unchanged.

For example, as shown in fig. 4, the actuator is a cantilever 410, and the image capturing device 420 is mounted on the cantilever 410, and when the cantilever 410 moves, the image capturing device 420 moves along with the movement of the cantilever 410, and the positional relationship between the two is unchanged.

In another case, the image capturing device is not provided on the actuator, the positional relationship between the image capturing device and the mobile robot is unchanged, and the positional relationship between the actuator and the mobile robot is variable, so the positional relationship between the image capturing device and the actuator is variable.

For example, as shown in fig. 5, the actuator is a gripper arm 210, and the image pickup device 220 is mounted on the mobile robot and not mounted on the gripper arm 210. As the gripper arm 210 moves, the positional relationship between the gripper arm 210 and the mobile robot changes, the positional relationship between the image capturing device 220 and the mobile robot does not change, and the positional relationship between the image capturing device 220 and the gripper arm 210 changes.

The above two situations may exist corresponding to the positional relationship between the image capturing device and the executing mechanism, and as one implementation manner of the embodiment of the present application, the step S302 described above, that is, the step of determining the motion policy of the executing mechanism based on the motion parameters corresponding to the image capturing device and the positional relationship between the image capturing device and the executing mechanism, also exists two implementation manners:

In one implementation manner, if the position relationship between the image acquisition device and the executing mechanism is unchanged, the motion parameter corresponding to the image acquisition device is used as the motion parameter of the executing mechanism, and the motion strategy of the executing mechanism is obtained.

Under the condition that the position relation between the image acquisition equipment and the executing mechanism is unchanged, the executing mechanism moves to drive the image acquisition equipment to move simultaneously with the image acquisition equipment, and the moving parameters such as the moving position and the moving angle are the same, namely, the movement of the image acquisition equipment represents the movement of the executing mechanism.

The motion parameters of the image acquisition equipment characterize the motion parameters which need to be changed when the image acquisition equipment wants to move from the current state to the state corresponding to the image acquisition equipment corresponding to the first calibration position relation, and the motion parameters such as the motion position and the motion angle of the execution mechanism and the image acquisition equipment are the same, so that the execution mechanism can drive the image acquisition equipment to move to the state of the image acquisition equipment corresponding to the first calibration position relation according to the motion parameters of the image acquisition equipment, and the execution mechanism can also simultaneously move to the state of the execution mechanism corresponding to the first calibration position relation, namely, the state of the preset action can be accurately executed on the target workpiece.

Based on the motion parameters, when the motion parameters corresponding to the image acquisition equipment are determined, the electronic equipment can take the motion parameters corresponding to the image acquisition equipment as the motion parameters of the execution mechanism, and determine the motion strategy of the execution mechanism based on the motion parameters of the execution mechanism, so that the motion strategy of the execution mechanism is obtained.

For example, the actuator is a cantilever 410, and the positional relationship between the cantilever 410 and the image capturing device 420 is shown in fig. 4. The calibration positional relationship between the image acquisition device 420 and the calibration fixture 430 may be obtained by calibrating in advance the calibration positional relationship between the image acquisition device 420 and the calibration fixture 430 in a state where the cantilever 410 performs a preset action on the calibration fixture 430 corresponding to the target workpiece, as shown in fig. 4. After determining the motion parameters corresponding to the image capturing device 420 based on the first current position relationship and the calibration position relationship between the image capturing device 420 and the target workpiece, the electronic device may use the motion parameters corresponding to the image capturing device 420 as the motion parameters of the cantilever 410, and determine the motion strategy of the cantilever 410 based on the motion parameters required to be changed by the cantilever 410.

After the motion parameters of the motion required by the execution mechanism are determined, the motion strategy of the execution mechanism can be planned according to the motion parameters, so that the motion parameters of the execution mechanism reach the motion parameters when the execution mechanism completes the motion according to the motion strategy.

In another implementation manner, if the position relationship between the image acquisition device and the executing mechanism is variable, and the calibration position relationship further includes a second calibration position relationship between the executing mechanism and the image acquisition device in a state that the executing mechanism calibrated in advance executes the preset action on the calibration tool corresponding to the target workpiece, the motion strategy of the executing mechanism is determined based on the motion parameter corresponding to the image acquisition device, the second position relationship, and the current position relationship between the image acquisition device and the executing mechanism.

Under the condition that the position relation between the image acquisition equipment and the executing mechanism is variable, when the calibration is performed in advance, the second calibration position relation between the executing mechanism and the image acquisition equipment can be also performed in a state that the executing mechanism performs a preset action on the calibration tool corresponding to the target workpiece in advance, namely the calibration position relation also comprises the second calibration position relation between the executing mechanism and the image acquisition equipment in a state that the executing mechanism performs the preset action on the calibration tool corresponding to the target workpiece in advance.

Because the position relation between the image acquisition equipment and the executing mechanism is variable, and the position relation between the image acquisition equipment and the executing mechanism should meet the second calibration position relation when the executing mechanism executes the preset action on the target workpiece, the position relation between the image acquisition equipment and the executing mechanism also needs to be considered when the motion strategy of the executing mechanism is determined according to the motion parameters of the image acquisition equipment.

Based on the above, after determining the motion parameters corresponding to the image acquisition device, the electronic device may determine the motion policy of the execution mechanism according to the motion parameters corresponding to the image acquisition device, the second calibration positional relationship between the execution mechanism and the image acquisition device, and the current positional relationship between the image acquisition device and the execution mechanism.

The electronic device may first calculate a second calibration positional relationship between the actuator and the image capturing device, and a difference between the second calibration positional relationship and a current positional relationship between the image capturing device and the actuator, where the difference characterizes a motion parameter corresponding to a required execution action of the actuator, where the positional relationship between the image capturing device and the actuator is intended to satisfy the second calibration positional relationship.

And then, according to the motion parameters corresponding to the image acquisition equipment and the differences, the electronic equipment can calculate the motion parameters of the execution mechanism. Because the motion parameters corresponding to the image acquisition equipment represent the motion parameters required to be changed when the image acquisition equipment wants to move from the current state to the state of the image acquisition equipment corresponding to the first calibration position relation, the difference and the motion parameters represent the motion parameters required by the state of the execution mechanism corresponding to the execution mechanism moving from the current state to the first calibration position relation, so that the motion parameters of the execution mechanism can be determined according to the motion parameters corresponding to the image acquisition equipment and the difference. Furthermore, the electronic equipment can determine the motion strategy of the executing mechanism according to the motion parameters of the executing mechanism.

For example, the executing mechanism is a clamping arm 210, the current position relationship between the clamping arm 210 and the image capturing device 220 is shown in fig. 2, and the position relationship between the clamping arm 210 and the image capturing device 220 in the state that the clamping arm 210 performs the preset action on the calibration tool 230 corresponding to the target workpiece may be calibrated in advance, so as to obtain the second calibration position relationship between the clamping arm 210 and the image capturing device 220 shown in fig. 5. After determining the motion parameters corresponding to the image capturing device 220, the electronic device may determine the current position relationship between the arm rest 210 and the image capturing device 220, and determine the motion strategy of the arm rest 210 according to the motion parameters corresponding to the image capturing device 220, the second calibration position relationship, and the current position relationship between the image capturing device 220 and the arm rest 210.

If the positional relationship between the image capture device and the actuator is variable, the change in the positional relationship between the actuator and the image capture device may be caused by movement of the actuator and/or the actuator performing other tasks. In this case, in order to accurately determine the motion parameter of the actuator, the above calibration positional relationship may further include a third calibration positional relationship between the image capturing device and the actuator in the case where the actuator moves to a preset execution position corresponding to the preset action.

As an implementation manner, if the position relationship between the image capturing device and the executing mechanism is variable, after determining the motion parameter corresponding to the image capturing device, the electronic device may determine the motion policy of the executing mechanism based on the motion parameter corresponding to the image capturing device, the pre-calibrated third calibration position relationship, the pre-calibrated preset motion parameter, and the current position relationship between the image capturing device and the executing mechanism.

The motion parameters required by the state of the execution mechanism corresponding to the first calibration position relation from the current state comprise the motion parameters of the execution mechanism from the current state to the preset execution position corresponding to the preset action and the preset action parameters corresponding to the preset action, based on the motion parameters corresponding to the image acquisition equipment, the pre-calibrated third calibration position relation and the current position relation between the image acquisition equipment and the execution mechanism, the motion parameters required by the state of the execution mechanism from the current state to the state of the execution mechanism corresponding to the third calibration position relation can be determined by the electronic equipment, and the motion strategy of the execution mechanism can be determined based on the motion parameters and the preset motion parameters.

It can be seen that, in this embodiment, when determining the motion policy of the actuator based on the motion parameters corresponding to the image capturing device, it is also required to consider whether the positional relationship between the image capturing device and the actuator can be changed. When the position relation between the image acquisition equipment and the executing mechanism is unchanged, the electronic equipment can directly determine the motion strategy of the executing mechanism according to the motion parameters corresponding to the image acquisition equipment; when the position relation between the image acquisition equipment and the executing mechanism is variable, the electronic equipment also needs to consider the difference between the current position relation between the executing mechanism and the image acquisition equipment and the second calibration position relation between the pre-calibrated executing mechanism and the image acquisition equipment when determining the motion strategy of the executing mechanism, namely, the electronic equipment can determine the motion strategy of the executing mechanism according to the motion strategy of the image acquisition equipment, the current position relation between the executing mechanism and the image acquisition equipment and the second calibration position relation between the pre-calibrated executing mechanism and the image acquisition equipment. Therefore, under the condition that whether the position relation between the image acquisition equipment and the executing mechanism can be changed or not is considered, the motion strategy of the executing mechanism determined by the electronic equipment is more accurate, and the executing mechanism is subjected to motion control according to the motion strategy, so that the motion precision of the executing mechanism can be improved.

In the actual production process, in order to facilitate the execution of the operation on each workpiece by using the image acquisition device, the actual characteristics of each workpiece may be determined, for example, feature points are marked on each workpiece as actual characteristics possessed by the workpiece, preset textures on each workpiece are used as actual characteristics, and the like; corresponding to the actual features that the target workpiece has, target features may be set on the calibration tooling corresponding to the target tool, for example, as shown in fig. 5, target features 240 may be set on the calibration tooling 230. In this way, the target characteristics set on the calibration tool can be utilized, and the calibration executing mechanism can calibrate the calibration position relationship between the calibration tool and the image acquisition equipment under the condition that the calibration executing mechanism executes the preset action on the calibration tool.

As one implementation manner of the embodiment of the present application, the calibration fixture has a target feature, where the target feature is set in advance based on an actual feature of the target workpiece;

as shown in fig. 6, before the step S101, that is, the step of acquiring the target image of the target workpiece acquired by the image acquisition device, the method further includes:

s601: controlling an actuating mechanism of the mobile robot to execute a preset action on a calibration tool corresponding to the target workpiece;

s602: acquiring a calibration image of the calibration tool currently acquired by the image acquisition equipment;

S603: and determining the position relation between the calibration tool and the image acquisition equipment based on the position of the target feature in the calibration image and the equipment parameter of the image acquisition equipment calibrated in advance, and taking the position relation as a calibration position relation.

When the calibration tool is utilized to calibrate the calibration position relationship between the calibration tool and the image acquisition equipment under the condition that the execution mechanism executes the preset action on the calibration tool, the electronic equipment can firstly control the mobile robot to move to the preset action execution position and control the execution mechanism to execute the preset action on the calibration tool corresponding to the target workpiece. And then, acquiring a calibration image of the calibration tool currently acquired by the image acquisition equipment.

Then, the electronic device can determine the position of the target feature on the calibration tool in the calibration image, and determine the position relationship between the target feature and the image acquisition device based on the position of the target feature and the device parameter of the image acquisition device calibrated in advance, so as to obtain the calibration position relationship between the target feature and the image acquisition device.

It can be seen that, in this embodiment, the target feature may be set on the calibration tool according to the actual feature of the target workpiece, and further, the positional relationship between the image capturing device and the calibration tool when the execution mechanism performs the preset action on the calibration tool corresponding to the target workpiece is converted into the calibration positional relationship between the image capturing device and the target feature of the calibration tool when the execution mechanism performs the preset action on the calibration tool corresponding to the target workpiece, thereby improving the accuracy of the determined calibration positional relationship. For each mobile robot, the calibration process can calibrate the corresponding calibration position relation of the image acquisition equipment of the mobile robot, so that the tolerance degree of the mobile robot on the structure machining assembly error and the delivery consistency of the execution mechanism of each mobile robot in the mass production process are improved.

Accordingly, as an implementation manner of the embodiment of the present application, the step S103, that is, the step of determining the first current positional relationship between the target workpiece and the image capturing device based on the target image and the device parameter of the image capturing device calibrated in advance, may include:

Because the calibration position relation between the image acquisition device and the target feature of the calibration tool, which is set based on the actual feature of the target workpiece, is calibrated in advance under the condition that the execution mechanism executes the preset action on the calibration tool corresponding to the target workpiece, in order to perform high-precision motion control on the execution mechanism based on the calibration position relation in the actual production process, after the target image is acquired, the electronic device can determine the position of the actual feature of the target workpiece in the target image, and then determine the current position relation between the actual feature of the target workpiece and the image acquisition device based on the position of the actual feature of the target workpiece included in the target image and the device parameter of the image acquisition device calibrated in advance, as the first current position relation between the target workpiece and the image acquisition device.

Therefore, in this embodiment, because the calibration position relationship between the image acquisition device and the target feature of the calibration tool corresponding to the target workpiece is calibrated in advance under the condition that the calibration tool corresponding to the target workpiece is executed by the execution mechanism, in an actual operation process, the electronic device can determine the first current position relationship between the actual feature of the target workpiece and the image acquisition device, so that the calibration position relationship and the first current position relationship are the position relationship between the image acquisition device and the same feature of the workpiece, and therefore, the relevance of the calibration position relationship and the first current position relationship is enhanced, and the motion strategy of the execution mechanism is determined based on the difference between the first current position relationship and the calibration position relationship, so that the accuracy of the motion strategy is improved, and the motion accuracy of the execution mechanism according to the motion strategy is further improved.

As shown in fig. 7, the step S104, that is, the step of controlling the mobile robot to move according to the movement strategy, may include:

S701: and in the process of moving the mobile robot according to the movement strategy, determining a second current position relation between the target workpiece and the image acquisition equipment based on the image of the target workpiece currently acquired by the image acquisition equipment and the equipment parameters.

In the process of controlling the mobile robot to move according to the movement strategy, in order to keep the executing mechanism to move with higher movement precision, the electronic equipment can verify the current movement state of the executing mechanism based on the current position relation between the image acquisition equipment and the target workpiece in the process of the mobile robot. The electronic equipment can acquire the image of the target workpiece currently acquired by the image acquisition equipment, and determine a second current position relationship between the target workpiece and the image acquisition equipment based on the image of the target workpiece currently acquired by the image acquisition equipment and equipment parameters of the image acquisition equipment.

S702: and calculating the current motion state of the executing mechanism based on the second current position relation and the current position relation between the image acquisition equipment and the executing mechanism.

After determining the second current positional relationship between the target workpiece and the image capturing device, the electronic device may determine a current positional relationship between the image capturing device and the actuator, and determine a current motion state of the actuator based on the second current positional relationship and the current positional relationship between the image capturing device and the actuator.

When the position relation between the image acquisition equipment and the executing mechanism is unchanged, the executing mechanism moves to drive the image acquisition equipment to move simultaneously with the image acquisition equipment, and the moving parameters such as the moving position, the moving angle and the like are the same, namely, the movement of the image acquisition equipment represents the movement of the executing mechanism, the electronic equipment can determine the current moving state of the image acquisition equipment based on the second current position relation, and determine the current moving state of the executing mechanism based on the current moving state of the image acquisition equipment and the position relation between the image acquisition equipment and the executing mechanism, wherein the current moving state comprises the position and the gesture of the executing mechanism.

When the positional relationship between the image capturing device and the executing mechanism is variable, the electronic device may determine a current positional relationship between the image capturing device and the executing mechanism, and then calculate a current motion state of the executing mechanism based on the second current positional relationship and the current positional relationship between the image capturing device and the executing mechanism. As an implementation manner, the electronic device may acquire an image of the mobile robot currently acquired by a preset image acquisition device that performs image acquisition on the mobile robot, calculate a current position relationship between the image acquisition device and the execution mechanism based on the image, and then determine a current motion state of the execution mechanism based on the second current position relationship and the current position relationship between the image acquisition device and the execution mechanism.

S703: and if the difference between the current motion state of the executing mechanism and the target motion state meets a first preset precision condition, controlling the mobile robot to continue to move according to the motion strategy. The target motion state is a motion state corresponding to the motion strategy when the executing mechanism is located at the current position.

S704: and if the difference between the current motion state and the target motion state does not meet the first preset precision condition, controlling the mobile robot to stop moving, and reporting abnormal information.

For evaluating the movement accuracy of the actuator, a first preset accuracy condition may be preset, wherein the first preset accuracy condition may be set based on the device structure error and the device installation error, and may include a position deviation, an attitude deviation, and the like, corresponding to the movement state, for example, the first preset accuracy condition may be that the position deviation is not greater than 1 cm, the attitude deviation is not greater than 0.2 degrees, and the like in the world coordinate system, which is not particularly limited herein.

The electronic device can determine a corresponding target motion state when the executing mechanism processes the current position according to a motion strategy of the executing mechanism in the process of controlling the motion of the executing mechanism. Thus, after determining the current motion state of the actuator, a difference between the current motion state of the actuator and the target motion state may be calculated, and it may be determined whether the difference satisfies a first preset accuracy condition.

If the difference meets the first preset precision condition, the motion precision of the actuating mechanism is higher, and the mobile robot can be controlled to continue to move according to the motion strategy.

If the difference between the current motion state and the target motion state does not meet the first preset precision condition, the motion state of the actuating mechanism is abnormal, the moving actuating mechanism is prevented from moving continuously in the abnormal state, the moving robot can be controlled to stop moving, the current motion state and the difference of the actuating mechanism are reported as abnormal information, and therefore when the abnormal information is received by a worker, the abnormal information is recorded and analyzed.

As an implementation manner, after the mobile robot is controlled to stop moving, the electronic device may update the movement strategy of the actuator based on the current movement state of the actuator and the difference between the current movement state and the target movement state, and continue to control the movement of the actuator according to the updated movement strategy of the actuator.

It can be seen that, in this embodiment, in the process of controlling the mobile robot to move according to the movement strategy, the electronic device may determine the second current position relationship between the image capturing device and the target workpiece based on the image of the target workpiece captured by the image capturing device, and calculate the current movement state of the actuator based on the second current position relationship and the current position relationship between the image capturing device and the actuator. And then, based on the target motion state of the actuating mechanism determined according to the motion precision and a first preset precision condition, performing precision verification on the current motion state. When the current motion state meets the first preset precision condition, the motion of the executing mechanism can be controlled continuously, and when the current motion state does not meet the first preset precision condition, the parking adjustment can be carried out on the mobile robot. Therefore, the motion precision of the actuating mechanism in the motion process is checked by utilizing the image acquired by the image acquisition equipment, so that the actuating mechanism can keep higher motion precision in the motion process, and the operation failure caused by the fact that the actuating mechanism continues to move under abnormal motion is avoided.

As an implementation manner of the embodiment of the present application, after step S104, that is, the step of controlling the executing mechanism to execute the preset action on the target workpiece, the method may further include:

After the control executing mechanism executes the preset action on the target workpiece, the electronic device can verify the accuracy of the execution mechanism executing the preset action on the target workpiece based on the position relation between the image acquisition device and the target workpiece and the first calibration position relation between the image acquisition device and the target workpiece.

After the control execution mechanism executes the preset action on the target workpiece, the electronic device can acquire the image of the target workpiece currently acquired by the image acquisition device, and then determine a third current position relationship between the target workpiece and the image acquisition device based on the image and the device parameters of the image acquisition device.

Then, the electronic device may calculate a difference between the third positional relationship and the first calibration positional relationship, and determine whether the difference satisfies a second preset accuracy condition. The second preset accuracy condition may be set based on the equipment machining error and the equipment installation error, and, in correspondence to the positional relationship, the second preset accuracy condition may include an angle deviation, a distance deviation, a horizontal deviation, a vertical deviation, and the like, for example, the second preset accuracy condition may be that the angle deviation is not greater than 0.2 degrees, the distance deviation is not greater than 1 cm, the horizontal deviation is not greater than 0.5 cm, the vertical deviation is not greater than 0.5 cm, and the like in the world coordinate system, which is not particularly limited herein.

When the difference between the calculated third current position relation and the first calibration position relation meets a second preset precision condition, determining that the precision of the preset action of the execution mechanism on the target workpiece is higher; and when the difference between the third current position relation and the first calibration position relation does not meet the second preset precision condition, determining that the precision of the preset action of the execution mechanism on the target workpiece is lower, reporting the third current position relation and the difference as abnormal data so as to facilitate a worker to receive the abnormal data and analyze and record the abnormal data.

As an implementation manner, when the difference between the third current position relationship and the first calibration position relationship does not meet the second preset accuracy condition, the electronic device may determine a position adjustment policy of the actuator based on the difference, and perform position adjustment on the actuator based on the position adjustment policy.

In one embodiment, the electronic device may acquire an image of the mobile robot currently acquired by a preset image acquisition device that performs image acquisition on the mobile robot, calculate a third current positional relationship between the image acquisition device and the executing mechanism based on the image, and then determine, according to a relationship between a difference between the third positional relationship and the first calibration positional relationship and a second preset precision condition, a precision of executing a preset action on the target workpiece by the executing mechanism.

It can be seen that, in this embodiment, after the execution mechanism is controlled to execute the preset action on the target workpiece, the electronic device may further determine a third current positional relationship between the target workpiece and the image capturing device based on the image currently captured by the image capturing device, and determine the accuracy of the execution mechanism to execute the preset action on the target workpiece based on the difference between the third positional relationship and the first calibration positional relationship, and the second preset accuracy condition. Thus, by means of the image acquisition equipment, the accuracy verification when the action is completed can be realized, the influence caused by lower action accuracy of the execution mechanism on the target workpiece for executing the preset action can be avoided, the action accuracy of the execution mechanism is improved, and further the operation accuracy of the mobile robot is improved.

As an implementation of the embodiment of the present application, the mobile robot further includes a sensor for detecting a motor parameter of the mobile robot; the step S104, that is, the step of controlling the mobile robot to move according to the motion policy, may include:

Determining the current motion state of an actuating mechanism of the mobile robot based on the motor parameters of the mobile robot acquired by the sensor in the process of moving the mobile robot according to the motion strategy; if the difference between the current motion state of the executing mechanism and the target motion state meets a first preset precision condition, controlling the mobile robot to continue to move according to the motion strategy, wherein the target motion state is a motion state corresponding to the executing mechanism included in the motion strategy when the executing mechanism is at the current position; and if the difference between the current motion state and the target motion state does not meet the first preset precision condition, controlling the mobile robot to stop moving, and reporting abnormal information.

The mobile robot is also provided with a sensor for detecting motor parameters of the mobile robot, and the motion of the executing mechanism is driven by the motor motion, based on which the motor parameters can characterize the motion parameters of the executing mechanism.

In the process of controlling the mobile robot to move according to the movement strategy, the electronic equipment can acquire the motor parameters of the mobile robot acquired by the sensor, and determine the current movement state of the executing mechanism according to the motor parameters of the mobile robot acquired by the sensor. The electronic device may then calculate a motion policy according to an actuator that processes the corresponding target motion state at the current location.

Then, the electronic device can calculate the difference between the current motion state and the target motion state of the executing mechanism, determine whether the difference meets a first preset precision condition, and if the difference between the current motion state and the target motion state meets the first preset precision condition, the motion precision of the executing mechanism is higher, so that the mobile robot can be controlled to continue to move according to a motion strategy.

If the difference between the current motion state and the target motion state does not meet the first preset precision condition, the motion state of the executing mechanism is abnormal, in order to avoid that the mobile executing mechanism continues to move in the abnormal state, the mobile robot can be controlled to stop moving, the current motion state and the difference of the executing mechanism are reported as abnormal information, and therefore when the abnormal information is received by a worker, the abnormal information is recorded and analyzed.

Correspondingly, the motor parameters of the mobile robot collected by the sensor can be used for verifying the action precision of the preset action of the executing mechanism on the target workpiece. As an implementation manner of the embodiment of the present application, in step S104, after the step of controlling the executing mechanism to execute the preset action on the target workpiece, the method may further include:

After the step of controlling the executing mechanism to execute the preset action on the target workpiece, the electronic device can acquire the motor parameters of the mobile robot acquired by the sensor. Then, a third current positional relationship between the target workpiece and the image capturing apparatus is determined based on the motor parameter. And then calculating the difference between the third position relation and the first calibration position relation, and determining whether the difference meets a second preset precision condition.

It can be seen that, in this embodiment, in the process of controlling the mobile robot to move according to the movement strategy and after controlling the executing mechanism to execute the preset action on the target workpiece, the electronic device may verify the action precision of the executing mechanism based on the motor parameter collected by the sensor, so as to keep the executing mechanism to move with higher action precision, and improve the operation precision of the mobile robot.

Next, an overall process of performing motion control on the actuator is illustrated with reference to fig. 8, as shown in fig. 8, where the vision sensor is an image acquisition device provided on the mobile robot in the embodiment of the present application; the main control is the electronic equipment in the embodiment of the application; the process of controlling the motion of the actuator may include:

S801: installing a visual sensor structure;

S802: parameter calibration under the condition of the current installation position of the visual sensor is carried out through a calibration tool;

S803: the vision sensor collects characteristic images and analyzes position relation information through the images;

s804: the master control analyzes action parameters required to be executed by the executing mechanism and outputs the action parameters to the corresponding control motor to execute the action;

s805: the electronic sensor feedback action is performed in place and completed.

In the structural design process of the actuating mechanism and the visual sensor, the perception precision of the visual sensor is improved in a finishing mode and the like, and the visual sensor is installed with higher installation precision in the installation stage of the visual sensor.

The execution mechanism of the main control mobile robot executes preset actions on the calibration fixture corresponding to the target workpiece, and obtains characteristic images of the calibration fixture acquired by the vision sensor. And then determining and analyzing the position relation between the target feature of the calibration tool and the image acquisition equipment based on the position of the target feature in the feature image and the equipment parameters of the pre-calibrated image acquisition equipment to obtain calibration parameters of the vision sensor under the condition of the current installation position, wherein the calibration parameters comprise parameters such as angles, positions and the like.

Under the condition that the mobile robot runs to an execution area, a characteristic image of a target workpiece acquired by a visual sensor is acquired, the position relation information of the visual sensor and the target workpiece is analyzed through the characteristic image, the parameters which are needed to be executed and adjusted by the execution mechanism at the moment are obtained, and the parameters are sent to a control motor for executing the motion, so that the control motor controls the execution mechanism to perform motion control on the execution mechanism according to the parameters.

In the motion process of the executing mechanism, whether the current motion state of the executing mechanism is correct or not is checked through image data acquired by the visual sensor, the executing mechanism can be controlled to stop motion due to abnormal data, and alarm information is sent, so that the executing mechanism is prevented from moving in the abnormal state to cause equipment damage.

After the motor is executed to execute the preset action, the motor parameters fed back by the motor sensor are obtained, and the motion parameters of the executing mechanism are checked based on the electronic parameters, so that whether the motion precision of the executing mechanism meets the preset precision requirement is determined.

Corresponding to the motion control method of the executing mechanism, the embodiment of the application also provides a motion control device of the executing mechanism, and the motion control device of the executing mechanism provided by the embodiment of the application is introduced below.

As shown in fig. 9, a motion control device of an actuator, the actuator is disposed on a mobile robot, the mobile robot further includes an image acquisition apparatus, the device includes:

An image acquisition module 901, configured to acquire a target image of a target workpiece acquired by the image acquisition device when the mobile robot moves to an execution area;

A position relationship determining module 902, configured to determine a first current position relationship between the target workpiece and the image capturing device based on the target image and a device parameter of the image capturing device calibrated in advance;

A motion policy determining module 903, configured to determine a motion policy of the executing mechanism based on the first current position relationship, a predetermined calibration position relationship, and a position relationship between the image capturing device and the executing mechanism, where the calibration position relationship includes a first calibration position relationship between the calibration tool and the image capturing device in a state where the executing mechanism calibrated in advance performs a preset action on a calibration tool corresponding to the target workpiece;

And the motion control module 904 is used for controlling the mobile robot to move according to the motion strategy and controlling the executing mechanism to execute the preset action on the target workpiece when the mobile robot reaches a motion end point.

As an implementation manner of the embodiment of the present application, the motion policy determining module 903 includes:

The motion parameter determining unit is used for determining motion parameters corresponding to the image acquisition equipment according to the difference between the first current position relation and a predetermined calibration position relation;

And the motion strategy determining unit is used for determining the motion strategy of the executing mechanism based on the motion parameters corresponding to the image acquisition equipment and the position relationship between the image acquisition equipment and the executing mechanism.

As an implementation manner of the embodiment of the present application, the motion policy determining unit includes:

A motion policy determining subunit, configured to obtain a motion policy of the executing mechanism by using, as a motion parameter of the executing mechanism, a motion parameter corresponding to the image capturing device if a positional relationship between the image capturing device and the executing mechanism is unchanged; or if the position relationship between the image acquisition equipment and the executing mechanism is variable, and the calibration position relationship further comprises a second calibration position relationship between the executing mechanism and the image acquisition equipment in a state that the executing mechanism performs the preset action on the calibration tool corresponding to the target workpiece, and the motion strategy of the executing mechanism is determined based on the motion parameter corresponding to the image acquisition equipment, the second position relationship and the current position relationship between the image acquisition equipment and the executing mechanism.

As one implementation manner of the embodiment of the present application, the calibration fixture has a target feature, where the target feature is set in advance based on an actual feature of the target workpiece; the apparatus further comprises:

the action execution module is used for controlling an execution mechanism of the mobile robot to execute a preset action on a calibration tool corresponding to the target workpiece before the step of acquiring the target image of the target workpiece acquired by the image acquisition equipment;

The calibration image acquisition module is used for acquiring a calibration image of the calibration tool currently acquired by the image acquisition equipment;

And the calibration position relation determining module is used for determining the position relation between the calibration tool and the image acquisition equipment based on the position of the target feature in the calibration image and the equipment parameter of the image acquisition equipment calibrated in advance, and taking the position relation as the calibration position relation.

As an implementation manner of the embodiment of the present application, the location relationship determining module 902 includes:

And the first determining unit is used for determining a first current position relation between the target workpiece and the image acquisition equipment based on the position of the actual feature included in the target image and the equipment parameter of the image acquisition equipment calibrated in advance.

As an implementation of the embodiment of the present application, the motion control module 904 includes:

The second determining unit is used for determining a second current position relation between the target workpiece and the image acquisition equipment based on the image of the target workpiece and the equipment parameters which are currently acquired by the image acquisition equipment in the process that the mobile robot moves according to the movement strategy;

a calculating unit, configured to calculate a current motion state of the executing mechanism based on the second current position relationship and a current position relationship between the image capturing device and the executing mechanism; if the difference between the current motion state and the target motion state of the executing mechanism meets a first preset precision condition, triggering a motion control unit; triggering a motion stop unit if the difference between the current motion state and the target motion state does not meet the first preset precision condition;

the motion control unit is used for controlling the mobile robot to continue to move according to the motion strategy, wherein the target motion state is a motion state corresponding to the motion strategy when the executing mechanism is at the current position;

And the motion stopping unit is used for controlling the mobile robot to stop moving and reporting abnormal information.

As an implementation manner of the embodiment of the present application, the apparatus further includes:

a third current position relationship determining module, configured to determine a third current position relationship between the target workpiece and the image capturing device based on the image of the target workpiece and the device parameter currently captured by the image capturing device after the step of controlling the executing mechanism to execute the preset action on the target workpiece;

The first precision determining module is used for determining the precision of the execution mechanism for executing the preset action on the target workpiece according to the relation between the difference of the third position relation and the first calibration position relation and the second preset precision condition.

As an implementation of the embodiment of the present application, the mobile robot further includes a sensor for detecting a motor parameter of the mobile robot; the motion control module 904 includes:

The motion stopping unit is used for determining the current motion state of an executing mechanism of the mobile robot based on the motor parameters of the mobile robot acquired by the sensor in the process that the mobile robot moves according to the motion strategy; if the difference between the current motion state of the executing mechanism and the target motion state meets a first preset precision condition, controlling the mobile robot to continue to move according to the motion strategy, wherein the target motion state is a motion state corresponding to the executing mechanism included in the motion strategy when the executing mechanism is at the current position; if the difference between the current motion state and the target motion state does not meet the first preset precision condition, controlling the mobile robot to stop moving, and reporting abnormal information; and/or

The apparatus further comprises:

The second precision determining module is used for determining a third current position relation between the target workpiece and the image acquisition equipment based on the motor parameters of the mobile robot acquired by the sensor after the step of controlling the executing mechanism to execute the preset action on the target workpiece; and determining the precision of the execution mechanism for executing the preset action on the target workpiece according to the relation between the difference of the third position relation and the first calibration position relation and a second preset precision condition.

Corresponding to the motion control method of the executing mechanism, the embodiment of the application also provides a mobile robot, and the mobile robot provided by the embodiment of the application is introduced below.

As shown in fig. 10, a mobile robot includes an image acquisition device 1001 and a processor 1002.

The image acquisition device 1001 is configured to acquire an image;

the processor 1002 is configured to execute steps of a motion control method of any one of the actuators provided in the embodiments of the present application.

The mobile robot provided by the embodiment of the application can comprise an image acquisition device and a processor. The image acquisition device is used for acquiring images, and the processor is used for acquiring target images of target workpieces acquired by the image acquisition device under the condition that the mobile robot moves to an execution area; determining a first current position relationship between a target workpiece and image acquisition equipment based on a target image and equipment parameters of the image acquisition equipment calibrated in advance; determining a motion strategy of an executing mechanism based on a first current position relation, a predetermined calibration position relation and a position relation between image acquisition equipment and the executing mechanism, wherein the calibration position relation comprises a first calibration position relation between the calibration tool and the image acquisition equipment in a state that the executing mechanism calibrated in advance executes a preset action on the calibration tool corresponding to a target workpiece; and controlling the mobile robot to move according to a movement strategy, and controlling the executing mechanism to execute a preset action on the target workpiece when the mobile robot reaches a movement end point.

The processor may be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU), a network processor (Network Processor, NP), etc.; but may also be a digital signal Processor (DIGITAL SIGNAL Processor, DSP), application SPECIFIC INTEGRATED Circuit (ASIC), field-Programmable gate array (Field-Programmable GATE ARRAY, FPGA) or other Programmable logic device, discrete gate or transistor logic device, discrete hardware components.

In yet another embodiment of the present application, a computer readable storage medium is provided, in which a computer program is stored, which when executed by a processor, implements the steps of the motion control method of any of the actuators described above.

In yet another embodiment of the present application, a computer program product containing instructions that, when run on a computer, cause the computer to perform the method of controlling the motion of any of the actuators of the above embodiments is also provided.

In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, produces a flow or function in accordance with embodiments of the present application, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another, for example, by wired (e.g., coaxial cable, optical fiber, digital Subscriber Line (DSL)), or wireless (e.g., infrared, wireless, microwave, etc.). The computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains an integration of one or more available media. The usable medium may be a magnetic medium (e.g., floppy disk, hard disk, tape), an optical medium (e.g., DVD), or a Solid state disk (Solid STATE DISK, SSD), etc.

It is noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.

In this specification, each embodiment is described in a related manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for system embodiments, since they are substantially similar to method embodiments, the description is relatively simple, as relevant to see a section of the description of method embodiments.

The foregoing description is only of the preferred embodiments of the present application and is not intended to limit the scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application are included in the protection scope of the present application.

Claims

1. A method for controlling movement of an actuator, wherein the actuator is disposed on a mobile robot, the mobile robot further comprising an image acquisition device, the method comprising:

2. The method of claim 1, wherein the step of determining a motion strategy of the actuator based on the first current positional relationship, a predetermined calibration positional relationship, and a positional relationship between the image capture device and the actuator comprises:

3. The method according to claim 2, wherein the step of determining the motion strategy of the actuator based on the motion parameters corresponding to the image capturing device and the positional relationship between the image capturing device and the actuator includes:

4. The method of claim 1, wherein the calibration fixture has target features that are set in advance based on actual features that the target workpiece has;

5. The method of claim 4, wherein the step of determining a first current positional relationship of the target workpiece and the image capturing device based on the target image and the device parameters of the image capturing device calibrated in advance comprises:

6. The method according to any one of claims 1-5, wherein said step of controlling said mobile robot to move according to said movement strategy comprises:

7. The method of any of claims 1-5, wherein after the step of controlling the actuator to perform the preset action on the target workpiece, the method further comprises:

8. The method according to any of claims 1-5, wherein the mobile robot further comprises a sensor for detecting a motor parameter of the mobile robot;

9. A motion control device for an actuator, wherein the actuator is disposed on a mobile robot, the mobile robot further comprising an image acquisition device, the device comprising:

10. A mobile robot comprising an image acquisition device and a processor;

The image acquisition equipment is used for acquiring images;

The processor being configured to perform the method of any of claims 1-8.

11. A computer readable storage medium, characterized in that the computer readable storage medium has stored therein a computer program which, when executed by a processor, implements the method of any of claims 1-8.