CN119299850A - Remote image acquisition method and related products - Google Patents

Abstract

The present application discloses a remote image acquisition method and related products, the remote image acquisition method is applied to a control device, the control device is used to remotely control the image acquisition device to acquire images, the image acquisition device is fixed on a mechanical arm, the control device drives the image acquisition device to move by controlling the movement of the mechanical arm, the method comprises: obtaining a first posture of the image acquisition device in a base coordinate system of the mechanical arm, the base coordinate system is a coordinate system constructed with the base of the mechanical arm; obtaining a first adjustment amount of the posture of the image acquisition device; based on the first adjustment amount, adjusting the first posture to obtain a second posture of the image acquisition device in the base coordinate system; obtaining a first conversion relationship, the first conversion relationship being a conversion relationship between the posture of the image acquisition device and the posture of a reference structure, the reference structure comprising a structure on the mechanical arm except the base; using the first conversion relationship to convert the second posture into a third posture of the reference structure in the base coordinate system.

Description

Remote image acquisition method and related products

Technical Field

The present application relates to the field of medical image processing technology, and in particular to a remote image acquisition method and related products.

Background Art

In the medical field, there is often a need for remote surgery, which involves remote control image acquisition. Therefore, how to perform remote image acquisition methods is of great significance.

Summary of the invention

The present application provides a remote image acquisition method and related products for remote image acquisition, wherein the related products include a remote image acquisition puncture device, an electronic device, a computer-readable storage medium, and a computer program product.

In a first aspect, a remote image acquisition method is provided, the method is applied to a control device, the control device is used to remotely control an image acquisition device to acquire images, the image acquisition device is fixed on a mechanical arm, the control device drives the image acquisition device to move by controlling the movement of the mechanical arm, the method comprises:

Acquire a first position of the image acquisition device in a base coordinate system of the robotic arm, wherein the base coordinate system is a coordinate system constructed based on a base of the robotic arm;

Acquire a first adjustment amount of the posture of the image acquisition device;

Based on the first adjustment amount, adjusting the first posture to obtain a second posture of the image acquisition device in the base coordinate system;

Acquire a first conversion relationship, where the first conversion relationship is a conversion relationship between a posture of the image acquisition device and a posture of a reference structure, where the reference structure includes a structure on the robotic arm except the base;

The second posture is converted into a third posture of the reference structure in the base coordinate system by using the first conversion relationship.

In combination with any embodiment of the present application, after using the first transformation relationship to transform the second posture into a third posture of the reference structure in the base coordinate system, the method further includes:

The movement of the robotic arm is controlled by the third posture so that the posture of the reference structure is the third posture.

In combination with any embodiment of the present application, the image captured by the image acquisition device includes an ultrasonic image, and after controlling the movement of the robotic arm through the third posture so that the posture of the reference structure is the third posture, the method includes:

receiving a target ultrasound image acquired by the image acquisition device in the third posture, wherein the target ultrasound image includes an object to be punctured;

A puncture path of the object to be punctured is determined based on the target ultrasound image.

In combination with any embodiment of the present application, the first posture is a posture at a first moment, and the step of obtaining a first adjustment amount of the posture of the image acquisition device includes:

Acquire a fourth posture of the control device at the first moment, the control device being used to remotely adjust the posture of the image acquisition device;

Acquire a fifth posture of the control device at a second moment, where the second moment is a moment later than the first moment;

The first adjustment amount is obtained based on a difference between the fifth posture and the fourth posture.

In combination with any embodiment of the present application, the first adjustment amount includes a first rotation amount and a first translation amount, and the using the first conversion relationship to convert the second posture into a third posture of the reference structure in the base coordinate system includes:

Decomposing the second posture into a second rotation and a second translation;

adjusting the second rotation amount by using the first rotation amount to obtain a third rotation amount;

adjusting the second translation amount by using the first translation amount to obtain a third translation amount;

The third posture is obtained based on the third rotation amount and the third translation amount.

In combination with any embodiment of the present application, the base coordinate system includes a horizontal axis, a vertical axis and a vertical axis, the first rotation amount includes a first horizontal rotation amount, a first vertical rotation amount and a first vertical rotation amount, the first horizontal rotation amount is a component of the first rotation amount on the horizontal axis, the first vertical rotation amount is a component of the first rotation amount on the vertical axis, and the first vertical rotation amount is a component of the first rotation amount on the vertical axis;

The step of adjusting the second rotation amount by using the first rotation amount to obtain a third rotation amount includes:

Determine the component of the second rotation amount on the horizontal axis to obtain a second horizontal rotation amount;

Determine the component of the second rotation amount on the longitudinal axis to obtain a second longitudinal rotation amount;

Determine the component of the second rotation amount on the vertical axis to obtain a second vertical rotation amount;

adjusting the second lateral rotation amount by using the first lateral rotation amount to obtain a third lateral rotation amount;

adjusting the second longitudinal rotation amount by using the first longitudinal rotation amount to obtain a third longitudinal rotation amount;

adjusting the second vertical rotation amount by using the first vertical rotation amount to obtain a third vertical rotation amount;

The third rotation amount is obtained based on the third lateral rotation amount, the third longitudinal rotation amount, and the third vertical rotation amount.

In a second aspect, a control device is provided, the control device is used to remotely control an image acquisition device to acquire images, the image acquisition device is fixed on a mechanical arm, the control device drives the image acquisition device to move by controlling the movement of the mechanical arm, and the control device includes:

An acquisition unit, used for acquiring a first position of the image acquisition device in a base coordinate system of the robotic arm, wherein the base coordinate system is a coordinate system constructed based on a base of the robotic arm;

The acquisition unit is further used to acquire a first adjustment amount of the posture of the image acquisition device;

an adjusting unit, configured to adjust the first posture to obtain a second posture of the image acquisition device in the base coordinate system based on the first adjustment amount;

The acquisition unit is further used to acquire a first conversion relationship, where the first conversion relationship is a conversion relationship between a posture of the image acquisition device and a posture of a reference structure, where the reference structure includes a structure on the robotic arm except the base;

A conversion unit is used to convert the second posture into a third posture of the reference structure in the base coordinate system by using the first conversion relationship.

In combination with any implementation manner of the present application, the control device further includes:

A control unit is used to control the movement of the robotic arm through the third posture so that the posture of the reference structure is the third posture.

In combination with any embodiment of the present application, the image captured by the image acquisition device includes an ultrasonic image, and the control device further includes: a receiving unit for receiving a target ultrasonic image captured by the image acquisition device in the third posture, wherein the target ultrasonic image includes the object to be punctured;

A determination unit is used to determine a puncture path of the object to be punctured based on the target ultrasound image.

In combination with any implementation manner of the present application, the acquisition unit is used to:

Acquire a fourth posture of the control device at the first moment, the control device being used to remotely adjust the posture of the image acquisition device;

Acquire a fifth posture of the control device at a second moment, where the second moment is a moment later than the first moment;

The first adjustment amount is obtained based on a difference between the fifth posture and the fourth posture.

In combination with any embodiment of the present application, the first adjustment amount includes a first rotation amount and a first translation amount, and the conversion unit is used to:

Decomposing the second posture into a second rotation and a second translation;

adjusting the second rotation amount by using the first rotation amount to obtain a third rotation amount;

adjusting the second translation amount by using the first translation amount to obtain a third translation amount;

The third posture is obtained based on the third rotation amount and the third translation amount.

In combination with any embodiment of the present application, the base coordinate system includes a horizontal axis, a vertical axis and a vertical axis, the first rotation amount includes a first horizontal rotation amount, a first vertical rotation amount and a first vertical rotation amount, the first horizontal rotation amount is a component of the first rotation amount on the horizontal axis, the first vertical rotation amount is a component of the first rotation amount on the vertical axis, and the first vertical rotation amount is a component of the first rotation amount on the vertical axis;

The conversion unit is used for:

Determine the component of the second rotation amount on the horizontal axis to obtain a second horizontal rotation amount;

Determine the component of the second rotation amount on the longitudinal axis to obtain a second longitudinal rotation amount;

Determine the component of the second rotation amount on the vertical axis to obtain a second vertical rotation amount;

adjusting the second lateral rotation amount by using the first lateral rotation amount to obtain a third lateral rotation amount;

adjusting the second longitudinal rotation amount by using the first longitudinal rotation amount to obtain a third longitudinal rotation amount;

adjusting the second vertical rotation amount by using the first vertical rotation amount to obtain a third vertical rotation amount;

The third rotation amount is obtained based on the third lateral rotation amount, the third longitudinal rotation amount, and the third vertical rotation amount.

In a third aspect, an electronic device is provided, comprising: a processor and a memory, the memory being used to store computer program code, the computer program code comprising computer instructions, and when the processor executes the computer instructions, the electronic device executes the method as described in the first aspect above and any possible implementation thereof.

In a fourth aspect, another electronic device is provided, comprising: a processor, a sending device, an input device, an output device and a memory, wherein the memory is used to store computer program code, and the computer program code includes computer instructions. When the processor executes the computer instructions, the electronic device executes the method as described in the first aspect above and any possible implementation method thereof.

In a fifth aspect, a computer-readable storage medium is provided, in which a computer program is stored. The computer program includes program instructions, and when the program instructions are executed by a processor, the processor is caused to execute the method as described in the first aspect above and any possible implementation method thereof.

In a sixth aspect, a computer program product is provided, which includes a computer program or instructions, and when the computer program or instructions are run on a computer, the computer is enabled to execute the method of the above-mentioned first aspect and any possible implementation thereof.

It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application.

In an embodiment of the present application, the image acquisition device is fixed on the robotic arm, wherein the control device drives the image acquisition device to move by controlling the movement of the robotic arm. After the control device obtains the first posture of the image acquisition device in the base coordinate system of the robotic arm, it obtains the first adjustment amount of the posture of the image acquisition device. Then, based on the first adjustment amount, the first posture is adjusted to obtain the second posture of the image acquisition device in the base coordinate system. Then, the first conversion relationship is obtained, wherein the first conversion relationship is the conversion relationship between the posture of the image acquisition device and the posture of the reference structure, and the reference structure is the structure of the robotic arm. Finally, the second posture is converted into the third posture of the reference structure in the base coordinate system using the first conversion relationship, so that the first adjustment amount of the posture of the image acquisition device can be converted into the posture of the reference structure of the robotic arm.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the technical solutions in the embodiments of the present application or the background technology, the drawings required for use in the embodiments of the present application or the background technology will be described below.

The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present application and are used together with the specification to illustrate the technical solution of the present application.

FIG1 is a schematic diagram of a flow chart of a remote image acquisition method provided in an embodiment of the present application;

FIG2 is a schematic diagram of a scenario in which an image acquisition device is controlled by a joystick to acquire an image, provided in an embodiment of the present application;

FIG3a is a schematic diagram of the horizontal axis and the vertical axis of a flange coordinate system provided in an embodiment of the present application;

FIG3 b is a schematic diagram of the longitudinal axis and the vertical axis of a flange coordinate system provided in an embodiment of the present application;

FIG4 is a schematic diagram of a scenario of puncture based on an ultrasound probe provided in an embodiment of the present application;

FIG5 is a schematic diagram of the structure of a control device provided in an embodiment of the present application;

FIG6 is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application.

DETAILED DESCRIPTION

In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

The terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.

Mentioning "embodiment" in this article means that the specific features, structures or characteristics described in conjunction with the embodiment may be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments. It should be understood that in this application, "at least one (item)" means one or more, "multiple" means two or more, and "at least two (items)" means two or three and more.

The embodiment of the present application provides a remote image acquisition method, which is applied to a control device, wherein the control device is used to remotely control an image acquisition device to acquire images, the image acquisition device is fixed on a mechanical arm, and the control device drives the image acquisition device to move by controlling the movement of the mechanical arm. By controlling the movement of the mechanical arm, the image acquisition device can be driven to move, and then the shooting angle of the image acquisition device can be adjusted, thereby adjusting the content of the image acquired by the image acquisition device.

In one possible implementation scenario, the image acquisition device includes an ultrasound probe, wherein the ultrasound probe is used to acquire ultrasound images. The ultrasound probe is fixed on a robotic arm, and the robotic arm can be deployed in an operating room. The control device can be any electronic device. Optionally, the control device includes one of the following: a mobile phone, a computer, and a tablet computer. The control device can be deployed outside the operating room. In this implementation scenario, a user can control the image acquisition device in the operating room to acquire images through the control device outside the operating room, thereby realizing remote image acquisition. For example, a doctor can control the image acquisition device in the operating room to acquire images through the control device outside the operating room. In the embodiment of the present application, the executor of the remote image acquisition method is the above-mentioned control device.

It should be understood that the method embodiment of the present application can also be implemented by a processor executing a computer program code. The present application embodiment is described below in conjunction with the accompanying drawings in the present application embodiment. Please refer to Figure 1, which is a flow chart of a remote image acquisition method provided in the present application embodiment.

101. Obtain the first position of the image acquisition device in the base coordinate system of the robotic arm.

In the embodiment of the present application, the base coordinate system is a coordinate system constructed by the base of the robot arm. The first pose is the pose of the image acquisition device in the base coordinate system. Optionally, the pose in the embodiment of the present application includes position and pose, and the first pose includes the position of the image acquisition device in the base coordinate system and the pose of the image acquisition device in the base coordinate system.

In an implementation method of obtaining the first posture, a control device receives the first posture input by the user through an input component to obtain the first posture, wherein the input component includes: a mouse, a keyboard, a touch screen, a touch pad, and an audio input device.

In another implementation method of obtaining the first posture, a control device receives the first posture sent by the user through a terminal to obtain the first posture, wherein the terminal includes: a mobile phone, a computer, a tablet computer, and a smart wearable device.

In another implementation method of obtaining the first pose, the control device includes a positioning imaging device, wherein the positioning imaging device includes an imaging device, and the positioning imaging device is used to determine the first pose. Optionally, the positioning imaging device is a laser radar. By scanning the image acquisition device through the laser radar, a reference pose of the image acquisition device in the world coordinate system can be obtained. Then, based on the conversion relationship between the world coordinate system and the base coordinate system, the reference pose is converted to obtain the first pose, wherein the conversion relationship includes a rotation relationship between the world coordinate system and the base coordinate system, and a translation relationship between the world coordinate system and the base coordinate system. After rotating the world coordinate system based on the rotation relationship and translating the world coordinate system based on the translation relationship, the world coordinate system can be aligned with the base coordinate system.

In an implementation scenario based on this implementation method, the control device is deployed outside the operating room, and the image acquisition device is deployed inside the operating room. Through this implementation method, the position and posture of the image acquisition device in the operating room can be obtained outside the operating room.

Optionally, the positioning imaging device is a two-dimensional imaging device. A two-dimensional scene image including an image acquisition device is acquired by the two-dimensional imaging device. Then, the two-dimensional scene image is input into a pose estimation model, so that the pose estimation model estimates the pose of the image acquisition device in the two-dimensional scene image in the base coordinate system to obtain a first pose. The pose estimation model is used to estimate the pose of an object in the two-dimensional image in the base coordinate system based on the two-dimensional image.

102. Obtain a first adjustment amount of the position and posture of the image acquisition device.

In the embodiment of the present application, the first adjustment amount is the adjustment amount of the posture of the image acquisition device. Adjusting the posture of the image acquisition device based on the first adjustment amount can change the posture of the image acquisition device, and then change the shooting angle and shooting range of the image acquisition device, thereby changing the content of the image captured by the image acquisition device.

In a possible implementation, the control device includes a joystick, wherein the joystick is used to adjust the posture of the image acquisition device. Specifically, the posture of the image acquisition device can be changed by adjusting the posture of the joystick. The user can input a first adjustment amount of the posture of the image acquisition device to the control device by moving the joystick. Optionally, the adjustment amount of the posture of the image acquisition device is based on the adjustment amount of the posture of the joystick. Please refer to Figure 2, which is a schematic diagram of a scene of controlling an image acquisition device to acquire an image by a joystick provided in an embodiment of the present application. As shown in Figure 2, the image acquisition device is fixed on a mechanical arm, and the operating lever can drive the image acquisition device to move by controlling the movement of the mechanical arm, thereby controlling the image acquisition device to acquire an image of the object to be acquired, wherein the image acquisition device can be an ultrasound probe, and the object to be acquired can be a patient. It should be understood that the joystick and the mechanical arm in Figure 2 may not be fixedly connected. In actual applications, the joystick and the mechanical arm can be deployed in different spaces respectively, for example, the joystick is deployed outside the operating room and the mechanical arm is deployed in the operating room.

103. Based on the first adjustment amount, adjust the first posture to obtain a second posture of the image acquisition device in the base coordinate system.

As described in step 102, the first adjustment amount is the adjustment amount of the posture of the image acquisition device, so the control device can adjust the first posture of the image acquisition device in the base coordinate system based on the first adjustment amount, and the posture of the image acquisition device in the base coordinate system after adjustment is the second posture. In a possible implementation, the control device obtains the second posture by determining the sum of the first posture and the first adjustment amount. In another possible implementation, after the control device adjusts the first posture based on the first adjustment amount to obtain the second posture, it also plans a reference path for the image acquisition device to adjust the posture from the first posture to the second posture based on the three-dimensional scene image, wherein the three-dimensional scene image is a three-dimensional image of the scene where the robot arm and the image acquisition device are located, and the three-dimensional scene image is acquired by the control device, and the reference path is the movement path of the robot arm, and the robot arm moves according to the reference path to adjust the posture of the image acquisition device on the robot arm from the first posture to the second posture. In this implementation, since the scene where the robot arm and the image acquisition device are located is an operating room, and there may be people or objects in the operating room, the movement path of the robot arm should avoid people or objects. The control device is outside the operating room, so by collecting the three-dimensional image of the scene in the operating room (that is, the scene where the robot arm and the image acquisition device are located) through the control device, the information of the scene in the operating room can be obtained more comprehensively and accurately. Then, based on the three-dimensional scene image, a reference path is planned, and when the robot arm moves according to the reference path to change the posture of the image acquisition device from the first posture to the second posture, the probability of the robot arm and the image acquisition device colliding with people or objects can be reduced, thereby improving safety.

104. Obtain a first conversion relationship.

In the embodiment of the present application, the first conversion relationship is the conversion relationship between the posture of the image acquisition device and the posture of the reference structure, that is, the posture of the image acquisition device can be converted into the posture of the reference structure using the first conversion relationship. For example, when it is known that the posture of the image acquisition device is posture 1, the posture of the reference structure can be obtained by converting posture 1 using the first conversion relationship. The reference structure is the structure of the robot arm. Optionally, the reference structure is the structure of the robot arm except the base. Optionally, the reference structure is the structure on the robot arm connected to the image acquisition device. For example, the image acquisition device is fixed on the flange of the robot arm, then the reference structure is the flange.

Optionally, in the case where the reference structure is a flange of a robotic arm, the first conversion relationship is a conversion relationship between a base coordinate system and a flange coordinate system, wherein the flange coordinate system is a coordinate system constructed based on the flange. Optionally, FIG. 3a is a schematic diagram of a horizontal axis and a vertical axis of a flange coordinate system provided in an embodiment of the present application, and FIG. 3b is a schematic diagram of a longitudinal axis and a vertical axis of a flange coordinate system provided in an embodiment of the present application. As shown in FIG. 3a and FIG. 3b, the flange includes a first end face and a second end face, wherein the first end face is an end face in contact with the image acquisition device. The origin of the flange coordinate system (i.e., O1 in FIG. _3a and _O1 in FIG. 3b) is the center of the first end face. The horizontal axis (i.e., the _X1 axis in FIG. 3a) and the longitudinal axis (i.e., the _Y1 axis in FIG. 3a and the _Y1 axis in FIG. 3b) are both parallel to the radial direction of the first end face, and the horizontal axis is perpendicular to the longitudinal axis. The longitudinal axis (i.e., the _Z1 axis in FIG. 3b) is perpendicular to the first end face.

When the image acquisition device is fixed on the flange, the first conversion relationship can be determined based on the structural relationship between the image acquisition device and the flange. At this time, the first conversion relationship can be used to convert the position and posture of the image acquisition device in the base coordinate system into the position and posture of the flange in the flange coordinate system. Based on the first conversion relationship, the robotic arm can convert the position and posture of the flange in the flange coordinate system into the position and posture of the flange in the base coordinate system. For example, the first conversion relationship is represented by Flange, where Flange is a matrix, and the position and posture of the image acquisition device in the base coordinate system can be converted into the position and posture of the flange in the flange coordinate system based on the first conversion relationship. Then, based on Flange ^-1 , the position and posture of the flange in the flange coordinate system can be converted into the position and posture of the flange in the base coordinate system, where Flange ^-1 is the inverse matrix of Flange.

Since the structure that the robot can control is the structure of the robot, and it cannot control objects other than the structure of the robot, in other words, the robot can control the position and posture of the structure of the robot, but cannot control the position and posture of objects fixed on the robot. The reference structure is the structure of the robot, and the image acquisition device is not the structure of the robot. Therefore, the robot can control the position and posture of the reference structure, but cannot control the position and posture of the image acquisition device. It should be understood that the robot controlling the position and posture of the structure of the robot can be understood as the robot includes a central processing unit (CPU), and the CPU can control the structure of the robot. Specifically, the CPU can adjust the position and posture of the structure of the robot in the base coordinate system of the robot by controlling the movement of the robot. For example, the CPU of the robot can adjust the position and posture of the reference structure of the robot in the base coordinate system by controlling the movement of each joint of the robot.

105. Use the first transformation relationship to transform the second posture into a third posture of the reference structure in the base coordinate system.

Since the first conversion relationship is the conversion relationship between the posture of the image acquisition device and the posture of the reference structure, the control device can use the first conversion relationship to convert the posture of the image acquisition device in the base coordinate system into the posture of the reference structure in the base coordinate system, that is, to convert the second posture into the posture of the reference structure in the base coordinate system, that is, the third posture. As described in step 104, the structure that the robot can control is the structure of the robot, and cannot control objects other than the structure of the robot. Therefore, the control device obtains the third posture by executing step 105, and the robot can adjust the posture of the image acquisition device to the second posture by adjusting the posture of the reference structure to the third posture.

In an embodiment of the present application, the image acquisition device is fixed on the robotic arm, wherein the control device drives the image acquisition device to move by controlling the movement of the robotic arm. After the control device obtains the first posture of the image acquisition device in the base coordinate system of the robotic arm, it obtains the first adjustment amount of the posture of the image acquisition device. Then, based on the first adjustment amount, the first posture is adjusted to obtain the second posture of the image acquisition device in the base coordinate system. Then, the first conversion relationship is obtained, wherein the first conversion relationship is the conversion relationship between the posture of the image acquisition device and the posture of the reference structure, and the reference structure is the structure of the robotic arm. Finally, the second posture is converted into the third posture of the reference structure in the base coordinate system using the first conversion relationship, so that the first adjustment amount of the posture of the image acquisition device can be converted into the posture of the reference structure of the robotic arm.

As an optional implementation, after the control device converts the second posture into the third posture of the reference structure in the base coordinate system using the first conversion relationship, the control device controls the movement of the robotic arm through the third posture so that the posture of the reference structure is the third posture. Optionally, after obtaining the third posture, the control device sends a control instruction carrying the third posture to the robotic arm, wherein the control instruction is used to instruct the robotic arm to adjust the posture of the reference structure to the third posture. Because when the posture of the reference structure in the base coordinate system is the third posture, the posture of the image acquisition device in the base coordinate system is the second posture, the control device can adjust the posture of the image acquisition device to the second posture based on the first adjustment amount by sending a control instruction carrying the third posture to the robotic arm.

In a possible scenario, the image acquisition device includes an ultrasound probe, wherein the ultrasound probe is used to acquire ultrasound images. The ultrasound probe is fixed on a robotic arm, the robotic arm can be deployed in an operating room, and the control device is deployed outside the operating room. The user can adjust the posture of the ultrasound probe in the operating room to the second posture outside the operating room by sending a control instruction carrying a third posture to the robotic arm through the control device.

As an optional embodiment, the image captured by the above-mentioned image acquisition device includes an ultrasonic image. After the control device controls the movement of the above-mentioned robotic arm through the above-mentioned third posture so that the posture of the above-mentioned reference structure is the above-mentioned third posture, the control device further performs the following steps: receiving a target ultrasonic image captured by the above-mentioned image acquisition device in the above-mentioned third posture, the above-mentioned target ultrasonic image includes the object to be punctured; and determining the puncture path of the above-mentioned object to be punctured based on the above-mentioned target ultrasonic image.

In one possible scenario, the image acquisition device includes an ultrasound probe, wherein the ultrasound probe is used to acquire ultrasound images. The ultrasound probe is fixed on a robotic arm, the robotic arm can be deployed in an operating room, and the control device is deployed outside the operating room. The user can adjust the posture of the ultrasound probe in the operating room to the second posture outside the operating room by sending a control instruction carrying a third posture to the robotic arm through the control device. After acquiring an ultrasound image of the patient through the ultrasound probe, the puncture path for the patient can be determined based on the ultrasound image, and then the needle insertion device can be controlled based on the puncture path to puncture the patient.

Please refer to Figure 4, which is a schematic diagram of a scenario of puncture based on an ultrasound probe provided in an embodiment of the present application. As shown in Figure 4, the scenario includes an ultrasound probe, a mechanical arm, and a needle insertion device, wherein the ultrasound probe and the needle insertion device are both fixed on the mechanical arm. The ultrasound probe is used to obtain an ultrasound image of the patient by scanning and collecting the patient, and the needle insertion device is used to control the puncture needle to perform puncture.

As shown in FIG4 , the scanning area of the ultrasonic probe is a fan-shaped area. When the ultrasonic probe scans the patient, the information in the scanning area can be imaged to obtain an ultrasonic image. After obtaining the ultrasonic image, the needle insertion point and the patient's lesion can be determined from the ultrasonic image. Then the position of the needle insertion point in the ultrasonic image is converted to the position in the world coordinate system, and the position of the lesion in the ultrasonic image is converted to the position in the world coordinate system, wherein O ₂ X ₂ Y ₂ Z ₂ shown in FIG4 is the world coordinate system, specifically, O ₂ is the origin of the world coordinate system, X ₂ is the horizontal axis of the world coordinate system, Y ₂ is the vertical axis of the world coordinate system, and Z ₂ is the vertical axis of the world coordinate system. Based on the position of the needle insertion point in the world coordinate system and the position of the lesion in the world coordinate system, the puncture path for puncturing the patient's lesion can be determined. Then, based on the puncture path, the mechanical arm is controlled to drive the needle insertion device to move, so that the needle insertion device controls the puncture needle to puncture along the puncture path.

As an optional implementation, the first posture is the posture at the first moment, and the control device obtains the first adjustment amount of the posture of the image acquisition device by executing the following steps: obtaining the fourth posture of the control device at the first moment, the control device is used to remotely adjust the posture of the image acquisition device; obtaining the fifth posture of the control device at the second moment, the second moment being a moment later than the first moment; based on the difference between the fifth posture and the fourth posture, obtaining the first adjustment amount.

In this embodiment, the control device is used to remotely adjust the posture of the image acquisition device, that is, the control device can control the posture of the image acquisition device, for example, the control device is the joystick in Figure 2. Optionally, the control device includes the control device. The first moment is the initial moment of adjusting the posture of the image acquisition device by the control device, and the second moment is the termination moment of adjusting the posture of the image acquisition device by the control device, wherein the first posture is the posture of the image acquisition device at the first moment, that is, at the first moment, the posture of the image acquisition device in the base coordinate system is the first posture. In a possible scenario, at the first moment, the posture of the image acquisition device in the base coordinate system of the manipulator is the first posture, and the posture of the control device is the fourth posture. It is used to adjust the posture of the control device from the first moment (for example, move the joystick) so that the posture of the control device at the second moment is the second posture. The control device converts the change amount of the posture of the control device from the first moment to the second moment into a first adjustment amount, and adjusts the posture of the image acquisition device based on the first adjustment amount.

Optionally, the fourth posture and the fifth posture are both postures of the manipulation device in the manipulation coordinate system, wherein the manipulation coordinate system is a coordinate system constructed based on the manipulation device. The control device uses the second transformation relationship to convert the fourth posture into a sixth posture in the base coordinate system, and uses the second transformation to convert the fifth posture into a seventh posture in the base coordinate system. Then, based on the difference between the seventh posture and the sixth posture, the first adjustment amount is obtained.

As an optional embodiment, the above-mentioned first adjustment amount includes a first rotation amount and a first translation amount. The control device performs the following steps during the execution of the step of "using the above-mentioned first transformation relationship to convert the above-mentioned second posture into the third posture of the above-mentioned reference structure in the above-mentioned base coordinate system": decomposing the above-mentioned second posture into a second rotation amount and a second translation amount; adjusting the above-mentioned second rotation amount using the above-mentioned first rotation amount to obtain a third rotation amount; adjusting the above-mentioned second translation amount using the above-mentioned first translation amount to obtain a third translation amount; and obtaining the above-mentioned third posture based on the above-mentioned third rotation amount and the above-mentioned third translation amount.

In this embodiment, since the first adjustment amount includes the first rotation amount and the first translation amount, correspondingly, when the control device adjusts the posture of the image acquisition device based on the first adjustment amount, it can be achieved by rotating the image acquisition device and translating the image acquisition device. Therefore, the control device first decomposes the second posture into the second rotation amount and the second translation amount, and then uses the first rotation amount and the first translation amount to adjust the rotation component and the translation component of the second posture respectively to obtain the third posture.

As an optional embodiment, the base coordinate system includes a horizontal axis, a vertical axis and a vertical axis, and the first rotation amount includes a first horizontal rotation amount, a first vertical rotation amount and a first vertical rotation amount. The first horizontal rotation amount is a component of the first rotation amount on the horizontal axis, the first vertical rotation amount is a component of the first rotation amount on the vertical axis, and the first vertical rotation amount is a component of the first rotation amount on the vertical axis.

The control device performs the following steps during the process of executing the step "adjusting the second rotation amount by using the first rotation amount to obtain a third rotation amount": determining the component of the second rotation amount on the horizontal axis to obtain a second horizontal rotation amount. Determining the component of the second rotation amount on the vertical axis to obtain a second vertical rotation amount. Determining the component of the second rotation amount on the vertical axis to obtain a second vertical rotation amount. Adjusting the second horizontal rotation amount by using the first horizontal rotation amount to obtain a third horizontal rotation amount. Adjusting the second vertical rotation amount by using the first vertical rotation amount to obtain a third vertical rotation amount. Adjusting the second vertical rotation amount by using the first vertical rotation amount to obtain a third vertical rotation amount. Based on the third horizontal rotation amount, the third vertical rotation amount and the third vertical rotation amount, the third rotation amount is obtained.

In this embodiment, the first rotation amount includes components on each coordinate axis of the base coordinate system (i.e., the first horizontal rotation amount, the first vertical rotation amount, and the first vertical rotation amount). Through the above steps, the control device decomposes the second rotation amount into components on each coordinate axis of the base coordinate system to obtain the second horizontal rotation amount, the second vertical rotation amount, and the second vertical rotation amount. Then, using the components of the first rotation amount on each coordinate axis, the components on the corresponding coordinate axis are adjusted to obtain the adjusted components on each coordinate axis (i.e., the third horizontal rotation amount, the third vertical rotation amount, and the third vertical rotation amount). Finally, based on the adjusted components on each coordinate axis, the adjusted rotation amount (i.e., the third rotation amount) can be obtained.

Optionally, the horizontal axis of the base coordinate system is expressed as , the vertical axis of the base coordinate system is expressed as , the vertical axis of the base coordinate system is expressed as ,in, It is represented as (1, 0, 0), It is represented as (0, 1, 0), Represented as (0, 0, 1). The control device is By multiplying the rotation amount, the component of the rotation amount on the horizontal axis can be obtained. For example, by multiplying the second rotation amount by (1, 0, 0), the second horizontal rotation amount can be obtained. By multiplying the rotation amount, the component of the rotation amount on the vertical axis can be obtained. Multiplying it by the amount of rotation gives the component of the rotation on the vertical axis.

Based on the remote image acquisition method provided above, the embodiment of the present application also provides a possible application scenario. Specifically, the image acquisition device can be an ultrasonic probe, which is used to scan the patient to obtain the ultrasonic image of the patient, so as to further determine the puncture path for puncturing the lesion in the patient's body based on the ultrasonic image. In order to avoid the aorta and the great vein in the patient's body when planning the puncture path. When the patient is scanned by the ultrasonic probe, a contrast agent is injected into the patient. In this way, if the ultrasonic probe scans and obtains the ultrasonic image of the patient within the development time of the contrast agent, then in the ultrasonic image, the aorta and the great vein are in a development state, that is, in the ultrasonic image, the aorta and the great vein can be highlighted. If the ultrasonic probe does not scan and obtain the ultrasonic image of the patient within the development time of the contrast agent, then in the ultrasonic image, the aorta and the great vein are not in a development state, that is, in the ultrasonic image, the aorta and the great vein may not be highlighted.

Based on the remote image acquisition method provided above, the control device controls the mechanical arm to drive the ultrasound probe to move, so that the ultrasound probe scans the patient multiple times, and multiple puncture ultrasound images of the patient can be obtained. When multiple puncture ultrasound images are displayed, the doctor can determine the lesion and the needle insertion point based on the multiple puncture ultrasound images, and further determine the puncture path based on the lesion and the needle insertion point.

Since different ultrasound images are obtained by scanning the patient at different positions and different angles, the lesion information contained in different ultrasound images is different, wherein the lesion information includes information used to determine the lesion. For example, if the lesion is a lesion in the lung, then the lesion information includes the tissue of the lung. For another example, if the lesion is located in the renal pelvis of the kidney, then the lesion information includes the renal pelvis. Obviously, the richer and clearer the lesion information contained in the ultrasound image, the higher the accuracy of the lesion determined based on the ultrasound image, and thus the higher the accuracy of the puncture path determined based on the lesion. Therefore, before scanning the patient with an ultrasound probe, the puncture posture of the ultrasound probe can be determined, wherein, when the posture of the ultrasound probe is the puncture posture, the ultrasound image obtained by scanning the ultrasound probe contains an ultrasound image whose information amount and clarity of the lesion information meet the preset requirements, for example, the information amount of the lesion information is greater than or equal to the information amount threshold, and the clarity of the lesion information is greater than or equal to the clarity threshold. Then the doctor can determine the lesion and the needle insertion point based on the puncture ultrasound image. Optionally, before scanning the patient, the ultrasound probe can be controlled by a remote image acquisition method to scan the test object in different postures to obtain multiple test ultrasound images. Then, based on the amount of information and clarity of the lesion information contained in the ultrasound image, a reference ultrasound image can be determined from the multiple ultrasound images, wherein the reference ultrasound image is an ultrasound image whose amount of information and clarity of the lesion information meet preset requirements, for example, the amount of information of the lesion information is greater than or equal to the information amount threshold, and the clarity of the lesion information is greater than or equal to the clarity threshold. Finally, the posture of the ultrasound probe when acquiring the reference ultrasound image is determined to be the puncture posture.

After the puncture position is determined, the ultrasound probe can be controlled to move through the remote image acquisition method provided above, so that the ultrasound probe is in the puncture position. Then the ultrasound probe can be controlled to scan the patient in the puncture position to obtain multiple puncture ultrasound images. The doctor can then determine the lesion and the needle insertion point based on the multiple puncture ultrasound images.

Considering the limited development time of contrast agent, multiple puncture ultrasound images include puncture ultrasound images acquired outside the development time of contrast agent. When the puncture ultrasound images are acquired outside the development time, the aorta and the great vein are not highlighted in the puncture ultrasound images, which may cause the doctor to ignore the aorta and the great vein when determining the lesion and the needle insertion point, and then cause the puncture path passing through the lesion and the needle insertion point to pass through the aorta or the great vein, which may cause the patient to be in fatal danger.

Since multiple puncture ultrasound images are all acquired by the ultrasound probe in the puncture posture, and the patient's position is usually unchanged during the process of the ultrasound probe acquiring multiple puncture ultrasound images. Therefore, the positions of the aorta and the great vein are the same in each puncture ultrasound image. Based on this, the control device first determines the developed ultrasound image whose acquisition time is within the development time of the contrast agent from the multiple ultrasound images. Then the position of the aorta and the great vein in the developed ultrasound image is determined as the developed position. Finally, for the non-developed ultrasound images other than the developed ultrasound images in the multiple ultrasound images, the area with the developed position is highlighted. In this way, the doctor determines the lesion and the needle insertion point based on the developed ultrasound images and the non-developed ultrasound images in the multiple puncture ultrasound images, which can reduce the probability that the puncture path determined based on the lesion and the needle insertion point passes through the aorta or the great vein.

After the lesion and the needle insertion point are determined from the puncture ultrasound image, the position of the lesion in the puncture ultrasound image is converted to a position in the world coordinate system based on the third conversion relationship, and the position of the needle insertion point in the puncture ultrasound image is converted to a position in the world coordinate system. Finally, the puncture path in the world coordinate system can be determined based on the position of the lesion in the world coordinate system and the position of the needle insertion point in the world coordinate system.

Those skilled in the art will appreciate that, in the above method of specific implementation, the order in which the steps are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of the steps should be determined by their functions and possible internal logic.

If the technical solution of this application involves personal information, the product using the technical solution of this application has clearly informed the personal information processing rules and obtained the individual's voluntary consent before processing the personal information. If the technical solution of this application involves sensitive personal information, the product using the technical solution of this application has obtained the individual's separate consent before processing the sensitive personal information, and at the same time meets the "explicit consent" requirement. For example, on personal information collection devices such as cameras, clear and prominent signs are set to inform that the personal information collection scope has been entered and personal information will be collected. If the individual voluntarily enters the collection scope, it is deemed that he or she agrees to the collection of his or her personal information; or on the device that processes personal information, the personal information processing rules are notified by obvious signs/information, and the individual's authorization is obtained through pop-up information or by asking the individual to upload his or her personal information; among which, personal information processing may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the type of personal information processed.

The method of the embodiment of the present application is described in detail above, and the device of the embodiment of the present application is provided below.

Please refer to FIG5 , which is a schematic diagram of the structure of a control device provided in an embodiment of the present application. The control device 1 is used to remotely control an image acquisition device to acquire images. The image acquisition device is fixed on a mechanical arm. The control device 1 drives the image acquisition device to move by controlling the movement of the mechanical arm. The control device 1 includes: an acquisition unit 11, an adjustment unit 12, and a conversion unit 13. Optionally, the control device 1 also includes: a control unit 14, a receiving unit 15, and a determination unit 16. Specifically:

An acquisition unit 11 is used to acquire a first pose of the image acquisition device in a base coordinate system of the robotic arm, where the base coordinate system is a coordinate system constructed based on a base of the robotic arm;

The acquisition unit 11 is further used to acquire a first adjustment amount of the posture of the image acquisition device;

An adjusting unit 12, configured to adjust the first posture to obtain a second posture of the image acquisition device in the base coordinate system based on the first adjustment amount;

The acquisition unit 11 is further used to acquire a first conversion relationship, where the first conversion relationship is a conversion relationship between the posture of the image acquisition device and the posture of a reference structure, where the reference structure includes a structure on the robotic arm except the base;

The conversion unit 13 is used to convert the second posture into a third posture of the reference structure in the base coordinate system by using the first conversion relationship.

In combination with any embodiment of the present application, the control device 1 further includes:

The control unit 14 is used to control the movement of the robot arm through the third posture so that the posture of the reference structure is the third posture.

In combination with any embodiment of the present application, the image captured by the image acquisition device includes an ultrasonic image, and the control device 1 further includes: a receiving unit 15, configured to receive a target ultrasonic image captured by the image acquisition device in the third posture, wherein the target ultrasonic image includes an object to be punctured;

The determination unit 16 is configured to determine a puncture path of the object to be punctured based on the target ultrasound image.

In combination with any embodiment of the present application, the acquisition unit 11 is used to:

Acquire a fourth posture of the control device at the first moment, the control device being used to remotely adjust the posture of the image acquisition device;

Acquire a fifth posture of the control device at a second moment, where the second moment is a moment later than the first moment;

The first adjustment amount is obtained based on a difference between the fifth posture and the fourth posture.

In combination with any embodiment of the present application, the first adjustment amount includes a first rotation amount and a first translation amount, and the conversion unit 13 is used to:

Decomposing the second posture into a second rotation and a second translation;

adjusting the second rotation amount by using the first rotation amount to obtain a third rotation amount;

adjusting the second translation amount by using the first translation amount to obtain a third translation amount;

The third posture is obtained based on the third rotation amount and the third translation amount.

In combination with any embodiment of the present application, the base coordinate system includes a horizontal axis, a vertical axis and a vertical axis, the first rotation amount includes a first horizontal rotation amount, a first vertical rotation amount and a first vertical rotation amount, the first horizontal rotation amount is a component of the first rotation amount on the horizontal axis, the first vertical rotation amount is a component of the first rotation amount on the vertical axis, and the first vertical rotation amount is a component of the first rotation amount on the vertical axis;

The conversion unit is used for:

Determine the component of the second rotation amount on the horizontal axis to obtain a second horizontal rotation amount;

Determine the component of the second rotation amount on the longitudinal axis to obtain a second longitudinal rotation amount;

Determine the component of the second rotation amount on the vertical axis to obtain a second vertical rotation amount;

adjusting the second lateral rotation amount by using the first lateral rotation amount to obtain a third lateral rotation amount;

adjusting the second longitudinal rotation amount by using the first longitudinal rotation amount to obtain a third longitudinal rotation amount;

adjusting the second vertical rotation amount by using the first vertical rotation amount to obtain a third vertical rotation amount;

The third rotation amount is obtained based on the third lateral rotation amount, the third longitudinal rotation amount, and the third vertical rotation amount.

In an embodiment of the present application, the image acquisition device is fixed on the robotic arm, wherein the control device drives the image acquisition device to move by controlling the movement of the robotic arm. After the control device obtains the first posture of the image acquisition device in the base coordinate system of the robotic arm, it obtains the first adjustment amount of the posture of the image acquisition device. Then, based on the first adjustment amount, the first posture is adjusted to obtain the second posture of the image acquisition device in the base coordinate system. Then, the first conversion relationship is obtained, wherein the first conversion relationship is the conversion relationship between the posture of the image acquisition device and the posture of the reference structure, and the reference structure is the structure of the robotic arm. Finally, the second posture is converted into the third posture of the reference structure in the base coordinate system using the first conversion relationship, so that the first adjustment amount of the posture of the image acquisition device can be converted into the posture of the reference structure of the robotic arm.

In some embodiments, the functions or modules included in the device provided in the embodiments of the present application can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments. For the sake of brevity, it will not be repeated here.

FIG6 is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application. The electronic device 2 includes a processor 21 and a memory 22. Optionally, the electronic device 2 also includes an input device 23 and an output device 24. The processor 21, the memory 22, the input device 23 and the output device 24 are coupled via a connector, and the connector includes various interfaces, transmission lines or buses, etc., which are not limited in the embodiments of the present application. It should be understood that in each embodiment of the present application, coupling refers to mutual connection in a specific manner, including direct connection or indirect connection through other devices, for example, it can be connected through various interfaces, transmission lines, buses, etc.

The processor 21 may be one or more graphics processing units (GPUs). When the processor 21 is a GPU, the GPU may be a single-core GPU or a multi-core GPU. Optionally, the processor 21 may be a processor group consisting of multiple GPUs, and the multiple processors are coupled to each other via one or more buses. Optionally, the processor may also be other types of processors, etc., which are not limited in the embodiments of the present application.

The memory 22 can be used to store computer program instructions and various computer program codes including the program code for executing the program code of the present application. Optionally, the memory includes but is not limited to random access memory (RAM), read-only memory (ROM), erasable programmable read only memory (EPROM), or portable read only memory (CD-ROM), which is used for related instructions and data.

The input device 23 is used to input data and/or signals, and the output device 24 is used to output data and/or signals. The input device 23 and the output device 24 can be independent devices or an integrated device.

It can be understood that in the embodiment of the present application, the memory 22 can be used not only to store relevant instructions, but also to store relevant data. The embodiment of the present application does not limit the specific data stored in the memory.

It is understandable that FIG6 merely shows a simplified design of an electronic device. In practical applications, the electronic device may further include other necessary components, including but not limited to any number of input/output devices, processors, memories, etc., and all electronic devices that can implement the embodiments of the present application are within the protection scope of the present application.

Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. Those skilled in the art can also clearly understand that the descriptions of the various embodiments of the present application have different focuses. For the convenience and brevity of description, the same or similar parts may not be repeated in different embodiments. Therefore, for parts not described or not described in detail in a certain embodiment, refer to the records of other embodiments.

In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions may be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a digital versatile disc (DVD)), or a semiconductor medium (eg, a solid state disk (SSD)).

A person skilled in the art can understand that to implement all or part of the processes in the above-mentioned embodiments, the processes can be completed by a computer program to instruct the relevant hardware, and the program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the above-mentioned method embodiments. The aforementioned storage medium includes: a read-only memory (ROM) or a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

Claims (10)

1. The remote image acquisition method is characterized in that the method is applied to control equipment, the control equipment is used for remotely controlling the image acquisition equipment to acquire images, the image acquisition equipment is fixed on a mechanical arm, and the control equipment drives the image acquisition equipment to move by controlling the mechanical arm to move, and the method comprises the following steps:

acquiring a first pose of the image acquisition equipment under a base coordinate system of the mechanical arm, wherein the base coordinate system is a coordinate system constructed by a base of the mechanical arm;

Acquiring a first adjustment amount of the pose of the image acquisition equipment;

Based on the first adjustment amount, adjusting the first pose to obtain a second pose of the image acquisition equipment under the base coordinate system;

acquiring a first conversion relation, wherein the first conversion relation is a conversion relation between the pose of the image acquisition equipment and the pose of a reference structure, and the reference structure is a structure of a mechanical arm;

and converting the second pose into a third pose of the reference structure under the base coordinate system by utilizing the first conversion relation.

2. The method of claim 1, wherein after converting the second pose to a third pose of the reference structure in the base coordinate system using the first conversion relationship, the method further comprises:

And controlling the mechanical arm to move through the third pose, so that the pose of the reference structure is the third pose.

3. The method of claim 2, wherein the image acquired by the image acquisition device comprises an ultrasound image, and wherein after controlling the robotic arm motion by the third pose to pose the reference structure to the third pose, the method comprises:

receiving a target ultrasonic image acquired by the image acquisition equipment under the third pose, wherein the target ultrasonic image comprises an object to be punctured;

and determining a puncture path of the object to be punctured based on the target ultrasonic image.

4. A method according to any one of claims 1 to 3, wherein the first pose is a pose at a first moment in time, and the obtaining a first adjustment of the pose of the image capturing device comprises:

acquiring a fourth pose of control equipment at the first moment, wherein the control equipment is used for remotely adjusting the pose of the image acquisition equipment;

Acquiring a fifth pose of the control equipment at a second moment, wherein the second moment is a moment later than the first moment;

and obtaining the first adjustment amount based on the difference between the fifth pose and the fourth pose.

5. A method according to any one of claims 1 to 3, wherein the first adjustment comprises a first rotation amount and a first translation amount, the converting the second pose into a third pose of the reference structure in the base coordinate system using the first conversion relationship comprising:

decomposing the second pose into a second rotation amount and a second translation amount;

Adjusting the second rotation amount by using the first rotation amount to obtain a third rotation amount;

adjusting the second translation amount by using the first translation amount to obtain a third translation amount;

and obtaining the third pose based on the third rotation amount and the third translation amount.

6. The method of claim 5, wherein the base coordinate system comprises a lateral axis, a longitudinal axis, and a vertical axis, the first rotation amount comprising a first lateral rotation amount, a first longitudinal rotation amount, a first vertical rotation amount, the first lateral rotation amount being a component of the first rotation amount on the lateral axis, the first longitudinal rotation amount being a component of the first rotation amount on the longitudinal axis, the first vertical rotation amount being a component of the first rotation amount on the vertical axis;

The adjusting the second rotation amount by using the first rotation amount to obtain a third rotation amount includes:

determining the component of the second rotation quantity on the transverse axis to obtain a second transverse rotation quantity;

Determining a component of the second rotation amount on the longitudinal axis to obtain a second longitudinal rotation amount;

determining the component of the second rotation quantity on the vertical shaft to obtain a second vertical rotation quantity;

The second transverse rotation amount is adjusted by utilizing the first transverse rotation amount, and a third transverse rotation amount is obtained;

adjusting the second longitudinal rotation amount by using the first longitudinal rotation amount to obtain a third longitudinal rotation amount;

adjusting the second vertical rotation amount by using the first vertical rotation amount to obtain a third vertical rotation amount;

the third rotation amount is obtained based on the third lateral rotation amount, the third longitudinal rotation amount, and the third vertical rotation amount.

7. The utility model provides a control equipment, its characterized in that, control equipment is used for remote control image acquisition equipment gathers the image, image acquisition equipment is fixed on the arm, control equipment is through control the arm motion drives image acquisition equipment moves, control equipment includes:

An acquisition unit, configured to acquire a first pose of the image acquisition device under a base coordinate system of the mechanical arm, where the base coordinate system is a coordinate system constructed with a base of the mechanical arm;

the acquisition unit is also used for acquiring a first adjustment amount of the pose of the image acquisition equipment;

the adjusting unit is used for adjusting the first pose based on the first adjustment amount to obtain a second pose of the image acquisition equipment under the base coordinate system;

The acquisition unit is further configured to acquire a first conversion relationship, where the first conversion relationship is a conversion relationship between a pose of the image acquisition device and a pose of a reference structure, and the reference structure includes a structure on the mechanical arm except the base;

and the conversion unit is used for converting the second pose into a third pose of the reference structure under the base coordinate system by utilizing the first conversion relation.

8. An electronic device comprising a processor and a memory for storing computer program code, the computer program code comprising computer instructions, which, when executed by the processor, cause the electronic device to perform the method of any one of claims 1 to 6.

9. A computer readable storage medium, characterized in that the computer readable storage medium has stored therein a computer program comprising program instructions which, when executed by a processor, cause the processor to perform the method of any of claims 1 to 6.

10. A computer program product, characterized in that the computer program product comprises a computer program which, when run on a computer, causes the computer to carry out the method according to any one of claims 1 to 6.