CN114404047B

CN114404047B - Positioning method, system, device, computer equipment and storage medium

Info

Publication number: CN114404047B
Application number: CN202111604601.XA
Authority: CN
Inventors: 请求不公布姓名; 何超; 李涛; 于海英
Original assignee: Suzhou Xiaowei Changxing Robot Co ltd
Current assignee: Suzhou Xiaowei Changxing Robot Co ltd
Priority date: 2021-12-24
Filing date: 2021-12-24
Publication date: 2024-06-14
Anticipated expiration: 2041-12-24
Also published as: CN114404047A; WO2023116823A1

Abstract

The application relates to a positioning method, a positioning system, a positioning device, computer equipment and a storage medium. The method comprises the following steps: planning an osteotomy face according to image information of a target part to obtain at least one osteotomy face of the target part; determining a target osteotomy plane from the osteotomy planes according to the pose relation between an osteotomy guide block device arranged at the tail end of the mechanical arm and each osteotomy plane in the target part; when the pose relation between the osteotomy guide block device and the target osteotomy face meets the positioning condition, the mechanical arm is controlled to position the osteotomy guide block device to the target osteotomy face in an automatic mode. The method can improve the positioning efficiency of the osteotomy face.

Description

Positioning method, system, device, computer equipment and storage medium

Technical Field

The present application relates to the field of robotics, and in particular, to a positioning method, system, apparatus, computer device, and storage medium.

Background

With the progress of science and technology, robots can be applied to various fields to assist people in carrying out operations.

For example: in the medical field, a doctor can be assisted by a robot to perform related operations such as diagnosis and treatment. Taking an orthopedic operation as an example, in the orthopedic operation process, a doctor can be assisted by an orthopedic operation robot to perform the operation. In the related art, in the bone cutting process of the bone surgery robot, after a doctor is required to plan an bone cutting surface to be positioned, a command operator adopts a cooperative mode to roughly position a mechanical arm to the bone cutting surface, and then adopts an automatic mode to enable the mechanical arm to execute automatic fine positioning and bone cutting operation, and under the condition that a plurality of bone cutting surfaces exist, the process is required to be repeated for each bone cutting surface until all the bone cutting surfaces are subjected to bone cutting operation.

The mechanical arm is used for positioning the osteotomy face, and the positioning process is complicated, so that the positioning efficiency is low.

Disclosure of Invention

In view of the foregoing, it is desirable to provide a positioning method, system, apparatus, computer device, and storage medium that can improve the positioning efficiency of a robot arm.

In a first aspect, the present application provides a positioning method, the method comprising:

Planning an osteotomy face according to image information of a target part to obtain at least one osteotomy face of the target part;

Determining a target osteotomy plane from the osteotomy planes according to the pose relation between an osteotomy guide block device arranged at the tail end of the mechanical arm and each osteotomy plane in the target part;

When the pose relation between the osteotomy guide block device and the target osteotomy face meets the positioning condition, the mechanical arm is controlled to position the osteotomy guide block device to the target osteotomy face in an automatic mode.

In one embodiment, the planning the osteotomy plane according to the image information of the target site, to obtain at least one osteotomy plane of the target site, includes:

Determining characteristic point information of a target part from image information of the target part;

and determining at least one osteotomy face of the target part according to the characteristic point information.

In one embodiment, the determining at least one osteotomy plane of the target site according to the feature point information includes:

Determining at least one initial osteotomy face of the target part according to the characteristic point information;

And adjusting each initial osteotomy plane according to the prosthesis device of the target part to obtain at least one osteotomy plane of the target part.

In one embodiment, the determining the target osteotomy plane from the osteotomy planes according to the pose relationship between the osteotomy guide block device mounted at the tail end of the mechanical arm and each of the osteotomy planes in the target site comprises:

Acquiring real-time image information of the target part through an image acquisition device;

according to the real-time image information and each osteotomy face, determining an osteotomy area aiming at the target part;

When the osteotomy guide block device mounted at the tail end of the mechanical arm is detected to move into the osteotomy region, determining the pose relationship between the osteotomy guide block device and each osteotomy face;

And determining a target osteotomy face from the osteotomy faces according to the pose relation between the osteotomy guide block device and each osteotomy face, wherein the pose relation is used for representing the distance between the osteotomy guide block device and the osteotomy faces and the presented angle relation.

In one embodiment, determining the osteotomy region of the target site according to the real-time image information and each osteotomy face includes:

Generating a corresponding initial osteotomy region according to the target part and each osteotomy surface;

according to the real-time image information of the target part, after the space registration is carried out on the target part and a three-dimensional model of the target part, the three-dimensional model is displayed in a display interface, wherein the three-dimensional model comprises the initial osteotomy region and each osteotomy surface, and each osteotomy surface is positioned in the initial osteotomy region;

And responding to the adjustment operation for the initial osteotomy region, and obtaining the osteotomy region of the target part.

In one embodiment, when the movement of the osteotomy guide block device mounted at the tail end of the mechanical arm into the osteotomy region is detected, determining the pose relationship of the osteotomy guide block device and each osteotomy face comprises:

acquiring section pose information of the section of the osteotomy guide block device installed at the tail end of the mechanical arm in real time under the coordinate system of image acquisition equipment;

determining whether the osteotomy guide block device moves into the osteotomy region according to the section pose information of the osteotomy guide block device;

When the osteotomy guide block device is determined to move into the osteotomy guide block region, determining the pose relationship between the osteotomy guide block device and each osteotomy face according to the section pose information of the osteotomy guide block device and the pose information of each osteotomy face.

In one embodiment, when the osteotomy guide device is determined to move into the osteotomy guide region, determining the pose relationship between the osteotomy guide device and each osteotomy face according to the cross-sectional pose information of the osteotomy guide device and the pose information of each osteotomy face, including:

When the osteotomy guide block device moves into the osteotomy guide block area, detecting whether external force control exists on the mechanical arm in real time;

When the mechanical arm is detected to be free from external force control, the pose relation between the osteotomy guide block device and each osteotomy face is determined according to the section pose information of the osteotomy guide block device and the pose information of each osteotomy face.

In one embodiment, the determining the pose relationship between the osteotomy guide block device and each osteotomy face according to the cross-sectional pose information of the osteotomy guide block device and the pose information of each osteotomy face includes:

For any osteotomy face, determining the angle and distance between the osteotomy guide block device and the osteotomy face according to the section pose information of the osteotomy guide block device and the pose information of the osteotomy face;

and determining the pose relation between the osteotomy guide block device and the osteotomy face according to the angle and the distance.

In a second aspect, the present application also provides a positioning system, the system comprising: the device comprises an image acquisition device, a mechanical arm and an osteotomy guide block device arranged at the tail end of the mechanical arm,

The image acquisition device is used for planning an osteotomy face according to the image information of the target part to obtain at least one osteotomy face of the target part;

The image acquisition device is also used for sending pose information of the target osteotomy face to the mechanical arm after determining the target osteotomy face from the osteotomy face according to the pose relation between the osteotomy guide block device and each osteotomy face in the target part;

the mechanical arm is used for positioning the osteotomy guide block device to the target osteotomy face according to pose information of the target osteotomy face in an automatic mode.

In one embodiment, the image acquisition device is further configured to acquire real-time image information of a target portion, determine an osteotomy region for the target portion according to the real-time image information and each osteotomy surface, and determine a target osteotomy surface from the osteotomy surfaces according to a pose relationship between the osteotomy guide block device and each osteotomy surface after determining that the osteotomy guide block device moves to the osteotomy region.

In one embodiment, the image capturing device is further configured to obtain, in real time, cross-sectional pose information of a cross section of the osteotomy guide block device under the coordinate system of the image capturing device, detect whether the osteotomy guide block device moves into the osteotomy region according to the cross-sectional pose information, and determine, when detecting that the osteotomy guide block device moves into the osteotomy guide block region, a pose relationship between the osteotomy guide block device and each osteotomy face according to the cross-sectional pose information of the osteotomy guide block device and the pose information of each osteotomy face.

In one embodiment, the osteotomy guide block device includes a clamping slot, and the system further includes a marking member mounted to the distal end of the robotic arm;

The image acquisition equipment is also used for determining pose information of the osteotomy guide block device under the coordinate system of the image acquisition equipment according to the coordinate conversion matrix of the osteotomy guide block device and the marking component and the pose information of the marking component under the coordinate system of the image acquisition equipment after acquiring the pose information of the marking component under the coordinate system of the image acquisition equipment;

The image acquisition equipment is also used for determining pose information of the cross section of the osteotomy guide block device under the coordinate system of the image acquisition equipment according to the coordinate transformation matrix of the osteotomy guide block device and the clamping groove and the pose information of the osteotomy guide block device under the coordinate system of the image acquisition equipment.

In a third aspect, the present application also provides a positioning device, the device comprising:

the planning module is used for planning the osteotomy face according to the image information of the target part to obtain at least one osteotomy face of the target part;

The determining module is used for determining a target osteotomy face from the osteotomy faces according to the pose relation between the osteotomy guide block device arranged at the tail end of the mechanical arm and each osteotomy face in the target part;

and the positioning module is used for controlling the mechanical arm to position the osteotomy guide block device to the target osteotomy face by adopting an automatic mode when the pose relation between the osteotomy guide block device and the target osteotomy face meets the positioning condition.

In one embodiment, the planning module is further configured to:

In one embodiment, the determining module is further configured to:

In a fourth aspect, the present application also provides a computer device. The computer device comprises a memory storing a computer program and a processor implementing the steps of the above positioning method when the processor executes the computer program.

In a fifth aspect, the present application also provides a computer-readable storage medium. The computer readable storage medium has stored thereon a computer program which, when executed by a processor, implements the steps of the above positioning method.

In a sixth aspect, the application also provides a computer program product. The computer program product comprises a computer program which, when executed by a processor, implements the steps of the above positioning method.

According to the positioning method, the system, the device, the computer equipment and the storage medium, the osteotomy face planning can be carried out according to the image information of the target part, at least one osteotomy face of the target part is obtained, the target osteotomy face is determined from the osteotomy face according to the pose relation between the osteotomy guide block device arranged at the tail end of the mechanical arm and each osteotomy face in the target part, and when the pose relation between the osteotomy guide block device and the target osteotomy face meets the positioning condition, the mechanical arm is controlled to position the osteotomy guide block device to the target osteotomy face in an automatic mode. The positioning method, the system, the device, the computer equipment and the storage medium provided by the embodiment of the application can plan the osteotomy face aiming at the target part before the operation, and select the target osteotomy face to automatically position through the visual positioning control mechanical arm, namely the positioning and the positioning process are all automatic, so that the positioning flow of the mechanical arm in the osteotomy process can be simplified, the positioning efficiency of the osteotomy face can be improved, the labor cost can be reduced, the time consumption of the osteotomy operation can be reduced, and the operation efficiency of the osteotomy operation can be improved.

Drawings

FIG. 1 is a flow chart of a positioning method in one embodiment;

FIG. 2 is a flow chart of a positioning method in one embodiment;

FIGS. 3 a-3 c are schematic diagrams of a positioning method in one embodiment;

FIG. 4 is a flow chart of a positioning method in one embodiment;

FIG. 5 is a schematic diagram of a positioning method in one embodiment;

FIG. 6 is a schematic diagram of a positioning method in one embodiment;

FIG. 7 is a flow chart of a positioning method in one embodiment;

FIG. 8 is a flow chart of a positioning method in one embodiment;

FIG. 9 is a schematic diagram of a positioning method in one embodiment;

FIG. 10 is a schematic diagram of a positioning method in one embodiment;

FIGS. 11 a-11 c are schematic diagrams of a positioning method in one embodiment;

FIG. 12 is a flow diagram of a positioning method in one embodiment;

FIG. 13 is a schematic diagram of a positioning method in one embodiment;

FIG. 14 is a flow diagram of a positioning method in one embodiment;

FIG. 15 is a schematic diagram of a positioning method in one embodiment;

FIG. 16 is a flow diagram of a positioning method in one embodiment;

FIG. 17 is a schematic diagram of a positioning method in one embodiment;

FIG. 18 is a schematic diagram of a positioning method in one embodiment;

FIG. 19 is a schematic diagram of a positioning method in one embodiment;

FIG. 20 is a schematic diagram of a positioning method in one embodiment;

FIG. 21 is a schematic diagram of a positioning method in one embodiment;

FIG. 22 is a schematic diagram of a positioning method in one embodiment;

FIG. 23 is a schematic diagram of a positioning method in one embodiment;

FIG. 24 is a schematic diagram of a positioning system in one embodiment;

FIG. 25 is a schematic diagram of a positioning system in one embodiment;

FIGS. 26 a-26 b are schematic diagrams of a positioning system in one embodiment;

FIGS. 27 a-27 c are schematic diagrams of a positioning system in one embodiment;

FIG. 28 is a schematic view of a positioning system in one embodiment;

FIG. 29 is a block diagram of a positioning device in one embodiment;

fig. 30 is an internal structural view of a computer device in one embodiment.

Detailed Description

The present application will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present application more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the application.

In an embodiment, as shown in fig. 1, a positioning method is provided, where the embodiment is applied to a robot for illustration, it is understood that the method may also be applied to a terminal device for controlling the robot, and may also be applied to a system including a terminal and a server, and implemented through interaction between the terminal and the server. In this embodiment, the method includes the steps of:

Step 102, performing osteotomy plane planning according to the image information of the target part to obtain at least one osteotomy plane of the target part.

In the embodiment of the application, the image information of the target part can be acquired before the osteotomy, and the image information can comprise CT (Computed Tomography, electronic computer tomography) scanning images, AR (Augmented Reality ) visual scanning images and other images. And can carry out the osteotomy plan based on the image information of the target site, plan at least one osteotomy of the target site. For example: at least one osteotomy face of the target site may be generated based on the feature point information of the target site in the image information.

And 104, determining a target osteotomy face from the osteotomy faces according to the pose relation between the osteotomy guide block device arranged at the tail end of the mechanical arm and each osteotomy face in the target part.

In the embodiment of the application, at least one osteotomy face of the target part can be planned and obtained before an osteotomy operation, an assistance mode can be adopted in the osteotomy operation process, and the mechanical arm can be controlled to move towards the osteotomy face of the target part by medical staff, or the mechanical arm can also be controlled to move towards the osteotomy face of the target part in an automatic mode. In the moving process of the mechanical arm, the pose information of the osteotomy guide block device arranged at the tail end of the mechanical arm can be detected in real time, and then the pose relation between the osteotomy guide block device and each osteotomy face in the target part can be determined in real time according to the pose information of the osteotomy guide block device arranged at the tail end of the mechanical arm, wherein the pose relation can comprise a distance relation and an angle relation formed by the cross section.

After determining the pose relationship of the osteotomy guide device mounted at the distal end of the robotic arm with each of the osteotomy faces in the target site, a target osteotomy face that best matches the pose relationship of the osteotomy guide device may be selected from the osteotomy faces, and illustratively, the osteotomy face that is closest to and/or at the smallest angle than the osteotomy guide device may be determined as the target osteotomy face that best matches the pose relationship of the osteotomy guide device.

And 106, when the pose relation between the osteotomy guiding block device and the target osteotomy face meets the positioning condition, the automatic mode is adopted to control the mechanical arm to position the osteotomy guiding block device to the target osteotomy face.

In the embodiment of the application, after the target osteotomy face is determined, the pose relation between the osteotomy guide block device and the target osteotomy face can be detected in real time through the image acquisition device, and under the condition that the pose relation between the osteotomy guide block device and the target osteotomy face meets the preset positioning condition, the pose information of the target osteotomy face can be sent to the mechanical arm, and the mechanical arm is controlled to perform fine positioning by adopting an automatic mode, so that the osteotomy guide block device is automatically positioned to the target osteotomy face according to the pose information of the target osteotomy face. Wherein the positioning condition may include that the distance and/or angle of the osteotomy guide device from the target osteotomy face is less than or equal to a preset distance threshold and/or angle threshold.

Illustratively, after the osteotomy guide device is positioned to the target osteotomy face, the robotic arm may be placed in a hold state, and a medical professional may install a pendulum saw or electric drill using the osteotomy guide slot and/or guide hole on the osteotomy guide device to perform the osteotomy and drilling operations.

According to the positioning method provided by the embodiment of the application, the osteotomy face planning can be carried out according to the image information of the target part, at least one osteotomy face of the target part is obtained, the target osteotomy face is determined from the osteotomy faces according to the pose relation between the osteotomy guide block device arranged at the tail end of the mechanical arm and each osteotomy face in the target part, and when the pose relation between the osteotomy guide block device and the target osteotomy face meets the positioning condition, the mechanical arm is controlled to position the osteotomy guide block device to the target osteotomy face by adopting an automatic mode. The positioning method provided by the embodiment of the application can plan the osteotomy face aiming at the target part before the operation, and automatically position the target osteotomy face by the vision positioning control mechanical arm, namely, the positioning and positioning processes are all automatic, so that the positioning process of the mechanical arm in the osteotomy process can be simplified, the positioning efficiency of the osteotomy face can be improved, the labor cost can be reduced, the time consumption of the osteotomy operation can be reduced, and the operation efficiency of the osteotomy operation can be improved.

In one embodiment, referring to fig. 2, in step 102, performing osteotomy plane planning according to image information of a target site to obtain at least one osteotomy plane of the target site includes:

step 202, determining characteristic point information of a target part from image information of the target part;

Step 204, determining at least one osteotomy face of the target part according to the feature point information.

In the embodiment of the application, the image information of the target part can be acquired before the osteotomy, the characteristic point information of the target part is determined in the image information of the target part in a mode of image recognition or manual marking of medical staff, and then at least one osteotomy face of the target part is determined in the image information of the target part according to the characteristic point information of the target part. For example, the feature point information for generating each osteotomy face of the target portion may be preset, and after identifying the feature point information corresponding to each osteotomy face, the corresponding osteotomy face may be generated based on the feature point information.

For example, taking knee joint replacement surgery as an example, after image information of tibia, femur and knee joint is obtained through an image device (such as a CT scanning device and an AR vision scanning device), a medical staff can manually divide a target area of the image information to obtain a target area in an image. Further, knee joint feature points (including a femur feature point including a femur rotation center, a knee joint center (femur force line), a lateral condyle highest point, a medial intercondylar notch lowest point (intercondylar line (TEA)), a intercondylar notch highest point, a trochlear lowest point (Whiteside line), a greater trochanter cusp, a intercondylar notch midpoint (femur anatomical axis), a medial condyle distal tangent point, a lateral condyle distal tangent point (femur distal axis), a posterior condyle medial tangent point, a posterior condyle lateral tangent point (posterior condyle axis)) are obtained from the target region.

By way of example, referring to fig. 3 a-3 c, through the characteristic points lateral condyle distal tangent point 301, medial condyle distal tangent point 302, and knee center 303, the femoral distal axis and femoral power line 304 may be identified, through the posterior condyle lateral tangent point 305 and posterior condyle medial tangent point 306, the posterior condyle axis may be identified, and a corresponding osteotomy plane may be generated based on the femoral distal axis, the femoral power line, and the posterior condyle axis. Illustratively, taking the distal femur section and the femoral canal as examples, the distal femur section is as perpendicular as possible to the femoral canal.

According to the positioning method provided by the embodiment of the application, before an osteotomy, the osteotomy surface can be planned according to the image information of the target part, so that at least one osteotomy surface of the target part is obtained, in the osteotomy process, the target osteotomy surface is selected by the vision positioning control mechanical arm to be automatically positioned, namely the positioning and positioning processes are all automated, the positioning process of the mechanical arm in the osteotomy process can be simplified, the positioning efficiency of the osteotomy surface is improved, the labor cost is reduced, the time consumption of the osteotomy is reduced, and the operation efficiency of the osteotomy is improved.

In one embodiment, referring to fig. 4, in step 204, determining at least one osteotomy plane of the target site based on the feature point information, comprises:

step 402, determining at least one initial osteotomy surface of the target part according to the characteristic point information;

Step 404, adjusting each initial osteotomy plane according to the prosthetic device of the target site, to obtain at least one osteotomy plane of the target site.

In the embodiment of the application, initial planning can be performed based on the characteristic point information of the target part, at least one initial osteotomy face of the target part is determined, and the initial osteotomy faces can be adjusted based on the selected prosthesis device of the target part, including the position, angle and other information of the initial osteotomy faces are adjusted, so that at least one osteotomy face of the target part is obtained.

The prosthetic device may be manually selected by a medical staff, or may also be automatically selected according to the osteotomy face or the feature point information, and the adjustment for the initial osteotomy face may be manually adjusted by the medical staff, or may also be automatically adjusted according to the osteotomy face information corresponding to the prosthetic device, which is not particularly limited in the embodiment of the present application.

For example, referring to fig. 5, knee joint image information may be acquired, and planning of the target region may be performed based on the knee joint image information, to obtain the target region. And selecting characteristic points of a target area in the knee joint image information to obtain characteristic point information of the knee joint image information. After the model of the prosthesis corresponding to the knee joint is selected, at least one initial osteotomy face of the knee joint can be generated based on the feature point information, and each initial osteotomy face is adjusted based on the model number of the prosthesis, so that each osteotomy face of the knee joint is obtained. For example, referring to fig. 6, each osteotomy plane may be shown as 601 being the position of the anterior resection surface of the distal femur, 602 being the position of the anterior resection surface of the distal femur, 603 being the position of the posterior resection surface of the distal femur, 604 being the position of the posterior resection surface of the distal femur, and 605 being the position of the distal resection surface of the femur.

According to the positioning method provided by the embodiment of the application, before an osteotomy, the osteotomy surface can be planned according to the image information of the target part, at least one initial osteotomy surface of the target part is obtained, and the initial osteotomy surface is adjusted based on the prosthesis device, so that the osteotomy surface of the target part is accurately obtained, and further, in the osteotomy process, the target osteotomy surface is automatically selected from the planned osteotomy surface by the vision positioning control mechanical arm to be automatically positioned, namely, the positioning and positioning processes are all automatic, so that the positioning flow of the mechanical arm in the osteotomy process can be simplified, the positioning efficiency of the osteotomy surface can be improved, the labor cost can be reduced, the time consumption of the osteotomy can be reduced, and the operation efficiency of the osteotomy can be improved.

In one embodiment, referring to fig. 7, in step 104, determining a target osteotomy surface from the osteotomy surfaces based on the pose relationship of the robotic arm tip mounted osteotomy guide block device to each osteotomy surface in the target site, comprises:

Step 702, acquiring real-time image information of the target part through an image acquisition device;

step 704, determining an osteotomy region of the target part according to the real-time image information and each osteotomy surface;

Step 706, when detecting that the osteotomy guide block device mounted at the tail end of the mechanical arm moves into the osteotomy region, determining the pose relationship between the osteotomy guide block device and each osteotomy face;

Step 708, determining a target osteotomy face from the osteotomy faces according to the pose relationship between the osteotomy guide block device and each osteotomy face, wherein the pose relationship is used for representing the distance and the presented angle relationship between the osteotomy guide block device and the osteotomy faces.

In the embodiment of the application, before the osteotomy operation is performed, the osteotomy region of the target part can be generated according to the positions of the osteotomy surface and the target part in the real-time image information after the real-time image information of the target part is acquired by the image acquisition equipment. Illustratively, the osteotomy region is centered about a center point of the target site, and each osteotomy face is located within the osteotomy region.

After the osteotomy region is determined, the mechanical arm starts to perform osteotomy positioning, and the osteotomy guide block device is controlled to gradually approach the osteotomy region, wherein in the osteotomy positioning process, the mechanical arm can move in a cooperative mode or an automatic mode, and the embodiment of the application is not particularly limited.

The pose information of the osteotomy guide block device arranged at the tail end of the mechanical arm is detected in real time, whether the osteotomy guide block device moves into an osteotomy region is determined according to the pose information of the osteotomy guide block device, when the osteotomy guide block device is detected to move into the osteotomy region, a target osteotomy face to be positioned can be automatically determined from a plurality of osteotomy faces according to the pose relation between the osteotomy guide block device and each osteotomy face, and the target osteotomy face is positioned in an automatic mode. The pose relationship can be determined according to the pose information of the osteotomy guide block device and the pose information of the osteotomy face, and can be used for representing the distance between the osteotomy guide block device and the osteotomy face and the presented angle relationship.

According to the positioning method provided by the embodiment of the application, in the process of osteotomy, whether the osteotomy guide block device arranged at the tail end of the mechanical arm is positioned in the osteotomy region can be determined through visual positioning, and the target osteotomy face is automatically selected from the osteotomy region according to the pose relation between the osteotomy guide block device and each osteotomy face for automatic positioning, namely, the coarse positioning and the fine positioning of the mechanical arm are fully automatic, so that the positioning flow of the mechanical arm in the osteotomy process can be simplified, the positioning efficiency of the osteotomy face can be improved, the labor cost can be reduced, the time consumption of osteotomy operation can be reduced, and the operation efficiency of osteotomy operation can be improved.

In one embodiment, referring to fig. 8, in step 704, determining an osteotomy region of the target site based on the real-time image information and the respective osteotomy faces may include:

step 802, generating a corresponding initial osteotomy region according to the target part and each osteotomy surface;

Step 804, after the three-dimensional model of the target part and the target part is spatially registered according to the real-time image information of the target part, the three-dimensional model is displayed in a display interface, wherein the three-dimensional model comprises an initial osteotomy region and various osteotomy faces, and the various osteotomy faces are positioned in the initial osteotomy region;

step 806, obtaining an osteotomy region of the target site in response to the adjustment operation for the initial osteotomy region.

In the embodiment of the present application, a three-dimensional model corresponding to the target site may be generated based on the feature point information in the image information of the target site, and an exemplary three-dimensional model is shown with reference to fig. 9. Referring to fig. 10, a target position may be selected in image information of a target portion, a marking member (e.g., an optical target or the like) may be disposed at each of the target positions in the target portion of the target object, real-time image information of the target portion may be acquired by an image acquisition device such as NDI, and the real-time image information and the image information may be registered based on the target positions, so as to spatially register the target portion and a three-dimensional model of the target portion.

For example, the size of the osteotomy region of the target site may be preset based on a priori knowledge of the target site, and then an initial osteotomy region may be generated according to the center of the target site and the preset osteotomy region size, and the initial osteotomy region may be displayed in the display interface. Or the initial osteotomy region may be generated according to the centers of the osteotomy surfaces and the target portion, and the method for generating the initial osteotomy region in the embodiment of the present application is not particularly limited.

In one example, still taking knee replacement surgery as an example, a calibration position may be selected from CT image information of a target portion, including a plurality of positions of a knee joint, a hip joint, and an ankle joint, a marker member is disposed on the target portion at the plurality of positions of the knee joint, the hip joint, and the ankle joint, and real-time image information is acquired by the NDI image acquisition apparatus, so as to complete registration of the real-time image information and the image information according to the marker member. After registration is completed, an initial osteotomy region may be generated based on the target site and the osteotomy regions, and a three-dimensional model including the initial osteotomy region may be presented in a display interface in which the initial osteotomy region includes the osteotomy regions.

In the display interface, the initial osteotomy region may be displayed as a polyhedral structure, such as: in fig. 11a, 1101 is an osteotomy region, 1102 is a knee joint center point, and 1103 is an osteotomy plane. The polyhedral structure of the osteotomy region is not particularly limited in the embodiment of the application. After the initial osteotomy region is generated, the osteotomy region of the target site can be obtained by adjusting the size of the initial osteotomy region on the display interface.

Still taking the above example as an example, a corresponding initial osteotomy region may be generated from the knee center. The size of the initial osteotomy region may be set according to the size of the patient's knee joint (for example, a sphere with a center radius of 10-15 cm may be generally taken). The initial osteotomy region can be adjusted according to the position of each osteotomy surface to obtain the osteotomy region of the knee joint. For example: in the process of operating the mechanical arm to coarsely position to each osteotomy face in the cooperative mode, confirming that the osteotomy guide block device is always in the initial osteotomy region, and when the osteotomy guide block device is found not to be in the initial osteotomy region, adjusting the initial osteotomy region to obtain an adjusted osteotomy region.

According to the positioning method provided by the embodiment of the application, the osteotomy face and the osteotomy region can be planned, and further in the osteotomy process, whether the osteotomy guide block device arranged at the tail end of the mechanical arm is positioned in the osteotomy region can be determined through visual positioning, so that the target osteotomy face is automatically selected from the osteotomy face to be positioned automatically, namely, the coarse positioning and the fine positioning processes of the mechanical arm are all automated, the positioning process of the mechanical arm in the osteotomy process can be simplified, the positioning efficiency of the osteotomy face can be improved, the labor cost can be further reduced, the time consumption of osteotomy operation can be reduced, and the operation efficiency of osteotomy operation can be improved.

In one embodiment, referring to fig. 12, when it is detected that the end-mounted osteotomy guide device of the robotic arm moves into the osteotomy region, determining the pose relationship of the osteotomy guide device to the respective osteotomy face, in step 806, may comprise:

step 1202, acquiring section pose information of a section of an osteotomy guide block device mounted at the tail end of a mechanical arm in real time under an image acquisition equipment coordinate system;

Step 1204, determining whether the osteotomy guide block device moves into the osteotomy region according to the sectional pose information of the osteotomy guide block device;

when it is determined that the osteotomy guide device moves into the osteotomy guide region, the pose relationship between the osteotomy guide device and each osteotomy face is determined according to the cross-sectional pose information of the osteotomy guide device and the pose information of each osteotomy face, step 1206.

In the embodiment of the application, the registration of the real-time image information acquired by the image acquisition equipment and the image information of the target part is realized in the process, so that the pose information of the osteotomy guide block device under the coordinate system of the image acquisition equipment can be obtained in real time according to the marking component fixedly connected with the osteotomy guide block device in the moving process of the mechanical arm. Further, according to the coordinate transformation matrix of the osteotomy guide block device and the clamping groove of the osteotomy guide block device, the pose information of the clamping groove of the osteotomy guide block device under the coordinate system of the image acquisition equipment can be obtained by transforming the pose information of the clamping groove of the osteotomy guide block device under the coordinate system of the image acquisition equipment, and the pose information of the clamping groove under the coordinate system of the image acquisition equipment can be used as the cross section pose information (hereinafter referred to as cross section pose information) of the cross section of the osteotomy guide block device under the coordinate system of the image acquisition equipment.

After the section pose information of the osteotomy guide block device is obtained, whether the osteotomy guide block device moves into the osteotomy region or not can be determined according to the section pose information of the osteotomy guide block device, and when the osteotomy guide block device is determined to move into the osteotomy region, the pose relation between the osteotomy guide block device and each osteotomy face is determined according to the osteotomy pose information of the osteotomy guide block device and the pose information of each osteotomy face. And then according to the pose relation between the osteotomy guide block device and each osteotomy face, determining a target osteotomy face from each osteotomy face, moving to the target osteotomy face, and starting an automatic mode to perform fine positioning under the condition that the osteotomy guide block device and the target osteotomy face meet the positioning condition.

For example, referring to fig. 13, the mechanical arm may be controlled to move into the osteotomy region (the mechanical arm may be controlled to move into the osteotomy region by a person in a cooperative mode, or may be controlled to move into the osteotomy region by an automatic mode), and the sectional pose information of the osteotomy guide device mounted at the end of the mechanical arm may be detected in real time, and the position information X of the osteotomy guide device in space may be obtained from the sectional pose information, and if the osteotomy region is (f _min,f_max), it may be determined that the osteotomy guide device has moved into the osteotomy region in the case of f _min<X<f_max, and the target osteotomy face may be determined from the osteotomy faces by determining the pose relationship between the osteotomy guide device and the osteotomy faces.

In one embodiment, referring to fig. 14, when it is determined that the osteotomy guide device moves into the osteotomy guide region, in step 1206, the pose relationship of the osteotomy guide device to each osteotomy face is determined based on the cross-sectional pose information of the osteotomy guide device and the pose information of each osteotomy face, comprising:

step 1402, detecting whether the mechanical arm has external force control in real time when the osteotomy guiding block device moves into the osteotomy guiding block region;

in step 1404, when it is detected that the mechanical arm is not controlled by external force, the pose relationship between the osteotomy guide block device and each osteotomy face is determined according to the cross-sectional pose information of the osteotomy guide block device and the pose information of each osteotomy face.

In the embodiment of the application, after the osteotomy face and the osteotomy region are planned, the mechanical arm can perform intelligent selection of the target osteotomy face. The mechanical arm may adopt a cooperative mode or an automatic mode in the process of selecting the target osteotomy face, which is not particularly limited in the embodiment of the present application. The target osteotomy face may be determined from the osteotomy face when the osteotomy guide device is moved into the osteotomy guide region and no longer under control of an external force.

For example, referring to fig. 15, in the collaborative mode, the robotic arm motion may be guided manually by a person (this process may also be accomplished using automatic positioning). Whether the osteotomy guide block device is positioned in the osteotomy region can be judged in real time, and when the osteotomy guide block device is positioned in the osteotomy region, whether the mechanical arm is controlled by external force is detected. When the external force control of the mechanical arm is detected, the mechanical arm can move in a compliant mode under the external force control, and whether the osteotomy guide block device is positioned in the osteotomy region or not is detected in real time in the moving process.

Until the mechanical arm is not controlled by external force and still positioned in the osteotomy region, the pose relationship between the osteotomy guide block device and each osteotomy face can be determined, the nearest target osteotomy face can be selected from each osteotomy face according to the pose relationship, and the mechanical arm can be automatically positioned to the target osteotomy face in an automatic mode.

In this way, after the osteotomy for the target osteotomy face is completed, when the other osteotomy faces are positioned, the user can give an external force control to the mechanical arm, the mechanical arm continues to move after receiving the external force control, and the processes are repeated, so that the automatic positioning and osteotomy for all osteotomy faces can be realized, and after the positioning and osteotomy for all osteotomy faces are completed, the osteotomy command can be triggered to stop through the display interface, and the osteotomy process is ended.

According to the positioning method provided by the embodiment of the application, whether the osteotomy guide block device arranged at the tail end of the mechanical arm is positioned in the osteotomy region can be determined through visual positioning, so that the target osteotomy face is automatically selected from the osteotomy faces to perform automatic positioning, namely, the coarse positioning and the fine positioning of the mechanical arm are fully automatic, the positioning process of the mechanical arm in the osteotomy process can be simplified, the positioning efficiency of the osteotomy face can be improved, the labor cost can be reduced, the time consumption of osteotomy operation can be reduced, and the operation efficiency of osteotomy operation can be improved.

In one embodiment, referring to fig. 16, in step 1206, determining a pose relationship between the osteotomy guide device and each osteotomy face according to the cross-sectional pose information of the osteotomy guide device and the pose information of each osteotomy face may include:

Step 1602, determining the angle and distance between the osteotomy guide block device and the osteotomy face according to the cross-section pose information of the osteotomy guide block device and the pose information of the osteotomy face for any osteotomy face;

step 1604, determining the pose relation of the osteotomy guide block device and the osteotomy face according to the angle and the distance.

In the embodiment of the application, the cross-section pose information of the osteotomy guide block device can be acquired in real time, and whether the osteotomy guide block device is in an osteotomy region or not can be judged according to the cross-section pose information. If the osteotomy guide device is within the osteotomy region and the robotic arm is not under the control of an external force, the angles α _i (exemplary, a _i generally ranges from 0 to 45 ° and i is an integer greater than 0 and less than 7) between the cross section of the osteotomy guide device and each osteotomy face (exemplary, if there are 6 osteotomy faces, each osteotomy face is numbered 1,2,3,4,5,6, respectively) are calculated. Referring to fig. 17, it is assumed that a direction vector of a distal femur osteotomy plane 1701 in the image capturing device coordinate system is available in the image capturing deviceAnd the orientation vector/>, of the osteotomy guide block 1702 in the image acquisition device coordinate systemThen passThe angle value of the angle alpha _i between the osteotomy guide block device and the ith osteotomy face can be calculated, and the distance d _i between the osteotomy guide block device 1702 and the distal femur osteotomy face can be obtained. Further, the pose relationship f _i between the osteotomy guide 1702 and the distal femoral resection surface can be determined by the pose relationship discriminant function f (α _i,d_i)＝kα_i+(1-k)d_i), where k is a weight coefficient of angle and distance, k e (0, 1), and so on, the pose relationship between the osteotomy guide 1702 and each resection surface can be determined, the resection surface with the minimum pose relationship f _i can be selected from each resection surface, the closest osteotomy surface to the osteotomy guide can be determined, and the osteotomy guide can be determined as the target osteotomy guide.

In one example, referring to fig. 18, after determining the target osteotomy face, an angle between the target osteotomy face and the osteotomy guide device may be displayed in a display interface, and when the angle between the osteotomy guide device and the target osteotomy face is less than an angle threshold, the user may be prompted by voice, text, AR, or the like. Still taking the example shown in fig. 17 as an example, the distance and the included angle between the distal femur section 1701 and the guide block 1702 are the smallest, that is, the distal femur section 1701 is the target osteotomy plane (assuming that the mechanical arm is not controlled by external force at this time), and the included angle between the distal femur section 1701 and the guide block 1702 is shown in the display interface. If the osteotomy guide block device continues to move so that the osteotomy guide block device is attached to the front cut section of the distal femur until the distance and the included angle between the osteotomy guide block device and the front cut section of the distal femur are minimum, the front cut section of the distal femur is determined to be the target osteotomy surface (if the mechanical arm is not controlled by external force at this time), and the included angle between the front cut section of the distal femur and the cross section of the osteotomy guide block device can be displayed in the display interface.

According to the positioning method provided by the embodiment of the application, the pose relation between the osteotomy guide block device arranged at the tail end of the mechanical arm and each osteotomy face can be determined through visual positioning, and the target osteotomy face is selected from the pose relation to perform automatic positioning, namely, the mechanical arm coarse positioning and the fine positioning processes are all automatic, so that the positioning flow of the mechanical arm in the osteotomy process can be simplified, the positioning efficiency of the osteotomy face can be improved, the labor cost can be reduced, the time consumption of osteotomy can be reduced, and the operation efficiency of osteotomy can be improved.

In one embodiment, referring to fig. 19, pose information of the osteotomy guide device may be detected in real time by the image capturing apparatus, and whether the osteotomy guide device is located in the osteotomy region may be determined based on the pose information. When the osteotomy guide block is not positioned in the osteotomy region, the mechanical arm can be controlled to move manually by a person (or can also be controlled to move in an automatic positioning mode) until the osteotomy guide block device is detected to be positioned in the osteotomy region, and corresponding prompt information can be displayed in the display interface. At this time, if the mechanical arm is manually controlled by a person, the person can release the control of the mechanical arm, and the mechanical arm is automatically positioned, or the mechanical arm can be controlled to move in a flexible mode.

Under the condition that the mechanical arm is not controlled by external force, the pose relation between the osteotomy guide block device and each osteotomy face can be judged through the image acquisition equipment, one target osteotomy face is selected from the osteotomy faces according to the pose relation, and the included angle between the osteotomy guide block device and the target osteotomy face is displayed in the display interface to achieve the position relation. When the included angle between the osteotomy guide block device and the target osteotomy face is smaller than the included angle threshold, corresponding prompt information can be displayed in the display interface. At the moment, the mechanical arm can be automatically positioned in an automatic mode, and after the mechanical arm is positioned to the target osteotomy face, corresponding prompt information is displayed in the display interface.

After the osteotomy guide block device reaches the target osteotomy face, if fine adjustment is needed, a terminal can be used for sending a fine adjustment instruction, the mechanical arm keeps the current posture unchanged, the mechanical arm can be controlled by a human hand or automatically fine adjusted in the target osteotomy face direction and the target osteotomy face normal direction, when the osteotomy guide block device moves to an osteotomy region in the movement process, bone edge information of a target part can be obtained through the image acquisition equipment, the collision detection function is provided in the movement process of the mechanical arm, and the collision to human legs or other objects in the operation process of the mechanical arm can be prevented. When the mechanical arm detects collision, the mechanical arm stops the motion in the current direction.

In one example, when a human hand drags the mechanical arm to move in the osteotomy region, a compliant positioning mode may be employed, and a model structure corresponding to the compliant positioning mode may be shown with reference to fig. 20. Fig. 20 shows a compliant positioning mode, where the difference between the position and velocity command 2002 and the position and velocity fed back by the actuator encoder 2004 can be used to output a control moment through the controller 2006, and the feedforward gravity compensation 2008 and the friction compensation are added to obtain a total moment, and the total moment is sent to each control joint of the mechanical arm body 2010 to realize the compliant positioning mode.

The structure of the controller 2006 described above may be as shown in fig. 21. The difference between the position and speed command 2002 and the speed fed back by the actuator encoder 2004 is multiplied by a coefficient Ki through an integration link, and the difference between the position and speed command 2002 and the speed fed back by the actuator encoder 2004 is directly multiplied by a gain coefficient Kp, and then the two are summed to be used as torque output of the controller 2006.

Alternatively, the structure of the controller 2006 may be as shown in fig. 22. In fig. 22, the friction force and gravity of each joint of the mechanical arm body 2010 are already compensated by feedforward, the whole mechanical arm is in a zero-force state, and the mechanical arm body follows the external force after the external force acts.

When the robotic arm is automatically positioned in the automatic mode, the controller 2006 is configured as shown in fig. 23, and in fig. 23, the position and speed command 2002 is differentiated from the position and speed fed back by the actuator encoder 2004, and then a PID controller with a speed filter is used to generate a control torque required for the whole positioning process as an output of the controller 2006.

In one embodiment, referring to FIG. 24, there is provided a positioning system, comprising: an image acquisition device 2402, a mechanical arm 2404 and an osteotomy guide block 2406 arranged at the tail end of the mechanical arm 2404, wherein,

The image acquisition device 2402 is used for planning an osteotomy face according to the image information of the target part to obtain at least one osteotomy face of the target part;

The image acquisition device 2402 is further configured to send pose information of the target osteotomy face to the mechanical arm 2404 after determining the target osteotomy face from the osteotomy faces according to the pose relation between the osteotomy guide block 2406 and each osteotomy face in the target site;

the robotic arm 2404 is configured to position the osteotomy guide block device to the target osteotomy face in accordance with pose information of the target osteotomy face in the automatic positioning mode.

In an embodiment of the present application, the image capturing device 2402 may be NDI navigation equipment. Before the osteotomy, the image capturing device 2402 may perform osteotomy plan through the acquired image information of the target portion to obtain at least one osteotomy plane of the target portion, and the specific planning process of the osteotomy plane may be described with reference to the foregoing embodiments, which are not repeated herein.

After planning the osteotomy, the image acquisition device 2402 can acquire real-time image information of the target site in the osteotomy process, and realize registration with the image information for planning the osteotomy based on a marking component (optical target, etc.) arranged on the marking position of the target site, so that pose information of the osteotomy guide block device 2406 mounted at the tail end of the mechanical arm 2404 can be acquired in real time after registration, and the pose relation between the osteotomy guide block device 2406 and each osteotomy surface in the target site can be determined according to the pose information of the osteotomy guide block device 2406, and further the target osteotomy surface can be determined from the osteotomy surfaces according to the pose relation between the osteotomy guide block device 2406 and each osteotomy surface in the target site, and the pose information of the target osteotomy surface can be sent to the mechanical arm 2404.

The mechanical arm 2404 can automatically position in an automatic mode, position in the target osteotomy face according to pose information of the target osteotomy face, enter a holding mode after the target osteotomy face is positioned, enable a doctor to use a pendulum saw or an electric drill to perform osteotomy and drilling operations through an osteotomy guide slot and a guide hole of an osteotomy guide block device, and enable the doctor to install a prosthesis and perform other operation operations after the osteotomy and drilling operations are completed.

The positioning system provided by the embodiment of the application is used for planning an osteotomy face according to the image information of the target part to obtain at least one osteotomy face of the target part, and the image acquisition device is also used for sending pose information of the target osteotomy face to the mechanical arm after determining the target osteotomy face from the osteotomy face according to the pose relation between the osteotomy guide block device and each osteotomy face in the target part in the osteotomy operation process. The mechanical arm is used for positioning the osteotomy guide block device to the target osteotomy face according to pose information of the target osteotomy face in an automatic mode. The positioning system provided by the embodiment of the application can plan the osteotomy face aiming at the target part before the operation, and automatically position the target osteotomy face by the vision positioning control mechanical arm, namely, the positioning and positioning processes are all automatic, so that the positioning process of the mechanical arm in the osteotomy process can be simplified, the positioning efficiency of the osteotomy face can be improved, the labor cost can be reduced, the time consumption of the osteotomy operation can be reduced, and the operation efficiency of the osteotomy operation can be improved.

In one embodiment, the image capturing device 2402 is further configured to capture real-time image information of the target site, determine an osteotomy region for the target site based on the real-time image information and each osteotomy surface, and determine a target osteotomy surface from the osteotomy surfaces based on the pose relationship of the osteotomy guide 2406 to each osteotomy surface when it is determined that the osteotomy guide is moved into the osteotomy region.

In the embodiment of the present application, the image capturing device 2402 determines the osteotomy region for the target portion according to the real-time image information and each osteotomy surface, and in the case of determining that the osteotomy guide block 2406 moves into the osteotomy region, the specific process of determining the target osteotomy surface from the osteotomy surfaces according to the pose relationship between the osteotomy guide block and each osteotomy surface may refer to the related description of the foregoing embodiment, which is not repeated herein.

In one embodiment, the image capturing device 2402 is further configured to obtain, in real time, cross-sectional pose information of the cross-section of the osteotomy guide 2406 under the coordinate system of the image capturing device, detect whether the osteotomy guide 2406 moves into the osteotomy region according to the cross-sectional pose information, and determine, when detecting that the osteotomy guide 2406 moves into the osteotomy guide region, a pose relationship between the osteotomy guide 2406 and each osteotomy face according to the cross-sectional pose information of the osteotomy guide 2406 and the pose information of each osteotomy face.

In the embodiment of the present application, the above operation procedure of the image capturing device 2402 may be specifically referred to the related description of the foregoing embodiment, which is not repeated herein.

In one embodiment, the osteotomy guide block device 2406 includes a slot, the system further including a marking member 2408 mounted to the end of the robotic arm;

the image capturing device 2402 is further configured to determine pose information of the osteotomy guide 2406 under the image capturing device coordinate system according to the coordinate transformation matrix of the osteotomy guide 2406 and the marking member 2408 and the pose information of the marking member 2408 under the image capturing device coordinate system after acquiring the pose information of the marking member 2408 under the image capturing device coordinate system;

The image capturing device 2402 is further configured to determine pose information of a cross section of the osteotomy guide 2406 under the image capturing device coordinate system according to the coordinate transformation matrix of the osteotomy guide 2406 and the clamping groove and the pose information of the osteotomy guide 2406 under the image capturing device coordinate system.

In an embodiment of the present application, osteotomy guide block assembly 2406 includes a slot, as shown in figure 25. The osteotomy guide 2406 may have a plurality of slots, or may be a single slot, wherein the osteotomy guide 2406 with multiple slots is shown in fig. 26a, and the osteotomy guide 2406 with single slot is shown in fig. 26 b. The number of the clamping grooves of the osteotomy guide block device 2406 is not particularly limited in the embodiment of the application. In addition, the osteotomy guide device 2406 may be a non-free osteotomy guide, or may be a multi-degree-of-freedom osteotomy guide as shown in fig. 27a to 27c, and the degree of freedom of the osteotomy guide device is not particularly limited in the embodiment of the application.

Referring to FIG. 24, the osteotomy guide 2406 may be an adjustable device with multiple degrees of freedom combined, or may be a guide with only one osteotomy slot or multiple osteotomy slots.

In the mechanical arm, the coordinate conversion relationship of the osteotomy guide 2406 and the marking member 2408 is fixed, and the coordinate conversion relationship of the osteotomy guide 2406 and the marking member 2408 can be obtained in advancePose information of the flag member 2408 in the image capturing apparatus 2402 may be obtained by the image capturing apparatus 2402 asPose information/>, of the osteotomy guide block 2406 in the image capture device 2402 can be convertedThe method comprises the following steps: /(I)

Referring to FIG. 25, the coordinate system conversion relationship of the card slot 2502 in the osteotomy guide block 2406 coordinate system obtained by measurement and calibration is as followsThen along with the movement of the mechanical arm 2404, the cross-section pose information/>, of the osteotomy guide block 2406 can be obtained in real timeThe method comprises the following steps: /(I)

When the image acquisition equipment performs cross section planning, the cross section pose information of each osteotomy face can be obtained asKnown cross-sectional pose informationFrom the position information of the osteotomy faceAnd pose information of osteotomy faceComposition, i.eSimilarly, the cross-sectional pose information of the osteotomy guide block 2406Also by the positional information/>, of the osteotomy guide block 2406And gesture informationComposition, i.e.Then according toWhether or not the osteotomy guide 2406 is within the osteotomy region is determined within the boundary of the osteotomy region, when the osteotomy guide 2406 is determined to be within the osteotomy region, the positional information/>, based on the osteotomy faceAnd section position information/>, of osteotomy guide block device 2406The distance d _i between the osteotomy guide 2406 and the osteotomy face is determined by reference to the following equation (one).

And so on, in determining the cross-sectional pose information of the osteotomy guide block 2406Pose information with osteotomy planeAnd the orientation vector of the osteotomy guide 2406 in the osteotomy guide coordinate system is known asThe direction vector of the osteotomy plane in the target site coordinate system isThe angle α _i between the cross-sectional pose information of the osteotomy guide 2406 and the pose information of the osteotomy face can be calculated, and the following formula (two) can be referred to for specific procedures.

For representing the vectorModulus ofFor representing the vectorIs a mold of (a).

After obtaining the included angle α _i and the distance d _i between the cross-sectional pose information of the osteotomy guide 2406 and the pose information of the osteotomy face, the pose relationship between the osteotomy guide 2406 and the osteotomy face can be determined according to the pose relationship conversion function and the included angle α _i and the distance between the cross-sectional pose information of the osteotomy guide 2406 and the pose information of the osteotomy face.

F (alpha _i,d_i)＝k*α_i+(1-k)*d_i formula (III))

According to the positioning system provided by the embodiment of the application, the characteristic point information of the bone of the target part can be obtained through the image acquisition device, and then the osteotomy region and the corresponding osteotomy face are set according to the characteristic point information of the bone of the target part. And determining whether the osteotomy guide block device reaches the osteotomy region by identifying position information of the osteotomy guide block device and the edge of the osteotomy region. When the osteotomy guide block device is determined to be positioned in the osteotomy region, the distance and angle between the osteotomy guide block device and each osteotomy face are determined, the target osteotomy face is selected according to the distance and angle between the osteotomy guide block device and each osteotomy face, the mechanical arm is operated in a compliant mode to enable the osteotomy guide block device to slowly approach the target osteotomy face, after the set included angle between the target osteotomy face and the osteotomy guide block device reaches an included angle threshold value, the mechanical arm is automatically positioned towards the target osteotomy face in an automatic mode, and after the target osteotomy face is positioned and corresponding osteotomy operation is completed, the processes are repeated through applying external force to the osteotomy guide block device, so that the positioning of other osteotomy faces is completed.

According to the invention, the corresponding osteotomy region and the corresponding osteotomy face in the operation process are obtained through the vision system, the target osteotomy face is automatically selected by detecting the pose relation between the osteotomy guide block device and the osteotomy face, and under the condition that the osteotomy guide block device and the target osteotomy face meet the positioning condition, the mechanical arm is controlled by adopting an automatic mode to automatically position the osteotomy guide block device to the target osteotomy face, so that the fine positioning of the osteotomy guide block device is realized, the positioning process is simplified, the used interaction is optimized, and the time consumed by the operation can be reduced.

In order that those skilled in the art will better understand the embodiments of the present application, specific examples will be described below.

Referring to fig. 28, knee replacement surgery is taken as an example. In fig. 28, there are included a surgical trolley 2801, a robotic arm 2802, a tool target 2803, an osteotomy guide block 2804, a pendulum saw 2805, ndi navigation equipment 2806, an auxiliary display 2807, a main display 2808, a navigation trolley 2809, a keyboard 2810, a femoral target 2811, a femur 2812, a tibial target 2813, a tibia 2814, and a base target 2815.

The surgical trolley 2801 and navigation trolley 2809 can be placed in the appropriate position beside the patient bed and the femoral targets 2811, tibial targets 2812, base targets 2815, sterile pouches, osteotomy guide 2804, tool targets 2803, etc. can be installed. The doctor imports the CT scan model of the patient bone into a computer for preoperative osteotomy face planning, for example: plan the plane coordinates of the osteotomy plane and select the appropriate model prosthesis and adjust the osteotomy plane, the computer includes a main display 8, a keyboard 10, and a controller located within the navigation cart 9.

The doctor uses the target pen to click characteristic points of the femur and the tibia of the patient, the NDI navigation equipment 2806 takes a base target 2815 as a reference, records the position of the characteristic points of the femur of the patient according to the click of the target pen, sends the bone characteristic points to the computer, then the computer obtains the actual positions of the femur and the tibia through a characteristic matching algorithm and corresponds to the CT image positions of the femur and the tibia, and then the navigation system links the actual positions of the femur and the tibia with corresponding targets arranged on the femur and the tibia, so that the femur target and the tibia target can track the actual positions of the bones in real time, and in the operation process, as long as the relative positions of the targets and the bones are fixed, the bone movement does not influence the operation effect.

The NDI navigation apparatus 2806 determines a target osteotomy plane according to pose information of the osteotomy guide 2804 and pose information of the osteotomy plane, and transmits osteotomy plane coordinates of the target osteotomy plane planned before the operation to the mechanical arm 2802, the mechanical arm 2802 automatically positions the target osteotomy plane through a tool target (mounted on the mechanical arm or the osteotomy guide) and moves to a predetermined position, the mechanical arm enters a holding state, a doctor can perform osteotomy and drilling operations through an osteotomy guide slot and a guide hole of the osteotomy guide module by using a pendulum saw or an electric drill, and after the osteotomy and drilling operations are completed, the doctor can mount a prosthesis and perform other operations.

The positioning method and the system provided by the embodiment of the application can simplify the bone cutting operation flow, can directly assist the mechanical arm by medical staff, can complete the automatic positioning of the bone cutting surface, can realize the whole process without additional operator back and forth cutting state, can automatically determine the selection of the cut surface by the operator or the medical staff or can automatically select the mechanical arm, and can select the target bone cutting surface to automatically position after the bone cutting guide block device of the mechanical arm is positioned in the bone cutting area.

It should be understood that, although the steps in the flowcharts related to the embodiments described above are sequentially shown as indicated by arrows, these steps are not necessarily sequentially performed in the order indicated by the arrows. The steps are not strictly limited to the order of execution unless explicitly recited herein, and the steps may be executed in other orders. Moreover, at least some of the steps in the flowcharts described in the above embodiments may include a plurality of steps or a plurality of stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of the steps or stages is not necessarily performed sequentially, but may be performed alternately or alternately with at least some of the other steps or stages.

Based on the same inventive concept, the embodiment of the application also provides a positioning device for realizing the positioning method. The implementation of the solution provided by the device is similar to that described in the above method, so the specific limitations in one or more embodiments of the positioning device provided below may be referred to above for limitations of the positioning method, which are not repeated here.

In one embodiment, as shown in fig. 29, there is provided a positioning device comprising: planning module 2902, determination module 2904, and positioning module 2906, wherein:

Planning module 2902, configured to perform osteotomy plan planning according to image information of a target location, so as to obtain at least one osteotomy of the target location;

A determining module 2904, configured to determine a target osteotomy plane from the osteotomy planes according to a pose relationship between an osteotomy guide block device mounted at the end of the robotic arm and each of the osteotomy planes in the target location;

The positioning module 2906 is used for controlling the mechanical arm to position the osteotomy guide block device to the target osteotomy face by adopting an automatic mode when the pose relation between the osteotomy guide block device and the target osteotomy face meets the positioning condition.

According to the positioning device, the osteotomy face planning can be carried out according to the image information of the target part, at least one osteotomy face of the target part is obtained, the target osteotomy face is determined from the osteotomy faces according to the pose relation between the osteotomy guide block device arranged at the tail end of the mechanical arm and each osteotomy face in the target part, and when the pose relation between the osteotomy guide block device and the target osteotomy face meets the positioning condition, the mechanical arm is controlled to position the osteotomy guide block device to the target osteotomy face in an automatic mode. The positioning device provided by the embodiment of the application can plan the osteotomy face aiming at the target part before the operation, and automatically position the target osteotomy face by the vision positioning control mechanical arm, namely, the positioning and positioning processes are all automatic, so that the positioning process of the mechanical arm in the osteotomy process can be simplified, the positioning efficiency of the osteotomy face can be improved, the labor cost can be reduced, the time consumption of the osteotomy operation can be reduced, and the operation efficiency of the osteotomy operation can be improved.

In one embodiment, the planning module 2902 is further configured to:

In one embodiment, the determining module 2904 is further configured to:

Determining an osteotomy region of the target part according to the real-time image information and each osteotomy surface;

In one embodiment, the determining module 2904 is further configured to:

The various modules in the positioning device described above may be implemented in whole or in part by software, hardware, and combinations thereof. The above modules may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.

In one embodiment, a computer device is provided, which may be a terminal, and an internal structure diagram thereof may be as shown in fig. 30. The computer device includes a processor, a memory, a communication interface, a display screen, and an input device connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The communication interface of the computer device is used for carrying out wired or wireless communication with an external terminal, and the wireless mode can be realized through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. The computer program is executed by a processor to implement a positioning method. The display screen of the computer equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, can also be keys, a track ball or a touch pad arranged on the shell of the computer equipment, and can also be an external keyboard, a touch pad or a mouse and the like.

It will be appreciated by those skilled in the art that the structure shown in FIG. 30 is merely a block diagram of some of the structures associated with the present inventive arrangements and is not limiting of the computer device to which the present inventive arrangements may be applied, and that a particular computer device may include more or fewer components than shown, or may combine some of the components, or have a different arrangement of components.

In an embodiment, there is also provided a computer device comprising a memory and a processor, the memory having stored therein a computer program, the processor implementing the steps of the method embodiments described above when the computer program is executed.

In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored which, when executed by a processor, carries out the steps of the method embodiments described above.

In an embodiment, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the steps of the method embodiments described above.

The user information (including but not limited to user equipment information, user personal information, etc.) and the data (including but not limited to data for analysis, stored data, presented data, etc.) related to the present application are information and data authorized by the user or sufficiently authorized by each party.

Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, database, or other medium used in embodiments provided herein may include at least one of non-volatile and volatile memory. The nonvolatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical Memory, high density embedded nonvolatile Memory, resistive random access Memory (ReRAM), magneto-resistive random access Memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric Memory (Ferroelectric Random Access Memory, FRAM), phase change Memory (PHASE CHANGE Memory, PCM), graphene Memory, and the like. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, and the like. By way of illustration, and not limitation, RAM can be in various forms such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), etc. The databases referred to in the embodiments provided herein may include at least one of a relational database and a non-relational database. The non-relational database may include, but is not limited to, a blockchain-based distributed database, and the like. The processor referred to in the embodiments provided in the present application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, or the like, but is not limited thereto.

The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.

The foregoing examples illustrate only a few embodiments of the application and are described in detail herein without thereby limiting the scope of the application. It should be noted that it will be apparent to those skilled in the art that several variations and modifications can be made without departing from the spirit of the application, which are all within the scope of the application. Accordingly, the scope of the application should be assessed as that of the appended claims.

Claims

1. A method of positioning, the method comprising:

Determining a target osteotomy face from the osteotomy face according to the pose relation between an osteotomy guide block device arranged at the tail end of the mechanical arm and each osteotomy face in the target part, wherein the pose relation comprises a distance relation and an angle relation of a section;

When the pose relation between the osteotomy guiding block device and the target osteotomy face meets a positioning condition, the mechanical arm is controlled to position the osteotomy guiding block device to the target osteotomy face in an automatic mode, and the positioning condition comprises that the distance and/or angle between the osteotomy guiding block device and the target osteotomy face is smaller than or equal to a preset distance threshold value and/or angle threshold value.

2. The method of claim 1, wherein the performing osteotomy plan based on image information of the target site to obtain at least one osteotomy of the target site comprises:

3. The method of claim 2, wherein determining at least one osteotomy plane of the target site based on the feature point information comprises:

4. A method according to any one of claims 1 to 3, wherein said determining a target osteotomy plane from said osteotomy planes based on the pose relationship of a robotic arm end mounted osteotomy guide block device to each of said osteotomies planes in said target site, comprises:

5. The method of claim 4, wherein determining an osteotomy region for the target site based on the real-time image information and each of the osteotomies comprises:

6. The method of claim 4, wherein said determining the pose relationship of said osteotomy guide device to each of said osteotomy faces when said robotic arm end mounted osteotomy guide device is detected to move into said osteotomy region, comprises:

7. The method of claim 6, wherein the determining the pose relationship of the osteotomy guide device to each of the osteotomy faces based on the cross-sectional pose information of the osteotomy guide device and pose information of each of the osteotomy faces when the osteotomy guide device is determined to move into the osteotomy guide region comprises:

8. The method of claim 6 or 7, wherein said determining the pose relationship of the osteotomy guide device to each of the osteotomy faces based on the cross-sectional pose information of the osteotomy guide device and pose information of each of the osteotomy faces comprises:

9. A positioning system, the system comprising: image acquisition equipment, a mechanical arm and an osteotomy guide block device arranged at the tail end of the mechanical arm, wherein,

The image acquisition equipment is used for planning an osteotomy face according to the image information of the target part to obtain at least one osteotomy face of the target part;

The image acquisition equipment is also used for sending pose information of the target osteotomy face to the mechanical arm after determining the target osteotomy face from the osteotomy face according to the pose relation between the osteotomy guide block device and each osteotomy face in the target part, wherein the pose relation comprises a distance relation and an angle relation of a section;

10. The system of claim 9, wherein the image capturing device is further configured to capture real-time image information of a target site, determine an osteotomy region for the target site based on the real-time image information and each of the osteotomies, and determine a target osteotomies from the osteotomies based on a pose relationship of the osteotomies to each of the osteotomies when the osteotomies are determined to be moved into the osteotomies.

11. The system of claim 10, wherein the image acquisition device is further configured to obtain in real time cross-sectional pose information of a cross-section of the osteotomy guide device under the image acquisition device coordinate system, detect whether the osteotomy guide device moves into the osteotomy region based on the cross-sectional pose information, and determine a pose relationship of the osteotomy guide device to each of the osteotomy faces based on the cross-sectional pose information of the osteotomy guide device and pose information of each of the osteotomy faces when the osteotomy guide device is detected to move into the osteotomy guide region.

12. The system of claim 11, wherein the osteotomy guide block device includes a slot, the system further comprising a marking member mounted to a distal end of the robotic arm;

The image acquisition equipment is also used for determining pose information of the osteotomy guide block device under the coordinate system of the image acquisition equipment according to the coordinate transformation matrix of the osteotomy guide block device and the marking component and the pose information of the marking component under the coordinate system of the image acquisition equipment after acquiring the pose information of the marking component under the coordinate system of the image acquisition equipment;

13. A positioning device, the device comprising:

The determining module is used for determining a target osteotomy face from the osteotomy face according to the pose relation between an osteotomy guide block device arranged at the tail end of the mechanical arm and each osteotomy face in the target part, wherein the pose relation comprises a distance relation and an angle relation formed by a section;

And the positioning module is used for controlling the mechanical arm to position the osteotomy guide block device to the target osteotomy surface by adopting an automatic mode when the pose relation between the osteotomy guide block device and the target osteotomy surface meets the positioning condition, wherein the positioning condition comprises that the distance and/or angle between the osteotomy guide block device and the target osteotomy surface is smaller than or equal to a preset distance threshold value and/or angle threshold value.

14. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor implements the steps of the method of any one of claims 1 to 8 when the computer program is executed.

15. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method of any of claims 1 to 8.