[go: up one dir, main page]

CN118383801B - Full-automatic breast ultrasound double-arm scanning method, device and control system - Google Patents

Full-automatic breast ultrasound double-arm scanning method, device and control system Download PDF

Info

Publication number
CN118383801B
CN118383801B CN202410844105.9A CN202410844105A CN118383801B CN 118383801 B CN118383801 B CN 118383801B CN 202410844105 A CN202410844105 A CN 202410844105A CN 118383801 B CN118383801 B CN 118383801B
Authority
CN
China
Prior art keywords
scanning
ultrasonic
track
mammary gland
breast
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202410844105.9A
Other languages
Chinese (zh)
Other versions
CN118383801A (en
Inventor
刘振
张乐
杨大鹏
黄琦
牛宝山
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hefei Hebin Intelligent Robot Co ltd
Original Assignee
Hefei Hebin Intelligent Robot Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hefei Hebin Intelligent Robot Co ltd filed Critical Hefei Hebin Intelligent Robot Co ltd
Priority to CN202410844105.9A priority Critical patent/CN118383801B/en
Publication of CN118383801A publication Critical patent/CN118383801A/en
Application granted granted Critical
Publication of CN118383801B publication Critical patent/CN118383801B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B8/00Diagnosis using ultrasonic, sonic or infrasonic waves
    • A61B8/08Clinical applications
    • A61B8/0825Clinical applications for diagnosis of the breast, e.g. mammography
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B8/00Diagnosis using ultrasonic, sonic or infrasonic waves
    • A61B8/52Devices using data or image processing specially adapted for diagnosis using ultrasonic, sonic or infrasonic waves
    • A61B8/5269Devices using data or image processing specially adapted for diagnosis using ultrasonic, sonic or infrasonic waves involving detection or reduction of artifacts
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B8/00Diagnosis using ultrasonic, sonic or infrasonic waves
    • A61B8/54Control of the diagnostic device
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B8/00Diagnosis using ultrasonic, sonic or infrasonic waves
    • A61B8/58Testing, adjusting or calibrating the diagnostic device
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00Three dimensional [3D] modelling, e.g. data description of 3D objects
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/0002Inspection of images, e.g. flaw detection
    • G06T7/0012Biomedical image inspection
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10132Ultrasound image
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30004Biomedical image processing

Landscapes

  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Radiology & Medical Imaging (AREA)
  • Biophysics (AREA)
  • Veterinary Medicine (AREA)
  • Public Health (AREA)
  • Animal Behavior & Ethology (AREA)
  • Surgery (AREA)
  • Pathology (AREA)
  • Biomedical Technology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Molecular Biology (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Quality & Reliability (AREA)
  • Software Systems (AREA)
  • Geometry (AREA)
  • Computer Graphics (AREA)
  • Ultra Sonic Daignosis Equipment (AREA)

Abstract

The invention relates to the technical field of breast ultrasonic scanning, in particular to a full-automatic breast ultrasonic double-arm scanning method, a full-automatic breast ultrasonic double-arm scanning device and a full-automatic breast ultrasonic double-arm scanning control system, which comprise the following steps: acquiring a depth image of a chest region, and performing three-dimensional reconstruction on the depth image to form a three-dimensional reconstruction model; generating a scanning track of an ultrasonic probe for scanning on the mammary gland according to the three-dimensional reconstruction model; two mechanical arms are symmetrically arranged on the mammary gland, and ultrasonic probes on the two mechanical arms perform symmetrical scanning on the mammary gland along a scanning track so as to realize double-arm scanning; the invention can avoid large displacement of the mammary gland caused by uneven stress in the process of ultrasonic scanning of the mammary gland, and improve the scanning accuracy.

Description

Full-automatic breast ultrasound double-arm scanning method, device and control system
Technical Field
The invention relates to the technical field of breast ultrasonic scanning, in particular to a full-automatic breast ultrasonic double-arm scanning method, device and control system.
Background
With advances in science and technology, screening for breast cancer has made significant progress, with breast ultrasound screening being one of the common breast cancer screening techniques.
The existing breast ultrasonic examination is generally manual scanning, and the manual scanning method has high requirements on a doctor's scanning method in the breast examination process, and has low scanning speed and influences the examination efficiency. In order to solve the problems associated with manual inspection, the skilled person has therefore devised a number of devices which allow automatic inspection. There is a device in the market that requires a person to be examined to lie on a breast examination bed with two breasts respectively placed in two examination holes, and then an ultrasonic probe is controlled to scan the breasts; and the probe transmits the acquired ultrasonic information to an ultrasonic image processing device for processing. The method can realize automatic scanning of the mammary glands, and is beneficial to improving the inspection efficiency. However, the examinee must take a prone position during the examination, and feel uncomfortable. And the probe moving device and the probe are immersed in water, the probe moving device is easy to fail and inconvenient to maintain, and the probe can agitate the water in the water tank in the moving process of the probe to influence the accuracy of inspection.
Therefore, in order to avoid the problems in the prior art, patent 201410349360.2 discloses an automatic breast ultrasonic scanning method, which can realize automatic scanning of breasts, and a person to be inspected adopts a supine position in the inspection process, so that the method is more comfortable and has high inspection accuracy.
In the actual inspection process, the defects that the breast is large in deformation and strong in fluidity are found, and the breast is easily displaced by the existing single-arm breast scanning method, so that the designed scanning track is invalid, and the risk of omission exists in the scanning process, so that the problem is to be solved.
Disclosure of Invention
In order to avoid and overcome the technical problems in the prior art, the invention provides a full-automatic breast ultrasound double-arm scanning method. The invention can avoid large displacement of the mammary gland caused by uneven stress in the process of ultrasonic scanning of the mammary gland, and improve the scanning accuracy.
In order to achieve the above purpose, the present invention provides the following technical solutions:
a full-automatic breast ultrasound double-arm scanning method comprises the following steps:
S1, acquiring a depth image of a chest region, and performing three-dimensional reconstruction on the depth image to form a breast three-dimensional reconstruction model;
s2, generating a scanning track of an ultrasonic probe for scanning on the mammary gland according to the mammary gland three-dimensional reconstruction model;
s3, symmetrically arranging two mechanical arms on the mammary gland, and symmetrically scanning the mammary gland along a scanning track by using ultrasonic probes on the two mechanical arms so as to realize double-arm scanning.
As still further aspects of the invention: the specific content of step S3 is as follows:
s31, symmetrically arranging two mechanical arms on a mammary gland by taking a vertical axis of a nipple of the mammary gland as a symmetry axis, wherein one mechanical arm is used as a main scanning arm, and performing a scanning task along a scanning track; the other mechanical arm is used as an auxiliary arm, and simultaneously presses the mammary gland along the other symmetrical scanning track with the same force; the two ultrasonic probes are in a symmetrical state;
S32, enabling a six-dimensional force sensor arranged on the mechanical arm to directly read the stress of the ultrasonic probe through gravity compensation, and then calculating a contact force difference value between an expected contact force and an actual contact force, wherein the calculation formula of the contact force difference value is as follows:
Wherein F Z represents the reading of the six-dimensional force sensor in the Z-axis direction, namely the actual contact force, and the positive direction of the Z-axis is vertically upward; f Ce represents the desired contact force; f Ze represents the contact force difference between the desired contact force and the actual contact force;
S33, obtaining the initial moving speed of the ultrasonic probe according to the speed difference value between two adjacent path points of the ultrasonic probe on the same scanning track;
S34, adjusting the required moving speed of the ultrasonic probe by combining the contact depth to realize a force control mode of force-speed, wherein the force control mode is expressed as follows:
Wherein V z (k+1) represents the normal speed of the ultrasonic probe at the scanning track path point at the time k+1; v z (k) represents the normal velocity of the ultrasound probe at the scan trajectory path point at time k; both C z1 and C z2 are adjustable parameters.
As still further aspects of the invention: the specific steps of step S1 are as follows:
s11, a patient to be inspected is laid on an inspection bed, the chest area is placed in a limited space which can be photographed by a depth camera, and then the chest area of the patient to be inspected is covered by surgical cloth and the mammary gland of the patient to be inspected is exposed;
s12, shooting a chest region of a patient to be inspected by using a depth camera to obtain a chest depth image containing a breast and surgical cloth, and converting the chest depth image into a corresponding chest point cloud;
s13, constructing a mechanical arm base coordinate system, wherein the mechanical arm base coordinate system forms a three-dimensional space; acquiring pose information of an ultrasonic probe positioned at the tail end of the mechanical arm in a mechanical arm base coordinate system through hand-eye calibration, and acquiring pose information of chest point cloud in the mechanical arm base coordinate system through coordinate conversion;
s14, registering the breast point cloud through an ICP algorithm to obtain registered breast point cloud;
and S15, performing three-dimensional reconstruction on the registered breast point cloud through a Poisson curved surface generation algorithm to obtain a breast three-dimensional reconstruction model.
As still further aspects of the invention: the specific content of step S2 is as follows:
S21, finding out the highest point of the breast three-dimensional model on the Z axis in a mechanical arm base coordinate system, taking the projection of the highest point on the XY plane along the Z axis as a circle center, wherein the radius of the circle where the circle center is located is the radius of the smallest circle surrounding the breast root in the breast three-dimensional model, and the radius of the smallest circle is the scanning radius;
S22, generating a scanning track on the basis of a scanning radius, wherein the generated scanning tracks are of three types, namely a parallel moving type ultrasonic scanning track, a rotary type ultrasonic scanning track and a radial type ultrasonic scanning track;
the generation process of the parallel moving type ultrasonic scanning track specifically comprises the following steps: firstly, an ultrasonic probe slides on the outer contour of a mammary gland along the X-axis direction or the Y-axis direction of a three-dimensional space to form a central curve scanning track, and the central curve scanning track passes through the highest point of a mammary gland three-dimensional model on the Z axis in the three-dimensional space; then, sequentially defining curve-type scanning tracks with the distance between the center curve-type scanning track and the scanning track which is set by the center curve-type scanning track along the direction of the center curve-type scanning track; generating a plurality of curve-type scanning tracks on two sides of the central curve-type scanning track in sequence according to the demarcation mode, wherein the distances between adjacent curve-type scanning tracks are the scanning track distances; a central curve type scanning track and a plurality of curve type scanning tracks are the parallel moving type ultrasonic scanning tracks;
The generation process of the rotary ultrasonic scanning track specifically comprises the following steps: generating a plurality of annular scanning tracks with equal height lines on the mammary gland outline along the direction from the highest point of the mammary gland three-dimensional model on the Z axis to the smallest circular edge in the three-dimensional space, wherein the distances between adjacent annular scanning tracks are the distance between the scanning tracks; the annular scanning track is the rotary ultrasonic scanning track;
The generation process of the radial ultrasonic scanning track specifically comprises the following steps: taking the highest point in the step S21 as a starting point, sliding from the highest point to the smallest circle on the surface of the mammary gland along the scanning radial direction to form a radial ultrasonic scanning track along the scanning radial direction; repeatedly moving in the mode, forming a plurality of radial ultrasonic scanning tracks along the circumferential direction of the minimum circle, wherein the included angles between the projections of adjacent radial ultrasonic scanning tracks on the XY plane are the same;
as still further aspects of the invention: the scan track pitch calculation formula is as follows:
wherein DeltaL is the scanning track interval, L is the scanning width of the ultrasonic probe, and alpha is the overlapping rate between adjacent scanning tracks.
As still further aspects of the invention: the calculation formula of the radial ultrasonic scanning track number is as follows:
Wherein N tr represents the number of radial ultrasonic scanning tracks, R is the scanning radius, ceil represents the upward rounding operation.
An apparatus includes an image acquisition module, a trajectory generation module, a scanning control module, and a diagnostic module;
an image acquisition module: for acquiring a depth image of a chest region of a user;
The track generation module: the method comprises the steps of carrying out three-dimensional reconstruction according to a depth image of a chest region to obtain a breast three-dimensional reconstruction model, and planning a corresponding parallel moving type ultrasonic scanning track, a rotary type ultrasonic scanning track and a radial type ultrasonic scanning track according to the breast three-dimensional reconstruction model;
And the scanning control module is used for: the ultrasonic scanning device is used for controlling the scanning mechanism to drive the ultrasonic probe to carry out ultrasonic scanning on the mammary gland of the user according to the parallel moving type ultrasonic scanning track, the rotary type ultrasonic scanning track and the radial type ultrasonic scanning track;
the scanning mechanism comprises a detection platform for a patient to lie on, two groups of mechanical arms are respectively arranged on two sides of the detection platform, and end effectors are respectively arranged at the free ends of the two groups of mechanical arms; the end effector comprises a depth camera shooting a mammary gland region, a clamp is connected to a fixed plate beside the depth camera through a six-dimensional force sensor controlling the compression force of the mammary gland, and an ultrasonic probe is arranged at the clamping end of the clamp in a clamping manner;
and a diagnosis module: the method comprises the steps of analyzing and processing an acquired ultrasonic image to generate a diagnosis result;
And a service module: and the system is used for generating a review plan and/or a health insurance scheme according to the diagnosis result.
A control system comprising a processor, an input device, an output device and a memory, the processor, the input device, the output device and the memory being connected in sequence, the memory being for storing a computer program comprising program instructions, the processor being configured to invoke the program instructions to perform the method as described above.
Compared with the prior art, the invention has the beneficial effects that:
1. According to the invention, the radial, rotary and parallel scanning tracks are planned by obtaining the breast three-dimensional model of the patient to be checked, so that the proper scanning track can be selected according to the size and shape characteristics of the breast of the patient, the individual difference is considered, the humanization is realized, and the accuracy of scanning diagnosis is improved.
2. According to the invention, according to the characteristic that the mammary gland has close symmetry, the double-arm opposite scanning and double-arm alternating motion planning is provided, so that the symmetrical stress of the mammary gland is realized, and the mammary gland is prevented from generating unidirectional large displacement deformation due to single-side stress.
3. The invention provides the motion planning of double arms to scanning, which greatly reduces the scanning time and improves the scanning efficiency.
4. The invention realizes a force-speed force control mode, and performs automatic breast ultrasound scanning with stable and constant force, thereby improving the comfort and safety of patients.
5. The invention can be applied to the mechanical arm to carry out automatic ultrasonic scanning on the mammary gland, and effectively relieves the resource shortage problem of an ultrasonic doctor.
Drawings
FIG. 1 is a flow chart of the method of the present invention.
Fig. 2 is a top view of a rotational ultrasound scanning trajectory in accordance with the present invention.
Fig. 3 is a plan view of a parallel moving ultrasound scanning trajectory in the present invention.
Fig. 4 is a top view of a radial ultrasound scanning trajectory in accordance with the present invention.
Fig. 5 is a graph of the difference between the desired contact force and the actual contact force for the present invention.
FIG. 6 is a graph showing force control accuracy of a nonlinear force control model and a linear force control model according to the present invention.
Fig. 7 is a schematic structural diagram of a fully automatic breast ultrasound dual-arm scanning device in the invention.
Fig. 8 is a schematic view of the structure of the end effector of the present invention.
Fig. 9 is a schematic structural diagram of a control system according to the present invention.
In the figure: 1. an end effector; 11. a depth camera; 12. a fixing plate; 13. a six-dimensional force sensor; 14. a clamp; 15. an ultrasonic probe; 2. a mechanical arm; 3. a patient; 20. a control system; 21. a processor; 22. a memory; 23. an input device; 24. and an output device.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Referring to fig. 1 to 6, in an embodiment of the present invention, a full-automatic breast ultrasound dual-arm scanning method includes the following steps:
1. a depth image of the chest region is acquired and three-dimensional reconstruction is performed.
In this step, the patient to be examined is first laid on an examination bed, the chest is placed in a limited space for camera vision, and then the upper body is covered with a blue surgical cloth and the chest is exposed. The depth image is shot at multiple angles through the depth camera at the tail end of the mechanical arm, then the pose of the ultrasonic probe at the tail end of the mechanical arm is obtained through hand-eye calibration, and the pose information of the chest point cloud in the mechanical arm base coordinate system is obtained through coordinate conversion, so that the track planning of the mechanical arm is facilitated. Because the number of the point clouds is large, the processing time is long, and in order to reduce the point clouds which are not important points of research, the chest point clouds of the space region which is important attention are separated through the direct filtering. The breast point cloud in the space area is partially breast, and partially blue operation cloth, so that the point cloud can be divided and independent breast point clouds can be obtained due to the difference of the color information of the breast point cloud and the blue operation cloth. Then, the point cloud of one angle is taken as the main point cloud, and the point clouds of other angles are registered to the main point cloud. And carrying out three-dimensional reconstruction through Poisson curved surface reconstruction according to the registered point cloud model.
The step of acquiring a depth image of the chest region and performing a three-dimensional reconstruction comprises:
1.1, covering the upper body with a blue surgical cloth and exposing the chest.
The patient to be examined is first laid on the examination bed, the chest is placed in a limited space for camera vision, and then the upper body is covered with a blue surgical cloth and the chest is exposed.
1.2, Shooting depth images by the depth camera at multiple angles.
And automatically shooting depth images of the chest of the patient to be inspected at multiple angles through a depth camera at the tail end of the mechanical arm.
And 1.3, converting the coordinates into a mechanical arm base coordinate system.
The pose of the ultrasonic probe at the tail end of the mechanical arm is obtained through hand-eye calibration, and then pose information of the chest point cloud in a mechanical arm base coordinate system is obtained through coordinate conversion, so that the track planning of the mechanical arm is facilitated.
1.4, Straight-pass filtering and point cloud segmentation.
Since the number of point clouds is large and the processing time is long, in order to reduce the chest point cloud which is not the focus of the study, that is, the chest point cloud which is not the focus of the study, the three-dimensional coordinates of the chest point cloud which is the subject of the study in the three-dimensional space are segmented by the straight-through filtering, and the chest point cloud which is the subject of the study is defined as the focus chest point cloud.
1.5, Three-dimensional reconstruction of mammary glands.
In order to obtain a breast three-dimensional reconstruction model, point cloud denoising is performed through a radius filter, and outlier point clouds are deleted. Then, the point cloud of one angle is taken as the main point cloud, and the point clouds of other angles are registered to the main point cloud. To avoid that iterations of ICP registration enter local minima due to shape symmetry, the main point cloud is rotated once every 10 degrees, starting ICP after each rotation. And carrying out three-dimensional reconstruction through a Poisson curved surface generation algorithm according to the registered point cloud model. And calculating the highest point coordinate as a starting point of the first scanning track, and then selecting the minimum circular radius containing the breast three-dimensional model by taking the highest point coordinate as the circle center.
2. And generating a scanning track of the ultrasonic probe according to the breast three-dimensional reconstruction model.
The scan trajectory spacing is calculated according to the following formula:
Wherein DeltaL is the scanning track interval, L is the scanning width of the ultrasonic probe, and alpha is the overlapping rate.
The generation process of the rotary ultrasonic scanning track specifically comprises the following steps:
As shown in fig. 2, the highest point P (0, 0) of the breast three-dimensional reconstruction model on the Z axis in the three-dimensional space is along the breast outer contour to the minimum circular edge direction, so as to generate a plurality of annular scanning tracks with equal height on the breast outer contour, and the distances between adjacent annular scanning tracks are all the scanning track distances. The two ultrasonic probes are respectively positioned at two sides of the highest point P (0, 0), and the two ultrasonic probes are positioned on the same rotary ultrasonic scanning track and rotate in the same direction. Meanwhile, the highest point and the two ultrasonic probes are always positioned on the same straight line in the moving process, so that the unilateral deformation of the mammary gland can be reduced, and the scanning accuracy is improved.
The generation process of the parallel moving type ultrasonic scanning track specifically comprises the following steps:
As shown in fig. 3, two ultrasonic probes are respectively positioned at two sides of the highest point P (0, 0), and the two ultrasonic probes are positioned on two parallel moving ultrasonic scanning tracks symmetrically positioned at two sides of the highest point and move reversely. Meanwhile, the highest point and the two ultrasonic probes are always positioned on the same straight line in the moving process, so that the unilateral deformation of the mammary gland can be reduced, and the scanning accuracy is improved.
The breast point clouds P (0, 1), P (1, 0), P (4, 0), P (2, 0), P (0, 2), etc. in fig. 2 and 3 are distributed on the breast in a set manner, and the set manner may be according to practical situations. The projection distances between adjacent tracks on the XY plane are equal, and the projection distances are the scanning track distances. The projection distance in the horizontal plane between the point clouds of adjacent trajectories is not equal, as it relates to the curvature of the breast surface. This allows equidistant paths to be planned even on body surfaces with large curvatures, thus achieving a complete and uniform coverage of the ultrasound scan, greatly reducing the risk of omission. Since the above search results in a huge and unordered number of point clouds on the trajectory, it is employed to uniformly downsample and rearrange the disk-shaped point clouds according to the polar angle of the points in either clockwise or counterclockwise direction.
For a radial ultrasonic scanning track, the number of the tracks is calculated according to the following formula:
Wherein N tr represents the number of radial scanning tracks, R is the scanning radius, ceil represents the upward rounding.
Each scan trajectory is scanned from the bottom of the breast toward the nipple, in sequence at intervals until the surface of the breast is completely covered, as shown in fig. 4.
3. And controlling double-arm scanning according to the scanning track.
The motion planning of the double-arm opposite scanning comprises the following steps:
Rotation type ultrasonic scanning track: because the mammary gland has the characteristic of being close to symmetry, in a rotary ultrasonic scanning track, one mechanical arm executes half track scanning tasks, the other mechanical arm simultaneously executes the other half track scanning tasks, the two ultrasonic probes are kept in opposite states, the highest point and the two ultrasonic probes are always positioned on the same straight line in the moving process, so that the symmetrical stress of the mammary gland can be realized, and the mammary gland is prevented from being deformed in a unidirectional large displacement due to the stress of a single side.
Parallel moving type ultrasonic scanning track: because the mammary gland has the characteristic of being close to symmetry, the two ultrasonic probes are positioned on two parallel moving ultrasonic scanning tracks which are symmetrically positioned at the two sides of the highest point and move reversely. And the highest point and the two ultrasonic probes are always positioned on the same straight line in the moving process. When one mechanical arm executes the scanning task of the parallel moving type ultrasonic scanning track, the other mechanical arm simultaneously executes the scanning task of the parallel moving type ultrasonic scanning track on the other side symmetrical to the mechanical arm, and the two ultrasonic probes are kept in a relative state and are simultaneously far away from the highest point or are simultaneously close to the highest point, so that the symmetrical stress of the mammary gland can be realized, and the mammary gland is prevented from being deformed in a unidirectional large displacement due to the stress of a single side.
Radial ultrasonic scanning track: because the mammary gland has the characteristic of being close to symmetry, the two ultrasonic probes are positioned on two radial ultrasonic scanning tracks which are symmetrically positioned at the two sides of the highest point and move in the same direction or in opposite directions. And the highest point and the two ultrasonic probes are always positioned on the same straight line in the moving process. When one mechanical arm executes the scanning task of a radial ultrasonic scanning track, the other mechanical arm simultaneously executes the scanning task of the radial ultrasonic scanning track on the other side which is symmetrical with the mechanical arm, and the two ultrasonic probes are kept in a relative state and are simultaneously far away from the highest point or are simultaneously close to the highest point, so that the symmetrical stress of the mammary gland can be realized, and the mammary gland is prevented from being deformed in a unidirectional large displacement way due to the stress of a single side.
The force control step of the double arms comprises the following steps:
The readings of the six-dimensional force sensor directly represent the stress of the ultrasonic probe through gravity compensation, and then the contact force difference between the expected contact force and the actual contact force is calculated according to the following calculation formula:
Wherein F Z represents the reading of the six-dimensional force sensor in the Z-axis direction, namely the actual contact force, and the positive direction of the Z-axis is vertically upward; f Ce represents the desired contact force; f Ze represents the contact force difference between the desired contact force and the actual contact force;
According to the difference value between two adjacent path points on the scanning track, including the position and the gesture, obtaining an initial speed, and then obtaining a required probe speed by combining with contact depth adjustment, realizing a force control mode of force-speed, wherein the two force control modes are expressed as follows:
Wherein V z (k+1) represents the normal speed of the ultrasonic probe at the scanning track path point at the time k+1; f Z represents normal force of the ultrasonic probe at the scanning track path point at the moment k; v z (k) represents the normal velocity of the ultrasound probe at the scan trajectory path point at time k; c z1 and C z2 are adjustable parameters; Representing an upper speed limit; Indicating the magnitude of the force at zero speed; Representing the slope of the curve. As shown in FIG. 5, the abscissa is the difference between the desired contact force and the actual contact force in newtons and the ordinate is velocity in velocity . As can be seen from fig. 5, the response of the force control mode of linear force-velocity is much smaller as the desired force is approached than the non-linear force control mode. If the desired force reaches 15N, the difference between the desired contact force and the actual contact force is maximum when the human body is not contacted, and the moving speed of the linear force control mode is 30Whereas the movement speed of the nonlinear force control mode is close to 6Therefore, the nonlinear force control mode is safer and more reliable than the linear mode, and the patient can be more psychologically accepted.
As shown in fig. 6, the abscissa is the number of acquisitions, and the ordinate is the actual contact force magnitude in newtons. As can be seen from fig. 6, the force control accuracy of the nonlinear force control model is higher than that of the linear force control model, and the nonlinear force control model can be more suitable for the characteristic of large curvature of the large curved surface of the mammary gland.
Based on the aforementioned proposed full-automatic breast ultrasound double-arm scanning method, the invention further provides a full-automatic breast ultrasound double-arm scanning device, as shown in fig. 7 and 8, comprising an end effector 1, a mechanical arm 2 and a mannequin 3 for replacing a patient.
The end effector comprises a depth camera 11, a fixed plate 12, a six-dimensional force sensor 13, a clamp 14, and an ultrasonic probe 15. The ultrasonic probe 15 is fixed inside the jig 14, and the jig 14 is fixed at one side of the six-dimensional force sensor 13. A six-dimensional force sensor 13 is fixed to one side of the fixed plate 12 for measuring the force applied to the ultrasonic probe 14. The depth camera 11 is fixed to one side of the fixed plate 12 for obtaining a depth image. The robot arm 2 adopts a UR3 robot arm. The mechanical arm 2 carries the end effector 1 to automatically carry out ultrasonic scanning tasks on the manikin 3.
As shown in fig. 9, the control system 20 includes one or more processors 21 and corresponding memory 22.
The processor 22 may be a central processing unit or other form of processing unit having data processing and/or instruction execution capabilities, and may control other components in the control system 20 to perform desired functions. Memory 22 may include one or more computer program products that may include various forms of computer storage media, such as volatile memory and/or non-volatile memory. The volatile memory may include, for example, random Access Memory (RAM) and/or cache memory (cache), and the like. The non-volatile memory may include, for example, read Only Memory (ROM), hard disk, flash memory, and the like. One or more computer program instructions may be stored on the computer storage medium that can be executed by the processor 22 to implement the decision making methods and/or other desired functions of the various embodiments of the present application described above.
In an example, the control system 20 may further include: an input device 23 and an output device 24, which are interconnected by a bus system and/or other forms of connection mechanisms (not shown). The input device 23 may also comprise, for example, a keyboard, a mouse, etc. The output means 24 may comprise, for example, a display, speakers, a printer, and a communication network and remote output devices connected thereto, etc.
Of course, only some of the components of the control system 20 that are relevant to the present application are shown in fig. 7 for simplicity, components such as buses, input/output interfaces, etc. are omitted. In addition, control system 20 may include any other suitable components depending on the particular application.
The foregoing is only a preferred embodiment of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art, who is within the scope of the present invention, should make equivalent substitutions or modifications according to the technical scheme of the present invention and the inventive concept thereof, and should be covered by the scope of the present invention.

Claims (7)

1. A full-automatic breast ultrasound double-arm scanning method is characterized by comprising the following steps:
S1, acquiring a depth image of a chest region, and performing three-dimensional reconstruction on the depth image to form a breast three-dimensional reconstruction model;
s2, generating a scanning track of an ultrasonic probe for scanning on the mammary gland according to the mammary gland three-dimensional reconstruction model;
s3, symmetrically arranging two mechanical arms on the mammary gland, and symmetrically scanning the mammary gland along a scanning track by using ultrasonic probes on the two mechanical arms so as to realize double-arm scanning;
s31, symmetrically arranging two mechanical arms on a mammary gland by taking a vertical axis of a nipple of the mammary gland as a symmetry axis, wherein one mechanical arm is used as a main scanning arm, and performing a scanning task along a scanning track; the other mechanical arm is used as an auxiliary arm, and simultaneously presses the mammary gland along the other symmetrical scanning track with the same force; the two ultrasonic probes are in a symmetrical state;
s32, enabling a six-dimensional force sensor arranged on the mechanical arm to directly read the stress of the ultrasonic probe through gravity compensation, and then calculating a contact force difference value between the expected contact force and the actual contact force;
S33, obtaining the initial moving speed of the ultrasonic probe according to the speed difference value between two adjacent path points of the ultrasonic probe on the same scanning track;
s34, adjusting the required moving speed of the ultrasonic probe by combining the contact depth to realize a force control mode of force-speed;
The contact force difference is calculated as follows:
Wherein F Z represents the reading of the six-dimensional force sensor in the Z-axis direction, namely the actual contact force, and the positive direction of the Z-axis is vertically upward; f Ce represents the desired contact force; f Ze represents the contact force difference between the desired contact force and the actual contact force;
the force control mode is represented as follows:
Wherein V z (k+1) represents the normal speed of the ultrasonic probe at the scanning track path point at the time k+1; v z (k) represents the normal velocity of the ultrasound probe at the scan trajectory path point at time k; both C z1 and C z2 are adjustable parameters.
2. The full-automatic breast ultrasound double-arm scanning method according to claim 1, wherein the specific steps of step S1 are as follows:
s11, a patient to be inspected is laid on an inspection bed, the chest area is placed in a limited space which can be photographed by a depth camera, and then the chest area of the patient to be inspected is covered by surgical cloth and the mammary gland of the patient to be inspected is exposed;
s12, shooting a chest region of a patient to be inspected by using a depth camera to obtain a chest depth image containing a breast and surgical cloth, and converting the chest depth image into a corresponding chest point cloud;
s13, constructing a mechanical arm base coordinate system, wherein the mechanical arm base coordinate system forms a three-dimensional space; acquiring pose information of an ultrasonic probe positioned at the tail end of the mechanical arm in a mechanical arm base coordinate system through hand-eye calibration, and acquiring pose information of chest point cloud in the mechanical arm base coordinate system through coordinate conversion;
s14, registering the breast point cloud through an ICP algorithm to obtain registered breast point cloud;
and S15, performing three-dimensional reconstruction on the registered breast point cloud through a Poisson curved surface generation algorithm to obtain a breast three-dimensional reconstruction model.
3. The full-automatic breast ultrasound double-arm scanning method according to claim 2, wherein the specific contents of step S2 are as follows:
S21, finding out the highest point of the breast three-dimensional model on the Z axis in a mechanical arm base coordinate system, taking the projection of the highest point on the XY plane along the Z axis as a circle center, wherein the radius of the circle where the circle center is located is the radius of the smallest circle surrounding the breast root in the breast three-dimensional model, and the radius of the smallest circle is the scanning radius;
S22, generating a scanning track on the basis of a scanning radius, wherein the generated scanning tracks are of three types, namely a parallel moving type ultrasonic scanning track, a rotary type ultrasonic scanning track and a radial type ultrasonic scanning track;
the generation process of the parallel moving type ultrasonic scanning track specifically comprises the following steps: firstly, an ultrasonic probe slides on the outer contour of a mammary gland along the X-axis direction or the Y-axis direction of a three-dimensional space to form a central curve scanning track, and the central curve scanning track passes through the highest point of a mammary gland three-dimensional model on the Z axis in the three-dimensional space; then, sequentially defining curve-type scanning tracks with the distance between the center curve-type scanning track and the scanning track which is set by the center curve-type scanning track along the direction of the center curve-type scanning track; generating a plurality of curve-type scanning tracks on two sides of the central curve-type scanning track in sequence according to the demarcation mode, wherein the distances between adjacent curve-type scanning tracks are the scanning track distances; a central curve type scanning track and a plurality of curve type scanning tracks are the parallel moving type ultrasonic scanning tracks;
The generation process of the rotary ultrasonic scanning track specifically comprises the following steps: generating a plurality of annular scanning tracks with equal height lines on the mammary gland outline along the direction from the highest point of the mammary gland three-dimensional model on the Z axis to the smallest circular edge in the three-dimensional space, wherein the distances between adjacent annular scanning tracks are the distance between the scanning tracks; the annular scanning track is the rotary ultrasonic scanning track;
The generation process of the radial ultrasonic scanning track specifically comprises the following steps: taking the highest point in the step S21 as a starting point, sliding from the highest point to the smallest circle on the surface of the mammary gland along the scanning radial direction to form a radial ultrasonic scanning track along the scanning radial direction; in this way, the movement is repeated, a plurality of radial ultrasonic scanning tracks are formed along the circumference of the minimum circle, and the included angles between projections of adjacent radial ultrasonic scanning tracks on the XY plane are the same.
4. A fully automatic breast ultrasound double-arm scanning method according to claim 3, wherein the scanning track spacing is calculated as follows:
wherein DeltaL is the scanning track interval, L is the scanning width of the ultrasonic probe, and alpha is the overlapping rate between adjacent scanning tracks.
5. The full-automatic breast ultrasound double-arm scanning method according to claim 4, wherein the calculation formula of the number of radial ultrasound scanning tracks is as follows:
Wherein N tr represents the number of radial ultrasonic scanning tracks, R is the scanning radius, ceil represents the upward rounding operation.
6. A device applying the full-automatic breast ultrasound double-arm scanning method according to any one of claims 1-5, comprising an image acquisition module, a track generation module, a scanning control module and a diagnosis module;
an image acquisition module: for acquiring a depth image of a chest region of a user;
The track generation module: the method comprises the steps of carrying out three-dimensional reconstruction according to a depth image of a chest region to obtain a breast three-dimensional reconstruction model, and planning a corresponding parallel moving type ultrasonic scanning track, a rotary type ultrasonic scanning track and a radial type ultrasonic scanning track according to the breast three-dimensional reconstruction model;
And the scanning control module is used for: the ultrasonic scanning device is used for controlling the scanning mechanism to drive the ultrasonic probe to carry out ultrasonic scanning on the mammary gland of the user according to the parallel moving type ultrasonic scanning track, the rotary type ultrasonic scanning track and the radial type ultrasonic scanning track;
the scanning mechanism comprises a detection platform for a patient to lie on, two groups of mechanical arms are respectively arranged on two sides of the detection platform, and end effectors are respectively arranged at the free ends of the two groups of mechanical arms; the end effector comprises a depth camera shooting a mammary gland region, a clamp is connected to a fixed plate beside the depth camera through a six-dimensional force sensor controlling the compression force of the mammary gland, and an ultrasonic probe is arranged at the clamping end of the clamp in a clamping manner;
and a diagnosis module: the method comprises the steps of analyzing and processing an acquired ultrasonic image to generate a diagnosis result; and a service module: and the system is used for generating a review plan and/or a health insurance scheme according to the diagnosis result.
7. A control system comprising a processor, an input device, an output device, and a memory, the processor, the input device, the output device, and the memory being connected in sequence, the memory being configured to store a computer program comprising program instructions, the processor being configured to invoke the program instructions to perform a fully automated breast ultrasound dual-arm scanning method according to any of claims 1-5.
CN202410844105.9A 2024-06-27 2024-06-27 Full-automatic breast ultrasound double-arm scanning method, device and control system Active CN118383801B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202410844105.9A CN118383801B (en) 2024-06-27 2024-06-27 Full-automatic breast ultrasound double-arm scanning method, device and control system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202410844105.9A CN118383801B (en) 2024-06-27 2024-06-27 Full-automatic breast ultrasound double-arm scanning method, device and control system

Publications (2)

Publication Number Publication Date
CN118383801A CN118383801A (en) 2024-07-26
CN118383801B true CN118383801B (en) 2024-09-06

Family

ID=91996067

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202410844105.9A Active CN118383801B (en) 2024-06-27 2024-06-27 Full-automatic breast ultrasound double-arm scanning method, device and control system

Country Status (1)

Country Link
CN (1) CN118383801B (en)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101166950A (en) * 2005-04-25 2008-04-23 瑞尼斯豪公司 Method for scanning workpiece surfaces
CN114748101A (en) * 2022-06-15 2022-07-15 深圳瀚维智能医疗科技有限公司 Ultrasonic scanning control method, system and computer readable storage medium

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6574499B1 (en) * 1998-11-25 2003-06-03 Xdata Corporation Mammography method and apparatus
CN113842165B (en) * 2021-10-14 2022-12-30 合肥合滨智能机器人有限公司 Portable remote ultrasonic scanning system and safe ultrasonic scanning compliance control method
CN115179279B (en) * 2022-06-21 2025-02-18 深圳瀚维智能医疗科技有限公司 Control method and device of mechanical arm, mechanical arm and readable storage medium
CN117392109A (en) * 2023-11-10 2024-01-12 哈尔滨工业大学 A three-dimensional reconstruction method and system for breast lesions
CN117351037B (en) * 2023-12-04 2024-02-09 合肥合滨智能机器人有限公司 Trajectory planning method for breast isometric scanning for rotational and parallel movements

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101166950A (en) * 2005-04-25 2008-04-23 瑞尼斯豪公司 Method for scanning workpiece surfaces
CN114748101A (en) * 2022-06-15 2022-07-15 深圳瀚维智能医疗科技有限公司 Ultrasonic scanning control method, system and computer readable storage medium

Also Published As

Publication number Publication date
CN118383801A (en) 2024-07-26

Similar Documents

Publication Publication Date Title
Yang et al. Automatic 3-D imaging and measurement of human spines with a robotic ultrasound system
US12453602B2 (en) Ultrasonic puncture guidance planning system based on multi-modal medical image registration using an iterative closest point algorithm
CN101371786B (en) A method and system for three-dimensional reconstruction of X-ray images
US20190295250A1 (en) Method, apparatus and system for reconstructing images of 3d surface
CN112270993B (en) An online decision-making method and system for ultrasound robots using diagnostic results as feedback
CN112151169B (en) A human-like operating ultrasonic robot autonomous scanning method and system
CN117351037B (en) Trajectory planning method for breast isometric scanning for rotational and parallel movements
CN112132805B (en) Ultrasonic robot state normalization method and system based on human body characteristics
Tan et al. Fully automatic dual-probe lung ultrasound scanning robot for screening triage
CN104771189B (en) Three-dimensional head image aligns method and device
CN102488528B (en) Correcting method for geometric parameters of tomography
CN113057678A (en) A breast ultrasound scanning method and system based on binocular vision and robot
CN117392109A (en) A three-dimensional reconstruction method and system for breast lesions
CN118628538B (en) Registration method between 3D sensor of spinal robot and DICOM image
CN111292311B (en) Method and system for calculating volume of macular center avascular zone based on OCTA retinal image
CN115067993A (en) Ultrasonic probe control method and device, terminal equipment and storage medium
CN103188997A (en) Imaging apparatus for imaging an object
CN110689080B (en) Planar atlas construction method of blood vessel structure image
CN103761767A (en) Quick three-dimensional ultrasound image reconstruction method based on sparse data
JP2004024739A (en) Method and apparatus for stereoscopic display and coordinate measurement of fundus
CN115922725B (en) A throat swab sampling robot positioning system
CN118383801B (en) Full-automatic breast ultrasound double-arm scanning method, device and control system
Tang et al. Autonomous ultrasound scanning robotic system based on human posture recognition and image servo control: an application for cardiac imaging
CN117731324A (en) Method and device for real-time force interactive control of ultrasonic probe on contact surface
CN112862947B (en) Image scanning method and system based on three-dimensional ultrasonic probe

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant