CN119423814A - Dynamic three-dimensional ultrasonic spine detection method and imaging system - Google Patents
Dynamic three-dimensional ultrasonic spine detection method and imaging system Download PDFInfo
- Publication number
- CN119423814A CN119423814A CN202310974513.1A CN202310974513A CN119423814A CN 119423814 A CN119423814 A CN 119423814A CN 202310974513 A CN202310974513 A CN 202310974513A CN 119423814 A CN119423814 A CN 119423814A
- Authority
- CN
- China
- Prior art keywords
- spine
- image
- ultrasonic
- dimensional
- dynamic
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Ultra Sonic Daignosis Equipment (AREA)
- Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
Abstract
本发明公开了一种动态三维超声脊柱检测方法与成像系统，属于医学领域，该方法采用成熟的三维脊柱超声成像系统，配合穿戴式无线超声探头采集实时超声图像，深度摄像机实时记录患者在弯曲脊柱时脊柱特征点的空间坐标变化，将三维超声图像、实时超声图像、背部形态动态三维图像进行融合，得到动态三维超声脊柱图像，并据此结合深度学习算法进行Lenke分型参数测算和对脊柱进行动态综合评估，使得脊柱侧弯的分型从传统的基于有辐射危害的X光平面评定方式上升到无辐射的动态三维超声分析，解决了现有技术中静态情况采集到的数据错误率高及采集参数不足的问题。
The present invention discloses a dynamic three-dimensional ultrasonic spine detection method and imaging system, which belongs to the field of medicine. The method adopts a mature three-dimensional spine ultrasonic imaging system, cooperates with a wearable wireless ultrasonic probe to collect real-time ultrasonic images, and a depth camera records in real time the spatial coordinate changes of the spine feature points of the patient when the spine is bent. The three-dimensional ultrasonic image, the real-time ultrasonic image, and the dynamic three-dimensional image of the back morphology are fused to obtain a dynamic three-dimensional ultrasonic spine image, and based on this, a deep learning algorithm is combined to calculate Lenke classification parameters and perform a dynamic comprehensive evaluation of the spine, so that the classification of scoliosis is upgraded from the traditional X-ray plane assessment method with radiation hazards to a radiation-free dynamic three-dimensional ultrasonic analysis, which solves the problems of high error rate and insufficient acquisition parameters of data collected in static conditions in the prior art.
Description
Technical Field
The invention relates to the field of medicine, in particular to a dynamic three-dimensional ultrasonic spine detection method and an imaging system for acquiring a Lenke typing parameter of scoliosis.
Background
Scoliosis refers to a spinal deformity in which one or more segments of the spine bend laterally or accompany rotation of the vertebral body. Adolescent idiopathic scoliosis (adolescent idiopathic scoliosis, AIS) is the most common form of scoliosis. Diagnosis of scoliosis is currently mainly performed by X-rays, and Cobb angle measurement is used to measure scoliosis angle.
Three-dimensional spinal ultrasound imaging has also found many clinical applications internationally, and a commercial three-dimensional spinal ultrasound system, represented by Scolioscan, is effective and reliable in measuring coronal deformities in scoliosis patients, and helps reduce radiation dose during continuous monitoring, as compared to conventional X-ray Cobb angle gold standards in screening a large number of patients, monitoring spinal changes, and evaluating treatment results.
However, the three-dimensional spine ultrasonic imaging system is used for detecting static indexes, namely, a patient is required to be unfolded and scanned under the condition of static state, the body state of a detected person can have adverse effect on the result in the scanning process, and the state of human body movement corresponding to the detected person cannot be confirmed when the detection result is judged, so that the error rate of data acquired under the static condition is high, and the guidance effect on scoliosis operation and non-operation treatment decisions is limited.
In recent years, the machine vision technology is also applied to scoliosis, a color image and a depth image are acquired through a depth camera, three-dimensional reconstruction is carried out on the back surface information of a human body, and the Cobb angle can be calculated according to the three-dimensional contour of the back. This method, which relies solely on machine vision, is severely demanding for the imaging environment and poorly suited for patients with insignificant lumbar convexity and insignificant back midline.
The static and dynamic comprehensive evaluation of the spine is needed for the surgical and non-surgical treatment decisions of scoliosis, and the static and dynamic comprehensive evaluation mainly depends on the overall balance degree of ① spines, the real-time activity degree of scoliosis, the flexibility and stability of spines, the opening degree of the intervertebral space near the top vertebrae and the upper and lower vertebrae during ② scoliosis activity, and the improvement of the rotation degree of the vertebral bodies of the top vertebral areas of ③. However, the evaluation of all the parameters cannot be performed at present by simply relying on the techniques of X-ray, three-dimensional spine ultrasonic imaging, machine vision and the like.
Disclosure of Invention
In view of the foregoing, it is necessary to provide a dynamic three-dimensional ultrasonic spine detection method and an imaging system, so as to solve the problems of high data error rate and insufficient acquisition parameters acquired in the static condition in the prior art, and better provide guidance for surgical and non-surgical treatment decisions of scoliosis.
To achieve the above object, the present invention provides a dynamic three-dimensional ultrasonic spine detection method using a computer terminal 4 communicatively connected to a three-dimensional spine ultrasonic imaging device 1, a wearable wireless ultrasonic scanning unit 2, a depth camera 3 with an optical sensor built therein, the method comprising the steps of:
S1) carrying out three-dimensional ultrasonic scanning on the spine of a patient by using the three-dimensional spine ultrasonic imaging equipment 1 to obtain a three-dimensional spine image 100 of the patient, wherein the three-dimensional spine image comprises a spine projection image of a coronal plane and a sagittal plane;
s2) acquiring the positions of one or more main curved top vertebrae and upper and lower vertebrae of scoliosis from a coronal plane spine projection image in the three-dimensional spine image 100, taking the fact that complete reflecting surfaces of vertebral plates on two sides are seen in the field of view of an ultrasonic image as the reference, respectively fixing ultrasonic probes of the wearable wireless ultrasonic scanning unit 2 at the positions of the acquired curved top vertebrae, upper vertebrae and lower vertebrae to acquire a real-time ultrasonic image 200 of the corresponding positions, wherein an optical marker is arranged on the outer end face of each ultrasonic probe;
s3) directing the depth camera 3 with the built-in optical sensor towards the back of the patient to obtain real-time video, wherein the real-time video comprises a multi-frame back depth image, an optical identifier and a dynamic back image 300 of a body surface image;
S4) the patient bends the trunk according to the guide until reaching the required bending degree, and the computer terminal acquires a dynamic back image 300 comprising a back depth image and a body surface image and a real-time ultrasonic image 200 in real time during the spinal bending action, wherein the body surface image is a color image or a gray scale image;
s5) acquiring real-time three-dimensional space positions and directions of each ultrasonic probe in the spinal bending action process by using the optical markers in the body surface map;
S6) the computer terminal 4 respectively carries out synchronization and fusion processing on the real-time ultrasonic image 200 and the back depth image in the dynamic back image 300 and the three-dimensional spine image 100 to obtain a dynamic three-dimensional ultrasonic image during the spine bending action, wherein the real-time three-dimensional space position and direction of the ultrasonic probe obtained in the step S5) are used as reference information in the image synchronization and fusion processing process;
S7) on a computer terminal, performing spine bending morphological analysis according to the dynamic three-dimensional ultrasonic image fused in the step S6), wherein the analysis comprises Lenke typing parameter calculation, analyzing the real-time activity degree, flexibility and stability of the scoliosis, and evaluating the opening degree of the vertebral gap and the rotating condition of the vertebral body near the top vertebral column and the upper and lower vertebral columns during the lateral bending activity.
The invention also provides a dynamic three-dimensional ultrasonic spine imaging system, which comprises the existing three-dimensional spine ultrasonic imaging equipment 1 and further comprises:
the wearable wireless ultrasonic scanning unit 2 is used for generating a local two-dimensional ultrasonic image frame of the spine and comprises three or more ultrasonic probes;
The depth camera unit 3 is used for recording the spatial position change of the ultrasonic probe and the back dynamic three-dimensional form, and comprises a depth camera with an optical sensor;
The computer terminal 4 is connected with the three-dimensional spine ultrasonic imaging device 1, the wearable wireless ultrasonic scanning unit 2 and the depth camera unit 3, and is used for receiving a three-dimensional spine image 100 of the three-dimensional spine ultrasonic imaging device 1, a real-time ultrasonic image 200 of the wearable wireless ultrasonic scanning unit 2 and a dynamic back image 300 of the depth camera unit 3, and performing spine bending morphological analysis by fusing the three-dimensional spine image 100, the real-time ultrasonic image 200 and the dynamic back image 300, including measuring a Lenke typing parameter of scoliosis and dynamically evaluating the spine in lateral flexion activities.
The method and the system adopt a mature three-dimensional spine ultrasonic imaging system, are matched with a wearable wireless ultrasonic probe to acquire real-time ultrasonic images, and are matched with a depth camera with an internal optical sensor to record the spatial coordinate change of spine characteristic points in side flexion of a patient in real time, and fuse the three-dimensional ultrasonic images acquired by the three-dimensional spine ultrasonic imaging system in upright and bending states, the real-time ultrasonic images acquired by the wearable wireless ultrasonic system and the back form dynamic three-dimensional images acquired by the depth camera to obtain dynamic three-dimensional ultrasonic spine images. According to dynamic three-dimensional ultrasonic spine images, computer vision real-time feedback of deep learning is combined, and the Lenke typing parameters of scoliosis are rapidly and accurately measured, and dynamic comprehensive evaluation is carried out on the overall balance degree of the spine, the real-time activity degree of scoliosis, the flexibility and stability of the spine, the opening degree of the vertebral gap near the top vertebra and the end vertebra during scoliosis and the improvement of the rotation degree of the vertebral body at the top vertebral area, so that the typing of scoliosis is increased from a traditional X-ray plane assessment mode based on radiation hazard to non-radiation dynamic three-dimensional ultrasonic analysis, and the problems of high data error rate and insufficient acquisition parameters acquired under the static condition in the prior art are solved. The invention may also be applied to lordosis and other spinal curvature tests where desired, particularly procedures and lateral flexion analogs.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description serve to explain the application and do not constitute a limitation on the application. In the drawings:
fig. 1 is a numbered schematic view of a human thoracic and lumbar vertebrae according to an embodiment of the present invention.
Fig. 2 is a logic block diagram of a dynamic three-dimensional ultrasound spinal imaging system provided by an embodiment of the present invention.
Fig. 3 is a schematic diagram of a key technology of a dynamic three-dimensional ultrasonic spine imaging system according to an embodiment of the present invention.
Fig. 4 is a schematic diagram of steps of a dynamic three-dimensional ultrasonic spine detection method according to an embodiment of the present invention.
Fig. 5 is a two-dimensional ultrasound image of a spinal cross-section provided in an embodiment of the present invention.
Fig. 6 is a schematic diagram of a connection line between transverse processes on two sides of a vertebra at a three-dimensional spine volume projection imaging image end and a schematic diagram of a connection line between transverse processes on two sides of a vertebra at an X-ray film end according to an embodiment of the present invention.
Fig. 7 is an effect diagram of identifying and labeling an ultrasonic probe position in a fused image of a depth image of a back of a depth camera body and three-dimensional ultrasonic volume projection imaging according to an embodiment of the present invention.
Detailed Description
The embodiment of the application solves the problems of high data error rate and insufficient acquisition parameters of static condition acquisition in the prior art by providing the dynamic three-dimensional ultrasonic spine detection method and the imaging system, so that the typing of scoliosis is improved from a traditional X-ray plane assessment mode based on radiation hazard to non-radiation dynamic three-dimensional ultrasonic analysis.
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments of the present invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Embodiments of the invention are described in further detail below with reference to the drawings. It is to be understood that the embodiments described herein are for illustration and explanation of the invention only and are not intended to limit the invention.
In practical clinical diagnosis, doctors have corresponding names for each vertebra of the spine, and the human spine has 33 vertebrae in total, wherein the number of the vertebrae is 7, the number of the thoracic vertebrae is 12, the number of the lumbar vertebrae is 5, and the total number of the sacrum and the coccyx is nine. In the scoliosis Lenke typing system, cobb angle measurement does not involve the cervical spine as well as the sacrum and coccyx, etc. As shown in FIG. 1, the numbering scheme of the thoracic vertebrae and lumbar vertebrae of a human body is shown, L5-L1 from bottom to top represents lumbar vertebrae, and T12-T1 represents thoracic vertebrae.
One embodiment of the present invention provides a dynamic three-dimensional ultrasonic spinal imaging system, as shown in fig. 2, which is a logic block diagram of the system, including a three-dimensional spinal ultrasonic imaging device 1, and further including:
the wearable wireless ultrasonic scanning unit 2 is used for generating a local two-dimensional ultrasonic image frame of the spine and comprises three or more ultrasonic probes;
The depth camera unit 3 is used for recording the spatial position change of the ultrasonic probe and the back dynamic three-dimensional form, and comprises a depth camera with an optical sensor;
The computer terminal 4 is connected with the three-dimensional spine ultrasonic imaging device 1, the wearable wireless ultrasonic scanning unit 2 and the depth camera unit 3, and is used for receiving a three-dimensional spine image 100 of the three-dimensional spine ultrasonic imaging device 1, a real-time ultrasonic image 200 of the wearable wireless ultrasonic scanning unit 2 and a dynamic back image 300 of the depth camera unit 3, obtaining a dynamic three-dimensional ultrasonic spine image by fusing the three-dimensional spine image, the real-time ultrasonic image and the dynamic back image, carrying out spine bending morphological analysis, measuring a Lenke typing parameter of scoliosis, and carrying out dynamic evaluation on a spine in lateral bending activities.
The three-dimensional spine ultrasonic imaging device 1 is mature three-dimensional spine ultrasonic imaging device, acquires a three-dimensional image 100 of a spine after volume reconstruction by acquiring ultrasonic pictures of the spine of a patient, and can further acquire projection pictures of the spine in a coronal plane and a sagittal plane by a volume projection imaging method.
The wearable wireless ultrasonic scanning unit 2 comprises ultrasonic probes and a wireless controller, and in the embodiment of the invention, the wearable wireless ultrasonic scanning unit 2 comprises 3 or more probes, and the ultrasonic probes acquire real-time ultrasonic images 200 of the spine.
The depth camera unit 3 includes a depth camera with built-in optical sensors and a support, on which the depth camera is positioned to be aligned with the back of the patient, and acquires a dynamic back image 300 of the patient.
The computer terminal comprises a display terminal for displaying the three-dimensional spine image 100, the real-time ultrasound image 200, the dynamic back image 300 and the fused image.
Fig. 3 is a schematic diagram of a dynamic three-dimensional ultrasonic spine imaging system according to an embodiment of the present invention, where the three-dimensional spine ultrasonic imaging system uses a volumetric projection imaging method to provide a projection view of a patient's spine on a coronal plane, from which the curvature of each vertebral body of the spine can be seen. For each main bend, positioning the positions of the top vertebra, the upper end vertebra and the lower end vertebra, installing 3 ultrasonic probes at the corresponding positions of the patient vertebra, respectively corresponding to the positions of the top vertebra, the upper end vertebra and the lower end vertebra, respectively positioning the top vertebra, the upper end vertebra and the lower end vertebra at the center positions of scanning windows of the 3 probes, and finely adjusting the positions of the probes by combining the real-time ultrasonic images of the 3 probes displayed by the terminal of a computer, so that complete reflecting surfaces of vertebral plates at two sides are seen in the field of ultrasonic images. The 3 probes are provided with optical markers on the outward cross section, and each probe is connected with the wireless controller by wires.
An embodiment of the present invention provides a dynamic three-dimensional ultrasonic spine detection method, as shown in fig. 4, which is a schematic diagram of steps of the dynamic three-dimensional ultrasonic spine detection method according to the embodiment of the present invention, including the following steps:
s1) three-dimensional ultrasound scanning of the spine of a patient with the three-dimensional spine ultrasound imaging apparatus 1, a three-dimensional spine image 100 thereof including a coronal plane and a sagittal plane of the spine projection image is obtained.
S2) acquiring the positions of one or more main curved vertebras, an upper end vertebra and a lower end vertebra of the scoliosis from the coronary surface spine projection image in the three-dimensional spine image 100, taking the view of the ultrasonic image as the standard of seeing complete reflecting surfaces of vertebral plates at two sides, respectively fixing the ultrasonic probe of the wearable wireless ultrasonic scanning unit 2 at the positions of the acquired curved vertebras, the upper end vertebra and the lower end vertebra to acquire a real-time ultrasonic image 200 at the corresponding positions, wherein the outer end surface of the ultrasonic probe is provided with an optical marker.
S3) the depth camera 3 with the built-in optical sensor is directed towards the back of the patient, and a real-time video is obtained, including a multi-frame back depth map, an optical marker and a dynamic back image 300 of the body surface map.
S4) the patient bends the trunk according to the guide until reaching the required bending degree, the computer terminal acquires a dynamic back image 300 of a dynamic back depth image and a body surface image during the spine bending action in real time and a real-time ultrasonic image 200, wherein the body surface image is a color image or a gray scale image, and the trunk is bent according to the guide, and comprises lateral bending, forward bending or any required test posture or bends to any bending degree suitable for examination.
S5) acquiring the real-time three-dimensional space position and direction of each ultrasonic probe in the spinal bending action process by using the optical markers in the body surface chart.
S6) the computer terminal 4 respectively carries out synchronization and fusion processing on the real-time ultrasonic image 200 and the back depth map in the dynamic back image 300 and the three-dimensional spine image 100 to obtain a dynamic three-dimensional ultrasonic image during the spine bending action, wherein the real-time three-dimensional space position and direction of the ultrasonic probe obtained in the step S5) are used as reference information in the image synchronization and fusion processing process.
S7) on a computer terminal, performing spine bending morphological analysis according to the dynamic three-dimensional ultrasonic image fused in the step S6), wherein the analysis comprises Lenke typing parameter calculation, analyzing the real-time activity degree, flexibility and stability of the scoliosis, and evaluating the opening degree of the vertebral gap and the rotating condition of the vertebral body near the top vertebral column and the upper and lower vertebral columns during the lateral bending activity.
The ultrasonic probe comprises a patch, a cable, a sticking fixing device and a wireless controller, wherein the wireless controller transmits patch ultrasonic signals conducted by the cable to a nearby computer terminal 4 through WiFi.
The ultrasound probe may be a patch ultrasound probe, a flexible ultrasound probe, a one-dimensional array or a two-dimensional array ultrasound probe.
The flexible ultrasonic probe can be an array probe and a unit probe, and in the embodiment of the invention, the flexible ultrasonic probe can be a one-dimensional array or a two-dimensional array probe, and can generate focused and clear sound beams in a local detection plane, so that higher resolution is obtained.
In the embodiment of the invention, the depth camera 3 with the built-in optical sensor can be arranged on a bracket, and the bracket can control the depth camera to move in 6 directions of front, back, up, down, left and right so that the back of a human body is always in the visual field of the camera when the spine is subjected to bending action.
In an embodiment of the method of the present invention, step S1 comprises performing an ultrasound scan of the spine of the patient in an upright standing position using the three-dimensional spine ultrasound imaging device 1, obtaining a three-dimensional spine image 100 of the patient in the upright standing position, including a projection image of the spine in the coronal and sagittal planes.
In another embodiment of the invention, step S1 further comprises ultrasonically scanning the patient' S spine in one or more different curved states using the three-dimensional spinal ultrasound imaging device 1 to obtain three-dimensional spinal images 100 of the patient in one or more different curved states, including coronal and sagittal spinal projection images. The spinal curvature is a dynamic three-dimensional shape of the spine of the patient when subjected to lateral flexion, anterior flexion, or other bending actions for spinal assessment.
In the embodiment of the present invention, if the main bending part obtained from the coronal plane spine projection image in step S2 is greater than one, the steps S2-S6 are repeated for each other bending part to acquire data.
In the embodiment of the present invention, in the step S2, the step of obtaining the positions of the scoliosis main curved top vertebra and the upper and lower end vertebrae from the coronal plane spine projection image in the three-dimensional spine image 100 includes the following steps:
s21) numbering each vertebral body;
s22) determining the number of primary bends;
s23) for each curvature, the numbers of the top, upper and lower vertebrae are identified and recorded.
The pictures taken by a common color camera can see all objects within the camera's view angle and record, but the data recorded does not contain the distance of these objects from the camera. Only by semantic analysis of the image can it be determined which objects are farther from us and which are closer, but there is no exact data. The invention adopts the depth camera to precisely solve the problem, the depth camera can acquire a color image (or a gray level image) and a depth image at the same time, and the distance between each point in the image and the camera can be accurately known through the data of the depth image, so that the three-dimensional space coordinate of each point in the image can be acquired by adding the (x, y) coordinate of the point in the 2D image. The real scene can be restored through the three-dimensional coordinates, and the application of scene modeling and the like is realized. Therefore, the real-time three-dimensional dynamic image of the patient shot by the depth camera can be effectively fused with the real-time ultrasonic image and the three-dimensional ultrasonic image acquired by the three-dimensional ultrasonic system, and spine parameter measurement and parting of the dynamic three-dimensional layer are performed.
Image fusion refers to fusing two or more images together to produce a fused image. In the process of image fusion, key information of each original image can be reserved, and meanwhile, detail information in different images is utilized for correction and enhancement, so that a clearer and clearer image is generated.
Image fusion is typically performed on multiple images acquired from different sources or different sensors. Through fusion, the noise problem existing in the images acquired by different sensors can be eliminated, the quality and resolution of the images can be improved, and meanwhile, the recognition and analysis capability of the images can be improved, so that a better foundation is provided for further processing and analysis of the images.
Image fusion may take a variety of approaches, such as pixel-level based fusion, feature-level based fusion, advanced neural network based fusion, and so forth. These methods rely on specific features of the image such as spatial resolution, color, gray scale, texture, etc.
In the embodiment of the present invention, the specific steps of fusing and synchronizing the real-time ultrasound image 200 with the back depth map in the dynamic back image 300 in the step S6 are as follows:
S61) identifying the optical identifier on each ultrasonic probe through an image identification algorithm, wherein the optical identifier in the embodiment of the invention is in a two-dimensional code form, and can also adopt other plane patterns or three-dimensional forms with obvious texture characteristics, and the invention is not limited to the two-dimensional patterns or the three-dimensional forms.
S62) obtaining the real-time three-dimensional space position and direction of each ultrasonic probe through image analysis;
S63) fusing the real-time ultrasonic image acquired by each probe with the back depth image according to the position and direction information, wherein the specific method of the fusion is to synchronize the time of the real-time ultrasonic image and the back depth image, unify the real-time ultrasonic image and the back depth image of the patient at the same time or similar time to the same coordinate system, acquire the corresponding depth information for the pixel points in the real-time ultrasonic image range, and realize the fusion of the real-time ultrasonic image and the back depth image;
The fusing of the real-time ultrasonic image 200 and the three-dimensional spine image 100 is based on the position of each ultrasonic probe relative to the spine, and the real-time ultrasonic image acquired by the ultrasonic probe is fused with the three-dimensional spine image, so that the real-time spine image characteristics of the local area corresponding to the ultrasonic probe and the overall spine information of the three-dimensional spine image are fused to obtain a dynamic three-dimensional spine ultrasonic image. The fusion method specifically comprises the steps of extracting characteristic points of spinous processes, vertebral lamina, transverse processes and the like from real-time ultrasonic images acquired by probes based on the positions of the probes relative to the spine, and realizing fusion matching of two-dimensional ultrasonic images and three-dimensional spine images by one-to-one correspondence between the extracted characteristic points and the three-dimensional spine images through characteristic matching and fusion methods.
Lenke typing is mainly aimed at idiopathic scoliosis, and plays an important role in diagnosis of scoliosis and treatment design. Lenke typing consists of three parts, coronal lateral curvature type (1-6), lumbar vertebra correction type (A, B, C), thoracic sagittal sequence correction type (-, N, +).
In the Lenke typing, the bending type is determined according to the apical section, and the corresponding relationship between the thoracic and lumbar bending type and the apical section is shown in table 1.
Table 1 correspondence table between thoracic and lumbar bending types and apical intervals
| Type of chest and waist bending | Top vertebral section |
| Upper chest curve (proximal thoracic PT) | T2-T5 |
| Main chest bend (main thoracic, MT) | T6-T11/12 |
| Chest and waist bend (thoracolumbar TL) | T12、L1 |
| Waist bend (lumbar, L) | L1/2-L4 |
After determining the type of curvature, the major and minor curvature, structural curvature and non-structural curvature are determined, and then the coronal lateral curvature type (1-6) is determined by referring to table 2.
TABLE 2Lenke coronal lateral bending type (1-6)
| Type(s) | Upper chest curve | Main chest curve | Chest and waist bend/waist bend | Type of bending |
| 1 | Unstructured feature | Structural x | Unstructured feature | Main chest curve |
| 2 | Structural design | Structural x | Unstructured feature | Double chest bend |
| 3 | Unstructured feature | Structural x | Structural design | Double main bend |
| 4 | Structural design | Structural x | Structural x | Three main bends |
| 5 | Unstructured feature | Unstructured feature | Structural x | Chest and waist bend/waist bend |
| 6 | Unstructured feature | Structural design | Structural x | Chest and waist bend/waist bend-main chest bend |
Note that the main bend position is shown
The specific steps of calculating the Lenke typing parameters according to the fused images in the step S6 in the computer software in the step S7 are as follows:
S71) obtaining a Cobb angle of a coronal plane and a Cobb angle of a sagittal plane when standing up according to the fusion image of the standing up posture, and determining the top vertebral position;
s72) obtaining the change condition of the Cobb angle during lateral buckling according to the fusion image during lateral buckling;
s73) judging the type of the bend, primary and secondary bends, structural bend and non-structural bend according to the top vertebral position in the step S71) and the Cobb angle in the step 72), and determining the side bend type of the Lenke parting by combining with the Lenke parting principle;
S74) obtaining a Lenke-typed lumbar vertebra correction type according to the relative relation between the position of the lumbar bending apex vertebra and the central perpendicular line of the sacrum, S75) determining a Lenke-typed thoracic vertebra sagittal sequence correction type according to the Cobb angle of the sagittal plane when standing upright.
The static and dynamic comprehensive evaluation of the spine is needed for surgical and non-surgical treatment decisions of scoliosis, and the static and dynamic comprehensive evaluation mainly depends on the overall balance degree of ① spines, the real-time activity degree of scoliosis, the flexibility and stability of spines, the opening degree of the intervertebral space near the top vertebrae and the end vertebrae during ② scoliosis activity, and the improvement of the rotation degree of the vertebral bodies of the top vertebral areas of ③.
Dynamic three-dimensional ultrasound is based on three-dimensional ultrasound, time tracking is added, continuous changes of each vertebra of a patient in the lateral flexion process can be observed through dynamic three-dimensional ultrasound images, and scoliosis evaluation parameters can be calculated in real time. In the step S8, the real-time activity, flexibility and stability of the scoliosis are analyzed, and the degree of opening of the intervertebral space near the top vertebra and the upper and lower end vertebrae during the scoliosis activity is evaluated based on the dynamic three-dimensional ultrasonic image constructed in the step S6 for the dynamic comprehensive evaluation of the spine. The result of the dynamic comprehensive evaluation can form a written report, and provides diagnosis basis for doctors.
Machine learning is classified according to a learning form and can be classified into supervised learning, unsupervised learning, semi-supervised learning, reinforcement learning, and the like. The difference is that supervised learning requires a sample set of labels, unsupervised learning does not require a sample set of labels, semi-supervised learning requires a small number of samples of labels, and reinforcement learning requires a feedback mechanism.
And (3) automatically identifying and automatically measuring and calculating scoliosis evaluation parameters of the dynamic three-dimensional ultrasonic images fused in the step (S6) through deep learning of a large number of the dynamic three-dimensional ultrasonic images fused in the step (S6) and the corresponding scoliosis evaluation parameters thereof.
In one embodiment of the invention, the deep learning includes detection range intra-frame two-dimensional spine cross-section ultrasound image marker identification. The method comprises the steps of acquiring a large number of ultrasonic images of the cross section of the spine, manually marking spinous processes and vertebral plates on two sides of the spine, taking the marked frames as a training set of a supervised deep learning algorithm, simulating manual identification markers and marking targets through training of a large amount of data, and accordingly automatically calculating the rotation degree of the vertebral body and realizing real-time activity tracking of the vertebral body in the bending action of the spine. The morphology of the spinous process in the two-dimensional spine cross section ultrasonic image and the inclination degree of the vertebral plates at two sides play an important role in judging the spatial rotation angle of the vertebral body and in finishing the real-time activity, flexibility and stability of the vertebral body in lateral flexion. As shown in FIG. 5, which is a two-dimensional spine cross-section ultrasonic image, the morphology of the spinous process and the two side vertebral plates is clearly distinguishable, so that the embodiment of the invention adopts a deep learning algorithm to automatically identify the spinous process and the vertebral plates in the two-dimensional spine cross-section ultrasonic image. After a large number of spine cross section ultrasonic images are acquired, the spinous process and the vertebral plates on two sides of the corresponding frame are marked manually, the marked frame is used as a training set of a supervised deep learning algorithm, and the aim of simulating the manual identification marker and marking is achieved through training of a large amount of data, so that the rotation degree of the vertebral body is automatically calculated, and the real-time activity tracking of the vertebral body in lateral flexion is realized.
In one embodiment of the invention, the depth learning further comprises angular measurement of the three-dimensional spine volume projection imaging ultrasound image. Specifically, after a large number of three-dimensional spine volume projection ultrasonic images are acquired, marks of transverse processes on two sides of each curve end are manually made, the marks are used as a training set of a supervised deep learning algorithm, and the aim of simulating artificial identification markers and making marks is achieved through training of a large amount of data, so that the angle of each section of spine bending line of a patient is automatically calculated, and further real-time activity tracking of the change of the angle of the spine bending line in lateral flexion is realized. Clinical three-dimensional spine volume projection imaging ultrasound images are used to measure scoliosis angle (analogous to X-ray Cobb angle). When a scoliosis patient has one or more scoliosis bending lines, the angle formed by the end vertebrae at the two ends of each curve is used for marking the severity of the curve. In the three-dimensional spine volume projection imaging ultrasonic image, the current manual marking mode can be quantified by adopting the connection line angle of transverse processes at two sides of the end vertebrae or the projection of the vertebral lamina of the upper lumbar vertebra and the upper articular processes of the lower lumbar vertebra. Fig. 6 (a) shows a schematic diagram of the connection lines of the transverse processes on two sides of the vertebral end of the three-dimensional spine volume projection imaging image, and fig. 6 (b) shows a schematic diagram of the connection lines of the transverse processes on two sides of the vertebral end of the X-ray film. After a large number of three-dimensional spine volume projection ultrasonic images are acquired, marks of transverse processes on two sides of each curve end are manually made, the marks are used as a training set of a supervised deep learning algorithm, and the aim of simulating manual identification markers and making marks is achieved through training of a large amount of data, so that the angle of each section of spine bending line of a patient is automatically calculated, and further real-time activity tracking of the change of the angle of the spine bending line in side bending is achieved.
In one embodiment of the invention, the deep learning further comprises real-time measurement of the spatial position change of the ultrasound probe. Specifically, an unsupervised learning algorithm is adopted, and the optical identifier is utilized to complete automatic identification, tracking and recording of the position of the ultrasonic probe. In the embodiment, the spatial position change of the ultrasonic probe is captured in real time by a depth camera, so that a spatial coordinate system of the detected intraspinal marker is established. Fig. 7 is an effect diagram of identifying and marking the position of the ultrasonic probe in the fused image of the depth image of the back of the human body of the depth camera and the three-dimensional ultrasonic volume projection imaging. Because the relative position change of the ultrasonic probe in the body surface space coordinate system is different from the anatomical marker observed in the body, and the shape and the material are different from the human body structure, the ultrasonic probe is easy to identify, an unsupervised learning algorithm can be adopted, and the automatic identification, tracking and recording of the position of the ultrasonic probe can be completed by utilizing the optical marker.
The medical image processing method has the advantages that the medical image processing method is more advantageous than the traditional algorithm due to the characteristics of blurring, non-uniformity, individual difference, complexity, diversity and the like, but the acquired limited sample data set needs to be expanded due to the fact that the medical image relates to patient privacy and the acquisition difficulty is high.
The method and the system are characterized in that based on a mature three-dimensional spine ultrasonic imaging system, a real-time ultrasonic image is acquired by matching with a wearable wireless ultrasonic probe, the spatial coordinate change of spine characteristic points when a patient bends sideways is recorded in real time by a depth camera with an optical sensor, and the three-dimensional ultrasonic image acquired by the three-dimensional spine ultrasonic imaging system in an upright and bending state, the real-time ultrasonic image acquired by the wearable wireless ultrasonic system and the back form dynamic three-dimensional image acquired by the depth camera are fused to obtain a dynamic three-dimensional ultrasonic spine image. According to dynamic three-dimensional ultrasonic spine images, computer vision real-time feedback of deep learning is combined, and the Lenke typing parameters of scoliosis are rapidly and accurately measured, and dynamic comprehensive evaluation is carried out on the overall balance degree of the spine, the real-time activity degree of scoliosis, the flexibility and stability of the spine, the opening degree of the vertebral gap near the top vertebra and the end vertebra during scoliosis and the improvement of the rotation degree of the vertebral body at the top vertebral area, so that the typing of scoliosis is increased from a traditional X-ray plane assessment mode based on radiation hazard to non-radiation dynamic three-dimensional ultrasonic analysis, and the problems of high data error rate and insufficient acquisition parameters acquired under the static condition in the prior art are solved. The invention may also be applied to lordosis and other spinal curvature tests where desired, particularly procedures and lateral flexion analogs.
The above description is only of embodiments of the present invention and should not be construed as limiting the scope of the present invention, and it should be understood that the present invention is not limited to the embodiments described above, but is intended to cover all modifications, equivalents, and alternatives known to those skilled in the art.
Claims (18)
1. A dynamic three-dimensional ultrasonic spine detection method is characterized by using a computer terminal (4) in communication connection with three-dimensional spine ultrasonic imaging equipment (1), a wearable wireless ultrasonic scanning unit (2) and a depth camera (3) with an optical sensor built in, and comprises the following steps:
s1) carrying out three-dimensional ultrasonic scanning on the spine of a patient by using the three-dimensional spine ultrasonic imaging equipment (1) to obtain a three-dimensional spine image (100) of the patient, wherein the three-dimensional spine image comprises a spine projection image of a coronal plane and a sagittal plane;
S2) acquiring the positions of one or more main curved vertebras, an upper end vertebra and a lower end vertebra of scoliosis from a coronal plane spine projection image in the three-dimensional spine image (100), taking the view of the complete reflecting surfaces of vertebral plates at two sides in the field of an ultrasonic image as the reference, respectively fixing an ultrasonic probe of the wearable wireless ultrasonic scanning unit (2) at the positions of the acquired curved vertebras, the upper end vertebra and the lower end vertebra to acquire a real-time ultrasonic image (200) at the corresponding positions, wherein an optical marker is arranged on the outer end surface of the ultrasonic probe;
S3) a depth camera (3) with a built-in optical sensor faces the back of a patient to obtain real-time video, wherein the real-time video comprises a multi-frame back depth image, an optical marker and a dynamic back image (300) of a body surface image;
S4) the patient bends the trunk according to the guide until reaching the required bending degree, the computer terminal acquires a dynamic back image (300) comprising a back depth image and a body surface image and a real-time ultrasonic image (200) in real time during the spinal bending action, wherein the body surface image is a color image or a gray scale image, and the trunk is bent according to the guide, comprises lateral bending, forward bending or any required test posture, or is bent to any bending degree suitable for examination;
s5) acquiring real-time three-dimensional space positions and directions of each ultrasonic probe in the spinal bending action process by using the optical markers in the body surface map;
S6) the computer terminal (4) respectively synchronizes and fuses the real-time ultrasonic image (200) with the back depth image in the dynamic back image (300) and the three-dimensional spine image (100) to obtain a dynamic three-dimensional ultrasonic image during the spine bending action, wherein the real-time three-dimensional space position and direction of the ultrasonic probe obtained in the step S5) are used as reference information in the image synchronization and fusion process;
S7) on a computer terminal, performing spine bending morphological analysis according to the dynamic three-dimensional ultrasonic image fused in the step S6), wherein the analysis comprises Lenke typing parameter calculation, analyzing the real-time activity degree, flexibility and stability of the scoliosis, and evaluating the opening degree of the vertebral gap and the rotating condition of the vertebral body near the top vertebral column and the upper and lower vertebral columns during the lateral bending activity.
2. The dynamic three-dimensional ultrasonic spine detection method according to claim 1, wherein the step S1 comprises performing an ultrasonic scan of the spine of the patient in the standing posture using the three-dimensional spine ultrasonic imaging device (1), and acquiring a three-dimensional spine image (100) of the patient in the standing posture, including a spine projection image of a coronal plane and a sagittal plane.
3. The method according to claim 1, wherein the step S1 comprises performing an ultrasonic scan of the spine of the patient in one or more different curved modalities using a three-dimensional spine ultrasonic imaging device (1), and acquiring three-dimensional spine images (100) of the patient in one or more different curved modalities, including coronal and sagittal spine projection images.
4. A dynamic three-dimensional ultrasound spinal column testing method according to claim 3, wherein the spinal curvature profile is a dynamic three-dimensional profile of the spinal column of the patient while performing lateral flexion, anterior flexion, or other bending actions for spinal assessment.
5. The method of claim 1, further comprising step S6 a) between steps S6) and S7), wherein if the number of curved portions obtained from the coronal plane spine projection image in step S2 is greater than one, repeating steps S2-S6 for each other curved portion to acquire data.
6. The method for dynamic three-dimensional ultrasonic spine detection according to claim 1, wherein the step S2 of acquiring the positions of the scoliosis main curved top vertebra and the upper and lower end vertebrae from the coronal plane spine projection image in the three-dimensional spine image (100) comprises the steps of:
s21) numbering each vertebral body;
s22) determining the number of primary bends;
s23) for each curvature, the numbers of the top, upper and lower vertebrae are identified and recorded.
7. The method for dynamic three-dimensional ultrasound spine detection according to claim 1, wherein the specific steps of fusing and synchronizing the real-time ultrasound image (200) with the back depth map in the dynamic back image (300) in step S6 are:
S61) identifying the optical identifier on each ultrasonic probe through an image identification algorithm;
s62) obtaining the real-time three-dimensional space position and direction of each ultrasonic probe through image analysis;
s63) fusing the real-time ultrasonic image acquired by each probe with the back depth map according to the position and direction information;
The fusing of the real-time ultrasonic image (200) and the three-dimensional spine image (100) is based on the position of each ultrasonic probe relative to the spine, and the real-time ultrasonic image acquired by the ultrasonic probe is fused with the three-dimensional spine image, so that the real-time spine image characteristics of the local area corresponding to the ultrasonic probe and the overall spine information of the three-dimensional spine image are fused to obtain a dynamic three-dimensional spine ultrasonic image.
8. The dynamic three-dimensional ultrasonic spine detection method according to claim 1, wherein the specific steps of performing Lenke typing parameter calculation on the computer terminal according to the fused image in step S6 in step S7 are as follows:
S71) obtaining a Cobb angle of a coronal plane and a Cobb angle of a sagittal plane when standing up according to the fusion image of the standing up posture, and determining the top vertebral position;
s72) obtaining the change condition of the Cobb angle during lateral buckling according to the fusion image during lateral buckling;
s73) judging main and secondary bends, structural bends and non-structural bends according to the top vertebral position in the step 71) and the Cobb angle in the step 72), and determining the side bend type of the Lenke parting by combining with the Lenke parting principle;
s74) obtaining the Lenke typed lumbar vertebra correction according to the relative relation between the position of the lumbar bending apex vertebra and the central perpendicular line of the sacrum;
s75) determining the lunke sagittal sequence correction type of the Lenke typing based on the Cobb angle of the sagittal plane at the time of erection.
9. The method for dynamic three-dimensional ultrasonic spine detection according to claim 1, further comprising the step of S8) automatically recognizing and automatically measuring scoliosis evaluation parameters of the dynamic three-dimensional ultrasonic images fused in the step S6 through deep learning of a large number of the acquired dynamic three-dimensional ultrasonic images fused in the step S6 and corresponding scoliosis evaluation parameters thereof.
10. The method for detecting dynamic three-dimensional ultrasonic spine according to claim 9, wherein the deep learning in the step S8 comprises identification of ultrasonic image markers of two-dimensional spine cross section in a detection range frame, and the method comprises the steps of manually marking spinous processes and two side vertebral plates of corresponding frames after a large number of ultrasonic images of the spine cross section are acquired, taking the marked frames as a training set of a supervised deep learning algorithm, achieving the aims of simulating the manual identification markers and marking through training of a large amount of data, so as to automatically calculate the rotation degree of the vertebral body, and realizing real-time activity tracking of the vertebral body in the bending action of the spine.
11. The method according to claim 9, wherein the deep learning in step S8 includes angle measurement of three-dimensional spine volume projection imaging ultrasonic images, including collecting a plurality of three-dimensional spine volume projection ultrasonic images, marking lateral processes on two sides of each curve end vertebra, or projection of vertebral lamina of an upper lumbar vertebra and upper articular processes of a lower lumbar vertebra, training the marked images to achieve the objective of simulating artificial identification markers and marking, thereby automatically calculating angles of each section of spine side bending line of a patient, and further realizing real-time activity tracking of angle change of the spine side bending line in side bending.
12. The method according to claim 9, wherein the deep learning in step S8 includes real-time measurement of spatial position change of the ultrasonic probe, including an algorithm of unsupervised learning, and the automatic identification, tracking and recording of the ultrasonic probe position is performed by using the optical markers.
13. A dynamic three-dimensional ultrasonic spinal column detection method according to any one of claims 1-12, characterized in that the ultrasonic probe (2) contains a patch, a cable, a patch fixture and a wireless controller, which transmits the patch ultrasonic signal conducted by the cable to a nearby computer terminal (4) via WiFi.
14. The method of claim 1, wherein the ultrasonic probe is a patch ultrasonic probe.
15. The method for dynamic three-dimensional ultrasound spinal column detection according to claim 1, wherein the ultrasound probe is a flexible ultrasound probe.
16. The method for dynamic three-dimensional ultrasonic spine detection according to claim 1, wherein the ultrasonic probe is a one-dimensional array or a two-dimensional array ultrasonic probe.
17. The dynamic three-dimensional ultrasonic spine detection method of claim 6 wherein the depth camera in step S3 is located on a support, and the support can control the depth camera to move in 6 directions, namely front, back, up, down, left and right, so that the back of the human body is always in the field of view of the camera when the spine is subjected to bending motion.
18. A dynamic three-dimensional ultrasound spinal imaging system comprising a three-dimensional spinal ultrasound imaging device (1), characterized by further comprising:
The wearable wireless ultrasonic scanning unit (2) is used for generating a local two-dimensional ultrasonic image frame of the spine and comprises three or more ultrasonic probes;
The depth camera unit (3) is used for recording the spatial position change of the ultrasonic probe and the back dynamic three-dimensional form, and comprises a depth camera with an optical sensor;
The computer terminal (4) is connected with the three-dimensional spine ultrasonic imaging equipment (1), the wearable wireless ultrasonic scanning unit (2) and the depth camera unit (3) and is used for receiving a three-dimensional spine image (100) of the three-dimensional spine ultrasonic imaging equipment (1), a real-time ultrasonic image (200) of the wearable wireless ultrasonic scanning unit (2) and a dynamic back image (300) of the depth camera unit (3), and performing spine bending morphological analysis by fusing the three-dimensional spine image (100), the real-time ultrasonic image (200) and the dynamic back image (300), including measuring a Lenke typing parameter of scoliosis, and dynamically evaluating the spine in lateral flexion activities.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310974513.1A CN119423814A (en) | 2023-08-03 | 2023-08-03 | Dynamic three-dimensional ultrasonic spine detection method and imaging system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310974513.1A CN119423814A (en) | 2023-08-03 | 2023-08-03 | Dynamic three-dimensional ultrasonic spine detection method and imaging system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN119423814A true CN119423814A (en) | 2025-02-14 |
Family
ID=94508109
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310974513.1A Pending CN119423814A (en) | 2023-08-03 | 2023-08-03 | Dynamic three-dimensional ultrasonic spine detection method and imaging system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN119423814A (en) |
-
2023
- 2023-08-03 CN CN202310974513.1A patent/CN119423814A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP5849048B2 (en) | Three-dimensional (3D) ultrasound imaging system for scoliosis evaluation | |
| CN107481228B (en) | Human back scoliosis angle measuring method based on computer vision | |
| JP4653737B2 (en) | Radiation imaging method for three-dimensional reconstruction and computer program and apparatus for implementing the method | |
| US7899220B2 (en) | Time-dependent three-dimensional musculo-skeletal modeling based on dynamic surface measurements of bodies | |
| US5891060A (en) | Method for evaluating a human joint | |
| CN102018527B (en) | Vertebral column three-dimensional dynamic measurement and analysis system and method | |
| CN110363765A (en) | Osteoporosis parameter automatic measurement method based on CT image | |
| US5954674A (en) | Apparatus for gathering biomechanical parameters | |
| CN110772255A (en) | Measurement method of human scoliosis angle based on attitude and position sensor | |
| AU2018301578A1 (en) | Method for detecting spinal deformity using three-dimensional ultrasonic imaging | |
| KR20120123370A (en) | Method and apparatus for measuring spinal characteristics of a patient | |
| Roy et al. | A noninvasive 3D body scanner and software tool towards analysis of scoliosis | |
| US20160180520A1 (en) | Quantitative method for 3-d joint characterization | |
| CN112037277A (en) | A 3D visualization method based on spine 3D ultrasound volume data | |
| RU2392855C1 (en) | Method of digital diagnostics of vertebral deformations | |
| Breen et al. | A digital videofluoroscopic technique for spine kinematics | |
| Wu et al. | The feasibility of a video-based motion analysis system in measuring the segmental movements between upper and lower cervical spine | |
| TWI681755B (en) | System and method for measuring scoliosis | |
| US5991701A (en) | Method for improved instantaneous helical axis determination | |
| CN119423814A (en) | Dynamic three-dimensional ultrasonic spine detection method and imaging system | |
| CN111184535A (en) | Handheld Unconstrained Scanning Wireless 3D Ultrasound Real-time Voxel Imaging System | |
| Zeng et al. | Automatic segmentation of vertebral features on ultrasound spine images using Stacked Hourglass Network | |
| JP6129145B2 (en) | Medical X-ray measuring device | |
| CN106805977A (en) | A kind of backbone interspace of lumbar vertebrae localization method | |
| CN114219889B (en) | Spinal column in-vivo quasi-static three-dimensional measurement method based on CT and ultrasonic fusion |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination |