CN115908717B - Method and system for 3D modeling of spine - Google Patents
Method and system for 3D modeling of spineInfo
- Publication number
- CN115908717B CN115908717B CN202211600511.8A CN202211600511A CN115908717B CN 115908717 B CN115908717 B CN 115908717B CN 202211600511 A CN202211600511 A CN 202211600511A CN 115908717 B CN115908717 B CN 115908717B
- Authority
- CN
- China
- Prior art keywords
- spine
- vertebra
- data
- model
- points
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Landscapes
- Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
- Image Processing (AREA)
Abstract
本发明提出一种脊柱3D建模的方法和系统，首先，在Unity软件中初始化Azure Kinect传感器的身体追踪功能捕获关节点坐标数据，并采用骨长约束法和无迹卡尔曼滤波法对数据进行处理生成控制点，再采用Catmull‑Rom样条插值得到拟合的脊柱曲线；其次，在场景中建立通用的3D脊柱模型并配准到拟合脊柱曲线上，得到实时更新的个性化3D脊柱模型；最后，根据拟合脊柱线的曲率分析算法自动计算冠状面的Cobb角和矢状面的胸椎后凸角、腰椎前凸角。本发明的优点是能够实时可视化3D脊柱模型并准确的评估脊柱角度参数，且无任何电离辐射、成本低廉以及便捷易部署，适用于脊柱侧弯筛查和脊柱康复训练评估。
The present invention proposes a method and system for 3D modeling of the spine. First, the body tracking function of the Azure Kinect sensor is initialized in Unity software to capture joint coordinate data, and the bone length constraint method and unscented Kalman filter method are used to process the data to generate control points. Catmull-Rom spline interpolation is then used to obtain a fitted spinal curve. Secondly, a universal 3D spinal model is established in the scene and aligned to the fitted spinal curve to obtain a personalized 3D spinal model that is updated in real time. Finally, the Cobb angle in the coronal plane and the thoracic kyphosis angle and lumbar lordosis angle in the sagittal plane are automatically calculated based on the curvature analysis algorithm of the fitted spinal line. The advantages of the present invention are that the 3D spinal model can be visualized in real time and the spinal angle parameters can be accurately evaluated. There is no ionizing radiation, low cost, and it is convenient and easy to deploy. It is suitable for scoliosis screening and spinal rehabilitation training evaluation.
Description
Technical Field
The invention belongs to the technical field of medical auxiliary equipment, and particularly relates to a method and a system for spine 3D modeling.
Background
Scoliosis is a high-onset spinal disease that occurs mainly in adolescent period, called adolescent idiopathic scoliosis. Clinical scoliosis detection is carried out by adopting X-ray photographing, a professional doctor manually positions the upper and lower vertebrae, calculates the included angle between the upper and lower edge planes to obtain a Cobb angle, judges the degree of scoliosis according to the angle, and calculates the thoracic posterior lobe and the lumbar anterior lobe of the sagittal plane. The method relies entirely on the subjective judgment of the physician and is susceptible to X-ray sharpness and rotation of the individual vertebrae. In addition, X-rays have radiation injury, and in the non-operative treatment process such as bracket correction and the like, a patient with scoliosis needs to shoot X-rays for examination every 3-6 months, and organs sensitive to the radiation can cause cancer lesions due to receiving overdose rays.
Accordingly, many studies are currently performed on patients for scoliosis detection and three-dimensional spine reconstruction using a inclinometer, a three-dimensional ultrasonic probe, a three-dimensional scanner, and the like, to perform continuous and objective examination of the patients for non-electric radiation. Adam anteversion test refers to the measurement of the rotation angle of the torso by manually holding a inclinometer, but the method has higher sensitivity and specificity, and the correlation coefficient between the measurement and Cobb angle is not ideal. The three-dimensional ultrasonic imaging method is to manually position the ultrasonic probe to finish three-dimensional spine reconstruction and obtain the Cobb angle, but the measured value has only moderate correlation with the result obtained by the X-ray photographing method. Both the above methods rely on manual operation, which can bring a large burden to the inspector and easily cause erroneous judgment due to fatigue. The DIERS spine analysis system utilizes a three-dimensional scanner to scan the three-dimensional contour of the back of a person in the motion process, reconstructs a spine model, calculates the Cobb angle in the motion process, and can realize harmless and rapid measurement. However, three-dimensional scanners are expensive and, depending on the particular site conditions and professional operations, are not suitable for everyday measurements by the public.
Human body posture estimation is a technology for capturing human body posture sequences and outputting joint point coordinates to reflect posture characteristics. The Vicon system enables high-precision human body pose estimation, but this method requires the placement of multiple capture cameras and the wearing of capture garments, and is costly. In recent years, a gesture estimation method based on computer vision is layered endlessly, videoPose D is a classical 3D gesture estimation method, and the method can capture human joints in real time through an RGB camera, but has the defects of low precision, no good open source and difficulty in integrating interaction with other software. The Azure Kinect is a third-generation depth sensor pushed out by Microsoft, bone tracking is realized by combining RGBD images by adopting a deep learning method, accuracy is greatly improved compared with the former two generations, the Azure Kinect is determined to be portable, economical and efficient equipment for gait evaluation, and all functions are packaged in Azure KINECT SDK and Azure Kinect Body TRACKING SDK, so that secondary development can be conveniently performed.
Although the bone tracking of Azure Kinect has achieved a high degree of accuracy, due to its depth camera being based on the time of flight ranging (TOF) principle of light, it is unavoidable to be affected by dust, light and shadows in the environment, so that white noise or spike jitter may occur in the joint data. The resulting data is therefore processed using smoothing algorithms, typically median filtering algorithms, exponential smoothing algorithms, and kalman filtering algorithms. The median filtering algorithm is used for eliminating the peak of data, but has poor effect of removing white noise, the exponential smoothing algorithm can have smooth effect but has larger delay, and the Kalman filter can provide more accurate estimation but is limited to a linear system. And these algorithms are limited to processing the data for each joint point, ignoring the relationship of bone length between the joint points.
Because the three-dimensional coordinates of the joint points output by the Azure Kinect have nonlinear transformation, the traditional Kalman filtering cannot effectively model the nonlinear transformation, and the Unscented Kalman Filtering (UKF) can process nonlinear state transition and transformation, so that a high-precision filtering effect can be achieved on white noise. And the bone length is taken as constraint, and on the basis of UKF processing, the three-dimensional coordinates of the joint points conforming to the real position are further screened, and the error coordinates causing peaks are removed, so that accurate estimation of the joint points is realized.
Disclosure of Invention
Therefore, aiming at the actual improvement requirement of the prior art, the invention provides a method and a system for spine 3D modeling, which utilize Azure Kinect to capture the coordinates of the joint points of a human body part, utilize a UKF method and a bone length constraint method to carry out smoothing treatment on the data of the joint points, generate control points from the joint points to obtain Catmull-Rom spline lines, represent the fitted human spine lines, register a 3D spine model on the spine lines and automatically calculate the coronal Cobb angle, the sagittal thoracic posterior lobe and the lumbar anterior lobe.
Firstly, initializing a body tracking function of an Azure Kinect sensor in Unity software, capturing joint point coordinate data, processing the data by adopting a bone length constraint method and a unscented Kalman filtering method to generate control points, then adopting Catmull-Rom spline interpolation to obtain a fitted spine curve, secondly, establishing a general 3D spine model in a scene, registering the general 3D spine model on the fitted spine curve to obtain a personalized 3D spine model updated in real time, and finally, automatically calculating a Cobb angle of a coronal plane, a thoracic posterior lobe and a lumbar anterior lobe of a sagittal plane according to a curvature analysis algorithm of the fitted spine line. The invention has the advantages of being capable of visualizing the 3D spine model in real time and accurately evaluating the spine angle parameter, free of any ionizing radiation, low in cost, convenient and easy to deploy, and suitable for scoliosis screening and spine rehabilitation training evaluation.
The technical scheme adopted for solving the technical problems is as follows:
A method for modeling a spine 3D (three-dimensional) is characterized in that firstly, the body tracking function of an Azure Kinect sensor is initialized in Unity software to capture coordinate data of joint points, a bone length constraint method and an unscented Kalman filtering method are adopted to process the data to generate control points, then Catmull-Rom spline interpolation is adopted to obtain a fitted spine curve, secondly, a general 3D spine model is built in a scene and registered to the fitted spine curve to obtain a real-time updated personalized 3D spine model, and finally, projections of spline lines on a coronal plane and a sagittal plane are respectively subjected to curvature analysis, and a Cobb angle, a thoracic rear lobe and a lumbar anterior lobe are obtained through calculation.
Further, an Azure Kinect environment is configured in the Unity, an avatar tracking function is initialized, original joint point data are arranged into a queue for input according to the form of an Azure Kinect data stream, control points are generated according to the original joint points and additional calculation added joint points to accurately model and position vertebrae, and the obtained joint point data are processed by adopting a UKF and bone length constraint method to improve accuracy.
Further, the control point generation method specifically comprises the steps of selecting four points related to the spine, namely Neck, SPINE CHEST, SPINE NAVAL and Pelvis, as basic control points for the Azure Kinect body tracking automatic tracking joint points, respectively connecting Left Shoulder with Left Hip, right Shoulder with Right Hip, representing shoulder-Hip connection vectors on the Left side and Right side of the body, and then taking the midpoints of connecting lines at 25% and 65% of the Left shoulder-Hip connection vectors and the Right shoulder-Hip connection vectors on the Right side and the midpoint of Left Shoulder and Right Shoulder as additional control points.
Further, the specific method for generating the control points comprises the following steps:
representing joint point data as Where k represents the number of frames captured by the sensor, i represents the number of joint points, and bone length between the joint points is expressed as Euclidean distance between two pointsWherein:
The joint point data with smaller fluctuation is taken to calculate the average value, and the calculated value of the L i,i+1 is finally obtained as follows:
Li,i+1∈Li,i+1±δi,i+1
The sensor has different accuracy to different joint points, so that different bone lengths correspond to different thresholds, different error thresholds delta i,i+1 are set for the length of the left and right shoulder joints and the length of the left and right hip joints, and when the error is larger than the set threshold, the data spike jitter is considered to appear, and the acquired position information of the joint points is corrected through a filter.
The control point generation method comprises the specific application mode of adopting an unscented Kalman filtering method to filter coordinate data of the joint points, changing a state transition matrix F and a measurement transition matrix H in Kalman filtering into nonlinear functions F (x) and H (x) by the unscented Kalman filtering method, calculating the mean value and covariance of random variables after nonlinear function transformation by a sampling method, establishing a state transition function by using the joint point data acquired by Azure Kinect previously and the data being acquired, obtaining the data from an unscented Kalman filtering algorithm module by iteration, and obtaining the coordinate of each joint point by using an observation function calculation.
Further, the specific process of obtaining the fitted spine curve by adopting the Catmull-Rom spline interpolation is that each control point of the Catmull-Rom spline is positioned on the spline, so that when the spine model is registered to the Catmull-Rom spline, each vertebra can move and rotate more accurately according to the transformation of the control point, and the Catmull-Rom spline function is generated as the fitted spine curve by adopting the Catmull-Rom interpolation algorithm for the control points obtained after the data processing.
Further, the specific process of establishing and registering the universal 3D spine model comprises the steps of representing a central contour curve of the spine by adopting Catmull-Rom spline obtained by interpolation of control points so as to reconstruct the spine vertebra model in three dimensions;
After the universal vertebra models are imported into Unity, a local coordinate system is established for each vertebra model, and the axis of the coordinate system is positioned at the center of the vertebra;
after that, first is the vertebral model position registration:
According to the vertebra model parameters, calculating to obtain the proportional relation of each vertebra relative to the whole spine length, and obtaining the percentage position of each vertebra on the fitted spine curve; fixing the axle center of each vertebra model to a point corresponding to the percentage position of the fitted spine line, setting the Z-axis direction as the tangential direction of the point on the fitted spine line, and setting the Y-axis direction as the normal direction of the point on the fitted spine line and towards the camera plane, so as to register each vertebra to the fitted spine line;
the next is automatically scaling the vertebral model:
The length of the spine line corresponding to the established general spine model is denoted as L normal, the length of the fitted spine curve calculated for each test of different subjects is denoted as L, and the scaling factor of the vertebra model is denoted as:
ScaleFactor =l normal L type 10
The obtained scaling factors are applied to the scaling assemblies of each vertebra model, so that the vertebra models can be automatically scaled according to the fitting spinal curve characteristics of different people, and finally the personalized 3D vertebra model is obtained.
Further, the specific process of calculating the Cobb angle is as follows:
The Catmull-Rom spline line is adopted to fit the spine curve, and since the Catmull-Rom spline function has C2 continuity, namely the second derivative function is continuous, the first derivative of the Catmull-Rom spline function is calculated to calculate the slope:
Calculating extreme points by solving second derivative:
P″ (α) = (2P i-2-5Pi-1+4P-iPi+1)+(-3Pi-2+9Pi-1-9P+3Pi+1) α type 12
Obtaining extreme points (x 1,z1)、(x2,z2)、(x3,z3) of the slope of the coronal plane and the slope k 1、k2、k3 corresponding to the points, and calculating the Cobb angle according to formula 13:
The largest of the two angles is defined as a scoliosis Cobb angle, and the degree of coronal curvature is evaluated and judged according to the angle.
Further, the specific process of calculating the thoracic and lumbar anterior lobes is:
The posterior lobe of the thoracic vertebra is defined as an included angle between the upper end vertebra of the fourth thoracic vertebra (T4) and the tangent line of the lower end vertebra of the twelfth thoracic vertebra (T12), and the anterior lobe of the lumbar vertebra is defined as an included angle between the upper end vertebra of the first lumbar vertebra (L1) and the tangent line of the upper end vertebra of the coccyx (S1);
Calculating a slope k 4 of a position where a T4 center point is located, a slope k 5 of a position where a T12 center point is located, a slope k 6 of a position where a L1 center point is located and a slope k 7 of a position where a L5 center point is located in the established 3D spine model by projection interpolation fit of a sagittal plane to obtain TK and LL:
And a system for 3D modeling of the spine, characterized in that it comprises, according to the method for 3D modeling of the spine as described above:
the system comprises a data capturing module, a data processing optimization module and a Unity scene modeling module;
the data capturing module is used for configuring an Azure Kinect use environment in Unity, calling built-in functions in Azure KINECT SDK and Azure Kinect Body TRACKING SDK, and inputting joint point data captured by the sensor into Unity in a form of a queue;
the data optimization processing module is used for generating control points for interpolation fitting through the joint points and the additional points, and sequentially processing data of the control points by adopting a bone length constraint method and an unscented Kalman filtering algorithm to reduce errors;
The Unity scene modeling module is used for carrying out Catmull-Rom spline interpolation according to control points obtained by data processing to obtain a fitted spine curve, registering and aligning an imported 3D general spine model with the fitted spine curve to obtain a personalized 3D spine model, evaluating parameters of the fitted spine curve through an angle analysis algorithm, calculating to obtain a coronal Cobb angle and sagittal TK angles and LL angles, and refreshing the Unity scene and continuously updating according to data transmission of Azure Kinect.
The invention and the design of the preferred proposal thereof can be applied to the fields of scoliosis screening and spine rehabilitation training, has the advantages of no ionizing radiation, high precision, automatic calculation of angle parameters, 3D spine visualization, low cost, convenient use and the like, and can replace the existing methods of adopting a dip angle side bending meter, a three-dimensional ultrasonic probe, a three-dimensional scanner and the like.
Compared with the prior art, the method has the following advantages and uses:
1. No ionizing radiation. The depth camera of Azure Kinect is based on the principle of time-of-difference ranging (TOF) to measure depth information without any ionizing radiation to the body.
2. The algorithm accuracy is high. The high-precision depth sensor Azure Kinect is adopted to capture the joint point, and the bone length constraint method and the unscented Kalman filtering method are adopted to reduce data errors, so that the result is accurate and reliable.
3. Automatically calculates the angle parameters and realizes the 3D spine visualization. And automatically calculating the coronal Cobb angle and the sagittal TK angle and the sagittal LL angle according to the data capturing processing result, and evaluating the degree of scoliosis of the human body, wherein the visualized three-dimensional spine is helpful for a doctor to explain the rehabilitation training action of the spine to a patient.
Low cost and convenient use. The invention only needs one Azure Kinect depth sensor, a camera tripod and a common notebook computer, has lower cost compared with a system using equipment such as a three-dimensional ultrasonic probe, a three-dimensional scanner and the like, and only needs to install Unity software in the notebook computer and introduce the project package of the invention, build the tripod and connect the sensor to a computer running system, can automatically calculate angle parameters and realize 3D spine visualization, does not need complicated manual operation, and has strong product deployment property.
Drawings
The invention is described in further detail below with reference to the attached drawings and detailed description:
FIG. 1 is a flow chart of a 3D spine modeling and system in accordance with an embodiment of the present invention;
FIG. 2 is a flowchart of a method for detecting and processing a joint point according to an embodiment of the present invention;
FIG. 3 is a schematic view of an Azure Kinect joint point according to an embodiment of the present invention;
FIG. 4 is a schematic view of a local coordinate system of a vertebral model according to an embodiment of the present invention;
FIG. 5 is a diagram of measuring Cobb angle according to the embodiment of the present invention, wherein (a) is a schematic diagram of principle of measuring Cobb angle of coronal plane (b) is a schematic diagram of measuring Cobb angle);
FIG. 6 is a schematic diagram of a sagittal plane parameter calculation according to an embodiment of the present invention;
FIG. 7 is a block diagram of a 3D spinal modeling and system design in accordance with an embodiment of the present invention.
Detailed Description
In order to make the features and advantages of the present patent more comprehensible, embodiments accompanied with figures are described in detail below:
It should be noted that the following detailed description is illustrative and is intended to provide further explanation of the application. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs.
It is noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of exemplary embodiments according to the present application. As used herein, the singular is also intended to include the plural unless the context clearly indicates otherwise, and furthermore, it is to be understood that the terms "comprises" and/or "comprising" when used in this specification are taken to specify the presence of stated features, steps, operations, devices, components, and/or combinations thereof.
In the body tracking function of Unity integrated Azure Kinect, the data of the joint points are processed smoothly by adopting a unscented Kalman filtering method and a bone length constraint method, a spine curve is fitted by adopting Catmull-Rom spline interpolation on control points obtained from the joint points, a general 3D spine model is established and registered on the fitted spine line to form an automatically calibrated personalized 3D spine model, and finally projections of the spline line on a coronal plane and a sagittal plane are respectively subjected to curvature analysis, and a Cobb angle, a thoracic rear lobe and a lumbar anterior lobe are obtained by calculation. The flow chart is shown in fig. 1.
1. Joint detection and processing
In the embodiment, an Azure Kinect environment is configured in Unity, an avatar tracking function is initialized, original joint point data is arranged into a queue for input according to the form of an Azure Kinect data stream, control points are generated according to the original joint point and additional calculation added joint points to accurately model and position vertebrae, and the obtained joint point data is processed by adopting a UKF and bone length constraint method to improve the accuracy, wherein a flow chart is shown in figure 2.
1.1 Control Point Generation
For the Azure Kinect body tracking, the automatically tracked joint points, four points related to the spine, neck, SPINE CHEST, SPINE NAVAL and Pelvis, are selected as basic control points. Left Shoulder and Left Hip, right Shoulder and Right Hip are connected, respectively, representing shoulder-Hip connection vectors on the Left and Right sides of the body. The midpoints of the connecting lines at the left and right shoulder-hip connection vectors 25% and 65%, and the midpoints of Left Shoulder and Right Shoulder were then taken as additional control points. As shown in fig. 3, gray middle with dots marked as base control points and gray middle without dots marked as additional added control points, act together to generate the spinal curve.
1.2 Bone Length constraint
When estimating the posture of the human body, peak fluctuation of the joint point data may occur due to the rapid movement of the human body or the occlusion of the object. In practice, the bone length of the human body is constant, and this error can be reduced by limiting the distance between the two joint points. Therefore, the invention measures the length of the left and right shoulder joints and the left and right hip joints, continuously collects the data with smaller fluctuation in multi-frame data, and calculates the average value to establish the bone length model.
The node data is expressed asWhere k represents the number of frames captured by the sensor and i represents the joint number. Bone length between articular points is expressed as Euclidean distance between two pointsWherein:
The joint point data with smaller fluctuation is taken to calculate the average value, and the calculated value of the L i,i+1 is finally obtained as follows:
Li,i+1∈Li,i+1±δi,i+1
Because the sensor performs differently for different nodes, different bone lengths correspond to different thresholds. And setting different error thresholds delta i,i+1 for the length of the left and right shoulder joints and the length of the left and right hip joints, and when the error is larger than the set threshold, considering that data spike jitter occurs, and correcting the acquired joint point position information by a filter.
1.3 Unscented Kalman Filter (UKF)
Because the three-dimensional coordinate of the joint point output by the Azure Kinect has nonlinear transformation, the traditional Kalman filtering cannot effectively model the nonlinear transformation, and therefore, the embodiment of the invention adopts the UKF method to carry out filtering treatment on the coordinate data of the joint point. The UKF changes a state transition matrix F and a measurement transition matrix H in Kalman filtering into nonlinear functions F (x) and H (x), and the mean value and covariance of the random variables after nonlinear function transformation are calculated through a sampling method. UKF state update has two stages, namely a prediction stage and an update stage.
In the prediction phase, based on the state vector X t and the covariance matrix P t at time t, 2n+1 sampling points { X i, i=0, 1,..2 n } and their corresponding weights are obtained using a sampling algorithmWhere n represents the dimension of the state vector x. Substituting the sampling point [ X i ] into f (X) can solve the corresponding function value { Y i }, and then the predicted state vector and covariance matrix at time t+1 are expressed as:
In the update phase, the sample mean and variance of { Z i } obtained by the sample point { Y i } after the nonlinear function h (x) is calculated first:
further, a residual vector and a Kalman gain are calculated:
Finally, the state vector x t+1 and covariance matrix P t+1 at time t+1 are updated:
And establishing a state conversion function by using the node data acquired previously by Azure Kinect and the data being captured, obtaining the data from the UKF system through iteration, and obtaining the coordinate of each node by using observation function calculation.
2. Interpolation fitting of spinal curves
The biggest difference between the Catmull-Rom spline and the Bessel spline is that each of its control points is located on the spline, which allows each vertebra to move and rotate more accurately according to the transformation of the control points when the spinal model is registered to the Catmull-Rom spline. Therefore, for the obtained control points after data processing, a Catmull-Rom spline function is generated by adopting a Catmull-Rom interpolation algorithm as a fitted spine curve.
The Catmul-Rom spline interpolation is essentially a piecewise polynomial interpolation, and the four points of each segment are respectively named as P i-2、Pi-1、Pi、Pi+1, so that a cubic polynomial curve can be constructed between P i-1、Pi according to the coordinates of the four points, and a complete continuous curve can be constructed by splicing the spline curves of each segment. Cubic polynomial for each segment:
P (t) =c 0+C1α+C2α2+C3α3 type 7
Wherein, floating point coordinates alpha epsilon [0,1], spline function coefficient C 0～C3 is obtained by constraint conditions of P (t) at endpoints alpha=0 and alpha=1:
Where T is called a shape factor, and the interpolation fitting effect is optimal when T is set to 0.5. Then, the coefficient C 0～C3 is obtained from the equation 8 and substituted into the equation 7, and a Catmull-Rom curve equation between the interpolation points P i-1、Pi can be obtained:
3. generic 3D spinal model creation and registration
3.1 General 3D spinal model creation
In order to make a judgment on the scoliosis condition more intuitively, the embodiment adopts a Catmull-Rom spline line obtained by interpolation of control points to represent the central contour curve of the spine so as to reconstruct the spine vertebra model in three dimensions. The structure of the spine is complex and consists of 7 cervical vertebrae, 12 thoracic vertebrae, 5 lumbar vertebrae, sacrum and coccyx. According to studies, scoliosis is shown to occur mainly in the thoracic (T1-T12) and lumbar (L1-L5) vertebrae.
Thus, a 3D model including 17 vertebrae in total of thoracic and lumbar vertebrae was pre-built according to the geometric proportion of vertebrae counted in the literature, and the built model was imported into Unity software. Wherein the relative percent heights of the center point height of each vertebra relative to the spinal column minus the disc height are shown in table 1.
Table 1 vertebral height relative percentages
In practice, intervertebral disc spaces exist between vertebrae of the human spinal column, and studies have shown that the height between all of the intervertebral discs is approximately 25% -30% of the total spinal column length. Thus, the disc height should be considered in creating a generic 3D spinal model, and the percentage of the height between the thoracic and lumbar discs is calculated from literature as shown in table 2.
Table 2 disc height percent
The actual percent height for each vertebra is calculated in combination with consideration of the vertebral height and the disc height as shown in table 3.
Table 3 actual vertebral height percentages
3.2 General 3D spinal model registration
(1) Vertebral model position registration
After the universal vertebral models are imported into Unity, a local coordinate system shown in fig. 4 is established for each vertebral model, the axes of the coordinate system are positioned at the center of the vertebrae (the junction of the vertebral body and the pedicle), the left is a sagittal plane view, the right is a coronal plane view, and the Z axis and the Y axis are respectively perpendicular and parallel to the upper plane of the vertebrae.
In the three-dimensional space, the position of an object can be fixed by determining the coordinate axes of the object and the directions of two coordinate axes. The proportional relationship of each vertebra in Table 3 relative to the entire spinal length was calculated based on the vertebra model parameters in tables 1 and 2 to obtain the percent position of each vertebra on the fitted spinal curve. The axes of each vertebral model are fixed to points at a percentage of the corresponding fitted spinal line, and the Z-axis direction is set to be tangential to the points on the fitted spinal line, and the Y-axis direction is set to be normal to the points on the fitted spinal line and toward the camera plane, thereby registering each vertebra to the fitted spinal line.
(2) Automatically scaling vertebral models
The length of the spine line corresponding to the established general spine model is denoted as L normal, the length of the fitted spine curve calculated for each test of different subjects is denoted as L, and the scaling factor of the vertebra model is denoted as:
ScaleFactor =l normal L type 10
The obtained scaling factors are applied to the scaling assemblies of each vertebra model, so that the vertebra models can be automatically scaled according to the fitting spinal curve characteristics of different people, and finally the personalized 3D vertebra model is obtained.
4. Automatic calculation of coronal and sagittal plane angle parameters
4.1 Coronal Cobb Angle
The Cobb angle is the most commonly used measure for judging the scoliosis degree at present, and is defined as the maximum included angle between the upper edge plane of the upper end vertebra and the lower edge plane of the lower end vertebra of the human coronal plane spine, as shown in fig. 5 (a). According to the method, the Cobb angle is calculated according to the fitted spine curve, the position with the largest spine curve bending degree can be automatically positioned, and meanwhile the influence caused by abnormal rotation of a single vertebra in the traditional method is avoided.
FIG. 5 (b) is a schematic diagram of Cobb angle measurement according to the present invention, in which the positions of the upper and lower vertebrae are the points of the spine curve with the greatest inclination, and the inclination is the extreme point of the curve inclination, so that the extreme point of the inclination can be obtained by calculating the point where the second derivative of the fitted spine curve is zero, thereby locating the positions of the upper and lower vertebrae.
According to the invention, the Catmull-Rom spline line is adopted to fit the spine curve, and as the Catmull-Rom spline function has C2 continuity, namely the second derivative function is continuous, the slope can be calculated by solving the first derivative of the Catmull-Rom spline function according to 9:
Calculating extreme points by solving second derivative:
P″ (α) = (2P i-2-5Pi-1+4P-iPi+1)+(-3Pi-2+9Pi-1-9P+3Pi+1) α type 12
The extreme points (x 1,z1)、(x2,z2)、(x3,z3) of the coronal slope are obtained, as well as the slope k 1、k2、k3 corresponding to these points. Cobb angle size is calculated according to equation 13:
The largest of the two angles is defined as a scoliosis Cobb angle, and the degree of coronal curvature is evaluated and judged according to the angle.
4.2 Sagittal plane parameters
The sagittal plane evaluation parameters of the invention are thoracic kyphosis (thoracic kyphosis, TK) and lumbar lordosis (lumbar lordosis, LL), and the prior study shows that the two have higher correlation with scoliosis degree.
As shown in fig. 6, the posterior thoracic lobe is defined as the angle between the upper end of the fourth thoracic vertebra (T4) and the tangent to the lower end of the twelfth thoracic vertebra (T12), and the anterior lumbar lobe is defined as the angle between the upper end of the first lumbar vertebra (L1) and the tangent to the upper end of the coccyx (S1). According to the invention, through projection interpolation fitted spine curves on a sagittal plane, the slope k 4 of the position of the T4 center point, the slope k 5 of the position of the T12 center point, the slope k 6 of the position of the L1 center point and the slope k 7 of the position of the L5 center point in the established 3D spine model are calculated, so that TK and LL are obtained:
5. System design
Based on the above method design, the system design provided by the embodiment of the invention mainly comprises three parts, namely a data capturing module, a data processing optimizing module and a Unity scene modeling module, as shown in fig. 7.
The data capturing module is used for configuring an Azure Kinect use environment in the Unity, calling built-in functions in Azure KINECT SDK and Azure Kinect Body TRACKING SDK and inputting the joint point data captured by the sensor into the Unity in a queue mode.
The data optimization processing module generates control points for interpolation fitting through the joint points and the additional points, and sequentially adopts a bone length constraint method and an unscented Kalman filtering algorithm to process the data of the control points so as to reduce errors.
The Unity scene modeling module carries out Catmull-Rom spline interpolation according to control points obtained by data processing to obtain a fitted spine curve, registers and aligns an imported 3D general spine model with the fitted spine curve to obtain a personalized 3D spine model, evaluates parameters of the fitted spine curve through an angle analysis algorithm, automatically calculates to obtain a coronal Cobb angle and sagittal TK angles and LL angles, and finally refreshes the Unity scene and continuously updates the model according to data transmission of Azure Kinect.
6. The use process of the scheme of the embodiment is specifically as follows:
1. Horizontally placing a camera tripod, adjusting the camera tripod to a height of 1.4-1.6 meters, and installing an Azure Kinect sensor;
2. the test subject stands upright and faces the sensor, and the distance is 0.25-2.0 meters;
3. The Unity software is started, the 3D spine modeling and system project is opened, and the playing button is clicked to operate the system, so that the spine form and the angle parameter can be monitored in real time;
4. clicking a pause button to export the data to a database and collecting static data of the subject;
and clicking the play button again to finish the system, thereby completing the acquisition.
It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The above description is only a preferred embodiment of the present invention, and is not intended to limit the invention in any way, and any person skilled in the art may make modifications or alterations to the disclosed technical content to the equivalent embodiments. However, any simple modification, equivalent variation and variation of the above embodiments according to the technical substance of the present invention still fall within the protection scope of the technical solution of the present invention.
The present patent is not limited to the above-mentioned preferred embodiments, and any person who can obtain other various types of methods and systems for modeling the 3D of the spine in the light of the present patent shall not be limited to the above-mentioned preferred embodiments, and all equivalent changes and modifications made according to the claims of the present patent shall be covered by this patent.
Claims (7)
1. A method for modeling a spine 3D is characterized in that firstly, the body tracking function of an Azure Kinect sensor is initialized in Unity software to capture coordinate data of joint points, a bone length constraint method and a unscented Kalman filtering method are adopted to process the data to generate control points, and then Catmull-Rom spline interpolation is adopted to obtain a fitted spine curve;
Configuring an Azure Kinect environment in Unity, initializing an avatar tracking function, arranging original joint point data into a queue for input according to the form of an Azure Kinect data stream, generating control points according to the original joint points and additional calculation added joint points to accurately model and position vertebrae, and processing the obtained joint point data by adopting a UKF and bone length constraint method to improve the precision;
The specific method for generating the control points comprises the steps of selecting four points which are related to the spine, namely Neck, SPINE CHEST, SPINE NAVAL and Pelvis, as basic control points for Azure Kinect body tracking and automatic tracking, respectively connecting Left Shoulder with Left Hip, right Shoulder with Right Hip, representing shoulder-Hip connection vectors on the Left side and Right side of the body, and then taking the midpoints of connecting lines at 25% and 65% of the Left shoulder-Hip connection vectors and the Right shoulder-Hip connection vectors on the Right side and the midpoints of Left Shoulder and Right Shoulder as additional control points;
the specific method for generating the control points comprises the following steps:
representing joint point data as Where k represents the number of frames captured by the sensor, i represents the number of joint points, and bone length between the joint points is expressed as Euclidean distance between two pointsWherein:
1 (1)
Averaging the joint point data with small fluctuation to obtainThe calculated values are:
2, 2
Because the accuracy of the sensor to different joint points is different, different bone lengths correspond to different thresholds, and different error thresholds are set for the length of the left and right shoulder joints and the length of the left and right hip jointsWhen the error is larger than the set threshold, the collected position information of the joint point is corrected through a filter in consideration of data spike jitter.
2. The method for modeling the 3D spine according to claim 1, wherein the control point is generated by using a method for filtering coordinate data of the nodes by using an unscented Kalman filtering method, the unscented Kalman filtering method changes a state transition matrix F and a measurement transition matrix H in Kalman filtering into nonlinear functions F (x) and H (x), a mean value and covariance of random variables after nonlinear function transformation are calculated by a sampling method, a state transition function is established by using node data previously acquired by Azure Kinect and data being acquired, the data is obtained from an unscented Kalman filtering algorithm module by iteration, and coordinates of each node are obtained by using an observation function calculation.
3. The method for 3D modeling of the spine according to claim 1, wherein the method for obtaining the fitted spine curve by interpolation of the Catmull-Rom spline is characterized in that each control point of the Catmull-Rom spline is located on the spline, so that each vertebra can move and rotate more accurately according to the transformation of the control point when the spine model is registered to the Catmull-Rom spline, and the Catmull-Rom spline function is generated as the fitted spine curve by the Catmull-Rom interpolation algorithm for the obtained control points after data processing.
4. The method for modeling the 3D spine of claim 1, wherein the general 3D spine model is established and registered by adopting Catmull-Rom spline obtained by interpolation of control points to represent a central contour curve of the spine so as to reconstruct the spine vertebra model in three dimensions;
After the universal vertebra models are imported into Unity, a local coordinate system is established for each vertebra model, and the axis of the coordinate system is positioned at the center of the vertebra;
after that, first is the vertebral model position registration:
According to the vertebra model parameters, calculating to obtain the proportional relation of each vertebra relative to the whole spine length, and obtaining the percentage position of each vertebra on the fitted spine curve; fixing the axle center of each vertebra model to a point corresponding to the percentage position of the fitted spine line, setting the Z-axis direction as the tangential direction of the point on the fitted spine line, and setting the Y-axis direction as the normal direction of the point on the fitted spine line and towards the camera plane, so as to register each vertebra to the fitted spine line;
the next is automatically scaling the vertebral model:
The length of the spine line corresponding to the established general spine model is expressed as The length of the fitted spinal curve calculated for each test of the different subjects is expressed asThe scaling factor of the vertebral model is expressed as:
10. The method of the invention
The obtained scaling factors are applied to the scaling assemblies of each vertebra model, so that the vertebra models can be automatically scaled according to the fitting spinal curve characteristics of different people, and finally the personalized 3D vertebra model is obtained.
5. The method for 3D modeling of the spine of claim 1 wherein the specific process of calculating the Cobb angle is:
The Catmull-Rom spline line is adopted to fit the spine curve, and since the Catmull-Rom spline function has C2 continuity, namely the second derivative function is continuous, the first derivative of the Catmull-Rom spline function is calculated to calculate the slope:
11. The method of the invention
Calculating extreme points by solving second derivative:
12. Fig.
Obtaining extreme points of the slope of the coronal plane、、And the slopes corresponding to these points、、Then, the Cobb angle size is calculated according to formula 13:
13 of the group
The largest of the two angles is defined as a scoliosis Cobb angle, and the degree of coronal curvature is evaluated and judged according to the angle.
6. The method of 3D modeling of the spinal column of claim 1, wherein the calculating of the posterior thoracic and anterior lumbar lobes is performed by:
The posterior lobe of the thoracic vertebra is defined as an included angle between the upper end vertebra of the fourth thoracic vertebra (T4) and the tangent line of the lower end vertebra of the twelfth thoracic vertebra (T12), and the anterior lobe of the lumbar vertebra is defined as an included angle between the upper end vertebra of the first lumbar vertebra (L1) and the tangent line of the upper end vertebra of the coccyx (S1);
Calculating the slope of the position of the T4 center point in the established 3D spine model through projection interpolation fitted spine curve on the sagittal plane Slope of the position of the T12 center pointSlope of the location of the center point L1Slope of the location of the L5 center pointTK and LL are obtained:
Formula 14.
7. A system for 3D modeling of the spine, characterized in that the method for 3D modeling of the spine according to any one of claims 1 to 6 comprises:
the system comprises a data capturing module, a data processing optimization module and a Unity scene modeling module;
the data capturing module is used for configuring an Azure Kinect use environment in Unity, calling built-in functions in Azure KINECT SDK and Azure Kinect Body TRACKING SDK, and inputting joint point data captured by the sensor into Unity in a form of a queue;
the data optimization processing module is used for generating control points for interpolation fitting through the joint points and the additional points, and sequentially processing data of the control points by adopting a bone length constraint method and an unscented Kalman filtering algorithm to reduce errors;
The Unity scene modeling module is used for carrying out Catmull-Rom spline interpolation according to control points obtained by data processing to obtain a fitted spine curve, registering and aligning an imported 3D general spine model with the fitted spine curve to obtain a personalized 3D spine model, evaluating parameters of the fitted spine curve through an angle analysis algorithm, calculating to obtain a coronal Cobb angle and sagittal TK angles and LL angles, and refreshing the Unity scene and continuously updating according to data transmission of Azure Kinect.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211600511.8A CN115908717B (en) | 2022-12-12 | 2022-12-12 | Method and system for 3D modeling of spine |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211600511.8A CN115908717B (en) | 2022-12-12 | 2022-12-12 | Method and system for 3D modeling of spine |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115908717A CN115908717A (en) | 2023-04-04 |
| CN115908717B true CN115908717B (en) | 2025-08-08 |
Family
ID=86472441
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211600511.8A Active CN115908717B (en) | 2022-12-12 | 2022-12-12 | Method and system for 3D modeling of spine |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115908717B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117152229A (en) * | 2023-08-31 | 2023-12-01 | 曾海涵 | Cobb angle measurement method and device |
| CN117379036A (en) * | 2023-12-07 | 2024-01-12 | 北京维卓致远医疗科技发展有限责任公司 | Scoliosis measurement system and equipment |
| CN120298415B (en) * | 2025-06-13 | 2025-08-19 | 中国科学院合肥物质科学研究院 | Three-dimensional spine assessment method and system based on RGB images and anatomical constraints |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113298917A (en) * | 2021-05-28 | 2021-08-24 | 上海曼恒数字技术股份有限公司 | Human-shaped animation multiplexing method and system |
| CN113631115A (en) * | 2019-02-05 | 2021-11-09 | 史密夫和内修有限公司 | Algorithm-based optimization, tools, and selectable simulation data for total hip arthroplasty |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8175349B2 (en) * | 2006-08-16 | 2012-05-08 | Siemens Medical Solutions Usa, Inc. | System and method for segmenting vertebrae in digitized images |
| CN113870098B (en) * | 2021-09-09 | 2024-09-24 | 武汉大学 | Automatic Cobb angle measurement method based on spine layered reconstruction |
-
2022
- 2022-12-12 CN CN202211600511.8A patent/CN115908717B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113631115A (en) * | 2019-02-05 | 2021-11-09 | 史密夫和内修有限公司 | Algorithm-based optimization, tools, and selectable simulation data for total hip arthroplasty |
| CN113298917A (en) * | 2021-05-28 | 2021-08-24 | 上海曼恒数字技术股份有限公司 | Human-shaped animation multiplexing method and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115908717A (en) | 2023-04-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN115908717B (en) | Method and system for 3D modeling of spine | |
| US7899220B2 (en) | Time-dependent three-dimensional musculo-skeletal modeling based on dynamic surface measurements of bodies | |
| Yamazaki et al. | Improvement of depth position in 2-D/3-D registration of knee implants using single-plane fluoroscopy | |
| Cappello et al. | Soft tissue artifact compensation in knee kinematics by double anatomical landmark calibration: performance of a novel method during selected motor tasks | |
| US9538940B2 (en) | Intelligent algorithms for tracking three-dimensional skeletal movement from radiographic image sequences | |
| Prabhu | Automatic quantification of spinal curvature in scoliotic radiograph using image processing | |
| CN116524124A (en) | Method and system for three-dimensional dynamic reconstruction of spine | |
| JP2017532165A (en) | System and method for measuring and evaluating spinal instability | |
| WO2015162158A1 (en) | Human motion tracking | |
| JP2004008419A (en) | Anatomically characteristic position detector and object structure measuring instrument | |
| Makki et al. | In vivo ankle joint kinematics from dynamic magnetic resonance imaging using a registration-based framework | |
| Kandasamy et al. | Posture and back shape measurement tools: a narrative literature review | |
| JP2004167109A (en) | Three-dimensional measurement method, three-dimensional measurement system, image processing device, and computer program | |
| JP2025084107A (en) | Personalized joint bone and cartilage morphology measurement system and method | |
| CHOI et al. | Measurement of vertebral rotation from moire image for screening of adolescent idiopathic scoliosis | |
| Groisser et al. | 3d reconstruction of scoliotic spines from stereoradiography and depth imaging | |
| CN119055193A (en) | A screening method for adolescent idiopathic scoliosis based on three-dimensional curvature asymmetry | |
| WO2025080516A1 (en) | Systems for planning joint alignment in orthopedic procedures | |
| Nuzzi et al. | Body measurement estimations using 3D scanner for individuals with severe motor impairments | |
| Cukovic et al. | Development of 3D kinematic model of the spine for idiopathic scoliosis simulation | |
| Tersi et al. | 3D elbow kinematics with monoplanar fluoroscopy: in silico evaluation | |
| Huang | A fast method for spine localization in x-ray images | |
| RU2836777C1 (en) | Method for assessing the functional state of the shoulder joint | |
| CN120298415B (en) | Three-dimensional spine assessment method and system based on RGB images and anatomical constraints | |
| Tanee et al. | Scoliosis screening using depth camera |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |