CN115619773A - A method and system for three-dimensional tooth multimodal data registration - Google Patents

Abstract

The invention discloses a three-dimensional tooth multimodal data registration method and system, belonging to the technical field of three-dimensional point cloud and grid processing. Including obtaining 3D tooth information and constructing a 3D tooth model; obtaining crown information and constructing a 3D crown model; preprocessing the 3D tooth model and the 3D crown model; performing sequential processing on the preprocessed 3D tooth model and 3D crown model Initial pose normalization, coarse registration and fine registration, and then perform secondary fine registration on each single crown model in the 3D crown model to obtain a single crown model after rigid transformation; according to the rigidity For each single crown model after transformation, non-rigid transformation is performed on the corresponding single three-dimensional tooth model in the three-dimensional tooth model to obtain a three-dimensional tooth model with high-precision crown information; it makes up for the limitations of single-type data , to obtain more comprehensive tooth information; solve the problem of "unable to display comprehensive information of teeth" in the prior art.

Description

A method and system for three-dimensional tooth multimodal data registration

technical field

The present application relates to the technical field of 3D point cloud and grid processing, in particular to a method and system for 3D tooth multimodal data registration.

Background technique

The statements in this section merely mention the background art related to this application, and do not necessarily constitute the prior art.

In order to improve the service capabilities of the oral medical market, new computer-aided medical technologies based on the Internet, big data and artificial intelligence technology have emerged, making oral medical treatment more accurate and efficient, and laying the foundation for adapting to the growing market demand. Among them, digital scanning technology has become a computer-aided medical technology that is gradually popularized in clinical medicine. The accuracy of digital imaging has been significantly improved and it is multi-modal.

Multi-modality refers to multiple data obtained from different perspectives or fields when describing the same object. Each perspective or field is called a modality, and multi-modal fusion is to combine information from multiple modalities. , to maximize the advantages of each modality, it is necessary to reduce the information loss in fusion to a certain extent.

Due to different diagnostic purposes and data acquisition methods, the oral cavity has multi-modal data forms, including oral scan data, CBCT (Cone-Beam Computed Tomography, dental cone-beam computerized tomography) data, oral X-ray panorama, cephalometric lateral view There are large differences between various types of data such as pictures and dental photos, and each has its own advantages and disadvantages. For example, although CBCT data can obtain complete three-dimensional information of teeth including tooth roots, due to problems such as adhesion of adjacent teeth and low contrast with surrounding alveolar bone, resulting in blurred boundaries and low resolution, it is difficult to accurately describe the occlusion of teeth Although the oral scan data provides high-resolution geometric feature information of the crown, it cannot check the state of the tooth root; the cephalometric image can clearly understand the structure and relative position of the soft and hard tissues; the oral X-ray panorama can provide insight Wisdom teeth, misalignment and other potential dental problems.

With the development of oral data collection technology, the orthodontic industry has become the forefront of digital applications in the dental field. The core part of orthodontic treatment is to predict the trend of tooth movement. Digital orthodontics can achieve precise correction through digital scanning equipment. , through 3D tooth modeling to arrange the teeth and plan the movement path, the personalized intermediate correction steps and final correction results suitable for the patient can be intuitively generated. In clinical orthodontic treatment, crown information obtained from oral scan data is generally used to operate, but due to the variable alveolar bone anatomy, wrong judgment of tooth position, lack of root information, etc., the speed of tooth movement in the orthodontic diagnosis and treatment plan If it is faster than the remodeling speed of the alveolar bone, it will cause damage such as bone fenestration and bone cracking.

For orthodontic problems, it is far from enough to rely solely on crown information obtained from oral scan data. It is necessary to combine multi-modal data for multi-modal data registration. Multimodal data registration is usually divided into two steps: coarse registration and fine registration. Coarse registration refers to the roughly aligned transformation of the data to provide a good initial pose, while fine registration focuses on the details of the differences between the data. , which can further reduce the registration error. There are various existing multimodal registration algorithms, and considerable registration results can be obtained on better quality data sets. However, for tooth data with more noise, inconspicuous geometric features, and missing information However, there is still no mature algorithm that can obtain relatively ideal registration results.

Contents of the invention

In order to solve the deficiencies of the existing technology, this application provides a three-dimensional tooth multi-modal data registration method, which makes full use of the complementarity between the multi-modal tooth data, and registers data with similar expressions to make up for their respective differences. Advantages and disadvantages, to maximize the comprehensive information of teeth.

In the first aspect, the present application provides a three-dimensional tooth multimodal data registration method;

A three-dimensional tooth multimodal data registration method, comprising:

Obtain 3D tooth information and construct a 3D tooth model; obtain crown information and construct a 3D crown model; preprocess the 3D tooth model and 3D crown model;

Perform initial pose normalization, rough registration, and fine registration on the preprocessed 3D tooth model and 3D crown model, and then perform secondary fine matching on each single crown model in the 3D crown model Accurate, obtain the single crown model after rigid transformation;

According to each single tooth crown model after the rigid transformation, non-rigid transformation is performed on the corresponding single three-dimensional tooth model in the three-dimensional tooth model to obtain a three-dimensional tooth model with high-precision tooth crown information.

In the second aspect, the present application provides a three-dimensional tooth multimodal data registration system;

A three-dimensional tooth multimodal data registration system, comprising:

The data preprocessing module is configured to: obtain three-dimensional tooth information, construct a three-dimensional tooth model; obtain dental crown information, and construct a three-dimensional dental crown model; preprocess the three-dimensional tooth model and the three-dimensional dental crown model;

The rigid transformation module is configured to: perform initial pose normalization, rough registration and fine registration on the preprocessed 3D tooth model and 3D crown model in sequence, and then perform each single tooth in the 3D crown model The tooth model is subjected to secondary fine registration to obtain a single crown model after rigid transformation;

The non-rigid transformation module is configured to: perform non-rigid transformation on the corresponding single three-dimensional tooth model in the three-dimensional tooth model according to each single tooth crown model after the rigid transformation, and obtain a three-dimensional crown with high-precision tooth crown information. Tooth model.

Compared with prior art, the beneficial effect of the present application is:

1. The present invention focuses on CBCT data and oral scan data, extracts three-dimensional tooth information and three-dimensional crown information, analyzes different characteristics of tooth multimodal data, and focuses on the key link in the process of orthodontics and restoration, that is, multimodal It is the basic data processing process of digital orthodontics and plays an important role in the development of digital dentistry;

2. The present invention proposes a global rigid transformation before the local rigid transformation, and the transformation of the upper/mandibular teeth as a whole can preserve the arrangement and posture of the original teeth, and avoid large-scale wrong posture transformation of a single tooth;

3. The present invention proposes an effective algorithm for registration by combining rigid transformation and non-rigid transformation. Compared with ordinary rigid transformation, non-rigid transformation constrained by distance term and stiffness term can obtain more accurate transformation results;

4. The present invention can reconstruct and generate a high-precision model, obtain more comprehensive tooth information, and use the reconstructed model to analyze various problems such as tooth arrangement, feature detection, and tooth root generation;

5. The present invention proposes a registration problem based on three-dimensional model data, and the method used can also be extended to other model data, which can meet the data processing requirements of other industries.

Description of drawings

The accompanying drawings constituting a part of the present application are used to provide further understanding of the present application, and the schematic embodiments and descriptions of the present application are used to explain the present application, and do not constitute improper limitations to the present application.

Fig. 1 is the schematic flow chart that the embodiment of the present application provides;

Fig. 2 is a schematic diagram of the pretreatment effect of the three-dimensional tooth model provided by the embodiment of the present application;

Fig. 3 is a schematic diagram of the pretreatment effect of the three-dimensional crown model provided by the embodiment of the present application;

FIG. 4 is a schematic diagram of an initial pose normalization process provided by an embodiment of the present application;

FIG. 5 is a schematic diagram of the initial pose normalization effect provided by the embodiment of the present application;

FIG. 6 is a schematic diagram of the effect of the global coarse registration provided by the embodiment of the present application;

Fig. 7 is a schematic diagram of the effect of another angle of the global coarse registration provided by the embodiment of the present application;

FIG. 8 is a schematic diagram of the effect of global fine registration provided by the embodiment of the present application;

FIG. 9 is a schematic diagram of the effect of another angle of global fine registration provided by the embodiment of the present application;

FIG. 10 is a schematic diagram of the non-rigid transformation process provided by the embodiment of the present application;

Fig. 11 is a schematic diagram of the effect of a three-dimensional tooth model with high-precision crown information provided by the embodiment of the present application;

Fig. 12 is a schematic diagram showing the effect of another angle of the three-dimensional tooth model with high-precision crown information provided by the embodiment of the present application.

detailed description

It should be pointed out that the following detailed description is exemplary and is intended to provide further explanation to the present application. Unless defined otherwise, all technical and scientific terms used in this application have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs.

It should be noted that the terminology used here is only for describing specific implementations, and is not intended to limit the exemplary implementations according to the present application. As used herein, unless the context clearly dictates otherwise, the singular is intended to include the plural, and it should also be understood that the terms "comprising" and "having" and any variations thereof are intended to cover a non-exclusive Comprising, for example, a process, method, system, product, or device comprising a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include steps or units not explicitly listed or for these processes, methods, Other steps or units inherent in a product or equipment.

In the case of no conflict, the embodiments and the features in the embodiments of the present invention can be combined with each other.

Embodiment one

In the prior art, the oral cavity has multi-modal data forms, and the differences between various types of data are large and each has its own advantages and disadvantages, so it is impossible to fully and accurately display tooth information; therefore, this application provides a 3D tooth multi-modal data registration method, making full use of the complementarity between the multimodal data of teeth, and registering data with similar expressions to make up for their respective advantages and disadvantages, so as to display the comprehensive information of teeth to the greatest extent.

A three-dimensional tooth multimodal data registration method, comprising:

Obtain 3D tooth information and construct a 3D tooth model; obtain crown information and construct a 3D crown model; preprocess the 3D tooth model and 3D crown model;

Perform initial pose normalization, rough registration, and fine registration on the preprocessed 3D tooth model and 3D crown model, and then perform secondary fine matching on each single crown model in the 3D crown model Accurate, obtain the single crown model after rigid transformation;

According to each single tooth crown model after the rigid transformation, non-rigid transformation is performed on the corresponding single three-dimensional tooth model in the three-dimensional tooth model to obtain a three-dimensional tooth model with high-precision tooth crown information.

Further, the specific steps for preprocessing the 3D tooth model and the 3D crown model are:

Segment the three-dimensional tooth model to obtain several single tooth models; segment the three-dimensional crown model to obtain several single tooth crown models;

Perform mesh encryption processing on each single tooth model in the 3D tooth model, and smooth boundary processing on each single tooth crown model in the 3D crown model;

Label each individual tooth model and each individual crown model.

Further, the specific steps for normalizing the initial pose of the preprocessed 3D tooth model and 3D crown model are as follows:

Rotate the three-dimensional tooth model and the three-dimensional crown model to be parallel to the XOY plane, and remove the root part of the three-dimensional tooth model according to the center of gravity of the three-dimensional tooth model after rotation;

Normalize the positions of the rotated 3D crown model and the 3D tooth model after removal of the root portion.

Further, the specific steps for rough registration of the 3D tooth model and the 3D crown model after the initial pose normalization are as follows:

Sampling the farthest point on the normalized 3D crown model and 3D tooth model to obtain a fast point feature histogram of the 3D tooth model and a fast point feature histogram of the 3D crown model;

According to the fast point feature histogram of the 3D tooth model and the fast point feature histogram of the 3D crown model, estimate the corresponding points of the 3D crown model in the 3D tooth model and remove the wrong point pairs, and iteratively reduce the distance between the corresponding points. Realize global coarse registration.

Further, the specific steps for fine registration of the 3D tooth model and the 3D crown model after rough registration are:

According to the 3D tooth model and the 3D crown model after the global coarse registration, calculate the closest point of each point in the 3D crown model in the 3D tooth model, and obtain the transformation matrix that minimizes the distance between the corresponding closest points. The 3D crown model before point sampling is transformed to achieve global fine registration.

Further, the specific steps for secondary fine registration of each single crown model in the three-dimensional crown model are:

According to the single tooth model in the 3D tooth model, calculate the nearest point of each point in each single tooth model in the 3D crown model after fine registration in the corresponding single tooth model, and obtain the corresponding The transformation matrix with the smallest distance between the closest points transforms the single crown model before the farthest point sampling to achieve local fine registration.

Further, according to each single tooth crown model after rigid transformation, the specific steps of performing non-rigid transformation on the corresponding single three-dimensional tooth model in the three-dimensional tooth model are as follows:

Assign an affine transformation to each point of the 3D tooth model, and set the distance item and the stiffness item according to the rigidly transformed single crown model and the 3D tooth model;

Set regularization constraints on the distance term and stiffness term to obtain the best affine transformation matrix;

Through the optimal affine transformation matrix, each point of the three-dimensional tooth model is transformed to obtain a three-dimensional tooth model with high-precision crown information.

Further, after setting the distance item, before setting the stiffness item, it also includes:

Find the boundary point of the three-dimensional dental crown model, and set the boundary point of the three-dimensional dental crown model to the nearest point that the three-dimensional tooth model cannot find;

Calculate the F norm of the normal of each point after each transformation of the 3D crown model and the normal of the corresponding point in the 3D tooth model, and perform normal constraint.

Further, both the three-dimensional tooth model and the three-dimensional crown model are three-dimensional mesh models.

Next, a three-dimensional tooth multimodal data registration method disclosed in this embodiment will be described in detail with reference to FIGS. 1-12 . The three-dimensional tooth multimodal data registration method includes the following steps:

Step 1. Obtain three-dimensional tooth information, construct a three-dimensional tooth model, obtain dental crown information, and construct a three-dimensional dental crown model, wherein the three-dimensional tooth model and the three-dimensional dental crown model are both three-dimensional mesh models; for the three-dimensional dental model and three-dimensional dental crown model Perform preprocessing; specific steps include:

Step 1.1. Obtain three-dimensional tooth information through CBCT data, and construct a three-dimensional tooth model; obtain crown information through oral scan data, and construct a three-dimensional crown model.

Specifically, to read the CBCT data, directly read the tooth DICOM (Digital Imaging and Communications in Medicine, medical digital imaging and communication) file and convert the file format, obtain the bitmap file corresponding to each slice, and reconstruct the 3D based on the bitmap file Volume data, and based on the 3D volume data, obtain complete 3D tooth information through the ToothNet network model, and reconstruct the 3D tooth model; read the oral scan data, directly read the STL (Stereolithography, stereolithography) file obtained by the oral scan scanner, The 3D crown model with the same number of point clouds is obtained through the sampling algorithm, and the 3D tooth model and the 3D crown model are unified in obj format.

Step 1.2, segment the three-dimensional tooth model to obtain several single tooth models; segment the three-dimensional crown model to obtain several single crown models; in this embodiment, the three-dimensional tooth model is segmented to obtain 28 single teeth Model, three-dimensional crown model to obtain 28 single crown models.

Step 1.3, performing mesh encryption processing on each single tooth model in the three-dimensional tooth model.

Specifically, mesh subdivision is performed on each single tooth model, and the number of points and surfaces is increased by adding new edge points and updating original points.

Since the mesh density of the divided 3D crown model and the 3D tooth model are different, the mesh of the 3D tooth model is relatively sparse. The idea of subdivision is to subdivide the mesh of the three-dimensional tooth model, that is, add a vertex to each edge of the mesh, and connect the vertices in the same triangular patch with the new vertices to form a new triangular patch , adjust the position of each vertex at the same time, according to whether the point is a new point and whether it is a boundary point, there are the following four adjustment methods:

，新位置

计算如下： (1) For vertices that already exist inside the grid

, the new location

Calculated as follows:

为顶点

的度，

为

的邻接顶点，

为邻接顶点的权重。 in,

for the vertex

degree,

for

Adjacent vertices of ,

is the weight of adjacent vertices.

，新位置

计算如下： (2) For vertices that already exist on the grid boundary

, the new location

Calculated as follows:

和

为与

相邻的两个顶点。 in,

and

for with

two adjacent vertices.

，新位置

计算如下： (3) For the newly added vertices inside the mesh

, the new location

Calculated as follows:

位于由

和

构成的边上，

和

为共享

边的两个三角形面片的另 外两个点。 in,

located by

and

constituted by the edge,

and

for sharing

The other two points of the two triangle patches of the edge.

，新位置

计算如下： (4) New vertices for mesh boundaries

, the new location

Calculated as follows:

和

为构成一条边界的两个顶点。 in,

and

are the two vertices forming a boundary.

In this embodiment, according to the position of different points, whether the point is a new point and whether it is a boundary point, a corresponding adjustment method is selected for adjustment.

Step 1.4, performing boundary smoothing processing on each single crown model in the three-dimensional crown model, and removing the curling information of the crown.

Specifically, a low-pass filter constructed with a windowed sinc function is used to smooth the single crown model. The process of constructing the low-pass filter is as follows:

(1) Perform inverse Fourier transform on the ideal low-pass filter to obtain the sinc function;

(2) Intercept a section of the sinc function to obtain a new sinc function with discontinuous points;

(3) Select and generate window functions of the same size;

(4) The window function is multiplied by the new sinc function, so that the multiplied windowed signal can better meet the periodicity requirements of the Fourier transform;

(5) Perform Fourier forward transform on the windowed signal to obtain the final low-pass filter, so as to avoid excessive shrinkage during the smoothing process and reduce the loss of details.

Step 1.5: Label each single tooth model in the three-dimensional tooth model and each single tooth crown model in the three-dimensional crown model, so as to determine the corresponding relationship.

Specifically, in order to find the corresponding relationship conveniently, the application marks the single tooth model and the corresponding single crown model respectively according to the FDI International Dental Federation recording method. Exemplarily, taking 28 teeth as an example, the upper and lower teeth are divided into 4 groups, each group has 7 teeth, and each tooth is represented by two Arabic numerals, where the first digit indicates the quadrant where the tooth is located. The upper left, lower left, and lower right are the 1st, 2nd, 3rd, and 4th quadrants respectively. The second digit indicates the position of the teeth, from the middle to the edge of the incisors, lateral incisors, canines, first premolars, and second premolars , first molars, and second molars are marked 1-7, respectively.

Step 2. Global rigid transformation with the preprocessed 3D tooth model as the target data and the preprocessed 3D crown model as the source data, specifically including:

Step 2.1, perform initial pose normalization on the preprocessed 3D tooth model and 3D crown model; the specific steps are as follows:

Step 2.11, fitting the planes of the three-dimensional tooth model and the three-dimensional crown model.

Specifically, use the RANSAC (Random Sample Consensus) algorithm to fit the plane equations of the 3D tooth model and the 3D crown model, the process is as follows:

(1) Randomly select three points in the given point set of the 3D tooth model and the point set of the 3D crown model, and calculate the corresponding plane equation:

为平面法向量，

为常数项。in,

is the plane normal vector,

is a constant term.

(2) Calculate the distance of all points to this plane:

为平面法向量，

为常数项。 in,

is the plane normal vector,

is a constant term.

，若

，则该点作为局内点（inliers），否则为局外点 （outliers），记录局内点的个数，局内点的个数表示为

；其中，

=0.01。 (3) Set the distance threshold

,like

, then the point is regarded as inliers, otherwise it is outliers, record the number of inliers, and the number of inliers is expressed as

;in,

=0.01.

最大时，所选取的拟合参数

最佳。 (4) Repeat the above steps, when

When the maximum, the selected fitting parameters

optimal.

Step 2.12, using the best fitting parameters to obtain the rotation axis and rotation angle to rotate the 3D tooth model and the 3D crown model to be parallel to the XOY plane, the specific steps are as follows:

，三维牙冠模型的归一化平面法向量表示为

， XOY平面的法向量表示为

。 (1) Normalize the plane normal vectors of the 3D tooth model and the 3D crown model obtained above, and the normalized plane normal vector of the 3D tooth model is expressed as

, the normalized plane normal vector of the 3D crown model is expressed as

, the normal vector of the XOY plane is expressed as

.

和

分别与

进行点积运算，求取的夹角作为三维牙齿模型和三 维牙冠模型的旋转角度。 (2) Will

and

respectively with

The dot product operation is performed, and the calculated included angle is used as the rotation angle of the three-dimensional tooth model and the three-dimensional crown model.

和

分别与

进行叉积运算，将求取的向量归一化，作为三维牙齿 模型和三维牙冠模型的旋转轴。 (3) Will

and

respectively with

Carry out the cross product operation, normalize the calculated vector, and use it as the rotation axis of the three-dimensional tooth model and the three-dimensional crown model.

。罗德里格斯（Rodrigues）旋转方程如下： (4) Combining the rotation axis and rotation angle, the Rodrigues (Rodrigues) rotation equation is used to solve the rotation matrix of the 3D tooth model and the 3D crown model respectively

. The Rodrigues rotation equation is as follows:

表示单位矩阵，

表示旋转角度，

表示旋转轴。 in,

represents the identity matrix,

represents the rotation angle,

Indicates the axis of rotation.

进行变换即可将其 旋转至与XOY平面平行。 (5) Multiply the rotation matrix to the left of the point cloud in the 3D tooth model and the 3D crown model

Doing a transform rotates it to be parallel to the XOY plane.

Step 2.13, performing conditional filtering on the rotated three-dimensional tooth model to remove the tooth root.

Specifically, calculate the center of gravity of the three-dimensional tooth model after rotation, and use the center of gravity as the boundary. Since it is parallel to the XOY plane, set the scope to the z-axis. When processing the upper jaw data, keep the part smaller than the center of gravity. When processing the lower jaw data, keep The part larger than the center of gravity, thereby removing the root.

Step 2.14, performing normalization processing on the three-dimensional tooth model after tooth root removal and the three-dimensional crown model after rotation.

Specifically, calculate the center of gravity of the three-dimensional tooth model after tooth root removal and the three-dimensional crown model after rotation, and translate the three-dimensional tooth model after tooth root removal on the x, y, and z axes respectively, so that the center of gravity of the three-dimensional tooth model after tooth root removal Coincide with the origin of the coordinates, translate the rotated 3D crown model on the x, y, and z axes respectively, so that the center of the rotated 3D crown model coincides with the origin of the coordinates, so that the 3D tooth model and the 3D crown model are basically on the same plane.

Step 2.2. In order to improve the calculation efficiency, rough registration is performed on the 3D tooth model and the 3D crown model after the initial pose normalization process; the specific steps are as follows:

Step 2.21: Sampling the farthest point on the 3D tooth model and the 3D crown model with the root removed; for the point cloud, the farthest point sampling can cover all points in the space to the greatest extent, which can achieve uniform sampling and realize the process as follows:

中随机选取一个点

作为初始点，依次 计算其他点与初始点的距离并存入数组

中，选取距离最远的点

加入到采样点集

中。 (1) The point set of the 3D tooth model with the root removed

Randomly pick a point in

As the initial point, calculate the distance between other points and the initial point in turn and store them in an array

, select the point farthest from

Add to sample point set

middle.

中各点的距离，选取最近距离的点 存入数组

中； (2) Select the next sampling point and calculate other points to the sampling point set

The distance of each point, select the point with the closest distance and store it in the array

middle;

中选取最远距离对应的点

加入到采样点集

中。 (3) from

Select the point corresponding to the farthest distance

Add to sample point set

middle.

(4) Repeat (2) and (3) until the number of selected sampling points meets the set requirements, and the number of points set in this embodiment is 5000.

The process of performing the farthest point sampling operation on the three-dimensional crown model is the same as the above, and will not be repeated here.

In step 2.22, the normal information of the point cloud data in the 3D tooth model and the 3D crown model after the farthest point sampling are calculated respectively.

个最近点拟合出的平面法向量作为当前查询点的法向量，因牙齿中的纹理信息较为 丰富，因此为尽可能体现点云细节，设置K近邻数为10，计算拟合平面即分析一个协方差矩 阵的特征矢量和特征值，协方差矩阵从查询点的近邻元素中创建，对于每一个点

,对应的 协方差矩阵

如下： Specifically, using the PCA (Principal Component Analysis, principal component analysis) method, using kd-tree to find

The plane normal vector fitted by the nearest point is used as the normal vector of the current query point. Because the texture information in the tooth is rich, so in order to reflect the details of the point cloud as much as possible, the number of K nearest neighbors is set to 10, and the calculation of the fitting plane is to analyze a The eigenvectors and eigenvalues of the covariance matrix, the covariance matrix is created from the nearest neighbors of the query point, for each point

, the corresponding covariance matrix

as follows:

表示点

邻近点的数目，

表示最近邻元素的三维质心，

表示协方差矩 阵的第

个特征值，

表示第

个特征向量。 in,

Represent a point

the number of neighboring points,

represents the three-dimensional centroid of the nearest neighbor element,

Represents the covariance matrix's first

eigenvalues,

Indicates the first

feature vector.

In step 2.23, the FPFH (FastPoint Feature Histograms, fast point feature histograms) of the midpoints of the three-dimensional tooth model and the three-dimensional crown model are respectively calculated by using the normal.

三个特 征元素，与PFH相比少了领域点之间的互联，因FPFH的计算量比较大，为保证体现细节的同 时还能提高计算效率，利用kd-tree将K近邻数设置为20以确定点的最近邻集，将SPFH通过 如下权重组合来赋值给FPFH： Specifically, as a point cloud feature descriptor, FPFH is improved from PFH (Point Feature Histogram, point feature histogram), and the optimal sample surface change is obtained by combining the angle relationship between the three-dimensional coordinate axis data and the normal vector , which can simplify the complexity of histogram feature calculation, and only calculate the SPFH (Simplified Point Feature Histogram, simplified point feature histogram) of each point in its spatial neighborhood, including

Compared with PFH, the three feature elements have fewer interconnections between domain points. Because FPFH has a relatively large amount of calculation, in order to ensure the details and improve calculation efficiency, use kd-tree to set the number of K-nearest neighbors to 20 or more. Determine the nearest neighbor set of the point, and assign SPFH to FPFH through the following weight combination:

为查询点，

为邻近点，

为查询点与邻近点之间的距离权重。 in,

is the query point,

is a neighboring point,

is the distance weight between the query point and its neighbors.

Step 2.24, according to the fast point feature histogram of the 3D tooth model and the fast point feature histogram of the 3D crown model, estimate the corresponding point of the 3D crown model after the farthest point sampling in the 3D tooth model after the farthest point sampling , and reduce the distance between corresponding points through loop iterations to achieve global coarse registration.

Specifically, firstly, based on the RANSAC (Random Sample Consensus, random sampling consistency) algorithm idea, the corresponding relationship between the 3D tooth model and the 3D crown model is estimated according to the FPFH feature, the corresponding point pairs are initially estimated, and the wrong point pairs are removed, that is, random Find one or more points in the 3D tooth model that have similar FPFH features in the 3D tooth model, randomly select a point from these similar points as the corresponding point of the 3D crown model in the 3D tooth model, and perform cyclic iteration Reduce the distance between corresponding points, calculate the transformation matrix between corresponding points, use the "distance error sum" function such as Huber to evaluate the performance of the transformation matrix, until the best measurement error result is achieved, and finally use the obtained rigid transformation matrix as a global coarse registration result. The Huber penalty function is calculated as follows:

为预先给定值，

为第

组对应点变换之后的距离差。 in,

is a predetermined value,

for the first

The distance difference after the transformation of the corresponding points of the group.

Step 2.3. Perform fine registration on the 3D tooth model and the 3D crown model after the rough registration.

和平移向量

，利用 获取的变换矩阵对最远点采样前的三维牙冠模型进行变换，便可得到全局精配准结果。误 差函数用均方误差（MSE）表示如下： Specifically, in order to improve the registration efficiency, the 3D crown model after the farthest point sampling is used as the source data, the 3D tooth model before the farthest point sampling is used as the target data, and the above global rough registration result is used as the initial state, using ICP (Iterative Closest Point, iterative closest point) algorithm for fine registration, that is, to calculate the closest point of each point in the 3D crown model in the 3D tooth model and take it as the corresponding point, and obtain the The transformation with the smallest error function, iterative calculation until the convergence condition is met, the convergence condition can also include the maximum number of iterations, the difference between the two change matrices, etc., the purpose is to find the rotation matrix between the source data and the target data

and translation vector

, using the obtained transformation matrix to transform the three-dimensional crown model before the farthest point sampling, the global fine registration result can be obtained. The error function is expressed as the mean square error (MSE) as follows:

为待配准点云的数据集，

为目标点云中与

的最近对应点，

为三维 牙冠模型中点云的数量，

表示旋转矩阵，

表示平移向量。 in,

is the data set of the point cloud to be registered,

For the target point cloud and

the nearest corresponding point of ,

is the number of point clouds in the 3D crown model,

represents the rotation matrix,

Represents a translation vector.

In step 2.4, secondary fine registration is performed on each single crown model in the three-dimensional crown model.

Specifically, in order to further reduce the error, the global fine registration result is taken as the initial value, and the above-mentioned ICP fine registration is sequentially performed on the single three-dimensional crown model.

Step 3: Perform non-rigid transformation on the corresponding single three-dimensional tooth model in the three-dimensional tooth model according to each single tooth crown model after the rigid transformation, and obtain a three-dimensional tooth model with high-precision tooth crown information. Specifically include:

Step 3.1. Taking the rigidly transformed single tooth crown model as the target data and the single tooth model as the source data, an affine transformation is determined for each point in each single tooth model.

Step 3.2, setting the distance item weight.

，距离项如下： Specifically, in order to find the optimal deformation for a given stiffness, similar to the ICP fine registration in the above steps, it is necessary to search the points in the 3D crown model that are closest to the 3D tooth model to find a set of preliminary correspondences, and at the same time It is necessary to ensure that the corresponding points are constantly approaching, but the difference is that this step sets a weight for the distance

, the distance term is as follows:

为三维牙齿模型中的各个点，

为每个点对应的权重，

为每个点对应 的仿射变换，

为三维牙冠模型中的对应点。 in,

For each point in the 3D tooth model,

is the weight corresponding to each point,

is the affine transformation corresponding to each point,

is the corresponding point in the 3D crown model.

Use KNN (K-Nearest Neighbor, nearest neighbor search) to find the nearest point in the 3D crown model for each query point in the 3D tooth model, sort the obtained distance values in ascending order, and set a Gaussian kernel for the sorted distance The function calculates the weight. The smaller the distance is, the closer the point weight is to 1, and the larger the distance is, the closer the point weight is to 0. The Gaussian kernel function is as follows:

为距离权重，

为三维牙齿模型与三维牙冠模型对应点间的最短距 离，

描述距离的离散程度，在两次迭代中分别设置为0.2和0.3。 in,

is the distance weight,

is the shortest distance between the corresponding points of the 3D tooth model and the 3D crown model,

Describes how discrete the distance is, set to 0.2 and 0.3 in two iterations.

Step 3.3, finding the boundary points of the three-dimensional crown model and processing the missing tooth root data.

Specifically, in order to make the transformation effect of the crown and root in the 3D tooth model smooth, it is necessary to find the small boundary point of the 3D circle massage; because the 3D crown model lacks root information, the root part in the 3D tooth model is in the 3D crown model. Most of the corresponding points are the boundary points of the three-dimensional dental crown model. Through iteration, the root part of the three-dimensional dental model will move to the boundary point of the three-dimensional dental crown model as a whole. Fit the 3D crown model as much as possible, while the root part remains as immobile as possible, so the 3D crown model boundary points are regarded as corresponding points that cannot be found in the 3D tooth model, and the distance term weights corresponding to these points are set to 0.

Step 3.4. Solve the problem of surface flipping during the transformation process through normal constraints.

Specifically, calculate the F norm of the normal of each point in the three-dimensional tooth model and the normal of the corresponding point in the three-dimensional crown model after each transformation, and compare the normal of the middle point of the three-dimensional tooth model with the normal of the corresponding point in the three-dimensional crown model The line F norm is multiplied by the weight, the formula is as follows:

为三维牙齿模型一点

每次变换后的法线，

为

在三维牙冠模型中对 应点的法线，

为Frobenius范数的标志。 in,

A point for the 3D tooth model

Normals after each transformation,

for

The normal of the corresponding point in the 3D crown model,

is the sign of the Frobenius norm.

Step 3.5, set the stiffness item, which can constrain the transformation range of the three-dimensional tooth midpoint, and perform cyclic iterations with decreasing stiffness parameters, so that the points in the three-dimensional tooth model can gradually move to the corresponding points in the three-dimensional crown model, so that the three-dimensional The crown portion of the tooth model is continuously fitted to the 3D crown model.

，使得三维牙齿模型随着循环迭代逐渐向三维牙冠模型移动变 形，相邻点的变换自由度越来越高，可达到更加贴合三维牙冠模型的效果。刚度项如下：Specifically, because a single correspondence cannot uniquely determine an affine transformation, a stiffness item is set for this purpose, which can constrain adjacent vertices to perform similar transformations, making the transformed model as smooth as possible, and setting a series of decreasing Stiffness term weight

, so that the three-dimensional tooth model gradually moves and deforms to the three-dimensional crown model as the cycle iterates, and the degree of freedom of transformation of adjacent points becomes higher and higher, which can achieve the effect of fitting the three-dimensional crown model more closely. The stiffness term is as follows:

为三维牙齿模型中的一条边，

为三维牙齿模型中同一条边上的两个顶 点，

和

为两个相邻顶点的仿射变换，

为加权矩阵用来约束仿射变换中旋转和平移的 比重，在本实施例中，

为单位矩阵，

为Frobenius范数的标志。 in,

is an edge in the 3D tooth model,

are two vertices on the same edge in the 3D tooth model,

and

is the affine transformation of two adjacent vertices,

is the weighting matrix used to constrain the proportion of rotation and translation in the affine transformation, in this embodiment,

is the identity matrix,

is the sign of the Frobenius norm.

Step 3.5, the distance term and the stiffness term constitute a loss function.

的反复最小化，可以获 取使得三维牙齿模型中各点变换最优的矩阵集合

，损失函数如下： Specifically, the loss function is composed of the above-mentioned distance term and stiffness term, by

The iterative minimization of can obtain the matrix set that makes the transformation of each point in the 3D tooth model optimal

, the loss function is as follows:

为刚度项权重，

为刚度项，

为距离项。 in,

is the stiffness term weight,

is the stiffness item,

is the distance item.

，即可将三维牙齿模型中的各点进行相应的变换，从而获得 带有高精度牙冠信息的完整三维牙齿模型。 Set by applying matrix

, each point in the 3D tooth model can be transformed accordingly, so as to obtain a complete 3D tooth model with high-precision crown information.

Embodiment two

This embodiment discloses a three-dimensional tooth multimodal data registration system, including:

The data preprocessing module is configured to: obtain three-dimensional tooth information, construct a three-dimensional tooth model; obtain dental crown information, and construct a three-dimensional dental crown model; preprocess the three-dimensional tooth model and the three-dimensional dental crown model;

The rigid transformation module is configured to: perform initial pose normalization, rough registration and fine registration on the preprocessed 3D tooth model and 3D crown model in sequence, and then perform each single tooth in the 3D crown model The tooth model is subjected to secondary fine registration to obtain a single crown model after rigid transformation;

The non-rigid transformation module is configured to: perform non-rigid transformation on the corresponding single three-dimensional tooth model in the three-dimensional tooth model according to each single tooth crown model after the rigid transformation, and obtain a three-dimensional crown with high-precision tooth crown information. Tooth model.

It should be noted here that the above-mentioned data preprocessing module, rigid transformation module, and non-rigid transformation module correspond to the steps in Embodiment 1, and the examples and application scenarios implemented by the above-mentioned modules and corresponding steps are the same, but are not limited to the above-mentioned implementation The content disclosed in Example 1. It should be noted that, as a part of the system, the above-mentioned modules can be executed in a computer system such as a set of computer-executable instructions.

The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each procedure and/or block in the flowchart and/or block diagram, and combinations of procedures and/or blocks in the flowchart and/or block diagram can be realized by computer program instructions. These computer program instructions may be provided to a general purpose computer, special purpose computer, embedded processor, or processor of other programmable data processing equipment to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing equipment produce a Means for realizing the functions specified in one or more steps of the flowchart and/or one or more blocks of the block diagram.

These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to operate in a specific manner, such that the instructions stored in the computer-readable memory produce an article of manufacture comprising instruction means, the instructions The device realizes the function specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram.

These computer program instructions can also be loaded onto a computer or other programmable data processing device, and a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device Steps are provided for realizing the functions specified in the flow chart or flow charts and/or the block diagram block or blocks.

The description of each embodiment in the foregoing embodiments has its own emphases, and for parts not described in detail in a certain embodiment, reference may be made to relevant descriptions of other embodiments.

The above descriptions are only preferred embodiments of the present application, and are not intended to limit the present application. For those skilled in the art, various modifications and changes may be made to the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of this application shall be included within the protection scope of this application.

Claims (10)

1. A three-dimensional tooth multi-mode data registration method is characterized by comprising the following steps:

acquiring three-dimensional tooth information and constructing a three-dimensional tooth model; obtaining crown information and constructing a three-dimensional crown model; preprocessing a three-dimensional tooth model and a three-dimensional dental crown model;

sequentially carrying out initial pose normalization, rough registration and fine registration on the preprocessed three-dimensional tooth model and the preprocessed three-dimensional dental crown model, and carrying out secondary fine registration on each single crown model in the three-dimensional dental crown model to obtain a rigidly transformed single crown model;

and according to each single crown model after rigid transformation, carrying out non-rigid transformation on the corresponding single three-dimensional tooth model in the three-dimensional tooth model to obtain the three-dimensional tooth model with high-precision crown information.

2. The method for multi-modal data registration of three-dimensional teeth as claimed in claim 1, wherein the pre-processing of the three-dimensional tooth model and the three-dimensional crown model comprises the following specific steps:

segmenting the three-dimensional tooth model to obtain a plurality of single tooth models; segmenting the three-dimensional dental crown model to obtain a plurality of single dental crown models;

carrying out grid encryption processing on each single tooth model in the three-dimensional tooth model, and carrying out smooth boundary processing on each single crown model in the three-dimensional crown model;

each single tooth model and each single crown model are numbered.

3. The three-dimensional multi-modal dental data registration method of claim 1, wherein the initial pose normalization of the preprocessed three-dimensional dental model and the three-dimensional dental crown model comprises the following specific steps:

rotating the three-dimensional tooth model and the three-dimensional dental crown model to be parallel to an XOY plane, and removing the root part of the three-dimensional tooth model according to the gravity center of the rotated three-dimensional tooth model;

and normalizing the positions of the rotated three-dimensional dental crown model and the three-dimensional dental model with the root part removed.

4. The three-dimensional multi-modal dental data registration method of claim 3, wherein the step of coarsely registering the three-dimensional dental model and the three-dimensional dental crown model after the initial pose normalization processing comprises the steps of:

sampling farthest points of the normalized three-dimensional dental crown model and the normalized three-dimensional dental model to obtain a fast point characteristic histogram of the three-dimensional dental model and a fast point characteristic histogram of the three-dimensional dental crown model;

and estimating corresponding points of the three-dimensional dental crown model in the three-dimensional dental model and removing error point pairs according to the fast point characteristic histogram of the three-dimensional dental model and the fast point characteristic histogram of the three-dimensional dental crown model, and circularly iterating to reduce the distance between the corresponding points to realize global rough registration.

5. The three-dimensional dental multi-modality data registration method of claim 4, wherein the specific steps of performing the fine registration of the coarsely registered three-dimensional dental model and the three-dimensional dental crown model are as follows:

and according to the three-dimensional tooth model and the three-dimensional dental crown model after the global rough registration, calculating the closest point of each point in the three-dimensional dental crown model in the three-dimensional tooth model, acquiring a transformation matrix which enables the distance between the corresponding closest points to be minimum, and transforming the three-dimensional dental crown model before sampling the farthest point to realize the global precise registration.

6. The multi-modal data registration method for three-dimensional teeth according to claim 1, wherein the step of performing the secondary fine registration on each single crown model in the three-dimensional crown models comprises:

according to the single tooth model in the three-dimensional tooth model, calculating the nearest point of each point in each single tooth crown model in the three-dimensional tooth crown model after precise registration in the corresponding single tooth model, acquiring a transformation matrix which enables the distance between the corresponding nearest points to be minimum, and transforming the single tooth crown model before sampling the farthest points to realize local precise registration.

7. The method according to claim 1, wherein the non-rigid transformation of the corresponding single three-dimensional tooth model in the three-dimensional tooth models according to each rigidly transformed single crown model comprises the following specific steps:

distributing affine transformation for each point of the three-dimensional tooth model, and setting a distance term and a rigidity term according to the rigidly transformed single crown model and the three-dimensional tooth model;

setting regularization on the distance term and the rigidity term for constraint to obtain an optimal affine transformation matrix;

and transforming each point of the three-dimensional tooth model through the optimal affine transformation matrix to obtain the three-dimensional tooth model with high-precision dental crown information.

8. The three-dimensional dental multi-modality data registration method of claim 7, wherein setting the distance term further comprises:

searching boundary points of the three-dimensional dental crown model, and setting the boundary points of the three-dimensional dental crown model as the closest points which cannot be found by the three-dimensional dental model;

and calculating the F norm of each point normal after the three-dimensional dental crown model is transformed each time and the corresponding point normal in the three-dimensional dental model, and carrying out normal constraint.

9. The three-dimensional dental multi-modal data registration method of claim 1, wherein the three-dimensional dental model and the three-dimensional crown model are each a three-dimensional mesh model.

10. A three-dimensional dental multi-modality data registration system, comprising:

a data pre-processing module configured to: acquiring three-dimensional tooth information and constructing a three-dimensional tooth model; obtaining information of the dental crown and constructing a three-dimensional dental crown model; preprocessing a three-dimensional tooth model and a three-dimensional dental crown model;

a rigid transformation module configured to: sequentially carrying out initial pose normalization, rough registration and fine registration on the preprocessed three-dimensional tooth model and the preprocessed three-dimensional dental crown model, and carrying out secondary fine registration on each single tooth model in the three-dimensional dental crown model to obtain a rigidly transformed single tooth crown model;

a non-rigid transformation module configured to: and according to each single crown model after rigid transformation, carrying out non-rigid transformation on the corresponding single three-dimensional tooth model in the three-dimensional tooth model to obtain the three-dimensional tooth model with high-precision crown information.

Images