CN112348735A - Method and device for determining dynamic image deformation process and processor - Google Patents
Method and device for determining dynamic image deformation process and processor Download PDFInfo
- Publication number
- CN112348735A CN112348735A CN202011063066.7A CN202011063066A CN112348735A CN 112348735 A CN112348735 A CN 112348735A CN 202011063066 A CN202011063066 A CN 202011063066A CN 112348735 A CN112348735 A CN 112348735A
- Authority
- CN
- China
- Prior art keywords
- image
- deformation
- initial
- determining
- initial image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/18—Image warping, e.g. rearranging pixels individually
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T11/00—2D [Two Dimensional] image generation
- G06T11/003—Reconstruction from projections, e.g. tomography
- G06T11/008—Specific post-processing after tomographic reconstruction, e.g. voxelisation, metal artifact correction
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
Abstract
本申请提供了一种动态图像变形过程的确定方法、确定装置与处理器。该确定方法包括：获取初始图像和末尾图像，初始图像和末尾图像是对动态变形的目标物体进行连续拍摄得到的，初始图像为拍摄时间在先的图像，末尾图像为拍摄时间在后的图像；确定第一变形场，第一变形场表征了从初始图像到末尾图像的变形；将第一变形场划分为多个子变形场；应用各子变形场对初始图像进行变形计算，得到多个中间图像，子变形场与中间图像一一对应；至少根据中间图像确定从初始图像至末尾图像的中间变形过程。应用本方案得到的中间图像可以获取新增的像素点的信息，且避免了消失点引起的边界问题。
The present application provides a determination method, determination device and processor of a dynamic image deformation process. The determining method includes: acquiring an initial image and a final image, the initial image and the final image are obtained by continuously photographing a dynamically deformed target object, the initial image is an image with an earlier shooting time, and the final image is an image with a later shooting time; Determine the first deformation field, which represents the deformation from the initial image to the final image; divide the first deformation field into multiple sub-deformation fields; apply each sub-deformation field to deform the initial image to obtain multiple intermediate images , the sub-deformation field corresponds to the intermediate image one-to-one; the intermediate deformation process from the initial image to the final image is determined at least according to the intermediate image. The intermediate image obtained by applying this scheme can obtain the information of the newly added pixel points, and avoid the boundary problem caused by the vanishing point.
Description
Technical Field
The present application relates to the field of dynamic image processing, and in particular, to a method and an apparatus for determining a dynamic image deformation process, a computer-readable storage medium, and a processor.
Background
In recent years, the traditional medicine is greatly improved by medical imaging technologies such as nuclear magnetic resonance technology, CT tomography technology and the like, the diagnosis accuracy is improved, a large amount of data is provided for formulation of a treatment scheme, and therefore the examination, diagnosis and treatment effects are greatly improved.
The Large Deformation differential homomorphic Metric Mapping method (LDDMM) is widely used in brain science image processing. The LDDMM simulates a point in space through a flow field, and the flow field at each moment is described by the velocity vector field of the point. At a given moment, the spatial point moves from the starting position to the end position, giving the self-differential homomorphism of the spatial point. Based on the characteristics, the LDDMM can better reflect the geometric large deformation of the object, thereby realizing the registration task of the image when the object in the image changes greatly.
The existing LDDMM technology can well complete the registration of moving objects in dynamic images at different moments and provide corresponding speed fields (given time) of the initial and final images. However, the intermediate process of the pattern change cannot give an accurate prediction. In order to obtain the intermediate change process of the graph, a specific intermediate image insertion mode needs to be provided, and the state of the graph in the intermediate insertion image is obtained through deformation calculation of the initial image. Because the situation that pixel points are increased or decreased may exist in the deformation process from the initial image to the last image, however, the intermediate image obtained by applying the scheme may not include the newly added pixel points, and the problem may cause the boundary problem caused by the fact that the speed of the newly generated point on the graph cannot be obtained and the vanishing point. As with the existing method, further image registration is required for the intermediate state graph as well as the initial graph to solve the point addition and subtraction problem (process shown in fig. 3). Each inserted intermediate state pattern in this scheme needs to be registered with the original pattern, thus requiring a large amount of computation. For continuously changing images (such as dynamic medical images of a beating heart, a vibrating blood vessel, and the like), a large number of intermediate images are required to be inserted to capture a dynamic change process in one cycle, and the existing continuous registration method is hardly realized.
Disclosure of Invention
The present application mainly aims to provide a method, an apparatus, a computer-readable storage medium, and a processor for determining a dynamic image deformation process, so as to solve the problem in the prior art that the efficiency of the dynamic image deformation process is low.
In order to achieve the above object, according to an aspect of the present application, there is provided a method of determining a dynamic image warping process, including: acquiring an initial image and a final image, wherein the initial image and the final image are obtained by continuously shooting a dynamically deformed target object, the initial image is an image with a shooting time being prior, and the final image is an image with the shooting time being later; determining a first deformation field characterizing a deformation from the initial image to the end image; dividing the first deformation field into a plurality of sub-deformation fields; performing deformation calculation on the initial image by applying each sub-deformation field to obtain a plurality of intermediate images, wherein the sub-deformation fields correspond to the intermediate images one to one; determining an intermediate warping process from the initial image to the end image based at least on the intermediate image.
Further, determining a first deformation field includes: registering the initial image and the final image to obtain a registration result; determining the first deformation field according to the registration result.
Further, determining an intermediate warping process from the initial image to the end image based at least on the intermediate image comprises: mapping the corresponding intermediate image to an initial coordinate space according to the sub-deformation field to obtain a mapped image, wherein the initial coordinate space is the coordinate space where the initial image is located; comparing the mapping image with the intermediate image to obtain a comparison result; and determining the intermediate deformation process from the initial image to the final image according to the comparison result.
Further, comparing the mapping image with the intermediate image to obtain a comparison result, including: determining a mapping position according to the mapping image, wherein the mapping position is the position of a pixel point on the intermediate image in the initial coordinate space; determining an intermediate position, wherein the intermediate position is the position of a pixel point on the intermediate image in an intermediate coordinate space, and the intermediate coordinate space is the coordinate space where the intermediate image is located; determining a first deformation amount, where the first deformation amount is a deformation amount from the mapping position to the intermediate position, and the first deformation amount is the comparison result.
Further, according to the comparison result, determining the intermediate deformation process from the initial image to the end image, including: determining the intermediate deformation process from the initial image to the end image according to at least the first deformation amount.
Further, determining the intermediate deformation process from the initial image to the end image according to at least the first deformation amount includes: determining a second deformation amount, wherein the second deformation amount is the deformation amount from the pixel point in the initial image to the corresponding pixel point in the tail image; acquiring a preset ratio which is the ratio of the first deformation to the second deformation; determining a second deformation field, said second deformation field being a product of said predetermined ratio and said first deformation field, said second deformation field characterizing a deformation from said initial image to said intermediate image; determining the intermediate deformation process from the initial image to the end image according to the second deformation field.
Further, after determining the intermediate deformation process from the initial image to the end image according to the second deformation field, the determination method further includes: acquiring deformation time, wherein the deformation time is the time from the initial image to the intermediate image; determining a velocity field, the velocity field being a ratio of the second deformation field to the deformation time.
According to another aspect of the present application, there is provided a dynamic image warping determination apparatus, including: a first obtaining unit, configured to obtain an initial image and an end image, where the initial image and the end image are obtained by continuously shooting a dynamically-deformed target object, the initial image is an image with a previous shooting time, and the end image is an image with a later shooting time; a first determining unit for determining a first deformation field characterizing a deformation from the initial image to the end image; a dividing unit configured to divide the first deformation field into a plurality of sub-deformation fields; the computing unit is used for carrying out deformation computation on the initial image by applying each sub-deformation field to obtain a plurality of intermediate images, and the sub-deformation fields correspond to the intermediate images one by one; a second determining unit for determining an intermediate warping process from the initial image to the end image at least based on the intermediate image.
According to still another aspect of the present application, there is provided a computer-readable storage medium including a stored program, wherein the program, when executed, controls an apparatus in which the computer-readable storage medium is located to execute any one of the determination methods of the dynamic image deformation process.
According to still another aspect of the present application, there is provided a processor for executing a program, wherein the program executes any one of the determination methods of the dynamic image warping process.
By applying the technical scheme, the initial image and the tail image are obtained, the first deformation field representing the deformation from the initial image to the tail image is determined, the first deformation field is divided to obtain a plurality of sub deformation fields, then the deformation calculation is carried out on the initial image according to the sub deformation fields to obtain a plurality of intermediate images, the accurate obtaining of the intermediate images is achieved, and the sub deformation fields in the scheme are obtained by dividing the first deformation field, so that the intermediate images obtained by applying the scheme can obtain the information of newly added pixel points, and the boundary problem caused by vanishing points is avoided. The method and the device avoid the operation of additionally registering a large number of images required for acquiring the intermediate images and the speed in the prior art, are suitable for continuously changing images, and realize the acquisition of the intermediate deformation process from the initial image to the final image without a large amount of calculation.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this application, illustrate embodiments of the application and, together with the description, serve to explain the application and are not intended to limit the application. In the drawings:
FIG. 1 shows a flow chart of a method for determining a dynamic image warping process according to an embodiment of the present application;
FIG. 2 shows a schematic diagram of a determination apparatus for a dynamic image warping process according to an embodiment of the present application;
FIG. 3 is a schematic diagram illustrating a method for determining a dynamic image deformation process in the prior art;
FIG. 4 is a schematic diagram illustrating another dynamic image warping process determination method according to an embodiment of the present application;
FIG. 5 shows a schematic diagram of an initial image, an intermediate image and an end image according to an embodiment of the application;
FIG. 6 shows a schematic diagram of a boundary velocity profile of an initial image, an intermediate image and an end image according to an embodiment of the application;
fig. 7 shows a first deformation field and a sub-deformation field schematic according to an embodiment of the application.
Detailed Description
It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict. The present application will be described in detail below with reference to the embodiments with reference to the attached drawings.
In order to make the technical solutions better understood by those skilled in the art, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only partial embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
It should be noted that the terms "first," "second," and the like in the description and claims of this application and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It should be understood that the data so used may be interchanged under appropriate circumstances such that embodiments of the application described herein may be used. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
It will be understood that when an element such as a layer, film, region, or substrate is referred to as being "on" another element, it can be directly on the other element or intervening elements may also be present. Also, in the specification and claims, when an element is described as being "connected" to another element, the element may be "directly connected" to the other element or "connected" to the other element through a third element.
As described in the background art, the deformation process for acquiring a dynamic image in the prior art is inefficient, and to solve the problem of inefficient deformation process for acquiring a dynamic image as described above, embodiments of the present application provide a method and an apparatus for determining a dynamic image deformation process, a computer-readable storage medium, and a processor.
According to an embodiment of the present application, there is provided a method of determining a dynamic image warping process.
Fig. 1 is a flowchart of a method for determining a dynamic image warping process according to an embodiment of the present application. As shown in fig. 1, the method comprises the steps of:
step S101, obtaining an initial image and a final image, wherein the initial image and the final image are obtained by continuously shooting a dynamically deformed target object, the initial image is an image with a shooting time prior, and the final image is an image with the shooting time after;
step S102, determining a first deformation field, wherein the first deformation field represents the deformation from the initial image to the end image;
step S103, dividing the first deformation field into a plurality of sub deformation fields;
step S104, carrying out deformation calculation on the initial image by applying each sub-deformation field to obtain a plurality of intermediate images, wherein the sub-deformation fields correspond to the intermediate images one by one;
step S105, determining an intermediate transformation process from the initial image to the final image at least based on the intermediate image.
In the scheme, the initial image and the tail image are obtained, the first deformation field representing deformation from the initial image to the tail image is determined, the first deformation field is divided to obtain a plurality of sub deformation fields, then deformation calculation is carried out on the initial image according to the sub deformation fields to obtain a plurality of intermediate images, accurate obtaining of the intermediate images is achieved, and the sub deformation fields are obtained by dividing the first deformation field, so that the intermediate images obtained by applying the scheme can obtain information of newly added pixel points, and the boundary problem caused by vanishing points is avoided. The method and the device avoid the operation of additionally registering a large number of images required for acquiring the intermediate images and the speed in the prior art, are suitable for continuously changing images, and realize the acquisition of the intermediate deformation process from the initial image to the final image without a large amount of calculation.
Specifically, the initial image and the final image are two consecutive images obtained by consecutively photographing the target object that is dynamically deformed.
Specifically, dividing the first deformation field into a plurality of sub-deformation fields includes: and in the time domain, dividing the first deformation field into a plurality of sub deformation fields in a linear division mode. For example, an initial image is acquired at the 1 st second, an end image is acquired at the 2 nd second, a first deformation field is determined according to the initial image and the end image, and a sub deformation field of 1.25s, a sub deformation field of 1.5s and a sub deformation field of 1.75s are obtained by adopting a linear interpolation mode. Of course, the way of acquiring the sub-deformation field may also be a way of non-linear interpolation.
It should be noted that the steps illustrated in the flowcharts of the figures may be performed in a computer system such as a set of computer-executable instructions and that, although a logical order is illustrated in the flowcharts, in some cases, the steps illustrated or described may be performed in an order different than presented herein.
In an embodiment of the present application, in a case where a deformation process of an image is relatively simple, only one intermediate image is acquired, and determination of an intermediate deformation process from the initial image to the final image is achieved.
In one embodiment of the present application, determining a first deformation field includes: registering the initial image and the final image to obtain a registration result; determining the first deformation field according to the registration result. Specifically, a large deformation differential homoembryo metric mapping method may be adopted to perform registration on the initial image and the end image to obtain the registration result. An accurate acquisition of the first deformation field is achieved, which accurately reflects the deformation from the initial image to the final image. Of course, the method for registering the initial image and the last image is not limited to the large deformation differential homoembryo metric mapping method, and other registration methods may be adopted to realize registration of the initial image and the last image.
In another embodiment of the present application, determining an intermediate warping process from the initial image to the final image at least based on the intermediate image includes: mapping the corresponding intermediate image into an initial coordinate space according to the sub-deformation field to obtain a mapped image, wherein the initial coordinate space is the coordinate space where the initial image is located; comparing the mapping image with the intermediate image to obtain a comparison result; and determining the intermediate deformation process from the initial image to the final image according to the comparison result. The intermediate image is mapped to the initial coordinate space to obtain a mapping image of the intermediate image in the initial coordinate space, and the intermediate image may have an increase of pixel points and/or a decrease of pixel points relative to the initial image, so that coordinates of pixel points on the mapping image corresponding to the intermediate image may be different, for example, a coordinate of a certain pixel point on the intermediate image is (5, 5, 5), and a coordinate of a pixel point on the mapping image corresponding to the pixel point is (4.5, 4.5, 4.5), and by comparing the mapping image with the intermediate image, the determination of the intermediate deformation process from the initial image to the final image is realized according to a comparison result.
In another embodiment of the present application, comparing the mapping image with the intermediate image to obtain a comparison result includes: determining a mapping position according to the mapping image, wherein the mapping position is the position of a pixel point on the intermediate image in the initial coordinate space; determining a middle position, wherein the middle position is the position of a pixel point on the middle image in a middle coordinate space, and the middle coordinate space is the coordinate space of the middle image; determining a first deformation amount, wherein the first deformation amount is a deformation amount from the mapping position to the intermediate position, and the first deformation amount is the comparison result. For example, the coordinate of a certain pixel point on the intermediate image in the intermediate coordinate space is (5, 5, 5), the coordinate of the pixel point in the initial coordinate space is (4.5, 4.5, 4.5), the first deformation is the change from the coordinate (5, 5, 5) to the coordinate (4.5, 4.5, 4.5), and according to the first deformation corresponding to the coordinates of all the pixel points on the intermediate image, the comparison result between the mapping image and the intermediate image can be obtained, so that the mapping image and the intermediate image are accurately compared.
In another embodiment of the present application, determining the intermediate transformation process from the initial image to the final image according to the comparison result includes: the intermediate deformation process from the initial image to the final image is determined at least according to the first deformation, and the intermediate deformation process from the initial image to the final image can be obtained according to the first deformation corresponding to the coordinates of all the pixel points on the intermediate image.
In one embodiment of the present application, the determining the intermediate transformation process from the initial image to the final image at least according to the first transformation amount includes: determining a second deformation amount, wherein the second deformation amount is the deformation amount from the pixel point in the initial image to the corresponding pixel point in the tail image; acquiring a preset ratio which is the ratio of the first deformation to the second deformation; determining a second deformation field, said second deformation field being a product of said predetermined ratio and said first deformation field, said second deformation field characterizing a deformation from said initial image to said intermediate image; determining the intermediate warping process from the initial image to the final image according to the second warping field. Specifically, the second distortion amount is a distortion amount from the initial image to the end image, the first distortion amount is substantially a distortion amount from the initial image to the intermediate image, then the first distortion field is multiplied by a ratio of the first distortion amount to the second distortion amount, so that a distortion field from the initial image to the intermediate image, namely the second distortion field, is obtained, and the intermediate distortion process from the initial image to the end image is determined according to the second distortion field, so that the accurate determination of the intermediate distortion process from the initial image to the end image is realized.
In an embodiment of the application, after determining the intermediate deformation process from the initial image to the final image according to the second deformation field, the determining method further includes: acquiring deformation time, wherein the deformation time is the time for changing from the initial image to the intermediate image; determining a velocity field, the velocity field being a ratio of the second deformation field to the deformation time. The velocity field from the initial image to the intermediate image is determined from the second deformation field and the deformation time. The method effectively solves the speed acquisition problem of the newly generated point of the object inserted into the image in the middle, avoids a large amount of image registration calculation required by the existing method when acquiring the speed field, and greatly improves the calculation efficiency.
The embodiment of the present application further provides a device for determining a dynamic image deformation process, and it should be noted that the device for determining a dynamic image deformation process according to the embodiment of the present application may be used to execute the method for determining a dynamic image deformation process according to the embodiment of the present application. The following describes a device for determining a moving image warping process according to an embodiment of the present application.
Fig. 2 is a schematic diagram of a determination device of a dynamic image warping process according to an embodiment of the present application. As shown in fig. 2, the apparatus includes:
a first acquiring unit 10 configured to acquire an initial image and an end image, the initial image and the end image being obtained by continuously capturing a target object that is dynamically deformed, the initial image being an image whose capturing time is earlier, and the end image being an image whose capturing time is later;
a first determining unit 20 for determining a first deformation field, said first deformation field representing a deformation from said initial image to said final image;
a dividing unit 30 for dividing the first deformation field into a plurality of sub-deformation fields;
a calculating unit 40, configured to perform a deformation calculation on the initial image by using each of the sub-deformation fields to obtain a plurality of intermediate images, where the sub-deformation fields correspond to the intermediate images one to one;
a second determining unit 50, configured to determine an intermediate warping process from the initial image to the final image at least according to the intermediate image.
In the scheme, the first obtaining unit obtains an initial image and a tail image, the first determining unit determines a first deformation field representing deformation from the initial image to the tail image, the dividing unit divides the first deformation field to obtain a plurality of sub deformation fields, the calculating unit performs deformation calculation on the initial image according to the sub deformation fields to obtain a plurality of intermediate images, accurate obtaining of the intermediate images is achieved, and the sub deformation fields are obtained by dividing the first deformation field, so that the intermediate images obtained by applying the scheme can obtain information of newly added pixel points, and boundary problems caused by vanishing points are avoided. The method and the device avoid the operation of additionally registering a large number of images required for acquiring the intermediate images and the speed in the prior art, are suitable for continuously changing images, and realize the acquisition of the intermediate deformation process from the initial image to the final image without a large amount of calculation.
Specifically, the initial image and the final image are two consecutive images obtained by consecutively photographing the target object that is dynamically deformed.
Specifically, the dividing unit is further configured to divide the first deformation field into a plurality of sub-deformation fields in a time domain by using a linear division manner. For example, an initial image is acquired at the 1 st second, an end image is acquired at the 2 nd second, a first deformation field is determined according to the initial image and the end image, and a sub deformation field of 1.25s, a sub deformation field of 1.5s and a sub deformation field of 1.75s are obtained by adopting a linear interpolation mode. Of course, the way of acquiring the sub-deformation field may also be a way of non-linear interpolation.
In an embodiment of the present application, the first determining unit includes a registration module and a first determining module, and the registration module is configured to register the initial image and the final image to obtain a registration result; the first determining module is used for determining the first deformation field according to the registration result. Specifically, a large deformation differential homoembryo metric mapping method may be adopted to perform registration on the initial image and the end image to obtain the registration result. An accurate acquisition of the first deformation field is achieved, which accurately reflects the deformation from the initial image to the final image. Of course, the method for registering the initial image and the last image is not limited to the large deformation differential homoembryo metric mapping method, and other registration methods may be adopted to realize registration of the initial image and the last image.
In another embodiment of the present application, the second determining unit includes a mapping module, a comparing module and a second determining module, where the mapping module is configured to map the corresponding intermediate image into an initial coordinate space according to the sub-deformation field to obtain a mapped image, and the initial coordinate space is a coordinate space where the initial image is located; the comparison module is used for comparing the mapping image with the intermediate image to obtain a comparison result; the second determining module is configured to determine the intermediate deformation process from the initial image to the final image according to the comparison result. The intermediate image is mapped to the initial coordinate space to obtain a mapping image of the intermediate image in the initial coordinate space, and the intermediate image may have an increase of pixel points and/or a decrease of pixel points relative to the initial image, so that coordinates of pixel points on the mapping image corresponding to the intermediate image may be different, for example, a coordinate of a certain pixel point on the intermediate image is (5, 5, 5), and a coordinate of a pixel point on the mapping image corresponding to the pixel point is (4.5, 4.5, 4.5), and by comparing the mapping image with the intermediate image, the determination of the intermediate deformation process from the initial image to the final image is realized according to a comparison result.
In another embodiment of the present application, the comparing module is further configured to determine a mapping position according to the mapping image, where the mapping position is a position of a pixel point on the intermediate image in the initial coordinate space; determining a middle position, wherein the middle position is the position of a pixel point on the middle image in a middle coordinate space, and the middle coordinate space is the coordinate space of the middle image; determining a first deformation amount, wherein the first deformation amount is a deformation amount from the mapping position to the intermediate position, and the first deformation amount is the comparison result. For example, the coordinate of a certain pixel point on the intermediate image in the intermediate coordinate space is (5, 5, 5), the coordinate of the pixel point in the initial coordinate space is (4.5, 4.5, 4.5), the first deformation is the change from the coordinate (5, 5, 5) to the coordinate (4.5, 4.5, 4.5), and according to the first deformation corresponding to the coordinates of all the pixel points on the intermediate image, the comparison result between the mapping image and the intermediate image can be obtained, so that the mapping image and the intermediate image are accurately compared.
In yet another embodiment of the present application, the second determining module is further configured to: the intermediate deformation process from the initial image to the final image is determined at least according to the first deformation, and the intermediate deformation process from the initial image to the final image can be obtained according to the first deformation corresponding to the coordinates of all the pixel points on the intermediate image.
In an embodiment of the present application, the second determining module is further configured to determine a second deformation amount, where the second deformation amount is a deformation amount from a pixel point in the initial image to a corresponding pixel point in the final image; acquiring a preset ratio which is the ratio of the first deformation to the second deformation; determining a second deformation field, said second deformation field being a product of said predetermined ratio and said first deformation field, said second deformation field characterizing a deformation from said initial image to said intermediate image; determining the intermediate warping process from the initial image to the final image according to the second warping field. Specifically, the second distortion amount is a distortion amount from the initial image to the end image, the first distortion amount is substantially a distortion amount from the initial image to the intermediate image, then the first distortion field is multiplied by a ratio of the first distortion amount to the second distortion amount, so that a distortion field from the initial image to the intermediate image, namely the second distortion field, is obtained, and the intermediate distortion process from the initial image to the end image is determined according to the second distortion field, so that the accurate determination of the intermediate distortion process from the initial image to the end image is realized.
In an embodiment of the present application, the determining apparatus further includes a second obtaining unit and a third determining unit, the second obtaining unit is configured to obtain a deformation time after determining the intermediate deformation process from the initial image to the end image according to the second deformation field, the deformation time being a time for changing from the initial image to the intermediate image; a third determining unit is configured to determine a velocity field, which is a ratio of the second deformation field to the deformation time, after determining the intermediate deformation process from the initial image to the end image based on the second deformation field. The velocity field from the initial image to the intermediate image is determined from the second deformation field and the deformation time. The scheme effectively solves the speed acquisition problem of the newly generated point of the object in the middle of the inserted image, avoids a large amount of image registration calculation required by the existing device when acquiring a speed field, and greatly improves the calculation efficiency.
The determination device of the dynamic image deformation process comprises a processor and a memory, wherein the first acquisition unit, the first determination unit, the dividing unit, the calculation unit, the second determination unit and the like are stored in the memory as program units, and the processor executes the program units stored in the memory to realize corresponding functions.
The processor comprises a kernel, and the kernel calls the corresponding program unit from the memory. The kernel can be set to be one or more than one, and the efficiency of the deformation process of acquiring the dynamic image is improved by adjusting the kernel parameters.
The memory may include volatile memory in a computer readable medium, Random Access Memory (RAM) and/or nonvolatile memory such as Read Only Memory (ROM) or flash memory (flash RAM), and the memory includes at least one memory chip.
The embodiment of the invention provides a computer-readable storage medium, which comprises a stored program, wherein when the program runs, a device where the computer-readable storage medium is located is controlled to execute the determination method of the dynamic image deformation process.
The embodiment of the invention provides a processor, which is used for running a program, wherein the program executes the method for determining the dynamic image deformation process when running.
The embodiment of the invention provides equipment, which comprises a processor, a memory and a program which is stored on the memory and can run on the processor, wherein when the processor executes the program, at least the following steps are realized:
step S101, obtaining an initial image and a final image, wherein the initial image and the final image are obtained by continuously shooting a dynamically deformed target object, the initial image is an image with a shooting time prior, and the final image is an image with the shooting time after;
step S102, determining a first deformation field, wherein the first deformation field represents the deformation from the initial image to the end image;
step S103, dividing the first deformation field into a plurality of sub deformation fields;
step S104, carrying out deformation calculation on the initial image by applying each sub-deformation field to obtain a plurality of intermediate images, wherein the sub-deformation fields correspond to the intermediate images one by one;
step S105, determining an intermediate transformation process from the initial image to the final image at least based on the intermediate image.
The device herein may be a server, a PC, a PAD, a mobile phone, etc.
The present application further provides a computer program product adapted to perform a program of initializing at least the following method steps when executed on a data processing device:
step S101, obtaining an initial image and a final image, wherein the initial image and the final image are obtained by continuously shooting a dynamically deformed target object, the initial image is an image with a shooting time prior, and the final image is an image with the shooting time after;
step S102, determining a first deformation field, wherein the first deformation field represents the deformation from the initial image to the end image;
step S103, dividing the first deformation field into a plurality of sub deformation fields;
step S104, carrying out deformation calculation on the initial image by applying each sub-deformation field to obtain a plurality of intermediate images, wherein the sub-deformation fields correspond to the intermediate images one by one;
step S105, determining an intermediate transformation process from the initial image to the final image at least based on the intermediate image.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). The memory is an example of a computer-readable medium.
Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in the process, method, article, or apparatus that comprises the element.
Examples
The embodiment relates to a specific method for determining a dynamic image deformation process, which comprises the following steps:
step S1 of acquiring an initial image and a final image, the initial image and the final image being obtained by continuously capturing a target object that is dynamically deformed, the initial image being an image whose capturing time is earlier and the final image being an image whose capturing time is later;
step S2, registering the initial image and the end image to obtain a registration result; determining said first deformation field based on said registration result, said first deformation field characterizing a deformation from said initial image to said final image;
step S3, dividing the first deformation field into a plurality of sub-deformation fields;
one expression of the registration control energy equation is shown in equation 1 (the coefficients and similarity terms are variable):
wherein epsilon is energy consumption, t is time, L is a differential operator, delta is a similarity coefficient, f is an initial image, mu is an end image, Id is unit deformation, v is velocity, and is defined as
Is the amount of deformation. Boundary condition being unit deformation
The solution of the obtained deformation amount is the deformation amount corresponding to the minimum energy
Namely, the first deformation field, the specific solving process is to perform derivation on ∈, the corresponding deformation amount when the derivation result is minimum is the first deformation field, and the process of solving equation 1 is the registration process in step S2.
For the deformation amount obtained in step S2
Which can be understood as a combination of a plurality of sub-deformation fields, as shown in equation 2:
wherein (Id + v)nV) is a single step small deformation, vnThe operator ° represents a combination of two variables for the speed values corresponding to a single step of small deformation. The combination mode meets the following four requirements:
1. all elements a, b, a DEG b belonging to the operation domain G also belong to G;
2. for all elements a, b, c belonging to G, (a ° b) ° c ═ a ° (b ° c);
g, there is a unit deformation Id, for all elements a in G, a ° Id ═ Id ° a ═ a;
4. for any one of a belonging to G, there is one element b satisfying a ° b ═ b ° a ═ Id.
The manner in which the sub-deformation quantities are obtained by splitting the first deformation field may be varied. The results of obtaining the sub-deformation amounts by three different splitting manners are shown in fig. 7. Part a in fig. 7 represents a first deformation field from the initial image (represented by squares in fig. 7) to the final image (represented by triangles in fig. 7). For this first deformation field, a plurality of sub-deformation fields can be obtained by linear splitting (part B in fig. 7, deformation amounts at equal time intervals, etc.), non-linear splitting at equal time intervals (part C in fig. 7, increasing first and decreasing second (gradient descent), decreasing first and increasing second (gradient ascent)), and non-linear splitting at equal time intervals (part D in fig. 7). The deformation amount from the deformation to the nth step corresponding to the splitting method adopted in the parts B to D in fig. 7 is shown in formula 3:
in fig. 7, part a represents the first deformation field from beginning to end; part B represents a sub-deformation field obtained by linear interpolation; part C represents a sub-deformation field obtained by adopting an equal-time nonlinear insertion mode; part D represents the sub-deformation field obtained by means of non-equal time non-linear interpolation.
Step S4, performing a deformation calculation on the initial image by using each of the sub-deformation fields to obtain a plurality of intermediate images, where the sub-deformation fields correspond to the intermediate images one-to-one, fig. 4 shows a plurality of intermediate images obtained by using the present scheme, i.e. intermediate image 1, intermediate image 2, …, intermediate image n, fig. 3 shows a plurality of intermediate images obtained by using a scheme in the prior art, i.e. first intermediate image, second intermediate image, and nth intermediate image, respectively, in order to solve the increase and decrease problem of pixel points in the prior art, it is necessary to perform a re-registration on each intermediate image and the initial image, so that a large amount of calculation is required, and the present scheme does not need to perform a re-registration on the intermediate images and the initial images, thereby greatly reducing the amount of calculation, fig. 5 shows schematic diagrams of the initial image, the intermediate image, and the final image according to an embodiment, a is a 3D model built based on an initial image; b is a 3D model built based on the last image; c is a model established by the intermediate insertion image (intermediate image) at 1/4, 1/2 and 3/4 time intervals; d is a partial enlarged view of the model built by the intermediate insertion image at 1/4, 1/2 and 3/4 time intervals, T0 corresponds to the initial image, T0.25 corresponds to the intermediate image built at 1/4 time intervals, T0.5 corresponds to the intermediate image built at 1/2 time intervals, T0.75 corresponds to the intermediate image built at 3/4 time intervals, T1 corresponds to the end image, and the partial enlarged view of the end image is obviously larger relative to the initial image as can be seen from FIG. 5;
a step S5 of determining an intermediate transformation process from the initial image to the final image based on at least the intermediate image;
step S5 includes the following steps:
step A, mapping the corresponding intermediate image into an initial coordinate space according to the sub-deformation field to obtain a mapping image, wherein the initial coordinate space is the coordinate space where the initial image is located, and determining a mapping position according to the mapping image, wherein the mapping position is the position of a pixel point on the intermediate image in the initial coordinate space;
step B, determining an intermediate position, wherein the intermediate position is the position of a pixel point on the intermediate image in an intermediate coordinate space, and the intermediate coordinate space is the coordinate space of the intermediate image;
step C of determining a first deformation amount, which is a deformation amount from the mapping position to the intermediate position;
step D, determining a second deformation amount, wherein the second deformation amount is the deformation amount from the pixel point in the initial image to the corresponding pixel point in the tail image;
step E, obtaining a preset ratio, wherein the preset ratio is the ratio of the first deformation to the second deformation;
step F of determining a second deformation field, said second deformation field being a product of said predetermined ratio and said first deformation field, said second deformation field characterizing a deformation from said initial image to said intermediate image;
and G, determining the intermediate deformation process from the initial image to the final image according to the second deformation field.
Step S6 of acquiring a transformation time from the initial image to the intermediate image; determining a velocity field, wherein the velocity field is a ratio of the second deformation field to the deformation time, fig. 6 shows a schematic boundary velocity distribution diagram of the initial image, the intermediate image and the end image according to an embodiment of the present application, an arrow in a part a in fig. 6 based on black on a 3D model created by the initial image represents a velocity field obtained from the initial image to the end image, a part B represents a partial enlarged view of the velocity distribution (velocity field) obtained from the initial image to the end image, and a part C represents a model created by the intermediate interpolated image at 1/4, 1/2, 3/4 time intervals; the D part represents the model of the intermediate interpolated image at 1/4, 1/2, 3/4 time intervals and the boundary velocity profile (i.e., the velocity field of the intermediate image relative to the initial image).
By adopting the method for determining the dynamic image deformation process, the accurate determination of the intermediate deformation process from the initial image to the final image is realized, the calculation amount is greatly reduced, the method is suitable for continuously changing images, and the intermediate deformation process from the initial image to the final image is obtained without a large amount of calculation. The method avoids a large amount of extra image registration operations required by the conventional method for acquiring the intermediate image and speed, and further improves the feasibility, operability and efficiency for acquiring the intermediate state and speed of the object motion in the dynamic medical image. Meanwhile, the scheme effectively solves the speed acquisition problem of the newly generated point of the object inserted into the image in the middle, avoids a large amount of image registration calculation required by the existing method for acquiring the speed field, and greatly improves the calculation efficiency.
The method provided by the embodiment can effectively solve the problem of value taking of the position and the movement speed of the object caused by unknown details of the object movement process recorded in the medical image (such as a CT image), and can provide reliable boundary conditions for a numerical simulation model for further analyzing the object movement. The method avoids a large amount of extra image registration operations required by the conventional method for acquiring the intermediate image and speed, and further improves the feasibility, operability and efficiency for acquiring the intermediate state and speed of the object motion in the dynamic medical image.
From the above description, it can be seen that the above-described embodiments of the present application achieve the following technical effects:
1) the method for determining the dynamic image deformation process comprises the steps of obtaining an initial image and a tail image, then determining a first deformation field representing deformation from the initial image to the tail image, dividing the first deformation field to obtain a plurality of sub deformation fields, then performing deformation calculation on the initial image according to the sub deformation fields to obtain a plurality of intermediate images, and achieving accurate obtaining of the intermediate images. The method and the device avoid the operation of additionally registering a large number of images required for acquiring the intermediate images and the speed in the prior art, are suitable for continuously changing images, and realize the acquisition of the intermediate deformation process from the initial image to the final image without a large amount of calculation.
2) The device for determining the dynamic image deformation process comprises a first obtaining unit, a first determining unit, a dividing unit, a calculating unit and a second determining unit, wherein the first obtaining unit obtains an initial image and a tail image, the first determining unit determines a first deformation field representing deformation from the initial image to the tail image, the dividing unit divides the first deformation field to obtain a plurality of sub deformation fields, the calculating unit performs deformation calculation on the initial image according to the sub deformation fields to obtain a plurality of intermediate images, accurate obtaining of the intermediate images is achieved, and the sub deformation fields in the scheme are obtained by dividing the first deformation field, so that the intermediate images obtained by applying the scheme can obtain information of newly added pixel points, and boundary problems caused by vanishing points are avoided. The method and the device avoid the operation of additionally registering a large number of images required for acquiring the intermediate images and the speed in the prior art, are suitable for continuously changing images, and realize the acquisition of the intermediate deformation process from the initial image to the final image without a large amount of calculation.
The above description is only a preferred embodiment of the present application and is not intended to limit the present application, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application shall be included in the protection scope of the present application.
Claims (10)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011063066.7A CN112348735A (en) | 2020-09-30 | 2020-09-30 | Method and device for determining dynamic image deformation process and processor |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011063066.7A CN112348735A (en) | 2020-09-30 | 2020-09-30 | Method and device for determining dynamic image deformation process and processor |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112348735A true CN112348735A (en) | 2021-02-09 |
Family
ID=74361468
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011063066.7A Pending CN112348735A (en) | 2020-09-30 | 2020-09-30 | Method and device for determining dynamic image deformation process and processor |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112348735A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114359356A (en) * | 2021-12-28 | 2022-04-15 | 上海联影智能医疗科技有限公司 | Training method of image registration model, image registration method, device and medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050190189A1 (en) * | 2004-02-25 | 2005-09-01 | Christophe Chefd'hotel | System and method for GPU-based 3D nonrigid registration |
| US20140010422A1 (en) * | 2012-07-09 | 2014-01-09 | Jim Piper | Image registration |
| US20150043794A1 (en) * | 2012-03-15 | 2015-02-12 | Koninklijke Philips N.V. | Multi-modality deformable registration |
| CN107633478A (en) * | 2016-07-19 | 2018-01-26 | 佳能株式会社 | Image processing apparatus, image processing method and computer-readable medium |
| CN110580691A (en) * | 2019-09-09 | 2019-12-17 | 京东方科技集团股份有限公司 | dynamic processing method, device and equipment of image and computer readable storage medium |
| CN111721225A (en) * | 2020-06-23 | 2020-09-29 | 清华大学 | A kind of dynamic measurement method and device for temperature deformation under high temperature environment |
-
2020
- 2020-09-30 CN CN202011063066.7A patent/CN112348735A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050190189A1 (en) * | 2004-02-25 | 2005-09-01 | Christophe Chefd'hotel | System and method for GPU-based 3D nonrigid registration |
| US20150043794A1 (en) * | 2012-03-15 | 2015-02-12 | Koninklijke Philips N.V. | Multi-modality deformable registration |
| US20140010422A1 (en) * | 2012-07-09 | 2014-01-09 | Jim Piper | Image registration |
| CN107633478A (en) * | 2016-07-19 | 2018-01-26 | 佳能株式会社 | Image processing apparatus, image processing method and computer-readable medium |
| CN110580691A (en) * | 2019-09-09 | 2019-12-17 | 京东方科技集团股份有限公司 | dynamic processing method, device and equipment of image and computer readable storage medium |
| CN111721225A (en) * | 2020-06-23 | 2020-09-29 | 清华大学 | A kind of dynamic measurement method and device for temperature deformation under high temperature environment |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114359356A (en) * | 2021-12-28 | 2022-04-15 | 上海联影智能医疗科技有限公司 | Training method of image registration model, image registration method, device and medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP7005669B2 (en) | Methods and systems for performing convolutional image transformation estimates | |
| CN108230367A (en) | A kind of quick method for tracking and positioning to set objective in greyscale video | |
| WO2016030305A1 (en) | Method and device for registering an image to a model | |
| CN113610752B (en) | Breast image registration method, computer device and storage medium | |
| CN114332156B (en) | Real-time three-dimensional motion completion method based on graph convolution neural network | |
| CN112734887A (en) | Face mixing-deformation generation method and device based on deep learning | |
| CN115564639A (en) | Background blurring method and device, computer equipment and storage medium | |
| CN112907733A (en) | Method and device for reconstructing three-dimensional model and three-dimensional model acquisition and reconstruction system | |
| CN112348735A (en) | Method and device for determining dynamic image deformation process and processor | |
| CN110830848B (en) | Image interpolation method, image interpolation device, computer equipment and storage medium | |
| US20230104702A1 (en) | Transformer-based shape models | |
| Ren et al. | Shape recovery of endoscopic videos by shape from shading using mesh regularization | |
| CN114549594A (en) | Image registration method and device and electronic equipment | |
| US20140032180A1 (en) | Method and apparatus for computing deformation of an object | |
| Kapusta et al. | Acceleration of image reconstruction in 3D Electrical Capacitance Tomography in heterogeneous, multi-GPU system using sparse matrix computations and Finite Element Method | |
| CN115953440A (en) | Medical image registration method and device, storage medium and electronic equipment | |
| CN110728746B (en) | Modeling method and system for dynamic texture | |
| CN115880378A (en) | Method and device for determining color information in radiation field | |
| CN108961161A (en) | A kind of image processing method, device and computer storage medium | |
| KR20240018722A (en) | A method of calculating a brain blood flow data and a training method of a neural network for calculating a brain blood flow data | |
| Han et al. | Hierarchical geodesic polynomial model for multilevel analysis of longitudinal shape | |
| CN118350987B (en) | Image registration method and device, storage medium and electronic equipment | |
| Fishbaugh et al. | Spatiotemporal modeling for image time series with appearance change: Application to early brain development | |
| Świta et al. | Thermal image approximation using B-spline surfaces | |
| CN118379383B (en) | Magnetic resonance image reconstruction method and device based on Fourier convolution |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB03 | Change of inventor or designer information | ||
| CB03 | Change of inventor or designer information |
Inventor after: Feng Hui Inventor after: Zhang Chao Inventor after: Mao Yijin Inventor after: Zhao Qinghua Inventor after: Liu Wei Inventor before: Feng Hui Inventor before: Zhang Chao Inventor before: Mao Yijin Inventor before: Zhao Qinghua Inventor before: Liu Yuqi |
|
| AD01 | Patent right deemed abandoned |
Effective date of abandoning: 20250718 |