JP7530184B2 - IMAGE PROCESSING APPARATUS, IMAGE PROCESSING METHOD, IMAGE PROCESSING SYSTEM, AND PROGRAM - Google Patents

Description

The present invention relates to an image processing device, an image processing method, an image processing system, and a program.

In the medical field, attempts have been made to visualize changes over time in lesions, etc. by presenting a user with a difference image generated from two images taken at different times using various modalities.

Non-Patent Document 1 discloses a technique for displaying a difference image generated from two three-dimensional images captured by a CT device as a two-dimensional tomographic image. It also discloses a technique for generating and displaying a two-dimensional projection image in which the pixel values of the difference image are projected in a direction parallel to the slice plane (in CT, this is generally a direction perpendicular to the subject's body axis).

R. Sakamoto, et. al, Temporal subtraction system for detecting bone metastasis using LDDMM: preliminary study, CARS2014.

However, in the technology of Non-Patent Document 1, when a difference image is generated from two 3D images (original images) with a coarse slice interval, the slice interval of the difference image also becomes coarse. Therefore, when a projection image in which the pixel values of the difference image are projected in a direction parallel to the slice plane is displayed on a display unit, a problem may arise in that visibility is reduced.

In view of the above problems, the present invention aims to provide an image processing technique capable of generating a highly visible difference image.

In order to achieve the object of the present invention, an image processing device according to one aspect of the present invention is an image processing device that generates a difference image to be output to a display unit from a first medical image and a second medical image, which are three-dimensional images constituted by a plurality of slices obtained by imaging a subject, and
an acquisition means for acquiring the first medical image and the second medical image;
a determination means for determining a slice interval of the difference image generated by dividing the three-dimensional image in a slice direction of the subject, the slice interval being determined such that a slice of one of the first medical image and the second medical image serving as a reference is included in the difference image , and the slice interval of the slice in the difference image is smaller than the slice interval of the one of the medical images;
a generating means for generating the difference image having the slice interval determined by the determining means, and generating at least one slice corresponding to a slice of the one medical image from the difference image as a derived difference image;
The image processing method further comprises: storing means for storing the difference image and the derived difference image in different storage units.
In order to achieve the object of the present invention, an image processing method according to another aspect of the present invention is an image processing method of an image processing device for generating a difference image to be output to a display unit from a first medical image and a second medical image, the first medical image and the second medical image being three-dimensional images constituted by a plurality of slices obtained by imaging a subject, the method comprising the steps of:
An acquisition step in which an acquisition means acquires the first medical image and the second medical image;
a determining step in which a determining means determines a slice interval of the difference image generated by dividing the three-dimensional image in a slice direction of the subject, the slice interval being determined such that a slice of one of the first medical image and the second medical image serving as a reference is included in the difference image , and the slice interval of the slice in the difference image is smaller than the slice interval of the one of the medical images;
a generating step in which a generating means generates the difference image having the slice interval determined in the determining step, and generates at least one slice corresponding to a slice of the one medical image from the difference image as a derived difference image;
The storage means has a storage step of storing the difference image and the derived difference image in different storage units.

According to the present invention, it is possible to generate a difference image with high visibility. In other words, the difference image can be displayed cross-sectionally in the same slice as the original image, and a highly visible projection image can be displayed even when the pixel values of the difference image are projected parallel to the slice plane.

The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the invention and, together with the description, serve to explain the principles of the invention.
FIG. 1 is a diagram showing the functional configuration of an image processing system according to an embodiment. FIG. 2 is a diagram showing an overall processing procedure according to the embodiment. FIG. 4 is a diagram for explaining slice intervals of a difference image. FIG. 4 is a diagram illustrating an axial cross section of an original image and a differential image. 1A and 1B are diagrams illustrating coronal sections of an original image and a differential projection image. FIG. 4 is a diagram illustrating an example of setting of coordinate axes in the embodiment.

The following describes in detail the embodiments of the present invention with reference to the attached drawings. Note that the following embodiments do not limit the scope of the present invention, and not all of the combinations of features described in the present embodiments are necessarily essential to the solution of the present invention.

[First embodiment]
(Image Processing System Configuration)
1 is a diagram showing a configuration of an image processing system 10 according to an embodiment. The image processing system 10 has an image processing device 100, which is connected to a data server 130 via a network 120. The image processing device 100 according to the embodiment is a device capable of performing registration between two images captured at different times and generating a difference image from two images (a first medical image and a second medical image) that are three-dimensional images obtained by capturing images of a subject.

The data server 130 holds a plurality of medical images. The data server 130 represents, for example, a medical image management system (PACS: Picture Archiving and Communication Systems) that receives medical image data captured by a modality and stores and manages the data through a network. In the following description, it is assumed that the data server 130 holds a plurality of three-dimensional tomographic images obtained by capturing images of a subject in advance under different conditions (different modalities, capture modes, dates, times, body positions, etc.) as the first medical image and the second medical image. In this embodiment, as an example of a medical image, the first medical image and the second medical image are described as three-dimensional tomographic images (three-dimensional medical images) captured by an X-ray CT scanner.

Figure 6 is a diagram illustrating the setting of coordinate axes in an embodiment. As shown in Figure 6, the axis representing the direction from the right hand to the left hand of the subject 600 is defined as the X axis, the axis representing the direction from the back to the front of the subject 600 is defined as the Y axis, and the axis representing the direction from the head to the feet of the subject is defined as the Z axis. The XY section is defined as the axial plane, the YZ section as the sagittal plane, and the ZX section as the coronal plane. That is, the X axis direction is the direction perpendicular to the sagittal plane (hereinafter, "sagittal direction"). The Y axis direction is the direction perpendicular to the coronal plane (hereinafter, "coronal direction"). Furthermore, the Z axis direction is the direction perpendicular to the axial plane (hereinafter, "axial direction"). In this case, in the case of a CT image, which is a three-dimensional image composed of a collection of two-dimensional tomographic images (slices), the slice plane of the image represents the axial plane, and the direction perpendicular to the slice plane (hereinafter, "slice direction") represents the axial direction. Note that the coordinate system setting shown in FIG. 6 is just one example, and it is possible to set the coordinate system using other definitions.

The modality for capturing the three-dimensional tomographic image may be, for example, an MRI device, an X-ray CT device, a three-dimensional ultrasound imaging device, a photoacoustic tomography device, a PET/SPECT device, an OCT device, or the like. Furthermore, the first medical image and the second medical image may be any images as long as they are three-dimensional tomographic images for which a difference image is to be generated. For example, they may be images captured at the same time using different modalities or different imaging modes. Furthermore, they may be images captured of the same patient using the same modality and in the same position on different dates and times for follow-up observation. Note that the first medical image and the second medical image are three-dimensional medical images (three-dimensional tomographic images) configured as a collection of two-dimensional tomographic images. The position and orientation of each two-dimensional tomographic image are converted into a reference coordinate system (a coordinate system in space based on the subject) and then stored in the data server 130. At this time, the first medical image and the second medical image expressed in the reference coordinate system are input to the image processing device 100 in response to instructions from a user who operates the instruction unit 140.

The image processing device 100 is a device that receives a processing request from a user via the instruction unit 140, performs image processing, and outputs the results of the image processing to the display unit 150, and functions as a terminal device for interpretation operated by a user such as a doctor. Specifically, based on instructions from the user via the instruction unit 140, the image processing device 100 acquires a first medical image and a second medical image to be subjected to image processing from the data server 130 as a pair of images (image pair) to be subjected to image processing. The image processing device 100 then performs alignment processing of the acquired first medical image and second medical image, generates a difference image or the like, and outputs it to the display unit 150.

The image processing device 100 is composed of the components described below. The functions of each of the components below are realized, for example, by one or more CPUs (central processing units) functioning as the control unit of the image processing device 100 executing programs. The components of the image processing device 100 may be composed of integrated circuits or the like as long as they perform similar functions.

The acquisition unit 101 functions as an interface that realizes communication between an external device (e.g., a data server 130, etc.) and the image processing device 100 via the network 120, and acquires information on a first medical image and a second medical image input to the image processing device 100 via the network 120. When a user instruction is input from the instruction unit 140, the acquisition unit 101 acquires, as an image pair, the first medical image and the second medical image to be subjected to image processing from the data server 130.

The conversion unit 102 converts the resolution of the first medical image and the second medical image to a predetermined resolution (hereinafter, "processing resolution"). The alignment unit 103 performs alignment processing between the first medical image and the second medical image after resolution conversion (hereinafter, "first converted image" and "second converted image"), and obtains a displacement field that associates the positions of the images. Note that the image resolution here represents the pixel size in each of the X, Y and Z axis directions.

The determination unit 104 determines the resolution in each of the X, Y and Z axial directions of the difference image to be output (hereinafter, "output resolution"). The slice interval of the difference image (i.e., the output resolution in the Z direction of the difference image) is determined to be an interval such that, within the range of the difference image in the slice direction of the three-dimensional medical image, slices corresponding to slices of either one of the first medical image and the second medical image serving as a reference are included in the difference image, and the interval is equal to or less than the slice interval of either one of the first medical image and the second medical image serving as a reference.

The difference image generating unit 105 generates a difference image at the output resolution determined by the determining unit 104. Specifically, the difference image generating unit 105 generates a difference image between an image (hereinafter, "second deformed image") obtained by deforming the second medical image to match the first medical image based on the displacement field acquired by the alignment unit 103 and the first medical image based on the output resolution. Note that the image deformed based on the displacement field acquired by the alignment unit 103 is not limited to the second medical image, and the first medical image may be deformed. For example, the difference image generating unit 105 can also generate a difference image between an image (first deformed image) obtained by deforming the first medical image to match the second medical image based on the displacement field acquired by the alignment unit 103 and the second medical image based on the output resolution.

The projection image generating unit 106 generates a projection image (hereinafter, "difference projection image") by projecting pixel values of the difference image in two dimensions. The display control unit 107 controls the output of cross-sectional images of the first medical image and the second medical image, cross-sectional images of the generated difference image, difference projection images, etc. to the display unit 150.

The display unit 150 is composed of any device such as an LCD or CRT, and displays medical images and the like for a doctor to interpret. Specifically, it displays cross-sectional images of the first medical image and the second medical image acquired from the image processing device 100. It also displays cross-sectional images of the difference image and the projection image (difference projection image) generated by the image processing device 100. Here, the display unit 150 is provided with a GUI for acquiring instructions from a user such as a doctor. The user can freely switch the cross-sectional images of the first medical image and the second medical image being displayed to the cross-sectional image of the difference image or the difference projection image using the GUI. In addition, the display control unit 107 can also control the display unit 150 to combine and display each image based on an instruction from the GUI.

Figure 2 is a flowchart showing the overall processing procedure performed by the image processing device 100.

(S201: Acquire image)
In step S201, the acquisition unit 101 acquires the first medical image and the second medical image designated by the user through the instruction unit 140 from the data server 130. Then, the first medical image and the second medical image are acquired as a pair of images to be subjected to image processing, and output to the conversion unit 102, the determination unit 104, the difference image generation unit 105, and the display control unit 107. The first medical image and the second medical image automatically acquired based on a predetermined rule may be confirmed by the user and finally acquired. Here, as an example of the process of automatically acquiring images based on a predetermined rule, for example, the first medical image may be automatically acquired as the latest examination image of the subject to be examined, and the second medical image may be automatically acquired as the second latest examination image of the same subject. The method of automatic acquisition is not limited to this, and for example, the second medical image may be automatically acquired as the oldest examination image of the same subject. Also, a configuration may be adopted in which only the first medical image is designated by the user, and the second medical image is automatically acquired based on a predetermined rule in the same manner as described above.

(S202: Convert image resolution)
In step S202, the conversion unit 102 converts the resolution of the first medical image and the second medical image acquired from the acquisition unit 101 to a predetermined processing resolution, and acquires a first converted image and a second converted image. For example, when the resolution (pixel size) of the first medical image and the second medical image that are the original images of the difference image is anisotropic, in order to perform the alignment between the images in the subsequent processing with high accuracy, the conversion unit 102 acquires the first converted image and the second converted image in which the resolution is made isotropic so that the resolution in each axial direction is equal. More specifically, for example, when the first medical image and the second medical image are CT images, the resolution in the slice plane (for example, the sagittal direction, the coronal direction) is higher than the resolution in the slice direction (axial direction). Therefore, the conversion unit 102 performs a process of upsampling the pixels in the slice direction to match the resolution in the slice plane. Note that a known image processing method can be used for the interpolation of density values during resolution conversion.

In this embodiment, the processing resolution is set to, for example, 1 mm in each three-dimensional axis direction so that the difference in details between the images can be calculated, and a first converted image and a second converted image whose resolution has been made isotropic by resolution conversion are acquired. The processing resolution does not have to be 1 mm as long as the difference in details between the images can be calculated sufficiently. Also, it is not necessary to make the images isotropic, and it is sufficient if the resolution can be converted to a level that allows the alignment between the images to be performed with high accuracy in the subsequent processing. Furthermore, if the resolution conversion process is not required, such as when the resolution of the first medical image and the second medical image at the stage of acquisition in step S201 is already the processing resolution, resolution conversion is not necessarily required. Then, the conversion unit 102 outputs the generated first converted image and second converted image to the alignment unit 103.

(S203: Align the two images)
In step S203, the registration unit 103 performs registration so that pixels representing the same part between the first converted image and the second converted image substantially coincide with each other, and obtains a displacement field that associates the positions between the images. Then, the registration unit 103 outputs information on the displacement field that is the result of the registration to the difference image generating unit 105.

In this embodiment, the alignment between images refers to a process of calculating a displacement field for displacing each pixel position of one image to the corresponding pixel position of the other image. For example, when there are two images captured at different times, the displacement field from one image to the other image can be calculated by estimating the corresponding pixel position of the other image for each pixel position of the one image used as a reference. In this embodiment, when the first medical image is used as a reference, the displacement vector at each pixel position is obtained by determining the pixel position on the second converted image corresponding to each pixel position on the first converted image. In other words, when the first medical image is used as a reference, the generated displacement field is an image that stores the displacement vector to the corresponding position on the second converted image at each pixel position of the first converted image, and has the same resolution and the same number of pixels as the first converted image used as a reference. Therefore, when the processing resolution is, for example, 1 mm isotropic, the displacement field is also a displacement vector field that is isotropized at 1 mm. As a result, the displacement field holds an amount of information that allows the difference in details between images to be calculated.

In this embodiment, the alignment unit 103 can perform alignment by a known image processing method. For example, alignment can be performed by obtaining a displacement field by deforming one image so that the image similarity between the images is high. As the image similarity, known methods such as the commonly used Sum of Squared Difference (SSD), mutual information, and cross-correlation coefficient can be used. In addition, as a model of image deformation, a deformation model based on a radial basis function such as Thin Plate Spline (TPS), Free Form Deformation (FFD), Large Deformation Diffeomorphic Metric Mapping (LDDMM), and other known deformation models can be used.

(S204: Determine output resolution)
In step S204, the determination unit 104 determines the output resolution based on the resolution of the first medical image and the processing resolution used in step S202. Then, the determination unit 104 outputs the determined value of the output resolution to the difference image generation unit 105.

Here, the details of the process in which the determination unit 104 determines the output resolution will be described. A common method for generating a difference image between two images is to generate a difference image in such a way that the medical image (the first medical image in this embodiment) that is the reference for alignment among the two images (the first medical image and the second medical image) and the difference image are associated with each other in position for each pixel. In this case, the difference image is generated with the same resolution as the medical image (the first medical image in this embodiment) that is the reference for alignment. In addition, within the range in which the difference image is generated, the difference image is generated with the same number of pixels as the image (the first medical image) that is the reference for alignment. This allows, for example, when it is desired to confirm the positions of the two-dimensional tomographic image included in the first medical image and the two-dimensional tomographic image included in the difference image in association with each other, to easily display them in association with each other.

However, when generating and displaying a differential projection image or a volume rendering image in which a differential image is two-dimensionally projected in a direction parallel to the slice plane, or a cross-sectional image (hereinafter, a differential projection image, etc.) in which a differential image is cut at any cross-section, including a sagittal plane or a coronal plane, different from the slice plane (axial plane), the following problem may occur. That is, if the slice interval of the first medical image serving as a reference is coarse, the slice interval of the differential image will also be coarse. As a result, a problem may occur in that, although the resolution is sufficient when a cross-sectional image of the slice plane of the differential image is displayed on a display unit, when the above-mentioned differential projection image, etc. is displayed on a display unit, the resolution is displayed at a coarse resolution in the slice direction. This problem may occur because, when three-dimensional tomographic images are acquired by various modalities, the resolution is often not necessarily the same in all axial directions. For example, in the case of an X-ray CT device, the resolution (slice interval) in the slice direction (axial direction) is often coarser than the resolution (slice in-plane resolution) in the slice in-plane direction (for example, the sagittal direction or coronal direction) in order to reduce the amount of radiation exposure and the amount of data.

Therefore, in this embodiment, the resolution in the slice plane (XY plane) of the output resolution is matched to the resolution in the slice plane of the first medical image. Then, the slice interval (value in the Z direction) of the output resolution is determined to be a value such that the slices corresponding to the slices of the first medical image are included in the difference image and the slice interval of the difference image is not coarse compared to the set predetermined resolution. That is, the determination unit 104 corresponds to an example of a determination means that determines, as the slice interval of the difference image, an interval in which the slices corresponding to the slices of either one of the first medical image and the second medical image serving as a reference are included in the difference image and which is equal to or less than the slice interval of either one of the first medical image and the second medical image serving as a reference. At this time, the predetermined resolution is a resolution at which the difference between the fine details of the images can be calculated, and is set as the upper limit value of the Z direction value (slice interval) of the output resolution (hereinafter, "upper limit resolution"). A specific setting method is described below.

In this embodiment, the determination unit 104 sets the Z-direction value of the output resolution to a value that satisfies the following two conditions:

<Condition 1> The slice interval of the first medical image must be divided by a natural number. <Condition 2> The maximum value equal to or less than the upper limit resolution. Here, if the Z-direction value of the output resolution is _sz , the slice interval of the first medical image is s1, the natural number is n, and the upper limit resolution is _sMAX , the following formula (1A) is established for the Z-direction value of the output resolution _sZ . According to formula (1A), the determination unit 104 obtains a value obtained by dividing the slice interval (s1) of the reference medical image (in this embodiment, the first medical image) by the natural number (n) as the slice interval of the difference image. Furthermore, the determination unit 104 determines the slice interval of the difference image with the set resolution (upper limit resolution _sMAX ) as the upper limit.

s _Z = s1/n ≦ s _MAX ...(1A)
Furthermore, from formula (1A), the following formula (1B) holds:

n ≧ s1/s _MAX ...(1B)
Here, in order to make the Z-direction value _sZ of the output resolution the maximum value that satisfies the formula (1A), the natural number n must be the minimum value that satisfies the formula (1B). If the slice interval s1 and the upper limit resolution _sMAX of the medical image are known, the natural number n can be calculated. For example, let s1 = 3.5 mm and _sMAX = 1.0 mm (= processing resolution). Here, the upper limit resolution _sMAX = The reason for using the processing resolution will be described later. In this case, equation (1B) is n≧3.5, and the smallest natural number n=4 that satisfies equation (1B) is found. Therefore, from equation (1A), the output resolution is The value of the Z direction can be calculated as s _Z =3.5/4=0.875 mm.

FIG. 3 is a diagram for explaining the Z-direction value of the output resolution (slice interval of the difference image). In FIG. 3, a three-dimensional tomographic image 300 represents a difference image. The thick line interval 301 in the Z direction represents the slice interval of the first medical image, and the dotted line interval 302 represents the Z-direction value _sZ of the output resolution. As described above, when the thick line interval 301 is s1=3.5 mm, the dotted line interval 302 is 0.875 mm, which is a value obtained by dividing the thick line interval 301 by n=4. At this time, in the slice direction (Z-axis direction in FIG. 3) of the difference image (three-dimensional tomographic image 300), the slice interval at the position of the thick line interval 301 is the same as that of the first medical image, that is, the slice corresponding to the slice of the first medical image is included in the difference image. At the position of the dotted line interval 302 other than the thick line interval 301, a finer slice interval (Z-direction value _sZ of the output resolution) is set compared to the upper limit resolution _sMAX .

In this way, since the difference image includes slices corresponding to slices of the reference medical image (the first medical image in this embodiment), when displaying a cross section, the first medical image and the difference image can be easily associated and observed on a slice-by-slice basis. Furthermore, since the slice interval of the difference image is not coarser than the upper limit value (upper limit resolution s _MAX ), even if the slice interval of the original image is coarse, it is possible to generate an image (such as a difference projection image) with high visibility having fine resolution in the slice direction (Z-direction value s _Z of the output resolution)

Here, the value of the processing resolution can be used as the upper limit resolution. In other words, the upper limit of the Z-direction value of the output resolution is set as the processing resolution. By matching the upper limit of the Z-direction value of the output resolution (upper limit resolution) to the processing resolution, as described in step S203, the upper limit resolution also matches the resolution of the displacement field obtained by the alignment process. As described in step S203, the displacement field is generated at a resolution (e.g., 1 mm) at which the fine differences between images can be calculated. Therefore, the upper limit of the Z-direction output resolution is not simply set so that the resolution in the body axis direction of the differential projection image does not become coarse, but is also set as a value that can express the fine differences between images in terms of the amount of information.

Note that the value used as the upper limit resolution is an example and is not limited to the above example. In other words, the upper limit resolution does not have to match the value of the processing resolution, and the upper limit resolution may be a value obtained by multiplying the processing resolution by a predetermined constant. For example, an example of the predetermined constant may be 1.5. Here, the predetermined constant can be empirically set to a value that displays the differential projection image, etc. with sufficient resolution. By setting the predetermined constant to 1 or more, the generation of the differential image can be accelerated. Alternatively, a predetermined constant may be set as the upper limit resolution. The upper limit resolution may be set to a predetermined resolution in the range of 0.5 mm or more and 2.0 mm or less, which is a resolution that is not unnecessarily fine and can depict sufficiently fine differential values.

In the calculation of the Z-direction value _sZ of the output resolution, a predetermined resolution is set as an upper limit value, and the value is always set to be equal to or less than that value, but such a setting is not necessarily required. That is, the predetermined resolution may be set as a reference value (hereinafter, "reference resolution"), and a value that satisfies a predetermined condition with respect to the reference resolution may be set as the Z-direction resolution. That is, the above upper limit resolution is an example of the reference resolution, and <Condition 2> is an example of a predetermined condition. For example, a value closest to the reference resolution may be set as the Z-direction resolution. That is, as a condition to be combined with <Condition 1>, it is also possible to set a value that satisfies the following <Condition 2'> instead of the above-mentioned <Condition 2>.

<Condition 2'> Value closest to the reference resolution In this case, if the reference resolution is _sB , the following formulas (2) to (5) hold for <Condition 2'>. According to formulas (2) to (5), the determination unit 104 sets the set resolution as the reference resolution ( _sB ) and determines the slice interval of the difference image based on a value ( _dMIN ) close to the reference resolution. The determination unit 104 compares a first value ( _d- ) acquired based on the difference between the value (s1/ _n- ) obtained by dividing the interval between slices of the reference medical image by a first natural number (n-) and the reference resolution ( _sB ) with a second value (d ₊ ) acquired based on the difference between the value (s1/n ₊ ) obtained by dividing the interval between slices of the reference medical image by a second natural number (n ₊ ) and the reference resolution ( _sB ) _, and sets the value with the smaller absolute value as the close value ( _dMIN ). Here, the first natural number ( _n- ) is a natural number that gives a maximum value equal to or less than the reference resolution when the interval between slices of the reference medical image is divided by the first natural number ( _n- ), and the second natural number (n ₊ ) is a natural number that gives a minimum value equal to or more than the reference resolution when the interval between slices of the reference medical image is divided by the second natural number (n ₊ ).

s _Z = s _B +d _MIN ...(2)
d _MIN = {d _- (if | d _- | ≦ | d ₊ |)
d ₊ (if | d _- | > | d ₊ |)}...(3)
d _- = s1/n _- - s _B ... (4)
d ₊ = s1/n ₊ - s _B ...(5)
Here, in formula (4), _n- is a natural number (first natural number) that gives the maximum value below the reference resolution when s1 is divided by itself, and in formula (5), n ₊ is the value obtained by dividing s1 by itself. d _- is the difference between the maximum value below the reference resolution and the reference resolution, and d ₊ is the difference between the maximum value below the reference resolution and the reference resolution. In addition, in formula (3), d _MIN is set to the smaller absolute value by comparing the absolute value of d _- with the absolute value of d ₊ . do.

For example, when s1=2.25 mm and _sB =1.0 mm, from equations (2) to (5), n- ₌ 3, n ₊ =2, _d- =0.25, and d ₊ =0.125. In this case, the _dMIN value in equation (3) is set to the smaller absolute value of the absolute values of _d- and d ₊ , resulting in _dMIN =0.125. When the reference resolution is _sB =1.0 mm, from equation (2), the Z-direction value of the output resolution ( _sZ ) is _sZ = _sB + _dMIN =1.0+0.125=1.125.

On the other hand, when the parameters are set to satisfy the above-mentioned <Condition 1> and <Condition 2>, in this case, the formula (1B) becomes n≧2.25, and the smallest natural number n=3 that satisfies the formula (1B) is obtained. Therefore, from the formula (1A), the Z-direction value of the output resolution can be obtained as _sZ =2.25/3=0.75 mm. The Z-direction value ( _{sZ )} of the output resolution that satisfies <Condition 1> and <Condition 2> is set to _sZ =0.75. In this case, the difference from the standard resolution _sB (=1.0) is 0.25, and compared to the difference from the standard resolution sB (0.125) when <Condition 2'> is used, when <Condition 2> is used, a value farther from the standard resolution _sB is set.

In this way, by treating the specified resolution as a reference value and setting the value closest to the reference value, it is possible to set a value closer to the specified resolution, compared to treating the specified resolution as the upper limit of the Z-direction resolution.

This is useful in the following cases, for example. As described above, the amount of information possessed by the displacement field due to the alignment is, for example, the amount of information possessed by the 1 mm isotropic resolution, which is the processing resolution. Therefore, when <Condition 2> is used, the Z-direction value (s _Z ) of the output resolution does not exceed the processing resolution, so that even if the slice interval of the original image is coarse, it is not set to a coarse value. However, in some cases, as in the above case, it may be set to a fine value (0.75 mm) far from 1 mm, and may be set to a value finer than the amount of information possessed by the displacement field. In such a case, by using <Condition 2'>, the slice interval of the difference image is determined to a value in the vicinity of the set resolution (reference resolution), that is, by setting the value closest to the processing resolution as the slice interval (output resolution value) of the difference image, it is possible to set the value of the output resolution (s _Z ) to a value (1.125 mm) that is coarser than the amount of information possessed by the displacement field, but closer to the processing resolution.

(S205: Generate differential image)
In step S205, the difference image generating unit 105 performs difference processing at the output resolution determined in step S204 between the first medical image and the second medical image to generate a difference image at the output resolution. The generated difference image is then output to the projection image generating unit 106 and the display control unit 107. A specific method for generating a difference image will be described below.

Generally, the generation of a difference image based on the difference processing between the first medical image and the second medical image is performed by the following method. That is, based on the displacement field acquired in step S203, the difference image generating unit 105 calculates, for each pixel on the first medical image, the position (coordinates) of the pixel on the second medical image that corresponds to the position (coordinates) of the pixel. Then, for each pixel on the first medical image, a difference image is generated by taking the difference value of the pixel value between the position (coordinates) of the corresponding pixel on the second medical image. As a result, the position (coordinates) of each pixel in the difference image corresponds to the first medical image, so that the difference image can be acquired at the same resolution as the first medical image.

In the difference processing of this embodiment, since the output resolution of the difference image does not necessarily match the resolution of the first medical image, the difference image generating unit 105 first converts the resolution of the first medical image to the output resolution, and then performs the same difference processing as described above to generate a difference image. The difference image generating unit 105 converts the resolution of the first medical image described in the difference processing method above to the output resolution, and then performs processing to generate a difference image based on the first medical image after the resolution conversion.

That is, for each pixel on the first medical image after resolution conversion, the difference image generating unit 105 calculates the position (coordinates) of the pixel on the second medical image that corresponds to the position (coordinates) of the pixel. Then, for each pixel on the first medical image after resolution conversion, a difference image is generated by taking the difference in pixel value between the position (coordinates) of the corresponding pixel on the second medical image. As a result, the position (coordinates) of each pixel in the difference image corresponds to the first medical image after conversion to the output resolution, so that the difference image can be obtained at the output resolution.

Note that the method of generating the difference image in this embodiment is not limited to this. For example, a method may be used in which the first medical image is not converted to the output resolution first. Specifically, an empty area for a difference image having the output resolution is first prepared. Next, for each pixel position (coordinate) in that area, a difference image may be generated by obtaining the difference in pixel value between the position (coordinate) on the first medical image to which the position is originally associated and the position (coordinate) on the second medical image to which the position is associated using the displacement field.

As described in FIG. 3, the difference image generated at the output resolution contains slices that correspond to slices of the first medical image at the same slice interval as the first medical image, represented by the interval 301 between the thick lines. Therefore, by extracting slices that correspond to the slices of the first medical image from the difference image, it is possible to display the slices of the first medical image and the slices of the difference image in correspondence with each other when displayed in step S208 described below. This can be achieved, for example, by sequentially acquiring slices of the difference image at the same slice interval as the first medical image.

In addition, instead of displaying the slices in units of slices, it is also possible to display the slices in correspondence by corresponding the first medical image and the subtraction image in the coordinate system of the physical space.

In addition, the method of making the slices of the first medical image and the slices of the difference image correspond to each other is not limited to the above method. For example, by adding correspondence information such as "which slice of the first medical image corresponds to which slice of the difference image" to the difference image, it is possible to display the slices of both images in a corresponding manner. That is, the difference image generating unit 105 adds correspondence information that makes the slices of the first medical image and the slices of the difference image correspond to each other to the difference image. Note that there may be cases where there is no slice of the first medical image that corresponds to a slice of the difference image, such as when the slice interval of the difference image is finer than the slice interval of the first medical image. In that case, it is not necessary to add correspondence information to a difference image for which there is no corresponding slice, or the closest slice of the first medical image may be added as correspondence information.

Figure 4 is a diagram illustrating axial sections of the original images (first medical image, second medical image) and the difference image. In Figure 4, axial section 400 of the first medical image, axial section 401 of the second medical image, and axial section 402 of the difference image are shown. Furthermore, region 403 on axial section 400 of the first medical image represents an abnormal area, and region 404 on axial section 402 of the difference image represents a difference area corresponding to region 403 (abnormal area).

The region 403 (abnormal region) is present in the first medical image, but the region of the second medical image corresponding to the region 403 (abnormal region) does not have an abnormal region. The difference region corresponding to the region 403 (abnormal region) is depicted as a difference value of pixel values in corresponding regions between the first medical image and the second medical image. At this time, the difference image and the first medical image are associated on a slice-by-slice basis. Therefore, when the original region 403 (abnormal region) is specified in the axial cross section 400 of the first medical image, the position of the corresponding region 404 (difference region) can be identified, and the corresponding positions between images can be easily displayed, such as by displaying the axial cross section 402 of the difference image including the identified region 404 (difference region).

The difference image is generated only in the area common to the first and second medical images as a result of aligning the two, and therefore there are in fact areas of the first medical image that are outside the range of the difference image. In other words, to express the above content strictly, the difference image and the first medical image correspond on a pixel-by-pixel basis only within the range of the difference image. Therefore, it goes without saying that the above-mentioned images can be easily merged and the corresponding positions between the images can be displayed within the range of the difference image.

In this embodiment, the generated difference image is stored in a storage unit (not shown). As a result, when it is desired to obtain the difference image again after the processing of the image processing device 100 is completed, the stored difference image can be read from the storage unit to easily obtain the difference image. However, it is not necessary to store the generated difference image in a storage unit (not shown).

(S206: Generate differential projection image)
In step S206, the projection image generating unit 106 generates a differential projection image by two-dimensionally projecting the pixel values of the differential image generated in step S205. Then, the generated differential projection image is output to the display control unit 107.

More specifically, the projection image generating unit 106 generates a projection image as a difference projection image by projecting the second difference image, which is a three-dimensional image, parallel to a slice plane (e.g., the xy plane in FIG. 6). When the original image of the difference image is an X-ray CT image, the slice direction coincides with the body axis direction (Z axis), so the difference projection image is an image projected in a direction perpendicular to the body axis direction (Z axis) of the subject 600 (FIG. 6) (e.g., the y direction in FIG. 6). For example, a difference projection image projected in the front direction (coronal direction) of the subject is generated as a direction perpendicular to the body axis direction (Z axis). This makes it easy to grasp the information of the difference image of the entire imaging area of the subject.

The projection image generating unit 106 can generate an image by, for example, calculating the average value of the maximum and minimum pixel values in the projection direction as a projection method for generating a difference projection image. Hereinafter, this is referred to as an MIP/MinIP image (maximum intensity projection/minimum intensity projection image). This allows a value that takes into account both the positive and negative difference values in the difference image to be reflected on the projection image. In addition, the projection method for generating a difference projection image is not limited to this method, and maximum intensity projection (MIP) or minimum intensity projection (MinIP) can also be used. At this time, the resolution of the difference projection image in the slice direction is the output resolution determined in step S204.

Figure 5 is a diagram illustrating an example of a coronal section image and a differential projection image of an original image. In Figure 5, a coronal section image 500 of an original image (first medical image) and a differential projection image 501 are shown. Furthermore, an area 502 on the coronal section image 500 of the first medical image represents an abnormal area, and an area 503 on the differential projection image 501 represents a differential projection area onto which the differential area corresponding to the area 502 (abnormal area) is projected.

Area 503 (difference projection area) corresponds to the area obtained by projecting the three-dimensional difference area including area 404 (difference area) on the slice (axial section 402 of the difference image) in Figure 4 in the coronal direction (e.g., the y direction in Figure 6).

In FIG. 5, the slice spacing of the original image (first medical image) is coarse (e.g., 5 mm), so the resolution of the coronal cross-sectional image 500 in the slice direction (Z-axis direction in FIG. 5) is coarse.

However, the resolution of the difference projection image 501 in the slice direction is the upper limit resolution (= processing resolution, for example 1 mm), and since it is generated as an image that retains the amount of information of the differences in the details of the three-dimensional image, it does not become coarse in the slice direction. In other words, since the slice interval of the original image (first medical image) is coarse, the display of the area 502 (abnormal area) on the coronal cross-sectional image 500 becomes coarse, but the area 503 (difference projection area) on the difference projection image 501 retains the amount of information of the differences in the details of the three-dimensional image, and is therefore depicted in detail.

In the above example, an example of generating a differential projection image projected in the front direction of the subject (e.g., y direction in FIG. 6) as a direction perpendicular to the body axis direction (Z axis) has been described, but the present invention is not limited to this example. It is also possible to generate multiple projection images by projecting while uniformly changing the projection direction so as to rotate around the body axis (e.g., z axis in FIG. 6) including this direction. It is also possible to generate (reconstruct) data in which a set of these projection images is combined into one. Here, for the sake of simplicity, the data of the set of individual projection images is also referred to as a differential projection image. For example, the projection image generating unit 106 generates, as a differential projection image, a set of data consisting of a total of 36 projection images divided around the body axis at 10° intervals. Note that the method of generating a differential projection image is not limited to this method, and may be a process of projecting pixel values of a differential image from any direction.

When the differential projection image is to be observed again after the processing of the image processing device 100 is completed, the projection image generating unit 106 reads the stored differential image from the storage unit, and then performs the processing of this step alone to generate a differential projection image, so that the differential projection image can be easily observed. For this reason, in this embodiment, it is not essential to store the generated differential projection image in a storage unit (not shown). Since it is sufficient that at least the differential image is stored in the storage unit, it is possible to prevent the storage capacity of the storage unit from increasing by further storing the differential projection image in the storage unit. In this case, it is sufficient to store the differential images together in the storage unit, which allows the storage capacity of the storage unit to be used efficiently and also makes data management easier. Note that the generated differential projection image may be stored in a storage unit (not shown), in which case it is possible to reduce the processing time required to generate the differential projection image when the differential projection image is to be observed again.

(S207: Display image)
In step S207, the display control unit 107 controls the display unit 150 to display the cross-sectional image of the difference image acquired from the difference image generating unit 105 and the difference projection image acquired from the projection image generating unit 106. The display control unit 107 also controls the display unit 150 to display the cross-sectional images of the first medical image and the second medical image acquired from the acquiring unit 101. In this manner, the processing of the image processing device 100 is performed.

According to this embodiment, the difference image can be displayed in cross section in the same slice as the original image, and when a projection image is generated by projecting the pixel values of the difference image parallel to the slice plane, a fine resolution can be generated in the slice direction even if the slice interval of the original image is coarse. As a result, when a projection image is generated by projecting the pixel values of the difference image parallel to the slice plane as in Non-Patent Document 1, a projection image with fine resolution in the slice direction can be generated even if the slice interval of the original image is coarse.

According to this embodiment, it is possible to generate a difference image with high visibility, and the difference image can be displayed cross-sectionally in the same slice as the original image, and even when the pixel values of the difference image are projected parallel to the slice plane, a projection image with high visibility can be displayed.

(Variation 1)
In the first embodiment, when the difference image is displayed, a slice corresponding to the first medical image is extracted from the difference image and displayed. However, it is not necessary to associate the slices of the first medical image with the slices of the difference image when displaying. For example, before displaying the difference image in step S207, a difference image (hereinafter, "derived difference image") may be generated in which only the slices corresponding to the first medical image are extracted. In this case, within the range of the difference image, the slices of the first medical image and the slices of the derived difference image are all in correspondence, so that it is not necessary to perform a process of extracting slices from the difference image every time the image is displayed in step S207. In addition, within the range of the difference image, all slices are in correspondence between the first medical image and the derived difference image, that is, the images are in correspondence with each other on a pixel-by-pixel basis, so that the images can be easily fused with each other. In this way, by generating a derived difference image, an image more suitable for cross-sectional display can be obtained.

In this modified example, when a derived difference image for cross-sectional display is acquired after the processing of the image processing device 100 is completed, the display control unit 107 reads the difference image saved in step S205 from the storage unit, and then performs the processing of this step alone to generate a derived difference image, thereby making it possible to easily acquire a derived difference image for cross-sectional display. For this reason, it is not essential to save the derived difference image generated in this modified example in a storage unit (not shown). Since it is sufficient that at least the difference image generated in step S205 is saved in the storage unit, it is possible to prevent the storage capacity of the storage unit from increasing by further saving the derived difference image for cross-sectional display in the storage unit. In this case, it is sufficient to save the difference images together in the storage unit, which allows efficient use of the storage capacity of the storage unit and also makes data management easier. The generated derived difference image may be saved in a storage unit (not shown), in which case it is possible to reduce the processing time required for generating the derived difference image.

(Variation 2)
In the first embodiment and the first modification, only the difference image generated at the output resolution is stored in a storage unit (not shown), and images derived from the difference image processed for display, such as the derived difference image and the difference projection image, are not stored. However, the storage method is not necessarily limited to this. For example, only the images processed for display, such as the derived difference image and the difference projection image, may be stored, and the difference image generated at the output resolution may not be stored. As a result, when it is desired to obtain a derived difference image or a difference projection image for cross-sectional display or projection display after the processing of the image processing device 100 is completed, the required image can be obtained by simply reading the stored derived difference image or the difference projection image from the storage unit without executing the processing of the first modification or the processing of step S206. In addition, although the stored data is divided into two, if the original first medical image is coarse, the derived difference image to be stored will also be coarse, so that the storage capacity of the storage unit can be reduced compared to the case where the difference image generated at the output resolution, which is a high resolution, is stored. In addition, the storage destination may be changed between the difference image generated at the output resolution and the derived difference image and the difference projection image. The image processing device 100 further includes a storage unit that stores the difference image and the derived difference image in different storage units. The storage unit can store the difference image in a storage unit included in the image processing device 100, and can store the derived difference image in a storage unit included in a data server connected to the image processing device 100 via a network. More specifically, the difference image generated at the output resolution can be stored in a storage unit (not shown) in the image processing device 100, and the derived difference image and the difference projection image can be stored in a storage unit (not shown) in a data server 130 such as a PACS.

As a result, the derived difference image and the difference image that is the basis of the difference projection image are stored in the image processing device 100. Therefore, based on the difference image, for example, it is possible to check slices that are not included in the generated derived difference image, or to generate and view a difference projection image again with parameters different from the generated difference projection image. On the other hand, a large number of images are usually stored in a data server 130 such as a PACS, and the storage capacity is limited. Therefore, by storing only the derived difference image and the difference projection image, it is possible to reduce the storage capacity while storing the minimum information required to view the difference image.

(Variation 3)
In the first embodiment, the upper limit (upper limit resolution) when determining the value of the output resolution in the slice direction is the same as the processing resolution. However, the upper limit of the output resolution does not necessarily have to be the same as the processing resolution. For example, the processing resolution may be a value close to the upper limit of the output resolution. In this modified example, the close value can be defined as a value within ±0.5 mm of the target resolution. That is, it is a value including the upper limit resolution based on the upper limit resolution. For example, in the output resolution determination process of step S204, the upper limit of the output resolution is set to a resolution (1 mm) at which the difference in details between images can be calculated, as described in step S202. On the other hand, in step S202, the processing resolution for resolution conversion is set to 1.5 mm, which is a value close to 1 mm.

Then, in the process of aligning the two images in step S203, alignment is performed between the first converted image whose resolution has been converted to 1.5 mm and the second converted image. This allows high-speed alignment without sacrificing alignment accuracy (without sacrificing the amount of information on the displacement field) by aligning at a resolution close to that when the resolution is converted to 1 mm.

Then, in the differential image generation process in step S205, a displacement field with a resolution of 1.5 mm is used to generate a differential image with an upper limit of the output resolution of 1 mm, and in step S206, a differential projection image is generated based on that differential image. At this time, since the displacement field used to generate the differential image does not lose much information compared to the case of 1 mm, the differential projection image can be generated without losing much information in the slice direction. In this way, if the processing resolution is coarser than the upper limit of the output resolution but is a nearby value, the differential projection image can be output at high speed with as little loss in quality as possible compared to the differential projection image generated in the first embodiment.

Other Embodiments
The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

The present invention is not limited to the above-described embodiments, and various modifications and variations are possible without departing from the spirit and scope of the present invention.

REFERENCE SIGNS LIST 101 Acquisition unit 102 Conversion unit 103 Alignment unit 104 Determination unit 105 Difference image generation unit 106 Projection image generation unit 107 Display control unit

Claims (16)

1. An image processing device that generates a difference image to be output to a display unit from a first medical image and a second medical image, the first medical image and the second medical image being three-dimensional images constituted by a plurality of slices obtained by imaging a subject,
an acquisition means for acquiring the first medical image and the second medical image;
a determination means for determining a slice interval of the difference image generated by dividing the three-dimensional image in a slice direction of the subject, the slice interval being determined such that a slice of one of the first medical image and the second medical image serving as a reference is included in the difference image , and the slice interval of the slice in the difference image is smaller than the slice interval of the one of the medical images;
a generating means for generating the difference image having the slice interval determined by the determining means, and generating at least one slice corresponding to a slice of the one medical image from the difference image as a derived difference image;
a storage means for storing the difference image and the derived difference image in different storage units;
An image processing device comprising: The image processing device according to claim 1, characterized in that the determining means determines a slice interval for generating the difference image to be a value obtained by dividing the slice interval of the slice of the one medical image by a natural number. The image processing device according to claim 1 or 2, characterized in that the determination means determines the slice interval with the set resolution as an upper limit value. The image processing device according to any one of claims 1 to 3, further comprising a conversion means for converting the resolution of the first medical image and the second medical image to a set resolution. The image processing device according to claim 4, characterized in that the determination means determines the slice interval of the difference image based on a value close to the reference resolution, using the set resolution as the reference resolution. The determining means is
a first value obtained based on a difference between a value obtained by dividing a slice interval between slices of the one medical image by a first natural number and the reference resolution;
The image processing device according to claim 5, characterized in that a value obtained by dividing a slice interval of the slices of the one medical image by a second natural number is compared with a second value obtained based on a difference between the slice interval and the reference resolution, and the value with the smaller absolute value is set as the neighboring value. the first natural number is a natural number that gives a maximum value, equal to or less than the reference resolution, when a slice interval between slices of the one medical image is divided by the first natural number, and
7. The image processing apparatus according to claim 6, wherein the second natural number is a natural number that gives a minimum value equal to or greater than the reference resolution when a slice interval between slices of the one medical image is divided by the second natural number. The image processing device according to any one of claims 1 to 7, characterized in that the generating means converts the reference medical image into a resolution based on the slice interval, and performs processing to generate the difference image based on the reference medical image after resolution conversion. The image processing device according to any one of claims 1 to 7, characterized in that the generating means adds correspondence information to the difference image that associates slices of the reference medical image with slices of the difference image. a registration unit that performs a registration process between the first medical image and the second medical image after the resolution conversion and obtains a displacement field that associates positions between the images;
The image processing device according to claim 4 or 5, characterized in that the generating means generates a difference image between an image obtained by deforming the second medical image based on the displacement field so as to match the first medical image and the first medical image. The image processing device according to claim 1, further comprising a projection image generating means for generating a projection image by two-dimensionally projecting pixel values of the difference image in a direction parallel to the slice plane. The image processing device according to claim 11, further comprising a display control means for displaying at least one of the difference image, the derived difference image, and the projected image on a display means. The image processing device according to claim 1, characterized in that the storage means stores the difference image in a storage means provided in the image processing device, and stores the derived difference image in a storage means provided in a data server connected to the image processing device via a network. An image processing system comprising the image processing device according to claim 1, an instruction unit that receives input of instructions from an operator who operates the image processing device and inputs the instructions to the image processing device, and a display unit that displays the generated difference image. 1. An image processing method of an image processing device for generating a difference image to be output to a display unit from a first medical image and a second medical image, the first medical image and the second medical image being three-dimensional images constituted by a plurality of slices obtained by imaging a subject, the method comprising:
An acquisition step in which an acquisition means acquires the first medical image and the second medical image;
a determining step in which a determining means determines a slice interval of the difference image generated by dividing the three-dimensional image in a slice direction of the subject, the slice interval being determined such that a slice of one of the first medical image and the second medical image serving as a reference is included in the difference image , and the slice interval of the slice in the difference image is smaller than the slice interval of the one of the medical images;
a generating step in which a generating means generates the difference image having the slice interval determined in the determining step, and generates at least one slice corresponding to a slice of the one medical image from the difference image as a derived difference image;
a storage step in which a storage unit stores the difference image and the derived difference image in different storage units;
13. An image processing method comprising the steps of: A program for causing a computer to execute each step of the image processing method described in claim 15.

Images