JP6419115B2 - Image processing apparatus, image processing method, and program - Google Patents

Description

  The present invention relates to a technique relating to high image quality of a tomographic image in an eye part.

  An ophthalmic tomographic imaging apparatus, such as an optical coherence tomography (hereinafter referred to as “OCT”), can observe the state inside the retinal layer in three dimensions. This tomographic imaging apparatus has attracted attention in recent years because it is useful for more accurately diagnosing diseases.

  In ophthalmologic diagnosis, there are cases where a volume image is used to grasp the state of the entire retinal layer and a high-quality two-dimensional tomographic image is used to grasp a layer that does not appear in a low-quality tomographic image. A volume image means a set of two-dimensional tomographic images.

  The image quality of a tomographic image obtained by OCT depends on the intensity of near infrared light incident on the retina. For this reason, in order to improve the image quality of a tomographic image, it is necessary to increase the intensity of light irradiated to the retina, but from the viewpoint of safety, there is a limit to the intensity of light that can be irradiated to the retina. For this reason, it is desired to generate a high-quality tomographic image while irradiating near-infrared light within a safe intensity range for safety. In response to such a requirement, a technique for generating a cross-sectional image with less noise by superimposing photographed two-dimensional tomographic image groups on each other is disclosed (see Patent Document 1).

JP 2008-237238 A

  However, in Patent Document 1, the entire plurality of tomographic images are simply averaged. For this reason, when the correlation between the added tomographic images is low, the reduction of the diagnostic information may be large. In particular, since eye movements and the like occur, the entire areas of adjacent images are not always similar.

  The present invention has been made in view of the above problems, and an object thereof is to improve the image quality of a tomographic image.

  In order to achieve the above object, an image processing apparatus according to an aspect of the present invention includes the following arrangement.

That is, after the plurality of tomographic images of the fundus of the eye to be examined obtained based on the measurement light of OCT (Optical Coherence Tomography) controlled to scan the same position a plurality of times, the plurality of tomographic images are aligned. Calculating means for calculating a similarity between partial areas, which are partial areas of the tomographic image having a predetermined width in a direction orthogonal to the A-scan direction, and the similarity calculated by the calculating means A selection unit that selects an A scan image included in the partial region higher than a predetermined value from a plurality of corresponding A scan images between the plurality of tomographic images, and one of the plurality of A scan images selected by the selection unit. Generating means for generating an A scan image of the A scan image, and the part in a direction orthogonal to the A scan direction Alignment means for aligning a plurality of tomographic images of the fundus of the eye to be examined, wherein the width of the region is larger than the width of the A-scan image, and the plurality of tomographic images after being aligned by the alignment means A calculation unit that calculates a similarity between them, a selection unit that selects a plurality of tomographic images whose similarity calculated by the calculation unit is higher than a predetermined value from the plurality of tomographic images, and a selection unit selected by the selection unit Generating means for generating one tomographic image from a plurality of tomographic images.

  According to the present invention, the image quality of a tomographic image can be improved.

It is a figure which shows the structure of an image processing system. It is a flowchart which shows the flow of the tomographic imaging process in an image processing device. It is a figure for demonstrating a superimposition image generation process. It is a figure for demonstrating a superimposition area | region. It is a figure for demonstrating the determination process of superimposition. It is a figure which shows the structure of an image processing system. It is a flowchart which shows the flow of the tomographic imaging process in an image processing device. It is a figure for demonstrating a superimposition area | region.

Example 1
Hereinafter, an image processing system including the image processing apparatus according to the present embodiment will be described with reference to the drawings.

  FIG. 1 is a diagram illustrating a configuration of an image processing system 100 including an image processing apparatus 110 according to the present embodiment. As shown in FIG. 1, the image processing system 100 is configured by connecting an image processing apparatus 110 to a tomographic imaging apparatus 120 via an interface.

  The tomographic imaging apparatus 120 is an apparatus that captures a tomographic image of the eye, and includes, for example, a time domain type OCT or a Fourier domain type OCT. Since the tomographic imaging apparatus 120 is a known apparatus, detailed description thereof is omitted.

  The image processing apparatus 110 includes an acquisition unit 111, a storage unit 112, an image processing unit 113, and a display control unit 114.

  The acquisition unit 111 acquires a tomographic image captured by the tomographic image capturing apparatus 120 and stores it in the storage unit 112. The image processing unit 113 generates a new two-dimensional tomographic image from the tomographic images stored in the storage unit 112. The display control unit 114 performs control for displaying the processed image on a monitor (not shown).

  Although a plurality of locations may be scanned sequentially, as an example in which the image quality is particularly improved, FIG. 3 (a) shows a second example of a macular portion in an example in which the tomographic image capturing apparatus 120 continuously scans almost the same location repeatedly. The schematic diagram of a two-dimensional tomographic image group is shown. Here, the direction in which the measurement light is scanned to capture a two-dimensional tomographic image is referred to as a main scanning direction. A direction orthogonal to the main scanning direction is called a sub-scanning direction.

  In general, the tomographic imaging apparatus 120 performs imaging in the main scanning direction while shifting the measurement light in the sub-scanning direction. That is, the present invention can also be applied when shifting in the sub-scanning direction.

In FIG. 3A, x and z represent coordinate axes, and t represents a time axis. T _{1 to} T _n are two-dimensional tomographic images of the macular region taken at different time points. That is, the two-dimensional tomographic image group is formed by a set of two-dimensional tomographic images obtained by photographing almost the same part.

  Here, the improvement in image quality means that the S / N ratio is improved.

  The improvement in image quality means an improvement in the S / N ratio.

Next, a processing procedure of the image processing apparatus 110 of this embodiment will be described with reference to the flowcharts of FIGS. 2 (a) and 2 (b).
<Step S201>
In step S201, the tomographic image capturing apparatus 120 is controlled by a control unit (not shown) to capture the retinal layer. First, the positions of the depth direction (z direction in FIG. 3A), which is the direction in which the measurement light is irradiated, and the plane direction orthogonal to the z direction (x direction in FIG. 3A) are adjusted. Here, matching the position in the depth direction corresponds to matching the position of the coherent gate for obtaining the tomographic image.
<Step S202>
In step S201, the control unit (not shown) adjusts the position suitable for photographing the retinal layer, and in step S202, the photographing instruction unit (not shown) instructs the start of photographing.
<Step S203>
In step S203, when an operator gives an imaging instruction, a control unit (not shown) repeatedly scans substantially the same portion to capture a plurality of two-dimensional tomographic images.

Note that the control unit (not shown) also has a function of adjusting the interval of movement in the sub-scanning direction.
<Step S204>
In step S <b> 204, the image processing unit 113 generates a new two-dimensional tomographic image using the two-dimensional tomographic image group stored in the storage unit 112. A process for generating a high-quality two-dimensional tomographic image will be described with reference to FIG.
<Step S210>
In step S210, the first alignment unit 115 aligns two-dimensional tomographic images. As the alignment processing, for example, an evaluation function indicating the similarity between two two-dimensional tomographic images is defined in advance, and the tomographic image is deformed so that the value of this evaluation function becomes the best. Examples of the evaluation function include a method of evaluating with pixel values (for example, a method of performing evaluation using the correlation coefficient of Expression (1)). Examples of the image deformation process include a process of performing translation and rotation using affine transformation and changing an enlargement ratio. As the alignment process, the position may be aligned based on feature points. For example, features such as each retinal layer and lesion are extracted from the two-dimensional tomographic image. The inner boundary membrane, nerve fiber layer, photoreceptor inner / outer joint junction, and retinal pigment epithelium layer have high luminance values, and the layer boundary has high contrast. Align.

Accordingly, the position deformation parameter that maximizes the evaluation function is calculated while deforming the two-dimensional tomographic image, and the positions between the two-dimensional tomographic images are matched. When the number of two-dimensional tomographic images for superposition is N, N-1 two-dimensional tomographic image alignment processing is performed on the reference two-dimensional tomographic image.
<Step S211>
In step S211, the determination unit 117 determines an A-scan image to be superimposed for each corresponding A-scan image. This process will be described with reference to FIG. T _i ′ and T _{i + 1} ′ are two-dimensional tomographic images captured at different points in time and are aligned in step S210. A _ij ′ and A _{(i + 1) j} ′ respectively represent A scan images corresponding to T _i ′ and T _{i + 1} ′ after alignment. In the A scan image in the present invention, one pixel column parallel to the z-axis direction in FIG. 4 is an A scan image.

  The A-scan image is an image that matches the incident direction of the measurement light, and each A-scan image obtained from the same location has almost the same image information. For this reason, even if the similarity of the entire two-dimensional tomographic image is low due to eye fixation micromotion, there may be data having high similarity in the A-scan images of different two-dimensional tomographic images.

The same hatched regions R _ij ′ and R _{(i + 1) j} ′ represent + −α regions in the x-axis direction centering on A _ij ′ and A _{(i + 1) j} ′. The calculation unit 116 calculates the similarity between the A scan images in the image regions R _ij ′, R _{(i + 1) j} ′ centered on the A scan image. When the reference region for performing overlay determination is R _ij ′, the calculation unit 116 calculates the similarity between all corresponding regions of R _1j ′ to R _nj ′. Here, an equation in the case of using a correlation coefficient as an evaluation function indicating the similarity between the A-scan images is shown in Equation 1.

In the case of FIG. 4, it is assumed that the region R _ij ′ is f (x, y) and R _{(i + 1) j} ′ is g (x, y).

  Respectively represent the average of the region f (x, y) and the region g (x, y).

  The determination unit 117 selects a region to be used for superposition for each region. The processing of the determination unit will be described with reference to FIG. FIG. 5 shows an example of the result of the calculation unit 116 calculating the similarity between regions.

  The horizontal axis is numbers 1 to N of the captured two-dimensional tomographic image. The vertical axis represents the similarity between the reference region and the region in another two-dimensional tomographic image. FIG. 5A shows an example in which a threshold Th is set and a region having a similarity equal to or higher than a predetermined value is selected. FIG. 5B is an example in which FIG. 5A is sorted in the order of similarity, and the top M sheets with similarities are selected. FIG. 5C shows a line graph representing the change rate of the similarity after the sort processing in the order of similarity. Here, it is an example in the case of selecting an image before the rate of change in similarity is significantly reduced. That is, the similarity where the change rate of the similarity after the sorting process shows a predetermined value is obtained, and an area having a value equal to or higher than the obtained similarity is selected.

Also, the rate of change between similarities is seen without sorting. When the similarity is lower than a predetermined value, the calculation unit 116 stops calculating the evaluation value. An image whose evaluation value is equal to or greater than a predetermined value and before the calculation is stopped is selected.
In the case of the threshold value (a), those with poor evaluation values are not used for superimposition, so that only regions with good evaluation values can be overlaid.

This is suitable when the movement distance per unit time in the sub-scanning direction is small. This is because there is little change in the tissue structure, but there is an effect of not selecting an image in a region where the tissue structure caused by specific eye movement or blinking is greatly different.
When M sheets are selected as the fixed number of sheets (b), there is no variation in the number of data to be subjected to the overlay averaging process in units of images. There is no variation in the number of data to be subjected to the overlay averaging process for each A-scan image unit. Since the degree of noise reduction can be made uniform, it is more suitable when the image quality is made comparable.
When viewing the rate of change in (c), there is a feature that similar regions can be selected even when the overall image quality is poor due to illness or the like and the similarity is low overall.
The method (d) is also suitable when the moving distance per unit time in the sub-scanning direction becomes large. This is because when the deviation in the sub-scanning direction exceeds a predetermined value, the tissue structure of the retina is different, so that the calculation unit 116 can be prevented from calculating the evaluation value unnecessarily. In other words, the change in organizational structure can be seen by looking at the rate of change.

  As described above, in each region, the region to be superimposed is selected based on the evaluation value. Therefore, when the retinal layer is deformed in the two-dimensional tomographic image due to fixation micromotion, etc., or the area where the image quality is partially degraded due to blinking or vignetting is not used for overlaying, it was newly generated The image quality is improved.

Further, the determination unit 117 performs a process of combining the above (a) to (d) according to the moving distance per unit time in the sub-scanning direction. For example, when the moving distance is less than a predetermined value, the process (a) or (b) is used, and when the distance exceeds the predetermined value, the process (c) or (d) is performed. The combination of (a) and (d) can perform processing with an emphasis on speed, and the combination of (b) and (d) can perform processing with an emphasis on image quality.
<Step S212>
In step S212, the generation unit 118 performs an overlay process. Here, the superposition in the case of two A-scan images will be described. FIG. 3B is a diagram for explaining processing for processing a two-dimensional tomographic image for each A scan image and generating one two-dimensional tomographic image as a composite image. That is, an explanation will be given by taking as an example an addition averaging process of A-scan images that are imaged at different times and located on different two-dimensional tomographic images. FIG. 3C is a two-dimensional tomographic image with high image quality generated by performing an averaging process using M (two in this example) pixels for each pixel. That is, in FIG. 3C, A _ij ″ is a new A scan image calculated by performing the averaging process on the corresponding A scan image.

In FIG. 3B, T _i ′ and T _{i + 1} ′ are two-dimensional tomographic images obtained by imaging the same cross section at different times. A _ij ′ and A _{(i + 1) j} ′ represent the respective A scan images in the tomographic images T _i ′ and T _{i + 1} ′. The generation unit 118 calculates A _ij ″ in FIG. 3C by performing addition averaging processing on the A scan images of A _ij ′ and A _{(i + 1) j} ′. Note that a high-quality two-dimensional tomographic image is obtained. The (composite image) generation process is not limited to the averaging process, and a median calculation process, a weighted averaging process, or the like may be used, which may be used as an A scan of the reference image T _i ′. This is performed for all images (from A _i1 ′ to A _im ′) For example, in the weighted addition averaging process, the above-mentioned similarity is used as a weight for addition.

In the present embodiment, the region in which the similarity is calculated is R _ij ′, and the overlay processing is performed in A _ij ′ (A scan image unit). However, the present invention is not limited to this. . For example, the overlay process may be performed in units of regions where the similarity is calculated. Further, the overlay process may be performed with a two-dimensional tomographic image. Furthermore, α may be set to 0 (in the case of α = 0, R _ij ′ = A _ij ′), the degree of similarity may be calculated and determined for each A-scan image unit, and superposition may be performed.
<Step S205>
In step S205, the display control unit 114 displays the high-quality two-dimensional tomographic image generated in step S204 on a display unit (not shown). Further, the example in which the scanning range of the measurement light of the tomographic imaging apparatus 120 is fixed and almost the same retina region is scanned has been described. However, as described above, the present invention can be applied even when the entire retina is sequentially scanned. It is.

  As is clear from the above description, in this embodiment, in a plurality of aligned two-dimensional tomographic images, the similarity between the regions is calculated using the corresponding A scan image and the peripheral region, and each region is calculated. The region used for the overlay process was determined in units. As a result, the entire image is in position, but even when the retinal layer in the two-dimensional tomographic image is deformed due to fixation micromotion, etc., the region used for the overlay process is determined in units of partial regions. Therefore, a high-quality two-dimensional tomographic image can be generated.

According to the present embodiment, it is possible to acquire a high-quality two-dimensional tomographic image even when the retinal layer in the two-dimensional tomographic image is deformed due to fixation micromotion or the like. Here, a high-quality image means an image in which the S / N ratio is improved as compared with one-time shooting. Or it refers to an image in which the amount of information necessary for diagnosis is increasing.
(Example 2)
In the first embodiment, in the two-dimensional tomographic image that has been aligned, the similarity between the regions is calculated using the corresponding A-scan image and the surrounding region, and the region used for the overlay process in units of each region Judgment was made. The present embodiment is characterized in that, in the aligned two-dimensional tomographic image, using the A-scan image and the peripheral region, a region having a high evaluation value in the vicinity region is searched for, and the overlay process is performed. . According to the present embodiment, alignment is performed using global features as a whole, and after performing alignment using local features, the overlay process is performed.

  FIG. 6 is a diagram illustrating a configuration of an image processing system 600 including the image processing apparatus 610 according to the present embodiment. As illustrated in FIG. 6, the image processing apparatus 610 includes an acquisition unit 111, a storage unit 112, an image processing unit 613, and a display control unit 114. Among these, since functions other than the image processing unit 613 have the same functions as those of the first embodiment, description thereof is omitted here.

  In the image processing unit 613, the second alignment unit 619 performs local alignment using the A-scan image and the surrounding area.

Hereinafter, a processing procedure of the image processing apparatus 610 according to the present embodiment will be described with reference to FIGS. 7 and 8. Since steps other than step S704 are the same as steps S201 to S203 and step S205 of the first embodiment, description thereof will be omitted.
<Step S704>
In step S704, the image processing unit 613 generates a high-quality two-dimensional tomographic image by performing global alignment and local alignment and performing image overlay processing. A high-quality two-dimensional tomographic image generation process will be described with reference to FIG.
<Step S710>
In step S710, the first alignment unit 115 aligns the two-dimensional tomographic images. Since this process is the same as step S210 of the first embodiment, a description thereof will be omitted.
<Step S711>
In the two-dimensional tomographic image that has been globally aligned in step S710, local alignment is performed by performing alignment for each corresponding A-scan image. This process will be described with reference to FIG. T _i ′ and T _{i + 1} ′ are images obtained by capturing the same cross section at different points in time, and are two-dimensional tomographic images that have been aligned in step S710. A _ij ′ represents an A-scan image at T _i ′. A hatched region R _ij ″ represents a range of + −α in the x-axis direction centering on A _ij ′. R _{(i + 1) j} ″ represents a rectangular region corresponding to R _ij ″ in T _{i + 1} ′. S _{(i + 1) j} ′ represents a search range for moving the rectangular region of R _{(i + 1) j} ″. When the reference A-scan image is A _ij ′ and the region for calculating the similarity is R _ij ″, calculation is performed by scanning a rectangular region of R _{(i + 1) j} ″ within the search range S _{(i + 1) j} ′. The unit 116 calculates an evaluation value.

As the evaluation value, the calculation unit 116 calculates the correlation between the pixel values of R _ij ″ and R _{(i + 1) j} ″, and the determination unit 117 performs the evaluation. Alternatively, the layer thickness is detected by detecting the retinal layer boundary, and the calculation unit 116 calculates the evaluation value of the similarity using the layer thickness. FIG. 8B shows a case where the similarity is evaluated using the layer thickness. FIG. 8B shows retinal layer boundaries (inner boundary film L ₁ , retinal pigment epithelial layer L ₂ ) in the search range S _{(i + 1) j} ′ and layer thicknesses 1 to 3 obtained from the retinal layer boundaries. Yes. In the rectangular region R _{(i + 1) j} ″, the layer thickness on each A-scan image is obtained. Then, the calculation unit 116 calculates the evaluation value by comparing the layer thicknesses in R _ij ″ and R _{(i + 1) j} ″. The layer used for calculating the layer thickness is not limited to the above, and the layer thickness may be compared using other layer boundaries such as a nerve fiber layer and a joint node between outer and outer segments of photoreceptor cells. .

In the present embodiment, the overlay determination process by the determination unit 117 is omitted. However, whether or not the overlay is performed based on the similarity as in the first embodiment after performing the alignment for each A-scan image. The determination may be performed.
<Step S712>
In step S <b> 712, the generation unit 118 performs overlapping processing of corresponding A scan images. In step S711, the generation unit 118 performs a superimposition process on the A scan image located at the center of the rectangular region R _{(i + 1) j} ″ where the evaluation value is maximum and the A scan image A _ij ′ of the reference image. Do.

In the present embodiment, R _ij ″ is set as a rectangular region with a range of + −α centered on the position of the A scan image. As this setting method, α is not a fixed value but is shown in the two-dimensional tomographic image. An example of region setting is shown in Fig. 8C. For example, when the shape of the retinal layer is planar, α Is set wide (R _{(i + 1) j} ″). When there is a characteristic portion (a large number of vertical and horizontal edges) in the two-dimensional tomographic image in which the shape of the retinal layer is curved, the range of α is set narrow (R _{(i + 1) k} ″). The setting of the region range may be changed for each case from the image characteristics, or may be changed for each A-scan image in one two-dimensional tomographic image.

As is clear from the above description, in the present embodiment, in the aligned two-dimensional tomographic image, an area where the evaluation value is high in the neighboring area is searched using the A-scan image and the peripheral area, and overlapped. The combination process was performed. As a result, the entire image is aligned, but even when the retinal layer in the two-dimensional tomographic image is deformed due to fixation micromotion, etc., the alignment processing is performed in local units, so that high-quality A dimensional tomographic image can be generated.
(Other embodiments)
Each of the above embodiments implements the present invention as an image processing apparatus. However, the embodiment of the present invention is not limited only to the image processing apparatus. The present invention can also be realized as software that runs on a computer. The CPU of the image processing apparatus controls the entire computer using computer programs and data stored in RAM and ROM. In addition, the execution of software corresponding to each unit of the image processing apparatus is controlled to realize the function of each unit.

DESCRIPTION OF SYMBOLS 110 Image processing apparatus 120 Tomography apparatus 112 Storage part 113 Image processing part 114 Display control part 115 1st alignment part 116 Calculation part 117 Judgment part 118 Generation part 619 2nd alignment part

Claims (5)

After a plurality of tomographic image measuring the light onto the eye obtained based fundus of the same position is controlled so as to scan a plurality of times OCT (Optical Coherence Tomography) are aligned, in each of the plurality of tomographic images Calculating means for calculating a similarity between partial areas that are partial areas of the tomographic image having a predetermined width in a direction orthogonal to the A-scan direction;
Selection means for selecting an A-scan image included in the partial area having a similarity calculated by the calculation means higher than a predetermined value from a plurality of corresponding A-scan images among the plurality of tomographic images;
Generating means for generating one A scan image from a plurality of A scan images selected by the selecting means;
An image processing apparatus, wherein a width of the partial area in a direction orthogonal to the A scan direction is larger than a width of the A scan image.   The image processing apparatus according to claim 1, wherein the partial area is an area centered on an A-scan image, and the A-scan image included in the partial area is an A-scan image at the center of the partial area. . The calculating means calculates the similarity in the partial regions at a plurality of different positions in a direction orthogonal to the A-scan direction;
The image processing apparatus according to claim 1, wherein the selection unit selects the A scan image in the partial areas at different positions in a direction orthogonal to the A scan direction. . After the plurality of tomographic images of the fundus of the eye to be examined obtained based on the measurement light of OCT (Optical Coherence Tomography) controlled to scan the same position multiple times by the processor, the plurality of tomographic images are aligned. A calculation step of calculating a similarity between partial areas that are partial areas of the tomographic image having a predetermined width in a direction orthogonal to the A-scan direction in each of the images ;
A selection step of selecting an A-scan image included in the partial area having a similarity calculated by the calculation unit higher than a predetermined value by a processor from a plurality of corresponding A-scan images among a plurality of tomographic images;
A generating step of generating one A-scan image from a plurality of A-scan images selected by the selection means by a processor;
An image processing method comprising: Program for executing the image processing method according to the computer to claim 4.