JP4202671B2 - Standardized image generation method and apparatus - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention detects re-radiation light of different wavelength bands generated from a living tissue by light irradiation as an image, adds a desired offset to at least one of the images of different wavelength bands, and adds the offset The present invention relates to a normalized image generation method and apparatus for generating a normalized image by obtaining a ratio of different images.
[0002]
[Prior art]
Conventionally, when the living tissue is irradiated with excitation light in the excitation light wavelength region of the living dye, the fluorescence intensity emitted between the normal tissue and the diseased tissue is different, thereby exciting the living tissue in the predetermined wavelength region. There has been proposed a fluorescence detection apparatus that recognizes the localization and infiltration range of a diseased tissue by irradiating light and detecting fluorescence emitted from a living body dye.
[0003]
Normally, when excitation light is irradiated, strong fluorescence is emitted from normal tissue as shown by a solid line in FIG. 10, and weaker fluorescence is emitted from lesion tissue than fluorescence emitted from normal tissue as shown by a broken line. By measuring the strength, it is possible to determine whether the living tissue is normal or in a lesion state.
[0004]
Furthermore, a method has been proposed in which fluorescence from excitation light is imaged by an imaging element or the like, and a fluorescence image corresponding to the intensity of the fluorescence is displayed to determine whether the living tissue is normal or in a lesion state. In this technique, since the living tissue has irregularities, the intensity of the excitation light irradiated to the living tissue is not uniform, and the fluorescence intensity emitted from the living tissue is proportional to the square of the distance between the light source and the living tissue. Will drop. Further, the distance between the fluorescence detection means and the living tissue also decreases in proportion to the square thereof. For this reason, strong fluorescence may be received from a lesion tissue closer to a normal tissue far from the light source or the detection means, and the tissue properties of the living tissue can be determined only by information on the intensity of the fluorescence from the excitation light. It cannot be accurately identified. In order to reduce such inconveniences, the ratio of fluorescence images based on two types of fluorescence intensities acquired from different wavelength bands (narrow band near 480 nm and wide band near 430 nm to 730 nm) is obtained by division, and the division value A standardized image generation method based on the difference in the shape of the fluorescence spectrum that reflects the tissue properties of a living body has been proposed. In addition, color information is assigned to the division values of fluorescence images in different wavelength bands, and the lesion state of a living tissue is indicated as a color image by the difference in color, and further, uniform absorption is performed for various living tissues. It is obtained by irradiating a living tissue with the received near-infrared light as reference light, detecting a reference image based on the intensity of reflected light from the living tissue due to irradiation of this reference light, and assigning luminance information to the reference image. A method has also been proposed in which a luminance image is combined with the color image to show an image with a sense of unevenness in which the shape of a living tissue is also reflected in the image.
[0005]
In addition, when a normalized fluorescence image is generated based on the division values of fluorescence images in different wavelength bands as described above, the fluorescence intensity from the living tissue used for the normalization calculation is weak, so this fluorescence intensity The S / N of the normalized fluorescent image based on the above is very low. In order to improve this, there has been proposed a method for improving the S / N of the normalized fluorescence image by adding the offset to at least one of the fluorescence images in different wavelength bands and calculating the division value.
[0006]
[Problems to be solved by the invention]
However, when obtaining a division value by adding an offset as described above, the offset value may be increased in order to obtain a good S / N. However, the fluorescence intensity depends on the distance between the detection means and the living tissue. Therefore, if the offset value is excessively increased, even if the biological tissue at the far point and the biological tissue at the near point are the same, the division value may be greatly different. Therefore, it becomes difficult to distinguish between the living tissue of the lesioned part and the living tissue of the normal part. As an example, FIG. 8 shows a division value of a narrow-band fluorescence image and a wide-band fluorescence image to which an offset is added (a narrow-band fluorescence image / (broadband fluorescence image + offset)) for a predetermined normal body tissue and a lesion tissue. The relationship between the division value and the above-mentioned distance when? The offset values are 5, 10, 15, and 20. The division value for the normal living tissue is indicated by a white symbol, and the division value for the lesion biological tissue is indicated by a black symbol. As shown in the figure, the division value varies with the distance, and the larger the offset value, the smaller the difference in the division value between the normal part and the lesioned part, making it difficult to identify. On the other hand, if the offset value is reduced, the change of the division value due to the distance is reduced, but naturally there is a problem that the effect of improving the S / N of the normalized fluorescent image by adding the offset cannot be obtained sufficiently. 9A and 9B, the normalized fluorescence image based on the division value when the distance between the detection means and the biological tissue is large (far point) for a predetermined biological tissue (FIG. 9A). FIG. 9B shows a normalized fluorescence image (FIG. 9B) based on the division value when the distance is small (near point). Note that the offset at this time is 20, and in each image, a portion that is relatively darker than the surrounding image means an image of a lesioned part. As shown in the figure, in the image of the far point and the near point, the image of the lesioned part or the image of the normal part must be displayed with the same brightness, but the far point image is closer to the near point. It is darker than the above image and the contrast between the normal part and the lesion part is also small, so that it is difficult to distinguish them.
[0007]
In view of the above-described problems of the prior art, the present invention adds a desired offset to at least one of images in different wavelength bands and performs a normalization operation to generate a standardized image. In a generation method and apparatus, a normalized image generation method and apparatus capable of taking a constant normalized calculation value even when the distance changes without depending on the distance between the detection means and the living tissue. It is intended.
[0008]
[Means for Solving the Problems]
The normalized image generation method of the present invention detects images of different wavelength bands based on re-radiated light generated from a living tissue by light irradiation, and detects at least one of images of different wavelength bands. In a standardized image generation method for generating a standardized image based on a ratio of images in different wavelength bands to which a desired offset is added to at least one of the predetermined offsets, A corrected normalized image is generated by performing a distance correction for correcting a variation amount of the normalized image caused by the distance between the living tissue and the image detecting means using the correction function.
[0009]
Here, the “image detection means” refers to, for example, an endoscope insertion portion that is inserted into a living body, an endoscope insertion in a fluorescence endoscope that detects fluorescence emitted from a living tissue by irradiation of excitation light. This means an image sensor that captures an image of the fluorescence guided by the unit and a light guide unit up to the image sensor. And about an image of a mutually different wavelength band, you may provide one part each, such as an image pick-up element and a light guide part, respectively, and may make it common.
[0010]
Further, “adding a desired offset to images in different wavelength bands” means adding an offset value to each pixel value of each image. By adding an offset value to each image, the variation in the pixel value of the normalized image after the normalization calculation is reduced, and the S / N of the normalized image can be improved. The portion to which the offset value is added may not be the entire image, and may be added only to a portion where the image is dark, that is, a portion having a low S / N. That is, when the normalization calculation is performed based on the ratio of each image, an offset value large enough to improve the S / N of the standardized image so as to improve the accuracy of identifying the tissue property. Add it.
[0011]
The “image ratio” means a ratio of pixel values on the same coordinates in images of different wavelength bands, and “normalized based on the image ratio”. “Generate an image” means to divide by each pixel value on the same coordinate or perform a standardization operation similar to it, and generate an image having the normalized value as a pixel value as a standardized image Means that.
[0012]
In addition, the above-mentioned “distance between the living tissue and the image detection means” means a distance that substantially affects the size of the pixel value of the detected image. Is the distance between the living tissue and the distal end of the endoscope insertion portion in the fluorescence endoscope. Furthermore, when the excitation light irradiating means for irradiating the biological tissue with the excitation light also moves in the positional relationship with the biological tissue, the pixel value of the fluorescence image varies depending on the movement, so the above distance is determined by the excitation light irradiation. The distance between the means and the living tissue is also included.
[0013]
In addition, the “variation amount of the standardized image” means a variation amount of the pixel value of the standardized image.
[0014]
Further, “correcting the amount of fluctuation of the standardized image due to the distance” means correcting the magnitude of the pixel value so that the pixel value of the standardized image does not vary depending on the distance. To do.
[0015]
Further, the normalized image generation method of the present invention detects fluorescence images in different wavelength bands based on fluorescence generated from a living tissue by irradiation of excitation light, respectively, and detects fluorescence images in different wavelength bands. A standard that adds a desired offset to at least one of the two and performs a normalization operation based on a ratio of fluorescent images in different wavelength bands to which the desired offset is added to at least one of the two to generate a normalized fluorescent image In the normalized image generation method, the normalized fluorescence image is subjected to a distance correction that corrects a variation amount of the normalized fluorescence image caused by the distance between the living tissue and the fluorescence image detection unit using a predetermined correction function, thereby correcting the normalized fluorescence image An image is generated.
[0016]
In addition, the fluorescent images having different wavelength bands can be a narrow-band fluorescent image and a broadband fluorescent image.
[0017]
In addition, the correction function can be calculated based on at least one of a broadband fluorescent image and a narrowband fluorescent image for a predetermined biological tissue whose property is known.
[0018]
Further, a reference image by reflected light reflected from a predetermined biological tissue whose property is known by reference light irradiation is detected by a reference image detection means, and a correction function is calculated based on the reference image. Can do.
[0019]
Here, the “reference light image detection means” refers to, for example, an endoscope insertion portion that is inserted into a living body in a fluorescence endoscope, and reflected light that is emitted by reference light guided by the endoscope insertion portion. Means that includes an image pickup device that picks up an image and a light guide portion up to the image pickup device. The reference light image detection means and the fluorescence image detection means are preferably a common endoscope insertion part and a light guide part, but since the wavelength bands are different, an optical filter for selecting the wavelength band of each image is provided. It is necessary to provide it. If the image sensor is also used in common, the exposure timing may be shifted to capture an image.
[0020]
Further, when the correction function is calculated based on the reference image, it is desirable that the distance between the reference image detecting unit and the living tissue and the distance between the fluorescent image detecting unit and the living tissue are equal.
[0021]
Further, the correction function can be expressed by the following equation (5), and the corrected normalized fluorescence image can be calculated by the following equation (6).
[0022]
(Nir + os2) / nir (5)
{N / (w + os1)} × {(nir + os2) / nir} (6)
However,
n: Narrow-band fluorescence image
w: Broadband fluorescence image
nir: reference image
os1: Offset
os2: correction coefficient, os2 = os1 × knir / kw
knir: distance between reference image / biological tissue and reference image detection means
kw: Broadband fluorescent image / distance between biological tissue and fluorescent image detection means
Here, n, w, and mir mean the pixel values of the narrow-band fluorescent image, the broadband fluorescent image, and the reference image, respectively. Further, knir is obtained by dividing the pixel value of the reference image by the distance between the living tissue and the reference image detecting means, that is, the pixel value of the reference image per unit distance. Kw is obtained by dividing the pixel value of the broadband fluorescent image by the distance between the living tissue and the fluorescent image detecting means, that is, the pixel value of the broadband fluorescent image per unit distance. Further, the correction coefficient os2 is calculated from the offset, knir and kw as shown in the above equation.
[0023]
Further, the os2 can be calculated based on knir and kw when the biological tissue of the normal part is imaged by the reference image detection unit and the fluorescence image detection unit.
[0024]
The os2 can be calculated based on knir and kw when the biological tissue of the lesion is imaged by the reference image detection unit and the fluorescence image detection unit.
[0025]
The os2 is based on each knir and each kw when the biological tissue at each stage when the progress from the normal part to the lesioned part is divided into s stages is imaged by the reference image detection unit and the fluorescence image detection unit. Can be calculated respectively.
[0026]
In addition, using the s correction functions based on os2 corresponding to each of the s-stage biological tissues, distance correction is performed on the standardized fluorescent image serving as a reference for each of the s-stage biological tissues, and s pieces are used as the reference. Are calculated, and boundary values are set based on the s reference correction normalized fluorescent images, while distance correction is performed on the normalized fluorescent image of the living tissue using s correction functions. To calculate a corrected standardized fluorescent image, extract a region corresponding to the s-stage biological tissue using the boundary value for the s corrected standardized fluorescent images, and extract the extracted s regions Can be superimposed and displayed.
[0027]
Here, s in the “s stage” means a natural number of 2 or more.
[0028]
The “standardized fluorescent image as a reference” is a standardized fluorescent image calculated by imaging an s-stage living tissue as a sample in order to set the boundary value. The “standardized fluorescence image” means an image obtained by subjecting the standardized fluorescence image as a reference to distance correction using a correction function calculated for each living tissue.
[0029]
The “normalized fluorescent image of the biological tissue” is a normalized fluorescent image calculated by imaging a biological tissue that is actually subjected to image diagnosis or the like.
[0030]
The normalized image generating apparatus of the present invention includes a light irradiating unit that irradiates light to a living tissue and an image detecting unit that detects images in different wavelength bands based on re-radiated light generated from the living tissue by light irradiation. And a standard that generates a standardized image based on a ratio of images in different wavelength bands to which a desired offset is added to at least one of images in different wavelength bands and a desired offset is added to at least one of the images. In a standardized image generation apparatus including a standardized image generation unit, distance correction is performed to correct a variation amount of the standardized image caused by the distance between the living tissue and the image detection unit using a predetermined correction function for the standardized image. And a correction means for generating a corrected standardized image.
[0031]
In addition, the normalized image generation apparatus according to the present invention detects excitation light irradiation means for irradiating biological tissue with excitation light and fluorescence images in different wavelength bands based on fluorescence generated from the biological tissue by irradiation of excitation light. And a fluorescent image detecting means for adding a desired offset to at least one of the fluorescent images in different wavelength bands and a ratio of the fluorescent images in different wavelength bands to which the desired offset is added to at least one of the fluorescent images. In a standardized image generating apparatus including a standardized fluorescent image generating unit that performs a standardization operation to generate a standardized fluorescent image, a normal tissue and a fluorescent image detecting unit are used by using a predetermined correction function for the standardized fluorescent image. And a correction means for generating a corrected standardized fluorescent image by performing a distance correction for correcting the fluctuation amount of the standardized fluorescent image caused by the distance of It is.
[0032]
Further, the correction means performs distance correction on s standardized fluorescent images serving as a reference for each biological tissue in the s stage using s correction functions based on os2 corresponding to each biological tissue in the s stage. s number of corrected standardized fluorescent images to be used as a reference, boundary value setting means for setting a boundary value based on the s reference corrected standardized fluorescent images, and normalization of living tissue by the correcting means A region extraction unit that extracts regions corresponding to s stages of biological tissue using boundary values from s correction standardized fluorescence images calculated using s correction functions for the fluorescence image, and a region extraction unit. It is characterized by comprising display means for displaying the extracted areas of the s corrected standardized fluorescent images in a superimposed manner.
[0033]
【The invention's effect】
According to the normalized image generation method and apparatus of the present invention, images in different wavelength bands are detected by the image detection means based on re-radiated light generated from the living tissue by light irradiation, and images in different wavelength bands are detected. A standardized image generation method for generating a standardized image based on a ratio of images of different wavelength bands to which a desired offset is added to at least one and a desired offset is added to at least one. Since a corrected standardized image is generated by performing a distance correction that corrects the variation amount of the standardized image due to the distance between the living tissue and the image detecting means using a predetermined correction function, by adding an offset The S / N of the normalized calculation value can be improved, and the distance is not dependent on the distance between the image detection means and the living tissue. It is possible to take certain of the normalized calculated values even when the change can be performed to identify the tissue state of the biological tissue more accurately.
[0034]
Further, when the images of different wavelength bands are a narrow-band fluorescent image and a broadband fluorescent image, and the correction function is calculated based on at least one of the broadband fluorescent image and the narrow-band fluorescent image, In particular, the correction function can be easily obtained only from the arithmetic processing without providing a mechanism for obtaining the correction function.
[0035]
In addition, when the reference image detection unit detects a reference image by reflected light reflected from a predetermined biological tissue whose properties are known by irradiation of the reference light, and the correction function is calculated based on the reference image In this case, the reference image does not depend on the properties of the living tissue and reflects the distance between the living tissue and the reference image detection unit as it is, so that it is possible to perform distance correction with higher accuracy.
[0036]
When the correction function is expressed by the following equation (7) and the corrected standardized fluorescence image is calculated by the following equation (8), the correction is performed by a simpler calculation process. A function can be obtained.
[0037]
(Nir + os2) / nir (7)
{N / (w + os1)} × {(nir + os2) / nir} (8)
However,
n: Narrow-band fluorescence image
w: Broadband fluorescence image
nir: reference image
os1: Offset
os2: correction coefficient, os2 = os1 × knir / kw
knir: distance between reference image / biological tissue and reference image detection means
kw: Broadband fluorescent image / distance between biological tissue and fluorescent image detection means
In addition, the s number of correction functions based on os2 corresponding to the respective biological tissues in the s stage are subjected to distance correction to the standardized fluorescent image serving as the reference for the respective biological tissues in the s stage, and are used as the reference. Each of the corrected normalized fluorescent images is calculated, and a boundary value is set based on the s reference corrected normalized fluorescent images. On the other hand, distance correction is performed on the normalized fluorescent image of the living tissue using s correction functions. S corrected standardized fluorescent images are respectively calculated, and a region corresponding to the s-stage biological tissue is extracted from the s corrected standardized fluorescent images using boundary values, and the extracted s pieces When the regions are superimposed and displayed, the progress of the diseased body can be more clearly confirmed on the same living tissue by an image.
[0038]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, specific embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a diagram showing a schematic configuration of a first embodiment of a fluorescence endoscope to which a standardized image generating apparatus that implements a standardized image generating method of the present invention is applied.
[0039]
The fluorescence endoscope according to the present embodiment processes an endoscope insertion unit 100 inserted into a site suspected of being a patient's lesion, and information obtained from a living tissue by the endoscope insertion unit 100 as an image signal. The image signal processing unit 1 includes a monitor 600 that displays a signal processed by the image signal processing unit 1 as a visible image.
[0040]
The image signal processing unit 1 includes an illumination unit 110 including three light sources that respectively emit normal image white light Lw, autofluorescence image excitation light Lr, and reference image reference light Ls, and irradiation of the excitation light. An image detection unit 300 that captures an autofluorescence image Zj generated from the living tissue 9 and a reference image Zs generated from the living tissue 9 by irradiation of the reference light, converts the digital image into digital values, and outputs the image as two-dimensional image data. A normalization calculation is performed from the two-dimensional image data of the autofluorescence image output from the detection unit 300, color information is allocated to the normalized calculation value, luminance information is allocated to the two-dimensional image data of the reference image, and An image arithmetic unit 400 that synthesizes and outputs image information, and converts a normal image into a digital value to obtain two-dimensional image data, and the two-dimensional image data and the image arithmetic unit. A display signal processing unit 500 that converts the output signal of the network 400 into a video signal and outputs it, a control computer 200 that is connected to each unit and controls the operation timing, and a normal image display state and a composite image display described later. It consists of a foot switch 2 for switching the state.
[0041]
The endoscope insertion portion 100 includes a light guide 101 that extends to the tip inside and an image fiber 102. An illumination lens 103 is provided at the distal end portion of the light guide 101, that is, the distal end portion of the endoscope insertion portion 100. Further, the image fiber 102 is a multi-component glass fiber, and includes an excitation light filter 104 and a condensing lens 105 at the tip thereof. In the light guide 101, a white light light guide 101a that is a multi-component glass fiber and an excitation light light guide 101b that is a quartz glass fiber are bundled and integrated into a cable shape, and the white light light guide 101a and the excitation light light guide are integrated. 101 b is connected to the lighting unit 110. The excitation light light guide 101b is also a light guide that guides reference light. One end of the image fiber 102 is connected to the image detection unit 300.
[0042]
The illumination unit 110 includes a GaN-based semiconductor laser 111 that emits excitation light Lr for autofluorescence images, a semiconductor laser power source 112 that is electrically connected to the GaN-based semiconductor laser 111, and a white light source that emits white light Lw for normal images. 114, a white light source 115 electrically connected to the white light source 114, a reference light source 117 that emits reference light Ls for reference images, a reference light source 118 that is electrically connected to the reference light source 117, and GaN The dichroic mirror 120 transmits the excitation light Lr output from the semiconductor laser 111 and reflects the reference light Ls output from the reference light source 117 in the right angle direction.
[0043]
An image fiber 102 is connected to the image detection unit 300, and an autofluorescent image propagated through the image fiber 102, a normal image, a collimating lens 301 that forms a reference image, and a normal image that has passed through the collimating lens 301 are perpendicular to each other. The fluorescence image and the reference image that have been totally reflected and transmitted through the collimator lens 301 are moved to the position indicated by the broken line 302 and the fluorescent image (light having a wavelength of 750 nm or less) transmitted through the collimator lens 301 is perpendicular to the movable mirror 302. Reflecting dichroic mirror 303, 50% of the amount of light of the autofluorescence image reflected by dichroic mirror 303 is transmitted, half mirror 308 reflecting 50% in the perpendicular direction, and autofluorescence image passing through half mirror 308 is reflected in the orthogonal direction The fluorescent image mirror 313 and the fluorescent image mirror 313 are arranged in a perpendicular direction. A broadband fluorescent image condensing lens 304 that forms a projected autofluorescent image, a broadband bandpass filter 305 that selects a wavelength of 430 nm to 730 nm from the autofluorescent image transmitted through the broadband fluorescent image condensing lens 304, and a broadband bandpass A high-sensitivity image sensor 306 for broadband fluorescent images that captures an auto-fluorescent image that has passed through the filter 305, and an auto-fluorescent image captured by the high-sensitivity image sensor 306 for broadband fluorescent images are converted into digital values and output as two-dimensional image data. From the auto-fluorescent image formed by the narrow-band fluorescent image condensing lens 309 and the narrow-band fluorescent image condensing lens 309 that forms an auto-fluorescent image reflected by the AD converter 307, the half mirror 308 in a right angle direction. Narrowband bandpass filter 310 and narrowband bandpass filter 31 for extracting wavelengths from 430 nm to 530 nm A high-sensitivity image sensor 311 for narrow-band fluorescence images that captures an auto-fluorescence image that has passed through the image, and an auto-fluorescence image captured by the high-sensitivity image sensor 311 for narrow-band fluorescence images is converted into digital values and output as two-dimensional image data An AD converter 312, a reference image condensing lens 314 that forms a reference image transmitted through the dichroic mirror 303, a reference image imaging device 315 that images a reference image formed by the reference image condensing lens 314, And an AD converter 316 that converts a reference image picked up by the reference image pickup element 315 into a digital value and outputs it as two-dimensional image data.
[0044]
The image calculation unit 400 includes an autofluorescence image memory 401 that stores digitized autofluorescence image signal data, a reference image memory 402 that stores reference image signal data, and 2 stored in the autofluorescence image memory 401. A pre-set offset value is added to each pixel value of the autofluorescence image in one wavelength band, and an operation according to the ratio of each pixel value to which the offset is added is performed to calculate a normalized operation value for each pixel. Based on the pixel value of the reference image stored in the standardized fluorescence image generation unit 403 and the reference image memory 402, a correction function to be described later is calculated and output from the standardized fluorescence image generation unit 403 using the correction function. A correction unit 404 that corrects the normalized calculation value to calculate a corrected normalization calculation value, and assigns color information to the correction standardization calculation value calculated by the correction unit 404. Color image calculation means 405, luminance image calculation means 406 for assigning luminance information to each pixel value of the reference image stored in the reference image memory 402, and an image signal having color information output from the color image calculation means 405 And an image synthesizing unit 407 for synthesizing image signals having luminance information output from the luminance image calculating unit 406 to generate and output a synthesized image.
[0045]
The autofluorescence image memory 401 is composed of a broadband autofluorescence image storage area and a narrowband autofluorescence image storage area (not shown). The broadband autofluorescence image captured by the high-sensitivity imaging device 306 for broadband fluorescence images is a broadband autofluorescence image. A narrowband fluorescent image stored in the fluorescent image storage area and captured by the high-sensitivity image sensor 311 for narrowband fluorescent images is stored in the narrowband fluorescent image storage area.
[0046]
The normalized fluorescence image generation means 403 adds an offset value to the pixel value of the autofluorescence image stored in the autofluorescence image memory 401, and performs a narrowband autofluorescence image and a broadband autofluorescence image according to the following equation (9). Calculate the ratio. In the present embodiment, the offset value is added to only the broadband autofluorescence image as shown in the following formula (9), and is a predetermined value stored in advance in the normalized fluorescence image generation means 403.
[0047]
n / (w + os1) (9)
n: Narrow-band autofluorescence image
w: Broadband autofluorescence image
os1: Offset
The correction unit 404 calculates a corrected normalized calculation value by multiplying the normalized calculation value calculated by the normalized fluorescent image generation unit 403 by the correction function calculated by the following equation (10). That is, the corrected standardization calculation value is calculated according to the following equation (11). In the present embodiment, the correction function is calculated from the reference image as shown in the following equation (10), and the correction unit 404 calculates the correction function from the pixel value of the reference image stored in the reference image memory 402. . Further, os2 is calculated according to the following expression (12), and knir is obtained by dividing the pixel value of the reference image by the distance from the distal end of the endoscope insertion unit 100 to the living tissue 9 when the reference image is detected. Further, kw is obtained by dividing the pixel value of the broadband autofluorescence image by the distance from the distal end of the endoscope insertion unit 100 to the living tissue 9 when the broadband autofluorescence image is detected.
[0048]
(Nir + os2) / nir (10)
nir: reference image
os2: Correction coefficient
{N / (w + os1)} × {(nir + os2) / nir} (11)
os2 = os1 × knir / kw (12)
knir: distance from the distal end of the reference image / endoscope insertion unit 100 to the living tissue 9
kw: Broadband fluorescent image / distance from the distal end of the endoscope insertion unit 100 to the living tissue 9
The color image calculation unit 405 assigns color information corresponding to the size of the correction pixel value calculated by the correction unit 404 and generates a color image.
[0049]
The luminance image calculation unit 406 assigns luminance information according to the size of the pixel value stored in the reference image memory 402 and generates a luminance image.
[0050]
The image synthesizing unit 407 synthesizes the color image output from the color image calculating unit 405 and the luminance image output from the luminance image calculating unit 406 and outputs the synthesized image to the video signal processing circuit 506 of the display signal processing unit 500 described later.
[0051]
The display signal processing unit 500 includes a normal image mirror 501 that reflects a normal image reflected by the movable mirror 302 in a right angle direction, and a normal image condensing lens 502 that forms a normal image reflected by the normal image mirror 501. The normal image imaging element 503 that captures the normal image formed by the normal image condenser lens 502, and the reference image captured by the normal image imaging element 503 is converted into a digital value and output as two-dimensional image data. The AD converter 504, the normal image memory 505 for storing the digitized normal image signal, the normal image signal output from the normal image memory 505, and the composite image signal output from the image composition unit 405 as a video signal. A video signal processing circuit 506 for converting and outputting is provided. The monitor 600 switches between a normal image and a composite image for display.
[0052]
Next, the operation of the fluorescence endoscope in the above embodiment will be described. First, an operation when an autofluorescence image and a reference image in two different wavelength bands are captured, and a composite image is generated and displayed from these images will be described.
[0053]
At the time of displaying the composite image, excitation light Lr is emitted from the GaN-based semiconductor laser 111 by the semiconductor laser power source 112 based on a signal from the control computer 200, and the excitation light Lr passes through the excitation light condensing lens 113. The light passes through the dichroic mirror 120, enters the excitation light guide 101 b, is guided to the distal end portion of the endoscope insertion unit 100, and is then irradiated onto the living tissue 9 from the illumination lens 103. The autofluorescence image from the living tissue 9 generated by the irradiation of the excitation light Lr is collected by the condensing lens 105, passes through the excitation light cut filter 104, is incident on the tip of the image fiber 102, passes through the image fiber 102, The light enters the collimating lens 301. The excitation light cut filter 104 is a long pass filter that transmits all fluorescence having a wavelength of 420 nm or more. Since the wavelength of the excitation light Lr is 410 nm, the excitation light reflected by the living tissue 9 is cut by the excitation light cut filter 104. The autofluorescence image transmitted through the collimator lens 301 is reflected by the dichroic mirror 303 in the right angle direction. Then, the light is transmitted by the half mirror 308 with a transmittance of 50% and reflected with a reflectance of 50%. The self-fluorescent image transmitted through the half mirror 308 is reflected by the fluorescent image mirror 313 in the perpendicular direction, formed by the broadband fluorescent image condensing lens 304, and transmitted through the broadband fluorescent image condensing lens 304. Is transmitted through the broadband band-pass filter 305 and imaged by the high-sensitivity imaging device 306 for broadband fluorescent images, and the video signal from the high-sensitivity imaging device 306 for broadband fluorescent images is input to the AD converter 307 and digitized. After that, it is stored in the wide-band autofluorescence image storage area of the autofluorescence image memory 401.
[0054]
In addition, the autofluorescence image reflected by the dichroic mirror 303 and reflected by the half mirror 308 is formed by the narrowband fluorescent image condenser lens 309, passes through the narrowband bandpass filter 310, and narrowband fluorescence image. The image signal from the high-sensitivity image sensor 311 for image pickup and the video signal from the high-sensitivity image sensor 311 for narrow-band fluorescence image are input to the AD converter 312 and digitized, and then the narrow-band private area of the self-fluorescence image memory 401 Stored in the fluorescence image area. The digital data of the autofluorescence image captured by the high-sensitivity image sensor 306 for broadband fluorescent images and the digital data of the autofluorescence image captured by the high-sensitivity image sensor 311 for narrowband fluorescence images are stored in different areas. . At this time, it is assumed that the movable mirror 302 is at the position of a broken line parallel to the optical axis of the autofluorescence image.
[0055]
Further, reference light Ls is emitted from the reference light source 117 by the reference light source 118, and this reference light Ls is transmitted through the reference light condensing lens 119 and is reflected by the dichroic mirror 120 in a right angle direction to generate the excitation light. After being incident on the guide 101b and guided to the distal end portion of the endoscope, the living tissue 9 is irradiated from the illumination lens 103. The reference image from the living tissue 9 generated by the irradiation of the reference light Ls is condensed by the condensing lens 105, and the reference image transmitted through the condensing lens 105 is transmitted through the excitation light cut filter 104 and the tip of the image fiber 102. And enters the collimating lens 301 via the image fiber 102. The excitation light cut filter is a long pass filter that transmits a reference image having a wavelength of 420 nm or more. The reference image that has passed through the collimator lens 301 passes through the dichroic mirror 303, is formed by the reference image condensing lens 314, is imaged by the reference image imaging device 315, and the video signal from the reference image imaging device 315 is After being input to the AD converter 316 and digitized, it is stored in the reference image memory 402. At this time, it is assumed that the movable mirror 302 is at a broken line position parallel to the optical axis of the reference image.
[0056]
In the autofluorescence image of the two wavelength bands stored in the autofluorescence image memory 401, the normalized fluorescence image generation means 403 adds an offset os1 to each pixel value of only the broadband autofluorescence image. Then, the ratio with the narrow-band autofluorescence image is calculated according to the following formula (13). Note that os1 is a predetermined value set in advance in the normalized fluorescent image generation unit 403.
[0057]
n / (w + os1) (13)
n: Narrow-band autofluorescence image
w: Broadband autofluorescence image
os1: Offset
Then, the normalized calculation value calculated by the normalized fluorescence image generation unit 403 is output to the correction unit 404, and the correction unit 404 calculates the corrected normalized calculation value according to the following equation (14).
[0058]
{N / (w + os1)} × {(nir + os2) / nir} (14)
os2 = os1 × knir / kw
knir: distance from the tip of the reference image / endoscope insertion unit 100 to the living tissue
kw: Broadband autofluorescence image / distance from the distal end of the endoscope insertion unit 100 to the living tissue Note that the correction function (nir + os2) / nir is a predetermined living tissue detected in advance by the fluorescence endoscope of the present embodiment. Calculated from the reference image and the broadband autofluorescence image and stored in the correction means 404 in advance. In this embodiment, a reference image and a broadband autofluorescence image are captured using a normal biological tissue as the predetermined biological tissue, and the correction function is calculated. After the correction function is set in advance as described above, the correction normalization calculation value and the distal end of the endoscope insertion unit 100 when the autofluorescence image and the reference image of the living tissue having the normal part and the lesion part are actually captured are taken. The relationship with the distance from the living tissue 9 is shown in FIG. In FIG. 2, white circles indicate correction standardization calculation values calculated for normal tissue and black circles indicate correction standardization calculation values calculated for lesional biological tissue. In the present embodiment, as described above, the correction function is calculated for the normal tissue of the normal part, so that the correction standardization calculation value calculated for the normal tissue of the normal part depends on the distance as shown in the figure. It is possible to take a substantially constant value. Therefore, at this time, for example, a boundary value is obtained based on the correction standardization calculation value indicated by a white circle, and the living tissue corresponding to the correction standardization calculation value equal to or higher than the boundary value is determined as a normal part, and is less than the boundary value The living tissue corresponding to the corrected normalization calculation value may be determined as a lesioned part. The boundary value is obtained, for example, by obtaining an average value of the corrected standardization calculation values indicated by the white circles for a plurality of types (a plurality of patients) of normal tissues, obtaining a standard deviation σ, and obtaining an average value −2σ or an average The value −3σ may be used as the boundary value. Whether to set −2σ or −3σ may be determined based on the relationship with the corrected normalization calculation value of the living tissue of the lesioned part.
[0059]
In addition, instead of calculating the correction function for the normal tissue of the normal part as in the present embodiment, the correction function may be calculated for the biological tissue of the lesioned part. The corrected standardization calculation value calculated for the living tissue can take a substantially constant value without depending on the distance. Therefore, at this time, for example, as described above, a boundary value is obtained based on the corrected normalized calculation value, and the living tissue corresponding to the corrected normalized calculation value larger than the boundary value is determined to be a normal part, and below this boundary value A living tissue corresponding to a certain correction normalization calculation value may be determined as a lesioned part. The boundary value is obtained, for example, by obtaining an average value of correction normalization calculation values of a living tissue of a lesioned part for a plurality of types (a plurality of patients) of normal tissues, obtaining a standard deviation σ, and calculating an average value + 2σ or an average The value + 3σ may be used as the boundary value.
[0060]
Then, color information is assigned to each pixel based on the result of the binary determination of the corrected normalized calculation value as described above by the color image calculation unit 405, and is output to the image synthesis unit 407 as a color image signal. On the other hand, for the reference image stored in the reference image memory 402, the luminance image calculation means 406 assigns luminance information to each pixel value, and generates and outputs a luminance image signal. The two image signals output from the color image calculating unit 405 and the luminance image calculating unit 406 are combined by the image combining unit 407. The synthesized image synthesized by the image synthesizing unit 407 is input to the monitor 600 after DA conversion by the video signal processing circuit 506, and the synthesized image is displayed.
[0061]
Next, the operation when displaying a normal image will be described. First, the white light source 114 emits white light Lw from the white light source 114 based on a signal from the control computer 200, and the white light Lw enters the white light light guide 101a via the white light condensing lens 116. After being guided to the distal end portion of the endoscope insertion portion 100, the living tissue 9 is irradiated from the illumination lens 103. The reflected light of the white light Lw is collected by the condenser lens 105, passes through the excitation light filter 104, enters the tip of the image fiber 102, enters the collimator lens 301 through the image fiber 102. The excitation light cut filter 104 is a long pass filter that transmits visible light having a wavelength of 420 nm or more. The reflected light that has passed through the collimator lens 301 is reflected by the movable mirror 302 and the normal image mirror 501 and is incident on the normal image condensing lens 502. The normal image transmitted through the normal image condensing lens 502 is formed on the normal image pickup element 503. The video signal from the normal image pickup device 503 is input to the AD converter 504, digitized, and stored in the normal image memory 505. The normal image signal stored in the normal image memory 505 is input to the monitor 600 after DA conversion by the video signal processing circuit 506 and displayed on the monitor 600 as a visible image.
[0062]
A series of operations relating to the combined image display operation and the normal image display operation are controlled by the control computer 200.
[0063]
The composite image display state and the normal image display state are switched by pressing the foot switch 2.
[0064]
According to the fluorescence endoscope in the above-described embodiment to which the normalized image generation method and apparatus according to the present invention is applied, the living tissue 9 and the endoscope are inserted using the correction function in the normalized fluorescence image (standardized calculation value). Since the correction of the normalized fluorescence image (corrected normalized calculation value) is performed by performing the distance correction for correcting the fluctuation amount of the normalized fluorescence image (normalized calculation value) due to the distance from the tip of the unit 100. The S / N of the normalized fluorescence image (standardized calculation value) can be improved by adding an offset, and without depending on the distance between the distal end of the endoscope insertion portion 100 and the living tissue 9, Even when the distance changes, a constant normalized calculation value can be obtained, so that the normal part and the lesion part in the living tissue can be more accurately identified by the image.
[0065]
For example, images obtained when distance correction is performed on the far point and near point images of the same sample shown in FIGS. 9A and 9B using the above correction function are shown in FIGS. ). Note that this image does not display the result of the binary determination as in the above-described embodiment, but is an image in which luminance is assigned according to the magnitude of the corrected standardization calculation value and displayed as a luminance image. As shown in the figure, compared to FIGS. 9A and 9B, the brightness difference between the image of the normal part between the image at the far point (FIG. 3A) and the image at the near point (FIG. 3B). It can be seen that the difference in contrast between the normal part and the lesion part is small.
[0066]
In the first embodiment, the correction standardization calculation value is calculated using the correction function based on the reference image according to the equations (9) and (10). A method for obtaining a correction function based on the reference image by another method and calculating a correction normalization calculation value using this correction function will be described below.
[0067]
First, as in the above embodiment, in order to obtain a correction function, a narrow-band autofluorescence image, a broadband autofluorescence image, and a reference image are captured in advance with respect to a normal tissue, and pixel values of each image are used for the autofluorescence image. The data is stored in the memory 401 and the reference image memory 402. The autofluorescence image of the two wavelength bands stored in the autofluorescence image memory 401 is added with the os1 to each pixel value of only the broadband autofluorescence image by the standardized fluorescence image generation means 403. Then, the ratio with the narrow-band autofluorescence image is calculated according to the following equation (15). Note that os1 is a predetermined value set in advance in the normalized fluorescent image generation unit 403.
[0068]
n / (w + os1) (15)
n: Narrow-band autofluorescence image
w: Broadband autofluorescence image
os1: Offset
The normalized calculation value calculated by the normalized fluorescent image generation unit 403 is output to the correction unit 404. On the other hand, the pixel value of the reference image stored in the reference image memory 402 is output to the correction unit 404. Based on the normalized calculation value output from the normalized fluorescent image generation unit 403 and the pixel value of the reference image output from the reference image memory 402, the correction unit 404 corrects the correction function hnf as shown in FIG. Find (NIR) and store. Here, hnf is a function based on the square of the reciprocal of the pixel value of the reference image captured at this time, and this function uses the distance NIR between the distal end of the endoscope insertion unit 100 and the living tissue as a variable. Is. The distance NIR is calculated from the square of the reciprocal of the pixel value of the reference image. Since the reference image is captured based on the reflected light by the irradiation of the reference light that hardly absorbs into the living tissue, the size of the pixel value is the distance between the tip of the endoscope insertion unit 100 and the living tissue. It depends on the size. Therefore, it can be said that hnf, which is a function based on the square of the reciprocal of the pixel value of the reference image, directly reflects the distance.
[0069]
When an actual biological tissue is imaged, the normalized calculation value is calculated according to the above equation (15) by the normalized fluorescence image generation means 403 from the actually captured narrowband autofluorescence image and broadband autofluorescence image. Then, the normalized calculation value and the pixel value of the reference image are output to the correcting unit 404, and the corrected normalized calculation value is output according to the following equation (16).
[0070]
nf × hnf (0) / hnf (NIR) (16)
Here, nf is a normalized calculation value based on the self-fluorescent image actually captured.
[0071]
By obtaining the correction standardization calculation value using the correction function as described above, as shown in FIG. 4B, the actual standardization calculation value (broken line) that depends on the distance is corrected without depending on the distance. It can be corrected to a value (solid line). The operation after calculating the corrected normalization calculation value is the same as that in the first embodiment.
[0072]
Next, a second embodiment of the fluorescence endoscope to which the standardized image generating apparatus that performs the standardized image generating method of the present invention is applied will be described. FIG. 5 is a diagram showing a schematic configuration of the present embodiment. In addition, about this Embodiment, the same number is attached | subjected about the element similar to 1st Embodiment, and the description is abbreviate | omitted unless there is particular need.
[0073]
The fluorescence endoscope according to the present embodiment has a configuration that does not use the reference light in the first embodiment.
[0074]
The image signal processing unit 3 includes an illumination unit 120 including two light sources that respectively emit normal image white light Lw and autofluorescence image excitation light Lr, and autofluorescence generated from the living tissue 9 by irradiation of the excitation light. An image detection unit 310 that captures an image Zj, converts it into a digital value and outputs it as two-dimensional image data, and performs a normalization operation from the two-dimensional image data of the autofluorescence image output from the image detection unit 310, An image calculation unit 410 that assigns color information to a normalized calculation value and outputs it as a color image signal, and converts a normal image into a digital value to form two-dimensional image data. The two-dimensional image data and an output signal of the image calculation unit 410 Display signal processing unit 500 for converting the video signal into a video signal and outputting the video signal, and for controlling the operation timing connected to each unit A computer 210, and a foot switch 2 for switching the normal image display state and the color image display state will be described later.
[0075]
The illumination unit 120 includes a GaN-based semiconductor laser 121 that emits excitation light Lr for autofluorescence images, a semiconductor laser power source 122 that is electrically connected to the GaN-based semiconductor laser 111, and a white light source that emits white light Lw for normal images. , And a white light power source 125 electrically connected to the white light source 124.
[0076]
The image fiber 102 is connected to the image detection unit 310, and the autofluorescence image propagated through the image fiber 102, the collimator lens 331 that forms a normal image, and the normal image that has passed through the collimator lens 331 are totally reflected at right angles. The fluorescent image transmitted through the collimating lens 331 transmits 50% of the light quantity of the movable mirror 332 that moves to and passes through the position indicated by the broken line, and the autofluorescent image (light having a wavelength of 750 nm or less) that has passed through the collimating lens 331, A self-image formed by a half-mirror 323 that reflects 50% in a perpendicular direction, a narrow-band fluorescent image condensing lens 324 that forms an auto-fluorescent image transmitted through the half-mirror 323, and a narrow-band fluorescent image condensing lens 324 Narrowband bandpass filter 325 for extracting wavelengths from 430 nm to 530 nm from the fluorescence image, narrowband band A high-sensitivity image sensor 326 for narrow-band fluorescence images that captures an auto-fluorescence image that has passed through the pass filter 325, and a two-dimensional image obtained by converting the auto-fluorescence image captured by the high-sensitivity image sensor 326 for narrow-band fluorescence images into a digital value. The AD converter 327 output as data, the self-fluorescent image reflected by the half mirror 323 in the perpendicular direction, the fluorescent image mirror 318 reflecting again in the perpendicular direction, and the auto-fluorescent image reflected in the perpendicular direction by the fluorescent image mirror 318 are combined. A broadband fluorescent image condensing lens 319 to be imaged, a broadband bandpass filter 320 for selecting a wavelength of 430 nm to 730 nm from an autofluorescent image transmitted through the broadband fluorescent image condensing lens 319, and autofluorescence transmitted through the broadband bandpass filter 320 High-sensitivity image sensor 321 for broadband fluorescent image that captures images, and high-sensitivity image sensor for broadband fluorescent image And a AD converter 322 which outputs autofluorescence image captured as the conversion to the two-dimensional image data into a digital value by 21.
[0077]
The image calculation unit 410 includes a broadband autofluorescence memory 411 that stores digitized broadband autofluorescence image signal data, a narrowband autofluorescence memory 412 that stores narrowband autofluorescence image signal data, and a broadband autofluorescence memory. A preset offset value is added to each pixel value of the broadband autofluorescence image stored in 411, and an operation is performed according to the ratio between the pixel value to which the offset is added and the pixel value of the narrowband autofluorescence image. Based on the pixel value of the broadband autofluorescence image stored in the standardized fluorescence image generation means 413 and the broadband autofluorescence memory 411 for calculating the standardization calculation value of each pixel, a correction function described later is calculated, and the correction function is calculated. Correction means 414 for correcting the normalized calculation value output from the normalized fluorescent image generation means 413 and calculating a corrected normalized calculation value; And a color image calculation unit 415 outputs the color image signals by assigning color information to the calculated corrected normalized calculation value by finely correcting means 414.
[0078]
The normalized fluorescence image generation means 413 adds an offset value to the pixel value of the autofluorescence image stored in the wideband autofluorescence memory 411, and performs a narrowband autofluorescence image and a broadband autofluorescence image according to the following equation (17). Calculate the ratio. In the present embodiment, the offset value is a predetermined value stored in the standardized fluorescent image generation unit 413 in advance.
[0079]
n / (w + os1) (17)
n: Narrow-band autofluorescence image
w: Broadband autofluorescence image
os1: Offset
The correction unit 414 obtains a correction function based on the pixel value of the broadband autofluorescence image and the normalized calculation value calculated by the normalized fluorescence image generation unit 413, and corrects the normalized calculation value by correcting the distance. The standardized operation value is output, details of which will be described later.
[0080]
The color image calculation unit 415 generates color images by assigning color information corresponding to the size of the correction pixel value calculated by the correction unit 414.
[0081]
Next, the operation of the fluorescence endoscope in the above embodiment will be described. First, an operation in the case where autofluorescence images in two different wavelength bands are taken and a color image is generated from these images and displayed will be described.
[0082]
At the time of displaying the color image, the excitation light Lr is emitted from the GaN-based semiconductor laser 121 by the semiconductor laser power source 122 based on the signal from the control computer 200, and the excitation light Lr passes through the excitation light condensing lens 123. After being incident on the excitation light guide 101 b and guided to the distal end portion of the endoscope insertion portion 100, the living tissue 9 is irradiated from the illumination lens 103. The autofluorescence image from the living tissue 9 generated by the irradiation of the excitation light Lr is collected by the condensing lens 105, passes through the excitation light cut filter 104, is incident on the tip of the image fiber 102, passes through the image fiber 102, The light enters the collimating lens 331. The excitation light cut filter 104 is a long pass filter that transmits all fluorescence having a wavelength of 420 nm or more. Since the wavelength of the excitation light Lr is 410 nm, the excitation light reflected by the living tissue 9 is cut by the excitation light cut filter 104. The autofluorescence image transmitted through the collimator lens 331 is transmitted by the half mirror 313 with a transmittance of 50% and reflected with a reflectance of 50%. The autofluorescence image reflected from the half mirror 313 is reflected by the fluorescence image mirror 318 in the right angle direction, formed by the broadband fluorescent image condenser lens 319, and transmitted through the broadband fluorescent image condenser lens 319. Is transmitted through the broadband bandpass filter 320 and imaged by the broadband fluorescent image high-sensitivity image sensor 321, and the video signal from the broadband fluorescent image high-sensitivity image sensor 321 is input to the AD converter 322 and digitized. And stored in the broadband autofluorescence memory 411.
[0083]
The self-fluorescent image transmitted through the half mirror 313 is formed by the narrow-band fluorescent image condensing lens 314, passes through the narrow-band band-pass filter 315, and is captured by the narrow-band fluorescent image high-sensitivity image sensor 316. The video signal from the high-sensitivity image sensor 316 for narrow-band fluorescent images is input to the AD converter 317, digitized, and stored in the narrow-band autofluorescence memory 412. The digital data of the autofluorescence image captured by the high-sensitivity image sensor 321 for the broadband fluorescent image and the digital data of the autofluorescence image captured by the high-sensitivity image sensor 316 for the narrowband fluorescence image are different areas in a common memory. It may be stored in. At this time, it is assumed that the movable mirror 332 is positioned at a broken line parallel to the optical axis of the autofluorescence image.
[0084]
Here, in the present embodiment, as in the above-described embodiment, in order to obtain a correction function, a narrowband autofluorescence image and a broadband autofluorescence image are captured in advance for a normal tissue, and the pixel value of each image is set to a wideband. The data is stored in the fluorescence memory 411 and the narrowband fluorescence memory 412. The broadband autofluorescence image stored in the broadband fluorescence memory 411 is added with the os1 to each pixel value in the normalized fluorescence image generation means 413. Then, the ratio with the narrow-band autofluorescence image is calculated according to the following equation (18). Note that os1 is a predetermined value set in advance in the normalized fluorescent image generation unit 413.
[0085]
n / (w + os1) (18)
n: Narrow-band autofluorescence image
w: Broadband autofluorescence image
os1: Offset
The normalized calculation value calculated by the normalized fluorescent image generation unit 413 is output to the correction unit 414. On the other hand, the pixel value of the broadband autofluorescence image stored in the broadband fluorescence memory 411 is output to the correction means 414. The correction unit 414 performs correction as shown in FIG. 6A based on the normalized calculation value output from the normalized fluorescence image generation unit 413 and the pixel value of the broadband autofluorescence image output from the broadband fluorescence memory 411. The function hnf (w) is obtained and stored. Here, w is a pixel value of the broadband autofluorescence image captured at this time. W _∞ Is the size of the pixel value of the broadband autofluorescence image when the distance between the distal end of the endoscope insertion portion 100 and the living tissue 9 is substantially 0 (when it is very close), and is a sufficiently large value. It is.
[0086]
When an actual biological tissue is imaged, a normalized calculation value is calculated by the normalized fluorescence image generation unit 413 according to the above equation (18) from the actually captured narrowband autofluorescence image and broadband autofluorescence image. Then, the normalized calculation value and the pixel value of the broadband autofluorescence image are output to the correcting means 414, and the corrected normalized calculation value is output according to the following equation (19).
[0087]
nf × hnf (w _∞ ) / Hnf (w) (19)
Here, nf is a normalized calculation value based on the actually captured autofluorescence image, and hnf (w) is a value of the correction function hnf (w) in the pixel value w of the actually captured broadband autofluorescence image. It is.
[0088]
By obtaining the correction standardization calculation value using the correction function as described above, as shown in FIG. 6B, the actual standardization calculation value (broken line) depending on the distance is corrected and the correction standardization calculation independent of the distance It can be corrected to a value (solid line).
[0089]
In the present embodiment, since the correction function is calculated for the living tissue of the normal part as described above, the boundary value is obtained based on the correction normalization calculation value as in the first embodiment, and this The biological tissue corresponding to the corrected standardized calculation value equal to or greater than the boundary value may be determined as a normal part, and the biological tissue corresponding to the corrected standardized calculation value that is less than the boundary value may be determined as a lesioned part. The boundary value may be obtained in the same manner as in the first embodiment. Further, the method for obtaining the boundary value when calculating the correction function for the living tissue of the lesioned part is the same as that in the first embodiment.
[0090]
Then, color information is assigned to each pixel based on the result of the binary determination of the corrected normalized calculation value as described above by the color image calculation means 415, and a color image signal is generated. The color image signal is input to the monitor 600 after DA conversion by the video signal processing circuit 506, and a color image is displayed. Other operations are the same as those in the first embodiment.
[0091]
In the second embodiment, the correction function is obtained from the relationship between the normalized calculation value based on the autofluorescence image and the pixel value of the wideband autofluorescence image. The correction function may be obtained from the relationship between the pixel value and the normalized calculation value, or the relationship between the sum of the pixel values of the broadband autofluorescence image and the narrowband autofluorescence image and the normalized calculation value.
[0092]
In the first and second embodiments, a correction function is obtained based on the living tissue of a normal part or a lesioned part, a correction normalization calculation value is calculated using the correction function, and this correction normalization is performed. Although the binary value is determined by setting the boundary value based on the calculated value, the correction function is based on the biological tissue at each stage when the progress from the normal part to the lesioned part is divided into s stages. S correction standardization calculation values serving as a reference may be calculated based on the correction function, and boundary values may be set based on the s correction standardization calculation values serving as a reference. . In this case, with respect to the actually captured biological tissue, correction standardization calculation values are calculated using s correction functions corresponding to the respective biological tissues in the s stages, and the s correction standardization calculation values are calculated. The region corresponding to the s-stage biological tissue is extracted from the s corrected standardized fluorescent images using the boundary values, and different color images are assigned to the s regions, for example, and the color images are overlaid. May be displayed. In this case, for example, when applied to the first embodiment, a boundary value setting unit 420 and a region extraction unit 421 are provided between the correction unit 404 and the color image calculation unit 405 as shown in FIG. The image composing unit 407 may superimpose regions corresponding to the s-stage biological tissue output from the color image calculating unit 405.
[0093]
In the first and second embodiments, the correction function is calculated using the normal tissue or the diseased tissue, and a plurality of patient tissues are used as the living tissue. May be. The correction function may be calculated and set in advance before actual image diagnosis, or may be calculated and set during actual image diagnosis, or may be updated. Good.
[0094]
In the first and second embodiments, the composite image and the normal image or the color image and the normal image are switched and displayed on one monitor, but may be displayed on separate monitors. Good. Further, the method of switching with one monitor may be automatically performed in time series from the control computer without using the foot switch as in the above embodiment.
[0095]
In addition, although the GaN-based semiconductor laser 114 and the white light source 111 are configured separately, a single light source can be used as both the excitation light source and the white light source by using an appropriate optical transmission filter.
[0096]
The excitation light source may be any wavelength of about 400 nm to 420 nm.
[0097]
In the first and second embodiments, the normalized image generation method of the present invention is applied when generating a normalized fluorescence image based on autofluorescence emitted from a biological tissue by irradiation with excitation light. However, the present invention is not limited to the case where the standardized fluorescence image is generated based on the autofluorescence as described above, but, for example, based on the reflected light reflected by the irradiation of the white light to the living tissue. The normalized image generation method of the present invention can also be applied when calculating the oxygen saturation of a living tissue and generating a normalized image based on the oxygen saturation. Specifically, this will be described below.
[0098]
First, a method for calculating the oxygen saturation will be described. The living tissue 9 is irradiated with white light from the endoscope insertion portion 100 of the fluorescence endoscope as described above. At this time, the reflected light reflected from the living tissue is imaged by the imaging device via the bandpass filters having different wavelength bands λ1 and λ2, thereby obtaining reflected images r1 and r2 having different wavelength bands λ1 and λ2. Then, with respect to the two reflected images r1 and r2, the absorbance for each pixel is calculated based on each pixel value and the intensity of the white light irradiated on the living tissue.
[0099]
Here, since the absorbance changes according to the oxygen saturation of the living tissue, the oxygen saturation can be calculated from the absorbance. However, the absorbance varies depending on the pulsation of the living tissue. Therefore, for example, a pulse meter is provided, and based on the pulsation measured by the pulse meter, for example, a time T1 at which the absorbance is the largest and a time T2 at which the absorbance is the smallest are obtained, and the time T1 and the time T2 are obtained. The change in absorbance is calculated, and the oxygen saturation is calculated based on this change.
[0100]
That is, for each pixel, the absorbance Iλ1 (T1) with respect to the irradiation light in the wavelength band λ1 at time T1 is obtained based on the reflected image r1 at time T1, and the absorbance Iλ2 (T1) with respect to the irradiation light in the wavelength band λ2 at time T1. Is obtained based on the reflected image r2 at time T1, and the absorbance Iλ1 (T2) for the irradiation light in the wavelength band λ1 at time T2 is obtained based on the reflected image r1 ′ at time T2, and the irradiation in the wavelength band λ2 at time T2. Absorbance Iλ2 (T2) with respect to light is obtained based on reflection image r2 ′ at time T2, and absorbance changes ΔIλ1 and ΔIλ2 are obtained as follows.
[0101]
ΔIλ1 = Iλ1 (T1) −Iλ (T2)
ΔIλ2 = Iλ2 (T1) −Iλ (T2)
Then, the oxygen saturation SaO based on the time variation ΔIλ1 and ΔIλ2 of the absorbance for the two wavelength bands λ1 and λ2. ₂ Is obtained by the following equation.
[0102]
SaO ₂ = F (Φ12)
Φ12 = ΔIλ1 / ΔIλ2
Note that f is Φ12 and SaO obtained by experiment. ₂ It is a function obtained based on the relationship.
[0103]
Here, since the absorbance changes ΔIλ1 and ΔIλ2 are very small values as in the case of the autofluorescence, Φ12 is obtained based on these values to obtain the oxygen saturation SaO. ₂ When this is calculated and imaged, an image with a poor S / N is obtained. Therefore, as in the case of imaging the autofluorescence, for example, it is conceivable to obtain Φ12 ′ by adding an offset as follows.
[0104]
Φ12 ′ = ΔIλ1 / (ΔIλ2 + os3)
However, os3: Offset
However, when Φ12 ′ is obtained by adding an offset as described above, the value of Φ12 ′ is set to the endoscope insertion unit 100 as in the case of calculating a normalized fluorescence image by adding an offset to the autofluorescence image. Therefore, it is difficult to appropriately indicate the oxygen saturation of the living tissue based on Φ12 ′. Therefore, as in the above embodiment, Φ12 ′ is corrected by the correction function, and the oxygen saturation SaO is based on this corrected value. ₂ By calculating and imaging, it is possible to display an image in which the oxygen saturation is appropriately shown without depending on the distance between the distal end of the endoscope insertion unit 100 and the living tissue. Further, as the correction function, for example, the following equation (20) is used, and Φ12 ′ is corrected by performing calculations according to equations (21) and (22). Good.
[0105]
(Wir + os4) / Wir (20)
Wir: Absorbance when irradiated with reference light
os4: Correction coefficient
{ΔIλ1 / (ΔIλ2 + os3) × {(Wir + os4) / Wir} (21)
os4 = os3 × hir / hw (22)
hir: Absorbance when the reference light is irradiated / Distance from the distal end of the endoscope insertion portion 100 to the living tissue 9
hw: ΔIλ2 / distance from the distal end of the endoscope insertion unit 100 to the living tissue 9
In addition, the correction function is not limited to the above equation (20), and other correction functions described in the above embodiments may be applied.
[0106]
As a method for obtaining the reflection images r1 and r2 of the different wavelength bands λ1 and λ2, the two reflection images r1 and r2 are time-sequentially used by using a field sequential filter including bandpass filters of the wavelength bands λ1 and λ2. Alternatively, two reflection images r1 and r2 may be obtained simultaneously using a mosaic filter composed of bandpass filters of wavelength bands λ1 and λ2.
[0107]
The reflected images r1 and r2 may be captured at the time of capturing a normal image that irradiates a living tissue with white light.
[Brief description of the drawings]
FIG. 1 is a schematic configuration diagram of a first embodiment of a fluorescence endoscope to which a normalized image generation method and apparatus according to the present invention is applied.
FIG. 2 shows a relationship between a corrected normalization calculation value and a distance from a distal end of an endoscope insertion portion to a living tissue when an autofluorescence image and a reference image of a living tissue having a normal part and a lesioned part are captured. Figure
3 is a diagram illustrating an image when distance correction is performed using a correction function for the far-point image (A) and the near-point image (B) illustrated in FIG. 9;
FIG. 4 is a diagram showing a relationship (A) between a normalized calculation value based on a narrowband autofluorescence image and a broadband autofluorescence image and a reciprocal NIR of a pixel value of a reference image, and correction by a correction function (B).
FIG. 5 is a schematic configuration diagram of a second embodiment of a fluorescence endoscope to which a normalized image generation method and apparatus according to the present invention is applied.
FIG. 6 is a diagram showing a relationship (A) between a normalized calculation value based on a narrowband autofluorescence image and a broadband autofluorescence image and a pixel value of the broadband autofluorescence image, and correction (B) using a correction function.
FIG. 7 sets a boundary value by a boundary value setting unit based on a corrected standardized calculation value corresponding to an s-stage biological tissue calculated by the correction unit, and based on the boundary value, the region extraction unit sets an s-stage. Configuration diagram of each means when extracting a region corresponding to a living tissue and assigning different color images to this region to generate a color image
FIG. 8 is a diagram showing the relationship between the normalized calculation value and the distance between the detection means and the biological tissue when the normalized calculation value is obtained by adding an offset to the normal tissue and the lesion biological tissue;
FIG. 9 shows an image (A) based on a division value when the distance between the detection means and the living tissue is large (far point) and an image based on a division value when the distance is small (near point) (B Figure showing
FIG. 10 is an explanatory view showing the intensity distribution of the fluorescence spectrum of the living tissue of the normal part and the lesion part.
[Explanation of symbols]
1 Fluorescence diagnostic equipment
2 Foot switch
9 Living tissue
100 Endoscope insertion part
101 Light guide
101a White light guide
101b Excitation light guide
102 Image fiber
103 Illumination lens
104 Excitation light cut filter
105 Objective lens
110, 120 lighting unit
111,121 GaN semiconductor laser
112,122 Power supply for semiconductor laser
113, 123 Condensing lens for excitation light
114,124 White light source
115,125 Power supply for semiconductor laser
116,126 Condensing lens for white light
200, 210 Control computer
300, 310 Image detection unit
301 collimating lens
302 Movable mirror
303 Dichroic Mirror
304,319 condenser lens for broadband fluorescent image
305, 320 Wideband bandpass filter
306,321 High-sensitivity image sensor for broadband fluorescent image
307, 312, 316, 322, 327, 504 AD converter
308,323 half mirror
309,324 Mirror for narrow band fluorescent image
310,325 Narrowband bandpass filter
311 and 326 High-sensitivity image sensor for narrow-band fluorescence image
313 Mirror for fluorescent image
314 Condensing lens for reference image
315 Image sensor for reference image
331 Collimating lens
332 Movable mirror
400 Image arithmetic unit
401 Self-fluorescent image memory
402 Reference image memory
403, 413 Normalized fluorescent image generation means
404, 414 Correction means
405,415 color image calculation means
406 Luminance image calculation means
407 Image composition means
500 Display signal processing unit
501 Normal image mirror
502 Condensing lens for normal image
503 Image sensor for normal images
505 Normal image memory
506 Video signal processing circuit
600 Monitor unit

Claims (9)

Based on the fluorescence generated from the living tissue by the irradiation of the excitation light , a broadband fluorescent image and a narrow-band fluorescent image are detected by the fluorescent image detection means, respectively, and a desired offset is added to the broadband fluorescent image, and the desired offset In a standardized image generation method for generating a standardized fluorescent image based on a ratio between the broadband fluorescent image to which is added and the narrowband fluorescent image ,
A reference image detection unit that detects a reference image by reflected light reflected from a predetermined biological tissue whose properties are known by irradiation of reference light is detected in advance.
Based on the detected reference image, a correction function represented by the following equation (1) is calculated:
A standardized image generating method characterized by generating a corrected standardized fluorescent image represented by the following equation (2).
(Nir + os2) / nir ... (1)
{N / (w + os1)} × {(nir + os2) / nir} ... (2)
However,
n: Narrow-band fluorescence image
w: Broadband fluorescence image
nir: reference image
os1: Offset
os2: correction coefficient, os2 = os1 × knir / kw
knir: distance between reference image / biological tissue and reference image detection means
kw: Broadband fluorescent image / distance between biological tissue and fluorescent image detection means Wherein os2 is the knir and normalized image generation method according to claim 1, characterized in that it is calculated based on the kw corresponding to the normal portion of the living tissue. Wherein os2 is the knir and normalized image generation method according to claim 1, characterized in that it is calculated based on the kw corresponding to the lesion of the biological tissue. The os2 is calculated based on each knir and each kw corresponding to the living tissue at each stage when the degree of progression from a normal part to a lesioned part is divided into s stages. Item 2. A normalized image generation method according to Item 1 . Using the s correction functions based on the os2 corresponding to the s-stage biological tissues, the distance correction is performed on the normalized fluorescent image serving as a reference for the s-stage biological tissues to obtain a reference. S number of the corrected normalized fluorescent images respectively, and a boundary value is set based on the s reference corrected normalized fluorescent images, while the s number of the corrected normalized fluorescent images of the living tissue By performing the distance correction using a correction function, s corrected standardized fluorescent images are calculated, and the boundary values are used for the s corrected standardized fluorescent images according to the s-stage biological tissue. 5. The standardized image generation method according to claim 4 , wherein the extracted area is extracted and the extracted s areas are superimposed and displayed. An excitation light irradiating means for irradiating the living tissue with excitation light, and a fluorescence image detecting means for detecting a broadband fluorescent image and a narrowband fluorescent image , respectively, based on the fluorescence generated from the biological tissue by the irradiation of the excitation light; A standardized fluorescent image generating means for adding a desired offset to the broadband fluorescent image and generating a standardized fluorescent image based on a ratio between the broadband fluorescent image to which the desired offset is added and the narrowband fluorescent image ; In the standardized image generating apparatus provided,
A reference light irradiating means for irradiating a predetermined biological tissue whose property is known;
Reference image detection means for detecting a reference image in advance based on reflected light reflected from a biological tissue whose properties are known by irradiation of reference light by the reference light irradiation means;
Using the correction function represented by the following equation (1 ) calculated based on the reference image detected in advance by the reference image detecting means , a corrected standardized fluorescent image represented by the following equation (2) is generated. A standardized image generating apparatus comprising a correcting unit that performs the correction.
(Nir + os2) / nir ... (1)
{N / (w + os1)} × {(nir + os2) / nir} ... (2)
However,
n: Narrow-band fluorescence image
w: Broadband fluorescence image
nir: reference image
os1: Offset
os2: correction coefficient, os2 = os1 × knir / kw
knir: distance between reference image / biological tissue and reference image detection means
kw: Broadband fluorescent image / distance between biological tissue and fluorescent image detection means The normalized image generation apparatus according to claim 6 , wherein the os2 is calculated based on the knir and the kw corresponding to a lesioned part of a living tissue of a lesioned part. The os2 is calculated based on each knir and each kw corresponding to the living tissue at each stage when the degree of progression from a normal part to a lesioned part is divided into s stages. Item 7. The normalized image generating device according to Item 6 . The correction means uses the s correction functions based on the os2 corresponding to the s-stage biological tissues to generate s standardized fluorescent images serving as a reference for the s-stage biological tissues. Performing the distance correction and calculating the corrected standardized fluorescence image as the s number of references,
Boundary value setting means for setting a boundary value based on the s reference correction normalized fluorescent images;
From the s corrected normalized fluorescent images calculated using the s correction functions for the normalized fluorescent image of the biological tissue by the correcting means, the s-stage biological tissue is respectively obtained using the boundary value. Area extracting means for extracting a corresponding area;
9. The normalized image generating apparatus according to claim 8, further comprising display means for displaying the s corrected normalized fluorescent image areas extracted by the area extracting means in a superimposed manner.

Images