JP2002172082A - Method and device for fluorescent image display - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a method and device for fluorescent image display, which displays an image including information relating to the shape of a measured part together with information based on fluorescent light reflecting the tissue characteristics of the measured part to reduce a sense of incompatibility of the observer. SOLUTION: Excitement light L2 is irradiated on a measured part 50 to image fluorescent images by different two wave lengths based on the intensity of fluorescent light L3 generated by the measured part 50. White light L1 is irradiated on the measured part 50 as reference light to image a reflection light image based on the intensity of the reflection light L4 reflected by the measured part 50. Hue is assigned to calculated images based on divided values of pixel values of fluorescent images in the different two wave lengths to generate a tissue characteristics image to mainly show tissue characteristics of the measured part 50. Brightness is assigned to reflection light images to generate a tissue shape image to mainly show the tissue shape of the measured part 50. The tissue characteristics image and the tissue shape image are integrated to generate a synthetic image to display the synthetic image.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a fluorescent image display method and a fluorescent image display device for detecting fluorescence generated from a living tissue by irradiation with excitation light and displaying the detected image as information representing information on the living tissue.

[0002]

2. Description of the Related Art Conventionally, when excitation light of a predetermined wavelength band is applied to a living body to be measured, a predetermined intensity is applied to the living body to be measured by taking advantage of the fact that the intensity of fluorescence emitted between normal tissue and diseased tissue is different. There has been proposed a technique of irradiating excitation light having a wavelength and receiving fluorescence emitted from a living body to be measured to display the localization / infiltration range of a diseased tissue as a fluorescence image.

Normally, when excitation light is irradiated, as shown in FIG. 21, normal tissue emits strong fluorescent light and diseased tissue emits weak fluorescent light. Can be determined.

[0004] This type of fluorescent image display apparatus basically includes an excitation light irradiating means for irradiating excitation light to a living body to be measured, a fluorescent image obtaining means for obtaining a fluorescent image from fluorescent light emitted from a living tissue, Display means for receiving the output of the fluorescence image acquisition means and displaying the fluorescence image, and is often incorporated into an endoscope inserted into a body cavity, a colposcope, a surgical microscope, or the like. It is composed in a shape.

In the above-described fluorescent image display apparatus, the distance from the excitation light irradiation system to the living body to be measured is not uniform due to the unevenness of the part of the living body. Are generally non-uniform. The fluorescence intensity is almost proportional to the illuminance of the excitation light, and the illuminance of the excitation light decreases in inverse proportion to the square of the distance. Therefore, a diseased tissue located closer to a normal tissue that is farther from the light source may emit stronger fluorescence.If the observer makes a determination based only on the fluorescence intensity, the determination of the lesion state may be erroneous. obtain.

In order to reduce such inconveniences, the present applicant has developed a narrow band fluorescent image near 480 nm in the wavelength band where the difference between the fluorescence intensity emitted from normal tissue and the fluorescence intensity emitted from diseased tissue is large, and a visible wavelength. The present invention proposes a fluorescent image display device that captures a broadband fluorescent image of a band, obtains a divided value of the light intensity of the narrowband fluorescent image and the light intensity of the broadband fluorescent image, and displays a pseudo color image based on the divided value. ing.

That is, the above-described division cancels the term of the fluorescence intensity depending on the distance between the excitation light source and the fluorescence receiving section and the living body measurement section, and a display reflecting only the difference in the shape of the fluorescence spectrum is obtained.

On the other hand, the applicant of the present application has proposed a ratio of the light intensity of the excitation light received by a part of the living tissue to the light intensity of the fluorescence emitted from the part by receiving the excitation light, ie, the excitation light. A method has also been proposed in which the tissue property of the measured portion is identified by obtaining a value that reflects the fluorescence yield, which is a value that is not affected by the irradiation distance or angle.

When obtaining a value reflecting the above fluorescence yield,
Since the excitation light is not uniformly absorbed by various living tissues, measuring the intensity distribution of the reflected excitation light does not mean that the intensity distribution of the excitation light received by the living tissue is correctly measured.

Therefore, as one measure for determining the fluorescence yield, near-infrared light that is uniformly absorbed by various living tissues is irradiated onto living tissues as reference light, and the reflected light of the reflected reference light is reflected. Is imaged as a reflection image, and the light intensity is used instead of the light intensity of the excitation light received by the living tissue,
There has been proposed a fluorescent image display device which obtains a division value of a light intensity of a fluorescence image and a reflection image and displays a pseudo color image based on the division value.

That is, the above-described division cancels the term of the fluorescence intensity depending on the distance between the excitation light source and the fluorescence receiving section and the living body measurement section, and a display reflecting only the difference in the fluorescence yield is obtained.

[0012]

However, as described above, by performing the division between the fluorescence images or the division between the fluorescence image and the reflection image, the pseudo-color image in which the distance information has been canceled is related to the fluorescence emitted from the living tissue. Although the information includes information, it becomes a composite image in which information relating to the shape of the living tissue, which is valuable information in determining the lesion state, is omitted. For the observer, a composite image giving a flat impression without any unevenness has a strong sense of discomfort.

The present invention has been made in view of the above circumstances, and displays an image including information on the shape of the measured section together with information on the fluorescence emitted from the measured section irradiated with the excitation light. It is an object of the present invention to provide a fluorescent image display method and apparatus which do not give an uncomfortable feeling to an observer.

Further, in order to display an image including information on the shape of the portion to be measured as described above, a luminance image is generated based on the light intensity of the reflection image, and the pseudo color image and the luminance image are synthesized. When the diagnostic image is obtained, when the intensity of the reflected light due to the irradiation of the reference light is weak,
Since the luminance image based on the intensity of the reflected light is a monochrome image, the entire image may be dark and the measurement target may not be visible. For example, when the fluorescence image display device according to the above technology is configured to be incorporated into an endoscope inserted into the body cavity, a colposop or a surgical microscope, or the like, the measurement target portion targeted for the body insertion portion When it is inserted to the vicinity, the distance from the part to be measured ranges from several mm to about 50 mm because the tip is not fixed. Therefore,
If the distance between the distal end of the insertion part and the part to be measured is large, the intensity of the reflected light due to the irradiation of the reference light becomes weak as described above, and as a result, the entire luminance image becomes dark and the measurement target is visually recognized. It will not be possible.

The present invention has been made in view of the above circumstances, and has a fluorescent image display capable of displaying a diagnostic image having sufficient brightness even when the intensity of reflected light due to irradiation of reference light is low. It is intended to provide a device.

[0016]

According to a first fluorescent image display method of the present invention, a fluorescent image based on the intensity of the fluorescent light generated from the measured portion by irradiating the measured portion with the excitation light is captured. At least one of the color information and the luminance information is assigned to the calculated image based on the image to generate a tissue property image mainly indicating the tissue property of the part to be measured, and the color information and the brightness information assigned to the tissue property image in the fluorescence image At least one of the color information and the luminance information according to is generated to generate a tissue shape image mainly indicating the tissue shape of the measured part, and the composite image is generated by combining the tissue property image and the tissue shape image. , And a composite image is displayed.

Here, the "calculated image based on the fluorescent image" can be, for example, an image calculated from the ratio of the fluorescent images in a plurality of different wavelength bands, and a division value is used as the ratio. Can be. Note that the division value includes a value calculated by an operation similar to division, such as a value obtained by adding a correction value to a pixel value of a fluorescent image and a value obtained by performing a mathematical process on the divided value. It is a thing. Further, the calculated image may be the fluorescent image itself.

The "color information" is, for example, a color system (HSB / HVC / Lab / Luv / La ^* b ^* / L).
u ^* v ^* color space) or hue, saturation, chromaticity (hue and saturation) of mixed color system (XYZ color space), color difference of video signal represented by TV signal, etc. (for example, IQ of YIQ of NTSC signal) , YCbCr
CbCr).

The "luminance information" is, for example, a color system (HSB / HVC / Lab / Luv / La ^* b ^* /
Lu ^* v ^* color space) and lightness of mixed color system (XYZ color space)
Luminance, luminance of a video signal represented by a TV signal, etc. (for example,
Y of the NTSC signal, Y of YCbCr, etc.).

The above-mentioned "assigning at least one of the color information and the luminance information to the operation image" means that the hue, saturation, chromaticity, color difference, lightness, etc., differ according to the pixel value of the operation image. This means that a numerical value indicating at least one is assigned to each pixel.

The above-mentioned "assigning at least one of the color information and the luminance information corresponding to the color information and the luminance information assigned to the tissue characterization image to the fluorescent image" is assigned to the operation image as described above. Considering the combination of color information and luminance information, the appropriate hue,
This means that at least one of saturation, chromaticity, color difference, lightness, and the like is assigned.

According to a second fluorescent image display method of the present invention, a fluorescent image based on the intensity of fluorescent light generated from a measured section is irradiated by irradiating excitation light to the measured section, and reference light is applied to the measured section. A reflection image based on the intensity of the reflected light reflected from the measured part by irradiation is captured, and at least one of the color information and the luminance information is assigned to the calculated image based on the fluorescence image, and mainly the tissue properties of the measured part are measured. Is generated, and at least one of color information and luminance information corresponding to the color information and the luminance information assigned to the tissue characteristic image is assigned to the reflection image, and the tissue mainly indicates the tissue shape of the portion to be measured. It is characterized in that a shape image is generated, a tissue property image and a tissue shape image are combined to generate a combined image, and the combined image is displayed.

Here, the "calculated image based on the fluorescent image" is, for example, an image calculated from the ratio of the fluorescent images in a plurality of different wavelength bands, or the ratio between the fluorescent image in a predetermined wavelength band and the anti-image. The ratio can be calculated, and a division value can be used as the ratio. The division value was calculated by an operation similar to division, such as a value obtained by adding a correction value to a pixel value of a fluorescence image or a reflection image and a value obtained by performing a mathematical process on the divided value. It also includes the value. Further, the calculated image may be the fluorescent image itself.

The above-mentioned "assigning at least one of the color information and the luminance information according to the color information and the luminance information assigned to the tissue characterization image to the reflection image" is assigned to the operation image as described above. Considering the combination of color information and luminance information, the hue,
This means that at least one of saturation, chromaticity, color difference, lightness, and the like is assigned.

In the first and second fluorescent image display methods, the statistic of the pixel value of one of the captured images is calculated, and the display gradation of the luminance information is assigned based on the statistic. You can do so.

Here, the "image of any one of the photographed images" means the calculated image or the fluorescent image in the first fluorescent image display method and the calculated image in the second fluorescent image display method. , The fluorescence image or the reflection image.

The "statistical value" means, for example, an average value, a standard deviation, a maximum value and a minimum value, but may be any value indicating a statistical amount.

"Assigning the display gradation of the luminance information based on the statistic" refers to the statistic or a combination of the statistic (for example, an average value and a standard deviation, an average value and a maximum value,
Average, maximum and minimum, combination of average, standard deviation, maximum and minimum, maximum and standard deviation, maximum and minimum, maximum, minimum and standard deviation) This means that a numerical value indicating the display gradation of the luminance information is assigned to each pixel of the photographed one of the images used in the calculation of the statistic according to the size of the pixel value.

Further, the statistic can be calculated from a desired region of a part of any one of the captured images.

Here, the "partial desired area" means an image area to be observed with particular attention in the range of any of the captured images.

Further, a predetermined coefficient is calculated based on the statistic, and the calculated coefficient is multiplied by one of the picked-up images, and luminance information is added to one of the picked-up images multiplied by the coefficient. Of display gradations can be assigned.

Here, "calculating a predetermined coefficient based on a statistic" means that the coefficient is calculated by, for example, equation (1). The average value of the pixel values of any of the captured images is m, the standard deviation is σ, an arbitrary constant a,
If b and c, the upper limit of the display gradation of the luminance information × a × (m + b × σ) × c (1) Then, the display of the luminance information on one of the captured images multiplied by the coefficient “Assign gradation” means, as shown in FIG. 19, a pixel value distribution 10 multiplied by a coefficient to obtain a pixel value distribution 20, and a value indicating the display gradation of luminance information for the value of the pixel value distribution 20. This means that a numerical value indicating luminance information is assigned according to the tone processing function 30.

Further, a gradation processing function indicating the display gradation of the luminance information is determined based on the statistics, and the display of the luminance information is performed on any of the captured images based on the determined gradation processing function. A gradation can be assigned.

Here, "determining a gradation processing function indicating a display gradation of luminance information based on a statistic" means, for example,
As shown in FIG. 20, when the pixel value distribution 10 of any one of the captured images with respect to the gradation processing function 30 is distributed as shown in FIG.
It means to change to 0. In other words, it means that the gradation processing function is rewritten so that the upper limit of the display gradation of the luminance information is Δm + b × σ, and the lower limit of the display gradation of the luminance information is Δm−b × σ. F (x), Min =
When m−b × σ and Max = m + b × σ, f (x)
To f (x-Min / (Max-Min)).

"To assign a display gradation of luminance information to any one of the captured images based on the determined gradation processing function" means that the gradation processing function 40 Means that a numerical value indicating the display gradation of the luminance information is assigned according to.

The first fluorescent image display device of the present invention comprises: a fluorescent image pick-up means for picking up a fluorescent image based on the intensity of the fluorescent light generated from the measured portion by irradiating the measured portion with excitation light; A tissue property image generating means for allocating at least one of the color information and the luminance information to the calculated image based on the image and generating a tissue property image mainly indicating the tissue property of the measured portion; and assigning the fluorescence property image to the tissue property image. A tissue shape image generating means for allocating at least one of the color information and the brightness information according to the color information and the brightness information to generate a tissue shape image mainly indicating a tissue shape of the measured part; a tissue property image and a tissue shape image And a display means for displaying a composite image generated by the composite image generation means.

The second fluorescent image display apparatus of the present invention comprises: a fluorescent image pick-up means for picking up a fluorescent image based on the intensity of the fluorescent light generated from the part to be measured by irradiating the part to be measured with excitation light; Reflection image capturing means for capturing a reflection image based on the intensity of the reflected light reflected from the measurement target by irradiating the measurement target with light, and applying at least one of the color information and the luminance information to the calculation image based on the fluorescence image. A tissue property image generating means for allocating and generating a tissue property image mainly indicating the tissue property of the measured portion, and at least color information and luminance information corresponding to the color information and the luminance information assigned to the tissue property image to the reflection image A tissue shape image generating means for allocating one and generating a tissue shape image mainly indicating the tissue shape of the measured portion, and a synthesis for generating a synthesized image by synthesizing the tissue property image and the tissue shape image An image generating means, is characterized in that a display means for displaying the synthesized image generated by the synthesized image generating means.

In the first and second fluorescent image display devices, the calculated image can be based on the ratio of the fluorescent images in a plurality of different wavelength bands.

In the second fluorescent image display device, the calculated image can be based on the ratio of the fluorescent image to the reflected image.

Further, in the first and second fluorescent image display devices, a statistic calculating means for calculating a statistic of a pixel value of one of the picked-up images, and a luminosity information based on the statistic. And a gradation processing means for assigning a display gradation.

Further, the statistic calculation means may calculate the statistic from a desired region of a part of any of the captured images.

Further, the gradation processing means calculates a predetermined coefficient based on the statistic, multiplies one of the captured images by a coefficient, and multiplies the coefficient by one of the captured images. To the display gradation of the luminance information.

Further, the gradation processing means determines a gradation processing function indicating the display gradation of the luminance information based on the statistical amount, and any one of the imaged images is determined based on the determined gradation processing function. The display gradation of the luminance information can be assigned to the image.

Further, when the pixel value of any one of the captured images is represented by 9 bits or more, there is provided bit shift means for bit-shifting the data so as to be represented by the upper 8 bits or less. A statistic may be calculated based on the bit-shifted data.

Here, "bit shift as indicated by the upper 8 bits or less" means that when data indicating the pixel value of the image is indicated by 9 bits or more, the bits of the upper 8 bits or less are rounded to 8 bits. The data is shown below, which means that it can be calculated by an 8-bit general-purpose statistical calculator.

Further, the gradation processing means can be turned on / off.

Here, turning on the gradation processing means means performing gradation processing, and turning off the gradation processing means.
Means that the gradation processing is not performed, and the “O”
"N / OFF possible" means that it is possible to switch between the case where gradation processing is performed and the case where gradation processing is not performed. The case where the gradation processing is not performed is, for example, a case where luminance information is assigned to the tissue property image. At this time, when the gradation processing is performed, the brightness or the like changes regardless of the tissue property (lesion or normal). However, it is not possible to determine the organizational properties. Also, at this time, a composite image based on the tissue texture image is displayed without performing the tone process, and after once determining the tissue texture, the tone processing means is switched on to perform the tone process on the tissue texture image. In this case, it is possible to see a detailed change in the tissue property in the observation screen.

In addition to the case other than the case where the luminance information is assigned to the tissue property image as described above, for example, it is displayed when the distance to the part to be measured suddenly becomes shorter or longer. The gradation processing means is turned off to prevent the display gradation of the luminance information of the image from largely changing.
It is desirable to be able to do so.

Further, when the composite image generating means generates the composite image by synthesizing the tissue property image and the tissue shape image,
When the number of pixels of the two images is different, the image may be converted to one of the numbers of pixels, and then the combined image may be generated.

Here, "when the number of pixels of both images is different, the number of pixels changes to either one of the pixels" means, for example, that the number of pixels of one image is 100 × 100 pixels and the number of pixels of the other image is 100 × 100. If the number of pixels is 500 × 500 pixels, 1
By converting one pixel of the image of 00 × 100 pixels into 5 × 5 pixels, the number of pixels of the image is reduced to 100 × 1
This means that enlargement processing is performed from 00 pixels to 500 × 500 pixels. Conversely, when converting to a pixel number of an image having a small number of pixels, reduction processing may be performed, and a general image processing method can be used for the above-described unit processing and reduction processing.

Further, an endoscope having an endoscope insertion portion to be inserted into a living body can be used.

The light source of the excitation light is a GaN-based semiconductor laser, and the wavelength band of the excitation light is from 400 nm to 42 nm.
It can be in the range up to 0 nm.

Further, the first and second fluorescent image display devices may be combined with a device which captures and displays a normal image based on reflected light reflected by irradiating a measured portion with white light.

[0054]

In general, the concept of "color" is classified into "color perceived by color" and "color perceived by color".
The “color perceived by color” is also referred to as a perceived color, and qualitatively defines a color perceived by a human as a psychological quantity using a symbol, a color table, or the like. On the other hand, “color based on color sensation” is also called psychophysical quantity, and is quantitatively determined as psychophysical quantity by a psychophysical experiment that measures the correspondence between the color as psychological quantity and the spectral characteristic of light as physical quantity to feel it. It is specified. The color system, which is a system for displaying colors, includes a developed color system for displaying perceived colors and a mixed color system for displaying psychophysical colors.

One of the representative color systems is the Munsell color system. In the Munsell color system, a color is expressed as a hue (H:
hue), saturation (S: saturation), and lightness (V: value). The hue (H) distinguishes between colors such as red, blue, and yellow. First, R (red),
The five colors Y (yellow), G (green), B (blue), and P (purple) are used as basic hues, and five circles are formed on the circumference of one circle (hue circle) as shown in FIG. Arrange in equal parts. Next, intermediate colors YR, GY, BG, PB, and RP are arranged in the middle of the basic hues. In general, the basic hue and the intermediate hue are given the number 5, and in many cases, 100 hues obtained by equally dividing adjacent hues into ten are used. However, if the rotation angle from the basic hue 5R is used, As a continuous value. In this case, for example, 5R is H = 0.
rad, 5Y is H = 1/3 rad, 5G is H = 2
/ 3rad.

The lightness (V) is defined as a scale indicating the brightness of a color, and is expressed as 0, where the ideal black lightness with a reflectance of 0% is 10, and the ideal white lightness with a reflectance of 100% is 10. In general, the human perception of brightness is not proportional to the reflectance, for example, an object having a reflectance of about 20% is recognized as an intermediate brightness. Therefore, the Munsell brightness scale is almost proportional to the square root of the reflectance. ing.

Saturation (S) is defined as a scale indicating the degree of color vividness. For each lightness and hue, an achromatic color (gray) is defined as a saturation of 0, and the most vivid color (monochromatic) is defined. )
Are equally spaced and represented by numbers.

When a hue circle in which the saturation is represented by the distance from the center is created for each lightness, and the hue circles are concentrically stacked in descending order of the lightness, the three attributes of the Munsell become cylindrical as shown in FIG. Color solid. All colors are located somewhere in this color solid. Generally, the perceived color is
One-dimensional coordinates (called the lightness index) corresponding to the lightness,
It is displayed in a three-dimensional coordinate space consisting of two-dimensional coordinates called perceived chromaticity, which is considered by integrating attributes of hue and saturation.

On the other hand, a typical color system representative of a color mixture system is a CIE (International Commission on Illumination) color system in which a color is treated quantitatively with three source stimuli (ie, light of three primary colors). There are mainly RGB color system and XYZ color system.
Empirically, it is known that any color can be created by mixing three source stimuli, independent three different colors, in appropriate proportions. The most common three
The two source stimuli are R (red), G (green), B (blue),
3 showing the mixing ratio of these source stimuli and these source stimuli
An RGB color system is configured from the stimulus values. However, in the RGB color system, there are inconveniences in mathematical processing, such as the fact that the color matching function plotting the stimulus value when creating continuous spectrum light by additive color mixing becomes negative depending on the wavelength. An XYZ color system is known in which X, Y, and Z, which are new virtual source stimuli for RGB, are subjected to coordinate conversion so that all values become positive. X
In the YZ color system, to facilitate mathematical handling,
Since the color matching function of Z (referred to as luminance) is defined to be equal to the relative luminous efficiency indicating the sensitivity of the human eye to the wavelength, the brightness of the color is entirely determined only by Z.
The color difference (chromaticity) is determined by chromaticity coordinates x and y obtained by the following equation.

X = X / (X + Y + Z) y = Y / (X + Y + Z) FIG. 3 shows the xy chromaticity diagram of the XYZ system.
A color obtained by mixing two points on the chromaticity is on a straight line connecting the two points, and all colors are in a curved curve (spectral locus).
And a point in a region surrounded by a straight line (pure purple locus) connecting both ends thereof. Note that the saturation decreases toward the white point at the center of the region. Arrows in the figure indicate changes in hue.

Further, when a color is represented by a video signal system represented by a TV signal or the like, R, G, and B are not transmitted independently. The signals I and Q and the luminance signal Y are converted and used. In addition, it is possible to represent and use a color by a video signal represented by the PAL system or the like.

According to the first method and apparatus for displaying a fluorescent image according to the present invention, a fluorescent image based on the intensity of the fluorescent light generated from the measured part by irradiating the measured part with the excitation light is captured, and the fluorescent image is displayed. At least one of the color information and the luminance information is assigned to the calculated image based on the image data, and a tissue property image mainly indicating the tissue property of the measured portion is generated, and the fluorescence image is assigned to the color information and the brightness information assigned to the tissue property image. And generating at least one of the color information and the luminance information, the tissue shape image mainly indicating the tissue shape of the measured part, and combining the tissue property image and the tissue shape image to generate a combined image. Is displayed, so that information on the fluorescence emitted from the measured portion (information on the tissue properties) and information on the tissue shape of the measured portion are displayed on one composite image. It can be, and is not to give a sense of discomfort to the observer.

Further, according to the second method and apparatus for displaying a fluorescent image according to the present invention, a fluorescent image based on the intensity of the fluorescent light generated from the measured part by irradiating the measured part with the excitation light is captured and referred to. By irradiating light to the measured part, a reflected image based on the intensity of the reflected light reflected from the measured part is captured, and at least one of the color information and the luminance information is assigned to the calculated image based on the fluorescent light image. A tissue property image indicating the tissue property of the part to be measured is generated, and at least one of color information and luminance information corresponding to the color information and the luminance information assigned to the tissue property image is assigned to the reflection image, and the measured property is mainly determined. Since the tissue shape image indicating the tissue shape is generated, the tissue property image and the tissue shape image are combined to generate a combined image, and the combined image is displayed. With fine apparatus and the same effect can be obtained by to produce a reflected image based on the tissue property image on irradiation of the reference light, it is possible to display a composite image that accurately reflects the more organized shape.

In the first and second fluorescent image display methods and apparatuses, when the calculated image is based on the ratio of the fluorescent images in a plurality of different wavelength bands, the fluorescent light emitted from the measured portion is Since a calculated image reflecting the difference in the shape of the fluorescence spectrum can be obtained, it is possible to display a composite image that more accurately reflects the tissue properties of the measured portion.

In the second method and apparatus for displaying a fluorescent image, when the calculated image is based on the ratio between the fluorescent image and the reflected image, the fluorescent yield of the fluorescent light emitted from the portion to be measured is reflected. Since the calculated image can be obtained, it is possible to display a composite image that more accurately reflects the tissue properties of the measured portion.

In the first and second fluorescent image display methods and apparatuses, the statistic of the pixel value of any one of the captured images is calculated, and the display gradation of the luminance information is assigned based on the statistic. Therefore, even when the pixel value of the image to which the luminance information is assigned is small,
A composite image with a predetermined brightness or higher can always be generated, and the dynamic range of the display gradation of the luminance information can be pseudo-expanded, so that a composite image that is always visible can be provided. . Furthermore, when an endoscope or the like inserted into a living body is used, a synthesized image with a predetermined brightness or higher can be generated even when the endoscope end portion is separated from the measured portion, so It is possible to provide a composite image that can be visually recognized over the measurement distance range.

When the statistic is calculated from a desired area of a part of the image, the display gradation of the luminance information of the desired area can be optimized. The amount of calculation can be reduced.

Further, a case where a predetermined coefficient is calculated based on the above-mentioned statistics, the calculated coefficient is multiplied to the image, and a display gradation of luminance information is assigned to the image multiplied by the coefficient. Can assign display gradations of appropriate luminance information by a simpler calculation method.

Further, a gradation processing function indicating the display gradation of the luminance information is determined based on the statistics, and the display gradation of the luminance information is assigned to the image based on the determined gradation processing function. In this case, the dynamic range of the display gradation of the luminance information in the composite image can be pseudo-expanded (equalized effect) by a simpler calculation method.

When the chromaticity of a color developing system, the chromaticity of a color mixing system or the color difference of a video signal system is used as the color information, a tissue property image or a tissue shape image is used. Can be easily assigned to the color information.

In the case where the brightness of the color system, the brightness of the color mixture system, or the brightness of the video signal system is used as the brightness information, the brightness of the tissue property image or the tissue shape image is used. Brightness information can be easily assigned.

Further, for example, if the Munsell color system, which is one of the color developing systems, is used, the color information can be determined in correspondence with the hue H in the Munsell hue circle shown in FIG. Can be made to correspond only to the hue.

For example, XY, which is one of the color mixing systems,
If the Z color system is used, color information can be determined in correspondence with the coordinates (x, y) on the xy chromaticity diagram shown in FIG.
Chromaticity can be easily matched.

Further, for example, when the color difference and luminance of the video signal system are used, the color difference signal and the luminance signal are determined from the above-mentioned calculated image and the like, and the color difference signal and the luminance signal are directly input to the video signal circuit and the like. , The color (color difference and luminance) of the composite image can be determined.

Further, the mathematical processing at the time of generating the composite image can be simplified.

In the case where the pixel value of the image for which the statistic is calculated is represented by 9 bits or more, the pixel value is set to the higher 8 bits.
When the statistic calculation means calculates the statistic based on the bit-shifted pixel value, an 8-bit general-purpose statistical calculator is provided. Can be used,
The speed of the arithmetic processing can be increased.

When the gradation processing for assigning the display gradation of the luminance information based on the statistic is enabled / disabled, for example, even when the luminance information is allocated to the tissue texture image, By turning off the tone processing, the determination of the tissue property can be accurately performed. After the tissue property is once determined, the tone processing means is turned on to execute the tone processing. If applied to an image, it is possible to see a detailed change in the tissue properties within the observation screen.

Further, in cases other than the case where the luminance information is assigned to the tissue characterization image as described above, for example, when the distance to the part to be measured suddenly becomes shorter or longer, the gradation processing is performed. Can be turned off, so that a large change in the display gradation of the luminance information of the displayed image can be avoided.

Further, when the statistic is a combination including the average value or the maximum value of the pixel values, the calculation of the statistic can be performed relatively easily, and the display gradation of appropriate brightness can be obtained. Can be assigned.

When a composite image is generated by synthesizing the tissue characterization image and the tissue shape image, if the two images have different numbers of pixels, the two images are converted into one of the numbers of pixels.
Since the combined image is generated, for example, after matching the number of pixels of the image with the smaller number of pixels to the number of pixels with the larger number of pixels, if a combined image is generated based on both images, for example, Since the amount of light such as fluorescent light is small, it is necessary to perform a binning process or the like when capturing a fluorescent image, and even when the number of pixels of the fluorescent image is smaller than the number of pixels of the reflected image, the number of pixels of the combined image is reflected. The display can be performed in accordance with the number of pixels of the image, and the tissue characteristics of the measured portion can be accurately displayed.

Further, for example, if the number of pixels of the image having the larger number of pixels is made to match the number of pixels having the smaller number of pixels, and then a combined image is generated based on both images, an extra operation process is required. The speed of the image processing can be increased without performing the processing.

Further, if a GaN-based semiconductor laser is used as the excitation light source, an inexpensive and compact light source can be obtained. If the wavelength band is within the range of 400 nm to 420 nm, the efficiency can be improved. It can be fluorescent.

[0083]

Embodiments of the present invention will be described below in detail with reference to the drawings. First, with reference to FIG. 4 and FIG. 5, a description will be given of a fluorescence endoscope apparatus as a first specific embodiment to which a fluorescence image display apparatus that performs a fluorescence image display method according to the present invention is applied. FIG. 4 is a schematic configuration diagram of a fluorescent endoscope apparatus to which the fluorescent image display device according to the present invention is applied. In this fluorescence endoscope apparatus, the fluorescence emitted from the measured part irradiated with the excitation light is two-dimensionally detected by an image fiber, and a narrow band fluorescence image of wavelength band light 430 nm to 530 nm and a wavelength band light 430 nm to 430 nm are obtained. A 730-nm broadband fluorescent image is captured, an operation image is generated based on the divided value of the pixel values of both images, and a hue H in the Munsell color system is assigned to the operation image, and a tissue characteristic image mainly indicating a tissue characteristic And an IR reflection image is captured from the reflected light of the part to be measured irradiated with white light, and a tissue property mainly indicating a tissue shape is obtained by assigning brightness V in the Munsell color system to pixel values of the IR reflection image. An image is generated, and a synthesized image obtained by synthesizing the tissue shape image and the tissue property image is displayed on a monitor.

The fluorescent endoscope apparatus according to the first embodiment of the present invention includes an endoscope insertion section 100 inserted into a site suspected of a lesion of a patient, a white light for capturing a normal image and an IR reflection image. And an illumination unit 110 including a light source that emits excitation light for capturing a fluorescence image, and an imaging unit 12 that captures two types of fluorescence images and IR reflection images having different wavelength bands.
0, calculating a division value between the fluorescent images, assigning a hue to an operation image based on the division value, and generating a tissue property image;
A composite image generating unit 130 that generates a tissue shape image by assigning lightness V to the pixel value of the IR reflection image, synthesizes the tissue property image and the tissue shape image to generate a composite image, and converts the normal image and the composite image into a visible image An image processing unit 140 that performs image processing for display as a display, a control computer 150 that is connected to each unit and controls operation timing, a monitor 160 that displays a normal image processed by the image processing unit 140 as a visible image, And a monitor 170 for displaying the composite image processed by the image processing unit 140 as a visible image.

The endoscope insertion section 100 includes a light guide 101, a CCD cable 102, and an image fiber 103 which extend to the distal end. Light guide 1
An illumination lens 104 and an objective lens 105 are provided at the end of the endoscope 01 and the CCD cable 102, that is, at the end of the endoscope insertion section 100. The image fiber 103 is a silica glass fiber, and has a condenser lens 106 at the tip. A CCD imaging device 107 is connected to a distal end portion of the CCD cable 102, and a prism 108 is attached to the CCD imaging device 107. The light guide 101 is bundled with a white light light guide 101a, which is a multi-component glass fiber, and an excitation light light guide 101b, which is a quartz glass fiber, and is integrated in a cable shape.
1a and the excitation light guide 101b are connected to the illumination unit 110. One end of the CCD cable 102 is connected to the image processing unit 140, and one end of the image fiber 103 is connected to the imaging unit 120.

[0086] The illumination unit 110 is used for normal image and IR.
A white light source 111 that emits white light L1 for imaging a reflected image,
A power supply 112 for a white light source electrically connected to the white light source 111, a GaN-based semiconductor laser 114 emitting excitation light L2 for capturing a fluorescent image, and the GaN-based semiconductor laser 11
Power supply 115 for semiconductor laser electrically connected to
It has.

The imaging unit 120 has a wavelength 42 near the excitation light from the fluorescent light L 3 passing through the image fiber 103.
An excitation light cut filter 121 for cutting a wavelength band of 0 nm or less, a switching filter 122 in which three types of optical filters are combined, a filter rotating device 124 for rotating the switching filter 122, and a fluorescent image or IR reflection transmitted through the switching filter 122. A CCD image sensor 125 for capturing an image,
Fluorescent image and IR captured by the CCD image sensor 125
A / D conversion circuit 126 for digitizing a reflected image and A / D
An image memory 127 for storing the image signal digitized by the conversion circuit 126 is provided.

The switching filter 122 includes an optical filter 123a, which is a band-pass filter for transmitting light of 430 nm to 730 nm, as shown in FIG.
An optical filter 123b is a bandpass filter that transmits light of nm, and an optical filter 123c is a bandpass filter that transmits light of 750 to 900 nm. The optical filter 123a is an optical filter for capturing a broadband fluorescent image, and the optical filter 123a
b is an optical filter for capturing a narrow band fluorescent image, and the optical filter 123c is an optical filter for capturing an IR reflected image. When the white light L1 is radiated, the switching filter 122 has an optical filter 123c on the optical path.
Are arranged, and when the excitation light L2 is irradiated, the control computer 150 controls the optical filter 123a or the optical filter 123b via the filter rotating device 124 so that the optical filters 123a and 123b are alternately arranged.

The CCD image pickup device 125 is 500 × 500
It is an image sensor of pixels, and performs normal reading when capturing an IR reflection image under the control of the control computer 150.However, when capturing a fluorescent image, in order to increase the signal intensity of the fluorescent image, Binning read is performed after adding outputs of 5 × 5 pixels. For this reason, when capturing a fluorescent image, it operates as an apparently 100 × 100 pixel image sensor.

The image memory 127 includes a narrow-band fluorescence image storage area, a broad-band fluorescence image storage area, and an IR reflection image storage area (not shown).
The fluorescent image captured in a state where the optical filter 123a for imaging the narrow band fluorescent image is arranged on the optical path is stored in the narrow band fluorescent image storage area, irradiated with the excitation light L2, and the optical filter for imaging the wide band fluorescent image. The fluorescent image captured in a state where 123b is arranged on the optical path is stored in the broadband fluorescent image storage area. In addition, the IR reflection image captured in a state where the white light L1 is irradiated and the optical filter 123c for imaging the IR reflection image is arranged on the optical path is stored in the IR reflection image storage area.

As described above, since the reading method is different, the number of pixels of the IR reflection image is 500 × 500 pixels.
The number of pixels of the narrow band fluorescent image and the wide band fluorescent image is 100
× 100 pixels.

The composite image generating unit 130 previously determines the range of the division value between the fluorescent images and the hue H (0 rad to 2/3 rad, Red to Yellow to Green area) in the Munsell hue circle.
And a tissue property image generating means 131 for assigning the hue H to the calculated image to generate a tissue property image, a range of pixel values of the IR reflection image, and a brightness in the Munsell color system in advance. V (Value)
And a tissue shape image generating means 132 for generating a tissue shape image by assigning brightness V to the reflection image, and a synthetic image generating means 133 for generating a synthetic image based on the two images. It is configured.

The image processing unit 140 includes an A / D conversion circuit 141 for digitizing a normal image picked up by the CCD image pickup device 107, a normal image memory 143 for storing the digitized normal image, and a memory for the normal image. There is provided a video signal processing circuit 144 for converting the normal image output from 143 and the synthesized image synthesized by the synthesized image generation means 133 into a video signal.

Hereinafter, the operation of the fluorescent endoscope apparatus having the above configuration to which the fluorescent display device according to the present invention is applied will be described.
In the fluorescence endoscope apparatus of the present embodiment, the imaging of the normal image and the IR reflection image and the imaging of the fluorescence image are alternately performed in a time-division manner. First, the operation at the time of imaging the fluorescence image will be described.

The present fluorescence endoscope apparatus uses a semiconductor laser power supply 115 based on a signal from the control computer 150.
Is driven, and the wavelength 41 is output from the GaN-based semiconductor laser 114.
The excitation light L2 of 0 nm is emitted. The excitation light L2 passes through the excitation light condenser lens 116, and the excitation light light guide 101
b, the light is guided to the distal end of the endoscope insertion portion, and then emitted from the illumination lens 104 to the living tissue 50.

The fluorescence L3 from the living tissue 50 generated by the irradiation of the excitation light L2 is condensed by the condenser lens 106, is incident on the tip of the image fiber 103, passes through the image fiber 103, and is collected by the lens 128. The light is transmitted through the excitation light cut filter 121 and the optical filter 123a or 123b of the switching filter 122.

The optical filter 123a has a wavelength band 430
It is a bandpass filter that transmits light of nm to 730 nm, and the fluorescent light transmitted through the optical filter 123a becomes a broadband fluorescent image. The optical filter 123b has a wavelength band 48
This is a bandpass filter that transmits light of 0 ± 50 nm, and the fluorescent light transmitted through the optical filter 123b becomes a narrow band fluorescent image.

The broadband fluorescent image and the narrow band fluorescent image are
After being received by the CCD image sensor 125 and photoelectrically converted,
Signals of 5 × 5 pixels are added and read out by binning readout, converted into digital signals by the A / D conversion circuit 126, and stored in the broadband fluorescent image storage area and the narrowband fluorescent image storage area of the image memory 127. You. By performing the binning readout as described above, a fluorescent image having a low light intensity can be accurately captured, but the number of pixels of the fluorescent image is 1/25 that of the normal readout.
00 × 100 pixels.

Next, the operation at the time of capturing an IR reflection image will be described. First, the white light source power supply 112 is driven based on a signal from the control computer 150, and the white light source 11
1 emits white light L1. The white light L1 is incident on the white light light guide 101a via the white light condenser lens 113, is guided to the distal end of the endoscope insertion portion, and is emitted from the illumination lens 104 to the living tissue 50. White light L1
Is reflected by the condenser lens 106, is incident on the tip of the image fiber 103, passes through the image fiber 103, is condensed by the lens 128, and is formed by the excitation light cut filter 121 and the optical filter 123 c of the switching filter 122. Through.

The optical filter 123c has a wavelength band of 750
Since the band-pass filter transmits only light having a wavelength of 900 nm to 900 nm, an IR reflection image transmitted through the optical filter 123c is an IR reflection image transmitted through only the near-infrared wavelength band in the reflected light L4.
It becomes a reflection image.

This IR reflected image is received by the CCD image sensor 125. IR photoelectrically converted by CCD image sensor 125
The reflection image is converted into a digital signal by the A / D conversion circuit 126 and then stored in the IR reflection image storage area of the image memory 127.

Next, the operation of generating a composite image will be described. First, in the tissue property image generating means 131 of the composite image generating unit 130, for each pixel of the narrow band fluorescent image and the wide band fluorescent image stored in the broad band fluorescent image storage area and the narrow band fluorescent image storage area of the image memory 127,
The pixel value in the narrow-band fluorescent image is divided by the pixel value in the wide-band fluorescent image, and the hue H (Hue) in the Munsell color system is assigned by using the divided value and a look-up table stored in advance. Produces
The image is output to the composite image generation unit 133.

The tissue shape image generating means 132
IR stored in the IR reflection image storage area of the image memory 127
For each pixel of the reflection image, a brightness V in the Munsell color system is assigned using the pixel value and the look-up table to generate a tissue shape image, which is output to the composite image generation unit 133.

The composite image generating means 133 converts the data of one pixel of the tissue characterization image into data of 5 × 5 pixels, and reduces the number of pixels of the tissue characterization image from 100 × 100 pixels to 5 × 5.
The image is enlarged to 00 × 500 pixels, and thereafter, the composite image is generated by synthesizing the tissue characteristic image and the tissue shape image based on the lightness V. Note that, when an image is displayed in color, three attributes of color, hue, lightness, and saturation, are required. Therefore, when combining images, the saturation S (Sa) in the Munsell color system is used.
The maximum value for each hue and lightness is set as the value of the "Turation".

Thereafter, RGB conversion is performed, a composite image is generated, and output to the video signal processing circuit 144.

The composite image converted to a video signal by the video signal processing circuit 144 is input to the monitor 170 and displayed on the monitor 170 as a visible image. The above series of operations are controlled by the control computer 150.

Next, the operation at the time of capturing a normal image will be described. When capturing a normal image, the control computer 150
The white light source power supply 112 is driven based on the signal from the white light source 111, and the white light source 111 emits white light L1. White light L1
Is incident on the white light light guide 101a via the white light condensing lens 113, is guided to the end of the endoscope insertion portion, and is then emitted from the illumination lens 104 to the living tissue 50.

The reflected light L 4 of the white light L 1 is transmitted to the objective lens 105.
And is reflected by the prism 108 so that the CCD
An image is formed on the image sensor 107.

The video signal from the CCD image sensor 107 is input to the A / D conversion circuit 142, digitized, and stored in the normal image memory 143. The normal image signal stored in the normal image memory 143 is converted into a video signal by the video signal processing circuit 144 and then input to the monitor 160, and displayed on the monitor 160 as a visible image. The above series of operations is performed by the control computer 1
50.

By the above-described operation, the hue of the displayed composite image is obtained by dividing the pixel value between the two types of fluorescent images,
That is, the difference reflects the difference in the shape of the fluorescence spectrum of the fluorescence emitted from the living tissue 50.
Since the pixel value of the reflection image, that is, the shape of the living tissue 50 is reflected, it is necessary to display information on the shape of the portion to be measured together with the information on the fluorescence emitted from the living tissue 50 in one image. And gives no discomfort to the observer. For this reason, the observer can easily determine the tissue property of the measured portion.

Further, since the hue H in the Munsell hue circle is determined based on the divided value of the pixel value between the fluorescent images, the divided value of the pixel value can be easily made to correspond to only the hue. Thus, the difference in the shape of the fluorescence spectrum of the fluorescence can be reflected in the composite image.

Further, since the fluorescent image is read out by binning readout at the time of imaging, the number of pixels of the fluorescent image is 100 × 100 pixels. Is converted into data of 5 × 5 pixels, and the number of pixels of the tissue characterization image is set to 100 × 1.
From 00 pixels to 500x500 pixels, then
Since the composite image is generated by synthesizing the tissue characteristic image and the tissue shape image based on the lightness V, the number of pixels of the display image is
This corresponds to 500 × 500 pixels, and the shape of the measured portion can be clearly displayed.

Further, since the GaN-based semiconductor laser 114 is used as the light source of the excitation light L2, the excitation light can be irradiated by a cheap and small light source. Further, since the wavelength of the excitation light is 410 nm, the living tissue 50 efficiently emits fluorescence.

As a modified example of the first embodiment, when a composite image is generated, the division value between the fluorescent images and
Instead of determining the hue using a look-up table in which the hue is stored in advance in correspondence with the division value between the fluorescent images, 2 is converted into an unsigned 16 bit as shown in Table 1.
It is conceivable that the hue H is assigned using a look-up table in which the pixel values and the hue H of the kinds of fluorescent images are stored. In this case, since division between the fluorescent images is unnecessary, stable mathematical processing can be performed even when the pixel values of the fluorescent images are small.

[0115]

[Table 1] Next, with reference to FIGS. 5 and 6, a description will be given of a fluorescence endoscope apparatus which is a second specific embodiment to which a fluorescence image capturing apparatus that performs the fluorescence image display method according to the present invention is applied. FIG. 6 is a schematic configuration diagram of a fluorescence endoscope apparatus to which the fluorescence image pickup device according to the present invention is applied. Note that, in the present embodiment, the same elements as those in the above-described first embodiment are denoted by the same reference numerals, and description thereof will be omitted unless necessary.

The fluorescence endoscope apparatus according to the second embodiment of the present invention includes an endoscope insertion portion 100 inserted into a site suspected of a lesion of a patient, a white light for capturing a normal image and an IR reflection image. And an illumination unit 110 having a light source that emits excitation light for fluorescent images, and an imaging unit 30 that captures two types of fluorescent images and IR reflected images having different wavelength bands.
0, calculates a division between the fluorescent images, assigns a hue H to an operation image based on the division value, generates a tissue property image, and assigns a brightness V to a pixel value of the IR reflection image to generate a tissue shape image. And a composite image generation unit 40 that composites the tissue property image and the tissue shape image to generate a composite image.
0, an image processing unit 500 that performs image processing for displaying a normal image and a composite image as a visible image, a control computer 200 that is connected to each unit and controls operation timing, and a normal image that is processed by the image processing unit 500. A monitor 601 displays an image as a visible image, and a monitor 602 displays a composite image processed by the image processing unit 500 as a visible image.

[0117] The image pickup unit 300 receives the wavelength 4 near the excitation light from the fluorescence L3 passing through the image fiber 103.
Excitation light cut filter 302 for cutting a wavelength band of 20 nm or less, switching filter 303 combining three types of optical filters, filter rotating device 304 for rotating switching filter 303, and fluorescent image or IR reflection transmitted through switching filter 303. CCD imaging device 30 for capturing an image
6. An A / D conversion circuit 307 for digitizing a signal imaged by the CCD image sensor 306 is provided.

The switching filter 303 includes three types of optical filters 303a and 3a, as in the first embodiment.
03b and 303c, and the optical filter 30
Reference numeral 3a denotes a bandpass filter that transmits a broadband fluorescent image having a wavelength of 430 nm to 730 nm, an optical filter 303b transmits a narrowband fluorescent image having a wavelength of 430 nm to 530 nm, and an optical filter 303c transmits a fluorescent image of 750 nm to 530 nm. This is a band-pass filter that transmits a 900 nm IR reflected image.

The synthesized image generating unit 400 includes a fluorescence image memory 401 for storing digitized fluorescence image signal data of two different wavelength bands, an IR reflection image memory 403 for storing IR reflection image signal data, and a fluorescence image. Property image generating means for performing a calculation according to the ratio of each pixel value of the fluorescence image of the two wavelength bands stored in the memory 401 and assigning the hue H to the calculated value of each pixel to generate a tissue property image 402, among the pixel values of the IR reflection image stored in the IR reflection image memory 403, for the pixel values of 9 bits or more, the bit shift means 409 for bit-shifting to 8 bits, and the pixels output from the bit shift means 409 A statistic calculation unit 404 provided with an 8-bit statistic calculator for calculating a predetermined statistic of the value, a statistic calculation unit 40
Coefficient calculating means 405 for calculating a predetermined coefficient based on the statistic output from No. 4, coefficient multiplying means 406 for multiplying each pixel value of the IR reflection image by the predetermined coefficient output from the coefficient calculating means 405, coefficient multiplying means Tissue shape image generating means 407 for generating a tissue shape image by assigning a display gradation of lightness V to each pixel value output from 406, and a tissue property image and a tissue shape image generating means output from tissue property image generating means 402 The image processing apparatus further includes a synthetic image generation unit 408 that synthesizes the tissue shape image output from the image forming unit 407 and outputs the synthesized image. In the present embodiment, the fluorescence images of two different wavelength bands are stored in the fluorescence image memory 401, and the IR reflection image is stored in the IR reflection image memory 403. IR reflection image memory 4
03 may be shared. In that case, the shared memory includes a narrow-band fluorescence image storage area, a broadband fluorescence image storage area, and an IR reflection image storage area, and the fluorescence image transmitted through the optical filter 303a is stored in the broadband fluorescence image storage area. The fluorescence image transmitted through the optical filter 303b may be stored in the narrow-band fluorescence image storage area, and the IR reflection image transmitted through the optical filter 303c may be stored in the IR reflection image storage area.

The image processing unit 500 includes an A / D conversion circuit 501 for digitizing a video signal obtained by the CCD image pickup device 107, a normal image memory 502 for storing a digitized normal image, and a normal image memory 502. A video signal processing circuit 503 is provided for converting the image signal output from 502 and the composite image signal output from the composite image generation means 408 into a video signal.

Next, the operation of the fluorescent endoscope apparatus to which the fluorescent image display device according to the present embodiment configured as described above is applied will be described. The fluorescence endoscope apparatus of the present embodiment also has a normal image and an IR image similarly to the first embodiment.
Although the imaging of the reflection image and the imaging of the fluorescence image are alternately performed in a time-division manner, the operation at the time of imaging the fluorescence image, the operation at the time of imaging the IR reflection image and the normal image are also the same as those in the above embodiment, The description is omitted, and the operation of generating a composite image different from that of the first embodiment will be described.

In the image pickup unit 300, the CCD image pickup element 306
, And the digitized broadband fluorescent image and narrowband fluorescent image are stored in the fluorescent image memory 401. The broadband fluorescent image captured by the CCD image sensor 306 is stored in a broadband fluorescent image area (not shown) of the fluorescent image memory 401, and the narrowband fluorescent image is stored in a narrowband fluorescent image area (not shown). . Then, the IR reflection image captured by the CCD unit 306 and digitized by the imaging unit 300 is stored in the reflection image memory 403.

The wide band fluorescent image and the narrow band fluorescent image stored in the fluorescent image memory 401 are divided by the tissue property image generating means 402 by each pixel value of each image in the same manner as in the first embodiment. An operation image is generated, and a hue H is assigned to a pixel value of the operation image to generate and output a tissue property image. The IR reflection image stored in the IR reflection image memory 403 is output to the statistic calculation means 404 after the data of 9 bits or more for all the pixel values is bit-shifted to 8-bit data by the bit shift means 409. Then, the average value m and the standard deviation σ of each pixel value are calculated by the statistic calculation means 404. Then, the average value m and the standard deviation σ are output to the coefficient calculating unit 405. In the coefficient calculating unit 405, the coefficient c is determined according to the following equation (2). The coefficient c is multiplied by each pixel value of the IR reflection image by the coefficient multiplying unit 406, and the operation value of each pixel is calculated by the tissue shape image generating unit 407. Is output to The tissue shape image generating means 407 generates and outputs a tissue shape image in which a display gradation of lightness V is assigned to the operation value of each pixel.

Upper limit of luminance display gradation × a ≒ (m + b + σ) × c (2) The tissue property image output from tissue property image generating means 402 and the tissue shape image output from tissue shape image generating means 407 are: In the same manner as in the first embodiment, the video signal processing circuit 503
Is output to

The composite image converted into a video signal by the video signal processing circuit 503 is input to the monitor 602 and displayed on the monitor 602 as a visible image. The above series of operations are controlled by the control computer 150. Other operations are the same as those in the first embodiment.

The operation of generating the tissue characterization image and the generation of the tissue morphology image are performed by serial processing, such as generating a tissue characterization image, generating a tissue morphology image, and combining the two images. Alternatively, the processing may be performed by parallel processing such that the generation of the tissue property image and the generation of the tissue shape image are performed at the same time and then combined. When the parallel processing is performed, the processing speed can be increased.

It is desirable that the gradation processing in the second embodiment can be turned on / off.

Further, in the second embodiment,
The average value and the standard deviation are calculated as statistics from the pixel values of the IR reflection image, and the predetermined coefficient c is calculated.However, when using a combination of the maximum value and the minimum value as the statistics, for example, What is necessary is just to calculate | require the coefficient c by Formula (3).

Assuming that the maximum value is max, the minimum value is min, the coefficient by which the IR reflection image is multiplied is c, and arbitrary constants are a and b, the upper limit of display gradation of luminance × aａ ((max + min) / 2 + b × ( (max-min) / 2) × c (3) In the second embodiment, the statistic calculated by the statistic calculation unit 404 is calculated based on the IR reflection image captured in real time in the same frame. It may not be based on the above, but may be based on the IR reflection image captured in the previous frame.

According to the fluorescence endoscope apparatus of the second embodiment, the statistic of the pixel value of the IR reflection image is calculated, and the display gradation of the luminance information is assigned based on the statistic. Therefore, even when the pixel value of the image to which the luminance information is assigned is small, it is possible to always generate a composite image having a predetermined brightness or higher, and to pseudo-expand the dynamic range of the display gradation of the luminance information. Therefore, it is possible to provide a composite image that is always visible.
Furthermore, even when the tip of the endoscope is separated from the measured portion, it is possible to generate a synthesized image with a predetermined brightness or more, so that it is possible to provide a synthesized image that can be visually recognized over a wide measurement distance range. it can.

Next, a description will be given of a fluorescent endoscope apparatus according to a third specific embodiment to which a fluorescent image pickup apparatus for implementing the fluorescent image display method according to the present invention is applied. In this embodiment, in the first embodiment or the second embodiment, the hue H is assigned to the operation image based on the divided value of the narrow-band fluorescence image and the wide-band fluorescence image to generate the tissue property image. Is generated by assigning a saturation S to the operation image to generate a tissue property image, and combining the image with the tissue shape image to generate and display a composite image. The tissue shape image is generated by assigning the lightness V to the reflection image as in the first and second embodiments. When the composite image generated according to the present embodiment is represented by the three attributes of color, it is displayed in the range of shaded areas in FIG. That is, for example, when the hue is green (a suitable hue may be used), for a normal tissue in which the distance from the distal end of the endoscope insertion section 100 to the measured section is short,
Vivid bright green, bright dark green for normal tissues at the distance above, bright white without color for lesions near the distance, and dark black without color for lesions at the far distance Will be done. Other configurations and operations are the same as those in the first or second embodiment.

In the first to third embodiments, the white light source 111 is common as a light source for white light and reference light for a normal image. However, each light source is provided separately. Is also good.

In the first to third embodiments, the calculation image is calculated based on the ratio between the narrow-band fluorescence image and the wide-band fluorescence image.
The calculation image may be calculated based on the ratio of the IR reflection images. Further, the hue or the saturation may be assigned to the fluorescent image itself without calculating the operation image as described above.

Although the CCD image pickup device for picking up the fluorescence image and the IR reflection image is commonly used, they may be provided separately. Further, a CCD image sensor may be separately provided for each of the narrow band fluorescent image and the wide band fluorescent image. By providing a separate CCD image sensor in this way, instead of capturing images in chronological order,
It is possible to take images in parallel.

Next, a description will be given of a fourth fluorescence endoscope apparatus to which a fluorescence image display device for implementing the fluorescence image display method according to the present invention is applied. FIG. 8 is a schematic configuration diagram of a fluorescent endoscope apparatus to which the fluorescent image display device according to the present invention is applied.

The fluorescent endoscope apparatus according to the present embodiment
The reflected light reflected from the portion to be measured by the irradiation of the surface sequential light (Lr, Lg, Lb) is captured as a normal image by the CCD image sensor 156 and displayed on the monitor 161. On the other hand, the portion to be measured by the excitation light irradiation A narrow band fluorescent image and a wide band fluorescent image are imaged by the CCD image sensor 156 based on the fluorescent light emitted from the CCD, and the chromaticity coordinates x and y in the XYZ color system are assigned based on the divided value of the narrow band fluorescent image and the broadband fluorescent image to obtain the tissue characteristic image Generate
In addition, the reflected light reflected from the part to be measured by the irradiation of the reference light is imaged as an IR reflected image, and the luminance is determined based on the pixel value.
A tissue shape image is generated by allocating Z, and a composite image based on the tissue property image and the tissue shape image is displayed on the monitor 162.

In the fluorescence endoscope apparatus according to the fourth embodiment of the present invention, a CCD image pickup device 156 is provided at the tip, and
Endoscope insertion part 3 inserted into suspected lesion of patient
50, plane-sequential light (R light L
r, G light Lg and B light Lb) and excitation light L for imaging a fluorescence image
2 and an illumination unit 310 that emits a reference light L5 for an IR reflection image, calculates a division value between the fluorescence images, and assigns chromaticity (hue and saturation) to an operation image based on the division value to obtain a tissue property. Generate an image and set the luminance to the pixel value of the IR reflection image
Assigning Z to generate a tissue shape image, synthesizing the tissue property image and the tissue shape image to generate a synthesized image, and performing image processing for displaying a normal image and a synthesized image as a visible image. Image processing unit 340, a control computer 360 connected to each unit for controlling the timing of operation, a monitor 161 for displaying a normal image processed by the image processing unit 340 as a visible image, and a process by the image processing unit 340 Monitor 1 that displays the synthesized image as a visible image
62.

[0138] The endoscope insertion section 350 includes a light guide 351 and a CCD cable 352 that extend to the inside thereof, and an illumination lens 154 is provided at the front end of the light guide 351.
Is provided, and an objective lens 155 is provided at the end of the CCD cable 352. At the tip of the CCD cable 352, a CCD image sensor 156 on which a mosaic filter 354 in which minute bandpass filters are combined in a mosaic shape is connected, and a prism 157 is attached to the CCD image sensor 156. I have.

The light guide 351 includes a light guide 351a for plane-sequential light, a light guide 351b for excitation light, and a light guide 351c for reference light which are bundled and integrated in a cable shape. It is connected to the.

The CCD cable 352 is connected to the CCD image sensor 1
The drive line 353a to which the drive signal of 56 is transmitted and the CCD
An output line 353 for reading out a signal from the image sensor 156
b, one end of the drive line 353a is connected to the control computer 360, and the output line 353
One end of “b” is connected to the composite image generation unit 330 and the image processing unit 340.

As shown in FIG. 9, the mosaic filter 354 is formed by alternately combining a narrow band filter 354a for transmitting light in a wavelength band of 430 nm to 530 nm and a full band filter 354b for transmitting light in all wavelength bands. The bandpass filters correspond one-to-one with the pixels of the CCD image sensor 156.

The illumination unit 310 includes a white light source 111 that emits white light, a power source 112 for a white light source that is electrically connected to the white light source 111, and outputs white light as R light Lr and G light Lg.
And the B light Lb, a switching filter 314 for sequentially separating colors, a filter rotating unit 315 for rotating the switching filter 314, a GaN-based semiconductor laser 211 that emits excitation light L2 for capturing a fluorescent image, and the GaN-based semiconductor laser 211. A power supply 212 for a semiconductor laser that is electrically connected, a reference light source 311 that is a semiconductor laser that emits a reference light L5 for imaging an IR reflected image, and a power supply 312 for a reference light source that is electrically connected to the reference light source 311 are provided. I have.

As shown in FIG. 10, the switching filter 314 includes an R filter 314a transmitting R light, a G filter 314b transmitting G light, a B filter 314c transmitting B light, and a mask portion 314d having a light shielding function. It is composed of

[0144] The composite image generation unit 330 generates the excitation light L
A / D conversion circuit 331 that digitizes an image signal captured by the CCD image sensor 156 when the light 2 or the reference light L5 is irradiated, and corresponds to the narrow band filter 354a of the mosaic filter 354 when the excitation light L1 is irradiated. A narrow-band fluorescent image received by a pixel to be irradiated, a wide-band fluorescent image received by a pixel corresponding to the full-band filter 354b when the excitation light L2 is irradiated, and a full-band filter of the mosaic filter 354 when the reference light L5 is irradiated. The image memory 332 that stores the reflection image received by the pixel corresponding to 354b in a different storage area, and the divided value of the narrow band fluorescent image and the wide band fluorescent image captured by the adjacent pixels stored in the image memory 332 A tissue property image generating unit 333 for calculating and assigning the chromaticity coordinates x and y to the operation value of each pixel to generate a tissue property image; 332 among the pixel values of the stored IR reflection image, the pixel value equal to or greater than 9bit is 8
A bit shift unit 335 that shifts bits to bits, a statistic calculation unit 336 including an 8-bit statistic calculator that calculates a predetermined statistic of each pixel value output from the bit shift unit 335, and a statistic calculation unit 336 A gradation processing function determination unit 337 that determines a gradation processing function based on the output statistics, and a luminance value is assigned to each pixel value of the IR reflection image based on the gradation processing function output from the gradation processing function determination unit 337. A tissue shape image generating means 338 for generating a tissue shape image by assigning a display gradation of Z, and a tissue property image output from the tissue property image generating means 333 and a tissue shape image output from the tissue shape image generating means 338 are combined. And a composite image generating unit 334 for outputting the composite image as a composite image.

The tissue property image generating means 333 has
Dividing value of fluorescent image and chromaticity coordinates x, y in XYZ color system
And a lookup table corresponding to.
In this lookup table, the division value is unsigned
In the value converted to 16bi and the xy chromaticity diagram shown in FIG.
The coordinates on the spectrum trajectory from ed (650 nm) to Yellow to Green (520 nm) correspond to each other as shown in Table 2.

[0146]

[Table 2] The image processing unit 340 includes an A / D conversion circuit 342 that digitizes an image signal of a pixel corresponding to the wideband filter 354b of the mosaic filter 354 when the R light Lr, the G light Lg, or the B light Lb is irradiated. The normal image memory 343 that stores the obtained normal image for each color, and when displaying the normal image, converts the three color image signals output from the normal image memory 343 in synchronization with each other into video signals. And a video signal processing circuit 344 that converts the composite image output from the composite image generation unit 330 into a video signal and outputs the video signal when displaying the fluorescent image.

Next, the operation of the fluorescent endoscope apparatus according to the above embodiment will be described. In the present fluorescence endoscope apparatus, imaging of a normal image, imaging of an IR reflection image, and imaging of a fluorescence image are performed in a time-division manner, and a normal image based on the normal image is displayed on the monitor 161 and the fluorescence L3 And reflected light L
4 is displayed on the monitor 162. In order to capture each image in a time-sharing manner, the illumination unit 310 sequentially emits the R light Lr, the G light Lg, the B light Lb, the excitation light L2, and the reference light L5.

First, the operation of capturing a fluorescence image and an IR reflection image, and generating and displaying a composite image based on both images will be described. When capturing a fluorescent image, the semiconductor laser power supply 2
12 is driven, and excitation light L2 having a wavelength of 410 nm is emitted from the GaN-based semiconductor laser 211. The excitation light L2 passes through the lens 213, enters the excitation light light guide 351b, is guided to the end of the endoscope insertion section, and is then emitted from the illumination lens 154 to the measurement target section 50.

The fluorescence from the measured section 50 generated by the irradiation of the excitation light L2 is condensed by the condenser lens 155, reflected by the prism 157, transmitted through the mosaic filter 354, and passed through the CCD image sensor 156. The image is formed as a fluorescent image on the top. At this time, since the reflected light of the excitation light L2 is cut by the excitation light cut filter 355, it does not enter the CCD image sensor 156.

The image signal photoelectrically converted by the CCD image sensor 156 is converted into a digital signal by the A / D conversion circuit 331 of the composite image generation unit 330, and the narrow band fluorescent image transmitted through the narrow band filter 354a and the whole band Filter 3
The image is divided into the broadband fluorescent images transmitted through 54b and stored in respective storage areas of the image memory 332.

Next, a description will be given of an operation when an IR reflection image is captured by irradiation of the reference light L5. Based on a signal from the controller 350, the reference light source power supply 312 is driven,
Reference light L5 is emitted from reference light source 311. Reference light L
5 is transmitted through the reference light lens 313, is incident on the reference light light guide 351 c, is guided to the end of the endoscope insertion portion, and is then emitted from the illumination lens 154 to the measured section 50.

The reflected light L 6 of the reference light L 5 reflected by the measured section 50 is condensed by the condenser lens 155, reflected by the prism 157, transmitted through the mosaic filter 354, and transferred onto the CCD image sensor 156. An image is formed as an IR reflected light image. In the image signal photoelectrically converted by the CCD image sensor 156, only the signal received by the pixel corresponding to the all-band filter 354b is converted into a digital signal by the A / D conversion circuit 331 of the composite image generation unit 330, and the image memory 33
IR to an area different from the area where the above-mentioned fluorescence image is stored
It is stored as a reflection image.

With respect to the narrow-band fluorescent image and the wide-band fluorescent image stored in the image memory 332, the pixel value of the narrow-band fluorescent image is divided by the pixel value of the wide-band fluorescent image for each pixel. After conversion to XYZ using a look-up table stored in advance.
A tissue property image is generated by assigning chromaticity coordinates x and y in the color system, and is output to the composite image generating means 233.

Further, after the IR reflection image stored in the image memory 332 is bit-shifted by the bit shift means 335, the average value m and the standard deviation σ of each pixel value are calculated by the statistic calculation means 336. . Then, the average value m and the standard deviation σ are output to the gradation processing function determining means 337. The gradation processing function determining means 337 sets f (x) to f (x−Min, where f (x) is the gradation processing function before the change.
/ (Max−Min)) (where Min = m−b ×
σ, Max = m + b × σ) to change and determine the gradation processing function.

The tissue shape image generating means 338 generates a tissue shape image by assigning the display gradation of the luminance Z in the XYZ color system to each pixel value of the IR reflection image based on the changed gradation processing function. Output.

The composite image generating means 334 generates a composite image by synthesizing the tissue property image and the tissue shape image based on the luminance Z.

In generating a composite image, first, X, Y, and Z are obtained from chromaticity coordinates x and y and luminance Z and the following equation.

X = X / (X + Y + Z) y = Y / (X + Y + Z) z = Z / (X + Y + Z) Then, RGB conversion is performed using the following equation to generate a composite image and output it to the video signal processing circuit 344. .

R = 0.41844X-0.15866Y-0.08283Z G = -0.09117X + 0.25242Y + 0.01570Z B = 0.00092X-0.00255Y-0.17858Z Input to the monitor 1
At 62, it is displayed as a visible image. The above series of operations
It is controlled by the control computer 360.

In the present embodiment, the XY image is added to the calculated image.
Although the chromaticity coordinates x and y in the Z color system are assigned to generate the tissue property image, a color difference signal may be used as color information. In this case, a chrominance signal IQ is assigned to the operation image to generate a tissue property image, and a luminance signal is assigned to the IR reflection image.
By assigning Y to generate a tissue shape image,
Since the color difference signal IQ determined from the operation image and the luminance signal Y determined from the IR reflection image can be directly input to the video signal processing circuit 344, there is no need to generate an RGB signal, and the apparatus can be simplified. .

Next, the operation for displaying a normal image will be described. This operation is substantially the same as that of the first embodiment except that the image is captured in a time-division manner. Therefore, the different operation will be mainly described.

First, the R light Lr is applied to the measured section, and the reflected light of the R light Lr reflected by the measured section 50 is formed on the CCD image pickup device 156 as an R light reflected image. Among the signals output from the CCD image sensor 156, the R image signal received by the pixel corresponding to the all-band filter 354 b of the mosaic filter 354 is converted into a digital signal by the A / D conversion circuit 342, and is stored in a normal image memory. 343 is stored in the storage area of the R image signal. Thereafter, the G image signal and the B image signal are obtained by the same operation, and stored in the G image signal storage area and the B image signal storage area of the normal image memory 343, respectively.

The three-color image signal is stored in the normal image memory 343.
Is output in synchronization with the display timing, converted into a video signal by the video signal processing circuit 344, output to the monitor 161 and displayed as a color image.

As a modification of the fourth embodiment, when a composite image is generated, the chromaticity coordinates x and y are stored in advance in correspondence with the division values between the fluorescent images. Instead of using the table to determine the chromaticity coordinates x and y, Red (650 nm) to Yellow correspond to the pixel values of each fluorescent image converted to unsigned 16 bits as shown in Table 3.
The chromaticity coordinates x and y may be determined by using a look-up table in which the hues in the range from 〜 to Green (520 nm) are stored.

[0165]

[Table 3] In the fourth embodiment, XYZ is used as the color space.
Although the color space is used, the HSV color space may be used. For example, when the hue and the saturation are assigned to the operation image and the tissue property image is generated, the composite image is a hatched portion in FIG. It is displayed in the color indicated by. That is, for a normal tissue having a short distance from the distal end of the endoscope insertion unit 100 to the measured portion, a bright bright green color is provided. For a normal tissue having a long distance, a bright dark green color is provided for a normal tissue. Is displayed as a bright red without a color, and the diseased tissue at the long distance is displayed as a dark red without a color. For the assignment of saturation,
Contrary to the above-mentioned assignment method, a low saturation may be assigned to normal tissue and a high saturation may be assigned to diseased tissue. In this case, the diseased tissue is more accurately imaged at diagnosis. Can be detected as

It is desirable that the gradation processing in the fourth embodiment can be turned on / off.

In the fourth embodiment, the average value and the standard deviation are calculated as statistics from the pixel values of the IR reflection image to change the gradation processing function. When a combination of a value and a minimum value is used, for example, if the maximum value of the pixel value of the IR reflection image is max, the minimum value is min, and any constants are α and β, the upper limit of the luminance display gradation ≒ ( max + min) / 2 + α ×
(Max-min) / 2 Lower limit of display gradation of luminance ≒ (max + min) / 2−β ×
The gradation processing function may be rewritten so as to be (max-min) / 2.

In the fourth embodiment, the statistic calculated by the statistic calculation means 336 does not have to be based on the IR reflection image captured in real time in the same frame. It may be based on an IR reflection image captured in a frame.

Next, a description will be given of a fluorescence endoscope apparatus according to a fifth specific embodiment to which the fluorescence image pickup apparatus according to the present invention is applied. In the present embodiment, the first to fourth embodiments are described.
In the embodiment, the tissue image is generated by assigning the hue H or the chromaticity (hue and saturation) to the calculation image based on the divided value of the narrow-band fluorescence image and the wide-band fluorescence image. A tissue property image is generated by assigning a lightness V (luminance Z), and is synthesized with the tissue shape image to generate and display a synthesized image. The tissue shape image may be generated by assigning lightness V (brightness Z) to the reflection image as in the first to fourth embodiments. When the composite image generated according to the present embodiment is represented by three attributes of color, it is displayed in a color in a range indicated by a thick arrow in FIG. In other words, for example, when the hue is green (an appropriate hue may be used), for a normal tissue in which the distance from the distal end of the endoscope insertion portion to the portion to be measured is short, bright green, and the distance is long The normal tissue is displayed in dark green, the diseased tissue at the short distance is displayed in dark green, and the diseased tissue in the long distance is displayed in darker green. Other configurations and operations are the same as those in the first to fourth embodiments.

Next, a description will be given of a fluorescent endoscope apparatus according to a sixth embodiment to which the fluorescent image display apparatus for performing the fluorescent image display method according to the present invention is applied. FIG. 13 is a schematic configuration diagram of a fluorescence endoscope apparatus to which the fluorescence image display device according to the present invention is applied. Note that, in this embodiment, the same elements as those in the above-described fourth embodiment are denoted by the same reference numerals, and description thereof will be omitted unless particularly necessary.

The fluorescent endoscope apparatus according to the present embodiment
The reflected light reflected from the portion to be measured by the irradiation of the surface sequential light (Lr, Lg, Lb) is captured as a normal image by the CCD image sensor 156 and displayed on the monitor 161. On the other hand, the portion to be measured by the excitation light irradiation A broadband fluorescence image is captured by the CCD image sensor 156 based on the fluorescence emitted from
The R light Lr is used as the reference light, and the reflected light L7 reflected from the measured section 50 by the irradiation of the R light Lr is imaged as an IR reflection image, and based on the divided value of the broadband fluorescence image and the IR reflection reflection image. To calculate a calculation image, assign a hue H based on the pixel value of the calculation image to generate a tissue characterization image, and assign a saturation S and a brightness V based on the pixel value of the IR reflection image to determine a tissue shape. An image is generated, and a composite image based on the tissue property image and the tissue shape image is displayed on the monitor 162.

In the fluorescence endoscope apparatus according to the sixth embodiment of the present invention, a CCD image pickup device 156 is provided at the tip, and
Endoscope insertion part 3 inserted into suspected lesion of patient
50, an illumination unit 370 for emitting plane-sequential light (R light Lr, G light Lg, and B light Lb) as illumination light for normal image capturing and reference light and excitation light L2 for fluorescent image capturing; Calculate the division value of the IR reflection image and assign the hue H to the operation image based on the division value to generate a tissue property image, and assign the saturation and brightness to the pixel value of the IR reflection image to generate the tissue shape image. A composite image generation unit 380 that generates and synthesizes the tissue property image and the tissue shape image to generate a composite image, and an image processing unit 3 that performs image processing for displaying a normal image and a composite image as a visible image
40, a control computer 360 connected to each unit to control the operation timing, the image processing unit 3
The monitor 161 includes a monitor 161 for displaying a normal image processed by the image processing unit 40 as a visible image and a monitor 162 for displaying a composite image processed by the image processing unit 340 as a visible image.

The light guide 351 has a light guide 351a for plane-sequential light and a light guide 351b for excitation light bundled and integrated into a cable. Each light guide is connected to the lighting unit 310. .

The illumination unit 370 includes a white light source 111 that emits white light, a power source 112 for a white light source that is electrically connected to the white light source 111, and white light that emits R light, G light, and B light.
A switching filter 314 for sequentially separating color into light, a filter rotating unit 315 for rotating the switching filter 314, and a GaN-based semiconductor laser 2 emitting excitation light L2 for capturing a fluorescent image.
And a semiconductor laser power supply 212 electrically connected to the GaN-based semiconductor laser 211.

The synthesized image generation unit 380 is configured to generate the excitation light L
2 or when the R light Lr is irradiated, the CCD image sensor 15
6, an A / D conversion circuit 381 for digitizing the image signal captured in step 6, an image memory 382 for storing the broadband fluorescent image and the IR reflection image in different storage areas, and a broadband fluorescent image and IR stored in the image memory 382. A tissue property image generation unit 383 for calculating a division value of the reflection image, assigning the hue H to the calculation value of each pixel to generate a tissue property image, and the image memory 3
A tissue shape image generating unit 385 that assigns saturation and lightness to each pixel value of the IR reflection image stored in 82 to generate a tissue shape image, and a tissue property image and a tissue shape image output from the tissue property image generating means 383. A composite image generation unit 384 that combines the tissue shape images output from the generation unit 385 and outputs the composite image is provided.

The image processing unit 340 includes the R light Lr, G light
A / D for digitizing an image signal captured by the CCD image sensor 156 when the light Lg or the B light Lb is irradiated
A conversion circuit 342, a normal image memory 343 for storing a digitized normal image for each color, and a three-color image signal synchronously output from the normal image memory 343 when displaying the normal image. And a video signal processing circuit 344 that converts the composite image output from the composite image generation unit 380 into a video signal and outputs the video signal when the fluorescent image is displayed. I have.

Next, the operation of the fluorescent endoscope apparatus according to the above embodiment will be described. In the present fluorescence endoscope apparatus, the imaging of the normal image and the imaging of the fluorescent image are performed in a time-division manner, and the IR reflection image is captured simultaneously with the irradiation of the R light Lr during the imaging of the normal image. A normal image based on the normal image is displayed on the monitor 161, and a composite image based on the fluorescent image and the IR reflected light image is displayed on the monitor 162. In order to capture each image in a time division manner, the illumination unit 310 outputs R
The light Lr, the G light Lg, the B light Lb, and the excitation light L are sequentially emitted.

First, the operation for displaying a normal image will be described. The R light Lr is applied to the measurement target, and the reflected light L7 of the R light Lr reflected by the measurement target 50 is the CCD image sensor 156.
An R light reflection image is formed thereon. CCD image sensor 156
The R image signal that has been photoelectrically converted is converted into an A / D conversion circuit 342.
Is converted into a digital signal, and stored in the storage area of the normal image memory 343 for the R image signal. Thereafter, the G image signal and the B image signal are obtained by the same operation, and stored in the G image signal storage area and the B image signal storage area of the normal image memory 343, respectively. The R image signal is also used as an IR reflection image, and the A / D conversion circuit 3
After being converted into a digital signal at 81, the image memory 382
Is stored in the storage area of the IR reflection image.

The three color image signals are stored in the normal image memory 343.
Is output in synchronization with the display timing, converted into a video signal by the video signal processing circuit 344, output to the monitor 161 and displayed as a color image.

Next, the operation of generating and displaying a composite image based on the fluorescence image and the IR reflection image will be described.
At the time of capturing a fluorescent image, the semiconductor laser power supply 212 is driven based on a signal from the control computer 360, and
Excitation light L having a wavelength of 410 nm from the aN-based semiconductor laser 211
2 is injected. The excitation light L2 passes through the lens 213, enters the excitation light light guide 351b, is guided to the end of the endoscope insertion section, and is then emitted from the illumination lens 154 to the measurement target section 50.

Fluorescence from the measured section 50 caused by irradiation with the excitation light L2 is condensed by the condenser lens 155, reflected by the prism 157, and reflected by the CCD image pickup device 15
6 is formed as a fluorescent image. At this time, the reflected light of the excitation light L2 is cut by the excitation light cut filter 355, and therefore does not enter the CCD image sensor 156.

The image signal photoelectrically converted by the CCD image sensor 156 is converted into a digital signal by the A / D conversion circuit 381 of the composite image generation unit 380, and
The two broadband fluorescent images are stored in the storage area.

On the other hand, in the image memory 382, an IR reflection image captured at the time of capturing the normal image is already stored.

For the broadband fluorescence image and the IR reflection image stored in the image memory 382, the pixel value of the broadband fluorescence image is divided by the pixel value of the IR reflection image for each pixel in the tissue property image generation means 382. Using a divided value and a look-up table stored in advance, a hue H in the Munsell color system is assigned to generate a tissue property image, and the resultant is output to the composite image generating means 334.

In the tissue shape image generating means 385,
IR stored in the IR reflection image storage area of the image memory 382
For each pixel of the reflection image, a brightness V and a saturation S in the Munsell color system are assigned using the pixel value and the look-up table to generate a tissue shape image, and output to the composite image generation unit 133.

The composite image generating means 133 generates a composite image by synthesizing the tissue property image with the tissue shape image based on the brightness V and the saturation S. When an image is to be displayed in color, RGB conversion is performed, a composite image is generated, and the composite image is output to the video signal processing circuit 344.

[0187] The composite image converted into a video signal by the video signal processing circuit 144 is input to the monitor 170 and displayed on the monitor 170 as a visible image. The above series of operations are controlled by the control computer 150.

When the composite image generated according to the present embodiment is represented by the three attributes of color, it is displayed in the shaded area of FIG. So, for example,
For normal tissue with a short distance from the tip of the endoscope insertion section to the measured part, vivid and bright green, for normal tissue with the long distance, dark green without color, and for diseased tissue with a short distance. The bright and bright red, and the diseased tissue at the above distance are displayed in dark red with no color.

Next, a description will be given of a fluorescent endoscope apparatus according to a seventh embodiment to which a fluorescent image pickup apparatus for performing the fluorescent image display method according to the present invention is applied. In the present embodiment, the tissue property image is generated by assigning the hue H to the calculation image based on the divided value of the broadband fluorescence image and the IR reflection image in the sixth embodiment. A tissue property image is generated by assigning saturation, and is synthesized with the tissue shape image to generate and display a synthesized image. When represented in the HSV space, the composite image generated according to the present embodiment is displayed in a color in a range indicated by a hatched portion in FIG. That is, for example, for a normal tissue in which the distance from the distal end of the endoscope insertion portion to the measured portion is short, bright and bright green, and for the normal tissue in which the distance is long, dark green with no tint and in which the distance is short The diseased tissue is displayed in bright red without color, and the diseased tissue at the long distance is displayed in darker red without color. Other configurations and operations are the same as in the sixth embodiment.

Next, a description will be given of a fluorescent endoscope apparatus according to an eighth embodiment to which a fluorescent image pickup apparatus for performing the fluorescent image display method according to the present invention is applied. This embodiment is different from the sixth embodiment in that the saturation S is added to the IR reflection image.
A tissue shape image was generated by assigning a brightness V and a tissue shape image was generated by assigning only the saturation S to the IR reflection image, and a composite image was generated by synthesizing the tissue characteristic image with the above-described tissue characteristic image and displayed. It is something to do. When the composite image generated according to the present embodiment is represented by three attributes of color, it is displayed in a color in a range indicated by a hatched portion in FIG. That is, for example, when the brightness is set to 100% (appropriate brightness may be used), for example, for a normal tissue in which the distance from the distal end of the endoscope insertion portion to the portion to be measured is short, bright green, The normal tissue at the long distance is displayed in green without color, the diseased tissue in the short distance is displayed in bright red, and the diseased tissue in the long distance is displayed in uncolored red. Other configurations and operations are the same as in the sixth embodiment.

Further, in addition to the assignment of the color information and the like to the tissue property image and the assignment of the color information and the like to the tissue shape image in each of the above embodiments, for example, the saturation is assigned to the tissue property image. Thus, saturation and lightness may be assigned to the tissue shape image. In this case, the composite image based on the tissue property image and the tissue shape image is as shown in FIG.
Will be displayed in the color indicated by the shaded area.
That is, for example, when the hue is green (an appropriate hue may be used), for a normal tissue in which the distance from the distal end of the endoscope insertion portion to the measured portion is short, bright bright green, Distant normal tissues are displayed in dark green with no color, diseased tissues with a short distance are displayed in bright white without color, and diseased tissues in the distant distance are displayed in dark black without color. . In this case, the color and the darkness are displayed in a color that is more emphasized in color and darkness than the color in the range illustrated in FIG. 7 used in the third embodiment. Further, when the saturation is assigned to both the tissue property image and the tissue fluorescence image as described above, for example, a predetermined function f is determined, and the saturation becomes f (tissue property image, tissue shape image). May be assigned a saturation. The predetermined function f may be, for example, f (tissue property image, tissue shape image) = tissue property image × tissue shape image.

Further, for example, lightness may be assigned to the tissue property image, and saturation and lightness may be assigned to the tissue shape image. In this case, the composite image based on the tissue property image and the tissue shape image is displayed in a color in a range indicated by a hatched portion in FIG. So, for example,
When the hue is green (the appropriate hue may be used), a normal tissue with a short distance from the distal end of the endoscope insertion portion to the measurement target is bright green, and a normal tissue with the above distance is far. , Dark green with no color, diseased tissue with the shortest distance is displayed in dark green, and diseased tissue with the longest distance is displayed in dark green without color (darker than normal tissue with the longest distance) become.

Further, in each of the above embodiments, an IR reflection image is used to generate a tissue shape image, but a fluorescence image may be used. In this case, the IR reflection image faithfully reflects the tissue shape, that is, the distance information between the tip of the endoscope insertion portion and the measured portion, while the fluorescence image reflects the distance information and information on the tissue properties. Therefore, a hue is assigned to the tissue property image, a luminance (brightness) is assigned to the tissue shape image based on the fluorescent image, and a synthetic image is generated based on both images. Since it is displayed in green and bright, and the diseased tissue is displayed in red and dark, the distinction between the normal tissue and the diseased tissue can be made clearer.

In each of the above-described embodiments, when there is color information and luminance information that cannot be assigned to the tissue characterization image or the tissue shape image, it is desirable to appropriately adjust the color and brightness information manually. In particular, when the hue of the color information is not assigned, it is more desirable to adopt the above-described form.

In each of the above embodiments, the hue and saturation of the developed color system are used as the color information. However, the chromaticity (XY) of the mixed color system, the color difference of the video signal (for example, the Y value of the NTSC signal).
For example, IQ of IQ) may be used.

The CCD image pickup device in the above embodiment may be a charge-doubling type CCD image pickup device.
The charge multiplying CCD image sensor is a CMD (Charge Multiplying D
(Electr) -CCD, which collides electrons with atoms in an intense electric field region, doubles the signal charge by a charge doubling effect generated by this ionization, and improves the imaging sensitivity of the imaging element.

In each of the above embodiments, the method for displaying the normal image and the composite image is such that the two images are displayed separately on two monitors. You may. At this time, the switching between the display of the normal image and the display of the composite image may be automatically performed in a time series by the control computer, or may be arbitrarily switched by an observer by a suitable switching unit.

In each of the above embodiments, the image fiber can be a multi-component glass fiber instead of a quartz fiber. At this time, when the excitation light is incident on the multi-component glass fiber, it emits fluorescence. Therefore, it is necessary to provide an excitation light cut filter between the condenser lens and the fluorescent image incident end of the image fiber. The cost can be reduced by changing from a quartz fiber to a multi-component glass fiber.

In the first to third embodiments, the CCD image pickup device for a normal image is installed at the end of the endoscope. However, by using an image fiber, it is installed in the image pickup unit. May be. Further, an image fiber and an image sensor for capturing a normal image, a fluorescence image, and a reflection image may be shared. In this case,
A filter means for obtaining a normal image may be provided on the front surface of the image sensor using a four-divided switching filter, a four-divided mosaic filter, or the like.

Further, the image processing relating to the display of a composite image in each of the above embodiments is not limited to being performed in units of pixels, but may be performed in units of n × m pixels as desired by the measurer.

If there is an area in which image processing related to composite image display has not been performed, the display color of that area is displayed in a predetermined color, so that the area in which processing relating to composite image display has been performed is clearly displayed. it can. For example, when pixels for which image processing is performed are thinned out, interpolation display may be performed based on a result of nearby image processing.

The excitation light source has a wavelength of 400 nm.
To about 420 nm.

Although the excitation light source and the white light source are separate, the light source may be shared by using an appropriate switching filter.

[Brief description of the drawings]

FIG. 1 is an explanatory diagram of a hue circle in the Munsell color system

FIG. 2 is an explanatory diagram of a Munsell color solid.

FIG. 3 is an xy chromaticity diagram of the XYZ color system.

FIG. 4 is a schematic configuration diagram of a fluorescence endoscope apparatus according to the first embodiment of the present invention.

FIG. 5 is a schematic configuration diagram of a switching filter used in the first and second embodiments.

FIG. 6 is a schematic configuration diagram of a fluorescence endoscope apparatus according to a second embodiment of the present invention.

FIG. 7 is a diagram illustrating a color range of a composite image displayed by the fluorescence endoscope apparatus according to the third embodiment of the present invention.

FIG. 8 is a schematic configuration diagram of a fluorescence endoscope apparatus according to a fourth embodiment of the present invention.

FIG. 9 is a schematic diagram of a mosaic filter.

FIG. 10 is a schematic configuration diagram of a switching filter used in a fourth embodiment.

FIG. 11 is a diagram showing a color range of a composite image displayed by the fluorescence endoscope apparatus according to the fourth embodiment of the present invention.

FIG. 12 is a diagram illustrating a color range of a composite image displayed by the fluorescence endoscope apparatus according to the fifth embodiment of the present invention.

FIG. 13 is a schematic configuration diagram of a fluorescence endoscope apparatus according to a sixth embodiment of the present invention.

FIG. 14 is a diagram illustrating a color range of a composite image displayed by the fluorescence endoscope apparatus according to the sixth embodiment of the present invention.

FIG. 15 is a diagram illustrating a color range of a composite image displayed by the fluorescence endoscope apparatus according to the seventh embodiment of the present invention.

FIG. 16 is a diagram showing a color range of a composite image displayed by the fluorescence endoscope apparatus according to the eighth embodiment of the present invention.

FIG. 17 is a diagram illustrating a color range of a composite image displayed by the fluorescence endoscope apparatus according to another embodiment;

FIG. 18 is a diagram illustrating a color range of a composite image displayed by the fluorescence endoscope apparatus according to another embodiment.

FIG. 19 is an explanatory diagram showing that display gradation of luminance information is assigned by multiplying an IR reflection image by a predetermined coefficient.

FIG. 20 is an explanatory diagram showing that display gradation of luminance information is assigned by changing a gradation processing function.

FIG. 21 is an explanatory diagram showing an intensity distribution of a fluorescence spectrum of a normal tissue and a diseased tissue.

[Explanation of symbols]

10, 20 Pixel value distribution 30, 40 Gradation processing function 50 Living tissue 100, 350 Endoscope insertion unit 101 Light guide 101a Light guide for white light 101b, 351b Light guide for excitation light 102 CCD cable 103 Image fiber 104, 154 Illumination lens 105, 155 Objective lens 106 Condenser lens 107, 125, 156, 306 CCD image pickup device 108, 157 Prism 110, 310, 370 Illumination unit 111 White light source 112 White light source power supply 113 White light condenser lens 114, 211 GaN-based semiconductor laser 115, 212 excitation light power supply 116, 213 excitation light condenser lens 150, 200, 360 control computer 120 imaging unit 121, 302, 355 excitation light cut filter 122, 303 Filter 123a, 123b, 123c Optical filter 124, 304 Filter rotation device 126, 141, 307, 342, 501 A / D conversion circuit 127, 332, 382 Image memory 128, 301 Lens 129, 305 Condensing lens for fluorescence 130 Synthetic image Generation unit 131,383 Tissue property image generation unit 132,385 Tissue shape image generation unit 133,384 Synthetic image generation unit 140,340,500 Image processing unit 143,343,502 Memory for normal image 144,344,503 Video signal processing Circuit 160, 161, 162, 170, 601, 602
Monitors 303a, 303b, 303c Optical filters 307, 331, 381, 501 A / D conversion circuit 311 Reference light source 312 Reference light source power supply 313 Reference light lens 314a R filter 314b G filter 315c B filter 330, 380 Synthetic image generation unit 333 Tissue property image generation means 334 Synthetic image generation means 335, 409 Bit shift means 336, 404 Statistical value calculation means 337 Gradation processing function determination means 338 Tissue shape image generation means 351 Light guide 351a Light guide for plane-sequential light 351c For reference light Light guide 352 CCD cable 353a Drive line 353b Output line 354 Mosaic filter 354a Narrow band filter 354b Full band filter 400, 800 Image operation unit 401 Memo for fluorescent image 402 tissue property image generation means 403 IR reflection image memory 405 coefficient calculation means 406 coefficient multiplication means 407 tissue shape image generation means 408 composite image generation means 415 gradation processing function determination means 502 normal image memory 503 video signal processing circuit L1 white Light L2 Excitation light L3 Fluorescence L4, L6, L7 Reflected light L5 Reference light Lr R light Lg G light Lb B light

Claims (26)

[Claims] 1. A fluorescence image based on the intensity of fluorescence generated from the measured part by irradiating the measured part with excitation light, and at least one of color information and luminance information is added to an arithmetic image based on the fluorescence image. And generating a tissue characterization image mainly indicating the tissue characterization of the measured part, and assigning the color information and the luminance information according to the color information and the luminance information assigned to the tissue characterization image to the fluorescence image. At least one is assigned to generate a tissue shape image mainly indicating the tissue shape of the measured portion, and the tissue property image and the tissue shape image are combined to generate a combined image, and the combined image is displayed. A method for displaying a fluorescent image, comprising: 2. A fluorescence image based on the intensity of fluorescence generated from the measured part by irradiating the measured part with excitation light, and irradiating a reference light to the measured part by irradiating a reference light to the measured part. A reflection image based on the intensity of the reflected light reflected from the image, and assigning at least one of color information and luminance information to an operation image based on the fluorescence image, and mainly indicating a tissue characteristic of the measurement target portion. And generating at least one of the color information and the luminance information corresponding to the color information and the luminance information assigned to the tissue characterization image to the reflection image, and mainly indicating the tissue shape of the measured portion. A fluorescent image display method, comprising: generating a shape image; synthesizing the tissue property image and the tissue shape image to generate a synthesized image; and displaying the synthesized image. 3. The fluorescence image display method according to claim 1, wherein the calculation image is based on a ratio of the fluorescence images in a plurality of different wavelength bands. 4. The fluorescent image display method according to claim 2, wherein the calculated image is based on a ratio between the fluorescent image and the reflected image. 5. The method according to claim 1, wherein a statistic of a pixel value of one of the captured images is calculated, and a display gradation of the luminance information is assigned based on the statistic. 2. The fluorescence image capturing method according to claim 1. 6. The fluorescent image display method according to claim 5, wherein the statistic is calculated from a desired region of a part of one of the captured images. 7. A predetermined coefficient is calculated based on the statistic, and the calculated coefficient is multiplied by any of the captured images, and the calculated coefficient is multiplied by any of the captured images. 7. The fluorescent image display method according to claim 5, wherein a display gradation of the luminance information is assigned. 8. A gradation processing function indicating a display gradation of the luminance information is determined based on the statistic, and the luminance is added to one of the captured images based on the determined gradation processing function. 7. The fluorescent image display method according to claim 5, wherein a display gradation of information is assigned. 9. The chromaticity of a color developing system,
9. The fluorescent image display method according to claim 1, wherein the chromaticity is a chromaticity of a color system or a color difference of a video signal system. 10. The brightness information according to claim 1, wherein the brightness information is lightness of a color system based on color development, brightness of a color system based on color mixture, or brightness of a video signal system. The fluorescent image display method described in the above. 11. A fluorescence image capturing means for capturing a fluorescent image based on the intensity of fluorescence generated from the measured section by irradiating the measured section with excitation light; and color information and a calculated image based on the fluorescent image. A tissue property image generating means for allocating at least one piece of luminance information and mainly generating a tissue property image indicating a tissue property of the measured part; and the color information and the color information allocated to the tissue property image in the fluorescence image. A tissue shape image generating means for allocating at least one of color information and brightness information according to the brightness information to generate a tissue shape image mainly indicating the tissue shape of the measured part; the tissue property image and the tissue shape image And a composite image generating means for generating a composite image by synthesizing the composite image and a display means for displaying the composite image generated by the composite image generating means. That the fluorescence image display apparatus. 12. A fluorescence image capturing means for capturing a fluorescence image based on the intensity of fluorescence generated from the measured section by irradiating the measured section with excitation light, and irradiating the measured section with reference light. A reflection image capturing means for capturing a reflection image based on the intensity of the reflected light reflected from the measurement target; and allocating at least one of color information and luminance information to an operation image based on the fluorescence image, and A tissue property image generating means for generating a tissue property image indicating a tissue property of the measuring unit; and at least one of color information and luminance information corresponding to the color information and the luminance information assigned to the tissue property image in the reflection image. A tissue shape image generating means for generating a tissue shape image mainly indicating the tissue shape of the measurement target portion, and combining and combining the tissue property image and the tissue shape image. And composite image generation means for generating an image, the fluorescence image display apparatus comprising the display means for displaying the synthesized image generated by the synthesized image generating means. 13. The fluorescent image display device according to claim 11, wherein the operation image is based on a ratio of the fluorescent images in a plurality of different wavelength bands. 14. The fluorescent image display device according to claim 12, wherein the calculated image is based on a ratio between the fluorescent image and the reflected image. 15. A statistic calculation means for calculating a statistic of a pixel value of any one of the captured images, and a gradation processing means for allocating a display gradation of the luminance information based on the statistic. 12. The method according to claim 11, wherein
15. The fluorescent image display device according to items 14 to 14. 16. The fluorescent image display apparatus according to claim 15, wherein said statistic calculating means calculates the statistic from a desired region of a part of one of the captured images. . 17. The gradation processing unit calculates a predetermined coefficient based on the statistic, multiplies any one of the captured images by the coefficient, and multiplies the coefficient by the coefficient. 17. The image according to claim 15, wherein a display gradation of said luminance information is assigned to said image.
The fluorescent image display device according to the above. 18. The gradation processing unit determines a gradation processing function indicating a display gradation of the luminance information based on the statistic, and the image pickup is performed based on the determined gradation processing function. The display gradation of the luminance information is assigned to one of the images.
The fluorescent image display device according to the above. 19. When the pixel value of any one of the captured images is represented by 9 bits or more, the pixel value is ranked in the top 8
16. The apparatus according to claim 15, further comprising: a bit shift unit that performs a bit shift as indicated by a bit or less, wherein the statistic calculation unit calculates the statistic based on the bit-shifted pixel value. 19. The fluorescent image display device according to any one of 18. 20. The apparatus according to claim 15, wherein said gradation processing means can be turned on / off.
Item 7. The fluorescent image display device according to Item 1. 21. The fluorescent image display device according to claim 15, wherein the statistic is a combination including an average value or a maximum value of the pixel values. 22. The color information according to claim 11, wherein the color information is a chromaticity of a color developing system, a chromaticity of a color mixing system, or a color difference of a video signal system. 2. The fluorescent image display device according to claim 1. 23. The brightness information according to claim 11, wherein the brightness information is lightness of a color system based on color development, brightness of a color system based on color mixture, or brightness of a video signal system. The fluorescent image display device according to the above. 24. When the synthetic image generating means generates the synthetic image by synthesizing the tissue property image and the tissue shape image, when the number of pixels of the two images is different,
The fluorescent image display device according to any one of claims 11 to 23, wherein the composite image is generated after the conversion into one of the pixel numbers. 25. An endoscope having an endoscope insertion portion to be inserted into a living body.
25. The fluorescent image display device according to any one of items 24 to 24. 26. A light source of the excitation light is a GaN-based semiconductor laser, and a wavelength band of the excitation light is from 400 nm to 4 nm.
2. The method according to claim 1, wherein the distance is up to 20 nm.
26. The fluorescent image display device according to any one of 1 to 25.