JP6272956B2 - Endoscope system, processor device for endoscope system, method for operating endoscope system, method for operating processor device - Google Patents

Description

  The present invention relates to an endoscope system for obtaining biological function information related to oxygen saturation of blood hemoglobin from an image signal obtained by imaging an observation target in a subject, a processor device of the endoscope system, and an endoscope system. The present invention relates to an operating method and a processor device operating method.

  In the medical field, diagnosis is generally performed using an endoscope system including a light source device, an endoscope, and a processor device. In recent years, lesions are being diagnosed using the oxygen saturation of blood hemoglobin in the biological function information. As a method for obtaining the oxygen saturation, first and second signal lights having different wavelength bands and different absorption coefficients of oxyhemoglobin and deoxyhemoglobin are alternately applied to the blood vessels in the mucous membrane, and the first and second A method is known in which each reflected light of signal light is detected by a sensor at the tip of an endoscope (Patent Documents 1 to 3).

  The ratio of the signal value of each pixel of the image signal corresponding to the reflected light of the first signal light detected by the sensor and the image signal corresponding to the reflected light of the second signal light (hereinafter referred to as the signal ratio) If there is no change in oxygen saturation, a constant value is maintained, but if there is a change in oxygen saturation, it also changes. Therefore, the oxygen saturation can be calculated based on the signal ratio of the image signal.

JP 2012-100800 A JP 2012-130429 A JP 2013-099464 A

  When the first signal light and the second signal light are irradiated to the observation target at different timings as in Patent Documents 1 to 3, the observation target is obtained between the timings for obtaining the image signals corresponding to the first and second signal lights. When there is no movement, the oxygen saturation can be calculated most accurately. However, the observation target is, for example, a living body such as the esophagus, stomach, intestine, and the like, and usually moves almost constantly due to a peristaltic movement or the like even during observation with the endoscope system. Even if the observation object itself does not move, if the operator moves the endoscope, such as when the operator moves the endoscope, it is equivalent to relatively moving the observation object.

  When a movement occurs in the observation target or the endoscope, a positional deviation may occur between the frames that obtain the image signals corresponding to the first and second signal lights. Further, the irradiation angle of the first and second illumination lights to the observation object changes due to the movement of the observation object and endoscope, and the light quantity ratio of the reflected lights of the first and second illumination lights changes. May end up. These positional deviations and changes in the light quantity ratio cause the accuracy of oxygen saturation calculation to deteriorate.

  The present invention relates to an endoscope system and an endoscope system that can improve the robustness with respect to the movement of an observation object and an endoscope, and can accurately calculate oxygen saturation even when the observation object and the endoscope are moving. PROBLEM TO BE SOLVED: To provide an operation method of an endoscope system, and an operation method of a processor device

  The endoscope system according to the present invention images a light source that emits first illumination light and second illumination light having different emission spectra, and images an observation target under illumination with the first illumination light, and outputs a first image signal. , An imaging device that images the observation target under illumination with the second illumination light and outputs a second image signal, a motion amount calculation unit that calculates the motion amount of the observation target, and the first when the motion amount falls within a specific range In the first mode in which the oxygen saturation is calculated using the first and second image signals, or in the second mode in which the oxygen saturation is calculated using the first image signal when the amount of motion does not fall within the specific range. An oxygen saturation calculation unit for calculating saturation.

  For example, the motion amount calculation unit calculates one motion amount for the entire imaging region, and the first and second modes are collectively changed for the entire imaging region.

  The motion amount calculation unit divides the imaging region into a plurality of regions, calculates the motion amount for each region, and the oxygen saturation calculation unit calculates the oxygen saturation in the first mode for a region where the motion amount falls within a specific range. And the oxygen saturation may be calculated in the second mode for a region where the amount of motion does not fall within a specific range. In this case, the motion amount calculation unit may calculate the motion amount for each pixel of the first image signal or the second image signal.

  A light source controller that controls the light emission timing of the first illumination light and the second illumination light and alternately emits the first illumination light and the second illumination light in both the first and second modes is provided. Also good.

  In addition, the emission timing of the first illumination light and the second illumination light is controlled, and the first illumination light and the second illumination light are alternately emitted in the first mode, and the first illumination in the second mode. You may provide the light source control part which light-emits only light.

  The motion amount calculation unit can calculate the motion amount based on a ratio between the red image signal included in the first image signal and the red image signal included in the second image signal.

  In the first mode, for example, the oxygen saturation is calculated based on the ratio of the blue image signal included in the first image signal and the green image signal included in the second image signal. The second mode is a mode for calculating the oxygen saturation based on, for example, the ratio of the blue and green image signals included in the first image signal.

In another endoscope system of the present invention, a light source that emits first illumination light and second illumination light having different emission spectra from each other, an observation target being illuminated with the first illumination light, and a first image signal are captured. An imaging device that outputs and images the observation target under illumination with the second illumination light and outputs a second image signal; a motion amount calculation unit that calculates a motion amount of the observation target; and the first and second image signals. An oxygen saturation calculation unit for calculating oxygen saturation in each of a first mode for calculating oxygen saturation using the first mode and a second mode for calculating oxygen saturation using the first image signal; The oxygen saturation calculation unit includes a first mode image generation unit that generates a first mode image as an oxygen saturation image representing the oxygen saturation calculated in the first mode, and an oxygen saturation calculated in the second mode. Generating a second mode image as an oxygen saturation image representing The first mode image and the second mode image are weighted so that the ratio of the second mode image is increased as the amount of motion is increased, and the ratio of the first mode image is increased as the amount of motion is decreased. A weighting synthesis unit that generates a synthetic oxygen saturation image by synthesizing the mode image;
Is provided.
The processor device of the present invention captures a light source that emits first illumination light and second illumination light having different emission spectra, images an observation target under illumination with the first illumination light, outputs a first image signal, and outputs a first image signal. A receiving unit that receives the first image signal and the second image signal, the processor device of the endoscope system comprising: an imaging device that images the observation target under illumination with two illumination lights and outputs a second image signal A motion amount calculation unit that calculates the motion amount of the observation target, and a first mode that calculates oxygen saturation using the first and second image signals when the motion amount falls within a specific range, or the motion amount is An oxygen saturation calculation unit that calculates oxygen saturation in a second mode that calculates oxygen saturation using the first image signal when it does not fall within the specific range.

  According to the operation method of the endoscope system of the present invention, the light source emits the first illumination light and the second illumination light whose emission spectra are different from each other, and the imaging element images the observation target under illumination with the first illumination light. An imaging step of outputting a first image signal, imaging an observation target under illumination with the second illumination light, and outputting a second image signal, and a motion amount calculation unit for calculating a motion amount of the observation target Step and the first mode in which the oxygen saturation calculation unit calculates the oxygen saturation using the first and second image signals when the motion amount falls within the specific range, or the motion amount does not fall within the specific range And an oxygen saturation calculating step for calculating oxygen saturation in a second mode for calculating oxygen saturation using the first image signal.

  According to the operating method of the processor device of the present invention, a light source that emits first and second illumination lights having different emission spectra, an observation target under illumination with the first illumination light, and a first image signal are output. And an image sensor that captures an image of an observation target under illumination with the second illumination light and outputs a second image signal. And a receiving step for receiving the second image signal, a motion amount calculating unit for calculating a motion amount of the observation target, and an oxygen saturation calculating unit for the first step when the motion amount falls within a specific range. And the first mode for calculating the oxygen saturation using the second image signal, or the second mode for calculating the oxygen saturation using the first image signal when the amount of motion does not fall within a specific range. Oxygen saturation calculation to calculate the degree Includes a step, a.

  According to the endoscope system, the processor device of the endoscope system, the operation method of the endoscope system, and the operation method of the processor device of the present invention, even if there is movement in the observation target or the endoscope, It is possible to calculate the oxygen saturation. That is, the robustness with respect to the observation object and the movement of the endoscope can be enhanced.

It is an external view of an endoscope system. It is a block diagram of an endoscope system. It is a graph which shows the spectrum of the light emitted at the time of normal observation mode. It is a graph which shows the spectrum of the light emitted at the time of special observation mode. It is a graph which shows the spectral transmittance of a RGB color filter. It is explanatory drawing which shows the imaging control at the time of normal observation mode. It is explanatory drawing which shows the imaging control at the time of special observation mode. It is a block diagram of an oxygen saturation image generation part. It is a graph which shows the correlation of a signal ratio and oxygen saturation. It is a graph which shows the light absorption coefficient of oxygenated hemoglobin and reduced hemoglobin. It is explanatory drawing which shows the method of calculating oxygen saturation. It is a flowchart which shows the effect | action of an endoscope system. It is a block diagram which shows the motion amount calculation part and oxygen saturation image generation part of 2nd Embodiment. It is explanatory drawing which switches the mode which calculates oxygen saturation for every some area | region. It is a block diagram which shows the oxygen saturation image generation part of 3rd Embodiment. It is explanatory drawing which produces | generates a synthetic | combination oxygen saturation image by weighting and synthesize | combining a 1st, 2nd mode image. It is a flowchart which shows the effect | action of 3rd Embodiment. It is a block diagram of the endoscope system of a 4th embodiment. It is explanatory drawing which shows the control aspect of 4th Embodiment. It is a block diagram of the endoscope system of a 5th embodiment. It is a graph which shows the light emission zone | band of LED, and the characteristic of HPF. It is explanatory drawing which shows the imaging control at the time of normal observation mode in 5th Embodiment. It is explanatory drawing which shows the imaging control at the time of the special observation mode in 5th Embodiment. It is a block diagram of the endoscope system of a reference example. It is a top view of a rotation filter. It is explanatory drawing which shows the imaging control of normal observation mode in the case of using a CMOS image sensor. It is explanatory drawing which shows the imaging control of the special observation mode in the case of using a CMOS image sensor. It is a block diagram of the oxygen saturation image generation part which memorize | stores the correlation for 1st, 2nd each mode.

[First Embodiment]
As shown in FIG. 1, the endoscope system 10 of the first embodiment includes an endoscope 12, a light source device 14, a processor device 16, a monitor 18 (display unit), and a console 20. The endoscope 12 is optically connected to the light source device 14 and electrically connected to the processor device 16. The endoscope 12 includes an insertion portion 21 to be inserted into a subject, an operation portion 22 provided at a proximal end portion of the insertion portion 21, a bending portion 23 and a distal end portion provided at the distal end side of the insertion portion 21. 24. By operating the angle knob 22a of the operation unit 22, the bending unit 23 performs a bending operation. With this bending operation, the distal end portion 24 can be directed in a desired direction.

  In addition to the angle knob 22a, the operation unit 22 is provided with an observation mode switching SW (observation mode switching switch) 22b, a zoom operation unit 22c, and a freeze button (not shown) for storing a still image. It has been. The mode switching SW 22b is used for switching operation between two types of modes, a normal observation mode and a special observation mode. The normal observation mode is a mode in which a normal light image in which an observation target in the subject is converted into a full color image is displayed on the monitor 18. The special observation mode is a mode in which an oxygen saturation image obtained by imaging the oxygen saturation of blood hemoglobin to be observed is displayed on the monitor 18. The zoom operation unit 22c is used for a zoom operation for driving the zoom lens 47 (see FIG. 2) in the endoscope 12 to enlarge the observation target.

  The processor device 16 is electrically connected to the monitor 18 and the console 20. The monitor 18 displays images such as normal light images and oxygen saturation images, and information related to these images (hereinafter referred to as image information and the like). The console 20 functions as a UI (user interface) that receives input operations such as function settings. Note that a recording unit (not shown) for recording image information or the like may be connected to the processor device 16.

  As shown in FIG. 2, the light source device 14 includes a first blue laser light source (473LD (laser diode)) 34 that emits a first blue laser beam having a center wavelength of 473 nm and a second blue laser beam that emits a second blue laser beam having a center wavelength of 445 nm. Two blue laser light sources (445LD) 36 are provided as light emission sources. The light emission amount and the light emission timing of each of the light sources 34 and 36 composed of these semiconductor light emitting elements are individually controlled by the light source control unit 40. For this reason, the light quantity ratio between the emitted light from the first blue laser light source 34 and the emitted light from the second blue laser light source 36 is freely changeable. The half width of the first and second blue laser beams is preferably about ± 10 nm. In addition, the first blue laser light source 34 and the second blue laser light source 36 can use broad area type InGaN laser diodes, and can also use InGaNAs laser diodes or GaNAs laser diodes. The light source may be configured to use a light emitter such as a light emitting diode.

  The light source control unit 40 turns on the second blue laser light source 36 in the normal observation mode. On the other hand, in the special observation mode, the first blue laser light source 34 and the second blue laser light source 36 are alternately turned on at intervals of one frame.

  The first and second blue laser beams emitted from the light sources 34 and 36 are transmitted to a light guide (LG) 41 via optical members (all not shown) such as a condenser lens, an optical fiber, and a multiplexer. Incident. The light guide 41 is built in the endoscope 12 and the universal cord 17 (see FIG. 1) that connects the light source device 14 and the endoscope 12. The light guide 41 propagates the first and second blue laser beams from the light sources 34 and 36 to the distal end portion 24 of the endoscope 12. A multimode fiber can be used as the light guide 41. As an example, a thin fiber cable having a core diameter of 105 μm, a cladding diameter of 125 μm, and a diameter of φ0.3 to 0.5 mm including a protective layer serving as an outer shell can be used.

  The distal end portion 24 of the endoscope 12 includes an illumination optical system 24a and an imaging optical system 24b. The illumination optical system 24a is provided with a phosphor 44 and an illumination lens 45. The first and second blue laser beams are incident on the phosphor 44 from the light guide 41. The phosphor 44 emits fluorescence when irradiated with the first or second blue laser light. Further, a part of the first or second blue laser light passes through the phosphor 44 as it is. The light emitted from the phosphor 44 is irradiated to the observation target through the illumination lens 45. Therefore, the first blue laser light source 34, the second blue laser light source 36, and the phosphor 44 constitute a light source for irradiating the observation target with illumination light.

  In the normal observation mode, since the second blue laser light is incident on the phosphor 44, white light having a spectrum shown in FIG. 3 (hereinafter referred to as second white light) is irradiated as illumination light onto the observation target. The second white light is composed of second blue laser light and green to red second fluorescence excited and emitted from the phosphor 44 by the second blue laser light. Therefore, the wavelength range of the second white light extends to the entire visible light range.

  On the other hand, in the special observation mode, when the first blue laser light and the second blue laser light are alternately incident on the phosphor 44, as shown in FIG. White light is alternately applied to the observation object as illumination light. The first white light is composed of first blue laser light and green to red first fluorescence that is excited and emitted from the phosphor 44 by the first blue laser light. Therefore, the first white light has a wavelength range covering the entire visible light range. The second white light is the same as the second white light irradiated in the normal observation mode. In the present embodiment, the first white light is the first illumination light, and the second white light is the second illumination light.

  The first fluorescence and the second fluorescence have substantially the same waveform (spectrum shape), and the ratio of the intensity of the first fluorescence (I1 (λ)) to the intensity of the second fluorescence (I2 (λ)) (hereinafter referred to as a frame). The intensity ratio) is the same at any wavelength λ. For example, I2 (λ1) / I1 (λ1) = I2 (λ2) / I1 (λ2). Since the inter-frame intensity ratio I2 (λ) / I1 (λ) affects the calculation accuracy of the oxygen saturation, the light source control unit 40 maintains a preset reference inter-frame intensity ratio. It is controlled with high accuracy.

The phosphor 44 absorbs a part of the first and second blue laser beams and excites and emits green to red light (for example, YAG phosphor or BAM (BaMgAl ₁₀ O ₁₇ )). It is preferable to use a material comprising a phosphor such as In addition, when the semiconductor light emitting element is used as an excitation light source of the phosphor 44 as in the present embodiment, high intensity first white light and second white light can be obtained with high luminous efficiency. In addition, the intensity of each white light can be easily adjusted, and changes in color temperature and chromaticity can be kept small.

  The imaging optical system 24b of the endoscope 12 includes an imaging lens 46, a zoom lens 47, and a sensor 48 (see FIG. 2). Reflected light from the observation object enters the sensor 48 via the imaging lens 46 and the zoom lens 47. As a result, a reflected image of the observation object is formed on the sensor 48. The zoom lens 47 moves between the tele end and the wide end by operating the zoom operation unit 22c. When the zoom lens 47 moves to the tele end side, the reflected image to be observed is enlarged. On the other hand, when the zoom lens 47 moves to the wide end side, the reflected image to be observed is reduced. Note that when not magnifying observation (non-magnifying observation), the zoom lens 47 is disposed at the wide end. When performing magnified observation, the zoom lens 47 is moved from the wide end to the tele end side by operating the zoom operation unit 22c.

  The sensor 48 is a color image pickup device, picks up a reflected image of an observation target, and outputs an image signal. As the sensor 48, for example, a charge coupled device (CCD) image sensor or a complementary metal-oxide semiconductor (CMOS) image sensor can be used. In the present embodiment, the sensor 48 is a CCD image sensor. The sensor 48 has RGB pixels provided with RGB color filters on the imaging surface, and outputs image signals of three colors of R, G, and B by performing photoelectric conversion with pixels of each color of RGB. .

  As shown in FIG. 5, the B color filter has a spectral transmittance of 380 to 560 nm, the G color filter has a spectral transmittance of 450 to 630 nm, and the spectral transmission of the R color filter 580 to 760 nm. Have a rate. Therefore, when the second white light is irradiated on the observation target in the normal observation mode, the second blue laser light and a part of the green component of the second fluorescence are incident on the B pixel, and the second light is incident on the G pixel. A part of the green component of the fluorescence is incident, and the red component of the second fluorescence is incident on the R pixel. The B image signal output from the B pixel is mostly occupied by the reflected light component of the second blue laser light because the second blue laser light has a much higher emission intensity than the second fluorescence.

  On the other hand, when the first white light is irradiated on the observation target in the special observation mode, the first blue laser light and a part of the green component of the first fluorescence are incident on the B pixel, and the first is applied to the G pixel. A part of the green component of the fluorescence and the first blue laser light attenuated by the G color filter enter, and the red component of the first fluorescence enters the R pixel. Since the first blue laser light has an emission intensity much higher than that of the first fluorescence, most of the B image signal output from the B pixel is occupied by the reflected light component of the first blue laser light.

  The light incident components at the RGB pixels when the second white light is irradiated onto the observation target in the special observation mode are the same as those in the normal observation mode.

  As the sensor 48, a so-called complementary color image sensor having complementary color filters of C (cyan), M (magenta), Y (yellow) and G (green) on the imaging surface may be used. When a complementary color image sensor is used as the sensor 48, the color conversion unit that performs color conversion from the CMYG four-color image signal to the RGB three-color image signal is any of the endoscope 12, the light source device 14, and the processor device 16. It should be provided in. In this way, even when a complementary color image sensor is used, it is possible to obtain RGB three-color image signals by color conversion from the four-color CMYG image signals.

The imaging control unit 49 performs imaging control of the sensor 48. As shown in FIG. 6, in the normal observation mode, the observation object illuminated with the second white light is imaged by the sensor 48 every one frame period (hereinafter simply referred to as one frame). Thereby, RGB image signals are output from the sensor 48 for each frame. In the present embodiment, since the sensor 48 is a CCD image sensor, one frame is, for example, from the end of the charge accumulation period (also referred to as the exposure period) (time T _A ) to the end of the next charge accumulation period (time T _B ). Further, since the sensor 48 is a CCD image sensor, the readout period and the charge accumulation period are separated in FIG. 6, but almost all of one frame is set as the charge accumulation period and is accumulated in the previous frame during the accumulation of signal charges. It is also possible to read out the signal charge. The imaging control unit 49 also performs control such as adjustment of the length of the charge accumulation period.

  The imaging control unit 49 controls the imaging of the sensor 48 in the special observation mode as in the normal observation mode. However, in the special observation mode, the first white light and the second white light are alternately irradiated on the observation target in synchronization with the imaging frame of the sensor 48. For this reason, as shown in FIG. 7, the sensor 48 reads out the signal charges obtained by imaging the observation object under the first white light during the readout period of the first frame, and outputs image signals for each of the RGB colors. To do. Then, the signal charges obtained by imaging the observation target under the second white light are read out during the readout period of the second frame, and image signals for each color of RGB are output. The sensor 48 outputs RGB image signals in both the first and second frames, but the spectrum of the white light on which it depends is different. Therefore, for the sake of distinction, the RGB output by the sensor 48 in the first frame will be described below. The image signals of the respective colors are referred to as R1 image signals, G1 image signals, and B1 image signals, respectively. The RGB image signals output in the second frame are referred to as R2 image signals, G2 image signals, and B2 image signals. The B1, G1, and R1 image signals output in the first frame are referred to as first image signals, and the B2, G2, and R2 image signals output in the second frame are the second image. This is called a signal.

  The oxygen saturation is calculated using, for example, a signal ratio B1 / G2 between the B1 image signal and the G2 image signal and a signal ratio R2 / G2 between the R2 image signal and the G2 image signal. Among these, the signal ratio essential for calculating the oxygen saturation is the signal ratio B1 / G2 between the B1 image signal and the G2 image signal. Therefore, the component that becomes the B1 image signal in the first white light (the first blue laser light transmitted through the phosphor 44) is the first signal light, and the component that becomes the G2 image signal in the second white light (the first blue light). 2 fluorescent green band component) is the second signal light. The endoscope system 10 may use a signal ratio B1 / G1 between the B1 image signal and the G1 image signal and a signal ratio R1 / G1 between the R1 image signal and the G1 image signal. In this case, the component that becomes the B1 image signal in the first white light is the first signal light, and the component (the green band component of the first fluorescence) that becomes the G1 image signal in the first white light is the second signal. Light.

  The image signals of the respective colors output from the sensor 48 are transmitted to a CDS (correlated double sampling) / AGC (automatic gain control) circuit 50 (see FIG. 2). The CDS / AGC circuit 50 performs correlated double sampling (CDS) and automatic gain control (AGC) on the analog image signal output from the sensor 48. The image signal that has passed through the CDS / AGC circuit 50 is converted into a digital image signal by the A / D converter 52. The digitized image signal is input to the processor device 16.

  The processor device 16 includes a receiving unit 54, an image processing switching unit 60, a normal observation image processing unit 62, a special observation image processing unit 64, a motion amount calculation unit 65, and an image display signal generation unit 66. Yes. The receiving unit 54 receives an image signal input from the endoscope 12. The receiving unit 54 includes a DSP (Digital Signal Processor) 56, a noise removing unit 58, and a signal converting unit 59.

  The DSP 56 performs various signal processing such as defect correction processing, offset processing, gain correction processing, linear matrix processing, gamma conversion processing, demosaic processing, and YC conversion processing on the received image signal. In the defect correction process, the signal of the defective pixel of the sensor 48 is corrected. In the offset process, the dark current component is removed from the image signal subjected to the defect correction process, and an accurate zero level is set. The gain correction process adjusts the signal level of each image signal by multiplying each RGB image signal after the offset process by a specific gain. The image signal of each color after the gain correction processing is subjected to linear matrix processing for improving color reproducibility. Thereafter, the brightness and saturation of each image signal are adjusted by gamma conversion processing. The image signal after the linear matrix processing is subjected to demosaic processing (also referred to as isotropic processing or simultaneous processing), and a color signal with missing pixels is generated by interpolation. Through the demosaic processing, all pixels have signals of RGB colors. The DSP 59 performs YC conversion processing on each image signal after the demosaic processing, and outputs the luminance signal Y and the color difference signals Cb and Cr to the noise removal unit 58.

  The noise removal unit 58 performs noise removal processing by, for example, a moving average method or a median filter method on the image signal that has been subjected to demosaic processing or the like by the DSP 56. The image signal from which noise has been removed is input to the signal conversion unit 59, reconverted into an RGB image signal, and input to the image processing switching unit 60.

  The image processing switching unit 60 inputs an image signal to the normal observation image processing unit 62 when the observation mode switching SW 22b is set to the normal observation mode. On the other hand, when the observation mode switching SW 22 b is set to the special observation mode, the image processing switching unit 60 inputs an image signal to the special observation image processing unit 64 and the motion amount calculation unit 65.

  The normal observation image processing unit 62 includes a color conversion unit 68, a color enhancement unit 70, and a structure enhancement unit 72. The color conversion unit 68 generates RGB image data in which the input RGB image signals for one frame are assigned to R pixels, G pixels, and B pixels, respectively. The RGB image data is further subjected to color conversion processing such as 3 × 3 matrix processing, gradation conversion processing, and three-dimensional LUT processing.

  The color enhancement unit 70 performs various color enhancement processes on the RGB image data that has been subjected to the color conversion process. The structure enhancement unit 72 performs structure enhancement processing such as spatial frequency enhancement on the RGB image data that has been subjected to color enhancement processing. The RGB image data subjected to the structure enhancement process by the structure enhancement unit 72 is input to the image display signal generation unit 66 as a normal observation image.

  The special observation image processing unit 64 includes an oxygen saturation image generation unit 76 and a structure enhancement unit 78. The oxygen saturation image generation unit 76 calculates the oxygen saturation and generates an oxygen saturation image representing the calculated oxygen saturation.

  The structure enhancement unit 78 performs structure enhancement processing such as spatial frequency enhancement processing on the oxygen saturation image input from the oxygen saturation image generation unit 76. The oxygen saturation image that has undergone the structure enhancement processing by the structure enhancement unit 72 is input to the image display signal generation unit 66.

  When the observation mode switching SW 22b is in the special observation mode, the motion amount calculation unit 65 acquires an image signal from the image processing switching unit 60 and calculates a motion amount using the acquired image signal. The movement refers to a change in movement, deformation, rotation, orientation, etc. due to a peristaltic motion of the observation object, a change in relative observation object due to a change in movement, rotation, orientation, etc. of the tip 24 within the subject, or This is a comprehensive change in the observation object as a result of combining the change in the observation object itself and the relative change in the observation object due to the change in the tip 24. The amount of movement is a value indicating the magnitude of the movement of the observation target.

  The motion amount calculation unit 65 calculates the motion amount based on the signal ratio between the R1 image signal obtained in the first frame and the R2 image signal obtained in the second frame. Specifically, for example, the ratio between the signal values of the R1 image signal and the R2 image signal is calculated for each pixel, and the average value is used as the motion amount. Therefore, one motion amount calculated by the motion amount calculation unit 65 is calculated for the entire imaging region represented by the first image signal (or the second image signal). Of course, the total value of the ratio of the signal values of the R1 image signal and the R2 image signal, and other statistical values such as the maximum value, minimum value, median value, and variance may be used as the motion amount. Since the R1 image signal and the R2 image signal are substantially the same when there is no movement in the observation target, the movement amount is substantially “1”. On the other hand, when a motion occurs in the observation target between the first frame and the second frame, a difference occurs between the R1 image signal and the R2 image signal, and the motion amount becomes a value deviated from “1”. The larger the movement of the observation target, the greater the deviation from “1”.

  The motion amount calculation unit 65 inputs the calculated motion amount to the oxygen saturation image generation unit 76. The oxygen saturation image generation unit 76 has a first mode and a second mode as modes for calculating the oxygen saturation. Based on the motion amount input from the motion amount calculation unit 65, at least these Calculate oxygen saturation in either mode. In the first mode, the observation target being illuminated with the first white light is imaged, and the first image signal obtained in the first frame and the observation target illuminated with the second white light are imaged and obtained in the second frame. In this mode, the oxygen saturation is calculated based on both of the second image signal and the second image signal. More specifically, in the first mode, the oxygen saturation is calculated based on the signal ratio B1 / G2 between the B1 image signal and the G2 image signal and the signal ratio R2 / G2 between the R2 image signal and the G2 image signal. On the other hand, the second mode is a mode in which the oxygen saturation is calculated based on only the first image signal without using the second image signal. More specifically, in the second mode, the oxygen saturation is calculated based on the signal ratio B1 / G1 between the B1 image signal and the G1 image signal and the signal ratio R1 / G1 between the R1 image signal and the G1 image signal. .

  The image display signal generation unit 66 converts the normal observation image or the oxygen saturation image into a display format signal (display image signal) and inputs it to the monitor 18. As a result, the normal observation image or the oxygen saturation image is displayed on the monitor 18.

  As shown in FIG. 8, the oxygen saturation image generation unit 76 includes a mode switching unit 80, a signal ratio calculation unit 81, a correlation storage unit 82, an oxygen saturation calculation unit 83, an image generation unit 84, It has.

  The mode switching unit 80 compares the motion amount input from the motion amount calculation unit 65 with a threshold value (a set of upper and lower limit values that define the range of motion amount), and calculates the oxygen saturation according to the result A mode setting signal for switching is input to the signal ratio calculation unit 81. Specifically, the mode setting for setting the mode for calculating the oxygen saturation to the first mode when it is determined that the movement amount is within a specific range determined by the threshold value and the movement of the observation target is small. The signal is input to the signal ratio calculation unit 81. On the other hand, when it is determined that the amount of movement does not fall within the specific range determined by the threshold and the movement of the observation target is large, the mode setting signal for setting the mode for calculating the oxygen saturation to the second mode is set to the signal ratio. Input to the calculation unit 81. The threshold and the specific range determined by the threshold are determined in advance by setting or the like. In addition, since one motion amount is calculated for the entire imaging region represented by the first image signal (or the second image signal), the switching of the calculation mode performed by the mode switching unit 80 is based on this one motion amount. The entire imaging area is changed all at once.

  The signal ratio calculation unit 81 calculates the signal ratio used by the oxygen saturation calculation unit 83 in accordance with the mode setting signal input from the mode switching unit 80. When the first mode is set, the signal ratio calculation unit 81 calculates the signal ratio B1 / G2 between the B1 image signal and the G2 image signal for each pixel and the signal ratio R2 / G2 between the R2 image signal and the G2 image signal. G2 is calculated for each pixel. When the signal ratio calculation unit 81 calculates the signal ratio B1 / G2, the signal value based on the first fluorescence is calculated from the B1 image signal by the inter-pixel calculation using the B1 image signal, the G1 image signal, and the R1 image signal. A B1 image signal that has been subjected to correction processing to improve the color separability and corrected to a signal value substantially using only the first blue laser beam is used.

  On the other hand, when the second mode is set, the signal ratio calculation unit 81 calculates the signal ratio B1 / G1 between the B1 image signal and the G1 image signal for each pixel, and the signal ratio between the R1 image signal and the G1 image signal. R1 / G1 is calculated for each pixel. When calculating the signal ratios B1 / G1 and R1 / G1, the signal ratio calculation unit 81 calculates the first fluorescence from the B1 image signal by an inter-pixel calculation using the B1 image signal, the G1 image signal, and the R1 image signal. Is removed, and the signal value of the first laser beam is removed from the G1 image signal, and correction processing is performed to improve the separation of each color. Then, the signal ratio B1 / G1 and the signal ratio R1 / are obtained by using the B1 image signal corrected to the signal value by only the first blue laser light and the G1 image signal corrected to the signal value by only the first fluorescence. G1 is calculated.

  Comparing the first mode and the second mode, since the first mode is only the B1 image signal that is subjected to correction processing when calculating the signal ratio used for calculating the oxygen saturation, the B1 image signal and the G1 image signal The accuracy of oxygen saturation is higher than that in the second mode in which correction processing is performed on these two types of image signals. On the other hand, in the second mode, since the oxygen saturation is calculated using only the first image signal obtained simultaneously in one frame (first frame), the calculation accuracy of the oxygen saturation is reduced even if the observation target moves. do not do.

  The correlation storage unit 82 stores a correlation between oxygen saturation and a set of two signal ratios calculated by the signal ratio calculation unit 81 in each mode. This correlation is stored in a two-dimensional table in which isolines of oxygen saturation are defined on the two-dimensional space shown in FIG. 9, and is used in common in the first and second modes in this embodiment. The positions and shapes of the isolines with respect to the signal ratio are obtained in advance by a physical simulation of light scattering, and the interval between the isolines changes according to the blood volume (horizontal axis in FIG. 9). The correlation between the signal ratio and the oxygen saturation is stored on a log scale.

  As shown in FIG. 10, the correlation is closely related to the light absorption characteristics and light scattering characteristics of oxyhemoglobin (graph 90) and reduced hemoglobin (graph 91). For example, information on oxygen saturation is easy to handle at a wavelength where the difference in absorption coefficient between oxygenated hemoglobin and reduced hemoglobin is large, such as the center wavelength of 473 nm of the first blue laser beam. However, the B1 image signal including a signal corresponding to 473 nm light is highly dependent not only on the oxygen saturation but also on the blood volume. Therefore, in the first mode, in addition to the B1 image signal, a signal obtained from an R2 image signal corresponding to light that changes mainly depending on the blood volume, and a G2 image signal serving as a reference signal for the B1 image signal and the R2 image signal. By using the ratio R2 / G2, the oxygen saturation can be accurately determined without depending on the blood volume. In the second mode, the G1 image signal is used as the reference signal for the B1 image signal and the R1 image signal, and the signal ratio R1 / G1 is used as the signal ratio representing the blood volume. As in the case of the 1 mode, the oxygen saturation can be accurately obtained without depending on the blood volume.

The oxygen saturation calculation unit 83 calculates the oxygen saturation based on the amount of movement by using the signal ratio calculated by the signal ratio calculation unit 81. More specifically, the oxygen saturation calculation unit 83 refers to the correlation stored in the correlation storage unit 82 and calculates the oxygen saturation corresponding to the signal ratio calculated by the signal ratio calculation unit 81 for each pixel. calculate. For example, in the first mode, when the signal ratio B1 / G2 and the signal ratio R2 / G2 in the specific pixel are B1 ^* / G2 ^* and R2 ^* / G2 ^* , respectively, referring to the correlation as shown in FIG. The oxygen saturation corresponding to the signal ratio B1 ^* / G2 ^* and the signal ratio R2 ^* / G2 ^* is “60%”. Therefore, the oxygen saturation calculation unit 83 calculates the oxygen saturation of this specific pixel as “60%”. The same applies to the second mode.

  In the first mode, the signal ratio B1 / G2 and the signal ratio R2 / G2 are hardly increased or decreased extremely. In other words, the values of the signal ratio B1 / G2 and the signal ratio R2 / G2 hardly exceed the lower limit line 93 with an oxygen saturation of 0%, or conversely fall below the upper limit line 94 with an oxygen saturation of 100%. However, when the calculated oxygen saturation falls below the lower limit line 93, the oxygen saturation calculation unit 83 sets the oxygen saturation to 0%. When the calculated oxygen saturation exceeds the upper limit line 94, the oxygen saturation is set to 100. %. Further, when the points corresponding to the signal ratio B1 / G2 and the signal ratio R2 / G2 deviate from between the lower limit line 93 and the upper limit line 94, it is understood that the reliability of oxygen saturation in the pixel is low. It is also possible not to display or calculate the oxygen saturation. The same applies to the second mode.

  The image generation unit 84 uses the oxygen saturation calculated by the oxygen saturation calculation unit 83 to generate an oxygen saturation image obtained by imaging the oxygen saturation. Specifically, in the case of the first mode, the image generation unit 84 acquires the B2 image signal, the G2 image signal, and the R2 image signal, and gains corresponding to the oxygen saturation for these image signals for each pixel. Apply. Then, RGB image data is generated using the gained B2 image signal, G2 image signal, and R2 image signal. For example, the image generation unit 84 multiplies all of the B2 image signal, the G2 image signal, and the R2 image signal by the same gain “1” for a pixel having a corrected oxygen saturation of 60% or more. On the other hand, for pixels with a corrected oxygen saturation of less than 60%, the B2 image signal is multiplied by a gain less than “1”, and the G2 image signal and the R2 image signal are gained by “1” or more. Multiply. The RGB image data generated using the B2 image signal, G2 image signal, and R2 image signal after the gain processing is an oxygen saturation image.

  In the second mode, the image generation unit 84 acquires the B1 image signal, the G1 image signal, and the R1 image signal, and applies a gain corresponding to the oxygen saturation to these image signals for each pixel. Then, RGB image data is generated as an oxygen saturation image using the gained B1 image signal, G1 image signal, and R1 image signal. The method of multiplying the gain is the same as in the first mode.

  In the oxygen saturation image generated by the image generator 84, the high oxygen region (region where the oxygen saturation is 60 to 100%) is expressed in the same color as the normal observation image. On the other hand, a low oxygen region where the oxygen saturation is lower than a specific value (region where the oxygen saturation is 0 to 60%) is represented by a color (pseudo color) different from that of the normal observation image.

  In the present embodiment, the image generation unit 84 multiplies the gain for pseudo-coloring only the low oxygen region, but the gain corresponding to the oxygen saturation is applied even in the high oxygen region, and the entire oxygen saturation image is obtained. A pseudo color may be used. Further, although the low oxygen region and the high oxygen region are separated by oxygen saturation 60%, this boundary is also arbitrary.

  Next, the flow of observation by the endoscope system 10 of this embodiment will be described along the flowchart of FIG. First, in the normal observation mode, screening is performed from the farthest view state (S10). In the normal observation mode, a normal observation image is displayed on the monitor 18. At the time of this screening, if a site (hereinafter, referred to as a possible lesion) such as a brownish area or redness is found (S11), the mode switching SW 22b is operated to switch to the special observation mode. (S12). Then, in this special observation mode, a possible lesion site is diagnosed.

  In the special observation mode, the first and second white lights are alternately irradiated on the observation target in synchronization with the imaging frame of the sensor 48, so that the sensor 48 detects the R1 image signal, the G1 image signal, and the B1 image signal in the first frame. And the R2 image signal, the G2 image signal, and the B2 image signal are output in the second frame (imaging step (S12)). Then, the processor device 16 receives these image pickup signals at the reception unit 54 (reception step (S12)), and the motion amount calculation unit 65 uses the R1 image signal and the R2 image signal among the image signals for these two frames. The signal ratio is calculated for each pixel, and the average value is calculated as the motion amount (S13: motion amount calculation step).

  The amount of motion is compared with a threshold in the mode switching unit 80 (S14), and when the amount of motion is small and falls within a specific range determined by the threshold, the mode for calculating the oxygen saturation is set to the first mode (S14). : YES). On the other hand, when the amount of motion is large and does not fall within the specific range determined by the threshold, the mode for calculating the oxygen saturation is set to the second mode (S14: NO).

  When the mode for calculating the oxygen saturation is set to the first mode, the signal ratio calculation unit 81 calculates the signal ratio B1 / G2 and the signal ratio R2 / G2 (S15: signal ratio calculation step). However, when the signal ratio B1 / G2 is calculated, the B1 image signal that has been subjected to the correction process for removing the signal value due to the first fluorescence from the B1 image signal acquired via the image processing switching unit 60 is used. . On the other hand, when the oxygen saturation calculation mode is set to the second mode, the signal ratio calculation unit 81 calculates the signal ratio B1 / G1 and the signal ratio R1 / G1 (S16: signal ratio calculation step). However, when calculating these signal ratios B1 / G1 and R1 / G1, a B1 image that has been subjected to correction processing for removing the signal value due to the first fluorescence from the B1 image signal acquired through the image processing switching unit 60 A signal and a G1 image signal that has been subjected to correction processing for removing the signal value of the first laser light from the G1 image signal are used.

  When the signal ratio calculated according to the calculation mode is calculated in this way, the oxygen saturation calculation unit 83 calculates the oxygen saturation for each pixel based on the calculated signal ratio (S17: oxygen saturation calculation step). ). When the oxygen saturation is calculated, the image generation unit 84 generates an oxygen saturation image (S18). When the mode for calculating the oxygen saturation is set to the first mode, the B2 image signal, the G2 image signal, according to the oxygen saturation calculated based on the signal ratio B1 / G2 and the signal ratio R2 / G2, An oxygen saturation image is generated by applying gain to the R2 image signal. On the other hand, when the mode for calculating the oxygen saturation is set to the second mode, the B1 image signal and the G1 image are selected according to the oxygen saturation calculated based on the signal ratio B1 / G1 and the signal ratio R1 / G1. An oxygen saturation image is generated by applying a gain to the signal and the R1 image signal. The generated oxygen saturation image is displayed on the monitor 18 (S19: display step). These operations are repeated until the normal observation mode is switched (S20) or the diagnosis is finished (S21).

  As described above, in the first mode set when the movement of the observation target is small, both the first image signal obtained in the first frame and the second image signal obtained in the second frame are used. The oxygen saturation is calculated and an oxygen saturation image is generated. On the other hand, in the second mode that is set when the movement of the observation target is large, the oxygen saturation level is determined using only the first image signal obtained in the first frame without using the second image signal obtained in the second frame. Calculated and an oxygen saturation image is generated. The switching between the first mode and the second mode is automatically performed based on the amount of movement.

  In the first mode, particularly accurate oxygen saturation can be calculated. However, since the image signal for two frames is required for calculation of oxygen saturation, the calculation accuracy of oxygen saturation is calculated when there is a movement in the observation target. May get worse. In the second mode, the signal ratios B1 / G1, R1 / G1 are calculated after correcting the two image signals of the B1 image signal and the G2 image signal in order to increase the calculation accuracy of the oxygen saturation. Since it is necessary, the calculation accuracy of oxygen saturation is inferior to that of the first mode. However, since only one frame of image signal is used, the calculation accuracy of oxygen saturation is difficult to be lowered by movement of the observation target. For this reason, the endoscope system 10 detects the movement of the observation target by calculating the movement amount of the observation target as described above, and detects the movement of the observation target according to the magnitude of the movement of the observation target calculated as the movement amount. The first mode and the second mode are automatically switched.

  As a result, when there is almost no movement in the observation target, such as during a close examination that is performed without substantially moving the position and orientation of the tip 24, the first mode is automatically selected, and oxygen with higher accuracy than during screening. Saturation can be calculated and presented. Furthermore, when there is a large movement in the observation target, such as during screening performed while changing the position and orientation of the distal end portion 24, the second mode is automatically selected, and accuracy that does not cause misdiagnosis is ensured. Accurate oxygen saturation can be calculated and presented. Of course, even if there is a movement in the observation target due to a peristaltic movement or the like at the time of detailed examination, if the movement is large, the mode is automatically switched to the second mode, and accurate oxygen saturation calculation and presentation are maintained. Even during screening, if there is no movement in the observation target, the mode is automatically switched to the first mode, and the oxygen saturation can be calculated and presented with higher accuracy than when the second mode is continued. That is, the endoscope system 10 can calculate and present the oxygen saturation as accurately as possible even when the observation target moves.

[Second Embodiment]
The endoscope system of the second embodiment includes the motion amount calculation unit 65 of the first embodiment, the mode switching unit 80, the signal ratio calculation unit 81, and the oxygen saturation calculation unit 83 of the oxygen saturation image generation unit 76. The motion amount calculation unit 265, the mode switching unit 280, the signal ratio calculation unit 281, and the oxygen saturation calculation unit 283 shown in FIG. Other configurations are the same as those of the endoscope system 10 of the first embodiment.

When the observation mode switching SW 22b is in the special observation mode, the motion amount calculation unit 265 acquires an image signal from the image processing switching unit 60, calculates a motion amount using the acquired image signal, and a method thereof. This is the same as the motion amount calculation unit 65 of the first embodiment. However, the motion amount calculation unit 265 divides the imaging region represented by the first image signal (or the second image signal) into a plurality of regions according to the setting, and calculates the motion amount in each region. For example, as illustrated in FIG. 14, the motion amount calculation unit 265 determines that the imaging region represented by the R1 image signal (reference R1) and the R2 image signal (reference R2) is a 3 × 3 area A _ij (i = 1 to 3, j = 1 to 3), and the motion amount M _ij is calculated for each region A _ij .

Further, the mode switching unit 280 compares the amount of movement with the threshold value for each region A _ij , and calculates a signal ratio calculation of a mode setting signal for switching the mode for calculating the oxygen saturation for each region A _ij according to the result. Input to the unit 281. This mode setting signal specifies the correspondence between each region A _ij and the mode setting for calculating the oxygen saturation. For example, as shown in FIG. 14 setting, if the motion amount M ₂₂ in the central region A ₂₂ is within the specific range defined by the threshold, the mode of calculating the oxygen saturation of the region A ₂₂ is the first mode Is done. Also, as in the upper left region A _11, if not within the specified range of motion amount M ₁₁ is determined by the threshold, the mode of calculating the oxygen saturation of the region A ₁₁ is set to the second mode . The threshold used by the mode switching unit 280 is the same as that used by the mode switching unit 80 of the first embodiment.

  The signal ratio calculation unit 281 calculates the signal ratio used by the oxygen saturation calculation unit 283 in accordance with the mode setting signal input from the mode switching unit 80. The signal ratio calculation method in each calculation mode is the same as that of the signal ratio calculation unit 81 of the first embodiment, but the signal ratio calculation unit 281 uses the calculation mode specified by the mode setting signal for each region. Calculate the signal ratio. Therefore, the signal ratio B1 / G2 and the signal ratio R2 / G2 are calculated in the region set in the first mode, and the signal ratio B1 / G1 and the signal ratio R1 / G1 are calculated in the region set in the second mode.

  The oxygen saturation calculation unit 283 calculates the oxygen saturation based on the signal ratio of each region input from the signal ratio calculation unit 281 and the correlation stored in the correlation storage unit 82. Therefore, in the region set in the first mode, the oxygen saturation is calculated using the signal ratio B1 / G2 and the signal ratio R2 / G2, and in the region set in the second mode, the signal ratio B1 / G1 and The oxygen saturation is calculated using the signal ratio R1 / G1.

As described above, dividing the image signal into a plurality of regions A _ij, calculates the motion amount M _ij in each region A _ij, calculating the oxygen saturation in each region A _ij based on the magnitude of the motion amount M _ij By setting the mode, it is possible to accurately calculate and present the oxygen saturation even when the movement of the observation target is partial. For example, in the region where there is no movement in the observation target, the first mode maintains particularly accurate calculation and presentation of the oxygen saturation, and in the region where the observation target moves, the accuracy of the oxygen saturation in the second mode is maintained. Is secured.

In FIG. 14, although the area is divided into 3 × 3 areas A _ij , the number and the number of areas are arbitrary, and the number of areas is different in the vertical and horizontal directions of the image signal, such as dividing into 4 × 5 areas. May be. Moreover, the shape of each area | region does not need to be a rectangle like FIG. 14, For example, you may divide | segment an image signal into several area | regions of a concentric ring.

  In addition, the minimum unit of the area where the image signal is divided is one pixel. In this case, each pixel is the region, and the mode for calculating the oxygen saturation is changed by calculating the amount of movement for each pixel. As described above, if the mode for calculating the oxygen saturation is changed by calculating the amount of movement for each pixel, the boundary between regions having different modes for calculating the oxygen saturation in the oxygen saturation image is not particularly noticeable.

[Third Embodiment]
The endoscope system according to the third embodiment includes a signal ratio calculation unit 81, an oxygen saturation calculation unit 83, and an image generation unit 84 according to the first embodiment, the signal ratio calculation unit 381, the oxygen saturation calculation unit illustrated in FIG. 383 and the image generation unit 384 are replaced. Further, in the endoscope system according to the third embodiment, the mode switching unit 80 is not provided, and the input destination of the motion amount calculated by the motion amount calculation unit 65 is the image generation unit 384. Other configurations are the same as those of the endoscope system 10 of the first embodiment.

  The signal ratio calculation unit 381 includes a first mode signal ratio calculation unit 381A and a second mode signal ratio calculation unit 381B. The first mode signal ratio calculation unit 381A calculates the first mode signal ratios B1 / G2 and R2 / G2, and the second mode signal ratio calculation unit 381B calculates the second mode signal ratios B1 / G1 and R1. / G1 is calculated. That is, the signal ratio calculation unit 381 always calculates both the signal ratios B1 / G2 and R2 / G2 for the first mode and the signal ratios B1 / G1 and R1 / G1 for the second mode regardless of the motion amount. To do. The method by which the first mode signal ratio calculation unit 381A and the second mode signal ratio calculation unit 381B calculate the signal ratio is the same as that of the signal ratio calculation unit 81 of the first embodiment.

  The oxygen saturation calculation unit 383 includes a first mode oxygen saturation calculation unit 383A and a second mode oxygen saturation calculation unit 383B. The first mode oxygen saturation calculation unit 383A is based on the signal ratios B1 / G2 and R2 / G2 calculated by the first mode signal ratio calculation unit 381A and the correlation stored in the correlation storage unit 82. The oxygen saturation of each pixel is calculated. The second mode oxygen saturation calculation unit 383B is based on the signal ratios B1 / G1 and R1 / G1 calculated by the second mode signal ratio calculation unit 381B and the correlation stored in the correlation storage unit 82. Thus, the oxygen saturation of each pixel is calculated. That is, the oxygen saturation calculation unit 383 always calculates the oxygen saturation in the first mode and calculates the oxygen saturation in the second mode regardless of the amount of motion.

  The image generation unit 384 includes a first mode image generation unit 384A, a second mode image generation unit 384B, and a weighting synthesis unit 385. The first mode image generation unit 384A generates an oxygen saturation image using the oxygen saturation calculated by the first mode oxygen saturation calculation unit 383A and the B2 image signal, the G2 image signal, and the R2 image signal. The oxygen saturation image (hereinafter referred to as a first mode image) generated by the first mode image generation unit 384A is an oxygen saturation image calculated when the first mode is set in the first embodiment. The second mode image generation unit 384B generates an oxygen saturation image using the oxygen saturation calculated by the second mode oxygen saturation calculation unit 383B and the B1 image signal, the G1 image signal, and the R1 image signal. To do. The oxygen saturation image calculated by the second mode image generation unit 384B (hereinafter referred to as a second mode image) is an oxygen saturation image calculated when the second mode is set in the first embodiment. Therefore, the image generation unit 384 always generates the first mode image and the second mode image regardless of the amount of motion.

  The weighting synthesis unit 385 generates a synthesized oxygen saturation image by synthesizing the first mode image and the second mode image. This synthesis is performed using a weighting coefficient K based on the amount of motion. Specifically, the weighting synthesis unit 385 first calculates the weighting coefficient K based on the motion amount input from the motion amount calculation unit 65. The weighting coefficient K is calculated as a value from “0” to “1”, and is “0” when the motion amount is “1” (the value of R2 / R1 when there is no motion in the observation target). When the value is the same as the threshold value, “1” is set. When the motion amount is between “1” and the threshold value, the value is set to a value within the range of “0” to “1” in proportion to the magnitude of the motion amount. If the amount of motion exceeds the threshold, the weighting coefficient K is set to “1”. When the weighting coefficient K is calculated, as shown in FIG. 16, the first mode image 391 is multiplied by “1-K”, the second mode image 392 is multiplied by “K”, and the signal values are summed for each pixel. Thus, a synthetic oxygen saturation image 393 is generated. The synthesized oxygen saturation image 393 generated in this way is output to the structure enhancement unit 78 as an oxygen saturation image for display and displayed on the monitor 18.

  When generating the synthetic oxygen saturation image 393 as described above, as shown in FIG. 17, as in the first embodiment, first, screening is performed in the normal observation mode (S310) to find a possible lesion site. In the case (S311), the mode switching SW 22b is operated to switch to the special observation mode (S312), and in this special observation mode, a possible lesion is diagnosed.

  In the special observation mode, the first and second white lights are alternately irradiated on the observation target in synchronization with the imaging frame of the sensor 48, so that the sensor 48 detects the R1 image signal, the G1 image signal, and the B1 image signal in the first frame. Since the R2 image signal, the G2 image signal, and the B2 image signal are output in the second frame, the motion amount is calculated by the motion amount calculation unit 65 as in the first embodiment (S313).

  Thereafter, in the first embodiment, the oxygen saturation is calculated and the oxygen saturation image is generated by switching between the first mode and the second mode based on the amount of movement. In this embodiment, the amount of movement is large. The signal ratios B1 / G2 and R2 / G2 for the first mode are calculated (S314: first signal ratio calculation step) and the signal ratios B1 / G1 and R1 / G1 for the second mode are Calculated (S315: second signal ratio calculation step). Then, based on the signal ratios for the first and second modes, the oxygen saturation of the first mode and the oxygen saturation of the second mode are calculated (S316: first oxygen saturation calculating step, S317). : Second oxygen saturation calculation step).

  When the oxygen saturation levels in the first and second calculation modes are thus calculated, a first mode image 391 is generated using the oxygen saturation level for the first mode (S318: first mode image generation step), and The second mode image 392 is generated using the oxygen saturation for the second mode (S319: second mode image generation step). In addition, the weighting synthesis unit 385 calculates a weighting coefficient K corresponding to the amount of motion calculated in step 313 (S320: weighting coefficient calculation step), and the first mode image 391 and the first mode image 391 at a ratio according to the weighting coefficient K. The two-mode image 392 is synthesized, and a synthetic oxygen saturation image 393 is generated (S321: synthesis step). In the present embodiment, the synthesized oxygen saturation image 393 generated as described above is displayed on the monitor 18 through structure enhancement or the like (S322: display step). Note that these operations are repeated until the normal observation mode is switched (S323) or the diagnosis is completed (S324), as in the first embodiment.

  As described above, both the first mode image 391 and the second mode image 392 are always generated, synthesized by applying the weighting coefficient K weighting based on the motion amount, and the synthesized oxygen saturation image 393 for display is synthesized. When generating, if the movement of the observation target is large, the ratio of the second mode image 392 in the synthetic oxygen saturation image 393 increases, and if the movement of the observation target is small, the first mode image 391 in the synthetic oxygen saturation image 393 is generated. The ratio of becomes higher. Of course, if the movement of the observation target is small, the synthetic oxygen saturation image 393 becomes the first mode image 391 itself, and if the movement of the observation target is large, the synthetic oxygen saturation image 393 becomes the second mode image 392 itself. When the magnitude of the movement of the observation object is between these values, the higher the reliability of each oxygen saturation calculated in the first and second modes, the stronger appears in the synthesized oxygen saturation image.

  According to this embodiment, it is possible to accurately calculate and present the oxygen saturation without making a doctor or the like aware of the switching between the first mode and the second mode. Further, compared to the case where the mode for calculating the oxygen saturation is switched for each region as in the second embodiment, the synthesized oxygen saturation image 393 of the present embodiment is an adjacent region having a different mode for calculating the oxygen saturation. Visibility is good because there is no boundary.

  Note that the image signal may be divided into a plurality of regions as in the second embodiment, and a composite image corresponding to the synthetic oxygen saturation image 393 may be generated in each region as described above. As described above, when this embodiment and the second embodiment are combined, the boundary of each region becomes inconspicuous, and an oxygen saturation image with good visibility can be provided.

[Fourth Embodiment]
As shown in FIG. 18, the endoscope system 400 of the fourth embodiment includes a motion amount calculation unit 465, an oxygen saturation image generation unit 476, and a light source control unit 440 that are different from those of the first embodiment. Other configurations are the same as those of the endoscope system 10 of the first embodiment.

  The motion amount calculation unit 465 stores at least one color of the first image signals obtained in the past frame, and the latest first frame image acquired via the past frame image and the image processing switching unit 60. Corresponding feature points are extracted from the image signal, a motion vector representing the direction and magnitude of the movement of the observation target is obtained from the positional relationship, and the magnitude is calculated as the amount of movement. Further, when a plurality of feature points are extracted, for example, an average value of the magnitudes of motion vectors obtained from each is calculated as a motion amount.

  The oxygen saturation image generation unit 476 calculates the oxygen saturation and generates the oxygen saturation image in the same manner as the oxygen saturation image generation unit 76 of the first embodiment, but the mode setting signal output from the mode switching unit 80 Is input not only to the signal ratio calculation unit 81 but also to the light source control unit 440.

  As shown in FIG. 19, when the mode setting signal input from the oxygen saturation image generation unit 76 is the first mode, the light source control unit 440 uses the first blue laser light source (473LD) as in the first embodiment. ) 34 and the second blue laser light source (445LD) 36 are alternately turned on in synchronization with the imaging frame. On the other hand, when the input mode setting signal is the second mode, the light source control unit 440 continues to receive the first blue laser light source until the mode setting signal for setting the first mode is received even in the special observation mode. Only (473LD) 34 is kept on. In this case, in both the first frame and the second frame, the observation target is irradiated with the first white light, and the sensor 48 outputs a B1 image signal, a G1 image signal, and an R1 image signal. For this reason, the oxygen saturation image generation unit 476 performs oxygen saturation calculation and oxygen saturation image generation in the first mode for each frame.

  According to the above control, the frame rate for displaying the image and the oxygen saturation image is automatically improved while accurately calculating and presenting the oxygen saturation while the observation target moves large (fast). The visibility of the observation target is improved.

  Also in the first to third embodiments, the motion amount may be calculated by the same method as the motion amount calculation unit 465 of the present embodiment.

[Fifth Embodiment]
As shown in FIG. 20, the light source device 14 of the endoscope system 700 includes an LED (Light Emitting Diode) light source unit 501 instead of the first and second blue laser light sources 34 and 36 and the light source control unit 40. An LED light source control unit 504 is provided. Further, the phosphor 44 is not provided in the illumination optical system 24 a of the endoscope system 500. Other than that, it is the same as the endoscope system 10 of the first embodiment.

  The LED light source unit 501 includes an R-LED 501a, a G-LED 501b, and a B-LED 501c as light sources that emit light limited to a specific wavelength band. As shown in FIG. 21, the R-LED 501a emits red band light (hereinafter simply referred to as red light) of about 600 to 650 nm, for example. The center wavelength of the red light is about 620 to 630 nm. The G-LED 501b emits about 500 to 600 nm of green band light (hereinafter simply referred to as green light) represented by a normal distribution. The B-LED 501c emits blue band light having a central wavelength of 445 to 460 nm (hereinafter simply referred to as blue light).

  Further, the LED light source unit 501 has a high-pass filter (HPF) 502 that is inserted into and extracted from the optical path of blue light emitted from the B-LED 501c. The high pass filter 502 cuts blue light having a wavelength band of about 450 nm or less and transmits light having a wavelength band longer than about 450 nm.

  The cutoff wavelength (about 450 nm) of the high-pass filter 502 is a wavelength at which the absorption coefficients of oxyhemoglobin and reduced hemoglobin are substantially equal (see FIG. 10), and the absorption coefficients of oxyhemoglobin and reduced hemoglobin are reversed at this wavelength. In the case of the present embodiment, the correlation stored in the correlation storage unit 82 is a case where the extinction coefficient of oxyhemoglobin is larger than the extinction coefficient of reduced hemoglobin. Therefore, a signal based on a wavelength band equal to or less than the cutoff wavelength is This can cause inaccurate oxygen saturation to be calculated. For this reason, when acquiring the B1 image signal for calculating at least the oxygen saturation, the high-pass filter 502 is used to prevent the observation target from being irradiated with light having a wavelength band equal to or less than the cutoff wavelength, thereby reducing the oxygen saturation. The calculation accuracy of is improved.

  Therefore, the high-pass filter 502 is inserted at the insertion position before the B-LED 501c in the special observation mode, and is retracted to the retraction position in the normal observation mode. The high-pass filter 502 is inserted / removed by the HPF insertion / extraction unit 503 under the control of the LED light source control unit 504.

  The LED light source control unit 504 controls turning on / off of the LEDs 501 a to 501 c of the LED light source unit 501 and insertion / extraction of the high-pass filter 502. Specifically, as shown in FIG. 22, in the normal observation mode, the LED light source control unit 504 lights all the LEDs 501a to 501c, and the high-pass filter 502 retracts from the optical path of the B-LED 501c. Thereby, white light on which blue light, green light, and red light are superimposed is irradiated on the observation target, and the sensor 48 images the observation target with the reflected light and outputs image signals of B, G, and R colors.

  On the other hand, as shown in FIG. 23, in the special observation mode, the LED light source control unit 504 inserts or retracts the high-pass filter 502 for each frame in a state where all the LEDs 501a to 501c are turned on. Accordingly, the observation target includes first mixed color light composed of blue light, green light, and red light with a wavelength band of 450 nm or less cut, and blue light and green light that is not cut with a wavelength band of 450 nm or less. The second mixed color light composed of red light is alternately irradiated. The first mixed color light corresponds to the first white light of the first embodiment, and the second mixed light corresponds to the second white light of the first embodiment.

  Then, the imaging control unit 49 reads out the signal charge obtained by imaging the observation target under the first mixed color light during the first frame readout period, and outputs the B1 image signal, the G1 image signal, and the R1 image signal. To do. Further, the signal charge obtained by imaging the observation target under the second mixed color light is read during the reading period of the second frame, and the B2 image signal, the G2 image signal, and the R2 image signal are output. Subsequent processing can be performed in the same manner as the endoscope system 10.

  The first and second mixed color lights are the first and second illumination lights having different emission spectra. The R-LED 501a, the G-LED 501b, the B-LED 501c, and the high-pass filter 502 have an emission spectrum on the observation target. A light source that emits different first and second illumination lights is configured.

[Reference example]
As shown in FIG. 24, the light source device 14 of the endoscope system 600 includes a broadband light source 601, a rotation filter 602, and a rotation instead of the first and second blue laser light sources 34 and 36 and the light source control unit 40. A filter control unit 603 is provided. In addition, the sensor 605 of the endoscope system 600 is a monochrome imaging element that is not provided with a color filter. For this reason, the DSP 56 does not perform processing peculiar to the color image sensor such as demosaic processing. About other than that, it is the same as the endoscope system 10 of 1st Embodiment.

  The broadband light source 601 includes, for example, a xenon lamp, a white LED, and the like, and emits white light whose wavelength band ranges from blue to red. The rotary filter 602 includes a normal observation mode filter 610 and a special observation mode filter 611 (see FIG. 25), and the white light emitted from the broadband light source 601 is normally on the optical path on which the light guide 41 is incident. It is movable in the radial direction between a first position for the normal observation mode where the observation mode filter 610 is disposed and a second position for the special observation mode where the special observation mode filter 611 is disposed. The mutual movement of the rotary filter 602 to the first position and the second position is controlled by the rotary filter control unit 603 according to the selected observation mode. The rotary filter 602 rotates in accordance with the imaging frame of the sensor 605 in a state where the rotary filter 602 is disposed at the first position or the second position. The rotation speed of the rotary filter 602 is controlled by the rotary filter control unit 603 according to the selected observation mode.

  As shown in FIG. 25, the normal observation mode filter 610 is provided on the inner periphery of the rotary filter 602. The normal observation mode filter 610 includes an R filter 610a that transmits red light, a G filter 610b that transmits green light, and a B filter 610c that transmits blue light. Therefore, when the rotary filter 602 is arranged at the first position for the normal light observation mode, white light from the broadband light source 601 is selected from the R filter 610a, the G filter 610b, and the B filter 610c according to the rotation of the rotary filter 602. Is incident on. For this reason, the observation target is sequentially irradiated with red light, green light, and blue light according to the transmitted filter, and the sensor 605 images the observation target with these reflected lights, thereby obtaining an R image signal, A G image signal and a B image signal are sequentially output.

  The special observation mode filter 611 is provided on the outer periphery of the rotary filter 602. The special observation mode filter 611 includes an R filter 611a that transmits red light, a G filter 611b that transmits green light, a B filter 611c that transmits blue light, and a narrow band that transmits 473 ± 10 nm narrow band light. And a filter 611d. Therefore, when the rotary filter 602 is arranged at the second position for the normal light observation mode, white light from the broadband light source 601 is converted into an R filter 611a, a G filter 611b, a B filter 611c, a narrow band according to the rotation of the rotary filter 602. The light enters one of the filters 611d. Therefore, the observation target is sequentially irradiated with red light, green light, blue light, and narrowband light (473 nm) according to the transmitted filter, and the sensor 605 images each of the observation targets with these reflected lights. Thus, an R image signal, a G image signal, a B image signal, and a narrowband image signal are sequentially output.

  The R image signal and the G image signal obtained in the special observation mode correspond to the R1 (or R2) image signal and the G1 (or G2) image signal of the first embodiment. Further, the B image signal obtained in the special observation mode corresponds to the B2 image signal of the first embodiment, and the narrowband image signal corresponds to the B1 image signal. Therefore, the subsequent processing can be performed in substantially the same manner as the endoscope system 10 of the first embodiment.

  However, in the endoscope system of the present reference example, the oxygen saturation calculation unit 83 calculates the oxygen saturation using three image signals of the narrowband image signal, the R image signal, and the G image signal. If it is determined that the movement of the observation target is small, the image generation unit 84 generates an oxygen saturation image using the oxygen saturation and R, G, and B image signals. That is, when the movement of the observation target is small, the oxygen saturation can be calculated, and an image signal for a total of four frames can be used to generate an oxygen saturation image, and an accurate oxygen saturation can be presented. This mode corresponds to the calculation of the oxygen saturation and the generation of the oxygen saturation image in the first mode of the first embodiment.

  On the other hand, when it is determined that the movement of the observation target is large, the image generation unit 84 generates an oxygen saturation image using the calculated oxygen saturation and the narrowband image signal, the R image signal, and the G image signal. . That is, when the movement of the observation target is large, only the image signals for a total of three frames are used to calculate the oxygen saturation and generate an oxygen saturation image. For this reason, accurate oxygen saturation can be presented while suppressing blurring of an observation target in the oxygen saturation image. This mode corresponds to the calculation of the oxygen saturation and the generation of the oxygen saturation image in the second mode of the first embodiment.

  The broadband light source 601 and the rotary filter 602 constitute a light source that emits first and second illumination lights having different emission spectra. In the case of this embodiment, a series of light irradiated to the observation target by using the special observation mode filter 611 is the first illumination light, and a series of light irradiated to the observation target by using the normal observation mode filter 610. Is the second illumination light.

  In the first to fifth embodiments and the reference example, the oxygen saturation is calculated based on the signal ratio B1 / G2 and the signal ratio R2 / G2, or the signal ratio B1 / G1 and the signal ratio R1 / G1. However, the oxygen saturation may be calculated based only on the signal ratio B1 / G2 or the signal ratio B1 / G1. In this case, the correlation storage unit 82 may store the correlation between the signal ratio B1 / G2 or the signal ratio B1 / G1 and the oxygen saturation.

  In the first to fifth embodiments and the reference example, an oxygen saturation image obtained by imaging oxygen saturation is generated and displayed. In addition, a blood volume image obtained by imaging blood volume is generated and displayed. You may do it. Since the blood volume is correlated with the signal ratio R2 / G2 (or R1 / G1), the blood volume image obtained by imaging the blood volume by assigning different colors according to the signal ratio R2 / G2 (or R1 / G1). Can be created.

  In the first to fifth embodiments and reference examples, oxygen saturation is calculated, but instead of this, or in addition to this, an oxygenated hemoglobin index obtained from “blood volume × oxygen saturation (%)”, Other biological function information such as a reduced hemoglobin index obtained from “blood volume × (1−oxygen saturation) (%)” may be calculated.

In the first to fifth embodiments and reference examples, a CCD image sensor is used as the sensor 48, but a CMOS image sensor may be used as the sensor 48. However, the CMOS image sensor is driven by a so-called rolling shutter system, and signal charges are accumulated and read out in order for each row of pixels (each of 1 to N rows). For this reason, since the timing of signal charge accumulation and readout in each row differs for each row, it is desirable to switch between the first white light and the second white light in accordance with the readout timing. For example, as shown in FIG. 26, in the normal observation mode, the second white light is irradiated from the start of accumulation of the Nth row (time T ₁ ) to the completion of accumulation of the first row (time T ₂ ). The irradiation of the second white light is stopped from the start of reading of the first row (time T ₂ ) to the completion of reading of the Nth row (time T ₃ ). As shown in FIG. 27, in the special observation mode, the second white light is irradiated from the start of accumulation of the Nth row (time T ₁ ) to the completion of accumulation of the first row (time T ₂ ). The irradiation of the second white light is stopped from the start of reading of the first row (time T ₂ ) to the completion of reading of the Nth row (time T ₃ ). Then, in the next frame, during the period from the start of accumulation of the Nth row (time T ₃ ) to the completion of accumulation of the first row (time T ₄ ), irradiation with the first white light is performed, while readout of the first row is started. The irradiation of the first white light is stopped from the time (time T ₄ ) to the completion of reading of the N-th row (time T ₅ ). This makes it possible to unify the length (exposure amount) of the substantial charge accumulation period of each row and prevent the signal from the first white light and the signal from the second white light from being mixed. Even when a CMOS image sensor is used, an accurate oxygen saturation can be calculated as in the above embodiments. The same applies when the LED light source unit 501, the broadband light source 601 and the rotary filter 602 are used instead of the first and second blue laser light sources 34 and 36.

  In the first to fifth embodiments and the reference example, the correlation stored in the correlation storage unit 82 is commonly used in the first mode and the second mode. However, as shown in FIG. The mode correlation 82A and the second mode correlation 82B may be stored in the correlation storage unit 82, and the corresponding correlation may be used in each calculation mode.

10, 400, 500, 600 Endoscope system 18 Monitor 65, 265 Motion amount calculation unit 76 Oxygen saturation image generation unit 80, 280 Mode switching unit 81, 281, 381 Signal ratio calculation unit 83, 283, 383 Oxygen saturation Calculation unit 84,384 Image generation unit

Claims (12)

A light source that emits first illumination light and second illumination light having different emission spectra;
An imaging device that images the observation target under illumination with the first illumination light and outputs a first image signal, images the observation target under illumination with the second illumination light, and outputs a second image signal;
A motion amount calculation unit for calculating a motion amount of the observation target;
A first mode in which oxygen saturation is calculated using the first and second image signals when the amount of movement falls within a specific range, or the first image signal when the amount of movement does not fall within the specific range. An oxygen saturation calculation unit for calculating the oxygen saturation by a second mode for calculating the oxygen saturation using
An endoscope system comprising: The motion amount calculation unit calculates one motion amount in the entire imaging region represented by the first image signal or the second image signal,
The endoscope system according to claim 1, wherein the first and second modes are collectively changed over the entire imaging region. The motion amount calculation unit divides an imaging region represented by the first image signal or the second image signal into a plurality of regions, calculates the motion amount for each region,
The oxygen saturation calculation unit calculates oxygen saturation in the first mode for the region where the motion amount falls within the specific range, and the second region for the region where the motion amount does not fall within the specific range. The endoscope system according to claim 1, wherein the oxygen saturation is calculated in a mode.   The endoscope system according to claim 3, wherein the motion amount calculation unit calculates the motion amount for each pixel of the first image signal or the second image signal.   Light source control for controlling the light emission timing of the first illumination light and the second illumination light and alternately emitting the first illumination light and the second illumination light in both cases of the first and second modes. The endoscope system according to any one of claims 1 to 4, further comprising a unit.   The emission timing of the first illumination light and the second illumination light is controlled. In the first mode, the first illumination light and the second illumination light are alternately emitted, and in the second mode. The endoscope system according to any one of claims 1 to 4, further comprising a light source control unit that emits only the first illumination light.   The motion amount calculation unit calculates the motion amount based on a ratio between a red image signal included in the first image signal and a red image signal included in the second image signal. The endoscope system according to any one of the above. The first mode is a calculation mode for calculating the oxygen saturation based on a ratio of a blue image signal included in the first image signal and a green image signal included in the second image signal,
The said 2nd mode is a calculation mode which calculates the said oxygen saturation based on the ratio of the blue and green image signal contained in the said 1st image signal, The inside of any one of Claims 1-7 Endoscopic system. A light source that emits first illumination light and second illumination light having different emission spectra;
An imaging device that images the observation target under illumination with the first illumination light and outputs a first image signal, images the observation target under illumination with the second illumination light, and outputs a second image signal;
A motion amount calculation unit for calculating a motion amount of the observation target;
The oxygen saturation is calculated in a first mode in which oxygen saturation is calculated using the first and second image signals and in a second mode in which oxygen saturation is calculated using the first image signal. An oxygen saturation calculator,
Further, the oxygen saturation calculation unit
A first mode image generator that generates a first mode image as an oxygen saturation image representing the oxygen saturation calculated in the first mode;
A second mode image generation unit that generates a second mode image as an oxygen saturation image representing the oxygen saturation calculated in the second mode;
The first mode image and the second mode are weighted such that the ratio of the second mode image is increased as the movement amount is increased, and the ratio of the first mode image is increased as the movement amount is decreased. A weighting synthesis unit that generates a synthetic oxygen saturation image by synthesizing the images;
An endoscope system comprising: A light source that emits first illumination light and second illumination light having different emission spectra from each other, and an image of an observation target being illuminated with the first illumination light, outputs a first image signal, and is illuminated with the second illumination light In an endoscope system processor comprising: an image pickup device that picks up an image of the observation target and outputs a second image signal;
A receiving unit for receiving the first image signal and the second image signal;
A motion amount calculation unit for calculating a motion amount of the observation target;
A first mode in which oxygen saturation is calculated using the first and second image signals when the amount of movement falls within a specific range, or the first image signal when the amount of movement does not fall within the specific range. An oxygen saturation calculation unit for calculating the oxygen saturation by a second mode for calculating the oxygen saturation using
A processor device of an endoscope system comprising: The light source emits first illumination light and second illumination light having emission spectra different from each other, the imaging element images the observation target under illumination with the first illumination light, outputs a first image signal, and the second An imaging step of imaging the observation object under illumination with illumination light and outputting a second image signal;
A motion amount calculating unit for calculating a motion amount of the observation target;
A first mode in which the oxygen saturation calculation unit calculates oxygen saturation using the first and second image signals when the amount of motion falls within a specific range, or the amount of motion does not fall within the specific range. An oxygen saturation calculating step for calculating the oxygen saturation by a second mode in which the oxygen saturation is calculated using the first image signal,
A method of operating an endoscope system comprising: A light source that emits first illumination light and second illumination light having different emission spectra from each other, and an image of an observation target being illuminated with the first illumination light, outputs a first image signal, and is illuminated with the second illumination light An image pickup device that picks up an image of the observation target and outputs a second image signal, and an operation method of a processor device used in an endoscope system,
A receiving step in which a receiving unit receives the first image signal and the second image signal;
A motion amount calculating unit for calculating a motion amount of the observation target;
A first mode in which the oxygen saturation calculation unit calculates oxygen saturation using the first and second image signals when the amount of motion falls within a specific range, or the amount of motion does not fall within the specific range. An oxygen saturation calculating step for calculating the oxygen saturation by a second mode in which the oxygen saturation is calculated using the first image signal,
A method of operating a processor device comprising:

Images