JP6218709B2 - Endoscope system, processor device, operation method of endoscope system, and operation method of processor device - Google Patents

Description

  The present invention relates to an endoscope system, a processor device, an operation method of an endoscope system, and an operation method of a processor device that irradiates a plurality of illumination lights having different wavelength bands and images the observation object with the return light. .

  In the medical field, diagnosis using an endoscope system including a light source device, an endoscope system, and a processor device is widely performed. In medical diagnosis using an endoscope system, an insertion portion of an endoscope is inserted into a subject, and illumination light is irradiated from the distal end portion to an observation target. Then, the observation target irradiated with the illumination light is imaged by the imaging sensor at the tip, and an image of the observation target is generated using the obtained image signal and displayed on the monitor.

  In an endoscope system, an observation object is usually imaged using white light as illumination light, and an image that can be observed in a natural color (hereinafter referred to as a normal observation image) is acquired and displayed. An endoscope system is known that obtains an image in which a specific tissue or the like included in an observation target is emphasized (hereinafter, referred to as a special observation image) by using light having a specific wavelength band as illumination light. For example, an endoscope system that displays a normal observation image and a special observation image to be observed side by side on a monitor is known (Patent Documents 1 and 2). Thus, when displaying the normal observation image and the special observation image side by side on the monitor, the doctor makes a diagnosis while comparing the normal observation image displayed on the monitor with the special observation image. It is required that the same observation object is imaged almost simultaneously and is clear enough to be used for diagnosis. For this reason, in the endoscope system of Patent Document 1, when there is a still image acquisition instruction, a pair of images with less image movement (observation target movement) within a certain period of time from the still image acquisition instruction. Selected and displayed. Further, the endoscope system of Patent Document 2 displays only an image usable for diagnosis on the monitor by stopping the update of the special observation image when the movement of the observation target is large.

JP 2012-239757 A Japanese Patent No. 5203861

  In recent endoscope systems, a plurality of illumination lights having different wavelength bands are used to sequentially image an observation target, and signal processing using the obtained image signals (hereinafter referred to as multiframe signal processing) is performed. Assist the doctor's diagnosis by extracting specific tissues and structures (hereinafter referred to as specific tissues, etc.) included in the observation target, and generating and displaying images in which specific tissues are emphasized There is a case. In addition, multi-frame signal processing is performed to obtain an index indicating the state of the observation target, such as oxygen saturation of blood hemoglobin and blood vessel density, and the result is simulated or numerically colored. In some cases, a doctor's diagnosis may be assisted. In order to appropriately perform multiframe signal processing for assisting diagnosis in this way, it is possible to perform signal processing more accurately than in the case where a plurality of types of images are simply displayed side by side as in Patent Documents 1 and 2. It is necessary to select a suitable image signal. For example, even with images that have the same degree of synchrony and fineness that can be used for diagnosis by a doctor's visual observation, the observation target is unclear due to the movement (blurring) of the observation target that is difficult to discern visually or due to a focus shift or the like. There is something wrong. Even if it is difficult to discriminate from the appearance of the image, if the multi-frame signal processing is performed using an image signal or an image signal with a sharp observation target, the result will be inaccurate. Cheap. In particular, local movement and unsharpness of the observation target are not conspicuous as a whole image, but greatly affect the result of multiframe signal processing.

  The present invention provides an endoscope system capable of obtaining appropriate results when performing multi-frame signal processing using a plurality of image signals obtained by irradiating a plurality of illumination lights having different wavelength bands and imaging an observation target. It is an object of the present invention to provide a processor device, an operation method of an endoscope system, and an operation method of a processor device.

An endoscope system of the present invention includes a light source, an image sensor, an image signal acquisition unit, a storage unit, an unsharpness calculation unit, a movement amount calculation unit, an image signal selection unit, a signal processing unit, Is provided. The light source emits illumination light. The imaging sensor images the observation target with the return light of the illumination light. The image signal acquisition unit acquires a first image signal corresponding to the first illumination light in the illumination light, and corresponds to the second illumination light having a wavelength band different from the first illumination light in the illumination light. A second image signal is acquired. The storage unit stores a plurality of first image signals and a plurality of second image signals. The unsharpness calculation unit calculates the first unsharpness that is the unsharpness of the observation target represented by the first image signal, and the second unsharpness that is the unsharpness of the observation target represented by the second image signal. Calculate the degree. The movement amount calculation unit uses a plurality of first image signals, a plurality of second image signals, or a first image signal and a second image signal acquired at different timings to display an observation target represented by the first image signal. A first movement amount representing the amount of movement is calculated, and a second movement amount representing the amount of movement of the observation target represented by the second image signal is calculated. The image signal selection unit uses the first unsharpness, the second unsharpness, the first movement amount, and the second movement amount to store a plurality of first image signals and a plurality of second images. A specific first image signal and a specific second image signal are selected from the signals. The signal processing unit performs signal processing using the specific first image signal and the specific second image signal selected by the image signal selection unit. The image signal selection unit calculates a first reference value that is a reference for selecting a specific first image signal using the first unsharpness and the first moving amount, and the second unsharpness And a reference value calculation unit for calculating a second reference value that is a reference for selecting a specific second image signal using the second movement amount, and using the first reference value, the specific first image signal is obtained. A specific second image signal is selected using the second reference value.

  The reference value calculation unit calculates the first reference value by setting the first priority for the first unsharpness and the first movement amount, and the second for the second unsharpness and the second movement amount. It is preferable to set the priority and calculate the second reference value.

  When the first reference value is smaller, the image signal selection unit has a lower first unsharpness and a smaller first movement amount, and the first image having the smallest first reference value among the plurality of first image signals. When the signal is selected and the first unsharpness is lower and the first moving amount is smaller as the first reference value is larger, the first image signal having the largest first reference value among the plurality of first image signals When the second unsharpness is low and the second moving amount is small as the second reference value is small, the second image signal having the smallest second reference value is selected from the plurality of second image signals. When the second reference value is larger and the second unsharpness is lower and the second movement amount is smaller, the second image signal having the largest second reference value is selected from the plurality of second image signals. It is preferable to do.

  The reference value calculation unit calculates the first reference value by adding the first unsharpness and the first movement amount, and calculates the second reference value by adding the second unsharpness and the second movement amount. It is preferable to do.

  A still image acquisition instruction unit for inputting a still image acquisition instruction for instructing acquisition of a still image is provided, and the image signal selection unit is configured to input a still image acquisition instruction before inputting a still image acquisition instruction, or after inputting a still image acquisition instruction. It is preferable to select a specific first image signal and a specific second image signal from among the plurality of first image signals and the plurality of second image signals acquired in the first specific period before and after the input of the acquisition instruction.

  The image signal selection unit selects one of the specific first image signal and the specific second image signal from the plurality of first image signals and the plurality of second image signals acquired in the first specific period, The other is preferably selected from a plurality of first image signals or a plurality of second image signals obtained in a second specific period shorter than the first specific period.

  The light source alternately generates the first illumination light and the second illumination light, and the imaging sensor alternately images the observation target with the return light of the first illumination light and the return light of the second illumination light, and selects an image signal. It is preferable that the unit selects the first image signal and the second image signal that are continuously captured by setting the second specific period.

The unsharpness calculating unit calculates the first unsharpness using the amount or ratio of the high-frequency component of the first image signal, and the second unsharpness using the amount or ratio of the high-frequency component of the second image signal. It is preferable to calculate the degree.

  The unsharpness calculating unit calculates the unsharpness of the blood vessel included in the observation target represented by the first image signal as the first unsharpness, and determines the unsharpness of the blood vessel included in the observation target represented by the second image signal. It is preferable to calculate the second unsharpness.

  The unsharpness calculation unit calculates the first unsharpness using the distribution of the thickness of the blood vessel included in the observation target represented by the first image signal, and the thickness of the blood vessel included in the observation target represented by the second image signal. It is preferable to calculate the second unsharpness using the thickness distribution.

A processor device of the present invention is a processor device for an endoscope system having a light source that emits illumination light and an imaging sensor that captures an observation target with return light of the illumination light, an image signal acquisition unit, and a storage unit And an unsharpness calculation unit, a movement amount calculation unit, an image signal selection unit, and a signal processing unit. The image signal acquisition unit acquires a first image signal corresponding to the first illumination light in the illumination light, and corresponds to the second illumination light having a wavelength band different from the first illumination light in the illumination light. The second image signal to be acquired is acquired. The storage unit stores a plurality of first image signals and a plurality of second image signals. The unsharpness calculation unit calculates the first unsharpness that is the unsharpness of the observation target represented by the first image signal, and the second unsharpness that is the unsharpness of the observation target represented by the second image signal. Calculate the degree. The movement amount calculation unit uses a plurality of first image signals, a plurality of second image signals, or a first image signal and a second image signal acquired at different timings to display an observation target represented by the first image signal. A first movement amount representing the amount of movement is calculated, and a second movement amount representing the amount of movement of the observation target represented by the second image signal is calculated. The image signal selection unit uses the first unsharpness, the second unsharpness, the first movement amount, and the second movement amount to store a plurality of first image signals and a plurality of second images. A specific first image signal and a specific second image signal are selected from the signals. The signal processing unit performs signal processing using the specific first image signal and the specific second image signal selected by the image signal selection unit. The image signal selection unit calculates a first reference value that is a reference for selecting a specific first image signal using the first unsharpness and the first moving amount, and the second unsharpness And a reference value calculation unit for calculating a second reference value that is a reference for selecting a specific second image signal using the second movement amount, and using the first reference value, the specific first image signal is obtained. A specific second image signal is selected using the second reference value.

The operation method of the endoscope system of the present invention includes an illumination light generation step, an imaging step, a first image signal acquisition step, a second image signal acquisition step, an image signal storage step, and an unsharpness calculation step. A moving amount calculating step, an image signal selecting step, and a signal processing step. In the illumination light generation step, the light source generates illumination light. In the imaging step, the imaging sensor images the observation target with the return light of the illumination light. In the first image signal acquisition step, the image signal acquisition unit acquires the first image signal corresponding to the first illumination light of the illumination light from the imaging sensor. In the second image signal acquisition step, the image signal acquisition unit acquires the second image signal corresponding to the second illumination light having a wavelength band different from that of the first illumination light from the imaging sensor. In the image signal storage step, the storage unit stores the first image signal and the second image signal acquired by the image signal acquisition unit. In the unsharpness calculating step, the unsharpness calculating unit calculates the first unsharpness, which is the unsharpness of the observation target represented by the first image signal, and the unsharpness of the observation target represented by the second image signal. The second unsharpness is calculated. In the movement amount calculation step, the movement amount calculation unit uses the plurality of first image signals, the plurality of second image signals, or the first image signal and the second image signal acquired at different timings to generate the first image. A first movement amount representing the amount of movement of the observation target represented by the signal is calculated, and a second movement amount representing the amount of movement of the observation target represented by the second image signal is calculated. In the image signal selection step, the image signal selection unit uses the first unsharpness, the second unsharpness, the first movement amount, and the second movement amount to store a plurality of first image signals stored in the storage unit. And a specific first image signal and a specific second image signal are selected from the plurality of second image signals. In the signal processing step, the signal processing unit performs signal processing using the specific first image signal and the specific second image signal selected by the image signal selection unit. The image signal selection unit calculates a first reference value that is a reference for selecting a specific first image signal using the first unsharpness and the first movement amount, and the second unsharpness and the A reference value calculation unit that calculates a second reference value that is a reference for selecting a specific second image signal using two movement amounts, and selects the specific first image signal using the first reference value; The specific second image signal is selected using the second reference value.

An operating method of a processor device according to the present invention is an operating method of a processor device for an endoscope system having a light source that emits illumination light and an imaging sensor that captures an observation object with return light of the illumination light. An image signal acquisition step, a second image signal acquisition step, an image signal storage step, an unsharpness calculation step, a movement amount calculation step, an image signal selection step, and a signal processing step are provided. In the first image signal acquisition step, the image signal acquisition unit acquires the first image signal corresponding to the first illumination light of the illumination light from the imaging sensor. In the second image signal acquisition step, the image signal acquisition unit acquires the second image signal corresponding to the second illumination light having a wavelength band different from that of the first illumination light from the imaging sensor. In the image signal storage step, the storage unit stores the first image signal and the second image signal acquired by the image signal acquisition unit. In the unsharpness calculating step, the unsharpness calculating unit calculates the first unsharpness, which is the unsharpness of the observation target represented by the first image signal, and the unsharpness of the observation target represented by the second image signal. The second unsharpness is calculated. In the movement amount calculation step, the movement amount calculation unit uses the plurality of first image signals, the plurality of second image signals, or the first image signal and the second image signal acquired at different timings to generate the first image. A first movement amount representing the amount of movement of the observation target represented by the signal is calculated, and a second movement amount representing the amount of movement of the observation target represented by the second image signal is calculated. In the image signal selection step, the image signal selection unit uses the first unsharpness, the second unsharpness, the first movement amount, and the second movement amount to store a plurality of first image signals stored in the storage unit. And a specific first image signal and a specific second image signal are selected from the plurality of second image signals. In the signal processing step, the signal processing unit performs signal processing using the specific first image signal and the specific second image signal selected by the image signal selection unit. The image signal selection unit calculates a first reference value that is a reference for selecting a specific first image signal using the first unsharpness and the first movement amount, and the second unsharpness and the A reference value calculation unit that calculates a second reference value that is a reference for selecting a specific second image signal using two movement amounts, and selects the specific first image signal using the first reference value; The specific second image signal is selected using the second reference value.

  In the present invention, when performing multi-frame signal processing using a plurality of image signals obtained by irradiating a plurality of illumination lights having different wavelength bands and imaging the observation target, the amount of movement and the unsharpness of the observation target are determined. Since an image signal to be used for multi-frame signal processing is selected by calculation and used, an endoscope system, a processor device, an operation method of the endoscope system, and an operation method of the processor device that can obtain appropriate results Can be provided.

It is an external view of an endoscope system. It is a block diagram which shows the function of an endoscope system. It is explanatory drawing which shows the image signal memorize | stored in memory. It is explanatory drawing which shows the range which selects the image signal used for signal processing. It is a flowchart which shows the operation | movement aspect of an endoscope system. It is a table | surface which shows the typical unsharpness of a 1st image signal, the moving amount | distance, and a reference value. It is a table | surface which shows the typical unsharpness, the moving amount | distance, and reference value of a 2nd image signal. It is explanatory drawing which shows a 1st specific period and a 2nd specific period. It is a table | surface which shows the selection method of the image signal of a modification. It is explanatory drawing of the 1st specific period set to the range before the input of a still image acquisition instruction | indication. It is explanatory drawing of the 1st specific period set to the range after the input of a still image acquisition instruction | indication. It is explanatory drawing of the 1st specific period set to the range before and behind the input of a still image acquisition instruction | indication. It is the schematic of a capsule endoscope.

  As shown in FIG. 1, the endoscope system 10 includes an endoscope 12, a light source device 14, a processor device 16, a monitor 18, and a console 19. The endoscope 12 is optically connected to the light source device 14 and electrically connected to the processor device 16. The endoscope 12 includes an insertion portion 12a to be inserted into a subject, an operation portion 12b provided at a proximal end portion of the insertion portion 12a, a bending portion 12c and a distal end portion provided at the distal end side of the insertion portion 12a. 12d. By operating the angle knob 12e of the operation unit 12b, the bending unit 12c performs a bending operation. By this bending operation, the distal end portion 12d is directed in a desired direction.

  In addition to the angle knob 12e, the operation unit 12b is provided with a still image acquisition instruction unit 13a and a zoom operation unit 13b. The still image acquisition instruction unit 13 a is used to input a still image acquisition instruction to the endoscope system 10. The still image acquisition instruction includes a freeze instruction for displaying the still image to be observed on the monitor 18 and a release instruction for saving the still image displayed on the monitor 18 in the storage. The endoscope system 10 performs multi-frame signal processing in response to an input of a still image acquisition instruction, but performs the same multi-frame signal processing whether the still image acquisition instruction is a freeze instruction or a release instruction. For this reason, in the present embodiment, the freeze instruction and the release instruction are not distinguished from each other and are simply referred to as a still image acquisition instruction. The zoom operation unit 13b is used to input an imaging magnification change instruction.

  The processor device 16 is electrically connected to the monitor 18 and the console 19. The monitor 18 outputs and displays an image to be observed and information attached to the image. The console 19 functions as a user interface that receives input operations such as function settings.

  As shown in FIG. 2, the light source device 14 includes a light source 20 that emits illumination light that irradiates an observation target, and a light source control unit 22 that controls the light source 20. The light source 20 includes, for example, a semiconductor light source such as a multi-color LED (Light Emitting Diode) or a lamp such as a xenon lamp. The light source 20 includes an optical filter for adjusting the wavelength band of light emitted from the LED or the like. The light source control unit 22 controls the wavelength band of the illumination light irradiated on the observation target by changing the light emission amount of the LED or the like or the optical filter. Thereby, the light source control part 22 controls the light source 20, and the 1st illumination light which has a 1st wavelength band, and the 2nd illumination light which has a 2nd wavelength band from which a wavelength band differs from 1st illumination light At least two types of illumination light are irradiated onto the observation object. In the present embodiment, the light source control unit 22 controls the light source 20 in accordance with the timing at which the observation target is imaged, and alternately irradiates the observation target with the first illumination light and the second illumination light.

  The wavelength bands of the first illumination light and the second illumination light may partially or entirely overlap. For example, as in the case where the first illumination light is white light and the second illumination light is blue light, the second illumination light (or first illumination light) is the first illumination light (second illumination light). The light having a part of the wavelength band may be used. Further, the first illumination light and the second illumination light only have to have substantially different spectral spectra. For example, in the case where white light having a large component in the green wavelength band and white light having a large component in the blue wavelength band are respectively used as the first illumination light and the second illumination light, strictly speaking, the first illumination light and the second illumination light are used. Even if the wavelength bands of the first illumination light and the second illumination light are substantially the same, the wavelength bands are substantially the same. Therefore, in this specification, the combination of the first illumination light and the second illumination light is also different in wavelength band. say.

  The illumination light emitted from the light source 20 is incident on the light guide 41 inserted into the insertion portion 12a. The light guide 41 is built in the endoscope 12 and the universal cord (the cord connecting the endoscope 12, the light source device 14, and the processor device 16), and the illumination light is transmitted to the distal end portion 12d of the endoscope 12. Propagate. A multimode fiber can be used as the light guide 41. As an example, a thin fiber cable having a core diameter of 105 μm, a clad diameter of 125 μm, and a diameter of φ0.3 to 0.5 mm including a protective layer serving as an outer skin can be used.

  The distal end portion 12d of the endoscope 12 is provided with an illumination optical system 30a and an imaging optical system 30b. The illumination optical system 30 a has an illumination lens 45, and the illumination light propagated by the light guide 41 is irradiated to the observation target through the illumination lens 45. The imaging optical system 30 b includes an objective lens 46, a zoom lens 47, and an imaging sensor 48. Various types of light (hereinafter referred to as return light) such as reflected light, scattered light, and fluorescence from the observation target enter the image sensor 48 via the objective lens 46 and the zoom lens 47. As a result, an image to be observed is formed on the image sensor 48. The zoom lens 47 is freely moved between the tele end and the wide end by operating the zoom operation unit 13b, and enlarges or reduces the reflected image of the observation target formed on the image sensor 48.

  The imaging sensor 48 images the observation target with the return light of the illumination light. The imaging sensor 48 is a color imaging sensor in which any one of R (red), G (green), and B (blue) color filters is provided for each pixel. Is output. The image signal output by the image sensor 48 when the illumination light is the first illumination light is a first image signal corresponding to the first illumination light, and the image sensor 48 outputs when the illumination light is the second illumination light. The image signal is a second image signal corresponding to the second illumination light. In the present embodiment, the imaging sensor 48 alternately irradiates the observation target with the first illumination light and the second illumination light in accordance with the timing at which the imaging target 48 images the observation target. Two image signals are alternately output.

  As the image sensor 48, a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal-Oxide Semiconductor) image sensor can be used. Further, instead of the primary color imaging sensor 48, a complementary color imaging sensor including complementary color filters of C (cyan), M (magenta), Y (yellow), and G (green) may be used. When the complementary color image sensor is used, CMYG four-color image signals are output. By converting the CMYG four-color image signals into RGB three-color image signals by complementary color-primary color conversion, An RGB image signal similar to that of the image sensor 48 can be obtained. Further, instead of the imaging sensor 48, a monochrome sensor without a color filter may be used.

  An image signal output from the image sensor 48 is transmitted to the CDS / AGC circuit 50. The CDS / AGC circuit 50 performs correlated double sampling (CDS) and automatic gain control (AGC) on an image signal that is an analog signal. The image signal that has passed through the CDS / AGC circuit 50 is converted into a digital image signal by an A / D (Analog to Digital) converter 52. The digital image signal after A / D conversion is input to the processor device 16.

  The processor device 16 includes an image signal acquisition unit 53, a DSP (Digital Signal Processor) 56, a noise removal unit 58, a memory 61, an unsharpness calculation unit 62, a movement amount calculation unit 63, and an image signal selection unit. 66, a multi-frame signal processing unit 71, a storage 72, and a video signal generation unit 76.

  The image signal acquisition unit 53 acquires a digital image signal from the endoscope 12. Specifically, the image signal acquisition unit 53 acquires a first image signal corresponding to the first illumination light in the illumination light, and acquires a second image signal corresponding to the second illumination light.

  The DSP 56 performs various signal processing such as defect correction processing, offset processing, gain correction processing, linear matrix processing, gamma conversion processing, and demosaicing processing on the acquired image signal. In the defect correction process, the signal of the defective pixel of the image sensor 48 is corrected. In the offset process, the dark current component is removed from the image signal subjected to the defect correction process, and an accurate zero level is set. In the gain correction process, the signal level is adjusted by multiplying the image signal after the offset process by a specific gain.

  The image signal after gain correction processing is subjected to linear matrix processing for improving color reproducibility. After that, brightness and saturation are adjusted by gamma conversion processing. The image signal after the gamma conversion processing is subjected to demosaic processing (also referred to as isotropic processing or synchronization processing), and a signal of a color that is insufficient at each pixel is generated by interpolation. By this demosaic processing, all the pixels have RGB signals. The noise removal unit 58 removes noise by performing noise removal processing (for example, using a moving average method, a median filter method, or the like) on the image signal that has been demosaiced by the DSP 56. The image signal from which noise has been removed is stored in the memory 61.

  The memory 61 is a storage unit that temporarily stores a plurality of image signals in the order of acquisition. The memory 61 has a storage capacity capable of storing a plurality of first image signals and a plurality of second image signals at the same time. The signals are deleted from the memory 61 in order from the signal, and the latest image signal is stored instead. The first image signal and the second image signal stored in the memory 61 are read by the unsharpness calculation unit 62, the movement amount calculation unit 63, and the multiframe signal processing unit 71.

In the following, as shown in FIG. 3, each acquisition time of a plurality of image signals is represented by t _N , t _N−1 , t _N−2 ,... Using an integer “N” that represents the image signal acquisition timing. , Represented by first image signals P _N , P _N-2 , P _N-4 ,... Acquired at each time, and represented by second image signals Q _N−1 , Q _N−3 , Q _N−5 ,. For example, a first image signal obtained at time _{t N} is _{P N,} the second imaging signal obtained at time _{t N-1} is a _{Q N-1.} Acquisition interval of the interval and the image signal the image sensor 48 images the observation target is equidistant at T _0.

The unsharpness calculation unit 62 reads the first image signal stored in the memory 61 and calculates the first unsharpness that is the unsharpness of the observation target represented by the first image signal. The unsharpness is, for example, a value proportional to the reciprocal of the sharpness or a value obtained by subtracting the sharpness from a specific value. In the case of an image signal obtained by the endoscope system 10, it is possible to determine the degree of unsharpness according to how clearly a blood vessel, a pit pattern, or the like on the surface layer to be observed can be observed. An image signal (image signal with high sharpness and low unsharpness) that can clearly observe an image of these specific fine tissues or the like contains a lot of corresponding high-frequency components. Therefore, the unsharpness calculating unit 62 calculates the amount or ratio of the high-frequency component corresponding to the surface blood vessel, the pit pattern, etc. included in the observation target represented by the first image signal from the first image signal, and the calculated high-frequency component The first unsharpness is calculated using the amount or ratio. The higher the value of the first unsharpness, the smaller the amount or ratio of the high-frequency component included in the first image signal, and the higher the unsharpness (the lower the sharpness). The first unsharpness is calculated for each first image signal, and only the first image signal for calculating the first unsharpness is used to calculate the first unsharpness. For example, the first unsharpness of the first image signal P _N acquired at time t _N is calculated using only the first image signal P _N.

Further, the unsharpness calculating unit 62 reads the second image signal stored in the memory 61 and calculates the second unsharpness that is the unsharpness of the observation target represented by the second image signal. The method for calculating the second unsharpness is the same as the method for calculating the first unsharpness relating to the first image signal. Further, the second unsharpness is calculated for each second image signal, and only the second image signal for calculating the second unsharpness is used for calculating the second unsharpness. For example, the second unsharpness of the second image signal Q _N−1 acquired at time t _N−1 is calculated using only the second image signal Q _N−1 .

The movement amount calculation unit 63 is a first movement that represents the amount of movement of the observation target represented by the first image signal acquired later with respect to the observation target represented by the first image signal or the second image signal acquired previously. Calculate the amount. That is, the movement amount calculation unit 63 uses a plurality of first image signals acquired at different timings, or a first image signal and a second image signal acquired at different timings. The amount of movement is calculated. For example, when calculating the first amount of movement of the first image signal obtained at time t _N, the shift amount calculating part 63 and the first image signal P _N from the memory 61, obtains just prior to the first image signal P _N The second image signal Q _N−1 thus read is read out. Then, the feature point of the observation target represented by the first image signal _PN is obtained, and the feature point of the observation subject represented by the second image signal Q _N-1 is obtained. Next, a vector representing the displacement of the corresponding feature point among the feature points of the first image signal _{PN and} the feature points of the second image signal Q _N−1 is obtained. Thereafter, a statistic such as the total value, average value, or median value of the size and rotation angle of each vector between corresponding feature points is calculated. Statistics calculated based on the displacement vector of the feature point thus, or evaluation values obtained by combining the calculated statistic is a first amount of movement of the first image signal P _N. The greater the value of the first movement amount, the greater the movement of the observation target represented by the first image signal.

Similarly, the movement amount calculation unit 63 represents the amount of movement of the observation target represented by the second image signal acquired later with respect to the observation target represented by the first image signal or the second image signal acquired previously. The second movement amount is calculated. That is, the movement amount calculation unit 63 uses a plurality of second image signals acquired at different timings, or the second image signal second and second image signals acquired at different timings. The amount of movement is calculated. For example, when calculating the second movement amount of the second image signal Q _N−1 acquired at time t _N−1 , the movement amount calculation unit 63 receives the second image signal Q _N−1 from the memory 61 and the second image signal Q _N−1 . The first image signal P _N-2 acquired immediately before the image signal Q _N-1 is read, and the second movement amount of the second image signal Q _N-1 is calculated using these. A specific method of calculating the second movement amount using the second image signal Q _N-1 and the first image signal P _N-2 is the same as the first movement amount. Accordingly, the larger the value of the second movement amount, the greater the movement of the observation target represented by the second image signal.

  In the present embodiment, as described above, the movement amount calculation unit 63 uses the second image signal acquired first and the first image signal acquired later, and the first image signal acquired later. Although one movement amount is calculated, the first movement amount of the first image signal acquired later may be calculated using the first image signal acquired earlier and the first image signal acquired later. . Similarly, the movement amount calculation unit 63 calculates the second movement amount of the second image signal acquired later using the first image signal acquired earlier and the second image signal acquired later. However, the second movement amount of the second image signal acquired later may be calculated using the second image signal acquired earlier and the second image signal acquired later. That is, the first movement amount may be calculated using the first image signals, and the second movement amount may be calculated using the second image signals. When calculating the first movement amount using the first image signals, the two first image signals used for calculating the first movement amount are the first image signals acquired closest in time. It is preferable that The same applies when the second movement amount is calculated using the second image signals.

  The image signal selection unit 66 uses the first unsharpness, the second unsharpness, the first movement amount, and the second movement amount to store a plurality of first image signals and a plurality of second image signals stored in the memory 61. A specific first image signal and a specific second image signal used in the multi-frame signal processing unit 71 are selected from the image signals. Further, the image signal selection unit 66 includes a reference value calculation unit 67. The reference value calculation unit 67 calculates the first reference value using the first unsharpness and the first movement amount, and calculates the second reference value using the second unsharpness and the second movement amount. . The first reference value is a reference for selecting a specific first image signal to be used by the multi-frame signal processing unit 71 from among a plurality of first image signals stored in the memory 61, and each first image signal Is calculated for each. Similarly, the second reference value is a reference for selecting a specific second image signal to be used by the multiframe signal processing unit 71 from among a plurality of second image signals stored in the memory 61. It is calculated for each of the two image signals. That is, the image signal selection unit 66 selects a specific first image signal and a specific second image signal using the first reference value and the second reference value.

  For example, the reference value calculation unit 67 calculates the first reference value of each first image signal by weighting and adding the first unsharpness and the first movement amount of each first image signal. When the first reference value is calculated, the first weighting coefficient to be applied to the first unsharpness and the first moving amount is a priority between the first unsharpness and the first moving amount (hereinafter referred to as the first priority). Degree). That is, the reference value calculation unit 67 calculates the first reference value by setting the first priority to the first unsharpness and the first movement amount. For example, when the first unsharpness is more important than the first movement amount, the first unsharpness is preferentially reflected in the first reference value by increasing the weight of the first unsharpness. The first weighting coefficient is set according to the observation object or according to the individual difference of the subject. Further, as is apparent from the calculation method, in this embodiment, the greater the value of the first reference value, the greater the first unsharpness and the greater the first movement amount. Therefore, it can be said that the smaller the value of the first reference value, the more suitable the first image signal is for multiframe signal processing.

  Further, the reference value calculation unit 67 calculates the second reference value of each second image signal by, for example, weighting and adding the second unsharpness and the second movement amount of each second image signal. To do. When the second reference value is calculated, the second weighting coefficient multiplied by the second unsharpness and the second moving amount is a priority between the second unsharpness and the second moving amount (hereinafter referred to as second priority). It is set according to the object to be observed. That is, the reference value calculation unit 67 sets the second priority to the second unsharpness and the second movement amount, and calculates the second reference value. The calculation method of the second reference value is the same as that of the first reference value. The larger the second reference value, the greater the second unsharpness and the greater the second movement amount. Therefore, it can be said that the smaller the value of the second reference value, the more suitable the second image signal for multiframe signal processing.

  The first weighting coefficient for calculating the first reference value may be set to the same value as described above, but the second weighting coefficient for calculating the second reference value may be different. . Since the wavelength bands of the first illumination light and the second illumination light are different, there is a difference in the structure that can be observed even if the same observation target is imaged. For this reason, for example, the first weighting coefficient and the second weighting coefficient may be set to different values according to the wavelength band of the illumination light (that is, according to the structure that can be observed). Further, the reference value calculation unit 67 may use a product obtained by weighting the first unsharpness and the first movement amount as the first reference value, and the first unsharpness and the first movement amount as arguments, and between them. The first reference value may be calculated according to other functions including weighting. The same applies to the calculation method of the second reference value.

As shown in FIG. 4, the time range in which the image signal selection unit 66 selects the first image signal and the second image signal is a specific period according to the input timing of the still image acquisition instruction (hereinafter referred to as the first specific period). ) it is defined in the _{T 1} that. The first specific period T1 is a period that can be regarded as almost simultaneous with the input of the still image acquisition instruction. The plurality of first image signals and the plurality of second image signals acquired in the first specific period T ₁ can be regarded as being acquired almost simultaneously with the input timing of the still image acquisition instruction, and in the first specific period T ₁ . It can be considered that the acquired image signals are acquired almost simultaneously. For example, after the acquisition time t _N-1 of the second image signal Q _N-1, the still image acquisition instruction until the acquisition time t _N of the first image signal P _N have been inputted, the image signal selecting section 66 Is a specific first image signal used in the multi-frame signal processing unit 71 from among the first image signals P _N , P _N-2 ,..., P _N-10 acquired within the first specific period T ₁ . The specific second image signal to be selected and used by the multi-frame signal processing unit 71 is the second image signal Q _N−1 , Q _N−3 ,... Q _N− acquired within the first specific period T ₁ . Select from ₉ . In the present embodiment, the image signal selection unit 66 compares the first reference values of the first image signals P _N , P _N-2 ,..., P _N-10 included in the first specific period T ₁ , The first image signal having the smallest first reference value is selected. Similarly, the image signal selection unit 66 compares the second reference values of the second image signals Q _N−1 , Q _N−3 ,... Q _N−9 included in the first specific period T ₁ and compares the second reference values. 2 The second image signal having the smallest reference value is selected. Thus, defining a first image signal and the time range for selecting the second image signal used in the multi-frame signal processing unit 71 to the first specific time period T _1, a first image signal obtained in the first specific time period T ₁ and By selecting the first image signal and the second image signal that are small and sharp in the movement of the observation target from the second image signal, respectively, without making a difference with the input timing of the still image acquisition instruction, The image signal selection unit 66 selects a specific first image signal and a specific second image signal that are kept synchronized and are suitable for multiframe signal processing.

When there are a plurality of first image signals having the smallest first reference value, the image signal selection unit 66 is the latest of the first image signals having the smallest first reference value that is closest to the input of the still image acquisition instruction. A first image signal is selected. The same applies when there are a plurality of second image signals having the smallest second reference value. 4 schematically shows six first image signals P _N , P _N-2 ,..., P _N-10 and five second image signals Q _N during the first specific period T _{1. −1} , Q _N-3 ,..., Q _N-9 are included, but a first specific period T ₁ including a larger number of first image signals and second image signals may be set, or a smaller number. the first may be set a specific time period T ₁ including the first and second image signals of.

  When a still image acquisition instruction is input, the multi-frame signal processing unit 71 reads the specific first image signal and the specific second image signal selected by the image signal selection unit 66 from the memory 61 and uses them. Perform multi-frame signal processing. For example, the multi-frame signal processing unit 71 uses the first image signal and the second image signal selected by the image signal selection unit 66 to calculate an index representing the state of the observation target for each pixel, and based on the calculated index. Then, a special observation image of the observation target that is colored is generated. Further, when generating the special observation image, the multi-frame signal processing unit 71 performs color conversion processing, color enhancement processing, and structure enhancement processing as necessary. In the color conversion processing, color conversion processing is performed on the image signal by 3 × 3 matrix processing, gradation conversion processing, three-dimensional LUT (look-up table) processing, and the like. The color enhancement process is performed on the image signal that has been subjected to the color conversion process. The structure enhancement process is a process for enhancing a structure to be observed such as a surface blood vessel or a pit pattern, and is performed on the image signal after the color enhancement process.

  The special observation image generated by the multiframe signal processing unit 71 through the multiframe signal processing is input to the video signal generation unit 76. The video signal generation unit 76 converts the special observation image generated by the multi-frame signal processing unit 71 into a video signal for output to the monitor 18. Further, the special observation image generated by the multiframe signal processing unit 71 by the multiframe signal processing is stored in the storage 72.

  Note that until the still image acquisition instruction is input and after the multiframe signal processing using the specific first image signal and the specific second image signal selected by the image signal selection unit 66 is completed. The multi-frame signal processing unit 71 reads the latest first image signal and second image signal from the memory 61, and performs multi-frame signal processing similar to the above using these to generate an image to be observed. For this reason, the moving image to be observed is displayed on the monitor 18 in real time even when no still image acquisition instruction is input.

  Next, the operation flow of the endoscope system 10 of the present embodiment will be described along the flowchart shown in FIG. First, the light source 20 generates first illumination light under the control of the light source control unit 22 (S11: first illumination light generation step), and the imaging sensor 48 images the observation target with the return light of the first illumination light (S12). : First imaging step). In the processor device 16, the image signal acquisition unit 53 acquires a first image signal corresponding to the first illumination light from the imaging sensor 48 (S <b> 13: first image signal acquisition step). The first image signal acquired by the image signal acquisition unit 53 is subjected to signal processing such as demosaic processing by the DSP 56, and after noise is removed by the noise removal unit 58, it is stored in the memory 61 (S14: image signal storage) Step).

  At this time, if the still image acquisition instruction unit 13a is not operated to input a still image acquisition instruction (S15), the light source 20 generates the second illumination light under the control of the light source control unit 22 (S21: second illumination). In the light generation step), the imaging sensor 48 images the observation target with the return light of the second illumination light (S22: second imaging step). When the observation target is imaged with the return light of the second illumination light in this way, the image signal acquisition unit 53 acquires a second image signal corresponding to the second illumination light from the imaging sensor 48 (S23: acquisition of the second image signal). Step). The second image signal acquired by the image signal acquisition unit 53 is subjected to demosaic processing, noise removal processing, and the like as in the case of the first image signal, and is additionally stored in the memory 61.

  The above steps are repeated until a still image acquisition instruction is input (S25), and the memory 61 stores a plurality of first image signals and a plurality of second image signals. During this time, the multi-frame signal processing unit 71 reads the latest first image signal and second image signal from the memory 61, performs multi-frame signal processing using them, and calculates an index representing the state of the observation target. In addition, an image to be observed representing the calculated index in color is generated and displayed on the monitor 18. While viewing the moving image displayed on the monitor 18 in real time, the doctor obtains a timing for acquiring and displaying a still image for detailed examination.

When a still image acquisition instruction is input (S15 or S25), the unsharpness calculation unit 62 receives the _first acquired from the plurality of first image signals stored in the memory 61 in the first specific period T1. the first unsharpness of the image signal are calculated, respectively, and, among the plurality of second image signal stored in the memory 61, the second unsharpness of the second image signal acquired in the first specific period T ₁ (S31: Unsharpness calculation step). The moving amount calculating unit 63, among the plurality of first image signal stored in the memory 61, and calculates a first amount of movement of the first image signal obtained in the first specific period T _1, respectively, and, in the second image signal stored in the memory 61, calculates a second movement amount of the second image signal acquired in the first specific period T _1, respectively (S32: movement amount calculating step).

Thereafter, in the image signal selection unit 66, the reference value calculation unit 67 uses the first unsharpness, the second unsharpness, the first movement amount, and the second movement amount to specify a specific frame used for multiframe signal processing. A first reference value and a second reference value for selecting the first image signal and the specific second image signal are calculated for each first image signal and each second image signal (S33: reference value calculating step). More specifically, as shown in FIG. 6, the reference value calculation unit 67 performs the first non-indication for each of the first image signals P _N , P _N-2 ,..., P _N-8 , P _N-10. The sharpness and the first movement amount are added to calculate a first reference value for each of the first image signals P _N , P _N-2 ,..., P _N-8 , P _N-10 . Further, as shown in FIG. 7, the reference value calculation unit 67 performs the second unsharpness for each of the second image signals Q _N−1 , Q _{N− 3} ,..., Q _N−7 , Q _N−9. And the second movement amount are added to calculate a second reference value for each of the second image signals Q _N−1 , Q _N−3 ,..., Q _N−7 , Q _N−9 .

The image signal selection unit 66 uses the first reference values calculated by the reference value calculation unit 67 as described above, and the first image signals P _N , P _N-2 ,..., P _N-8 , P _N-10. selecting a first image signal used for the multi-frame signal processing from among, and using the second reference value, the second image signal _{Q N-1, Q N-} 3, ..., Q N-7, Q _A second image signal to be used for multiframe signal processing is selected from _N-9 (S34: image signal selection step). The first reference values of the first image signals P _N , P _N-2 ,..., P _N-8 , P _N-10 are “43”, “42”, ..., “45”, “44”, respectively. In the case (see FIG. 6), the image signal selection unit 66 selects the first image signal _PN-2 having the smallest first reference value. Similarly, the second reference values of the second image signals Q _N−1 , Q _N−3 ,..., Q _N−7 , Q _N−9 are “44”, “42”,. In the case of “43” (see FIG. 7), the image signal selection unit 66 selects the second image signal Q _N-7 having the minimum second reference value.

The multi-frame signal processing unit 71 performs multi-frame signal processing using the first image signal P _N-2 and the second image signal Q _N-7 selected by the image signal selection unit 66 (S35: signal processing). Step), an index representing the state of the observation target is calculated. Then, a special observation image representing the calculated index in color is generated and displayed on the monitor 18 as a still image. The doctor makes a diagnosis by examining the special observation image displayed on the monitor 18.

  As described above, when the still image acquisition instruction is input, the endoscope system 10 estimates the movement amount and the unsharpness of the observation target that seriously affects the multiframe signal processing, and determines the still image acquisition instruction. The first image signal and the second image signal that are most suitable for multiframe signal processing are selected in terms of the amount of movement and the unsharpness of the observation target within a range that can be regarded as being acquired almost simultaneously with the input. As a result, the endoscope system 10 ensures that the first image signal and the second image signal are simultaneously synchronized, and further, an error or the like caused by the movement or unsharpness of the observation target is most unlikely to occur. The first image signal and the second image signal can be used for multiframe signal processing.

  For example, in the conventional endoscope systems such as Patent Document 1 and Patent Document 2, there are cases where the simultaneity of a plurality of types of images is maintained based on an amount similar to the movement amount of the present embodiment, but the simultaneity is simply ensured. However, the result of multiframe signal processing may include a large error. On the other hand, the endoscope system 10 selects an image signal that is suitable for multiframe signal processing in terms of both the amount of movement of the observation target and the sharpness within a range in which simultaneity is maintained. Therefore, an appropriate multiframe signal processing result can be obtained by selecting an image signal suitable for multiframe signal processing based on both the movement amount and the sharpness of the observation target as in the endoscope system 10. it can.

In the above embodiment, the smaller the first reference value, the lower the first unsharpness and the smaller the first movement amount. That is, as the first reference value is smaller, small movements of the observation target, and, because of the high sharpness, the image signal selecting section 66, from among the plurality of first image signal acquired in the first specific period T ₁ Although the first image signal having the smallest first reference value is selected, the reference value calculation unit 67 uses the first unsharpness and the first moving amount, and the larger the value, the lower the first unsharpness. In addition, when calculating the first reference value having a small first movement amount, the image signal selection unit 66 selects the first image signal having the largest first reference value. Similarly, as the second reference value is smaller second unsharpness is low and, when the second movement amount is small, the image signal selector 66, a plurality of second image signal acquired in the first specific period T ₁ The second image signal having the smallest second reference value is selected from among the two, and if the second unsharpness is lower and the second movement amount is smaller as the second reference value is larger, the image signal selection unit 66 is selected. Selects the second image signal having the largest second reference value.

In the above embodiment, to select the specific first image signal used for multi-frame signal processing from among a plurality of first image signal acquired in the first specific time period T ₁ to be set based on the input timing of the still picture acquisition instruction and, although selecting a particular second image signal used for multi-frame signal processing from among the same first specific time period T ₁ a plurality of second image signal acquired in the first image signal or the second image signal whereas the after selecting from a plurality of first image signal or the plurality of second image signal acquired in the first specific time period T _1, the first image signal or the second image signal of the other of the first specific time period T it is preferably selected from a plurality of first image signal or the plurality of second image signal obtained in the second specific time period T ₂ shorter than _1. For example, as shown in FIG. 8, the first image signal P _N-4 having the smallest first reference value is selected from the first image signals P _N ,..., P _N-10 acquired in the first specific period T1. When the second image signal is selected before the second image signal, the second image signal used for the multi-frame signal processing is the entire second image signal Q _N−1 ,... Q _N-9 acquired in the first specific period T _1. The second image signals Q _N-3 and Q _N-5 acquired in the second specific period T ₂ that includes the first image signal P _N-4 selected earlier, but is shorter than the first specific period T _1. The second image signal having the smaller second reference value is selected from either of the two. In this case, the second image signal Q _N−1 ,... Q _N-9 acquired in the first specific period T ₁ is selected before the second image signal used for the multiframe signal processing is selected. The second image signal having higher synchronization with the first image signal _PN-4 can be used for multiframe signal processing.

Further, the second specific time period T _2, imaging interval and the image signal acquisition interval T ₀ or more imaging sensors 48, it is preferably set to less than 3 times the acquisition interval T ₀ of the imaging interval and the image signal. In this way, the second image signals Q _N-3 and Q _N-5 acquired in succession to the first image signal _PN-4 either before or after the previously selected first image signal _PN-4 . The second image signal used for multiframe signal processing is always selected from either one. Therefore, the second image signal having the highest synchronization with the previously selected first image signal _PN-4 can be used for multiframe signal processing together with the first image signal _PN-4 .

As described above, when the second specific period T ₂ is set in addition to the first specific period T ₁ and the first image signal and the second image signal are selected, the second specific period T ₂ falls within the range of the second specific period T ₂ . It is preferable to compare the reference value with the threshold value for the image signal selected in step (b). As shown in FIG. 8, when selecting the second image signal using either of the second image signal Q _N-3 or the second image signal Q _N-5 obtained in the second specific time period T ₂ in the multi-frame signal processing The second reference values of the second image signals Q _N-3 and Q _N-5 that are selection candidates are respectively compared with threshold values. Then, if there is a second image signal in which the moving amount and the unsharpness of the observation target are small enough to be used for multiframe signal processing and the second reference value is equal to or less than the threshold value, among the selection candidates, the second reference is selected. A second image signal to be used in multiframe signal processing is selected according to the value.

On the other hand, when the second image signal a second reference value is equal to or lower than a threshold is not in the selection candidate, the second image signal acquired in the second specific time period T ₂ are, the more unsuitable for use in all multi-frame signal processing It can be determined that the unsharpness or movement of the observation target is large. Therefore, when there is no second image signal whose second reference value is equal to or less than the threshold among the selection candidates, the first image signals P _N , P _N-2 ,..., P acquired in the first specific period T _{1 The} selection of the first image signal _PN-2 having the smallest first reference value selected first from _N-10 is canceled, and the first image signal having the second smallest first reference value is selected. Thereafter, the second specific period T ₂ is newly set according to the acquisition timing of the first image signal having the second first reference value that is newly selected, and acquired in the newly set second specific period T ₂ . Each second reference value of the second image signal is compared with a threshold value. Then, if there is a second image signal whose second reference value is equal to or less than the threshold, among the selection candidates, the second image signal used for multiframe signal processing is selected from the selection candidates, and the second image signal whose second reference value is equal to or less than the threshold If the two image signals are not included in the selection candidates, the first image signal having the third smallest first reference value is selected again as described above. By repeating this operation, by setting the second specific time period T _2, even if the selection candidate (an image signal for selecting later) the second image signal is small, the first image signal suitable to a multi-frame signal processing And a second image signal can be selected.

In the above modification, the first image signal ahead from the first image signal acquired in the first specific period T ₁ selected earlier, later, in the second image signal acquired in the second specific time period T ₂ However, conversely, the second image signal may be selected first, and the first image signal may be selected later.

In the above embodiment, the first image signal used for the multi-frame signal processing is selected using the first reference value of the first image signal. Separately, the second image signal used for the multi-frame signal processing is the first image signal. Although the second image signal is selected using the second reference value, the first image signal and the second image signal used for multiframe signal processing may be selected by a different method. For example, as shown in FIG. 9, the image signal selector 66, the sum of the first reference value and second reference value of the first image signal and the second image signal acquired in succession by the first inner specified period T ₁ And a pair of the first image signal and the second image signal having the smallest sum of the first reference value and the second reference value is selected as the first image signal and the second image signal used for multi-frame signal processing. Also good.

In the case of FIG. 9, when the first image signal having the smallest first reference value and the second image signal having the smallest second reference value are selected separately, the first image signal P _N-2 and the second image signal Q _{N are selected. -7} is selected, but when a pair of the first image signal and the second image signal that minimizes the sum of the first reference value and the second reference value is selected, the first image signal _PN-2 and the second image signal are selected. A pair of signals Q _N-3 is selected. In this way, when the first image signal and the second image signal are selected as a pair using the sum of the first reference value and the second reference value of the first image signal and the second image signal acquired successively, The simultaneity between the first image signal and the second image signal to be selected has more certain synchronism than selecting the first image signal and the second image signal separately, and the unsharpness of the observation target A pair of the first image signal and the second image signal that can be said to be most suitable for multiframe signal processing in terms of movement is selected.

  As described above, when the pair of the first image signal and the second image signal used for multiframe signal processing is selected using the sum of the first reference value and the second reference value, the first reference value and A priority may be set for the second reference value, and a sum of the first reference value and the second reference value calculated by weighting corresponding to the priority may be calculated. For example, depending on the contents of multiframe signal processing, the amount of movement and the unsharpness of the observation target represented by the first image signal have more influence on the processing result than the amount of movement and the unsharpness of the observation target represented by the second image signal. May be large. In such a case, the sum of the first reference value and the second reference value is calculated by weighting corresponding to the priority, and when a pair of the first image signal and the second image signal is selected based on the sum, A pair of the first image signal and the second image signal can be selected in consideration of the degree of influence on the multiframe signal processing result.

  As described above, when the first image signal and the second image signal acquired in succession are selected as a pair, the sum of the first unsharpness and the second unsharpness is calculated, and the first unsharpness is calculated. A pair of the first image signal and the second image signal that minimizes the sum of the degree and the second unsharpness may be selected as the first image signal and the second image signal used for multiframe signal processing. Similarly, when the first image signal and the second image signal acquired in succession are selected as a pair, the sum of the first movement amount and the second movement amount is calculated, and the calculated first movement amount and second movement are calculated. A pair of the first image signal and the second image signal that minimizes the sum of the amounts may be selected as the first image signal and the second image signal used for multiframe signal processing.

In the above-described embodiment, the image signal selection unit 66 acquires the image signal (first image signal P _N ) acquired immediately after the input of the still image acquisition instruction and the image signal (first image signal acquired before the input at the time of acquiring the still image). The first specific period T ₁ including the image signals P _N-2 ,... P _N-10 and the second image signals Q _N-1 ,... Q _N-9 ) is set, but as shown in FIG. the first specific time period T ₁ may be set before the input of the still image acquisition instruction. Further, as shown in FIG. 11, may be set first specified time period T ₁ after the input of the still image acquisition instruction. Then, as in the above embodiment, when setting the first specific time period T ₁ before and after the input of the still image acquisition instruction, as shown in FIG. 12, respectively before and after the input of the still image acquisition instruction multiple first images a first specific time period T ₁ including a signal and a plurality of second image signal may be set.

  The unsharpness calculator 62 calculates the first unsharpness using the amount or ratio of the high-frequency component of the first image signal, and the second unsharpness using the amount or ratio of the high-frequency component of the second image signal. Although the sharpness is calculated, in the case of an endoscope image signal, the sharpness of the observation target can be evaluated by the sharpness of a specific structure such as a blood vessel or a pit pattern. Instead of the ratio, the unsharpness of a specific structure such as a blood vessel or a pit pattern may be calculated as the first unsharpness and the second unsharpness. Similarly, for example, it can be evaluated that the observation target is sharper as the contrast of the superficial blood vessel is larger. Therefore, even when the first sharpness and the second sharpness are calculated using the contrast of the specific structure. good.

  In addition, the unsharpness calculating unit 62 may calculate the first unsharpness and the second unsharpness based on the distribution of a specific structure such as a blood vessel or a pit pattern. For example, when blood vessels are extracted from the image signal, thin surface blood vessels and thick intermediate / deep blood vessels can be extracted. And if there are few thin blood vessels in the extracted blood vessel, since fine surface blood vessels are not visible, it can be evaluated that the unsharpness of the observation target is high (the sharpness is low). If the extracted blood vessels contain a certain proportion of thin ones or more, fine surface blood vessels can be seen, so it can be evaluated that the unsharpness of the observation target is low (sharpness is high). Therefore, if a blood vessel is extracted and the distribution of the thickness of the blood vessel is used, the first unsharpness and the second unsharpness can be calculated.

  The fine specific structures such as blood vessels and pit patterns look different depending on the wavelength band of the illumination light. Therefore, the first unsharpness of the first image signal corresponding to the first illumination light and the second illumination light are supported. The second unsharpness of the second image signal may be calculated by different methods. Moreover, you may set the 1st weighting coefficient and 2nd weighting coefficient at the time of calculating a 1st reference value and a 2nd reference value according to the wavelength band of 1st illumination light and 2nd illumination light.

  In the above embodiment, the multi-frame signal processing is performed using the first image signal corresponding to the first illumination light and the second image signal corresponding to the second illumination light, but depending on the processing content of the multi-frame signal processing. May require more than two types of image signals. Even in such a case, the present invention is applicable. Specifically, illumination light having three or more different wavelength bands is generated, image signals corresponding to each illumination light are respectively acquired, and the reference value of each acquired image signal is calculated in the same manner as in the above embodiment. Thus, multi-frame signal processing may be selected based on the calculated reference value.

  Moreover, in the said embodiment, although the 1st image signal corresponding to 1st illumination light and the 2nd image signal corresponding to 2nd illumination light are acquired alternately, a 1st image signal and a 2nd image signal are acquired. Acquisitions need not be alternating. For example, after the first image signal is acquired twice, the first image signal and the second image signal may be acquired in a cycle in which one second image signal is acquired. The present invention is also applicable to such a case. The same applies when acquiring three or more types of image signals.

  In the above embodiment, the multi-frame signal processing unit 71 uses the first image signal and the second image signal selected by the image signal selection unit 66 to calculate an index that represents the state of the observation target for each pixel. Multi-frame signal processing that generates a special observation image that is colored based on the index, which is referred to as a specific tissue, etc., or that generates and displays an image in which the specific tissue is emphasized You can go.

  An example of the index calculated by the multi-frame signal processing unit 71 of the above embodiment is oxygen saturation. The oxygen saturation is obtained, for example, by imaging an image signal (hereinafter referred to as a B1 image signal) obtained by irradiating blue light having a wavelength band having a difference in absorption coefficient between oxyhemoglobin and reduced hemoglobin, and white light. The image signal corresponding to the green wavelength band (hereinafter referred to as G2 image signal) and the image signal corresponding to the red wavelength band (hereinafter referred to as R2 image signal) can be calculated. . More specifically, the oxygen saturation of the observation object is closely related to the signal ratio B1 / G2 between the B1 image signal and the G1 image signal and the signal ratio R2 / G2 between the R2 image signal and the G1 image signal. Therefore, the oxygen saturation is calculated from the signal ratio B1 / G2 and the signal ratio R2 / G2 by obtaining the correlation between the signal ratio B1 / G2, the signal ratio R2 / G2 and the oxygen saturation in advance through experiments or the like. can do. For this reason, when the oxygen saturation is calculated, multiframe signal processing for calculating the signal ratio B1 / G2 and the signal ratio R2 / G2 is performed. In this case, blue light having a wavelength band with a difference in absorption coefficient between oxyhemoglobin and deoxyhemoglobin is the first illumination light, and the B1 image signal output from the blue pixel is the first image signal. Further, white light is the second illumination light, and corresponds to the G2 image signal corresponding to the green wavelength band or the red wavelength band among the image signals of each color obtained when the observation target is imaged by irradiating the white light. Either of the R2 image signals can be used as the second image signal of the above embodiment.

  In the above embodiment, when a still image acquisition instruction is input, the first image signal and the second image signal used for multiframe signal processing are selected according to the first reference value and the second reference value, and the moving image is monitored. In the case of displaying on 18, multi-frame signal processing is performed using the latest first image signal and second image signal. However, when displaying a moving image on the monitor 18, as in the above embodiment, The first image signal and the second image signal to be used for multi-frame signal processing may be selected according to the 1 reference value and the second reference value.

  In the above embodiment, the present invention is implemented by the endoscope system 10 that performs observation by inserting the endoscope 12 provided with the imaging sensor 48 into the subject. However, the present invention is also applied to a capsule endoscope system. Is preferred. For example, as shown in FIG. 13, the capsule endoscope system includes at least a capsule endoscope 300 and a processor device (not shown). The capsule endoscope 300 includes a light source 302, a light source control unit 303, an image sensor 304, an image signal acquisition processing unit 306, and a transmission / reception antenna 308. The light source 302 emits first illumination light and second illumination light under the control of the light source control unit 303. The image signal acquisition processing unit 306 is an image signal acquisition unit that acquires an image signal from the imaging sensor 304 and is a multi-frame signal processing unit that performs multi-frame signal processing. The processor device of the capsule endoscope system is configured in substantially the same manner as the processor device 16 of the above embodiment, but the image signal acquisition processing unit 306 that performs multi-frame signal processing is provided in the capsule endoscope 300. The special observation image generated by the image signal acquisition processing unit 306 is transmitted to the processor device via the transmission / reception antenna 308.

DESCRIPTION OF SYMBOLS 10 Endoscope system 12 Endoscope 14 Light source apparatus 16 Processor apparatus 20,302 Light source 48,304 Image sensor 53 Image signal acquisition part 61 Memory 62 Unsharpness calculation part 63 Movement amount calculation part 66 Image signal selection part 67 Reference value Calculation unit 71 Multi-frame signal processing unit 300 Capsule endoscope

Claims (13)

A light source that emits illumination light;
An imaging sensor for imaging an observation object with the return light of the illumination light;
A first image signal corresponding to the first illumination light of the illumination light is obtained, and a second of the illumination light corresponding to the second illumination light having a wavelength band different from that of the first illumination light. An image signal acquisition unit for acquiring an image signal;
A storage unit for storing the plurality of first image signals and the plurality of second image signals;
A first unsharpness that is the unsharpness of the observation target represented by the first image signal is calculated, and a second unsharpness that is the unsharpness of the viewing object represented by the second image signal is calculated. An unsharpness calculation unit,
A plurality of the first image signals acquired at different timings, a plurality of the second image signals, or the observation target represented by the first image signal using the first image signal and the second image signal. A movement amount calculation unit that calculates a first movement amount that represents the amount of movement and that calculates a second movement amount that represents the amount of movement of the observation target represented by the second image signal;
Using the first unsharpness, the second unsharpness, the first movement amount, and the second movement amount, the plurality of first image signals and the plurality of second images stored in the storage unit. An image signal selection unit for selecting the specific first image signal and the specific second image signal from image signals;
A signal processing unit that performs signal processing using the specific first image signal and the specific second image signal selected by the image signal selection unit;
Equipped with a,
The image signal selection unit calculates a first reference value that is a reference for selecting the specific first image signal using the first unsharpness and the first movement amount, and the second A reference value calculation unit that calculates a second reference value that is a reference for selecting the specific second image signal using the unsharpness and the second movement amount is provided, and is specified using the first reference value. select the first image signal, you select a specific second image signal using the second reference value,
Endoscope system. The reference value calculation unit calculates the first reference value by setting a first priority to the first unsharpness and the first movement amount, and the second unsharpness and the second The endoscope system according to claim 1 , wherein the second reference value is calculated by setting a second priority to a movement amount. The image signal selection unit
The first image having the smallest first reference value among a plurality of the first image signals when the first unsharpness is low and the first moving amount is small as the first reference value is small. Select the signal
The first image having the largest first reference value among a plurality of the first image signals when the first unsharpness is lower and the first movement amount is smaller as the first reference value is larger. Select the signal
The second image having the smallest second reference value among a plurality of the second image signals when the second unsharpness is low and the second movement amount is small as the second reference value is small. Select the signal
If the second unsharpness is lower and the second movement amount is smaller as the second reference value is larger, the second image having the largest second reference value among the plurality of second image signals. The endoscope system according to claim 1 or 2 , wherein a signal is selected. The reference value calculation unit calculates the first reference value by adding the first unsharpness and the first moving amount, and adds the second unsharpness and the second moving amount. The endoscope system according to any one of claims 1 to 3 , wherein the second reference value is calculated. A still image acquisition instruction unit for inputting a still image acquisition instruction for instructing acquisition of a still image;
The image signal selection unit includes a plurality of the first image signals acquired before input of the still image acquisition instruction, after input of the still image acquisition instruction, or in a first specific period before and after input of the still image acquisition instruction. The endoscope system according to any one of claims 1 to 4 , wherein the specific first image signal and the specific second image signal are selected from the plurality of second image signals. The image signal selection unit includes the specific first image signal or the specific second image signal among the plurality of first image signals and the plurality of second image signals acquired in the first specific period. selects one of, and the other to claim 5 selected from a plurality of the first image signal or a plurality of the second image signal obtained at the shorter second specific period than the first specific time period The endoscope system described. The light source alternately generates the first illumination light and the second illumination light,
The imaging sensor images the observation object alternately with the return light of the first illumination light and the return light of the second illumination light,
The endoscope system according to claim 6 , wherein the image signal selection unit selects the first image signal and the second image signal that are continuously captured according to the setting of the second specific period. The unsharpness calculating unit calculates the first unsharpness using the amount or ratio of the high-frequency component of the first image signal, and uses the amount or ratio of the high-frequency component of the second image signal. The endoscope system according to any one of claims 1 to 7 , wherein the second unsharpness is calculated. The unsharpness calculating unit calculates the unsharpness of a blood vessel included in the observation target represented by the first image signal as the first unsharpness and is included in the observation target represented by the second image signal. The endoscope system according to any one of claims 1 to 7 , wherein a blood vessel unsharpness is calculated as the second unsharpness. The unsharpness calculating unit calculates the first unsharpness using a distribution of the thickness of blood vessels included in the observation object represented by the first image signal, and the observation object represented by the second image signal. The endoscope system according to any one of claims 1 to 7 , wherein the second unsharpness is calculated using a distribution of the thickness of blood vessels included in the blood vessel. In a processor device for an endoscope system, comprising: a light source that emits illumination light; and an imaging sensor that captures an observation target with return light of the illumination light.
A first image signal corresponding to the first illumination light of the illumination light is obtained, and a second of the illumination light corresponding to the second illumination light having a wavelength band different from that of the first illumination light. An image signal acquisition unit for acquiring an image signal;
A storage unit for storing the plurality of first image signals and the plurality of second image signals;
A first unsharpness that is the unsharpness of the observation target represented by the first image signal is calculated, and a second unsharpness that is the unsharpness of the viewing object represented by the second image signal is calculated. An unsharpness calculation unit,
A plurality of the first image signals acquired at different timings, a plurality of the second image signals, or the observation target represented by the first image signal using the first image signal and the second image signal. A movement amount calculation unit that calculates a first movement amount that represents the amount of movement and that calculates a second movement amount that represents the amount of movement of the observation target represented by the second image signal;
Using the first unsharpness, the second unsharpness, the first movement amount, and the second movement amount, the plurality of first image signals and the plurality of second images stored in the storage unit. An image signal selection unit for selecting the specific first image signal and the specific second image signal from image signals;
A signal processing unit that performs signal processing using the specific first image signal and the specific second image signal selected by the image signal selection unit;
Equipped with a,
The image signal selection unit calculates a first reference value that is a reference for selecting the specific first image signal using the first unsharpness and the first movement amount, and the second A reference value calculation unit that calculates a second reference value that is a reference for selecting the specific second image signal using the unsharpness and the second movement amount is provided, and is specified using the first reference value. select the first image signal, you select a specific second image signal using the second reference value,
Processor device. An illumination light generation step in which the light source generates illumination light;
An imaging step in which the imaging sensor images the observation object with the return light of the illumination light; and
A first image signal acquisition step in which an image signal acquisition unit acquires a first image signal corresponding to the first illumination light of the illumination light from the imaging sensor;
A second image signal acquisition step in which an image signal acquisition unit acquires a second image signal corresponding to a second illumination light having a wavelength band different from that of the first illumination light from the illumination sensor;
An image signal storage step in which a storage unit stores the first image signal and the second image signal acquired by the image signal acquisition unit;
The unsharpness calculation unit calculates the first unsharpness that is the unsharpness of the observation target represented by the first image signal, and is the unsharpness of the observation target represented by the second image signal. An unsharpness calculating step for calculating a second unsharpness;
The movement amount calculation unit uses the plurality of first image signals, the plurality of second image signals, or the first image signal and the second image signal acquired at different timings to generate the first image signal. A movement amount calculating step of calculating a first movement amount representing the amount of movement of the observation object represented by the second movement amount, and calculating a second movement amount representing the amount of movement of the observation object represented by the second image signal; ,
The image signal selection unit uses the first unsharpness, the second unsharpness, the first movement amount, and the second movement amount to store a plurality of the first image signals stored in the storage unit. And an image signal selection step of selecting the specific first image signal and the specific second image signal from the plurality of the second image signals;
A signal processing step in which a signal processing unit performs signal processing using the specific first image signal and the specific second image signal selected by the image signal selection unit;
Equipped with a,
The image signal selection unit calculates a first reference value that is a reference for selecting the specific first image signal using the first unsharpness and the first movement amount, and the second A reference value calculation unit that calculates a second reference value that is a reference for selecting the specific second image signal using the unsharpness and the second movement amount is provided, and is specified using the first reference value. Selecting the first image signal and selecting the specific second image signal using the second reference value.
How to operate the endoscope system. In an operating method of a processor device for an endoscope system, comprising: a light source that emits illumination light; and an imaging sensor that images an observation target with return light of the illumination light.
A first image signal acquisition step in which an image signal acquisition unit acquires a first image signal corresponding to the first illumination light of the illumination light from the imaging sensor;
A second image signal acquisition step in which the image signal acquisition unit acquires, from the imaging sensor, a second image signal corresponding to a second illumination light having a wavelength band different from the first illumination light among the illumination light; ,
An image signal storage step in which a storage unit stores the first image signal and the second image signal acquired by the image signal acquisition unit;
The unsharpness calculation unit calculates the first unsharpness that is the unsharpness of the observation target represented by the first image signal, and is the unsharpness of the observation target represented by the second image signal. An unsharpness calculating step for calculating a second unsharpness;
The movement amount calculation unit uses the plurality of first image signals, the plurality of second image signals, or the first image signal and the second image signal acquired at different timings to generate the first image signal. A movement amount calculating step of calculating a first movement amount representing the amount of movement of the observation object represented by the second movement amount, and calculating a second movement amount representing the amount of movement of the observation object represented by the second image signal; ,
The image signal selection unit uses the first unsharpness, the second unsharpness, the first movement amount, and the second movement amount to store a plurality of the first image signals stored in the storage unit. And an image signal selection step of selecting the specific first image signal and the specific second image signal from the plurality of the second image signals;
A signal processing step in which a signal processing unit performs signal processing using the specific first image signal and the specific second image signal selected by the image signal selection unit;
Equipped with a,
The image signal selection unit calculates a first reference value that is a reference for selecting the specific first image signal using the first unsharpness and the first movement amount, and the second A reference value calculation unit that calculates a second reference value that is a reference for selecting the specific second image signal using the unsharpness and the second movement amount is provided, and is specified using the first reference value. Selecting the first image signal and selecting the specific second image signal using the second reference value.
A method of operating a processor unit.

Images