CN110381807A - The working method of endoscopic system, processor device and endoscopic system - Google Patents

Abstract

The present invention provides the working method of a kind of endoscopic system, processor device and endoscopic system, the endoscopic system inhibits processing load the undesirable moment in users such as doctors, and the processing such as differentiation is executed at the time of appropriate desired by the users such as doctor automatically, so as to obtain the information for facilitating diagnosis.Endoscopic system (10) has image acquiring unit (54), differentiation processing unit (82) and differentiates processing control unit (83).Image acquiring unit (54), which obtains, utilizes endoscopic images obtained by endoscope (12) shooting observation object.Differentiation processing unit (82) differentiates the part with special characteristic in observation object by carrying out differentiation processing using endoscopic images.Differentiate that processing control unit (83) differentiates the beginning or end of processing according to the variation of the movement of endoscope (12) or endoscopic images control.

Description

Endoscope system, processor device and working method of endoscope system

technical field

The present invention relates to an endoscope system, a processor device, and an operation method of the endoscope system for discriminating a portion having a specific characteristic using an endoscope image obtained by photographing an observation object using an endoscope image.

Background technique

In the medical field, diagnosis is generally performed using an endoscope system including a light source device, an endoscope, and a processor device. In recent years, an endoscope system has been known which not only displays an endoscopic image on a display unit such as a monitor, but also uses the endoscopic image to discriminate a portion having a specific characteristic such as a lesion. For example, Patent Document 1 describes an endoscope system that calculates a feature quantity using an endoscopic image, and then discriminates a lesion using the calculated feature quantity.

In addition, there are known endoscopic systems for extracting blood vessels and the like important for diagnosis using endoscopic images. For example, Patent Document 2 describes an endoscope system that changes the execution frequency according to the amount of movement and performs blood vessel extraction processing.

Previous technical literature

Patent Literature

Patent Document 1: Japanese Patent Laid-Open No. 2012-040075

Patent Document 2: Japanese Patent Laid-Open No. 2016-144507

SUMMARY OF THE INVENTION

The technical problem to be solved by the invention

As described above, there have been known endoscopes that provide information useful for diagnosis in addition to the endoscopic image by using the endoscopic image to discriminate or identify a portion having a specific feature such as a lesion. system.

However, the process of discriminating a portion having a specific characteristic such as a lesion is usually a heavy process requiring a very high processing capability. Therefore, it is actually difficult to continuously perform processing such as discrimination of a portion having a specific characteristic such as a lesion while continuously consuming a large amount of resources of the processor device in real time during observation.

Therefore, it is realistic to appropriately perform a process such as discriminating a portion having a specific characteristic such as a lesion, as necessary, according to a user's request based on a button operation or the like. On the other hand, since the operation of the endoscope system is inherently complicated, it is not preferable to further impose an operational load on a user such as a doctor.

Therefore, an object of the present invention is to provide an endoscope system, a processor device, and an operation method of the endoscope system that suppress the processing load when the user such as the doctor does not want it, and that the user such as the doctor can reduce the processing load when the user such as the doctor does not want it. Processing such as discrimination is automatically performed at a desired and appropriate timing, and information useful for diagnosis can be acquired.

Means for solving technical problems

The endoscope system of the present invention includes an image acquisition unit, a determination processing unit, and a determination processing control unit. The image acquisition unit acquires an endoscopic image obtained by imaging an observation object with an endoscope. The discrimination processing unit discriminates a portion of the observation object having a specific feature by performing discrimination processing using the endoscopic image. The determination processing control unit controls the determination processing to start or end based on changes in the operation of the endoscope or endoscopic images.

The discrimination processing control unit includes a feature amount calculation unit and a comparison unit. The feature amount calculation unit calculates the feature amount using the endoscopic image. The comparison unit compares the feature amount with a threshold value. In this case, it is preferable that the determination processing control unit starts or ends the determination processing based on the comparison result in the comparison unit.

Preferably, the determination processing control unit starts or ends the determination processing when the observation object is enlarged or reduced by the endoscope.

Preferably, the determination processing control unit starts the determination processing when the illumination light used when the observation object is photographed by the endoscope is switched to the specific illumination light, and the illumination light used when the observation object is photographed by the endoscope is switched to illumination other than the specific illumination light In the case of light, the determination process ends.

Preferably, the determination processing control unit starts the determination processing when the illumination light is switched to illumination light having a wavelength of 450 nm or less.

Preferably, the determination processing control unit starts or ends the determination processing when the object passes through the passage through which the endoscope is inserted.

Preferably, the determination processing control unit starts the determination processing when the water passes through the passage.

Preferably, the determination processing control unit starts the determination processing when the coloring agent passes through the passage.

Preferably, the determination processing control unit starts or ends the determination processing when the position of the observation object captured by the endoscope changes or when the position of the observation object captured by the endoscope does not change.

Preferably, the determination processing control unit detects the position at which the observation object is captured by the endoscope based on a change in the observation object captured in the endoscopic image.

Preferably, the determination processing control unit uses a position sensor to detect the position of the observation object captured by the endoscope.

Preferably, the determination processing control unit ends the determination processing when the position of the observation object captured by the endoscope has changed.

Preferably, the determination processing control unit starts the determination processing based on a change in the operation of the endoscope or an endoscope image, and ends the determination processing after a predetermined time has elapsed after the determination processing is started.

Preferably, the determination processing control unit starts or ends the determination processing when a still image is captured by the endoscope.

It is preferable to include a notification control unit that controls the start or end of notification of the determination result, which is a result of the determination process, based on a change in the operation of the endoscope or an endoscope image.

Preferably, the notification control unit includes a feature amount calculation unit that calculates the feature amount using the endoscopic image, and a comparison unit that compares the feature amount with a threshold, and the notification control unit starts or ends notification of the determination result based on the comparison result in the comparison unit.

Preferably, the notification control unit starts or ends notification of the determination result when the observation object is enlarged or reduced by the endoscope.

Preferably, the notification control unit starts the notification of the determination result when the illumination light used when the observation object is photographed by the endoscope is switched to the specific illumination light, and the illumination light used when the observation object is photographed by the endoscope is switched to other than the specific illumination light. In the case of illumination light, the notification of the determination result ends.

Preferably, the notification control unit starts notification of the determination result when the illumination light is switched to illumination light having a wavelength of 450 nm or less.

Preferably, the notification control unit starts or ends notification of the determination result when the object passes through the channel inserted into the endoscope.

Preferably, the notification control unit starts notification of the determination result when the water passes through the channel.

Preferably, the notification control unit starts notification of the determination result when the dye passes through the passage.

Preferably, the notification control unit starts or ends the notification of the determination result when the position of the observation object captured by the endoscope changes or when the position of the observation object captured by the endoscope does not change.

Preferably, the notification control unit detects the position at which the observation object is captured by the endoscope based on a change in the observation object captured in the endoscopic image.

Preferably, the notification control unit uses the position sensor to detect the position of the observation object captured by the endoscope.

Preferably, the notification control unit ends the notification of the determination result when the position of the observation object captured by the endoscope has changed.

Preferably, the notification control unit starts the notification of the determination result based on a change in the operation of the endoscope or the endoscopic image, and ends the notification of the determination result after a predetermined period of time elapses after the notification of the determination result is started.

Preferably, the notification control unit starts or ends notification of the determination result when the endoscope captures a still image.

Preferably, the notification control unit notifies the determination result using any one of voice, image, or message, or a combination of these.

The processor device of the present invention includes an image acquisition unit, a determination processing unit, and a determination processing control unit. The image acquisition unit acquires an endoscopic image obtained by imaging an observation object with an endoscope. The discrimination processing unit discriminates a portion of the observation object having a specific feature by performing discrimination processing using the endoscopic image. The determination processing control unit controls the determination processing to start or end based on changes in the operation of the endoscope or endoscopic images.

The operating method of the endoscope system of the present invention includes: an image acquisition unit acquiring an endoscopic image obtained by photographing an observation object with an endoscope; and a discrimination processing unit discriminating the observation object by performing discrimination processing using the endoscopic image Steps in the section with specific characteristics. The operating method of the endoscope system of the present invention includes a step of determining the start or end of the process based on the change in the operation of the endoscope or the endoscope image control by the process control unit.

Invention effect

According to the endoscope system, the processor device, and the operation method of the endoscope system of the present invention, the processing load is suppressed when the user, such as a doctor, does not want it, and processing such as determination is automatically performed at the appropriate time when the user, such as a doctor, desires. Information to aid in diagnosis is available.

Description of drawings

FIG. 1 is an external view of an endoscope system.

2 is a block diagram of an endoscope system.

FIG. 3 is a block diagram of an image processing unit.

FIG. 4 is a flowchart showing the flow of the start and end of the discrimination process.

FIG. 5 is a display example when the discrimination process is performed.

FIG. 6 is a display example when the discrimination process is not performed.

7 is a block diagram of an image processing unit in the second embodiment.

8 is a flowchart showing the flow of the start and end of the determination process in the second embodiment.

9 is a block diagram of an image processing unit in the third embodiment.

10 is a block diagram of an image processing unit in the third embodiment.

FIG. 11 is a block diagram of an image processing unit and the like in the fourth embodiment.

FIG. 12 is a flowchart showing the flow of the start and end of the determination process in the fourth embodiment.

FIG. 13 is an explanatory diagram of a modification example.

FIG. 14 is a block diagram of an image processing unit in the fifth embodiment.

FIG. 15 is a flowchart showing the flow of the start and end of the determination process in the fifth embodiment.

FIG. 16 is an explanatory diagram of a modification.

FIG. 17 is a block diagram of an image processing unit in the sixth embodiment.

18 is a flowchart showing the flow of the start and end of the determination process in the sixth embodiment.

FIG. 19 is a block diagram of an image processing unit in the seventh embodiment.

FIG. 20 is a display example of the display in the seventh embodiment.

Fig. 21 is a schematic diagram of a capsule endoscope.

Detailed ways

[First Embodiment]

As shown in FIG. 1 , the endoscope system 10 includes an endoscope 12 , a light source device 14 , a processor device 16 , a display 18 , and a console 19 . The endoscope 12 images the observation object. The light source device 14 generates illumination light. The processor device 16 performs system control, image processing, and the like of the endoscope system 10 . The display 18 is a display unit that displays a display image (hereinafter, referred to as an endoscopic image. For example, refer to an endoscopic image 101 of FIG. 5 or an endoscopic image 102 of FIG. 6 ) generated by the processor device 16 . The console 19 is an input device that performs setting input and the like to the processor device 16 and the like.

The endoscope 12 includes an insertion portion 12a inserted into the subject, an operation portion 12b provided at a proximal end portion of the insertion portion 12a, a bending portion 12c and a distal end portion 12d provided at the distal end side of the insertion portion 12a. By operating the angle button 12e of the operation part 12b, the bending part 12c is bent. The front end portion 12d is directed in a desired direction by the bending of the curved portion 12c. Further, the distal end portion 12d is provided with a forceps port (not shown) for protruding a treatment tool such as forceps toward the observation target, or a jet port (not shown) for spraying air, water, or the like toward the observation target.

Moreover, in addition to the angle button 12e, the operation part 12b is provided with the zoom operation part 13a, the mode switching switch 13b, and the like. By operating the zoom operation unit 13a, it is possible to zoom in or zoom out the observation object to perform imaging. The observation mode can be switched by the operation of the mode switching switch 13b. The endoscope system 10 has a plurality of observation modes, such as a normal observation mode and a special observation mode. In the normal observation mode, white light is used as illumination light to observe the observation object in a natural color. , emphasizing blood vessels located in the mucosal surface, etc. These plurality of observation modes can be appropriately changed at any timing by operating the mode changeover switch 13b.

As shown in FIG. 2 , the light source device 14 includes a light source unit 20 that emits illumination light, and a light source control unit 22 that controls driving of the light source unit 20 .

The light source unit 20 includes, for example, a plurality of LEDs (Light Emitting Diodes) that emit light having different center wavelengths or wavelength ranges (hereinafter, simply referred to as "different wavelengths") as light sources, and the light emission or lighting of the respective LEDs and the amount of light are determined. Adjustment, etc., can emit a variety of illumination lights of different wavelengths. For example, the light source unit 20 can respectively emit violet light, blue light, green light, and red light having a relatively wide wavelength range and a wide frequency band as illumination light. Furthermore, the light source unit 20 can emit violet light, blue light, green light, and red light in a narrow frequency band (meaning a wavelength range of about 10 nm to 20 nm) in addition to violet light, blue light, green light, and red light in a wide band. Light as illumination light.

In addition, as the light source unit 20, a combination of an LD (Laser Diode), a phosphor, and a band-limiting filter, a combination of a lamp such as a xenon lamp, and a band-limiting filter can be used instead of the LED. Of course, when the light source unit 20 is constituted by LEDs, it can also be used in combination with a phosphor or a band-limiting filter.

The light source control unit 22 independently controls the timing of turning on or turning off each of the light sources constituting the light source unit 20 , the amount of light emitted at the time of turning on, and the like. As a result, the light source unit 20 can emit various types of illumination light of different wavelengths. Then, the light source control unit 22 controls the light source unit 20 according to the imaging timing (so-called frame) of the image sensor 48 .

The illumination light emitted by the light source unit 20 is incident on the light guide 41 . The light guide 41 is built in the endoscope 12 and the universal cord, and transmits the illumination light to the distal end portion 12 d of the endoscope 12 . The universal cord is a cord that connects the endoscope 12 with the light source device 14 and the processor device 16 . In addition, as the light guide 41, a multimode optical fiber can be used. As an example, a thin optical fiber cable having a diameter of 0.3 to 0.5 mm in diameter including a core diameter of 105 μm, a coating diameter of 125 μm, and a protective layer serving as a sheath can be used.

The distal end portion 12d of the endoscope 12 is provided with an illumination optical system 30a and an imaging optical system 30b. The illumination optical system 30a includes an illumination lens 45 and emits illumination light toward the observation object via the illumination lens 45 . The imaging optical system 30 b includes an objective lens 46 , a zoom lens 47 , and an image sensor 48 . The image sensor 48 utilizes, through the objective lens 46 and the zoom lens 47 , reflected light of the illumination light returned from the observation object, and the like (in addition to the reflected light, flash light, fluorescence emitted by the observation object, and light caused by drug administration to the observation object are also included. Fluorescence caused by drugs, etc.), and photograph the observation object. The zoom lens 47 is moved by the operation of the zoom operation unit 13 a to enlarge or reduce the observation object photographed by the image sensor 48 .

The image sensor 48 is, for example, a color sensor having a primary color filter, and includes a B pixel (blue pixel) having a blue filter, a G pixel (green pixel) having a green filter, and a red filter. The R pixel (red pixel) of these 3 kinds of pixels. The blue color filter mainly transmits violet to blue light. The green color filter mainly transmits green light. The red color filter mainly transmits red light. When the object to be observed is photographed by the image sensor 48 of the primary color system as described above, the B image (blue image) obtained from the B pixel, the G image (green image) obtained from the G pixel, and the R pixel can be obtained at the same time. The obtained R image (red image) of these 3 kinds of images.

In addition, as the image sensor 48, a CCD (Charge Coupled Device) sensor or a CMOS (Complementary Metal Oxide Semiconductor) sensor can be used. In addition, the image sensor 48 of the present embodiment is a color sensor of a primary color system, but a color sensor of a complementary color system can also be used. The color sensor of the complementary color system has, for example, a cyan pixel provided with a cyan color filter, a magenta pixel provided with a magenta color filter, a yellow pixel provided with a yellow color filter, and a green (green) color filter provided. Green pixels. Complementary-primary color conversion is performed on an image obtained from the pixels of each of the above-mentioned colors when a complementary color system color sensor is used, so that it can be converted into a B image, a G image, and an R image. Also, a monochrome sensor without a color filter can be used as the image sensor 48 instead of the color sensor. In this case, images of the above-mentioned colors can be obtained by sequentially photographing the observation object with illumination light of each color such as BGR.

The processor device 16 includes a control unit 52 , an image acquisition unit 54 , an image processing unit 61 , and a display control unit 66 .

The control unit 52 performs overall control of the endoscope system 10 , such as synchronization control of the irradiation timing of the illumination light and the imaging timing. For example, the control unit 52 performs synchronization control of the irradiation timing of the illumination light and the imaging timing, and the like. In addition, when the zoom operation unit 13a is operated, the control unit 52 receives an operation signal from the zoom operation unit 13a, and may send a control signal indicating the imaging size of the observation object (that is, the enlargement or reduction of the observation object and the degree thereof) (hereinafter , referred to as the imaging size signal) is input to the image processing unit 61 . The image processing unit 61 may or may not perform predetermined processing, adjust the intensity of predetermined processing, or the like according to the imaging size of the observation object based on the imaging size signal.

Then, when the mode changeover switch 13b is operated, the control unit 52 receives an operation signal from the mode changeover switch 13b, and inputs a control signal for designating the observation mode (hereinafter, referred to as an observation mode designation signal) to the light source control unit 22 and the image processing unit 22 Section 61. The observation mode designation signal input from the control unit 52 to the light source control unit 22 and the image processing unit 61 designates the type of illumination light to be used, the light emission order, and the like. The light source control unit 22 emits illumination light according to the observation mode designation signal by the light source unit 20 according to the observation mode designation signal. The image processing unit 61 may or may not perform predetermined processing or adjust the intensity of predetermined processing according to the observation mode according to the observation mode designation signal.

The image acquisition unit 54 acquires an image of the observation object from the image sensor 48 . The image of the observation object acquired by the image acquisition unit 54 from the image sensor 48 is an endoscopic image. In addition, the image generated for display using the image of the observation object acquired from the image sensor 48 is also an endoscopic image. Hereinafter, when it is necessary to distinguish, the image of the observation object acquired from the image sensor 48 is referred to as a "captured endoscopic image" or simply an "image", and an image generated for display is referred to as a "display endoscope" image" or simply "endoscopic image". In the present embodiment, since the image sensor 48 has a color filter, the image acquisition unit 54 acquires an image (photographed endoscopic image) for each illumination light and each color filter.

The image acquisition unit 54 includes a DSP (Digital Signal Processor) 56 , a noise reduction unit 58 , and a conversion unit 59 , and by using these, various kinds of processing are performed on the acquired image as necessary.

The DSP 56 performs various processes such as defect correction processing, offset processing, gain correction processing, linear matrix processing, gamma conversion processing, demosaic processing, and YC conversion processing on the acquired image as necessary.

The defect correction process is a process of correcting the pixel value of the pixel corresponding to the defective pixel of the image sensor 48 . The offset process is a process of reducing the dark current component from the image to which the defect correction process has been performed and setting an accurate zero level. The gain correction process is a process of adjusting the signal level of each image by multiplying the offset-processed image by a gain. The linear matrix processing is processing for improving the color reproducibility of the image subjected to the offset processing, and the gamma conversion processing is processing for adjusting the lightness and saturation of the image after the linear matrix processing. Demosaic processing (also referred to as isotropic processing or synchronization processing) is processing for interpolating pixel values of missing pixels, and is performed on an image after gamma conversion processing. A missing pixel refers to a pixel that has no pixel value due to the arrangement of the color filters (due to the arrangement of pixels of other colors in the image sensor 48 ). For example, since the B image is an image obtained by photographing the observation object at the B pixel, there is no pixel value in the pixels at the positions corresponding to the G pixel and the R pixel. The demosaic process interpolates the B image to generate pixel values of pixels located at the positions of the G pixel and the R pixel of the image sensor 48 . The YC conversion process is a process of converting the image after the demosaic process into a luminance channel Y, a color-difference channel Cb, and a color-difference channel Cr.

The noise reduction unit 58 performs noise reduction processing on the luminance channel Y, the color difference channel Cb, and the color difference channel Cr, for example, by a moving average method, a median filter method, or the like. The conversion unit 59 re-converts the luminance channel Y, the color difference channel Cb, and the color difference channel Cr after the noise reduction processing into the image of each color of BGR.

The image processing unit 61 uses the image acquired by the image acquisition unit 54 to generate an endoscopic image to be displayed on the display 18 . Then, the image processing unit 61 executes a determination process of determining a portion having a specific characteristic of the observation object using the endoscopic image, as necessary. For this purpose, as shown in FIG. 3 , the image processing unit 61 includes an image generation unit 81 , a determination processing unit 82 , and a determination processing control unit 83 .

The image generation unit 81 generates an endoscopic image according to the observation mode using the image acquired by the image acquisition unit 54 . When the observation mode is the normal observation mode, the image generation unit 81 generates an endoscopic image in which the observation object can be observed in natural colors. In addition, when the observation mode is the special observation mode, use a combination of images different from those of the normal observation mode, or assign the used images to color channels in a method different from the normal observation mode, etc., to generate the purpose of the special observation mode. of endoscopic images. For example, an endoscopic image in which blood vessels and the like located on the mucosal surface are emphasized is generated.

In addition, when generating an endoscopic image according to the observation mode, the image processing unit 61 performs color conversion processing, color enhancement processing, and structure enhancement processing on the image acquired by the image acquisition unit 54 as necessary. The color conversion processing performs 3×3 matrix processing, gradation conversion processing, three-dimensional LUT (look-up table) processing, and the like on the image of each color of the BGR. The color enhancement processing is processing for enhancing the color of an image, and the structure enhancement processing is processing for enhancing, for example, a tissue or structure of an observation target such as a blood vessel or a pit pattern.

The discrimination processing unit 82 performs discrimination processing using the endoscopic image generated by the image generating unit 81 to discriminate a portion having a specific characteristic (hereinafter, referred to as a specific portion) in the observation object. The "specific part" can be determined according to the presence or absence of tissue or structure, the shape (thickness, length, the state of the boundary with other tissues, that is, the state of the edge, the disorder of the tissue, etc., the meandering degree of the linear tissue, etc.), the amount ( area or density, etc.), distribution, or color, or state of tissue or structure (e.g., the depth of the submucosal or internal parameters such as the oxygen saturation of hemoglobin contained in blood vessels), in contrast to other parts (e.g., normal mucosa ) to be differentiated. As described above, the "specific feature" that specifies a specific portion refers to a feature that is known from observation of an endoscopic image or information obtained by operation using an endoscopic image or an image used to generate an endoscopic image. Hereinafter, information such as a quantity related to a specific feature specifying a specific part is referred to as "feature amount".

"Discrimination" means further obtaining information useful for diagnosis based on the characteristics of a specific part or the like, and "discrimination processing" means processing performed by the discrimination processing unit 82 for discrimination. For example, the identification process of identifying whether or not a specific portion is a lesion based on a feature amount representing a feature of the specific portion is a discrimination process. In addition, processing of calculating information indicating the possibility of a lesion, such as an index indicating the possibility that the specific part is a lesion, is also included in the determination process, based on the feature quantity representing the feature of the specific part. In addition, when the specific portion is a lesion, processing such as the classification or classification of the lesion, the identification of benign or malignant, or the identification of the degree of progression or the degree of invasion of the lesion is also a determination processing.

The discrimination processing unit 82 uses, for example, a CNN (Convolution Neural Network) 71 (refer to FIG. 2 ), a neural network (Neural Network), SVM (Support Vector Machine, Support Vector Macine) or a classifier (Adaboost (Adaptive Boosting)) or the like performs the above discrimination processing. The result of the determination process (hereinafter, referred to as determination result) is, for example, "whether or not a specific part is a lesion" (including information that can determine whether or not a specific part is a lesion). Therefore, when the specific part is a lesion, the type or classification of the lesion, the identification of benign or malignant, the degree of progression or the degree of invasion of the lesion, and the like are the results of discrimination. Then, a score indicating the accuracy of the determination of whether or not the specific portion is a lesion or the certainty of the classification of the lesion can be used as the determination result. In the present embodiment, the determination processing unit 82 determines whether or not a specific portion is a lesion, and uses a score indicating the certainty of the determination as a determination result. The fraction is, for example, a numerical value of 0.0 to 1.0. Further, when the determination processing unit 82 performs the determination processing, the information of the endoscopic image used in the determination processing is stored in the storage unit 72 (refer to FIG. 2 ). The information of the endoscopic image used in the discrimination processing refers to, in addition to the endoscopic image itself, a frame number, a score as a discrimination result, and the like. The determination processing unit 82 associates the endoscopic image, the frame number, the score as the determination result, and the like with each other, and stores it in the storage unit 72 . The storage unit 72 is, for example, a memory or a memory such as a hard disk.

In addition, the determination processing unit 82 uses the endoscopic image (display endoscopic image) generated by the image generating unit 81 in the determination processing, but the display endoscopic image is generated using the captured endoscopic image. Therefore, as a result, the determination processing unit 82 performs determination processing using the endoscopic image obtained by imaging the observation object with the endoscope. In addition, the determination processing unit 82 can directly perform determination processing using the captured endoscopic image instead of the display endoscopic image.

The determination processing control unit 83 controls the start or end of the determination processing in the determination processing unit 82 according to changes in the operation of the endoscope 12 or endoscopic images. The “change in the operation of the endoscope 12 ” refers to the transition or switching of the operation mode of the endoscope 12 , and generally, the determination process refers to the transition or switching of the operation mode that is not directly related. For example, enlargement or reduction of the observation object, switching of illumination light due to switching of the observation mode, and the like are changes in the operation of the endoscope 12 . "According to the endoscopic image" refers to a change in accordance with a feature of an observation object captured in an endoscopic image, for example, a change in accordance with a feature quantity.

In addition, the "start of the determination process" means that, when the determination process is not performed on the endoscopic image of a certain arbitrary frame, the determination process is performed on the endoscopic image of at least one or more consecutive frames. That is, the "start of the determination process" also includes the case where the determination process is performed only on an endoscopic image for one frame when the frames for which the determination process is not performed are consecutive. In addition, "the end of the determination process" means that when the determination process is performed on the endoscopic image of a certain arbitrary frame, the determination process is not performed on the endoscopic image of one or more consecutive frames. That is, the "end of the determination process" includes a case where the determination process is not performed for only one frame of an endoscopic image when the frames for which the determination process is performed are consecutive.

In the present embodiment, the determination processing control unit 83 controls the start and end of the determination processing in the determination processing unit 82 based on the endoscopic image. For this purpose, the determination processing control unit 83 includes a feature amount calculation unit 86 and a comparison unit 87 .

The feature amount calculation unit 86 acquires the endoscopic image from the image generation unit 81 . Then, the feature amount calculation unit 86 calculates the feature amount of the observation object captured in the endoscopic image using the acquired endoscopic image. The calculation of the feature amount is performed for a part (for example, every few frames, etc.) or all of the endoscopic images generated by the image generation unit 81 . Then, the feature amount calculation unit 86 calculates the feature amount for all or a part of the endoscopic image.

The comparison unit 87 compares the feature value calculated by the feature value calculation unit 86 with a preset feature value of an endoscopic image of a normal observation object. The result of the comparison performed by the comparison unit 87 (hereinafter, referred to as the comparison result) indicates whether the value of the feature amount or the distribution of the values calculated by the feature amount calculation unit 86 is within a normal range, and indicates that the image was captured in the sequentially acquired range. Changes in the characteristics of the observation object of the speculum image. In the present embodiment, the comparison unit 87 compares the feature value calculated by the feature value calculation unit 86 with the threshold value using a preset feature value of an endoscopic image of a normal observation object as a threshold value. The feature amount calculation unit 86 sequentially compares the feature amount with the threshold value every time the feature amount is calculated. The case where the feature amount is equal to or greater than the threshold value is, for example, a case where redness or the like is captured in an endoscopic image, and basically a case where determination processing is desired. Conversely, when the feature amount becomes smaller than the threshold value, for example, there is no redness or the like, the observation object is normal, and the determination process is basically unnecessary.

The determination processing control unit 83 starts or ends the determination processing based on the comparison result of the comparison unit 87 . As a result of this, the determination processing control unit 83 controls the start or end of the determination processing in the determination processing unit 82 based on the endoscopic image. More specifically, in the present embodiment, the determination process is started when the feature amount becomes equal to or greater than the threshold value, and the determination process is terminated when the feature amount becomes smaller than the threshold value. As a result, the determination processing unit 82 automatically executes the determination process when the feature amount is equal to or greater than the threshold value, and automatically ends the determination process when the feature amount becomes smaller than the threshold value.

The display control unit 66 acquires the endoscopic image from the image processing unit 61 , converts the acquired endoscopic image into a format suitable for display, and sequentially outputs and displays it on the display 18 . Thereby, a doctor or the like can observe the observation object using the endoscopic image. Then, when the determination processing unit 82 performs the determination processing, the display control unit 66 acquires the score as the determination result from the determination processing unit 82 in addition to acquiring the endoscopic image. Then, the score as the determination result is displayed on the display 18 together with the endoscopic image.

Next, the start and end of the determination process in the endoscope system 10 will be described along the flowchart shown in FIG. 4 . First, an observation mode is selected and an observation object is photographed (S11). Thereby, the image acquisition unit acquires the captured endoscopic image from the image sensor 48, and the image generation unit 81 generates an endoscopic image for display using the captured endoscopic image.

When the endoscopic image is acquired, the determination processing control unit 83 calculates the feature amount in the feature amount calculation unit 86 ( S12 ), and then compares the feature amount with a threshold in the comparison unit 87 ( S13 ). As a result of the comparison in the comparison unit 87, when the feature amount is equal to or greater than the threshold value (S13: YES), the determination processing control unit 83 executes the determination processing in the determination processing unit 82 (S14). Usually, since the imaging rate is sufficiently fast compared with the change of the endoscopic image, once the feature amount becomes equal to or greater than the threshold value, the feature amount becomes equal to or greater than the threshold value for a certain period of time thereafter. Therefore, in practice, when the feature amount becomes equal to or larger than the threshold value, the determination process starts. When the determination process is started, as shown in FIG. 5 , the display control unit 66 sequentially displays the endoscopic image 101 used in the determination process and the score 99 as the determination result on the display 18 .

On the other hand, when the feature amount is smaller than the threshold value as a result of the comparison by the comparison unit 87 ( S13 : NO), the determination processing control unit 83 skips the determination processing by the determination processing unit 82 . Usually, since the imaging rate is sufficiently fast compared with the change of the endoscopic image, once the feature amount becomes smaller than the threshold value, the feature amount becomes smaller than the threshold value for a certain period of time thereafter. Therefore, actually, when the feature amount becomes smaller than the threshold value, the determination process ends. When the determination process ends, as shown in FIG. 6 , the display control unit 66 sequentially displays the endoscopic images 102 for which the determination process has not been performed on the display 18 .

Then, the above-described start and end of the determination process are repeated until the observation is completed ( S15 ). Therefore, when the feature amounts of the sequentially generated endoscopic images are sequentially calculated and the calculated feature amounts are sequentially compared with the threshold value, if the feature amount is smaller than the threshold value and the feature amount is greater than or equal to the threshold value, even if the button is not pressed It is also possible to automatically start the determination process by operating or the like, and to provide a score as a determination result to a doctor or the like to support diagnosis. In addition, if the need for the determination process becomes smaller when the feature amount becomes smaller than the threshold value from the state in which the determination process has been started, the determination process can be automatically terminated without performing a button operation or the like, thereby reducing the processing load of the processor device 16 ( consumption of resources such as memory).

[Second Embodiment]

In the above-described first embodiment, the determination processing control unit 83 starts and ends the determination processing based on the change in the feature quantity (the change in the endoscopic image), but instead of this, the determination processing can be started or ended when the endoscope 12 enlarges or reduces the observation object. Discriminate processing.

In this case, as shown in FIG. 7 , the image processing unit 61 is provided with a determination processing control unit 283 instead of the determination processing control unit 83 of the first embodiment. The determination processing control unit 283 acquires, via the control unit 52, information related to the operation status of the zoom operation unit 13a. Specifically, the determination processing control unit 283 acquires an imaging size signal indicating the imaging size of the observation object. Then, the determination process is started or ended according to the presence or absence or degree of enlargement or reduction of the observation object known from the imaging size signal.

For example, as shown in FIG. 8 , when the observation object is photographed ( S211 ) and an endoscopic image is acquired, the determination processing control unit 283 detects whether the observation object is being magnified and observed based on the imaging size signal ( S212 ). Then, when the magnification observation is performed ( S212 : YES), the determination processing control unit 83 executes the determination processing in the determination processing unit 82 ( S213 ). This actually becomes the start of the discrimination process when the discrimination process has not been performed in the previous frame. On the other hand, when magnification observation is not performed ( S212 : NO), the determination processing control unit 283 skips the determination processing in the determination processing unit 82 . When the discrimination process was performed in the previous frame, this actually becomes the end of the discrimination process.

By repeating the above operation until the observation is completed ( S214 ), the determination process is automatically performed while the magnification observation is being performed, and when the magnification observation ends, the linked determination process also automatically ends. This is because when a doctor or the like magnifies the observation object, the enlarged portion is a lesion or there is a possibility of a lesion. Therefore, by starting and ending the determination process based on the presence or absence of enlargement as described above, the determination process can be performed at an appropriate timing when the doctor or the like desires the determination process, and the score 99 as the determination result can be provided. In addition, unnecessary determination processing is not performed at unnecessary timing, whereby the processing load of the processor device 16 can be reduced.

In the above-described second embodiment, the determination processing control unit 283 starts the determination processing when the observation object is enlarged, but may start the determination process when the observation object is reduced in size. The case of narrowing down the observation object and photographing the observation object with a so-called "pull" is, for example, the case of confirming the presence or absence of a lesion or a portion likely to have a lesion. In this case, if the discrimination processing is performed to display the score as the discrimination result, it is possible to easily confirm whether or not there is a lesion within the observation range, or a portion where there is a possibility of a lesion, or the like.

In the above-mentioned second embodiment and the above-mentioned modification example, the determination process is started when the observation object is enlarged or reduced, but it can be determined according to whether the enlargement is larger than a specific enlargement ratio, or when it is reduced to a certain reduction ratio or less, or the enlargement ratio (reduction ratio) is between The determination process starts or ends when the degree of enlargement or reduction of the observation object is within a specific range.

In the above-described second embodiment and the above-described modification, the determination processing control unit 283 starts or ends the determination processing based on the imaging size signal indicating the operation of the zoom operation unit 13a, but the determination processing control unit 283 can use the endoscope instead of the imaging size signal. The mirror image detects the enlargement or reduction of the observation object, or the degree thereof, and starts or ends the discrimination process according to the detection result. At this time, the determination processing control unit 283 acquires the endoscopic image from the image acquisition unit 54 or the image generation unit 81 , and holds at least the previous endoscopic image for one frame or more. Then, when an endoscopic image is newly acquired, the newly acquired endoscopic image is compared with the held conventional endoscopic image, and the enlargement or reduction of the observation object, or the degree thereof is detected. At this time, the determination processing control unit 283 also functions as an enlargement/reduction detection unit that detects the enlargement or reduction of the observation object, or the degree thereof.

[Third Embodiment]

The endoscope system 10 can start or end the determination process based on the "change in the operation of the endoscope 12 or the endoscopic image", which is different from the above-described first and second embodiments. For example, the determination process can be started or ended when the observation mode is switched, that is, when the illumination light used when the endoscope 12 images the observation object is switched.

In this case, as shown in FIG. 9 , the image processing unit 61 is provided with a determination processing control unit 383 instead of the determination processing control unit 83 of the first embodiment or the determination processing control unit 283 of the second embodiment. The determination processing control unit 383 acquires, via the control unit 52 , information related to the operation status of the mode changeover switch 13 b. Specifically, the determination processing control unit 383 acquires an observation mode designation signal that designates an observation mode. Then, switching of the illumination light is detected from the type of the observation mode designated by the observation mode designation signal, and the determination process is started or ended.

For example, as shown in FIG. 10, when an observation object is photographed (S311) and an endoscopic image is acquired, the determination processing control unit 383 determines the type of observation mode specified by the observation mode specifying signal based on the presence or absence of input of the observation mode specifying signal. It is detected whether or not specific illumination light is being used as illumination light (S312). The specific illumination light is the illumination light of the observation mode in which a doctor or the like often wishes to perform the discrimination process. When the illumination light is switched to the specific illumination light ( S312 : YES), the determination processing control unit 383 executes the determination processing in the determination processing unit 82 ( S313 ). This actually becomes the start of the discrimination process when the discrimination process has not been performed in the previous frame. On the other hand, when the illumination light is switched to illumination light other than the specific illumination light ( S312 : NO), the determination processing control unit 383 skips the determination in the determination processing unit 82 . When the discrimination process was performed in the previous frame, this actually becomes the end of the discrimination process.

By repeating the above operation until the observation is completed (S314), the determination process is automatically performed while the observation mode is the observation mode using the specific illumination light, and when the observation mode is changed to another observation mode that does not use the specific illumination light, it is linked to the observation mode. The discrimination process also ends automatically. In this way, if the determination process is started when switching to the “observation mode using specific illumination light” in which there are many cases in which doctors and the like wish to use the determination process at the same time, and the determination process is terminated when the determination process is switched to the observation mode using other illumination light, it is possible to The determination process is performed at an appropriate timing when the doctor or the like desires the determination process, and a score 99 is provided as the determination result. In addition, unnecessary determination processing is not performed at unnecessary timing, whereby the processing load of the processor device 16 can be reduced.

In the third embodiment described above, the determination processing control unit 383 starts or ends the determination processing based on the observation mode designation signal, but the determination processing control unit 383 can detect switching of the illumination light and the observation mode using the endoscopic image instead of the observation mode designation signal. At this time, the determination processing control unit 383 acquires the endoscopic image from the image acquisition unit 54 or the image generation unit 81 . Then, switching of the illumination light and the observation mode is detected based on the characteristics (color matching, etc.) of the acquired endoscopic image.

In addition, the specific illumination light is, for example, illumination light having a wavelength of 450 nm or less. That is, it is preferable to start the determination process when the illumination light is switched to illumination light having a wavelength of 450 nm or less. Using illuminating light with a wavelength of 450 nm or less makes it easier to observe fine structures such as blood vessels and dimple patterns in the mucosal surface layer. Therefore, when using illuminating light with a wavelength of 450 nm or less, doctors and the like want detailed observation and determination results for diagnosis, etc. more cases. "Using illuminating light with a wavelength of 450 nm or less" means actually using the illuminating light with a wavelength of 450 nm or less alone and photographing the observation object only by reflected light or the like of the illuminating light with a wavelength of 450 nm or less.

[4th Embodiment]

The endoscope system 10 can start or end the determination process based on "change in operation of the endoscope 12 or endoscopic image", which is different from the above-described first, second, and third embodiments. For example, the determination process can be started when the object passes through the channel inserted into the endoscope 12 . The channels inserted into the endoscope 12 are, for example, a forceps channel 401 for inserting treatment instruments such as forceps, and an air and water supply channel 402, which protrude from the front end portion 12d toward the observation object, and the air supply and water supply channel 402. For inserting water, air, or the like, water, air, or the like is ejected from the front end portion 12d toward the observation object (see FIG. 11 ).

When the determination process is started or terminated when the object passes through the channel inserted into the endoscope 12 , as shown in FIG. 11 , for example, in the middle of the forceps channel 401 (for example, near the distal end 12 d ), a device for detecting a treatment tool such as forceps is installed. Passed through sensor 403 . In addition, the image processing unit 61 is provided with a determination processing control unit 483 in place of the determination processing control unit 83 and the like of the first embodiment. The determination processing control unit 483 acquires the detection signal of the treatment tool from the passage sensor 403 via the control unit 52 . Then, when it is detected that the distal end of the treatment tool or the like has passed through the forceps channel 401, the determination process is started or ended.

Specifically, as shown in FIG. 12 , when an observation object is photographed ( S411 ) and an endoscopic image is acquired, the determination processing control unit 483 acquires a detection signal from the passing sensor 403 , and detects whether or not the surgical instrument has passed the forceps based on the acquired detection signal. channel 401 (S412). In addition, when the treatment tool has not passed through the forceps channel 401 (S412: NO), the determination process is skipped. Then, from the state where the treatment tool is not inserted into the forceps channel 401, it is detected that the treatment tool has passed through the forceps channel 401 (S412: YES), and the treatment tool has protruded from the distal end portion 12d toward the observation object (S413: YES), and the processing control is discriminated. The section 483 executes the discrimination processing in the discrimination processing section 82 (S414). This actually becomes the start of the discrimination process when the discrimination process has not been performed in the previous frame. On the other hand, when the treatment tool has passed through the forceps channel 401 ( S412 : YES), once the passage of the treatment tool is detected, the case where the treatment tool is detected again is the case where the treatment tool is retracted ( S413 : NO), so The determination processing control unit 483 skips the determination processing in the determination processing unit 82 . When the discrimination process was performed in the previous frame, this actually becomes the end of the discrimination process.

By repeating the above operation until the observation is completed (S415), the determination process is automatically performed only when the treatment tool protrudes from the distal end portion 12d, and the determination process is automatically ended when the treatment tool is retracted. In addition, when the treatment tool has not been used from the beginning, the determination process is not performed. In the case of using a treatment instrument, there are lesions or the like, and treatment is required, and therefore there are many cases where discrimination treatment is desired. Therefore, if the determination process is performed when the treatment instrument is used as described above, the determination process can be performed at an appropriate timing when the doctor or the like desires the determination process, and the score 99 as the determination result can be automatically provided. In addition, unnecessary determination processing is not performed at unnecessary timing, whereby the processing load of the processor device 16 can be reduced.

In the third embodiment described above, the passage sensor 403 is provided in the forceps passage 401 to detect passage of the treatment instrument, but instead of this, passage of water or the like in the air supply and water supply passage 402 may be detected. At this time, as shown in FIG. 13, the passage sensor 404 which detects the passage of water etc. is provided in the midway (for example, near the front-end|tip part 12d) of the air supply and water supply passage 402. As shown in FIG. Then, the determination processing control unit 483 starts the determination processing when water or the like passes through the air supply and water supply passage 402 . In the case of spouting water or the like, for example, a portion in contact with water or the like or a peripheral portion thereof is observed in more detail. Therefore, there are cases in which it is desirable to perform the determination process at the same time. Therefore, if the discharge of water or the like is detected as described above, and the judgment processing is performed when the discharge of water or the like is detected, the judgment processing can be performed at an appropriate timing when the doctor or the like desires the judgment processing, and the judgment result can be automatically provided. Score 99. In addition, unnecessary determination processing is not performed at unnecessary timing, whereby the processing load of the processor device 16 can be reduced.

In addition, the object detected by the sensor 404 in the air supply and water supply passage 402 is preferably water or a coloring agent that colors the structure of the observation object or the like. This is because the coloring agent is used for easy observation of the structure and the like of the object to be observed. Therefore, when the coloring agent is sprayed, it is also the time when a doctor or the like wishes to determine the treatment.

Further, as described above, when the determination process is started when the water, dye, etc. passes through the air supply and water supply passage 402 , it is preferable that the determination process is terminated after a predetermined period of time has elapsed after the determination process is started. This is because the time for the water or the colorant to pass through the air supply and water supply channel 402 is extremely short, and a certain period of time after the water or the colorant is ejected from the front end portion 12d is for obtaining the effect of the sprayed water, the colorant, and the like. The time at which the observation was made by the subject.

In the fourth embodiment and the modification described above, the passage sensor 403 or the passage sensor 404 is used, but the determination processing control unit 483 can detect water, dye, etc. using the endoscopic image instead of the passage sensor 403 or the passage sensor 404 . At this time, the determination processing control unit 483 acquires the endoscopic image from the image acquisition unit 54 or the image generation unit 81 . Then, ejection of water, dye, or the like is detected based on changes in the characteristics of the acquired endoscopic image (disappearance of residues and the like, change in the color of the observation object due to the dye, and the like).

[Fifth Embodiment]

The endoscope system 10 can start or end the determination based on the "change in the operation of the endoscope 12 or the endoscopic image", which is different from the first, second, third, and fourth embodiments described above. deal with. For example, the determination process can be started or ended when the position at which the endoscope 12 captures the observation object changes or when the position at which the endoscope 12 captures the observation object does not change. The change in the imaging position referred to here includes a change in the imaging position due to a change in the direction of the distal end portion 12d relative to the observation object, in addition to the change in the imaging position caused by the insertion and removal of the insertion portion 12a.

As described above, when the position at which the endoscope 12 captures the observation object changes or when the position at which the endoscope 12 captures the observation object does not change, when the determination process starts or ends, as shown in FIG. 14 , the image processing unit 61 , In place of the determination processing control unit 83 and the like in the first embodiment, a determination processing control unit 583 is provided. The determination processing control unit 583 includes a position detection unit 586 . The position detection unit 586 detects the change or non-change of the position at which the endoscope 12 images the observation target (hereinafter, referred to as the imaging position) using the endoscopic image. For example, the position detection unit 586 acquires an endoscopic image from the image acquisition unit 54 or the image generation unit 81 , and holds at least one previous endoscopic image for one frame or more. Then, when the endoscopic image is newly acquired, the newly acquired endoscopic image is compared with the held conventional endoscopic image, and the position of the imaging position is detected based on the change of the observation object captured in the endoscopic image. Variety.

The determination processing control unit 583 uses the detection result of the position detection unit 586, and as a result, detects a change in the imaging position based on a change in the observation object captured in the endoscopic image. Then, the determination processing control unit 583 controls the start and end of the determination processing in the determination processing unit 82 based on the change or non-change of the detected imaging position. In the present embodiment, the determination processing control unit 583 starts the determination processing when the imaging position changes, and ends the determination processing when the imaging position does not change.

Specifically, as shown in FIG. 15 , when the observation object is photographed ( S511 ), and the determination processing control unit 583 and the position detection unit 586 acquire an endoscopic image, the position detection unit 586 compares the stored conventional endoscopic image with the endoscope image. The newly acquired endoscopic images are compared to detect changes in the shooting position. When the photographing position has not changed or when the photographing position has stopped changing ( S512 : NO), the discrimination processing control unit 583 executes the discrimination processing in the discrimination processing unit 82 ( S513 ). This actually becomes the start of the discrimination process when the discrimination process has not been performed in the previous frame. On the other hand, when there is a change in the imaging position ( S512 : YES), the determination processing control unit 583 skips the determination processing in the determination processing unit 82 . When the discrimination process was performed in the previous frame, this actually becomes the end of the discrimination process.

By repeating the above operation until the observation is completed ( S514 ), the determination process is automatically performed when there is no change in the imaging position, and the determination process is automatically terminated in conjunction with the change in the imaging position. The reason why the determination processing is performed when there is no change in the imaging position in this way is that when there is no change in the imaging position, it is necessary to observe in detail whether the observation object at the imaging position is a lesion or the like, and it is desirable to perform the determination processing at the same time. Therefore, if the determination process is started and terminated according to the change of the imaging position as described above, the determination process can be performed at an appropriate timing when the doctor or the like desires the determination process, and the score 99 can be given as the determination result. In addition, unnecessary determination processing is not performed at unnecessary timing, whereby the processing load of the processor device 16 can be reduced.

In the fifth embodiment described above, the determination process is performed when there is no change in the imaging position. Conversely, the determination process may be started when there is a change in the imaging position, and the determination process may be terminated when the change in the imaging position disappears. In this way, the specific content of the determination processing is mainly useful in the search for the presence or absence of lesions or the like (so-called screening), and screening can be supported by performing the determination processing at the time of screening in which the imaging position changes greatly. Of course, when performing both the discrimination processing suitable for detailed observation and the discrimination processing suitable for screening, it is possible to perform discrimination processing suitable for detailed observation when there is no change in the imaging position, and perform screening suitable for when there is a change in the imaging position discriminative processing.

In addition, "there is no change in the photographing position (no change in the photographing position)" means that the change in the photographing position is small compared to the time of screening or the like, and there is substantially no change in the photographing position. "There is a change in the photographing position" means that the photographing position has changed more than when observed in detail, and the photographing position is not substantially stationary.

In the fifth embodiment and the modification described above, the determination processing control unit 583 detects the imaging position using the endoscopic image, but as shown in FIG. 16 , instead of using the endoscopic image, the position sensor 591 (refer to FIG. 16 ) can be used to detect the imaging position Location. The position sensor 591 is provided in, for example, the distal end portion 12d of the endoscope 12, and detects the relative position of the distal end portion 12d to a table (not shown) or the like. Then, the determination processing control unit 583 detects a change (or no change) in the imaging position based on a change (or no change) in the position of the distal end portion 12d detected by the position sensor 591 . The position sensor 591 may be a sensor that detects the insertion length of the insertion portion 12a into the subject.

[Sixth Embodiment]

The endoscope system 10 can be based on "changes in the operation of the endoscope 12 or endoscopic images" which are different from the first, second, third, fourth, and fifth embodiments described above. , to start or end the discrimination process. For example, the determination process can be started or ended when the endoscope 12 captures a still image of the observation object (that is, when there is a change in the operation of the endoscope 12 ).

As shown in FIG. 17 , when the endoscope 12 captures a still image of the observation object, the endoscope 12 is provided with a still image capturing instruction unit 601 for inputting an image capturing instruction of the still image. The still image capturing instruction unit 601 is, for example, a so-called freeze button, a release button, or the like provided in the operation unit 12a. In addition, an optical switch (not shown) or the like can be used as the still image capturing instruction unit 601 . The still image capturing instruction unit 601 inputs a still image capturing instruction to the control unit 52 . As a result, the control unit 52 uses the endoscope 12 to capture images of the observation object under conditions for still image capturing, and uses the image processing unit 61 to generate at least one still image of the observation object.

In addition, the image processing unit 61 is provided with a determination processing control unit 683 in place of the determination processing control unit 83 and the like in the above-described embodiments. The determination processing control unit 683 acquires a still image capturing instruction from the still image capturing instruction unit 601 via the control unit 52 . As a result, the determination processing control unit 683 detects that the endoscope 12 has captured a still image. The determination processing control unit 683 starts the determination processing when the endoscope 12 captures a still image.

Specifically, as shown in FIG. 18 , when the endoscope 12 captures a still image of the observation target while the observation target is captured for moving images ( S611 ) ( S612 : YES), the determination processing control unit 683 executes Discrimination processing (S613). This actually becomes the start of the discrimination process when the discrimination process has not been performed in the previous frame. On the other hand, when the endoscope 12 does not capture a still image ( S612 : NO), the determination processing control unit 683 skips the determination processing ( S613 ). Therefore, when a still image is captured in the previous frame and the determination process is performed, this actually ends the determination process.

By repeating the above operation until the observation is completed ( S614 ), the determination process is automatically performed when the endoscope 12 captures the still image of the observation object, and the determination process is not performed when the endoscope 12 does not capture the still image of the observation object . The case where the endoscope 12 captures a still image of the observation target is when the observation target has a lesion or the like and requires detailed observation. Therefore, if the determination process is started and terminated when the endoscope 12 captures a still image as described above, the determination process can be performed at an appropriate timing when the doctor or the like desires the determination process, and the score 99 can be given as the determination result. In addition, unnecessary determination processing is not performed at unnecessary timing, whereby the processing load of the processor device 16 can be reduced.

In addition, the respective determination processing control units such as the determination processing control unit 83 in the first embodiment, the second embodiment, the third embodiment, the fourth embodiment, and the fifth embodiment described above are based on changes in the operation of the endoscope 12 or The endoscopic image controls the start and end of the discrimination processing in the discrimination processing unit 82, but the discrimination processing control units in the above-described embodiments can control only the discrimination processing based on changes in the operation of the endoscope 12 or endoscopic images. Either the start or the end of the discrimination process in the section 82 .

In addition, the respective determination processing control units in the first, second, third, fourth, and fifth embodiments described above start determination based on changes in the operation of the endoscope 12 or endoscopic images. At the time of processing, the determination process can be terminated when a predetermined time elapses after the determination process is started. In this way, if the determination process is terminated after a predetermined time elapses after the determination process is started, the change in the operation of the endoscope 12 or the change in the endoscopic image does not need to be detected in order to end the determination process. The processing of each discrimination processing control unit is reduced in weight.

[Seventh Embodiment]

In the first embodiment, the second embodiment, the third embodiment, the fourth embodiment, the fifth embodiment, and the sixth embodiment described above, according to the change in the operation of the endoscope 12 or the endoscope image control determination process It is started or ended, but the notification of the determination result can be started or ended according to the change in the operation of the endoscope 12 or the endoscope image. At this time, as shown in FIG. 19 , the image processing unit 61 is provided with a notification control unit 690 in addition to the determination processing control unit 83 and the like in the first embodiment.

The notification control unit 690 acquires the determination result from the determination processing unit 82 . Then, the notification control unit 690 inputs the acquired determination result to the display control unit 66 or to other output devices (eg, a speaker that outputs sound or voice, or a light-emitting indicator). As a result, the notification control unit 690 notifies the user of the determination result using, for example, any one of, or a combination of, voice, image, or message.

In addition, the notification control unit 690 does not always output the acquired determination result to the display control unit 66 or the like, but, as in any of the determination processing control units in the above-described embodiments, according to the operation of the endoscope 12 changes or endoscopic images, it is determined whether or not to output the acquired determination result to the display control unit 66 or the like. As a result, the notification control unit 690 starts or ends notification of the determination result according to the change in the operation of the endoscope 12 or the endoscope image. For example, when a score of 99 is acquired as the determination result, and it is determined that the determination result is suitable for notification based on the change in the motion of the endoscope 12 or the endoscopic image, the notification control unit 690 inputs the acquired score of 99 to the display control unit 66, and the result is , the judgment result is notified by displaying the score 99 on the display 18 (refer to FIG. 5 ).

In addition, when a plurality of determination results are acquired, the notification control unit 690 can control the start or end of notification according to the change in the operation of the endoscope 12 or the endoscope image for each determination result. In addition, the notification control unit 690 can start or end notification of the determination result based on the same reference as the reference for the start or end of the determination process. In addition, the notification control unit 690 can also set the criterion for starting or ending the notification of the determination result to a criterion different from the criterion for starting or ending the determination process. For example, when the determination processing control unit 83 controls the start or end of the determination processing by the method of the first embodiment, the notification control unit 690 can start or end notification of the determination result based on the same reference as the determination processing control unit 283 of the second embodiment. .

The notification control unit 690 can notify the determination result together with the information related to the determination result. For example, as shown in FIG. 20 , the notification control unit 690 can notify, as an example of the determination result, the score 692 and the degree of invasion of the lesion 695 , as information related to the determination result, the outline 691 of the specific part (lesion), An image 694 of a similar case or a message 693 such as "There is a possibility of cancer in this area".

Regarding the contour 691 of the specific part (lesion), for example, the determination processing unit 82 can know the position information of the specific part in the endoscopic image for which the determination process has been performed. The message 693 can be obtained by comparing the judgment result with a message database (not shown) prepared in advance. In FIG. 20 , the message 693 is displayed on the display 18 for notification, but the message 693 may be notified by voice. The images 694 of similar cases can be combined with the discrimination results, for example, obtained from a case database (not shown). In FIG. 20 , an image 694 of one similar case is displayed, but when there are a plurality of images 694 of similar cases, the images of a plurality of similar cases can be displayed on the display 18 .

In addition, in FIG. 20, the various scores 692 are displayed in the form of a graph, but the same display can also be performed in the above-described embodiments such as the first embodiment. In addition, in FIG. 20 , the degree of invasion 695 of the lesion, which is an example of the determination result, is shown on the display 18, but when the determination result is displayed in a graph, table, or other form, in each of the above-described embodiments such as the first embodiment, The judgment results can also be displayed in graphs, tables or other forms. The same applies to the judgment results other than the score 692 and the degree of invasion of the lesion 695 . The outline 691 of the lesion, the message 693, and the image 694 of the similar case are examples of information related to the determination result, and the notification control unit 690 can notify other information related to the determination result. For example, in addition to the above, the notification control unit 690 can notify the position of the specific part in the subject, the endoscopic image (conventional image) captured in the past diagnosis, and the like as the information related to the determination result.

In addition, in the seventh embodiment, the image processing unit 61 includes the determination processing control unit 83 in the first embodiment and other determination processing control units and the notification control unit 690 of the respective embodiments, and the image processing unit 61 can replace the determination processing control unit 83 and the like, but only the notification control unit 690 is provided. In this case, the endoscope system 10 includes an image acquisition unit 54 that acquires an endoscopic image obtained by photographing an observation object with an endoscope, and a discrimination processing unit 82 that performs discrimination processing using the endoscopic image to discriminate among the observation objects. A portion having a specific characteristic; and the notification control unit 690 controls the start or end of notification of the determination result, which is the result of the determination process, based on the change in the motion of the endoscope 12 or the endoscopic image.

Note that the notification control unit 61 includes the notification control unit 690 together with the determination processing control unit 83 and the like, or the image processing unit 61 includes only the notification control unit 690 instead of the determination processing control unit 83 or the like. Each of the units 690 can be configured in the same manner as the determination processing control unit 83 and the like in the first embodiment.

For example, it is preferable that the notification control unit 690 includes: a feature amount calculation unit 86 that calculates a feature amount using the endoscopic image; or notification of the end of the judgment result.

In addition, the notification control unit 690 can start or end notification of the determination result when the endoscope 12 zooms in or out of the observation object.

The notification control unit 690 can start the notification of the determination result when the illumination light used when the endoscope 12 images the observation object is switched to the specific illumination light, and can switch the illumination light used when the endoscope 12 images the observation object to other than the specific illumination light. Notification of the end of the judgment result when the light is illuminated.

The notification control unit 690 can start notification of the determination result when the illumination light is switched to the illumination light with a wavelength of 450 nm or less.

The notification control unit 690 can start or end notification of the determination result when the object passes through the channel inserted into the endoscope 12 .

The notification control unit 690 can start notification of the determination result when the water passes through the channel.

The notification control unit 690 can start notification of the determination result when the coloring agent passes through the passage.

The notification control unit 690 can start or end notification of the determination result when the position of the observation object captured by the endoscope changes or when the position of the observation object captured by the endoscope does not change.

The notification control unit 690 can detect the position of the observation object captured by the endoscope based on the change of the observation object captured in the endoscopic image.

The notification control unit 690 can detect the position of the observation object in the endoscope imaging using the position sensor.

The notification control unit 690 can terminate the notification of the determination result when the position where the observation object is captured by the endoscope 12 changes.

The notification control unit 690 can start the notification of the determination result according to the change in the operation of the endoscope 12 or the endoscopic image, and can end the notification of the determination result after a predetermined period of time elapses after the notification of the determination result is started.

The notification control unit 690 can start or end notification of the determination result when the endoscope 12 captures a still image.

In each of the above-described embodiments, the present invention is implemented in the endoscope system 10 in which the endoscope 12 provided with the image sensor 48 is inserted into the subject for observation, but the present invention is also applicable to a capsule endoscope system middle. As shown in FIG. 17 , for example, the capsule endoscope system includes at least a capsule endoscope 700 and a processor device (not shown).

The capsule endoscope 700 includes a light source unit 702 , a control unit 703 , an image sensor 704 , an image processing unit 706 , and a transmission/reception antenna 708 . The light source unit 702 corresponds to the light source unit 20 . The control unit 703 functions similarly to the light source control unit 22 and the control unit 52 . In addition, the control unit 703 can communicate with the processor device of the capsule endoscope system by wireless using the transmission/reception antenna 708 . The processor device of the capsule endoscope system is substantially the same as the processor device 16 of the above-described embodiment, but an image processing unit 706 corresponding to the image acquisition unit 54 and the image processing unit 61 is provided in the capsule endoscope 700, and the endoscope The mirror image is sent to the processor device via the transceiver antenna 708 . The image sensor 704 has the same configuration as the image sensor 48 .

Symbol Description

10-endoscope system, 12-endoscope, 12a-insertion part, 12b-operation part, 12c-bending part, 12d-tip part, 12e-angle knob, 13a-zoom operation part, 13b-mode switch , 14-light source device, 16-processor device, 18-display, 19-console, 20-light source part, 22-light source control part, 30a-illumination optical system, 30b-camera optical system, 41-light guide, 45 -Illumination lens, 46-Objective lens, 47-Zoom lens, 48-Image sensor, 52-Control section, 54-Image acquisition section, 56-DSP (Digital-Signal-Processor), 58-Noise reduction section , 59-conversion part, 61-image processing part, 66-display control part, 71-CNN (Convolution-Neural Network, Convolution-Neural-Network), 72-storage part, 81-image generation part, 82-discrimination processing Section, 83, 283, 383, 483, 583, 683 - Discrimination processing control section, 86 - Feature calculation section, 87 - Comparison section, 99, 692 - Score, 101, 102 - Endoscopic image, 401 - Forceps channel , 402-air supply and water supply channel, 403, 404-pass sensor, 586-position detection part, 591-position sensor, 601-still image camera instruction part, 690-notification control part, 691-outline, 693-message, 694- Images of similar cases, 695-invasion degree of the lesion, 700-capsule endoscope, 702-light source part, 703-control part, 704-image sensor, 706-image processing part, 708-transmitting antenna.

Claims (31)

1. An endoscope system comprising: an image acquisition unit for acquiring an endoscopic image obtained by photographing an observation object with an endoscope; a discrimination processing unit for discriminating a portion of the observation object having a specific feature by performing discrimination processing using the endoscopic image; and The determination processing control unit controls the start or end of the determination processing based on changes in the operation of the endoscope or the endoscope image. 2. The endoscopic system of claim 1, wherein, The discrimination processing control unit includes: a feature quantity calculation unit that calculates a feature quantity using the endoscopic image; and a comparison unit that compares the feature quantity with a threshold value, The discrimination processing control unit starts or ends the discrimination processing based on the comparison result in the comparison unit. 3. The endoscopic system of claim 1, wherein, The determination processing control unit starts or ends the determination processing when the endoscope enlarges or reduces the observation object. 4. The endoscopic system of claim 1, wherein, The discrimination processing control unit starts the discrimination processing when the illumination light used when the observation object is photographed by the endoscope is switched to a specific illumination light, and is used when the endoscope photographs the observation object When the illumination light of , is switched to illumination light other than the specific illumination light, the determination process ends. 5. The endoscopic system of claim 4, wherein, The discrimination processing control unit starts the discrimination processing when the illumination light is switched to illumination light having a wavelength of 450 nm or less. 6. The endoscopic system of claim 1, wherein, The determination processing control unit starts or ends the determination processing when the object passes through the channel inserted into the endoscope. 7. The endoscopic system of claim 6, wherein, The discrimination processing control unit starts the discrimination processing when water passes through the passage. 8. The endoscopic system of claim 6, wherein, The discrimination processing control unit starts the discrimination processing when the coloring agent passes through the passage. 9. The endoscopic system of claim 1, wherein, The discrimination processing control unit starts or ends the discrimination processing when the position at which the observation object is captured by the endoscope changes or when the position at which the endoscope captures the observation object does not change . 10. The endoscopic system of claim 9, wherein, The determination processing control unit detects a position where the endoscope imaged the observation object based on a change in the observation object captured in the endoscopic image. 11. The endoscopic system of claim 9, wherein, The discrimination processing control unit detects a position at which the observation object is imaged by the endoscope using a position sensor. 12. The endoscopic system of claim 9, wherein, The determination processing control unit ends the determination processing when the position where the observation object is captured by the endoscope has changed. 13. The endoscopic system of claim 1, wherein, The discrimination processing control unit starts the discrimination processing based on a change in the motion of the endoscope or the endoscope image, and ends the discrimination processing after a predetermined period of time elapses after the discrimination processing is started. Discriminate processing. 14. The endoscopic system of claim 1, wherein, The determination processing control unit starts or ends the determination processing when the endoscope captures a still image. 15. The endoscopic system of claim 1, comprising: The notification control unit controls the start or end of notification of the determination result, which is the result of the determination process, based on the change in the operation of the endoscope or the endoscope image. 16. The endoscopic system of claim 15, wherein, The notification control unit includes: a feature quantity calculation unit that calculates a feature quantity using the endoscopic image; and a comparison unit that compares the feature quantity with a threshold value, The notification control unit starts or ends notification of the determination result based on the comparison result in the comparison unit. 17. The endoscopic system of claim 15, wherein, The notification control unit starts or ends notification of the determination result when the endoscope enlarges or reduces the observation object. 18. The endoscopic system of claim 15, wherein, The notification control unit starts notification of the determination result when the illumination light used when the observation object is imaged by the endoscope is switched to a specific illumination light, and when the endoscope images the observation object When the illumination light to be used is switched to illumination light other than the specific illumination light, the notification of the determination result is terminated. 19. The endoscopic system of claim 18, wherein, The notification control unit starts notification of the determination result when the illumination light is switched to illumination light having a wavelength of 450 nm or less. 20. The endoscopic system of claim 15, wherein, The notification control unit starts or ends the notification of the determination result when the object passes through the channel inserted into the endoscope. 21. The endoscopic system of claim 20, wherein, The notification control unit starts notification of the determination result when water passes through the passage. 22. The endoscopic system of claim 20, wherein, The notification control unit starts notification of the determination result when the coloring agent passes through the passage. 23. The endoscopic system of claim 15, wherein, The notification control unit starts or ends the determination result when the position at which the observation object is captured by the endoscope changes or when the position at which the endoscope captures the observation object does not change. Notice. 24. The endoscopic system of claim 23, wherein, The notification control unit detects a position where the endoscope imaged the observation object based on a change in the observation object captured in the endoscopic image. 25. The endoscopic system of claim 23, wherein, The notification control unit detects a position at which the observation object is imaged by the endoscope using a position sensor. 26. The endoscopic system of claim 23, wherein, The notification control unit ends the notification of the determination result when the position where the observation object is captured by the endoscope has changed. 27. The endoscopic system of claim 15, wherein, The determination processing control unit starts notification of the determination result based on a change in the motion of the endoscope or the endoscope image, and a predetermined time elapses after the notification of the determination result is started. After that, the notification of the determination result ends. 28. The endoscopic system of claim 15, wherein, The notification control unit starts or ends notification of the determination result when the endoscope captures a still image. 29. The endoscopic system of claim 15, wherein, The notification control unit notifies the determination result using any one of voice, image, or message, or a combination of these. 30. A processor device comprising: an image acquisition unit for acquiring an endoscopic image obtained by photographing an observation object with an endoscope; a discrimination processing unit for discriminating a portion of the observation object having a specific feature by performing discrimination processing using the endoscopic image; and The determination processing control unit controls the start or end of the determination processing based on changes in the operation of the endoscope or the endoscope image. 31. An operating method of an endoscope system, comprising: an image acquisition unit acquiring an endoscopic image obtained by photographing an observation object with an endoscope; and a determination processing unit performing determination by using the endoscopic image processing, the step of judging the part of the observation object having a specific characteristic, in the working method of the endoscope system, A determination processing control unit includes a step of controlling the start or end of the determination processing based on changes in the operation of the endoscope or the endoscope image.