JP6834019B2 - Medical image processing equipment and endoscopic equipment - Google Patents

Description

The present invention relates to a medical image processing device and an endoscopic device that use the analysis results of medical images.

Conventionally, among devices related to medical treatment (hereinafter referred to as medical devices), those for acquiring an image including a subject image (hereinafter referred to as a medical image) present the acquired medical image to a doctor. Then, the doctor uses the medical image obtained from the medical device as one of the judgment materials to make a diagnosis or the like. As a matter of course, the discrimination of the condition of the subject using the medical image at the time of diagnosis is based on the skill and experience of the doctor.

In recent years, since image analysis technology has advanced, it is possible to obtain objective information or quantitative information from medical images by analyzing medical images. For this reason, the number of medical devices that support discrimination, diagnosis, and the like by presenting the analysis results of medical images to doctors and the like is increasing. For example, the endoscope device described in Patent Document 1 is abnormal by using a fluorescence image taken by fluorescence or a narrow band optical image taken by using light having a specific narrow wavelength band (so-called narrow band light). Identify the location of an area. Then, the position of the region where the abnormality exists is displayed on the endoscopic image for display.

Japanese Unexamined Patent Publication No. 2006-198106

In a conventional medical device, a notable area (so-called area of interest, area of interest) including one or more parts (lesion part or part having a possibility of lesion) in which an abnormality such as a lesion is recognized by analyzing a medical image is included. , Or an area called an abnormal area or the like. Hereinafter, simply referred to as an abnormal area), the position or the like is presented to support diagnosis or the like.

Doctors and the like discriminate the type and degree of progression of lesions and the like in the abnormal region presented by the medical device, but the discrimination of the type and degree of progression of lesions and the like is still a heavy burden. For this reason, in recent years, in order to further reduce the burden on doctors and the like, it has been required to provide detailed information that can be used as a material for differentiation regarding the type and degree of progression of lesions and the like.

An object of the present invention is to provide a medical image processing device and an endoscopic device capable of presenting more detailed and accurate information about lesions and the like than before.

The medical image processing apparatus of the present invention uses a medical image acquisition unit that acquires a medical image including a subject image, and a first medical image taken by using a first illumination light having a specific spectrum among the medical images. The image was taken using a first discrimination processing unit that performs a first discrimination process for detecting one or a plurality of types of first abnormal regions, and a second illumination light having a spectrum different from that of the first illumination light among medical images. A second discrimination processing unit that performs a second discrimination process for identifying the type of abnormality included in each first abnormal region using a second medical image, and a first illumination light and a second illumination light among the medical images. Includes a third discrimination processing unit that performs a third discrimination process for identifying the progress of an abnormality included in the first abnormality region using a third medical image taken using a third illumination light having a different spectrum. ..

The second discrimination processing unit identifies the type of abnormality included in the first abnormal region by performing the second discrimination processing on the second abnormal region included in the second medical image corresponding to the first abnormal region. Is preferable.

The third discrimination processing unit performs the third discrimination processing on the third abnormal region included in the third medical image corresponding to at least one of the first abnormal region and the second abnormal region, so that the first abnormal region becomes It is preferable to identify the degree of progression of the including abnormality.

It is preferable that the third discrimination processing unit selects the third medical image using the result of the second discrimination processing.

The third discrimination processing unit preferably selects a third medical image acquired in the spectrum of the third illumination light associated with the result of the second discrimination processing.

The medical image acquisition unit acquires the observation medical image in addition to the first medical image, the second medical image, and the third medical image, and while displaying the observation medical image on the display unit, the first It is preferable that the 1 discrimination processing unit performs the 1st discrimination processing, the 2nd discrimination processing unit performs the 2nd discrimination processing, and the 3rd discrimination processing unit performs the 3rd discrimination processing.

The medical image acquisition unit sets any one of the first medical image, the second medical image, and the third medical image as the observation medical image, and while the observation medical image is displayed on the display unit. It is preferable that the first discrimination processing unit performs the first discrimination processing, the second discrimination processing unit performs the second discrimination processing, and the third discrimination processing unit performs the third discrimination processing.

It is preferable to include a setting unit for changing the type of abnormality specified in the second discrimination process or the setting of the classification of the degree of progress specified in the third discrimination process.

It is preferable that the setting unit changes the setting of each spectrum or combination of the first illumination light, the second illumination light, or the third illumination light.

It is preferable that the medical record acquisition unit for acquiring the medical record is provided, and the setting unit makes settings using the information contained in the medical record.

It is preferable that the setting unit includes a specific lesion determined by the information contained in the medical record in the type of abnormality specified in the second discrimination process.

It is preferable to include a discrimination result correction unit that corrects the result of the first discrimination process, the result of the second discrimination process, or the result of the third discrimination process.

It is preferable to include a discrimination result notification unit that notifies the result of the first discrimination process, the result of the second discrimination process, or the result of the third discrimination process.

The endoscope device of the present invention was taken by using an endoscopic image acquisition unit that acquires an endoscopic image including a subject image and a first illumination light having a specific spectrum among the endoscopic images. The spectrum of the first discrimination processing unit that performs the first discrimination processing for detecting one or a plurality of types of the first abnormal regions using the first endoscopic image and the first illumination light of the endoscopic images are different from each other. A second discrimination processing unit that performs a second discrimination process for identifying the type of abnormality included in each first abnormal region using a second endoscope image taken by using the second illumination light, and an endoscope. Among the images, the third medical image taken by using the third illumination light having a spectrum different from that of the first illumination light and the second illumination light is used to identify the progress of the abnormality included in the first abnormality region. It is equipped with three discrimination processing units.

It is preferable that the third discrimination processing unit selects the spectrum of the illumination light used when acquiring the third medical image by using the result of the second discrimination processing.

The medical image processing apparatus and the endoscopic apparatus of the present invention can provide more detailed and accurate information about lesions and the like than before.

It is a block diagram of a medical image processing apparatus. It is a block diagram of an endoscope device. It is a graph which shows the spectrum of the illumination light which a light source device can emit. It is a graph which shows the spectrum of the illumination light for observation. It is a graph which shows the spectrum of the illumination light for discrimination of pattern 1. It is a graph which shows the spectrum of the illumination light for discrimination of pattern 2. It is a graph which shows the spectrum of the illumination light for discrimination of pattern 3. It is a graph which shows the spectrum of the illumination light for discrimination of pattern 4. It is a graph which shows the spectrum of the illumination light for discrimination of pattern 5. It is a graph which shows the spectrum of the illumination light for discrimination of pattern 6. It is a graph which shows the spectrum of the illumination light for discrimination of pattern 7. It is a graph which shows the spectrum of the illumination light for discrimination of pattern 8. It is a graph which shows the spectrum of the illumination light for discrimination of pattern 9. It is a block diagram of a medical image analysis processing unit. It is a first endoscopic image (first medical image). It is a second endoscopic image (second medical image). It is a block diagram of the 2nd discrimination processing part. It is a third endoscopic image (third medical image). It is a block diagram of the 3rd discrimination processing part. It is explanatory drawing which shows the content of setting in a setting part. It is a flowchart which shows the operation of a medical image processing apparatus. This is an example of a display screen showing the discrimination result. It is a block diagram of the medical image analysis processing part which provided the medical record acquisition part. It is a block diagram of the medical image analysis processing part which provided the discrimination result correction part. It is a block diagram of the medical image analysis processing unit provided with the discrimination result notification unit. It is a block diagram of an endoscope apparatus including a medical image processing apparatus. It is a flowchart when the discrimination process is performed in real time in an endoscope apparatus. It is a block diagram of an endoscope apparatus including a medical image processing apparatus. It is a table which shows the observation condition and the frequency of photography. It is explanatory drawing of the diagnosis support apparatus including a medical image processing apparatus. It is explanatory drawing of the medical business support apparatus including a medical image processing apparatus.

[First Embodiment]
As shown in FIG. 1, the medical image processing device 10 includes a medical image acquisition unit 11, a medical image analysis processing unit 12, a display unit 13, a display control unit 15, an input / reception unit 16, a general control unit 17, and a storage unit. 18 is provided.

The medical image acquisition unit 11 includes a medical image including a subject image directly from an endoscope device 21 or the like which is a medical device, or via a management system such as a PACS (Picture Archiving and Communication System) 22 or another information system. To get. The medical image is a still image or a moving image (so-called inspection moving image). When the medical image is a moving image, the medical image acquisition unit 11 can acquire a frame image constituting the moving image as a still image after the examination. When the medical image is a moving image, the display of the medical image includes displaying a still image of one representative frame constituting the moving image and playing the moving image one or more times. Further, in the medical image acquired by the medical image acquisition unit 11, in addition to the image taken by the doctor using the medical device such as the endoscope device 21, the medical device such as the endoscope device 21 is instructed by the doctor to take a picture. Includes images taken automatically regardless.

When the medical image acquisition unit 11 can acquire a plurality of medical images, the medical image acquisition unit 11 can selectively acquire one or a plurality of these medical images. In addition, the medical image acquisition unit 11 can acquire a plurality of medical images acquired in a plurality of different tests. For example, one or both of a medical image acquired by a past examination and a medical image acquired by the latest examination can be acquired. That is, the medical image acquisition unit 11 can arbitrarily acquire a medical image.

In the present embodiment, the medical image processing device 10 is connected to the endoscope device 21 and acquires a medical image from the endoscope device 21. That is, in the present embodiment, the medical image is an endoscopic image.

Further, discrimination of the presence or absence of an abnormal region such as an abnormal region 102 (see FIG. 15 and the like) (detection of the abnormal region 102), discrimination of the type of lesion and the like included in the abnormal region 102, and progression of the lesion and the like contained in the abnormal region 102. In order to discriminate the degree, the medical image acquisition unit 11 acquires at least a plurality of endoscopic images (medical images) having different imaging conditions. In the present embodiment, the medical image acquisition unit 11 is referred to as an observation endoscope image displayed on the display unit 13 (hereinafter referred to as an observation endoscope image 101 (observation medical image); see FIG. 22). , An endoscopic image used to distinguish the presence or absence of the abnormal region 102 (hereinafter referred to as a first endoscopic image 61; see FIG. 15) and an endoscopic image used to distinguish the type of lesion or the like contained in the abnormal region 102. (Hereinafter referred to as a second endoscopic image 63; see FIG. 16) and an endoscopic image used for distinguishing the degree of progression of lesions and the like included in the abnormal region 102 (hereinafter referred to as a third endoscopic image 66). 18.) and. The imaging conditions of the observation endoscope image 101, the first endoscope image 61, the second endoscope image 63, and the third endoscope image 66 are different from each other in order to match their respective purposes of use. ..

The imaging conditions are conditions related to the imaging of medical images, such as the spectrum of illumination light, the presence / absence or intensity of image processing when generating a medical image, and the like. The spectrum of illumination light is an intensity distribution for each wavelength, and includes the concept of wavelength band and center wavelength. The image processing when generating a medical image is, for example, a process related to adjustment of a color or the like that emphasizes a specific tissue or lesion.

As shown in FIG. 2, the endoscope device 21 to which the medical image processing device 10 is connected in the present embodiment irradiates at least one of light in a white wavelength band or light in a specific wavelength band to photograph a subject. The endoscope 31, the light source device 32 that irradiates the subject with illumination light through the endoscope 31, the processor device 33, and the monitor 34 that displays an endoscope image taken by using the endoscope 31. Has.

The endoscope 31 includes an image sensor 41 that captures a subject by using illumination light reflected or scattered by the subject, fluorescence emitted by the subject or a drug administered to the subject, or the like. The image sensor 41 is, for example, a CMOS (Complementary Metal-Oxide-Semiconductor) color sensor (sensor having a color filter).

The light source device 32 includes a light source unit 42 and a light source control unit 47. The light source unit 42 emits a plurality of types of illumination light having different spectra. The light source unit 42 includes, for example, a light emitting device such as an LED (Light Emitting Diode), an LD (Laser Diode), or a xenon lamp. Further, the light source unit 42 includes a prism, a mirror, an optical fiber, an optical filter for adjusting the wavelength band, the amount of light, or the like, if necessary. In the present embodiment, the light source unit 42 includes a V-LED43 that emits purple light having a central wavelength of about 405 nm, a B-LED44 that emits blue light having a central wavelength of about 450 nm, and a green light having a central wavelength of about 540 nm. It includes a G-LED 45 that emits light and an R-LED 46 having a center wavelength of about 630 nm.

The light source control unit 47 controls the light emitting source included in the light source unit 42, and generates the illumination light used by the endoscope 31 for photographing the subject. Further, when the light source unit 42 includes a plurality of light emitting devices, the light source control unit 47 can control the light emitting timing and the light emitting amount of each light emitting device. Therefore, the light source device 32 can supply a plurality of types of illumination light having different spectra to the endoscope 31 at an arbitrary timing and at an arbitrary intensity. That is, in the present embodiment, the light source device 32 emits two or more of purple light, blue light, green light, red light, or light of each of these colors, in addition to white light, under the control performed by the light source control unit 47. Light or the like mixed at an arbitrary intensity ratio can be emitted as illumination light at an arbitrary timing and at an arbitrary intensity.

For example, in the present embodiment, as shown in FIG. 3, the light source device 32 emits purple light emitted by the V-LED43 (represented by the symbol “V”; the same applies hereinafter) and blue light emitted by the B-LED44. (Represented by the symbol "B"; the same shall apply hereinafter), the green light emitted by the G-LED45 (represented by the symbol "G". The same applies hereinafter.) And can emit illumination light including. The light source control unit 47 arbitrarily controls the amount of each color light of the purple light V, the blue light B, the green light G, and the red light R contained in the illumination light according to the purpose of photographing and the like.

For example, when obtaining the observation endoscope image 101, the light source device 32 emits illumination light having the observation spectrum shown in FIG. 4 (hereinafter, referred to as observation illumination light). The illumination light for observation is so-called white light. Further, when obtaining the first endoscopic image 61, the second endoscopic image 63, or the third endoscopic image 66 for various discrimination processes, the light source device 32 is, for example, an illumination having the spectrum shown in FIG. Light (hereinafter referred to as pattern 1 identification illumination light), illumination light having the spectrum shown in FIG. 6 (hereinafter referred to as pattern 2 identification illumination light), and illumination light having the spectrum shown in FIG. 7 (hereinafter, pattern 3). The identification illumination light having the spectrum shown in FIG. 8 (hereinafter referred to as the identification illumination light of the pattern 4), and the illumination light having the spectrum shown in FIG. 9 (hereinafter referred to as the identification illumination light of the pattern 5). ), The illumination light having the spectrum shown in FIG. 10 (hereinafter referred to as the identification illumination light of the pattern 6), the illumination light having the spectrum shown in FIG. 11 (hereinafter referred to as the identification illumination light of the pattern 7), shown in FIG. Illumination light having a spectrum or illumination light having a spectrum shown in FIG. 13 (hereinafter, referred to as identification illumination light of pattern 9) is appropriately switched to emit light.

The identification illumination light of pattern 1 (see FIG. 5) is used when acquiring the first endoscope image 61. The identification illumination light of pattern 2 (see FIG. 6) is used when acquiring the second endoscopic image 63. The identification illumination light of the pattern 6 (see FIG. 10), the identification illumination light of the pattern 3 (see FIG. 7), and the identification illumination light of the pattern 4 (see FIG. 4) are the third endoscope image 66. Used when acquiring. More specifically, the identification illumination light of pattern 6 is used when acquiring the third endoscopic image 66 for differentiating the progress of the polyp, and the identification illumination light of pattern 3 is ulcerative. A third endoscope used to acquire the third endoscope image 66 for distinguishing the progress of colitis, and the illumination light for discrimination of pattern 4 is a third endoscope for distinguishing the progress of Crohn's disease. It is used when acquiring the image 66.

In addition, the light source device 32 can emit light having a specific narrow wavelength band (so-called narrow band light) as illumination light depending on the characteristics of the light emitting device or by using an optical filter. For example, light in a wavelength band shorter than the green wavelength band, particularly light in the blue band or purple band in the visible region can be emitted.

The processor device 33 acquires an endoscopic image from the image sensor 41, or an endoscopic image generation unit 48 that generates an endoscopic image obtained by applying image processing to the endoscopic image acquired from the image sensor 41. Be prepared. The image sensor 41 and the endoscopic image generation unit 48 constitute an "endoscopic image acquisition unit" in the endoscope device 21. The endoscopic image acquisition unit acquires an endoscopic image including a subject image by photographing the subject using illumination light. The medical image processing device 10 is connected to the processor device 33. Then, the medical image acquisition unit 11 acquires the endoscopic image directly from the endoscopic image generation unit 48 of the endoscope device 21.

The medical image analysis processing unit 12 performs analysis processing using an endoscopic image which is a medical image acquired by the medical image acquisition unit 11. Specifically, as shown in FIG. 14, the first discrimination processing unit 51, the second discrimination processing unit 52, the third discrimination processing unit 53, and the setting unit 56 are included.

The first discrimination processing unit 51 discriminates the presence or absence of the abnormal region 102 using a medical image. Specifically, the first discrimination processing unit 51 uses the first medical image taken by using the first illumination light having a specific spectrum among the medical images acquired by the medical image acquisition unit 11 and 1 or The first discrimination process for detecting a plurality of types of abnormal regions 102 is performed. In the present embodiment, the "first illumination light having a specific spectrum" is the identification illumination light of the pattern 1, and as shown in FIG. 15, the first medical image is the first endoscopic image 61. is there. Hereinafter, when it is necessary to distinguish from the abnormal region 102 in other medical images, the abnormal region 102 in the first endoscopic image 61 (first medical image) is referred to as a “first abnormal region”. For example, the first discrimination processing unit 51 calculates one or a plurality of predetermined feature amounts using the first endoscopic image 61, and detects the abnormal region 102 using the calculated feature amounts. The predetermined feature amount is, for example, color or brightness, shape, length, thickness, depth from mucosal surface, density, size, distribution, etc. of a specific tissue or structure such as blood vessels, number of branches, etc. , Complexity, disorder of regularity, or other features. Instead of using the feature amount, the first discrimination processing unit 51 can perform the first discrimination processing by, for example, texture analysis or pattern matching using a regular shape.

When the medical image acquisition unit 11 acquires an endoscopic image, the target detected by the first discrimination processing unit 51 as an abnormal region 102 has a characteristic of color or shape different from one or more lesions, surrounding tissues, and the like. Area to have, area where drug is sprayed, or treatment (biopsy, endoscopic mucosal resection (EMR)), or endoscopic submucosal dissection (ESD) ) Etc.), etc. are the areas that include the target. When the medical image is an endoscopic image, the lesion or the like is, for example, inflammation related to an inflammatory disease such as ulcerative colitis, Crohn's disease, or a polyp (elevated lesion). More specifically, the polyp is a hyperplastic polyp (HP), SSA / P (sessile serrated adenoma / polyp), adenoma, cancer, or the like. Further, the region having a color or shape characteristic different from that of the surrounding tissue or the like is redness, atrophy, diverticulum, treatment scar, or the like of the subject.

The first discrimination processing unit 51 detects the abnormal region 102 for a part or all of the medical image. That is, the first discrimination processing unit 51 performs the first discrimination processing for each pixel, for each small area when the endoscopic image is divided into small areas, or for the entire endoscopic image. It can be carried out. In the present embodiment, the first discrimination processing unit 51 determines whether or not it is an abnormal region 102 for each small region composed of a predetermined number of pixels. Therefore, the first discrimination processing unit 51 can detect the abnormal region 102 at one or a plurality of locations of the medical image. The first discrimination processing unit 51 can include "probability" indicating the certainty of the processing result in the result of the first discrimination processing.

The second discrimination processing unit 52 discriminates the type of abnormality included in the first abnormal region (abnormal region 102 in FIG. 15). Specifically, the second discrimination processing unit 52 captures a second medical image captured by using a second illumination light having a spectrum different from that of the first illumination light among the medical images acquired by the medical image acquisition unit 11. The second discrimination process for identifying the type of abnormality contained in each first abnormal region is performed. In the present embodiment, the "second illumination light having a spectrum different from that of the first illumination light" is the identification illumination light of the pattern 2, and as shown in FIG. 16, the second medical image is in the second. It is an endoscopic image 63. The second discrimination processing unit 52 performs the second discrimination processing on at least the "second abnormal region included in the second medical image" corresponding to the first abnormal region, so that the abnormality included in the first abnormal region is present. Identify the type. The second abnormal region is an abnormal region 102 in the second endoscopic image 63 (second medical image). In the second abnormal region (abnormal region 102 in FIG. 16), the position in the subject and other features correspond to the first abnormal region. Since it is sufficient that the target of the second discrimination processing includes the second abnormal region, the second discrimination processing unit 52 does not specify the position or the like of the second abnormal region, for example, and the second endoscope The second discrimination process can be performed on the entire image 63 to identify the type of abnormality included in the first abnormal region.

The second discrimination processing unit 52 includes one or a plurality of type discrimination units according to the number of “abnormal types” to be discriminated. The type discrimination unit discriminates whether or not the abnormality contained in the abnormal region 102 is a specific lesion. In the present embodiment, as shown in FIG. 17, the second type discrimination processing unit 52 includes the first type discrimination unit 71, the second type discrimination unit 72, the third type discrimination unit 73, and other type discrimination units ( Not shown). The first-type discrimination unit 71 includes abnormalities included in the abnormal regions 102 detected in the first discrimination processing (when a plurality of abnormal regions 102 are detected, they are specific abnormal regions 102. The same shall apply hereinafter). Distinguish whether or not is a "polyp". The second type discrimination unit 72 discriminates whether or not the abnormality contained in the abnormality region 102 detected in the first discrimination process is “ulcerative colitis”. The third type discrimination unit 73 discriminates whether or not the abnormality contained in the abnormal region 102 detected in the first discrimination process is “Clone's disease”. Similarly, the other type discrimination unit also discriminates whether or not the abnormality contained in the abnormal region 102 is a specific lesion.

The result of the second discrimination process includes at least information indicating the type of abnormality contained in the abnormal region 102 (“polyp”, “ulcerative colitis”, “Clone's disease”, etc.). The result of the second discrimination treatment does not necessarily definitively identify one type of abnormality, such as "40% probability of ulcerative colitis and 40% probability of Crohn's disease". In addition, it is information that can narrow down the plausible types of abnormalities together with the probability (probability) indicating the certainty of the discrimination result in each type discrimination unit. Therefore, in the second discrimination process, "identifying the type of abnormality" means obtaining information capable of narrowing down the type of abnormality.

The third discrimination processing unit 53 discriminates the progress of the abnormality included in the first abnormal region (abnormal region 102 in FIG. 15). Specifically, the third discrimination processing unit 53 took a picture using the third illumination light having a spectrum different from that of the first illumination light and the second illumination light among the medical images acquired by the medical image acquisition unit 11. Using the third medical image, a third discrimination process for identifying the progress of the abnormality included in the abnormal region 102 is performed. In the present embodiment, the "third illumination light having a spectrum different from that of the first illumination light and the second illumination light" is the identification illumination light of any of patterns 3 to 9, and as shown in FIG. In addition, the third medical image is the third endoscopic image 66. In addition, the third discrimination processing unit 53 performs the third discrimination processing on the "third abnormal region included in the third medical image" corresponding to at least one of the first abnormal region and the second abnormal region. The progress of the abnormality included in the first abnormal region is specified. The third abnormal region is an abnormal region 102 in the third endoscopic image 66 (third medical image). In the third abnormal region (abnormal region 102 in FIG. 18), the position in the subject and other features correspond to the first abnormal region or the second abnormal region. Since it is sufficient that the target of the third discrimination processing includes the third abnormal region, the third discrimination processing unit 53 does not specify the position or the like of the third abnormal region, for example, and the third endoscope The third discrimination process can be performed on the entire image 66 to identify the type of abnormality included in the first abnormal region.

The third discrimination processing unit 53 includes one or a plurality of progress discrimination units according to the number of “types of abnormality” to be discriminated. The progress discrimination unit discriminates the progress of a specific type of abnormality. In the present embodiment, as shown in FIG. 19, the third progress discrimination processing unit 53 includes a first progress discrimination section 81, a second progress discrimination section 82, a third progress discrimination section 83, and other progress. It is equipped with a degree discrimination unit (not shown). The first progress discrimination unit 81 discriminates the progress of the polyp. The second progression degree discrimination unit 82 discriminates the progression degree of ulcerative colitis. The third progression degree discrimination unit 83 discriminates the progression degree of Crohn's disease. The same applies to other progress discrimination departments.

The result of the third discrimination treatment is the degree of progression of the lesion or the like. When the medical image is an endoscopic image, the degree of progression is discriminated according to the so-called endoscopic finding classification. The endoscopic findings classification for polyps (including cancer) is, for example, NICE (The Narrow-band imaging International Colorectal Endoscopic) classification or JNET (The Japan NBI (Narrow Band Imaging) Expert Team). There are classifications, etc. In the present embodiment, when the progress of the polyp is discriminated in the third discrimination process, it is classified into "NICE Type 1", "NICE Type 2", or "NICE Type 3" according to the NICE classification. "NICE Type 1" represents a hyperplastic lesion, "NICE Type 2" represents an adenoma (adenoma) to an intramucosal cancer (so-called M cancer), and "NICE Type 3" represents a submucosal invasive cancer (so-called SM cancer). ..

The endoscopic findings classification of ulcerative colitis includes, for example, Mayo classification, Matts classification, UCEIS (Ulcerative Colitis Endoscopic. Index of Severity) classification and the like. Among these endoscopic findings classification, in this embodiment, the degree of progression of ulcerative colitis is classified according to the Mayo classification. There are four grades in the Mayo classification: Mayo0 (grade 0), Mayo1 (grade 1), Mayo2 (grade 2), and Mayo3 (grade 3). Mayo0 is a grade that represents normal or inactive (including in remission). Mayo1 is a grade showing mild illness, and is generally a condition in which redness, unclear blood vessel image, or mild bleeding is observed. Mayo2 is a grade indicating moderate disease, and is generally a condition in which marked redness, loss of blood vessel image, easy bleeding, adhesion of purulent secretions, rough mucosa, erosion, or partial ulcer are observed. Is. Mayo3 is a grade indicating severe illness (active phase), and is generally a condition in which obvious spontaneous bleeding, edema, ulcer (including extensive ulcer) and the like are observed. In addition, in the third differential treatment, dysplasia associated with ulcerative colitis is differentiated.

The endoscopic findings classification of Crohn's disease includes, for example, SESCD (Simple Endoscopic Score for Crohn's Disease). In this embodiment, the degree of progression of Crohn's disease is classified into "SESCD 0", "SESCD 1", "SESCD 2", or "SESCD 3" according to SESCD.

When the result of the second discrimination treatment is the result of narrowing down the types of lesions to a plurality of types, in the third discrimination treatment, the degree of progression of each type of lesion is differentiated, and the degree of progression of one or more lesions is determined. Output as a result of the third discrimination process. For example, when the result of the second differential treatment is "40% probability of ulcerative colitis and 40% probability of Crohn's disease", in the third differential treatment, the degree of progression of ulcerative colitis and Differentiate from the degree of progression of Crohn's disease. Then, the degree of progression of each lesion narrowed down in the second differential treatment is, for example, "20% probability of Mayo1 in the case of ulcerative colitis" and "95% probability of SESCD1 in the case of Crohn's disease". If so, among these results, the result relating to the latter Crohn's disease is output as the result of the third discrimination process.

The third discrimination processing unit 53 selects one or a plurality of third medical images using the result of the second discrimination processing. In addition, the third discrimination processing unit 53 selects a third medical image acquired in the spectrum of the third illumination light associated with the result of the second discrimination processing. For example, when the result of the second discrimination process includes a "polyp", the endoscope image taken by using the discrimination illumination light of the pattern 6 is selected and acquired as the third endoscope image 66. Similarly, when the result of the second discrimination treatment includes "ulcerative colitis", an endoscopic image taken using the discrimination illumination light of pattern 3 is selected and acquired as the third endoscopic image 66. When the result of the second discrimination process includes "Crohn's disease", the endoscopic image taken by using the discrimination illumination light of the pattern 4 is selected and acquired as the third endoscope image 66.

The medical image acquisition unit 11 can acquire medical images in advance that may be requested by the second discrimination processing unit 52 and the third discrimination processing unit 53. Further, when the medical image acquisition unit 11 has not acquired the medical image requested by the second discrimination processing unit 52 or the third discrimination processing unit 53, the medical image acquisition unit 11 requests the second discrimination processing unit 52 or the third discrimination processing unit 53. Therefore, the medical image used for the second discrimination process or the third discrimination process can be acquired.

Further, the first discrimination process, the second discrimination process, and the third discrimination process are performed in the background while displaying the medical image for observation on the display unit 13. That is, while the medical image for observation (endoscopic image 101 for observation in the present embodiment) is displayed on the display unit 13, the first discrimination processing unit 51 performs the first discrimination processing and the second discrimination processing. The second discrimination processing unit 52 performs the second discrimination processing, and the third discrimination processing unit 53 performs the third discrimination processing. In the present embodiment, the medical image acquisition unit 11 acquires an observation medical image in addition to the first medical image, the second medical image, and the third medical image, and the medical image acquisition unit 11 is the first. When any one of the 1 medical image, the 2nd medical image, and the 3rd medical image is set as the observation medical image (the 1st medical image, the 2nd medical image, or the 3rd medical image is the observation medical image). The same applies when using as). The display control unit 15 can appropriately update the observation endoscope image 101 displayed on the display unit 13 during the first discrimination process, the second discrimination process, or the third discrimination process.

The setting unit 56 can store and change the setting of the type of abnormality (discrimination target) specified in the second discrimination process or the classification of the degree of progress (discrimination content) specified in the third discrimination process. .. For example, the setting unit 56 displays the setting change menu window shown in FIG. 20 on the display unit 13. When a doctor or the like uses a mouse or the like to check the check box 91 (“re”), the corresponding discrimination target and discrimination content can be activated. In FIG. 17, "polyps" and "ulcerative colitis" are enabled as discrimination targets, and "NICE Type 1", "NICE Type 2", and "NICE Type 3" are enabled for polyps, and ulcerative colitis. For sexual colitis, "Mayo0", "Mayo1", and "Mayo2" are enabled. In the case of this setting, the second discrimination processing unit 52 discriminates whether or not the abnormality contained in the abnormal region 102 is a polyp and whether or not it is ulcerative colitis. Further, when the abnormality included in the abnormal region 102 is a polyp, the third discrimination processing unit 53 discriminates the degree of progress into "NICE Type 1", "NICE Type 2", or "NICE Type 3". Similarly, when the abnormality included in the abnormal region 102 is ulcerative colitis, the third discrimination processing unit 53 discriminates the degree of progression into "Mayo0", "Mayo1", or "Mayo2".

In addition, instead of setting the type of abnormality specified in the second discrimination process (discrimination target) or the classification of the degree of progress specified in the third discrimination process (discrimination content), or changing these settings, the setting unit 56 can set each spectrum or combination of the first illumination light, the second illumination light, or the third illumination light, or can change these settings. Setting the type of abnormality specified in the second discrimination process (discrimination target) or the classification of the degree of progress specified in the third discrimination process (discrimination content), and the first illumination light, the second illumination light, or Setting each spectrum or combination of the third illumination light exerts substantially the same effect on the first discrimination process, the second discrimination process, and the third discrimination process.

The display unit 13 is a display that displays the medical image acquired by the medical image acquisition unit 11. The display unit 13 displays at least a medical image for observation. In addition, the display unit 13 can display one or more of the result of the first discrimination process, the result of the second discrimination process, and the result of the third discrimination process. The monitor or display included in the device or the like to which the medical image processing device 10 is connected can be shared and used as the display unit 13 of the medical image processing device 10.

The display control unit 15 controls the display mode of the observation medical image or the like on the display unit 13. In the present embodiment, the abnormal region 102 (first abnormal region) detected by the first discrimination processing unit 51 is superimposed and displayed on the observation endoscope image 101. Further, the display control unit 15 displays on the display unit 13 the type of abnormality included in the abnormal area 102 and characters (character strings or messages) indicating the progress of the abnormality included in the abnormal area 102. Depending on the settings, the display control unit 15 may use any other method such as sound (including voice), light (partial blinking of the observation endoscope image 101, etc.), coordinate display, or the like to display an abnormal area. The position of 102, the type of abnormality included in the abnormal region 102, and the progress of the abnormality included in the abnormal region 102 can be indicated.

The input receiving unit 16 receives input from a mouse, keyboard, or other operating device connected to the medical image processing device 10. The operation of each part of the medical image processing device 10 can be controlled by using these operating devices.

The integrated control unit 17 comprehensively controls the operation of each unit of the medical image processing apparatus 10. When the input receiving unit 16 receives the operation input using the operation device, the overall control unit 17 controls each unit of the medical image processing device 10 according to the operation input.

The storage unit 18 may be used as a storage device (not shown) such as a memory included in the medical image processing device 10, a medical device such as an endoscope device 21 or a storage device (not shown) included in the PACS 22, if necessary. The position of the abnormal area 102, the type of the abnormality included in the abnormal area 102, the progress of the abnormality included in the abnormal area 102, and the like are saved.

The operation flow of the medical image processing apparatus 10 will be described below. As shown in FIG. 21, the medical image acquisition unit 11 appropriately acquires the observation endoscope image 101, which is an observation medical image (step S110), and the display control unit 15 appropriately acquires the observation endoscope image 101. Is displayed on the display unit 13 as appropriate (step S111). In the background of the process of displaying the observation endoscope image 101 on the display unit 13 in this way, the medical image acquisition unit 11 acquires the first endoscope image 61 which is the medical image for the first discrimination process (step). S112), and the first discrimination processing unit 51 executes the first discrimination processing for discriminating the presence or absence of the abnormal region 102 using the first endoscopic image 61 (step S113).

When the abnormal region 102 is detected in the first discrimination processing, the medical image acquisition unit 11 acquires the second endoscopic image 63, which is a medical image for the second discrimination processing (step S114), and the second discrimination processing unit. 52 executes a second discrimination process for discriminating the type of abnormality included in the abnormal region 102 using the second endoscopic image 63 (step S115). After that, the medical image acquisition unit 11 acquires the third endoscopic image 66, which is a medical image for the third discrimination processing (step S116), and the third discrimination processing unit 53 obtains the third endoscopic image 66. A third discrimination process for discriminating the progress of the abnormality included in the abnormal region 102 is executed (step S117).

When the first discrimination process, the second discrimination process, and the third discrimination process are executed in this way, the display control unit 15 displays these results on the display unit 13 (step S118). For example, as shown in FIG. 22, the display control unit 15 displays the position of the abnormal region 102 by showing the outline of the abnormal region 102 in the observation endoscope image 101. Further, the display control unit 15 displays that the abnormality included in the abnormal region 102 is, for example, "ulcerative colitis" in the specific display field 103 for showing the result of the second discrimination processing, and also displays the third discrimination. In the specific display field 104 for showing the result of the process, it is displayed that the progress is, for example, "Mayo1".

As described above, when the medical image processing apparatus 10 detects the abnormal region 102 and discriminates the type and progress of the abnormality contained in the detected abnormal region 102, the medical image processing apparatus 10 takes an image with illumination light suitable for each of these discriminations. You are using a medical image. Specifically, in the first discrimination process, the first endoscopic image 61 suitable for discriminating the presence or absence of the abnormal region 102 is used, and in the second discrimination process, the type of abnormality included in the abnormal region 102 is discriminated. A suitable second endoscopic image 63 is used, and in the third discrimination process, a third endoscopic image 66 suitable for discriminating the degree of progress is used. Therefore, the medical image processing apparatus 10 can provide accurate information about the position of the abnormal region 102, the type of the abnormality included in the abnormal region 102, and the progress of the abnormality included in the abnormal region 102. That is, the medical image processing apparatus 10 can provide more detailed information about lesions and the like than before, and can more effectively support diagnosis and the like.

The first discrimination processing unit 51, the second discrimination processing unit 52, and the third discrimination processing unit 53 in the first embodiment use a so-called AI (Artificial Intelligence) program learned by machine learning or deep learning. Can be configured.

When the types of abnormalities are narrowed down to a plurality of types in the second discrimination process, the medical image acquisition unit 11 obtains a third endoscopic image 66 suitable for the third discrimination process for each type of abnormality. The third discrimination processing unit 53 discriminates the progress of each type of abnormality. For example, when it is specified in the second discrimination process that "the probability of having ulcerative colitis is 40% and the probability of having Crohn's disease is 40%", the medical image acquisition unit 11 distinguishes the degree of progression of ulcerative colitis. The necessary third endoscopic image 66 and the necessary third endoscopic image 66 for differentiating the degree of progression of Crohn's disease are acquired, and the third discrimination processing unit 53 obtains each third endoscopic image 66. Using, the degree of progression of ulcerative colitis (the degree of progression when the type of abnormality is ulcerative colitis) and the degree of progression of Crohn's disease (the degree of progression when the type of abnormality is Crohn's disease) ) And.

Further, when the types of abnormalities are narrowed down to a plurality of types in the second discrimination process and the progress of each is specified in the third discrimination process, the display control unit 15 displays at least the most accurate discrimination result. .. For example, in the second differential treatment, "the probability of having ulcerative colitis is 40% and the probability of having Crohn's disease is 40%", and in the third differential treatment, "the probability that the degree of progression of ulcerative colitis is Mayo1". When "20%" and "probability that the degree of progression of Crohn's disease is SESCD1 is 95%" is specified, the display control unit 15 displays at least the type of abnormality "Crohn's disease" and the degree of progression "SESCE1".

As shown in FIG. 23, the medical image analysis processing unit 12 includes a medical record acquisition unit 121 in addition to the first discrimination processing unit 51, the second discrimination processing unit 52, the third discrimination processing unit 53, and the setting unit 56. You may prepare. The medical record acquisition unit 121 acquires a medical record of a patient to be examined from a medical device such as an endoscope device 21 or a PACS 22 or the like. A medical record is information that records the progress of medical treatment or examination for a patient, and includes, for example, a record of the patient's name, gender and age, disease name, major symptoms, prescription or treatment content, or medical history. ..

As described above, when the medical record acquisition unit 121 is provided, the setting unit 56 uses the information contained in the medical record to specify the type of abnormality (identification target) specified in the second discrimination process or the third discrimination process. It is possible to set the classification of progress (discrimination content). For example, the setting unit 56 can include a specific lesion determined by the information contained in the medical record in the type of abnormality specified in the second discrimination process. Specifically, if there is a history of ulcerative colitis, even if ulcerative colitis is not the main lesion to be examined, the setting unit 56 distinguishes ulcerative colitis as a secondary lesion. For example, set it as a target.

As shown in FIG. 24, the medical image analysis processing unit 12 includes the discrimination result correction unit 131 in addition to the first discrimination processing unit 51, the second discrimination processing unit 52, the third discrimination processing unit 53, and the setting unit 56. May be provided. The discrimination result correction unit 131 corrects the result of the first discrimination process, the result of the second discrimination process, or the result of the third discrimination process based on the operation input using the input receiving unit 16 of a doctor or the like. By providing the discrimination result correction unit 131, it is possible to correct erroneous discrimination in the first discrimination process, the second discrimination process, or the third discrimination process. Specifically, a doctor or the like can arbitrarily modify the position, size, range, number, etc. of the abnormal region 102 detected in the first discrimination process. In addition, the type of abnormality specified in the second discrimination process can be arbitrarily reselected, or the degree of progress specified in the third discrimination process can also be arbitrarily re-differentiated by a doctor or the like.

When the discrimination result correction unit 131 corrects the result of the first discrimination processing, the second discrimination processing unit 52 executes the second discrimination processing by reflecting the result of the corrected first discrimination processing, and also performs the third discrimination. The processing unit 53 executes the third discrimination process by reflecting the result of the second discrimination process reflecting the result of the modified first discrimination process. When the discrimination result correction unit 131 corrects the result of the first discrimination process, if the second discrimination process, the third discrimination process, or both of them have already been completed, the operation input automatically or by a doctor or the like. The second discrimination processing unit 52 can redo the second discrimination processing reflecting the result of the first discrimination processing after the correction, and the third discrimination processing unit 53 can redo the second discrimination processing based on the instruction by the above. The third discrimination process can be redone by reflecting the result of the second discrimination process reflecting the result of the first discrimination process.

When the discrimination result correction unit 131 corrects the result of the second discrimination process, the third discrimination processing unit 53 executes the third discrimination process by reflecting the result of the corrected second discrimination process. When the discrimination result correction unit 131 corrects the result of the second discrimination processing, if the third discrimination processing has already been completed, the third discrimination is automatically performed or based on an instruction by a doctor or the like. The processing unit 53 can redo the third discrimination process by reflecting the result of the second discrimination process after the correction.

As shown in FIG. 25, the medical image analysis processing unit 12 includes the discrimination result notification unit 141 in addition to the first discrimination processing unit 51, the second discrimination processing unit 52, the third discrimination processing unit 53, and the setting unit 56. May be provided. The discrimination result notification unit 141 notifies the result of the first discrimination process, the result of the second discrimination process, or the result of the third discrimination process at an arbitrary timing. In the first embodiment, when the third discrimination process is completed, the result of the first discrimination process, the result of the second discrimination process, and the result of the third discrimination process are displayed on the display unit 13 to notify the user. However, for example, the discrimination result notification unit 141 notifies the result of the first discrimination process in real time after the completion of the first discrimination process, and after the completion of the second discrimination process, the result of the second discrimination process is reported in real time. Notify to. The discrimination result notification unit 141 can display each discrimination result by using the display control unit 15 and the display unit 13 in the same manner as in the first embodiment, and also has sound (including voice) and light (observation endoscope image 101). The result of the first discrimination process, the result of the second discrimination process, and the result of the third discrimination process can be notified by any other method such as partial blinking of the above), display of coordinates, and the like.

[Second Embodiment]
In the first embodiment and the modification, the medical image processing device 10 and the endoscope device 21 are separate devices, but the endoscope device 21 can include the medical image processing device 10. In this case, as in the endoscope device 510 shown in FIG. 26, each part 520 constituting the medical image processing device 10 is provided in the processor device 33. However, the display unit 13 can share the monitor 34 of the endoscope device 21. Further, the medical image acquisition unit 11 corresponds to the "endoscopic image acquisition unit" formed by the image sensor 41 and the endoscopic image generation unit 48. Therefore, it is sufficient to provide the processor device 33 with each unit other than the medical image acquisition unit 11 and the display unit 13. The configuration of each other part is the same as that of the first embodiment. In addition, a new endoscope device can be configured as a whole of the medical image processing device 10 of each of the above-described embodiments and other modifications, and the endoscope device 21 of FIG.

The endoscope device 510 including the medical image processing device 10 is basically a device for observing a subject in real time. Therefore, the endoscope device 510 performs the acquisition of the endoscopic image which is a medical image, the first discrimination processing, the second discrimination processing, the third discrimination processing, the display processing of these results, and the like with the endoscope. It can be executed in real time while taking an image, or at any timing caused by the operation of various operation units.

Hereinafter, the flow of operations when the endoscope device 510 including the medical image processing device 10 detects a lesion or the like and discriminates the degree of progression in real time will be described. As shown in FIG. 27, the endoscope device 510 photographs the subject using the observation illumination light, appropriately acquires the observation endoscope image 101 (step S210), and the display control unit 15 observes. The endoscope image 101 for use is appropriately displayed on the monitor 34 (step S211).

Further, in the background of the process of displaying the observation endoscope image 101 on the monitor 34, the general control unit 17 receives the observation endoscope image 101 during a series of shooting of the observation endoscope image 101. The shooting for the first discrimination process is interrupted intermittently at a predetermined timing that does not interfere with the continuous display. That is, the endoscope device 510 appropriately switches the illumination light to the identification illumination light of the pattern 1 in accordance with the imaging timing of the image sensor 41 (step S212), and is between frames for capturing the observation endoscope image 101. A frame for photographing the subject using the identification illumination light of the pattern 1 is appropriately inserted. As a result, the endoscope device 510 acquires the first endoscope image 61 for the first discrimination processing (step S213), and the first discrimination processing unit 51 uses the first endoscope image 61. The first discrimination process is executed (step S214).

When the abnormal region 102 is detected in the first discrimination process, the integrated control unit 17 determines that the continuous display of the observation endoscope image 101 is not hindered during the shooting of the series of observation endoscope images 101. At the timing, the shooting for the second discrimination process is interrupted intermittently. That is, the endoscope device 510 appropriately switches the illumination light to the identification illumination light of the pattern 2 in accordance with the imaging timing of the image sensor 41 (step S215), and is between frames for capturing the observation endoscope image 101. A frame for photographing the subject using the identification illumination light of the pattern 2 is appropriately inserted. As a result, the endoscope device 510 acquires the second endoscope image 63 for the second discrimination processing (step S216), and the second discrimination processing unit 52 uses the second endoscope image 63. The second discrimination process is executed (step S217).

After that, the overall control unit 17 intermittently performs the third discrimination at a predetermined timing that does not interfere with the continuous display of the observation endoscope image 101 during the shooting of the series of observation endoscope images 101. Interrupt the shooting for processing. That is, the endoscope device 510 appropriately switches to the identification illumination light of a specific pattern having a spectrum suitable for the third discrimination process according to the result of the second discrimination process (step S218). At this time, the third discrimination processing unit 53 selects the spectrum of the illumination light to be used when acquiring the third medical image using the result of the second discrimination processing. As a result of switching the illumination light, a frame for photographing the subject using the illumination light for the third discrimination processing is appropriately inserted between the frames for photographing the observation endoscope image 101. For example, if the abnormality included in the abnormal region 102 is "polyp" as a result of the second discrimination processing, a frame for photographing the subject is inserted using the identification illumination light of pattern 6, and the abnormality included in the abnormal region 102 is "ulcer". In the case of "sexual colitis", a frame for photographing the subject is inserted using the identification illumination light of pattern 3, and if the abnormality included in the abnormal region 102 is "Clone disease", the identification illumination light of pattern 4 is used. Insert a frame to shoot the subject using. As a result, the endoscope device 510 acquires the third endoscope image 66 for the third discrimination processing (step S219), and the third discrimination processing unit 53 uses the third endoscope image 66. The third discrimination process is executed (step S220). When the third discrimination process is completed, the display control unit 15 displays at least the result of the third discrimination process on the display unit 13 (step S221).

As described above, the endoscope device 510 including the medical image processing device 10 may perform the first discrimination process, the second discrimination process, and the third discrimination process in real time to support diagnosis and the like in real time. it can.

As shown in FIG. 28, the endoscope device 510 including the medical image processing device 10 further includes a zoom mechanism 531, a zoom control unit 541, a light amount control unit 542, a motion detection unit 543, an imaging control unit 545, and the like. Is preferable.

The zoom mechanism 531 is a mechanical mechanism or an electronic mechanism that changes the photographing magnification. The mechanical mechanism is a so-called optical zoom, and the electronic mechanism is a so-called electronic zoom. The zoom control unit 541 controls the zoom mechanism 531 to set the shooting magnification. The endoscope device 510 can acquire an imaging magnification by using the zoom control unit 541.

The light amount control unit 542 is appropriate according to the observation distance (the distance between the tip of the endoscope 31 and the subject, including the actual distance between the tip of the endoscope 31 and the subject due to a change in the photographing magnification). Set the amount of illumination light. The light source control unit 47 uses the light source unit 42 to emit the observation illumination light and the identification illumination light of each pattern at the amount of light set by the light amount control unit 542.

The motion detection unit 543 uses an endoscope image such as the observation endoscope image 101 to move the subject, or the relative motion between the subject and the endoscope 31 (hereinafter, simply referred to as “movement”). Detects direction, speed, movement amount, etc.

The imaging control unit 545 controls the frequency of imaging for the first discrimination process, the frequency of imaging for the second discrimination process, or the frequency of imaging for the third discrimination process according to the observation conditions. The observation condition is a situation related to the shooting magnification, the amount of illumination light, or the movement when taking an endoscopic image. The shooting control unit 545 can acquire the shooting magnification from the zoom control unit 541, the amount of illumination light from the light amount control unit 542, and the movement from the motion detection unit 543. In this modified example, the photographing control unit 545 determines the frequency of photographing for the first discrimination processing, the photographing for the second discrimination processing, and the photographing according to the observation conditions in which the photographing magnification, the amount of illumination light, and the movement are combined. The frequency of shooting for the third discrimination process and the frequency of shooting are controlled.

As shown in FIG. 29, “observation condition X1” in which the imaging magnification is relatively small, the amount of illumination light is relatively large, and the movement is relatively large is observed with a high frequency of occurrence in a so-called screening situation. It is a condition. Therefore, the imaging control unit 545 relatively increases the frequency of imaging of the observation endoscope image 101 and the frequency of imaging for the first discrimination processing, and the second discrimination processing and the third Relatively reduce the frequency of imaging for discrimination processing. This is because screening generally aims at the overall observation of the subject or the detection of the abnormal region 102.

On the other hand, "observation condition X2", in which the shooting magnification is relatively large, the amount of illumination light is relatively small, and the movement is relatively small, occurs frequently in a situation where the subject is observed in detail for diagnosis or the like. It is a condition. Therefore, the imaging control unit 545 relatively reduces the imaging frequency of the observation endoscope image 101 and the imaging frequency for the first discrimination process, and is used for the second discrimination process and the third discrimination process. Increase the frequency of shooting relatively. In a situation where a subject is observed in detail for diagnosis, etc., it is required to quickly and accurately present the results of the second and third discrimination processes by following a slight change in the appearance of the subject. Because it is possible.

As described above, using the observation conditions, the frequency of photographing the observation endoscopic image 101, the frequency of photographing for the first discrimination processing, the frequency of photographing for the second discrimination processing, and the frequency of the third discrimination processing. By changing the shooting frequency of the above, it is possible to present the discrimination result such as the result of the third discrimination processing at an appropriate update frequency according to the shooting situation (purpose of shooting).

Note that relatively small means, for example, smaller than a predetermined threshold value, and relatively large means, for example, greater than or equal to a predetermined threshold value. Further, “relatively increasing or decreasing the frequency of photographing” means increasing or decreasing the frequency of photographing based on the frequency of specific photographing under the observation condition X1, the observation condition X2, or other observation conditions. .. In this modification, it is a mutual comparison between the frequency of imaging under the observation condition X1 and the frequency of imaging under the observation condition X2.

In the second embodiment, the endoscope device 510 includes the medical image processing device 10, but as shown in FIG. 30, the diagnostic support device 610 used in combination with the endoscope device 21 and other modalities is The medical image processing apparatus 10 of the above embodiment and other modifications can be included. Further, as shown in FIG. 31, various inspection devices such as the first inspection device 621, the second inspection device 622, ..., The Nth inspection device 623, including the endoscope device 21, and any network 626 are used. The medical work support device 630 to be connected can include the medical image processing device 10 of the above embodiment and other modifications.

In addition, the medical image processing device 10, various devices including the medical image processing device 10, and various devices or systems including the functions of the medical image processing device 10 can be used by making the following various changes and the like. ..

As the medical image, a white band light or a normal light image obtained by irradiating light of a plurality of wavelength bands as white band light can be used.

When an image obtained by irradiating light in a specific wavelength band is used as the medical image, the specific wavelength band can be narrower than the white wavelength band.

The specific wavelength band is, for example, a blue band or a green band in the visible region.

When the specific wavelength band is the blue band or the green band in the visible region, the specific wavelength band includes the wavelength band of 390 nm or more and 450 nm or less or 530 nm or more and 550 nm or less, and the light of the specific wavelength band is 390 nm or more. It is preferable to have a peak wavelength in the wavelength band of 450 nm or less or 530 nm or more and 550 nm or less.

The specific wavelength band is, for example, the red band in the visible range.

When the specific wavelength band is the red band in the visible region, the specific wavelength band includes a wavelength band of 585 nm or more and 615 nm or 610 nm or more and 730 nm or less, and the light of the specific wavelength band is 585 nm or more and 615 nm or less or 610 nm. It is preferable to have a peak wavelength in the wavelength band of 730 nm or less.

The specific wavelength band includes, for example, a wavelength band in which the absorption coefficient differs between the oxidized hemoglobin and the reduced hemoglobin, and the light in the specific wavelength band has a peak wavelength in the wavelength band in which the absorption coefficient differs between the oxidized hemoglobin and the reduced hemoglobin. Can have.

A specific wavelength band includes a wavelength band in which the absorption coefficient differs between the oxidized hemoglobin and the reduced hemoglobin, and light in the specific wavelength band has a peak wavelength in the wavelength band in which the absorption coefficient differs between the oxidized hemoglobin and the reduced hemoglobin. In this case, the specific wavelength band includes a wavelength band of 400 ± 10 nm, 440 ± 10 nm, 470 ± 10 nm, or 600 nm or more and 750 nm or less, and the light of the specific wavelength band is 400 ± 10 nm, 440 ± 10 nm, It is preferable to have a peak wavelength in a wavelength band of 470 ± 10 nm or 600 nm or more and 750 nm or less.

When the medical image is an in-vivo image of the inside of a living body, the in-vivo image can have information on fluorescence emitted by a fluorescent substance in the living body.

Further, as the fluorescence, fluorescence obtained by irradiating the living body with excitation light having a peak wavelength of 390 nm or more and 470 nm or less can be used.

When the medical image is an in-vivo image of the inside of a living body, the wavelength band of infrared light can be used as the specific wavelength band described above.

When the medical image is an in-vivo image of the inside of a living body and the wavelength band of infrared light is used as the above-mentioned specific wavelength band, the specific wavelength band is a wavelength band of 790 nm or more and 820 nm or 905 nm or more and 970 nm or less. Light in a specific wavelength band preferably has a peak wavelength in a wavelength band of 790 nm or more and 820 nm or less or 905 nm or more and 970 nm or less.

The medical image acquisition unit 11 acquires a special optical image having a signal in a specific wavelength band based on a normal light image obtained by irradiating light in a white band or light in a plurality of wavelength bands as light in the white band. It can have a special optical image acquisition unit. In this case, a special light image can be used as a medical image.

A signal in a specific wavelength band can be obtained by an operation based on RGB or CMY color information included in a normal optical image.

By calculation based on at least one of a normal light image obtained by irradiating light in a white band or light in a plurality of wavelength bands as light in a white band and a special light image obtained by irradiating light in a specific wavelength band. A feature amount image generation unit for generating a feature amount image can be provided. In this case, a feature image can be used as a medical image.

As for the endoscope device 21, a capsule endoscope can be used as the endoscope 31. In this case, the light source device 32 and a part of the processor device 33 can be mounted on the capsule endoscope.

In each of the above embodiments and modifications, the medical image acquisition unit 11, the medical image analysis processing unit 12 (each unit constituting the medical image analysis processing unit 12), the display control unit 15, the input receiving unit 16, the integrated control unit 17, and the control unit 17 The hardware structure of the processing unit that executes various processes such as the endoscope image generation unit 48 of the endoscope device 21 is various processors as shown below. For various processors, the circuit configuration is changed after manufacturing the CPU (Central Processing Unit), FPGA (Field Programmable Gate Array), which is a general-purpose processor that executes software (program) and functions as various processing units. It includes a programmable logic device (PLD), which is a possible processor, a dedicated electric circuit, which is a processor having a circuit configuration specially designed for executing various processes, and the like.

One processing unit may be composed of one of these various processors, or may be composed of a combination of two or more processors of the same type or different types (for example, a plurality of FPGAs or a combination of a CPU and an FPGA). May be done. Further, a plurality of processing units may be configured by one processor. As an example of configuring a plurality of processing units with one processor, first, as represented by a computer such as a client or a server, one processor is configured by a combination of one or more CPUs and software. There is a form in which this processor functions as a plurality of processing units. Secondly, as typified by System On Chip (SoC), there is a form in which a processor that realizes the functions of the entire system including a plurality of processing units with one IC (Integrated Circuit) chip is used. is there. As described above, the various processing units are configured by using one or more of the above-mentioned various processors as a hardware-like structure.

Further, the hardware structure of these various processors is, more specifically, an electric circuit (circuitry) in which circuit elements such as semiconductor elements are combined.

10 Medical image processing device 11 Medical image acquisition unit 12 Medical image analysis processing unit 13 Display unit 15 Display control unit 16 Input / reception unit 17 General control unit 18 Storage unit 21,510 Endoscope device 22 PACS
31 Endoscope 32 Light source device 33 Processor device 34 Monitor 41 Image sensor 42 Light source unit 43 V-LED
44 B-LED
45 G-LED
46 R-LED
47 Light source control unit 48 Endoscopic image generation unit 51 1st discrimination processing unit 52 2nd discrimination processing unit 53 3rd discrimination processing unit 56 Setting unit 61 1st endoscopic image (1st medical image)
63 Second endoscopic image (second medical image)
66 Third endoscopic image (third medical image)
71 1st type discrimination part 72 2nd type discrimination part 73 3rd type discrimination part 81 1st progress discrimination part 82 2nd progress discrimination part 83 3rd progress discrimination part 91 Check box 101 Observation endoscopic image 102 Abnormal area 103 Display column showing the result of the second discrimination process 104 Display column showing the result of the third discrimination process 121 Medical record acquisition section 131 Discrimination result correction section 141 Discrimination result notification section 520 Each section constituting the medical image processing device 531 Zoom mechanism 541 Zoom control unit 542 Light amount control unit 543 Motion detection unit 545 Imaging control unit 610 Diagnosis support device 621 1st inspection device 622 2nd inspection device 623 Nth inspection device 626 Network 630 Medical business support device S110-S118, S210-S221 Operation Steps X1, X2 Observation conditions

Claims (15)

A medical image acquisition unit that acquires medical images including subject images,
A first discrimination process for detecting one or a plurality of types of first abnormal regions using a first medical image taken using a first illumination light having a specific spectrum among the medical images. Department and
Among the medical images, the second medical image taken by using the second illumination light having a spectrum different from that of the first illumination light is used to identify the type of abnormality included in each of the first abnormality regions. 2 The second discrimination processing unit that performs discrimination processing and
Among the medical images, the progress of the abnormality included in the first abnormal region is used by using the third medical image taken by using the first illumination light and the third illumination light having a spectrum different from that of the second illumination light. The third discrimination processing unit that performs the third discrimination processing to specify the degree, and
A medical image processing device comprising. The second discrimination processing unit performs the second discrimination processing on the second abnormality region included in the second medical image corresponding to the first abnormality region, so that the abnormality included in the first abnormality region can be detected. The medical image processing apparatus according to claim 1, which specifies the type. The third discrimination processing unit performs the third discrimination processing on the third abnormal region included in the third medical image corresponding to at least one of the first abnormal region or the second abnormal region. The medical image processing apparatus according to claim 2, wherein the degree of progress of the abnormality included in the first abnormal region is specified. The medical image processing apparatus according to any one of claims 1 to 3, wherein the third discrimination processing unit selects the third medical image using the result of the second discrimination processing. The medical image processing apparatus according to claim 4, wherein the third discrimination processing unit selects the third medical image acquired by the spectrum of the third illumination light associated with the result of the second discrimination processing. The medical image acquisition unit acquires a medical image for observation in addition to the first medical image, the second medical image, and the third medical image.
While the observation medical image is displayed on the display unit, the first discrimination processing unit performs the first discrimination processing, the second discrimination processing unit performs the second discrimination processing, and the first discrimination processing unit performs the second discrimination processing. 3. The medical image processing apparatus according to any one of claims 1 to 5, wherein the discrimination processing unit performs the third discrimination processing. The medical image acquisition unit sets any one of the first medical image, the second medical image, and the third medical image as the observation medical image, and sets the medical image for observation.
While the observation medical image is displayed on the display unit, the first discrimination processing unit performs the first discrimination processing, the second discrimination processing unit performs the second discrimination processing, and the first discrimination processing unit performs the second discrimination processing. 3. The medical image processing apparatus according to any one of claims 1 to 5, wherein the discrimination processing unit performs the third discrimination processing. The medical image according to any one of claims 1 to 7, further comprising a setting unit for changing the type of abnormality specified in the second discrimination process or the setting of the classification of the degree of progress specified in the third discrimination process. Processing equipment. The medical image processing apparatus according to claim 8, wherein the setting unit changes the setting of each spectrum or combination of the first illumination light, the second illumination light, or the third illumination light. Equipped with a medical record acquisition department to acquire medical records
The medical image processing apparatus according to claim 8 or 9, wherein the setting unit makes the setting using the information contained in the medical record. The medical image processing apparatus according to claim 10 , wherein the setting unit includes a specific lesion determined by the information contained in the medical record in the type of abnormality specified in the second discrimination process. The medical image processing according to any one of claims 1 to 11, further comprising a discrimination result correction unit that corrects the result of the first discrimination process, the result of the second discrimination process, or the result of the third discrimination process. apparatus. The medical image processing according to any one of claims 1 to 12, further comprising a discrimination result notification unit that notifies the result of the first discrimination process, the result of the second discrimination process, or the result of the third discrimination process. apparatus. An endoscopic image acquisition unit that acquires an endoscopic image including a subject image,
Among the endoscopic images, the first discrimination process for detecting one or a plurality of types of first abnormal regions is performed using the first endoscopic image taken by using the first illumination light having a specific spectrum. 1st discrimination processing department and
Among the endoscopic images, the type of abnormality included in each of the first abnormal regions using the second endoscopic image taken by using the second illumination light having a spectrum different from that of the first illumination light. The second discrimination processing unit that performs the second discrimination processing to identify
Among the endoscopic images, an abnormality included in the first abnormal region is used by using a third medical image taken by using the first illumination light and the third illumination light having a spectrum different from that of the second illumination light. The third discrimination processing unit that identifies the progress of
Endoscope device equipped with. The endoscope device according to claim 14, wherein the third discrimination processing unit selects a spectrum of illumination light to be used when acquiring the third medical image using the result of the second discrimination processing.

Images