JP6459410B2 - Diagnostic device, image processing method in the diagnostic device, and program thereof - Google Patents

Description

  The present invention relates to a diagnostic apparatus, an image processing method in the diagnostic apparatus, and a program thereof.

  Visual inspection is always performed as a diagnosis of skin lesions, and a lot of information can be obtained. However, even with the naked eye or loupe alone, it is difficult to distinguish between moles and spots, and it is also difficult to distinguish benign and malignant tumors. Therefore, dermoscopy diagnosis is performed in which lesions are imaged using a camera with a dermoscope.

  A dermoscope is a non-invasive diagnostic instrument that brightly illuminates a lesion with a halogen lamp or the like, and makes an echo gel, a polarizing filter or the like free from reflected light, and magnifies and observes about 10 times. The observation method using this instrument is called dermoscopy. The dermoscopy diagnosis is described in detail in the Internet URL (http://www.twmu.ac.jp/DNH/department/dermatology/dermoscopy.html) <browse September 1, 2014>. According to the dermoscopy diagnosis, the pigment distribution from the epidermis to the superficial dermis can be seen well by eliminating the irregular reflection due to the stratum corneum.

  For example, Patent Document 1 discloses a technique of a remote diagnosis system for a pigmentation site that performs diagnosis using skin tone images taken with the above-described dermoscope using values such as color tone, texture, asymmetry, and circularity. ing. It uses a camera phone with a dermoscope to photograph skin with benign pigmented nevus, which is a concern for melanoma, through the dermoscope. Then, the mobile phone is connected to the Internet using the network connection function, and the photographed skin image is transmitted to the remote diagnosis device to request diagnosis. The remote diagnostic device that has received the skin image uses the melanoma diagnostic program to diagnose from the skin image whether or not it is a melanoma, and if it is a melanoma, the stage of melanoma, and as a result Reply to the doctor.

JP-A-2005-192944

  For dermatoses, diagnosis using the above-mentioned dermoscope image is spreading, but it is often impossible to obtain clear shape changes and patterns, and image observation and lesion determination depend on the skill level of the doctor. Currently. Therefore, the emergence of tools that can be easily and accurately diagnosed by image processing, such as emphasizing a lesion part of a dermoscope image, has been expected.

  The present invention has been made to solve the above-described problems, and provides a diagnostic device, an image processing method in the diagnostic device, and a program thereof that facilitate diagnosis by a doctor and improve diagnostic accuracy. Objective.

One aspect of the present invention, with reference to the affected area of the Damosukopu image by Damosukopu with imaging apparatus, a diagnostic apparatus for improving the diagnostic accuracy as well as facilitating the diagnosis of disease by the doctor, captured by the imaging device A processing unit that processes a captured image that is a dermoscope image, and the processing unit includes an extraction unit that extracts a region selected as a diagnosis target, and the extraction unit indicates the likelihood of the region. The plane coordinates composed of the red hue direction and the blue hue direction of the color space composed of the luminance component and the color information component of the color system are reflected with respect to a specific point on the red hue direction axis. Rotate a predetermined angle clockwise, limit the luminance component in a specific range, calculate the likelihood indicating the likelihood of the part, and apply the calculated likelihood to the luminance component image. A diagnostic device, characterized in that by multiplying the luminance image obtained by performing Muhatto process of extracting highlights the selected site.

  According to the present invention, it is possible to provide a diagnostic apparatus, an image processing method in the diagnostic apparatus, and a program thereof that facilitate diagnosis of a doctor and improve diagnostic accuracy.

It is a block diagram which shows the structure of the diagnostic apparatus which concerns on embodiment of this invention. It is a flowchart which shows the basic processing operation | movement of the diagnostic apparatus which concerns on embodiment of this invention. 3 is a flowchart illustrating a blood vessel extraction processing operation of the captured image of FIG. 2. It is a flowchart which shows the processing operation which obtains a brightness | luminance emphasis image by the bottom hat process from L image of FIG. It is a figure which shows an example of the expansion process image of FIG. FIG. 4 is a flowchart illustrating an example of a processing operation for extracting the likelihood of blood vessels in FIG. 3 as likelihood A. FIG. 10 is a flowchart showing another example of the processing operation for extracting the likelihood of blood vessels in FIG. It is a figure which shows an example of the display screen structure of the diagnostic apparatus which concerns on embodiment of this invention.

  DESCRIPTION OF EMBODIMENTS Hereinafter, embodiments for carrying out the present invention (hereinafter referred to as “embodiments”) will be described in detail with reference to the accompanying drawings. Note that the same number is assigned to the same element throughout the description of the embodiment.

(Configuration of the embodiment)
FIG. 1 is a block diagram illustrating a configuration of a diagnostic apparatus 100 according to the present embodiment. As shown in FIG. 1, an imaging device 110 with a dermoscope is connected to the diagnostic device 100 according to the present embodiment. The imaging device 110 with a dermoscope performs imaging in accordance with an instruction from the diagnostic device 100 (processing unit 101), stores the captured image ( dermoscope image) in the image storage unit 102, and displays the image on the display device 120. In addition, the captured image is emphasized by the processing unit 101 and stored in the image storage unit 102 and displayed on the display device 120. The input device 130 performs an instruction to start photographing a dermoscope image, a part selection operation in the dermoscope image, which will be described later, and the like.

  Note that the display device 120 is constituted by, for example, an LCD (Liquid Crystal Display) monitor, and the input device 130 is constituted by a mouse or the like.

  The processing unit 101 processes a captured image stored in the image storage unit 102, and includes a separation unit 101a, an extraction unit 101b, and a generation unit 101c as shown in FIG.

  The separating unit 101a functions as a unit that separates the captured image into a luminance component and a color information component.

  The extraction unit 101b functions as a unit that extracts a region selected as a diagnosis target, and at least the first extraction unit 101b-1 that extracts a region candidate using the luminance component, and the likelihood of the region as a luminance component And a second extraction unit 101b-2 that is extracted by a color space composed of color information components.

  As will be described later, the first extracting unit 101b-1 uses a morphological process (here, a bottom hat process), and extracts a candidate for a region based on a luminance component. Further, as will be described later, the second extraction unit 101b-2 extracts, for example, using a color information component corresponding to the red hue direction of the CIELab color space, and extracts the red hue direction and the blue hue direction. The plane coordinates composed of the hue direction are rotated by a predetermined angle counterclockwise around a specific point in the red hue direction, and the luminance component is limited in a specific value range to indicate the likelihood of the part. Calculate the degree. Then, the selected likelihood is extracted by multiplying the calculated likelihood by a luminance image obtained by performing bottom hat processing on the luminance component image.

  The generation unit 101c functions as a unit that generates a part extraction result alone or in combination with a background image. At this time, the background image is at least one of a captured image and a gray-converted image of the captured image.

  The above-described separation unit 101a, extraction unit 101b (first extraction unit 101b-1, second extraction unit 101b-2), and generation unit 101c all sequentially execute the program according to the present embodiment that the processing unit 101 has. By executing the reading, the above functions are realized.

(Operation of the embodiment)
Hereinafter, the operation of the diagnostic apparatus 100 according to the present embodiment shown in FIG. 1 will be described in detail with reference to FIG.

  FIG. 2 shows a flow of basic processing operations of the diagnostic apparatus 100 according to the present embodiment. According to FIG. 2, the processing unit 101 first obtains a photographed image of an affected part, such as a skin lesion site, photographed by the photographing apparatus 110 with a dermoscope (step S11). The acquired captured image is stored in a predetermined area of the image storage unit 102 and displayed on the display device 120 (step S12). Subsequently, the processing unit 101 performs blood vessel extraction E processing from the photographed image (step S13), emphasizes the extracted blood vessel, displays the processed image side by side on the display device 120 together with the photographed image previously displayed, The diagnosis is entrusted to the doctor (step S14).

  FIG. 8 shows an example of a display screen image displayed on the display device 120. A captured image display area 121 is allocated to the left and an emphasized image display area 122 is allocated to the right, and a captured image and a blood vessel emphasized image are displayed in each area. When the doctor uses the input device 130 and clicks the “start imaging” button 123 assigned to the lower right of the screen of the display device 120, imaging of the affected area is started by the imaging device 110 with a dermoscope. Then, by the blood vessel extraction processing by the processing unit 101, the captured image is displayed in the captured image display area 121 of the display device 120, and the enhanced image in which the extracted blood vessels in the captured image are emphasized is displayed in the enhanced image display area 122. The

  FIG. 3 shows a detailed procedure of the “blood vessel extraction E process” in step S13 of FIG. According to FIG. 3, in the processing unit 101, first, the separation unit 101 a converts the captured image in the RGB color space into the Lab color space (more precisely, the CIE 1976 L * a * b color space) (step S <b> 131). ). Details of the Lab color space can be found on the Internet URL (http://ja.wikipedia.org/wiki/Lab%E8%89%B2%E7%A9%BA%E9%96%93) <September 1, 2014 >.

  Next, in the processing unit 101, the extraction unit 101b extracts a part selected as a diagnosis target. Specifically, the first extraction unit 101b-1 extracts the selected part candidate (blood vessel candidate) from the luminance component. Therefore, the first extraction unit 101b-1 uses the L image corresponding to the luminance in the Lab color space that has been color space converted by the separation unit 101a, and performs contrast enhancement by morphological processing (here, Bottom hat processing). An image BH is obtained (step S132). Morphology processing applies structuring elements to the input image and generates BH as an output image of the same size. Each value of the output image is compared with the corresponding pixel in the input image and neighboring pixels. Based on.

  The most basic morphological processes are expansion and contraction. Dilation adds pixels to the boundary of the object in the input image, and shrinkage removes the boundary pixels. The number of pixels added to or deleted from the object varies depending on the size and shape of the structuring element used for image processing.

  Here, a method of executing a morphological process using a bottom hat and extracting a region (blood vessel candidate) selected as a diagnosis target from luminance components will be described. The detailed procedure of the bottom hat process is shown in FIG.

  According to FIG. 4, the first extraction unit 101b-1 performs an expansion process on the L image, and obtains a processed luminance image L1 (step S132a). Details of the expansion process can be found on the Internet URL (http://www.mathworks.co.jp/jp/help/images/morphology-fundamentals-dilation-and-erosion.html) <browse September 1, 2014> Have been described.

  Next, the first extraction unit 101b-1 performs a contraction process on the brightness image L1 after the expansion process, and obtains a brightness image L2 (step S132b). Subsequently, the first extraction unit 101b-1 subtracts the L image from the luminance image L2 after the contraction process (BH = L2-L) to obtain the image BH after the bottom hat process (step S133). This is shown in FIG. 5A is an L image, FIG. 5B is an image L1 after the expansion process, and FIG. 5C is an image BH after the bottom hat process.

  Here, the expansion process will be supplemented. For example, consider a 5 × 5 pixel structuring element. The expansion processing is to perform processing for all the pixels with the maximum value within the range of the structuring element of the target pixel as the value of the target image. That is, the output target pixel value is the maximum value of all the pixels in the vicinity of the input pixel. On the other hand, in the reduction process, the minimum value within the range of the structuring element of the target pixel is set as the value of the target pixel. That is, the target pixel value is the minimum value of all the pixels in the vicinity of the input pixel. As described above, an image BH in which contrast enhancement is performed by the bottom hat process can be obtained from the L image.

  Returning to FIG. Next, in the processing unit 101, the second extraction unit 101b-2 extracts the likelihood (blood vessel likelihood) of the selected part using a color space composed of a luminance component and a color information component. For this reason, the second extraction unit 101b-2 calculates the likelihood of blood vessels as the likelihood A (step S133). An example of how to obtain the likelihood A is shown in FIG.

  According to FIG. 6, in the processing unit 101, the second extraction unit 101b-2 responds to the value of the a axis that is a color information component according to the red hue direction of the color space, and the blue hue direction. The b-axis value that is the color information component is extracted. That is, the second extraction unit 101b-2 generates LH1 by performing the following calculation from the values of the a axis and the b axis of the Lab color space (step S133b).

ad = (a−ca) * cos (r) + b * sin (r) + ca
bd = − (a−ca) * sin (r) + b * cos (r)
LH1 = exp (-((ad * ad) / sa / sa + (bd * bd)
/ Sb / sb))

  Here, ad and bd are obtained by rotating the ab plane by r radians counterclockwise around (ca, 0). Also, the value of r is set to about 0.3 to 0.8 radians. ca is set between 0 and 50. sa and sb are the reciprocal of the sensitivity in the a-axis direction and the reciprocal of the sensitivity in the b-axis direction, respectively. Here, it is set as sa> sb.

Next, the second extraction unit 101b-2 limits the obtained LH1 with the luminance L. The LH1 you 0 if the luminance L is the threshold value TH1 or more and LH2 (step S133c), the LH2 luminance L is obtained by the 0 if the threshold value TH2 or less and LH3 (step S133d). Here, the threshold value TH1 is set between 60 and 100, and the threshold value TH2 is set between 0 and 40. The LH3 obtained here is set as a likelihood A indicating the likelihood of blood vessels (step S133e).

  Returning to FIG. The second extraction unit 101b-2 extracts the likelihood of blood vessels as the likelihood A according to the above-described procedure (step S133), and then obtains each element of the image BH after the bottom hat processing and the likelihood A indicating the likelihood of blood vessels. Multiply and divide by coefficient N (step S134). Further, an image of the blood vessel extraction E emphasized by performing clipping processing at 1 is generated (step S135).

According to the example described above, the image of the blood vessel extraction E, although a multi-level image having a value of up to 0-1, since the through ball Tomuhatto processing, the boundary of the extracted blood vessel is steeper. If it is desired to obtain a steeper boundary, binarization may be performed with a desired threshold value.

  As described above, the second extraction unit 101b-2 uses the plane coordinates formed by the red hue direction of the color space and the blue hue direction as the center of a specific point on the red hue direction axis. Then, the likelihood A indicating the blood vessel likeness of the selected part is calculated by rotating a predetermined angle counterclockwise and limiting the luminance component in a specific value range. Then, the selected likelihood is emphasized by multiplying the calculated likelihood A by the luminance image obtained by performing the bottom hat process on the luminance component image.

  A modification in which the likelihood of blood vessels is extracted as likelihood A will be described with reference to the flowchart of FIG. The extraction means acquires the value of the a axis that is the red hue direction of the Lab color space (step S133x), sets S to 80 (step S133y), and sets the value of blood vessel likelihood A in the range of 0 to 80. Normalization (A ← max (min (a, S), 0) / S) is performed with a restriction, and a value range of 0 to 1 is set (step S133z). Here, the limit is given by a value from 0 to 80, but this value is an example and is not limited to this value.

  Finally, in the processing unit 101, the generation unit 101c re-synthesizes the extracted result of the part with the background image and displays the result on the display device 120. At this time, it is either a photographed image or a gray converted image of the photographed image as a background image to the doctor.

(Effect of embodiment)
As described above, according to the diagnostic apparatus 100 according to the present embodiment, the processing unit 101 causes the first extraction unit 101b-1 to extract the selected part candidate using the luminance component, and the second extraction. The means 101b-2 extracts the likelihood of the selected part by a color space composed of the luminance component and the color information component, and displays the extracted result on the display device 120. For this reason, the doctor can easily and accurately make a diagnosis by visually recognizing the screen highlighted for the site to be diagnosed, and as a result, the diagnostic accuracy is improved.

  Further, according to the diagnostic apparatus 100 according to the present embodiment, the processing unit 101 causes the first extraction unit 101b-1 to extract a blood vessel candidate using a morphological process (bottom hat process), and the second extraction unit. Since 101b-2 calculates the likelihood of blood vessels as a likelihood and extracts a blood vessel region from the likelihood A and the blood vessel candidate, it is possible to reproduce a clear shape change or pattern of the blood vessel. In the processing unit 101, the generation unit 101c re-synthesizes the extracted result of the part with the background image and displays the result on the display device 120. At this time, by providing the doctor with a captured image or a gray-converted image of the captured image as a background image, the display form according to the diagnostic purpose can be dynamically changed, and as a result, the doctor can more easily Diagnosis can be performed accurately and accurately, and the diagnostic accuracy is further improved.

  As mentioned above, although this invention was demonstrated using each embodiment, it cannot be overemphasized that the technical scope of this invention is not limited to the range as described in the said embodiment. It will be apparent to those skilled in the art that various modifications and improvements can be made to the above-described embodiments and examples. Further, it is apparent from the description of the scope of claims that embodiments with such changes or improvements can be included in the technical scope of the present invention.

  The invention described in the scope of claims attached to the application of this application will be added below. The item numbers of the claims described in the appendix are as set forth in the claims attached to the application of this application.

[Appendix]
[Claim 1]
A diagnostic device for diagnosing a lesion using a photographed image of an affected area,
An image storage unit for storing the captured image;
A processing unit that processes the captured image stored in the image storage unit,
The processing unit is
Separating means for separating the captured image into a luminance component and a color information component;
An extraction means for extracting a site to be diagnosed,
The extraction means comprises:
At least a first extraction means for extracting the candidate for the part by the luminance component, and a second extraction means for extracting the likelihood of the part by a color space composed of the luminance component and the color information component; A diagnostic device comprising one.
[Claim 2]
The diagnostic apparatus according to claim 1, wherein the first extraction unit uses morphological processing.
[Claim 3]
The diagnostic apparatus according to claim 2, wherein the morphology process is a bottom hat process.
[Claim 4]
The diagnostic apparatus according to claim 1, further comprising a generation unit configured to generate the extracted result of the part alone or in combination with a background image.
[Claim 5]
The diagnostic apparatus according to claim 4, wherein the background image is at least one of the captured image and a gray converted image of the captured image.
[Claim 6]
The diagnostic apparatus according to claim 1, wherein the second extraction unit performs extraction using a color information component corresponding to a red hue direction of a color space.
[Claim 7]
The second extracting means determines a plane coordinate composed of a red hue direction and a blue hue direction in a color space counterclockwise around a specific point on the red hue direction axis. Rotate the angle, limit the size of the specific point with the luminance component, calculate the likelihood indicating the likelihood of the part, and apply the calculated likelihood to the luminance component image in a bottom hat process The diagnostic apparatus according to claim 1, wherein the selected portion is emphasized and extracted by multiplying a luminance image obtained by applying the luminance image.
[Claim 8]
The diagnostic apparatus according to claim 6 or 7, wherein the color space is a CIELab color space.
[Claim 9]
An image processing method in a diagnostic apparatus for diagnosing a lesion using a photographed image of an affected area,
Separating the stored captured image into a luminance component and a color information component;
Extracting a region to be diagnosed by the luminance component or the color information component;
Generating the extracted result of the part alone by at least one selected from the group consisting of recombining with the captured image and recombining with the gray-converted image of the captured image,
The extraction step includes at least one of extracting the part candidates by the luminance component and extracting the likelihood of the part by a color space constituted by the luminance component and the color information component. An image processing method.
[Claim 10]
An image processing program in a diagnostic apparatus for diagnosing a lesion using a photographed image of an affected area,
On the computer,
A separation function for separating the stored captured image into a luminance component and a color information component;
An extraction function for extracting a region to be diagnosed by the luminance component or the color information component;
A generation function that generates the extracted result of the part by at least one selected from the group consisting of recombining with the captured image and recombining with the gray-converted image of the captured image; ,
The extraction function includes at least one of a function of extracting the part candidates by the luminance component and a function of extracting the likelihood of the part by a color space composed of the luminance component and the color information component. A program characterized by that.

  DESCRIPTION OF SYMBOLS 100 ... Diagnostic apparatus 101 ... Processing part 101a ... Separation means 101b ... Extraction means 101b-1 ... First extraction means 101b-2 ... Second extraction means 101c ... Generation means 110 ... Photography with dermoscope Device 120, display device 121, photographed image display area 122, highlight image display region 130, input device

Claims (11)

Using an affected part of Damosukopu image by Damosukopu with imaging apparatus, a diagnostic apparatus for improving the diagnostic accuracy as well as facilitating the diagnosis of a lesion by a doctor,
A processing unit that processes a captured image that is a dermoscope image captured by the imaging device ;
The processing unit is
Comprising an extraction means for extracting a site selected as a diagnostic target;
The extraction means comprises:
The redness direction of the red color direction of the color space composed of the luminance component and the color information component of the photographed image, and the plane coordinates composed of the blue color direction are represented by the red color direction. Rotate a predetermined angle counterclockwise around a specific point on the axis, calculate the likelihood indicating the likelihood of the part by limiting the luminance component in a specific value range, the calculated likelihood, A diagnostic apparatus, wherein the selected portion is emphasized and extracted by multiplying a luminance image obtained by performing a bottom hat process on the luminance component image. The diagnostic apparatus according to claim 1 , further comprising a generation unit configured to generate the extracted result of the part alone or in combination with a background image. The diagnostic apparatus according to claim 2 , wherein the background image is at least one of the captured image and a gray-converted image of the captured image. The diagnostic apparatus according to claim 1 , wherein the extraction unit performs extraction using a color information component corresponding to a red hue direction of a color space. Diagnostic device according to any one of claims 1 to 4, wherein the color space is a CIELab color space. The diagnostic apparatus according to claim 1 , further comprising: an image storage unit that stores the captured image; and a separation unit that separates the captured image into the luminance component and the color information component. Using an affected part of Damosukopu image by Damosukopu with imaging apparatus, an image processing method in the diagnosis apparatus for improving the diagnostic accuracy as well as facilitating the diagnosis of a lesion by a doctor,
A photographed image that is a dermoscope image photographed by the photographing device is converted into a CIELab color space, a blood vessel candidate that is a part selected as a diagnosis target is extracted from the L image, and a likelihood indicating the likelihood of a blood vessel is obtained from the Lab image. An image processing method, wherein blood vessel extraction is performed by multiplying the luminance image obtained by extraction and L-axis bottom hat processing by the likelihood. The image processing method according to claim 7 , wherein the likelihood indicating the likelihood of blood vessels is obtained by multiplying the value of the a-axis value of the Lab image by a value in a desired range. A diagnostic device for facilitating diagnosis of a lesion by a doctor and improving diagnostic accuracy using a dermoscope image of an affected part by an imaging device with a dermoscope,
A processing unit that processes a captured image that is a dermoscope image captured by the imaging device;
The processing unit is
The photographed image of the photographing device is converted into the CIELab color space, a blood vessel candidate that is a part selected as a diagnosis target is extracted from the L image, the likelihood indicating the blood vessel is extracted from the Lab image, and the bottom of the L axis A diagnostic apparatus, wherein a blood vessel is extracted by multiplying a luminance image obtained by hat processing by the likelihood. An image processing program in a diagnostic apparatus for facilitating diagnosis of a lesion by a doctor and improving diagnostic accuracy using a dermoscope image of an affected part by an imaging apparatus with a dermoscope,
On the computer,
Execute the extraction function to extract the part selected for diagnosis,
The extraction function is
The redness direction of the red color direction of the color space composed of the luminance component and the color information component of the photographed image, and the plane coordinates composed of the blue color direction are represented by the red color direction. Rotate a predetermined angle counterclockwise around a specific point on the axis, calculate the likelihood indicating the likelihood of the part by limiting the luminance component in a specific value range, the calculated likelihood, A program that emphasizes and extracts the selected portion by multiplying a luminance image obtained by performing bottom hat processing on the luminance component image . An image processing program in a diagnostic apparatus for facilitating diagnosis of a lesion by a doctor and improving diagnostic accuracy using a dermoscope image of an affected part by an imaging apparatus with a dermoscope,
On the computer,
A function of converting a captured image, which is a dermoscope image captured by the imaging apparatus, into a CIELab color space and extracting a blood vessel candidate that is a part selected as a diagnosis target from the L image, and a likelihood indicating the likelihood of a blood vessel from the Lab image And a function of performing blood vessel extraction by multiplying the luminance image obtained by L-axis bottom hat processing by the likelihood.

Images