JP6036790B2 - Diagnostic device, image processing method in the diagnostic device, and program thereof - Google Patents

Description

  The present invention relates to a diagnostic apparatus, an image processing method in the diagnostic apparatus, and a program thereof.

  Visual inspection is always performed as a diagnosis of skin lesions, and a lot of information can be obtained. However, even with the naked eye or loupe alone, it is difficult to distinguish between moles and spots, and it is also difficult to distinguish benign and malignant tumors. Therefore, dermoscopy diagnosis is performed in which lesions are imaged using a camera with a dermoscope.

  A dermoscope is a non-invasive diagnostic instrument that brightly illuminates a lesion with a halogen lamp or the like, and makes an echo gel, a polarizing filter or the like free from reflected light, and magnifies and observes about 10 times. The observation method using this instrument is called dermoscopy. The dermoscopy diagnosis is described in detail in the Internet URL (http://www.twmu.ac.jp/DNH/department/dermatology/dermoscopy.html) <browse September 1, 2014>. According to the dermoscopy diagnosis, the pigment distribution from the epidermis to the superficial dermis can be seen well by eliminating the irregular reflection due to the stratum corneum.

  For example, Patent Document 1 discloses a technique of a remote diagnosis system for a pigmentation site that performs diagnosis using skin tone images taken with the above-described dermoscope using values such as color tone, texture, asymmetry, and circularity. ing. It uses a camera phone with a dermoscope to photograph skin with benign pigmented nevus, which is a concern for melanoma, through the dermoscope. Then, the mobile phone is connected to the Internet using the network connection function, and the photographed skin image is transmitted to the remote diagnosis device to request diagnosis. The remote diagnostic device that has received the skin image uses the melanoma diagnostic program to diagnose from the skin image whether or not it is a melanoma, and if it is a melanoma, the stage of melanoma, and as a result Reply to the doctor.

JP-A-2005-192944

  For dermatoses, diagnosis using the above-mentioned dermoscope image is spreading, but it is often impossible to obtain clear shape changes and patterns, and image observation and lesion determination depend on the skill level of the doctor in charge. Is the current situation. Therefore, the emergence of tools that can be easily and accurately diagnosed by image processing, such as emphasizing a lesion part of a dermoscope image, has been expected.

  The present invention has been made to solve the above-described problems, and provides a diagnostic device, an image processing method in the diagnostic device, and a program thereof that facilitate diagnosis by a doctor and improve diagnostic accuracy. Objective.

In order to solve the above-described problem, the present invention provides a diagnostic apparatus for diagnosing a lesion using a captured image of an affected area, the image storage unit storing the captured image, and the imaging stored in the image storage unit A processing unit for processing an image, wherein the processing unit separates the captured image into a luminance component and a color information component, and separates the luminance component into a skeleton component and a detailed component. A luminance component is restored from the second separation unit, an enhancement unit that performs enhancement processing on the detailed component, the skeleton component and the enhanced detailed component, and an enhanced image is generated using the color information component Generating means for enhancing the detailed component in accordance with the likelihood of the region to be diagnosed.

  According to the present invention, it is possible to provide a diagnostic apparatus, an image processing method in the diagnostic apparatus, and a program thereof that facilitate diagnosis of a doctor and improve diagnostic accuracy.

It is a block diagram which shows the structure of the diagnostic apparatus which concerns on embodiment of this invention. It is a flowchart which shows the basic processing operation | movement of the diagnostic apparatus which concerns on embodiment of this invention. 3 is a flowchart showing details of a captured image enhancement processing operation of FIG. 2. It is a flowchart which shows the detail of the blood vessel likelihood extraction process operation | movement of FIG. It is a figure which shows an example of the display screen structure of the diagnostic apparatus which concerns on embodiment of this invention.

  DESCRIPTION OF EMBODIMENTS Hereinafter, a mode for carrying out the present invention (hereinafter referred to as this embodiment) will be described in detail with reference to the accompanying drawings. Note that the same numbers or symbols are assigned to the same elements throughout the description of the embodiment.

(Configuration of the embodiment)
FIG. 1 is a block diagram illustrating a configuration of a diagnostic apparatus 100 according to the present embodiment. As shown in FIG. 1, an imaging device 110 with a dermoscope is connected to the diagnostic device 100 according to the present embodiment. The imaging device 110 with a dermoscope performs imaging in accordance with an instruction from the diagnostic device 100 (processing unit 101), and stores the captured image (dermoscopy image) in the image storage unit 102 and displays it on the display device 120. In addition, the captured image is emphasized by the processing unit 101 and stored in the image storage unit 102 and displayed on the display device 120. The input device 130 performs a dermoscope image capturing start instruction, a part selection operation in a dermoscopy image, which will be described later, and the like.

  Note that the display device 120 is constituted by, for example, an LCD (Liquid Crystal Display) monitor, and the input device 130 is constituted by a mouse or the like.

  The processing unit 101 processes a captured image stored in the image storage unit 102. As illustrated in FIG. 1, the processing unit 101 generates a first separation unit 101a, a second separation unit 101b, an enhancement unit 101c, and a generation unit. Means 101d.

  The first separation unit 101a functions as a unit that separates a captured image into a luminance component and a color information component. The second separation unit 101b functions as a unit that separates the luminance component into a skeleton component and a detailed component.

  The emphasizing unit 101c functions as a unit that performs emphasis processing on the detailed component, and emphasizes the detailed component in accordance with the likelihood of the site to be diagnosed. At this time, the enhancement unit 101c obtains a color information component corresponding to the red hue direction of the first color space (CIELab color space), normalizes a predetermined range of the color information component, and obtains it by normalization. A detailed image that is emphasized may be generated by reflecting the likelihood indicating the likelihood of a given portion in the enhancement coefficient of the detailed component.

  The generation unit 101d restores the luminance by adding the skeleton component separated by the second separation unit and the detailed component enhanced by the enhancement unit 101c, and restores the restored luminance and the first color space (CIELab). It functions as a means for generating an enhanced image by converting the color information component corresponding to the red hue direction and the blue hue direction of the (color space) into the second color space (RGB color space).

  In each of the first separation unit 101a, the second separation unit 101b, the enhancement unit 101c, and the generation unit 101d described above, the processing unit 101 sequentially reads and executes the program according to the present embodiment included in the processing unit 101. As a result, the functions described above are realized.

(Operation of the embodiment)
Hereinafter, the operation of the diagnostic apparatus 100 according to the present embodiment shown in FIG. 1 will be described in detail with reference to FIG.

  FIG. 2 shows a flow of basic processing operations of the first embodiment. According to FIG. 2, the processing unit 101 first acquires a captured image of an affected part (skin lesion site) captured by the imaging device 110 with a dermoscope (step S <b> 11). The acquired captured image is stored in a predetermined area of the image storage unit 102 and displayed on the display device 120 (step S12). Subsequently, the processing unit 101 performs enhancement processing on the captured image (step S13), displays the processed image along with the previously displayed captured image on the display device 120, and leaves it to the doctor's diagnosis (step S14). .

  FIG. 5 shows an example of a display screen image displayed on the display device 120. A captured image display area 121 in which a captured image is displayed on the left side of the screen is assigned, and an enhanced image display area 122 in which an enhanced image of a blood vessel is displayed on the right side, for example. When the doctor operates (clicks) the input device 130 on the “start imaging” button 123 assigned to the lower right of the drawing, imaging of the affected area by the imaging device 110 with a dermoscope is started. Then, by the processing of the processing unit 101 described above, the captured image and, for example, an emphasized image of a blood vessel part among the captured images are displayed side by side in the display areas 121 and 122 of the display device 120.

  FIG. 3 shows a detailed processing procedure of “image enhancement processing” in step S13 of FIG. Here, an example is shown in which the processing unit 101 separates a skeleton component and a detailed component from a captured image of an affected part (skin lesion site) captured by the imaging device 110 with a dermoscope, and performs different enhancement processing on each. Yes.

  The processing unit 101 first performs color space conversion. The processing unit 101 uses the CIELAB color space (hereinafter simply referred to as the Lab color space, but to be precise, the CIE 1976 L) for the RGB color space captured image obtained by the first separation unit 101a from the imaging device 110 with a dermoscope. * A * b * color space) (step S131). For the Lab color space, for example, an Internet URL (http://Ja.wikipedia.org/wiki/Lab%E8%89%B2%E7%A9%BA%E9%96%93)<September 2014 The details are described in <Browsing Date>.

  Next, in the processing unit 101, the second separation unit 101b performs edge preserving filter processing on the L image in order to separate the captured image into a skeleton component and a detailed component (step S132). Here, for example, a bilateral filter is used as the edge preserving filter. The details of the bilateral filter are described in, for example, an Internet URL (http://en.wikipedia.org/wiki/Bilateralfilter) <browse September 1, 2014>.

  Next, the processing unit 101 acquires a B image B = bilateral_filter (L) obtained by the enhancement unit 101c performing bilateral filter processing on the L image. Here, the B image is a skeleton component. Next, the enhancement unit 101c acquires a D image that is a detailed component. The D image can be acquired by D = L image−B image (step S133).

  Subsequently, the enhancement unit 101c (first enhancement unit) obtains the enhanced skeleton image B1 by raising the skeleton image B to the p factorial (step S134). At this time, p is p <= 1. At this time, the enhancement unit 101c performs processing so that the maximum value and the minimum value that the skeleton image B can take are the same before and after conversion. Specifically, since the value range of the luminance L is 0 to 100 in the Lab color space, B1 can be obtained by B1 = (B ^ p) / (100 ^ p) * 100. Next, the enhancement unit 101c multiplies B1 by K1 with the value Z as a reference to obtain a compressed image B2 (S135).

  The compressed image B2 can be obtained by B2 = (B1-Z) * K1 + Z. Here, the coefficient K1 is set to 1 or less in terms of the compression rate, and here is a value of about 0.2 to 0.8. Z is set brighter than the center C. Here, C is a center position for compression, and C = (50 ^ p) / (100 ^ p) * 100. A value obtained by increasing this by about 5% to 50% is Z. That is, the emphasis unit 101c compresses and emphasizes the skeleton component brighter than the center value.

  Next, the enhancement unit 101c performs sharpness filter processing on the compressed image B2 to obtain a sharpened image B3 (step S136: B3_ ← _sharpnessFilter (B2)). When executing the sharpness filter processing, the enhancement unit 101c performs a convolution operation (convolution) with the following kernel M on the compressed image B2. The convolution matrix (value of the convolution kernel M) shown below is an example.

| −0.1667 −0.6667 −0.1667 |
M = | −0.6667 4.3333−0.6667 |
| −0.1667 −0.6667 −0.1667 |

  The emphasis unit 101c has been described as performing the compression emphasis process and the subsequent sharpness filter process. However, it is not essential to execute both the compression emphasis process and the sharpness filter process. At least the compression emphasis process and the sharpness filter process are performed. On the other hand.

  Next, the enhancement unit 101c executes a process of extracting the likelihood of blood vessels as the likelihood A and reflecting it in the degree of enhancement of the detailed image (step S307). The blood vessel likelihood (likelihood A) has information of the same dimension as that of the compressed image B2 of the skeleton image from which noise is removed, and has blood vessel likelihood information of 0 to 1 for each pixel. It approaches 1 when the blood vessel quality increases. FIG. 5 is a flowchart showing the flow of processing for extracting the likelihood of blood vessels as the likelihood A.

  According to FIG. 4, the enhancement unit 101 c acquires the value of the a axis that is the red hue direction of the Lab color space (step S <b> 137 a), and sets the value of a from 0 for the likelihood of blood vessels (likelihood A). Normalization is performed with a restriction in the range of S, and a value range of 0 to 1 is set (steps S137b and S137c). Here, S is 80, for example. Here, the limit is given by a value from 0 to 80, but this value is an example, and the value is not limited to this value.

  The description returns to the flowchart of FIG. The enhancement unit 101c extracts the blood vessel likelihood as the likelihood A as described above (step S137), and then obtains the enhancement coefficient K3 of the detailed image D using the likelihood A (step S3138). The enhancement coefficient K3 can be obtained by K3 = A * K2. Here, the lower limit of the enhancement coefficient K3 is LM1 times the coefficient K2. Here, LM1 is in the range of 0 to 1, for example, 0.5. That is, K3 = max (K3, LM1), and max () is a function that returns the maximum value of two arguments for each element. Since LM1 is a scalar, processing is performed with the same value as the enhancement coefficient K3 expanded with the same value. The enhancement unit 101c performs enhancement processing on the detail image D using the enhancement coefficient K3, and sets the enhancement image D1 of the detail image D (step S139). That is, the emphasized image D1 of the detailed component is obtained by D1 = D * K3.

  Subsequently, the processing unit 101 converts the skeleton image B1 that has been enhanced and converted by the enhancement unit 101c and the enhanced image D1 that has been enhanced and converted (L ″ = B3 + D1). The obtained luminance image L ″ is acquired (step S140). Subsequently, a final enhanced image E is generated by converting into the RGB color space using the obtained enhanced luminance L ″ and the values of the a-axis that is the red hue component and the b-axis that is the blue hue component. That is, the generation unit 101d restores the luminance from the emphasized skeleton component and the detailed component image, and generates an emphasized image using the color information component, and the processing unit 101 displays the display device. For example, as shown in the display screen of FIG. 5, a captured image display area 121 and an emphasized image display area 122 are displayed side by side at 120.

  As described above, the enhancement unit 101c can emphasize both the skeleton component and the detailed component, and the skeleton component is compressed brightly or compressed and sharpened by a sharpness filter process. About, the emphasis process was performed according to the vascularity. When the generation unit 101d receives this and obtains the converted luminance image L ″ (step S140), it does not necessarily require both the emphasized skeleton component and the emphasized detailed component, and uses at least one of them. For example, the generation unit 101d adds the skeleton component (B image) separated by the second separation unit 101b and the detailed component (D1 image) enhanced by the enhancement unit 101c. Thus, the converted luminance image L ″ can be acquired.

(Effect of embodiment)
As described above, according to the diagnostic apparatus 100 according to the present embodiment, the processing unit 101 separates a captured image stored in the image storage unit 102 into a luminance component and a color information component, and the luminance component is a skeleton component. The emphasis unit 101c performs the emphasis process on the detailed component according to the characteristics of the region to be diagnosed, and the generation unit 101d restores the luminance from the skeleton component and the emphasized detail component. By generating the emphasized image using the color information component, for example, as shown in the display screen of FIG. 5, the captured image display area 121 and the emphasized image display area 122 can be displayed side by side. By emphasizing the detailed components according to the vascularity, the periphery of the blood vessel becomes clear without changing the overall noise sensation. Can be easily and accurately diagnosed, and as a result, the diagnostic accuracy is improved.

  In addition, according to the diagnostic apparatus 100 according to the present embodiment, the bilateral filter processing is used when the luminance component is separated into the skeleton component and the detailed component, but not limited to the bilateral filter, epsilon filter, etc. Any edge-preserving smoothing filter can be substituted. In addition, as a display example, the shooting screen and the highlight screen are displayed side by side, but the same effect can be obtained by switching and displaying on the same screen. In addition, although the Lab color space is used to obtain the luminance image, the luminance signal Y in the YUV color space expressed using the luminance signal and the two color difference signals is used regardless of the Lab color space. Good. For the YUV color space, Internet URL: http: // Ja. wikipedia. The details are disclosed in org / wiki / YUV (accessed on September 1, 2014).

  Moreover, according to the diagnostic apparatus 100 according to the present embodiment, the a-axis of the Lab color space is used as the blood vessel, but the a-axis is rotated in the plus direction of the b-axis and (a1, b1) is the center. An axis may be used. In that case, it is necessary that a1 is a value of 10 to 50, b1 is 0, and the rotation amount is about 0.3 to 0.8 radians.

  As mentioned above, although this invention was demonstrated using embodiment, it cannot be overemphasized that the technical scope of this invention is not limited to the range as described in the said embodiment. It will be apparent to those skilled in the art that various modifications or improvements can be added to the above-described embodiments. Further, it is apparent from the scope of the claims that the embodiments added with such changes or improvements can be included in the technical scope of the present invention.

  The invention described in the scope of claims attached to the application of this application will be added below. The item numbers of the claims described in the appendix are as set forth in the claims attached to the application of this application.

[Appendix]
[Claim 1]
A diagnostic device for diagnosing a lesion using a photographed image of an affected area,
An image storage unit for storing the captured image;
A processing unit that processes the captured image stored in the image storage unit,
The processing unit is
First separation means for separating the captured image into a luminance component and a color information component;
Second separation means for separating the luminance component into a skeleton component and a detailed component;
An emphasizing means for emphasizing the detailed components;
Generating means for restoring luminance from the skeleton component and the emphasized detailed component, and generating an enhanced image using the color information component;
The highlighting means is
The diagnostic apparatus characterized in that the detailed component is emphasized in accordance with the likelihood of a site to be diagnosed.
[Claim 2]
The highlighting means is
Obtaining the color information component corresponding to the red hue direction of the first color space, normalizing a predetermined range of the color information component, and a likelihood indicating the likelihood of the part obtained by the normalization The diagnostic apparatus according to claim 1, wherein the detailed image enhanced by reflecting the enhancement coefficient of the detailed component is generated.
[Claim 3]
The generating means is
The luminance information is restored by adding the skeleton component and the emphasized detailed component, and the color information corresponding to the restored luminance and the red hue direction and the blue hue direction of the first color space The diagnostic apparatus according to claim 1, wherein the emphasized image is generated by converting the component into a second color space.
[Claim 4]
4. The diagnostic apparatus according to claim 2, wherein the first color space is a CIELab color space.
[Claim 5]
4. The diagnostic apparatus according to claim 3, wherein the second color space is an RGB color space.
[Claim 6]
An image processing method in a diagnostic apparatus for diagnosing a lesion using a photographed image of an affected area,
A first separation step of separating the stored captured image into a luminance component and a color information component;
A second separation step of separating the luminance component into a skeleton component and a detailed component;
An emphasizing step of emphasizing the detailed component;
Generating a luminance image from the skeletal component and the enhanced detail component, and generating an enhanced image using the color information component, and
The emphasis step comprises:
An image processing method comprising: emphasizing the detailed component according to the likelihood of the site to be diagnosed.
[Claim 7]
An image processing program in a diagnostic apparatus for diagnosing a lesion using a photographed image of an affected area,
On the computer,
A first separation function for separating a stored captured image into a luminance component and a color information component;
A second separation function for separating the luminance component into a skeleton component and a detailed component;
An emphasis function for emphasizing the detailed components;
Generating a luminance image from the skeletal component and the enhanced detail component, and generating a highlighted image using the color information component;
The emphasis function is
A program characterized by emphasizing the detailed component according to the likelihood of the site to be diagnosed.

  DESCRIPTION OF SYMBOLS 100 ... Diagnostic apparatus 101 ... Processing part 101a ... 1st separation means 101b ... 2nd separation means 101c ... Emphasis means 101d ... Extraction means 111e ... Designation means 110 ... Imaging device with a dermoscope, 120 ... Display device 121... Captured image display area 122. Highlighted image display area 130 130 Input device

Claims (8)

A diagnostic device for diagnosing a lesion using a photographed image of an affected area,
An image storage unit for storing the captured image;
A processing unit that processes the captured image stored in the image storage unit,
The processing unit is
First separation means for separating the captured image into a luminance component and a color information component;
Second separation means for separating the luminance component into a skeleton component and a detailed component;
An emphasizing means for emphasizing the detailed components;
Generating means for reconstructing a luminance component from the skeleton component and the enhanced detailed component, and generating an enhanced image using the color information component;
The highlighting means is
The diagnostic apparatus characterized in that the detailed component is emphasized in accordance with the likelihood of a site to be diagnosed. The highlighting means is
Obtaining the color information component corresponding to the red hue direction of the first color space, normalizing a predetermined range of the color information component, and a likelihood indicating the likelihood of the part obtained by the normalization The diagnostic apparatus according to claim 1, wherein the detailed image enhanced by reflecting the enhancement coefficient of the detailed component is generated. The diagnostic apparatus according to claim 2, wherein the likelihood indicating the likelihood of the part obtained by the normalization indicates the likelihood of a blood vessel. The generating means is
The luminance information is restored by adding the skeleton component and the emphasized detailed component, and the color information corresponding to the restored luminance and the red hue direction and the blue hue direction of the first color space The diagnostic apparatus according to claim 1, wherein the emphasized image is generated by converting the component into a second color space. The diagnostic apparatus according to claim 4 , wherein the first color space is a CIELab color space. The diagnostic apparatus according to claim 4 , wherein the second color space is an RGB color space. An image processing method in a diagnostic apparatus for diagnosing a lesion using a photographed image of an affected area,
A first separation step of separating the stored captured image into a luminance component and a color information component;
A second separation step of separating the luminance component into a skeleton component and a detailed component;
An emphasizing step of emphasizing the detailed component;
Generating a luminance component from the skeleton component and the emphasized detail component, and generating an enhanced image using the color information component, and
The diagnostic device comprises:
An image processing method in a diagnostic apparatus , wherein in the enhancement step , the detailed component is emphasized in accordance with the quality of a part to be diagnosed. An image processing program in a diagnostic apparatus for diagnosing a lesion using a photographed image of an affected area,
On the computer,
A first separation function for separating a stored captured image into a luminance component and a color information component;
A second separation function for separating the luminance component into a skeleton component and a detailed component;
An emphasis function for emphasizing the detailed components;
A luminance component is restored from the skeleton component and the emphasized detail component, and a generation function that generates an enhanced image using the color information component is executed, and
The emphasis function is
A program characterized by emphasizing the detailed component according to the likelihood of the site to be diagnosed.

Images