CN113112475B - Traditional Chinese medicine ear five-organ region segmentation method and device based on machine learning - Google Patents

Abstract

The invention discloses a traditional Chinese medicine ear five-zang-organ region segmentation method and a device based on machine learning, wherein the method comprises the following steps: collecting ear images and establishing an image data set; labeling the ear image to obtain a label image; respectively expanding the ear images and the tag images; carrying out Canny operator edge detection on the expansion tag image; respectively preprocessing the extended ear image, the extended tag image and the tag edge image; generating edge images of each single region of the heart, the liver, the spleen, the lung and the kidney by utilizing the gray level images and the binary images; inputting the RGB three-channel image into a learning network to obtain a predicted image; obtaining a predicted image of each single region of the heart, the liver, the spleen, the lung and the kidney from the predicted image; respectively carrying out expansion and corrosion treatment on the predicted image to obtain a single-region rough edge prediction image; calculating a total loss function; and calculating the minimum value of the total loss function to make the learning network converge, and segmenting the ear image by using the learning network to obtain the ear five-organ region segmentation image.

Description

A method and device for segmenting ear and five internal organs regions of traditional Chinese medicine based on machine learning

technical field

The invention relates to the field of image processing, in particular to a machine learning-based method and device for segmenting ear five viscera regions in traditional Chinese medicine.

Background technique

In recent years, machine learning has widely included various fields of images and achieved superior performance, such as image recognition, object detection, image segmentation and so on. It is worth mentioning that the application of deep learning, which belongs to the category of machine learning, to medical imaging research is of great significance for assisting doctors in clinical diagnosis. Many results have been achieved in tongue diagnosis image processing in traditional Chinese medicine inspection. However, the digital assisted diagnosis of ear diagnosis is still rare, especially the digitization of ear inspection has not appeared in any form, among which image segmentation is a necessary step in clinical application. The ear area segmentation of deep learning is less, and because the boundaries of each segmented area are based on the theory of traditional Chinese medicine, the area boundary is not the inherent boundary outline of the object, which has a certain degree of ambiguity, which brings difficulty in segmentation.

Contents of the invention

The present invention aims to solve at least one of the technical problems existing in the prior art. For this reason, the present invention proposes a method and device for segmenting the ear five internal organs area based on machine learning in traditional Chinese medicine. Automatic segmentation.

In the first aspect, the embodiment of the present invention proposes a method for segmenting ear and five viscera regions in traditional Chinese medicine with the above-mentioned functions. include:

Gathering ear images; said gathering ear images includes:

Make full use of the existing public ear data sets, and select ear images that meet the requirements of TCM ear diagnosis as the original data of the data set;

The other part is to collect raw data by itself according to the requirements of TCM ear diagnosis. The self-collected raw data is that the collection device is a TCM tongue image collection instrument. ) to a suitable angle of about 30°, so that the five internal organs of the ear are not blocked during the collection process. The original collected image includes the ear, face and the edge of the collection instrument. The original collected image passes through the trained Opencv ear detector The model detects the position of the ear, and the detection box does not completely cover the ear area. Therefore, the detection box is centered and cut at 1.6 times the width and height of the detection box. In the cropped image, the complete ear The ear region accounts for about half of the entire image area, and the ear image is obtained.

Annotating the ear image to obtain a label image;

Expanding the ear image and the label image respectively to obtain the expanded ear image and the expanded label image;

Carry out Canny operator edge detection to described extended label image, obtain label edge image;

Preprocessing the expanded ear image, the expanded label image and the label edge image respectively to obtain an RGB three-channel image, a grayscale image and a binary image;

Using the grayscale image and the binary image to generate edge images of individual regions of the heart, liver, spleen, lung, and kidney;

Input the RGB three-channel image into the learning network to obtain a predicted image;

obtaining prediction images of individual regions of the heart, liver, spleen, lung, and kidney from the prediction images;

Carrying out dilation and erosion processing on the prediction images of the single regions of the heart, liver, spleen, lung and kidney respectively, to obtain a single region rough edge prediction map of the corresponding regions;

calculating a total loss function, the total loss function including a first loss function of the grayscale image and the predicted image and a second loss function of the single region edge image and the single region rough edge prediction map;

The minimum value of the total loss function is calculated to make the learning network converge, and the learning network is used to segment the ear image to obtain a segmented image of the five viscera regions of the ear.

According to the machine learning-based method for segmenting the five internal organs of the ears of traditional Chinese medicine according to the embodiments of the present invention, at least the following technical effect is achieved: the corresponding regions of the five internal organs of the ears of traditional Chinese medicine can be automatically segmented.

According to the machine learning-based method for segmenting five viscera regions of the ear in traditional Chinese medicine according to an embodiment of the present invention, the expansion of the ear image and the label image to obtain the expanded ear image and the expanded label image includes:

Rotating the ear image, changing the size of the ear image, horizontally flipping the ear image, and performing gamma transformation on the ear image to obtain the expanded ear image;

Perform the same rotation on the label image as the ear image, change the size of the label image, flip the label image horizontally, and make it still only have five pixel values corresponding to heart, liver, spleen, The five areas of lung and kidney, and the gray pixel values are 50, 100, 150, 200, 250 respectively, except the above five areas are background areas, the pixel value is 0, and the extended label image is obtained.

According to the machine learning-based method for segmenting the ear five internal organs region of traditional Chinese medicine according to an embodiment of the present invention, the ear image is marked to obtain a label image, including:

According to the Chinese standard auricular point positioning schematic diagram, the ear image is marked using the LabelMe toolbox;

Convert the labeling information of the ear images to obtain a label image corresponding to each of the ear images, in which the center, liver, spleen, lung, kidney and other areas of the label image correspond to the Chinese standard auricular point positioning schematic diagram as the background Each of the six regions is distinguished by different pixel values.

The ear image and the corresponding data set composed of the label image checked by a professional TCM physician are divided into a training set, a verification set and a test set according to a ratio of 60%, 20%, and 20%, and the deep neural network U- Net is used as a training model, and Intersection over Union (Intersection over Union) IoU is used as a technical indicator. On the one hand, during the training process, the response sequence of the intersection ratio of the five labeled regions (heart, liver, spleen, lung, and kidney) in the verification set is lung, kidney, liver, spleen, and heart; The average intersection ratio of the five labeled regions on the test set is above 0.38. The data set that meets the above two indicators is determined as the final data set.

According to the machine learning-based method for segmenting five viscera regions of ears in traditional Chinese medicine according to an embodiment of the present invention, the preprocessing of the expanded ear image and the label edge image is performed respectively, including:

Carry out position cut and normalization operation on the expanded ear image to obtain the RGB three-channel image with a size of W*H.

Cutting the extended label image corresponding to the extended ear image to obtain the grayscale image with a size of W*H, the grayscale image is divided into background, heart, liver, spleen, lung, There are six areas of the kidney, and the gray pixel values of the six areas are 0, 1, 2, 3, 4, and 5, respectively.

Cutting the label edge image corresponding to the expanded ear image to obtain the binary image with a size of W*H.

According to the machine learning-based method for segmenting the five viscera regions of the ear in traditional Chinese medicine, the grayscale image and the binary image are used to generate edge images of individual regions of the heart, liver, spleen, lung, and kidney, including:

Let f _label (x, y) represent the pixel value at any coordinate (x, y) of the grayscale image, let f _edge (x, y) represent the pixel value at any coordinate (x, y) of the binary image The pixel value of , g _i (x, y) represents the corresponding transformed pixel value at these coordinates, there is

In the formula, i∈{1,2,…,5}, x∈{1,2,…,W}, y∈{1,2,…,H}, W and H are the grayscale image and The width and height of the binary image, g ₁ , g ₂ , g ₃ , g ₄ , and g ₅ are the edge images of the single regions of the heart, liver, spleen, lung, and kidney, respectively.

According to claim 1, the traditional Chinese medical science ear five internal organs region segmentation method based on machine learning, is characterized in that, said RGB three-channel image is input into learning network, obtains predicted image, comprises:

Pass the RGB three-channel image through two convolution layers with 3*3 convolution kernels to obtain a feature F _origin with 64 channels and a size of H×W;

Pass the feature F _origin through the PAM module to obtain the feature F ₁ .

The feature F ₁ is passed through the PAM module to obtain the feature F ₂ .

The two results obtained by multiplying the feature F ₁ and the feature F ₂ by the coefficients 0.75 and 0.25 respectively are spliced on the channel, and then passed through the convolution layer with a convolution kernel of 3*3 to obtain the channel as 6. The prediction image whose size is H×W.

According to the machine learning-based method for segmenting five viscera regions of ears in traditional Chinese medicine according to an embodiment of the present invention, the predicted images of individual regions of the heart, liver, spleen, lung, and kidney are obtained from the predicted images, including:

Take the maximum value of the predicted image in the dimension of the channel to obtain a dimension grayscale image with a size of H×W and only six pixel values of 0, 1, 2, 3, 4, and 5, wherein the six types of pixels The areas corresponding to the values in the dimension grayscale image are respectively the predicted segmentation areas of the background, heart, liver, spleen, lung, and kidney;

Let P(x,y) denote the pixel value at any coordinate (x,y) of the predicted grayscale image, we have

In the formula, i∈{1,2,…,5}, x∈{1,2,…,W}, y∈{1,2,…,H}, W and H are grayscale images of the dimensions respectively width and height, P(x,y)={0,1,...,5}, p ₁ , p ₂ , p ₃ , p ₄ , p ₅ are the heart, liver, spleen, lung and kidney respectively A single region predicts an image.

According to the machine learning-based segmentation method of TCM ears and five viscera regions, the prediction images of the individual regions of the heart, liver, spleen, lung, and kidney are respectively expanded and eroded to obtain the corresponding regions. Single region rough edge prediction map, including:

Define the structure element B, the size of the structure element B is 3*3, the origin is at the center, and the element values are all 1;

Using the structural element B to expand and corrode each of the single-region predicted images of the heart, liver, spleen, lung, and kidney to obtain the expanded single-region predicted image and the corroded single-region predicted image, the Subtracting the single-region prediction image from the corroded single-region prediction image to obtain its corresponding rough edge prediction map of the region;

Let pe _i (x, y) denote the pixel value at any coordinate of the predicted image at the rough edge of the region, we have

为膨胀运算，符号

为腐蚀运算，i∈{1,2,…,5}，x∈{1,2,…,W},y∈{1,2,…,H}，W和H分别是图像的宽和高，pe_i(x,y)∈{0,1}，，pe₁、pe₂、pe₃、pe₄、pe₅分别是心、肝、脾、肺、肾的所述单区域粗略边缘预测图像。In the formula, the symbol

is the dilation operation, the symbol

For the corrosion operation, i∈{1,2,…,5}, x∈{1,2,…,W}, y∈{1,2,…,H}, W and H are the width and height of the image respectively , pe _i (x, y)∈{0,1},, pe ₁ , pe ₂ , pe ₃ , pe ₄ , pe ₅ are the single-region rough edge prediction images of the heart, liver, spleen, lung, and kidney, respectively .

According to the machine learning-based method for segmenting the five viscera regions of TCM ears, the total loss function is calculated, and the total loss function includes the first loss function of the grayscale image and the predicted image and the edge of the single region The second loss function for the image and the single-region rough edge prediction map:

The first loss function is

对于类别c经过one-hot编码的所述二值图像，

q^c(x,y)是坐标(x,y)像素属于类别c的预测概率；In the formula, x∈{1,2,…,W}, y∈{1,2,…,H}, W and H are the width and height of the grayscale image and the predicted image respectively, c∈{ 1,...,C}, C is the number of split categories,

For the binary image of category c after one-hot encoding,

q ^c (x, y) is the predicted probability that the pixel at coordinates (x, y) belongs to category c;

The second loss function is

In the formula, i∈{1,2,…,5}, x∈{1,2,…,W}, y∈{1,2,…,H}, W and H are the edge images of the single area respectively and the width and height of the single region rough edge prediction map, g _i (x, y) is the i-th region edge image, g _i (x, y)∈{0,1}, AND is the intersection operation, pe _i (x,y) is the rough edge prediction image of the i-th region, pe _i (x,y)∈{0,1}.

The total loss function is

In the formula, the parameter N is the number of l _i not 0.

In the second aspect, the embodiment of the present invention also provides a machine learning-based device for segmenting ear five viscera regions in traditional Chinese medicine, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor, the processor When the program is executed, the machine learning-based method for segmenting ear five viscera regions in traditional Chinese medicine described in the first aspect of the present invention is realized.

Additional aspects and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention.

Description of drawings

The above and/or additional aspects and advantages of the present invention will become apparent and comprehensible from the description of the embodiments in conjunction with the following drawings, wherein:

Fig. 1 is the flow chart of the method for segmenting five viscera regions of ears in traditional Chinese medicine according to an embodiment of the present invention;

Fig. 2 is the flow chart of segmentation processing of five viscera regions of ears in traditional Chinese medicine according to an embodiment of the present invention;

Fig. 3 is the Chinese standard auricular point positioning schematic diagram of the embodiment of the present invention;

Detailed ways

Embodiments of the present invention are described in detail below, examples of which are shown in the drawings, wherein the same or similar reference numerals designate the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the figures are exemplary only for explaining the present invention and should not be construed as limiting the present invention.

In the description of the present invention, it should be understood that the orientation descriptions, such as up, down, front, back, left, right, etc. indicated orientations or positional relationships are based on the orientations or positional relationships shown in the drawings, and are only In order to facilitate the description of the present invention and simplify the description, it does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus should not be construed as limiting the present invention.

In the description of the present invention, several means one or more, and multiple means more than two. Greater than, less than, exceeding, etc. are understood as not including the original number, and above, below, within, etc. are understood as including the original number. If the description of the first and second is only for the purpose of distinguishing the technical features, it cannot be understood as indicating or implying the relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features relation.

In the description of the present invention, unless otherwise clearly defined, words such as setting, installation, and connection should be understood in a broad sense, and those skilled in the art can reasonably determine the specific meanings of the above words in the present invention in combination with the specific content of the technical solution.

Referring to FIG. 1 , FIG. 2 and FIG. 3 , a method for segmenting ear five internal organs regions based on machine learning in traditional Chinese medicine according to a first embodiment of the present invention will be described. include:

S101: Collecting ear images; the collecting ear images includes:

Make full use of the existing public ear data sets, and select ear images that meet the requirements of TCM ear diagnosis as the original data of the data set;

The other part is to collect raw data by itself according to the requirements of TCM ear diagnosis. The self-collected raw data is that the collection device is a TCM tongue image collection instrument. ) to a suitable angle of about 30°, so that the five internal organs of the ear are not blocked during the collection process. The original collected image includes the ear, face and the edge of the collection instrument. The original collected image passes through the trained Opencv ear detector The model detects the position of the ear, and the detection box does not completely cover the ear area. Therefore, the detection box is centered and cut at 1.6 times the width and height of the detection box. In the cropped image, the complete ear The ear region accounts for about half of the entire image area, and the ear image is obtained.

S102: Label the ear image to obtain a label image;

S103: Expand the ear image and the label image respectively to obtain the expanded ear image and the expanded label image;

S104: Perform Canny operator edge detection on the extended label image to obtain a label edge image;

S105: Perform preprocessing on the extended ear image, the extended label image, and the label edge image, respectively, to obtain an RGB three-channel image, a grayscale image, and a binary image;

S106: Using the grayscale image and the binary image to generate edge images of individual regions of the heart, liver, spleen, lung, and kidney;

S107: Input the RGB three-channel image into the learning network to obtain a predicted image;

S108: Obtain prediction images of individual regions of the heart, liver, spleen, lung, and kidney from the prediction images;

S109: Perform dilation and erosion processing on the single-region prediction images of the heart, liver, spleen, lung, and kidney respectively, to obtain a single-region rough edge prediction image of the corresponding region;

S110: Calculate a total loss function, where the total loss function includes a first loss function of the grayscale image and the predicted image and a second loss function of the single region edge image and the single region rough edge prediction map;

S111: Calculate the minimum value of the total loss function, make the learning network converge, use the learning network to segment the ear image, and obtain a segmented image of the five viscera regions of the ear.

According to the machine learning-based method for segmenting the five internal organs of the ears of traditional Chinese medicine according to the embodiments of the present invention, at least the following technical effect is achieved: the corresponding regions of the five internal organs of the ears of traditional Chinese medicine can be automatically segmented.

Referring to Fig. 1 and Fig. 2, according to the method for segmenting five viscera regions of ears based on machine learning in traditional Chinese medicine according to the first embodiment of the present invention, the ear images are marked to obtain label images, including: pairing according to the Chinese standard auricular point positioning schematic diagram The ear image is marked using the LabelMe toolbox; the label information of the ear image is converted to obtain a label image corresponding to each of the ear images, and the label image corresponds to the center of the Chinese standard auricular point positioning schematic diagram, Liver, spleen, lung, kidney and other regions are distinguished by different pixel values of the six background regions.

The ear image and the corresponding data set composed of the label image checked by a professional TCM physician are divided into a training set, a verification set and a test set according to a ratio of 60%, 20%, and 20%, and the deep neural network U- Net is used as a training model, and Intersection over Union (Intersection over Union) IoU is used as a technical indicator. On the one hand, during the training process, the intersection and ratio response order of the five labeled regions (heart, liver, spleen, lung, and kidney) in the validation set is lung, kidney, liver, spleen, and heart. On the other hand, after the training is completed, the average intersection ratio of the five labeled regions on the test set is above 0.38. The data set that meets the above two indicators is determined as the final data set.

Referring to Fig. 1 and Fig. 2, according to the TCM ear five internal organs region segmentation method based on machine learning according to the first embodiment of the present invention, the expansion of the ear image and the label image obtains the expanded ear image and the expanded label image ,include:

Rotating the ear image, changing the size of the ear image, horizontally flipping the ear image, and gamma transforming the ear image to obtain the expanded ear image;

Perform the same rotation on the label image as the ear image, change the size of the label image, flip the label image horizontally, and make it still only have five pixel values corresponding to heart, liver, spleen, The five areas of lung and kidney, and the gray pixel values are 50, 100, 150, 200, 250 respectively, except the above five areas are background areas, the pixel value is 0, and the extended label image is obtained.

Referring to Fig. 1 and Fig. 2, according to the TCM ear five internal organs region segmentation method based on machine learning according to the first embodiment of the present invention, preprocessing is performed on the expanded ear image and the label edge image respectively, including:

Carry out position cut and normalization operation on the expanded ear image to obtain the RGB three-channel image with a size of W*H.

Cutting the extended label image corresponding to the extended ear image to obtain the grayscale image with a size of W*H, the grayscale image is divided into background, heart, liver, spleen, lung, There are six areas of the kidney, and the gray pixel values of the six areas are 0, 1, 2, 3, 4, and 5, respectively.

Cutting the label edge image corresponding to the expanded ear image to obtain the binary image with a size of W*H.

Referring to Fig. 1 and Fig. 2, according to the machine learning-based method for segmenting the five viscera regions of the ear in traditional Chinese medicine according to the first embodiment of the present invention, the grayscale image and the binary image are used to generate the heart, liver, spleen, lung, and kidney. Individual region edge images, including:

Let f _label (x, y) represent the pixel value at any coordinate (x, y) of the grayscale image, let f _edge (x, y) represent the pixel value at any coordinate (x, y) of the binary image The pixel value of , g _i (x, y) represents the corresponding transformed pixel value at these coordinates, there is

In the formula, i∈{1,2,…,5}, x∈{1,2,…,W}, y∈{1,2,…,H}, W and H are the grayscale image and The width and height of the binary image, g ₁ , g ₂ , g ₃ , g ₄ , and g ₅ are the edge images of the single regions of the heart, liver, spleen, lung, and kidney, respectively.

Referring to Fig. 1 and Fig. 2, according to the machine learning-based TCM ear five internal organs region segmentation method according to the embodiment of the present invention, each single region of the heart, liver, spleen, lung and kidney is generated by using the grayscale image and the binary image Edge images, including:

Referring to Fig. 1 and Fig. 2, according to the method for segmenting the ear five viscera region of traditional Chinese medicine based on machine learning according to the first embodiment of the present invention, it is characterized in that the RGB three-channel image is input into the learning network to obtain a predicted image, including:

Pass the RGB three-channel image through two convolution layers with 3*3 convolution kernels to obtain a feature F _origin with 64 channels and a size of H×W;

Pass the feature F _origin through the PAM module to obtain the feature F ₁ ;

Pass the feature _F1 through the PAM module to obtain the feature _F2 ;

The two results obtained by multiplying the feature F ₁ and the feature F ₂ by the coefficients 0.75 and 0.25 respectively are spliced on the channel, and then passed through the convolution layer with a convolution kernel of 3*3 to obtain the channel as 6. The prediction image whose size is H×W.

Referring to Fig. 1 and Fig. 2, according to the machine learning-based method for segmenting five viscera regions of the ear in traditional Chinese medicine according to the first embodiment of the present invention, the prediction images of individual regions of the heart, liver, spleen, lung, and kidney are obtained from the prediction images, include:

Take the maximum value of the predicted image in the dimension of the channel to obtain a dimension grayscale image with a size of H×W and only six pixel values of 0, 1, 2, 3, 4, and 5, wherein the six types of pixels The areas corresponding to the values in the dimension grayscale image are respectively the predicted segmentation areas of the background, heart, liver, spleen, lung, and kidney;

Let P(x,y) denote the pixel value at any coordinate (x,y) of the predicted grayscale image, we have

In the formula, i∈{1,2,…,5}, x∈{1,2,…,W}, y∈{1,2,…,H}, W and H are grayscale images of the dimensions respectively width and height, P(x,y)={0,1,...,5}, p ₁ , p ₂ , p ₃ , p ₄ , p ₅ are the heart, liver, spleen, lung and kidney respectively A single region predicts an image.

Referring to Fig. 1 and Fig. 2, according to the machine learning-based TCM ear five internal organs region segmentation method according to the first embodiment of the present invention, the prediction images of the single regions of the heart, liver, spleen, lung and kidney are respectively expanded, Corrosion processing to obtain a single-region rough edge prediction map of its corresponding region, including:

Define the structure element B, the size of the structure element B is 3*3, the origin is at the center, and the element values are all 1;

Using the structural element B to expand and corrode each of the single-region predicted images of the heart, liver, spleen, lung, and kidney to obtain the expanded single-region predicted image and the corroded single-region predicted image, the Subtracting the single-region prediction image from the corroded single-region prediction image to obtain its corresponding rough edge prediction map of the region;

Let pe _i (x, y) denote the pixel value at any coordinate of the predicted image at the rough edge of the region, we have

为膨胀运算，符号

为腐蚀运算，i∈{1,2,…,5}，x∈{1,2,…,W},y∈{1,2,…,H}，W和H分别是图像的宽和高，pe_i(x,y)∈{0,1}，，pe₁、pe₂、pe₃、pe₄、pe₅分别是心、肝、脾、肺、肾的所述单区域粗略边缘预测图像。In the formula, the symbol

is the dilation operation, the symbol

For the corrosion operation, i∈{1,2,…,5}, x∈{1,2,…,W}, y∈{1,2,…,H}, W and H are the width and height of the image respectively , pe _i (x, y)∈{0,1},, pe ₁ , pe ₂ , pe ₃ , pe ₄ , pe ₅ are the single-region rough edge prediction images of the heart, liver, spleen, lung, and kidney, respectively .

Referring to Fig. 1 and Fig. 2, according to the first embodiment of the present invention, the method for segmenting the five internal organs of ears based on machine learning in traditional Chinese medicine is characterized in that it includes: calculating a total loss function, the total loss function including the grayscale image and The first loss function of the predicted image and the second loss function of the single region edge image and the single region rough edge prediction map:

The first loss function is

对于类别c经过one-hot编码的所述二值图像，

q^c(x,y)是坐标(x,y)像素属于类别c的预测概率；In the formula, x∈{1,2,…,W}, y∈{1,2,…,H}, W and H are the width and height of the grayscale image and the predicted image respectively, c∈{ 1,...,C}, C is the number of split categories,

For the binary image of category c after one-hot encoding,

q ^c (x, y) is the predicted probability that the pixel at coordinates (x, y) belongs to category c;

The second loss function is

In the formula, i∈{1,2,…,5}, x∈{1,2,…,W}, y∈{1,2,…,H}, W and H are the edge images of the single area respectively and the width and height of the single region rough edge prediction map, g _i (x, y) is the i-th region edge image, g _i (x, y)∈{0,1}, AND is the intersection operation, pe _i (x,y) is the rough edge prediction image of the i-th region, pe _i (x,y)∈{0,1}.

The total loss function is

In the formula, the parameter N is the number of l _i not 0.

A device for segmenting ear five viscera regions based on machine learning, characterized in that it includes at least one processor, and a memory connected to at least one processor; wherein, the memory stores information that can be executed by the at least one processor. Instructions, the instructions are executed by the at least one processor, so that the at least one processor can perform any one of the methods for segmenting ear and five viscera regions in traditional Chinese medicine.

It should be recognized that the method steps in the embodiments of the present invention may be implemented or implemented by computer hardware, a combination of hardware and software, or by computer instructions stored in a non-transitory computer-readable memory. The methods can use standard programming techniques. Each program can be implemented in a high-level procedural or object-oriented programming language to communicate with the computer system. However, the programs can be implemented in assembly or machine language, if desired. In any case, the language may be a compiled or interpreted language. Furthermore, the program can be run on an application specific integrated circuit programmed for this purpose.

In addition, operations of processes described herein may be performed in any suitable order unless otherwise indicated herein or otherwise clearly contradicted by context. The processes described herein (or variations and/or combinations thereof) can be performed under the control of one or more computer systems configured with executable instructions, and as code that collectively executes on one or more processors (e.g. , executable instructions, one or more computer programs or one or more applications), hardware or a combination thereof. The computer program comprises a plurality of instructions executable by one or more processors.

Further, the method can be implemented in any type of computing platform operably connected to a suitable one, including but not limited to personal computer, minicomputer, main frame, workstation, network or distributed computing environment, stand-alone or integrated computer platform, or communicate with charged particle tools or other imaging devices, etc. Aspects of the invention can be implemented as machine-readable code stored on a non-transitory storage medium or device, whether removable or integrated into a computing platform, such as a hard disk, optically read and/or written storage medium, RAM, ROM, etc., such that they are readable by a programmable computer, when the storage medium or device is read by the computer, can be used to configure and operate the computer to perform the processes described herein. Additionally, the machine-readable code, or portions thereof, may be transmitted over a wired or wireless network. The invention described herein includes these and other various types of non-transitory computer-readable storage media when such media include instructions or programs that implement the steps described above in conjunction with a microprocessor or other data processor. The invention also includes the computer itself when programmed according to the methods and techniques described herein.

Computer programs can be applied to input data to perform the functions described herein, thereby transforming the input data to generate output data stored to non-volatile memory. Output information may also be applied to one or more output devices such as a display. In a preferred embodiment of the invention, the transformed data represents physical and tangible objects, including specific visual depictions of physical and tangible objects produced on a display.

The embodiments of the present invention have been described in detail above in conjunction with the accompanying drawings, but the present invention is not limited to the above embodiments. Within the scope of knowledge possessed by those of ordinary skill in the technical field, various modifications can be made without departing from the gist of the present invention. kind of change.

In the description of this specification, references to the terms "one embodiment," "some embodiments," "exemplary embodiments," "example," "specific examples," or "some examples" are intended to mean that the implementation A specific feature, structure, material, or characteristic described by an embodiment or example is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

Although the embodiments of the present invention have been shown and described, those skilled in the art can understand that various changes, modifications, substitutions and modifications can be made to these embodiments without departing from the principle and spirit of the present invention. The scope of the invention is defined by the claims and their equivalents.

Claims (9)

1. A method for segmenting the five internal organs of the ears of traditional Chinese medicine based on machine learning, characterized in that, comprising: Collect ear images and build image datasets; Annotating the ear image to obtain a label image; Expanding the ear image and the label image respectively to obtain the expanded ear image and the expanded label image; Carry out Canny operator edge detection to described extended label image, obtain label edge image; Preprocessing the expanded ear image, the expanded label image, and the label edge image respectively to obtain an RGB three-channel image, a grayscale image, and a binary image, wherein the RGB three-channel image, the Both the size of the grayscale image and the binary image are W×H; Using the grayscale image and the binary image to generate edge images of individual regions of the heart, liver, spleen, lung, and kidney; Input the RGB three-channel image into the learning network to obtain a predicted image; obtaining prediction images of individual regions of the heart, liver, spleen, lung, and kidney from the prediction images; Carrying out dilation and erosion processing on the single area prediction images of the heart, liver, spleen, lung, and kidney respectively, to obtain a single area rough edge prediction map of the corresponding area; calculating a total loss function, the total loss function including a first loss function of the grayscale image and the predicted image and a second loss function of the single region edge image and the single region rough edge prediction map, wherein, The first loss function is

In the formula, x∈{1, 2,..., W}, y∈{1, 2,..., H}, W are the widths of the grayscale image and the predicted image respectively, and H is The height of the grayscale image and the predicted image, c ∈ {1, 2, ..., X}, C is the number of segmentation categories,

For the binary image of category c after one-hot encoding

q ^c (x, y) is the predicted probability that the pixel at coordinate (x, y) belongs to class c; The second loss function is

In the formula, i∈{1,2,...,5}, x∈{1,2,...,W}, y∈{1,2,...,H}, W are respectively the The width of the single region edge image and the single region rough edge prediction map, H is the height of the single region edge image and the single region rough edge prediction map, g _i (x, y)∈{0, 1} is the edge image of the i-th area, g _i (x, y) ∈ {0, 1}, AND is an intersection operation, pe _i (x, y) is the rough edge prediction image of the i-th area, pe _i (x, y ) ∈ {0, 1}, The total loss function is

In the formula, the parameter N is the number that l _i is not 0; The minimum value of the total loss function is calculated to make the learning network converge, and the learning network is used to segment the ear image to obtain a segmented image of the five viscera regions of the ear. 2. a kind of traditional Chinese medicine ear five internal organs region segmentation method based on machine learning according to claim 1, is characterized in that, described ear image is marked, obtains label image, comprises: According to the Chinese standard auricular point positioning schematic diagram, the ear image is marked using the LabelMe toolbox; Convert the labeling information of the ear images to obtain a label image corresponding to each of the ear images, in which the center, liver, spleen, lung, kidney and other areas of the label image correspond to the Chinese standard auricular point positioning schematic diagram as the background Each of the six regions is distinguished by different pixel values. 3. a kind of traditional Chinese medical science ear five viscera region segmentation method based on machine learning according to claim 1, is characterized in that, described expanding described ear image and described label image, obtain expanded ear image and expanded label image ,include: Rotating the ear image, changing the size of the ear image, horizontally flipping the ear image, and performing gamma transformation on the ear image to obtain the expanded ear image; Perform the same rotation on the label image as the ear image, change the size of the label image, flip the label image horizontally, and make it still only have five pixel values corresponding to heart, liver, spleen, The five areas of lung and kidney, and the gray pixel values are 50, 100, 150, 200, 250 respectively, except the above five areas are background areas, the pixel value is 0, and the extended label image is obtained. 4. a kind of TCM ear five internal organs region segmentation method based on machine learning according to claim 1, is characterized in that, preprocessing is carried out to described expansion ear image and described label edge image respectively, comprises: Carrying out position cutting and normalization operation on the expanded ear image to obtain the RGB three-channel image whose size is W×H; Cutting the extended label image corresponding to the extended ear image to obtain the grayscale image with a size of W×H, the grayscale image is divided into background, heart, liver, spleen, lung, Six areas of the kidney, the gray pixel values of the six areas are 0, 1, 2, 3, 4, 5 respectively; Perform position clipping corresponding to the extended ear image on the label edge image to obtain the binary image with a size of W×H. 5. A machine learning-based method for segmenting the five viscera regions of the ears of traditional Chinese medicine according to claim 1, characterized in that, using the grayscale image and the binary image to generate the heart, liver, spleen, lung, and kidney Individual region edge images, including: Let f _label (x, y) represent the pixel value at any coordinate (x, y) of the grayscale image, let f _edge (x, y) represent the pixel value at any coordinate (x, y) of the binary image The pixel value of , g _i (x, y) represents the corresponding transformed pixel value at these coordinates, there is

In the formula, i∈{1,2,...,5}, x∈{1,2,...,W}, y∈{1,2,...,H}, W are respectively the The width of the grayscale image and the binary image, H is the height of the grayscale image and the binary image respectively, g ₁ , g ₂ , g ₃ , g ₄ , g ₅ are the heart, liver, spleen , the single region edge images of lung and kidney. 6. a kind of traditional Chinese medicine ear five internal organs region segmentation method based on machine learning according to claim 1, is characterized in that, said RGB three-channel image input learning network, obtains predicted image, comprises: Pass the RGB three-channel image through two convolution layers with 3*3 convolution kernels to obtain a feature F _origin with 64 channels and a size of H×W; Pass the feature F _origin through the PAM module to obtain the feature F ₁ ; Pass the feature _F1 through the PAM module to obtain the feature _F2 ; The two results obtained by multiplying the feature F ₁ and the feature F ₂ by the coefficients 0.75 and 0.25 respectively are spliced on the channel, and then passed through the convolution layer with a convolution kernel of 3*3 to obtain the channel as 6. The prediction image whose size is H×W. 7. A kind of method for segmenting five internal organs regions of the ears of traditional Chinese medical science based on machine learning according to claim 1, is characterized in that, obtains each single region predictive image of heart, liver, spleen, lung, kidney from described predictive image, include: Take the maximum value of the predicted image in the dimension of the channel to obtain a dimensional grayscale image with a size of H×W and only six pixel values of 0, 1, 2, 3, 4, and 5, wherein the six types of pixels The areas corresponding to the values in the dimension grayscale image are respectively the predicted segmentation areas of the background, heart, liver, spleen, lung, and kidney; Let P(x, y) denote the pixel value at any coordinate (x, y) of the predicted grayscale image, we have

In the formula, i∈{1,2,...,5}, x∈{1,2,...,W}, y∈{1,2,...,H}, W and H are respectively The width and height of the dimensional grayscale image, P(x, y)={0, 1, ..., 5}, p ₁ , p ₂ , p ₃ , p ₄ , p ₅ are heart, liver, The single region prediction images of spleen, lung and kidney. 8. A kind of machine learning-based TCM ear five internal organs region segmentation method according to claim 7, characterized in that, said single region prediction images of the heart, liver, spleen, lung and kidney are respectively expanded, Corrosion processing to obtain a single-region rough edge prediction map of its corresponding region, including: Define the structure element B, the size of the structure element B is 3*3, the origin is at the center, and the element values are all 1; Use the structural element B to dilate and corrode each of the single-region predicted images of the heart, liver, spleen, lung, and kidney to obtain the expanded predicted single-region image and the corroded single-region predicted image, and after dilation Subtracting the predicted image of the single region and the predicted image of the single region after erosion to obtain the corresponding rough edge prediction map of the region; Let pe _i (x, y) denote the pixel value at any coordinate of the predicted image at the rough edge of the region, we have

In the formula, the symbol

is the dilation operation, the symbol

For corrosion operation, i∈{1, 2,...,5}, x∈{1, 2,...,W}, y∈{1, 2,...,H}, W and H respectively is the width and height of the single region rough edge prediction image, pe _i (x, y) ∈ {0, 1}, pe ₁ , pe ₂ , pe ₃ , pe ₄ , pe ₅ are heart, liver, spleen, The single region rough edge prediction image of the lung and kidney, p _i (x, y) represents the pixel value of any coordinate (x, y) of the corresponding single region prediction image in the heart, liver, spleen, lung and kidney. 9. A traditional Chinese medical science ear five internal organs region segmentation device based on machine learning, characterized in that it comprises at least one processor, and a memory connected to said at least one processor; wherein said memory stores information that can be used by said at least Instructions executed by a processor, the instructions are executed by the at least one processor, so that the at least one processor can execute the method according to any one of claims 1-8.

Images