CN118351110B - Chloasma severity objectively evaluating method based on multitask learning - Google Patents

Abstract

The present invention discloses an objective evaluation method for the severity of chloasma based on multi-task learning, which relates to the application of graphic image processing technology in the field of dermatological diagnosis, including: S1, multi-angle image acquisition of the patient's face through an image acquisition module; S2, segmentation processing of the image acquired by S1 based on a face and chloasma region segmentation model; S3, simultaneous calculation of the uniformity distribution, chromaticity value, and skin lesion area of chloasma in the segmented image based on a multi-task learning model; S4, integration of skin lesion area, uniformity distribution, and chromaticity value, and comprehensive evaluation of the severity of chloasma using MASI evaluation rules. The present invention provides an objective evaluation method for the severity of chloasma based on multi-task learning, which introduces uniformity evaluation and chromaticity value evaluation in chloasma evaluation, and the method can not only improve the calculation speed and reduce the amount of calculation, but also more accurately and comprehensively evaluate the severity of chloasma.

Description

Objective assessment method of melasma severity based on multi-task learning

Technical Field

The present invention relates to the application of graphic image processing technology in the field of dermatological diagnosis. More specifically, the present invention relates to a method for objectively evaluating the severity of melasma based on multi-task learning, which utilizes computer vision and deep learning technology to achieve objective and automated evaluation of the severity of melasma.

Background Art

Melasma is a common skin disease characterized by symmetrically distributed yellow-brown patches on the face. It is more common in women, especially during pregnancy and in women taking oral contraceptives. The formation of melasma is related to many factors, including genetic factors, ultraviolet exposure, hormonal changes, and skin type. Currently, the severity assessment of melasma mainly relies on the clinical experience and visual observation of dermatologists. Although the current existing technology has made attempts to objectively assess the severity of melasma, it only focuses on the area and color depth detection of melasma, and lacks an assessment of the uniformity of melasma, which limits the accuracy of its assessment. That is, the existing technology cannot accurately and objectively assess the severity of melasma, resulting in the following problems:

Blindness in treatment selection: Due to the lack of accurate severity assessment, doctors may not be able to choose the most appropriate treatment plan for patients, resulting in poor treatment effects and failure to help patients recover quickly.

Difficulty in monitoring therapeutic efficacy: It is impossible to accurately monitor changes in treatment effects and it is difficult to adjust the treatment plan in a timely manner, which may result in prolonged treatment time and increase the financial and psychological burden on patients.

Clinical research is limited: The lack of objective and standardized evaluation data restricts the in-depth progress of clinical research and the innovation of treatment methods.

Therefore, the development and use of accurate objective assessments are essential to improving the scientificity and effectiveness of melasma treatment.

Summary of the invention

An object of the present invention is to solve at least the above-mentioned problems and/or disadvantages and to provide at least the advantages which will be described hereinafter.

In order to achieve these purposes and other advantages of the present invention, a method for objectively evaluating the severity of melasma based on multi-task learning is provided, which is characterized by comprising:

S1, collecting multi-angle images of the patient's face through an image acquisition module;

S2, segmenting the image collected by S1 based on the face and chloasma area segmentation model;

S3, based on the multi-task learning model, the uniformity distribution, chromaticity value and skin lesion area of chloasma in the segmented image are simultaneously calculated;

S4. The lesion area, uniform distribution and chromaticity value were integrated, and the severity of melasma was comprehensively evaluated using the MASI evaluation rule.

Preferably, the multi-task learning model includes a chloasma lesion area calculation model and a shared pixel extraction model;

Wherein, the chloasma skin lesion area calculation model calculates the chloasma area of the segmented chloasma area;

The shared pixel extraction model performs chromaticity value calculation and uniformity detection on the segmented chloasma area.

Preferably, in S2, the face and chloasma region segmentation model uses a facial contour automatic recognition algorithm and a region segmentation algorithm to segment the image collected in S1 to obtain a corresponding training set and a test set, and the segmentation process includes:

S210, automatically identifying and segmenting the key points of the human body to obtain the areas corresponding to the forehead, the mandible, the left cheek, and the right cheek in the human face;

S211, using Labelme software to label the melasma in each area to obtain a JSON file;

S212, converting the JSON file into a grayscale image to obtain a face segmentation mask image and a chloasma segmentation mask image;

S213, dividing the face segmentation mask image, the chloasma segmentation mask image and the original image to obtain corresponding training sets and test sets.

Preferably, in S2, the deep learning network model is used as a shared network of the face and chloasma region segmentation model, and the training set is loaded into the shared network for segmentation training, thereby obtaining the face and chloasma region segmentation model;

In the face and chloasma region segmentation model, the image is divided into six regions according to categories: background, forehead, mandible, left cheek, right cheek and chloasma. During training, the size of the image input and the number of image data input simultaneously are selected, and the cross entropy loss function L ( X, Y ) in the following formula is used to complete the parameter setting:

Among them, X is the input image set of the model, Y is the true label or real data corresponding to X , C is the total number of categories, is the one-hot encoding vector corresponding to the true label of the nth instance, is the original output of the model corresponding to the nth instance before the softmax function; is the predicted probability distribution of the class corresponding to the nth instance.

Preferably, when the categories are unbalanced, the cross entropy loss function L ( X,Y ) is optimized by assigning different weights to different categories. The optimized cross entropy loss function L ( X,Y ) is as follows:

In the above formula, Wc _is the weight assigned to category c .

Preferably, in S3, the process of calculating the chromaticity value by the shared pixel extraction model includes:

S310, reading the original image and the segmented chloasma RGB mask image, mapping the pixels corresponding to the original image to the chloasma RGB mask image, so as to restore the pixels corresponding to the original image;

S311, performing a difference operation on the average pixel value of the chloasma area and the average pixel value of the normal skin area to obtain a corresponding difference value;

S312, inputting the difference into the CIEDE2000 color difference calculation formula to obtain the corresponding chloasma area chromaticity evaluation score.

Preferably, in S3, a uniformity detection algorithm is provided in the shared pixel extraction model, and the processing method of the uniformity detection algorithm includes:

S20, obtaining the actual average distance AAD of all melasma distributions based on the centroid of each melasma;

S21. Calculate the density of the centroid distribution using the following formula:

In the above formula, face area represents the area of the face corresponding to the forehead, mandible, left cheek, and right cheek, and centroid numbers represents the number of centroids of the tan spots;

S22, based on the density calculated by S21, the theoretical average distance TUAD of the center of mass is calculated using the following formula:

S23, based on the theoretical average distance TUAD calculated in S22, calculating the ratio R of the actual average distance AAD to the theoretical average distance TUAD;

If the R value is close to 1, it indicates a uniform distribution. If the R value is less than 1, it indicates a clustered distribution. Otherwise, it indicates a skewed distribution.

Preferably, in S4, the MASI judgment value for comprehensively evaluating the severity of melasma is obtained by the calculation result of the following formula:

In the above formula, A refers to the proportion of melasma lesion area, D refers to the chroma score, H is the uniformity score, A _ET indicates the proportion of melasma lesion area in the forehead area, D _ET indicates the color depth of melasma in the forehead area, H _ET indicates the uniformity of melasma in the forehead area, A _MR indicates the proportion of melasma lesion area in the right cheek area, D _MR indicates the color depth of melasma in the right cheek area, H _MR indicates the uniformity of melasma in the right cheek area, A _ML indicates the proportion of melasma lesion area in the left cheek area, D _ML indicates the color depth of melasma in the left cheek area, H _ML indicates the uniformity of melasma in the left cheek area, A _XH indicates the proportion of melasma lesion area in the mandibular area, D _XH indicates the color depth of melasma in the mandibular area, and H _XH indicates the uniformity of melasma in the mandibular area.

The present invention includes at least the following beneficial effects: Compared with the prior art based on single-task evaluation, the chloasma severity assessment method provided by the present application constructs a chloasma multi-task learning model including a chloasma lesion area calculation model and a shared pixel extraction model, and introduces uniformity assessment and chromaticity value evaluation in the chloasma evaluation. This method can not only improve the calculation speed and reduce the amount of calculation, but also more accurately and comprehensively assess the severity of chloasma, and provide a theoretical basis for the clinical formulation of personalized treatment measures and effect monitoring. Specifically, the present invention is based on automatic facial contour recognition and chloasma area segmentation algorithm, through the chloasma lesion area calculation model and the shared pixel extraction model, to simultaneously calculate the chloasma uniformity distribution, chloasma chromaticity value and chloasma lesion area, and finally integrates the lesion area, uniformity distribution and chromaticity value to accurately and comprehensively assess the severity of chloasma.

Other advantages, objectives and features of the present invention will be embodied in part through the following description, and in part will be understood by those skilled in the art through study and practice of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a flow chart of the evaluation of the severity of chloasma in the present invention;

FIG2 is a schematic diagram of a feature extraction network structure of the present invention;

FIG3 is a schematic diagram of the labeling process of the chloasma area of the present invention (it should be noted that (a) is the original image, (b) is the labeled image, and (c) is the chloasma mask image);

FIG4 is a schematic diagram of the chloasma region segmentation result of the present invention (it should be noted that (a) is the original image, and (b) is the chloasma mask image);

FIG5 is a schematic diagram of calculating the chromaticity value of a chloasma region according to the present invention (it should be noted that (a) is a chloasma mask image, (b) is an image after mapping chloasma and normal skin, and (c) is an image after chromaticity value calculation);

FIG6 is a schematic diagram of chloasma uniformity calculation according to the present invention (it should be noted that (a) is a chloasma mask image, (b) is an image after the chloasma centroid is calculated, and the black dot in (b) represents the centroid of the chloasma).

DETAILED DESCRIPTION

The present invention is further described in detail below in conjunction with the accompanying drawings so that those skilled in the art can implement the invention with reference to the description.

The study of this program not only evaluates the area and color of melasma, but also conducts an in-depth evaluation of the uniformity of melasma. This is the core competitive advantage of this program's research. Through the evaluation of uniformity, this program can provide a more accurate and comprehensive diagnosis of melasma, which helps doctors to formulate the best and comprehensive treatment plan. If only the area and color are considered, doctors may underestimate the severity of melasma, which may affect the choice of treatment, delay the patient's recovery time, and even aggravate the condition. However, if this program adds a uniformity assessment on the basis of the existing evaluation method, this program can perform a more comprehensive scoring. A more comprehensive scoring can reflect the true severity of melasma, so that the prescriptions and drug recommendations issued by doctors are more accurate, greatly improving the treatment effect and patient satisfaction of patients.

Therefore, accurate and comprehensive assessment of the severity of melasma can help medical practice in the following aspects:

Accurate treatment plan: Accurate and comprehensive objective assessment can help doctors more accurately determine the severity of melasma, thereby formulating a more personalized and targeted treatment plan. This helps to improve the effectiveness of treatment and reduce unnecessary medical interventions.

Monitoring of treatment effects: Through accurate and comprehensive objective evaluation, doctors can more accurately track treatment progress and adjust treatment strategies in a timely manner. This helps achieve the best treatment effect and avoid long-term treatment failure.

Research and comparison: Accurate and comprehensive objective evaluation provides a standardized data basis to facilitate comparison of the effects of different treatments in clinical studies and promote the development and optimization of melasma treatment methods.

Patient communication: Accurate, comprehensive and objective assessments can help doctors and patients communicate more effectively about their condition and treatment outcomes, and enhance patients’ confidence and participation in the treatment process.

The present invention provides an objective assessment method for the severity of melasma based on multi-task learning. The method uses computer vision and deep learning technology, combined with multi-task learning models such as melasma region segmentation, chromaticity analysis, and uniformity detection, to achieve automatic segmentation and severity assessment of melasma.

To achieve the above objectives, the present invention adopts a method combining a multi-task learning (MTL) algorithm and deep learning technology to improve the accuracy and efficiency of chloasma severity assessment. Specifically, the method of the present invention comprises:

The patient's face is imaged in high definition, with multiple angles including the frontal face (forehead and jaw), left cheek (referred to as left cheek) and right cheek (referred to as right cheek) captured under sufficient lighting conditions. These images are then processed, including:

Firstly, an automatic facial contour recognition algorithm is proposed to accurately identify the forehead, mandible, left and right cheeks. Then, based on the recognized facial contour, the facial contour and melasma area are accurately segmented through a deep learning network.

Based on the segmented facial contour and melasma area, multiple subtasks are performed simultaneously, including task one (melasma uniformity detection model), task two (melasma chromaticity calculation model) and task three (melasma lesion area calculation model).

A chloasma uniformity detection model is provided, based on the facial contour and chloasma segmentation, to propose an algorithm for detecting the uniformity distribution of chloasma.

A chloasma chromaticity calculation model is proposed based on the facial contour and chloasma segmentation, and a pixel mapping algorithm is proposed to calculate the color depth of chloasma.

The chloasma skin lesion area calculation model calculates the percentage of the chloasma area to the total area of the face region based on the face region and the chloasma segmentation network.

The severity of melasma was calculated based on the MASI severity score of melasma, which was combined with the detection of melasma uniformity, chromaticity calculation and area calculation.

Example:

This embodiment discloses a chloasma severity assessment system based on deep learning. The system is mainly composed of image acquisition and image processing modules. Image acquisition mainly uses VISIA and dermatoscope to obtain the left side, right side and front side of the face, and then sends this information to the image processing module for face area recognition and chloasma area segmentation, area calculation, chromaticity value calculation and uniformity calculation, and finally generates a chloasma severity assessment result, as shown in Figure 1, where it should be noted that in Figure 1, W represents width (English Width); H represents height (English Height); × is a multiplication sign; W × H represents image size; W/4 represents a quarter of the width of the original image; H/4 represents a quarter of the height of the original image; W/8 represents an eighth of the width of the original image; H/8 represents an eighth of the height of the original image; W/16 represents a sixteenth of the width of the original image; H/16 represents a sixteenth of the height of the original image; W/32 represents a thirty-second of the width of the original image; H/32 represents a thirty-second of the height of the original image.

The image processing module mainly includes facial contour automatic recognition algorithm, face and chloasma area segmentation model, chloasma lesion area calculation model, chloasma chromaticity calculation model and chloasma uniformity detection model. Automatic face region recognition is mainly used to divide the left cheek, right cheek, forehead and mandible of the face; the chloasma region segmentation task is mainly used to detect and segment the chloasma region in the face region. The two tasks share the same model. The model architecture is shown in Figure 2, where it should be noted that in Figure 2, W represents width; H represents height; × represents the multiplication sign; W × H represents the image size; W/4 represents the width is one-fourth of the original image; H/4 represents the height is one-fourth of the original image; W/8 represents the width is one-eighth of the original image; H/8 represents the height is one-eighth of the original image; W/16 represents the width is one-sixteenth of the original image; H/16 represents the height is one-sixteenth of the original image; W/32 represents the width is one-thirty-second of the original image; H/32 represents the height is one-thirty-second of the original image.

Chloasma chromaticity calculation is mainly used to calculate the difference between the color depth of the chloasma area and the color depth of normal skin, so as to obtain the relative color depth of the chloasma area; Chloasma uniformity detection is mainly used to calculate the uniformity of the distribution of chloasma in the face area, and the two tasks share the same model; Chloasma lesion area calculation is mainly used to calculate the percentage of chloasma in the corresponding face area, specifically:

First, the present invention proposes an automatic facial contour recognition algorithm, which can automatically identify and segment various areas of the face based on key points of the human body, such as the forehead, mandible, and left and right cheeks, and then based on this, use Labelme annotation software to annotate the face area and the chloasma in the corresponding area, and annotate along the edge of the chloasma to obtain a JSON file. Afterwards, the JSON file is converted into a grayscale image using an algorithm to obtain a face segmentation mask image and a chloasma segmentation mask image. The face and chloasma mask images and the original image are divided into a training set and a test set, and the UNet+ deep learning network model is used as a shared network of the face and chloasma area segmentation model, and the training set is loaded into the model for segmentation training to obtain the training results. The specific segmentation process and effect are shown in Figures 3 and 4.

Secondly, when training the face and chloasma area segmentation model, set the parameters according to the model requirements. In this model, the model segmentation categories are six, divided into background, forehead, jaw, left cheek, right cheek and chloasma area. Select the image input size and the number of data input at the same time according to the model requirements. In this example, the input image size is selected to be 512*512 pixels, and 3 images are loaded at a time, of which the frontal image includes the forehead and jaw. In parameter setting, this scheme proposes a new cross entropy loss function to complete multi-class segmentation tasks:

Wherein, X is the input image set of the model, Y is the real label or real data corresponding to X , and C is the total number of categories (6 in the present invention); is the one-hot encoding vector corresponding to the true label of the nth instance, is the original output of the model corresponding to the nth instance before the softmax function; is the predicted probability distribution of the class corresponding to the nth instance.

In addition, if the classes are unbalanced (for example, there are significantly more background pixels than melasma pixels), different weights will be assigned to different classes to offset this imbalance. The weight of each class can be incorporated into the loss function as follows:

Among them, W _c is the weight assigned to category c. Based on this, a larger weight can be assigned to categories with fewer pixels to make their categories relatively balanced and improve the segmentation results of the model.

In addition, this solution selects RMSprop as the optimizer. RMSprop is an optimization algorithm for training neural networks. It aims to adjust the learning rate of each parameter based on the average of the most recent gradients of that parameter. This helps prevent the learning rate from being too high or too low, especially when dealing with non-stationary targets.

In the evaluation of training results, this scheme selects IOU value and mIOU value as the result evaluation indicators. IOU value is the overlap between the self-verified predicted area and the actual marked melasma area during model training, and mIOU is the average value of IOU value during model training.

Right now:

Among them, “prediction area” and “annotation area” represent two rectangular boxes respectively, ∩ represents the intersection of the two rectangular boxes, and ∪ represents the union of the two rectangular boxes.

When training for chloasma segmentation, a 512*512 image is loaded into the segmentation model. The segmentation model first performs four-layer downsampling on the image data to obtain a 64*64 feature map, and then extracts the eigenvalues. The eigenvalues are then upsampled four layers to generate a 512*512 chloasma segmentation result map.

In addition, the trained model is used to perform a segmentation test on the test set data. If the mIOU of the model segmentation result and the test data annotation result is greater than 85%, the segmentation model is considered to meet the requirements.

Again, the melasma area calculation model was used to calculate the melasma area segmented by the melasma area segmentation model. Taking the left cheek as an example, the segmented melasma RGB mask image was first read by computer, and then the image was converted from PIL image format to NumPy array format to calculate the number of all melasma pixels and the number of normal area pixels. Finally, the percentage of melasma pixels in the total number of melasma and normal pixels was calculated.

Furthermore, the shared pixel extraction model is used to calculate the chromaticity value of the melasma area segmented by the melasma area segmentation model. First, the original image and the segmented melasma RGB mask image are read by a computer, and then the pixels corresponding to the original image are mapped to the mask images of the face and melasma to restore them to the pixels corresponding to the original image. Then the average pixel value of the melasma area and the average pixel value of the normal skin area are calculated (the difference between the two is taken), and then the average difference is input into the CIEDE2000 color difference calculation formula, and the result is the chromaticity evaluation score of the melasma area (as shown in Figure 5). The larger the score, the greater the color difference between melasma and normal skin, which means that the melasma is more serious.

Furthermore, the shared pixel extraction model is used to perform uniformity detection on the melasma area segmented by the melasma area segmentation model. First, the segmented melasma RGB mask image is read by a computer. For multiple melasma patches in the face area (for example, there are five melasma patches on the left cheek), based on the proposed uniformity detection algorithm, the centroid of each melasma is first found (as shown in Figure 6), and then the distance between adjacent centroids is calculated, and finally the actual average distance of all melasma distributions is calculated. This actual average distance is then compared with the average distance of the theoretical uniform distribution of melasma in the face area, and then the uniform distribution of melasma is calculated. For only one melasma patch in the face area, the uniform distribution is calculated using the area ratio.

According to the proposed uniformity detection algorithm, the density of the centroid distribution Density is first calculated:

Face area refers to the area of the corresponding face (such as forehead, jaw and left and right cheeks), and centroid numbers refers to the number of centroids of melasma.

Assuming that these centroids are evenly distributed in the image, theoretically, the distance around each point should be the same. Based on the density, the theoretical uniform average distance (TUAD) of the centroids can be calculated:

Finally, calculate the ratio R of the actual average distance (Actual Average Distance, AAD) to the theoretical uniform average distance:

If the R value is close to 1, it means that the actual average distance observed is roughly equal to the average distance of the theoretical uniform distribution, suggesting that the distribution is relatively uniform. If the R value is less than 1, it means that the actual average distance observed is less than the average distance of the theoretical uniform distribution, which means that the points tend to be closer to each other, showing a clustered distribution. If the R value is greater than 1, it means that the actual average distance observed is greater than the average distance of the theoretical uniform distribution, indicating that the distance between points is usually farther than expected under a uniform distribution, showing a biased distribution (i.e., a more dispersed distribution).

The uniform distribution is scored based on the Melasma Area Severity Index (MASI). The MASI score is based on the partition score and then summed according to the proportion of each area. The face is divided into four areas: ET represents the forehead area, MR represents the right cheek area, ML represents the left cheek area, and XH represents the mandibular area. The formula for evaluating melasma is:

Table 1 is a scoring table for the area and severity of melasma:

Table 1

Among them, A refers to the proportion of chloasma lesions, D refers to the chromaticity score, H is the uniformity score, and the color and uniformity of chloasma lesions are divided into five levels: none, mild, moderate, obvious and maximum. The subscript of each variable represents the area. In the above formula, A _ET represents the proportion of chloasma lesions in the forehead area, D _ET represents the depth of chloasma color in the forehead area, H _ET represents the uniformity of chloasma in the forehead area, A _MR represents the proportion of chloasma lesions in the right cheek area, D _MR represents the depth of chloasma color in the right cheek area, H _MR represents the uniformity of chloasma in the right cheek area, A _ML represents the proportion of chloasma lesions in the left cheek area, D _ML represents the depth of chloasma color in the left cheek area, H _ML represents the uniformity of chloasma in the left cheek area, A _XH represents the proportion of chloasma lesions in the mandibular area, D _XH represents the depth of chloasma color in the mandibular area, and H _XH represents the uniformity of chloasma in the mandibular area. In addition, according to the MASI score, the minimum score is 0, indicating no melasma, and the maximum score is 48, indicating severe melasma. Among them, 1-48 points are divided into 4 equal parts, indicating mild, mild, moderate, and severe. That is: mild: 1-12 points; mild: 13-24 points; moderate: 25-36 points; 37-48 points. Specific examples are as follows:

For example, when the proportion of chloasma area in the forehead area is between 30% and 49%, the score is 3 points; when the color depth of chloasma in the forehead area is moderate, the score is 2 points, and when the uniformity of chloasma in the forehead area is mild, the score is 1 point; when the area of chloasma in the right cheek area is between 10% and 29%, the score is 2 points; when the color depth of chloasma in the right cheek area is obvious, the score is 3 points, and when the uniformity of chloasma in the right cheek area is moderate, the score is 2 points; when the area of chloasma in the left cheek area is between 10% and 29%, the score is 2 points; when the color depth of chloasma in the left cheek area is moderate, the score is 2 points, and when the uniformity of chloasma in the left cheek area is obvious, the score is 3 points; when the area of chloasma in the mandibular area is between <10, the score is 1 point; when the color depth of chloasma in the mandibular area is mild, the score is 1 point, and when the uniformity of chloasma in the mandibular area is mild, the score is 1 point. The sum of each target sub-score resulted in a chloasma assessment result of 8.9 points, indicating that the severity of the patient's chloasma was mild.

The effects of this example include:

(1) This solution proposes an automatic facial contour recognition algorithm that can automatically recognize and segment different parts of the face based on key points of the human body, such as using the corners of the eyes, eyebrows, and nose as the defining criteria to distinguish the cheek, chin, and forehead areas. Compared with existing methods, this technology avoids the problem of image overlap caused by different shooting angles, thereby improving data quality and significantly enhancing the performance of the model.

(2) Based on the existing technology, the present invention proposes a unique algorithm for uniformity detection, which provides a more accurate, comprehensive and personalized evaluation effect for the severity of melasma. In addition, the algorithm further enhances the comprehensiveness and accuracy of melasma diagnosis and medication management.

(3) The present invention provides a chromaticity mapping scoring method: first, the original image and the chloasma RGB mask image are read by a computer, and then the pixels corresponding to the original image are mapped to the mask image to restore the corresponding pixel points. Then, the difference in the average pixel value between the chloasma area and the normal skin color area is calculated, and the difference is applied to the CIEDE2000 color difference formula to obtain the numerical value of the chromaticity evaluation of the chloasma area. The higher the value, the greater the color difference between the chloasma and normal skin, and the more serious the degree of the lesion.

The color mapping scoring method provides the following assistance for the assessment of melasma:

① Quantitative analysis: This method provides a quantitative way to determine the color difference between melasma and normal skin, thereby providing an objective assessment standard for the severity of the lesion.

② Reproducibility and consistency: This method can be repeatedly applied at different times and on different devices to obtain consistent scoring results, which is crucial for long-term clinical monitoring and efficacy evaluation.

③Objectivity: The CIEDE2000 color difference calculation formula is used to ensure the objectivity and scientificity of the evaluation results and avoid the errors that may be caused by pure visual comparison.

(4) The present invention has the capability of processing multiple images simultaneously, that is, it can process three photos of the patient from the front and side at the same time, automatically segment the facial area and chloasma in these three photos, and evaluate the severity of the results. Doctors can provide personalized treatment recommendations for patients based on the evaluation results.

Through the comprehensive application of these technologies, the method of this program improves the objectivity and accuracy of the diagnostic procedure, and provides strong data support for clinical treatment decisions, ultimately achieving precise and personalized evaluation of melasma treatment methods.

The above solution is only an illustration of a preferred embodiment, but is not limited thereto. When implementing the present invention, appropriate replacement and/or modification can be performed according to user needs.

Although the embodiments of the present invention have been disclosed above, they are not limited to the applications listed in the specification and the embodiments. It can be fully applied to various fields suitable for the present invention. For those familiar with the art, additional modifications can be easily realized. Therefore, without departing from the general concept defined by the claims and equivalent scope, the present invention is not limited to the specific details and the illustrations shown and described here.

Claims (3)

1. A method for objectively evaluating the severity of melasma based on multi-task learning, comprising: S1, collecting multi-angle images of the patient's face through an image acquisition module; S2, segmenting the image collected by S1 based on the face and chloasma area segmentation model; S3, based on the multi-task learning model, the uniformity distribution, chromaticity value and skin lesion area of chloasma in the segmented image are simultaneously calculated; S4. Integrate the lesion area, uniform distribution and chromaticity value, and use the MASI evaluation rule to comprehensively evaluate the severity of melasma; In S2, the deep learning network model is used as a shared network of the face and melasma region segmentation model, and the training set is loaded into the shared network for segmentation training, thereby obtaining the face and melasma region segmentation model; In the face and chloasma region segmentation model, the image is divided into six regions according to categories: background, forehead, mandible, left cheek, right cheek and chloasma. During training, the size of the image input and the number of image data input simultaneously are selected, and the cross entropy loss function L ( X, Y ) in the following formula is used to complete the parameter setting: Among them, X is the input image set of the model, Y is the true label or real data corresponding to X , C is the total number of categories, is the one-hot encoding vector corresponding to the true label of the nth instance, is the original output of the model corresponding to the nth instance before the softmax function; is the predicted probability distribution of the class corresponding to the nth instance; When the categories are unbalanced, the cross entropy loss function L ( X,Y ) is optimized by assigning different weights to different categories. The optimized cross entropy loss function L ( X,Y ) is as follows: In the above formula, W _c is the weight assigned to category c ; The multi-task learning model includes a chloasma lesion area calculation model and a shared pixel extraction model; Wherein, the chloasma skin lesion area calculation model calculates the chloasma area of the segmented chloasma area; The shared pixel extraction model calculates the chromaticity value and performs uniformity detection on the segmented chloasma area; In S3, the process of calculating the chromaticity value using the shared pixel extraction model includes: S310, reading the original image and the segmented chloasma RGB mask image, mapping the pixels corresponding to the original image to the chloasma RGB mask image, so as to restore the pixels corresponding to the original image; S311, performing a difference operation on the average pixel value of the chloasma area and the average pixel value of the normal skin area to obtain a corresponding difference value; S312, inputting the difference into the CIEDE2000 color difference calculation formula to obtain the corresponding chloasma area chromaticity evaluation score; In S3, a uniformity detection algorithm is provided in the shared pixel extraction model, and the processing method of the uniformity detection algorithm includes: S320, obtaining the actual average distance AAD of all melasma distributions based on the centroid of each melasma; S321. Calculate the density of the centroid distribution using the following formula: In the above formula, face area represents the area of the face corresponding to the forehead, mandible, left cheek, and right cheek, and centroid numbers represents the number of centroids of melasma; S322. Based on the density calculated in S321, the theoretical average distance TUAD of the center of mass is calculated using the following formula: S323, based on the theoretical average distance TUAD calculated in S322, calculating the ratio R of the actual average distance AAD to the theoretical average distance TUAD; If the R value is close to 1, it indicates a uniform distribution. If the R value is less than 1, it indicates a clustered distribution. Otherwise, it indicates a skewed distribution. 2. The objective assessment method for the severity of chloasma based on multi-task learning according to claim 1 is characterized in that, in S2, the face and chloasma region segmentation model uses a facial contour automatic recognition algorithm and a region segmentation algorithm to segment the image collected in S1 to obtain corresponding training sets and test sets, and the segmentation process includes: S210, automatically identifying and segmenting the key points of the human body to obtain the areas corresponding to the forehead, the mandible, the left cheek, and the right cheek in the human face; S211, using Labelme software to label the melasma in each area to obtain a JSON file; S212, converting the JSON file into a grayscale image to obtain a face segmentation mask image and a chloasma segmentation mask image; S213, dividing the face segmentation mask image, the chloasma segmentation mask image and the original image to obtain corresponding training sets and test sets. 3. The objective assessment method for the severity of melasma based on multi-task learning according to claim 1, characterized in that, in S4, the MASI judgment value for comprehensively assessing the severity of melasma is obtained by the calculation result of the following formula: In the above formula, A refers to the proportion of melasma lesion area, D refers to the chroma score, H is the uniformity score, A _ET indicates the proportion of melasma lesion area in the forehead area, D _ET indicates the color depth of melasma in the forehead area, H _ET indicates the uniformity of melasma in the forehead area, A _MR indicates the proportion of melasma lesion area in the right cheek area, D _MR indicates the color depth of melasma in the right cheek area, H _MR indicates the uniformity of melasma in the right cheek area, A _ML indicates the proportion of melasma lesion area in the left cheek area, D _ML indicates the color depth of melasma in the left cheek area, H _ML indicates the uniformity of melasma in the left cheek area, A _XH indicates the proportion of melasma lesion area in the mandibular area, D _XH indicates the color depth of melasma in the mandibular area, and H _XH indicates the uniformity of melasma in the mandibular area.