CN120070899B

CN120070899B - Training method and system for dermoscope image segmentation model

Info

Publication number: CN120070899B
Application number: CN202510541618.7A
Authority: CN
Inventors: 高丽
Original assignee: Sanya Central Hospital Hainan Third People's Hospital Sanya Central Hospital Medical Group General Hospital
Current assignee: Sanya Central Hospital Hainan Third People's Hospital Sanya Central Hospital Medical Group General Hospital
Priority date: 2025-04-28
Filing date: 2025-04-28
Publication date: 2025-10-17
Anticipated expiration: 2045-04-28
Also published as: CN120070899A

Abstract

The invention relates to the technical field of image processing, in particular to a training method and system for a dermoscope image segmentation model. The system comprises an image color correction and noise removal module, an image sample enhancement balancing module, a segmentation model construction module and a segmentation model training optimization module, wherein a multisource skin mirror image can be obtained to carry out color offset correction processing, loss and blur removal and image sample enhancement are carried out at the same time, a skin mirror enhanced image sample is generated, the skin mirror enhanced image sample is subjected to undersampling balancing processing, a skin mirror proportion balancing image sample is generated, a lesion region segmentation network is constructed by combining a multiscale feature fusion module and an attention mechanism, segmentation model training optimization is carried out by combining an adaptive learning rate adjustment strategy, and a skin mirror image segmentation optimization model is generated to output a skin mirror image lesion region segmentation result at the last layer in the lesion region segmentation network. The invention improves the accuracy of skin lesion region segmentation based on the image processing technology.

Description

Training method and system for dermoscope image segmentation model

Technical Field

The invention relates to the technical field of image processing, in particular to a training method and system for a dermoscope image segmentation model.

Background

The skin microscopy provides detailed information of skin lesion areas through high magnification and optical imaging technology, and with the continuous development of the skin microscopy, the application of the skin microscopy in early detection of skin cancers (particularly melanoma) is more and more widespread. In dermatoscopic image analysis, image segmentation is one of the key steps, the goal of which is to separate the lesion region of interest from the dermatoscopic image for subsequent feature extraction and classification. In recent years, rapid development of deep learning technology has led to a Convolutional Neural Network (CNN) having an excellent effect in an image segmentation task. However, although various methods for medical image segmentation have been proposed, there are problems in particular application to dermoscopic images, for example, existing models tend to perform poorly when dealing with complex backgrounds, different lighting conditions and skin types, resulting in reduced segmentation accuracy, further affecting the generalization ability and application of the model.

Disclosure of Invention

Accordingly, the present invention is directed to a training method and system for a dermoscope image segmentation model, which solve at least one of the above-mentioned problems.

To achieve the above object, a training system for a dermoscope image segmentation model includes the following modules:

The system comprises an image color correction and noise removal module, a mirror color correction and noise removal module and a mirror color correction module, wherein the image color correction and noise removal module is used for acquiring a multi-source mirror image, acquiring a mirror spectrum image corresponding to R, G and B color channels through the multi-source mirror image, and performing color offset correction processing on the multi-source mirror image based on the mirror spectrum image corresponding to R, G and B color channels to generate a mirror color offset correction image;

The image sample enhancement balancing module is used for enhancing the image sample of the skin mirror standardized image to obtain a skin mirror enhanced image sample;

The segmentation model construction module is used for constructing a lesion area segmentation network based on a convolutional neural network and combining the multi-scale feature fusion module and an attention mechanism so as to generate a dermoscope image segmentation model;

The segmentation model training optimization module is used for carrying out segmentation model training optimization on the skin mirror image segmentation model based on the skin mirror proportion balance image sample and combining with the self-adaptive learning rate adjustment strategy so as to generate a skin mirror image segmentation optimization model, and outputting a corresponding skin mirror image lesion region segmentation result at the last layer in the lesion region segmentation network.

Further, the image color correction and noise removal module comprises the following functions:

Acquiring a multisource dermoscope image;

Acquiring corresponding dermatoscopic spectral images under R, G and B color channels through the multi-source dermatoscopic image, and performing color shift correction processing on the multi-source dermatoscopic image based on the corresponding dermatoscopic spectral images under R, G and B color channels to generate a dermatoscopic color shift correction image;

performing contrast enhancement on the skin mirror color shift correction image by using histogram equalization to obtain a skin mirror contrast enhancement image;

performing pixel ambiguity analysis on the dermatoscope contrast enhanced image to obtain the pixel ambiguity of the dermatoscope image;

and performing loss-removing and pixel normalization processing on each pixel block in the dermatoscope contrast enhancement image based on the dermatoscope image pixel ambiguity so as to generate a dermatoscope normalization image.

Further, the obtaining the corresponding skin mirror spectral images under R, G and B color channels through the multi-source skin mirror image, and performing color shift correction processing on the multi-source skin mirror image based on the corresponding skin mirror spectral images under R, G and B color channels includes:

performing RGB color channel decomposition on the multi-source skin mirror image under different color channel wavelengths to generate corresponding skin mirror spectrum images under R, G and B color channels;

Carrying out local area brightness analysis on the corresponding dermatoscope spectrum images under R, G and B color channels to obtain local area brightness characteristics corresponding to R, G and B color channels, wherein the local area brightness characteristics comprise local area brightness distribution mean value and local area brightness distribution variance;

performing image color shift calculation on the corresponding skin mirror spectral image based on the corresponding local area brightness characteristics under R, G and B color channels to obtain corresponding skin mirror image color shift under R, G and B color channels;

Performing channel color adjustment analysis on the corresponding skin mirror spectrum image based on the corresponding skin mirror image color offset under R, G and B color channels to obtain corresponding skin mirror image color adjustment amplitude and adjustment direction under R, G and B color channels;

And performing color shift correction processing on the corresponding skin mirror spectrum image under the corresponding color channel based on the color adjustment amplitude and the adjustment direction of the corresponding skin mirror image under the R, G and B color channels, so as to correct the color value corresponding to each pixel block in the skin mirror spectrum image according to the corresponding color adjustment amplitude and the adjustment direction of the skin mirror image, and generate a skin mirror color shift correction image.

Further, the performing color shift correction processing on the corresponding skin mirror spectrum image under the corresponding color channel based on the color adjustment amplitude and the adjustment direction of the corresponding skin mirror image under the color channel R, G and the color channel B includes:

performing channel adjustment evaluation calculation according to the corresponding dermoscope image color adjustment amplitude and adjustment direction under R, G and B color channels to obtain corresponding dermoscope color channel adjustment coefficients under R, G and B color channels;

Acquiring corresponding skin mirror image spectral characteristics under corresponding color channels through corresponding skin mirror spectral images under R, G and B color channels, and performing color shift simulation on the corresponding color channels based on the corresponding skin mirror image spectral characteristics under the corresponding color channels and combined with corresponding skin mirror color channel adjustment coefficients under R, G and B color channels to generate corresponding skin mirror image color shift mechanisms under R, G and B color channels;

Performing color value shift correction on color values corresponding to each pixel block in the corresponding skin mirror spectral image under the corresponding color channel based on the corresponding skin mirror image color shift mechanism under R, G and the B color channel and according to the corresponding skin mirror color channel adjustment coefficient to generate a local color shift correction matrix corresponding to R, G and the B color channel, including color shift values corresponding to each pixel block under the corresponding color channel;

each pixel block within the corresponding dermatological spectral image for the respective color channel is subjected to a pixel-by-pixel color correction reconstruction process based on the corresponding local color shift correction matrix for the respective color channel R, G and B to generate a dermatological color shift corrected image.

Further, the removing the loss and blurring of each pixel block in the dermatoscope contrast enhancement image based on the pixel blurring degree of the dermatoscope image and the pixel standardization processing comprise:

Performing pixel blurring division on each pixel block in the skin mirror contrast enhancement image based on the pixel blurring degree of the skin mirror image, comparing and judging the pixel blurring degree corresponding to each pixel block according to a preset pixel blurring threshold value, if the pixel blurring degree is larger than the preset pixel blurring threshold value, determining the corresponding pixel block as a scattering blurring pixel block, if the pixel blurring degree is equal to the preset pixel blurring threshold value, determining the corresponding pixel block as a noise blurring pixel block, and if the pixel blurring degree is smaller than the preset pixel blurring threshold value, determining the corresponding pixel block as a clear pixel block;

Performing fuzzy block noise removal on a corresponding scattered fuzzy pixel block and a noise fuzzy pixel block in the skin mirror contrast enhancement image to calculate the light scattering degree for the scattered fuzzy pixel block to compensate the light scattering loss corresponding to the pixel block, enclosing the noise fuzzy pixel block into a corresponding noise fuzzy region, and performing noise interference removal on the noise fuzzy region by utilizing median filtering to generate a skin mirror fuzzy denoising image;

Performing pixel mean and variance analysis on the dermoscope fuzzy denoising image to obtain a dermoscope image pixel mean and a dermoscope image pixel variance;

and performing pixel normalization processing on the dermoscope blurring denoising image based on the dermoscope image pixel mean value and the dermoscope image pixel variance to generate a dermoscope normalization image.

Further, the calculating the light scattering degree for the scattering blurred pixel block to compensate the light scattering loss corresponding to the pixel block includes:

Performing spectral scattering analysis on the scattering fuzzy pixel block to statistically analyze the scattering wavelength and the scattering fuzzy distortion degree corresponding to the pixel block, and forming a characteristic vector by the scattering fuzzy pixel block to generate a spectral scattering fuzzy characteristic vector;

Acquiring environmental factors, object surface materials and light source angles corresponding to the pixel block through the scattering fuzzy pixel block, and carrying out scattering intensity prediction calculation based on the environmental factors, the object surface materials and the light source angles corresponding to the pixel block and combining the spectrum scattering fuzzy feature vector so as to obtain the light scattering degree corresponding to the pixel block;

Acquiring theoretical light scattering intensity corresponding to the pixel block through the scattering fuzzy pixel block, and performing scattering loss compensation calculation based on the light scattering degree corresponding to the pixel block and the corresponding theoretical light scattering intensity to obtain a scattering loss compensation factor;

The light scattering loss corresponding to the pixel block is compensated based on the scattering loss compensation factor.

Further, the image sample enhancement balancing module includes the following functions:

Image sample enhancement is carried out on the skin mirror standardized image so as to enhance and simulate corresponding skin mirror image samples under different conditions through random rotation, overturning, scaling, cutting, brightness adjustment and contrast adjustment operation, so that skin mirror enhanced image samples are obtained;

Performing lesion area proportion analysis on the dermatoscope enhanced image sample to quantitatively calculate a proportion value between a lesion sample and a normal sample in the dermatoscope image sample, so as to obtain a dermatoscope image sample lesion proportion;

And performing over-sampling balance processing on the skin-mirror enhanced image sample based on the skin-mirror image sample lesion proportion, wherein the over-sampling balance processing specifically comprises performing over-sampling operation on the corresponding lesion sample in the skin-mirror enhanced image sample to increase the number of samples corresponding to a minority class by using the samples corresponding to the minority class generated against network replication, and performing under-sampling operation on the corresponding normal sample in the skin-mirror enhanced image sample to reduce the number of samples corresponding to a majority class, so as to generate the skin-mirror proportion balanced image sample.

Further, the segmentation model construction module includes the following functions:

Designing a 3-layer 5x5 convolution layer, a 3x3 convolution layer and a 1x1 pooling layer through a convolution neural network, and connecting each layer through a connecting channel layer to construct a corresponding dermoscope image segmentation network architecture;

The method comprises the steps of combining different convolution layer designs in a dermoscope image segmentation network architecture with corresponding multi-scale feature fusion modules, introducing jump connection and feature fusion operation through the multi-scale feature fusion modules to fuse lesion high-resolution features corresponding to 3x3 shallow layers and lesion semantic features corresponding to 5x5 deep layers, so that the pooling layers can adjust feature images of different scales to the same size through up-sampling and down-sampling operation to perform stitching fusion, and improving segmentation capability of lesion areas of different scales through introducing attention mechanisms in connection channel layers between the layers to construct a lesion area segmentation network, wherein the attention mechanisms consist of channel attention modules and spatial attention modules, the channel attention modules are arranged in the connection channel layers between each layer of convolution layer and pooling layer to automatically learn importance between each connection channel layer, and the spatial attention modules are arranged in each connection channel layer between 3 layers of network segmentation architecture space to focus on corresponding lesion areas in a dermoscope image sample to enhance task expression of the network corresponding to the lesion areas so as to generate a dermoscope image segmentation model.

Further, the segmentation model training optimization module comprises the following functions:

dividing the skin mirror proportion balance image sample into a training image sample, a verification image sample and a test image sample according to the dividing proportion of 7:2:1;

Inputting a training image sample into a skin mirror image segmentation model for segmentation model training, inputting a verification image sample into the trained skin mirror image segmentation model for training loss analysis, calculating cross entropy loss for a lesion classification task and Dice loss for a lesion segmentation task through training, and carrying out weighted summation according to the cross entropy loss and the Dice loss to obtain skin mirror image segmentation training loss;

And (3) carrying out learning rate adjustment on the trained skin mirror image segmentation model based on the skin mirror image segmentation training loss and combining with an adaptive learning rate adjustment strategy, so that when the skin mirror image segmentation training loss is determined to not be obviously reduced in 10 iteration cycles according to the adaptive learning rate adjustment strategy, the learning rate corresponding to the skin mirror image segmentation model is reduced to 0.1 times of the original learning rate, the learning rate convergence verification is carried out on the skin mirror image segmentation model subjected to the learning rate adjustment by utilizing a test image sample, and meanwhile, model training optimization is carried out by adopting an L2 regularization method, so that a skin mirror image segmentation optimization model is generated, and a corresponding skin mirror image lesion region segmentation result is output at the last layer in a lesion region segmentation network.

Further, the present invention also provides a training method for a skin mirror image segmentation model, the method is implemented based on the training system for a skin mirror image segmentation model, and the training method for a skin mirror image segmentation model comprises:

Obtaining a corresponding skin mirror spectrum image under R, G and B color channels through the multi-source skin mirror image, and carrying out color shift correction processing on the multi-source skin mirror image based on the skin mirror spectrum image under R, G and B color channels to generate a skin mirror color shift correction image;

performing undersampling balance processing on the skin mirror enhanced image sample to generate a skin mirror proportion balance image sample;

Constructing a lesion area segmentation network based on a convolutional neural network and combining a multi-scale feature fusion module and an attention mechanism to generate a dermatoscope image segmentation model;

And performing segmentation model training optimization on the skin mirror image segmentation model based on the skin mirror proportion balance image sample and combining with the self-adaptive learning rate adjustment strategy to generate a skin mirror image segmentation optimization model, and outputting a corresponding skin mirror image lesion region segmentation result at the last layer in the lesion region segmentation network.

The invention has the beneficial effects that:

Compared with the prior art, the training system for the split model of the skin mirror image has the beneficial effects that the color components of the skin mirror image can be more accurately acquired by acquiring the multi-source skin mirror image and utilizing the spectral information of the images under three color channels of R (red), G (green) and B (blue), and the key point of the step is that the color channel information is converted into the spectral image of the skin mirror, so that important color and spectral data support is provided for subsequent image processing and lesion analysis, and different skin mirror images are often subjected to different acquisition environments, The device is set and the light source conditions influence, so that the images are required to be subjected to color shift correction, the color shift correction can effectively eliminate color differences caused by factors such as the color temperature of the light source, the illumination conditions and the like, so that the color distribution of the skin mirror images of different sources is more consistent, the accuracy and consistency of image analysis are ensured, the color distortion of the images is corrected after the color shift correction, and the subsequent image processing such as segmentation, feature extraction and the like can obtain better effects. In addition, in order to further improve the quality of the image, the corrected image needs to be subjected to loss-dispersion and blurring removal and standardization treatment, and the loss-dispersion and blurring removal can eliminate blurring caused by acquisition equipment or environmental factors in the image, so that the dermoscope image is clearer, and subsequent detail analysis is facilitated. The standardized processing can unify the characteristics of the size, the brightness and the like of the images, reduce the influence of the difference of the size or the illumination among different images, and further ensure the stability and the precision of the subsequent model training and reasoning. Secondly, through carrying out various data enhancement processes (such as rotation, overturning, scaling, noise adding and the like) on the skin mirror image, a training data set can be effectively expanded, so that the model is contacted with richer features in the learning process, the generalization capability of model training is improved, and in addition, the problem of model bias caused by too many or too few samples in a certain type in training can be avoided through over-undersampling balance processing. for example, if a certain type of lesion image is significantly more than other types, the model may tend to identify only a greater number of lesion types, and by balancing the sample scale such that the weight of each class is balanced, the model's ability to identify on different lesion types may be improved, and such enhancement and scale balancing processes are critical to high precision lesion region segmentation. Then, a lesion region segmentation network is constructed based on a Convolutional Neural Network (CNN) so as to more accurately identify and segment a lesion region by utilizing the strong characteristic of deep learning, and a multi-scale feature fusion module is combined, so that the network can extract features from different scales (from small to large and from local to global), and the identification precision of a complex lesion structure is improved. In addition, the attention mechanism can effectively strengthen the attention of the network to important features, inhibit the interference of irrelevant information, and the self-adaptively adjust the feature weight so that the network can be more effectively focused on areas with larger influence on the segmentation result, thereby obviously improving the segmentation precision and efficiency, and the finally generated dermoscope image segmentation model can not only improve the efficiency in the model training process, but also improve the segmentation precision of the lesion area of the dermoscope image. Finally, training and optimizing the segmentation model by utilizing the image samples with balanced proportion to generate a final segmentation optimization model, and in the training process, adopting a self-adaptive learning rate adjustment strategy, the learning rate can be dynamically adjusted according to training progress and model performance, so that the problems of instability caused by overlarge learning rate, slow convergence speed caused by overlarge learning rate and the like can be avoided, the training efficiency and effect are improved, and after the optimization, the model can better capture key characteristics and data distribution, thereby improving the segmentation precision of a lesion region.

Drawings

Other features, objects and advantages of the invention will become more apparent upon reading of the detailed description of a non-limiting implementation, made with reference to the accompanying drawings in which:

FIG. 1 is a block diagram of a training system for a dermoscope image segmentation model of the present invention;

FIG. 2 is a functional flow diagram of the image color correction and noise removal module of FIG. 1;

Fig. 3 is a functional flow diagram of the image sample enhancement balancing module in fig. 1.

Detailed Description

The following is a clear and complete description of the technical method of the present invention, taken in conjunction with the accompanying drawings, and it is evident that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the present invention without making any inventive effort, are intended to fall within the scope of the present invention.

Furthermore, the drawings are merely schematic illustrations of the present invention and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and thus a repetitive description thereof will be omitted. Some of the block diagrams shown in the figures are functional entities and do not necessarily correspond to physically or logically separate entities. The functional entities may be implemented in software or in one or more hardware modules or integrated circuits or in different networks and/or processor methods and/or microcontroller methods.

It will be understood that, although the terms "first," "second," etc. may be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another element. For example, a first element could be termed a second element, and, similarly, a second element could be termed a first element, without departing from the scope of example embodiments. The term "and/or" as used herein includes any and all combinations of one or more of the associated listed items.

To achieve the above object, referring to fig. 1 to 3, the present invention provides a training system for a dermoscope image segmentation model, the system comprising the following modules:

In the embodiment of the present invention, please refer to fig. 1, which is a schematic diagram of a training system for a skin mirror image segmentation model according to the present invention, in this example, the training system for a skin mirror image segmentation model includes the following modules:

The system comprises an image color correction and noise removal module, a speculum standard image, a speculum color correction module, a speculum image correction module and a speculum standard image, wherein the image color correction and noise removal module is used for acquiring a multi-source speculum image, acquiring a speculum spectrum image corresponding to R, G and B color channels through the multi-source speculum image, and performing color offset correction processing on the multi-source speculum image based on the speculum spectrum image corresponding to R, G and B color channels to generate a speculum color offset correction image;

In the embodiment of the invention, through collecting the skin mirror images from multiple channels such as a professional skin disease hospital, a scientific research institution, a medical equipment provider and the like, carrying out RGB color channel decomposition on the images, generating a spectrum image under R, G, B color channels, carrying out local area brightness analysis on the spectrum images of each channel, calculating the mean value and the variance, obtaining the color offset by comparing a reference value, determining the adjustment amplitude and the adjustment direction, carrying out color offset correction, then carrying out histogram equalization to enhance the contrast, analyzing the pixel ambiguity by using a Laplace operator, dividing pixel blocks according to the ambiguity threshold value, respectively processing the scattered ambiguity and the noise ambiguity pixel blocks, finally calculating the pixel mean value and the variance of the processed images, realizing standardization of each pixel, and finally generating the skin mirror standardized image.

S2, an image sample enhancement balancing module, which is used for enhancing the image sample of the skin mirror standardized image to obtain a skin mirror enhanced image sample;

In the embodiment of the invention, the skin mirror enhanced image sample is obtained by performing random rotation (-90 degrees to 90 degrees), turnover (horizontal and vertical), scaling (0.8 to 1.2 times), cutting, brightness adjustment (-0.2 to 0.2) and contrast adjustment (-0.2 to 0.2) on the skin mirror standardized image, simulating images under different conditions, classifying the enhanced image sample pixel by using a two-class model, counting the total number of pixels in a lesion and a normal region, calculating the lesion proportion, when the lesion proportion is smaller than 0.3, oversampling the lesion sample by using a generated countermeasure network, randomly deleting 50% of the normal sample, and finally generating the skin mirror proportion balanced image sample.

S3, a segmentation model construction module is used for constructing a lesion area segmentation network based on a convolutional neural network and combining a multi-scale feature fusion module and an attention mechanism so as to generate a dermoscope image segmentation model;

According to the embodiment of the invention, a 3-layer 5x5 convolution layer, a 3x3 convolution layer and a 1x1 pooling layer are designed according to a convolution neural network principle, all the layers are connected through a connecting channel layer to construct a basic network architecture, a multi-scale feature fusion module is designed on different convolution layers, high-resolution features of the 3x3 shallow layer and semantic features of the 5x5 deep layer are fused through jump connection, the feature pattern size is adjusted through up and down sampling and then spliced, an attention mechanism is introduced on the connecting channel layer, the channel attention module is arranged between the convolution layer and the pooling layer, the channel importance is learned through a global average pooling layer and a full connecting layer, the space attention module is arranged on the connecting channel layer between the 3-layer network architecture space, a lesion region is focused, and finally a lesion region segmentation network is constructed, so that a skin mirror image segmentation model is generated.

And S4, a segmentation model training optimization module is used for carrying out segmentation model training optimization on the skin mirror image segmentation model based on the skin mirror proportion balance image sample and combining with the self-adaptive learning rate adjustment strategy so as to generate a skin mirror image segmentation optimization model, and outputting a corresponding skin mirror image lesion region segmentation result at the last layer in the lesion region segmentation network.

In the embodiment of the invention, a skin mirror proportion balance image sample is divided into training, verifying and testing image samples according to a ratio of 7:2:1, the training image samples are input into a skin mirror image segmentation model for training, the verifying image samples are used for training loss analysis, cross entropy loss of lesion classification and Dice loss of lesion segmentation are calculated, the training loss is obtained through weighted summation, in the training process, if the decreasing amplitude of the training loss is smaller than 0.001 in 10 continuous iteration cycles, the learning rate is reduced to 0.1 times of the original training loss, the learning rate adjusting effect is verified by using the testing image samples, an L2 regularization optimization model is adopted, the skin mirror image segmentation optimization model is finally generated, and a lesion region segmentation result is output at the last layer of a lesion region segmentation network.

s11, acquiring a multisource skin mirror image;

Step S12, obtaining a corresponding dermatoscope spectrum image under R, G and a B color channel through the multi-source dermatoscope image, and carrying out color shift correction processing on the multi-source dermatoscope image based on the corresponding dermatoscope spectrum image under R, G and the B color channel so as to generate a dermatoscope color shift correction image;

step S13, performing contrast enhancement on the skin mirror color shift correction image by using histogram equalization to obtain a skin mirror contrast enhancement image;

step S14, carrying out pixel ambiguity analysis on the dermatoscope contrast enhancement image to obtain the pixel ambiguity of the dermatoscope image;

And S15, carrying out loss-removing and pixel standardization processing on each pixel block in the dermatoscope contrast enhancement image based on the pixel ambiguity of the dermatoscope image so as to generate a dermatoscope standardization image.

As an embodiment of the present invention, referring to fig. 2, a functional flow diagram of the image color correction and noise removal module in fig. 1 is shown, where the image color correction and noise removal module in this embodiment includes the following functions:

S11, acquiring a multisource dermoscope image;

In an embodiment of the invention, the dermatological image data of a patient is acquired by collecting dermatological image data from a plurality of sources, including a professional dermatological hospital, a scientific research institution and a medical equipment provider, in cooperation with the dermatological department of the hospital, by means of the dermatological examination equipment thereof, at a fixed image acquisition frequency, such as 100 images per day, the dermatological image of the patient is acquired, the image data accumulated in the dermatological research process of the patient is acquired from the scientific research institution, the data comprise dermatological images of different types of dermatological diseases, the images collected in the equipment testing and verification stage are obtained from the medical equipment provider, and the dermatological images from different channels are uniformly stored in a data storage center for subsequent processing.

S12, acquiring corresponding skin mirror spectral images under R, G and B color channels through the multi-source skin mirror image, and performing color shift correction processing on the multi-source skin mirror image based on the corresponding skin mirror spectral images under R, G and B color channels to generate a skin mirror color shift correction image;

In the embodiment of the invention, RGB color channel decomposition is carried out on a multisource skin mirror image, a skin mirror spectrum image under R, G, B color channels is generated, local area brightness analysis is carried out on the spectrum image of each channel, the image is divided into local areas with 32 multiplied by 32 pixels, the average value and variance of brightness distribution of each area are calculated, the average value and variance of standard brightness distribution are set as references, the average value and variance of each local area are compared with reference values, the color offset of each color channel is obtained through weighted summation, the color adjustment amplitude and direction are determined according to the offset, if the offset is positive and the adjustment direction is brightness reduction, the color value of each pixel block of the channel is subtracted by the adjustment amplitude, the adjusted color value is limited to be between 0 and 255, and finally, the corrected three channel images are recombined, and finally the skin mirror color offset correction image is generated.

S13, performing contrast enhancement on the skin mirror color shift correction image by using histogram equalization to obtain a skin mirror contrast enhancement image;

In the embodiment of the invention, the pixel value of the skin mirror color offset correction image is regarded as a gray value set, the occurrence frequency of each gray value in the image is counted, a gray histogram is constructed, a cumulative distribution function of the gray values is calculated, the cumulative distribution function is normalized to be in a value range of 0-255, each pixel value in the image is mapped according to the normalized cumulative distribution function, and the original pixel value is replaced by a new mapped pixel value. For example, if the original pixel value is 100, a new pixel value is 150 after mapping by the cumulative distribution function, the gray value of the pixel is updated to 150, and the contrast of the image is enhanced by mapping all the pixels in the image, so as to finally obtain the dermatoscope contrast enhancement image.

S14, carrying out pixel ambiguity analysis on the dermatoscope contrast enhanced image to obtain the pixel ambiguity of the dermatoscope image;

in the embodiment of the invention, the Laplace operator template (such as [0, 1, 0], [1, -4, 1], [0, 1, 0 ]) is subjected to sliding convolution operation on the image by adopting the Laplace operator to process the dermatoscope contrast enhancement image, the second derivative of each pixel point is calculated, the second derivative reflects the change rate of the pixel value, the larger the change rate is, the greater the difference of the pixel values around the pixel is, the clearer the image is, the smaller the change rate is, the more blurred the image is, the result obtained after the convolution operation is subjected to statistical analysis, and the average value of the second derivatives of all the pixel points is calculated, namely the pixel ambiguity of the dermatoscope image.

And S15, performing loss-removing and pixel standardization processing on each pixel block in the dermatoscope contrast enhancement image based on the pixel ambiguity of the dermatoscope image so as to generate a dermatoscope standardization image.

In the embodiment of the invention, the image is divided into pixel blocks of 5X 5 pixels by the contrast enhancement of the skin mirror, a pixel blurring threshold value is set, for example, 0.5, the pixel blurring degree of each pixel block is compared with the threshold value, if the pixel blurring degree is larger than the threshold value, the pixel blocks are determined to be the blurring pixel blocks, the light scattering degree is calculated by analyzing the pixel value distribution, the light scattering loss is compensated by adopting a linear interpolation method, if the pixel blurring pixel blocks are determined to be the threshold value, the adjacent pixel blocks are enclosed into a noise blurring area, the noise is removed by median filtering, the pixel mean value and the variance of the processed image are calculated, and a formula is used for each pixel valuePerforming normalization treatment, whereinAs the mean value of the pixels,And finally generating a skin mirror standardized image for an evolution value corresponding to the pixel variance, namely the pixel standard deviation. The blurring removal processing is carried out on the pixel blocks, blurring of the skin mirror image caused by various factors (such as equipment imaging quality, uneven skin surface and the like) can be effectively reduced, details in the image are clearer and more discernable, characteristics of skin lesions such as boundaries, colors and textures are more accurately observed, and when the pixel standardization processing is carried out, the gray scale range of the image can be adjusted, so that differences among different pixel values are more obvious, and the contrast of the image is enhanced. This helps to highlight the distinction of the diseased region of the skin from surrounding normal tissue, improving the detection and diagnostic accuracy of the lesions.

In the embodiment of the invention, through regarding the multi-source skin mirror image as a three-dimensional matrix formed by three color channels of red (R), green (G) and blue (B), each pixel of the image is separated on the three channels according to the wavelength characteristics of the color channels by utilizing an image channel separation algorithm, specifically, for each pixel point in the image, the value of the pixel point in the R channel is extracted to form a two-dimensional matrix only containing R channel information, namely a skin mirror spectrum image under the R color channel, and similarly, the values of the G and B channels are respectively extracted to generate corresponding skin mirror spectrum images under the G and B color channels, for example, if the resolution of the image is 512×512 pixels, the spectrum image of each channel after separation is also a two-dimensional matrix of 512×512, and the two-dimensional matrix respectively represents skin mirror image information under different color channels.

Preferably, local area brightness analysis is performed on the corresponding skin mirror spectrum image under R, G and the B color channel to obtain local area brightness characteristics corresponding to R, G and the B color channel, wherein the local area brightness characteristics comprise local area brightness distribution mean value and local area brightness distribution variance;

In the embodiment of the invention, the spectral image of the skin mirror under R, G, B color channels is divided into a plurality of non-overlapping local areas, for example, each local area is 32×32 pixels, the sum of brightness values of each local area is calculated, taking an R channel as an example, the R channel values of all pixels in the local area are added and divided by the total number of pixels in the area to obtain the average value of brightness distribution of the local area, then the square of the difference between the R channel value of each pixel and the average value is calculated, the square values are summed and divided by the total number of pixels to obtain the variance of brightness distribution of the local area, and the spectral images of the G and B channels are processed according to the same method to respectively obtain the average value and the variance of brightness distribution of the local area corresponding to R, G and B color channels, and the values form the brightness characteristics of the local area.

Preferably, image color shift calculation is performed on the corresponding skin mirror spectral image based on the corresponding local area brightness features under R, G and B color channels to obtain corresponding skin mirror image color shift under R, G and B color channels;

In the embodiment of the invention, a standard mean value and a standard variance of brightness distribution are set as reference values, the reference values represent brightness characteristics of the skin mirror image in each color channel under an ideal state, for the skin mirror spectrum image in the R color channel, the mean value and the variance of brightness distribution of each local area are compared with the reference values, the difference value of the mean value and the reference mean value of brightness distribution of the local area and the difference value of the variance of brightness distribution of the local area and the reference variance of the reference value are calculated, the two difference values are weighted and summed according to a certain weight (for example, the mean value difference weight is 0.6 and the variance difference weight is 0.4), the color offset of the local area is obtained, the same calculation is carried out on all local areas of the R color channel, the average value is taken as the color offset of the skin mirror image in the R color channel, and the color offsets of the skin mirror image in the G color channel and the B color channel are calculated respectively in the same mode.

Preferably, channel color adjustment analysis is performed on the corresponding skin mirror spectral image based on the corresponding skin mirror image color offset under R, G and the B color channel, so as to obtain the corresponding skin mirror image color adjustment amplitude and adjustment direction under R, G and the B color channel;

In the embodiment of the invention, the color offset of the skin mirror image under the R color channel is positive, the overall brightness of the channel is higher, the adjustment direction is reduced, the overall brightness is lower, the adjustment direction is improved, the adjustment amplitude is determined according to the absolute value of the offset, for example, a scaling factor k (k=0.1) is set, the absolute value of the offset is multiplied by k to obtain the primary adjustment amplitude, meanwhile, the primary adjustment amplitude is further corrected in consideration of the overall contrast and color balance of the image, for example, when the offset is larger, the adjustment amplitude is properly reduced to avoid excessive adjustment, the skin mirror image color offset under the G and B color channels is respectively analyzed according to the same method, the corresponding color adjustment amplitude and adjustment direction are determined, and finally the corresponding skin mirror image color adjustment amplitude and adjustment direction under the R, G and B color channels are obtained.

Preferably, the color shift correction process is performed on the corresponding skin mirror spectral image under the corresponding color channel based on the corresponding skin mirror image color adjustment amplitude and adjustment direction under R, G and B color channels, so as to correct the color value corresponding to each pixel block in the skin mirror spectral image according to the corresponding skin mirror image color adjustment amplitude and adjustment direction, so as to generate the skin mirror color shift correction image.

In the embodiment of the invention, the color value of the R channel of each pixel block is added with the adjustment amplitude by adding the adjustment amplitude to the skin mirror spectrum image under the R color channel if the adjustment direction is brightness enhancement, the adjustment amplitude is subtracted if the adjustment direction is brightness reduction, the range of the adjusted color value is limited to ensure that the color value is between 0 and 255 when the adjustment is performed, for example, if the R channel value of a certain pixel block is 200, the adjustment amplitude is 10, the adjusted R channel value is min (200+10, 255) =210, the skin mirror spectrum images under the G and B color channels are subjected to color shift correction processing in the same manner, and finally, the corrected R, G, B-channel spectrum images are recombined into a complete image to finally generate the skin mirror color shift correction image.

In the embodiment of the present invention, by using the OpenCV library of Python and the related mathematical calculation library (e.g., numpy), the channel adjustment evaluation calculation is performed according to the corresponding color adjustment range and adjustment direction of the skin mirror image under R, G and B color channels, and the data is read from the file stored with the adjustment record, where the adjustment operation of the multi-source skin mirror image on each color channel is recorded, for example, for R channel of a certain image, the record shows that the color adjustment range is increased by 10%, the direction is forward (i.e., the color is more biased toward red), for each color channel, taking R channel as an example, if the adjustment range is (Assuming 0.1) direction is a sign functionThe representation (if in the forward direction,The opposite direction) For example, then the formula can be usedCalculating the dermoscope color channel adjustment coefficient, assuming that in a certain adjustment, the R channel adjustment amplitude is increased by 15% (i.e) The direction is forward [ ]) ThenThe same calculation is also performed for G, B channels, resulting in corresponding dermoscope color channel adjustment coefficients for R, G and B color channels.

Preferably, the corresponding skin mirror image spectral characteristics under the corresponding color channels are obtained through the corresponding skin mirror spectral images under R, G and the B color channels, and the corresponding color channels are subjected to color shift simulation based on the corresponding skin mirror image spectral characteristics under the corresponding color channels and combined with the corresponding skin mirror color channel adjustment coefficients under R, G and the B color channels so as to generate corresponding skin mirror image color shift mechanisms under R, G and the B color channels;

In the embodiment of the present invention, by using the OpenCV library of Python and the spectrum analysis related library (such as the module related to spectrum analysis in scipy library), the corresponding speculum spectrum image under R, G and B color channels is read, and the R channel is taken as an example, and the R channel spectrum image is analyzed to obtain the spectrum characteristics, such as the spectrum peak position, the spectrum bandwidth, etc., and it is assumed that the spectrum peak of the R channel spectrum image is located at the wavelength by analysis Where the bandwidth isAnd obtain the corresponding dermoscope color channel adjustment coefficients under R, G and B color channels, to obtain R channel adjustment coefficientsFor example, (assuming 1.2), the color shift is simulated by building a mathematical model, e.g., assuming that the spectral peak position has a linear relationship with the color shift, the shifted spectral peak position(WhereinFor the proportionality constant determined based on a large number of experiments, it is assumed that) Similar simulation is also carried out on G, B channels, and the generated color shift mechanism of the corresponding skin mirror image under R, G and B color channels is generated, wherein the simulation process and related parameters of the color shift of each channel are elaborated, and finally the color shift mechanism of the corresponding skin mirror image under R, G and B color channels is generated.

Preferably, the color value shift correction is performed on the color value corresponding to each pixel block in the corresponding skin mirror spectral image under the corresponding color channel based on the corresponding skin mirror image color shift mechanism under R, G and B color channels and according to the corresponding skin mirror color channel adjustment coefficient, so as to generate a corresponding local color shift correction matrix under R, G and B color channels, including the color shift value corresponding to each pixel block under the corresponding color channel;

in the embodiment of the invention, by utilizing the OpenCV library and numpy library of Python, based on the color shift mechanism of the corresponding dermatological image and the corresponding dermatological color channel adjustment coefficient under R, G and B color channels, the color value shift correction is performed on the color value corresponding to each pixel block in the dermatological spectral image corresponding to the corresponding color channel, so as to read the dermatological spectral image, the image is divided into a plurality of pixel blocks, each pixel block is assumed to be 8×8 pixels, and for each pixel in each pixel block, the R channel is taken as an example, the original color value is According to the color shift mechanism and the adjustment coefficient, the adjustment coefficient is assumedBy the formula(WhereinFor the scale factor determined according to the color shift mechanism, assume that) Calculating color values after offset, calculating differences between color values after offset and original color values of all pixels in each pixel block, obtaining color offset values, arranging the color offset values into a matrix form, generating a corresponding local color offset correction matrix under an R channel, performing the same operation on a G, B channel to store matrix data by using a npy format of numpy, and finally generating a corresponding local color offset correction matrix under R, G and a B color channel.

Preferably, each pixel block within the corresponding dermatological spectral image under the respective color channel is subjected to a pixel-by-pixel color correction reconstruction process based on the corresponding local color shift correction matrix under R, G and B color channels to generate a dermatological color shift corrected image.

In the embodiment of the present invention, by using the OpenCV library and numpy library of Python, the pixel-by-pixel color correction reconstruction process is performed on each pixel block in the corresponding skin-mirror spectral image under the corresponding color channel based on the corresponding local color shift correction matrix under R, G and B color channels, so as to read the corresponding local color shift correction matrix file under R, G and B color channels, and for each pixel block, for example, for each pixel block, the color value of each pixel in the pixel block is corrected according to the color shift value in the corresponding local color shift correction matrix, for example, if the color shift value of a certain pixel position in the local color shift correction matrix isThe corrected color value of the pixel, wherein,After such correction processing is performed on all pixel blocks for the initial color value, the corrected R-channel image and the also corrected G, B-channel image are combined, and a merge function of OpenCV is used to finally generate a skin mirror color shift correction image, so that color corrected high-quality image data is provided for subsequent training of a skin mirror image segmentation model.

In the embodiment of the invention, the skin mirror contrast enhancement image is divided into a plurality of pixel blocks with the same size, for example, each pixel block is 5x5 pixels, the pixel block is processed by using the Laplacian, the pixel ambiguity is calculated, the Laplacian reflects the change condition among pixels by calculating the second derivative of the pixel value in the pixel block, the more severe the change is, the higher the pixel ambiguity is, the preset pixel ambiguity threshold is set, for example, 0.5, and the calculated pixel ambiguity is compared with the threshold for each pixel block. If the pixel ambiguity of a certain pixel block is larger than 0.5, the pixel block is determined to be a scattering blurred pixel block because the pixel block is greatly influenced by factors such as light scattering, if the pixel ambiguity is equal to 0.5, the pixel block is determined to be a noise blurred pixel block because the pixel block is mainly interfered by noise, and if the pixel ambiguity is smaller than 0.5, the pixel block is considered to be clear and is determined to be a clear pixel block, and in this way, pixel ambiguity division is carried out on all the pixel blocks in the skin mirror contrast enhanced image.

Preferably, the corresponding scattered blurred pixel blocks and noise blurred pixel blocks in the skin mirror contrast enhancement image are subjected to blur block noise removal, so that the light scattering degree is calculated for the scattered blurred pixel blocks to compensate the light scattering loss corresponding to the pixel blocks, the noise blurred pixel blocks are enclosed into corresponding noise blurred areas, and the noise blurred areas are subjected to noise interference removal by utilizing median filtering to generate the skin mirror blurred denoising image;

In the embodiment of the invention, the light scattering degree is calculated by analyzing the distribution condition of the pixel values of the scattering blurred pixel block, the standard deviation of the pixel values in the scattering blurred pixel block is assumed to be sigma, the light scattering degree is calculated by a specific formula, for example, the light scattering degree=k×sigma (k is a coefficient determined according to experiments), the pixel value of the pixel block is adjusted by adopting a linear interpolation method and the like according to the calculated light scattering degree so as to compensate the loss caused by light scattering, the brightness and the color of the pixel block are more approximate to the actual condition, the noise blurred pixel block is combined to form a noise blurred region, the noise blurred region is processed by using a median filtering method, the pixel values in a window are ordered by taking a neighborhood window of 3x3 as an example, the central pixel value of the window is replaced by the middle value, the noise interference is removed by the mode, and the noise blurred pixel block are processed to finally generate the skin mirror denoising image.

Preferably, performing pixel mean and variance analysis on the dermoscope fuzzy denoising image to obtain a dermoscope image pixel mean and a dermoscope image pixel variance;

in the embodiment of the invention, the image is treated as a two-dimensional pixel matrix by regarding the blurred and denoised image of the skin mirror, the width of the image is W, the height is H, the pixel mean value is calculated first, and all pixel values in the image are added, namely Then dividing the total pixel number W multiplied by H to obtain the pixel mean value of the dermoscope imageThen, the pixel variance is calculated for each pixel value in the imageCalculate its and pixel mean valueSquare of differenceThe squares of these differences for all pixels are added, i.eDividing by the total pixel number W×H to obtain the pixel variance of the skin mirror imageBy such calculation, the dermoscope image pixel mean value and the dermoscope image pixel variance are accurately obtained.

Preferably, the dermoscope blur-denoised image is subjected to pixel normalization processing based on the dermoscope image pixel mean value and the dermoscope image pixel variance to generate the dermoscope normalization image.

In an embodiment of the invention, the denoising image is blurred by blurring each pixel value in the image for the dermatoscopeUsing the formulaPerforming pixel normalization processing in whichFor the previously calculated dermoscope image pixel mean value,For the standard deviation corresponding to the pixels of the skin mirror image, the formula carries out normalization transformation on each pixel value, so that the distribution of the pixel values of the image has zero mean and unit variance, the pixel values of the image are adjusted to a uniform scale range by carrying out such normalization processing on all the pixel values in the skin mirror fuzzy denoising image, the pixel value difference caused by factors such as illumination and equipment among different images is eliminated, and finally, a skin mirror standardized image is generated, so that more standard and easier-to-process data is provided for subsequent skin mirror image segmentation model training.

In the embodiment of the invention, the spectrum analysis is performed on the scattered fuzzy pixel block by utilizing the spectrum analysis library (such as a module related to spectrum analysis in scikit-image) of Python, the scattered fuzzy pixel block data is read from the dermatoscope image data set marked with the scattered fuzzy pixel block, the pixel block is stored as an independent image file, the spectrum information of the pixel block is processed by using the spectrum analysis function in the library such as the Fourier transformation related function, the scattered wavelength corresponding to the pixel block is determined by analyzing the spectrum data, for example, after Fourier transformation, the frequency corresponding to the energy peak value is found in the frequency spectrum, and then the conversion relation between the frequency and the wavelength is calculated (WhereinIn order to achieve the light velocity, the light beam is,For frequency), assuming that the calculated scattering wavelength is 550nm, for the scattering ambiguity distortion, calculating by comparing the spectrum characteristic difference of the pixel block and the clear pixel block, constructing a spectrum characteristic template library of the clear pixel block, selecting the spectrum characteristic of the clear pixel block which is most similar to the current scattering ambiguity pixel block from the template library, calculating the Mean Square Error (MSE) between the two, wherein the larger the MSE value is used for indicating the higher the scattering ambiguity distortion, assuming that the scattering ambiguity distortion is 0.3 after calculation, forming the scattering wavelength and the scattering ambiguity distortion into a characteristic vector [550,0.3], and finally generating the spectrum scattering ambiguity characteristic vector.

Preferably, the environment factors, the object surface materials and the light source angles corresponding to the pixel block are obtained through the scattering fuzzy pixel block, and the scattering intensity prediction calculation is carried out based on the environment factors, the object surface materials and the light source angles corresponding to the pixel block and by combining the spectrum scattering fuzzy feature vector, so as to obtain the light scattering degree corresponding to the pixel block;

In the embodiment of the invention, by utilizing a data analysis library (such as pandas) of Python and an algorithm library related to some domain knowledge, from a file storing information related to the shooting of a skin mirror image, environmental factors (such as humidity and temperature during shooting), object surface materials (such as texture type of skin, whether scales exist or not and the like) and light source angle information corresponding to a scattering blurred pixel block are found according to the pixel block numbers, the environmental humidity corresponding to the pixel block is assumed to be 50%, the temperature is 25 ℃, the object surface materials are smooth and skin with a small amount of scales, the light source angle is 45 degrees with the skin surface, so as to read a spectrum scattering blurred feature vector, a pre-established light scattering prediction model is utilized to calculate scattering intensity, the model is obtained through training a large amount of experimental data, for example, a model is built by using a Support Vector Regression (SVR) algorithm, the model is input into the environmental factors, the object surface materials, the light source angles, the scattering wavelengths and the scattering blurred degree in the spectrum blurred feature vector, the scattering degree is output into a light degree, the relevant data is assumed to be input into the model, the corresponding to obtain the corresponding scattering degree of the pixel block is assumed to be 0.6.

Preferably, the theoretical light scattering intensity corresponding to the pixel block is obtained through the scattering blurred pixel block, and scattering loss compensation calculation is carried out based on the light scattering degree corresponding to the pixel block and the theoretical light scattering intensity corresponding to the pixel block, so as to obtain a scattering loss compensation factor;

In the embodiment of the invention, the theoretical light scattering intensity corresponding to the scattered blurred pixel block is obtained by utilizing a mathematical calculation library (such as numpy) of Python and starting from the optical theory knowledge and the dermatoscope imaging principle, by referring to related data or referring to optical specialists, the theoretical light scattering intensity is calculated to be 0.8 according to factors such as the material, the light source condition and the imaging distance of the current pixel block, the light scattering intensity corresponding to the pixel block is obtained, the light scattering degree is calculated to be 0.6, and the scattering loss compensation factor is calculated according to the formula of scattering loss compensation factor = theoretical light scattering intensity/light scattering degree, namely 0.8/0.6 is about 1.33, and finally the scattering loss compensation factor is obtained.

Preferably, the light scattering loss corresponding to the pixel block is compensated based on the scattering loss compensation factor.

In the embodiment of the present invention, by using the OpenCV library and numpy library of Python, the corresponding scattering loss compensation factor is obtained from the previous time, which is assumed to be 1.33, and the scattering blurred pixel block image data is read, and for each pixel in the pixel block, the original light intensity value is set to beAccording to the formulaCalculating a compensated light intensity value, for example, an original light intensity value of a certain pixel is 100, the compensated light intensity value is 100×1.33=133, performing such calculation on all pixels in a pixel block, completing compensation of light scattering loss corresponding to the pixel block, covering original files with compensated pixel block image data, finally compensating light scattering loss corresponding to the pixel block, providing high-quality pixel block data after light scattering loss compensation for subsequent skin mirror image segmentation model training, and improving segmentation accuracy of the model on skin mirror images.

As an embodiment of the present invention, referring to fig. 3, a functional flowchart of the image sample enhancement balancing module in fig. 1 is shown, where the image sample enhancement balancing module in this embodiment includes the following functions:

S21, performing image sample enhancement on the skin mirror standardized image so as to enhance and simulate corresponding skin mirror image samples under different conditions through random rotation, overturning, scaling, cutting, brightness adjustment and contrast adjustment operations, so as to obtain skin mirror enhanced image samples;

In the embodiment of the invention, various image enhancement operations are sequentially carried out on a standardized image of a skin mirror, in random rotation operation, an angle is randomly selected from the center of the image as a rotation point in the range of-90 degrees to rotate the image, so as to simulate the skin mirror image under different visual angles, the image is respectively horizontally turned and vertically turned for random turning, the diversity of the image is increased, during zooming operation, the image is randomly zoomed in the proportion range of 0.8 to 1.2, the observation effect under different distances is simulated, the cutting operation is to randomly select an area in the image for cutting, the image characteristics of different parts are reserved, the brightness value is randomly adjusted in the range of-0.2 to 0.2 on the basis of the original brightness value, the contrast is also randomly changed in the range of-0.2 to 0.2, and the skin mirror image sample under different conditions is simulated through the operations, so that the skin mirror enhanced image sample is finally obtained.

S22, performing lesion area proportion analysis on the dermatoscope enhanced image sample to quantitatively calculate a proportion value between a lesion sample and a normal sample in the dermatoscope image sample, so as to obtain a dermatoscope image sample lesion proportion;

In the embodiment of the invention, through carrying out pixel-by-pixel analysis on the dermatoscopic enhanced image sample, classifying each pixel by utilizing a pre-trained two-classification model to judge whether the pixel belongs to a lesion area or a normal area, wherein the two-classification model is trained based on a large number of labeled dermatoscopic image data, has higher classification accuracy, counts the total number of pixels of the lesion area and the total number of pixels of the normal area and is respectively recorded as AndThen, proportional value by formulaAnd calculating a ratio value between the lesion sample and the normal sample to obtain a lesion ratio of the dermatological image sample, for example, if the total number of pixels of a lesion area is 5000 and the total number of pixels of the normal area is 20000, the lesion ratio of the dermatological image sample is 5000/20000=0.25.

And S23, performing over-sampling balance processing on the skin mirror enhanced image sample based on the skin mirror image sample lesion proportion, wherein the over-sampling balance processing specifically comprises the steps of performing over-sampling operation on the corresponding lesion sample in the skin mirror enhanced image sample if the skin mirror image sample lesion proportion shows that the lesion sample is far less than the normal sample, so as to increase the sample number corresponding to the minority class by using the generation of the sample corresponding to the minority class against network duplication, and performing under-sampling operation on the corresponding normal sample in the skin mirror enhanced image sample, so as to reduce the sample number corresponding to the majority class, and generate the skin mirror proportion balanced image sample.

In the embodiment of the invention, when the lesion proportion of the skin mirror image sample is smaller than 0.3, the lesion sample is judged to be much less than the normal sample, the generation countermeasure network (GAN) is used for carrying out oversampling operation on the lesion sample, the GAN is composed of a generator and a discriminator, random noise is taken as input by the generator to try to generate an image similar to a real lesion sample, the discriminator judges whether the input image is the real lesion sample or the image generated by the generator, the generator can generate high-quality lesion sample images through continuously training the generator and the discriminator, so that the number of the lesion samples is increased, a random undersampling method is adopted for the normal sample, a certain proportion of normal samples are randomly selected for deletion, for example, 50% of the normal samples are deleted, the number of samples corresponding to most categories is reduced, the number of the lesion samples and the normal samples reach relative balance through the oversubsampled balance treatment, and finally the skin mirror proportion balance image samples are generated.

In the embodiment of the invention, when the structure of the skin mirror image segmentation network is constructed, firstly, design is carried out according to the principle of a convolutional neural network, a 5x5 convolutional layer is constructed for a first layer, the convolutional kernel size of the layer is 5x5, the input skin mirror image is subjected to feature extraction operation by setting a proper quantity (such as 32) of the convolutional kernels, specifically, the convolutional kernels slide on the image and carry out convolution operation with pixel values of the image to generate a feature image, then a second layer is designed to be a 3x3 convolutional layer, a certain quantity of the convolutional kernels (such as 64) are also arranged, the feature image output by the first layer is further extracted to obtain finer features, then a third layer is constructed to be a 1x1 pooling layer, the 1x1 pooling layer is used for carrying out dimension reduction operation on the feature image, important feature information is reserved at the same time, each layer is connected through a connecting channel layer, the connecting channel layer ensures that the feature image can be smoothly transferred from one layer to the next layer, for example, the feature image output by the first layer 5x5 convolutional layer is transferred to the 3x3 convolutional layer for final image processing, and the structure of the skin mirror image is constructed.

Preferably, the 3x3 shallow layer corresponding lesion high resolution features and 5x5 deep layer corresponding lesion semantic features are fused by combining corresponding multi-scale feature fusion modules through different convolution layer designs in the dermoscope image segmentation network architecture, jump connection and feature fusion operations are introduced through the multi-scale feature fusion modules, so that the pooling layer adjusts feature images with different scales to the same size through up-sampling and down-sampling operations to execute splicing fusion, segmentation capacity of the different scale lesion areas is improved through introducing attention mechanisms in connection channel layers between the layers to construct a lesion area segmentation network, wherein the attention mechanisms consist of channel attention modules and spatial attention modules, the channel attention modules are arranged in the connection channel layers between the convolution layers and the pooling layer in each layer to automatically learn the importance between each connection channel layer, and the spatial attention modules are arranged in each connection channel layer between the 3-layer network architecture space to focus on the corresponding lesion areas in the dermoscope image sample so as to enhance task expression of the network for segmenting the lesion areas corresponding to generate the dermoscope image segmentation network model.

In the embodiment of the invention, a multiscale feature fusion module is designed for different convolution layers in a constructed dermoscope image segmentation network architecture, for a 3x3 shallow convolution layer, an output feature image of the 3x3 shallow convolution layer contains high resolution features of lesions, a feature image output by a 5x5 deep convolution layer contains semantic features of lesions, the output of the 3x3 shallow convolution layer is directly connected to a subsequent proper layer in a corresponding jump connection mode through the multiscale feature fusion module, then fusion operation is carried out on the features of the 5x5 deep convolution layer, in a pooling layer, the feature images of different scales are adjusted to the same size by utilizing an up-sampling (such as a bilinear interpolation method) and a down-sampling (such as a maximum pooling) operation, then splicing fusion is carried out, so that the fused feature image simultaneously has high resolution and semantic information.

In the embodiment of the invention, by assuming that a lot of N skin mirror proportion balance image samples are provided, firstly, calculating the number of samples for division, the number of training image samples is n×0.7, the number of verification image samples is n×0.2, the number of test image samples is n×0.1, randomly extracting n×0.7 samples from N samples as training image samples by adopting a random sampling method to ensure the randomness of sampling to cover images with different characteristics, then randomly extracting n×0.2 samples from the rest samples as verification image samples, finally, taking the rest n×0.1 samples as test image samples, for example, if 1000 skin mirror proportion balance image samples are provided, 700 training image samples are provided, 200 verification image samples are provided, 100 test image samples are provided, and a proper data set is provided for subsequent model, verification and test by the definite proportion division.

Preferably, a training image sample is input into a skin mirror image segmentation model for segmentation model training, a verification image sample is input into the trained skin mirror image segmentation model for training loss analysis, cross entropy loss for a lesion classification task and Dice loss for a lesion segmentation task are calculated through training, and weighted summation is carried out according to the cross entropy loss and the Dice loss to obtain skin mirror image segmentation training loss;

In the embodiment of the invention, through sequentially inputting training image samples into a skin mirror image segmentation model, each convolution layer, pooling layer and other components in the model perform feature extraction and processing on the image, in the training process, for a lesion classification task, cross entropy loss is calculated by using a cross entropy loss function according to classification results output by the model and real lesion class labels, for example, if the model predicts that a certain skin lesion is malignant probability of 0.8 and the real labels are malignant (the label value is set to be 1), the cross entropy loss is calculated by a specific formula, for the lesion segmentation task, the segmentation results output by the model are compared with the real lesion segmentation areas, the Dice loss is calculated by using a Dice loss function, the real lesion areas are assumed to be A, the lesion areas predicted by the model are assumed to be B, the Dice loss is calculated according to the proportional relation between intersection sets and union sets of the two, weights are respectively set for the cross entropy loss and the Dice loss, for example, the cross entropy loss weight is 0.6, the Dice loss weight is 0.4, and the weight is summed up, so that the skin mirror image segmentation loss is finally obtained.

Preferably, the training loss of the skin mirror image segmentation is based on and combined with an adaptive learning rate adjustment strategy to perform learning rate adjustment on the trained skin mirror image segmentation model, so that when the training loss of the skin mirror image segmentation is determined to not be obviously reduced in 10 iteration cycles according to the adaptive learning rate adjustment strategy, the learning rate corresponding to the skin mirror image segmentation model is reduced to 0.1 times of the original learning rate, the learning rate convergence verification is performed on the skin mirror image segmentation model subjected to the learning rate adjustment by using a test image sample, and meanwhile, the model training optimization is performed by using an L2 regularization method to generate a skin mirror image segmentation optimization model, and a corresponding skin mirror image lesion region segmentation result is output at the last layer in a lesion region segmentation network.

In the embodiment of the invention, the training loss of the skin mirror image segmentation model is continuously monitored in the model training process, each time an iteration period is completed, the current training loss value is recorded, when the decreasing amplitude of the training loss value is smaller than a certain preset minimum value (such as 0.001) in 10 iteration periods, the training loss is judged not to be obviously decreased in 10 iteration periods, at the moment, the current learning rate of the skin mirror image segmentation model is multiplied by 0.1 to obtain a new learning rate, for example, the original learning rate is 0.001, the new learning rate is changed into 0.0001 after adjustment, then a test image sample is input into the model with the learning rate adjusted, the change condition of the model output result is observed, whether the learning rate adjustment is effective or not is verified, meanwhile, an L2 regularization method is adopted in the model training process, a regularization term is added in the loss function, the model parameters are restrained, the fitting is prevented, the skin mirror image segmentation optimization model is finally generated, and the accurate skin mirror image lesion region segmentation result is output in the last layer of a lesion region segmentation network after the training process.

The present embodiments are, therefore, to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein.

The foregoing is only a specific embodiment of the invention to enable those skilled in the art to understand or practice the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A training system for a dermoscope image segmentation model, comprising the following modules:

Wherein, the segmentation model construction module comprises the following functions:

Combining corresponding multi-scale feature fusion modules through different convolution layer designs in a dermoscope image segmentation network architecture, so as to fuse lesion high-resolution features corresponding to 3x3 shallow layers and lesion semantic features corresponding to 5x5 deep layers through jump connection and feature fusion operations introduced by the multi-scale feature fusion modules, so that the pooling layer adjusts feature images with different scales to the same size through up-sampling and down-sampling operations to execute splicing fusion, and improves segmentation capacity of lesion areas with different scales through introducing attention mechanisms in connection channel layers between the layers to construct a lesion area segmentation network, wherein the attention mechanisms consist of a channel attention module and a spatial attention module, the channel attention module is arranged in the connection channel layers between each layer convolution layer and the pooling layer to automatically learn the importance between each connection channel layer, and the spatial attention module is arranged in each connection channel layer between 3 layers of network architecture spaces to focus on the corresponding lesion areas in the dermoscope image samples to enhance the segmentation task expression of the network corresponding to the lesion areas so as to generate a dermoscope image model;

2. The training system for a dermatological image segmentation model of claim 1, wherein the image color correction and noise removal module includes the following functions:

Acquiring a multisource dermoscope image;

3. The training system for a dermatological image segmentation model of claim 2, wherein the acquiring, by the multi-source dermatological image, the dermatological spectral image corresponding at R, G and B color channels and performing a color shift correction process on the multi-source dermatological image based on the dermatological spectral image corresponding at R, G and B color channels comprises:

4. A training system for a dermatological image segmentation model according to claim 3, wherein the performing color shift correction processing on the corresponding dermatological spectral image under the respective color channel based on the corresponding dermatological image color adjustment magnitudes and adjustment directions under R, G and B color channels comprises:

5. The training system for a dermatological image segmentation model of claim 2, wherein the performing a speckle blur removal and pixel normalization process on each pixel block within a dermatological contrast enhanced image based on dermatological image pixel blur comprises:

6. The training system for a dermoscope image segmentation model of claim 5, wherein calculating the degree of light scattering for a block of scattered blur pixels to compensate for the loss of light scattering corresponding to the block of pixels comprises:

7. The training system for a dermatological image segmentation model of claim 1, wherein the image sample enhancement balancing module includes the following functions:

8. The training system for a dermatological image segmentation model of claim 1, wherein the segmentation model training optimization module includes the following functions:

And (3) carrying out learning rate adjustment on the trained skin mirror image segmentation model based on the skin mirror image segmentation training loss and combining with an adaptive learning rate adjustment strategy, so that when the skin mirror image segmentation training loss is determined to not be obviously reduced in 10 iteration cycles according to the adaptive learning rate adjustment strategy, the learning rate corresponding to the skin mirror image segmentation model is reduced to 0.1 times of the original learning rate, learning rate recovery verification is carried out on the skin mirror image segmentation model subjected to the learning rate adjustment by utilizing a test image sample, model training optimization is carried out by adopting an L2 regularization method, so as to generate a skin mirror image segmentation optimization model, and a corresponding skin mirror image lesion region segmentation result is output at the last layer in a lesion region segmentation network.

9. A training method for a skin mirror image segmentation model, characterized in that the method is implemented based on the training system for a skin mirror image segmentation model according to claim 1, the training method for a skin mirror image segmentation model comprising: