CN119006449B - Scoliosis recognition system based on machine learning - Google Patents
Scoliosis recognition system based on machine learning Download PDFInfo
- Publication number
- CN119006449B CN119006449B CN202411468054.0A CN202411468054A CN119006449B CN 119006449 B CN119006449 B CN 119006449B CN 202411468054 A CN202411468054 A CN 202411468054A CN 119006449 B CN119006449 B CN 119006449B
- Authority
- CN
- China
- Prior art keywords
- spine
- image
- scoliosis
- unit
- weight
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 206010039722 scoliosis Diseases 0.000 title claims abstract description 78
- 238000010801 machine learning Methods 0.000 title claims abstract description 13
- 230000011218 segmentation Effects 0.000 claims abstract description 38
- 238000003062 neural network model Methods 0.000 claims abstract description 31
- 238000003709 image segmentation Methods 0.000 claims abstract description 30
- 230000007246 mechanism Effects 0.000 claims abstract description 17
- 238000007781 pre-processing Methods 0.000 claims abstract description 17
- 239000002131 composite material Substances 0.000 claims abstract description 16
- 238000013480 data collection Methods 0.000 claims abstract description 14
- 238000010276 construction Methods 0.000 claims description 43
- 230000006870 function Effects 0.000 claims description 33
- 238000004364 calculation method Methods 0.000 claims description 27
- 238000000605 extraction Methods 0.000 claims description 21
- 238000001514 detection method Methods 0.000 claims description 18
- 238000010606 normalization Methods 0.000 claims description 15
- 238000012545 processing Methods 0.000 claims description 13
- 238000005457 optimization Methods 0.000 claims description 9
- 238000012549 training Methods 0.000 claims description 9
- 230000006872 improvement Effects 0.000 claims description 8
- 238000013527 convolutional neural network Methods 0.000 claims description 7
- 230000004913 activation Effects 0.000 claims description 6
- 230000004927 fusion Effects 0.000 claims description 3
- 238000011176 pooling Methods 0.000 claims description 3
- 230000009286 beneficial effect Effects 0.000 abstract description 5
- 238000000034 method Methods 0.000 description 15
- 230000008569 process Effects 0.000 description 9
- 230000009471 action Effects 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 230000002708 enhancing effect Effects 0.000 description 2
- 230000004075 alteration Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000000873 masking effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000000877 morphologic effect Effects 0.000 description 1
- 208000035851 morphological anomaly Diseases 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10116—X-ray image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30008—Bone
- G06T2207/30012—Spine; Backbone
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Software Systems (AREA)
- Medical Informatics (AREA)
- Computing Systems (AREA)
- Artificial Intelligence (AREA)
- Databases & Information Systems (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Molecular Biology (AREA)
- Computational Linguistics (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Radiology & Medical Imaging (AREA)
- Quality & Reliability (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a scoliosis recognition system based on machine learning, which belongs to the technical field of medical image recognition and comprises a data collection module, a data preprocessing module, a spine image segmentation module, a scoliosis recognition module and a comprehensive report generation module. The invention adopts the improved mask region convolution neural network model to segment the spine image, introduces a composite connection network, ensures that the expression capability of the model on a complex image is better, and a channel attention mechanism can better pay attention to key information in the image, inhibit irrelevant information in the background, obviously improve the accuracy of segmentation of the spine image, adopts a self-adaptive weight adjustable enhancement algorithm to identify scoliosis, introduces adjustable parameters, dynamically adjusts a weight update mechanism, improves the expression of the model on complex data, reduces the influence of a difficult-to-classify sample, thereby improving the overall stability of the model, reducing iteration times, improving the overall efficiency of the model and being beneficial to realizing efficient and accurate scoliosis identification.
Description
Technical Field
The invention belongs to the technical field of medical image recognition, and particularly relates to a scoliosis recognition system based on machine learning.
Background
The scoliosis recognition system based on machine learning is a system for automatically analyzing related medical images of the spine and recognizing spine conditions by utilizing a machine learning technology, and aims to automatically recognize scoliosis, help doctors to improve work efficiency and reduce work load.
However, in the existing scoliosis identification process, there are technical problems that the image segmentation method has lower accuracy of segmentation results when processing complex spine structures due to the fact that more irrelevant information is mixed in spine X-ray films, and the technical problems that scoliosis images of different patients have larger difference, are easy to generate observation errors by manually identifying scoliosis, and take longer time.
Disclosure of Invention
Aiming at the technical problems that in the existing scoliosis recognition process, more irrelevant information is mixed in a spine X-ray film, so that the accuracy of a segmentation result is lower when a complex spine structure is processed by an image segmentation method, the scheme creatively adopts an improved mask region convolutional neural network model to segment a spine image, introduces a composite connection network, enables the performance of the model in the complex image to be better, enables a channel attention mechanism to better pay attention to key information in the image, suppresses irrelevant information in a background, remarkably improves the accuracy of spine image segmentation, aims at the technical problems that in the existing scoliosis recognition process, the difference of spine scoliosis images of different patients is larger, observation errors are easy to generate by manually recognizing the spine scoliosis, and a longer time is required, and the scheme creatively adopts a self-adaptive weight adjustable enhancement algorithm to conduct spine scoliosis recognition, introduces adjustable parameters, dynamically adjusts an updating mechanism, improves the performance of the model on complex data, reduces the influence of difficult classification, improves the overall stability and accuracy of the model, and is beneficial to the accuracy of the overall scoliosis recognition, thereby improving the overall accuracy of the model, and realizing the accuracy of the scoliosis recognition.
The scoliosis recognition system based on machine learning comprises a data collection module, a data preprocessing module, a spine image segmentation module, a scoliosis recognition module and a comprehensive report generation module;
The data collection module is used for obtaining spine original image data and spine condition labels through data collection, sending the spine original image data to the data preprocessing module and sending the spine condition labels to the scoliosis recognition module;
the data preprocessing module performs data preprocessing through the image enhancement unit, the image adjustment unit and the normalization unit to obtain spine standard image data, and sends the spine standard image data to the spine image segmentation module;
the spine image segmentation module adopts an improved mask region convolutional neural network model to segment a spine image to obtain a spine region segmentation result, and sends the spine region segmentation result to the scoliosis recognition module and the comprehensive report generation module;
The scoliosis recognition module is used for performing scoliosis recognition by adopting a self-adaptive weight adjustable enhancement algorithm to obtain spine classification information, and sending the spine classification information to the comprehensive report generation module;
and the comprehensive report generation module is used for generating a comprehensive report by combining the spine region segmentation result and the spine classification information.
Further, in the data collection module, the data collection is used for collecting original image data required by scoliosis recognition, specifically, collecting front and rear spine X-ray films from a hospital system to obtain spine original image data and a spine condition label.
Further, in the data preprocessing module, an image enhancement unit, an image adjustment unit and a normalization unit are provided, including the following contents:
The image enhancement unit is used for adjusting the contrast, saturation and brightness of an image in the original image data of the spine, and denoising the image through a median filter to obtain the enhanced image data of the spine;
The image adjusting unit is used for adjusting the image size in the spine enhanced image data to be uniform size to obtain spine adjusted image data;
and the normalization unit is used for carrying out image normalization on the images in the spine adjustment image data to obtain spine standard image data.
Further, in the spine image segmentation module, an improved mask region convolution neural network model is adopted to segment the spine image, so that a spine region segmentation result is obtained;
The improved mask region convolution neural network model comprises an input layer, a feature extraction backbone network, a feature pyramid network, a region proposal optimization network, a candidate region alignment layer and a detection head;
The feature extraction backbone network is used for generating rich multi-level feature graphs;
The feature pyramid network is used for processing multi-scale features in the multi-level feature map;
the region proposal optimizing network is used for generating candidate regions containing the spine;
The candidate region alignment layer is used for aligning candidate regions;
the spine image segmentation module is provided with a model improvement unit, an input layer construction unit, a feature extraction main network construction unit, a feature pyramid network construction unit, a region proposal optimization network construction unit, a candidate region alignment layer construction unit, a detection head construction unit, an improved mask region convolution neural network model construction unit and an image segmentation result generation unit, and comprises the following contents:
The model improvement unit is used for introducing the composite connection network into a main network of the mask area convolution neural network model, introducing a channel attention mechanism into an area proposal network of the mask area convolution neural network model, and improving the mask area convolution neural network model to obtain an improved mask area convolution neural network model;
the input layer construction unit is used for receiving an input image, in particular receiving an image in the spine standard image data through the input layer as an input image;
The feature extraction main network construction unit is used for setting ResNet-101 networks and a composite connection network in the feature extraction main network, extracting deep features of an input image through the feature extraction main network to obtain a multi-level feature map, wherein the calculation formula is as follows:
;
Wherein Ig res is a ResNet-101 network output feature map, f ResNet-101 (-) is a ResNet-101 network computing function, ig input is an input image, ig cb is a multi-level feature map, specifically a composite connection network output feature map, and f CB-Net (-) is a composite connection network computing function;
The feature pyramid network construction unit is used for carrying out multi-scale fusion on the multi-level feature images through the feature pyramid network to obtain the multi-scale feature images, wherein the calculation formula is as follows:
;
Wherein Igp is a multi-scale feature map, and f fpn (DEG) is a feature pyramid network computing function;
The area proposal optimizing network construction unit comprises the following contents:
The key channel information in the multi-scale feature map is enhanced by using a channel attention mechanism, specifically, each channel in the multi-scale feature map is subjected to global average pooling, then the channel attention weight is learned through a full-connection layer, and then the multi-scale feature map is weighted by using the channel attention weight to generate the channel enhanced feature map, wherein the calculation formula is as follows:
;
Where z c is the global average of the c-th channel in the multi-scale feature map, c is the channel index, H is the multi-scale feature map height, i is the multi-scale feature map vertical direction index, W is the multi-scale feature map width, j is the multi-scale feature map horizontal direction index, Is the pixel value of the c-th channel in the multi-scale feature map at the position (i, j), W c is the attention weight of the c-th channel in the multi-scale feature map, sig (·) is an S-type activation function, W one is the first weight of the full-connection layer, reLU (·) is the ReLU activation function, W two is the second weight of the full-connection layer, pic c is the feature value of the c-th channel in the channel enhancement feature map, igp c is the feature value of the c-th channel in the multi-scale feature map;
Processing the channel enhancement feature map through a region proposal network to generate candidate regions;
The candidate region alignment layer construction unit aligns the candidate regions through the candidate region alignment layer to obtain candidate alignment regions;
The detection head construction unit is used for setting segmentation, classification and frame regression tasks in the detection head, and processing the candidate alignment area through the detection head to obtain a final segmentation result;
The improved mask region convolution neural network model building unit is used for building an improved mask region convolution neural network model through an input layer building unit, a feature extraction main network building unit, a feature pyramid network building unit, a region proposal optimization network building unit, a candidate region alignment layer building unit and a detection head building unit;
And the image segmentation result generating unit is used for carrying out image segmentation through the improved mask region convolution neural network model to obtain a spine region segmentation result.
Further, in the scoliosis recognition module, according to the spine region segmentation result, a self-adaptive weight adjustable enhancement algorithm is adopted to perform scoliosis recognition, so as to obtain spine classification information;
The scoliosis recognition module is provided with a self-adaptive enhancement algorithm optimizing unit, a self-adaptive weight adjustable enhancement algorithm constructing unit and a scoliosis recognition result generating unit, and comprises the following contents:
The self-adaptive enhancement algorithm optimizing unit is used for enhancing the flexibility of the algorithm and improving the adaptability to complex data, specifically, an adjustable parameter is introduced into the self-adaptive enhancement algorithm to adjust a sample weight updating mechanism, and the self-adaptive enhancement algorithm is optimized to obtain a self-adaptive weight adjustable enhancement algorithm;
the self-adaptive weight adjustable enhancement algorithm construction unit comprises the following contents:
Initializing sample weights, namely taking a spine region segmentation result as an input sample, and equally setting the sample weights, wherein a calculation formula is as follows:
;
In the formula, Is the initial sample weight, specifically the weight of the kth input sample in the 1 st iteration, k is the input sample index, and N is the number of input samples;
Training the weak classifier, specifically, training the weak classifier once by using the current sample weight, and calculating a classification error, wherein the calculation formula is as follows:
;
Where err m is the classification error of the mth weak classifier, m is the weak classifier iteration common index, Is the weight of the kth input sample in the mth iteration, I (·) is the indicator function whenWhen the indication function takes a value of 1, otherwise, the indication function takes a value of 0,G m (·) which is a weak classifier prediction function, x k is the kth input sample, y k is a true label of the kth input sample, and the true label specifically refers to a spine condition label;
the weak classifier weight is calculated, specifically, the weak classifier weight is calculated according to the classification error and the adjustable parameter, and the calculation formula is as follows:
;
where A m is the mth weak classifier weight, Is an adjustable parameter, log (·) is a logarithmic function;
Updating the sample weight, specifically controlling the sample weight updating amplitude through adjustable parameters, and updating the sample weight, wherein the calculation formula is as follows:
;
In the formula, The updated sample weight is specifically the weight of the kth input sample in the (m+1) th iteration, Z m is a normalization factor, and e is a natural constant;
Constructing a weak classifier, specifically, constructing the weak classifier by repeatedly executing the initialization sample weight, the training weak classifier, the calculation weak classifier weight and the updating sample weight until the maximum iteration number is reached;
Constructing a combined classifier, namely, carrying out weighted linear combination on all weak classifiers to obtain the combined classifier, wherein the calculation formula is as follows:
;
wherein Q (·) is a combined classifier predictive function, M is the number of weak classifiers equal to the number of iterations;
And the scoliosis recognition result generating unit is used for performing scoliosis recognition by using the combined classifier to obtain the spine classification information.
Further, in the integrated report generation module, a scoliosis identification integrated report is generated by combining the spine region segmentation result and the spine classification information, and treatment advice is provided.
By adopting the scheme, the beneficial effects obtained by the invention are as follows:
(1) Aiming at the technical problem that the accuracy of the segmentation result is lower when the image segmentation method is used for processing a complex spine structure due to the fact that more irrelevant information is mixed in a spine X-ray film in the existing scoliosis recognition process, the improved mask region convolution neural network model is creatively adopted to segment the spine image, and a composite connection network is introduced, so that the expressive capacity of the model in the complex image is better, and a channel attention mechanism can pay attention to key information in the image better, inhibit irrelevant information in the background and remarkably improve the accuracy of segmentation of the spine image;
(2) Aiming at the technical problems that in the existing scoliosis recognition process, scoliosis images of different patients are large in difference, the scoliosis is easily observed by manually recognizing the scoliosis, and long time is required to be consumed, the technical scheme creatively adopts a self-adaptive weight adjustable enhancement algorithm to recognize the scoliosis, introduces adjustable parameters, dynamically adjusts a weight updating mechanism, improves the expression of a model on complex data, reduces the influence of a sample difficult to classify, and accordingly improves the overall stability and accuracy of the model, reduces the iteration times, improves the overall efficiency of the model, and is beneficial to realizing efficient and accurate scoliosis recognition.
Drawings
FIG. 1 is a schematic flow chart of a scoliosis recognition system based on machine learning provided by the invention;
FIG. 2 is a schematic flow diagram of a data preprocessing module;
FIG. 3 is a flow diagram of a spine image segmentation module;
fig. 4 is a flow diagram of a scoliosis identification module.
The accompanying drawings are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate the invention and together with the embodiments of the invention, serve to explain the invention.
Detailed Description
The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention, and it is apparent that the described embodiments are only some embodiments of the present invention, but not all embodiments, and all other embodiments obtained by those skilled in the art without making any inventive effort based on the embodiments of the present invention are within the scope of protection of the present invention.
In the description of the present invention, it should be understood that the terms "upper," "lower," "front," "rear," "left," "right," "top," "bottom," "inner," "outer," and the like indicate orientation or positional relationships based on those shown in the drawings, merely to facilitate description of the invention and simplify the description, and do not indicate or imply that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, and thus should not be construed as limiting the invention.
Referring to fig. 1, the scoliosis recognition system based on machine learning provided by the invention comprises a data collection module, a data preprocessing module, a spine image segmentation module, a scoliosis recognition module and a comprehensive report generation module;
The data collection module is used for obtaining spine original image data and spine condition labels through data collection, sending the spine original image data to the data preprocessing module and sending the spine condition labels to the scoliosis recognition module;
the data preprocessing module performs data preprocessing through the image enhancement unit, the image adjustment unit and the normalization unit to obtain spine standard image data, and sends the spine standard image data to the spine image segmentation module;
the spine image segmentation module adopts an improved mask region convolutional neural network model to segment a spine image to obtain a spine region segmentation result, and sends the spine region segmentation result to the scoliosis recognition module and the comprehensive report generation module;
The scoliosis recognition module is used for performing scoliosis recognition by adopting a self-adaptive weight adjustable enhancement algorithm to obtain spine classification information, and sending the spine classification information to the comprehensive report generation module;
and the comprehensive report generation module is used for generating a comprehensive report by combining the spine region segmentation result and the spine classification information.
Referring to fig. 1, the second embodiment is based on the above embodiment, and in the data collecting module, the data collecting module is configured to collect original image data required for scoliosis identification, specifically, collect front-back spine X-ray films from a hospital system, to obtain original image data of a spine and a spine status tag;
The spine condition label comprises a normal spine, a C-shaped lateral curve and an S-shaped lateral curve.
An embodiment III, referring to FIG. 1 and FIG. 2, is based on the above embodiment, in the data preprocessing module, there are provided an image enhancement unit, an image adjustment unit and a normalization unit, including the following:
The image enhancement unit is used for adjusting the contrast, saturation and brightness of an image in the original image data of the spine, and denoising the image through a median filter to obtain the enhanced image data of the spine;
The image adjusting unit is used for adjusting the image size in the spine enhanced image data to be uniform size to obtain spine adjusted image data;
and the normalization unit is used for carrying out image normalization on the images in the spine adjustment image data to obtain spine standard image data.
In a fourth embodiment, referring to fig. 1 and 3, in the spine image segmentation module, an improved mask region convolutional neural network model is adopted to segment a spine image, so as to obtain a spine region segmentation result;
The improved mask region convolution neural network model comprises an input layer, a feature extraction backbone network, a feature pyramid network, a region proposal optimization network, a candidate region alignment layer and a detection head;
The feature extraction backbone network is used for generating rich multi-level feature graphs;
The feature pyramid network is used for processing multi-scale features in the multi-level feature map;
the region proposal optimizing network is used for generating candidate regions containing the spine;
The candidate region alignment layer is used for aligning candidate regions;
the spine image segmentation module is provided with a model improvement unit, an input layer construction unit, a feature extraction main network construction unit, a feature pyramid network construction unit, a region proposal optimization network construction unit, a candidate region alignment layer construction unit, a detection head construction unit, an improved mask region convolution neural network model construction unit and an image segmentation result generation unit, and comprises the following contents:
The model improvement unit is used for introducing the composite connection network into a main network of the mask area convolution neural network model, introducing a channel attention mechanism into an area proposal network of the mask area convolution neural network model, and improving the mask area convolution neural network model to obtain an improved mask area convolution neural network model;
the input layer construction unit is used for receiving an input image, in particular receiving an image in the spine standard image data through the input layer as an input image;
The feature extraction main network construction unit is used for setting ResNet-101 networks and a composite connection network in the feature extraction main network, extracting deep features of an input image through the feature extraction main network to obtain a multi-level feature map, wherein the calculation formula is as follows:
;
Wherein Ig res is a ResNet-101 network output feature map, f ResNet-101 (-) is a ResNet-101 network computing function, ig input is an input image, ig cb is a multi-level feature map, specifically a composite connection network output feature map, and f CB-Net (-) is a composite connection network computing function;
The feature pyramid network construction unit is used for carrying out multi-scale fusion on the multi-level feature images through the feature pyramid network to obtain the multi-scale feature images, wherein the calculation formula is as follows:
;
Wherein Igp is a multi-scale feature map, and f fpn (DEG) is a feature pyramid network computing function;
The area proposal optimizing network construction unit comprises the following contents:
The key channel information in the multi-scale feature map is enhanced by using a channel attention mechanism, specifically, each channel in the multi-scale feature map is subjected to global average pooling, then the channel attention weight is learned through a full-connection layer, and then the multi-scale feature map is weighted by using the channel attention weight to generate the channel enhanced feature map, wherein the calculation formula is as follows:
;
Where z c is the global average of the c-th channel in the multi-scale feature map, c is the channel index, H is the multi-scale feature map height, i is the multi-scale feature map vertical direction index, W is the multi-scale feature map width, j is the multi-scale feature map horizontal direction index, Is the pixel value of the c-th channel in the multi-scale feature map at the position (i, j), W c is the attention weight of the c-th channel in the multi-scale feature map, sig (·) is an S-type activation function, W one is the first weight of the full-connection layer, reLU (·) is the ReLU activation function, W two is the second weight of the full-connection layer, pic c is the feature value of the c-th channel in the channel enhancement feature map, igp c is the feature value of the c-th channel in the multi-scale feature map;
Processing the channel enhancement feature map through a region proposal network to generate candidate regions;
The candidate region alignment layer construction unit aligns the candidate regions through the candidate region alignment layer to obtain candidate alignment regions;
The detection head construction unit is used for setting segmentation, classification and frame regression tasks in the detection head, and processing the candidate alignment area through the detection head to obtain a final segmentation result;
The improved mask region convolution neural network model building unit is used for building an improved mask region convolution neural network model through an input layer building unit, a feature extraction main network building unit, a feature pyramid network building unit, a region proposal optimization network building unit, a candidate region alignment layer building unit and a detection head building unit;
The image segmentation result generating unit is used for carrying out image segmentation through the improved mask region convolution neural network model to obtain a spine region segmentation result;
The spine region segmentation result comprises a spine region segmentation mask, a boundary box and a category label;
the category labels comprise morphological normals and morphological anomalies;
table 1 is a comparison of model performance before and after the improvement of the mask area convolutional neural network model in the fourth embodiment of the present invention, for example, a table, an FPS is used to represent the number of images processed per second, and a higher FPS represents a faster model processing speed;
Table 1 model performance comparison of masking region convolutional neural network models before and after improvement
By executing the operation, aiming at the technical problem that the accuracy of the segmentation result is lower when the image segmentation method is used for processing a complex spine structure due to the fact that more irrelevant information is mixed in the spine X-ray film in the existing scoliosis recognition process, the improved mask region convolution neural network model is creatively adopted to segment the spine image, and a composite connection network is introduced, so that the performance of the model in the complex image is better, a channel attention mechanism can pay attention to key information in the image better, the irrelevant information in the background is restrained, and the accuracy of segmentation of the spine image is remarkably improved.
Fifth, referring to fig. 1 and fig. 4, in the scoliosis identification module, according to the spine region segmentation result, a self-adaptive weight adjustable enhancement algorithm is adopted to perform scoliosis identification to obtain spine classification information;
The scoliosis recognition module is provided with a self-adaptive enhancement algorithm optimizing unit, a self-adaptive weight adjustable enhancement algorithm constructing unit and a scoliosis recognition result generating unit, and comprises the following contents:
The self-adaptive enhancement algorithm optimizing unit is used for enhancing the flexibility of the algorithm and improving the adaptability to complex data, specifically, an adjustable parameter is introduced into the self-adaptive enhancement algorithm to adjust a sample weight updating mechanism, and the self-adaptive enhancement algorithm is optimized to obtain a self-adaptive weight adjustable enhancement algorithm;
the self-adaptive weight adjustable enhancement algorithm construction unit comprises the following contents:
Initializing sample weights, namely taking a spine region segmentation result as an input sample, and equally setting the sample weights, wherein a calculation formula is as follows:
;
In the formula, Is the initial sample weight, specifically the weight of the kth input sample in the 1 st iteration, k is the input sample index, and N is the number of input samples;
Training the weak classifier, specifically, training the weak classifier once by using the current sample weight, and calculating a classification error, wherein the calculation formula is as follows:
;
Where err m is the classification error of the mth weak classifier, m is the weak classifier iteration common index, Is the weight of the kth input sample in the mth iteration, I (·) is the indicator function whenWhen the indication function takes a value of 1, otherwise, the indication function takes a value of 0,G m (·) which is a weak classifier prediction function, x k is the kth input sample, y k is a true label of the kth input sample, and the true label specifically refers to a spine condition label;
the weak classifier weight is calculated, specifically, the weak classifier weight is calculated according to the classification error and the adjustable parameter, and the calculation formula is as follows:
;
where A m is the mth weak classifier weight, Is an adjustable parameter, log (·) is a logarithmic function;
Updating the sample weight, specifically controlling the sample weight updating amplitude through adjustable parameters, and updating the sample weight, wherein the calculation formula is as follows:
;
In the formula, The updated sample weight is specifically the weight of the kth input sample in the (m+1) th iteration, Z m is a normalization factor, and e is a natural constant;
Constructing a weak classifier, specifically, constructing the weak classifier by repeatedly executing the initialization sample weight, the training weak classifier, the calculation weak classifier weight and the updating sample weight until the maximum iteration number is reached;
Constructing a combined classifier, namely, carrying out weighted linear combination on all weak classifiers to obtain the combined classifier, wherein the calculation formula is as follows:
;
wherein Q (·) is a combined classifier predictive function, M is the number of weak classifiers equal to the number of iterations;
The scoliosis recognition result generating unit is used for performing scoliosis recognition by using the combined classifier to obtain spine classification information;
By executing the operation, aiming at the technical problems that in the existing scoliosis recognition process, the scoliosis images of different patients have larger difference, the scoliosis is easily observed by manually recognizing the scoliosis, and longer time is required to be consumed, the technical scheme creatively adopts the self-adaptive weight adjustable enhancement algorithm to recognize the scoliosis, introduces adjustable parameters, dynamically adjusts the weight updating mechanism, improves the expression of the model on complex data, reduces the influence of difficult-classification samples, thereby improving the overall stability and accuracy of the model, reducing the iteration times, improving the overall efficiency of the model, and being beneficial to realizing efficient and accurate scoliosis recognition.
Embodiment six, referring to fig. 1, based on the above embodiment, in the integrated report generation module, a scoliosis identification integrated report is generated by combining the spine region segmentation result and the spine classification information, and a treatment suggestion is provided.
It is noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus.
Although embodiments of the present invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made hereto without departing from the spirit and principles of the present invention.
The invention and its embodiments have been described above with no limitation, and the actual construction is not limited to the embodiments of the invention as shown in the drawings. In summary, if one of ordinary skill in the art is informed by this disclosure, a structural manner and an embodiment similar to the technical solution should not be creatively devised without departing from the gist of the present invention.
Claims (4)
1. The scoliosis recognition system based on machine learning is characterized by comprising a data collection module, a data preprocessing module, a spine image segmentation module, a scoliosis recognition module and a comprehensive report generation module;
The data collection module is used for obtaining spine original image data and spine condition labels through data collection, sending the spine original image data to the data preprocessing module and sending the spine condition labels to the scoliosis recognition module;
the data preprocessing module performs data preprocessing through the image enhancement unit, the image adjustment unit and the normalization unit to obtain spine standard image data, and sends the spine standard image data to the spine image segmentation module;
the spine image segmentation module adopts an improved mask region convolutional neural network model to segment a spine image to obtain a spine region segmentation result, and sends the spine region segmentation result to the scoliosis recognition module and the comprehensive report generation module;
The improved mask region convolution neural network model comprises an input layer, a feature extraction backbone network, a feature pyramid network, a region proposal optimization network, a candidate region alignment layer and a detection head;
The feature extraction backbone network is used for generating rich multi-level feature graphs;
The feature pyramid network is used for processing multi-scale features in the multi-level feature map;
the region proposal optimizing network is used for generating candidate regions containing the spine;
The candidate region alignment layer is used for aligning candidate regions;
the spine image segmentation module is provided with a model improvement unit, an input layer construction unit, a feature extraction main network construction unit, a feature pyramid network construction unit, a region proposal optimization network construction unit, a candidate region alignment layer construction unit, a detection head construction unit, an improved mask region convolution neural network model construction unit and an image segmentation result generation unit, and comprises the following contents:
The model improvement unit is used for introducing the composite connection network into a main network of the mask area convolution neural network model, introducing a channel attention mechanism into an area proposal network of the mask area convolution neural network model, and improving the mask area convolution neural network model to obtain an improved mask area convolution neural network model;
the input layer construction unit is used for receiving an input image, in particular receiving an image in the spine standard image data through the input layer as an input image;
The feature extraction main network construction unit is used for setting ResNet-101 networks and a composite connection network in the feature extraction main network, extracting deep features of an input image through the feature extraction main network to obtain a multi-level feature map, wherein the calculation formula is as follows:
;
Wherein Ig res is a ResNet-101 network output feature map, f ResNet-101 (-) is a ResNet-101 network computing function, ig input is an input image, ig cb is a multi-level feature map, specifically a composite connection network output feature map, and f CB-Net (-) is a composite connection network computing function;
The feature pyramid network construction unit is used for carrying out multi-scale fusion on the multi-level feature images through the feature pyramid network to obtain the multi-scale feature images, wherein the calculation formula is as follows:
;
Wherein Igp is a multi-scale feature map, and f fpn (DEG) is a feature pyramid network computing function;
The area proposal optimizing network construction unit comprises the following contents:
The key channel information in the multi-scale feature map is enhanced by using a channel attention mechanism, specifically, each channel in the multi-scale feature map is subjected to global average pooling, then the channel attention weight is learned through a full-connection layer, and then the multi-scale feature map is weighted by using the channel attention weight to generate the channel enhanced feature map, wherein the calculation formula is as follows:
;
Where z c is the global average of the c-th channel in the multi-scale feature map, c is the channel index, H is the multi-scale feature map height, i is the multi-scale feature map vertical direction index, W is the multi-scale feature map width, j is the multi-scale feature map horizontal direction index, Is the pixel value of the c-th channel in the multi-scale feature map at the position (i, j), W c is the attention weight of the c-th channel in the multi-scale feature map, sig (·) is an S-type activation function, W one is the first weight of the full-connection layer, reLU (·) is the ReLU activation function, W two is the second weight of the full-connection layer, pic c is the feature value of the c-th channel in the channel enhancement feature map, igp c is the feature value of the c-th channel in the multi-scale feature map;
Processing the channel enhancement feature map through a region proposal network to generate candidate regions;
The candidate region alignment layer construction unit aligns the candidate regions through the candidate region alignment layer to obtain candidate alignment regions;
The detection head construction unit is used for setting segmentation, classification and frame regression tasks in the detection head, and processing the candidate alignment area through the detection head to obtain a final segmentation result;
The improved mask region convolution neural network model building unit is used for building an improved mask region convolution neural network model through an input layer building unit, a feature extraction main network building unit, a feature pyramid network building unit, a region proposal optimization network building unit, a candidate region alignment layer building unit and a detection head building unit;
The image segmentation result generating unit is used for carrying out image segmentation through the improved mask region convolution neural network model to obtain a spine region segmentation result;
the scoliosis recognition module is used for performing scoliosis recognition by adopting a self-adaptive weight adjustable enhancement algorithm according to the spine region segmentation result to obtain spine classification information, and sending the spine classification information to the comprehensive report generation module;
The scoliosis recognition module is provided with a self-adaptive enhancement algorithm optimizing unit, a self-adaptive weight adjustable enhancement algorithm constructing unit and a scoliosis recognition result generating unit, and comprises the following contents:
The self-adaptive enhancement algorithm optimizing unit is used for adjusting a sample weight updating mechanism by introducing adjustable parameters into the self-adaptive enhancement algorithm, and optimizing the self-adaptive enhancement algorithm to obtain the self-adaptive weight adjustable enhancement algorithm;
the self-adaptive weight adjustable enhancement algorithm construction unit comprises the following contents:
Initializing sample weights, namely taking a spine region segmentation result as an input sample, and equally setting the sample weights, wherein a calculation formula is as follows:
;
In the formula, Is the initial sample weight, specifically the weight of the kth input sample in the 1 st iteration, k is the input sample index, and N is the number of input samples;
Training the weak classifier, specifically, training the weak classifier once by using the current sample weight, and calculating a classification error, wherein the calculation formula is as follows:
;
Where err m is the classification error of the mth weak classifier, m is the weak classifier iteration common index, Is the weight of the kth input sample in the mth iteration, I (·) is the indicator function whenWhen the indication function takes a value of 1, otherwise, the indication function takes a value of 0,G m (·) which is a weak classifier prediction function, x k is the kth input sample, y k is a true label of the kth input sample, and the true label specifically refers to a spine condition label;
the weak classifier weight is calculated, specifically, the weak classifier weight is calculated according to the classification error and the adjustable parameter, and the calculation formula is as follows:
;
where A m is the mth weak classifier weight, Is an adjustable parameter, log (·) is a logarithmic function;
Updating the sample weight, specifically controlling the sample weight updating amplitude through adjustable parameters, and updating the sample weight, wherein the calculation formula is as follows:
;
In the formula, The updated sample weight is specifically the weight of the kth input sample in the (m+1) th iteration, Z m is a normalization factor, and e is a natural constant;
Constructing a weak classifier, specifically, constructing the weak classifier by repeatedly executing the initialization sample weight, the training weak classifier, the calculation weak classifier weight and the updating sample weight until the maximum iteration number is reached;
Constructing a combined classifier, namely, carrying out weighted linear combination on all weak classifiers to obtain the combined classifier, wherein the calculation formula is as follows:
;
wherein Q (·) is a combined classifier predictive function, M is the number of weak classifiers equal to the number of iterations;
The scoliosis recognition result generating unit is used for performing scoliosis recognition by using the combined classifier to obtain spine classification information;
and the comprehensive report generation module is used for generating a comprehensive report by combining the spine region segmentation result and the spine classification information.
2. The scoliosis recognition system based on machine learning according to claim 1, wherein the data preprocessing module is provided with an image enhancement unit, an image adjustment unit and a normalization unit, and the system comprises the following contents:
The image enhancement unit is used for adjusting the contrast, saturation and brightness of an image in the original image data of the spine, and denoising the image through a median filter to obtain the enhanced image data of the spine;
The image adjusting unit is used for adjusting the image size in the spine enhanced image data to be uniform size to obtain spine adjusted image data;
and the normalization unit is used for carrying out image normalization on the images in the spine adjustment image data to obtain spine standard image data.
3. The scoliosis recognition system based on machine learning of claim 2, wherein in the data collection module, the data collection is used for collecting original image data required by scoliosis recognition, in particular to collecting front and back spine X-ray films from a hospital system to obtain spine original image data and a spine condition label.
4. The machine-learning based scoliosis recognition system of claim 3 wherein the integrated report generation module combines the segmentation results of the spinal region and the classification information of the spinal column to generate a scoliosis recognition integrated report and provide treatment advice.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202411468054.0A CN119006449B (en) | 2024-10-21 | 2024-10-21 | Scoliosis recognition system based on machine learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202411468054.0A CN119006449B (en) | 2024-10-21 | 2024-10-21 | Scoliosis recognition system based on machine learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN119006449A CN119006449A (en) | 2024-11-22 |
| CN119006449B true CN119006449B (en) | 2024-12-31 |
Family
ID=93474697
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202411468054.0A Active CN119006449B (en) | 2024-10-21 | 2024-10-21 | Scoliosis recognition system based on machine learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN119006449B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN119338833B (en) * | 2024-12-24 | 2025-03-11 | 浙江师范大学 | Identification method and system for scoliosis aiming at gait |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114287915A (en) * | 2021-12-28 | 2022-04-08 | 深圳零动医疗科技有限公司 | Noninvasive scoliosis screening method and system based on back color image |
| CN114742751A (en) * | 2022-03-01 | 2022-07-12 | 深圳视见医疗科技有限公司 | Scoliosis detection method and device, terminal device and readable storage medium |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US12100115B2 (en) * | 2021-08-10 | 2024-09-24 | GE Precision Healthcare LLC | System and method for cohesive multi-regional functional-anatomical medical image registration |
| CN118334334A (en) * | 2024-04-12 | 2024-07-12 | 刘晶 | Spine vertebra CT image segmentation method and system based on mixed attention |
-
2024
- 2024-10-21 CN CN202411468054.0A patent/CN119006449B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114287915A (en) * | 2021-12-28 | 2022-04-08 | 深圳零动医疗科技有限公司 | Noninvasive scoliosis screening method and system based on back color image |
| CN114742751A (en) * | 2022-03-01 | 2022-07-12 | 深圳视见医疗科技有限公司 | Scoliosis detection method and device, terminal device and readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN119006449A (en) | 2024-11-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112069874B (en) | Method, system, device and storage medium for identifying cells in embryonic light microscope images | |
| CN119006449B (en) | Scoliosis recognition system based on machine learning | |
| CN111882560B (en) | Lung parenchyma CT image segmentation method based on weighted full convolution neural network | |
| CN113052795B (en) | X-ray chest radiography image quality determination method and device | |
| CN112802019B (en) | Leke typing method based on spine AIS image | |
| CN108320306B (en) | Video target tracking method fusing TLD and KCF | |
| CN112699868A (en) | Image identification method and device based on deep convolutional neural network | |
| CN111598875A (en) | Method, system and device for building thyroid nodule automatic detection model | |
| CN111160407A (en) | Deep learning target detection method and system | |
| CN112053325A (en) | Breast mass image processing and classifying system | |
| CN104063713A (en) | Semi-autonomous on-line studying method based on random fern classifier | |
| CN114170212A (en) | A method and system for false-positive detection of small pulmonary nodules in CT images | |
| CN111091101A (en) | High-precision pedestrian detection method, system and device based on one-step method | |
| CN116959099A (en) | Abnormal behavior identification method based on space-time diagram convolutional neural network | |
| CN111667469B (en) | Lung disease classification method, device and equipment | |
| CN119400433B (en) | Postpartum hemorrhage risk assessment system based on medical information processing | |
| CN115908298B (en) | Target prediction method, model and storage medium for polyp in endoscopic image | |
| CN113139928B (en) | Training method of lung nodule detection model and lung nodule detection method | |
| CN112288765A (en) | Image processing method for vehicle-mounted infrared pedestrian detection and tracking | |
| CN113435469A (en) | Kidney tumor enhanced CT image automatic identification system based on deep learning and training method thereof | |
| CN111899214A (en) | Pathological section scanning analysis device and pathological section scanning method | |
| CN116403074A (en) | Semi-automatic image labeling method and device based on active labeling | |
| CN114648094A (en) | Machine learning apparatus and method | |
| CN114049629A (en) | Bamboo fiber bundle identification method and device and electronic equipment | |
| CN119941731B (en) | Lung nodule analysis method, system, equipment and medium based on large model |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |