Summary of the invention
In view of this, the present invention provides a kind of medical image abnormal area dividing method, include the following steps:
Obtain medical image;
The medical image is divided into multiple image to be classified blocks, and using the first machine learning model respectively to each
The image to be classified block is classified, and classification results are normal image block or abnormal image block;
It is based respectively on the abnormal image block and takes image block to be split in the medical image, and utilize the second engineering
It practises model and semantic segmentation is carried out to each image block to be split respectively, segmentation result is extraordinary image vegetarian refreshments;
The extraordinary image vegetarian refreshments is marked in the medical image.
Optionally, the adjacent image to be classified block overlaps.
Optionally, the medical image is divided into multiple image to be classified blocks, comprising:
The medical image is traversed using the first sliding window for being sized and obtains image to be classified block, in ergodic process
Glide direction include horizontal direction and vertical direction, the sliding step in two directions is respectively less than the sliding window
Length and width.
Optionally, first machine learning model exports bianry image block, for indicating the normal picture block and institute
State abnormal image block.
Optionally, it is based respectively on the abnormal image block and takes image block to be split in the medical image, comprising:
Identify the abnormal communication region being made of abnormal image block described at least one;
The second image block to be split being sized is taken in the medical image centered on the abnormal communication region.
Optionally, second machine learning model output is that the extraordinary image vegetarian refreshments at least one channel covers figure, wherein
Including at least one extraordinary image vegetarian refreshments.
Optionally, second machine learning model output is that the extraordinary image vegetarian refreshments in two channels covers figure, including extremely
Few two kinds of extraordinary image vegetarian refreshments.
Optionally, the extraordinary image vegetarian refreshments is marked in the medical image, comprising:
Corresponding semantic segmentation result is marked in each image block to be split respectively;
Whole annotation results are mapped back in the medical image.
Optionally, the medical image is eye fundus image, the method be used to divide blutpunkte in eye fundus image and/or
Exudation point.
Correspondingly, the present invention also provides a kind of medical image abnormal area splitting equipments, comprising: at least one processor with
And the memory being connect at least one described processor communication;Wherein, the memory be stored with can by it is described at least one
The instruction that processor executes, described instruction are executed by least one described processor, so that at least one described processor executes
Above-mentioned medical image abnormal area dividing method.
Medical image is divided into multiple wait divide by the medical image abnormal area dividing method provided according to the present invention first
Class image block, and using a machine learning model respectively to each image to be classified block classify determining normal picture block and
Abnormal image block realizes the coarse positioning to abnormal position, background content is excluded, so that can be no longer to back in subsequent processes
Scape content carries out identifying processing, improves recognition efficiency;Then image to be split is obtained in medical image according to abnormal image block
Block, includes extraordinary image vegetarian refreshments in these image blocks to be split, and accounting of the extraordinary image vegetarian refreshments in image block to be split is much big
In the accounting in whole medical image, segmented image block is treated using another machine learning model on this basis and carries out language
Justice segmentation, finds extraordinary image vegetarian refreshments, it is possible thereby to improve the accuracy to abnormal area identification.
Specific embodiment
Technical solution of the present invention is clearly and completely described below in conjunction with attached drawing, it is clear that described implementation
Example is a part of the embodiment of the present invention, instead of all the embodiments.Based on the embodiments of the present invention, ordinary skill
Personnel's every other embodiment obtained without making creative work, shall fall within the protection scope of the present invention.
In the description of the present invention, it should be noted that term " center ", "upper", "lower", "left", "right", "vertical",
The orientation or positional relationship of the instructions such as "horizontal", "inner", "outside" be based on the orientation or positional relationship shown in the drawings, merely to
Convenient for description the present invention and simplify description, rather than the device or element of indication or suggestion meaning must have a particular orientation,
It is constructed and operated in a specific orientation, therefore is not considered as limiting the invention.In addition, term " first ", " second " are only
For descriptive purposes, it is not understood to indicate or imply relative importance.
As long as in addition, the non-structure each other of technical characteristic involved in invention described below different embodiments
It can be combined with each other at conflict.
The embodiment of the invention provides a kind of medical image abnormal area dividing method, this method can be by computer kimonos
The electronic equipments such as business device execute.Machine learning model has been used to identify image in the method, the machine learning model can be with
It is the neural network of multiple types and structure.This method comprises the following steps as shown in Figure 1:
S1A obtains medical image.In one embodiment, acquisition is single channel image, such as CT (Computed
Tomography, CT scan) image, ultrasound examination image etc.;In another embodiment, acquisition
It is multichannel image, e.g. fundus photograph.
Medical image can be the original image acquired by medical detection device, is also possible to by processing or treated
Image.
Medical image is divided into multiple image to be classified blocks by S2A, and using the first machine learning model respectively to each
Image to be classified block is classified, and classification results are normal image block or abnormal image block.The size of image to be classified block can
It is set according to the size of medical image, for most cases, the size of image to be classified block should be significantly less than entire doctor
Treat the size of image.Such as the size of medical image is 1000*1000 (pixel), the size of the image to be classified block marked off
It can be 100*100 (pixel).
The size of different image to be classified blocks can be same or different.Adjacent image to be classified block boundary can
To be immediately adjacent to each other, can also overlap.
Sample data should be used to be trained it before using the first machine learning model, and to have it certain
Classification capacity.Specifically can be used largely be known as normal image block and be known as abnormal image block to initial model into
Row training, training data include abnormal and normal two classifications, model, that is, distinguishable input image block category after training
In normal class or exception class.
The first machine learning model in this step only needs to carry out image block two classification, for an image to be classified
Block output is one in both normal or abnormal classification results.Do not include any lesion content in normal picture block, it is different
Including in normal image block at least partly may be the content of lesion, but the first machine learning model need not identify specific lesion position
It sets, need not also identify lesion type.Most of background content can be removed after this step is classified.
The output of first machine learning model is for expressing the information of image to be classified block type, such as can be image, retouches
State information etc..
S3A is based respectively on abnormal image block and takes image block to be split in medical image, and utilizes the second machine learning mould
Type carries out semantic segmentation to each image block to be split respectively, and segmentation result is extraordinary image vegetarian refreshments.Abnormal image block and to be split
The size of image block can be it is same or different, such as can medical image content directly to abnormal image block carry out language
Justice segmentation, further determines that abnormal pixel therein;Based on the abnormal image block that one or more can also be connected to really
Fixed one takes figure range comprising these abnormal image blocks, then takes figure range to take image to be split in medical image with this
Block includes that picture material corresponding with abnormal image block and normal picture block are corresponding in the image block to be split obtained in this way
Picture material.
For a medical image, multiple image blocks to be split may be obtained, these image blocks to be split there can be weight
Folded part.
Sample data should be used to be trained it before using the second machine learning model, and to have it certain
Semantic segmentation ability, the specific sample image block training that handmarking's abnormal area can be used obtain.The step in
Two machine learning models can identify the position of extraordinary image vegetarian refreshments, and abnormal type.The output of second machine learning model is used for
The information of abnormal expression pixel, e.g. various colors are covered the intuitive image information such as figure, lines, wire frame, are also possible to
The descriptive information such as pixel coordinate, type.
Abnormal particular content is depending on different types of medical image, such as CT image, can be extremely lump,
Canceration area, hyperplastic tissue etc.;For fundus photograph, blutpunkte, exudation point etc. can be extremely.
S4A marks abnormal pixel point in medical image.The extraordinary image vegetarian refreshments label of all image blocks to be split is being cured
The all position of exception and type can be shown by treating in image.There is the case where overlapping for multiple image blocks to be split can be with
Union is taken, pixel is divided into exception in any one image block to be split, then marks the pixel in medical image.Mark
There are many forms, and various colors can be used for example covers icon note extraordinary image vegetarian refreshments, and it is different that lines, wire frame mark also can be used
Normal pixel etc. form.
Medical image, is divided into more by the medical image abnormal area dividing method provided according to embodiments of the present invention first
A image to be classified block, and determining normogram of classifying is carried out to each image to be classified block respectively using a machine learning model
As block and abnormal image block, the coarse positioning to abnormal position is realized, background content is excluded, so that can not in subsequent processes
Identifying processing is carried out to background content again, improves recognition efficiency;Then it is obtained in medical image according to abnormal image block wait divide
Image block is cut, includes extraordinary image vegetarian refreshments in these image blocks to be split, accounting of the extraordinary image vegetarian refreshments in image block to be split is wanted
The accounting being far longer than in whole medical image, treats segmented image block using another machine learning model on this basis
Semantic segmentation is carried out, extraordinary image vegetarian refreshments is found, it is possible thereby to improve the accuracy to abnormal area identification.
In conjunction with Fig. 2-Figure 10, the embodiment of the present invention provides a kind of eye fundus image abnormal area dividing method, and this method is used for
Identify the blutpunkte in eye fundus image and exudation point.As shown in Fig. 2, this method comprises the following steps:
S1B pre-processes eye fundus image to enhance pixel point feature.Such as image shown in Fig. 3, it can reinforce
Its contrast obtains image shown in Fig. 4.Specifically Gauss enhancing will can be carried out again after original eye fundus image Gaussian smoothing, drawn high
The contrast in the region of abnormal point.The algorithm that Gauss enhancing uses is Adaptive contrast enhancement algorithm, using unsharp masking
Technology: eye fundus image is divided into two parts first, first is that the unsharp masking part of low frequency, can pass through the low pass filtered of image
Wave (smooth, fuzzy technology) obtains;Second is that radio-frequency component, can cross original image and subtract unsharp masking acquisition.Then high frequency section
It is amplified and is added in unsharp masking, finally obtain enhancing image as shown in Figure 4.
Picture blutpunkte and exudation point after processed can contrast be stronger on entire eye fundus image, and shows more
Apparent feature.
S2B, the exclusive segment region in the eye fundus image of enhancing contrast, wherein the partial region can be with right and wrong eyeground
Region (peripheral background area) and certain known certain final identification mesh target area is not included.E.g. peripheral background
With optic disk region, their shape, position and pixel value tag has obvious feature in whole image, can pass through
Setting pixel threshold finds and excludes these regions.More complicated and accurate method can certainly be used, such as uses one
Trained machine learning model identifies these interference regions.
Final purpose is to be partitioned into blutpunkte and exudation point in embodiments of the present invention, due to being not in these in optic disk
Lesion, therefore selection excludes optic disk region to obtain image as shown in Figure 5.
S3B obtains image to be classified block using sliding window traversal eye fundus image, and the glide direction in ergodic process includes
Horizontal direction and vertical direction, the sliding step in both direction are respectively less than the length and width of sliding window.
The size of sliding window is 128*128 (pixel), the ruler of the image to be classified block thus obtained in the present embodiment
Very little is 128*128 (pixel), much smaller than the size of entire eye fundus image.Fig. 6 shows the mistake of sliding window traversal eye fundus image
Journey schematic diagram, using the window that moves right after sliding window acquirement image to be classified block a, step-length is 64 (pixels), is obtained wait divide
Class image block b, so that the adjacent image to be classified block of horizontal direction overlaps and (has half overlapping in the present embodiment);It is similar
Ground moves down window in vertical direction, and step-length is 64 (pixels), obtains image to be classified block c, vertically to adjacent to be sorted
Image block overlaps and (has half overlapping in the present embodiment), while image to be classified block b and image to be classified block c also have portion
Divide overlapping (having a quarter overlapping in the present embodiment).Entire eye fundus image can be divided after traversing image in this manner
At multiple images block, partly overlapping image to be classified block can eliminate boundary effect in subsequent classification to the shadow of classification results
It rings.
In other embodiments, sliding step can also be made identical as image block length and width, so that image to be classified block does not have
Overlapping, or overlapping or nonoverlapping image to be classified block are obtained using other modes.
S4B respectively classifies to each image to be classified block using the first convolutional neural networks, exports bianry image
Block, for indicating normal picture block and abnormal image block.What the first convolutional neural networks in the present embodiment exported is pixel value
The image block that image block or pixel value for 0 are 255.It is available such as Fig. 7 after classifying to all image to be classified blocks
Shown in cover figure, it is normal picture block that wherein white, which is abnormal image block, black,.This image is shown to clearly demonstrate
Intermediate result out, when practical application, need not show this image.
S5B identifies the abnormal communication region being made of at least one abnormal image block.Such as have one in image shown in Fig. 7
The biggish white area of a little areas, this region are made of multiple abnormal image blocks, they constitute a big connection
Region;Simultaneously there are also the lesser white area of area, the smallest white area is an abnormal image block, is also regarded as one
A connected region.
S6B takes image block to be split centered on abnormal communication region in eye fundus image.At this with one in Fig. 7 compared with
For big connected region, it can determine that one takes figure range (white dashed line frame in figure) centered on the connected region, at this
Taken in embodiment figure range be the size of image block preset namely to be split be it is preset, having a size of 224*
224 (pixels).Fig. 8 is in eye fundus image according to the schematic diagram for taking figure range to obtain image block 81 to be split.For other companies
Logical region, no matter its size, the image block to be split of 224*224 (pixel) is all taken centered on connected region.This makes
There is also overlapped situations between some image blocks to be split.
The size of image block 81 to be split is 224*224 (pixel) in this embodiment, and when practical application can be according to final
Its size of the curriculum offering to be divided, the size should be at least more than the sizes of an image to be classified block.
S7B carries out semantic segmentation, output abnormality pixel to each image block to be split using the second convolutional neural networks
Cover figure.In order to clearly show that the blutpunkte in eye fundus image and exudation point, the second convolutional neural networks in the present embodiment are defeated
Out be two channel images, it can pass through color value abundant show two kinds of extraordinary image vegetarian refreshments.It in other embodiments, can also be with
Export it is single pass cover figure, such as binary map can identify a kind of extraordinary image vegetarian refreshments, and single pass image can be used different
Gray scale identifies a variety of extraordinary image vegetarian refreshments.
Second convolutional neural networks are made of convolutional layer, active coating, pond layer and up-sampling layer.Convolutional layer is using size
It is weighted for the local data in a window in the convolution kernel and input data of 3*3, then sliding is rolled up on the image
Product window, until the complete all input datas of convolution;Active coating is introduced to neuron non-using ReLu line rectification function
Linear factor is handled input data by activation primitive max { 0, x };Pond layer uses maximum Chi Huafa.Network it is defeated
Enter be triple channel image block to be split, output is the figure of covering in two channels, and each pixel of correspondence image is prospect (exception) and back
The probability value of scape (normal).
S8B marks corresponding semantic segmentation result in each image block to be split respectively.It is with image block 81 to be split
Result shown in Fig. 9 can be obtained in covering after figure is combined with image block 81 to be split for second convolutional neural networks output by example.
S9B maps back whole annotation results in eye fundus image.After being handled respectively each image block to be split, it will tie
Fruit maps back in original image, obtains the accurate segmentation result of the abnormal area of final eyeground figure.As shown in Figure 10, wherein it is dark
(practical is red) Regional Representative hemorrhagic areas, light color (practical is green) Regional Representative's seepage areas.
The eye fundus image abnormal area dividing method provided according to embodiments of the present invention removes boundary in pretreatment stage
With optic disk region, reservation area-of-interest can reduce the generation in part puppet region, reduce noise jamming;In pretreatment stage,
Using Adaptive contrast enhancement algorithm, the contrast of image can be drawn high, highlights the contrast of abnormal point, improves subsequent mesh
Mark the accuracy rate of detection;In the coarse positioning processing in abnormal point region, is operated, can effectively be traversed in image using sliding window
All areas, improve to the recall rate of abnormal area;In the coarse positioning module in abnormal point region, convolutional neural networks are used
Model is classified, and the powerful ability to express of deep learning, the classification of each image block of accurate judgement can be used;To region
It in the processing for carrying out semantic segmentation mould, is split using convolutional neural networks model, the powerful table of deep learning can be used
Danone power, classification belonging to each pixel in accurate judgement image block;It is handled using coarse positioning and semantic segmentation handles phase
In conjunction with mode, enable in the training of parted pattern, anomalous content accounting is big, and it is higher to be easy to train segmentation accuracy
Network, to improve the accuracy divided to eye fundus image abnormal area.
One embodiment of the present of invention additionally provides a kind of medical image abnormal area segmenting device, comprising:
Acquiring unit, for obtaining medical image;
Division unit, for the medical image to be divided into multiple image to be classified blocks;
First machine learning model, for classifying respectively to each image to be classified block, classification results are positive
Normal image block or abnormal image block;
Figure unit is taken, takes image block to be split in the medical image for being based respectively on the abnormal image block;
Second machine learning model, for carrying out semantic segmentation, segmentation result to each image block to be split respectively
For extraordinary image vegetarian refreshments;
Unit is marked, for marking the extraordinary image vegetarian refreshments in the medical image.
As a preferred embodiment, the figure unit is taken to include:
Connected region recognition unit, the abnormal communication region being made of for identification abnormal image block described at least one;
Connected region takes figure unit, for taking second to set in the medical image centered on the abnormal communication region
The image block to be split of scale cun.
As a preferred embodiment, marking unit includes:
Image block marks unit, for marking corresponding semantic segmentation knot in each image block to be split respectively
Fruit;
Map unit, for mapping back whole annotation results in the medical image.
One embodiment of the present of invention additionally provides a kind of medical image abnormal area splitting equipment, comprising:
At least one processor and the memory being connect at least one described processor communication;Wherein, the storage
Device is stored with the instruction that can be executed by least one described processor, and described instruction is executed by least one described processor, with
At least one described processor is set to execute above-mentioned medical image or eye fundus image abnormal area dividing method.
It should be understood by those skilled in the art that, the embodiment of the present invention can provide as method, system or computer program
Product.Therefore, complete hardware embodiment, complete software embodiment or reality combining software and hardware aspects can be used in the present invention
Apply the form of example.Moreover, it wherein includes the computer of computer usable program code that the present invention, which can be used in one or more,
The computer program implemented in usable storage medium (including but not limited to magnetic disk storage, CD-ROM, optical memory etc.) produces
The form of product.
The present invention be referring to according to the method for the embodiment of the present invention, the process of equipment (system) and computer program product
Figure and/or block diagram describe.It should be understood that every one stream in flowchart and/or the block diagram can be realized by computer program instructions
The combination of process and/or box in journey and/or box and flowchart and/or the block diagram.It can provide these computer programs
Instruct the processor of general purpose computer, special purpose computer, Embedded Processor or other programmable data processing devices to produce
A raw machine, so that being generated by the instruction that computer or the processor of other programmable data processing devices execute for real
The device for the function of being specified in present one or more flows of the flowchart and/or one or more blocks of the block diagram.
These computer program instructions, which may also be stored in, is able to guide computer or other programmable data processing devices with spy
Determine in the computer-readable memory that mode works, so that it includes referring to that instruction stored in the computer readable memory, which generates,
Enable the manufacture of device, the command device realize in one box of one or more flows of the flowchart and/or block diagram or
The function of being specified in multiple boxes.
These computer program instructions also can be loaded onto a computer or other programmable data processing device, so that counting
Series of operation steps are executed on calculation machine or other programmable devices to generate computer implemented processing, thus in computer or
The instruction executed on other programmable devices is provided for realizing in one or more flows of the flowchart and/or block diagram one
The step of function of being specified in a box or multiple boxes.
Obviously, the above embodiments are merely examples for clarifying the description, and does not limit the embodiments.It is right
For those of ordinary skill in the art, can also make on the basis of the above description it is other it is various forms of variation or
It changes.There is no necessity and possibility to exhaust all the enbodiments.And it is extended from this it is obvious variation or
It changes still within the protection scope of the invention.