[go: up one dir, main page]

CN113012129A - System and device for counting area positioning and marked nerve cells of brain slice image - Google Patents

System and device for counting area positioning and marked nerve cells of brain slice image Download PDF

Info

Publication number
CN113012129A
CN113012129A CN202110291087.2A CN202110291087A CN113012129A CN 113012129 A CN113012129 A CN 113012129A CN 202110291087 A CN202110291087 A CN 202110291087A CN 113012129 A CN113012129 A CN 113012129A
Authority
CN
China
Prior art keywords
brain
atlas
brain slice
nerve cells
slice
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110291087.2A
Other languages
Chinese (zh)
Inventor
王松伟
牛克
王宇航
李倩
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhengzhou University
Original Assignee
Zhengzhou University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhengzhou University filed Critical Zhengzhou University
Priority to CN202110291087.2A priority Critical patent/CN113012129A/en
Publication of CN113012129A publication Critical patent/CN113012129A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/0002Inspection of images, e.g. flaw detection
    • G06T7/0012Biomedical image inspection
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/045Combinations of networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00Geometric image transformations in the plane of the image
    • G06T3/02Affine transformations
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/50Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/13Edge detection
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/30Determination of transform parameters for the alignment of images, i.e. image registration
    • G06T7/33Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/70Determining position or orientation of objects or cameras
    • G06T7/73Determining position or orientation of objects or cameras using feature-based methods
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10064Fluorescence image
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20081Training; Learning
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20084Artificial neural networks [ANN]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20212Image combination
    • G06T2207/20221Image fusion; Image merging
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30004Biomedical image processing
    • G06T2207/30016Brain
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30004Biomedical image processing
    • G06T2207/30024Cell structures in vitro; Tissue sections in vitro
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30242Counting objects in image

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Evolutionary Computation (AREA)
  • Computational Linguistics (AREA)
  • Molecular Biology (AREA)
  • Biophysics (AREA)
  • General Engineering & Computer Science (AREA)
  • Biomedical Technology (AREA)
  • Mathematical Physics (AREA)
  • Data Mining & Analysis (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Radiology & Medical Imaging (AREA)
  • Quality & Reliability (AREA)
  • Medical Informatics (AREA)
  • Image Analysis (AREA)

Abstract

The invention relates to a system and a device for area positioning of brain slice images and counting of marked nerve cells, and belongs to the field of image registration and target detection. The method comprises the following steps: acquiring a plurality of groups of fluorescence labeling brain slices, Average Template brain maps and Atlas brain maps; roughly registering the brain atlas and the brain slice in a global way and carrying out mode conversion; inputting the single-mode brain slice and the single-mode Average Template brain map into a registration network to obtain the Atlas brain map transformation of the deformation field; mapping the acquired brain area mark information and the space coordinate information onto the brain slice to complete the brain slice area positioning; and detecting the marked nerve cells on the brain slice by using the target detection network, comparing the marked nerve cells with the space coordinate information of the brain area to obtain the quantity of the nerve cells of each brain area, and counting the nerve cells of the brain slice. The invention realizes full-automatic, accurate and rapid area positioning of the brain section image and accurate counting of the marked nerve cells, and has simple and convenient operation.

Description

System and device for counting area positioning and marked nerve cells of brain slice image
Technical Field
The invention relates to a system and a device for area positioning of brain slice images and counting of marked nerve cells, and belongs to the technical field of image registration and target detection.
Background
The brain, the most complex organ in the animal body, is composed of a large number of nerve cells with different functions and morphologies, and the psychological and physiological activities of the animal are dominated by the different nerve cells and nerve networks. The research on the brain has important significance for optimizing artificial intelligence and treating neuropsychiatric diseases. With the great application of methods for staining nerve tissues such as acetylcholinesterase staining, Nie staining and immunohistochemical staining, people can draw more detailed and accurate standard brain maps of various animals such as mice and rats by understanding the brain structures of the animals. With the development of nerve staining technology and nerve loop labeling technology and the application of magnetic resonance imaging technology, the quantitative analysis of the number of nerve cells, the molecular expression level and the fluorescence signal intensity in brain slices is required, and then the research on the composition of a brain nerve network and the distribution characteristics of nerve cells is required.
The regional localization of fluorescence-labeled brain slices with reference to standard brain maps is the basis of brain region studies, and the accuracy of regional localization affects the statistical results of labeled nerve cells in each brain region. When the region is positioned, the brain slice image is required to be registered with the standard brain atlas, and when the nerve cell is marked for counting, a target detection technology is required to be adopted. The currently common methods for locating brain slice regions include: firstly, referring to a standard brain atlas, the outline of a brain area is hand-drawn on a brain slice image, but the hand-drawing needs expert experience, is suitable for drawing a small amount of samples and cannot be subjected to large-scale operation; secondly, semi-automatic region positioning is carried out on the brain slice image through Photoshop image processing software, and the method also needs expert experience and cannot be carried out on scale operation. Thirdly, the traditional multi-mode registration method is adopted, but the registration accuracy is low, so that the brain slice region positioning error is large. The traditional target detection method comprises three parts of region selection, feature extraction and classifier. The region selection is to locate the position of the target, and since the size of the target and the position appearing in the image cannot be determined, the whole image is traversed by adopting a sliding window method to determine the candidate region. The feature extraction is to extract features of the candidate regions, and common feature extraction methods include SIFT, HOG and the like. The classifier is used for classifying the detected target, and commonly used classifiers include SVM, Adaboost and the like. Because the traditional target detection method mostly adopts the characteristics of manual design, the robustness of the traditional target detection method to the diversity change of the target is poor. In addition, the traditional target detection method adopts a sliding window to select the region without pertinence, has the defects of high time complexity, window redundancy and the like, and causes high classification error rate. The marked nerve cells in the fluorescence-marked brain section are special in shape, and have polluted staining reagent spots, and the phenomena of omission and false detection are easy to occur by adopting the traditional target detection method.
Disclosure of Invention
The invention aims to provide a brain slice image area positioning and marked nerve cell counting system, which is used for solving the problems of inaccurate brain slice area positioning and missed detection and false detection of marked nerve cells in the prior art; meanwhile, the brain slice image area positioning and marked nerve cell counting device is also provided, and is used for solving the problems of inaccurate brain slice area positioning and missed detection and false detection of marked nerve cells.
In order to achieve the above object, the present invention provides a system for area localization and labeled neural cell counting of brain slice images, comprising the following steps:
acquiring a plurality of groups of images, wherein each group of images comprises a fluorescence labeling brain slice and an Average Template brain map and an Atlas brain map corresponding to the fluorescence labeling brain slice;
carrying out global coarse registration on the Average Template brain map, the Atlas brain map and the fluorescence labeled brain slice to obtain the Average Template brain map and the Atlas brain map after the coarse registration;
performing mode conversion on the fluorescence-labeled brain slice and the coarsely registered Average Template brain map to obtain a single-mode fluorescence-labeled brain slice and a single-mode Average Template brain map;
inputting the single-mode fluorescence labeling brain slice and the single-mode Average Template brain map into a registration network for training, and transforming the Atlas brain map by the obtained registration space transformation grid to obtain brain area labeling information and space coordinate information of the deformed Atlas brain map;
mapping the brain area mark information and the space coordinate information to a fluorescence mark brain slice to complete the area positioning of the brain slice image;
and detecting the marked nerve cells on the brain slice by using the target detection network model, comparing the obtained coordinate information of the marked nerve cells with the space coordinate information of the brain area to obtain the number of the marked nerve cells of each brain area, and finishing counting of the marked nerve cells of the brain slice.
In addition, the invention also provides a device for area positioning and labeled nerve cell counting of brain slice images, which comprises a memory, a processor and a computer program stored in the memory and capable of running on the processor, wherein the processor realizes the area positioning and labeled nerve cell counting of the brain slice images when executing the computer program.
The beneficial effects are that: the method registers the fluorescence labeling brain slice and the Average Template brain Atlas, the obtained registration network acts on the Atlas brain Atlas to realize the registration of the Atlas brain Atlas, and the outline of the Atlas brain Atlas after the registration is extracted and fused with the fluorescence labeling brain slice, thereby obtaining the area positioning of the brain slice image. And detecting the marked nerve cells in the fluorescence marked brain slice through the target detection network model, and comparing the brain area positioning map to complete the counting of the marked nerve cells in each brain area of the brain slice image. The invention realizes the full-automatic and rapid area positioning of the multi-modal brain slice image and the counting of the marked nerve cells, has simple and convenient operation and accurate area positioning and counting of the marked nerve cells.
Furthermore, in the above system and apparatus for area localization of brain slice image and counting of labeled nerve cells, in order to obtain fluorescence labeled brain slice, it is necessary to perform injection of neurotropic tool virus, material taking, sample preparation, immunohistochemistry and confocal imaging on animals.
Furthermore, in the system and the device for area positioning of the brain slice image and counting of the labeled nerve cells, in order to realize global coarse registration of the Average Template brain map, the Atlas brain map and the fluorescence labeled brain slice, the fluorescence labeled brain slice is taken as a reference, and affine transformation is carried out on the Average Template brain map and the Atlas brain map.
Furthermore, in the system and the device for area localization and labeled neural cell counting of brain slice images, in order to realize the same-mode registration between the fluorescence labeled brain slice and the coarsely registered Average Template brain atlas, the fluorescence labeled brain slice and the coarsely registered Average Template brain atlas are converted into the same mode through a PCANet-SR mode conversion network.
Further, in the system and the device for area localization and labeled neurocyte counting of brain slice images, the registration network comprises a net and a spatial transformation network.
Furthermore, in the system and the device for area positioning of the brain slice image and counting of the labeled nerve cells, in order to realize the area positioning of the brain slice image, the edge contour of the Atlas brain Atlas which is well registered is extracted by a Canny detector and is fused with the fluorescence labeled brain slice.
Furthermore, in the area positioning and labeled nerve cell counting system and device of the brain slice image, in order to realize the counting of the labeled nerve cells in the brain slice image, the labeled nerve cells in the brain slice are detected through IRN Faster R-CNN.
Drawings
FIG. 1 is a flow chart of a region localization and labeled neural cell counting system for brain slice images in accordance with the present invention;
FIG. 2-1 shows a mouse fluorescence-labeled brain slice and labeled nerve cells according to the present invention;
FIG. 2-2 is an Average Template standard brain map of a mouse of the present invention;
FIGS. 2-3 are standard brain maps of the mouse Atlas of the invention;
FIG. 3-1 is an image of a brain slice before and after affine transformation in accordance with the present invention;
FIG. 3-2 is an Average Template brain map before and after affine transformation in accordance with the present invention;
FIGS. 3-3 are Atlas brain atlases before and after affine transformation in accordance with the invention;
FIG. 4 illustrates a PCANet-SR network architecture according to the present invention;
FIG. 5 is a Reg-Net network architecture of the present invention;
FIG. 6-1 is a single mode brain slice image of the present invention;
FIG. 6-2 is a single mode Average Template brain map of the present invention;
FIG. 6-3 is an Average Template brain map after registration in accordance with the present invention;
6-4 are Atlas brain Atlas images after affine transformation in accordance with the present invention;
FIGS. 6-5 are Atlas brain atlases after registration according to the invention;
FIGS. 6-6 are graphs of the results of the area division of the fluorescence-labeled brain slices of the present invention;
FIG. 7 is an IRN Faster R-CNN network model of the present invention;
FIG. 8 shows the result of detecting the neural cells labeled by the brain slice according to the present invention;
FIG. 9-1 is an example of the result of the examination of the brain slice labeled neural cells according to the present invention;
FIG. 9-2 is an example of an Atlas brain Atlas after registration in accordance with the invention;
FIG. 10-1 shows the fusion of the brain slice image of the present invention with the original Atlas brain map;
FIG. 10-2 shows the Atlas brain Atlas fusion result after the brain slice image of the present invention is registered with the B-spline;
FIG. 10-3 shows the fusion result of the brain slice image and the Atlas brain Atlas after affine transformation;
FIG. 10-4 shows the fusion result of Atlas brain Atlas of B-spline registration after affine transformation and PSR processing of brain slice image according to the present invention;
FIGS. 10-5 show the Atlas brain Atlas fusion results after the brain slice images of the present invention were registered with the Voxelmorph method;
fig. 10-6 show the fusion result of the brain slice image region localization method proposed by the present invention.
Detailed Description
Area localization of brain slice images and labeled neural cell counting system embodiment:
the system for area localization and labeled neural cell counting of brain slice images proposed in this embodiment, as shown in fig. 1, includes the following steps:
1) an image is acquired.
The purpose of the step is to obtain a plurality of groups of images, wherein each group of images comprises a fluorescence labeling brain slice and an Average Template standard brain map and an Atlas standard brain map which correspond to the fluorescence labeling brain slice.
The acquisition of the fluorescence labeling brain section is mainly completed by three steps of neurotropic tool virus injection, material taking, sample preparation, immunohistochemistry and confocal imaging, and the Average Template standard brain map and Atlas standard brain map corresponding to the fluorescence labeling brain section are obtained from an ARA database.
2) And (4) global coarse registration.
The step aims to preprocess the fluorescence labeling brain slice, the Average Template standard brain map and the Atlas standard brain map, and improve the accuracy of the brain slice image area positioning.
And cutting and zooming the fluorescence-labeled brain slice to enable the resolution of the fluorescence-labeled brain slice to be unified with a standard brain atlas, and removing background information which has interference influence on registration in the fluorescence-labeled brain slice through background removing operation. And taking the processed fluorescence labeling brain slice as a reference, carrying out affine transformation on the Average Template brain map to obtain three parameters of affine transformation rotation, translation and scaling, and applying the three parameters to the Atlas brain map to carry out affine transformation, thereby realizing the global coarse registration of the Average Template brain map and the Atlas brain map to the fluorescence labeling brain slice.
3) And (5) modality conversion.
Obtaining a plurality of groups of brain slice images and corresponding affine-transformed Average Template brain maps in the same way in the step 2), and converting the brain slice images in different modes and the Average Template brain maps into the same mode by using a PCANet-SR model.
The two hidden layers of the cascade principal component analysis component in the PCANet-SR model extract a main feature output feature map of an input image, the output component converts the feature map, the structure representation component performs multi-stage feature fusion on the outputs of the two hidden layers, and an interested feature map of the PCANet-SR network is output. Finally obtaining a single-mode brain slice image and a single-mode Average Template brain map.
4) And (5) image registration.
Inputting the single-mode brain slice image and the single-mode Average Template brain map obtained in the step 3) into an image registration network based on the Reg-Net and the STN space transformation network for training to obtain the optimal transformation parameters of the registration network. Inputting a new reference image and a new floating image into the trained registration network, obtaining a space deformation field through Reg-Net, and then carrying out interpolation by using an STN space transformation network to obtain a registered image.
In a data set of the training image registration network, a reference image is a single-mode brain slice image, and a floating image is a single-mode Average Template brain map. The registration network training is realized by Keras based on Tensorflow as a rear end, an Adam optimization strategy is adopted, and the learning rate is set to be 0.01. And when the loss function does not significantly decline, stopping training to obtain the spatial deformation field parameters. And inputting the single-mode brain slice image and the single-mode Average Template brain map into the trained registration network to obtain a spatial deformation field phi. And (4) applying phi to the Atlas brain Atlas after affine transformation to obtain the registered Atlas brain Atlas.
5) Brain slice area localization.
The edge contours of the post-registration Atlas brain Atlas obtained in step 4) were extracted using the Canny detector in the MATLAB image processing toolkit. And fusing the original fluorescence brain slice without mode conversion with the edge contour map to complete the area positioning of the brain slice image.
6) And detecting and counting the marked nerve cells.
And making a data set required by labeled nerve cell detection, and training an IRN Faster R-CNN target detection network. And verifying the target detection model by using the verification data set, and storing the optimal model parameters. Inputting the fluorescence labeling brain slice to be detected into the trained detection model to obtain the coordinate position of the labeling nerve cell in the brain slice. Comparing the coordinate position with the brain slice area positioning map obtained in the step 5), and determining the brain area to which each marked nerve cell belongs, thereby obtaining the number of the marked nerve cells in each brain area.
The data set for detecting the marked nerve cells is completed by labeling the fluorescence-labeled brain slices by an image labeling tool, and comprises a training set, a verification set and a test set. The training of the target detection network is realized by Keras with Tensorflow as a rear end, an Adam optimization strategy is adopted, and a Swish activation function is used. And when the loss function is not obviously reduced, stopping training, verifying and adjusting the model parameters by using the verification data set, and storing the optimal model parameters for testing. And evaluating the accuracy and the generalization performance of the target detection model by using the test data set.
In this example, the region segmentation method and the labeled neural cell counting method of the present invention will be described in detail with reference to a mouse fluorescence-labeled brain slice as an example.
The method of the invention is utilized to carry out the regional localization and the detection of the marked nerve cells on the fluorescence marked brain section of the mouse:
mouse fluorescence labeling brain sections are obtained through three steps of neurotropic tool virus injection, material taking, sample preparation, immunohistochemistry and confocal imaging, and nerve cells in the brain sections are subjected to fluorescence staining, as shown in figure 2-1. The Average Template standard brain map and Atlas standard brain map were obtained from the ARA database as shown in FIGS. 2-2 and 2-3, respectively.
Because the resolution of the obtained fluorescence labeling brain slices is not uniform, the background comprises information such as a magnification coefficient and a scale, and the resolution of the standard brain atlas is 11400 × 8000, in order to reduce the calculation cost and ensure the registration effect, the brain slice image needs to be cut, zoomed and the background is removed, the standard brain atlas needs to be cut and zoomed, and the resolution of the brain slice image and the standard brain atlas is uniformly zoomed to 570 × 400. And taking the processed brain slice image as a reference image, carrying out affine transformation on the corresponding Average Template brain map to obtain translation, rotation and scaling parameters of the affine transformation, and acting the parameters on the processed Atlas brain map to obtain the Atlas brain map after the affine transformation. As shown in FIGS. 3-1, 3-2, and 3-3, the top surface is the brain slice before affine transformation, the Average Template brain map, and the Atlas brain map, and the bottom surface is the brain slice after affine transformation, the Average Template brain map, and the Atlas brain map.
The brain slice image and the affine-transformed Average Template brain map are input to a PCANet-SR network as shown in fig. 4, which is composed of a cascade principal component analysis module, an output module, and a structure representation module. The cascade principal component analysis component is composed of two hidden layers, wherein the first hidden layer extracts the features of an input image, an output feature map is used as the input of the second hidden layer, and the second hidden layer is similar to the first hidden layer in structure. And the output component converts the output of the second hidden layer by using a Sigmoid function, the structure representation component fuses the output of the two hidden layers by using an exponential function, and finally, a single-mode image is output. As shown in FIGS. 6-1 and 6-2, the single mode Average Template brain atlas and the single mode brain slice image are obtained after PCANet-SR network processing.
Processing PCANet-SR network to obtain single-mode Average Template brain map (floating image M)PSR) And single-mode brain slice image (reference image F)PSR) As a dataset for training a registration network, M in the datasetPSRAnd FPSRThe deformation field phi with the size of 570 x 400 x 2 is fused, input into a Reg-Net network shown in figure 5, and finally output after convolution, up-sampling and jumping connection. Using STN spatial transform network and spatial warping field phi to transform MPSRDeformation to MPSR(phi), calculating M from the similarity measurePSR(phi) and FPSRAnd updating the convolution kernel parameters of the registration network by optimizing the local normalized correlation coefficient NCC of the loss function. And when the loss function is not obviously reduced, stopping training, and further obtaining the spatial deformation field parameters and the registered Average Template brain atlas shown in the figure 6-3. Applying the obtained deformation field parameters to the Atlas brain Atlas after affine transformation as shown in fig. 6-4 to obtain the registered Atlas brain Atlas as shown in fig. 6-5Atlas brain Atlas.
And (3) extracting the edge contour of the Atlas brain Atlas after registration by using a Canny edge detector in an MATLAB image processing toolkit, fusing the extracted edge contour with the original brain slice to obtain a fused image shown in figures 6-6, and finishing the brain region positioning of the fluorescence labeling brain slice image.
The detection of the marked nerve cells in the brain slice image belongs to supervised deep learning, and when a data set is manufactured, the target area in the brain slice needs to be deducted firstly in consideration of the small occupied space of the marked nerve cells in the brain slice image, so that the computation amount of a model is reduced. Labeling the labeled nerve cells in the target region using a LabelImg labeling tool, and exporting an xml file. Obtaining 2000 marked nerve cell pictures, dividing 1400 training sets, 200 verification sets and 400 testing sets according to the proportion of 7:1: 2. Inputting the training set into the IRN Faster R-CNN network shown in FIG. 7, using the pre-training model parameters under the COCO data set as the initialization weight of the detection model, and using the Adam optimization algorithm to optimize the loss function. Setting the initial learning rate to be 0.001, continuously adjusting the learning rate in the training process, and reducing the learning rate when the learning is stopped. And when the error of the verification set is not obviously reduced, stopping training, and saving the optimal training weight as the test model. The original brain slice is firstly divided into small image blocks, the small image blocks are input into a detection model, and the detected image blocks are spliced according to the original direction to form the brain slice with the size of the original image. The results of the examination of the brain section labeled nerve cells are shown in FIG. 8. And obtaining coordinate values of the marked nerve cells in the brain slice image through an IRN Faster R-CNN target detection algorithm, and counting the marked nerve cells in each brain area in the brain slice by combining the corresponding Atlas brain Atlas after registration. Taking one group of brain slices as an example, as shown in fig. 9-1 and 9-2, the labeled nerve cell detection results of the brain slices and the Atlas brain Atlas after registration are respectively shown, and the number of labeled nerve cells in each brain area is counted and shown in table one.
TABLE statistical example of the number of nerve cells marked in each brain region of the brain slice of the present invention
Figure BDA0002981972510000081
Comparing the area positioning method of the brain slice image of the invention with the existing area positioning method, as shown in fig. 10-1, 10-2, 10-3, 10-4, 10-5 and 10-6, fig. 10-1 is the fusion result of the brain slice image and the original Atlas brain Atlas, fig. 10-2 is the fusion result of the Atlas brain Atlas after the brain slice image is registered with the B-spline, fig. 10-3 is the fusion result of the brain slice image and the Atlas brain Atlas after affine transformation, fig. 10-4 is the fusion result of the brain slice image and the Atlas brain Atlas after affine transformation and PSR processing, fig. 10-5 is the fusion result of the brain slice image and the Atlas brain Atlas after Voxelmorph method registration, and fig. 10-6 is the fusion result of the area positioning method of the brain slice image provided by the invention. Commonly used registration accuracy indicators are: root Mean Square Error (RMSE), Correlation Coefficient (CC), Mutual Information (MI). The smaller the RMSE value, the better the registration effect, and the larger the CC value and MI value, the better the registration effect. The registration accuracy index comparison of the registration method of the present invention and the existing registration method is shown in table two. The real-time property of registration is also an important factor for measuring the quality of the registration algorithm, and as shown in table three, the registration speed of each registration algorithm is compared.
Table two: the registration accuracy index comparison of the registration method of the invention and the existing registration method
Figure BDA0002981972510000091
Table three: registration speed comparison of registration algorithms
Figure BDA0002981972510000092
As can be found from the second table and the third table, the registration algorithm provided by the invention has the advantages of minimum RMSE value, maximum CC value and MI value, best registration effect, high registration speed and capability of realizing region positioning of batch brain slice images.
Comparing the marked nerve cell detection method with the existing detection method, selecting a Faster R-CNN target detection network model taking VGG16 as a characteristic extraction network and a Faster R-CNN target detection network model taking ResNet-101 as a characteristic extraction network as a control group, and selecting the IRN Faster R-CNN target detection network model as an experimental group. ReLU and Swish are respectively used as activation functions of the three target detection network models to evaluate the performance of different target detection network model algorithms. And (3) using the average accuracy average value (mAP) as an evaluation index of the three target detection network models on the detection result of the marked nerve cells, wherein the larger the mAP value is, the more accurate the detection result of the marked nerve cells is. The mAP values detected by the three target detection network models on the marked nerve cells are shown in the fourth table.
Table four: the marked nerve cell detection method of the invention is compared with the mAP index of the existing detection method
Figure BDA0002981972510000101
The table shows that the IRN Faster R-CNN target detection network model provided by the invention has the largest mAP value on the brain slice labeled nerve cell detection result, and the detection result is the most accurate.
The brain section image area positioning and marked nerve cell counting device embodiment:
the device for locating and labeling a brain slice image area and counting nerve cells provided by the embodiment comprises a memory, a processor and a computer program which is stored in the memory and can run on the processor, wherein the processor realizes a system for locating and labeling a brain slice image area and counting nerve cells when executing the computer program.
The specific implementation process of the brain slice image area locating and labeled neural cell counting system is described in the above embodiment of the brain slice image area locating and labeled neural cell counting system, and is not described herein again.

Claims (8)

1.一种脑切片图像的区域定位及标记神经细胞计数系统及装置,其特征在于,包括以下步骤:1. a regional positioning and marking nerve cell counting system and device of a brain slice image, is characterized in that, comprises the following steps: 获取多组图像,每组图像包括荧光标记脑切片和与其对应的Average Template脑图谱、Atlas脑图谱;Obtain multiple sets of images, each of which includes fluorescently labeled brain slices and their corresponding Average Template brain atlas and Atlas brain atlas; 将Average Template脑图谱、Atlas脑图谱与荧光标记脑切片进行全局粗配准,得到粗配准后的Average Template脑图谱、Atlas脑图谱;Perform global rough registration of the Average Template brain atlas, Atlas brain atlas and fluorescently labeled brain slices to obtain the Average Template brain atlas and Atlas brain atlas after rough registration; 将荧光标记脑切片和粗配准后的Average Template脑图谱进行模态转换,得到单模荧光标记脑切片和单模Average Template脑图谱;Perform modal conversion between the fluorescently labeled brain slices and the coarsely registered Average Template brain atlas to obtain single-mode fluorescently labeled brain slices and single-mode Average Template brain atlas; 将单模荧光标记脑切片和单模Average Template脑图谱输入到配准网络中进行训练,将得到的配准空间变换网格对Atlas脑图谱进行变换,得到变形后的Atlas脑图谱的脑区标记信息和空间坐标信息;Input the single-mode fluorescently labeled brain slice and the single-mode Average Template brain map into the registration network for training, and transform the Atlas brain map with the obtained registration space transformation grid to obtain the brain region markers of the deformed Atlas brain map. information and spatial coordinate information; 将脑区标记信息和空间坐标信息映射到荧光标记脑切片上,完成荧光标记脑切片的区域定位;Map the brain region marker information and spatial coordinate information to the fluorescently labeled brain slices to complete the regional positioning of the fluorescently labeled brain slices; 使用目标检测网络模型检测脑切片上的标记神经细胞,将得到的标记神经细胞的坐标信息与脑区空间坐标信息进行对比,得到每个脑区的标记神经细胞数量,完成脑切片标记神经细胞的计数。The target detection network model is used to detect the labeled nerve cells on the brain slice, and the obtained coordinate information of the labeled nerve cells is compared with the spatial coordinate information of the brain region to obtain the number of labeled nerve cells in each brain region. count. 2.根据权利要求1所述的脑切片图像的区域定位及标记神经细胞计数系统,其特征在于,通过对动物进行嗜神经工具病毒注射、取材及样本制备、免疫组化及共聚焦成像三个步骤获取荧光标记脑切片。2. The system for regional positioning and labeling of neuronal cells in brain slice images according to claim 1, characterized in that, by carrying out three steps of neurotropic virus injection, material sampling and sample preparation, immunohistochemistry and confocal imaging on animals Steps to obtain fluorescently labeled brain slices. 3.根据权利要求1所述的脑切片图像的区域定位及标记神经细胞计数系统,其特征在于,通过仿射变换对Average Template脑图谱、Atlas脑图谱与荧光标记脑切片进行全局粗配准。3 . The system for regional positioning and labeling of neuronal cells in brain slice images according to claim 1 , wherein the global rough registration of the Average Template brain atlas, Atlas brain atlas and fluorescently labeled brain slices is performed by affine transformation. 4 . 4.根据权利要求1所述的脑切片图像的区域定位及标记神经细胞计数系统,其特征在于,通过PCANet-SR模态转换网络将荧光标记脑切片和粗配准后的Average Template脑图谱转换为同一模态。4. The system for regional localization and labeled neuron counting of brain slice images according to claim 1, wherein the fluorescently labeled brain slice and the Average Template brain atlas after rough registration are converted by PCANet-SR modal conversion network for the same mode. 5.根据权利要求1所述的脑切片图像的区域定位及标记神经细胞计数系统,其特征在于,所述配准网络包括Unet和空间变换网络。5 . The system for regional localization and labeling of neuronal cells in brain slice images according to claim 1 , wherein the registration network comprises Unet and spatial transformation network. 6 . 6.根据权利要求1所述的脑切片图像的区域定位及标记神经细胞计数系统,其特征在于,通过Canny检测器提取已配准好的Atlas脑图谱的边缘轮廓,并将边缘轮廓与荧光标记脑切片融合。6. The regional positioning and marking neuron counting system of the brain slice image according to claim 1, wherein the edge contour of the registered Atlas brain atlas is extracted by the Canny detector, and the edge contour and the fluorescent marker are extracted. Brain slice fusion. 7.根据权利要求1所述的脑切片图像的区域定位及标记神经细胞计数系统,其特征在于,所述的目标检测网络模型为IRN Faster R-CNN,包括Inception-Res-Net特征提取网络、RPN网络、ROI Pooling和全连接网络。7. the regional location of the brain slice image according to claim 1 and the marking nerve cell counting system, it is characterized in that, described target detection network model is IRN Faster R-CNN, comprises Inception-Res-Net feature extraction network, RPN network, ROI Pooling and fully connected network. 8.一种脑切片图像的区域定位及标记神经细胞计数装置,包括存储器、处理器以及存储在所述存储器中并可在处理器上运行的计算机程序,其特征在于,所述处理器在执行所述计算机程序时实现如权利要求1-7中任意一项所述的脑切片图像的区域定位及标记神经细胞计数。8. A regional positioning and marking neuron counting device of a brain slice image, comprising a memory, a processor and a computer program stored in the memory and running on the processor, wherein the processor is executing The computer program realizes the regional localization of the brain slice image and the counting of labeled nerve cells according to any one of claims 1-7.
CN202110291087.2A 2021-03-18 2021-03-18 System and device for counting area positioning and marked nerve cells of brain slice image Pending CN113012129A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110291087.2A CN113012129A (en) 2021-03-18 2021-03-18 System and device for counting area positioning and marked nerve cells of brain slice image

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110291087.2A CN113012129A (en) 2021-03-18 2021-03-18 System and device for counting area positioning and marked nerve cells of brain slice image

Publications (1)

Publication Number Publication Date
CN113012129A true CN113012129A (en) 2021-06-22

Family

ID=76409716

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110291087.2A Pending CN113012129A (en) 2021-03-18 2021-03-18 System and device for counting area positioning and marked nerve cells of brain slice image

Country Status (1)

Country Link
CN (1) CN113012129A (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113571163A (en) * 2021-07-05 2021-10-29 新乡医学院 Method for constructing whole brain neuron network connection map
CN115049626A (en) * 2022-06-27 2022-09-13 华中科技大学苏州脑空间信息研究院 Map-assisted drawing method, computer-readable storage medium and electronic device
CN115345829A (en) * 2022-07-12 2022-11-15 江苏诺鬲生物科技有限公司 Acid-resistant mycobacterium identification method and device based on artificial intelligence algorithm
CN116798523A (en) * 2023-06-01 2023-09-22 南京金域医学检验所有限公司 Pattern recognition and judgment system for anti-neutrophil cytoplasmic antibody

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106920228A (en) * 2017-01-19 2017-07-04 北京理工大学 The method for registering and device of brain map and brain image
CN110555835A (en) * 2019-09-04 2019-12-10 郑州大学 brain slice image region division method and device

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106920228A (en) * 2017-01-19 2017-07-04 北京理工大学 The method for registering and device of brain map and brain image
CN110555835A (en) * 2019-09-04 2019-12-10 郑州大学 brain slice image region division method and device

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
赵秋阳: "脑切片图像的区域定位及标记神经细胞计数系统研究", 《中国优秀硕士学位论文全文数据库 基础科学辑》 *

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113571163A (en) * 2021-07-05 2021-10-29 新乡医学院 Method for constructing whole brain neuron network connection map
CN115049626A (en) * 2022-06-27 2022-09-13 华中科技大学苏州脑空间信息研究院 Map-assisted drawing method, computer-readable storage medium and electronic device
CN115345829A (en) * 2022-07-12 2022-11-15 江苏诺鬲生物科技有限公司 Acid-resistant mycobacterium identification method and device based on artificial intelligence algorithm
CN116798523A (en) * 2023-06-01 2023-09-22 南京金域医学检验所有限公司 Pattern recognition and judgment system for anti-neutrophil cytoplasmic antibody

Similar Documents

Publication Publication Date Title
Kromp et al. Evaluation of deep learning architectures for complex immunofluorescence nuclear image segmentation
Das et al. Computer-aided histopathological image analysis techniques for automated nuclear atypia scoring of breast cancer: a review
JP7422235B2 (en) Non-tumor segmentation to aid tumor detection and analysis
CN113012129A (en) System and device for counting area positioning and marked nerve cells of brain slice image
JP7197584B2 (en) Methods for storing and retrieving digital pathology analysis results
Pan et al. Cell detection in pathology and microscopy images with multi-scale fully convolutional neural networks
Marzahl et al. Deep learning-based quantification of pulmonary hemosiderophages in cytology slides
CN110033032B (en) Tissue slice classification method based on microscopic hyperspectral imaging technology
CN112380900A (en) Deep learning-based cervical fluid-based cell digital image classification method and system
CN110555835B (en) A brain slice image region division method and device
CN108846828A (en) A kind of pathological image target-region locating method and system based on deep learning
CN106780498A (en) Based on point depth convolutional network epithelium and matrix organization's automatic division method pixel-by-pixel
JP2022547722A (en) Weakly Supervised Multitask Learning for Cell Detection and Segmentation
CN115546605A (en) Training method and device based on image labeling and segmentation model
CN113902669B (en) Method and system for reading urine exfoliated cell liquid-based smear
CN114782372B (en) DNA fluorescence in situ hybridization BCR/ABL fusion state detection method and detection system
CN113658151B (en) Breast lesion magnetic resonance image classification method, equipment and readable storage medium
CN112348059A (en) Deep learning-based method and system for classifying multiple dyeing pathological images
CN109886346A (en) A Cardiac MRI Image Classification System
CN113627522B (en) Image classification method, device, equipment and storage medium based on relational network
Hasan et al. Real-time segmentation and classification of whole-slide images for tumor biomarker scoring
CN114972263A (en) A real-time ultrasound image follicle measurement method and system based on intelligent image segmentation
CN112597907B (en) Identification method of citrus red spider pests based on deep learning
Khamael et al. Using adapted JSEG algorithm with fuzzy C mean for segmentation and counting of white blood cell and nucleus images
Wang et al. A novel dataset and a two-stage deep learning method for breast cancer mitosis nuclei identification

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
WD01 Invention patent application deemed withdrawn after publication

Application publication date: 20210622

WD01 Invention patent application deemed withdrawn after publication