Abstract
For large-scale vision tasks in biomedical images, the labeled data is often limited to train effective deep models. Active learning is a common solution, where a query suggestion method selects representative unlabeled samples for annotation, and the new labels are used to improve the base model. However, most query suggestion models optimize their learnable parameters only on the limited labeled data and consequently become less effective for the more challenging unlabeled data. To tackle this, we propose a two-stream active query suggestion approach. In addition to the supervised feature extractor, we introduce an unsupervised one optimized on all raw images to capture diverse image features, which can later be improved by fine-tuning on new labels. As a use case, we build an end-to-end active learning framework with our query suggestion method for 3D synapse detection and mitochondria segmentation in connectomics. With the framework, we curate, to our best knowledge, the largest connectomics dataset with dense synapses and mitochondria annotation. On this new dataset, our method outperforms previous state-of-the-art methods by 3.1% for synapse and 3.8% for mitochondria in terms of region-of-interest proposal accuracy. We also apply our method to image classification, where it outperforms previous approaches on CIFAR-10 under the same limited annotation budget. The project page is https://zudi-lin.github.io/projects/#two_stream_active.
X. Chen, L. Kamentsky, A. Peleg, D. Haehn, T. Jones and T. Parag—Works were done at Harvard University.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
Architecture details are shown in Fig. S-1 in the supplementary document.
References
Abramson, Y., Freund, Y.: Active learning for visual object detection (2006)
Becker, C., Ali, K., Knott, G., Fua, P.: Learning context cues for synapse segmentation. IEEE TMI 32, 1864–1877 (2013)
Belkin, M., Niyogi, P.: Using manifold stucture for partially labeled classification. In: NIPS (2003)
Bietti, A.: Active learning for object detection on satellite images. Technical report, Caltech (2012)
Buhmann, J., et al.: Synaptic partner prediction from point annotations in insect brains. arXiv preprint arXiv:1806.08205 (2018)
Cheng, H.C., Varshney, A.: Volume segmentation using convolutional neural networks with limited training data. In: ICIP (2017)
Dorkenwald, S., et al.: Automated synaptic connectivity inference for volume electron microscopy. Nat. Methods 14(4), 435 (2017)
Ducoffe, M., Precioso, F.: Adversarial active learning for deep networks: a margin based approach. In: ICML (2018)
Dutt Jain, S., Grauman, K.: Active image segmentation propagation. In: CVPR (2016)
Freytag, A., Rodner, E., Denzler, J.: Selecting influential examples: active learning with expected model output changes. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8692, pp. 562–577. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10593-2_37
Funke, J., Saalfeld, S., Bock, D., Turaga, S., Perlman, E.: Circuit reconstruction from electron microscopy images (2016). https://cremi.org
He, K., Gkioxari, G., Dollár, P., Girshick, R.: Mask R-CNN. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2961–2969 (2017)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR (2016)
Heinrich, L., Funke, J., Pape, C., Nunez-Iglesias, J., Saalfeld, S.: Synaptic cleft segmentation in non-isotropic volume electron microscopy of the complete Drosophila brain. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11071, pp. 317–325. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00934-2_36
Hoffer, E., Ailon, N.: Deep metric learning using triplet network. In: International Workshop on Similarity-Based Pattern Recognition (2015)
Hu, J., Shen, L., Sun, G.: Squeeze-and-excitation networks. In: CVPR (2018)
Huang, G.B., Plaza, S.: Identifying synapses using deep and wide multiscale recursive networks. arXiv preprint arXiv:1409.1789 (2014)
Huang, G.B., Scheffer, L.K., Plaza, S.M.: Fully-automatic synapse prediction and validation on a large data set. arXiv preprint arXiv:1604.03075 (2016)
Jagadeesh, V., Anderson, J., Jones, B., Marc, R., Fisher, S., Manjunath, B.: Synapse classification and localization in electron micrographs. Pattern Recogn. Lett. 43, 17–24 (2014)
Kao, C.-C., Lee, T.-Y., Sen, P., Liu, M.-Y.: Localization-aware active learning for object detection. In: Jawahar, C.V., Li, H., Mori, G., Schindler, K. (eds.) ACCV 2018. LNCS, vol. 11366, pp. 506–522. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-20876-9_32
Kasthuri, N., et al.: Saturated reconstruction of a volume of neocortex. Cell 162(3), 648–661 (2015)
Kingma, D.P., Welling, M.: Auto-encoding variational bayes. ICLR (2013)
Kreshuk, A., Funke, J., Cardona, A., Hamprecht, F.A.: Who is talking to whom: synaptic partner detection in anisotropic volumes of insect brain. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9349, pp. 661–668. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24553-9_81
Kreshuk, A., Koethe, U., Pax, E., Bock, D.D., Hamprecht, F.A.: Automated detection of synapses in serial section transmission electron microscopy image stacks. PLoS One 9(2), e87351 (2014)
Kreshuk, A., et al.: Automated detection and segmentation of synaptic contacts in nearly isotropic serial electron microscopy images. PLoS One 6(10), e24899 (2011)
Lichtman, J.W., Sanes, J.R.: Ome sweet ome: what can the genome tell us about the connectome? Curr. Opin. Neurobiol. 18(3), 346–353 (2008)
Lucchi, A., Li, Y., Fua, P.: Learning for structured prediction using approximate subgradient descent with working sets. In: CVPR (2013)
Lucchi, A., Li, Y., Smith, K., Fua, P.: Structured image segmentation using kernelized features. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7573, pp. 400–413. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-33709-3_29
Lucchi, A., et al.: Learning structured models for segmentation of 2D and 3D imagery. IEEE TMI 34, 1096–1110 (2015)
Lucchi, A., Smith, K., Achanta, R., Knott, G., Fua, P.: Supervoxel-based segmentation of mitochondria in EM image stacks with learned shape features. IEEE TMI 31, 474–486 (2012)
Morgan, J.L., Lichtman, J.W.: Why not connectomics? Nat. Methods 10(6), 494 (2013)
Narasimha, R., Ouyang, H., Gray, A., McLaughlin, S.W., Subramaniam, S.: Automatic joint classification and segmentation of whole cell 3D images. Pattern Recogn. 42, 1067–1079 (2009)
Oztel, I., Yolcu, G., Ersoy, I., White, T., Bunyak, F.: Mitochondria segmentation in electron microscopy volumes using deep convolutional neural network. In: Bioinformatics and Biomedicine (2017)
Parag, T., et al.: Detecting synapse location and connectivity by signed proximity estimation and pruning with deep nets. arXiv preprint arXiv:1807.02739 (2018)
Parag, T., Ciresan, D.C., Giusti, A.: Efficient classifier training to minimize false merges in electron microscopy segmentation. In: ICCV (2015)
Perez, A.J., et al.: A workflow for the automatic segmentation of organelles in electron microscopy image stacks. Front. Neuroanat. 8, 126 (2014)
Plaza, S.M., Parag, T., Huang, G.B., Olbris, D.J., Saunders, M.A., Rivlin, P.K.: Annotating synapses in large EM datasets. arXiv preprint arXiv:1409.1801 (2014)
Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. In: Advances in Neural Information Processing Systems, pp. 91–99 (2015)
Roncal, W.G., et al.: VESICLE: volumetric evaluation of synaptic interfaces using computer vision at large scale. arXiv preprint arXiv:1403.3724 (2014)
Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Roy, S., Namboodiri, V.P., Biswas, A.: Active learning with version spaces for object detection. arXiv preprint arXiv:1611.07285 (2016)
Scheffer, T., Decomain, C., Wrobel, S.: Active hidden Markov models for information extraction. In: Hoffmann, F., Hand, D.J., Adams, N., Fisher, D., Guimaraes, G. (eds.) IDA 2001. LNCS, vol. 2189, pp. 309–318. Springer, Heidelberg (2001). https://doi.org/10.1007/3-540-44816-0_31
Sener, O., Savarese, S.: Active learning for convolutional neural networks: a core-set approach. In: ICLR (2018)
Settles, B.: Active learning literature survey. Technical report, University of Wisconsin-Madison Department of Computer Sciences (2009)
Settles, B.: Active learning literature survey. 2010. Computer Sciences Technical Report (2014)
Seyedhosseini, M., Ellisman, M.H., Tasdizen, T.: Segmentation of mitochondria in electron microscopy images using algebraic curves. In: ISBI, pp. 860–863. IEEE (2013)
Simonyan, K., Zisserman, A.: Two-stream convolutional networks for action recognition in videos. In: NIPS (2014)
Sivaraman, S., Trivedi, M.M.: Active learning for on-road vehicle detection: a comparative study. Mach. Vis. Appl. 25, 599–611 (2014)
Staffler, B., Berning, M., Boergens, K.M., Gour, A., van der Smagt, P., Helmstaedter, M.: SynEM, automated synapse detection for connectomics. Elife (2017)
Vazquez-Reina, A., Gelbart, M., Huang, D., Lichtman, J., Miller, E., Pfister, H.: Segmentation fusion for connectomics. In: ICCV (2011)
Vijayanarasimhan, S., Grauman, K.: Large-scale live active learning: training object detectors with crawled data and crowds. IJCV 108, 97–114 (2014)
Wang, K., Zhang, D., Li, Y., Zhang, R., Lin, L.: Cost-effective active learning for deep image classification. IEEE TCSVT 27, 2591–2600 (2017)
Yoo, D., Kweon, I.S.: Learning loss for active learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 93–102 (2019)
Yu, K., Bi, J., Tresp, V.: Active learning via transductive experimental design. In: Proceedings of the 23rd International Conference on Machine Learning, pp. 1081–1088 (2006)
Zhang, Y., Lease, M., Wallace, B.C.: Active discriminative text representation learning. In: AAAI (2017)
Zheng, Z., et al.: A complete electron microscopy volume of the brain of adult drosophila melanogaster. Cell 174, 730–743 (2018)
Zhu, X., Ghahramani, Z., Lafferty, J.D.: Semi-supervised learning using Gaussian fields and harmonic functions. In: ICML (2003)
Acknowledgment
This work has been partially supported by NSF award IIS-1835231 and NIH award 5U54CA225088-03.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Lin, Z. et al. (2020). Two Stream Active Query Suggestion for Active Learning in Connectomics. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, JM. (eds) Computer Vision – ECCV 2020. ECCV 2020. Lecture Notes in Computer Science(), vol 12363. Springer, Cham. https://doi.org/10.1007/978-3-030-58523-5_7
Download citation
DOI: https://doi.org/10.1007/978-3-030-58523-5_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-58522-8
Online ISBN: 978-3-030-58523-5
eBook Packages: Computer ScienceComputer Science (R0)