Abstract
In situ cryo-electron tomography enables investigation of macromolecules in their native cellular environment. Samples have become more readily available owing to recent software and hardware advancements. Data collection, however, still requires an experienced operator and appreciable microscope time to carefully select targets for high-throughput tilt series acquisition. Here, we developed smart parallel automated cryo-electron tomography (SPACEtomo), a workflow using machine learning approaches to fully automate the entire cryo-electron tomography process, including lamella detection, biological feature segmentation, target selection and parallel tilt series acquisition, all without the need for human intervention. This degree of automation will be essential for obtaining statistically relevant datasets and high-resolution structures of macromolecules in their native context.
This is a preview of subscription content, access via your institution
Access options
Access Nature and 54 other Nature Portfolio journals
Get Nature+, our best-value online-access subscription
$29.99 / 30 days
cancel any time
Subscribe to this journal
Receive 12 print issues and online access
$259.00 per year
only $21.58 per issue
Buy this article
- Purchase on Springer Link
- Instant access to full article PDF
Prices may be subject to local taxes which are calculated during checkout
Similar content being viewed by others
Data availability
Lamella montages and raw tilt series frames produced during SPACEtomo development were deposited in the Electron Microscopy Public Image Archive under accession code EMPIAR-11841. Example tomograms used for Fig. 2 are available under EMDB accession number EMD-38295. Labeled training datasets for deep learning models are available at https://doi.org/10.5281/zenodo.10360315 (ref. 39) and https://doi.org/10.5281/zenodo.10360344 (ref. 40). Trained deep learning models were deposited at https://doi.org/10.5281/zenodo.10360489 (ref. 41) and https://doi.org/10.5281/zenodo.10360540 (ref. 42).
Code availability
All SPACEtomo Python scripts are available at https://github.com/eisfabian/SPACEtomo. SPACEtomo is currently still in an alpha stage and will be updated to improve performance and reliability and add additional features and models.
References
Tegunov, D., Xue, L., Dienemann, C., Cramer, P. & Mahamid, J. Multi-particle cryo-EM refinement with M visualizes ribosome–antibiotic complex at 3.5 Å in cells. Nat. Methods 18, 186–193 (2021).
Wang, Z. et al. Structures from intact myofibrils reveal mechanism of thin filament regulation through nebulin. Science 375, eabn1934 (2022).
Gemmer, M. et al. Visualization of translation and protein biogenesis at the ER membrane. Nature 614, 160–167 (2023).
Hoffmann, P. C. et al. Structures of the eukaryotic ribosome and its translational states in situ. Nat. Commun. 13, 7435 (2022).
Xue, L. et al. Visualizing translation dynamics at atomic detail inside a bacterial cell. Nature 610, 205–211 (2022).
Buckley, G. et al. Automated cryo-lamella preparation for high-throughput in-situ structural biology. J. Struct. Biol. 210, 107488 (2020).
Zachs, T. et al. Fully automated, sequential focused ion beam milling for cryo-electron tomography. eLife 9, e52286 (2020).
Tacke, S. et al. A streamlined workflow for automated cryo focused ion beam milling. J. Struct. Biol. 213, 107743 (2021).
Klumpe, S. et al. A modular platform for automated cryo-FIB workflows. eLife 10, e70506 (2021).
Cleeve, P. et al. OpenFIBSEM: a universal API for FIBSEM control. J. Struct. Biol. 215, 107967 (2023).
Klumpe, S. et al. Developments in cryo-FIB sample preparation: targeting in cryo-lift-out preparation of tissues and machine learning models for fully automated on-grid lamella preparation. Microsc. Microanal. 29, 511–513 (2023).
Schiøtz, O. H. et al. Serial Lift-Out: sampling the molecular anatomy of whole organisms. Nat. Methods https://doi.org/10.1038/s41592-023-02113-5 (2023).
Dung, N. H. T. et al. Serialized on-grid lift-in sectioning for tomography (SOLIST). Preprint at bioRxiv https://doi.org/10.1101/2023.05.11.540146 (2023).
Bouvette, J. et al. Beam image-shift accelerated data acquisition for near-atomic resolution single-particle cryo-electron tomography. Nat. Commun. 12, 1957 (2021).
Eisenstein, F. et al. Parallel cryo electron tomography on in situ lamellae. Nat. Methods 20, 131–138 (2023).
Kim, P. T., Noble, A. J., Cheng, A. & Bepler, T. Learning to automate cryo-electron microscopy data collection with Ptolemy. IUCrJ 10, 90–102 (2023).
Cheng, A. et al. Fully automated multi-grid cryoEM screening using Smart Leginon. IUCrJ 10, 77–89 (2023).
Bouvette, J. et al. Automated systematic evaluation of cryo-EM specimens with SmartScope. eLife 11, e80047 (2022).
Mastronarde, D. N. Automated electron microscope tomography using robust prediction of specimen movements. J. Struct. Biol. 152, 36–51 (2005).
Schorb, M., Haberbosch, I., Hagen, W. J. H., Schwab, Y. & Mastronarde, D. N. Software tools for automated transmission electron microscopy. Nat. Methods 16, 471–477 (2019).
Jocher, G., Chaurasia, A. & Qiu, J. YOLO by Ultralytics https://ultralytics.com (2023).
Ronneberger, O., Fischer, P. & Brox, T. U-Net: convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisted Intervention — MICCAI 2015 (eds Navab, N. et al.) 234–241 (Springer International, 2015).
Isensee, F., Jaeger, P. F., Kohl, S. A. A., Petersen, J. & Maier-Hein, K. H. nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation. Nat. Methods 18, 203–211 (2021).
Bieber, A. et al. In situ structural analysis reveals membrane shape transitions during autophagosome formation. Proc. Natl Acad. Sci. USA 119, e2209823119 (2022).
Balyschew, N. et al. Streamlined structure determination by cryo-electron tomography and subtomogram averaging using TomoBEAR. Nat. Commun. 14, 6543 (2023).
Zheng, S. et al. AreTomo: an integrated software package for automated marker-free, motion-corrected cryo-electron tomographic alignment and reconstruction. J. Struct. Biol. X 6, 100068 (2022).
Liu, H.-F. et al. nextPYP: a comprehensive and scalable platform for characterizing protein variability in situ using single-particle cryo-electron tomography. Nat. Methods 20, 1909–1919 (2023).
Erdmann, P. S. et al. In situ cryo-electron tomography reveals gradient organization of ribosome biogenesis in intact nucleoli. Nat. Commun. 12, 5364 (2021).
Gupta, T. K. et al. Structural basis for VIPP1 oligomerization and maintenance of thylakoid membrane integrity. Cell 184, 3643–3659 (2021).
Buchholz, T.-O. et al. Content-aware image restoration for electron microscopy. In Methods in Cell Biology Vol. 152 (eds Müller-Reichert, T. & Pigino, G.) Ch. 13 (Academic Press, 2019).
Kremer, J. R., Mastronarde, D. N. & McIntosh, J. R. Computer visualization of three-dimensional image data using IMOD. J. Struct. Biol. 116, 71–76 (1996).
Mastronarde, D. N. Correction for non-perpendicularity of beam and tilt axis in tomographic reconstructions with the IMOD package. J. Microsc. 230, 212–217 (2008).
Ansel, J. et al. PyTorch 2: faster machine learning through dynamic Python bytecode transformation and graph compilation. In Proceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems 929–947 (Association for Computing Machinery, 2024); https://doi.org/10.1145/3620665.364036
Burnley, T., Palmer, C. M. & Winn, M. Recent developments in the CCP-EM software suite. Acta Crystallogr. D Struct. Biol. 73, 469–477 (2017).
Hunter, J. D. Matplotlib: a 2D graphics environment. Comput. Sci. Eng. 9, 90–95 (2007).
Harris, C. R. et al. Array programming with NumPy. Nature 585, 357–362 (2020).
van der Walt, S. et al. scikit-image: image processing in Python. PeerJ 2, e453 (2014).
Virtanen, P. et al. SciPy 1.0: fundamental algorithms for scientific computing in Python. Nat. Methods 17, 261–272 (2020).
Eisenstein, F. & Danev, R. SPACEtomo training dataset for lamella detection using YOLOv8. Zenodo https://doi.org/10.5281/zenodo.10360315 (2023).
Eisenstein, F., Bieber, A., Capitanio, C. & Danev, R. SPACEtomo training dataset for yeast lamella map segmentation using nnU-Netv2. Zenodo https://doi.org/10.5281/zenodo.10360344 (2023).
Eisenstein, F. & Danev, R. SPACEtomo lamella detection model using YOLOv8. Zenodo https://doi.org/10.5281/zenodo.10360489 (2023).
Eisenstein, F. & Danev, R. SPACEtomo lamella segmentation model for yeast. Zenodo https://doi.org/10.5281/zenodo.10360540 (2023).
Acknowledgements
We are grateful to A. Bieber and C. Capitanio for providing the yeast lamella montages used to create the training dataset. Additionally, we thank M. Pöge, G. Weiss and S. Klumpe for providing training data for the lamella-detection model and H. Kashihara and S. Tsukita for providing the EpH4 cell sample. We express our appreciation for fruitful discussions about deep learning approaches with S. Klumpe, Y. Hirabayashi, S. Suga and H. Kawai. We extend our gratitude to M. Kikkawa for providing feedback and support. Further thanks go to Y. Sakamaki, T. Wang and K. Nakamura for the support of the Graduate School of Medicine cryo-EM facility. D. Böhringer and M. Peterek supported the setup of SPACEtomo at ETH Zürich; J. Hugener, T. Zachs, M. Petersen, M. Weber, L. Rettberg and other members of the group of M. Pilhofer at ETH Zürich helped with troubleshooting of new features. F.E. was an International Research Fellow of the Japan Society for the Promotion of Science (P20764) and was a recipient of a Grant-in-Aid for Scientific Research (KAKENHI, 21F20764). Y.F. (KAKENHI, JP19H05707) and R.D. (KAKENHI, 22H02554) were supported by Japan Society for the Promotion of Science grants.
Author information
Authors and Affiliations
Contributions
F.E. developed SPACEtomo, labeled the training data and conducted all cryo-FIB and cryo-ET experiments. Y.F. prepared yeast samples and helped with troubleshooting. F.E. and R.D. designed experiments. F.E. wrote the manuscript with contributions from all authors.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing interests.
Peer review
Peer review information
Nature Methods thanks Jonathan Bouvette and the other, anonymous, reviewer(s) for their contribution to the peer review of this work. Primary Handling Editor: Rita Strack, in collaboration with the Nature Methods team. Peer reviewer reports are available.
Additional information
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Extended data
Extended Data Fig. 1 Example of lamella detection on a whole grid montage LM map.
Shown is a cryoFIB-milled grid of Yeast cells with 16 detected lamellae. Bounding boxes of detected lamellae are coloured according to their predicted class (green: ‘good’, red: ‘broken’, yellow: ‘thick’). The label for each bounding box shows the class label and the confidence score of the model.
Extended Data Fig. 2 Segmentation examples of S. cerevisiae lamellae medium magnification montages.
Shown are 3 examples of medium magnification montages of lamellae milled from Yeast cells (left) and their segmentations using the SPACEtomo model trained on S. cerevisiae (right).
Extended Data Fig. 3 Segmentation examples of C. reinhardtii lamellae medium magnification montages.
Shown are 3 examples of medium magnification montages of lamellae milled from C. reinhardtii cells (left) and their segmentations using the SPACEtomo model trained on S. cerevisiae (right). While some classes (for example ice contamination, dark and bright areas, coating) are successfully segmented by the Yeast model, cellular classes are principally misclassified.
Extended Data Fig. 4 Transferability of the SPACEtomo target selection to C. reinhardtii.
Shown are 3 examples of C. reinhardtii lamella MM maps on the left and the automated SPACEtomo target selection using a target list of all classes except the organism independent classes to be avoided (for example, black, white, ice, crack). Beam shape at maximum tilt angle is marked by yellow ellipse. Camera fields of view are outlined in red for the tracking target and blue for other targets. The dashed white line represents the tilt axis orientation.
Extended Data Fig. 5 Segmentation examples of EpH4 mouse epithelial cells lamellae medium magnification montages.
Shown are 3 examples of medium magnification montages of lamellae milled from EpH4 mouse epithelial cells (left) and their segmentations using the SPACEtomo model trained on S. cerevisiae (right). While some classes (for example ice contamination, dark and bright areas, coating) are successfully segmented by the Yeast model, cellular classes are not reliably detected, and even large areas of cytoplasm are classified as background.
Extended Data Fig. 6 Transferability of the SPACEtomo target selection to EpH4 mouse epithelial cells.
Shown are 3 examples of EpH4 cell lamella MM maps on the left and the automated SPACEtomo target selection using a target list of all classes except the organism independent classes to be avoided (for example, black, white, ice, crack). Beam shape at maximum tilt angle is marked by yellow ellipse. Camera fields of view are outlined in red for the tracking target and blue for other targets. The dashed white line represents the tilt axis orientation.
Extended Data Fig. 7 Segmentation classes.
Examples for all 17 classes used to train the nnU-Net for lamella segmentation. Class names in quotes are accompanied by short descriptions, where necessary, and an example image showing the respective class segmentation in red.
Extended Data Fig. 8 Training and inference performance of the nnU-Net model.
a. Loss values for training dataset (train loss) and validation dataset (val loss) of the 5-fold cross-validated training of the nnU-Net model. All 5 folds (fold_0-4) were provided by nnU-Net during the training process for 1000 epochs. b. Mean segmentation inference time in minutes per 100 µm2 obtained from inferences of MM maps (n = 8) of sizes between 300 and 1000 µm2. Error bars show the standard deviation.
Extended Data Fig. 9 Schematic of target selection steps.
a. Sparse target selection targeting mitochondria. b. Exhaustive target selection using a hexagonal target arrangement to minimise beam exposure overlap targeting cytoplasm.
Extended Data Fig. 10 The SPACEtomo score threshold can improve precision of targets.
Top: Precision and recall relative to used threshold (0.0) for 299 tilt series collected targeting mitochondria. Setting the threshold to 0.01 would have raised the precision to 90 % while losing only 2 % of mitochondria targets. Bottom: Precision and recall relative to used threshold (0.25) for 187 tilt series collected targeting the nuclear envelope. Even high thresholds did not yield a precision of 100 % indicating the presence of misclassifications in the segmentations that could be improved with more training data.
Supplementary information
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Eisenstein, F., Fukuda, Y. & Danev, R. Smart parallel automated cryo-electron tomography. Nat Methods (2024). https://doi.org/10.1038/s41592-024-02373-9
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/s41592-024-02373-9
