Abstract
Purpose
Optical imaging is evolving as a key technique for advanced sensing in the operating room. Recent research has shown that machine learning algorithms can be used to address the inverse problem of converting pixel-wise multispectral reflectance measurements to underlying tissue parameters, such as oxygenation. Assessment of the specific hardware used in conjunction with such algorithms, however, has not properly addressed the possibility that the problem may be ill-posed.
Methods
We present a novel approach to the assessment of optical imaging modalities, which is sensitive to the different types of uncertainties that may occur when inferring tissue parameters. Based on the concept of invertible neural networks, our framework goes beyond point estimates and maps each multispectral measurement to a full posterior probability distribution which is capable of representing ambiguity in the solution via multiple modes. Performance metrics for a hardware setup can then be computed from the characteristics of the posteriors.
Results
Application of the assessment framework to the specific use case of camera selection for physiological parameter estimation yields the following insights: (1) estimation of tissue oxygenation from multispectral images is a well-posed problem, while (2) blood volume fraction may not be recovered without ambiguity. (3) In general, ambiguity may be reduced by increasing the number of spectral bands in the camera.
Conclusion
Our method could help to optimize optical camera design in an application-specific manner.
Similar content being viewed by others
References
Alerstam E, Lo WCY, Han TD, Rose J, Andersson-Engels S, Lilge L (2010) Next-generation acceleration and code optimization for light transport in turbid media using GPUs. Biomed Opt Express 1(2):658–675
Ardizzone L, Kruse J, Rother C, Köthe U (2019) Analyzing inverse problems with invertible neural networks. In: International conference on learning representations. https://openreview.net/forum?id=rJed6j0cKX. Accessed 3 Mar 2019
Clancy NT, Arya S, Stoyanov D, Singh M, Hanna GB, Elson DS (2015) Intraoperative measurement of bowel oxygen saturation using a multispectral imaging laparoscope. Biomed Opt Express 6(10):4179–4190
Cox B, Laufer J, Beard P (2009) The challenges for quantitative photoacoustic imaging. In: Photons plus ultrasound: imaging and sensing 2009, vol 7177. International Society for Optics and Photonics, p 717713
Dinh L, Sohl-Dickstein J, Bengio S (2016) Density estimation using real NVP. arXiv preprint arXiv:1605.08803
Feindt M (2004) A neural Bayesian estimator for conditional probability densities. arXiv:physics/0402093
Gal Y, Ghahramani Z (2016) Dropout as a Bayesian approximation: representing model uncertainty in deep learning. In: International conference on machine learning, pp 1050–1059
Gretton A, Borgwardt KM, Rasch MJ, Schölkopf B, Smola A (2012) A kernel two-sample test. J Mach Learn Res 13(Mar):723–773
Gu X, Han Z, Yao L, Zhong Y, Shi Q, Fu Y, Liu C, Wang X, Xie T (2016) Image enhancement based on in vivo hyperspectral gastroscopic images: a case study. J Biomed Opt 21(10):101412. https://doi.org/10.1117/1.JBO.21.10.101412
Han Z, Zhang A, Wang X, Sun Z, Wang MD, Xie T (2016) In vivo use of hyperspectral imaging to develop a noncontact endoscopic diagnosis support system for malignant colorectal tumors. J Biomed Opt 21(1):016001
Jacques SL (2013) Optical properties of biological tissues: a review. Phys Med Biol 58(11):R37
Kaneko K, Yamaguchi H, Saito T, Yano T, Oono Y, Ikematsu H, Nomura S, Sato A, Kojima M, Esumi H, Ochiai A (2014) Hypoxia imaging endoscopy equipped with laser light source from preclinical live animal study to first-in-human subject research. PLoS ONE 9(6):e99055
Kirchner T, Gröhl J, Maier-Hein L (2018) Context encoding enables machine learning-based quantitative photoacoustics. J Biomed Opt 23(5):056008. https://doi.org/10.1117/1.JBO.23.5.056008
Kohl S.A, Romera-Paredes B, Meyer C, De Fauw J, Ledsam JR, Maier-Hein KH, Eslami S, Rezende DJ, Ronneberger O (2018) A probabilistic U-Net for segmentation of ambiguous images. arXiv preprint arXiv:1806.05034
Lakshminarayanan B, Pritzel A, Blundell C (2017) Simple and scalable predictive uncertainty estimation using deep ensembles. In: Guyon I, Luxburg UV, Bengio S, Wallach H, Fergus R, Vishwanathan S, Garnett R (eds) Advances in neural information processing systems, vol 30. Curran Associates, Inc., Red Hook, pp 6402–6413
Leibig C, Allken V, Ayhan MS, Berens P, Wahl S (2017) Leveraging uncertainty information from deep neural networks for disease detection. Sci Rep 7(1):17816. https://doi.org/10.1038/s41598-017-17876-z
Li Y, Gal Y (2017) Dropout inference in Bayesian neural networks with alpha-divergences. arXiv:1703.02914 [cs, stat]
Maier-Hein L, Eisenmann M, Reinke A, Onogur S, Stankovic M, Scholz P, Arbel T, Bogunovic H, Bradley AP, Carass A, Feldmann C, Frangi AF, Full PM, van Ginneken B, Hanbury A, Honauer K, Kozubek M, Landman BA, März K, Maier O, Maier-Hein K, Menze BH, Müller H, Neher PF, Niessen W, Rajpoot N, Sharp GC, Sirinukunwattana K, Speidel S, Stock C, Stoyanov D, Taha AA, van der Sommen F, Wang CW, Weber MA, Zheng G, Jannin P, Kopp-Schneider A (2018) Why rankings of biomedical image analysis competitions should be interpreted with care. Nat Commun 9(1):5217
Moccia S, Wirkert SJ, Kenngott H, Vemuri AS, Apitz M, Mayer B, De Momi E, Mattos LS, Maier-Hein L (2018) Uncertainty-aware organ classification for surgical data science applications in laparoscopy. IEEE Trans Biomed Eng 65(11):2649–2659
Niculescu-Mizil A, Caruana R (2005) Predicting good probabilities with supervised learning. In: Proceedings of the 22nd international conference on machine learning. ACM, New York, pp 625–632
Nouri D, Lucas Y, Treuillet S (2014) Efficient tissue discrimination during surgical interventions using hyperspectral imaging. In: International conference on information processing in computer-assisted interventions. Springer, Berlin, pp 266–275. http://link.springer.com/chapter/10.1007/978-3-319-07521-1_28
Nouri D, Lucas Y, Treuillet S (2016) Hyperspectral interventional imaging for enhanced tissue visualization and discrimination combining band selection methods. Int J Comput Assist Radiol Surg 11(12):2185–2197. https://doi.org/10.1007/s11548-016-1449-5
Smith L, Gal Y (2018) Understanding measures of uncertainty for adversarial example detection. arXiv:1803.08533 [cs, stat]
Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R (2014) Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 15(1):1929–1958
Waibel D, Gröhl J, Isensee F, Kirchner T, Maier-Hein K, Maier-Hein L (2018) Reconstruction of initial pressure from limited view photoacoustic images using deep learning. In: Photons plus ultrasound: imaging and sensing 2018, vol 10494. International Society for Optics and Photonics, p 104942S
Wang L, Jacques SL, Zheng L (1995) MCML: Monte Carlo modeling of light transport in multi-layered tissues. Comput Methods Programs Biomed 47(2):131–146
Wirkert SJ, Clancy NT, Stoyanov D, Arya S, Hanna GB, Schlemmer HP, Sauer P, Elson DS, Maier-Hein L (2014) Endoscopic Sheffield Index for unsupervised in vivo spectral band selection. In: Luo X, Reichl T, Mirota D, Soper T (eds) Computer-assisted and robotic endoscopy, vol 8899. Springer, Cham, pp 110–120
Wirkert SJ, Isensee F, Vemuri AS, Maier-Hein K, Fei B, Maier-Hein L (2018) Domain and task specific multispectral band selection (conference presentation). In: Design and quality for biomedical technologies XI, p. nil. https://doi.org/10.1117/12.2287824
Wirkert SJ, Kenngott H, Mayer B, Mietkowski P, Wagner M, Sauer P, Clancy NT, Elson DS, Maier-Hein L (2016) Robust near real-time estimation of physiological parameters from megapixel multispectral images with inverse Monte Carlo and random forest regression. Int J Comput Assist Radiol Surg 11(6):909–917
Wirkert SJ, Vemuri AS, Kenngott HG, Moccia S, Götz M, Mayer BF, Maier-Hein KH, Elson DS, Maier-Hein L (2017) Physiological parameter estimation from multispectral images unleashed. In: International conference on medical image computing and computer-assisted intervention. Springer, Berlin, pp 134–141
Zhu Y, Zabaras N (2018) Bayesian deep convolutional encoder-decoder networks for surrogate modeling and uncertainty quantification. J Comput Phys 366:415–447. https://doi.org/10.1016/j.jcp.2018.04.018
Acknowledgements
This study has received funding from the European Unions Horizon 2020 research and innovation program through the ERC starting grant COMBIOSCOPY under Grant Agreement No. ERC-2015-StG-37960.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Ethical standard
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. All applicable international, national and/or institutional guidelines for the care and use of animals were followed. All procedures performed in studies involving animals were in accordance with the ethical standards of the institution or practice at which the studies were conducted. For this type of study, formal consent is not required.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Adler, T.J., Ardizzone, L., Vemuri, A. et al. Uncertainty-aware performance assessment of optical imaging modalities with invertible neural networks. Int J CARS 14, 997–1007 (2019). https://doi.org/10.1007/s11548-019-01939-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11548-019-01939-9