[go: up one dir, main page]

skip to main content
research-article

Near-infrared Imaging for Information Embedding and Extraction with Layered Structures

Published: 12 August 2022 Publication History

Abstract

Non-invasive inspection and imaging techniques are used to acquire non-visible information embedded in samples. Typical applications include medical imaging, defect evaluation, and electronics testing. However, existing methods have specific limitations, including safety risks (e.g., X-ray), equipment costs (e.g., optical tomography), personnel training (e.g., ultrasonography), and material constraints (e.g., terahertz spectroscopy). Such constraints make these approaches impractical for everyday scenarios. In this article, we present a method that is low-cost and practical for non-invasive inspection in everyday settings. Our prototype incorporates a miniaturized near-infrared spectroscopy scanner driven by a computer-controlled 2D-plotter. Our work presents a method to optimize content embedding, as well as a wavelength selection algorithm to extract content without human supervision. We show that our method can successfully extract occluded text through a paper stack of up to 16 pages. In addition, we present a deep-learning-based image enhancement model that can further improve the image quality and simultaneously decompose overlapping content. Finally, we demonstrate how our method can be generalized to different inks and other layered materials beyond paper. Our approach enables a wide range of content embedding applications, including chipless information embedding, physical secret sharing, 3D print evaluations, and steganography.

References

[1]
Fadel Adib and Dina Katabi. 2013. See through walls with WiFi! In Proceedings of the ACM Conference of the Special Interest Group on Data Communications (SIGCOMM’13). 75–86.
[2]
Leili Afsah-Hejri, Elnaz Akbari, Arash Toudeshki, Taymaz Homayouni, Azar Alizadeh, and Reza Ehsani. 2020. Terahertz spectroscopy and imaging: A review on agricultural applications. Comput. Electr. Agric. 177 (2020), 105628.
[3]
Erkki Alarousu, Leszek Krehut, Tuukka Prykäri, and Risto Myllylä. 2005. Study on the use of optical coherence tomography in measurements of paper properties. Measure. Sci. Technol. 16, 5 (2005), 1131.
[4]
T. Arecchi, M. Bellini, C. Corsi, R. Fontana, M. Materazzi, L. Pezzati, and A. Tortora. 2006. A new tool for painting diagnostics: Optical coherence tomography. Optics Spectrosc. 101, 1 (2006), 23–26.
[5]
Hidenobu Arimoto, Mariko Egawa, and Yukio Yamada. 2005. Depth profile of diffuse reflectance near-infrared spectroscopy for measurement of water content in skin. Skin Res. Technol. 11, 1 (2005), 27–35.
[6]
Caroline R. Baumal. 1999. Clinical applications of optical coherence tomography. Curr. Opin. Ophthalmol. 10, 3 (1999), 182–188.
[7]
Amos Beimel. 2011. Secret-sharing schemes: A survey. In Proceedings of the International Conference on Coding and Cryptology. Springer, 11–46.
[8]
David A. Boas, Dana H. Brooks, Eric L. Miller, Charles A. DiMarzio, Misha Kilmer, Richard J. Gaudette, and Quan Zhang. 2001. Imaging the body with diffuse optical tomography. IEEE Signal Process. Mag. 18, 6 (2001), 57–75.
[9]
Alan C. Bovik. 2009. Basic binary image processing. In The Essential Guide to Image Processing. Elsevier, 69–96.
[10]
Ginger Butcher. 2010. Tour of the Electromagnetic Spectrum. National Aeronautics and Space Administration.
[11]
Perumal Varun Chadalavada, Goutham Palaniappan, Vimal Kumar Chandran, Khai Truong, and Daniel Wigdor. 2018. ID’em: Inductive sensing for embedding and extracting information in robust materials. Proc. ACM Interact. Mobile Wear. Ubiq. Technol. 2, 3 (2018), 1–28.
[12]
Vincent Chan and Anahi Perlas. 2011. Basics of ultrasound imaging. In Atlas of Ultrasound-guided Procedures in Interventional Pain Management. Springer, 13–19.
[13]
Jakub Czajkowski, Tuukka Prykäri, Erkki Alarousu, Jaakko Palosaari, and Risto Myllylä. 2010. Optical coherence tomography as a method of quality inspection for printed electronics products. Optic. Rev. 17, 3 (2010), 257–262.
[14]
Jana Dambrogio, Amanda Ghassaei, Daniel Starza Smith, Holly Jackson, Martin L. Demaine, Graham Davis, David Mills, Rebekah Ahrendt, Nadine Akkerman, David van der Linden, et al. 2021. Unlocking history through automated virtual unfolding of sealed documents imaged by X-ray microtomography. Nature Commun. 12, 1 (2021), 1–10.
[15]
Niranjan Damera-Venkata and Nelson L. Chang. 2009. Display supersampling. ACM Trans. Graph. 28, 1 (2009), 1–19.
[16]
Yiming Deng and Xin Liu. 2011. Electromagnetic imaging methods for nondestructive evaluation applications. Sensors 11, 12 (2011), 11774–11808.
[17]
Nicolas Ducros, Juan Felipe Perez-Juste Abascal, Bruno Sixou, Simon Rit, and Françoise Peyrin. 2017. Regularization of nonlinear decomposition of spectral x-ray projection images. Med. Phys. 44, 9 (2017), e174–e187.
[18]
Adolf F. Fercher, Wolfgang Drexler, Christoph K. Hitzenberger, and Theo Lasser. 2003. Optical coherence tomography-principles and applications. Reports Progr. Phys. 66, 2 (2003), 239.
[19]
S. Geetha, K. K. Satheesh Kumar, Chepuri R. K. Rao, M. Vijayan, and D. C. Trivedi. 2009. EMI shielding: Methods and materials-A review. J. Appl. Polymer Sci. 112, 4 (2009), 2073–2086.
[20]
S. Gholizadeh. 2016. A review of non-destructive testing methods of composite materials. Procedia Struct. Integr. 1 (2016), 50–57.
[21]
Mayank Goel, Eric Whitmire, Alex Mariakakis, T. Scott Saponas, Neel Joshi, Dan Morris, Brian Guenter, Marcel Gavriliu, Gaetano Borriello, and Shwetak N. Patel. 2015. HyperCam: Hyperspectral imaging for ubiquitous computing applications. In Proceedings of the ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp’15). ACM, New York, NY, 145–156.
[22]
Lewis D. Griffin, Matthew Caldwell, Jerone T. A. Andrews, and Helene Bohler. 2018. “unexpected item in the bagging area” Anomaly detection in x-ray security images. IEEE Trans. Info. Forensics Secur. 14, 6 (2018), 1539–1553.
[23]
E. J. Hall and D. J. Brenner. 2008. Cancer risks from diagnostic radiology. Brit. J. Radiol. 81, 965 (2008), 362–378.
[24]
M. Hegedüs, K. Fedorov, I. Antonov, P. Karataev, and V. N. Antonov. 2020. Detection of black body radiation using a compact terahertz imager. Appl. Phys. Lett. 117, 23 (2020), 231106.
[25]
Cristian Herrojo, Ferran Paredes, and Ferran Martín. 2019a. Double-stub loaded microstrip line reader for very high data density microwave encoders. IEEE Trans. Microw. Theory Techn. 67, 9 (2019), 3527–3536.
[26]
Cristian Herrojo, Ferran Paredes, Javier Mata-Contreras, and Ferran Martín. 2019b. Chipless-RFID: A review and recent developments. Sensors 19, 15 (2019), 3385.
[27]
Yoko Hoshi and Yukio Yamada. 2016. Overview of diffuse optical tomography and its clinical applications. J. Biomed. Optics 21, 9 (2016), 091312.
[28]
David Huang, Eric A. Swanson, Charles P. Lin, Joel S. Schuman, William G. Stinson, Warren Chang, Michael R. Hee, Thomas Flotte, Kenton Gregory, Carmen A. Puliafito et al. 1991. Optical coherence tomography. Science 254, 5035 (1991), 1178–1181.
[29]
Christian Jansen, Steffen Wietzke, Ole Peters, Maik Scheller, Nico Vieweg, Mohammed Salhi, Norman Krumbholz, Christian Jördens, Thomas Hochrein, and Martin Koch. 2010. Terahertz imaging: Applications and perspectives. Appl. Optics 49, 19 (2010), E48–E57.
[30]
Weiwei Jiang, Gabriele Marini, Niels van Berkel, Zhanna Sarsenbayeva, Zheyu Tan, Chu Luo, Xin He, Tilman Dingler, Jorge Goncalves, Yoshihiro Kawahara, and Vassilis Kostakos. 2019. Probing sucrose contents in everyday drinks using miniaturized near-infrared spectroscopy scanners. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 3, 4, Article 136 (Dec. 2019), 25 pages.
[31]
Weiwei Jiang, Zhanna Sarsenbayeva, Niels van Berkel, Chaofan Wang, Difeng Yu, Jing Wei, Jorge Goncalves, and Vassilis Kostakos. 2021. User trust in assisted decision-making using miniaturized near-infrared spectroscopy. In Proceedings of the CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, 1–16.
[32]
Yoshihiro Kawahara, Steve Hodges, Benjamin S. Cook, Cheng Zhang, and Gregory D. Abowd. 2013. Instant inkjet circuits: Lab-based inkjet printing to support rapid prototyping of ubicomp devices. In Proceedings of the ACM International Joint Conference on Pervasive and Ubiquitous Computing. 363–372.
[33]
Aditya Khosla, Nityananda Jayadevaprakash, Bangpeng Yao, and Li Fei-Fei. 2011. Novel dataset for fine-grained image categorization. In Proceedings of the 1st Workshop on Fine-Grained Visual Categorization, IEEE Conference on Computer Vision and Pattern Recognition.
[34]
Su-Hwan Kim, Jee-Hyun Kim, and Shin-Won Kang. 2011. Nondestructive defect inspection for LCDs using optical coherence tomography. Displays 32, 5 (2011), 325–329.
[35]
Simon Klakegg, Jorge Goncalves, Chu Luo, Aku Visuri, Alexey Popov, Niels van Berkel, Zhanna Sarsenbayeva, Vassilis Kostakos, Simo Hosio, Scott Savage, et al. 2018. Assisted medication management in elderly care using miniaturised near-infrared spectroscopy. In Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies. 69.
[36]
Simon Klakegg, Jorge Goncalves, Niels van Berkel, Chu Luo, Simo Hosio, and Vassilis Kostakos. 2017. Towards commoditised near infrared spectroscopy. In Proceedings of the Conference on Designing Interactive Systems (DIS’17). ACM, New York, NY, 515–527.
[37]
Simon Klakegg, Chu Luo, Jorge Goncalves, Simo Hosio, and Vassilis Kostakos. 2016. Instrumenting smartphones with portable NIRS. In Proceedings of the ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct (UbiComp’16). ACM, New York, NY, 618–623.
[38]
Marcin Kowalski, Mariusz Kastek, Michal Walczakowski, Norbert Palka, and Mieczyslaw Szustakowski. 2015. Passive imaging of concealed objects in terahertz and long-wavelength infrared. Appl. Optics 54, 13 (2015), 3826–3833.
[39]
Dingzeyu Li, Avinash S. Nair, Shree K. Nayar, and Changxi Zheng. 2017. Aircode: Unobtrusive physical tags for digital fabrication. In Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology. 449–460.
[40]
Ai Liu, Gang Li, Zhigang Fu, Yang Guan, and Ling Lin. 2018. Non-linearity correction in NIR absorption spectra by grouping modeling according to the content of analyte. Sci. Rep. 8, 1 (2018), 1–10.
[41]
Jonathan Long, Evan Shelhamer, and Trevor Darrell. 2015. Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’15).
[42]
Ilya Loshchilov and Frank Hutter. 2017. Decoupled weight decay regularization. Retrieved from https://arXiv:1711.05101.
[43]
Yongsen Ma, Gang Zhou, and Shuangquan Wang. 2019. WiFi sensing with channel state information: A survey. ACM Comput. Surveys 52, 3 (2019), 1–36.
[44]
Harry E. Martz, Clint M. Logan, Daniel J. Schneberk, and Peter J. Shull. 2016. X-ray Imaging: Fundamentals, Industrial Techniques and Applications. CRC Press.
[45]
Peter Meszaros. 2006. Gamma-ray bursts. Rep. Progr. Phys. 69, 8 (2006), 2259.
[46]
Peter G. Morris. 1986. Nuclear magnetic resonance imaging in medicine and biology. (1986).
[47]
E. W. T. Ngai, Karen K. L. Moon, Frederick J. Riggins, and Y. Yi Candace. 2008. RFID research: An academic literature review (1995–2005) and future research directions. Int. J. Product. Econ. 112, 2 (2008), 510–520.
[48]
M.-E. Nilsback and A. Zisserman. 2006. A visual vocabulary for flower classification. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06), Vol. 2. 1447–1454.
[49]
Vasilis Ntziachristos. 2010. Going deeper than microscopy: The optical imaging frontier in biology. Nature Methods 7, 8 (2010), 603–614.
[50]
Sonia H. Contreras Ortiz, Tsuicheng Chiu, and Martin D. Fox. 2012. Ultrasound image enhancement: A review. Biomed. Signal Process. Control 7, 5 (2012), 419–428.
[51]
Vishal M. Patel, Joseph N. Mait, Dennis W. Prather, and Abigail S. Hedden. 2016. Computational millimeter wave imaging: Problems, progress, and prospects. IEEE Signal Process. Mag. 33, 5 (2016), 109–118.
[52]
Stevan Preradovic and Nemai Chandra Karmakar. 2010. Chipless RFID: Bar code of the future. IEEE Microw. Mag. 11, 7 (2010), 87–97.
[53]
Albert Redo-Sanchez, Barmak Heshmat, Alireza Aghasi, Salman Naqvi, Mingjie Zhang, Justin Romberg, and Ramesh Raskar. 2016. Terahertz time-gated spectral imaging for content extraction through layered structures. Nature Commun. 7, 1 (2016), 1–7.
[54]
Gabriele Reich. 2005. Near-infrared spectroscopy and imaging: Basic principles and pharmaceutical applications. Non-Invasive Spectroscopic and Imaging Techniques in Drug Delivery. Adv. Drug Deliv. Rev. 57, 8 (2005), 1109–1143.
[55]
Yves Roggo, Pascal Chalus, Lene Maurer, Carmen Lema-Martinez, Aurélie Edmond, and Nadine Jent. 2007. A review of near infrared spectroscopy and chemometrics in pharmaceutical technologies. J. Pharm. Biomed. Anal. 44, 3 (2007), 683– 700.
[56]
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. 2015. U-Net: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisted Intervention (MICCAI’15), Nassir Navab, Joachim Hornegger, William M. Wells, and Alejandro F. Frangi (Eds.). Springer International Publishing, Cham, 234–241.
[57]
Kurt Rossmann. 1969. Point spread-function, line spread-function, and modulation transfer function. Radiology 93, 2 (1969), 257–272. PMID: 5822701.
[58]
Z. Sabetsarvestani, Barak Sober, Catherine Higgitt, Ingrid Daubechies, and M. R. D. Rodrigues. 2019. Artificial intelligence for art investigation: Meeting the challenge of separating X-ray images of the Ghent Altarpiece. Sci. Adv. 5, 8 (2019), eaaw7416.
[59]
T. W. L. Scheeren, P. Schober, and L. A. Schwarte. 2012. Monitoring tissue oxygenation by near infrared spectroscopy (NIRS): Background and current applications. J. Clin. Monitor. Comput. 26, 4 (2012), 279–287.
[60]
Quan Z. Sheng, Xue Li, and Sherali Zeadally. 2008. Enabling next-generation RFID applications: Solutions and challenges. Computer 41, 9 (2008), 21–28.
[61]
Lawrence A. Shepp and Joseph Bernard Kruskal. 1978. Computerized tomography: The new medical X-ray technology. Amer. Math. Monthly 85, 6 (1978), 420–439.
[62]
Heinz W. Siesler, Yukihiro Ozaki, Satoshi Kawata, and H. Michael Heise. 2008. Near-infrared Spectroscopy: Principles, Instruments, Applications. John Wiley & Sons.
[63]
Douglas A. Skoog, Donald M. West, F. James Holler, and Stanley Crouch. 2013. Fundamentals of Analytical Chemistry. Nelson Education.
[64]
Erin Treacy Solovey, Audrey Girouard, Krysta Chauncey, Leanne M. Hirshfield, Angelo Sassaroli, Feng Zheng, Sergio Fantini, and Robert J. K. Jacob. 2009. Using fNIRS brain sensing in realistic HCI settings: Experiments and guidelines. In Proceedings of the 22nd Annual ACM Symposium on User Interface Software and Technology. ACM, 157–166.
[65]
Joo-Hiuk Son, Seung Jae Oh, and Hwayeong Cheon. 2019. Potential clinical applications of terahertz radiation. J. Appl. Phys. 125, 19 (2019), 190901.
[66]
David Stifter. 2007. Beyond biomedicine: A review of alternative applications and developments for optical coherence tomography. Appl. Phys. B 88, 3 (2007), 337–357.
[67]
D. Stromer, V. Christlein, T. Schön, W. Holub, and A. Maier. 2017. Browsing through closed books: Evaluation of preprocessing methods for page extraction of a 3D CT book volume. In Proceedings of the IOP Conference Series: Materials Science and Engineering, Vol. 229. IOP Publishing, 012005.
[68]
Piotr Targowski, Bogumiła Rouba, Maciej Wojtkowski, and Andrzej Kowalczyk. 2004. The application of optical coherence tomography to non-destructive examination of museum objects. Studies Conserv. 49, 2 (2004), 107–114.
[69]
George J. Tserevelakis, Margarita Tsagkaraki, Panagiotis Siozos, and Giannis Zacharakis. 2019. Uncovering the hidden content of layered documents by means of photoacoustic imaging. Strain 55, 2 (2019), e12289.
[70]
Andrew M. Wallace, Abderrahim Halimi, and Gerald S. Buller. 2020. Full waveform lidar for adverse weather conditions. IEEE Trans. Vehic. Technol. 69, 7 (2020), 7064–7077.
[71]
Xintao Wang, Ke Yu, Shixiang Wu, Jinjin Gu, Yihao Liu, Chao Dong, Yu Qiao, and Chen Change Loy. 2018. ESRGAN: Enhanced super-resolution generative adversarial networks. In Proceedings of the European Conference on Computer Vision (ECCV’18).
[72]
Zhou Wang and Alan C. Bovik. 2009. Mean squared error: Love it or leave it? A new look at signal fidelity measures. IEEE Signal Process. Mag. 26, 1 (2009), 98–117.
[73]
Karl D. D. Willis and Andrew D. Wilson. 2013. InfraStructs: Fabricating information inside physical objects for imaging in the terahertz region. ACM Trans. Graph. 32, 4 (2013), 1–10.
[74]
Cort J. Willmott and Kenji Matsuura. 2005. Advantages of the mean absolute error (MAE) over the root mean square error (RMSE) in assessing average model performance. Climate Res. 30, 1 (2005), 79–82.
[75]
Honami Yamane, Kazuo Tanaka, and Yoshiki Chujo. 2018. Synthesis of a near-infrared light-absorbing polymer based on thiophene-substituted Aza-BODIPY. Polymer J. 50, 3 (2018), 271–275.
[76]
Li Yang, Amin Rida, Rushi Vyas, and Manos M. Tentzeris. 2007. RFID tag and RF structures on a paper substrate using inkjet-printing technology. IEEE Trans. Microw. Theory Techn. 55, 12 (2007), 2894–2901.
[77]
Ali Zamanian and C. J. H. F. E. Hardiman. 2005. Electromagnetic radiation and human health: A review of sources and effects. High Freq. Electr. 4, 3 (2005), 16–26.
[78]
Yanzi Zhu, Yuanshun Yao, Ben Y. Zhao, and Haitao Zheng. 2017. Object recognition and navigation using a single networking device. In Proceedings of the 15th Annual International Conference on Mobile Systems, Applications, and Services. 265–277.
[79]
Boris Zimmermann and Achim Kohler. 2013. Optimizing Savitzky-Golay parameters for improving spectral resolution and quantification in infrared spectroscopy. Appl. Spectrosc. 67, 8 (2013), 892–902.
[80]
Zhengxia Zou, Sen Lei, Tianyang Shi, Zhenwei Shi, and Jieping Ye. 2020. Deep adversarial decomposition: A unified framework for separating superimposed images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR’20). 12803–12813.

Cited By

View all
  • (2024)Illumination-Aware Low-Light Image Enhancement with Transformer and Auto-Knee CurveACM Transactions on Multimedia Computing, Communications, and Applications10.1145/366465320:8(1-23)Online publication date: 29-Jun-2024
  • (2024)Mobile Near-infrared Sensing—A Systematic Review on Devices, Data, Modeling, and ApplicationsACM Computing Surveys10.1145/365259656:8(1-36)Online publication date: 10-Apr-2024
  • (2024)Rip-NeRF: Anti-aliasing Radiance Fields with Ripmap-Encoded Platonic SolidsACM SIGGRAPH 2024 Conference Papers10.1145/3641519.3657402(1-11)Online publication date: 13-Jul-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics
ACM Transactions on Graphics  Volume 42, Issue 1
February 2023
211 pages
ISSN:0730-0301
EISSN:1557-7368
DOI:10.1145/3555791
Issue’s Table of Contents

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 August 2022
Online AM: 02 May 2022
Accepted: 01 April 2022
Revised: 01 March 2022
Received: 01 August 2021
Published in TOG Volume 42, Issue 1

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Near-infrared spectroscopy
  2. information embedding
  3. information extraction
  4. non-invasive inspection

Qualifiers

  • Research-article
  • Refereed

Funding Sources

  • Melbourne Research Scholarships
  • Doreen Thomas Postdoctoral Fellowship

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)170
  • Downloads (Last 6 weeks)13
Reflects downloads up to 10 Oct 2024

Other Metrics

Citations

Cited By

View all
  • (2024)Illumination-Aware Low-Light Image Enhancement with Transformer and Auto-Knee CurveACM Transactions on Multimedia Computing, Communications, and Applications10.1145/366465320:8(1-23)Online publication date: 29-Jun-2024
  • (2024)Mobile Near-infrared Sensing—A Systematic Review on Devices, Data, Modeling, and ApplicationsACM Computing Surveys10.1145/365259656:8(1-36)Online publication date: 10-Apr-2024
  • (2024)Rip-NeRF: Anti-aliasing Radiance Fields with Ripmap-Encoded Platonic SolidsACM SIGGRAPH 2024 Conference Papers10.1145/3641519.3657402(1-11)Online publication date: 13-Jul-2024
  • (2024)PSC diffusion: patch-based simplified conditional diffusion model for low-light image enhancementMultimedia Systems10.1007/s00530-024-01391-z30:4Online publication date: 21-Jun-2024
  • (2023)InfoPrintProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36109337:3(1-29)Online publication date: 27-Sep-2023
  • (2023)Interactive NeRF Geometry Editing With Shape PriorsIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2023.331506845:12(14821-14837)Online publication date: 15-Sep-2023

View Options

Get Access

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Full Text

View this article in Full Text.

Full Text

HTML Format

View this article in HTML Format.

HTML Format

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media