research-article

Public Access

Thru-the-wall Eavesdropping on Loudspeakers via RFID by Capturing Sub-mm Level Vibration

Authors:

Sanglu LuAuthors Info & Claims

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Volume 5, Issue 4

Article No.: 182, Pages 1 - 25

https://doi.org/10.1145/3494975

Published: 30 December 2021 Publication History

Abstract

The unprecedented success of speech recognition methods has stimulated the wide usage of intelligent audio systems, which provides new attack opportunities for stealing the user privacy through eavesdropping on the loudspeakers. Effective eavesdropping methods employ a high-speed camera, relying on LOS to measure object vibrations, or utilize WiFi MIMO antenna array, requiring to eavesdrop in quiet environments. In this paper, we explore the possibility of eavesdropping on the loudspeaker based on COTS RFID tags, which are prevalently deployed in many corners of our daily lives. We propose Tag-Bug that focuses on the human voice with complex frequency bands and performs the thru-the-wall eavesdropping on the loudspeaker by capturing sub-mm level vibration. Tag-Bug extracts sound characteristics through two means: (1) Vibration effect, where a tag directly vibrates caused by sounds; (2) Reflection effect, where a tag does not vibrate but senses the reflection signals from nearby vibrating objects. To amplify the influence of vibration signals, we design a new signal feature referred as Modulated Signal Difference (MSD) to reconstruct the sound from RF-signals. To improve the quality of the reconstructed sound for human voice recognition, we apply a Conditional Generative Adversarial Network (CGAN) to recover the full-frequency band from the partial-frequency band of the reconstructed sound. Extensive experiments on the USRP platform show that Tag-Bug can successfully capture the monotone sound when the loudness is larger than 60dB. Tag-Bug can efficiently recognize the numbers of human voice with 95.3%, 85.3% and 87.5% precision in the free-space eavesdropping, thru-the-brick-wall eavesdropping and thru-the-insulating-glass eavesdropping, respectively. Tag-Bug can also accurately recognize the letters with 87% precision in the free-space eavesdropping.

Supplementary Material

wang (wang.zip)

Supplemental movie, appendix, image and software files for, Thru-the-wall Eavesdropping on Loudspeakers via RFID by Capturing Sub-mm Level Vibration

Download
299.41 KB

References

[1]

2018. USRP SDR reader. https://github.com/nkargas/Gen2-UHF-RFID-Reader.

[2]

2019. EPC Gen2, EPCglobal. https://www.gs1.org/epcglobal.

[3]

2019. Impinj, Inc. http://www.impinj.com/.

[4]

2019. USRP reader. https://github.com/nkargas/Gen2-UHF-RFID-Reader.

[5]

Zhenlin An, Qiongzheng Lin, and Lei Yang. 2018. Cross-Frequency Communication: Near-Field Identification of UHF RFIDs with WiFi!. In Proc. of ACM Mobicom.

Digital Library

[6]

S Abhishek Anand and Nitesh Saxena. 2018. Speechless: Analyzing the threat to speech privacy from smartphone motion sensors. In 2018 IEEE Symposium on Security and Privacy (SP). IEEE, 1000--1017.

[7]

Dmitri Asonov and Rakesh Agrawal. 2004. Keyboard acoustic emanations. In Proceedings of IEEE Symposium on Security and Privacy. IEEE, 3--11.

[8]

Michael Backes, Markus Dürmuth, Sebastian Gerling, Manfred Pinkal, and Caroline Sporleder. 2010. Acoustic Side-Channel Attacks on Printers. In USENIX Security symposium. 307--322.

Digital Library

[9]

P Castellini, M Martarelli, and EP Tomasini. 2006. Laser Doppler Vibrometry: Development of advanced solutions answering to technology's needs. Mechanical systems and signal processing 20, 6 (2006), 1265--1285.

[10]

Abe Davis, Michael Rubinstein, Neal Wadhwa, Gautham J Mysore, Frédo Durand, and William T Freeman. 2014. The visual microphone: passive recovery of sound from video. (2014).

[11]

Han Ding, Chen Qian, Jinsong Han, Ge Wang, Zhiping Jiang, Jizhong Zhao, and Wei Xi. 2016. Device-free detection of approach and departure behaviors using backscatter communication. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, 167--177.

Digital Library

[12]

Han Ding, Longfei Shangguan, Zheng Yang, Jinsong Han, Zimu Zhou, Panlong Yang, Wei Xi, and Jizhong Zhao. 2015. Femo: A platform for free-weight exercise monitoring with rfids. In Proc. of ACM SenSys.

Digital Library

[13]

Daniel M Dobkin. 2012. The rf in RFID: uhf RFID in practice. Newnes.

[14]

Chunhui Duan, Lei Yang, Huanyu Jia, Qiongzheng Lin, Yunhao Liu, and Lei Xie. 2018. Robust spinning sensing with dual-rfid-tags in noisy settings. Proc. of IEEE INFOCOM.

Digital Library

[15]

Chengkun Jiang, Yuan He, Xiaolong Zheng, and Yunhao Liu. 2018. Orientation-aware RFID tracking with centimeter-level accuracy. In Proceedings of the 17th ACM/IEEE International Conference on Information Processing in Sensor Networks. IEEE Press, 290--301.

Digital Library

[16]

Wolfgang Klippel and Joachim Schlechter. 2006. Measurement and visualization of loudspeaker cone vibration. In Audio Engineering Society Convention 121. Audio Engineering Society.

[17]

Andrew Kwong, Wenyuan Xu, and Kevin Fu. 2019. Hard Drive of Hearing: Disks that Eavesdrop with a Synthesized Microphone. In 2019 IEEE Symposium on Security and Privacy (SP). IEEE, 905--919.

[18]

Yann LeCun, Bernhard Boser, John S Denker, Donnie Henderson, Richard E Howard, Wayne Hubbard, and Lawrence D Jackel. 1989. Backpropagation applied to handwritten zip code recognition. Neural computation 1, 4 (1989), 541--551.

Digital Library

[19]

Huining Li, Chenhan Xu, Aditya Singh Rathore, Zhengxiong Li, Hanbin Zhang, Chen Song, Kun Wang, Lu Su, Feng Lin, Kui Ren, et al. 2020. VocalPrint: exploring a resilient and secure voice authentication via mmWave biometric interrogation. In Proceedings of the 18th Conference on Embedded Networked Sensor Systems. 312--325.

Digital Library

[20]

Ping Li, Zhenlin An, Lei Yang, and Panlong Yang. 2019. Towards Physical-Layer Vibration Sensing with RFIDs. In Proc. of IEEE INFOCOM.

Digital Library

[21]

Jia Liu, Min Chen, Shigang Chen, Qingfeng Pan, and Lijun Chen. 2017. Tag-compass: Determining the spatial direction of an object with small dimensions. In Proceedings of IEEE Conference on Computer Communications (INFOCOM). 1--9.

[22]

Jia Liu, Xingyu Chen, Shigang Chen, Xiulong Liu, Yanyan Wang, and Lijun Chen. 2019. TagSheet: Sleeping posture recognition with an unobtrusive passive tag matrix. In Proceedings of IEEE Conference on Computer Communications (INFOCOM). 874--882.

Digital Library

[23]

Jian Liu, Yan Wang, Gorkem Kar, Yingying Chen, Jie Yang, and Marco Gruteser. 2015. Snooping keystrokes with mm-level audio ranging on a single phone. In Proceedings of the 21st Annual International Conference on Mobile Computing and Networking. ACM, 142--154.

Digital Library

[24]

Jia Liu, Feng Zhu, Yanyan Wang, Xia Wang, Qingfeng Pan, and Lijun Chen. 2017. RF-scanner: Shelf scanning with robot-assisted RFID systems. In Proceedings of IEEE Conference on Computer Communications (INFOCOM). 1--9.

[25]

Yunfei Ma, Nicholas Selby, and Fadel Adib. 2017. Minding the billions: Ultra-wideband localization for deployed rfid tags. In Proceedings of the 23rd Annual International Conference on Mobile Computing and Networking. ACM, 248--260.

Digital Library

[26]

Yan Michalevsky, Dan Boneh, and Gabi Nakibly. 2014. Gyrophone: Recognizing speech from gyroscope signals. In Proc. of 23rd USENIX Security Symposium (USENIX Security 14). 1053--1067.

[27]

Mehdi Mirza and Simon Osindero. 2014. Conditional generative adversarial nets. arXiv 2014. arXiv preprint arXiv:1411.1784 (2014).

[28]

Olaf Ronneberger, Philipp Fischer, and Thomas Brox. 2015. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention. Springer, 234--241.

[29]

Pascal Scalart et al. 1996. Speech enhancement based on a priori signal to noise estimation. In Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing Conference, Vol. 2. IEEE, 629--632.

[30]

Longfei Shangguan and Kyle Jamieson. 2016. Leveraging Electromagnetic Polarization in a Two-Antenna Whiteboard in the Air. In Proceedings of the 12th International on Conference on emerging Networking EXperiments and Technologies. ACM, 443--456.

Digital Library

[31]

Claude Elwood Shannon. 1948. A mathematical theory of communication. Bell system technical journal 27, 3 (1948), 379--423.

[32]

Michael Vorländer. 2007. Auralization: fundamentals of acoustics, modelling, simulation, algorithms and acoustic virtual reality. Springer Science & Business Media.

Digital Library

[33]

Chuyu Wang, Lei Xie, Wei Wang, Tao Xue, and Sanglu Lu. 2016. Moving Tag Detection via Physical Layer Analysis for Large-Scale RFID Systems. In Proc. of IEEE INFOCOM.

Digital Library

[34]

Jue Wang, Deepak Vasisht, and Dina Katabi. 2015. RF-IDraw: virtual touch screen in the air using RF signals. In Proc. of ACM SIGCOMM.

Digital Library

[35]

Junjue Wang, Kaichen Zhao, Xinyu Zhang, and Chunyi Peng. 2014. Ubiquitous keyboard for small mobile devices: harnessing multipath fading for fine-grained keystroke localization. In Proceedings of the 12th annual international conference on Mobile systems, applications, and services. ACM, 14--27.

Digital Library

[36]

Zhongqin Wang, Fu Xiao, Ning Ye, Ruchuan Wang, and Panlong Yang. 2018. A see-through-wall system for device-free human motion sensing based on battery-free RFID. ACM Transactions on Embedded Computing Systems (TECS) 17, 1 (2018), 6.

Digital Library

[37]

Teng Wei, Shu Wang, Anfu Zhou, and Xinyu Zhang. 2015. Acoustic eavesdropping through wireless vibrometry. In Proc. of ACM Mobicom. ACM, 130--141.

Digital Library

[38]

Teng Wei and Xinyu Zhang. 2016. Gyro in the air: tracking 3D orientation of batteryless internet-of-things. In Proc. of ACM Mobicom. 55--68.

Digital Library

[39]

Lei Yang, Yekui Chen, Xiang-Yang Li, Chaowei Xiao, Mo Li, and Yunhao Liu. 2014. Tagoram: Real-time tracking of mobile RFID tags to high precision using COTS devices. In Proc. of ACM MobiCom.

Digital Library

[40]

Lei Yang, Yao Li, Qiongzheng Lin, Xiang-Yang Li, and Yunhao Liu. 2016. Making sense of mechanical vibration period with sub-millisecond accuracy using backscatter signals. In Proc. of ACM Mobicom. ACM, 16--28.

Digital Library

[41]

Panlong Yang, Yuanhao Feng, Jie Xiong, Ziyang Chen, and Xiang-Yang Li. 2020. RF-Ear: Contactless Multi-device Vibration Sensing and Identification Using COTS RFID. In Proc. of IEEE INFOCOM.

Digital Library

[42]

Yilun Zheng, Yuan He, Meng Jin, Xiaolong Zheng, and Yunhao Liu. 2018. RED: RFID-based eccentricity detection for high-speed rotating machinery. In IEEE INFOCOM 2018-IEEE Conference on Computer Communications. IEEE, 1565--1573.

Digital Library

Cited By

Rong CDing JLi Y(2024)An Interdisciplinary Survey on Origin-destination Flows Modeling: Theory and TechniquesACM Computing Surveys10.1145/368205857:1(1-49)Online publication date: 26-Jul-2024
https://dl.acm.org/doi/10.1145/3682058
Chen YYu JChen YKong LZhu YChen YOkoshi TKo JLiKamWa R(2024)RFSpy: Eavesdropping on Online Conversations with Out-of-Vocabulary Words by Sensing Metal Coil Vibration of Headsets Leveraging RFIDProceedings of the 22nd Annual International Conference on Mobile Systems, Applications and Services10.1145/3643832.3661887(169-182)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3643832.3661887
Duan DSun ZNi TLi SJia XXu WLi TOkoshi TKo JLiKamWa R(2024)F2Key: Dynamically Converting Your Face into a Private Key Based on COTS Headphones for Reliable Voice InteractionProceedings of the 22nd Annual International Conference on Mobile Systems, Applications and Services10.1145/3643832.3661860(127-140)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3643832.3661860
Show More Cited By

Index Terms

Thru-the-wall Eavesdropping on Loudspeakers via RFID by Capturing Sub-mm Level Vibration
1. Networks
  1. Network types
    1. Cyber-physical networks
2. Security and privacy
  1. Network security
    1. Mobile and wireless security

Recommendations

Practical eavesdropping and skimming attacks on high-frequency RFID tokens
2010 Workshop on RFID Security (RFIDSec'10 Asia)

RFID systems often use near-field magnetic coupling to implement communication channels. The advertised operational range of these channels is less than 10 cm and therefore several implemented systems assume that the communication channel is location ...
Practical eavesdropping and skimming attacks on high-frequency RFID tokens
2010 Workshop on RFID Security (RFIDSec'10 Asia)

RFID systems often use near-field magnetic coupling to implement communication channels. The advertised operational range of these channels is less than 10 cm and therefore several implemented systems assume that the communication channel is location ...
RFSpy: Eavesdropping on Online Conversations with Out-of-Vocabulary Words by Sensing Metal Coil Vibration of Headsets Leveraging RFID
MOBISYS '24: Proceedings of the 22nd Annual International Conference on Mobile Systems, Applications and Services

Eavesdropping on human sound is one of the most common but harmful ways to threaten personal privacy. As one of the most essential accessories, headsets have been widely used in common online conversations, such as online calls, video meetings, etc. The ...

Comments

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies Volume 5, Issue 4

Dec 2021

1307 pages

EISSN:2474-9567

DOI:10.1145/3508492

Issue’s Table of Contents

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 30 December 2021

Published in IMWUT Volume 5, Issue 4

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

National Natural Science Foundation of China
the Key K&D Program of Jiangsu Province
JiangSu Natural Science Foundation
National Science Foundation

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

69
Total Citations
View Citations
1,075
Total Downloads

Downloads (Last 12 months)409
Downloads (Last 6 weeks)44

Reflects downloads up to 16 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Rong CDing JLi Y(2024)An Interdisciplinary Survey on Origin-destination Flows Modeling: Theory and TechniquesACM Computing Surveys10.1145/368205857:1(1-49)Online publication date: 26-Jul-2024
https://dl.acm.org/doi/10.1145/3682058
Chen YYu JChen YKong LZhu YChen YOkoshi TKo JLiKamWa R(2024)RFSpy: Eavesdropping on Online Conversations with Out-of-Vocabulary Words by Sensing Metal Coil Vibration of Headsets Leveraging RFIDProceedings of the 22nd Annual International Conference on Mobile Systems, Applications and Services10.1145/3643832.3661887(169-182)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3643832.3661887
Duan DSun ZNi TLi SJia XXu WLi TOkoshi TKo JLiKamWa R(2024)F2Key: Dynamically Converting Your Face into a Private Key Based on COTS Headphones for Reliable Voice InteractionProceedings of the 22nd Annual International Conference on Mobile Systems, Applications and Services10.1145/3643832.3661860(127-140)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3643832.3661860
Wang CFeng YZhong LZhu SZhang CZheng SLiang CWang YHe CYu CShi Y(2024)UbiPhysioProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36435528:1(1-27)Online publication date: 6-Mar-2024
https://dl.acm.org/doi/10.1145/3643552
Wang ZWang YYang J(2024)EarSlideProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36435158:1(1-29)Online publication date: 6-Mar-2024
https://dl.acm.org/doi/10.1145/3643515
Ahmad NLeung H(2024)HyperHARProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36435118:1(1-29)Online publication date: 6-Mar-2024
https://dl.acm.org/doi/10.1145/3643511
Xu ZLiu TJiang RHu PGuo ZLiu C(2024)AFaceProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36435108:1(1-33)Online publication date: 6-Mar-2024
https://dl.acm.org/doi/10.1145/3643510
Jiang FWang HLi Y(2024)VesNet: A Vessel Network for Jointly Learning Route Pattern and Future TrajectoryACM Transactions on Intelligent Systems and Technology10.1145/363937015:2(1-25)Online publication date: 28-Mar-2024
https://dl.acm.org/doi/10.1145/3639370
Zhang XZhang DXie YWu DLi YZhang D(2024)WaffleProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36314587:4(1-29)Online publication date: 12-Jan-2024
https://dl.acm.org/doi/10.1145/3631458
Duan DChen YXu WLi T(2024)EarSEProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36314477:4(1-33)Online publication date: 12-Jan-2024
https://dl.acm.org/doi/10.1145/3631447
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

Media

Figures

Other

Tables

View Issue’s Table of Contents