Abstract
An experimental analysis is proposed concerning the use of physiological signals, specifically remote Photoplethysmography (rPPG), as a potential means for detecting Deepfakes (DF). The study investigates the effects of different variables, such as video compression and face swap quality, on rPPG information extracted from both original and forged videos. The experiments aim to understand the impact of face forgery procedures on remotely-estimated cardiac information, how this effect interacts with other variables, and how rPPG-based DF detection accuracy is affected by these quantities. Preliminary results suggest that cardiac information in some cases (e.g. uncompressed videos) may have a limited role in discriminating real videos from forged ones, but the effects of other physiological signals cannot be discounted. Surprisingly, heart rate related frequencies appear to deliver a significant contribution to the DF detection task in compressed videos.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Lee, S.-H., Yun, G.-E., Lim, M.Y., Lee, Y.K.: A study on effective use of bpm information in deepfake detection. In: 2021 International Conference on Information and Communication Technology Convergence (ICTC), pp. 425–427. IEEE (2021)
Lee, D.: Deepfake Salvador Dalí takes selfies with museum visitors. The Verge (2019)
Bursic, S., D’Amelio, A., Granato, M., Grossi, G., Lanzarotti, R.: A quantitative evaluation framework of video de-identification methods. In: 2020 25th International Conference on Pattern Recognition (ICPR), pp. 6089–6095. IEEE (2021)
Mirsky, Y., Lee, W.: The creation and detection of deepfakes: a survey. ACM Comput. Surv. (CSUR) 54(1), 1–41 (2021)
Suwajanakorn, S., Seitz, S.M., Kemelmacher-Shlizerman, I.: Synthesizing obama: learning lip sync from audio. ACM Trans. Graph. (ToG) 36(4), 1–13 (2017)
Tolosana, R., Vera-Rodriguez, R., Fierrez, J., Morales, A., Ortega-Garcia, J.: Deepfakes and beyond: a survey of face manipulation and fake detection. Inf. Fusion 64, 131–148 (2020)
Nguyen, T.T., Nguyen, C.M., Nguyen, D.T., Nguyen, D.T., Nahavandi, S.: Deep learning for deepfakes creation and detection: a survey. arXiv preprint arXiv:1909.11573 (2019)
McDuff, D.: Camera measurement of physiological vital signs. ACM Comput. Surv. 55(9), 1–40 (2023)
Boccignone, G., Conte, D., Cuculo, V., D’Amelio, A., Grossi, G., Lanzarotti, R.: An open framework for remote-PPG methods and their assessment. IEEE Access 8, 216083–216103 (2020)
Boccignone, G., et al.: pyvhr: a python framework for remote photoplethysmography. PeerJ Comput. Sci. 8, e929 (2022)
McDuff, D.J., Blackford, E.B., Estepp, J.R.: The impact of video compression on remote cardiac pulse measurement using imaging photoplethysmography. In: 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017), pp. 63–70. IEEE (2017)
Qi, H., et al.: DeepRhythm: exposing deepfakes with attentional visual heartbeat rhythms. In: Proceedings of the 28th ACM International Conference on Multimedia, pp. 4318–4327 (2020)
Liang, J., Deng, W.: Identifying rhythmic patterns for face forgery detection and categorization. In: 2021 IEEE International Joint Conference on Biometrics (IJCB), pp. 1–8 (2021)
Hernandez-Ortega, J., Tolosana, R., Fierrez, J., Morales, A.: DeepFakesON-Phys: deepfakes detection based on heart rate estimation. arXiv preprint arXiv:2010.00400 (2020)
Ciftci, U.A., Demir, I., Yin, L.: FakeCatcher: detection of synthetic portrait videos using biological signals. IEEE Trans. Pattern Anal. Mach. Intell. (2020)
Ciftci, U.A., Demir, I., Yin, L.: How do the hearts of deep fakes beat? deep fake source detection via interpreting residuals with biological signals. In: 2020 IEEE International Joint Conference on Biometrics (IJCB), pp. 1–10. IEEE (2020)
Boccignone, G., et al.: Deepfakes have no heart: a simple rppg-based method to reveal fake videos. In: Image Analysis and Processing-ICIAP 2022: 21st International Conference, Lecce, Italy, 23–27 May 2022, Proceedings, Part II, pp. 186–195. Springer, Heidelberg (2022). https://doi.org/10.1007/978-3-031-06430-2_16
Liang, J., Deng, W.: Identifying rhythmic patterns for face forgery detection and categorization. In: 2021 IEEE International Joint Conference on Biometrics (IJCB), pp. 1–8. IEEE (2021)
Wu, J., Zhu, Y., Jiang, X., Liu, Y., Lin, J.: Local attention and long-distance interaction of rppg for deepfake detection. Visual Comput., 1–12 (2023)
Rössler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., Nießner, M.: FaceForensics++: learning to detect manipulated facial images. In: International Conference on Computer Vision (ICCV) (2019)
Dolhansky, B., et al.: The deepfake detection challenge (DFDC) dataset. arXiv preprint arXiv:2006.07397 (2020)
Li, Y., Yang, X., Sun, P., Qi, H., Lyu, S.: Celeb-df: a large-scale challenging dataset for deepfake forensics. In: IEEE Conference on Computer Vision and Patten Recognition (CVPR) (2020)
Jiang, L., Li, R., Wu, W., Qian, C., Loy, C.C.: DeeperForensics-1.0: a large-scale dataset for real-world face forgery detection. In: CVPR (2020)
Bobbia, S., Macwan, R., Benezeth, Y., Mansouri, A., Dubois, J.: Unsupervised skin tissue segmentation for remote photoplethysmography. Pattern Recogn. Lett. 124, 82–90 (2019)
Liu, M.-Y., Breuel, T., Kautz, J.: Unsupervised image-to-image translation networks. Adv. Neural Inf. Process. Syst. 30, 1–9 (2017)
Boccignone, G., D’Amelio, A., Ghezzi, O., Grossi, G., Lanzarotti, R.: An evaluation of non-contact photoplethysmography-based methods for remote respiratory rate estimation. Sensors 23(7), 3387 (2023)
Verkruysse, W., Svaasand, L.O., Nelson, J.S.: Remote plethysmographic imaging using ambient light. Opt. Express 16(26), 21434–21445 (2008)
Wang, W., den Brinker, A.C., Stuijk, S., De Haan, G.: Algorithmic principles of remote PPG. IEEE Trans. Biomed. Eng. 64(7), 1479–1491 (2016)
De Haan, G., Jeanne, V.: Robust pulse rate from chrominance-based rPPG. IEEE Trans. Biomed. Eng. 60(10), 2878–2886 (2013)
Kruschke, J.K.: Bayesian estimation supersedes the t test. J. Exp. Psychol. Gener. 142(2), 573 (2013)
Benavoli, A., Corani, G., Demšar, J., Zaffalon, M.: Time for a change: a tutorial for comparing multiple classifiers through bayesian analysis. J. Mach. Learn. Res. 18(1), 2653–2688 (2017)
Benavoli, A., Corani, G., Mangili, F., Zaffalon, M., Ruggeri, F.: A Bayesian Wilcoxon signed-rank test based on the Dirichlet process. In: International Conference on Machine Learning, pp. 1026–1034. PMLR (2014)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
D’Amelio, A. et al. (2023). On Using rPPG Signals for DeepFake Detection: A Cautionary Note. In: Foresti, G.L., Fusiello, A., Hancock, E. (eds) Image Analysis and Processing – ICIAP 2023. ICIAP 2023. Lecture Notes in Computer Science, vol 14234. Springer, Cham. https://doi.org/10.1007/978-3-031-43153-1_20
Download citation
DOI: https://doi.org/10.1007/978-3-031-43153-1_20
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-43152-4
Online ISBN: 978-3-031-43153-1
eBook Packages: Computer ScienceComputer Science (R0)