Multilingual Handwritten Signature Recognition Based on High-Dimensional Feature Fusion
<p>Offline signature authentication system model.</p> "> Figure 2
<p>Some multilingual offline signatures; an example of partial genuine signatures in CEDAR, Chinese(Han), Uyghur, and Kazakh languages.</p> "> Figure 3
<p>Preprocessing signature image: (<b>a</b>) initial image; (<b>b</b>)normalization image; (<b>c</b>) denoise image; (<b>d</b>) binary image.</p> "> Figure 4
<p>Offline handwritten signature recognition process.</p> "> Figure 5
<p>LBP operator coding process diagram.</p> "> Figure 6
<p>SILTP algorithm coding process.</p> "> Figure 7
<p>HOG feature extraction flowchart.</p> "> Figure 8
<p>Schematic diagram of the decision tree.</p> "> Figure 9
<p>Steps of random forest construction.</p> "> Figure 10
<p>Histogram of recognition results of different values for LOMO feature selection.</p> "> Figure 11
<p>Recognition results of HOG features of different dimensions.</p> "> Figure 12
<p>Experimental results of CEDAR signature extraction with different features.</p> ">
Abstract
:1. Introduction
2. Offline Handwritten Signature Recognition
2.1. Signature Datasets
2.2. Preprocessing
3. Proposed Method
3.1. Feature Extraction
3.1.1. Local Maximal Occurrence Representation
3.1.2. Histogram of Oriented Gradients
3.2. Classification
3.2.1. Principal Component Analysis
3.2.2. Random Forest
3.2.3. K Nearest Neighbors
4. Experimental Results
4.1. Evaluation Criteria
4.2. Analysis of Experimental Results
4.2.1. Experimental Results Based on the LOMO Method
4.2.2. Recognition Results Based on HOG Feature Extraction
4.2.3. Results of a Handwriting Signature Recognition Approach Based on Fused Features
4.3. Comparison of Experimental Results
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Engin, D.; Kantarci, A.; Arslan, S.; Ekenel, H.K. Offline signature verification on real-world documents. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, Seattle, WA, USA, 14–19 June 2020; pp. 808–809. [Google Scholar]
- Liang, Y.; Samtani, S.; Guo, B.; Yu, Z. Behavioral biometrics for continuous authentication in the internet-of-things era: An artificial intelligence perspective. IEEE Internet Things J. 2020, 7, 9128–9143. [Google Scholar] [CrossRef]
- Barrere, K.; Soullard, Y.; Lemaitre, A.; Coüasnon, B.B. A Light Transformer-Based Architecture for Handwritten Text Recognition. In Proceedings of the 15th IAPR International Workshop on Document Analysis Systems (DAS 2022), La Rochelle, France, 22–25 May 2022; pp. 275–290. [Google Scholar] [CrossRef]
- Soleimani, A.; Araabi, B.N.; Fouladi, K. Deep multitask metric learning for offline signature verification. Pattern Recognit. Lett. 2016, 80, 84–90. [Google Scholar] [CrossRef]
- Sharif, M.; Khan, M.A.; Faisal, M.; Yasmin, M.; Fernandes, S.L. A framework for offline signature verification system: Best features selection approach. Pattern Recognit. Lett. 2020, 139, 50–59. [Google Scholar] [CrossRef]
- Hadjadji, B.; Chibani, Y.; Nemmour, H. An efficient open system for offline handwritten signature identification based on curvelet transform and one-class principal component analysis. Neurocomputing 2017, 265, 66–77. [Google Scholar] [CrossRef]
- Mshir, S.; Kaya, M. Signature recognition using machine learning. In Proceedings of the 2020 8th International Symposium on Digital Forensics and Security (ISDFS), Beirut, Lebanon, 1–2 June 2020; IEEE: Piscataway, NJ, USA, 2020; pp. 1–4. [Google Scholar]
- Elssaedi, M.M.; Salih, O.M.; Ahmeed, A. Comparing the Effectiveness of Different Classifiers of Data Mining for Signature Recognition System. In Proceedings of the 6th International Conference on Engineering & MIS, Almaty Kazakhstan, 14–16 September 2020; pp. 1–4. [Google Scholar]
- Jagtap, A.B.; Hegadi, R.S. Offline handwritten signature recognition based on the upper and lower envelope using eigen values. In Proceedings of the 2017 World Congress on Computing and Communication Technologies (WCCCT), Tiruchirappalli, Nadu, India, 2–4 February 2017; IEEE: Piscataway, NJ, USA, 2017; pp. 223–226. [Google Scholar]
- Matsuda, K.; Ohyama, W.; Wakabayashi, T. Multilingual-signature verification by verifier fusion using random forests. In Proceedings of the 2017 4th IAPR Asian Conference on Pattern Recognition (ACPR), Nanjing, China, 26–29 November 2017; IEEE: Piscataway, NJ, USA, 2017; pp. 941–946. [Google Scholar]
- Zhang, S.J.; Aysa, Y.; Ubul, K. BOVW based feature selection for Uyghur offline signature verification. In Chinese Conference on Biometric Recognition; Springer: Cham, Switzerland, 2018; pp. 700–708. [Google Scholar]
- Kalera, M.K.; Srihari, S.; Xu, A. Offline signature verification and identification using distance statistics. Int. J. Pattern Recognit. Artif. Intell. 2004, 18, 1339–1360. [Google Scholar] [CrossRef] [Green Version]
- Tuncer, T.; Aydemir, E.; Ozyurt, F.; Dogan, S. A deep feature warehouse and iterative MRMR based handwritten signature verification method. Multimed. Tools Appl. 2022, 81, 3899–3913. [Google Scholar] [CrossRef]
- Ding, C.; Peng, H. Minimum redundancy feature selection from microarray gene expression data. J. Bioinform. Comput. Biol. 2005, 3, 185–205. [Google Scholar] [CrossRef]
- Li, H.; Wei, P.; Hu, P. AVN: An Adversarial Variation Network Model for Handwritten Signature Verification. IEEE Trans. Multimed. 2021, 24, 594–608. [Google Scholar] [CrossRef]
- Zhang, X.; Wu, Z.; Xie, L.; Li, Y.; Li, F.; Zhang, J. Multi-Path Siamese Convolution Network for Offline Handwritten Signature Verification. In Proceedings of the 8th International Conference on Computing and Data Engineering, Bangkok, Thailand, 11–13 January 2022; pp. 51–58. [Google Scholar]
- Zeng, Z. Multi-scale Attention-based Individual Character Network for Handwritten Signature Verification. In Proceedings of the 3rd International Conference on Computer Vision, Image and Deep Learning & International Conference on Computer Engineering and Applications (CVIDL & ICCEA), Changchun, China, 20–22 May 2022; IEEE: Piscataway, NJ, USA, 2022; pp. 1–5. [Google Scholar]
- Jain, V.; Gupta, P.; Chaudhry, A.; Batra, M.; Hemanth, D.J. A Modified Deep Convolution Siamese Network for WRITER-Independent Signature Verification. Int. J. Uncertain. Fuzziness Knowl.-Based Syst. 2022, 30, 479–498. [Google Scholar] [CrossRef]
- Tsourounis, D.; Theodorakopoulos, I.; Zois, E.N.; Economou, G. From text to signatures: Knowledge transfer for efficient deep feature learning in offline signature verification. Expert Syst. Appl. 2022, 189, 116136. [Google Scholar] [CrossRef]
- Krizhevsky, A.; Sutskever, I.; Hinton, G.E. Imagenet classification with deep convolutional neural networks. Commun. ACM 2017, 60, 84–90. [Google Scholar] [CrossRef] [Green Version]
- Hafemann, L.G.; Sabourin, R.; Oliveira, L.S. Learning features for offline handwritten signature verification using deep convolutional neural networks. Pattern Recognit. 2017, 70, 163–176. [Google Scholar] [CrossRef] [Green Version]
- Pal, S.; Alaei, A.; Pal, U. Blumenstein, Performance of an off-line signature verification method based on texture features on a large indic-script signature dataset. In Proceedings of the 2016 12th IAPR workshop on document analysis systems (DAS), Santorini, Greece, 11–14 April 2016; pp. 72–77. [Google Scholar]
- Ferrer, M.A.; Vargas, J.F.; Morales, A.; Ordonez, A. Robustness of offline signature verification based on gray level features. IEEE Trans. Inf. Forensics Secur. 2012, 7, 966–977. [Google Scholar] [CrossRef]
- Liao, S.; Hu, Y.; Zhu, X.; Li, S.Z. Person re-identification by local maximal occurrence representation and metric learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA, 7–12 June 2015; pp. 2197–2206. [Google Scholar]
- Li, Y.; Li, Z.; Li, J.; Jing, Z. A Hierarchical Scheme for Video-Based Person Re-identification Using Lightweight PCANet and Handcrafted LOMO Features. Chin. J. Electron. 2021, 30, 289–295. [Google Scholar]
- Dalal, N.; Triggs, B. Histograms of oriented gradients for human detection. In Proceedings of the 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR’05), San Diego, CA, USA, 21–23 September 2005; IEEE: Piscataway, NJ, USA, 2005; Volume 1, pp. 886–893. [Google Scholar]
- Liao, S.; Zhao, G.; Kellokumpu, V.; Pietikäinen, M.; Li, S.Z. Modeling pixel process with scale invariant local patterns for background subtraction in complex scenes. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Francisco, CA, USA, 17–19 June 2010; p. 3. [Google Scholar]
- Kuang, Y.; Guo, M.; Peng, Y.; Pei, Z. Learner posture recognition via a fusing model based on improved SILTP and LDP. Multimed. Tools Appl. 2019, 78, 30443–30456. [Google Scholar] [CrossRef]
- Song, M.; Jung, H.; Lee, S.; Kim, D.; Ahn, M. Diagnostic classification and biomarker identification of Alzheimer’s disease with random forest algorithm. Brain Sci. 2021, 11, 453. [Google Scholar] [CrossRef] [PubMed]
- Fan, Z.; Xie, J.K.; Wang, Z.Y.; Liu, P.C.; Qu, S.J.; Huo, L. Image classification method based on improved KNN algorithm. J. Phys. Conf. Ser. 2021, 1930, 012009. [Google Scholar] [CrossRef]
- Gheni, Z.; Mahpirat, N.Y.; Ubul, K. Uyghur off-line signature verification based on the directional features. Image Signal Process 2017, 6, 121–129. [Google Scholar]
- Ubul, K.; Adler, A.; Abliz, G.; Yasheng, M.; Hamdulla, A. Off-line Uyghur signature recognition based on modified grid information features. In Proceedings of the 2012 11th International Conference on Information Science, Signal Processing and their Applications (ISSPA), Montreal, QC, Canada, 2–5 July 2012; pp. 1056–1061. [Google Scholar]
- Xamxidin, N.; Yao, Z.; Aysa, A.; Ubul, K. Multilingual Offline Signature Verification Based on Improved Inverse Discriminator Network. Information 2022, 13, 293. [Google Scholar] [CrossRef]
- Batool, F.E.; Attique, M.; Sharif, M.; Javed, K.; Nazir, M.; Abbasi, A.A.; Iqbal, Z.; Riaz, N. Offline signature verification system: A novel technique of fusion of GLCM and geometric features using SVM. Multimed Tools Appl. 2020, 1–20. [Google Scholar] [CrossRef]
- Souza, V.L.F.; Oliveira, A.L.I.; Cruz, R.M.O.; Sabourin, R. An investigation of feature selection and transfer learning for writer-independent offline handwritten signature verification. In Proceedings of the 2020 25th International Conference on Pattern Recognition (ICPR), Milan, Italy, 10–15 January 2021; IEEE: Piscataway, NJ, USA, 2021; pp. 7478–7485. [Google Scholar]
- Kumari, K.; Rana, S. Offline Signature Recognition Using Deep Features. In Machine Learning for Predictive Analysis; Springer: Singapore, 2021; pp. 405–421. [Google Scholar]
- Culqui-Culqui, G.; Sanchez-Gordon, S.; Hernández-Álvarez, M. An Algorithm for Classifying Handwritten Signatures Using Convolutional Networks. IEEE Lat. Am. Trans. 2022, 20, 465–473. [Google Scholar] [CrossRef]
Dataset | Signers | Genuine Signature/Signers | Forged Signatures/Signers | Numbers of Genuine Signatures |
---|---|---|---|---|
BHSig-B [22] | 100 | 24 | 30 | 2400 |
BHSig-H [22] | 160 | 24 | 30 | 3840 |
CEDAR [12] | 55 | 24 | 24 | 1320 |
GPDS [23] | 4000 | 24 | 30 | 96,000 |
Uyghur (Ours) | 160 | 24 | – | 3840 |
Kazakh (Ours) | 151 | 24 | – | 3624 |
Han (Ours) | 160 | 24 | – | 3840 |
Dataset | Dimension | Train Numbers | ||||
---|---|---|---|---|---|---|
100 | 128 | S = 14 | S = 15 | S = 16 | S = 18 | |
CEDAR | √ | 97.2 | 97.6 | 97.8 | 98.3 | |
√ | 97 | 97.3 | 97.2 | 97.7 | ||
Han (Chinese) | √ | 88.64 | 88.83 | 88.85 | 89.96 | |
√ | 88.01 | 88.1 | 88.22 | 88.91 | ||
Uyghur | √ | 90.49 | 91.1 | 91.7 | 92.24 | |
√ | 90.4 | 91.1 | 91.95 | 92.45 | ||
Kazakh | √ | 92.38 | 92.57 | 92.3 | 94.15 | |
√ | 92.85 | 92.94 | 92.72 | 94.59 | ||
Uyghur + Kazakh | √ | 86.33 | 86.68 | 87.37 | 87.83 | |
√ | 86.94 | 87.66 | 88.55 | 89.04 | ||
Han (Chinese) + Uyghur | √ | 82.49 | 83.23 | 83.13 | 83.91 | |
√ | 83.07 | 83.8 | 83.39 | 83.89 |
Dataset | Classifier | |||||
---|---|---|---|---|---|---|
S = 14 | S = 15 | S = 16 | ||||
KNN | RF | KNN | RF | KNN | RF | |
Han(Chinese) | 93.47 | 91.5 | 93.53 | 91.75 | 94.49 | 96.46 |
CEDAR | 98.32 | 95.19 | 94.65 | 95.99 | 94.21 | 93.48 |
Uyghur | 95.78 | 94.17 | 96.44 | 92.38 | 95.97 | 95.35 |
Kazakh | 97.09 | 95.18 | 97 | 92.38 | 96.74 | 94.88 |
Dataset | Dimension | Train Numbers | ||||
---|---|---|---|---|---|---|
100 | 128 | S = 14 | S = 15 | S = 16 | S = 18 | |
Han + Uyghur | √ | 93.86 | 93.83 | 94.36 | 94.46 | |
√ | 93.42 | 93.52 | 93.92 | 93.83 | ||
Uyghur + Kazakh | √ | 95.02 | 95.5 | 95.72 | 96.49 | |
√ | 94.13 | 94.68 | 94.8 | 95.83 |
Dataset | Train Numbers | |||
---|---|---|---|---|
S = 14 | S = 15 | S = 16 | S = 18 | |
CEDAR | 94.35 | 95.54 | 95.44 | 95.74 |
Han(Chinese) | 91.95 | 92.38 | 92.86 | 93.1 |
Uyghur | 93 | 94.55 | 94.47 | 96.65 |
Kazakh | 95.02 | 95.93 | 94.88 | 95.54 |
Han + Uyghur | 90.73 | 91.23 | 91.58 | 92.87 |
Han + Uyghur + Kazakh | 88.33 | 89.57 | 90.21 | 91.21 |
Uyghur + Kazakh | 90.37 | 91.23 | 91.58 | 92.87 |
Han + Kazakh | 91.13 | 91.2 | 91.63 | 92.48 |
S = 14 | S = 15 | S = 16 | S = 18 | |
---|---|---|---|---|
CEDAR | 94.22 | 94.08 | 94.33 | 93.99 |
Han (Chinese) | 93.74 | 93.95 | 95.11 | 94.98 |
Uyghur | 95.81 | 96.23 | 96.65 | 96.05 |
Kazakh | 96.9 | 96.28 | 96.8 | 96.76 |
Han + Uyghur | 94.74 | 95.34 | 95.46 | 96.38 |
Han + Uyghur + Kazakh | 93.62 | 94.4 | 94.89 | 95.11 |
Uyghur + Kazakh | 94.74 | 95.34 | 95.46 | 96.38 |
Han + Kazakh | 94.53 | 94.37 | 94.66 | 95.92 |
Method | Dimension | Train Numbers | ||||||
---|---|---|---|---|---|---|---|---|
100 | 128 | S = 15 | S = 16 | S = 18 | ||||
RF | KNN | RF | KNN | RF | KNN | |||
HOG | √ | 95.99 | 94.65 | 96.46 | 94.21 | 96.77 | 94.62 | |
√ | 96.26 | 92.78 | 96.14 | 92.28 | 97.85 | 91.4 | ||
LomoHOG | √ | 95.54 | 94.33 | 95.44 | 94.08 | 95.74 | 94.22 | |
√ | 95.55 | 91.3 | 94.76 | 91.34 | 96.66 | 90.58 | ||
LOMO | √ | 97.3 | 97.6 | 96.5 | 97.8 | 98.4 | 98.3 | |
√ | 97 | 97.3 | 97.1 | 97.2 | 97.8 | 97.7 |
Dataset | Method | Train Numbers | ||
---|---|---|---|---|
Han (Chinese) + Uyghur | S = 14 | S = 15 | S = 16 | |
LomoHOG | 90.73 | 91.23 | 91.58 | |
Lomo | 76.35 | 76.7 | 77.81 | |
HOG | 88.8 | 89.5 | 90.72 |
Dataset | Train Numbers | |||||
---|---|---|---|---|---|---|
S = 15 | S = 16 | S = 18 | ||||
RF | KNN | RF | KNN | RF | KNN | |
CEDAR | 95.55 | 91.3 | 94.76 | 91.34 | 96.66 | 90.58 |
Han (Chinese) | 90.92 | 93.01 | 91.98 | 93.86 | 92.47 | 94.77 |
Uyghur | 95.71 | 96.65 | 94.84 | 96.23 | 96.23 | 95.18 |
Kazakh | 95.54 | 97.29 | 96.16 | 96.98 | 95.54 | 97.67 |
Han + Uyghur | 91.01 | 95.39 | 91.12 | 95.46 | 91.22 | 94.46 |
Han + Uyghur + Kazakh | 89.18 | 94.75 | 90.43 | 95.06 | 91.06 | 95.32 |
Uyghur + Kazakh | 91.01 | 95.39 | 91.12 | 95.46 | 93.42 | 96.38 |
Han + Kazakh | 90.67 | 94.31 | 91.44 | 94.47 | 92.8 | 95.27 |
Reference | Database | Dataset Size/Training Data | Feature | Classifier | ACC |
---|---|---|---|---|---|
Aini, Z. et al. [31] | Uyghur | 600 samplestrain 300 samples | Orientation feature | Euclidean distance | 92.58% |
Shu-Jing, Z., et al. [11] | Uyghur | train 480 samples | MRMR | SVM | 93.81% |
Ubul, K., et al. [32] | Uyghur | 2500 samples, train 2000 samples | 4MCLF-48 and LCDC | weighted Manhattan distance | 94.60% |
Xamxidin, N. [33] | Uyghur | 4000 sample, train 2900 samples | - | IDN | 94.32% |
Ours | Uyghur | 3355 samples train 2240 samples | LomoHOG | KNN | 96.41% |
Ours | Uyghur + Kazakh | 6397 samples train 4269 samples | LomoHOG | KNN | 94.74% |
Reference | Database | Dataset Size/Training Data | Feature | Classifier | ACC |
---|---|---|---|---|---|
Batool, F. E., et al. [34] | CEDAR | 1320 samples Train 840 samples | GLCM | SVM | 96.45% |
Souza, V. L. F. et al. [35] | CEDAR | – | – | DCNN | 96.73% |
Kumari, K. et al. [36] | CEDAR | - | - | SVM | 94.9% |
Culqui-Culqui, G. et al. [37] | CEDAR | - | - | CNN-HDR | 93.92% |
Ours | CEDAR | 1320 samples train 825 samples | LomoHOG | RF | 96.66% |
Ours | CEDAR | 1320 samples train 825 samples | Lomo | RF | 98.4% |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Rexit, A.; Muhammat, M.; Xu, X.; Kang, W.; Aysa, A.; Ubul, K. Multilingual Handwritten Signature Recognition Based on High-Dimensional Feature Fusion. Information 2022, 13, 496. https://doi.org/10.3390/info13100496
Rexit A, Muhammat M, Xu X, Kang W, Aysa A, Ubul K. Multilingual Handwritten Signature Recognition Based on High-Dimensional Feature Fusion. Information. 2022; 13(10):496. https://doi.org/10.3390/info13100496
Chicago/Turabian StyleRexit, Aliya, Mahpirat Muhammat, Xuebin Xu, Wenxiong Kang, Alimjan Aysa, and Kurban Ubul. 2022. "Multilingual Handwritten Signature Recognition Based on High-Dimensional Feature Fusion" Information 13, no. 10: 496. https://doi.org/10.3390/info13100496