Exploiting Character Class Information in Forensic

Writer Identification

Fernando Alonso-Fernandez, Julian Fierrez, Javier Galbally, and Javier Ortega-Garcia

Biometric Recognition Group - ATVS, Escuela Politecnica Superior

Universidad Autonoma de Madrid, Avda. Francisco Tomas y Valiente, 11
Campus de Cantoblanco, 28049 Madrid, Spain
{fernando.alonso,julian.fierrez,
javier.galbally,javier.ortega}@uam.es
http://atvs.ii.uam.es

Abstract. Questioned document examination is extensively used by forensic spe-

cialists for criminal identification. This paper presents a writer recognition sys-
tem based on contour features operating in identification mode (one-to-many) and
working at the level of isolated characters. Individual characters of a writer are
manually segmented and labeled by an expert as pertaining to one of 62
alphanumeric classes (10 numbers and 52 letters, including lowercase and
uppercase letters), being the particular setup used by the forensic laboratory partic-
ipating in this work. Three different scenarios for identity modeling are proposed,
making use to a different degree of the class information provided by the alphanu-
meric samples. Results obtained on a database of 30 writers from real forensic
documents show that the character class information given by the manual analy-
sis provides a valuable source of improvement, justifying the significant amount
of time spent in manual segmentation and labeling by the forensic specialist.

1 Introduction
Analysis of handwritten documents with the aim of determining the writer is an impor-
tant application area in forensic casework, with numerous cases in courts over the years
that have dealt with evidence provided by these documents [1]. Handwriting is consid-
ered individual, as shown by the wide social and legal acceptance of signatures as a
mean of identity validation, which is also supported by experimental studies [2]. The
goal of writer recognition is to determine whether two handwritten documents, referred
as to the known and the questioned document, were written by the same person or not.
For this purpose, computer vision and pattern recognition techniques have been applied
to this problem to support forensic experts [3,4].
The forensic scenario present some difficulties due to their particular characteris-
tics in terms of [5]: frequently reduced number of handwriting samples, variability of
writing style, pencil or type of paper, the presence of noise patterns, etc. or the unavail-
ability of online information. As a result, this application domain still heavily relies on
human-expert interaction. The use of semi-automatic recognition systems is very useful
to, given a questioned handwriting sample, narrow down a list of possible candidates
which are into a database of known identities, therefore making easier the subsequent
confrontation for the forensic expert [5,4].

H. Sako, K. Franke, and S. Saitoh (Eds.): IWCF 2010, LNCS 6540, pp. 31–42, 2011.

c Springer-Verlag Berlin Heidelberg 2011
32 F. Alonso-Fernandez et al.

Fig. 1. Connected components from a handwritten sample

In the last years, several writer recognition algorithms have been described in liter-
ature based on different group of features [7]: i) general geometric script features, like
word or line spacing; ii) textural features capturing for example slant and curvature of
the script; iii) placement features, i.e. writers placement preferences in the process of
placing ink elements across the page; iv) micro level features measuring ink deposition
characteristics; and v) character-fragment features measuring writer’ preferred use of
allographic elements.
A machine expert for off-line writer recognition making use of textural features
based on contour information has been built in this work. It is focused on discrimi-
nating writers by capturing the distinctive visual appearance of the samples. Previous
works following this direction used connected-component images or contours [8,9] us-
ing automatic segmentation. Perfect automatic segmentation of individual characters
still remains an unsolved problem [5], but connected components encompassing sev-
eral characters or syllables can be easily segmented, and the elements generated (see
Figure 1) also capture shape details of the visual appearance of the samples used by the
writer [9]. The system in this paper, however, makes use of individual characters seg-
mented manually by a forensic expert or a trained operator which are also assigned to
one of the 62 alphanumeric classes among digits “0”∼“9”, lowercase letters “a”∼“z”,
and uppercase letters “A”∼“Z”. This is the setup used by the Spanish forensic group
participating in this work. For a particular individual, the authenticated document is
scanned and next, a dedicated software tool for character segmentation is used. Seg-
mentation is done manually by a trained operator, who draw a character selection with
the computer mouse and label the corresponding sample according to the 62 classes
mentioned. We depict in Figure 2 (right) some examples of the manual selection of
characters. In this work, we adapt the recognition method based on contour features
from [9] to work with this setup. Additionally, the system is evaluated using a database
created from real forensic documents (i.e. confiscated to real criminals or authenticated
in the presence of a police officer), which is an important point compared with exper-
iments of other works where the writing samples are obtained with the collaboration
of volunteers under controlled conditions [10]. We evaluate in this paper three different
 Exploiting Character Class Information in Forensic Writer Identification 33

Fig. 2. Left: the 62 classes of alphanumeric characters used in this paper (digits “0”∼“9”, low-
ercase letters “a”∼“z”, and uppercase letters “A”∼“Z”). Right: manual selection of individual
characters with the computer mouse using a dedicated software tool. Images from [6].

scenarios for identity modeling, exploiting to a different degree the class information
provided by the manual segmentation of alphanumeric samples: modeling per indi-
vidual sample, modeling per alphanumeric channel, and modeling per writer. Results
show that the class information provides a considerable improvement, justifying the
writer identification approach used in our forensic system, where a significant amount
of time is spent every time a new writer is included.
The system is evaluated in identification mode, in which an individual is recog-
nized by searching the reference models of all the subjects in the database for a match
(one-to-many). As a result, the system returns a ranked list of candidates. Ideally, the
first ranked candidate (Top 1) should correspond with the correct identity of the in-
dividual, but one can choose to consider a longer list (e.g. Top 10) to increase the
chances of finding the correct identity. Identification is a critical component in negative
34 F. Alonso-Fernandez et al.

Table 1. Features used in this work

Feature Explanation Dimensions Source

f1 p(φ) Contour-direction PDF 12 contours
f2 p(φ1 , φ2 ) Contour-hinge PDF 300 contours
f3h p(φ1 , φ3 )h Direction co-occurrence PDF, horizontal run 144 contours
f3v p(φ1 , φ3 )v Direction co-occurrence PDF, vertical run 144 contours
f5h p(rl)h Run-length on background PDF, horizontal run 60 binary image
f5v p(rl)v Run-length on background PDF, vertical run 60 binary image

recognition applications (or watchlists) where the aim is checking if the person is
who he/she (implicitly or explicitly) denies to be, which the typical situation in foren-
sic/criminal cases [11].
The rest of the paper is structured as follows. In Section 2 we describe the main stages
of our recognition system. Section 3 describes the database, the scenarios for identity
modeling and the experimental results. Finally, conclusions are drawn in Section 4.

2 System Description
The writer recognition system of this paper makes use of the contour features presented
in [9], which are adapted to the particular setup of this paper. It includes three main
stages: i) preprocessing of the individual characters, ii) feature extraction, and iii)
feature matching. These stages are described next.

2.1 Pre-processing Stage

The writer identification method used by the forensic group participating in this work
is based on manually reviewing the handwritten material, as mentioned in Section 1.
After manual segmentation and labeling of alphanumeric characters from a given docu-
ment, they are binarized using the Otsu algorithm [12], followed by a margin drop and a
height normalization to 120 pixels, preserving the aspect ratio. Elimination of noise of
the binary image is then carried out through a morphological opening plus a closing op-
eration [13]. Next, a connected component detection, using 8-connectivity, is done. In
the last step, internal and external contours of the connected components are extracted
using the Moore’s algorithm [13]. Beginning from a contour pixel of a connected com-
ponent, which is set as the starting pixel, this algorithm seeks a pixel boundary around
it following the meaning clockwise, and repeats this process until the starting pixel is
reached for the same position from which it was agreed to begin the algorithm. The
result is a sequence with the pixels coordinates of the boundary of the component. This
vectorial representation is very effective because it allows a rapid extraction of many of
the features used later.

2.2 Feature Extraction Stage

Features are calculated from two representations of the handwritten samples extracted
during the preprocessing stage: the binary image without noise and the contours of the
 Exploiting Character Class Information in Forensic Writer Identification 35

connected components. The features used in this work are summarized in Table 1, in-
cluding the image representation used by each one. A handwritten sample is shaped like
a texture that is described with probability distribution functions (PDFs). Probability
distribution functions used here are grouped in two different categories: direction PDFs
(features f1, f2, f3h, f3v) and length PDFs (features f5h, f5v). A graphical description
of the extraction of these features is depicted in Figure 3. To be consistent with the work
in which these features where proposed [9], we follow the same nomenclature used in it.

Contour direction (f1) Contour hinge (f2)

Horizontal direction co-occurrence (f3h) Horizontal run-length on background (f5h)

Fig. 3. Graphical description of the feature extraction: contour direction (f1), contour hinge (f2),
horizontal direction co-occurrence (f3h) and horizontal run-length (f5h)

Contour-Direction PDF (f1)

This directional distribution is computed very fastly using the contour representation,
with the additional advantage that the influence of the ink-trace width is eliminated.
The contour-direction distribution f1 is extracted by considering the orientation of lo-
cal contour fragments. A fragment is determined by two contour pixels (xk ,yk ) and
(xk+ ,yk+ ) taken a certain distance  apart. The angle that the fragment makes with
the horizontal is computed using
yk+ − yk
φ = arctan( ) (1)
xk+ − xk
As the algorithm runs over the contour, the histogram of angles is built. This angle his-
togram is then normalized to a probability distribution f1 which gives the probability
of finding in the handwritten sample a contour fragment oriented with each φ. The an-
gle φ resides in the first two quadrants because, without online information, we do not
36 F. Alonso-Fernandez et al.

0.35 0.35

0.30 0.30

0.25 0.25
Probability

Probability
0.20 0.20

0.15 0.15

0.10 0.10

0.05 0.05

00 20 40 60 80 100 120 140 160 180 00 20 40 60 80 100 120 140 160 180
Degrees Degrees

Fig. 4. Example of the countour direction feature (f1) for two different handwritten characters

know which inclination the writer signed with. The histogram is spanned in the interval
0◦ -180◦, and is divided in n = 12 sections (bins). Therefore, each section spans 15◦ ,
which is a sufficiently detailed and robust description [9]. The parameter  controls the
length of the analyzing contour fragment, which is set to  = 5. These settings will be
used for all of the directional features presented in this paper. An example of extraction
of this feature for two handwritten characters is depicted in Figure 4.

Contour-Hinge PDF (f2)

In order to capture the curvature of the contour, as well as its orientation, the “hinge”
feature f2 is used. The main idea is to consider two contour fragments attached at a
common end pixel and compute the joint probability distribution of the orientations
φ1 and φ2 of the two sides. A joint density function is obtained, which quantifies the
chance of finding two “hinged” contour fragments with angles φ1 and φ2 , respectively.
It is spanned in the four quadrants (360◦ ) and there are 2n sections for every side of the
“contour-hinge”, but only non-redundant combinations are considered (i.e. φ2 ≥ φ1 ).
For n = 12, the resulting contour-hinge feature vector has 300 dimensions [9].

Direction Co-occurrence PDFs (f3h, f3v)

Based on the same idea of combining oriented contour fragments, the directional co-
occurrence is used. For this feature, the combination of contour-angles occurring at the
ends of run-lengths on the background are used, see Figure 3. Horizontal runs along the
rows of the image generate f3h and vertical runs along the columns generate f3v. They
are also joint density functions, spanned in the two first quadrants, and divided into n2
sections. These features give a measure of a roundness of the written characters and/or
strokes.

Run-Length PDFs (f5h, f5v)

These features are computed from the binary image of the handwritten sample taking
into consideration the pixels corresponding to the background. They capture the regions
 Exploiting Character Class Information in Forensic Writer Identification 37

03001

03002

Fig. 5. Training samples of two different writers of the forensic database used in this paper

250
SAMPLES PER WRITER TEST 160
TRAINING SAMPLES PER CHARACTER TEST
TRAINING
140
200
120
150 100

80
100
60

50 40

20
0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
0
WRITER ID 0 1 2 3 4 5 6 7 8 9 aAbB c CdDe E f F gG hH i I j J k K l L mMnNoOpPqQr R s S t TuUvVwWxX yY z Z

Fig. 6. Distribution of samples per writer (left) and per character (right) of the forensic database
used in this paper

enclosed inside the letters and strokes and also the empty spaces between them. The
probability distributions of horizontal and vertical lengths are used. These features gives
the probability of finding in the handwritten sample an enclosed region with each length.

2.3 Feature Matching Stage

Each writer is represented in the system by a PDF or set of PDFs (depending on the
experiment at hand, see 3). To compute the similarity between two PDFs o and µ from
two different writers, the χ2 distance is used:
N
 (oi − μi )2
χ2oµ = (2)
i=1
oi + μi
where N is the dimensionality of the vectors o and µ.

3 Experimental Framework

3.1 Database

To evaluate the system, we use a real forensic database from original confiscated/
authenticated documents provided by the Spanish forensic laboratory of the Dirección
General de la Guardia Civil (DGGC). Alphanumeric characters of the handwritten sam-
ples are segmented and labeled by a trained operator of the DGGC. The whole database
contains 9,297 character samples of real forensic cases from 30 different writers, with
38 F. Alonso-Fernandez et al.

around 300 samples on average per writer distributed between a training and a testing
data set. In Figure 5 we plot the training samples of two different writers of the database.
For each writer, training and testing data are extracted from different confiscated doc-
uments, meaning that they were “acquired” at different moments. Given the nature of
the database, it does not contain uniformly distributed samples of every character, nor
time span between training and testing data. Figure 6 shows the distribution of samples
per writer and per character of our database.

3.2 Writer Identity Modeling

Given a writer of the test set, identification experiments are done by outputting the N
closest identities of the training set. An identification is considered successful if the
correct identity is among the N outputted ones.
For a particular writer, several samples of individual characters pertaining to one of
the 62 alphanumeric classes among digits “0”∼“9”, lowercase letters “a”∼“z”, and up-
percase letters “A”∼“Z” are available thanks to the manual segmentation and labeling.
For each feature described in Section 2.2, we evaluate the following three scenarios for
writer identity modeling:
1. Modeling per individual sample (channel dependent). For example, if a writer has
x samples of the digit “0”, features for each of the x samples are computed. This
process is repeated with all the 62 alphanumeric channels. This modeling captures
particular variations in each alphanumeric character (e.g. if the writer has different
“a”, “b”, etc.) Due to the nature of the database, it will not be a uniform number
of features among the different channels. It could also be the case that a writer
many not have samples in a particular channel, in whose case no features will be
extracted. For each individual sample, we find the closest identity by comparing
with all the training samples pertaining to the same channel. We then compute the
closest identity to each alphanumeric channel based on the majority rule: the win-
ning identity will be the writer having the maximum number of winning samples. In
case of writers having the same number of winning samples, they are subsequently
ranked using the next 2 criteria, listed in descending order of weight: 1) average of
winning sub-distances, and 2) minimum winning sub-distance. Finally, identifica-
tion is based again on the majority rule, applied in this case to the alphanumeric
channels: the winning output identity will be the writer having the maximum num-
ber of winning alphanumeric channels, the second winning identity will be the next
writer, and so on. In case of writers having the same number of winning channels,
we apply the same above criteria.
2. Modeling per alphanumeric channel (channel dependent). For example, if a writer
has x samples of the digit “0”, histograms of the feature are combined (added)
to obtain a unique probability distribution. This process is repeated for all the 62
alphanumeric classes. This modeling averages the different variations of a given al-
phanumeric character. Therefore, we obtain 62 sub-distances between two writers,
one per channel. We then compute the closest identity to each alphanumeric chan-
nel based on its distance. Identification is based on the majority rule: the winning
output identity will be the writer having the maximum number of winning alphanu-
meric channels, the second winning identity will be the next writer, and so on. This
 Exploiting Character Class Information in Forensic Writer Identification 39

Modeling per individual sample

100

80
Sucess Rate (%)

60

40 f1
f2
f3h
20 f3v
f5h
f5v
0
0 5 10 15 20 25 30

Modeling per alphanumeric channel

100

80
Sucess Rate (%)

60

40

20

0
0 5 10 15 20 25 30

Modeling per writer

100

80
Sucess Rate (%)

60

40

20

0
0 5 10 15 20 25 30
TopN

Fig. 7. Writer identification rates for the three scenarios of identity modeling considered
40 F. Alonso-Fernandez et al.

results in 62×30×30=55,800 computed distances. In case of writers having the

same number of winning channels, we apply the same above criteria.
3. Modeling per writer (channel independent). This case computes a unique proba-
bility distribution per writer by combining all the available samples of all the al-
phanumeric characters. In this case, we do not use the character class information,
obtaining a unique writing identity model that averages information from the 62
channels. Only one distance between two writers is obtained, which is used for
identification. This results in 30×30=900 computed distances.

3.3 Results
We plot in Figure 7 results of the identification experiments varying the size of the hit
list from N =1 (Top 1) to N =30 (Top 30). Results are shown for the different features
described in Section 2.2 and for the three identity modeling scenarios considered.
We observe that, in general, working with the class information provided by the al-
phanumeric channels (top and medium plot in Figure 7) results in considerable better
performance with respect to using a unique single identity model that does not ex-
ploit this information (bottom plot in Figure 7). Thus, the class information given by
the character segmentation and labeling carried out by the trained operator provides
a considerable improvement. This justifies the writer identification approach used in
our forensic system, in which a considerable amount of time is spent every time a new
writer is included in the database. It can be seen in Figure 7, for example, that a success
rate of 80% is already achieved with some features for a hit list size of N =5 or less
when using channel information. However, when using a channel independent identity
modeling, it is not achieved until a hit list size of N =13 is considered.
It is worth noting that directional features (f1, f2, f3h, f3v) work consistently bet-
ter that features based on length properties (f5h, f5v). This suggests that the length of
the regions enclosed inside the letters and strokes is not a good distinctive feature in
the setup presented in this paper, where we are using a database of isolated alphanu-
meric handwritten characters. Better results are obtained in other studies making use of
complete lines or pages of handwritten material [9].
Finally, by comparing the two scenarios for writer identity modeling that make use
of channel information (top and medium plot in Figure 7), it can be seen that the best
results are obtained when using identity models per alphanumeric channel. In this case,
for a hit list size of N =5, all the directional features achieve a success rate of 8̃0%. On
the other hand, when using identity models per individual sample and a hit list size of
N =5, the success rate exhibited by the directional features are between 60% and 80%.
Thus, averaging all the samples of a given channel provides more robustness than using
the samples separately.

4 Conclusions and Future Work

A machine expert for off-line writer identification based on contour features has been
evaluated. It encodes several directional properties of contour fragments as well as the
length of the regions enclosed inside letters. The system presented in this work is based
 Exploiting Character Class Information in Forensic Writer Identification 41

on manual review of the handwritten material, in which segmentation and labeling of

characters is made using a dedicated software tool according to 62 alphanumeric classes
(10 numbers and 52 letters, including lowercase and uppercase letters). This particular
setup is used by the Spanish forensic group participating in this work, which has also
provided us with a database of real forensic documents from 30 different writers, an
important point in comparison with other works where data is obtained from collabo-
rative writers under controlled conditions. Experiments are done in identification mode
(one-to-many), which the typical situation in forensic/criminal cases.
The system of this paper is evaluated in three different scenarios for identity mod-
eling which exploit to a different degree the class information provided by the manual
segmentation of alphanumeric samples: i) modeling per individual sample, ii) model-
ing per alphanumeric channel, and iii) modeling per writer. The two first scenarios
make use of the class information given by the manual labeling, whereas the third one
is channel independent (i.e. does not use the character class information). Results show
that much better performance is obtained by using channel information, justifying the
considerable amount of time spent by the trained operator in the segmentation and la-
beling process. The best scenario is based on identity modeling per alphanumeric chan-
nel, meaning that averaging all the samples of a given channel provides more robustness
than using the samples separately. The latter approach may work better if enough sam-
ples representative of writer’ particular variations are included in the database, or for
specific channels commonly used in the language of the database (as can be seen in
Figure 6, characters like ’w’ and ’W’ are not often used in the Spanish language, while
“a”, “A”, “r” or “t” are quite common).
A drawback found in our experiments is that a success rate of 100% is never achieved
with some features and/or identity modeling scenarios. It means that there are some
writers in the database whose identity are never found, and test samples from this writer
are assigned as pertaining to someone else. It could be due to the majority rule used for
identification, as well as the decision criteria when several writers have the same number
of winning samples (see Section 3.2).
The analysis of these results with a limited database suggest that the proposed ap-
proach can be used for forensic writer identification, pointing out the advantages of
manual segmentation and labeling by a trained operator. Future work includes evaluat-
ing of our system with a bigger forensic database and improving the performance by
applying advanced alphanumeric channel combination methods [14]. Another source of
future work is the use of advanced approaches for user-dependent selection and combi-
nation of alphanumeric channels [15,16], so that the most discriminative channels for
each user are used in the fusion.

Acknowledgements
This work has been partially supported by projects Bio-Challenge (TEC2009-11186),
BBfor2 (FP7 ITN-2009-238803) and ”Cátedra UAM-Telefónica”. Postdoctoral work
of author F. A.-F. is supported by a Juan de la Cierva Fellowship from the Spanish
MICINN. The authors would like to thank to the forensic “Laboratorio de Grafı́stica”
of the ’Dirección General de la Guardia Civil’ for its valuable support.
42 F. Alonso-Fernandez et al.

References
1. Srihari, S., Huang, C., Srinivasan, H., Shah, V.: Biometric and Forensic Aspects of Digi-
tal Document Processing. In: Digital Document Processing, ch. 17, pp. 379–406. Springer,
Heidelberg (2007)
2. Srihari, S.N., Cha, S.H., Arora, H., Lee, S.: Individuality of handwriting. Journal of Forensic
Sciences 47(4), 856–872 (2002)
3. Plamondon, R., Srihari, S.: On-line and off-line handwriting recognition: A comprehensive
survey. IEEE Trans. on Pattern Analysis and Machine Intelligence 22(1), 63–84 (2000)
4. Srihari, S., Leedham, G.: A survey of computer methods in forensic document examination.
In: Proc. 11th International Graphonomics Society Conference, IGS, pp. 278–281 (Novem-
ber 2003)
5. Schomaker, L.: Writer identification and verification. In: Sensors, Systems and Algorithms,
Advances in Biometrics. Springer, Heidelberg (2008)
6. Tapiador, M.: Análisis de las Caracterı́sticas de Identificación Biométrica de la Escrit-
ura Manuscrita y Mecanográfica. PhD thesis, Escuela Politécnica Superior, Universidad
Autónoma de Madrid (2006)
7. Schomaker, L.: Advances in writer identification and verification. In: Proc. Intl. Conference
on Document Analysis and Recognition, ICDAR, vol. 2, pp. 1268–1273 (2007)
8. Schomaker, L., Bulacu, M.: Automatic writer identification using connected-component con-
tours and edge-based features of upper-case western script. IEEE Trans. on Pattern Analysis
and Machine Intelligence 26(6), 787–798 (2004)
9. Bulacu, M., Schomaker, L.: Text-independent writer identification and verification using tex-
tural and allographic features. IEEE Trans. on Pattern Analysis and Machine Inteligence 29(4),
701–717 (2007)
10. Tapiador, M., Sigüenza, J.: Writer identification method based on forensic knowledge. In:
Zhang, D., Jain, A.K. (eds.) ICBA 2004. LNCS, vol. 3072, pp. 555–560. Springer, Heidelberg
(2004)
11. Jain, A., Flynn, P., Ross, A. (eds.): Handbook of Biometrics. Springer, Heidelberg (2008)
12. Otsu, N.: A threshold selection method for gray-level histograms. IEEE Trans. on Systems,
Man and Cybernetics 9, 62–66 (1979)
13. Gonzalez, R., Woods, R.: Digital Image Processing. Addison-Wesley, Reading (2002)
14. Jain, A., Nandakumar, K., Ross, A.: Score Normalization in Multimodal Biometric Systems.
Pattern Recognition 38(12), 2270–2285 (2005)
15. Fierrez-Aguilar, J., Garcia-Romero, D., Ortega-Garcia, J., Gonzalez-Rodriguez, J.: Adapted
user-dependent multimodal biometric authentication exploiting general information. Pattern
Recognition Letters 26, 2628–2639 (2005)
16. Galbally, J., Fierrez, J., Freire, M.R., Ortega-Garcia, J.: Feature selection based on genetic
algorithms for on-line signature verification. In: Proc. IEEE Workshop on Automatic Identi-
fication Advanced Technologies, AutoID, pp. 198–203 (2007)