IEICE TRANS. INF. & SYST., VOL.E100–D, NO.

1 JANUARY 2017
42

PAPER Special Section on Enriched Multimedia —New Technology Trends in Creation, Utilization and Protection of Multimedia Information—

Aesthetic QR Code Based on Modified Systematic Encoding

Function
Minoru KURIBAYASHI†a) and Masakatu MORII†† , Senior Members

SUMMARY Quick Response (QR) code is a two dimensional barcode drawback is that it sacrifices the readability of original QR
widely used in many applications. A standard QR code consists of black codes with small displayable size. The second class modi-
and white square modules, and it appears randomized patterns. By modify-
fies the size and color of modules according to the pixel of
ing the modules using certain rule, it is possible to display a logo image on
the QR code. Such a QR code is called an aesthetic QR code. In this paper, image. Visualead [7] and LogoQ [8] keep a concentric re-
we change the encoding method of the Reed-Solomon (RS) code to produce gion of modules untouched and uniformly blend the neigh-
an aesthetic QR code without sacrificing its error correcting capability. The boring regions with pixels of an image preserving the origi-
proposed method randomly produces candidates of RS blocks and finds the nal contrast. The third class changes the padding codewords
best one during encoding. Considering an image to be displayed, we also
introduce a weighting function during random selection that classifies the
to display an image on the QR code [9], [10]. The possi-
visually important regions in the image. We further investigate the shape of ble area on which the image is displayed is restricted due to
modules which represents the image and consider the trade-off between the the standardized structure of QR code. A free software of
visual quality and its readability. As a result, we can produce a beautiful the QR-JAM MAKER [11] is openly available. Its main re-
aesthetic QR code, which still can be decoded by standard QR code reader.
striction is the standardized structure of Reed-Solomon (RS)
key words: 2-dimensional code, aesthetic code, parity check matrix
code [12] employed in QR code.
Some other works studied combination of above
1. Introduction
classes and image processing techniques to effectively en-
hance the visual quality. Chu et al. [13] applied the half-
QR code is a two-dimensional code consisting of black toning technique for displaying image to produce an aes-
and white square modules. It is originally developed by thetic QR code. Fang et al. [14] considered the sampling
Denso Wave [1] as an effective way for accessing digital process and error correction to optimize the generation pro-
data by forming a machine-readable format. It can carry cedure. Zhang et al. [15] proposed two-stage approach that
more information than conventional barcodes and can be is a combination of module-based and pixel-based encod-
read very quickly by software installed in smart phones. The ings. Even though these methods can enhance the visual ap-
QR code was standardized as JIS (Japanese Industrial Stan- pearance, they try to reduce the noise-like appearance at the
dards) standard in 2004 (JIS X0510) [2] and ISO/IEC stan- center of image by using error correcting capability or shape
dard in 2000 (ISO/IEC 18004) [3]. of modules. The basic white/black patterns at the important
A standard QR code contains only black and white region of an image are not removed in a literature.
square modules and its appearance has noisy looks. There In this paper, we develop a fourth class for generating
are many approaches for producing aesthetic QR codes by an aesthetic QR code based on the encoding method of RS
displaying images on a QR code. For a given message code which allows us to select the display position without
and image, it is desirable to automatically generate such a sacrificing the error correcting capability. The preliminary
QR code because manually designed barcodes are too ex- version of such an encoding method was presented in [16]∗ .
pensive. It is also required that the aesthetic QR codes be As the RS code belongs to a family of linear code over a
readable by standardized decoders. The conventional ap- finite field [12], the set of RS blocks generated by using a
proaches satisfying these requirements can be categorized certain parity check matrix becomes just equal to the set of
into three classes. the RS blocks generated from the matrix which is calcu-
The first class globally transforms an input image, and lated by column operation of the parity check matrix. Even
superimposes it on part of the QR code [4]–[6]. It modifies though the correspondence between the information and RS
several modules of an original QR code within a possible er- block is different, the set of RS blocks are equal. Based on
ror correcting capability to display a small logo image. The this property, the positions of information symbols in a RS
Manuscript received March 21, 2016. block are changed in the proposed method. Since the stan-
Manuscript revised August 1, 2016. dard QR code reader only extract first successive symbols
Manuscript publicized October 7, 2016. in RS blocks, the proposed method fixes first such positions
†
The author is with the Graduate School of Natural Science and for representing data and controls the positions of the other
Technology, Okayama University, Okayama-shi, 700–8530 Japan.
††
The author is with the Graduate School of Engineering, Kobe
∗
University, Kobe-shi, 657–8501 Japan. At the early stage of this study, we regarded the method as
a) E-mail: kminoru@okayama-u.ac.jp non-systematic encoding, but as explained in Sect. 2.2 it is a kind
DOI: 10.1587/transinf.2016MUP0002 of systematic encoding by definition of error correcting code.

Copyright 
c 2017 The Institute of Electronics, Information and Communication Engineers
KURIBAYASHI and MORII: AESTHETIC QR CODE BASED ON MODIFIED SYSTEMATIC ENCODING FUNCTION
43

information symbols. As the results, the produced aesthetic

QR code can be read by a standard QR code reader.
We propose two encoding methods in this paper; one is
a random selection from candidates, and the other is the se-
lection based on weights for candidates. We called them
“random method” and “weighting method” in this paper.
The random method randomly selects the positions of in-
formation symbols to generate RS blocks. Then, it deter-
mines the best RS block among candidates under the con-
straint that the Hamming distance with the original pixel Fig. 1 Function patterns of QR code with version 8.
values of an image becomes minimum. Based on the
method, the visual appearance is improved in the weight-
ing method, whose preliminary version was presented at tal and vertical coordinates of finder patterns. The timing
IWDW2015 [17]. The weighting method is based on the patterns are two lines, one horizontal and one vertical, of al-
following two tendencies for generic images. 1) The outer ternating black and white modules. A dark module which
end of an image is less important than the center region. 2) is a single black module is always placed beside the bottom
The noisy region of an image is less sensitive to the modi- left finder pattern.
fication of pixels. Based on these tendencies, we propose a The data bits are placed starting at the bottom-right of
weighting function to determine the positions of information the matrix and proceeding upward in a column with 2 mod-
symbols in the encoding method. ules wide. When the column reaches the top, the next 2-
We also investigate the module which represents the module column starts to the left of the previous column and
image to be displayed. As the number of modules in a QR continues downward. Whenever the current column reaches
code is relatively small, the resolution of an image should the edge of the matrix, it moves on to the next 2-module
be scaled down in order to adjust the QR code. We pro- column and change direction. At the position of function
pose sub-modules that express each module with a matrix patterns, the data bit is placed in the next module.
description to increase the resolusion. Using some examples The QR code uses the RS code for error correction
of such sub-modules, we measure the trade-off between the whose capability is classified into four levels: L, M, Q, and
readability and visual quality of generated QR code. H in the increasing order. Each error correction level can
This paper is organized as follows. In Sect. 2, the struc- correct symbols up to about 7%, 15%, 25% and 30% of all
ture of QR code and the employed error correcting code are symbols in the QR code. Due to the characteristic of the RS
briefly reviewed. The proposed method to produce an aes- code, each symbol in RS block has 8 bits, and it is arranged
thetic QR code is described in Sect. 3. In Sect. 4, the vi- in neighbors of 8 modules in a QR code. Considering dirt
sual appearance is further enhanced by changing the mod- on a QR code, the symbols of the RS block are arranged in
ule shape. In Sect. 5, we conclude this paper with some re- an interleaved fashion.
marks. Given the version number and the error correction
level, the number of RS blocks and their constructions ex-
2. Preliminaries pressed by RS (n, k, d) are uniquely determined in a QR
code, where n, k, and d are the code length, the number of
2.1 Overview of QR Code information symbols, and the minimum distance satisfying
d = n − k + 1. For instance, 5L is composed of a single
The module represented by black or white pattern is the min- RS block of RS (134, 108, 13), and 5M is composed of two
imum component of a QR code and each module is assigned RS blocks of RS (67, 43, 12). The code length n is increased
a single bit value. The size of QR code is determined by with the version number, while k is decreased with the error
a version number from 1 to 40. The number of modules correction level.
representing an encoded data in a QR code of version v is Since each symbol of the RS block is composed of
(17 + 4v) × (17 + 4v). 8 bits, at most 8k bits (k bytes) are represented by a sin-
The QR code has function patterns which allows reader gle RS block. If the amount of information to be encoded is
to identify the position in which an encoded data is involved. less than the capacity, padding codewords are attached with
The patterns are composed of three main parts; finder pat- the information symbols. The padding codewords have no
tern, alignment pattern, and timing pattern. Figure 1 shows information, so they can be changed freely. The remaining
the function patterns of QR code with version 8. The finder n − k symbols are called parity symbols in a RS code, and
patterns are the three square blocks in the corners of the QR its redundancy contributes on an error correcting capability.
code at the top left, top right, and bottom left. They are In the systematic encoding, the parity symbols are derived
placed irrespective of QR code version. Beside the finder uniquely from the information symbols.
patterns, white modules are placed to separate other mod- Considering the readability by optical camera devices,
ules. The alignment patterns are small square blocks, and the number of black and white modules should be balanced
their positions are defined in a table related to the horizon- in the QR code. There are eight mask patterns that change
 IEICE TRANS. INF. & SYST., VOL.E100–D, NO.1 JANUARY 2017
44

the bits according to their coordinates in order to make the Let δ = (δ1 , . . . , δk ) be information symbols of length
QR code easily readable. During encoding, each element of k, Then, the parity symbols p = (p1 , . . . , pn−k ) is calculated
a selected mask pattern is XORed with a current module. from δ using a systematic encoding function enc(), and the
generated RS block c is represented by
2.2 Encoding of Reed-Solomon Code
c = enc(δ) = (c1 , . . . , ck , ck+1 , . . . , cn ) (5)
In a standard QR code, the systematic encoding method of = (δ1 , . . . , δk , p1 , . . . , pn−k ). (6)
RS code is employed to generate the RS block. The system-
atic encoding means that the information symbols appear The information symbols δ can be decoded by performing a
directly as part of the RS block. For the convenience, dur- decoder dec() from a distorted RS block c if the bit errors
ing decoding, the information appeares only at the first k are within an error correcting capability.
symbols of the RS block in the QR code.
The RS code is based on the polynomial g(x) = x8 + 3. Proposed Aesthetic QR Code
x + x3 + x2 + 1 over finite field GF(28 ), and the code is
4

punctured through the consistent deletion of parity coordi- In this section, we propose two encoding methods for gen-
nates from each RS block. As the RS code is maximum erating a better RS block in which binary string placed on
distance separable (MDS) code, the elimination of a coordi- a QR code is similar to the module patterns of logo image.
nate in a code can reduce the minimum distance of the code The best RS block can be found by an exhaustive search
by 1 with each successive puncturing operation. In the QR among candidates. However, the number of candidates is
code, the eliminated coordinate of parity check matrix is the extremely large, it is computationally difficult. Instead, the
first 255 − n rows out of 255. When the minimum distance proposed method randomly and uniformly selects the candi-
is d and the root of the polynomial g(x) is α, the punctured dates and checks the similarity with the module pattern, and
parity check matrix H is given by it is called as “random method”. The other method consid-
ers the local characteristics of a logo image by calculating
⎡ ⎤
⎢⎢⎢ α0×(n−1) ... α0×0 ⎥⎥
⎥⎥⎥ weights in order to control the visual quality, and it is called
⎢⎢ .. .. ..
H = ⎢⎢⎢⎢ . . .
⎥⎥⎥ ,
⎥⎥ (1) as “weighting method”.
⎢⎣
(d−2)×0 ⎦
α(d−2)×(n−1)
... α
3.1 Basic Idea
where α = 1. Then, for any RS block c, the following
255

equation must be satisfied. As mentioned in Sect. 2.2, the code space generated from
H coincides with the code space generated from H (sys) . It
cH T = 0. (2) means that for a given δ, the RS block generated from H (sys)
is one of the RS blocks of H. Therefore, the code space is
Here, we suppose that a code space contains all RS same, but the assignment from information symbols is dif-
blocks generated from the matrix H. From the coding the- ferent if H (sys) is modified from H by using column opera-
ory, two codes that differ only in the arrangement of sym- tions. Even though the parity check matrix is different, the
bols have the same probability of error, namely the same encoding function based on the matrix generates a certain
error correcting property. Specifically, the matrix H can be systematic RS block. It implies that the information sym-
modified by the combination of column operations without bols appear not at the first k symbols of the RS block, but
changing the code space. It means that the RS block gener- at certain positions of the RS block. Such a systematic en-
ated from such a modified parity check matrix is one of the coding function can be designed by carefully selecting the
RS blocks generated from the matrix H. In this case, it is column operations to the parity check matrix.
possible to use the same decoder for a given RS block, but Let C be the group of all RS blocks generated by enc(),
the decoded information is different. Based on such a prop- namely, ∀c ∈ C. Then, the RS blocks c encoded by a cer-
erty, the systematic encoding method prepares the following tain systematic encoding method satisfies ∀c ∈ C. Thus,
matrix H (sys) : we can obtain certain information symbols from dec(c ). If

H (sys) = − PT In−k , (3) no error occurrs, then the first k symbols of the RS block
c is decoded as information symbols. For a given index
where, P is a k × (n − k) matrix, In−k is an (n − k) × (n − k) θ = (θ1 , θ2 , . . . , θk ), the k symbols of the RS block c is rep-
identify matrix, and T stands for a transposition. The gen- resented by
erator matrix G(sys) can be easily calculated from the matrix
H (sys) . cθi = δi , (1 ≤ i ≤ k), (7)

G(sys) = Ik P (4) where 1 ≤ θi ≤ n and θi  θ j for i  j. The other n − k
symbols are calculated according to these k symbols using
By multiplying the matrix G(sys) with a vector of informa- the employed systematic encoding function encθ ().
tion symbols, the systematic RS block is obtained. For con- Suppose that the first k̂ (< k) symbols are used to rep-
venience, such an encoding function is denoted by enc(). resent a data to be encoded and the remain k − k̂ symbols are
KURIBAYASHI and MORII: AESTHETIC QR CODE BASED ON MODIFIED SYSTEMATIC ENCODING FUNCTION
45

regarded as padding codewords. In such a case, it is possible from the pixel values of logo image to be displayed.
to use the following index.
3.3 Color Translation
θ k̂ = (1, 2, . . . , k̂, θk̂+1 , . . . , θk ) (8)
According to the standardized format of QR code, each
It means that there are C(n− k̂, k− k̂) candidates for the index
module is basically represented by black and white. Nev-
θ k̂ , where C(a, b) stands for the number of b-combinations
ertheless, it is possible to use color for modules because of
from a given set of a elements. In short, we can freely select
the following reason. When a QR code is taken by a certain
the positions for padding codewords within the range [k̂ +
camera device, the brightness of each module is first calcu-
1, n] at the RS block.
lated in the decoding algorithm, and then, the color space
is translated into luminance component. As the translation
3.2 Random Method algorithm is not defined in the specification of QR code de-
coder, there may be some methods to determine the black
When the amount of encoding data is small, the designable and white. In the proposed method, we heuristically ex-
area becomes large in the method. If a QR code has m RS amine the decodability of color space by using some smart
blocks whose information symbols are kt , (1 ≤ t ≤ m) bytes phones and develop the following color translation proce-
and the data to be encoded is σ bytes, then the size of des- dure at the production of aesthetic QR code.
ignable area is ( m t=1 kt ) − σ bytes. Suppose that an image is square size with L × L pix-
Without loss of generality, the data to be encoded is els and each pixel is represented by RGB color components.
composed of σ bytes and it is partitioned into m segment Using a certain conversion algorithm equipped at a QR code
of length k̂t , (1 ≤ t ≤ m), namely m t=1 k̂t = σ. For each reader, a binary matrix Bi, j , (1 ≤ i, j ≤ 17 + 4v) is calculated
segment, the RS block ct is calculated by the systematic from the input image for the given version v of QR code. In
encoding function encθ k̂ t . our method, the following algorithm is used as the conver-
First, each pixel of an image is regarded as a module sion.
of ideal aesthetic QR code except for the function patterns.
Step 1 The scale of the input image is changed to be the
The pixel values are classified into black/white module at
same size of the given version v of QR code.
the encoding, and the binarized pixels are produced. The bi-
Step 2 The RGB color components are translated into YUV
narized pixels may not coincide with the binary represention
color components, and the luminance (Y) components
of RS blocks. Allowing some errors, we generate several RS
Yi, j , (1 ≤ i, j ≤ 17 + 4v) are obtained.
blocks by changing θ k̂ t and choose the best one whose Ham-
Step 3 The average Ȳ of the values at its center square,
ming distance is minimum. The procedure to make aesthetic
which size is the quarter of the original image size, is
QR codes is described as follows.
calculated.
Step 1 Assign pixels of an image to modules of a QR code, 3L 3L
4 4
and binarize the modules with the threshold determined 4
Ȳ = 2 Yi, j (9)
by the average of all RGB values. L
i= L4 j= L4
Step 2 Operate a predetermined masking pattern to the bi-
nary matrix, and replace σ symbols of the masked Step 4 The binary matrix Bi, j is determined by the follow-
image with information symbols, which are never ing rule:
changed in the following steps.
Step 3 Partition the masked image into m sequences which 1 if Yi, j > Ȳ
Bi, j = (10)
are corresponding to the RS blocks placed on the QR 0 otherwise
code in an interleaved order. We denote the m se-
Although the binarization of the reduced-size image is
quences by η t , (1 ≤ t ≤ m).
slightly varied according to the resizing algorithm, the im-
Step 4 For 1 ≤ t ≤ m, find the best RS block ċt , whose
pact on the produced aesthetic QR code is small because of
Hamming distance from η t becomes minimum, by
the low resolution. In case of scaling-up, the visual quality
changing the index θ k̂ t .
is strongly affected by the resizing alrogithm. In practical,
Step 5 Replace the best RS block ċt with η t for 1 ≤ t ≤ m.
however, an input image is usually larger than a QR code.
Step 6 Operate the predetermined masking pattern again to
Notice that the binarized ideal QR code is composed of
cancel the masking operation at step2.
the binary matrix Bi, j except for the function patterns. At the
There are C(n − k̂t , kt − k̂t ) candidates for the index θ k̂ t Step 3 in the random method, the sequences η t are extracted
for (1 ≤ t ≤ m). It is computationally difficult to check from the binary matrix Bi, j . After the determination of RS
all candidates for the search in Step 4. In the above pro- blocks ċt , each module of QR code is modified according to
cedure, the index θ k̂ t are randomly and uniformly selected the original RGB color components in order not to change
from all possible candidates, and the trial is bounded to N the color balance. Let βi, j , (1 ≤ i, j ≤ 17 + 4v) be the binary
times. Among N candidates of QR codes, we select the best represented RS blocks placed on the matrix of QR code. It
one which has the minimum number of modules different is noted that βi, j must satisfy the black and white function
 IEICE TRANS. INF. & SYST., VOL.E100–D, NO.1 JANUARY 2017
46

patterns at the corresponding positions. We finally generate η t . By changing the index θ k̂ t under the constraint of such a
a QR code involving a given image as follows. rule, the best RS blocks ċt are selected under the measure-
If βi, j = 1, then ment of Hamming distance.
Among kt information symbols in t-th RS block, the
Yi, j if Yi, j > Ȳ + 
Yi, j = , (11) first k̂t symbols are fixed to represent data and the remain
Ȳi, j +  otherwise kt − k̂t symbols are assumed to be padding codewords. The
otherwise, weight sequence indicates the priority for the selection of
kt − k̂t symbols from n − kt symbols. If we choose kt − k̂t
Yi, j if Yi, j < Ȳ −  symbols which have highest kt − k̂t weights in w t , the cor-
Yi, j = , (12)
Ȳi, j −  otherwise responding RS block can be calculated. However, in such
a case, the Hamming distance is not always small. In or-
where  is a threshold for ensuring the readability. The RGB der to find the best RS block which has minimum Hamming
color components are translated from the modified lumi- distance among some candidates, we choose symbols which
nance components Y  as well as the original U and V com- have highest kt − k̂t + α weights, where α = 1, 2, · · ·. By
ponents, and the color QR code is finally obtained. increasing α, the number of candidates is exponentially in-
creased. For a given number N, α is adjusted in the proposed
3.4 Weighting Method
method. For example, when the version of QR code is 5L,
k = 108 and n = 134. In case of k̂ = 50, the number of
In the random method, the index θ k̂ t is randomly generated
candidates is C(59, 58) = C(59, 1) = 59 if α = 1, and it is
without any strategies. Considering the characteristics of an
C(60, 58) = C(60, 2) = 1770 if α = 2. When N = 500, we
image to be displayed, we introduce a weighting function to
determine α = 2, and generate 500 candidates by randomly
classify perceptually important regions with the others.
choosing kt − k̂t = 58 symbols among 60 symbols which
It is widely recognized that noisy regions of an image
have highest 60 weights. Finally we determine the best RS
are less sensitive to modification than flat regions. It is also
block among 500 candidates.
reasonable to assume that the center region of an image is
much more important than its surroundings. Based upon the
3.5 Numerical Evaluation
characteristics, we calculate weights for modules.
After the color translation, the Laplacian filter LF() is
In order to evaluate the performance, we implement two pro-
operated to the matrix Bi, j , which discrete convolution ker-
posed methods and an original method for generating RS
nel is given by
block using a same color translation algorithm. Here, the
⎡ ⎤
⎢⎢⎢ −1 −1 −1 ⎥⎥⎥ original method changes the padding codewords to display
⎢⎢⎢ −1 8 −1 ⎥⎥⎥ . an image on the QR code such as QR-JAM and [9], [10].
⎢⎣ ⎥⎦
−1 −1 −1 For convenience, the method explained in Sect. 3.2 is called
“Prop.I” and the method in Sect. 3.4 is called “Prop.II”.
The weight matrix Wi, j is calculated by We use four RGB color images of different size, which
Wi, j = |LF(Bi, j )|, (13) are shown in Fig. 2. We use the OpenCV library [18] for
resizing an input image. The encoding data is the URL:
where |a| returns the absolute value of a. It is noticed that http://www.okayama-u.ac.jp, which is represented by
0 ≤ Wi, j ≤ 8 because Bi, j ∈ {0, 1}. At the γ-th outer end of
the matrix for 0 ≤ γ ≤ 7, if Wi, j < 8 − γ, then the value is
replaced as Wi, j = 8 − γ.
The coordinate of the matrix Wi, j is corresponding to
that of QR code. As the symbols of t-th RS block η t are
placed on the matrix of QR code in an interleaved order, the
symbols of the weight sequence w t is also placed on the ma-
trix Wi, j . Namely, w t is selected from the matrix Wi, j in the
same rule as the placement of RS block in the matrix of QR
code. Remember that the symbols of RS block employed in
a QR code are not binary, and represented by 8 bits. In addi-
tion, 8 modules are placed on a certain neighboring modules
which positions depend on the version v. Considering such
neighboring positions and their values w t , the index θ k̂ t is
determined to mainly select flat regions at the center of QR
code for the assignment of as many as possible information
symbols. It is worth-mentioning that the modules at noisy
regions and the outer end of the QR code tend to be the par-
ity symbols of RS block, and their values may differ from Fig. 2 Original images for displaying on a QR code.
KURIBAYASHI and MORII: AESTHETIC QR CODE BASED ON MODIFIED SYSTEMATIC ENCODING FUNCTION
47

Fig. 3 Comparison of the visual appearance of aesthetic QR codes when the version is 10M, where
(a)–(e) are produced by the Prop.I method and (f)–(j) are by the Prop.II method.

Table 1 Number of different modules when version is 10L.

bles 1 and 2 show the number of different modules when
image method 1 10 100 1000 10000
momiji original 280 — — — —
the version of QR code is 10. It is observed that the Ham-
Prop.I 276 248 230 210 196 ming distance of Prop.II converges much faster than that of
Prop.II 260 232 222 218 212 Prop.I. It is because the Prop.I selects θ k̂ t at random while
flower original 293 — — — — the Prop.II considers the characteristics of an image to be
Prop.I 295 243 227 215 199 displayed for the selection. Therefore, the number of can-
Prop.II 293 269 231 219 207
present original 287 — — — —
didates for the Prop.II is smaller than that of the Prop.I. In
Prop.I 298 254 236 206 200 the measurements of Hamming distance, the Prop.I seems
Prop.II 287 251 241 223 223 better under a same number N of candidate RS blocks. It
soccer original 297 — — — — is worth-mentioning that it is possible to make the proposed
Prop.I 296 240 230 214 200 methods deterministic by setting N = 1.
Prop.II 298 255 245 243 243
Figure 3 shows the visual appearance of generated
QR codes when the version is 10M. The perceptual qual-
Table 2 Number of different modules when version is 10M. ity is slightly improved with the increase of N in the Prop.I
image method 1 10 100 1000 10000 method, while there is no clear changes in the Prop.II
momiji original 523 — — — — method. As the Prop.II method keeps perceptually impor-
Prop.I 538 450 438 402 378 tant regions unchanged introducing a weight matrix Wi, j , the
Prop.II 529 473 431 425 417
modules which are different from the original color appears
flower original 515 — — — —
Prop.I 520 476 430 400 378 mainly at the outer edge and noisy regions. Even though
Prop.II 541 469 443 423 421 the Prop.I method is better than the Prop.II in terms of the
present original 515 — — — — measurement of Hamming distance, the visual appearance
Prop.I 505 457 431 403 383 of the Prop.II is better as observed from these figures.
Prop.II 501 475 449 447 445
soccer original 509 — — — —
Figure 4 shows the generated QR codes of version 10
Prop.I 518 470 426 404 386 with error correction level L and M. As the encoding data
Prop.II 523 478 433 427 427 is 26 bytes, the number of padding codewords is 248 and
190 bytes, respectively for level L and M. As a comparison,
we show in Fig. 4 that the aesthetic QR codes generated by
26 bytes. The QR code of version 10 composed of 57 × 57 the original method as well as the proposed methods. In this
modules, and the maximum number of information symbols case, it is possible to display an image at the center of QR
for level L and M is 274 and 216 bytes, respectively. The code for the original method. However, the left side of the
threshold for the conversion algorithm is fixed by  = 64. QR code appears as random patterns because they are deter-
When an encoding method outputs an aesthetic QR mined by the RS encoding function. It is observed from the
code, the Hamming distance between the QR code and the figure that the improvement of the visual quality becomes
ideal one is measured. Since the arrangement of symbols much remarkable for Prop.II method when the error correct-
is fixed in the original method, the output is uniquely deter- ing level is increased.
mined, namely, the method is deterministic. On the other The amount of data to be encoded can be varied at
hand, the proposed methods can change the arrangement by most to the actual capacity of a given version of QR code.
selecting the index θ k̂ t . Thus, we produce N candidates and If k̂ = k, then there is only one candidate in the proposed
select the best one with minimum Hamming distance. Ta- method because C(n − k, 0) = 1. In such a case, what we can
 IEICE TRANS. INF. & SYST., VOL.E100–D, NO.1 JANUARY 2017
48

Fig. 4 Comparison of the visual appearance of aesthetic QR codes, where (a)–(e) are version 10L and
(f)–(j) are version 10M.

do is the modification of module shape as well as the color resolution of the obtained QR code is improved. For conve-
according to a logo image, such a method is equivalent to nience, such an operation is denoted by M1 . For ensuring
the conventional methods like Visualead [7], [8]. the readability, however,  is uniformly added to these sub-
modules. In case of βi, j = Bi, j , the original luminance value
4. Enrichment of Visual Quality Yi, j basically judges correct value by itself. Therefore, the
value of  can be decreased without seriously degrading the
Although the visual quality is improved by the introduc- readabiliy. As the center sub-module is important for most
tion of weight matrix at the encoding of RS block, the readers to determine black/white,  is introduced only for
low resolution of the displayed image still reduces its ben- such a sub-module in the operation M2 . These operations
efit. The reason is due to use of one pixel for representing M1 and M2 can be represented by the following matrices.
one module. The resolution of the displayed logo image is ⎧⎡ ⎤ ⎡ ⎤⎫
(17 + 4v) × (17 + 4v) pixels (e.g. only 37 × 37 pixels for ⎪
⎪ ⎢ 1 1 1 ⎥⎥⎥ ⎢⎢⎢ 1 1 1 ⎥⎥⎥⎪ ⎪
⎨⎢⎢⎢⎢
⎪ ⎪
⎬
version 5). The square shape of modules also degrades the M1 = ⎪⎪ ⎢⎣ 1 1 1 ⎥⎥⎥⎥⎦ , ⎢⎢⎢⎢⎣ 1 1 1 ⎥⎥⎥⎥⎦⎪
⎢ ⎪
⎪
⎩ 1 1 1 ⎪
visual appearance. 1 1 1 ⎭
One simple method to increase the resolution is to use ⎧⎡ ⎤ ⎡ ⎤⎫
⎪
⎪ ⎢ 0 0 0 ⎥⎥⎥ ⎢⎢⎢ 1 1 1 ⎥⎥⎥⎪ ⎪
higher version of QR code. However, higher the version of ⎨⎢⎢⎢⎢
⎪ ⎥⎥⎥ , ⎢⎢⎢ 1 ⎥⎥⎥⎪⎬
QR code becomes, the more difficult it is to read by an op- M2 = ⎪
⎪ ⎢⎢ 0 1 0 ⎥⎦ ⎢⎣ 1 1 ⎥⎦⎪
⎪
⎩⎣ 0
⎪
0 0 1 1 1
⎪
⎭
tical device as the module size becomes small under a con-
stant size of QR code. In addition, most modules become
padding codewords wasting a space. The left matrix is used for the modules satisfying Bi, j = βi, j ,
Our idea is to assign 3 × 3 sub-modules for each mod- where “1” stands for the sub-module that is modified ac-
ule except for the finder pattern modules placed at corners cording to Eq. (11) and Eq. (12), and “0” is the sub-module
of QR code. The QR code reader estimates the module size that uses the original luminance value Yi, j . The right matrix
from the finder pattern, the module shape is not modified. It is the modules that become Bi, j  βi, j .
is possible to only change the center of 3 × 3 sub-modules if In case of βi, j  Bi, j , it is also possible change the strat-
necessary to generate the aesthetic QR code because most of egy for performing Eq. (11) and Eq. (12) to sub-modules. As
QR code readers check the center region of module to deter- mentioned above, the center sub-module is the most impor-
mine black/white. However, depending on the performance tant one among 9 sub-modules. Considering the symmetric
of optical device, the readability must be degraded. On the structure, the following methods M3 and M4 can be good
other hand, if the luminance values of 3 × 3 sub-modules are candidates.
⎧⎡ ⎤ ⎡ ⎤⎫
uniformly represented by the luminance value of the module ⎪
⎪ ⎢ 0 0 0 ⎥⎥⎥ ⎢⎢⎢ 0 1 0 ⎥⎥⎥⎪ ⎪
⎨⎢⎢⎢⎢
⎪ ⎪
⎥⎥⎥⎥ , ⎢⎢⎢⎢ 1 1 1 ⎥⎥⎥⎥⎬
Yi, j and are modified by using Eqs. (11) and (12), the visual M3 = ⎪⎪ ⎢
⎢
⎣
0 1 0
⎦ ⎣ ⎦⎪
⎪
⎪
⎩ 0 0 0 ⎪
quality is still low. Considering the characteristics of popu- 0 1 0 ⎭
lar QR code readers, we examine some types of sub-module
⎧⎡ ⎤ ⎡ ⎤⎫
representation method. ⎪
⎪ ⎢ 0 0 0 ⎥⎥⎥ ⎢⎢⎢ 0 0 0 ⎥⎥⎥⎪ ⎪
⎨⎢⎢⎢⎢
⎪ ⎥⎥⎥ , ⎢⎢⎢ 0 ⎥⎥⎥⎪⎬
Suppose that the luminance values of sub-modules M4 = ⎪
⎪ ⎢⎢ 0 1 0 ⎥⎦ ⎢⎣ 1 0 ⎥⎦⎪
⎪
are respectively derived from a given logo image, and the ⎩⎣ 0
⎪
0 0 0 0 0
⎪
⎭
Eqs. (11) and (12) are applied respectively for each sub-
module to generate an aesthetic QR code. In such a case, the
KURIBAYASHI and MORII: AESTHETIC QR CODE BASED ON MODIFIED SYSTEMATIC ENCODING FUNCTION
49

Fig. 6 Examples of generated QR code using module type M3 , where

the version is 5 and the error correction level is M.

Fig. 5 Hi-resolution of QR code of version 5 with error correction

level M.

Fig. 7 Examples of generated QR code using module type M3 , where

5. Experiments the version is 5 and the error correction level is M.

For the Prop.II method, we change the module shapes as

specified by the matrices M1 to M4 . The version of QR code The purpose of the sub-module in [13] is basically dif-
is v = 5, and the same encoding data in Sect. 3.5 is encoded. ferent from the proposed method. In [13], an input image is
The threshold is  = 64 and the number of candidates is represented by black/white sub-modules while our method
N = 500. use multi-level values for sub-modules to keep the original
color and the quality of image. The readability is assured
5.1 Visual Appearance by keeping the center sub-module unchanged from the in-
put image in [13]. It makes it possible to select types of
Figure 5 shows the generated QR code using four module 3 × 3 sub-modules to represent a certain gray-scaled pixel
types, where the version of QR code is 5 and error correction value. On the other hand, our method assures the readabil-
level is M. It is remarkable that only one center sub-module ity by controlling the color of four neighboring sub-modules
is modified for module types M2 , M3 , and M4 if Bi, j = βi, j . in type M3 . Due to the multi-level values for sub-modules,
The modules satisfying Bi, j = βi, j are mainly flat regions, the visual quality of the proposed method is better than [13].
which are selected as information symbols of RS blocks in In addition, the proposed method can keep the RGB color
order not to be modified by the Prop.II method. Thus, these values while [13] uses only black/white values.
module types manage to preserve the original pixel values
at flat regions as many as possible. As the result, the visual 5.2 Readability
appearance of the displayed image becomes very close to
the original image. These QR codes are printed out to EPSON’s super fine paper
In the experiment, it is extremely difficult to read the using the EPSON ink jet printer PM-A900 without tuning
data for the M4 method. The readability of type M3 is not the printing quality. The printed QR codes are 3.4 × 3.4[cm]
seriously decreased from that of M1 though its visual quality and the threshold is  = 64. Under a fluorescent lighted
is much better as observed from Fig. 5 (c). Figure 6 show environment, each QR code is read using free Apple Store
the results of other images. In some cases, a QR code reader app and Google play applications such as [19] and [20]. We
takes more time to read the data with error correction level at use 9 images and try to read the printed QR codes within 5
L. The main reason is the noise at the optical device. When seconds. The success probabilities for some smart phones
higher error correction level is employed, the readability can are enumerated in Table 3.
be improved, but it also increases the number of different Generally, the optical device is sensitive to external
modules in a QR code. noise including LCD display’s performance, scaling factor,
It is also possible to display a portrait image on QR brightness, printing quality and so on. The readability can
codes. For instance, the “lena” image of 512 × 512 pixels be improved by increasing the threshold  and the module
with 24-bit color image is used in Fig. 7. As the resolution of shape, the scale of displayed/printed size. Such parameters
the original image is higher than the QR code, some block- should be adjusted considering the applications of aesthetic
ing effects appeares in case of version 5. With the increase QR code if each module is not represented by black/white
of the version, the visual appearance can be improved. square. In our experiments, when  ≤ 32, the aesthetic QR
 IEICE TRANS. INF. & SYST., VOL.E100–D, NO.1 JANUARY 2017
50

Table 3 Test of readability from printed QR codes.

case-based reasoning,” Proc. NaBIC’10, pp.401–406, 2010.
device success probability [%] [6] D. Samretwit and T. Wakahara, “Measurement of reading char-
(software) L M actristics of multiplexed image in QR code,” Proc. INCoS’11,
iphone 6 [19] 100 100 pp.552–557, 2011.
iphone 6 [20] 93.7 99.6 [7] Visualead, http://www.visualead.com. [Online; accessed 9-March-
iphone 5s 73.9 85.6 2016].
Xperia Z3 98.3 98.3 [8] logoQ. http://logoq.net. [Online; accessed 9-March-2016].
GALAXY S5 100 100 [9] T. Wakahara, N. Yamamoto, and H. Ochi, “Image processing of dot-
ted picture in the QR code of cellular phone,” Proc. 3PGCIC’10,
pp.454–458, 2010.
code cannot be read by iphone 6 [19], but it depends on the [10] T. Wakahara and N. Yamamoto, “Image processing of 2-dimensional
barcode,” Proc. NBiS’11, pp.484–490, 2011.
optical device and the environmental condition. In practical, [11] QR-JAM. https://www.risec.aist.go.jp/project/qrjam. [Online; ac-
 should be in the range, 64 ≤  ≤ 128. cessed 9-March-2016].
[12] S. Lin and D.J. Costello, Error Control Coding, Second Edition,
6. Concluding Remarks Prentice-Hall, Inc., Upper Saddle River, NJ, USA, 2004.
[13] H.-K. Chu, C.-S. Chang, R.-R. Lee, and N.J. Mitra, “Halftone QR
codes,” ACM Trans. Graph., vol.32, no.6, pp.217:1–217:8, 2013.
In this paper, we investigate the aesthetic QR code by dis-
[14] C. Fang, C. Zhang, and E.-C. Chang, “An optimization model for
playing a color image. It is possible to change the positions aesthetic two-dimentional barcodes,” MMM2013, LNCS, vol.8325,
of information symbols in RS block while keeping the de- pp.278–290, Springer, Heidelberg, 2014.
codability of standard RS decoder. If the first information [15] Y. Zhang, S. Deng, Z. Liu, and Y. Wang, “Aesthetic QR codes
symbols are fixed as the data to be encoded in a QR code, based on two-stage image blending,” MMM2015, LNCS, vol.8936,
the other symbols can be freely selected as the rest of infor- pp.183–194, Springer, Heidelberg, 2015.
[16] K. Fujita, M. Kuribayashi, and M. Morii, “Expansion of image dis-
mation symbols of RS block. Based on such a characteris- playable area in design QR code and its applications,” Proc. FIT’11,
tic of RS code, we propose a modified systematic encoding pp.517–5520, 2011.
method to produce aesthetic QR code. Among several can- [17] M. Kuribayashi and M. Morii, “Enrichment of visual appearance
didates of such QR code, the proposed method selects the of aesthetic QR code,” IWDW2015, LNCS, vol.9569, pp.220–231,
best one. Even though the visual quality can be improved Springer, Heidelberg, 2016.
[18] OpenCV Library, http://opencv.org/. [Online; accessed 22-July-
with the increase of the candidates, the random selection of
2016].
candidates are not suitable. [19] QR reader for iPhone/Android, http://qrreader.blogspot.sg. [Online;
Based on the following two characteristics on logo im- accessed 9-March-2016].
ages, we proposed a weighting function to control the sym- [20] QR Code Reader & Barcode Scanner, https://www.appannie.com/
bols of RS block at the flat and center regions of an image. apps/ios/app/qr-code-reader-barcode-scanner-free-download.
It based on the fact that the outer end of an image is less [Online; accessed 9-March-2016].
important than its center region, and the changes at noisy
region are less sensitive than the flat region. For the im-
provement of the image resolution, we also proposed some
module types and investigated the readability as well as the
visual quality.
In order to increase the resolution of displayed image
Minoru Kuribayashi received B.E., M.E.,
on a QR code, the module should be represented by more and D.E degrees from Kobe University, Kobe,
than 1 pixel. In the proposed method, each module is repre- Japan, in 1999, 2001, and 2004. From 2002 to
sented by 3 × 3 pixels, but it can be increased. If the num- 2007, he was a Research Associate in the De-
ber of sub-modules increases, the visual quality can be im- partment of Electrical and Electronic Engineer-
proved. The appearance can be further improved by consid- ing, Kobe University. In 2007, he was appointed
as an Assistant Professor at the Division of Elec-
ering the characteristic of image by adaptively selecting the
trical and Electronic Engineering, Kobe Univer-
threshold  for each sub-module. sity. Since 2015, he has been an Associate Pro-
fessor in the Graduate School of Natural Science
References and Technology, Okayama University. His re-
search interests include digital watermarking, information security, cryp-
[1] QR code.com, http://www.qrcode.com/en. [Online; accessed 9- tography, and coding theory. He received the Young Professionals Award
March-2016]. from IEEE Kansai Section in 2014.
[2] JIS X0510. http://www.jisc.go.jp/app/pager?id=2738494. [Online;
accessed 22-July-2016].
[3] ISO/IEC 18004. http://www.iso.org/iso/catalogue detail.htm?
csnumber=62021. [Online; accessed 22-July-2016].
[4] S. Ono, K. Morinaga, and S. Nakayama, “Two-dimensional barcode
decoration based on real-coded genetic algorithm,” Proc. CEC’08,
pp.1068–1073, 2008.
[5] S. Ono and S. Nakayama, “A system for decorating QR code with
facial image based on interactive evolutionary computation and
KURIBAYASHI and MORII: AESTHETIC QR CODE BASED ON MODIFIED SYSTEMATIC ENCODING FUNCTION
51

Masakatu Morii received the B.E. de-

gree in electrical engineering and the M.E. de-
gree in electronics engineering from Saga Uni-
versity, Saga, Japan, and the D.E. degree in com-
munication engineering from Osaka University,
Osaka, Japan, in 1983, 1985, and 1989, respec-
tively. From 1989 to 1990 he was an Instruc-
tor in the Department of Electronics and Infor-
mation Science, Kyoto Institute of Technology,
Japan. From 1990 to 1995 he was an Associate
Professor at the Department of Computer Sci-
ence, Faculty of Engineering at Ehime University, Japan. From 1995 to
2005 he was a Professor at the Department of Intelligent Systems and In-
formation Science, Faculty of Engineering at the University of Tokushima,
Japan. Since 2005, he has been a Professor at the Department of Electri-
cal and Electronic Engineering, Faculty of Engineering at Kobe University,
Japan. His research interests are in error correcting codes, cryptography,
discrete mathematics and computer networks and information security.