CN102054178A - Chinese painting image identifying method based on local semantic concept - Google Patents

Abstract

The present invention relates to a kind of Chinese painting image recognition method based on partial semantic concept, comprising the following steps: 1) using a scanning device to collect images of Chinese painting works to be recognized, and storing them in a computer; 2) using a random extractor to extract the collected Chinese paintings Work images are divided into training sample set and test sample set; 3) Extract training sample set and test sample set through the visual attention model respectively in the image of the salient area in the traditional Chinese painting image in the sample set; 4) The Chinese painting image in the training sample set and corresponding 5) According to the bag-of-words model space pyramid model, and generate two corresponding space pyramid feature histograms; 6) adopt the method of serial merging to generate in step 5) Two spatial pyramid feature histograms are fused; 7) Utilize more than one classification method in clustering method, K nearest neighbor method, neural network and support vector machine method to identify the traditional Chinese painting image to be identified in the test sample set, and identify accurately Output recognition results in the form of rate and confusion matrix.

Description

A Chinese Painting Image Recognition Method Based on Partial Semantic Concept

technical field

The invention relates to an image recognition method, in particular to a traditional Chinese painting image recognition method based on local semantic concepts.

Background technique

In recent years, semantic-based image classification and annotation technology has become a research hotspot closely related to CBIR (Content-Based Image Retrieval). Performance, and can bridge the "semantic gap" to a certain extent. Different from traditional digital images, the semantic information covered by traditional Chinese paintings is richer and more abstract. If the automatic classification and labeling of Chinese painting images can be realized, it can be widely used in the field of digital calligraphy and painting museums, and become a key technology in major research projects such as digital libraries.

The purpose of image scene classification is to classify the image as a whole into a certain scene category. Most of the existing research work at home and abroad focuses on the scene classification of natural images, that is, how to automatically classify the images to be recognized into a set of semantic categories (such as beaches, mountains, etc.). For the research field of digital images of Chinese calligraphy and painting, the classification methods of Chinese painting images mainly focus on the representation based on low-level visual features, and use support vector machines, decision tree algorithms, etc. to realize the automatic classification of Chinese painting images. Image Classification Approaches for Semantic Modeling.

In order to bridge the semantic gap and deal with image recognition problems in complex scenes, researchers achieve the purpose of image scene classification by implementing semantic modeling of image scenes. Layer representation methods in images based on local semantic concepts have become the mainstream methods because they do not depend on the results of image segmentation and show good classification performance.

In 2005, Fei-Fei proposed a new Bayesian hierarchical model for natural scene classification. Different from previous work, this method does not require a specially labeled training sample set, but forms a bag of words through clustering of local regions to represent images, and is finally satisfied with the experiment on a large-scale complex scene set containing 13 categories classification performance. Quelhas in 2005 and Bosch in 2006 respectively proposed the combination of Bag of words and pLSA models. The difference between the two lies in the different methods of extracting local descriptors. The former is based on sparse SIFT descriptors and the latter is dense SIFT descriptors. In 2006, Perronin proposed an image classification idea based on Bag of words and GMM (Gaussian Mixture Models, Gaussian Mixture Model). This method can describe the image content of all recognized image categories and can be improved by training typical class sample data learning. Post-adaptive dictionary-like. Previous methods based on visual dictionaries used a single histogram to describe an image, but the innovation of this method is to use a series of histograms to describe an image.

Although the above methods are effective, they do not consider and utilize the spatial structure information in the image. In the complex natural image scene classification system, the context information of this spatial structure (such as the spatial relationship between adjacent local objects or a certain The absolute position of objects in some scenes) can further improve the performance of the classifier and help to get better classification results. In 2006, Lazebnik proposed a classification algorithm for spatial pyramid matching higher than Bag of Words. This method divides the image into gradually smaller sub-regions and calculates the local feature histogram of each sub-region block, and then uses these local feature histograms to represent the image. The "Spatial Pyramid" is a simple, computationally efficient extension of image representations to unordered feature bags, and demonstrates important, far-reaching improvements in performance on the very challenging scene classification problem. However, for the image sample library with a large background area, the classification results of this method will be biased.

Contents of the invention

In view of the above problems, the purpose of the present invention is to propose a Chinese painting image recognition method based on local semantic concepts that combines global image features and local image features.

In order to achieve the above object, the present invention adopts the following technical schemes: 1, a kind of Chinese painting image recognition method based on local semantic concept, it comprises the following steps: 1) utilize scanning equipment to carry out image collection of the Chinese painting work to be recognized, and store in the computer ; 2) divide the collected Chinese painting works image into a training sample set and a test sample set by a random extractor; 3) extract the salient area images in the training sample set and the test sample set in the Chinese painting works image respectively by a visual attention model; 4 ) To the Chinese painting works image and the corresponding salient region image in the training sample set, set up the word bag model of the Chinese painting work image respectively; 5) According to the Chinese painting work image bag word model established in the training sample set and the corresponding salient region image word bag model, respectively constructing the spatial pyramid model of the image of traditional Chinese painting and the spatial pyramid model of the corresponding salient region image, and generating two corresponding spatial pyramid feature histograms; 7) Utilize more than one classification method in clustering method, K nearest neighbor method, neural network and support vector machine method to identify the traditional Chinese painting image to be identified in the test sample set, and use the recognition accuracy rate Output recognition results in the form of confusion matrix.

A_i为其中的一幅国画图像，P₂表示人物画，记为P₂＝{B₁，B₂，...，B_i}，B_i为其中的一幅国画图像，P₃表示山水画，记为P₃＝{C₁，C₂，...，C_i}，C_i为其中的一幅国画图像；③分别从P₁、P₂和P₃中随机选取设定数量的图像作为训练样本集Q，记为{P₁′，P₂′，P₃′}，用于生成国画图像识别的模型；将P₁、P₂和P₃中剩余的图像作为测试样本集

，用于校准。Described step 2) in the generation method of training sample set and test sample set include: 1. define the category of Chinese painting image, category number is 1～n, and n is a natural number; , recorded as {P ₁ , P ₂ , P ₃ }. Among them, P ₁ represents flower-and-bird painting, denoted as

A _i is one of the traditional Chinese painting images, P ₂ represents figure painting, recorded as P ₂ ={B ₁ , B ₂ ,...,B _i }, B _i is one of the traditional Chinese painting images, P ₃ represents landscape painting , recorded as P ₃ ={C ₁ , C ₂ ,...,C _i }, where C _i is one of the traditional Chinese painting images; ③Randomly select a set number of images from P ₁ , P ₂ and P ₃ respectively As a training sample set Q, denoted as {P ₁ ′, P ₂ ′, P ₃ ′}, it is used to generate a model for Chinese painting image recognition; the remaining images in P ₁ , P ₂ and P ₃ are used as a test sample set

, for calibration.

Said step 4) sets up the bag of words model of Chinese painting image, comprises the following steps: 1. the grayscale of Chinese painting image carries out grayscale processing to the color Chinese painting image in the training sample set and the salient region image respectively according to the following formula: Gray( i, j)=0.11*R(i, j)+0.59*G(i, j)+0.3*B(i, j); where i, j is the position of a pixel in the image, R(i, j) is the red component of the pixel color represented by i, j, G(i, j), B(i, j) represent the green and blue components respectively, and Gray(i, j) represents the converted gray of the point ② select the key points of the SIFT (Scale-invariant feature transform) descriptor for the grayscale image obtained in step ① respectively, and use the gradient direction distribution characteristics of the key point neighborhood pixels to generate Specify the direction parameters, generate SIFT feature vectors, and perform illumination normalization processing on the SIFT feature vectors as needed; ③Construct the visual vocabulary according to the SIFT feature vectors of the original Chinese painting image and the prominent area image of Chinese painting obtained in step ②; The vocabulary contains K visual words, K is a natural number, the general value is 500-1200, and K is recommended to be 1000; ④Use the two obtained visual vocabularies to extract and represent local semantic concept features, that is, to calculate a certain The Euclidean distance between the SIFT feature in the neighborhood of the SIFT key point and the SIFT feature corresponding to each visual word in the visual vocabulary, using the nearest neighbor visual word to define the SIFT key point, and mapping all SIFT key points to the visual vocabulary In the table, the image is described by the label of the visual word, that is, the local semantic concept feature of the image is obtained. The histogram feature representation is used to represent the local semantic concept feature of the image.

The step of selecting the key point of the SIFT descriptor in 2. of said step 4) is as follows: A, the original image of the traditional Chinese painting is sampled by grid sampling;

Described step 5) constructing space pyramid model comprises the following steps: 1. traditional Chinese painting image is divided into sub-image areas of different sizes in two-dimensional image space, forms space pyramid sub-block; Space pyramid layer number is 2-5; 2. The formed spatial pyramid block image constructs the corresponding spatial pyramid feature histogram.

In the step 6), the fusion of the two spatial pyramid feature histograms includes one of the following two methods: one is to connect two groups of feature vectors end to end to generate a joint vector as a new feature vector, in a higher dimension The other is to use complex vectors to combine two sets of feature vectors of the same sample, and perform feature extraction in complex vector space, that is, parallel combination.

The step when adopting support vector machine method to classify in described step 7) is as follows: 1. the generation of classifier model; Adopt LIBSVM-fast tool kit to carry out recognition experiment, training generates the required parameter of classifier model as options='-t4 -s0-b1-c1', which means that the kernel function is a cross kernel function, and the SVM type is C-svc; the C-svc penalty coefficient is 1, and probability estimation is required; ② output the Chinese painting image to be recognized in the test sample set Result; Utilize steps 3)～6) to process the Chinese painting image to be identified in the test sample set, obtain the corresponding feature vector, and input it into the trained classifier model, the classification result of the image can be obtained according to the formula of the classifier model; ③Recognition result evaluation methods include recognition accuracy and confusion matrix.

The present invention has the following advantages due to taking the above technical scheme: compared with the natural scene image classification method proposed by people such as Lazebnik, the present invention introduces and extracts the local salient region image (partial image) in the global traditional Chinese painting image (global image), for The global image and the local image use different methods to extract the SIFT descriptor key point information in the image, and realize the fusion of the local semantic concept features of the global image and the local image, so that the local semantic concept features of the global Chinese painting image and the local salient image are simultaneously The analysis of information can obtain more feature information that is helpful for classification and recognition and is more discriminative, so it can improve the accuracy of classification and recognition of traditional Chinese painting images. Compared with the traditional Chinese painting scene image classification method realized by using low-level visual features proposed by James.Wang and Jiang Shuqiang, the present invention has stronger expansibility. And the middle-level semantic modeling classification method is extended to the application field of traditional Chinese painting images.

Description of drawings

Fig. 1 is a block diagram of the present invention

Fig. 2 is the original picture of Chinese painting imported by the present invention

Fig. 3 is the salient area image in the original picture of traditional Chinese painting extracted by the present invention

Fig. 4 is the word bag representation model flowchart of the present invention

Fig. 5 is the algorithm flow chart of the local descriptor SIFT feature of the original picture of traditional Chinese painting in the present invention

Fig. 6 is a schematic diagram of the uniform grid sampling method of the present invention

Fig. 7 is a schematic diagram of a Chinese painting image uniform grid sampling example of the present invention

Fig. 8 is the algorithm flow chart of the local descriptor SIFT feature of the salient region image of traditional Chinese painting in the present invention

Fig. 9 is that the present invention constructs the Chinese painting image space pyramid legend

Fig. 10 is a schematic diagram of the flow chart of classification and recognition in the present invention

Detailed ways

The present invention will be described in detail below in conjunction with the accompanying drawings and embodiments.

According to the content of the painting, Chinese painting can be roughly divided into three categories: figure painting, landscape painting and flower, bird and animal painting. Each of these major categories can be divided into different subcategories. For example, figure painting is a painting subject that mainly depicts people. Portrait painting, story painting, genre painting, etc. The present invention is based on the traditional Chinese painting image recognition method of local semantic concept and comprises the following steps:

1) As shown in Figure 1, utilize the scanning device to scan several pieces of Chinese painting works to be identified, and store them in the computer. The scanning device can adopt various prior art devices. The scanning device used in this embodiment is an Expression10000XL flatbed scanner The parameters for saving the image are: 24-bit color depth, 400dpi resolution, and JPEG image file format.

2) each piece of traditional Chinese painting images collected is input into a random extractor, and the random extractor is divided into a training sample set and a test sample set to the imported Chinese painting images, and the classification steps are as follows:

1. define the category of Chinese painting image, category numbering is 1, 2, ..., n, and n is a natural number, and present embodiment divides Chinese painting into flower-and-bird painting, figure painting and landscape painting three major classes according to the content of painting, i.e. n=3, (In the following, n=3 is taken as an example for illustration, but not limited thereto).

A_i为其中的一幅国画图像，P₂表示人物画，记为P₂＝{B₁，B₂，...，B_i}，B_i为其中的一幅国画图像，P₃表示山水画，记为P₃＝{C₁，C₂，...，C_i}，C_i为其中的一幅国画图像，其中i为对应图像的数量。。② Assume that the image representative set of Chinese painting works to be recognized is P, denoted as {P ₁ , P ₂ , P ₃ }. Among them, P ₁ represents flower-and-bird painting, denoted as

A _i is one of the traditional Chinese painting images, P ₂ represents figure painting, recorded as P ₂ ={B ₁ , B ₂ ,...,B _i }, B _i is one of the traditional Chinese painting images, P ₃ represents landscape painting , recorded as P ₃ ={C ₁ , C ₂ ,...,C _i }, where C _i is one of the traditional Chinese painting images, where i is the number of corresponding images. .

，用于校准。③Respectively randomly select a set number of images from P ₁ , P ₂ and P ₃ as the training sample set Q, denoted as {P ₁ ′, P ₂ ′, P ₃ ′}, which is used to generate a model for Chinese painting image recognition; Use the remaining images in P ₁ , P ₂ and P ₃ as the test sample set

, for calibration.

3) Input the original image (as shown in Figure 2) of the training sample set and test sample set separated in step 2) into the visual attention model, and the visual attention model extracts the salient regions in the Chinese painting image from the training sample set Q Image set Q _display (as shown in Figure 3): The process of finding salient objects in the image is in line with the biological mechanism of the human visual nervous system to select salient objects in the visual scene, and retain as many main semantic regions in Chinese paintings that are helpful for classification , remove some redundant regions, the visual attention model among the present invention can adopt the visual attention model of Itti-Koch (personal name), also can adopt GBVS (Graph-Based Visual Saliency based on the salience of graph) of Jonathan Harel (personal name) Analysis) algorithm, but not limited to this. The Itti-Koch model is mainly divided into two steps: the extraction of visual features and the calculation of the saliency map. The GBVS algorithm is an improvement of the classic Itti-Koch model.

4) as shown in Figure 4, according to the training sample set Q obtained in step 3) and the salient region image _set Q, the process of setting up the word bag model of the traditional Chinese painting image is as follows:

①Grayscale of traditional Chinese painting images, respectively carry out grayscale processing on the color traditional Chinese painting images in the training sample set Q and the significant area _image Q, which are respectively recorded as Q′ and Q _′ , and the specific grayscale processing steps include:

To convert a color image into a grayscale image, convert it according to the following general formula:

Gray(i,j)=0.11*R(i,j)+0.59*G(i,j)+0.3*B(i,j)

Where i, j is the position of a pixel in the image, R(i, j) is the red component of the pixel color represented by i, j, similarly G(i, j), B(i, j) respectively Represents the green and blue components, and Gray(i, j) represents the converted gray level of the point. Finally, the RGB component values of the pixel point are all set to Gray(i, j). The color image can be converted into a grayscale image according to the above method.

② Select the key points of the SIFT (Scale-invariant feature transform, scale invariant feature transform) descriptor for the grayscale images in Q′ and _Qdisplay obtained in step ① respectively, and use the gradient direction distribution characteristics of the neighborhood pixels of the key points as Each key point specifies a direction parameter, generates a SIFT feature vector, and performs illumination normalization processing on the SIFT feature vector as required.

Among them, the extraction method of the local descriptor SIFT feature is divided into two parts, one is to describe the original image of traditional Chinese painting locally, and the other is to describe the image of the salient area of traditional Chinese painting locally:

As shown in Figure 5, the uniform grid sampling method is used to select SIFT feature key points for the image in Q′, and the grid sampling method is to sample the image according to the grid of M*M pixel size (as shown in Figure 6) , where M is an integer power of 2. It is recommended that M be 8 or 16 to ensure that the image is divided into a certain number of grids; if the width and height of the image are Width and Hight respectively, then:

X=(Width%M)/2+1;

Y=(Hight%M)/2+1;

X, Y are the starting point coordinates for grid sampling. A total of (Width/X)*(Hight/Y) uniform grids are generated. The intersection points of the uniform grids are used as the key points of SIFT features, and the key points are selected as the center of the circle. , the circle with M as the radius is the neighborhood, and the gradient direction distribution characteristics of the pixels in the neighborhood of the key point are used to specify the direction parameter for each key point to generate the SIFT feature vector (as shown in Figure 7).

As shown in 8, the method of scale space extremum detection is used to select the SIFT feature key points for the Q- _display image, that is, the two-dimensional plane space of the image and the DoG (Difference-of-Gaussian Gaussian kernel difference) scale space are simultaneously detected. The local extremum is used as the key point of the feature to make the feature have good uniqueness and stability. The DoG operator is defined as two Gaussian difference kernels of different scales, which has the characteristics of simple calculation and is an approximation of the normalized LoG (Laplacian-of-Gaussian) operator.

The DoG operator is shown in the following formula:

D(x,y,σ)=(G(x,y,kσ)-G(x,y,σ))*I(x,y)=L(x,y,kσ)-L(x,y , σ)

In the formula, G(x, y, kσ) is a two-dimensional Gaussian function, k represents the scaling factor of the scale factor, σ represents the variance of the Gaussian normal distribution, I(x, y) represents the original image, and L represents the scale space of the image .

Among them, L(x, y, σ) is defined as follows:

L(x,y,σ)=G(x,y,σ)*I(x,y)

The purpose of convolving the image I(x, y) with the Gaussian kernel G(x, y, σ) under different scale factors is to obtain stable feature points in different scale spaces.

Among them, G(x, y, kσ) is defined as follows:

$\mathrm{<span\; class="notranslate">\; G</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; \&sigma;</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; 2</span>}\mathrm{<span\; class="notranslate">\; \&pi;</span>}{\mathrm{<span\; class="notranslate">\; \&sigma;</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}{\mathrm{<span\; class="notranslate">\; e</span>}}^{<span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="2"\; label="x"\; filenames="HDA0000044640250000032.png"\; state="\{\{state\}\}">x</figure-callout></span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; 2</span>}{\mathrm{<span\; class="notranslate">\; \&sigma;</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}$

In the formula, (x, y) represents the pixel position of the image, and σ is called the scale space factor. The smaller the value, the less the image is smoothed, and the smaller the corresponding scale. The large scale corresponds to the general appearance of the image Features, the small scale corresponds to the detailed features of the image.

③As shown in Figure 4, the SIFT feature vector obtained in step ② is used to construct a visual vocabulary: use the K-Means algorithm to cluster all SIFT feature vectors generated on the training sample set Q, and each cluster center is regarded as a visual vocabulary words, thus generating a visual vocabulary consisting of K visual words. The numbers of the visual words in the visual vocabulary can also be called local semantic concepts. K is a natural number, the general value is 500-1200, and it is recommended that K be 1000. This visual vocabulary is only generated during training. In the same way, a visual vocabulary is also built on the Q _display .

④ Utilize the visual vocabulary obtained after processing the training sample set Q in the above step ③, and the visual vocabulary _obtained by processing the salient area image Q, and extract and represent the local semantic concept features of Q and Q _displayed images respectively:

First, calculate the Euclidean distance between the SIFT feature in the neighborhood of a certain SIFT key point and the SIFT feature corresponding to each visual word in the visual vocabulary, and use the nearest neighbor visual word to define the SIFT key point. Then, each SIFT key point in a given image is processed sequentially, all SIFT key points are mapped to the visual vocabulary, and the above-mentioned image is described with the label of the visual word, that is, the local semantic concept feature of the image is obtained. Finally, the histogram feature representation is used to represent the local semantic concept features of the image, that is, the visual word distribution probability histogram of the image.

5) As shown in Figure 9, in step 3) the training sample set Q obtained and the salient area image _set Q are respectively constructed spatial pyramid models, which includes the following steps:

①Space pyramid blocks of traditional Chinese painting images, specifically:

The whole Chinese painting image is divided into sub-image areas of different sizes in the two-dimensional image space to form an image space pyramid G.

Let the number of layers of G be L, l represents the lth layer of the space pyramid G, l=0, 1, . . . , L-1. The number of sub-image areas is D, r represents the label of the sub-image area, r=0, 1, . . . , D-1.

D＝(2 ^l )×(2 ^l )

When l=0, it means that it is at the bottom of the pyramid, and the number of blocks in the image is 1 at this time. L generally takes a value of 3-5, and 4 is recommended.

② Construct a spatial pyramid feature histogram, specifically;

Firstly, each sub-block image of each layer in the image space pyramid G is expressed as a local semantic concept feature histogram, and then these feature histograms are given appropriate weights and combined serially to form a total feature histogram, That is, the spatial pyramid feature histogram. In this embodiment, the weight value is 2 ^L-l+1 .

(其中r表示子图像区域标号，r＝0，1，......，D-1)，H^l代表空间金字塔G第l层上串行组合后的直方图；H为该图像经过空间金字塔分块表示后，形成总的特征直方图。Let the histogram of the rth sub-image area on the lth layer of the spatial pyramid G be

(where r represents the sub-image area label, r=0, 1, ..., D-1), H ¹ represents the histogram after serial combination on the first layer of space pyramid G; H is that the image passes through After the space pyramid is divided into blocks, a total feature histogram is formed.

${\mathrm{<span\; class="notranslate">\; h</span>}}^{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; =</span><span\; class="notranslate">\; [</span>{\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}^{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}^{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; ,</span><span\; class="notranslate">\; .</span><span\; class="notranslate">\; .</span><span\; class="notranslate">\; .</span><span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; r</span>}}^{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; ]</span>$

$\mathrm{<span\; class="notranslate">\; h</span>}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{{\mathrm{<span\; class="notranslate">\; 2</span>}}^{\mathrm{<span\; class="notranslate">\; L</span>}}<span\; class="notranslate">\; *</span>{\mathrm{<span\; class="notranslate">\; h</span>}}^{\mathrm{<span\; class="notranslate">\; 0</span>}}}<span\; class="notranslate">\; +</span>\underset{\mathrm{<span\; class="notranslate">\; l</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; L</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}<span\; class="notranslate">\; (</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{{\mathrm{<span\; class="notranslate">\; 2</span>}}^{\mathrm{<span\; class="notranslate">\; L</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; l</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; *</span>{\mathrm{<span\; class="notranslate">\; h</span>}}^{\mathrm{<span\; class="notranslate">\; l</span>}}}<span\; class="notranslate">\; )</span>$

l=1, 2, ..., L-1

In the same way, the spatial pyramid feature histogram of the image of the salient area of traditional Chinese painting is also constructed on the Q _display .

6) Fuse the spatial pyramid feature histogram constructed on the training sample set Q in ② of step 5) with the spatial pyramid feature histogram _constructed on the salient region image set Q. It includes the following steps:

① Use steps 4) and 5) in turn to process the Q obtained in step 2) and _{the display} of Q obtained in step 3).

Step 2) obtains the training sample set Q; step 3) generates the salient region image _Q , uses step 4) and step 5), respectively processes Q and Q _to obtain the corresponding spatial pyramid feature histogram, step 5) Realize the fusion of spatial pyramid feature histograms. The purpose of this fusion is to include both global features and local features to obtain better recognition results.

According to the training sample set Q={q ₁ , q ₂ , . . . , q _e }, e represents the number of images in the training sample set. Then, the feature histogram obtained after processing Q by step 4) and step 5) is _Hyuan ={ _Hyuan1 , _Hyuan2 , . . . , _Hyuane }.

In the same way, the feature histogram obtained after _processing the salient area Q of the traditional Chinese painting image by step 4) and step 5) is H _significant ={H _{significant 1} , H _{significant 2} ,..., H _{significant e} }.

② Serially merge the feature histogram H _original and H _display generated in ①.

At present, there are feature fusion methods, one is to connect two sets of feature vectors end to end to generate a joint vector as a new feature vector, and perform feature extraction in a higher-dimensional vector space, that is, serial combination; the other is to use complex The vector merges two sets of feature vectors of the same sample together, and performs feature extraction in the complex vector space, that is, parallel combination. What the present invention uses here is the method of serial combination, the result after final fusion: H={H _original , H _shows }.

7) As shown in Figure 10, select and utilize one or several classification methods in the methods such as existing clustering method, K nearest neighbor method, neural network and support vector machine, the traditional Chinese painting image to be identified in the test sample set is identified , output the recognition results in the form of recognition accuracy and confusion matrix, the specific steps are as follows:

① Generation of classifier model

The feature vector H extracted from the training sample set Q, the category label H_label corresponding to the training sample set Q, and related parameter options are used as the input of the training classifier model, and the classifier model model is output as the result. The invention adopts the LIBSVM-fast toolkit to carry out the recognition experiment, but is not limited thereto. On the simulation environment MatlabR2008A software platform, the following function model can be used to represent:

model = svmtrain(H, H_label, options);

Among them, H_label={label1, label2, ..., label _e }, the value range of label _e is 1~n, where n=3, representing flower-and-bird painting, landscape painting and figure painting respectively.

Options (operating parameters): The available options indicate the following meanings:

-t kernel function type: set the kernel function type. Optional types are

0——linear kernel 1——polynomial kernel

2——RBF kernel 3——sigmoid kernel

4——intersection core

-s set svm type

0——C-svc 1——V-svc

2——One-class-svm 3——ε-SVR

4——γ-SVR

-b probability estimate: Whether to calculate the probability estimate of SVC or SVR, the optional value is 0 or 1, and the default is 0.

-c cost: Set the penalty coefficient C in C-svc, ε-SVR, γ-SVR, the default value is 1.

The parameter options='-t4-s0-b1-c1' means that the kernel function is intersectionkernel, the SVM type is C-svc; the C-svc penalty coefficient is 1, and probability estimation is required.

② Output the result of the Chinese painting image to be recognized in the test sample set, which is specifically:

Use steps 3) to 6) to process the Chinese painting images to be recognized in the test sample set to obtain the corresponding feature vectors, and input them into the trained classifier model, and the image classification results can be obtained according to the formula of the classifier model.

Use steps 3) to 6) in sequence to process the Chinese painting images to be recognized in the test sample set C _p Q to obtain the corresponding feature histogram vectors H and H_label. H, H_label of the test sample set C _p Q and the model generated in ① of step 7) are used as input, and the test result of the test sample set C _p Q is the recognition accuracy. The invention adopts the LIBSVM-fast toolkit to carry out the recognition experiment, but is not limited thereto. In the simulation environment MatlabR2008A software platform, the following function model can be used to represent:

[VP]=svmpredict(H_label, H, model, libsvm_options);

Here, libsvm_options='-b 1' means that probability estimation is required. The meaning of the output result V is to predict the category label of the test sample set, and P is to predict the recognition accuracy of the test sample set.

③Recognition result evaluation method

There are two ways to evaluate the final recognition results, recognition accuracy and confusion matrix. Assuming that the recognition accuracy is p, define the following formula:

P=n/N;

Among them, n is the number of correctly recognized images, and N is the total number of images to be recognized.

Confusion matrix is a commonly used accuracy evaluation tool in pattern recognition. In image accuracy evaluation, it is mainly used to compare classification results and real results, and the accuracy of classification results can be displayed in a confusion matrix. A perfect classification model is that if a target object actually belongs to category A, it is also predicted to be category A, and if it is in category B, it is also predicted to be B. But in fact, the model often predicts that the object of category A is category B, and some objects that are originally category B are predicted as category A. Then, how much the model predicts is right and how much it is wrong, the confusion matrix puts all this information into a table (as shown in Table 2):

Table 2 confusion matrix

Among them, n _AA , n _BB , and n _CC on the diagonal are the number of correct predictions for each class; those on the non-diagonal are the number of errors in predicting the corresponding class for this class, such as n _BA is the number of B predicted as A Number; n _AC is the number of A predicted as C.

The present invention will be used in the classification of Chinese painting images, and its performance can be provided by the classification experiment that following actual Chinese painting images complete, the experimental data set is the Chinese painting image library that is derived from the scanning of the "Complete Works of Chinese Painting" album, and it contains 1303 pieces of Chinese painting images ( Among them, the number of training samples is 639, and the number of test samples is 664, approximately satisfying 1:1), the size of each image is 512* (the maximum length and width do not exceed 512), color image, jpg format. Design the sample set for the classification experiment, the details are as follows, where A represents flowers and birds, B represents people, and C represents landscapes.

Training sample set: A262; B157; C220;

Test sample set: A261; B103; C300;

The classifier uses the current mainstream support vector machine classifier, the version is Fast-Libsvm-2.84-1, the parameter options in the experiment='-t4-s0-b1-c1', the experimental results are as follows (as shown in Table 3):

Table 3 Chinese painting image classification and recognition result table

Method 1 in the table is a recognition method using only global original image features of Chinese paintings; Method 2 is a recognition method using only local salient area image features of local Chinese paintings; Method 3 is a method proposed by this method, a recognition method that combines global features and local features.

Table 4, Table 5, and Table 6 are the confusion matrices corresponding to the recognition results of Method 1, Method 2, and Method 3, respectively.

Table 4 Confusion Matrix Corresponding to Method 1

Table 5 Confusion Matrix Corresponding to Method 2

Table 6 Confusion Matrix Corresponding to Method 3

Known from Table 3, the method proposed by the method of the present invention comprehensively utilizes the global feature information and the local feature information of the Chinese painting image to carry out the recognition of the Chinese painting image, and compared with the previous two methods, the recognition accuracy of the Chinese painting has been improved. Known by table 4, table 5, table 6, method 3 has all improved than method 1 and method 2 on the correct recognition number of A class and B class traditional Chinese painting image, also can be step 4 simultaneously) ③ middle parameter K and step 5) The selection of the parameter L in ① provides a basis, that is, the selection of the optimal K and L is aimed at higher recognition accuracy and more correct recognition numbers.

Although the present invention has obtained more ideal recognition results, if the color, texture and other features can be considered, the accuracy rate will be further improved, and it will help the automatic classification, labeling and retrieval of traditional Chinese painting images.

Claims (10)

1. traditional Chinese Painting image-recognizing method based on local semantic concept, it may further comprise the steps:

1) utilizes scanning device that traditional Chinese Painting works to be identified are carried out image acquisition, and deposit in the computing machine;

2) by randomly drawing device the traditional Chinese Painting works image that collects is divided into training sample set and test sample book collection;

3) extract the marking area image in the traditional Chinese Painting works image in training sample set and the test sample book collection respectively by visual attention model;

4), set up the speech bag model of traditional Chinese Painting works image respectively to traditional Chinese Painting works image in the training sample set and corresponding marking area image;

5) according to traditional Chinese Painting works image speech bag model of setting up in the training sample set and corresponding marking area image speech bag model, make up the space pyramid model of traditional Chinese Painting works image and the space pyramid model of corresponding marking area image respectively, and generate corresponding two space pyramid feature histograms;

6) method that adopts serial to merge merges two space pyramid feature histograms that generate in the step 5);

7) utilize more than one sorting techniques in clustering method, k nearest neighbor method, neural network and the support vector machine method to concentrate traditional Chinese Painting image to be identified to discern, export recognition result with the mode of recognition accuracy and confusion matrix to test sample book.

2. a kind of traditional Chinese Painting image-recognizing method based on local semantic concept as claimed in claim 1 is characterized in that: the generation method of training sample set and test sample book collection comprises described step 2):

1. define the classification of traditional Chinese Painting image, classification is numbered 1～n, and n is a natural number;

2. hypothesis is used for traditional Chinese Painting works image representative collection to be identified for P, is designated as { P ₁, P ₂, P ₃.P wherein ₁The expression flower-and-bird painting is designated as

A _iBe a width of cloth traditional Chinese Painting image wherein, P ₂The expression figure painting is designated as P ₂={ B ₁, B ₂..., B _i, B _iBe a width of cloth traditional Chinese Painting image wherein, P ₃The expression landscape painting is designated as P ₃={ C ₁, C ₂..., C _i, C _iBe a width of cloth traditional Chinese Painting image wherein;

3. respectively from P ₁, P ₂And P ₃The image that middle picked at random is set quantity is designated as { P as training sample set Q ₁', P ₂', P ₃', be used to generate the model of traditional Chinese Painting image recognition; With P ₁, P ₂And P ₃In remaining image as the test sample book collection

, be used for calibration.

3. a kind of traditional Chinese Painting image-recognizing method based on local semantic concept as claimed in claim 1 or 2 is characterized in that: set up the speech bag model of traditional Chinese Painting image in the described step 4), comprise following steps:

1. the gray processing of traditional Chinese Painting image, carry out gray processing to colored traditional Chinese Painting image in training sample set and the marking area image by following formula respectively and handle:

Gray(i，j)＝0.11*R(i，j)+0.59*G(i，j)+0.3*B(i，j)

I wherein, j is the position of pixel in image, R (i j) is i, the red component of the pixel color that j is represented, G (i, j), (i j) represents green and blue component respectively to B, and (i j) represents grey level after this some conversion to Gray;

2. the gray level image that respectively 1. step is obtained is chosen SIFT (Scale-invariant feature transform, the conversion of yardstick invariant features) key point of descriptor, utilize the gradient direction distribution character of key point neighborhood territory pixel to be each key point assigned direction parameter, generate the SIFT proper vector, and as required the SIFT proper vector is carried out unitary of illumination and handle;

3. the SIFT proper vector of former figure of traditional Chinese Painting that 2. obtains according to step and traditional Chinese Painting marking area image is come, and makes up the visual vocabulary table respectively; The visual vocabulary table comprises K vision word, and K is a natural number, and general value is 500-1200, and suggestion K is taken as 1000;

4. utilize two visual vocabulary tables that obtain, carry out local semantic concept Feature Extraction and expression, promptly calculate each vision word in SIFT feature and the visual vocabulary table in some SIFT key point neighborhoods the Euclidean distance of corresponding SIFT feature, vision word with arest neighbors defines this SIFT key point, all SIFT key points are mapped in the visual vocabulary table, describe this width of cloth image with the label of vision word, the local semantic concept feature that promptly obtains this image adopts the histogram feature representation to represent the local semantic concept feature of this image.

4. a kind of traditional Chinese Painting image-recognizing method based on local semantic concept as claimed in claim 3 is characterized in that: described step 4) 2. in to choose the step of key point of SIFT descriptor as follows:

A, adopt the grid sampling method to sample to the former figure of traditional Chinese Painting;

B, traditional Chinese Painting marking area image is adopted metric space extreme value detection method.

5. a kind of traditional Chinese Painting image-recognizing method based on local semantic concept as claimed in claim 1 is characterized in that: make up the space pyramid model in the described step 5) and may further comprise the steps:

1. the traditional Chinese Painting image is divided into the sub-image area of different sizes in the two dimensional image space, forms space pyramid piecemeal; The space pyramid number of plies is 2～5;

2. the space pyramid block image that forms is made up corresponding space pyramid feature histogram.

6. as claim 1 or 2 or 4 or 5 described a kind of traditional Chinese Painting image-recognizing methods, it is characterized in that: in the described step 6), the fusion of two space pyramid feature histograms is comprised one of following two kinds of methods based on local semantic concept:

A kind of is two eigenvectors to be joined end to end generate an associating vector as new proper vector, carries out feature extraction, i.e. serial combination in the vector space of higher-dimension more;

Another kind is to utilize complex vector that two eigenvectors of same sample are combined, and carries out feature extraction, i.e. The parallel combined at complex vector space.

7. a kind of traditional Chinese Painting image-recognizing method based on local semantic concept as claimed in claim 3 is characterized in that: in the described step 6), the fusion of two space pyramid feature histograms is comprised one of following two kinds of methods:

A kind of is two eigenvectors to be joined end to end generate an associating vector as new proper vector, carries out feature extraction, i.e. serial combination in the vector space of higher-dimension more;

Another kind is to utilize complex vector that two eigenvectors of same sample are combined, and carries out feature extraction, i.e. The parallel combined at complex vector space.

8. as claim 1 or 2 or 4 or 5 or 7 described a kind of traditional Chinese Painting image-recognizing methods based on local semantic concept, it is characterized in that: it is as follows to adopt support vector machine method to carry out the step of branch time-like in the described step 7):

1. the generation of sorter model

Adopt the LIBSVM-fast kit to discern experiment, it is options='-t4-s0-b1-c1 ' that training generates the required parameter of sorter model, and the implication of its expression is that kernel function is the intersection kernel function, and the SVM type is C-svc; The C-svc penalty coefficient is 1, and needs probability estimate;

2. export test sample book and concentrate the result of traditional Chinese Painting image to be identified;

Utilize step 3)～6) handle the concentrated traditional Chinese Painting image to be identified of test sample book, obtain the characteristic of correspondence vector, and, can obtain the classification results of image according to the formula of sorter model the sorter model that its input trains;

3. the recognition result evaluation method comprises recognition accuracy and two kinds of methods of confusion matrix.

9. a kind of traditional Chinese Painting image-recognizing method based on local semantic concept as claimed in claim 3 is characterized in that: it is as follows to adopt support vector machine method to carry out the step of branch time-like in the described step 7):

1. the generation of sorter model

Adopt the LIBSVM-fast kit to discern experiment, it is options='-t4-s0-b1-c1 ' that training generates the required parameter of sorter model, and the implication of its expression is that kernel function is the intersection kernel function, and the SVM type is C-svc; The C-svc penalty coefficient is 1, and needs probability estimate;

2. export test sample book and concentrate the result of traditional Chinese Painting image to be identified;

Utilize step 3)～6) handle the concentrated traditional Chinese Painting image to be identified of test sample book, obtain the characteristic of correspondence vector, and, can obtain the classification results of image according to the formula of sorter model the sorter model that its input trains;

3. the recognition result evaluation method comprises recognition accuracy and two kinds of methods of confusion matrix.

10. a kind of traditional Chinese Painting image-recognizing method based on local semantic concept as claimed in claim 6 is characterized in that: it is as follows to adopt support vector machine method to carry out the step of branch time-like in the described step 7):

1. the generation of sorter model

Adopt the LIBSVM-fast kit to discern experiment, it is options='-t4-s0-b1-c1 ' that training generates the required parameter of sorter model, and the implication of its expression is that kernel function is the intersection kernel function, and the SVM type is C-svc; The C-svc penalty coefficient is 1, and needs probability estimate;

2. export test sample book and concentrate the result of traditional Chinese Painting image to be identified;

Utilize step 3)～6) handle the concentrated traditional Chinese Painting image to be identified of test sample book, obtain the characteristic of correspondence vector, and, can obtain the classification results of image according to the formula of sorter model the sorter model that its input trains;

3. the recognition result evaluation method comprises recognition accuracy and two kinds of methods of confusion matrix.

Images