CN109685076A - A kind of image-recognizing method based on SIFT and sparse coding - Google Patents

Abstract

The present invention provides an image recognition method based on SIFT and sparse coding, comprising: extracting dense SIFT features and pixel block features from each image of an RGB-D object respectively; and using K-SVD algorithm according to the extracted dense SIFT features Calculate the SIFT feature dictionary, and use the matching pursuit MP algorithm to solve the sparse coding of SIFT to obtain the first image feature; use K-SVD to learn the dictionary of pixel blocks, and use the OMP algorithm to obtain the sparse coding of pixels, and obtain the unit through the maximum pooling algorithm. feature, connect multiple unit features into block features, calculate sparse coding based on block features, and link block features and corresponding sparse coding to obtain second image features; use pyramid pooling algorithm to link all features to obtain fused image features Perform image recognition. By adopting the image recognition method of the present invention, the image recognition accuracy rate can be improved.

Description

An Image Recognition Method Based on SIFT and Sparse Coding

technical field

The invention relates to the technical field of image processing, in particular to an image recognition method based on SIFT and sparse coding.

Background technique

Object recognition based on RGB-D information is an important topic in the field of computer vision and machine vision, and has related applications such as face recognition, gesture recognition, text recognition and vehicle recognition.

Among the current object recognition algorithms based on traditional algorithms, many only use SIFT to extract object features and combine sparse coding in the initial stage, and the recognition effect is not good. For example, the existing Chinese patent application No. 201510567889.6 - Image Fast Feature Representation Method Based on Normalized Non-negative Sparse Encoder, and the Chinese Patent Application No. 201510874639.7 - Image Based on Multi-directional Context Information and Sparse Coding Model Classification methods, which can effectively extract object gradient information but ignore features such as color and shape. There are also some algorithms that directly extract object features from the image using sparse coding, such as the HMP algorithm. This algorithm can effectively extract the color and shape information of the object in the image, but this algorithm ignores the gradient information of the object and is accurate in feature extraction and recognition. The rate is not high, and it cannot well identify common objects in smart home life.

SUMMARY OF THE INVENTION

The technical problem to be solved by the present invention is to provide an image recognition method based on SIFT and sparse coding to improve the image recognition accuracy.

The present invention is realized as follows: an image recognition method based on SIFT and sparse coding, comprising the following steps:

Step 10. Extract dense SIFT features and pixel block features from each image of the RGB-D object;

Step 20, according to the extracted dense SIFT features, adopt the K-SVD algorithm to calculate the SIFT feature dictionary, and adopt the matching pursuit MP algorithm to solve the sparse coding of the SIFT to obtain the first image feature;

Step 30, using K-SVD to learn the dictionary of pixel blocks, and using the OMP algorithm to obtain the sparse coding of the pixels, obtaining the unit features through the maximum pooling algorithm, connecting the plurality of unit features as block features, and calculating the sparse coding based on the block features, Linking the block feature and the corresponding sparse coding to obtain the second image feature;

Step 40, adopt the pyramid pooling algorithm to be applied to the first image feature and the second image feature respectively, link all the features, and obtain the fused image feature for image recognition;

The above steps 20 and 30 are performed in no particular order.

Further, the method of extracting the dense SIFT feature in the step 10 is: extracting an image block every four pixels on the image, and extracting a SIFT feature from each image block;

The matrix of the dense SIFT features is expressed as: Y={y ₁ , y ₂ , ..., y _u }, where y _i is the i-th SIFT feature, and u is the number of image blocks.

Further, the solution process of the sparse coding of SIFT in the step 20 is as follows:

Step 21. Collect a plurality of SIFT features on each image, use Y _s ={y _s1 ,y _s2 ,y _s3 ,...,y _sp } to represent the sample matrix, and use D _s ={d _s1 ,d _s2 ,... ,d _sm }∈R ^h×m represents the learned SIFT feature dictionary, where d _si is the learned word or basis, X _s ={x _s1 ,x _s2 ,…x _sp }∈R ^p×m is Sparse coding, where x _s represents the sparse coding of y _s ;

Step 22. Calculate the dictionary by the following formula:

Among them, ||.|| _F represents the F norm, ||.|| ₀ represents the number of non-zero elements in the vector, o is a non-zero number, representing the upper limit of the sparsity level;

Step 23, using the K-Means algorithm to generate the initial SIFT feature dictionary D in the calculation process of the SIFT feature dictionary;

Step 24: Decompose the dictionary solution problem into a plurality of sub-problems for step-by-step solution, specifically using the following formula:

The SVD decomposition algorithm is used to update the optimization dictionary step by step. The update optimization process for the i-th word is as follows:

The value range represented by i is (1, 1024) for the SIFT feature. For j and i, they have the same meaning and are both words in the dictionary. _xi represents the sparse coding of the relevant feature corresponding to the i-th word. the value of the dimension;

in, represents the row of X _s , E _i represents the residual matrix, the optimized word d _i and the optimized coefficients It is obtained by applying the SVD algorithm to the residual matrix E _i , using the rows involved in the basis when updating, and repeating this process until convergence or reaching the preset number of iterations;

Step 25 , using the finally updated SIFT feature dictionary to solve the sparse coding of the SIFT by using the matching pursuit algorithm to obtain the first image feature.

Further, the step 30 further includes:

Step 31, using K-SVD to learn the dictionary of pixel blocks;

Step 32, adopt the OMP algorithm to obtain the sparse coding of the pixel;

Step 33, using the maximum pooling algorithm to obtain unit features;

Step 34, connect a plurality of unit features as block features, and the block features include color block features, depth block features and surface normal vector block features;

Step 35, calculate the sparse coding based on the block feature;

Step 36: Link the block feature and the corresponding sparse coding to obtain a second image feature, where the second image feature P _v is expressed as: P _v ={p _v1 ,p _v2 ,...,p _v3 }, where v Represents a color, depth, or normal vector.

Further, the step 40 is specifically as follows: the pyramid pooling adopts three layers of 3×3, 2×2, and 1×1, which are divided into 14 sub-areas, and the maximum pooling algorithm is used to obtain the sub-areas on each sub-area. feature HP _vb , b=1, 2, . . . , 14;

Let Θ _v ={HP _v1 ,HP _v2 ,...,HP _v14 }, where v represents the color, depth and surface normal vector, then the fused object image features are expressed as:

ψ = {Θ _rgb , Θ _depth , Θ _normal , φ};

Then divide ψ by Perform normalization to obtain the final fused image features, where ε=0.001.

The present invention has the following advantages:

1. The algorithm framework is designed for the combination of improved SIFT sparse coding and HMP algorithm to improve the accuracy of object recognition;

2. The algorithm designed by the present invention and HMP or SIFT-based sparse coding feature can extract rich features, and use RGB-D information to add gradient information to HMP to obtain a representation with more abundant features;

3. A multi-channel feature fusion scheme is proposed to adapt to SIFT-based sparse coding by modifying the HMP algorithm.

Description of drawings

The present invention will be further described below with reference to the accompanying drawings and embodiments.

FIG. 1 is an execution flow chart of an image recognition method based on SIFT and sparse coding according to the present invention.

Detailed ways

As shown in Figure 1, an image recognition method based on SIFT and sparse coding of the present invention includes the following steps:

Step 10. Extract dense SIFT features and pixel block features from each image of the RGB-D object;

Step 20, according to the extracted dense SIFT features, adopt the K-SVD algorithm to calculate the SIFT feature dictionary, and adopt the matching pursuit MP algorithm to solve the sparse coding of the SIFT to obtain the first image feature;

Step 30, using K-SVD to learn the dictionary of pixel blocks, and using the OMP algorithm to obtain the sparse coding of the pixels, obtaining the unit features through the maximum pooling algorithm, connecting the plurality of unit features as block features, and calculating the sparse coding based on the block features, Linking the block feature and the corresponding sparse coding to obtain the second image feature;

Step 40, adopt the pyramid pooling algorithm to be applied to the first image feature and the second image feature respectively, link all the features, and obtain the fused image feature for image recognition;

The above steps 20 and 30 are performed in no particular order.

Preferably, the extraction method of the dense SIFT feature in the step 10 is: extracting an image block every four pixels on the image, and extracting a SIFT feature from each image block;

The matrix of the dense SIFT features is expressed as: Y={y ₁ , y ₂ , ..., y _u }, where y _i is the i-th SIFT feature, and u is the number of image blocks.

Preferably, the solution process of the sparse coding of SIFT in the step 20 is as follows:

Step 21. Collect a plurality of SIFT features on each image, use Y _s ={y _s1 ,y _s2 ,y _s3 ,...,y _sp } to represent the sample matrix, and use D _s ={d _s1 ,d _s2 ,... ,d _sm }∈R ^h×m represents the learned SIFT feature dictionary, where d _si is the learned word or basis, X _s ={x _s1 ,x _s2 ,…x _sp }∈R ^p×m is Sparse coding, where x _s represents the sparse coding of y _s , where p and m of R ^p×m represent the number of features and the corresponding dimension of sparse coding, respectively, and h and m of R ^h×m represent the number of dictionaries and the dictionary, respectively dimension, H takes 1024.

Step 22. Calculate the dictionary by the following formula:

Among them, ||.|| _F represents the F norm, ||.|| ₀ represents the number of non-zero elements in the vector, o is a non-zero number, representing the upper limit of the sparsity level;

Step 23, using the K-Means algorithm to generate the initial SIFT feature dictionary D in the calculation process of the SIFT feature dictionary;

Step 24: Decompose the dictionary solution problem into a plurality of sub-problems for step-by-step solution, specifically using the following formula:

The SVD decomposition algorithm is used to update the optimization dictionary step by step. The update optimization process for the i-th word is as follows:

The value range represented by i is (1, 1024) for the SIFT feature. For j and i, they have the same meaning and are both words in the dictionary. _xi represents the sparse encoding of the relevant feature corresponding to the i-th word. the value of the dimension;

in, represents the row of X _s , E _i represents the residual matrix, the optimized word d _i and the optimized coefficients is obtained by applying the SVD algorithm to the residual matrix E _i , using the rows involved in the basis when updating, and repeating this process until convergence or reaching the preset number of iterations;

Step 25 , using the finally updated SIFT feature dictionary to solve the sparse coding of the SIFT by using the matching pursuit algorithm to obtain the first image feature.

Preferably, the step 30 further includes:

Step 31, using K-SVD to learn the dictionary of pixel blocks;

Step 32, adopt the OMP algorithm to obtain the sparse coding of the pixel;

Step 33, using the maximum pooling algorithm to obtain unit features;

Step 34, connect a plurality of unit features as block features, and the block features include color block features, depth block features and surface normal vector block features;

Step 35. Calculate sparse coding based on block features

Step 36: Link the block feature and the corresponding sparse coding to obtain a second image feature, where the second image feature P _v is expressed as: P _v ={p _v1 ,p _v2 ,...,p _v3 }, where v represents the color , depth and surface normal vector, where the colors refer to red, green and blue;

Preferably, the step 40 is as follows: the pyramid pooling adopts three layers of 3×3, 2×2, and 1×1, which are divided into 14 sub-regions, and the maximum pooling algorithm is used to obtain the sub-regions on each sub-region. Region feature HP _vb , b=1, 2, . . . , 14;

Let Θ _v ={HP _v1 ,HP _v2 ,...,HP _v14 }, where v represents the color, depth and surface normal vector, then the fused object image features are expressed as:

ψ = {Θ _rgb , Θ _depth , Θ _normal , φ};

where _Θrgb refers to the combination of Θred, Θgreen and Θblue, Θred refers to the red channel image feature, Θgreen refers to the green channel image feature and Θblue refers to the blue channel image feature, and the image feature refers to the The pyramid pooling algorithm is used to link the block features on each channel with the corresponding sparse coding.

Then divide ψ by Perform normalization processing to obtain the final fused image features, where ε=0.001.

Since the SIFT feature can extract the gradient direction histogram of the object, dense sampling with SIFT can preserve the detailed features of the object enhancement. The SIFT algorithm itself has scale and rotation invariance, and also has a certain affine invariance. Relying on this characteristic, a stable image block feature representation can be obtained. The present invention adopts the SIFT feature to perform intensive sampling, uses the K-SVD algorithm to learn the dictionary according to the obtained feature matrix, and then adopts the MP (matchingpursuit) algorithm to obtain the sparse representation of the block when solving the block feature. The purpose is to retain as much as possible The detailed features of the image block, and finally the pyramid pooling algorithm is used to link all the features to obtain the feature representation of the image.

Each image is divided into several 16*16 image block pixels, and each image block pixel is divided into 16 4*4 units. The advantage of using sparse coding for feature learning directly on the pixel block is that the dictionary learned by this pixel block can save the overall characteristics of the pixel block, including the spatial position relationship between pixels, the included geometric features, and the included pixel values. contact, etc. In this way, with the help of a pixel block-based dictionary, the sparse coding of pixels can be obtained. Applying the max pooling algorithm to the sparse coding corresponding to all pixels in the unit can obtain the unit feature (here the unit feature refers to a 4*4 pixel block), and then link the unit features to obtain the block feature. The unit features obtained in this way can retain the most salient features in the unit, and linking the unit features together can retain the spatial location information while retaining the salient information. For block features, feature learning is continued to obtain block-based dictionaries and sparse coding, which is to extract more abstract features. Finally, the pyramid pooling algorithm is applied to SIFT-based sparse coding, block-based feature and block-based sparse coding, respectively, and linking them together can get the final feature representation. Among them, SIFT extracts grayscale information, and directly uses sparse coding to extract color, depth and normal vector information.

The present invention learns SIFT feature and block feature from RGB-D information, and block feature can extract color, shape feature, spatial geometric feature and directional feature from RGB-D information. Dense SIFT features are able to capture gradient features. Then use the K-SVD algorithm to calculate the SIFT feature dictionary. In this way, more gradient information can be obtained. Match pursuit (MP) is then used to compute SIFT and sparse coding of block features. The block features and the corresponding sparse coding are concatenated, and on the basis of SIFT sparse coding, a simple three-layer segmentation 3 × 3, 2 × 2, 1 × 1 spatial pyramid pooling algorithm is used, and the block features and related sparse coding are used. The encoded features generate final image features, thereby improving image recognition accuracy.

In one embodiment, the present invention compares the sparse representation of the multi-feature combination with some classical object recognition algorithms on the Washington RGB-D object dataset, which contains 51 categories and 300 instances, each category contains multiple Example, the example is the experiment taken from different angles of 30°, 45°, 60°, 41788 RGB-D images are used in the experiment, the data design view, rotation, scaling, texture and texture less lighting problems, for objects Recognition, through training and testing processes, is performed on datasets using comparative methods, and there are two types of object recognition tasks that need to be performed: classification recognition and instance recognition. Invisible objects are identified by name in class recognition. One instance was randomly stripped from each class for testing, and the remaining 300-51=249 objects were used for training in each trace. Average accuracy on 10 random train/test splits. For example recognition, images obtained from 30° and 60° angles were used for training, and images from 45° angles were used for testing.

The comparison results of classification and recognition experiments are shown in Table 1:

Table 1

RGB RGB-D Kernel descriptor 80.7±2.1 86.5±2.1 HMP 74.7±2.5 82.1±3.3 Improved HMP 82.4±3.1 87.5±2.9 VGG 88.9±2.1 91.8±2.4 MJSR 91.5±1.5 92.8±1.3

The comparison results of instance recognition experiments are shown in Table 2:

Table 2

RGB RGB-D Kernel descriptor 90.8 91.2 HMP 75.8 78.9 Improved HMP 92.1 92.8 MJSR 92.6 92.2

Through experimental comparison, it can be seen that the object recognition method of the present invention can obtain competitive results with the Upgraded HMP by using color or combining color and depth information. Overall, it exceeds the original algorithm of HMP by more than 13 percentage points. Whether it is combining color information or combining color and depth information, it is also competitive with the kernel function. Depth is 1 percentage point higher.

Although the specific embodiments of the present invention have been described above, those skilled in the art should understand that the specific embodiments we describe are only illustrative, rather than used to limit the scope of the present invention. Equivalent modifications and changes made by a skilled person in accordance with the spirit of the present invention should be included within the scope of protection of the claims of the present invention.

Claims (5)

1. An image identification method based on SIFT and sparse coding is characterized in that: the method comprises the following steps:

step 10, extracting dense SIFT features and pixel block features from each image of the RGB-D object respectively;

step 20, calculating an SIFT feature dictionary by adopting a K-SVD algorithm according to the extracted dense SIFT features, and solving sparse coding of SIFT by adopting a matching pursuit MP algorithm to obtain first image features;

step 30, learning a dictionary of the pixel block by adopting K-SVD, obtaining sparse codes of the pixels by utilizing an OMP algorithm, obtaining unit features by utilizing a maximum pooling algorithm, connecting a plurality of unit features into block features, calculating sparse codes based on the block features, and linking the block features and the corresponding sparse codes to obtain second image features;

step 40, respectively applying a pyramid pooling algorithm to the first image feature and the second image feature, linking all the features, obtaining fused image features, and performing image recognition;

the step 20 and the step 30 are not performed successively.

2. The image recognition method based on SIFT and sparse coding as claimed in claim 1, wherein: the extraction method of the dense SIFT features in the step 10 is as follows: extracting an image block every four pixels on the image, wherein each image block extracts an SIFT feature;

the matrix of dense SIFT features is represented as: y ═ Y₁,y₂,…,y_uIn which y_iIs the ith SIFT feature and u is the image block number.

3. The image recognition method based on SIFT and sparse coding as claimed in claim 1, wherein: the solving process of the sparse coding of SIFT in the step 20 is as follows:

step 21, collecting a plurality of SIFT features on each picture, and using Y_s＝{y_s1,y_s2,y_s3,…,y_spDenotes the sample matrix, with D_s＝{d_s1,d_s2,…,d_sm}∈R^h×mRepresents a dictionary of learned SIFT features, wherein d_siFor learning the word or radical, X_s＝{x_s1,x_s2,…x_sp}∈R^p×mIs sparse coding, where x_sRepresents y_sSparse coding of (2);

step 22, calculating the dictionary by the following formula:

wherein | |_FRepresents F norm, | |. | luminance₀The number of non-zero elements in the vector is represented, and o is a non-zero number and represents the upper limit of the sparse level;

step 23, generating an initial SIFT feature dictionary D by adopting a K-Means algorithm in the calculation process of the SIFT feature dictionary;

step 24, decomposing the dictionary solving problem into a plurality of subproblems for stepwise solving, and specifically adopting the following formula:

and (3) updating the optimized dictionary step by adopting an SVD (singular value decomposition) algorithm, wherein the updating optimization process for the ith word is as follows:

wherein,represents X_sLine of (E), E_iRepresenting residual matrices, optimized word d_iAnd optimized coefficientsIs to apply SVD algorithm to residual matrix E_iIf so, the process is repeated by using the rows related in the base during updating until convergence or preset iteration times are reached;

and 25, solving the sparse coding of SIFT by using the finally updated SIFT feature dictionary and adopting a matching pursuit algorithm to obtain first image features.

4. The image recognition method based on SIFT and sparse coding as claimed in claim 1, wherein: said step 30 further comprises:

step 31, learning a dictionary of pixel blocks by adopting K-SVD;

step 32, obtaining sparse coding of the pixels by adopting an OMP algorithm;

step 33, obtaining unit characteristics by adopting a maximum pooling algorithm;

step 34, connecting the plurality of unit features into block features, wherein the block features comprise color block features, depth block features and surface normal vector block features;

step 35, calculating sparse coding based on block characteristics;

step 36, linking the block features and the corresponding sparse codes to obtain second image features, wherein the second image features P_vExpressed as: p_v＝{p_v1,p_v2,...,p_v3Where v represents a color, depth, or normal vector.

5. The SIFT and sparse coding based image recognition method according to claim 4, wherein: the step 40 is specifically as follows: the gold subzone pooling adopts three layers of 3 × 3, 2 × 2 and 1 × 1, which are totally divided into 14 sub-zones, and the maximum pooling algorithm is used for obtaining the sub-zone characteristics HP on each sub-zone_vb，b＝1，2，…，14；

Let Θ be_v＝{HP_v1,HP_v2,...,HP_v14And v represents color, depth and surface normal vectors, the fused object image features are represented as:

ψ＝{Θ_rgb,Θ_depth,Θ_normal,φ}；

after which psi is divided byAnd (5) carrying out normalization processing to obtain the finally fused image characteristics, wherein epsilon is 0.001.