CN112115780A - Semi-supervised pedestrian re-identification method based on deep multi-model cooperation - Google Patents

Abstract

The invention discloses a semi-supervised pedestrian re-identification method based on deep multi-model collaboration, comprising the steps of: 1) fine-tuning a plurality of pre-trained deep neural networks on ImageNet by using some labeled training data samples, and using these models as initial 2) Use these initial models to extract features for unlabeled training samples, and then propose an adaptive weighted multi-view clustering method to generate pseudo-labels for unlabeled training samples, labeled training samples and pseudo-labeled training samples The samples are combined into the overall training data, and these training data are used to fine-tune and train multiple deep neural networks with heterogeneous structures; 3) Alternate iterative pseudo-label generation and deep network training until the pseudo-labels do not change.

Description

A semi-supervised person re-identification method based on deep multi-model collaboration

technical field

The invention belongs to the field of image feature representation and semi-supervised learning, in particular to a semi-supervised pedestrian re-identification method based on deep multi-model collaboration.

Background technique

With the continuous development of the social economy and the continuous progress of computer vision technology, the construction of intelligent security and smart city has been continuously promoted, and the intelligent processing of video data of different scenes has become a problem of widespread concern in the field of computer vision. Pedestrian re-identification is a key technology for realizing smart security and smart city strategies. Given one or more images of pedestrians in a scene, pedestrian re-recognition technology requires the ability to find images that are related to the image in other non-adjacent scenes. Match all images of pedestrians. In different scenes, different lighting conditions, changes in pedestrian poses, changes in image background, and differences in imaging quality usually lead to changes within the same line of people being greater than changes between different people, which brings about pedestrian re-identification tasks. a serious challenge.

In recent years, benefiting from the powerful performance of deep convolutional neural networks, person re-identification techniques have achieved excellent recognition accuracy on large-scale datasets. But most methods are proposed on the basis of fully supervised learning. The value of supervised learning in real environments and industrial scenarios is inherently limited due to the need for large amounts of manually labeled training data. With the development of smart security, the need for smart cities is becoming more and more urgent, and the implementation and application of existing methods in practical scenarios has become a widespread concern. However, it is very difficult to label a large amount of data in all application environments (such as large shopping malls, urban community streets) in real situations. It is conceivable that the labelers can watch a set of cameras with different parameters to take pictures at different times and at different places. The video records and searches how difficult it is to locate the same pedestrian. Therefore, the first problem that pedestrian re-identification needs to face in the actual scene is the serious lack of labeled data. To overcome the heavy reliance of fully supervised learning methods on large-scale data, some semi-supervised or unsupervised learning methods have emerged. The semi-supervised person re-identification method combines a small amount of labeled data and a large amount of unlabeled data to maximize the use of supervised information and fully mine unsupervised information to achieve the goal of pedestrian retrieval.

SUMMARY OF THE INVENTION

The purpose of the present invention is to provide a semi-supervised pedestrian re-identification method based on deep multi-model collaboration in view of the above-mentioned shortcomings of the prior art.

The present invention adopts following technical scheme to realize:

A semi-supervised person re-identification method based on deep multi-model collaboration, including the following steps:

1) Use some labeled training data samples to fine-tune multiple pre-trained deep neural networks on ImageNet, and use these models as initial models;

2) Use these initial models to extract features for unlabeled training samples, and then propose an adaptive weighted multi-view clustering method to generate pseudo-labels for unlabeled training samples, a combination of labeled training samples and pseudo-labeled training samples The whole training data is used to fine-tune the training of multiple deep neural networks with heterogeneous structures;

3) Alternate iterative pseudo-label generation and deep network training until the pseudo-label is no longer changing.

A further improvement of the present invention is that the concrete realization method of step 1) is as follows:

101) First train multiple neural networks with different structures as feature extractors under multiple views;

102) Clustering the features of multiple heterogeneous neural networks using the proposed adaptive weight multi-view clustering method to obtain pseudo-labels of unlabeled data;

103) Use labeled data and unlabeled data with pseudo-labels to fine-tune multiple heterogeneous neural networks, and the updates of multiple deep networks and pseudo-labels are alternately performed.

The further improvement of the present invention is, the concrete realization method of step 2) is as follows:

201) using part of the labeled data to train multiple heterogeneous neural networks as initial parameters;

202) subsequently using the multiple deep neural networks trained in the first step to extract features from the unlabeled data, and then utilizing the adaptive weight multi-view clustering method to cluster the features of the unlabeled data to obtain the initial pseudo-label of the unlabeled data;

203) Integrate the unlabeled data with pseudo-labels obtained in the second step and the labeled data to train multiple deep neural networks again. The deep neural network training and clustering training are alternately performed until the pseudo-labels do not change, and the final result is obtained. pseudo-label.

The present invention at least has the following beneficial technical effects:

1. In the process of training a deep neural network, the present invention only needs to use a part of the accurately labeled data, and then helps network training by pseudo-labeling a large amount of unlabeled data.

2. The present invention uses multiple deep neural networks to extract features from unlabeled data, exerts the diversity of multiple deep network features, and uses the proposed adaptive weight multi-view clustering method to cluster heterogeneous network features, thereby obtaining Pseudo-labels with better accuracy.

Description of drawings

Figure 1 is a flow chart of the framework of the present invention.

Detailed ways

The present invention will be further described below with reference to the accompanying drawings and embodiments.

As shown in Figure 1, assuming that there are M types of features, the algorithm for υ = 1, 2, ..., M., can be written in the following form:

为深度神经网络的损失函数，

代表多视图估计伪标签损失函数。where x _l and x _u represent labeled and unlabeled training samples, respectively. N _l and N _u represent the number of labeled and unlabeled training samples, respectively. w ^υ represents the parameters in the _υth deep neural network, and yl and _yu represent labeled labels and pseudo-labels of unlabeled data, respectively.

is the loss function of the deep neural network,

Represents the multi-view estimation pseudo-label loss function.

The loss function of the deep neural network is mainly based on two different tasks, which are the recognition task and the verification task. The loss function can be written in the following form:

可以表示为：For the basic feature discriminative learning, the recognition task is regarded as a multi-classification task.

It can be expressed as:

为预测概率，p为目标概率。in

is the predicted probability, and p is the target probability.

可以表示为：For the verification part, the contrastive loss function is not used, and the contrastive loss function forces the same categories to be as close as possible. This may make deep neural networks prone to overfitting when the training dataset has few samples of each class. The validation loss in this paper is a binary logistic regression loss function, which defines the image feature pair as (φ(x _a , w), φ(x _b , w)).

It can be expressed as:

是预测概率。假如图像特征对预测结果为同一个人，那么q₁＝1，q₂＝0，反之q₁＝0，q₂＝1。in

is the predicted probability. If the prediction result of the image feature pair is the same person, then q ₁ =1, q ₂ =0, otherwise q ₁ =0, q ₂ =1.

The multi-view pseudo-label estimation part uses multiple heterogeneous deep neural network feature clustering of unlabeled data to obtain pseudo-labels of unlabeled data. One of the most straightforward methods is to concatenate features from multiple views of unlabeled data into a single feature and then perform standard clustering algorithms. However, in this case, the features under the important view and the less important view are treated equally, resulting in suboptimal clustering algorithm results. The ideal situation is to cluster the features from different views at the same time and combine the results from each view to get the final result. To achieve this goal, the multi-view pseudo-label estimation loss function of the present invention can be written in the following form:

代表无标签数据组合成的矩阵，矩阵的每一列为一个无标签数据。

代表第υ个视图的深度卷积网络特征。

为第υ个视图下的中心点矩阵。

满足1-of-K_u的形式。K_u为期望聚类的数目。α^υ为第υ个视图下的权重因子。in

Represents a matrix composed of unlabeled data, and each column of the matrix is an unlabeled data.

Represents the deep convolutional network features of the vth view.

is the center point matrix under the vth view.

Satisfy the form of 1-of-K _u . _Ku is the number of expected clusters. α ^υ is the weighting factor under the υth view.

The optimization process of the inventive method is as follows:

The present invention adopts the alternate optimization iterative algorithm to optimize the proposed model, and the optimization steps are as follows:

Initialization: Initialize w ^υ to train multiple deep convolutional networks of different architectures with a small fraction of labeled data. B is initialized by Kmeans clustering of a single view, with a weighting factor ^αν = 1/M.

Update B: Update B by minimizing the following subproblems:

To optimize equation (6), write it as:

in

H ^υ =Tr{(Φ ^υ -C ^υ BT)D ^υ ((Φ ^υ -C ^υ BT) ^T }, (8)

where e ^(υ)i is the ith row of the following matrix:

E ^υ =(Φ ^υ ) ^T -B(C ^υ ) ^T . (10)

1) Fix parameters B, D ^υ , α ^υ , update the cluster center C ^υ under each view, and calculate the reciprocal of C ^υ for J, we can get:

in

Let formula (11) be zero, C ^υ can be expressed as follows:

2) Fix the parameters C ^υ , D ^υ , α ^υ , update the cluster center matrix B:

最小化如下的问题：To optimize equation (14), fix i, the vector

Minimize the following problems:

是对角矩阵

的第i个元素，b满足1-of-K_u的形式，对于式(15)有K_u个后选值，每一个为矩阵的第k_u列，具体来说，做一个详尽的搜索找出式(15)的最优解：in

is a diagonal matrix

The i-th element of , b satisfies the form of 1-of-K _u . For formula (15), there are K _u post-selection values, each of which is the k _- th column of the matrix. Specifically, do an exhaustive search to find The optimal solution of formula (15) is:

where _ku is:

3) Fix parameters C ^υ , B, α ^υ , and update D ^υ according to equations (9) and (10).

4) Fix parameters C ^υ , B, D ^υ , update α ^υ .

To make equation (18) achieve a local minimum, α ^υ is expressed as follows:

Alternately iterate C ^υ , B, D ^υ , α ^υ , and repeat the above process until equation (6) converges.

Update _yu : This step is used to update the pseudo-labels of unlabeled data. When B is obtained, the pseudo _- label ku of each unlabeled sample is obtained. The total number of categories of unlabeled samples is K _u , and the total number of categories of labeled samples is K _l . y _u has the following representation:

y _u =k _u +K _l . (20)

Update w ^υ : Train a deep convolutional neural network with the labeled data as well as the residual data with pseudo-labels. Update w ^υ by minimizing:

For equation (21), stochastic gradient descent is used to optimize w ^υ .

The present invention proposes a semi-supervised pedestrian re-identification algorithm based on deep multi-model collaboration, which can realize feature learning of labeled data and pseudo-label estimation of unlabeled data in an end-to-end learning process. In order to improve the accuracy of pseudo-label estimation, the present invention proposes a learning strategy of weak model coordination, which can label more quality data to improve the performance of feature learning.

Claims (3)

1. A semi-supervised pedestrian re-identification method based on deep multi-model cooperation is characterized by comprising the following steps:

1) fine-tuning a plurality of pre-trained deep neural networks on ImageNet by adopting part of labeled training data samples, and taking the models as initial models;

2) extracting features of the training samples without labels by using the initial models, then providing a self-adaptive weight multi-view clustering method to generate pseudo labels for the training samples without labels, combining the training samples with labels and the training samples with the pseudo labels into integral training data, and finely tuning and training a plurality of deep neural networks with different structures by using the training data;

3) alternating iterations of pseudo-label generation and deep network training until the pseudo-label is not changing.

2. The semi-supervised pedestrian re-identification method based on deep multi-model cooperation according to claim 1, wherein the specific implementation method of the step 1) is as follows:

101) firstly, training a plurality of neural networks with different structures as feature extractors under a plurality of views;

102) clustering the characteristics of a plurality of heterogeneous neural networks by using the proposed self-adaptive weight multi-view clustering method to obtain pseudo labels of label-free data;

103) and finely adjusting the heterogeneous neural networks by using labeled data and label-free data with pseudo labels, wherein the updating of the deep networks and the updating of the pseudo labels are alternately carried out.

3. The semi-supervised pedestrian re-identification method based on deep multi-model cooperation according to claim 2, wherein the specific implementation method of the step 2) is as follows:

201) training a plurality of heterogeneous neural networks by adopting part of data with labels as initial parameters;

202) extracting features of the unlabeled data by adopting a plurality of deep neural networks trained in the first step, and clustering the features of the unlabeled data by using a self-adaptive weight multi-view clustering method to obtain an initial pseudo label of the unlabeled data;

203) and fusing the unmarked data with the pseudo label and the marked data obtained in the second step, and training a plurality of deep neural networks again, wherein the deep neural network training and the clustering training are alternately carried out until the pseudo label is not changed, so as to obtain the final pseudo label.

Images