CN111783753B - Person Re-identification Method Based on Semantically Consistent Horizontal Bars and Foreground Modification - Google Patents

Abstract

The invention belongs to the field of computer vision and pattern recognition, in particular to a pedestrian re-identification method based on semantically consistent horizontal bars and foreground correction, and aims to solve the problem of poor robustness of existing pedestrian re-identification methods. The method of the present invention includes: acquiring an image to be recognized as an input image; extracting features of the input image as a first feature; and based on the first feature, respectively acquiring foregrounds corresponding to pedestrians in the input image through a row classifier in a pedestrian re-identification model. The feature is used as the second feature, and the feature of the horizontal strip area of each set part of the pedestrian in the input image is obtained as the third feature; the second feature and the third feature are multiplied point-to-point, and spliced with the first feature to obtain the fourth feature ; Calculate the Euclidean distance between the fourth feature and the corresponding features of each image in the image library and sort, and output the sorting result as the re-identification result. The present invention improves the robustness of pedestrian re-identification.

Description

Person Re-identification Method Based on Semantically Consistent Horizontal Bars and Foreground Modification

technical field

The invention belongs to the field of computer vision and pattern recognition, and in particular relates to a pedestrian re-identification method, system and device based on semantically consistent horizontal bars and foreground correction.

Background technique

Person re-identification is a sub-problem in the field of image retrieval. Given a pedestrian image, the pedestrian re-identification task aims to find this pedestrian image in other scenes. However, due to the transformation of perspective, the difference in posture and the occlusion of objects, the parts of the human body may appear anywhere in the picture. Therefore, it is very important to learn a method that can effectively locate each part of the human body and extract the part features with sufficient discriminative power.

There are roughly four categories of existing part-alignment-based person re-identification: horizontal bar-based methods, bounding box-based methods, attention-based methods, and methods based on additional semantic information. Among them, horizontal bar-based methods are particularly popular due to their convenience and relatively high performance. Among them, the more popular ones are PCB, MGN, Pyramid, etc. PCB (YifanSun, Liang Zheng, Yi Yang, Qi Tian, Shengjin Wang.Beyond Part Models: PersonRetrieval with Refined Part Pooling (and A Strong Convolutional Baseline).ECCV, 2018) first proposed to divide pedestrian images into horizontal strips of equal height, Then average pooling is performed on each horizontal bar separately to obtain features, and the loss is calculated separately. MGN (Guanshuo Wang, Yufeng Yuan, Xiong Chen, Jiwei Li. Learning discriminative features with multiple granularities for person re-identification. ACM MM, 2018) and Pyramid (Zheng F , Deng C , Sun X , et al. Pyramidal Person Re-IDentification. via Multi-Loss Dynamic Training.CVPR, 2019.) On the basis of PCB, multi-granularity and overlapping horizontal bars are designed, which greatly improves the robustness of the algorithm. However, none of the above methods solve the following two problems: (1) The height and position of the horizontal bar are fixed. Due to differences in poses, perspectives, and occlusions, there is no guarantee that the semantics within each horizontal bar are consistent. But the above method uses a fixed horizontal bar and does not try to solve this problem. (2) Interference of background noise. Inside each horizontal bar, there will inevitably be interference of background information. How to remove the background noise inside the horizontal bar has no solution at present. Based on this, the present invention proposes a pedestrian re-identification method based on semantically consistent horizontal bars and foreground correction.

SUMMARY OF THE INVENTION

In order to solve the above problems in the prior art, that is, in order to solve the problem that the existing pedestrian re-identification method has a fixed height and position of the horizontal bar, and the background noise is not eliminated, the re-identification robustness is poor. A person re-identification method based on semantically consistent horizontal bars and foreground correction, which includes:

Step S10, acquiring the image to be recognized as an input image;

Step S20, extracting the feature of the input image through the feature extraction layer of the pedestrian re-identification model as the first feature;

Step S30, based on the first feature, obtain the foreground feature corresponding to the pedestrian in the input image as the second feature through the pre-trained line classifier in the pedestrian re-identification model, and obtain each set part of the pedestrian in the input image. The feature of the horizontal bar area is used as the third feature;

Step S40, performing point-to-point multiplication of the second feature and the third feature, and splicing with the first feature to obtain a fourth feature;

Step S50, calculating the Euclidean distance between the fourth feature and the corresponding feature of each image in the image library and sorting, and outputting the sorting result as a re-identification result;

Wherein, the pedestrian re-identification model is constructed based on a deep convolutional neural network; the row classifier is constructed based on a fully connected layer and a softmax layer.

In some preferred embodiments, the training method of the row classifier is:

Step A10, obtaining a training sample image set;

Step A20, for any image in the training sample image set, extract its row feature and pool it to obtain its corresponding average feature;

Step A30, judge whether the current number of iterations M is a multiple of N, if yes, then execute step A40, otherwise jump to step A50; wherein, N and M are natural numbers;

Step A40, extracting row features of each training sample image in the training sample set, obtaining pseudo-labels corresponding to each set part through self-similar clustering, and executing Step A50;

Step A50: Calculate the loss of the local features obtained in Step A20 and the pseudo-label, and update the parameters of the row classifier.

In some preferred embodiments, the self-similar clustering is a k-means clustering method.

In some preferred embodiments, "respectively obtain the foreground features corresponding to pedestrians in the input image as second features through the pre-trained line classifier in the pedestrian re-identification model", the method is:

Obtain the confidence of each pixel in the input image on the human foreground semantics through the row classifier;

The pixels whose confidence is greater than the first set threshold are used as foreground pixels, and the pixels whose straightness is less than the second set threshold are used as background pixels;

The feature constructed based on the extracted foreground pixels is used as the foreground feature corresponding to the pedestrian in the input image.

In some preferred embodiments, "acquiring the features of the horizontal strip area of each set part of the pedestrian in the input image through the pre-trained line classifier in the pedestrian re-identification model as the third feature", the method is:

Semantic segmentation is performed on the input image by a line classifier, and a confidence map of the horizontal bar area of each set part of the pedestrian in the input image is obtained;

Perform a point-to-point product operation on each confidence map and the first feature to obtain the feature of the horizontal strip area of each set part of the pedestrian in the input image.

In some preferred embodiments, the loss function of the pedestrian re-identification model during training is:

表示行人重识别模型的损失值，

表示行人重识别模型在训练时 一批次的训练样本图像的数量，

表示批次，

表示行人重识别模型在训练时一批次 的训练样本图像中的任一图像，

表示图像集A中图像特征与的

的特征欧式距离最大的一 张训练样本图像，

表示图像集B中图像特征与

的特征的欧式距离最小的一张训练样本图 像，

表示预设的距离间隔，

表示包含与

相同ID的所有图像的图像集，

表示当前批次 中除了

中包含的图像外所有图像构建的图像集，

表示欧氏距离。 in,

represents the loss value of the person re-identification model,

Represents the number of training sample images in a batch of the person re-identification model during training,

represents the batch,

Represents any image in a batch of training sample images of the person re-identification model during training,

Represents the image features in the image set A and the

A training sample image with the largest Euclidean distance of features,

Represents the image features in the image set B and

A training sample image with the smallest Euclidean distance of the features,

represents the preset distance interval,

means including

an image set of all images with the same ID,

Indicates that the current batch except

An image set constructed from all images except those contained in ,

represents the Euclidean distance.

In the second aspect of the present invention, a pedestrian re-identification system based on semantically consistent horizontal bars and foreground correction is proposed. The system includes: an image acquisition module, a global feature extraction module, a local feature extraction module, a feature splicing module, and a recognition output module. ;

The image acquisition module is configured to acquire an image to be recognized as an input image;

The global feature extraction module is configured to extract the feature of the input image through the feature extraction layer of the pedestrian re-identification model as the first feature;

The local feature extraction module is configured to, based on the first feature, obtain foreground features corresponding to pedestrians in the input image as second features through a pre-trained row classifier in the pedestrian re-identification model, and obtain the input image The characteristics of the horizontal bar area of each set part of the pedestrian are taken as the third characteristic;

The feature splicing module is configured to perform point-to-point multiplication of the second feature and the third feature, and splicing with the first feature to obtain a fourth feature;

The recognition output module is configured to calculate the Euclidean distance between the third feature and the corresponding feature of each image in the image library and sort it, and output the sorting result as a re-identification result;

Wherein, the pedestrian re-identification model is constructed based on a deep convolutional neural network; the row classifier is constructed based on a fully connected layer and a softmax layer.

In a third aspect of the present invention, a storage device is provided in which a plurality of programs are stored, and the program applications are loaded and executed by a processor to implement the above-mentioned method for pedestrian re-identification based on semantically consistent horizontal bars and foreground correction.

In a fourth aspect of the present invention, a processing device is proposed, including a processor and a storage device; the processor is adapted to execute various programs; the storage device is adapted to store multiple programs; the programs are adapted to be loaded by the processor And execute to achieve the above-mentioned semantically consistent horizontal bar and foreground correction based pedestrian re-identification method.

Beneficial effects of the present invention:

The present invention improves the robustness of pedestrian re-identification. The present invention divides each line into specific semantics through the pre-trained line classifier to form a horizontal bar with consistent semantics, and can adjust the height and position of the horizontal bar adaptively to ensure that the semantics contained in each horizontal bar are Consistent, solves the problem of semantic consistency of horizontal bars.

At the same time, each pixel is also assigned to foreground or background semantics. By taking the intersection of the semantics of the horizontal bar and the foreground area, we can approximate the position of each part of the human body, solve the interference of background information, and improve the accuracy of positioning of each part and the discrimination of local features.

Description of drawings

Other features, objects and advantages of the present application will become more apparent upon reading the detailed description of non-limiting embodiments taken with reference to the following drawings.

1 is a schematic flowchart of a pedestrian re-identification method based on semantically consistent horizontal bars and foreground correction according to an embodiment of the present invention;

2 is a schematic diagram of the framework of a pedestrian re-identification system based on semantically consistent horizontal bars and foreground correction according to an embodiment of the present invention;

3 is a schematic structural diagram of a pedestrian re-identification method based on semantically consistent horizontal bars and foreground correction according to an embodiment of the present invention;

FIG. 4 is a schematic diagram of a comparison effect between the row classifier of the present invention and the existing horizontal bar height and position fixed row classifier according to an embodiment of the present invention.

Detailed ways

In order to make the purposes, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments These are some embodiments of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

The present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the related invention, but not to limit the invention. It should be noted that the embodiments in the present application and the features of the embodiments may be combined with each other in the case of no conflict.

A pedestrian re-identification method based on semantically consistent horizontal bars and foreground correction according to the first embodiment of the present invention, as shown in FIG. 1 , the method includes the following steps:

Step S10, acquiring the image to be recognized as an input image;

Step S20, extracting the feature of the input image through the feature extraction layer of the pedestrian re-identification model as the first feature;

Step S30, based on the first feature, obtain the foreground feature corresponding to the pedestrian in the input image as the second feature through the pre-trained line classifier in the pedestrian re-identification model, and obtain each set part of the pedestrian in the input image. The feature of the horizontal bar area is used as the third feature;

Step S40, performing point-to-point multiplication of the second feature and the third feature, and splicing with the first feature to obtain a fourth feature;

Step S50, calculating the Euclidean distance between the fourth feature and the corresponding feature of each image in the image library and sorting, and outputting the sorting result as a re-identification result;

Wherein, the pedestrian re-identification model is constructed based on a deep convolutional neural network; the row classifier is constructed based on a fully connected layer and a softmax layer.

In order to more clearly describe the pedestrian re-identification method based on semantically consistent horizontal bars and foreground correction of the present invention, each step in an embodiment of the method of the present invention will be described in detail below.

In the following embodiments, the training process of the pedestrian re-identification model will be described in detail first, and then the process of obtaining the pedestrian re-identification result by the pedestrian re-identification method based on the semantically consistent level bar and foreground correction will be described in detail.

1. The training process of the pedestrian re-identification model

Step B10, pre-training the pedestrian re-identification model

In the present invention, the pedestrian re-identification model is constructed based on the deep convolutional neural network, and the deep convolutional neural network is preferably the document "Sun K, Xiao B, Liu D, et al. Deep High-Resolution Representation Learning for Human Pose Estimation in the present invention. [J]. 2019." The proposed HRNet network, HRNet contains multi-scale semantic information, which is more suitable for the common network of human semantic analysis and pedestrian re-identification. The pedestrian re-identification model is shown in Figure 3, where the neural network model in Figure 3 refers to a convolutional neural network, which is used to extract features. The aligned features of each part of the pedestrian represent the sequence of each part of the human body. The features obtained after feature splicing are explained in detail below.

，每 一次迭代输入64张图像，迭代训练6000次，在其他实施例中，可以根据实际需求选取模型预 训练的迭代次数以及每次迭代输入的样本图像数量。 In this embodiment, the ImageNet data set is used to pre-train the pedestrian re-identification model, and the network parameters of the pedestrian re-identification model are initialized. During pre-training, the selected sample image size is compressed to

, 64 images are input in each iteration, and the iterative training is performed 6000 times. In other embodiments, the number of iterations of model pre-training and the number of sample images input in each iteration can be selected according to actual requirements.

Step B20, obtain a training sample image set

In this embodiment, training sample images containing pedestrians are acquired, and a training sample image set is constructed.

Step B30, extracting the features of each training sample image in the training sample image set as global features

In this embodiment, the features of the training sample images are extracted through the feature extraction layer of the person re-identification model as global features, and the feature extraction layer is a feature extraction layer constructed based on convolutional neural networks. Among them, the final output feature (feature map) size of HRNet is 64322048. In the present invention, the feature output by HRNet is reduced to 6432512 by 11 convolution, and then row division and foreground correction are performed.

Step B40 , based on the global features, obtain foreground features corresponding to pedestrians in each training sample image and features of horizontal bar areas of each set part of pedestrians in each training sample image through the pre-trained row classifier in the pedestrian re-identification model.

The invention proposes a semantically consistent horizontal bar and foreground correction classifier, which firstly divides the pedestrian image into semantically consistent horizontal bars, and then removes the background inside the horizontal bar. The network contains semantically consistent horizontal bars and a foreground correction module. The former generates pseudo-labels of horizontal bars by iterative clustering, and then guides the learning of horizontal bar division; the latter uses the learned horizontal bar classifier to obtain the foreground response map, and uses the foreground response map to guide the division of the foreground and background. Finally, efficient pedestrian feature descriptions are obtained by combining global and local features. details as follows:

In this embodiment, the division of semantically consistent horizontal bars, ie row division, mainly includes a row classifier, which is constructed by a fully connected layer and a softmax layer, and can classify each row into different semantics. First, perform the pooling operation on each row of the training sample image (that is, row unit pooling in Figure 3) to obtain the average feature of each row, that is, the row feature, and then use the row classifier to classify the average feature of each row. The category into which the feature is classified represents the semantic part of the line, that is, the line classifier is used to perform semantic segmentation on the body part area of the pedestrian set in the training sample image, and the confidence map of the pedestrian set body part area is obtained. The point-to-point product operation is performed on the global features to form weighted feature maps corresponding to different parts of the body (features of the horizontal bar area (row area) of each set part of the pedestrian).

In the present invention, it is preferable to obtain the horizontal bar features of the five body parts of the pedestrian, which correspond to the features of the head, chest, abdomen, legs and feet respectively, which are represented as M1, M2, M3, M4 and M5. As shown in Figure 4, (a) (b) (c) in Figure 4 are the feature maps obtained by the existing horizontal bar height and position fixed row classifier. It can be found that the features obtained by using the existing human analysis model The graph (that is, the feature map after splicing) cannot use effective information such as the backpack to affect the performance.

Among them, the row classifier uses an iterative clustering method to assign pseudo-labels to each row during training. That is, for every n training stages, we cluster the feature mean of each row of the image (the horizontal bar area of the pedestrian setting part), and then assign semantics according to the position from top to bottom. In the subsequent training process, the assigned semantic pseudo-labels are used to supervise the learning of the row classifier. In this way, the present invention can adaptively divide the different semantic parts in the picture into horizontal stripes to obtain horizontal strips with consistent semantics. The training process of the row classifier is as follows:

Step B41, for any image in the training sample image set, extract its row feature and pool it to obtain its corresponding average feature;

Step B42, judging whether the current number of iterations M is a multiple of N, if so, execute step B43, otherwise jump to step B44; wherein, N and M are natural numbers; the current number of iterations M is also the current pedestrian re-identification training model. the number of iterations;

Step B43, extract the average feature of all the training sample images in the training sample set, and obtain the pseudo-label corresponding to each row through self-similar clustering to update, and perform step B44;

Step B44: Calculate the loss of the average feature obtained in Step B41 and the updated pseudo-label, and update the parameters of the row classifier.

In step B45, steps B41-B44 are executed cyclically until a trained row classifier is obtained.

Self-similar clustering is the k-means clustering algorithm in the present invention.

After the line classifier obtains the local features of each set part of the pedestrian, in order to further remove the background pixels in the horizontal bar of each set part, the noise interference is reduced. The present invention designs a foreground-guided part refinement method, that is, a foreground-background classifier is added to predict whether each pixel of the training sample image belongs to the foreground or the background. In view of the fact that the line classifier has been learned before, the present invention preferably uses the line classifier to distinguish the confidence of each pixel on each set part of the human body. The present invention preferably uses the confidence degree greater than 0.8 as the foreground pixel, and the confidence degree less than 0.2 as the foreground pixel Background pixels, others as neutral pixels (ie neutral in Figure 3). The features constructed based on the extracted foreground pixels are used as the foreground features corresponding to the pedestrians in the training sample images (that is, the foreground/background features are extracted in units of pixels in Figure 3).

Step B50, feature splicing

In this embodiment, first, M1-5 is compressed into five 256-dimensional vectors through global feature pooling, which is denoted as S1-5. Then, M1-5 are added to obtain the foreground feature map, which is compressed into a 256-dimensional vector by the average pooling map, denoted as S6. The global feature is compressed into a 256-dimensional vector by the average pooling map, denoted as S7, this feature vector can well convey the overall abstract feature. Finally, the above three feature vectors are spliced, and a 7×256-dimensional feature is finally obtained to represent the feature after pedestrian fusion.

Among them, S6 can also directly obtain the foreground features of each training sample image through the line classifier. If the foreground features of each training sample image are obtained through the line classifier, S1-5 and S6 are multiplied point-to-point and then spliced with S7 to get Characterize the features after pedestrian fusion.

Step B60: Calculate the Euclidean distance between the spliced feature and the corresponding feature of each image in the image library and sort, and output the sorting result as a re-identification result.

In this embodiment, the Euclidean distances corresponding to the spliced pedestrian features and the images in the image database are calculated and sorted in ascending order. The higher the matching rate of Rank-1 (ranked first) and the higher ranking, indicates that the target is re-identified. The better the task is. The image library is a database that stores multiple pedestrian images.

Based on the re-identified results, the present invention uses triple loss to supervise the training of the entire network. The core idea of this loss is to separate the mismatched pedestrian pairs from the matched pedestrian pairs by distance interval, so as to increase the inter-class difference and reduce the intra-class difference, as shown in formula (1):

（1）

(1)

表示行人重识别模型的损失值，

表示行人重识别模型在训练时 一批次的训练样本图像的数量，

表示批次，

表示行人重识别模型在训练时一批次 的训练样本图像中的任一图像，

表示图像集A中图像特征与的

的特征欧式距离最大的一 张训练样本图像，即最不像的正样本，

表示图像集B中图像特征与

的特征的欧式距离最 小的一张训练样本图像，即最像的负样本，

构成一个三元组，

表示预设的距离 间隔，

表示包含与

相同ID的所有图像的图像集，

表示当前批次中除了

中包含的图像 外所有图像构建的图像集，

表示欧氏距离。 in,

represents the loss value of the person re-identification model,

Represents the number of training sample images in a batch of the person re-identification model during training,

represents the batch,

Represents any image in a batch of training sample images of the person re-identification model during training,

Represents the image features in the image set A and the

A training sample image with the largest Euclidean distance of the features, that is, the most dissimilar positive sample,

Represents the image features in the image set B and

A training sample image with the smallest Euclidean distance of the feature, that is, the most similar negative sample,

form a triple,

represents the preset distance interval,

means including

an image set of all images with the same ID,

Indicates that the current batch except

An image set constructed from all images except those contained in ,

represents the Euclidean distance.

Update the network parameters of the pedestrian re-identification model based on the above loss, and jump to step B20 until the trained pedestrian re-identification model is obtained.

2. Pedestrian re-identification method based on semantically consistent horizontal bars and foreground correction

Step S10, acquiring the image to be recognized as an input image;

In this embodiment, the recognized pedestrian image is obtained first as the input image.

Step S20, extracting the feature of the input image through the feature extraction layer of the pedestrian re-identification model as the first feature;

In this embodiment, the global features of pedestrians in the input image are obtained through the trained pedestrian re-identification model, that is, the features of the input image are extracted based on the feature extraction layer of the pedestrian re-identification model as the first features.

Step S30, based on the first feature, obtain the foreground feature corresponding to the pedestrian in the input image as the second feature through the pre-trained line classifier in the pedestrian re-identification model, and obtain each set part of the pedestrian in the input image. The feature of the horizontal bar area is used as the third feature;

In this embodiment, the line classifier is used to obtain the confidence of each pixel in the input image on the human foreground semantics, the pixels with the confidence greater than the first set threshold are regarded as foreground pixels, and the confidence is less than the second set threshold. The pixel points are used as background pixels, and the feature constructed based on the extracted foreground pixels is used as the foreground feature corresponding to the pedestrian in the input image, as the second feature.

The input image is semantically segmented by the row classifier, and the confidence map of the horizontal bar area of each set part of the pedestrian in the input image is obtained. The feature of the horizontal bar area, as the third feature.

Step S40, performing point-to-point multiplication of the second feature and the third feature, and splicing with the first feature to obtain a fourth feature;

In this embodiment, the acquired characteristics of pedestrians are spliced together.

Step S50: Calculate the Euclidean distance between the fourth feature and the feature corresponding to each image in the image library and sort, and output the sorting result as a re-identification result.

In this embodiment, the Euclidean distance between the spliced fourth feature and the feature corresponding to each pedestrian image in the image library is calculated, and sorted, and the sorting result is output as the re-identification result. In the present invention, ascending order is preferably used, and the higher the order is, the higher the matching rate is.

A pedestrian re-identification system based on semantically consistent horizontal bars and foreground correction according to the second embodiment of the present invention, as shown in FIG. 2, includes: an image acquisition module 100, a global feature extraction module 200, a local feature extraction module 300, and a feature splicing module. Module 400, identifying and outputting module 500;

The image acquisition module 100 is configured to acquire an image to be recognized as an input image;

The global feature extraction module 200 is configured to extract the feature of the input image through the feature extraction layer of the pedestrian re-identification model as the first feature;

The local feature extraction module 300 is configured to, based on the first feature, obtain the foreground feature corresponding to the pedestrian in the input image as the second feature through the pre-trained row classifier in the pedestrian re-identification model, respectively, and obtain the input image. The feature of the horizontal strip area of each set part of the pedestrian in the image is used as the third feature;

The feature splicing module 400 is configured to perform point-to-point multiplication of the second feature and the third feature, and splicing with the first feature to obtain a fourth feature;

The recognition output module 500 is configured to calculate the Euclidean distance between the fourth feature and the corresponding features of each image in the image library, and sort them, and output the sorting result as a re-identification result;

Wherein, the pedestrian re-identification model is constructed based on a deep convolutional neural network; the row classifier is constructed based on a fully connected layer and a softmax layer.

Those skilled in the technical field can clearly understand that, for the convenience and brevity of description, for the specific working process and related description of the system described above, reference may be made to the corresponding process in the foregoing method embodiments, which will not be repeated here.

It should be noted that the pedestrian re-identification system based on semantically consistent horizontal bars and foreground correction provided by the above-mentioned embodiments is only illustrated by the division of the above-mentioned functional modules. It can be completed by different functional modules, that is, the modules or steps in the embodiments of the present invention are decomposed or combined. For example, the modules in the above-mentioned embodiments can be combined into one module, and can also be further split into multiple sub-modules, so as to complete the above description. All or part of the functionality. The names of the modules and steps involved in the embodiments of the present invention are only for distinguishing each module or step, and should not be regarded as an improper limitation of the present invention.

A storage device according to a third embodiment of the present invention stores a plurality of programs, and the programs are suitable for being loaded by a processor and implementing the above-mentioned pedestrian re-identification method based on semantically consistent horizontal bars and foreground correction.

A processing device according to a fourth embodiment of the present invention includes a processor and a storage device; the processor is adapted to execute various programs; the storage device is adapted to store multiple programs; the programs are adapted to be loaded and executed by the processor In order to realize the above-mentioned pedestrian re-identification method based on semantically consistent horizontal bars and foreground correction.

Those skilled in the technical field can clearly understand that, for the convenience and brevity of description, the specific working process and related description of the storage device and processing device described above can refer to the corresponding process in the foregoing method example, which is not repeated here. Repeat.

Those skilled in the art should be aware that the modules and method steps of each example described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, computer software or a combination of the two, and the programs corresponding to the software modules and method steps Can be placed in random access memory (RAM), internal memory, read only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or as known in the art in any other form of storage medium. In order to clearly illustrate the interchangeability of electronic hardware and software, the components and steps of each example have been described generally in terms of functionality in the foregoing description. Whether these functions are performed in electronic hardware or software depends on the specific application and design constraints of the technical solution. Skilled artisans may use different methods of implementing the described functionality for each particular application, but such implementations should not be considered beyond the scope of the present invention.

The terms "first," "second," "third," etc. are used to distinguish between similar objects, and are not used to describe or indicate a particular order or sequence.

So far, the technical solutions of the present invention have been described with reference to the preferred embodiments shown in the accompanying drawings, however, those skilled in the art can easily understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

Claims (8)

1. a pedestrian re-identification method based on semantically consistent horizontal bars and foreground correction, is characterized in that, the method comprises: Step S10, acquiring the image to be recognized as an input image; Step S20, extracting the feature of the input image through the feature extraction layer of the pedestrian re-identification model as the first feature; Step S30, based on the first feature, obtain the foreground feature corresponding to the pedestrian in the input image as the second feature through the pre-trained line classifier in the pedestrian re-identification model, and obtain each set part of the pedestrian in the input image. The feature of the horizontal bar area is used as the third feature; Step S40, performing point-to-point multiplication of the second feature and the third feature, and splicing with the first feature to obtain a fourth feature; Step S50, calculating the Euclidean distance between the fourth feature and the corresponding feature of each image in the image library and sorting, and outputting the sorting result as a re-identification result; Wherein, the pedestrian re-identification model is constructed based on a deep convolutional neural network; the row classifier is constructed based on a fully connected layer and a softmax layer, and the training method is: Step A10, obtaining a training sample image set; Step A20, for any image in the training sample image set, extract its row feature and pool it to obtain its corresponding average feature; Step A30, judge whether the current number of iterations M is a multiple of N, if yes, then execute step A40, otherwise jump to step A50; wherein, N and M are natural numbers; Step A40, extract the average feature of all training sample images in the training sample image set, and obtain the pseudo-label corresponding to each row through self-similar clustering, and execute Step A50; In step A50, the average feature obtained in step A20 and the loss of the pseudo-label are calculated, and the parameters of the row classifier are updated. 2 . The pedestrian re-identification method based on semantically consistent horizontal bars and foreground correction according to claim 1 , wherein the self-similar clustering is a k-means clustering method. 3 . 3. The pedestrian re-identification method based on semantically consistent horizontal bars and foreground correction according to claim 1, is characterized in that, "respectively obtain the corresponding pedestrians in the input image through the pre-trained row classifier in the pedestrian re-identification model. The foreground feature is used as the second feature", and its method is: Obtain the confidence of each pixel in the input image to each part of the human body through the row classifier; The pixels whose confidence is greater than the first set threshold are regarded as foreground pixels, and the pixels whose confidence is less than the second set threshold are regarded as background pixels; The feature constructed based on the extracted foreground pixels is used as the foreground feature corresponding to the pedestrian in the input image. 4. The pedestrian re-identification method based on semantically consistent horizontal bars and foreground correction according to claim 1, is characterized in that, "obtain each setting of pedestrians in the input image through the pre-trained row classifier in the pedestrian re-identification model. The feature of the horizontal bar area of the part is used as the third feature", and its method is: Semantically segment the input image by the line classifier, and obtain a confidence map of the horizontal bar area of each set part of the pedestrian in the input image; Perform a point-to-point product operation on each confidence map and the first feature to obtain the feature of the horizontal strip area of each set part of the pedestrian in the input image. 5. the pedestrian re-identification method based on semantically consistent horizontal bar and foreground correction according to claim 1, is characterized in that, described pedestrian re-identification model, its loss function during training is:

in,

represents the loss value of the person re-identification model,

Represents the number of training sample images in a batch of the person re-identification model during training,

represents the batch,

Represents any image in a batch of training sample images of the person re-identification model during training,

Represents the image features in the image set A and

A training sample image with the largest Euclidean distance of the features,

Represents the image features in the image set B and

A training sample image with the smallest Euclidean distance of the features,

represents the preset distance interval,

means including

an image set of all images with the same ID,

Indicates that the current batch except

An image set constructed from all images except those contained in ,

represents the Euclidean distance. 6. A pedestrian re-identification system based on semantically consistent horizontal bars and foreground correction, characterized in that the system comprises: an image acquisition module, a global feature extraction module, a local feature extraction module, a feature splicing module, and a recognition output module; The image acquisition module is configured to acquire an image to be recognized as an input image; The global feature extraction module is configured to extract the feature of the input image through the feature extraction layer of the pedestrian re-identification model as the first feature; The local feature extraction module is configured to, based on the first feature, obtain foreground features corresponding to pedestrians in the input image as second features through a pre-trained row classifier in the pedestrian re-identification model, and obtain the input image The characteristics of the horizontal bar area of each set part of the pedestrian are taken as the third characteristic; The feature splicing module is configured to perform point-to-point multiplication of the second feature and the third feature, and splicing with the first feature to obtain a fourth feature; The recognition output module is configured to calculate the Euclidean distance between the fourth feature and the corresponding features of each image in the image library and sort them, and output the sorting result as a re-identification result; Wherein, the pedestrian re-identification model is constructed based on a deep convolutional neural network; the row classifier is constructed based on a fully connected layer and a softmax layer, and the training method is: Step A10, obtaining a training sample image set; Step A20, for any image in the training sample image set, extract its row feature and pool it to obtain its corresponding average feature; Step A30, judge whether the current number of iterations M is a multiple of N, if yes, then execute step A40, otherwise jump to step A50; wherein, N and M are natural numbers; Step A40, extract the average feature of all training sample images in the training sample image set, and obtain the pseudo-label corresponding to each row through self-similar clustering, and execute Step A50; In step A50, the average feature obtained in step A20 and the loss of the pseudo-label are calculated, and the parameters of the row classifier are updated. 7. A storage device, wherein a plurality of programs are stored, wherein the programs are adapted to be loaded and executed by a processor to implement the semantically consistent horizontal bar and foreground correction according to any one of claims 1-5 Pedestrian Re-ID method. 8. A processing device, comprising a processor and a storage device; the processor is adapted to execute each program; the storage device is adapted to store a plurality of programs; characterized in that the program is adapted to be loaded and executed by the processor to Implement the pedestrian re-identification method based on semantically consistent horizontal bars and foreground correction according to any one of claims 1-5.

Images