CN112801020B - Pedestrian re-identification method and system based on background grayscale - Google Patents

Abstract

The invention relates to a pedestrian re-identification method and a system based on background graying, wherein the method comprises the following steps: s1, carrying out background graying processing on an original image to obtain a processed BGg image; s2, based on a ResNet50 two-way network, one way is background gray scale Stream BGg-Stream, feature extraction is carried out on BGg images, and the other way is global Stream G-Stream, feature extraction is carried out on original images; s3, interacting the two paths of networks through cascade connection; s4, connecting the SCAB module with the characteristic diagrams obtained by the two networks at each stage to serve as input characteristic diagrams of the next layer convolution of the BGg-stream, so that the two networks are connected, and the two characteristics are combined; and S5, updating network parameters by adopting a triple loss function, calculating similarity, and finally outputting a permutation sequence. The method and the system are favorable for weakening background interference and improving the accuracy of pedestrian re-identification.

Description

Pedestrian re-identification method and system based on background grayscale

technical field

The invention belongs to the field of computer vision, and in particular relates to a pedestrian re-identification method and system based on background grayscale.

Background technique

Person re-identification (ReID) is a well-researched problem in computer vision, which aims to retrieve images of a specific person from a large number of images captured by different cameras. It is the most basic visual recognition problem in video surveillance and has broad application prospects. The traditional methods proposed by ReID mostly use low-level features of color histogram and texture histogram, and use metric learning to find a distance function that minimizes the distance between images from the same class and maximizes the distance between different classes change. ReID has been studied in academia for many years, and it was only in recent years that with the development of deep learning, a very huge breakthrough was made. However, the problem that cannot be ignored is that, as a high-dimensional feature, deep convolution features are easily affected by factors such as people's poses, object occlusions, differences in light intensity, and background clutter.

Many methods have been proposed in the past few years to obtain more robust features. But these methods often directly take the entire image as input. Such global information contains not only pedestrian features but also background clutter features. At present, the effective methods that can alleviate the influence of background clutter are mainly divided into two types: 1) methods based on body region detection, such as using pose and key point estimation, and using part region detection methods to extract human information in images. 2) Methods based on human body segmentation. In recent years, existing image segmentation methods including FCN, mask R-CNN, jppnet, etc. have been able to obtain excellent results in terms of background removal.

But in practical unconstrained situations, person re-identification is still an extremely challenging task. How to extract discriminative and robust features to background clutter invariance is the core issue, because non-pedestrian parts in the background can greatly interfere with the features of foreground information.

Most of the existing methods to solve background interference are based on body region detection or use segmentation to filter out the background, but they still have one of the following limitations. First, pre-training of additional detection models and segmentation models is required, as well as additional data collection work. Second, the potential dataset bias between pose estimation and reid can cause partition errors and destroy the original complete body shape characteristics of pedestrians. Third, although the background is partially truncated, it still exists around the pedestrian and participates in model training with the same weight as the pedestrian area. According to this analysis, these methods do not really propose a fundamental solution to the background clutter problem. Fourth, the disadvantages brought by the tough segmentation not only destroy the original structure and smoothness of the image, but also completely abandon all the background information. Some background information can sometimes be useful as context information, and ignoring all background information will ignore some clues about the task of pedestrian re-identification.

SUMMARY OF THE INVENTION

The purpose of the present invention is to provide a method and system for pedestrian re-identification based on background grayscale, the method and system are conducive to weakening background interference and improving the accuracy of pedestrian re-identification.

In order to achieve the above object, the technical solution adopted in the present invention is: a method for re-identification of pedestrians based on background grayscale, comprising the following steps: S1, performing background grayscale processing on the original image, while the foreground information of the image remains unchanged, Obtain the processed BGg image, that is, the background grayscale image;

S2. A two-way network based on ResNet50, one is the background grayscale stream BGg-Stream, and the feature extraction is performed on the background grayscale image obtained in step S1, and the other is the global stream G-Stream, which performs feature extraction on the original image;

S3, the two-way network obtained in step S2 is interacted with each other by cascading;

S4. The feature map obtained by the two-way network at each stage, through the space-channel attention mechanism module, that is, the input feature map of the next layer of convolution of the BGg-stream after the SCAB module, so that the two-way network can be connected. Road features are combined;

S5. After feature extraction, use triple loss function to update network parameters, then perform similarity calculation, and finally output permutation sequence.

Further, the input of the BGg-stream is a BGg image, and the background grayscale formula of the image is:

BGg(i,j)=0.299×R(i,j)+0.587×G(i,j)+0.114×B(i,j)

Among them, BGg(i,j) is the pixel value of the BGg image in the i-th row and the j-th column, and R, G, and B are the three channels of the RGB image.

Further, the SCAB module includes a channel attention module and a spatial attention module.

Further, the channel attention module is implemented as follows: given an input feature map F _i ∈ R ^C*H*W , first use the average pooling operation to aggregate the spatial information of the feature map to generate a spatial context descriptor F _i ∈R ^C*1*1 , compresses and transforms the spatial information into channels; then sets the hidden activation size to F _i ∈ R ^C/r*1*1 to reduce parameter overhead, where R is the reduction rate; thus , the channel attention module is expressed as:

F _ii =σ(θ(R(ζ(δ(F _i ))))))

表示element-wise乘法。where F _i is the BGg-Stream feature of each stage, σ is the sigmoid activation function, R is the ReLU activation function; δ is the average number of pools; θ and ζ are represented as two different connection layers;

Represents element-wise multiplication.

Further, the spatial attention module is implemented as follows: for the input feature map F _i ∈ R ^C*H*W , where C is the total number of channels and H*W is the size of the feature map, the spatial attention module is expressed as:

F _iii =σ(C(F _i ))

where σ is the sigmoid activation function, and the output of the spatial attention module is F _iii ∈ R ^1*H*W .

Further, the local feature response maximization strategy, that is, the LRM strategy, is applied in the testing phase; in the testing process, the feature map is divided into n regions of an appropriate number horizontally, and the feature with the largest response is extracted for each part of the feature. as a feature of this section.

The present invention also provides a pedestrian re-identification system based on background grayscale, comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor. When the processor runs the computer program, all described method steps.

Compared with the prior art, the present invention has the following beneficial effects:

1. The present invention does not require additional training and data sets.

2. The present invention can maintain the integrity and effectiveness of the pedestrian's original body shape information.

3. The present invention can accurately locate the human body area and process all backgrounds including the range of the pedestrian's body.

4. The present invention can leave useful information without being disturbed by strong colors in the background.

5. The present invention enables the model to focus on the learning of foreground information during the learning process, and further weakens background interference.

Description of drawings

FIG. 1 is a flow chart of a method implementation according to an embodiment of the present invention.

FIG. 2 is a schematic structural diagram of a channel attention module in an embodiment of the present invention.

FIG. 3 is a schematic structural diagram of a spatial attention module in an embodiment of the present invention.

Detailed ways

The present invention will be further described below with reference to the accompanying drawings and embodiments.

It should be noted that the following detailed description is exemplary and intended to provide further explanation of the application. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs.

It should be noted that the terminology used herein is for the purpose of describing specific embodiments only, and is not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly dictates otherwise, the singular is intended to include the plural as well, furthermore, it is to be understood that when the terms "comprising" and/or "including" are used in this specification, it indicates that There are features, steps, operations, devices, components and/or combinations thereof.

As shown in FIG. 1 , this embodiment provides a method for pedestrian re-identification based on background grayscale, including the following steps:

S1. Use the mask to perform background grayscale processing on the original image, while the foreground information of the image remains unchanged to obtain a processed BGg image, that is, a background grayscale image (an RGB image in the foreground and an image in which the background is a grayscale image).

S2. A two-way network based on ResNet50, one is the background grayscale stream BGg-Stream, and the feature extraction is performed on the background grayscale image obtained in step S1, and the other is the global stream G-Stream, which performs feature extraction on the original image;

S3, the two-way network obtained in step S2 is interacted with each other by cascading;

S4. The feature map obtained by the two-way network at each stage, through the space-channel attention mechanism module, that is, the input feature map of the next layer of convolution of the BGg-stream after the SCAB module, so that the two-way network can be connected. Road features are combined;

S5. After feature extraction, use triple loss function to update network parameters, then perform similarity calculation, and finally output permutation sequence.

In this embodiment, the input of the BGg-stream is a BGg image, and the background grayscale formula of the image is:

BGg(i,j)=0.299×R(i,j)+0.587×G(i,j)+0.114×B(i,j)

Among them, BGg(i,j) is the pixel value of the BGg image in the i-th row and the j-th column, and R, G, and B are the three channels of the RGB image.

The SCAB module includes a channel attention module and a space attention module.

In this embodiment, the complete structure of the channel attention module is shown in FIG. 2 . The channel attention module is implemented as follows: given an input feature map F _i ∈ R ^C*H*W , first use the average pooling operation to aggregate the spatial information of the feature map to generate a spatial context descriptor F _i ∈ R ^{C *1*1} , the spatial information is compressed and transformed into the channel; then the hidden activation size is set to F _i ∈ R ^C/r*1*1 to reduce the parameter overhead, where R is the reduction rate; therefore, the channel pays attention to Modules are represented as:

F _ii =σ(θ(R(ζ(δ(F _i ))))))

表示element-wise乘法。where F _i is the BGg-Stream feature of each stage, σ is the sigmoid activation function, R is the ReLU activation function; δ is the average number of pools; θ and ζ are represented as two different connection layers;

Represents element-wise multiplication.

In this embodiment, the complete structure of the spatial attention module is shown in FIG. 3 . The spatial attention module is implemented as follows: for the input feature map F _i ∈ R ^C*H*W , where C is the total number of channels and H*W is the size of the feature map, the spatial attention module is expressed as:

F _iii =σ(C(F _i ))

where σ is the sigmoid activation function, and the output of the spatial attention module is F _iii ∈ R ^1*H*W .

In this embodiment, the local feature response maximization strategy, that is, the LRM strategy, is applied in the testing phase; in the testing process, the feature map is divided into n regions of an appropriate number horizontally, and n=8 in this embodiment , and extract the feature with the largest response for each part of the feature as the feature of this part.

This embodiment also provides a pedestrian re-identification system based on background grayscale, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor. When the processor runs the computer program, the the above method steps.

As will be appreciated by those skilled in the art, the embodiments of the present application may be provided as a method, a system, or a computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present application. It will be understood that each process and/or block in the flowchart illustrations and/or block diagrams, and combinations of processes and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to the processor of a general purpose computer, special purpose computer, embedded processor or other programmable data processing device to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing device produce Means for implementing the functions specified in a flow or flow of a flowchart and/or a block or blocks of a block diagram.

These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory result in an article of manufacture comprising instruction means, the instructions The apparatus implements the functions specified in the flow or flow of the flowcharts and/or the block or blocks of the block diagrams.

These computer program instructions can also be loaded on a computer or other programmable data processing device to cause a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process such that The instructions provide steps for implementing the functions specified in the flow or blocks of the flowcharts and/or the block or blocks of the block diagrams.

The above are only preferred embodiments of the present invention, and are not intended to limit the present invention in other forms. Any person skilled in the art may use the technical content disclosed above to make changes or modifications to equivalent changes. Example. However, any simple modifications, equivalent changes and modifications made to the above embodiments according to the technical essence of the present invention without departing from the content of the technical solutions of the present invention still belong to the protection scope of the technical solutions of the present invention.

Claims (4)

1. a pedestrian re-identification method based on background graying, is characterized in that, comprises the following steps: S1. Perform background grayscale processing on the original image, while the foreground information of the image remains unchanged, to obtain a processed BGg image, that is, a background grayscale image; S2. A two-way network based on ResNet50, one is the background grayscale stream BGg-Stream, and the feature extraction is performed on the background grayscale image obtained in step S1, and the other is the global stream G-Stream, which performs feature extraction on the original image; S3, the two-way network obtained in step S2 is interacted with each other by cascading; S4. The feature map obtained by the two-way network at each stage, through the space-channel attention mechanism module, that is, the input feature map of the next layer of convolution of the BGg-stream after the SCAB module, so that the two-way network can be connected. Road features are combined; S5. After feature extraction, use triple loss function to update network parameters, then perform similarity calculation, and finally output permutation sequence; The SCAB module includes a channel attention module and a space attention module; The channel attention module is implemented as follows: given an input feature map F _i ∈ R ^C*H*W , first use the average pooling operation to aggregate the spatial information of the feature map to generate a spatial context descriptor F _i ∈ R ^{C *1*1} , the spatial information is compressed and transformed into the channel; then the hidden activation size is set to F _i ∈ R ^C/r*1*1 to reduce the parameter overhead, where R is the reduction rate; therefore, the channel pays attention to Modules are represented as:

Among them, F _i is the BGg-Stream feature of each stage, σ is the sigmoid activation function, R is the ReLU activation function; δ is the average pool number;

and ζ are represented as two different connection layers;

represents element-wise multiplication; The spatial attention module is implemented as follows: for the input feature map F _i ∈ R ^C*H*W , where C is the total number of channels and H*W is the size of the feature map, the spatial attention module is expressed as: F _iii =σ(C(F _i )) where σ is the sigmoid activation function, and the output of the spatial attention module is F _iii ∈ R ^1*H*W . 2. the pedestrian re-identification method based on background graying according to claim 1, is characterized in that, the input of described BGg-stream is BGg image, and the background graying formula of image is: BGg(i,j)=0.299×R(i,j)+0.587×G(i,j)+0.114×B(i,j) Among them, BGg(i,j) is the pixel value of the BGg image in the i-th row and the j-th column, and R, G, and B are the three channels of the RGB image. 3. the pedestrian re-identification method based on background graying according to claim 1, is characterized in that, in testing stage, apply local feature response maximization strategy, namely LRM strategy; It is divided into an appropriate number of n regions, and the feature with the largest response is extracted for each part of the feature as the feature of the part. 4. a pedestrian re-identification system based on background graying, is characterized in that, comprises memory, processor and the computer program that is stored on memory and can run on processor, when processor runs this computer program, realizes. The method steps of any one of claims 1-3.

Images