CN118196700A - Method and device for identifying evacuated people under indoor smoke shielding - Google Patents

Abstract

The present invention relates to the technical field of video image processing and pattern recognition, and in particular to a method and device for identifying evacuees under indoor smoke cover, the method comprising: obtaining a sample set of evacuees under indoor smoke cover; constructing an evacuee feature extraction network model, comprising: a local to global transition parallel module, a local and global enhancement interaction module, and a multi-scale feature parallel extraction module; inputting the sample set into the local to global transition parallel module, the local and global enhancement interaction module, and the multi-scale feature parallel extraction module to obtain an evacuee feature map; training the evacuee feature extraction network model according to the evacuee feature map to obtain a trained evacuee feature extraction network model; obtaining image data of the evacuees to be identified and inputting it into the trained evacuee feature extraction network model to obtain an evacuee identification result. The present invention can improve the accuracy of identifying evacuees under indoor smoke scenes.

Description

A method and device for identifying evacuated personnel under indoor smoke cover

Technical Field

The invention relates to the technical field of video image processing and pattern recognition, and in particular to a method and a device for identifying evacuated personnel under the cover of indoor smoke.

Background technique

Accurate personnel detection is very important in emergency rescue. Currently, the commonly used human detection instruments in disaster environments include infrared detection, audio detection and radar detection, but because these methods have a short detection distance and are easily affected by temperature, it is difficult for these devices to effectively detect personnel in the case of indoor fires and smoke. Therefore, it is crucial to use the factual information of the fire scene transmitted by the surveillance video to use video/image processing technology to detect evacuated personnel in indoor smoke scenes to help emergency rescue.

Personnel detection is an important research topic in computer vision and can be applied to the fields of robots, smart cars, and video surveillance. Currently, most studies tend to study general scenarios, and there are few studies on personnel detection in special scenarios under smoke and occlusion. Most of the personnel detection methods in smoke occlusion scenarios are to defog first and then detect. Smoke is a non-rigid gas with dynamic and uneven changes. When using the defog method to process smoke, the clarity of the processed image is not high, resulting in low accuracy of the recognition result and a long recognition process, which is not suitable for emergency fire rescue scenarios.

Summary of the invention

In order to solve the technical problems of low accuracy of recognition results and long recognition process in the prior art, the embodiment of the present invention provides a method and device for identifying evacuees under indoor smoke cover. The technical solution is as follows:

On the one hand, a method for identifying evacuees under indoor smoke cover is provided, the method is implemented by an evacuee identification device under indoor smoke cover, and the method comprises:

S1. Obtain sample data of people evacuated under indoor smoke cover;

S2, constructing a feature extraction network model for evacuees; the feature extraction network model for evacuees includes: a local to global transition parallel module, a local and global enhanced interaction module, and a multi-scale feature parallel extraction module;

S3, inputting the sample data into the local to global transition parallel module, generating a global feature map and a local feature map in parallel; inputting the global feature map and the local feature map into the local and global enhancement interaction module, obtaining a feature map after the interaction of local and global information; inputting the feature map after the interaction of local and global feature information into the multi-scale feature parallel extraction module, obtaining a feature map of evacuated personnel;

S4, training the evacuee feature extraction network model according to the evacuee feature graph to obtain a trained evacuee feature extraction network model;

S5, obtaining image data of the person to be evacuated;

S6. Inputting the image data of the evacuees to be identified into the trained evacuees feature extraction network model to obtain the identification results of the evacuees.

Optionally, the sample data of people evacuating under indoor smoke cover in S1 includes: image data sets of different scene types, different human postures and different smoke types.

Optionally, the local-to-global transition parallel module includes: a local attention module and a global attention module.

Optionally, the step S3 of inputting the sample data into the local-to-global transition parallel module to generate a global feature map and a local feature map in parallel includes:

S31, dividing the input sample data into a first sub-feature graph and a second sub-feature graph;

S32. Use a dual-path parallel method to input the first sub-feature map into a local attention module to generate a local feature map; input the second sub-feature map into a global attention module to generate a global feature map.

Optionally, the step of inputting the global feature map and the local feature map into a local and global enhancement interaction module to obtain a feature map after the interaction of local and global feature information includes:

S33, mapping the local feature map into a plurality of local feature image blocks, inputting the plurality of local feature image blocks into a global attention module, exchanging information between the local feature image blocks, and obtaining local feature image blocks having global feature information;

S34, mapping the local feature image block with global feature information to the local feature map, and obtaining a feature map after the local and global feature information interact.

Optionally, the step of inputting the feature map after the interaction of the local and global feature information into a multi-scale feature parallel extraction module to obtain a feature map of evacuated personnel includes:

S35. Use three parallel depth convolutions of different scales to perform convolution operations on the feature map after the interaction of local and global feature information in parallel to obtain the feature map of evacuated personnel.

On the other hand, a device for identifying evacuees under indoor smoke cover is provided, and the device is applied to a method for identifying evacuees under indoor smoke cover, and the device comprises:

A first acquisition unit is used to acquire sample data of people evacuated under the cover of indoor smoke;

A construction unit is used to construct a network model for extracting features of evacuated personnel; the network model for extracting features of evacuated personnel includes: a local to global transition parallel module, a local and global enhanced interaction module, and a multi-scale feature parallel extraction module;

A feature extraction unit is used to input the sample data into the local-to-global transition parallel module to generate a global feature map and a local feature map in parallel; input the global feature map and the local feature map into the local and global enhancement interaction module to obtain a feature map after the local and global feature information interacts; input the feature map after the local and global feature information interacts into the multi-scale feature parallel extraction module to obtain an evacuated personnel feature map;

A training unit, used for training the evacuee feature extraction network model according to the evacuee feature graph to obtain a trained evacuee feature extraction network model;

A second acquisition unit is used to acquire image data of the evacuee to be identified;

The recognition unit is used to input the image data of the evacuees to be identified into the trained evacuees feature extraction network model to obtain the recognition results of the evacuees.

Optionally, the image dataset of people evacuated under indoor smoke cover includes: image datasets of different scene types, different human postures and different smoke types.

Optionally, the local-to-global transition parallel module includes: a local attention module and a global attention module.

Optionally, the sample data is input into the local-to-global transition parallel module to generate a global feature map and a local feature map in parallel, including:

Dividing the input sample data into a first sub-feature graph and a second sub-feature graph;

A dual-path parallel method is used to input the first sub-feature map into the local attention module to generate a local feature map; and the second sub-feature map is input into the global attention module to generate a global feature map.

Optionally, inputting the global feature map and the local feature map into a local and global enhancement interaction module to obtain a feature map after the local and global feature information are interacted, including:

Mapping the local feature map into several local feature image blocks, inputting the several local feature image blocks into the global attention module, enabling information exchange between the local feature image blocks, and obtaining local feature image blocks with global feature information;

The local feature image block with global feature information is mapped to the local feature map to obtain the feature map after the interaction of local and global feature information.

Optionally, the feature map after the interaction of the local and global feature information is input into a multi-scale feature parallel extraction module to obtain a feature map of evacuated personnel, including:

Three parallel depth convolutions of different scales are used to perform convolution operations on the feature map after the interaction of local and global feature information in parallel to obtain the feature map of evacuated personnel.

On the other hand, a device for identifying evacuees under indoor smoke cover is provided, and the device for identifying evacuees under indoor smoke cover comprises: a processor; a memory, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, any one of the above-mentioned methods for identifying evacuees under indoor smoke cover is implemented.

On the other hand, a computer-readable storage medium is provided, wherein at least one instruction is stored in the storage medium, and the at least one instruction is loaded and executed by a processor to implement any one of the above-mentioned methods for identifying evacuees under indoor smoke cover.

The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

According to an embodiment of the present invention, sample data of evacuees under indoor smoke cover is first obtained; secondly, a network model for extracting features of evacuees is constructed; the network model for extracting features of evacuees includes: a local-to-global transition parallel module, a local-to-global enhancement interaction module, and a multi-scale feature parallel extraction module; the sample data is input into the local-to-global transition parallel module to generate a global feature map and a local feature map in parallel; the global feature map and the local feature map are input into the local-to-global enhancement interaction module to obtain a feature map after the interaction of local and global feature information; the feature map after the interaction of local and global feature information is input into the multi-scale feature parallel extraction module to obtain a feature map of evacuees; according to the feature map of evacuees, the network model for extracting features of evacuees is trained to obtain a trained network model for extracting features of evacuees; finally, image data of evacuees to be identified are obtained; the image data of evacuees to be identified are input into the trained network model for extracting features of evacuees to obtain an identification result of evacuees. Compared with the prior art, the embodiments of the present invention effectively improve the ability of the feature recognition network model to extract features of evacuated personnel in the smoke generated by indoor fires, avoid the limitation of too many local features and the degradation of global performance caused by too many non-local features, and extract multi-scale personnel feature information through a multi-scale feedforward module, thereby reducing the interference of smoke on the predicted structure of the human body; the embodiment of the present invention can avoid the interference of temperature in the scene, can perform effective personnel detection in the case of indoor fire smoke, and can provide guiding suggestions for firefighters to rescue and evacuate personnel.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

FIG1 is a flow chart of a method for identifying personnel to be evacuated under indoor smoke cover provided by an embodiment of the present invention;

FIG2 is a flow chart of extracting global features by a global attention module provided by an embodiment of the present invention;

3 is a flow chart of a local to global transition parallel module provided by an embodiment of the present invention;

FIG4 is a flow chart of a local and global enhanced interaction module provided by an embodiment of the present invention;

FIG5 is a structural diagram of a multi-scale feature parallel extraction module provided in an embodiment of the present invention;

FIG6 is a structural diagram of a global and local enhancement interaction block provided in an embodiment of the present invention;

7 is a schematic diagram of the overall structure of a feature extraction network model based on global and local enhanced interaction provided by an embodiment of the present invention;

FIG8 is a block diagram of a device for identifying evacuees under indoor smoke cover provided by an embodiment of the present invention;

FIG. 9 is a schematic diagram of the structure of a device for identifying personnel to be evacuated under indoor smoke cover provided by an embodiment of the present invention.

Detailed ways

The technical solution of the present invention is described below in conjunction with the accompanying drawings.

In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "example" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "example" is intended to present the concept in a specific way. In addition, in the embodiments of the present invention, the meaning expressed by "and/or" can be both, or it can be either of the two.

In the embodiments of the present invention, "image" and "picture" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same. "of", "corresponding, relevant" and "corresponding" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same.

In the embodiments of the present invention, sometimes a subscript such as _W1 may be mistakenly written as a non-subscript such as W1. When the difference is not emphasized, the meanings to be expressed are consistent.

In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

The embodiment of the present invention provides a method for identifying evacuees under indoor smoke cover, which can be implemented by an indoor smoke cover evacuee identification device, which can be a terminal or a server. As shown in the flowchart of the method for identifying evacuees under indoor smoke cover in FIG1 , the processing flow of the method can include the following steps:

S1. Obtain sample data of people evacuated under indoor smoke cover.

Optionally, the sample data of people evacuated under indoor smoke cover in S1 includes: image datasets of different scene types, different human postures and different smoke types.

Among them, the scene types can be selected from some densely populated places such as schools, shopping malls, and office buildings with a large number of people and a relatively large proportion of buildings as fire scenes.

Among them, people will adopt different postures and speeds to escape from the fire scene during the fire evacuation process, and the human body posture can select the commonly used elements of bending over and covering the mouth.

Among them, the type of smoke can be selected from light white smoke to thick black smoke according to the development process of the fire.

In a feasible implementation, images of people evacuating in real indoor smoke scenes collected on the Internet are used as training sample data, and images of people evacuating in real indoor smoke scenes are used as test data.

S2. Construct a feature extraction network model for evacuees; the feature extraction network model for evacuees includes: a local to global transition parallel module, a local and global enhanced interaction module, and a multi-scale feature parallel extraction module.

Optionally, the local-to-global transition parallel module includes: a local attention module and a global attention module.

In a feasible implementation, the local attention module aggregates local features of the feature map using a dynamic deep convolution method to obtain a local feature map. The specific implementation steps may include:

(1) The feature map of evacuated personnel under indoor smoke cover is input into the local attention module, and the spatial context of the feature map is aggregated by adaptive average pooling operation, and the spatial dimension of the feature map is compressed to obtain the feature map after the compressed spatial dimension; the feature map after the compressed spatial dimension is input into the 1×1 convolution function to obtain the attention feature map. The specific process of obtaining the attention feature map can be expressed by the following formula (1):

Among them, A' represents the attention feature map, G represents the number of attention groups, X represents the input feature map X∈R ^H×W×C , R represents the matrix image, which means it is composed of 3 H×W matrices; C represents the number of image channels, H represents the height of the input feature map, W represents the width of the input feature map, r represents the equal fraction, Conv function represents the convolution function, and AdaptivePool represents the adaptive average pooling operation.

(2) Generate the attention weight of the attention feature map through the softmax function, multiply the attention weight with each element in a preset set of learnable parameters to obtain multiple product results; sum all the product results to obtain the deep convolution kernel. The attention weight and the deep convolution kernel can be expressed by the following formula (2) and formula (3):

A＝Softmax(Reshape(A')) (2)

Among them, A represents the attention weight, A' represents the attention feature map, and the Reshape function is used to give a new shape to the array without changing the data.

Among them, A represents the attention weight, A' represents the attention feature map, and P represents a set of preset learnable parameters R represents the matrix image, which is composed of 3 H×W matrices; G represents the number of attention groups, C represents the number of image channels, K ² represents the compressed spatial dimension, W represents the size of the convolution kernel, and the value range of i is 0,...,G.

Among them, since different input feature maps will produce different attention weights, the size of the convolution kernel will also change with the change of the input feature map; among them, dynamic deep convolution and adaptive pooling operations are used to extract the feature map of evacuated personnel under indoor smoke cover, and the local feature map is obtained by calculating the attention weight. The calculation of the local attention module can retain the distribution information of the features, which can improve the expression ability of the evacuated personnel feature recognition network model and the feature discrimination ability.

FIG2 shows a flowchart of extracting global features by the global attention module. In a feasible implementation, a patch sampling operation is adopted to perform adaptive maximum pooling and average pooling operations on the input indoor smoke-occluded evacuation personnel feature map, so that the spatial resolution of the feature map is reduced to obtain a global feature map. Among them, H _gw , W _gw represent the size of the global window; R represents the matrix image, which is composed of 3 H×W matrices; Among them, the method of combining average pooling operation and maximum pooling operation can be used to extract the feature description of the evacuated personnel in the feature map of evacuated personnel under indoor smoke occlusion.

In a feasible implementation, the global attention module uses convolution operations to aggregate the feature layers of the feature map of evacuated personnel under indoor smoke cover to obtain a fused feature map, focuses on the global features based on the obtained fused feature map, and outputs a feature map with strong correlation between single-point features and global features in each channel.

S3. Input the sample data into the local to global transition parallel module to generate the global feature map and the local feature map in parallel; input the global feature map and the local feature map into the local and global enhancement interaction module to obtain the feature map after the interaction of local and global feature information; input the feature map after the interaction of local and global feature information into the multi-scale feature parallel extraction module to obtain the evacuated personnel feature map.

Optionally, S3 inputs the sample data into a local-to-global transition parallel module to generate a global feature map and a local feature map in parallel, including:

S31, dividing the input sample data into a first sub-feature graph and a second sub-feature graph;

S32. Use a dual-path parallel method to input the first sub-feature map into a local attention module to generate a local feature map; input the second sub-feature map into a global attention module to generate a global feature map.

FIG3 shows a flow chart of the local-to-global transition parallel module. In a feasible implementation, the input feature map of evacuated personnel in an indoor smoke scene is evenly divided into a first sub-feature map and a second sub-feature map along the channel dimension, and the first sub-feature map and the second sub-feature map are respectively input into the global attention module and the local attention module to obtain the global feature map _X'2 and the local feature _X'1 , which can be expressed by the following formulas (4) and (5):

X ₁ ,X ₂ =Split(X) (4)

X'＝Concat(X ₁ ,X ₂ ) (5)

Among them, X∈R ^H×W×C represents the input feature map, R represents the matrix image, which is composed of 3 H×W matrices; H represents the height of the feature map, W is the width of the feature map, C represents the number of channels of the feature map, _X1 represents the first sub-feature map, _X2 represents the second sub-feature map, and X' represents the result of the local to global transition parallel module output, including _X'1 and _X'2 , where _X'1 represents the local feature map and _X'2 represents the global feature map; Among them, the Concat function represents connecting different feature maps, and the Split function represents dividing the input image along the channel dimension.

In a feasible implementation, the local-to-global transition parallel module uses a shifted self-attention mechanism to exchange different feature information, so that local features and global features can be generated in parallel; the local-to-global transition parallel module can avoid the inability of information from different regions to be exchanged with each other and avoid each image block being restricted to a fixed interaction area, and can fuse the global information and local information in the feature map of evacuated personnel in indoor smoke scenes, balance the weights between local and global information, and enable the evacuated personnel feature recognition network model to establish a short-distance to long-distance dependency relationship of information.

Optionally, S3 inputs the global feature map and the local feature map into a local and global enhancement interaction module to obtain a feature map after the local and global feature information are interacted, including:

S33, mapping the local feature map into a plurality of local feature image blocks, inputting the plurality of local feature image blocks into a global attention module, exchanging information between the local feature image blocks, and obtaining local feature image blocks having global feature information;

Among them, the number of local feature image blocks is determined by the size of the input local features;

S34, mapping the local feature image block with global feature information to the local feature map, and obtaining a feature map after the local and global feature information interact.

Fig. 4 shows a flow chart of the local and global enhancement interaction module. The local and global enhancement interaction module consists of two transmission processes, including: a feedback process from local feature information to global feature information and a feedback process from global feature information to local feature information.

Among them, the local and global enhancement interaction module uses representative and transitive image patches to communicate between local and global information.

Among them, the local and global enhancement module can interactively aggregate the global coarse-grained information and local fine-grained information in the evacuee feature map under smoke occlusion, which can improve the accuracy of evacuee feature recognition network model in identifying evacuees.

Optionally, S3 inputs the feature map after the interaction of local and global feature information into a multi-scale feature parallel extraction module to obtain a feature map of evacuated personnel, including:

S35. Use three parallel depth convolutions of different scales to perform convolution operations on the feature map after the interaction of local and global feature information in parallel to obtain the feature map of evacuated personnel.

Figure 5 shows the structure of the multi-scale feature parallel extraction module. In a feasible implementation, three parallel depth convolutions of different scales are used to perform convolution operations on the feature map after the interaction of local and global feature information in parallel, each convolution processes one-third of the channels, and the sizes of the depth convolution kernels are {1, 3, 5} respectively.

Among them, due to the perspective effect problem caused by the shooting angle of the surveillance video, multi-scale feature extraction is performed on the feature map after the interaction of local and global feature information by using parallel deep convolutions of different scales, which can improve the ability to extract the features of evacuated personnel under smoke cover.

Among them, smoke scenes are different from ordinary scenes. The special escape posture of pedestrians and the different distances from the camera will cause obvious scale problems in the video or image. Using a multi-scale feedforward module to process the feature map can reduce the image scale difference. At the same time, using a multi-scale feedforward module to extract multi-scale personnel features can reduce the interference of smoke on the predicted structure of the human body.

In a feasible implementation, a global and local enhancement interaction block is constructed based on the above-mentioned local attention module, global attention module, local to global transition parallel module, local and global enhancement interaction module and multi-scale feedforward module.

Figure 6 shows a schematic diagram of the structure of the global and local enhancement interaction block, where each global and local enhancement interaction block consists of two consecutive blocks, the first block consists of a local attention module, a global attention module, and a local-to-global transition parallel module, and the second block consists of a local and global enhancement interaction module and a multi-scale feedforward module. The evacuation personnel feature extraction network model is constructed based on the global and local enhancement interaction block;

FIG7 is a schematic diagram showing the overall structure of a network model for extracting features of evacuated personnel based on global and local enhanced interaction. In a feasible implementation, the specific feature extraction process may include:

(1) Use convolution and linear mapping layers to transform the input image Preprocessing is performed, where R represents the matrix image, which is composed of 3 H×W matrices; H represents the height of the image, W represents the width of the image, and 3 is the number of channels. The image is convolved to divide the image into non-overlapping 4×4 image blocks, and the number of channels is converted into the number of channels of the feature map through a linear mapping layer;

(2) The evacuation personnel feature extraction network model is divided into four stages, each of which consists of several iterative global and local enhancement interaction blocks. The two consecutive stages are connected by image block merging. Before entering the next stage, the number of channels of the feature map is adjusted to twice the original number through linear mapping. The feature map of each stage can be expressed as Among them, i represents the number of stages, H is the height of the image, W is the width of the image, C represents the number of channels, and R represents the matrix image;

(3) Input feature map Among them, R represents the matrix image. After the first stage of processing, the first stage feature map is obtained./> The evacuee characteristic map obtained in the first stage/> As input, the second stage is carried out to obtain the characteristic map of evacuees in the second stage/> The second stage evacuation personnel characteristic map/> As input, obtain the third stage evacuation personnel feature map/> Until the fourth stage is completed, the characteristic map of evacuees in the fourth stage is obtained/> Complete feature extraction.

Among them, the evacuee feature extraction network model can fully combine the global information and local information of the input image; the evacuee feature extraction network first extracts the local feature information and global information of the evacuee image under smoke cover, and secondly, combines the local feature information with the global feature information, and finally extracts multi-scale feature information to enable multi-scale information interaction between the local and the global.

S4. According to the evacuee feature graph, the evacuee feature extraction network model is trained to obtain a trained evacuee feature extraction network model.

In a feasible implementation manner, the specific training process may include:

(1) Use the image annotation tool labelImg as the training set sample annotation tool to annotate the sample data, and use the deep learning image annotation software labelme to complete the annotation of the self-built data set of evacuated personnel in the indoor smoke scene. Save the annotated image as a .json format file and convert it into a .txt format that can be recognized by the target detection model;

(2) Setting parameters of the evacuee feature extraction network, including learning rate and number of training rounds;

(3) The evacuee feature extraction network model is trained according to the obtained evacuee feature map and the set training parameters to obtain a trained evacuee feature extraction network model.

S5. Obtain image data of the persons to be identified and evacuated.

S6. Input the image data of the evacuees to be identified into the trained evacuees feature extraction network model to obtain the identification results of the evacuees.

In a feasible implementation, the trained evacuee feature extraction network model analyzes the personnel features in the evacuee images in indoor smoke scenes, marks the evacuees while judging whether there are evacuees in the image data of the evacuees to be identified, and outputs the specific location information of the evacuees; by outputting the specific location information of the evacuees, the information of the trapped persons can be accurately and quickly provided to the fire rescue.

According to an embodiment of the present invention, sample data of evacuees under indoor smoke cover is first obtained; secondly, a network model for extracting features of evacuees is constructed; the network model for extracting features of evacuees includes: a local-to-global transition parallel module, a local-to-global enhancement interaction module, and a multi-scale feature parallel extraction module; the sample data is input into the local-to-global transition parallel module to generate a global feature map and a local feature map in parallel; the global feature map and the local feature map are input into the local-to-global enhancement interaction module to obtain a feature map after the interaction of local and global feature information; the feature map after the interaction of local and global feature information is input into the multi-scale feature parallel extraction module to obtain a feature map of evacuees; according to the feature map of evacuees, the network model for extracting features of evacuees is trained to obtain a trained network model for extracting features of evacuees; finally, image data of evacuees to be identified are obtained; the image data of evacuees to be identified are input into the trained network model for extracting features of evacuees to obtain an identification result of evacuees. Compared with the prior art, the embodiments of the present invention effectively improve the ability of the feature recognition network model to extract features of evacuated personnel in the smoke generated by indoor fires, avoid the limitation of too many local features and the degradation of global performance caused by too many non-local features, and extract multi-scale personnel feature information through a multi-scale feedforward module, thereby reducing the interference of smoke on the predicted structure of the human body; the embodiment of the present invention can avoid the interference of temperature in the scene, can perform effective personnel detection in the case of indoor fire smoke, and can provide guiding suggestions for firefighters to rescue and evacuate personnel.

FIG8 is a block diagram of a device for identifying evacuees under indoor smoke cover according to an exemplary embodiment, and the device is used in a method for identifying evacuees under indoor smoke cover. Referring to FIG8 , the device includes a first acquisition unit 310, a construction unit 320, a feature extraction unit 330, a training unit 340, a second acquisition unit 350, and an identification unit 360. For ease of explanation, FIG8 only shows the main components of the full-process visualization device 600:

The first acquisition unit 310 is used to acquire sample data of people evacuated under the cover of indoor smoke;

A construction unit 320 is used to construct a network model for extracting features of evacuees; the network model for extracting features of evacuees includes: a local to global transition parallel module, a local and global enhancement interaction module, and a multi-scale feature parallel extraction module;

The feature extraction unit 330 is used to input the sample data into the local to global transition parallel module to generate a global feature map and a local feature map in parallel; input the global feature map and the local feature map into the local and global enhancement interaction module to obtain a feature map after the local and global feature information interacts; input the feature map after the local and global feature information interacts into the multi-scale feature parallel extraction module to obtain an evacuated personnel feature map;

The training unit 340 is used to train the evacuee feature extraction network model according to the evacuee feature graph to obtain a trained evacuee feature extraction network model;

A second acquisition unit 350 is used to acquire image data of the evacuee to be identified;

The identification unit 360 is used to input the image data of the evacuees to be identified into the trained evacuee feature extraction network model to obtain the identification results of the evacuees.

Optionally, the image dataset of people evacuating under indoor smoke cover includes: image datasets of different scene types, different human postures and different smoke types.

Optionally, the local-to-global transition parallel module includes: a local attention module and a global attention module.

Optionally, the sample data is input into a local-to-global transition parallel module to generate a global feature map and a local feature map in parallel, including:

Dividing the input sample data into a first sub-feature graph and a second sub-feature graph;

A dual-path parallel method is used to input the first sub-feature map into the local attention module to generate a local feature map; and the second sub-feature map is input into the global attention module to generate a global feature map.

Optionally, the global feature map and the local feature map are input into a local and global enhancement interaction module to obtain a feature map after the local and global feature information are interacted, including:

Mapping the local feature map into several local feature image blocks, inputting the several local feature image blocks into the global attention module, enabling information exchange between the local feature image blocks, and obtaining local feature image blocks with global feature information;

The local feature image block with global feature information is mapped to the local feature map to obtain the feature map after the interaction of local and global feature information.

Optionally, the feature map after the interaction of local and global feature information is input into a multi-scale feature parallel extraction module to obtain a feature map of evacuated personnel, including:

Three parallel depth convolutions of different scales are used to perform convolution operations on the feature map after the interaction of local and global feature information in parallel to obtain the feature map of evacuated personnel.

According to an embodiment of the present invention, sample data of evacuees under indoor smoke cover is first obtained; secondly, a network model for extracting features of evacuees is constructed; the network model for extracting features of evacuees includes: a local-to-global transition parallel module, a local-to-global enhancement interaction module, and a multi-scale feature parallel extraction module; the sample data is input into the local-to-global transition parallel module to generate a global feature map and a local feature map in parallel; the global feature map and the local feature map are input into the local-to-global enhancement interaction module to obtain a feature map after the interaction of local and global feature information; the feature map after the interaction of local and global feature information is input into the multi-scale feature parallel extraction module to obtain a feature map of evacuees; according to the feature map of evacuees, the network model for extracting features of evacuees is trained to obtain a trained network model for extracting features of evacuees; finally, image data of evacuees to be identified are obtained; the image data of evacuees to be identified are input into the trained network model for extracting features of evacuees to obtain an identification result of evacuees. Compared with the prior art, the embodiments of the present invention effectively improve the ability of the feature recognition network model to extract features of evacuated personnel in smoke generated by indoor fires, avoid the limitation of too many local features and the degradation of global performance caused by too many non-local features, and extract multi-scale personnel feature information through a multi-scale feedforward module, thereby reducing the interference of smoke on the predicted structure of the human body; the embodiment of the present invention can avoid the interference of temperature in the scene, can effectively detect personnel in the case of indoor fire smoke, and can provide guiding suggestions for firefighters to rescue and evacuate personnel.

FIG9 is a schematic diagram of the structure of a device for identifying evacuees under indoor smoke cover provided by an embodiment of the present invention. As shown in FIG9 , the device for identifying evacuees under indoor smoke cover may include the device for identifying evacuees under indoor smoke cover shown in FIG8 . Optionally, the device for identifying evacuees under indoor smoke cover 410 may include a first processor 2001.

Optionally, the device for identifying people to be evacuated under indoor smoke cover 410 may further include a memory 2002 and a transceiver 2003 .

The first processor 2001, the memory 2002 and the transceiver 2003 may be connected via a communication bus.

The following is a detailed introduction to the various components of the indoor smoke-shielded evacuation personnel identification device 410 in conjunction with FIG. 9 :

The first processor 2001 is the control center of the indoor smoke-shielded evacuation personnel identification device 410, which can be a processor or a general term for multiple processing elements. For example, the first processor 2001 is one or more central processing units (CPUs), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention, such as one or more microprocessors (digital signal processors, DSPs), or one or more field programmable gate arrays (field programmable gate arrays, FPGAs).

Optionally, the first processor 2001 can perform various functions of the indoor smoke-covered personnel evacuation identification device 410 by running or executing a software program stored in the memory 2002 and calling data stored in the memory 2002 .

In a specific implementation, as an embodiment, the first processor 2001 may include one or more CPUs, such as CPU0 and CPU1 shown in FIG. 9 .

In a specific implementation, as an embodiment, the indoor smoke-shielded evacuee identification device 410 may also include multiple processors, such as the first processor 2001 and the second processor 2004 shown in FIG9 . Each of these processors may be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). The processor here may refer to one or more devices, circuits, and/or processing cores for processing data (such as computer program instructions).

The memory 2002 is used to store the software program for executing the solution of the present invention, and is controlled to be executed by the first processor 2001. The specific implementation method can refer to the above method embodiment, which will not be repeated here.

Optionally, the memory 2002 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 2002 may be integrated with the first processor 2001, or may exist independently, and be coupled to the first processor 2001 through the interface circuit (not shown in FIG. 9 ) of the indoor smoke-shielded evacuation personnel identification device 410, which is not specifically limited in the embodiment of the present invention.

The transceiver 2003 is used to communicate with a network device or a terminal device.

Optionally, the transceiver 2003 may include a receiver and a transmitter (not shown separately in FIG. 9 ), wherein the receiver is used to implement a receiving function, and the transmitter is used to implement a sending function.

Optionally, the transceiver 2003 can be integrated with the first processor 2001, or can exist independently and be coupled to the first processor 2001 through the interface circuit of the evacuation personnel identification device 410 under indoor smoke cover (not shown in Figure 9), which is not specifically limited in this embodiment of the present invention.

It should be noted that the structure of the indoor smoke-covered evacuee identification device 410 shown in FIG9 does not constitute a limitation on the router, and the actual knowledge structure identification device may include more or fewer components than shown in the figure, or a combination of certain components, or a different arrangement of components.

In addition, the technical effects of the device 410 for identifying evacuees under indoor smoke cover can refer to the technical effects of the method for identifying evacuees under indoor smoke cover described in the above method embodiment, which will not be repeated here.

It should be understood that the first processor 2001 in the embodiment of the present invention may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

It should also be understood that the memory in the embodiments of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

The above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

It should be understood that the term "and/or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and/or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A and B can be singular or plural. In addition, the character "/" in this article generally indicates that the associated objects before and after are in an "or" relationship, but it may also indicate an "and/or" relationship. Please refer to the context for specific understanding.

In the present invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can be represented by: a, b, c, a-b, a-c, b-c, or a-b-c, where a, b, and c can be single or multiple.

It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described equipment, devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROM), random access memories (RAM), magnetic disks or optical disks.

The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art who is familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims (10)

1. A method for identifying evacuees under indoor smoke shielding, the method comprising:

s1, acquiring sample data of evacuated people under the shielding of indoor smoke;

S2, constructing an evacuated personnel feature extraction network model; the evacuation personnel feature extraction network model comprises: the system comprises a local-global transition parallel module, a local-global enhancement interaction module and a multi-scale feature parallel extraction module;

S3, inputting the sample data into the local-to-global transition parallel module, and generating a global feature map and a local feature map in parallel; inputting the global feature map and the local feature map into a local and global enhancement interaction module to obtain a feature map after local and global feature information interaction; inputting the feature map after the interaction of the local and global feature information into a multi-scale feature parallel extraction module to obtain an evacuated personnel feature map;

S4, training the evacuator feature extraction network model according to the evacuator feature map to obtain a trained evacuator feature extraction network model;

s5, acquiring image data of evacuees to be identified;

S6, inputting the image data of the evacuees to be identified into the trained evacuee feature extraction network model, and obtaining the identification result of the evacuees.

2. The method of claim 1, wherein the sample data of evacuees under the indoor smoke occlusion of S1 comprises: image datasets of different scene types, different human poses and different smoke types.

3. The method of claim 1, wherein the local-to-global transitional parallelism module comprises: a local attention module and a global attention module.

4. The method according to claim 3, wherein the inputting the sample data into the local-to-global transitional parallel module in S3 generates a global feature map and a local feature map in parallel, and includes:

s31, dividing input sample data into a first sub-feature map and a second sub-feature map;

S32, inputting the first sub-feature map into a local attention module by adopting a dual-path parallel method to generate a local feature map; and inputting the second sub-feature map into a global attention module to generate a global feature map.

5. The method according to claim 4, wherein the inputting the global feature map and the local feature map into the local and global enhancement interaction module in S3 obtains a feature map after the local and global feature information interaction, includes:

S33, mapping the local feature map into a plurality of local feature image blocks, inputting the plurality of local feature image blocks into a global attention module, and enabling information exchange between the local feature image blocks to obtain the local feature image blocks with global feature information;

and S34, mapping the local feature image block with the global feature information to the local feature map to obtain a feature map after local and global feature information interaction.

6. The method of claim 5, wherein the step S3 of inputting the feature map after the interaction of the local and global feature information into a multi-scale feature parallel extraction module to obtain an evacuated person feature map includes:

And S35, carrying out convolution operation on the feature map after the interaction of the local and global feature information by utilizing three parallel depth convolutions with different scales, and obtaining the sparse personnel feature map.

7. An indoor smoke-shielded evacuee recognition apparatus for implementing the indoor smoke-shielded evacuee recognition method according to any one of claims 1 to 6, characterized in that the apparatus comprises:

the first acquisition unit is used for acquiring sample data of evacuated people under the shielding of indoor smoke;

The construction unit is used for constructing an evacuated personnel feature extraction network model; the evacuation personnel feature extraction network model comprises: the system comprises a local-global transition parallel module, a local-global enhancement interaction module and a multi-scale feature parallel extraction module;

the feature extraction unit is used for inputting the sample data into the local-to-global transition parallel module and generating a global feature map and a local feature map in parallel; inputting the global feature map and the local feature map into a local and global enhancement interaction module to obtain a feature map after local and global feature information interaction; inputting the feature map after the interaction of the local and global feature information into a multi-scale feature parallel extraction module to obtain an evacuated personnel feature map;

The training unit is used for training the evacuator feature extraction network model according to the evacuator feature map to obtain a trained evacuator feature extraction network model;

The second acquisition unit is used for acquiring image data of evacuation personnel to be identified;

The recognition unit is used for inputting the image data of the evacuees to be recognized into the trained evacuee characteristic extraction network model to obtain the recognition result of the evacuees.

8. The apparatus of claim 7, wherein the indoor smoke masks sample data of lower evacuees, comprising:

image datasets of different scene types, different human poses and different smoke types.

9. An evacuated person identification device under an indoor smoke shield, the evacuated person identification device under an indoor smoke shield comprising:

A processor;

A memory having stored thereon computer readable instructions which, when executed by the processor, implement the method of any of claims 1 to 6.

10. A computer readable storage medium having stored therein program code which is callable by a processor to perform the method of any one of claims 1 to 6.