CN113160050B - Small target identification method and system based on space-time neural network - Google Patents

Abstract

The invention discloses a small target identification method and a system based on a space-time neural network, wherein the method comprises the following steps: preprocessing an original blurred image by using a super-resolution algorithm to obtain a high-quality image sequence; performing logic subtraction operation on adjacent frames of the high-quality image sequence by using a space-time attention mechanism, capturing and highlighting a suspicious region; extracting depth features in the suspicious region to obtain a feature map time sequence; inputting the time sequence of the feature map into a mapping device of confidence output by adopting an LSTM state transition subnet to obtain a transition state; and classifying the transfer state by using a classifier to obtain a final recognition result, wherein the final recognition result is the target type and the confidence coefficient. The method is characterized in that the model is self-corrected along with continuous reading of the frame sequence, and the model is gradually corrected to be of a correct category and the confidence coefficient is continuously improved.

Description

Small target recognition method and system based on spatio-temporal neural network

technical field

The invention relates to the technical field of computer vision, in particular to a small target recognition method and system based on a spatio-temporal neural network.

Background technique

With the development of computer vision, target recognition technology has become a research hotspot and has been widely used in intelligent security, automatic driving, medical aided diagnosis and other fields. In practical applications, it is often unrealistic to require the target to be clear and easy to distinguish, which makes the small target recognition technology receive more and more attention in recent years. There are some difficulties in practical application scenarios, such as: extremely small target size, long target distance, low resolution of image source, etc., which pose a serious challenge to traditional algorithms based on single-frame image recognition.

Contemporary deep network-based pervasive target recognition algorithms basically use the mainstream deep network model as the backbone network and automatic feature extractor, and then the final recognition result is given by the classifier. Because of the use of data sets containing a large number of pictures for training, these universal algorithms can often achieve good results in the face of clearly identifiable objects, but due to the use of convolution and other operations in the backbone network to varying degrees, it will inevitably lead to convolution. The feature resolution on the channel decreases, which directly causes the performance of these algorithms to seriously degrade in the face of small target problems.

In recent years, related work has been carried out around the problem of small target recognition. One way of thinking is to improve the model itself through operations such as fusion of different sizes of features, expansion of the receptive field, and introduction of image context from the perspective of the recognition model to improve the recognition of small targets by the model. ability. Another way of thinking is to consider from the perspective of image source restoration, and restore small targets into clear and identifiable signals as much as possible through data enhancement and super-resolution processing. Although the above two types of methods have achieved certain effects, they are still far from meeting the real-time and accurate requirements in real scenes because they only work on a single frame of image.

Contents of the invention

The present invention aims to solve one of the technical problems in the related art at least to a certain extent.

Therefore, an object of the present invention is to propose a small target recognition method based on a spatio-temporal neural network.

Another object of the present invention is to propose a small target recognition system based on spatio-temporal neural network.

In order to achieve the above purpose, an embodiment of the present invention proposes a small target recognition method based on a spatio-temporal neural network, including the following steps: step S1, obtaining the original blurred image at the current moment; step S2, using a super-resolution algorithm to process the original Preprocessing the blurred image to obtain a high-quality image sequence; step S3, using the spatio-temporal attention mechanism to perform a logical subtraction operation between adjacent frames of the high-quality image sequence, capturing and highlighting suspicious areas; step S4, Extracting the depth features in the suspicious area to obtain a time-series sequence of feature maps; step S5, using the LSTM state transition subnetwork to input the time-series sequence of feature maps into a confidence output mapping device to obtain a time-series sequence of feature maps after correction; Step S6, using a classifier to classify the corrected time series of feature maps to obtain a final recognition result, wherein the final recognition result is the target category and confidence rate.

The small target recognition method based on the spatio-temporal neural network in the embodiment of the present invention solves the existing problem of recognition performance degradation caused by single-frame image target recognition. The recognition confidence rate will be gradually improved through continuous time-series image capture for a certain period of time; at the same time, with the gradual operation of the model, some erroneous conclusions obtained in the early stage will also be corrected, so that the model has a certain error correction performance.

In addition, the spatio-temporal neural network-based small target recognition method according to the above-mentioned embodiments of the present invention may also have the following additional technical features:

Further, in one embodiment of the present invention, the LSTM state transition subnetwork part adopts an important variant LSTM of the RNN cyclic neural network as a main component, wherein a complete important variant LSTM cell structure includes an input gate, an output gate, Doors and Forgotten Doors.

Further, in an embodiment of the present invention, the cell structure of a complete important variant LSTM is:

Among them, i is the input gate, f is the forget gate, o is the output gate, g is the gate, the sigmod function σ(x)=1/(1+e ^-x ), φ(x)=(e ^x -e ^{- x} )/(e ^x +e ^-x ), W is the weight matrix, B is the bias vector, x _t and h _t-1 are the current input.

Further, in one embodiment of the present invention, the transition state is:

Among them, h _t is the output state of the current sequence, t is the sequence, o is the output gate, c _t is the hidden state of the current sequence, f is the forgetting gate, c _t-1 is the hidden state of the previous sequence, and i is the input gate , g is the gate.

Further, in one embodiment of the present invention, any one of the deep convolutional models is used as the backbone network in the step S4 and the step S6.

In order to achieve the above object, another embodiment of the present invention proposes a small target recognition system based on spatio-temporal neural network, including: acquisition module, super-resolution module, spatio-temporal attention module, feature extraction module, LSTM state transition subnetwork and classification module, wherein the acquisition module is used to acquire the original blurred image at the current moment; the super-resolution module is used to preprocess the original blurred image to obtain a high-quality image sequence; the spatiotemporal attention module, It is used to perform a logical subtraction operation between adjacent frames of the high-quality image sequence, to capture and highlight suspicious areas; the feature extraction module is used to extract depth features in the suspicious areas, and obtain the sequence of feature maps Sequence; the LSTM state transfer subnetwork is used to input the time series sequence of the feature map into the mapping device of the confidence output to obtain the time series sequence of the feature map after correction; the classification module is used to process the corrected time series The time series of feature maps are classified to obtain a final recognition result, wherein the final recognition result is a category and a confidence rate.

The small target recognition system based on spatio-temporal neural network in the embodiment of the present invention solves the existing problem of recognition performance degradation caused by single-frame image target recognition. The recognition confidence rate will be gradually improved through continuous time-series image capture for a certain period of time; at the same time, with the gradual operation of the model, some erroneous conclusions obtained in the early stage will also be corrected, so that the model has a certain error correction performance.

In addition, the small target recognition system based on the spatio-temporal neural network according to the above-mentioned embodiments of the present invention may also have the following additional technical features:

Further, in one embodiment of the present invention, the LSTM state transition subnetwork part adopts an important variant LSTM of the RNN cyclic neural network as a main component, wherein a complete important variant LSTM cell structure includes an input gate, an output gate, Doors and Forgotten Doors.

Further, in an embodiment of the present invention, the cell structure of a complete important variant LSTM is:

Among them, i is the input gate, f is the forget gate, o is the output gate, g is the gate, the sigmod function σ(x)=1/(1+e ^-x ), φ(x)=(e ^x -e ^{- x} )/(e ^x +e ^-x ), W is the weight matrix, B is the bias vector, x _t and h _t-1 are the current input.

Further, in one embodiment of the present invention, the transition state is:

Among them, h _t is the output state of the current sequence, t is the sequence, o is the output gate, c _t is the hidden state of the current sequence, f is the forgetting gate, c _t-1 is the hidden state of the previous sequence, and i is the input gate , g is the gate.

Further, in one embodiment of the present invention, any deep convolution model is used as the backbone network in the feature extraction module and the classification module.

Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention.

Description of drawings

The above and/or additional aspects and advantages of the present invention will become apparent and easy to understand from the following description of the embodiments in conjunction with the accompanying drawings, wherein:

Fig. 1 is the flow chart of the small target recognition method based on spatio-temporal neural network of an embodiment of the present invention;

Fig. 2 is an intuitive schematic diagram of the relationship between attention distribution and recognition accuracy of a specific target type according to an embodiment of the present invention, wherein (a) is distraction and wrong recognition; (b) is concentration of attention and correct recognition;

Fig. 3 is a schematic diagram of the structure of an LSTM cell unit according to an embodiment of the present invention;

Fig. 4 is a schematic diagram of an ATSETC4 sample diagram of an embodiment of the present invention;

Fig. 5 is a schematic diagram of model self-correction capability according to an embodiment of the present invention;

Fig. 6 is a schematic diagram of SRGAN processing results of images of different sizes according to a specific embodiment of the present invention;

Fig. 7 is a schematic structural diagram of a small target recognition system based on a spatio-temporal neural network according to an embodiment of the present invention.

Detailed ways

Embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the figures are exemplary and are intended to explain the present invention and should not be construed as limiting the present invention.

The spatio-temporal neural network-based small target recognition method and system according to the embodiments of the present invention will be described below with reference to the accompanying drawings. First, the spatio-temporal neural network-based small target recognition method according to the present invention will be described with reference to the accompanying drawings.

Fig. 1 is a flow chart of a small target recognition method based on a spatio-temporal neural network according to an embodiment of the present invention.

As shown in Figure 1, the small target recognition method based on spatio-temporal neural network includes the following steps:

In step S1, the original blurred image at the current moment is acquired.

In step S2, a super-resolution algorithm is used to preprocess the original blurred image to obtain a high-quality image sequence.

Specifically, a super-resolution algorithm with a complete training model is used to initially enhance the original blurred image to obtain a data source with better image quality, wherein the super-resolution algorithm can refer to any effective super-resolution method, the present invention In the embodiment, SRGAN is used, and those skilled in the art can choose different super-resolution methods according to actual conditions, which are not specifically limited here.

In step S3, the spatial-temporal attention mechanism is used to perform logical subtraction between adjacent frames of the high-quality image sequence, capture and highlight suspicious areas, so that subsequent computing resources can be more accurately allocated to actual targets.

Formally, let the model's attention score Y for a certain target be the inner product of weight w and feature map A:

Among them, A is the feature map, w is the weight of the neural network model, Y is the attention distribution score of the model reasoning process, relu is the linear rectification function, is the weight gradient of the model, specifically:

In formula (2), is the gradient weighted sum of each feature element. Combine formula (1) and formula (2) to get formula (3), which is the final form of the model's attention score for fixed categories:

In fact, the distribution of attention is closely related to the correct rate of recognition. As shown in Figure 2, when misrecognition occurs, the attention of the model becomes extremely scattered. On the contrary, when the recognition is correct, the attention is almost complete with the target outline fit.

In step S4, the deep features in the suspicious area are extracted to obtain the time series of feature maps.

That is, the suspicious regions output by the spatio-temporal attention mechanism are accepted and their deep features are extracted as the input of the LSTM state transfer sub-network.

In step S5, the LSTM state transition subnetwork is used to input the time-series sequence of feature maps into the confidence output mapping device to obtain the time-series sequence of feature maps after correction.

Furthermore, the LSTM state transfer subnet part adopts LSTM, an important variant of RNN recurrent neural network, as the main component. Due to the problem of gradient disappearance, the traditional RNN cyclic neural network unit has a time limit for storing content and is not easy to train. As shown in Figure 3, LSTM is a variant of cyclic neural network specially designed to solve such problems. A complete LSTM cell structure includes input gate, output gate, gate gate and forget gate, which can transfer the current hidden state Participate in the fusion calculation at the next moment, and at the same time avoid the memory storage time limit caused by the gradient disappearance problem of the ordinary cyclic neural network. The specific formula is:

Among them, i is the input gate, f is the forget gate, o is the output gate, g is the gate, the sigmod function σ(x)=1/(1+e ^-x ), φ(x)=(e ^x -e ^{- x} )/(e ^x +e ^-x ), W is the weight matrix, B is the bias vector, x _t and h _t-1 are the current input.

The output state and hidden state calculation process is:

Among them, h _t is the output state of the current sequence, t is the sequence, o is the output gate, c _t is the hidden state of the current sequence, f is the forgetting gate, c _t-1 is the hidden state of the previous sequence, and i is the input gate , g is the gate;

Then, according to the output state and the hidden state, the time sequence of the feature map is corrected.

In step S6, a classifier is used to classify the corrected time series of feature maps to obtain a final recognition result, wherein the final recognition result is the target category and confidence rate.

Specifically, the time series output of LSTM cells (modified time series of feature maps) is input into the classifier to obtain the final classification result.

It should be noted that in step S4 and step S6, any of the mainstream deep convolutional models can be used as the backbone network. Similarly, in fact, any effective feature extractor and classifier can also be replaced. The embodiment of the present invention The feature extractor composed of the VGG backbone network is used in the method, and the explicit fully connected layer is used as the classifier, but those skilled in the art can choose different super-resolution methods according to the actual situation, and no specific limitation is made here.

In addition, in order to carry out quantitative experiments, the present invention constructs an empty small target sequence image challenge dataset ATSETC4 as the basis for deep network training, and solves the shortcoming of lack of dedicated datasets for serialized neural network training in the current field . ATSETC4 contains 2400 video clips from real capture and network resources, considering many scenarios including field, city, virtual environment and complex meteorological environment. As shown in Fig. 4, the present invention has been provided with four kinds of flight target types in ATSETC4: flying bird, hot-air balloon, fixed-wing unmanned aerial vehicle and rotary-wing unmanned aerial vehicle. At the same time, six standard size image subsets are set: 224×224, 112×112, 56×56, 28×28, 14×14, 7×7 for multi-scale comparison test (the reason for using this setting is that it needs Adapt to the parameter requirements of deep networks with fully connected layers). Specifically, the small-scale subset is down-sampled from the large-scale subset, and in the initialization stage of the large-scale subset, the original targets of different sizes are properly fused to increase the challenge of the dataset. The final ATSETC4 contains 2400 sequences, each sequence length is 25 frames, a total of 60000 frame images. Generally speaking, objects whose size is less than or equal to 28×28 can be considered as small objects.

Therefore, the specific working process of the small target recognition method proposed by the present invention can be shown in Table 1 below.

Furthermore, in the actual test, the present invention has a remarkable effect on the recognition of continuous target frames, and has a strong self-correction ability. As shown in Figure 5, the model in the early stage of the recognition process is short-lived due to reasons such as blurred signals, small targets, and complex backgrounds. With the continuous reading of frame sequences, the model performs self-correction, gradually correcting to the correct category and continuously increasing the confidence rate.

The small target recognition method based on the spatio-temporal neural network proposed by the present invention will be further described below through a specific embodiment.

First, the embodiment of the present invention uses the SRGAN super-resolution model with a trained model to directly process the image.

Next, use the cross-entropy loss function as the optimization function, and set the minimum batchsize to 16. Specifically, a sequence with 25 frames is set as the minimum batch unit. The initial learning rate is 10 ^-4 , and when the experimental accuracy of the verification set stops increasing significantly, the learning rate decreases according to the scaling factor of 100. In particular, the fully connected layer in the model is regularized using the dropout mode during training, and the dropout factor is 0.5 (that is, to prevent overfitting, a part of the fully connected layer parameters are randomly frozen). In addition, the present invention is mainly based on the pre-trained VGG11 deep convolutional network on ImageNet as a feature extractor, and the training process keeps most of the parameters of the convolutional backbone network in a frozen state. Correspondingly, ATSETC4 is split into training set and test set according to the ratio of 8:2. In the end, an average of 55 to 65 rounds of training is required on subsets of different sizes, and it takes an average of 90 minutes to complete the training of a subset model of one size. The experimental equipment is a single-card NVIDIA GTX1080Ti GPU, and the machine learning framework uses Pytorch. In the comparative experiment, default parameters are used for other models, and the ATSETC4 proposed by the present invention is still used as the benchmark in the testing stage.

In the experiment, the embodiment of the present invention sets the simplified spatio-temporal neural network as Simple_STNet (without super-resolution module), and the full version model is named STNet. Compared with some cutting-edge target recognition network performance, as shown in Table 2 below.

Table 2: Performance comparison of Simple STNet, STNet and various advanced recognition algorithms

It can be seen from Table 2 that both the simplified version of Simple STNet and the full version of STNet achieve the best performance on almost all size subsets on ATSETC4. The degradation of the full version of STNet on the 7 scale is due to the super-resolution downsampling by 32 times Under these conditions, the theoretical limit has been exceeded, and an error occurs in the image restoration process, which leads to performance degradation. As shown in Figure 6, it is the original images of different sizes and the results after SRGAN processing. The three columns from left to right are: 224-size high-definition images, 7-size low-resolution images, and 7-size SRGAN processing results.

Therefore, the small target recognition method based on the spatio-temporal neural network proposed by the embodiment of the present invention solves the problem of the recognition performance degradation caused by the single-frame image target recognition. Resources are concentrated and continuously allocated to suspicious targets in this area, and the recognition confidence rate is gradually improved through continuous time-series image capture for a certain period of time; at the same time, with the gradual operation of the model, some erroneous conclusions obtained in the early stage will also be corrected, making the model It has certain error correction performance.

Next, the small target recognition system based on spatio-temporal neural network proposed according to the embodiment of the present invention will be described with reference to the accompanying drawings.

FIG. 7 is a schematic structural diagram of a small target recognition system based on a spatio-temporal neural network according to an embodiment of the present invention.

As shown in FIG. 7 , the system 10 includes: an acquisition module 100 , a super-resolution module 200 , a spatiotemporal attention module 300 , a feature extraction module 400 , an LSTM state transfer subnetwork 500 and a classification module 600 .

Wherein, the acquisition module 100 is used to acquire the original blurred image at the current moment. The super-resolution module 200 is used to preprocess the original blurred image to obtain a high-quality image sequence. The spatio-temporal attention module 300 is used to perform a logical subtraction operation between adjacent frames of a high-quality image sequence to capture and highlight suspicious areas. The feature extraction module 400 is used to extract the deep features in the suspicious area to obtain the time series of feature maps. The LSTM state transition sub-network 500 is used to input the time series sequence of the feature map into the confidence output mapping device to obtain the transition state. The classification module 600 is used to classify transition states to obtain a final recognition result, wherein the final recognition result is a category and a confidence rate.

Further, in one embodiment of the present invention, the LSTM state transfer subnet part adopts an important variant LSTM of the RNN cyclic neural network as a main component, wherein a complete important variant LSTM cell structure includes an input gate, an output gate, a gate gate and Forgotten Gate.

Further, in one embodiment of the present invention, a complete important variant LSTM cell structure is:

Among them, i is the input gate, f is the forget gate, o is the output gate, g is the gate, the sigmod function σ(x)=1/(1+e ^-x ), φ(x)=(e ^x -e ^{- x} )/(e ^x +e ^-x ), W is the weight matrix, B is the bias vector, x _t and h _t-1 are the current input.

Further, in one embodiment of the present invention, the transfer status is:

Among them, h _t is the output state of the current sequence, t is the sequence, o is the output gate, c _t is the hidden state of the current sequence, f is the forgetting gate, c _t-1 is the hidden state of the previous sequence, and i is the input gate , g is the gate.

Optionally, in an embodiment of the present invention, any deep convolution model is used as the backbone network in the feature extraction module and the classification module.

It should be noted that the foregoing explanations of the embodiment of the small target recognition method based on the spatio-temporal neural network are also applicable to this system, and will not be repeated here.

The spatio-temporal neural network-based small target recognition system proposed according to the embodiment of the present invention solves the problem of recognition performance degradation caused by single-frame image target recognition. After roughly locking the target area, the visual capture device and computing component resources Concentrated and continuous allocation to suspicious targets in this area, through continuous time-series image capture for a certain period of time, the recognition confidence rate is gradually improved; at the same time, with the gradual operation of the model, some wrong conclusions obtained in the early stage will also be corrected, so that the model has Certain error correction performance.

In addition, the terms "first" and "second" are used for descriptive purposes only, and cannot be interpreted as indicating or implying relative importance or implicitly specifying the quantity of indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include at least one of these features. In the description of the present invention, "plurality" means at least two, such as two, three, etc., unless otherwise specifically defined.

In the description of this specification, descriptions referring to the terms "one embodiment", "some embodiments", "example", "specific examples", or "some examples" mean that specific features described in connection with the embodiment or example , structure, material or characteristic is included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms are not necessarily directed to the same embodiment or example. Furthermore, the described specific features, structures, materials or characteristics may be combined in any suitable manner in any one or more embodiments or examples. In addition, those skilled in the art can combine and combine different embodiments or examples and features of different embodiments or examples described in this specification without conflicting with each other.

Although the embodiments of the present invention have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limiting the present invention, those skilled in the art can make the above-mentioned The embodiments are subject to changes, modifications, substitutions and variations.

Claims (4)

1. a small target recognition method based on spatio-temporal neural network, is characterized in that, comprises the following steps: Step S1, obtaining the original blurred image at the current moment; Step S2, using a super-resolution algorithm to preprocess the original blurred image to obtain a high-quality image sequence; Step S3, using the spatio-temporal attention mechanism to perform a logical subtraction operation between adjacent frames of the high-quality image sequence, capturing and highlighting suspicious areas; Let the model's attention score Y for a certain target be the inner product of the weight w and the feature map A: Among them, A is the feature map featuremap, w is the weight of the neural network model, Y is the attention distribution score of the model reasoning process, relu is the linear rectification function, is the weight gradient of the model, specifically: In formula (2), it is the gradient weighted sum of each feature element, combining formula (1) and formula (2) to get formula (3), which is the final form of the model's attention score for fixed categories: Step S4, extracting the depth features in the suspicious area to obtain a time series of feature maps; Step S5, using the LSTM state transfer subnetwork to input the feature map time series sequence into the confidence output mapping device to obtain the corrected feature map time series sequence, the LSTM state transfer subnetwork part adopts an important variant of the RNN cyclic neural network LSTM is used as the main component, wherein, a complete important variant LSTM cell structure includes an input gate, an output gate, a gate gate and a forgetting gate, and the complete important variant LSTM cell structure is: Among them, i is the input gate, f is the forget gate, o is the output gate, g is the gate, the sigmod function σ(x)=1/(1+e ^-x ), φ(x)=(e ^x -e ^{- x} )/(e ^x +e ^-x ), W is the weight matrix, B is the bias vector, x _t and h _t-1 are the current input; The transfer status is: Among them, h _t is the output state of the current sequence, t is the sequence, o is the output gate, c _t is the hidden state of the current sequence, f is the forgetting gate, c _t-1 is the hidden state of the previous sequence, and i is the input gate , g is the gate; Step S6, using a classifier to classify the corrected time series of feature maps to obtain a final recognition result, wherein the final recognition result is the target category and confidence rate. 2. The small target recognition method based on spatio-temporal neural network according to claim 1, characterized in that, any one of the deep convolution models is used as the backbone network in the step S4 and the step S6. 3. A small target recognition system based on spatio-temporal neural network, characterized in that, comprising: acquisition module, super-resolution module, spatio-temporal attention module, feature extraction module, LSTM state transfer subnetwork and classification module, wherein said An acquisition module, configured to acquire the original blurred image at the current moment; The super-resolution module is used to preprocess the original blurred image to obtain a high-quality image sequence; The spatio-temporal attention module is used to perform a logical subtraction operation between adjacent frames of the high-quality image sequence to capture and highlight suspicious areas; Let the model's attention score Y for a certain target be the inner product of the weight w and the feature map A: Among them, A is the feature map featuremap, w is the weight of the neural network model, Y is the attention distribution score of the model reasoning process, relu is the linear rectification function, is the weight gradient of the model, specifically: In formula (2), it is the gradient weighted sum of each feature element, combining formula (1) and formula (2) to get formula (3), which is the final form of the model's attention score for fixed categories: The feature extraction module is used to extract the depth features in the suspicious area to obtain a time series of feature maps; The LSTM state transfer subnetwork is used to input the time series sequence of the feature map into the mapping device of the confidence output, and obtain the time series sequence of the feature map after correction, wherein the LSTM state transfer subnetwork part adopts RNN cyclic neural network The important variant of LSTM is used as the main component, and a complete important variant of LSTM cell structure includes input gate, output gate, gate gate and forget gate; A complete important variant of the LSTM cell structure is: Among them, i is the input gate, f is the forget gate, o is the output gate, g is the gate, the sigmod function σ(x)=1/(1+e ^-x ), φ(x)=(e ^x -e ^{- x} )/(e ^x +e ^-x ), W is the weight matrix, B is the bias vector, x _t and h _t-1 are the current input; The transfer status is: Among them, h _t is the output state of the current sequence, t is the sequence, o is the output gate, c _t is the hidden state of the current sequence, f is the forgetting gate, c _t-1 is the hidden state of the previous sequence, and i is the input gate , g is the gate; The classification module is configured to classify the corrected time series of feature maps to obtain a final recognition result, wherein the final recognition result is a category and a confidence rate. 4. The small target recognition system based on spatio-temporal neural network according to claim 3, characterized in that any deep convolution model is used as the backbone network in the feature extraction module and the classification module.