CN118297984A - Multi-target tracking method and system for smart city camera - Google Patents

Abstract

The invention relates to the technical field of computer vision, in particular to a multi-target tracking method and system for a smart city camera, comprising the following steps of: based on the monitoring image, the depth separation convolutional neural network and the scale invariant feature transformation are adopted to perform image processing and preliminary target recognition, and preliminary detection of targets is performed. In the invention, the feature pyramid network and the attention mechanism accurately identify targets with different sizes, enhance multi-scale feature extraction and fusion, adjust and optimize target tracking flexibility and accuracy by self-adaptive scale, solve target shielding by fusion of a mask region convolution neural network and monocular depth estimation, enhance multi-camera data integration by a graph convolution neural network, improve space-time correlation construction efficiency, combine a time convolution neural network with a long-short-term memory neural network, and use a single support vector machine for abnormal behavior identification and early warning, improve behavior analysis accuracy and early warning timeliness, and enhance response capability and event processing efficiency of a smart city monitoring system.

Description

Smart city camera multi-target tracking method and system

Technical Field

The present invention relates to the field of computer vision technology, and in particular to a multi-target tracking method and system for smart city cameras.

Background technique

Computer vision is a field of science and engineering that studies how to enable computers to understand and interpret images or videos. It involves the use of algorithms and models to process, analyze, and understand digital image or video data to identify, track, and understand objects, actions, and events in a scene.

Among them, the smart city camera multi-target tracking method aims to track multiple targets using the smart city camera system. The purpose of this method is to automatically detect, track and identify multiple targets by analyzing the moving objects in the video surveillance screen, thereby providing information about the target location, motion trajectory and behavior pattern. In order to achieve this effect, this method is generally achieved through means such as target detection, target tracking, multi-target tracking and behavior analysis. These technical means are usually based on algorithms and methods in related fields such as deep learning, image processing and pattern recognition, and improve the accuracy and robustness of tracking by training a large amount of labeled data.

In the existing smart city camera multi-target tracking methods, the existing methods often lack effective multi-scale feature fusion and adaptive scale adjustment mechanisms, resulting in limited recognition capabilities when processing multi-size targets, and low tracking accuracy for long-distance or small-size targets. In addition, traditional methods often lack effective deep information analysis tools when dealing with target occlusion problems, making it difficult to accurately obtain occlusion relationships, resulting in insufficient understanding of the relationship between targets. In terms of multi-camera data integration, existing methods often cannot fully utilize information from different perspectives due to the lack of efficient network structures, affecting the establishment of spatiotemporal associations. At the same time, in terms of behavior learning and early warning, traditional methods are often limited to surface features and ignore deep-level pattern recognition, making the early warning of abnormal behavior not timely or accurate enough, limiting the response speed of the overall monitoring system and the ability to handle complex events.

Summary of the invention

The purpose of the present invention is to solve the shortcomings of the prior art and propose a smart city camera multi-target tracking method and system.

In order to achieve the above object, the present invention adopts the following technical solution: a multi-target tracking method of a smart city camera, comprising the following steps:

S1: Based on the surveillance image, a deep separation convolutional neural network and scale-invariant feature transformation are used for image processing and preliminary target recognition, and preliminary target detection is performed to generate preliminary target recognition information;

S2: Based on the preliminary target recognition information, a feature pyramid network and an attention mechanism are used to perform multi-scale feature fusion, and the recognition capability of multi-scale targets is enhanced to generate multi-scale target feature information;

S3: Based on the multi-scale target feature information, adaptive scale adjustment is performed using a scale estimation module and an anchor frame mechanism to optimize the size of the target tracking frame and generate tracking information after adaptive scale adjustment;

S4: Based on the tracking information after the adaptive scale adjustment, a mask area convolutional neural network and a monocular depth estimation algorithm are used to process the target occlusion problem, obtain the occlusion relationship between targets and the scene depth information, and generate the target occlusion relationship and depth information;

S5: Based on the target occlusion relationship and depth information, the data of multiple camera perspectives are integrated through a graph convolutional neural network, and the spatiotemporal correlation of the target in the field of view of multiple cameras is established to generate cross-camera target tracking information;

S6: Based on the cross-camera target tracking information, the target pattern is learned by combining the temporal convolutional neural network and the long short-term memory neural network, and the abnormal behavior is identified and warned by using a single-class support vector machine, and an abnormal behavior identification and warning report is generated;

The preliminary target recognition information includes the position, size and shape information of the target. The multi-scale target feature information is specifically a multi-level, multi-size target feature description, including the details and structural information of the target at multiple scales. The tracking information after adaptive scale adjustment is specifically the tracking frame and feature information of the matching target at multiple scales. The target occlusion relationship and depth information specifically refers to the recognition information of the occluded target and the depth position of the target in the scene. The cross-camera target tracking information includes the position, time association and behavior information of key targets between multiple cameras. The abnormal behavior recognition and warning report is specifically the discovered illegal behavior or abnormal behavior, and its time, place and nature are described and warned.

As a further solution of the present invention, based on the monitoring image, a deep separation convolutional neural network and a scale-invariant feature transformation are used to perform image processing and preliminary target recognition, and preliminary detection of the target is performed. The steps of generating preliminary target recognition information are specifically as follows:

S101: Based on the original monitoring image, histogram equalization and Gaussian blur algorithm are used to perform image enhancement and noise reduction to generate an enhanced image;

S102: Based on the enhanced image, a depthwise separable convolution algorithm is used to extract basic features, reduce the amount of calculation and maintain performance, and generate a basic feature map;

S103: Based on the basic feature map, multi-scale spatial pyramid pooling is used to enhance scale-invariant feature extraction to generate a vector feature map;

S104: Based on the vector feature map, a region proposal network is used in combination with bounding box regression to perform preliminary detection of the target, accurately locate the target, and generate preliminary target recognition information.

As a further solution of the present invention, based on the preliminary target recognition information, a feature pyramid network and an attention mechanism are used to perform multi-scale feature fusion, and the recognition capability of multi-size targets is enhanced. The steps of generating multi-scale target feature information are specifically as follows:

S201: Based on the preliminary target recognition information, a feature pyramid network is used to achieve fusion between feature layers, enhance the detection capability of multi-scale targets, and generate a fusion feature map;

S202: Based on the fused feature map, an attention mechanism is used to enhance the expressiveness of features and generate an attention-weighted feature map;

S203: Based on the attention weighted feature map, an anchor frame optimization algorithm is used to optimize and adjust the anchor frame, so as to improve the fitting accuracy of the model for multi-size objects and generate optimized anchor frame feature information;

S204: Based on the optimized anchor frame feature information, features are refined through a cascaded convolutional network to enhance the representation of multi-scale features and generate multi-scale target feature information.

As a further solution of the present invention, based on the multi-scale target feature information, the scale estimation module and the anchor frame mechanism are used to perform adaptive scale adjustment, optimize the size of the target tracking frame, and generate the tracking information after adaptive scale adjustment in the following steps:

S301: Based on the multi-scale target feature information, a real-time scale estimation algorithm is used to dynamically predict the target size, adapt to changes in the target scale, and generate scale adjustment information;

S302: Based on the scale adjustment information, an anchor frame generation algorithm is used to adjust the size of the tracking frame so that the tracking frame fits the actual scale of the target, and a tracking frame with an adaptive scale is generated;

S303: Based on the tracking frame of the adaptive scale, a non-maximum suppression algorithm is used to eliminate overlapping tracking frames, reduce redundancy, and generate a non-maximum suppressed tracking frame;

S304: Based on the non-maximum suppressed tracking frame, the final target tracking frame is located in combination with a tracking learning algorithm to generate tracking information after adaptive scale adjustment.

As a further solution of the present invention, based on the tracking information after the adaptive scale adjustment, the mask area convolutional neural network and the monocular depth estimation algorithm are used to process the target occlusion problem, obtain the occlusion relationship between targets and the scene depth information, and generate the target occlusion relationship and depth information in the following steps:

S401: Based on the tracking information after the adaptive scale adjustment, a mask region convolutional neural network enhanced by a feature pyramid network is used to perform target instance segmentation, and occlusion boundaries are accurately determined to generate target segmentation and occlusion boundary information;

S402: Based on the target segmentation and occlusion boundary information, a monocular depth estimation method based on deep learning is used to perform three-dimensional reconstruction of the scene to generate a scene depth map;

S403: Based on the scene depth map, pixel-level fusion technology is used to analyze the occlusion relationship between objects and generate an object occlusion relationship map;

S404: Based on the target occlusion relationship graph, combined with the depth sorting algorithm and the occlusion processing technology, the occlusion level and depth between the targets are determined, and the target occlusion relationship and depth information are generated.

As a further solution of the present invention, based on the target occlusion relationship and depth information, the data of multiple camera perspectives are integrated through a graph convolutional neural network, and the spatiotemporal association of the target in the multi-camera field of view is established, and the steps of generating cross-camera target tracking information are specifically as follows:

S501: Based on the target occlusion relationship and depth information, a graph convolutional neural network is applied, and feature extraction and optimization strategies are combined to integrate multi-view data to generate preliminary cross-camera target association information;

S502: Based on the preliminary cross-camera target association information, a multi-camera geometric calibration method is used to refine the target position relationship through a calibration technology based on epipolar constraints to generate multi-view target position information;

S503: Based on the multi-view target position information, using time series analysis technology, performing time series data analysis through dynamic time warping, optimizing target spatiotemporal association, and generating time-optimized target association information;

S504: Based on the time-optimized target association information, the multi-camera data is integrated, and the spatiotemporal association is completed through a multi-view tracking fusion algorithm to generate cross-camera target tracking information.

As a further solution of the present invention, based on the cross-camera target tracking information, the target pattern is learned by combining the temporal convolutional neural network and the long short-term memory neural network, and the abnormal behavior is identified and warned by using a single-class support vector machine. The steps of generating abnormal behavior identification and warning reports are specifically as follows:

S601: Based on the cross-camera target tracking information, deep feature learning is performed in combination with a temporal convolutional neural network and a long short-term memory network to generate a target behavior feature learning result;

S602: Based on the target behavior feature learning result, a feature clustering algorithm is used to perform spatiotemporal feature analysis, extract the target behavior pattern, and generate spatiotemporal clustered behavior features;

S603: Based on the behavior characteristics after the spatiotemporal clustering, using a single-class support vector machine to identify and model abnormal behaviors, and generate an abnormal behavior identification model;

S604: Based on the abnormal behavior recognition model and in combination with real-time monitoring data, abnormal behavior warning is performed to generate an abnormal behavior recognition and warning report.

A smart city camera multi-target tracking system, the smart city camera multi-target tracking system is used to execute the above-mentioned smart city camera multi-target tracking method, the system includes an image preprocessing module, a basic feature extraction module, a preliminary target recognition module, a feature enhancement and optimization module, a target tracking frame preprocessing module, a target association module, and a behavior learning and early warning module.

As a further solution of the present invention, the image preprocessing module uses histogram equalization and Gaussian blur algorithm based on the original monitoring image to perform image enhancement and noise reduction to generate an enhanced image;

The basic feature extraction module extracts basic features based on the enhanced image, adopts a depth-separable convolution algorithm to reduce the amount of calculation and maintain performance, and generates a basic feature map;

The preliminary target recognition module performs preliminary target recognition based on the basic feature map, adopts multi-scale spatial pyramid pooling to enhance scale-invariant feature extraction, and adopts a region proposal network combined with bounding box regression to locate the target and generate preliminary target recognition information;

The feature enhancement and optimization module performs feature enhancement and optimization based on preliminary target recognition information, uses a feature pyramid network to achieve fusion between feature layers and adopts an attention mechanism to enhance the expressiveness of features, optimizes and adjusts the frame by using an anchor frame optimization algorithm, and extracts features through a cascaded convolutional network to generate multi-scale target feature information;

The target tracking frame preprocessing module processes the tracking frame based on multi-scale target feature information, uses a real-time scale estimation algorithm to dynamically predict the target size, adjusts the tracking frame size through an anchor frame generation algorithm, and combines a non-maximum suppression algorithm to eliminate overlapping tracking frames, thereby generating tracking information after adaptive scale adjustment;

The target association module runs a target association algorithm based on the tracking information after adaptive scale adjustment, uses a mask area convolutional neural network enhanced by a feature pyramid network to perform target instance segmentation, reconstructs the scene in three dimensions through a deep learning monocular depth estimation method, uses pixel-level fusion technology for processing, and determines the occlusion level and depth between targets through a depth sorting algorithm and occlusion processing technology, and generates target occlusion relationships and depth information;

The behavior learning and warning module performs behavior learning and warning based on target occlusion relationship and depth information, uses temporal convolutional neural network and long short-term memory network for deep feature learning, performs spatiotemporal feature analysis through feature clustering algorithm, uses single-class support vector machine to identify and model abnormal behavior, combines real-time monitoring data for abnormal behavior warning, and generates abnormal behavior identification and warning report.

As a further solution of the present invention, the image preprocessing module includes a noise reduction submodule, an image enhancement submodule, an abnormality filtering submodule, and an image normalization submodule;

The basic feature extraction module includes a deep convolution submodule, a feature extraction submodule, a feature compression submodule, and a feature noise screening submodule;

The preliminary target recognition module includes a target positioning submodule, a target detection submodule, a target screening submodule, and a target confirmation submodule;

The feature enhancement and optimization module includes a feature fusion submodule, a feature attention weighting submodule, an anchor frame optimization submodule, and a feature extraction submodule;

The target tracking frame preprocessing module includes a real-time scale estimation submodule, an anchor frame generation submodule, a non-maximum suppression submodule, and a target tracking frame positioning submodule;

The target association module includes a target instance segmentation submodule, a scene 3D reconstruction submodule, a target occlusion relationship analysis submodule, and an occlusion level and depth determination submodule;

The behavior learning and warning module includes a deep feature learning submodule, a spatiotemporal feature analysis submodule, an abnormal behavior recognition submodule, and a real-time abnormal warning submodule.

Compared with the prior art, the advantages and positive effects of the present invention are:

In the present invention, by integrating the deep separation convolutional neural network and the scale-invariant feature conversion, the accuracy of image processing is effectively improved, and the ability of preliminary target recognition is enhanced. The application of feature pyramid network and attention mechanism makes the recognition of targets of different sizes more accurate, and enhances the extraction and fusion of multi-scale target features. Adaptive scale adjustment optimizes the size of the tracking frame through the scale estimation module and the anchor frame mechanism, improving the flexibility and accuracy of target tracking. The fusion of mask area convolutional neural network and monocular depth estimation algorithm provides an effective means to solve the problem of target occlusion and ensures the accurate acquisition of scene depth information. The introduction of graph convolutional neural network strengthens the integration ability of multi-camera data and improves the efficiency of building spatiotemporal association. Finally, the combination of time convolutional neural network and long short-term memory neural network, as well as the use of single-class support vector machine in abnormal behavior recognition and early warning, greatly improves the accuracy of behavior analysis and the timeliness of early warning, and significantly enhances the overall responsiveness and event processing efficiency of smart city monitoring system to complex scenes.

BRIEF DESCRIPTION OF THE DRAWINGS

Fig. 1 is a schematic diagram of the workflow of the present invention;

FIG2 is a flow chart of the refinement of S1 of the present invention;

FIG3 is a flow chart of the refinement of S2 of the present invention;

FIG4 is a flow chart of the refinement of S3 of the present invention;

FIG5 is a flow chart of the refinement of S4 of the present invention;

FIG6 is a flow chart of the refinement of S5 of the present invention;

FIG7 is a detailed flow chart of S6 of the present invention;

FIG8 is a system flow chart of the present invention;

FIG. 9 is a schematic diagram of a system framework of the present invention.

Detailed ways

In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

In the description of the present invention, it should be understood that the terms "length", "width", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, in the description of the present invention, "plurality" means two or more, unless otherwise clearly and specifically defined.

Embodiment 1

Please refer to FIG1 . The present invention provides a technical solution: a multi-target tracking method of a smart city camera, comprising the following steps:

S1: Based on the surveillance image, a deep separation convolutional neural network and scale-invariant feature transformation are used for image processing and preliminary target recognition, and preliminary target detection is performed to generate preliminary target recognition information;

S2: Based on the preliminary target recognition information, the feature pyramid network and attention mechanism are used to fuse multi-scale features, enhance the recognition ability of multi-size targets, and generate multi-scale target feature information;

S3: Based on the multi-scale target feature information, the scale estimation module and anchor frame mechanism are used to perform adaptive scale adjustment, optimize the size of the target tracking frame, and generate tracking information after adaptive scale adjustment;

S4: Based on the tracking information after adaptive scale adjustment, the mask region convolutional neural network and monocular depth estimation algorithm are used to deal with the target occlusion problem, obtain the occlusion relationship between targets and scene depth information, and generate target occlusion relationship and depth information;

S5: Based on the target occlusion relationship and depth information, the graph convolutional neural network is used to integrate the data from multiple camera perspectives, and the spatiotemporal correlation of the target in the multi-camera field of view is established to generate cross-camera target tracking information;

S6: Based on cross-camera target tracking information, the target pattern is learned by combining the temporal convolutional neural network and the long short-term memory neural network, and a single-class support vector machine is used to identify and warn abnormal behaviors, and generate abnormal behavior identification and warning reports;

Preliminary target recognition information includes the position, size and shape information of the target. Multi-scale target feature information is specifically a multi-level, multi-size target feature description, including the details and structural information of the target at multiple scales. The tracking information after adaptive scale adjustment is specifically the tracking frame and feature information of the matching target at multiple scales. The target occlusion relationship and depth information specifically refers to the recognition information of the occluded target and the depth position of the target in the scene. The cross-camera target tracking information includes the position, time correlation and behavior information of key targets between multiple cameras. The abnormal behavior recognition and warning report is specifically the discovered illegal or abnormal behavior, and its time, place and nature are described and warned.

By using the smart city camera system for multi-target tracking, multiple targets can be monitored and identified in real time, providing rich information on target locations, motion trajectories, and behavior patterns. This method uses deep separation convolutional neural networks and scale-invariant feature transformation to improve tracking accuracy. At the same time, multi-scale feature fusion is performed through feature pyramid networks and attention mechanisms to enhance the recognition ability of multi-size targets. In order to solve the problem of target occlusion, this method uses mask region convolutional neural networks and monocular depth estimation algorithms to process target occlusion relationships and scene depth information. In addition, the graph convolutional neural network integrates data from multiple camera perspectives to achieve cross-camera target tracking and obtain more comprehensive target information. Finally, the temporal convolutional neural network and long short-term memory neural network are combined to learn the target pattern, and a single-class support vector machine is used to identify and warn abnormal behaviors.

Please refer to Figure 2. Based on the surveillance image, the deep separation convolutional neural network and scale-invariant feature transformation are used to perform image processing and preliminary target recognition, and preliminary target detection is performed. The specific steps of generating preliminary target recognition information are as follows:

S101: Based on the original monitoring image, histogram equalization and Gaussian blur algorithm are used to perform image enhancement and noise reduction to generate an enhanced image;

S102: Based on the enhanced image, a depth-separable convolution algorithm is used to extract basic features, reduce the amount of calculation and maintain performance, and generate a basic feature map;

S103: Based on the basic feature map, multi-scale spatial pyramid pooling is used to enhance scale-invariant feature extraction to generate a vector feature map;

S104: Based on the vector feature map, a region proposal network is used in combination with bounding box regression to perform preliminary target detection, accurately locate the target, and generate preliminary target recognition information.

First, the original surveillance image is processed by histogram equalization and Gaussian blur algorithm to enhance the contrast and clarity of the image and reduce the influence of noise. This can generate an enhanced image and provide better input for subsequent target recognition.

Next, the enhanced image is subjected to feature extraction using the depthwise separable convolution algorithm. This algorithm decomposes the traditional convolution operation into two steps: depthwise convolution and pointwise convolution, which reduces the amount of computation while maintaining good performance. Through this step, a basic feature map containing basic feature information can be generated.

In order to enhance the recognition ability of objects of different scales, a multi-scale spatial pyramid pooling method is used to process the basic feature map. This method extracts scale-invariant features from feature maps at different levels and fuses them together to generate a vector feature map. This can obtain a richer and more discriminative feature representation.

Based on the generated vector feature map, the region proposal network is used in combination with bounding box regression to perform preliminary detection of the target. The region proposal network can generate a series of candidate boxes, and these candidate boxes are adjusted and optimized through bounding box regression to accurately locate the position and size of the target. Through this step, preliminary target recognition information can be generated, including key information such as the position, size and shape of the target.

In the present invention, after the preliminary target identification information is generated, in order to better lock the preliminary target identification information and improve the accuracy of the selected target, it is necessary to calculate the accuracy of the preliminary target selected in the candidate frame based on the preliminary target identification information, that is, calculate the possibility that the preliminary target selected is the final target, and judge it through a target matching index.

Specifically, the present invention further comprises the following steps:

S104a: Based on the preliminary target recognition information, obtain the position, size and shape of the target;

S104b: performing comparisons in a preset target database according to the position of the target, the size of the target, and the shape of the target, respectively, to obtain the similarity of the position item of the target, the similarity of the size item of the target, and the similarity of the shape item of the target;

S104c: Calculate the target matching index of the preliminary target according to the target position item similarity, the target size item similarity and the target shape item similarity.

In this step, the calculation formula of the target matching index of the preliminary target is expressed as:

W＝W ₀ +η _l ·w _l +η _d ·w _d +η _s ·w _s ;

Among them, W represents the target fit index of the preliminary target, _W0 represents the baseline value of the target fit index of the preliminary target, _ηl represents the similarity of the target's position item, _wl represents the weight factor of the target's position item, _ηd represents the similarity of the target's size item, _wd represents the weight factor of the target's size item, _ηs represents the similarity of the target's shape item, and _ws represents the weight factor of the target's shape item.

It can be understood that by comparing the calculated target match index of the preliminary target with a preset target match index threshold, preliminary targets with very low probabilities can be screened out, thereby increasing the possibility that the preliminary target is the final target.

Please refer to Figure 3. Based on the preliminary target recognition information, the feature pyramid network and attention mechanism are used to fuse multi-scale features and enhance the recognition ability of multi-size targets. The specific steps of generating multi-scale target feature information are as follows:

S201: Based on the preliminary target recognition information, a feature pyramid network is used to achieve fusion between feature layers, enhance the detection capability of multi-scale targets, and generate a fusion feature map;

S202: Based on the fused feature map, an attention mechanism is used to enhance the expressiveness of the features and generate an attention-weighted feature map;

S203: Based on the attention weighted feature map, an anchor frame optimization algorithm is used to optimize and adjust the anchor frame, improve the fitting accuracy of the model for multi-size targets, and generate optimized anchor frame feature information;

S204: Based on the optimized anchor frame feature information, features are refined through a cascaded convolutional network to enhance the representation of multi-scale features and generate multi-scale target feature information.

First, a feature pyramid network is constructed based on the preliminary target recognition information. Feature pyramid is a multi-scale feature representation method, which usually includes feature maps of multiple resolutions. A convolutional neural network (CNN) is used to extract features of each resolution and ensure that these features have the same semantic information. In this way, feature representations of different scales can be obtained, thereby enhancing the detection ability of multi-sized targets. The generated fused feature map will contain information from different scales.

Based on the fused feature map, an attention mechanism is introduced to enhance the expressiveness of features. With the attention mechanism, the model can automatically focus on the most important parts for target recognition. The attention weights for each position are calculated, and then these weights are applied to the fused feature map to generate an attention-weighted feature map. This helps the model better focus on key areas and improves the recognition of multi-sized targets.

The generated attention-weighted feature map is optimized using the anchor box optimization algorithm. Anchor boxes are predefined bounding boxes used for object detection, and their size and position are critical for objects of different sizes. By optimizing the anchor boxes, the model's fitting accuracy for objects of multiple sizes can be improved. This step involves adjusting the size and position of the anchor boxes to better adapt them to the sizes and shapes of different objects.

Based on optimizing the anchor box feature information, a cascaded convolutional network is introduced to further refine the features and enhance the representation ability of multi-scale features. The cascaded convolutional network is a stack of convolutional layers to capture higher-level semantic information. This helps to further improve the recognition performance of multi-scale objects. The generated multi-scale object feature information will include information from each step, providing a more comprehensive object representation.

Please refer to FIG. 4 . Based on the multi-scale target feature information, the scale estimation module and the anchor frame mechanism are used to perform adaptive scale adjustment, optimize the size of the target tracking frame, and generate the tracking information after adaptive scale adjustment. Specifically, the steps are as follows:

S301: Based on multi-scale target feature information, a real-time scale estimation algorithm is used to dynamically predict the target size, adapt to changes in the target scale, and generate scale adjustment information;

S302: Based on the scale adjustment information, an anchor frame generation algorithm is used to adjust the size of the tracking frame so that the tracking frame fits the actual scale of the target, and a tracking frame with an adaptive scale is generated;

S303: Based on the adaptive scale tracking frame, a non-maximum suppression algorithm is used to eliminate overlapping tracking frames, reduce redundancy, and generate a non-maximum suppressed tracking frame;

S304: Based on the tracking frame after non-maximum suppression, the final target tracking frame is located in combination with the tracking learning algorithm to generate tracking information after adaptive scale adjustment.

Based on the multi-scale target feature information, a real-time scale estimation algorithm is used to dynamically predict the target size. The algorithm can predict the size change of the target in the next frame based on the target feature information of the current frame and the target feature information of the historical frame. Through this step, scale adjustment information can be generated for subsequent adaptive scale adjustment.

Based on the generated scale adjustment information, an anchor box generation algorithm is used to adjust the size of the tracking box. The algorithm adjusts the size of the tracking box to a size that matches the actual scale of the target based on the scale adjustment information. Through this step, an adaptive scale tracking box can be generated to better adapt to changes in the target scale.

Based on the generated adaptive scale tracking frame, the non-maximum suppression algorithm is used to eliminate overlapping tracking frames. The algorithm compares the confidence of different tracking frames, retains the tracking frame with the highest confidence, and suppresses other overlapping tracking frames. Through this step, redundant tracking frames can be reduced and the accuracy and efficiency of tracking can be improved.

Based on the tracking frame after non-maximum suppression, the final target tracking frame is located in combination with the tracking learning algorithm. The algorithm uses the target position information and motion information between the previous and next frames to further optimize and locate the tracking frame. Through this step, adaptively scaled tracking information can be generated, including key information such as the target's position, size, and shape.

Please refer to Figure 5. Based on the tracking information after adaptive scale adjustment, the mask area convolutional neural network and the monocular depth estimation algorithm are used to deal with the target occlusion problem, obtain the occlusion relationship between targets and the scene depth information, and the specific steps of generating the target occlusion relationship and depth information are as follows:

S401: Based on the tracking information after adaptive scale adjustment, a mask region convolutional neural network enhanced by a feature pyramid network is used to perform target instance segmentation, and occlusion boundaries are accurately determined to generate target segmentation and occlusion boundary information;

S402: Based on the target segmentation and occlusion boundary information, a monocular depth estimation method based on deep learning is used to perform three-dimensional reconstruction of the scene and generate a scene depth map;

S403: Based on the scene depth map, pixel-level fusion technology is used to analyze the occlusion relationship between targets and generate a target occlusion relationship map;

S404: Based on the target occlusion relationship graph, combined with the depth sorting algorithm and the occlusion processing technology, the occlusion level and depth between the targets are determined, and the target occlusion relationship and depth information are generated.

Based on the tracking information after adaptive scale adjustment, the mask region convolutional neural network enhanced by the feature pyramid network is used to segment the target instance. This algorithm can accurately determine the boundary between the target and the background and generate target segmentation and occlusion boundary information. Through this step, the target can be accurately segmented and the location of the occlusion boundary can be determined.

Based on the generated target segmentation and occlusion boundary information, the scene is reconstructed in 3D using a deep learning monocular depth estimation method. This method can infer the depth information of objects in the scene by analyzing the pixel information in the image and generate a scene depth map. Through this step, the relative position and depth information of each object in the scene can be obtained.

Based on the generated scene depth map, the pixel-level fusion technology is used to analyze the occlusion relationship between targets. This technology can determine the occlusion relationship between targets based on the depth information in the scene depth map and generate a target occlusion relationship map. Through this step, the occlusion situation between targets can be accurately described.

Based on the generated target occlusion relationship graph, the depth sorting algorithm and occlusion processing technology are combined to determine the occlusion level and depth between targets. The algorithm can sort the targets according to the information in the target occlusion relationship graph and determine the occlusion level and depth of each target. Through this step, a detailed description of the occlusion relationship between targets and the scene depth information can be obtained.

Please refer to Figure 6. Based on the target occlusion relationship and depth information, the graph convolutional neural network is used to integrate the data of multiple camera perspectives, and the spatiotemporal correlation of the target in the multi-camera field of view is established. The specific steps for generating cross-camera target tracking information are as follows:

S501: Based on the target occlusion relationship and depth information, a graph convolutional neural network is applied, combined with feature extraction and optimization strategies to integrate multi-view data and generate preliminary cross-camera target association information;

S502: Based on the preliminary cross-camera target association information, a multi-camera geometric calibration method is used to refine the target position relationship through a calibration technology based on epipolar constraints to generate multi-view target position information;

S503: Based on the multi-view target position information, using time series analysis technology, performing time series data analysis through dynamic time warping, optimizing the target spatiotemporal association, and generating time-optimized target association information;

S504: Based on the time-optimized target association information, the multi-camera data is integrated, and the spatiotemporal association is completed through a multi-view tracking fusion algorithm to generate cross-camera target tracking information.

Based on the target occlusion relationship and depth information, a graph convolutional neural network is used in combination with feature extraction and optimization strategies to integrate multi-view data. This algorithm can fuse the view data from different cameras and generate preliminary cross-camera target association information. Through this step, the view information of multiple cameras can be fully utilized to improve the accuracy and robustness of target tracking.

Based on the generated preliminary cross-camera target association information, a multi-camera geometric calibration method is used to refine the target position relationship through the calibration technology based on epipolar constraints to generate multi-view target position information. This method can further improve the accuracy and consistency of target position by calibrating the geometric relationship between multiple cameras.

Based on the generated multi-view target position information, time series analysis technology is used to analyze time series data through dynamic time warping, optimize the target time-space association, and generate time-optimized target association information. The algorithm can model and predict the target's motion trajectory based on the information in the time dimension, further improving the accuracy and stability of target tracking.

Based on the generated time-optimized target association information, the multi-camera data is integrated, and the spatiotemporal association is completed through the multi-view tracking fusion algorithm to generate cross-camera target tracking information. This algorithm can fuse the target association information from different cameras to obtain the final cross-camera target tracking result. Through this step, global tracking and association analysis of targets under multiple cameras can be achieved.

Please refer to Figure 7. Based on the cross-camera target tracking information, the target pattern is learned by combining the temporal convolutional neural network and the long short-term memory neural network, and the abnormal behavior is identified and warned using a single-class support vector machine. The specific steps of generating the abnormal behavior identification and warning report are as follows:

S601: Based on the cross-camera target tracking information, a temporal convolutional neural network and a long short-term memory network are combined to perform deep feature learning to generate target behavior feature learning results;

S602: Based on the target behavior feature learning result, a feature clustering algorithm is used to perform spatiotemporal feature analysis, extract the target behavior pattern, and generate spatiotemporal clustered behavior features;

S603: Based on the behavior characteristics after spatiotemporal clustering, a single-class support vector machine is used to identify and model abnormal behaviors to generate an abnormal behavior identification model;

S604: Based on the abnormal behavior recognition model and in combination with real-time monitoring data, abnormal behavior warning is performed to generate an abnormal behavior recognition and warning report.

In this embodiment, based on the above abnormal behavior recognition model, the monitoring data will be analyzed and classified, and the corresponding abnormal behavior comprehensive value will be calculated by calculating the abnormal index corresponding to each type of abnormal behavior. Specifically, the calculation formula of the abnormal behavior comprehensive value is expressed as:

Where Y represents the comprehensive value of abnormal behavior, _Y0 represents the reference value of the comprehensive value of abnormal behavior, _λi represents the weight factor of the i-th abnormal behavior, _εi represents the calibration factor of the i-th abnormal behavior, _yi represents the abnormal index of the i-th abnormal behavior, represents the baseline anomaly index of the i-th type of abnormal behavior, and I represents the maximum number of categories of abnormal behavior.

It can be understood that when the calculated comprehensive value of abnormal behavior is greater than the preset abnormal behavior threshold, an early warning can be triggered to assist in discovering dangerous situations as early as possible and making emergency plans in a timely manner.

Based on cross-camera target tracking information, deep feature learning is performed by combining temporal convolutional neural networks and long short-term memory networks. This algorithm can learn and extract the target's behavior patterns and generate target behavior feature learning results. Through this step, the target's behavior characteristics in different time and space can be obtained, providing a basis for subsequent abnormal behavior identification.

Based on the generated target behavior feature learning results, a feature clustering algorithm is used to perform spatiotemporal feature analysis, extract target behavior patterns, and generate spatiotemporal clustered behavior features. This algorithm can cluster targets with similar behavior patterns to obtain target behavior features of different categories. Through this step, the target behavior features can be further refined to improve the accuracy of abnormal behavior recognition.

Based on the generated spatiotemporal clustering behavior features, a single-class support vector machine is used to identify and model abnormal behaviors and generate an abnormal behavior recognition model. This algorithm can distinguish between normal and abnormal behaviors by training a binary classifier and generate a corresponding abnormal behavior recognition model. Through this step, automatic identification and early warning of abnormal behaviors can be achieved.

Based on the generated abnormal behavior recognition model, combined with real-time monitoring data, abnormal behavior warning is carried out, and abnormal behavior recognition and warning reports are generated. The algorithm can analyze and predict the target's behavior in real time based on real-time monitoring data. Once abnormal behavior is found, a warning signal can be issued and a corresponding abnormal behavior recognition and warning report can be generated.

Please refer to Figure 8, a smart city camera multi-target tracking system, which is used to execute the above-mentioned smart city camera multi-target tracking method. The system includes an image preprocessing module, a basic feature extraction module, a preliminary target recognition module, a feature enhancement and optimization module, a target tracking frame preprocessing module, a target association module, and a behavior learning and early warning module.

The image preprocessing module uses histogram equalization and Gaussian blur algorithm to perform image enhancement and noise reduction based on the original surveillance image to generate an enhanced image;

The basic feature extraction module extracts basic features based on the enhanced image, adopts the depth-separable convolution algorithm to reduce the amount of calculation and maintain performance, and generates a basic feature map;

The preliminary target recognition module performs preliminary target recognition based on the basic feature map, uses multi-scale spatial pyramid pooling to enhance scale-invariant feature extraction, and uses a region proposal network combined with bounding box regression to locate the target and generate preliminary target recognition information;

The feature enhancement and optimization module performs feature enhancement and optimization based on the preliminary target recognition information. It uses the feature pyramid network to achieve the fusion between feature layers and adopts the attention mechanism to enhance the expressiveness of features. It optimizes the frame by using the anchor frame optimization algorithm and refines the features through the cascade convolutional network to generate multi-scale target feature information.

The target tracking frame preprocessing module processes the tracking frame based on multi-scale target feature information, uses a real-time scale estimation algorithm to dynamically predict the target size, adjusts the tracking frame size through an anchor frame generation algorithm, and combines a non-maximum suppression algorithm to eliminate overlapping tracking frames, generating tracking information after adaptive scale adjustment;

The target association module runs the target association algorithm based on the tracking information after adaptive scale adjustment, uses the mask area convolutional neural network enhanced by the feature pyramid network to perform target instance segmentation, reconstructs the scene in three dimensions through the monocular depth estimation method based on deep learning, uses pixel-level fusion technology to process, and determines the occlusion level and depth between targets through the depth sorting algorithm and occlusion processing technology, and generates target occlusion relationship and depth information;

The behavior learning and warning module performs behavior learning and warning based on target occlusion relationships and depth information, uses temporal convolutional neural networks and long short-term memory networks for deep feature learning, performs spatiotemporal feature analysis through feature clustering algorithms, and uses single-class support vector machines to identify and model abnormal behaviors. It combines real-time monitoring data to perform abnormal behavior warnings and generate abnormal behavior identification and warning reports.

The image preprocessing module uses histogram equalization and Gaussian blur algorithms for image enhancement and noise reduction, which can effectively improve the accuracy and stability of subsequent feature extraction. The basic feature extraction module uses a deep separable convolution algorithm to reduce the amount of calculation while maintaining performance and improving the efficiency of feature extraction. The preliminary target recognition module uses a multi-scale spatial pyramid pooling and region proposal network combined with a bounding box regression method to accurately locate and identify targets, providing a reliable foundation for subsequent target tracking.

In addition, the feature enhancement and optimization module uses a feature pyramid network to achieve fusion between feature layers and introduces an attention mechanism to enhance the expressiveness of features, further improving the accuracy of target recognition.

The target tracking frame preprocessing module uses a real-time scale estimation algorithm to dynamically predict the target size, and adjusts the tracking frame size through the anchor frame generation algorithm. It combines the non-maximum suppression algorithm to eliminate overlapping tracking frames, and realizes the generation of tracking information after adaptive scale adjustment.

The target association module uses a mask area convolutional neural network enhanced by a feature pyramid network to perform target instance segmentation, reconstructs the scene in three dimensions through a deep learning monocular depth estimation method, processes it using pixel-level fusion technology, and determines the occlusion level and depth between targets through a depth sorting algorithm and occlusion processing technology, generating target occlusion relationships and depth information.

The behavior learning and warning module uses temporal convolutional neural networks and long short-term memory networks for deep feature learning, performs spatiotemporal feature analysis through feature clustering algorithms, uses single-class support vector machines to identify and model abnormal behaviors, combines real-time monitoring data for abnormal behavior warnings, and generates abnormal behavior identification and warning reports.

Please refer to FIG9 , the image preprocessing module includes a noise reduction submodule, an image enhancement submodule, an abnormality filtering submodule, and an image normalization submodule;

The basic feature extraction module includes a deep convolution submodule, a feature extraction submodule, a feature compression submodule, and a feature noise screening submodule;

The preliminary target recognition module includes a target positioning submodule, a target detection submodule, a target screening submodule, and a target confirmation submodule;

The feature enhancement and optimization module includes feature fusion submodule, feature attention weighting submodule, anchor box optimization submodule, and feature extraction submodule;

The target tracking frame preprocessing module includes a real-time scale estimation submodule, an anchor frame generation submodule, a non-maximum suppression submodule, and a target tracking frame positioning submodule;

The target association module includes a target instance segmentation submodule, a scene 3D reconstruction submodule, a target occlusion relationship analysis submodule, and an occlusion level and depth determination submodule;

The behavior learning and warning module includes a deep feature learning submodule, a spatiotemporal feature analysis submodule, an abnormal behavior recognition submodule, and a real-time abnormal warning submodule.

In the image preprocessing module, the noise reduction submodule uses the Gaussian blur algorithm to reduce the noise of the original monitoring image; the image enhancement submodule uses the histogram equalization algorithm to enhance the image; the abnormal filtering submodule is used to remove abnormal values in the image; and the image normalization submodule normalizes the image data.

In the basic feature extraction module, the deep convolution submodule uses the deep separable convolution algorithm to extract basic features; the feature extraction submodule is used to extract key features from the enhanced image; the feature compression submodule reduces the amount of calculation by reducing the feature dimension; and the feature noise screening submodule is used to screen out effective features.

In the preliminary target recognition module, the target localization submodule uses a region proposal network combined with bounding box regression to locate the target; the target detection submodule is used to detect multiple targets; the target screening submodule screens out reliable targets based on preset thresholds and confidence levels; and the target confirmation submodule is used to confirm the identity information of the target.

In the feature enhancement and optimization module, the feature fusion submodule uses the feature pyramid network to achieve fusion between feature layers; the feature attention weighting submodule enhances the expressiveness of features by introducing the attention mechanism; the anchor box optimization submodule optimizes the target position estimation by adjusting the tracking box size; the feature extraction submodule further refines features through a cascaded convolutional network.

In the target tracking frame preprocessing module, the real-time scale estimation submodule uses a real-time scale estimation algorithm to dynamically predict the target size; the anchor frame generation submodule adjusts the tracking frame size through the anchor frame generation algorithm; the non-maximum suppression submodule is used to eliminate overlapping tracking frames; and the target tracking frame positioning submodule is used to determine the position of the tracking frame.

In the target association module, the target instance segmentation submodule uses the mask area convolutional neural network enhanced by the feature pyramid network to perform target instance segmentation; the scene 3D reconstruction submodule reconstructs the scene in 3D through the monocular depth estimation method based on deep learning; the target occlusion relationship analysis submodule uses pixel-level fusion technology to process, and determines the occlusion level and depth between targets through the depth sorting algorithm and occlusion processing technology; the occlusion level and depth determination submodule is used to determine the occlusion relationship and depth information between targets.

In the behavior learning and warning module, the deep feature learning submodule uses temporal convolutional neural networks and long short-term memory networks for deep feature learning; the spatiotemporal feature analysis submodule performs spatiotemporal feature analysis through feature clustering algorithms; the abnormal behavior recognition submodule uses a single-class support vector machine to identify and model abnormal behaviors; the real-time abnormal warning submodule combines real-time monitoring data to warn of abnormal behaviors and generates abnormal behavior recognition and warning reports.

The above are only preferred embodiments of the present invention and are not intended to limit the present invention in other forms. Any technician familiar with the profession may use the technical contents disclosed above to change or modify them into equivalent embodiments with equivalent changes and apply them to other fields. However, any simple modification, equivalent change and modification made to the above embodiments based on the technical essence of the present invention without departing from the technical solution of the present invention still falls within the protection scope of the technical solution of the present invention.

Claims (10)

1. The multi-target tracking method for the smart city camera is characterized by comprising the following steps of:

Based on the monitoring image, performing image processing and preliminary target identification by adopting a depth separation convolutional neural network and scale invariant feature transformation, and performing preliminary detection of targets to generate preliminary target identification information;

based on the preliminary target identification information, carrying out multi-scale feature fusion by adopting a feature pyramid network and an attention mechanism, enhancing the identification capability of multi-size targets, and generating multi-scale target feature information;

Based on the multi-scale target characteristic information, performing self-adaptive scale adjustment by using a scale estimation module and an anchor frame mechanism, optimizing the size of a target tracking frame, and generating tracking information after the self-adaptive scale adjustment;

based on the tracking information after the self-adaptive scale adjustment, a mask region convolutional neural network and a monocular depth estimation algorithm are used for processing a target shielding problem, shielding relation and scene depth information between targets are obtained, and target shielding relation and depth information are generated;

integrating the data of multiple camera view angles through a graph convolution neural network based on the target shielding relation and the depth information, and establishing space-time correlation of targets in the multiple camera view fields to generate target tracking information crossing cameras;

Based on the target tracking information of the cross-camera, learning a target mode by combining a time convolution neural network and a long-term and short-term memory neural network, and identifying and early warning abnormal behaviors by using a single support vector machine to generate an abnormal behavior identification and early warning report;

The preliminary target identification information comprises position, size and shape information of a target, the multi-scale target characteristic information is specifically multi-level and multi-scale target characteristic description, the multi-scale target characteristic information comprises details and structure information of the target under the multi-scale, the tracking information after the self-adaptive scale adjustment is specifically tracking frames and characteristic information of the matched target under the multi-scale, the target shielding relation and depth information is specifically identification information comprising shielding targets and depth positions of the targets in a scene, the cross-camera target tracking information comprises positions, time relations and behavior information of key targets among multiple cameras, and the abnormal behavior identification and early warning report is specifically found illegal behaviors or abnormal behaviors, and describes and early warns time, place and property of the illegal behaviors or abnormal behaviors.

2. The smart city camera multi-target tracking method of claim 1, wherein the steps of performing image processing and preliminary target recognition using a depth-separation convolutional neural network and scale-invariant feature transform based on the monitored image, and performing preliminary detection of the target, and generating preliminary target recognition information are specifically as follows:

based on the original monitoring image, carrying out image enhancement and noise reduction by adopting a histogram equalization and Gaussian blur algorithm to generate an enhanced image;

based on the enhanced image, extracting basic features by adopting a depth separable convolution algorithm, reducing calculated amount and maintaining performance, and generating a basic feature map;

Based on the basic feature map, adopting a multi-scale space pyramid pooling enhancement scale invariant feature extraction to generate a vector feature map;

based on the vector feature map, the area proposal network is adopted to combine with frame regression to perform preliminary detection of the target, the target is accurately positioned, and preliminary target identification information is generated.

3. The smart city camera multi-target tracking method of claim 1, wherein the step of generating multi-scale target feature information by performing multi-scale feature fusion using a feature pyramid network and an attention mechanism based on the preliminary target identification information and enhancing the identification capability of multi-scale targets comprises the steps of:

based on the preliminary target identification information, realizing fusion between feature layers by adopting a feature pyramid network, enhancing the detection capability of a multi-scale target, and generating a fusion feature map;

based on the fusion feature map, adopting an attention mechanism to enhance the expression capacity of the features and generating an attention weighted feature map;

based on the attention weighted feature map, an anchor frame optimization algorithm is adopted to perform optimization adjustment of the anchor frame, fitting precision of the model to the multi-size targets is improved, and optimized anchor frame feature information is generated;

And extracting the characteristics through a cascade convolution network based on the optimized anchor frame characteristic information, enhancing the representation force of the multi-scale characteristics, and generating multi-scale target characteristic information.

4. The smart city camera multi-target tracking method according to claim 1, wherein the step of adaptively scaling the size of the target tracking frame and optimizing the size of the target tracking frame based on the multi-scale target feature information by using a scale estimation module and an anchor frame mechanism, and generating the tracking information after adaptively scaling is specifically as follows:

Based on the multi-scale target characteristic information, adopting a real-time scale estimation algorithm to dynamically predict the size of the target, adapting to the change of the target scale and generating scale adjustment information;

Based on the scale adjustment information, adjusting the size of the tracking frame by adopting an anchor frame generation algorithm, enabling the tracking frame to be attached to the actual scale of the target, and generating a tracking frame with a self-adaptive scale;

based on the tracking frames of the self-adaptive scale, using a non-maximum suppression algorithm to eliminate overlapped tracking frames, reducing redundancy and generating tracking frames after non-maximum suppression;

and based on the tracking frame after non-maximum suppression, carrying out final target tracking frame positioning by combining a tracking learning algorithm, and generating tracking information after self-adaptive scale adjustment.

5. The smart city camera multi-target tracking method according to claim 1, wherein the step of processing a target occlusion problem using a mask area convolutional neural network and a monocular depth estimation algorithm based on the tracking information after the adaptive scale adjustment to obtain an occlusion relationship between targets and scene depth information, and generating the target occlusion relationship and the depth information is specifically as follows:

Based on the tracking information after the self-adaptive scale adjustment, a mask area convolutional neural network enhanced by a characteristic pyramid network is adopted to segment a target instance, and an occlusion boundary is accurately judged to generate target segmentation and occlusion boundary information;

based on the target segmentation and shielding boundary information, performing three-dimensional reconstruction on the scene by using a monocular depth estimation method of depth learning to generate a scene depth map;

Based on the scene depth map, analyzing the shielding relation among targets by using pixel-level fusion technology processing to generate a target shielding relation map;

And determining the shielding level and depth between targets based on the target shielding relation graph by combining a depth ordering algorithm and a shielding processing technology, and generating target shielding relation and depth information.

6. The smart city camera multi-target tracking method of claim 1, wherein integrating data of multiple camera angles of view through a graph convolutional neural network based on the target occlusion relationship and depth information, and establishing space-time correlation of targets in multiple camera fields of view, the step of generating inter-camera target tracking information specifically comprises:

Based on the target shielding relation and the depth information, a graph convolutional neural network is applied, and multi-view data integration is carried out by combining feature extraction and optimization strategies, so that preliminary cross-camera target association information is generated;

based on the preliminary cross-camera target association information, a multi-camera geometric calibration method is adopted, and the target position relation is refined through a calibration technology based on polar line constraint, so that multi-view target position information is generated;

based on the multi-view target position information, time sequence analysis technology is applied, time sequence data analysis is carried out through dynamic time warping, target space-time correlation is optimized, and time optimized target correlation information is generated;

And integrating the multi-camera data based on the time-optimized target association information, and completing space-time association through a multi-view tracking fusion algorithm to generate inter-camera target tracking information.

7. The smart city camera multi-target tracking method according to claim 1, wherein based on the cross-camera target tracking information, a target mode is learned by combining a time convolution neural network and a long-term and short-term memory neural network, and abnormal behaviors are identified and early-warned by using a single support vector machine, and the steps of generating an abnormal behavior identification and early-warning report are specifically as follows:

based on the target tracking information of the cross-camera, performing deep feature learning by combining a time convolution neural network and a long-term and short-term memory network, and generating a target behavior feature learning result;

based on the target behavior feature learning result, performing space-time feature analysis by adopting a feature clustering algorithm, extracting a target behavior mode, and generating behavior features after space-time clustering;

based on the behavior characteristics after space-time clustering, carrying out recognition and modeling of abnormal behaviors by using a single-class support vector machine, and generating an abnormal behavior recognition model;

And based on the abnormal behavior recognition model, carrying out abnormal behavior early warning by combining real-time monitoring data, and generating abnormal behavior recognition and early warning reports.

8. The smart city camera multi-target tracking system according to any one of claims 1-7, wherein the system comprises an image preprocessing module, a basic feature extraction module, a preliminary target recognition module, a feature enhancement and optimization module, a target tracking frame preprocessing module, a target association module, and a behavior learning and early warning module.

9. The smart city camera multi-target tracking system of claim 8, wherein the image pre-processing module performs image enhancement and noise reduction using histogram equalization and gaussian blur algorithm based on the original monitored image to generate an enhanced image;

The basic feature extraction module extracts basic features based on the enhanced image, adopts a depth separable convolution algorithm, reduces calculated amount and maintains performance, and generates a basic feature map;

The preliminary target recognition module carries out preliminary target recognition based on the basic feature map, adopts multi-scale space pyramid pooling to enhance scale invariant feature extraction, adopts a region proposal network to combine with frame regression, positions a target and generates preliminary target recognition information;

The feature enhancement and optimization module performs feature enhancement and optimization based on preliminary target identification information, realizes fusion between feature layers by using a feature pyramid network, adopts a attention mechanism to enhance the expression capability of features, performs optimization adjustment of frames by adopting an anchor frame optimization algorithm, refines the features by using a cascade convolution network, and generates multi-scale target feature information;

The target tracking frame preprocessing module processes the tracking frame based on the multi-scale target characteristic information, dynamically predicts the size of the target by adopting a real-time scale estimation algorithm, adjusts the size of the tracking frame by adopting an anchor frame generation algorithm, and eliminates overlapped tracking frames by combining a non-maximum suppression algorithm to generate tracking information after self-adaptive scale adjustment;

The target association module runs a target association algorithm based on tracking information after self-adaptive scale adjustment, performs target instance segmentation by adopting a mask region convolutional neural network enhanced by a feature pyramid network, performs three-dimensional reconstruction on a scene by a monocular depth estimation method of deep learning, processes by using a pixel level fusion technology, determines the shielding level and depth between targets by a depth ordering algorithm and a shielding processing technology, and generates a target shielding relation and depth information;

The behavior learning and early warning module performs behavior learning and early warning based on the target shielding relation and the depth information, performs deep feature learning by using a time convolution neural network and a long-term and short-term memory network, performs space-time feature analysis by using a feature clustering algorithm, performs abnormal behavior recognition and modeling by using a single support vector machine, performs abnormal behavior early warning by combining real-time monitoring data, and generates an abnormal behavior recognition and early warning report.

10. The smart city camera multi-target tracking system of claim 8, wherein the image pre-processing module comprises a noise reduction sub-module, an image enhancer sub-module, a filtering anomaly sub-module, an image normalization sub-module;

The basic feature extraction module comprises a depth convolution sub-module, a feature extraction sub-module, a feature compression sub-module and a feature noise screening sub-module;

The preliminary target recognition module comprises a target positioning sub-module, a target detection sub-module, a target screening sub-module and a target confirmation sub-module;

The feature enhancement and optimization module comprises a feature fusion sub-module, a feature attention weighting sub-module, an anchor frame optimization sub-module and a feature extraction sub-module;

the target tracking frame preprocessing module comprises a real-time scale estimation sub-module, an anchor frame generation sub-module, a non-maximum value suppression sub-module and a target tracking frame positioning sub-module;

the target association module comprises a target instance segmentation sub-module, a scene three-dimensional reconstruction sub-module, a target occlusion relationship analysis sub-module, an occlusion level and depth judgment sub-module;

The behavior learning and early warning module comprises a deep feature learning sub-module, a space-time feature analysis sub-module, an abnormal behavior identification sub-module and a real-time abnormal early warning sub-module.