CN116994337A - Appearance detection action recognition device and method for dense objects in narrow space - Google Patents

Abstract

The present invention provides an appearance detection action recognition device and method for dense objects in a small space, and for the first time proposes different computer vision algorithm models to perform cyclic reading processing of original encoded video frame image data stored in shared memory according to time sequence. A multi-target tracking algorithm is used to correspond the ID of dense object tracking to the recognition result of the object's appearance detection behavior. The recognition results are regularly added to the data set for self-training of the model, effectively reducing the coupling between systems and data reading time. delay, greatly improving the real-time, robustness and efficiency of video processing. This solves the problems of low efficiency and strong subjectivity in the appearance detection of dense objects by existing methods.

Description

Appearance detection action recognition device and method for dense objects in small space

Technical field

The invention belongs to the technical fields of artificial intelligence and computer vision, and specifically relates to an appearance detection action recognition device and method for dense objects in a small space.

Background technique

In recent years, under the trend of industrial intelligent upgrading, more and more companies are trying to build smart factories through technologies such as robots and artificial intelligence. Factory surveillance cameras can produce several terabytes of effective factory video data every day, and most of these videos are only used to monitor worker production. In fact, these factory video data contain a large number of workers and machine operating behaviors, as well as the production operation modes contained in them, etc., which can be further used for action recognition, workflow mining, abnormal event monitoring, etc. However, at present, the work quality assessment of production line workers or mechanical devices in certain production operations still relies on manual inspection or random inspection, which consumes a lot of manpower, material and financial resources, and does not effectively combine surveillance video and artificial intelligence technology. of combined.

It is understood that there are currently very few studies that attempt to automatically determine whether the behavior of appearance detection of dense objects in a small space is compliant, and most methods still rely on manual monitoring. However, this manual inspection method is limited by manpower and subjective factors. When factory production capacity is expanded, it becomes more difficult to judge whether the appearance inspection of intensive objects is compliant.

In the past, most of the technologies used to recognize the actions of people were to recognize the actions currently performed by the person as a whole. For example, in Patent Document 1: Invention Disclosure CN115410268A Action Recognition Method and Action Recognition Method Based on Action Skeleton Point Data, just Input the action key point data into the model for recognition and output the results. However, in the small space of the factory, multiple appearance detection actions need to be performed on multiple objects. Existing action recognition methods cannot directly perform action recognition on surveillance video data and effectively bind the recognition results to dense objects correctly.

Contents of the invention

In order to overcome the above problems in the prior art, the purpose of the present invention is to provide an appearance detection and action recognition device and method for dense objects in a small space. It proposes for the first time different computer vision algorithm models to analyze the original data stored in the shared memory based on time sequence. The encoded video frame image data is read and processed cyclically, and a multi-target tracking algorithm is used to correspond the ID of the dense object tracking to the appearance detection behavior recognition result of the object. The recognition results are regularly added to the data set for self-training of the model, effectively reducing the It reduces the coupling between systems and the data reading delay, and greatly improves the real-time, robustness and efficiency of video processing. This solves the problems of low efficiency and strong subjectivity in the appearance detection of dense objects by existing methods.

In order to achieve the above objects, the technical solution adopted by the present invention is:

An appearance detection action recognition method for dense objects in a small space, including the following steps:

S1: The camera sends the original encoded video to the edge device and the background display system through multicast mode, and writes the frame image data to the shared memory in real time; if the writing position reaches the end of the shared memory queue, it will be written again from the head of the queue, overwriting Original frame image data, that is, cyclic writing of shared memory;

S2: Cyclically read from the shared memory according to the frame sequence information generated and transmitted by the camera, and obtain the corresponding frame image data as input. The frame sequence information specifies the specific storage location of the frame image data in the shared memory; use damo- The yolo target detection model performs target detection on the frame image data retrieved from the shared memory, and outputs detection results such as detection frame coordinates and object categories to the target tracking model;

S3: Read the frame image data from the shared memory according to the frame sequence information passed by the damo-yolo target detection model, combine the detection results, fuse the real-time position and texture changes of the object for multi-target tracking processing and assign object ids, and output dense objects Tracking coordinates and ID tracking results;

S4: Use the action segmentation model to perform appearance detection action behavior segmentation, segment the cached long video obtained in the shared memory into isolated action segments, mark the start frame number start and end frame number end of the segments, and forward them to the action recognition model through messages;

S5: The action recognition model obtains the image data of the isolated action clip from the shared memory based on the start frame number start and the end frame number end obtained by message forwarding, and uses the spatiotemporal regional attention model and large convolution kernel to extract spatiotemporal features of the video sequence. Then complete the action classification, output the recognition results and forward the results message;

S6: The background display system receives the detection, tracking, behavior segmentation and action recognition results of the front-end edge device through socket communication, and overlays the results on the real-time video display;

S7: The recognition results are automatically cached in the storage device and merged with the original training data to generate a new data set, which is triggered by a scheduled task. The incremental data set is used for action recognition model training, and the latest action recognition model file is automatically Make a substitution.

Further, in the S1, the original encoded video is I={I _s } ^T , I _s represents the sth frame, s represents the index value of each frame, and T represents the timestamp of the sth frame; the camera obtains the sth frame s real-time images I _s of size M, and write the real-time images into a pre-applied shared memory of size N, where a total of n images can be written into the shared memory, that is, N=nM.

Further, in S2, the cyclic reading in the shared memory means that after the camera writes the shared memory, the target detection model reads the corresponding picture data from the location of the shared memory according to the frame sequence information of the frame picture data transmitted by the queue. .

Further, in the S2, the object detection result of the frame image data is calculated as follows:

D _t =F(I _t );

In the formula, D _t represents the object detection result of the current frame I _t , specifically expressed as [c, w, h, x, y, thresh], where c represents the object category, w represents the width of the detection frame, and h represents the height of the detection frame, x represents the x-axis coordinate of the center point of the detection frame, y represents the y-axis coordinate of the center point of the detection frame, thresh represents the confidence of the detection frame; t represents the index value of the image data of each frame in the shared memory; function F represents the current frame I _t The operation process of the damo-yolo target detection model.

Further, in S3, the calculation process of multi-target tracking processing of the real-time position and texture changes of the fusion object is as follows:

X _t =G(I _t ,D _t )

In the _{formula ,} represents the width of the tracking detection frame, h represents the height of the tracking detection frame, x represents the x-axis coordinate of the center point of the tracking detection frame, y represents the y-axis coordinate of the center point of the tracking detection frame, thresh represents the confidence of the tracking detection frame; t represents each The index value of the frame's image data in the shared memory; the function G represents the operation process of the current frame _It and the detection result _Dt of the frame through the target tracking algorithm.

Further, in the S4, the action segmentation model performs the calculation process of appearance detection action behavior segmentation, as shown in the following formula:

{L ₁ ,…,L _n }＝trim(V _m )

(I _start ,I _end )=P({L ₁ ,…,L _n })

In the formula, V _m represents the cached long video data obtained from the shared memory; the function trim represents the operation process of the current cached video V _m through the action segmentation algorithm; L _i represents the action category output by each frame of the cached long video, and i represents The number of categories set by the system; function P represents the operation process of obtaining the isolated action start frame I _start and end frame I _end based on the frame data action category _Li .

Further, in S5, the spatiotemporal regional attention model and large convolution kernel are used to extract spatiotemporal features of the video sequence, and then complete the calculation process of action classification, as shown in the following formula:

Y＝W(I _start ,…,I _end )

In the formula, Y represents the action recognition prediction result, I _start ,...,I _end represents all picture data of the input image as the start point and end point defined by the action segmentation algorithm, obtained from the shared memory; the function W represents the current input frame sequence The operation process of the action recognition algorithm.

Further, in S6, the background display system receives the detection, tracking, behavior segmentation and action recognition results of the front-end edge device through socket communication, obtains pictures from the shared memory at the same time, and aligns the picture timestamp with the detection information timestamp. Overlay results on live video.

Further, in S7, the system automatically caches the recognition results to the storage device, and merges them with the original training data to generate a new data set, which is triggered by the scheduled task, and uses the incremental data set for action recognition model training, and Automatically replace the latest motion recognition model files.

Another object of the present invention is to provide a device that includes a processor, a communication interface, a memory, a display, and a communication bus, wherein the processor, the communication interface, the memory, and the display complete communication with each other through the communication bus;

Memory, used to store computer programs;

A display used to display the execution results and real-time images of the processor;

The processor is used to implement the above method steps when executing the program stored in the memory.

Beneficial effects of the present invention:

(1) The video input frame access method of the present invention stores video images in shared memory, and camera sampling and each algorithm model access the shared memory in chronological order. The camera writes to the shared memory cyclically, and each algorithm model reads the frame image data from the shared memory in an orderly manner and processes it accordingly. Only the frame image processing results, such as the storage location and timestamp of the frame in the shared memory, are transmitted between the models. Wait for a small amount of data. It effectively reduces the delay problems caused by large data transmission and data reading in each model such as images.

(2) Loop reading in the shared memory of the present invention means that each algorithm model reads the picture from the storage location of the shared memory according to the picture frame after the camera writes it into the shared memory. If the read position reaches the end of the shared memory queue, read again from the head of the queue. The target detection model reads images from the shared memory for processing, and transmits a small amount of information such as the storage location of the frame in the shared memory and detection results to the target tracking model through the queue. The target tracking model reads pictures from the shared memory according to the storage location of the frame in the shared memory, and processes it based on the detection results. It transmits a small amount of information such as the storage location of the frame in the shared memory and tracking results through the queue. The action segmentation model obtains the cached long video data from the shared memory for processing, and transmits the isolated action start frame I _start and end frame I _end through the queue. The action recognition model reads the image sequence from the shared memory as model input based on the transmitted isolated action start frame number start and end frame number end.

(3) The present invention introduces a target tracking model, and corresponds the ID of the object target tracking with the appearance detection behavior recognition result of the object, effectively solving the problem that multiple results of dense objects in a small space are difficult to correspond after appearance detection, and achieves Dynamic binding of dense objects and detection results.

(4) The present invention updates the data set by caching the recognition data, and uses scheduled tasks to iteratively train the model based on the new data set, and updates the action recognition model file in the system, effectively improving the performance of the model. Accuracy and robustness.

(5) Compared with the existing technology, the present invention uses existing 2D cameras to sample the appearance detection behavior of dense objects in small spaces without adding additional hardware equipment, and uses computer vision algorithms to automatically determine whether the current detection behavior is compliant. Effectively reducing manpower and material resources, greatly increasing work efficiency and objectivity of inspection.

Description of the drawings

Figure 1 is the framework of an appearance detection action recognition system for dense objects in a small space according to an embodiment of the present invention.

Figure 2 is a schematic diagram of system data interaction according to an embodiment of the present invention.

Figure 3 is a schematic diagram of the deployment of the system framework constructed according to the embodiment of the present invention.

Figure 4 is a flow chart of object appearance detection logic processing according to the embodiment of the present invention.

Detailed ways

The present invention will be further described in detail below in conjunction with the accompanying drawings and examples.

As shown in Figures 1 to 3, embodiments of the present invention provide an appearance detection and action recognition device and method for dense objects in a small space, including a camera, a target detection model, a target tracking model, an action segmentation model, and an action recognition model. The interface displays serial execution in chronological order and accesses shared memory data in parallel.

Specifically, it includes the following steps:

Step 1: The camera sends the original encoded video to the edge device and the background display system through multicast, and writes the frame image data to the shared memory in real time, so that the subsequent algorithm model can obtain the corresponding image data in an orderly manner based on the frame sequence information. The camera only transmits the location of the image in the shared memory and the image timestamp to the target detection model through the queue, thereby reducing the delay time caused by transmitting image data between models. The edge device is the server required for model running and training, and the background display system includes a visual interface and a monitor.

Step 2: Read cyclically from the shared memory according to the frame sequence information generated and transmitted by the camera, obtain the corresponding frame image data as input, use the damo-yolo target detection model to perform target detection on the frame image data taken out from the shared memory, and Output detection results such as detection frame coordinates and object categories to the target tracking model.

Step 21, input the original encoded video data I = {I _t } ^T into the pre-processing of the damo-yolo target detection model, where I _t represents the t-th frame, t represents the index value of each frame, and T represents the timestamp. The input static images are scaled, cropped, denoised, etc. to adapt to the input requirements of the model.

Step 22: Input each cropped small image into a convolutional neural network for feature extraction, and connect each feature map to form a feature map of the entire image.

Step 23, apply a fully connected layer to the large feature map to output a 13x13x (5+K) tensor, where K is the number of categories and 5 is the bounding box parameters (center coordinates, width, height and confidence). For each 13x13 cell, a predicted bounding box is generated based on the bounding box parameters and class confidence. Perform threshold filtering and non-maximum suppression on the predicted bounding box to obtain the final detection result.

Step 3: Cyclically read frame image data from the shared memory according to the frame sequence information passed by the damo-yolo target detection model, combine the detection results, fuse the real-time position and texture changes of the object for multi-target tracking processing and assign object ids, and output dense objects Tracking results such as coordinates and IDs

Step 31: Obtain the original encoded video I={I _t } ^T from the shared memory according to the frame sequence information transmitted by the target detection model and input it to the tracking model for processing.

Step 32: Based on the detection frame and other information transmitted from the damo-yolo target detection model, use the feature extraction network to extract features from the images in each frame to obtain the feature vector of each target.

Step 33: Use Kalman Filter to predict the state (position and velocity) of each target, using an 8-dimensional state space vector expressed as x=[cx,cy,w,h,vx,vy, vr, vh], each speed value is initialized to 0. As a direct observation model of the object state, (cx, cy) represents the center point coordinates of the detection frame, the width w, the height h and the relative speed corresponding to the image coordinates. Prediction is to predict the state of the track at time t based on its state at time t-1. The calculation formula is as follows:

x′＝Fx (1)

P′＝FPF ^T +Q (2)

In Formula 1, x is the mean value of track at time t-1, and F is called the state transition matrix. This formula predicts x' at time t:

dt in matrix F is the difference between the current frame and the previous frame.

In Formula 2, P is the covariance of track at time t-1, Q is the noise matrix of the system, which represents the reliability of the entire system. It is generally initialized to a very small value. This formula predicts P' at time t.

Step 34: Use the Hungarian Algorithm to match the predicted state and the detected bounding box, and update the tracker state according to the matching result.

Step 35: For unmatched bounding boxes, use cosine distance (Cosine Distance) to calculate the similarity between them and the feature vector of the existing tracker. If the similarity is higher than a certain threshold, it is assigned to the corresponding tracking tracker, otherwise create a new tracker. For unmatched trackers, if they do not match a bounding box for multiple consecutive frames, they are deleted.

Step 36, use Kalman filter for position update. The update is based on the detection detected at time t, correcting the status of the track associated with it, and obtaining a more accurate result. The calculation formula is as follows:

y＝z-Hx′ (3)

S＝HP′H ^T +R (4)

K＝P′H ^T S ^-1 (5)

x＝x′+Ky (6)

P＝(I-KH)P′ (7)

In Formula 3, z is the mean vector of detection, which does not include the speed change value, that is, z = [cx, cy, w, h]. H is called the measurement matrix, which maps the mean vector x' of track to the detection space. This formula calculates the mean error of detection and track;

In Formula 4, R is the noise matrix of the detector. It is a 4x4 diagonal matrix. The values on the diagonal are the two coordinates of the center point and the noise of the width and height. They are initialized with any value. Generally, the width and height are set. The noise of is greater than the noise of the center point. This formula first maps the covariance matrix P' to the detection space, and then adds the noise matrix R;

Formula 5 calculates the Kalman gain K, which is used to estimate the importance of the error;

Formula 6 and Formula 7 obtain the updated mean vector x and covariance matrix P.

Step 4: Use the action segmentation model to perform appearance detection action behavior segmentation, segment the cached long video into isolated action segments, mark the start frame number start and end frame number end of the segments, and forward them to the action recognition model through messages.

Step 41: Obtain the cached long video from the shared memory and use a prediction generation stage (PredictionGeneration Stage) to generate initial action segmentation predictions. This stage uses a dual dilated layer (DualDilated Layer, DDL), which can capture both large-scale and small-scale timing information at the same time;

Step 42: Use multiple refinement stages to gradually improve the action segmentation prediction. Each refinement stage uses a Self-Supervised Temporal Convolutional Network (SS-TCN), which can use information from previous and previous frames to correct the prediction of the current frame;

Step 43: average the outputs of all stages to obtain the final action segmentation result, and assign an action category label to each frame of picture. Action segmentation is performed based on the action category label, and the start frame number start and the end frame number end of the isolated action fragment are marked.

Step 44: Send the start frame number start and the end frame number end to the action recognition model through message forwarding.

Step 5: The action recognition model obtains the image data of the isolated action clip from the shared memory based on the start frame number start and the end frame number end obtained by message forwarding, and uses the spatiotemporal regional attention model and large convolution kernel to extract spatiotemporal features of the video sequence. , and then complete the action classification, output the recognition results and forward the results message.

Step 51: Based on the obtained start frame number start and end frame number end, obtain all the image data of the action clip from the shared memory, averagely sample the image according to the NUM_FRAMES set by the model, and randomly scale and scale the image after the sampling is completed. Crop.

Step 52: For each segment, sample different numbers of frames (for example, 8 frames and 32 frames) from the slow path and the fast path respectively. For each path, a 3D convolutional neural network (such as ResNet-50) is used to extract features from the sampled frames to obtain a feature map.

Step 53: Use spatial attention mechanism (Spatial attention) to select salient areas in the feature map. It generates an output feature map that only contains salient areas by calculating the attention score for the input feature map and using the score as the weight coefficient of the values of different channels of the input feature map at the same pixel position. The calculation formula of Spatial attention attention score is as follows:

M _s (F)＝f(conv _k (concatenation(MaxPooling(F),AveragePooling(F))))

Among them, F is the input feature map with size (B, C, T, H, W), where B refers to the batch size, C refers to the number of channels, T refers to the time dimension, and H refers to Height, W refers to width. MaxPooling(·) and AveragePooling(·) in the above formula respectively refer to using maximum pooling and average pooling on F in the channel dimension, thereby obtaining two sizes of (B, 1, T, H, W) The tensor is then subjected to the concatenation(·) operation to obtain a tensor of size (B, 2, T, H, W). conv _k (·) in the above formula refers to the width and height of the convolution kernel, which is specifically implemented as a three-dimensional convolution operation of k*k. As a hyperparameter, k can be adjusted according to the actual situation. In the above formula, f(·) can be any activation function. In the present invention, the sigmoid activation function is selected. The spatial weight M _s (F) is thus obtained. The value of each pixel in M _s (F) represents the importance of the corresponding position in the input feature F. Applying this module in the model can suppress irrelevant background information.

Step 54: Use the spatial weight M _s (F) to mask the input feature F. The specific formula is as follows:

After masking, the number of channels of the input features will not change.

Step 55: At a specific level, use Bidirectional Channel-wiseConnections to fuse the feature maps of the slow path and the fast path. For each path, global average pooling is used to compress the features into a one-dimensional vector. The feature vectors of the two paths are concatenated to obtain the final video representation.

Step 56: Use the fully connected layer (Fully Connected Layer) and the softmax layer to classify the video representation to obtain action recognition results.

As shown in Figure 2, the data interaction schematic diagram explains in detail the data transmission methods between each model and the front-end and back-end data communication methods.

Step 6: The background display system receives the detection, tracking, behavior segmentation and action recognition results of the front-end edge device through socket communication, and overlays the results on the real-time video for display.

Step 61: The camera uses multicast to push video data to the background display system, and the background calls the camera driver to obtain the frame image data and write it into the shared memory.

Step 62: Detection, tracking, action recognition and other results share a socket connection, and the transmission data is distinguished through the result type field and related processing is performed.

Step 63: Obtain the frame image data from the shared memory according to the frame sequence information of the tracking result, and obtain the detection frame coordinates, tracking id, log timestamp and other information of all objects in the frame. Obtain the recognition result of the object's detection action and the corresponding tracking ID information based on the action recognition result.

Step 64: Bind the detection results to the object position coordinates based on the tracking ID; align the tracking result timestamp with the image timestamp, and perform frame processing based on the detection frame information after alignment, and display the processed image in the system interface show.

Step 7: The system automatically caches the recognition results to the storage device and merges them with the original training data to generate a new data set, which is triggered by a scheduled task. The incremental data set is used for action recognition model training, and the latest action recognition Model files are automatically replaced.

As shown in Figure 4, the embodiment of the present invention provides an object appearance detection logic processing flow chart, which introduces the processing logic of the system when the detection result is OK or NG, where the result OK means that the object appearance detection meets the appearance detection standard, The result NG means that the object's appearance inspection behavior is unqualified. At this time, the operator or the robot arm needs to conduct another appearance inspection of the object, that is, the object is re-inspected. The specific process is as follows:

The camera acquires real-time surveillance video, passes it into the target detection model of the front-end edge device, performs target detection processing, and outputs the detection results, which are output to the target tracking model. The target tracking model makes judgments based on the current trajectory status: If it is an NG object re-inspection, the trajectory status maintained by the target tracking model does not increase steadily and continuously, and a jump phenomenon occurs. The system resets the trajectory and gives priority to finding the missing id from the non-continuous trajectories currently maintained for redistribution. After completion, the id increases steadily and continuously; if it is a new object detection, the trajectory status currently maintained by the system increases steadily without jumping. , then a new trajectory is incrementally generated and assigned a new id. The action segmentation model divides continuous long videos into isolated action segments, and the action recognition model is passed in to determine the appearance detection compliance, and the recognition results are passed to the background display system. The background receives the detection, tracking, behavior segmentation and actions of the front-end edge device. Recognize the results and display the results overlay on the real-time video. The operator or mechanical equipment re-inspects the objects with NG results based on the results, and repeats the above operation process until the recognition result is OK.

By conducting time-consuming analysis of data access between models, this invention proposes for the first time to store video images in shared memory, and camera sampling and each algorithm model read frame images from the shared memory in an orderly manner in chronological order. The data is processed accordingly. Only frame sequence information is transmitted between each model, which includes a small amount of data such as the position of the frame in the shared memory, timestamp, etc. This method effectively solves the delay problem of each model caused by big data transmission and data reading such as images.

In the matching stage of detection results of dense objects in a small space, the present invention pioneered the introduction of a target tracking model and proposed to bind the ID of the target tracking with the appearance detection behavior recognition result of the object, effectively solving the problem of the appearance of dense objects in a small space. The problem is that it is difficult to correspond to multiple results after testing.

The present invention updates the data set by caching the recognition data, and uses scheduled tasks to iteratively train the model based on the new data set, and updates the action recognition model file in the system, effectively improving the accuracy and accuracy of the model. robustness.

Compared with the existing technology, the present invention uses existing 2D cameras to sample dense object appearance detection behaviors without adding additional hardware equipment, and uses computer vision algorithms to automatically determine whether the current detection behavior is compliant, effectively reducing manpower. material resources, greatly increasing work efficiency and objectivity of inspection.

It should be understood that the above-described embodiments are only illustrative. The described embodiments of the invention may be implemented in hardware, software, firmware, middleware, microcode, or any combination thereof. For hardware implementation, the processing unit may be in one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays ( FPGA), processor, controller, microcontroller, microprocessor and/or other electronic unit designed to perform the functions described in the present invention, or a combination thereof.

Each embodiment in this specification is described in a related manner. The same and similar parts between the various embodiments can be referred to each other. Each embodiment focuses on its differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple. For relevant details, please refer to the partial description of the method embodiment.

The above descriptions are only preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention are included in the protection scope of the present invention.

Claims (10)

1. An appearance detection action recognition method for dense objects in a small space, which is characterized by including the following steps: S1: The camera sends the original encoded video to the edge device and the background display system through multicast mode, and writes the frame image data to the shared memory in real time; if the writing position reaches the end of the shared memory queue, it will be written again from the head of the queue, overwriting Original frame image data, that is, cyclic writing of shared memory; S2: Cyclically read from the shared memory according to the frame sequence information generated and transmitted by the camera, and obtain the corresponding frame image data as input. The frame sequence information specifies the specific storage location of the frame image data in the shared memory; use damo- The yolo target detection model performs target detection on the frame image data retrieved from the shared memory, and outputs the detection frame coordinates and object category detection results to the target tracking model; S3: Read the frame image data from the shared memory according to the frame sequence information passed by the damo-yolo target detection model, combine the detection results, fuse the real-time position and texture changes of the object for multi-target tracking processing and assign object ids, and output dense objects Tracking coordinates and ID tracking results; S4: Use the action segmentation model to perform appearance detection action behavior segmentation, segment the cached long video obtained in the shared memory into isolated action segments, mark the start frame number start and end frame number end of the segments, and forward them to the action recognition model through messages; S5: The action recognition model obtains the image data of the isolated action clip from the shared memory based on the start frame number start and the end frame number end obtained by message forwarding, and uses the spatiotemporal regional attention model and large convolution kernel to extract spatiotemporal features of the video sequence. Then complete the action classification, output the recognition results and forward the results message; S6: The background display system receives the detection, tracking, behavior segmentation and action recognition results of the front-end edge device through socket communication, and overlays the results on the real-time video display; S7: The recognition results are automatically cached in the storage device and merged with the original training data to generate a new data set, which is triggered by a scheduled task. The incremental data set is used for action recognition model training, and the latest action recognition model file is automatically Make a substitution. 2. A method of appearance detection and action recognition for dense objects in a small space according to claim 1, characterized in that in the S1, the original encoded video is I={I _s } ^T , and I _s represents the third s frame, s represents the index value of each frame, and T represents the timestamp of the s-th frame; the camera obtains the s-th real-time image I _s of size M, and writes the real-time image into the pre-applied shared memory of size N , in which the shared memory can write a total of N images, that is, N=nM. 3. A method of appearance detection action recognition for dense objects in a small space according to claim 1, characterized in that, in the S2, the cyclic reading in the shared memory refers to the target detection model being written into the shared memory by the camera. Finally, according to the frame sequence information of the frame picture data transmitted by the queue, the corresponding picture data is read from the location of the shared memory. 4. A method for appearance detection action recognition of dense objects in a small space according to claim 1, characterized in that, in the S2, the object detection result of the frame image data is calculated as shown in the following formula: D _t =F(I _t ); In the formula, D _t represents the object detection result of the current frame I _t , specifically expressed as [c, w, h, x, y, thresh], where c represents the object category, w represents the width of the detection frame, and h represents the height of the detection frame, x represents the x-axis coordinate of the center point of the detection frame, y represents the y-axis coordinate of the center point of the detection frame, thresh represents the confidence of the detection frame; t represents the index value of the image data of each frame in the shared memory; function F represents the current frame I _t The operation process of the damo-yolo target detection model. 5. A method of appearance detection and action recognition for dense objects in a small space according to claim 1, characterized in that, in the S3, the calculation process of the real-time position and texture changes of the fused object is carried out for multi-target tracking processing, As shown in the following formula: X _t =G(I _t ,D _t ) In the _{formula ,} represents the width of the tracking detection frame, h represents the height of the tracking detection frame, x represents the x-axis coordinate of the center point of the tracking detection frame, y represents the y-axis coordinate of the center point of the tracking detection frame, thresh represents the confidence of the tracking detection frame; t represents each The index value of the frame's image data in the shared memory; the function G represents the operation process of the current frame _It and the detection result _Dt of the frame through the target tracking algorithm. 6. An appearance detection action recognition method for dense objects in a small space according to claim 1, characterized in that, in the S4, the action segmentation model performs the calculation process of appearance detection action behavior cutting, as follows: Shown: {L ₁ ,…,L _n }＝trim(V _m ) (I _start ,I _end )=P({L ₁ ,…,L _n }) In the formula, V _m represents the cached long video data obtained from the shared memory; the function trim represents the operation process of the current cached video V _m through the action segmentation algorithm; L _i represents the action category output by each frame of the cached long video, and i represents The number of categories set by the system; function P represents the operation process of obtaining the isolated action start frame I _start and end frame I _end based on the frame data action category _Li . 7. An appearance detection action recognition method for dense objects in a small space according to claim 1, characterized in that, in the S5, the spatiotemporal region attention model and a large convolution kernel are used to perform spatiotemporal analysis of the video sequence. Feature extraction, and then complete the calculation process of action classification, as shown in the following formula: Y＝W(I _start ,…,I _end ) In the formula, Y represents the action recognition prediction result, I _start ,...,I _end represents all picture data of the input image as the start point and end point defined by the action segmentation algorithm, obtained from the shared memory; the function W represents the current input frame sequence For the operation process of the action recognition algorithm, please refer to step 5 of the specific implementation manner for its specific process. 8. A method of appearance detection and action recognition for dense objects in a small space according to claim 1, characterized in that in said S6, the background display system receives detection, tracking and behavior segmentation of front-end edge devices through socket communication. and action recognition results. At the same time, the image is obtained from the shared memory, the image timestamp is aligned with the detection information timestamp, and the results are overlaid and displayed on the real-time video. 9. An appearance detection action recognition method for dense objects in a small space according to claim 1, characterized in that, in the S7, the system automatically caches the recognition results into a storage device and merges them with the original training data to generate The new data set is triggered by a scheduled task. The incremental data set is used for action recognition model training, and the latest action recognition model file is automatically replaced. 10. Provide a device based on the method of any one of claims 1 to 9, characterized in that it includes a processor, a communication interface, a memory, a display, and a communication bus, wherein the processor, the communication interface, the memory, and the display communicate through The bus completes mutual communication; Memory, used to store computer programs; A display used to display the execution results and real-time images of the processor; The processor is used to implement the method steps described in claims 1-9 when executing a program stored in the memory.