CN110933520B - A surveillance video display method and storage medium based on spiral abstract - Google Patents

Abstract

The invention discloses a monitoring video display method and storage medium based on a spiral abstract. The method includes: 1) extracting key frames from the monitoring video to be processed to obtain a key frame set; 2) interested in each key frame Region extraction; 3) Generate a spiral video summary of the surveillance video according to the determined region of interest information; perform moving target detection on key frames, and use the key frames extracted from the region of interest on the spiral timeline to determine whether there is a moving target , divided into multiple sections; Count the various categories of targets appearing in the surveillance video, and locate the effective information area that the user is interested in; 4) Based on the effective information area, navigate and locate the surveillance video through the spiral video summary; 5) Through the spiral video summary The hyperlinks are used to construct the association between the surveillance video scenes; by selecting and merging the spiral video abstracts, the editing and merging of the associated scenes of the surveillance video are realized, and the surveillance video for display is obtained.

Description

A surveillance video display method and storage medium based on spiral abstract

technical field

The invention belongs to the field of human-computer interaction, and in particular relates to a monitoring video display method and a storage medium based on a spiral abstract.

Background technique

With the development of the Internet, pure text content is no longer the main content of data interaction, and most of the data used for interaction is in image or video format. Therefore, how to quickly retrieve the content that users are interested in from video data and navigate to the corresponding area is a hot issue of video summarization. Video is a streaming media composed of a series of interrelated pictures in a certain chronological order. The amount of information provided by the video is very huge, and it is usually difficult for users to obtain the main content of the video in a short period of time. Current mainstream video applications often provide users with a way to interact with videos through a horizontal timeline. Users can watch videos by clicking on the timeline or by fast-forwarding. However, this interaction can easily lead to users skipping important shots and scenes. . Therefore, summarizing the main content of the video through the video summary is an effective way to help users quickly obtain the video content.

Surveillance video data usually has the following characteristics: large amount of data, various data formats, slow processing speed, high cost and low density of video usable information. At the same time, different from videos such as movies and animations, because movies, animations and other videos have to meet the needs of the audience and give the audience a better audio-visual effect, the main target in the video picture is usually located in the center of the lens, and the quality such as clarity and contrast will be affected. relatively high. However, due to the camera placement, shooting angle, light and other reasons, the video quality of surveillance video will be significantly worse than that of movies and animations, and the target in the lens is often not located in the center of the lens, and some are not very obvious. In addition, surveillance video has a lot of redundant information, often hours of surveillance video, only a few minutes of effective information. The traditional way of obtaining effective information from surveillance video often requires a lot of manpower and material resources, and the super cognitive load makes it easy for staff to miss key information. Therefore, visual analysis of surveillance video content is very necessary.

Compared with the common video abstract form that arranges the video abstracts in a straight line or grid, the spiral video abstract can present more video information in a limited space on the one hand; The keyframes are arranged for the time axis, and there is no grid-like arrangement, which maintains the visual continuity of the user, and makes the presentation of the video content more in line with the user's cognitive habits. The monitoring video visual analysis method based on the spiral abstract technology is based on the video abstract in the spiral form, combined with the moving target detection result data to display the monitoring video information. Interactive functions such as video summary navigation and positioning video, spiral video summary multi-scale browsing, sketch annotation, etc., to achieve fast and effective acquisition of surveillance video content, based on spiral video summary hyperlinks and fusion operations, to achieve convenient construction of related scene connections in surveillance video .

SUMMARY OF THE INVENTION

The purpose of the present invention is to provide a monitoring video display method based on a spiral summary, which can efficiently display and visualize the monitoring video content, and provide a convenient and efficient interactive mode for monitoring video browsing, so as to speed up the acquisition process of the monitoring video content. The method proposed by the present invention mainly includes adaptive threshold key frame extraction for surveillance video content, region of interest extraction based on object detection, moving object detection and surveillance video statistical information generation, surveillance video effective information area search based on spiral abstraction, The process of quick browsing of surveillance video based on spiral summary and construction of surveillance video correlation scene based on spiral summary. The goal of the present invention is to display the monitoring video content in a form that is easy for users to understand through a method for visualizing monitoring video content based on spiral abstracts, supplemented by interactive methods such as navigation, positioning operations, and sketch interaction for spiral abstracts. The rapid and effective acquisition of content can solve the problem that the current monitoring video has more redundant information and it is difficult to effectively obtain key information. The advantages of surveillance video content analysis based on the spiral summary are: the main content of the video is described by the video summary in the form of a spiral combined with the detection results of moving objects, and the video browsing is convenient and fast; the effective information area of the surveillance video is quickly found, and the retrieval and positioning process of the target is accelerated; fast Browsing surveillance video content to speed up the macro acquisition process of surveillance video content; based on the spiral video summary, the correlation analysis of related scenes between surveillance videos is implemented to assist users in decision-making.

In order to realize the above-mentioned purpose of the invention, the present invention adopts the following technical scheme:

A monitoring video display method based on spiral summary, the steps are:

1) Aiming at the real-time and accuracy requirements of the surveillance video visual analysis system, an algorithm for extracting key frames based on color histograms provides an adaptive threshold real-time key frame extraction algorithm based on surveillance video. The set of keyframes;

2) In view of the complex characteristics of surveillance video scenes, based on the target detection algorithm yolov3, a region of interest extraction algorithm for surveillance video is given, and 1) the important foreground information of the determined key frames is extracted, and the extracted key frames are further processed. , which provides a prerequisite for the generation of spiral video abstracts;

3) use 2) the determined region of interest information to generate the spiral video summary for surveillance video; and analyze the spiral video summary line based on the moving target detection algorithm, and divide the spiral time axis into a plurality of regions with whether the moving target is included, To improve the user's analysis and retrieval efficiency of surveillance video content; generate statistical information of various types of objects in surveillance video through target detection of surveillance video content, and visualize the statistical information from different setting angles to reduce users' cognition load;

4) Through the target distribution pie chart and the spiral summary target type distribution map generated by the statistical information, combined with the moving target detection results, the effective information area (including the moving target in the area where the user is interested in the target category belongs to) is carried out on the spiral video abstract. area) search to help users filter out a large amount of useless information, quickly locate effective information areas that users are interested in, and improve user browsing efficiency while ensuring accuracy;

5) After determining the effective information area that the user is interested in in 4), the surveillance video is located through the spiral summary navigation, and the functions such as multi-scale browsing and sketch annotation based on the spiral summary help the user to quickly and accurately obtain the surveillance video content in the effective information area. Realize quick browsing of surveillance video content;

6) Through the hyperlink based on the spiral summary (the quick jump of related scenes in multiple videos is realized through the simple sketch interaction on the spiral summary) and the editing and merging operation of the spiral summary, it is possible to realize the multiple monitoring between different surveillance videos or within the surveillance video. Correlation analysis of plots between scenes.

Further, by improving the algorithm for extracting key frames based on color histogram proposed by Kumthekar et al. 207-214, ISSN 2278–0882, July-2013), gives an adaptive threshold real-time key frame extraction algorithm based on surveillance video to extract surveillance video key frames; in the process of key frame extraction, the threshold is adaptively adjusted according to different surveillance video scenes , to achieve fast and accurate extraction of effective content of surveillance video.

Further, based on the target detection algorithm yolov3 (Redmon J, Farhadi A. YOLOv3: AnIncremental Improvement [J]. 2018.), a region-of-interest extraction algorithm for surveillance video is given; the important foreground information in key frames is detected by yolov3, The background information is filtered out, and the region of interest of the key frame in the complex scene of the surveillance video is extracted.

Further, on the basis of the determined key frame and region of interest information, through the SpiralTape algorithm (Liu Y, Ma C, Zhao G et al. An interactive SpiralTape video summarization [J]. IEEE Trans. Multimedia, vol. 18, no.7, pp.1269–1282, Jul. 2016.), generate a spiral video summary for surveillance video; based on the moving object detection algorithm motionNet (Use pytorch to do image semanticsegmentation. https://github.com/ISCAS007/torchseg ) Perform moving target detection on key frames, distinguish key frame sets based on whether there is a moving target, and divide the key frames on the spiral timeline after interest extraction into multiple sections to improve user awareness. Monitor the analysis and retrieval efficiency of video content.

Further, the target detection algorithm yolov3 is used to extract the statistical information of each category of targets (including five categories of "person", "rider", "car", "bus", and "truck") appearing in the surveillance video. The extracted statistical information generates a target distribution pie chart, a spiral summary target category distribution chart, and a target number-time change line chart.

Further, through the target distribution pie chart, the spiral summary target category distribution chart, combined with the moving target detection results, the effective information area search based on the spiral summary is realized; the target distribution pie chart is used to macroscopically understand the various occurrences in the surveillance video. The target of the category and its proportion of the number, to confirm whether there is an interesting target category in the surveillance video and the proportion of the number of interesting object categories that appear in the surveillance video.

Further, the distribution of each category of targets in the surveillance video on the spiral timeline is further understood from the distribution map of the spiral summary target category; the five colors of red, white, green, yellow and blue are used in the spiral summary target type distribution map. The dots represent the five types of targets "person", "rider", "car", "bus", and "truck" that often appear in surveillance videos. The larger the radius of the dot, the greater the current time period. The distribution area of the target category of interest to the user on the spiral time axis can be quickly located by the spiral summary target type distribution map, and the effective information search area can be narrowed down.

Further, the spiral time axis is divided into multiple areas according to whether the moving target is included in the moving target detection result. The gray area indicates that there is no moving target in the current area on the time axis, and the colored area indicates that there is a moving target area on the time axis. The area where the static target is located, further narrows the effective information area and realizes the fast search of the effective information area.

Further, on the basis of determining the effective information area, around the spiral video summary, the surveillance video is located through the spiral summary navigation, multi-scale browsing based on the spiral summary, sketch annotation and other functions help users quickly and accurately obtain the content of the surveillance video; based on the spiral video summary , Navigate and locate surveillance videos through the spiral summary, based on the multi-scale browsing of the spiral summary, and the sketch annotation function to quickly and accurately browse the surveillance video content in the effective information area; when browsing the surveillance video through the spiral video summary, you can navigate to the surveillance video from the interesting segment on the spiral summary. Learn more about the corresponding clips in the video, and use the sketch annotation function to record your understanding of the surveillance video. When browsing the area of interest, you can view the video summary at different granularities through the multi-scale browsing function provided by the system, from a global overview to a local detailed inspection, to fully understand the video content at multiple levels; you can also use the target quantity-time change The trend graph can quickly get the fluctuation of the number of targets of each category in the current camera location with time, so as to quickly browse the area of interest in the surveillance video.

Further, the construction of the connection between the surveillance video related scenes is realized through the spiral abstract hyperlink; specifically, the hyperlink construction of the associated scene is realized through the interaction of sketches, and through these associations, multiple surveillance videos are realized between different surveillance videos or within the surveillance video. Quickly jump between scenes to achieve correlation analysis of surveillance video plots.

Further, the clipping and merging of the relevant scenes of the surveillance video can also be realized by selecting and merging the spiral abstracts; through the sketch interaction, the sketch interaction can be used to select the related scene fragments on the multiple spiral abstracts, and generate a corresponding preview, and then Through the merging operation of the spiral video summary, the fast fusion of the related scenes in the surveillance video is realized.

The present invention also provides a computer-readable storage medium storing a computer program, wherein the computer program includes instructions for executing each step in any of the above-mentioned methods.

The main contents of the present invention include:

1. Adaptive threshold key frame extraction for surveillance video content

In view of the real-time requirement of surveillance video analysis system, the present invention provides an adaptive threshold real-time key frame extraction algorithm based on surveillance video by improving the color histogram-based algorithm of Kumthekar et al. The algorithm defines the similarity of the two images by comparing the difference in the color histogram of the two images, and each time only the video frame whose similarity with the last frame in the existing key frame set is less than a threshold threshold is reserved as a new key frame, Insert a keyframe collection.

For surveillance videos in different scenarios, they often correspond to different thresholds. If the threshold thresh is fixed, the number of extracted key frames will either be too large, resulting in the existence of a large amount of redundant information, or too little, resulting in the loss of a large amount of key information. Therefore, the present invention proposes a key frame extraction algorithm with an adaptive threshold, which can adaptively adjust the threshold thresh in different scenarios, so as to ensure that the main information of the surveillance video is not missed, and the redundant information in the extracted key frames is as little as possible. . Specifically, define the parameter minFrames, the minimum interval frame number, to ensure that at most one key frame appears in every minFrames frame picture (the default setting is 25), and define the parameter maximum interval frame number maxFrames to ensure that there is at least one key frame per maxFrames frame picture (default setting). is 80).

Different surveillance video targets have different occurrence probabilities. Set the time interval parameter distance, that is, take a frame every distance to calculate the similarity with the previous key frame. When there are few targets in the surveillance video, increase the value of distance. In order to speed up the processing speed, when there are many targets in the video, reduce the value of distance to make the result more accurate. See Algorithm 1 for the specific implementation:

Algorithm 1. Key frame extraction algorithm.

Input: Surveillance video V.

Output: The keyframe set keyFrameSet extracted from surveillance video.

1). Initialize the adaptive adjustment threshold thresh to 0.5, and define the key frame set.

2). Extract a frame from the surveillance video V in chronological order and record it as frame. If it is the first frame, save it as a key frame, record the frame as preKeyFrame, and record its ordinal number in the original video frame as pre_cnt. Otherwise, starting from the second frame, according to the principle of taking one frame every distance, a frame is extracted from the video frequency, and its ordinal number in the original video frame is recorded as cnt.

3). Calculate the respective color histograms of the current frame frame and the latest key frame preKeyFrame as frameHist and preFrameHist respectively, and normalize the histograms, and then calculate the histogram similarity between frameHist and preFrameHist as the difference between the frames Record it as score.

4). The adaptive threshold thresh update principle is as follows:

if score>thresh&&thresh<0.99&&(cnt-pre_cnt)>=maxFrames:thresh=score

if(cnt-pre_cnt)<minFrames&&score<thresh:thresh=thresh-0.05

If score<thresh, it is considered that the current frame frame is quite different from the previous key frame preKeyFrame, that is, the current frame frame is regarded as a key frame, the current frame frame is added to the keyFrameSet, and the

5). Determine whether the current frame extraction process from the video reaches the end of the video, if so, the key frame extraction process ends, and returns to the key frame set keyFrameSet, otherwise returns to 2), and continues to perform key frame extraction.

2. Region of interest extraction

De-bounding and blending between adjacent keyframes on the spiral. In this process, it is possible to dispose of important foreground information if no region of interest (ROI) extraction is performed on key frames. The present invention performs ROI extraction through key frames to highlight important foreground information of video key frames.

For surveillance video, the number of targets in the shot is often large, the individuals are small, and the distribution is wide, and they are usually not located in the central area of the shot. The traditional region of interest extraction algorithm performs poorly on surveillance video. In addition, existing image segmentation algorithms such as the classic GraphCuts algorithm and deeplabv3+, the current best-effective deep learning segmentation algorithm, perform generally in complex scenes of surveillance video, and often miss a lot of important foreground information.

The present invention is aimed at monitoring video, based on the yolov3 detection result to extract the key frame region of interest (ROI), that is, the target position in the current key frame is first located by target detection (mainly includes five categories: "person", "car", "bus" ", "truck", "rider"), and then calculate the minimum bounding box of all targets in the current frame. If the area of the minimum bounding box is greater than 200px, crop the image in the area where the minimum bounding box is located in the original image, and scale the size to 150x100 , and then output as ROI, otherwise it is determined that the key frame does not have a target, and the key frame is discarded (secondary screening of key frames), and the region of interest that meets the requirements can be obtained. Thanks to the excellent performance of yolov3 in surveillance video, almost all objects appearing in surveillance video can be accurately positioned, so the accuracy of the extracted region of interest (ROI) is much higher than that of segmentation and traditional ROI extraction algorithms.

3. Search for valid information area

When using traditional methods to browse videos, a lot of energy is often expended on monitoring redundant video information, and while consuming a lot of manpower and time, it is easy to miss important information. The invention takes the spiral video abstract as the center, uses the pie chart, the target type distribution map of the spiral abstract, and the detection result of the moving target to search for the effective information area on the spiral video abstract to help the user filter out a large amount of useless information, and quickly locate the user's feeling. The effective information area of interest can improve user browsing efficiency while ensuring accuracy.

The monitoring video analysis system proposed by the present invention can macroscopically understand the targets of each category appearing in the monitoring video and their proportions through the target distribution pie chart, and confirm whether there is an interesting target category in the monitoring video. After determining that the target of interest is contained in the video, it is necessary to further determine the exact position of the target in the surveillance video. The surveillance video analysis system organizes the surveillance video content based on the spiral summary, counts the target information in each key frame image on the spiral summary timeline, and designs the spiral summary target type distribution map to further visualize the targets in the surveillance video. , which is convenient for users to understand the distribution of various types of targets in the surveillance video on the timeline of the spiral video summary. In the spiral summary target type distribution diagram, five colored dots of red, white, green, yellow and blue are used to represent the five types of targets "person", "rider", "car", and "bus" that often appear in surveillance videos. , and "truck". The larger the radius of the dot, the more targets of this type exist in the current time period. Therefore, the distribution area of the target category of interest to the user on the spiral time axis can be quickly located by the spiral summary target type distribution map, so as to achieve the effect of reducing the effective information area.

In practice, the target that the user is interested in is often a moving target. Too many static targets will cause a certain amount of interference while wasting the energy of the staff. The present invention is based on the spiral video summary, and the effective information is further narrowed by moving target detection. area. According to the detection results of moving objects, the spiral timeline is divided into multiple regions according to whether moving objects are included, and the effective information region can be further narrowed by excluding the region where the static object is located.

The monitoring video analysis system designed by the present invention effectively organizes the monitoring video content based on the spiral video summary technology, and visualizes the target statistical information in the monitoring video by means of two visualization methods: pie chart and spiral summary target type distribution map. The moving target detection results of the spiral video summary can effectively narrow the search information area, and ensure that the user can quickly locate the target on the spiral video summary through a simple interactive way.

4. Video Quick View

After the effective information area of the surveillance video is determined by searching, there is still the problem of how to quickly browse the effective information area, so as to obtain the video content efficiently and accurately. The invention revolves around the spiral video abstract, locates the monitoring video through the spiral abstract navigation, multi-scale browsing based on the spiral abstract, and the sketch annotation function helps the user to quickly and accurately obtain the monitoring video content.

When learning about the surveillance video through the video summary, you can navigate to the corresponding segment in the surveillance video from the segment of interest on the helix summary for details. At the same time, in order to understand the video content more conveniently, the system provides the preview function of the segment of interest and the preview function of the 2s video near the segment of interest. When browsing the area of interest, you can view video summaries at different granularities through the multi-scale browsing function provided by the system, from a global overview to a local detailed inspection, to fully understand the video content at multiple levels. When browsing the video through the spiral video summary, the sketch annotation function can be used to record its understanding of the surveillance video. In addition, you can also quickly obtain the fluctuation of the number of targets of each category in the current camera location over time by monitoring the number of targets in the video-time trend line chart, such as the time periods of peak and trough periods. The fluctuation law of the number of various targets in the lot on the time axis can be focused on when individual abnormal situations occur.

5. Video scene association construction

In practical applications, there may sometimes be temporal or spatial connections between surveillance video scenes, such as several surveillance videos at the same location with different shooting times, or several surveillance videos with the same shooting time and location but different viewing angles, or even within the same surveillance video. There may be some associations in some scenarios. How to quickly construct the connection between these associated video scenes is a difficult problem.

The invention realizes the construction of the association between the surveillance video scenes through the spiral abstract hyperlinks; realizes the construction of the hyperlinks of the associated scenes through the interaction of sketches, and realizes the rapid jump among multiple scenes between different surveillance videos or within the surveillance videos through these associations , to realize the correlation analysis of the surveillance video plot. The invention also realizes the editing and merging of the surveillance video related scenes by selecting and merging the spiral abstracts, so as to realize the construction of the connection between the monitoring video related scenes; by using the sketches on the spiral abstracts to interactively select the related scene segments , and generate the corresponding preview, and then realize the rapid fusion of the related scenes in the surveillance video through the merging operation of the spiral video summary.

To sum up, compared with the prior art, the advantages and positive effects that the present invention has are as follows:

1. The present invention applies the video summarization technology in the spiral form to the analysis of surveillance video content. On the one hand, the use of the video summarization in the spiral form can save screen space. There is the problem of line spacing in the traditional grid-like arrangement, which maintains the visual continuity of the user and makes the content presentation more in line with the user's cognitive habits.

2. The present invention improves the algorithm for extracting key frames based on image histogram proposed by Kumthekar et al. for complex scenes of surveillance video, and provides an area of interest extraction algorithm based on the target detection results of yolov3, which can better adapt to complex scenes. surveillance video.

3. The present invention is based on the display advantages of the spiral video abstract, combined with the moving target detection result data, visualizes the statistical information of the video target from multiple angles, and is supplemented with interactive functions such as video abstract navigation and positioning video, spiral video abstract multi-scale browsing, sketch annotation and other interactive functions. Realize fast and effective acquisition of surveillance video content.

Description of drawings

Figure 1 is a schematic diagram of the system;

Fig. 2 is a flow chart of region of interest extraction based on spiral abstract;

Fig. 3 is a schematic diagram of an effective information area search process based on a spiral summary;

Fig. 4 is a schematic diagram of a monitoring video playback jump based on a spiral abstract hyperlink;

Fig. 5 is a schematic diagram of scene editing and merging based on spiral summary.

Detailed ways

In order to make those skilled in the art better understand the present invention, the following is a further detailed description of the surveillance video visual analysis technology based on the spiral abstract provided by the present invention with reference to the accompanying drawings, but it does not constitute a limitation of the present invention.

1. Select the video resources to be processed. In this example, the surveillance videos of 3 intersections in the same period of about 1 hour are selected from the existing surveillance video materials as the object;

2. Use the methods described in the previous steps 1 and 2 to perform key frame extraction and interest extraction processing on the video;

3. Use the method described in the previous step 3 to generate a spiral video summary for the surveillance video, and combine the moving target detection results to visualize the statistical information of the objects appearing in the surveillance video from multiple angles, forming a surveillance video visual analysis system based on the spiral summary. interface;

4. The method described in step 4 is used to realize the quick search of the effective information area in the surveillance video, and the search process is shown in Figure 3;

5. Use the method described in the previous step 5 to quickly browse the valid information area determined in the step 4;

6. Using the method described in the previous step 6, based on the spiral video summary, the correlation analysis of the plots between different surveillance videos or multiple scenes within the surveillance videos is realized, as shown in FIG. 4 and FIG. 5 .

The method and technology for visual analysis of surveillance video based on the spiral summary of the present invention have been described above in detail, but it is obvious that the specific implementation form of the present invention is not limited to this. For those skilled in the art, various obvious changes to the method without departing from the spirit of the present invention and the scope of the claims are within the protection scope of the present invention.

Claims (8)

1. a monitoring video display method based on spiral abstract, the steps of which comprise: 1) Extract key frames from the monitoring video to be processed to obtain a set of key frames; wherein the method for extracting key frames from the monitoring video to be processed is: a) initialize an adaptive adjustment threshold thresh; b) extract a frame in time sequence from the monitoring video V to be processed, if the frame is the first frame of the monitoring video V, then save it as a key frame; otherwise, take a frame at every set time interval distance principle, extract a frame from the surveillance video V; c) Calculate the color histogram of the current frame and the latest key frame and normalize the histogram, and then calculate the difference score between the current frame and the latest key frame according to the histogram; d) If the current difference score is greater than the current adaptive adjustment threshold thresh, and the current adaptive adjustment threshold thresh is less than the set maximum threshold H, and the frame sequence number of the current frame in the monitoring video V minus the latest key frame in the monitoring video V If the frame ordinal number is not greater than the set maximum interval frame number maxFrames, the current adaptive threshold thresh is updated to the current difference score; if the frame ordinal number of the current frame in the surveillance video V minus the frame of the latest key frame in the surveillance video V If the ordinal number is less than the set minimum interval frame number minFrames and the current difference score is less than the current adaptive threshold thresh, then the current adaptive threshold thresh is reduced by a set value a; e) If the difference score of the current frame is less than the current adaptive threshold thresh, the current frame is used as the latest key frame; 2) extracting a region of interest for each of the key frames; 3) generate the spiral video summary of the surveillance video according to the determined region of interest information; and carry out moving target detection to the key frame, and use the key frame after the extraction of the region of interest on the spiral time axis to determine whether there is a moving target, Divided into multiple sections; make statistics on various types of targets appearing in the surveillance video, and locate the effective information area that the user is interested in; 4) Based on the effective information area that the user is interested in the positioning, the surveillance video is positioned through the spiral video summary navigation; 5) The association between surveillance video scenes is constructed through the hyperlinks of the spiral video abstracts; the clipping and merging of the associated scenes of the surveillance videos are realized by selecting and merging the spiral video abstracts to obtain the surveillance video for display. 2 . The method according to claim 1 , wherein the statistics of various categories of targets appearing in the surveillance video are performed to generate a target distribution pie chart and a spiral summary target category distribution diagram to locate the effective information area that the user is interested in. 3 . 3. The method according to claim 2, wherein, based on the target distribution pie chart and the spiral summary target category distribution chart, the method for locating the effective information area that the user is interested in is: first obtain the target distribution pie chart through the target distribution pie chart. The targets of each category appearing in the surveillance video and their proportions, to confirm whether there is any target category of interest in the surveillance video, if there is an object of interest, organize the surveillance video content based on the spiral video summary, and analyze the spiral video summary timeline. The target information in each key frame image on the image is counted, and the distribution area of the target category of interest on the spiral time axis is located by using the spiral summary target type distribution map. 4. The method according to claim 3, characterized in that, in the helical summary target type distribution diagram, dots of different colors are used to represent different set targets in the surveillance video; the larger the radius of the dots, the presence of the current time period. The more corresponding goals. 5. The method according to claim 1, characterized in that, statistics of various categories of targets appearing in the surveillance video are performed to generate a target number-time trend line graph, which is used to obtain the number of targets of each category in the area where each camera is located. time fluctuations. 6. method as claimed in claim 1, it is characterized in that, extract key frame area of interest based on target detection algorithm yolov3: first by target detection to locate the target position of each setting target in the current key frame, then calculate all in the current frame. The minimum bounding box of the target. If the area of the minimum bounding box is greater than the set value, the image of the area where the minimum bounding box is located will be cropped and adjusted to the set size and output as the region of interest. Otherwise, it is determined that there is no target in the current keyframe and discarded. The current keyframe. 7 . The method of claim 1 , wherein the user determines the region of interest by clicking and delineating operations, and realizes the construction of connections between temporally or spatially related scenes through sketch interaction. 8 . 8. A computer-readable storage medium, characterized by storing a computer program, wherein the computer program includes instructions for executing each step in the method according to any one of claims 1-7.

Images