CN103824284B - Key frame extraction method based on visual attention model and system - Google Patents

Abstract

The invention discloses a key frame extraction method and system based on a visual attention model. The extraction method includes: In the space domain, the method uses the binomial coefficient to filter the global contrast to detect the saliency, and uses the adaptive threshold to extract the target area. The algorithm can not only keep the boundary of the salient object area well, but also the saliency degree in the area is relatively uniform. Then, in the time domain, the method defines the saliency of the motion, estimates the target motion through the homography matrix, uses the key points instead of the target to detect the saliency, and then fuses the data of the saliency in the space domain to propose a boundary based on the energy function. The extended method obtains bounding boxes as salient object regions in the temporal domain. Finally, the method reduces the richness of the video by salient target regions, and uses a shot-adaptive method combined with online clustering for keyframe extraction.

Description

A key frame extraction method and system based on visual attention model

technical field

The invention relates to the technical field of video analysis, in particular to a method and system for extracting key frames based on a visual attention model.

Background technique

With the rapid development of Internet technology, we have entered the era of information explosion, and various network applications and the rapid development of multimedia technology have been widely used. As a common network information carrier, video is vivid and intuitive, with strong appreciation and expressiveness, and thus has been widely used in various fields, resulting in a massive increase in video data. Taking the famous video website YouTube as an example, every There are about 60 hours of videos uploaded by users per minute (data taken from January 23, 2012), and it still maintains a growing trend. How to quickly and effectively store, manage and access massive video resources has become an important issue in the current video application field. Due to the time-domain correlation of video, in the traditional way, users need to browse the entire video from beginning to end to grasp a piece of video information. While irrelevant videos take up a lot of time for users, they also waste a lot of network bandwidth. Therefore, we need to add auxiliary information to the video to help users filter better. At present, the traditional text annotation method is generally used in mature systems, which are manually classified by manual methods, and artificial semantics are given to videos with text such as titles and descriptions. Faced with a large number of videos, this task is not only a lot of work, but also different people have different understandings of videos, and others cannot judge whether the videos meet their interests through the author's text annotation.

Therefore, there is an urgent need for an automated way to effectively summarize videos.

Contents of the invention

In order to solve the deficiencies of the prior art, the present invention firstly provides a video key frame extraction method based on a visual attention model, which can effectively obtain key frames that are very representative of video shots.

Another object of the present invention is to propose a video key frame extraction system based on a visual attention model.

In order to achieve the above object, the technical solution of the present invention is:

A video key frame extraction method based on a visual attention model, comprising:

In the spatial domain, the binomial coefficient is used to filter the global contrast for saliency detection, and the adaptive threshold is used to extract the target area; this method can not only better maintain the boundary of the salient target area, but also the saliency in the area is relatively uniform .

In the time domain, define the saliency of the motion, estimate the target motion through the homography matrix, use the key points instead of the target to detect the saliency, fuse the data of the saliency in the space domain, and propose a method based on the boundary expansion of the energy function to obtain the bounding box as a salient target area in the temporal domain;

The richness of the video is reduced by salient target regions, and a shot-adaptive method combined with online clustering is used for keyframe extraction.

A video key frame extraction system based on a visual attention model, the system includes a salient area extraction module and a key frame extraction module;

Specifically, the salient region extraction module includes:

The airspace salient area extraction module is used to extract the salient area on the airspace;

The time-domain key point saliency acquisition module is used to extract the saliency value of the key point on the time domain;

The fusion module is used to fuse the salient areas in the air domain and the key points in the time domain, and finally obtain the salient areas.

The key frame extraction module includes:

Static lens key frame extraction module, used for key frame extraction of static lens;

Dynamic lens key frame extraction module, used for key frame extraction of dynamic lens;

The lens adaptive module is used for the control between the static lens key frame extraction module and the dynamic lens key frame extraction module.

Compared with the prior art, the beneficial effects of the present invention are: adopting the present invention can automatically and effectively summarize the video, and effectively obtain key frames that are very representative of the video shots.

Description of drawings

Fig. 1 is a flow chart of key frame extraction of a static shot in the present invention.

Fig. 2 is a flow chart of key frame extraction of a dynamic shot in the present invention.

Fig. 3 is a flow chart of key frame extraction of an adaptive lens according to the present invention.

detailed description

The present invention will be described in further detail below in conjunction with the accompanying drawings.

A kind of video key frame extraction method based on visual attention model disclosed by the present invention, the specific implementation is as follows:

First, in the spatial domain, the binomial coefficient is used to filter the global contrast for saliency detection, and the adaptive threshold is used to extract the target area. The specific method is as follows:

(11) The binomial coefficient is constructed according to the Yang Hui triangle, and the normalization factor of the N layer is 2 ^N . Select the fourth layer, so the filter coefficient B ₄ =(1/16)[1 4 6 4 1];

(12) Let I be the original stimulus intensity, is the mean value of the surrounding stimulus intensity, It is the convolution of I and B4 _; the pixels are measured in the form of vectors in the CIELAB color space to measure the strength of the stimulus, and the contrast of the stimulus is the Euclidean distance between the two CIELAB vectors. Therefore, for the stimulus detection of the pixel (x, y) for

(13) After obtaining the saliency measurement set S _s =(s ₁₁ ,s ₁₂ ,...,s _NM ), use the adaptive threshold to extract the target area, where s _ij (0≤i≤N,0≤ j≤M) is the salient degree of the pixel point (i, j), M and N are the width and height of the image respectively.

Specifically, the adaptive threshold is used to extract the target area through the following methods:

(21) Define the global saliency detection calculation formula of pixel point (x, y)

where A is the detected area, is the stimulus intensity of the pixel (x, y ₎ after the original image is filtered by filter B4, I(i, j) is the original stimulus intensity of the pixel (i, j), M, N are the width and height of the image respectively ;

(22) Accelerate the operation through the histogram, and map the original stimulus intensity I to the stimulus space In the end, the stimulus felt by the user The significance of

where D is the stimulus The distance between the m nearest stimuli m is an artificial control parameter, m is taken as 8 in the present embodiment;

(23) Specify the foreground and background regions by changing the threshold T _s , and then obtain the threshold of the smallest energy function as the optimal threshold; the definition of the energy function with T _s as the threshold is as follows:

Among them, S _n is obtained by formula (2), λ is the weight of significant target energy, in this embodiment, λ=1.0, N is the total number of pixels of the image, f(T _s ,S _n )=max(0,sign (S _n -T _s )), V(I,T _s ,s) is a measure of the similarity of surrounding stimuli, select the salient point under the current T _s and its 8 neighboring pixels to form a point to calculate the Pair, dist(p,q) is the spatial distance between two points, σ is a human control parameter, and σ=10.0 is taken in this embodiment.

Therefore, given an image and a saliency map, T _s is estimated by minimizing the energy function. When a pixel belongs to a salient object, it is marked as 1, and the rest are marked as 0. The parameters λ and σ need to be manually set in advance.

Then, in the time domain, the saliency of motion is defined, the target motion is estimated through the homography matrix, and key points are used to replace the target for saliency detection, and then the data of spatial saliency is fused, and a method based on energy function boundary extension is proposed Obtain the bounding box as the salient target area in the time domain, the specific method is as follows:

(31) Given an image, use the FAST (Features from Accelerated SegmentTest) feature point detection algorithm with good real-time performance to obtain the key points of the image;

(32) Given two adjacent frames of images, use FLANN (Fast Library for Approximate Nearest Neighbor) for fast correlation point matching;

(33) Use the homography matrix (Homography Matrix) H to describe the motion of key points. Since one H only describes one motion form, and the motion forms in the same video are diverse, multiple H pairs of different Movement is described. In this embodiment, the RANSAC algorithm is used to obtain a series of homography matrix estimates H={H ₁ ,H ₂ ,...,H _n } through continuous iterations;

(34) Define the temporal saliency of key points as

Among them, A _m is the distribution area of all key points in the motion state H _m , and W and H are the width and height of the video image;

(35) Fusing the saliency value of the spatial domain with the temporal saliency value of the acquired key points;

(36) adopt an energy function-based boundary extension method to obtain bounding boxes as salient object regions in the temporal domain.

Specifically, the saliency value of the spatial domain and the temporal saliency value of the obtained key points are fused by the following methods:

(41) Defining the contrast of a motion salience Among them, the time-domain saliency value S _t of key points is obtained by formula (5), is the mean value of the time-domain saliency value of key points;

(42) The salience of motion should be aimed at targets that still have a strong degree of discrimination in the airspace, so the statistical range of the time-domain saliency S _t should be limited. Let p _i be the i-th key point of S _t , Then p _i should satisfy in is the mean value of the spatial significance value;

(43) Define the weight of the time domain weight of airspace The temporal domain and spatial domain saliency values of the key points satisfying (42) are added according to the weight.

Specifically, the time-domain salient target region extraction is realized by the following methods:

The prominent key point p of the airspace is used as the seed point, and the seed area adopts a rectangular bounding box B. Let b _i be the four sides of the bounding box B, i∈{1,2,3,4} is the number of up, down, left, and right, and the boundary is extended The algorithm is as follows:

Initialization: The upper, lower, left, and right vertices of the bounding box B are set to the position of the key point p, and point p is the internal point of the bounding box B.

Step 1: Calculate the saliency energy E _outer (i) on the outer boundary of bi and the saliency energy E _inner (i) on the inner boundary of bi in increasing order from _i =1, the calculation of the energy function is as formula (4), and then The weight that can be expanded to calculate the boundary is Where l _i is the length of the i-th side of the current bounding box B.

Step 2: If w(i)≥ε, the i-th side extends outward by one pixel unit. ε is the threshold for extending the decision, which needs to be set in advance. In the experiments in this paper, it is set to 0.8T _s ′, where T _s ′ is the mean value of the spatial saliency in the bounding box.

Step 3: If no new edge is extended in step 2, stop the algorithm and output bounding box B; otherwise, repeat step 1 and step 2.

Finally, the richness of the video is reduced by the significant target area, and the key frame extraction is performed by a shot adaptive method combined with online clustering. The specific method is as follows:

(51) Convert the RGB color space of the salient area to the HSV color space, and take the H component (hue) and S component (saturation) to calculate the hue-saturation histogram (Hue-saturation Histogram). Note that Hp(i) is the i-th bin value of the hue-saturation histogram of the salient target area in frame _p , and this embodiment uses the Bhattacharyya distance to measure the visual distance between two frames

(52) A shot adaptive method combined with online clustering is used for key frame extraction, the clustering method of static shots is the main method, and the clustering method of dynamic shots is supplemented. For static shots, online clustering is performed based on the hue-saturation histogram of the salient area, and any frame in the cluster is selected as a key frame. For dynamic shots, firstly, the significant moving objects are tracked, and then the tracking of the significant moving objects is used as the basis for online clustering, and the location information of the prominent objects is used as the basis for extracting key frames from the clustering.

Specifically, as shown in Figure 1, the online clustering of static shots is realized through the following steps:

Initialization: Calculate the hue-saturation histogram of the first frame of the static shot The initial cell number N=1, and the As the vector of the centroid C ₁ of the cell cell Cell ₁ , C ₁ =f ₁ .

S11: If the current frame p belongs to a static shot, calculate the hue-saturation histogram H _p of the current frame.

S12: Calculate the visual distance between p and the centroid of each cell, and obtain the smallest visual distance cell where m is the index number of the cell.

S13: Compare D _sal (p, C _m ) with the threshold ε _c , when D _sal (p, C _m ) ≤ ε _c , classify p into Cell _m , and then replace Cell _m with H _p Centroid. Otherwise, add cell _N+1 , use H _p as the vector of centroid C _N+1 of cell _N +1, and finally update cell number N=N+1.

S14: Repeat S11, S12 and S13 for all still shot frames.

Specifically, as shown in Figure 2, the key frame extraction of dynamic shots is realized through the following steps:

Initialization: Get the first frame of the dynamic camera.

S21: Obtain the tracking target area, initialize particles or resample, extract the next frame of the video, judge whether the frame is empty, and if it is empty, end.

S22: Obtain FAST feature vectors, use the FLANN algorithm for matching, update the weights of feature vectors, and end if the feature vectors are insufficient.

S23: Update the weights of each particle, calculate the key frame weights and the target area, and jump to S21.

A key frame extraction system based on a visual attention model disclosed in the present invention includes a salient area extraction module and a key frame extraction module.

Salient region extraction modules include:

The airspace salient area extraction module is used to extract the salient area on the airspace;

The time-domain key point saliency acquisition module is used to extract the saliency value of the key point on the time domain;

The fusion module is used to fuse the salient areas in the air domain and the key points in the time domain, and finally obtain the salient areas.

The keyframe extraction module includes:

Static lens key frame extraction module, used for key frame extraction of static lens;

Dynamic lens key frame extraction module, used for key frame extraction of dynamic lens;

The lens adaptive module is used for the control between the static lens key frame extraction module and the dynamic lens key frame extraction module.

The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the protection scope of the present invention. It should be understood that the present invention is not limited to the implementation solutions described here. The purpose of these implementation solutions descriptions is to help those skilled in the art Those skilled in the art practice the present invention. Any person skilled in the art can easily make further improvements and improvements without departing from the spirit and scope of the present invention, so any modifications, equivalent replacements and improvements made within the spirit and principles of the present invention should be Included within the protection scope of the claims of the present invention.

Claims (8)

1. A key frame extraction method based on visual attention model, for extracting the key frame of video, it is characterized in that, comprising: In the spatial domain, the binomial coefficient is used to filter the global contrast for saliency detection, and the adaptive threshold is used to extract the target area; In the time domain, define the saliency of the motion, estimate the target motion through the homography matrix, use the key points instead of the target to detect the saliency, fuse the data of the spatial saliency, and propose a method based on the boundary expansion of the energy function to obtain the bounding box as a salient target area in the temporal domain; Reduce the richness of the video through significant target areas, and use a shot adaptive method combined with online clustering for key frame extraction; In the spatial domain, saliency detection is performed by filtering the global contrast with the binomial coefficient, and the target area is extracted by using an adaptive threshold. The specific method is as follows: (11) The binomial coefficient is constructed according to the Yang Hui triangle, and the normalization factor of the N layer is 2 ^N ; the fourth layer is selected, and the filter coefficient B ₄ =(1/16)[1 4 6 4 1]; (12) Let I be the original stimulus intensity, is the mean value of the surrounding stimulus intensity, It is the convolution of I and B4 _; the pixels are measured in the form of vectors in CIELAB color space to measure the strength of the stimulus, and the contrast of the stimulus is the Euclidean distance between the two CIELAB vectors, so for the stimulus detection of the pixel (x, y) for $\mathrm{<span\; class="notranslate">\; S</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span><span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; I</span>}}_{{\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; 4</span>}}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{\mathrm{<span\; class="notranslate">\; I</span>}}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>$ (13) After obtaining the saliency measurement set S _s =(s ₁₁ ,s ₁₂ ,…,s _NM ), use the adaptive threshold to extract the target area, where s _ij is the saliency of the pixel point (i,j) , 0≤i≤N, 0≤j≤M, M, N are the width and height of the image respectively; implement the adaptive threshold to extract the target area by the following method: (21) Define the global saliency detection calculation formula of pixel point (x, y) ${\mathrm{<span\; class="notranslate">\; S</span>}}_{\mathrm{<span\; class="notranslate">\; g</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; A</span>}}\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; N</span>}}{<span\; class="notranslate">\; \&Sigma;</span>}}\underset{\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}}{<span\; class="notranslate">\; \&Sigma;</span>}}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; I</span>}}_{{\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; 4</span>}}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; I</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; )</span>$ where A is the detected area, is the stimulus intensity of pixel (x, y ₎ after the original image is filtered by filter B4, I(i, j) is the original stimulus intensity of pixel (i, j), M, N are the width and height of the image respectively ; (22) Accelerate the operation through the histogram, and map the original stimulus intensity I to the stimulus space In the end, the stimulus felt by the user The significance of $\mathrm{<span\; class="notranslate">\; S</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; I</span>}}_{{\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; 4</span>}}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; I</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>\mathrm{<span\; class="notranslate">\; D.</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; I</span>}}_{{\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; 4</span>}}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; I</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; )</span>}\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}}{<span\; class="notranslate">\; \&Sigma;</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; D.</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; I</span>}}_{{\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; 4</span>}}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; I</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; I</span>}}_{{\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; 4</span>}}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; I</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; I</span>}}_{{\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; 4</span>}}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; I</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span><span\; class="notranslate">\; )</span>{\mathrm{<span\; class="notranslate">\; S</span>}}_{\mathrm{<span\; class="notranslate">\; g</span>}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; I</span>}}_{{\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; 4</span>}}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; I</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 3</span>}<span\; class="notranslate">\; )</span>$ where D is the stimulus The distance between the m nearest stimuli (23) Designate the foreground and background regions by changing the threshold T _s , and then obtain the threshold of the smallest energy function as the optimal threshold; the definition of the energy function with T _s as the threshold is as follows: $\mathrm{<span\; class="notranslate">\; E.</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; I</span>}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; T</span>}}_{\mathrm{<span\; class="notranslate">\; the\; s</span>}}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; \&lambda;</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; \&sigma;</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; \&lambda;</span>}\underset{\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; N</span>}}{<span\; class="notranslate">\; \&Sigma;</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; T</span>}}_{\mathrm{<span\; class="notranslate">\; the\; s</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; S</span>}}_{\mathrm{<span\; class="notranslate">\; no</span>}}<span\; class="notranslate">\; )</span>{\mathrm{<span\; class="notranslate">\; S</span>}}_{\mathrm{<span\; class="notranslate">\; no</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; V</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; I</span>}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; T</span>}}_{\mathrm{<span\; class="notranslate">\; the\; s</span>}}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; \&sigma;</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 4</span>}<span\; class="notranslate">\; )</span>$ where S _n is obtained by formula (2), λ is the weight of salient target energy, N is the total number of pixels in the image, f(T _s ,S _n )=max(0,sign(S _n -T _s )), V (I, T _s , σ) is a measure of the similarity of the surrounding stimuli, select the salient points under the current T _s and the pixel points in its 8 neighborhoods to calculate the Pair, dist(p,q) is the spatial distance between two points, and σ is the control parameter. 2. The method according to claim 1, characterized in that, in the time domain, the saliency of the motion is defined, the target motion is estimated through the homography matrix, the key points are used instead of the target for saliency detection, and then the spatial domain is fused For saliency data, a method based on energy function boundary extension is proposed to obtain the bounding box as the salient target area in the time domain. The specific method is as follows: (31) Given an image, use the FAST feature point detection algorithm with good real-time performance to obtain the key points of the image; (32) Given two adjacent frames of images, use FLANN to perform fast correlation point matching; (33) Use multiple homography matrices H to describe the movement of key points, and use the RANSAC algorithm to obtain a series of homography matrix estimates H={H ₁ ,H ₂ ,...,H _n through continuous iteration }; (34) Define the temporal saliency of key points as ${\mathrm{<span\; class="notranslate">\; S</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; p</span>}}_{\mathrm{<span\; class="notranslate">\; m</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; A</span>}}_{\mathrm{<span\; class="notranslate">\; m</span>}}}{\mathrm{<span\; class="notranslate">\; W</span>}<span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; h</span>}}\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; no</span>}}{<span\; class="notranslate">\; \&Sigma;</span>}}{\mathrm{<span\; class="notranslate">\; A</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}\mathrm{<span\; class="notranslate">\; D.</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; p</span>}}_{\mathrm{<span\; class="notranslate">\; m</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 5</span>}<span\; class="notranslate">\; )</span>$ Among them, A _m is the distribution area of all key points in the motion state H _m , and W and H are the width and height of the video image; (35) A method based on energy function boundary expansion is used to obtain bounding boxes as salient object regions in the time domain. 3. The method according to claim 2, characterized in that, the saliency value of the space domain is fused with the time domain saliency value of the key point obtained by the following method: (41) Define a motion salience contrast Among them, the time-domain saliency value S _t of key points is obtained by formula (5), is the mean value of the time-domain saliency value of key points; (42) Let p _i be the ith key point of S _t , then p _i should satisfy in is the mean value of the spatial significance value; (43) Define the weight of the time domain weight of airspace The time-domain and space-domain saliency values satisfying the key points in step (42) are added according to weights. 4. The method according to claim 2, characterized in that, the extraction of the salient target region in time domain is realized by the following method: The prominent key point p of the airspace is used as the seed point, and the seed area adopts a rectangular bounding box B. Let b _i be the four sides of the bounding box B, i∈{1,2,3,4} is the number of up, down, left, and right, and the boundary is extended The algorithm is as follows: Initialization: The upper, lower, left, and right vertices of the bounding box B are set to the position of the key point p, and point p is the internal point of the bounding box B; Step 1: Calculate the saliency energy E _outer (i) on the outer boundary of bi and the saliency energy E _inner (i) of the inner boundary in increasing order from _i =1, the calculation of the energy function is as formula (4), and then The weight to calculate the boundary extension is Where l _i is the length of the i-th side of the current bounding box B; Step 2: If w(i)≥ε, then the i-th edge is extended outward by one pixel unit; ε is the set threshold for expanding the judgment, set to 0.8T _s ′, and T _s ′ is the significant space in the bounding box degree mean; Step 3: If no new edge is extended in step 2, stop the algorithm and output bounding box B; otherwise, repeat step 1 and step 2. 5. method according to claim 1, is characterized in that, reduces the richness of video by significant target area, adopts the shot adaptive method in conjunction with online clustering to carry out key frame extraction, and concrete method is as follows: (51) Convert the RGB color space of the salient area to the HSV color space, take the H component and the S component to calculate the hue-saturation histogram, record H _p (i) as the hue-saturation histogram of the p-th frame salient target area For the i-th bin value, the Bhattacharyya distance is used to measure the visual distance between the p and q frames (52) Using the lens adaptive method combined with online clustering to extract key frames, the clustering method of static shots is the main method, and the clustering method of dynamic shots is supplemented; For static shots, online clustering is performed based on the hue-saturation histogram of the salient area, and any frame in the cluster is selected as a key frame; For dynamic shots, firstly, the significant moving objects are tracked, and then the tracking of the significant moving objects is used as the basis for online clustering, and the location information of the prominent objects is used as the basis for extracting key frames from the clustering. 6. The method according to claim 5, characterized in that, the online clustering of static shots is realized by the following steps: Initialization: Calculate the hue-saturation histogram of the first frame of the static shot The initial cell number N=1, and the As the vector of the centroid C ₁ of the cell cavity Cell ₁ , C ₁ =f ₁ ; S11: If the current frame p belongs to a static shot, calculate the hue-saturation histogram H _p of the current frame; S12: Calculate the visual distance between p and the centroid of each cell, and obtain the smallest visual distance cell Where m is the index number of the cell; S13: Compare D _sal (p, C _m ) with the threshold ε _c , when D _sal (p, C _m ) ≤ ε _c , classify p into Cell _m , and then replace Cell _m with H _p Centroid; otherwise, increase the cell cavity Cell _N+1 , use H _p as the vector of the centroid C _N+1 of the cell cavity Cell _N+1 , and finally update the cell number N=N+1; S14: Repeat S11, S12 and S13 for all still shot frames. 7. method according to claim 5, is characterized in that, realizes the key frame extraction of dynamic shot by the following steps: Initialization: Get the first frame of the dynamic shot; S21: Obtain the tracking target area, initialize particles or resampling, extract the next frame of the video, judge whether the frame is empty, and if it is empty, end; S22: Obtain FAST eigenvectors, use the FLANN algorithm for matching, update the weights of the eigenvectors, and end if the eigenvectors are insufficient; S23: Update the weights of each particle, calculate the key frame weights and the target area, and jump to S21. 8. A system of the key frame extraction method based on the visual attention model described in any one of claims 1 to 7, characterized in that, comprising a salient region extraction module, a key frame extraction module; The salient region extraction module includes: The airspace salient area extraction module is used to extract the salient area on the airspace; The time-domain key point saliency acquisition module is used to extract the saliency value of the key point on the time domain; The fusion module is used to fuse the salient areas in the airspace and the key points in the time domain, and finally obtain the salient areas; The key frame extraction module includes: Static lens key frame extraction module, used for key frame extraction of static lens; Dynamic lens key frame extraction module, used for key frame extraction of dynamic lens; The lens adaptive module is used for the control between the static lens key frame extraction module and the dynamic lens key frame extraction module.