CN101493890B - Dynamic vision caution region extracting method based on characteristic - Google Patents

Abstract

The invention relates to a feature-based dynamic visual attention area extraction method in the technical field of machine vision. The steps are: use the independent component analysis method to sparsely decompose a large number of natural images, obtain a set of filtering basis functions and a corresponding set of reconstruction basis functions, divide the input image into m×m RGB small blocks, and project them to these On the group basis, the features of the graph are obtained; using the principle of effective coding, the incremental encoding length index is measured for each feature; the third step, according to these incremental encoding length indicators, the energy of each feature is redistributed to process each small The saliency of the block, so that the saliency map is finally obtained. The invention can eliminate "time slice" and continuous sampling, so that the data of different frames can guide the processing of salience together, solve the problem that the saliency of different frames needs to be processed independently, and realize dynamics.

Description

Feature-based Dynamic Visual Attention Region Extraction Method

technical field

The present invention relates to a method in the technical field of image processing, in particular to a feature-based dynamic visual attention region extraction method.

Background technique

With the continuous development of artificial intelligence technology, machine vision is used more and more in real life. It mainly uses computers to simulate human visual functions, but it is not just a simple extension of the human eye, but more importantly, it has the human brain. A part of the function - to extract information from the image of objective things, process and understand it, and finally use it for actual detection, measurement and control. Because machine vision has the characteristics of fast speed, large amount of information, and multiple functions, it is widely used in quality inspection, identity authentication, object detection and recognition, robots, and automatic cars.

At present, engineering has been able to produce sensors that exceed the capabilities of the human eye in all aspects (including viewing angle, visual acuity, broad-spectrum, and dynamic characteristics). It can be said that the exploration of "vision" has reached a certain level, but machine vision The system needs not only "vision", but also "feeling". Since the human selective attention mechanism ensures the high efficiency of human eyes to obtain information, it has attracted people's attention and research, and various extraction techniques of visual attention regions have been proposed and widely used. For example, using the visual attention region extraction technology based on the selective attention mechanism to find the regions of interest in the image, and then preferentially search in these regions, thereby improving the efficiency of object detection and recognition; using the region of interest found, Efficient image compression (low compression ratio in the region of interest, high compression ratio in other regions), and image scaling (the zoom ratio of the region of interest is larger than that of other regions), and so on. The extraction technology of visual attention area has a huge advantage in the efficiency of information acquisition, so it often appears in the process of machine vision system processing.

According to the literature search of the existing technology, the extraction technology of visual attention area is the saliency map (Saliency Map) proposed by Koch and Ullman in 1985, which was later perfected by Itti and Koch to form a whole set of saliency map system. For details, please refer to the literature: "LAURENT I, CHRISTOF K, ERNST N.A model of saliency-based visual attention for rapid scene anlysis[J]. IEEE Transactions on PAMI, 1998, 20(11): 1254-1259", (Author: LAURENT I , CHRISTOF K, ERNST N, Title: A saliency-based visual attention model for fast scene analysis, Journal: IEEE Transactions on Pattern Analysis and Machine Intelligence, 1998, Vol. 20, No. 11, 1254-1259 Page). This method is a space-based extraction technology. First, the image is divided into several parallel channels such as color, direction, brightness, and texture, and then information is extracted for each channel to form a picture that retains the topological structure of the image, but at the same time extracts the information of the features. Response to feature maps with recorded strengths and weaknesses. Next, each feature map is filtered by a series of "Mexican hat (Difference of Gaussian)" functions, which are two different scales of Gaussian Function obtained after function difference. This function is very sensitive to detect changes, but very weak to the signal of a mass, which has general biological significance. Next, use the Winner-Take-All model of the competition network to compare different attention regions, and finally generate a map of the saliency of each point in the world, called a saliency map. Although this method and the subsequent spatial-based analysis techniques performed well in many scenes, they almost inevitably face the following problems: 1) only pay attention to a specific part of visual cues, 2) the distribution of attention It is not continuous in time. For example, when observing continuous images, the system cannot consider the situation of multiple frames, which leads to the need to re-analyze the saliency map separately at each moment, which greatly reduces the continuity and reliability of the system. Moreover, when the viewing angle and the position of the object change, since there is no tracking mechanism for features, the prediction of the new saliency map is likely to be offset from the previous frame. Furthermore, a range of visual attentional behaviors, such as inhibition of return, and viewpoint shift, are not well represented by spatially-based analysis techniques.

Contents of the invention

The purpose of the present invention is to overcome the deficiencies of the prior art and provide a method for extracting dynamic visual attention regions based on features. The definition of salience in this method is based on the features themselves rather than the spatial distribution differences of the features, which can eliminate the "time slice" ", continuous sampling, so that the data of different frames (time) can guide the processing of saliency together, which solves the problem that the saliency of different frames (time) needs to be processed independently, and realizes dynamics.

The present invention is achieved through the following technical solutions, comprising the following steps:

In the first step, the independent component analysis method is used to sparsely decompose a large number of natural images, and a set of filtering basis functions and a corresponding set of reconstruction basis functions are obtained. The input image is divided into m×m RGB small blocks, and projected to Based on this group, the features of the graph are obtained;

The second step is to use the effective encoding principle, that is, when a system is effectively encoded, its entropy is the largest, and measure the incremental encoding length index for each feature;

In the third step, according to these incremental encoding length indicators, the saliency of each small block is processed by redistribution of the energy of each feature, so as to finally obtain a saliency map.

The first step, as described, is as follows:

① Divide the training picture into several RGB color blocks of m×m pixel size, and vectorize each small block. Sampling natural pictures to obtain a large number of m×m RGB color blocks, which are used as training samples. The value of m can be 8, 16 or 32. m is the side length of each RGB color block.

②Basic functions (A, W) are trained through the standard independent component analysis (ICA) method. The number of basis functions is m×m×3=3m ² , namely $W=[{\mathrm{<figure-callout\; id="1"\; label="w"\; filenames="H2009100466886E00012.png"\; state="\{\{state\}\}">w</figure-callout>}}_1,w_2,...w_{3m^2}],$ Where w _i is the i-th filtering basis function (the size of A is the same as W, 1≤i≤3m ² ). A and W are the basis functions trained by the ICA method, and their values can be in any range, determined by the input.

③For any picture X, divide it into n m×m RGB small blocks to form a sampling matrix X=[x ₁ , x ₂ ,…x _n ], where x _k is the vectorization of the kth image block Indicates (1≤k≤n), linear transformation of x _k $S_k=W{x}_k=[{\mathrm{the\; s}}_{k,1},{\mathrm{the\; s}}_{k,2},...,{\mathrm{the\; s}}_{k,3m^2}],$ where W is the trained filtering basis function. Then S _k is the corresponding coefficient of the basis function, that is, the feature of the image block x _k , and s _k,i is the corresponding coefficient of the i-th basis function, that is, the value of the i-th feature. Do the same process for all x _k to get the feature S=[S ₁ , S ₂ , . . . , S _n ] of X. n is the number of RGB blocks that the input image X is divided into, and its value is determined by the size of the input image X and the value of m.

After the first step of processing, for the input image X, 3m ² features S have been constructed, and then the second step will be performed.

The second step is as follows:

① For each feature, calculate the activation rate p _i

${\mathrm{<span\; class="notranslate">\; p</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}{{\mathrm{<span\; class="notranslate">\; the\; s</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; i</span>}}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}{{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}{{\mathrm{<span\; class="notranslate">\; the\; s</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 2.1</span>}<span\; class="notranslate">\; )</span>$

This amount represents the average energy release level for that feature.

变成：②Considering the change of entropy on the activation rate p _i of the i-th feature, that is, the incremental encoding length index of the i-th feature, let $p=\{{\mathrm{<figure-callout\; id="1"\; label="p"\; filenames="H2009100466886E00012.png"\; state="\{\{state\}\}">p</figure-callout>}}_1,p_2,p_{3m^2}\},$ is the probability distribution of a random variable. Assuming that the feature activation rate distribution at a specific moment is p, when the i-th feature is activated, it will bring a small disturbance ε to p _i , so the new distribution

become:

$\stackrel{<span\; class="notranslate">\; ^</span>}{\mathrm{<span\; class="notranslate">\; p</span>}}<span\; class="notranslate">\; =</span>\left\{\begin{array}{cc}\frac{{\mathrm{<span\; class="notranslate">\; p</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; \&epsiv;</span>}}{\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; \&epsiv;</span>}}<span\; class="notranslate">\; ,</span>& \mathrm{<span\; class="notranslate">\; if\; j</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="i\; p\; j"\; filenames="H2009100466886E00012.png"\; state="\{\{state\}\}">i</figure-callout></span><figure-callout\; id="1"\; label="i\; p\; j"\; filenames="H2009100466886E00012.png"\; state="\{\{state\}\}">\; </figure-callout>}& \\ \frac{{\mathrm{<span\; class="notranslate">\; p</span>}}_{}}{}\frac{{\mathrm{<span\; class="notranslate">\; j</span>}}_{}}{\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; \&epsiv;</span>}}<span\; class="notranslate">\; ,</span>& \mathrm{<span\; class="notranslate">\; if\; j</span>}<span\; class="notranslate">\; \&NotEqual;</span>\mathrm{<span\; class="notranslate">\; i</span>}\end{array}\right.<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 2.2</span>}<span\; class="notranslate">\; )</span>$

Therefore, the incremental encoding length of the i-th feature is:

$\mathrm{<span\; class="notranslate">\; ICL</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; p</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{<span\; class="notranslate">\; \&PartialD;</span>\mathrm{<span\; class="notranslate">\; h</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; )</span>}{<span\; class="notranslate">\; \&PartialD;</span>{\mathrm{<span\; class="notranslate">\; p</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; h</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; p</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; log</span>}{\mathrm{<span\; class="notranslate">\; p</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; p</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}\mathrm{<span\; class="notranslate">\; log</span>}{\mathrm{<span\; class="notranslate">\; p</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 2.3</span>}<span\; class="notranslate">\; )</span>$

The present invention links energy, features and salience by means of the principle of predictive coding. Incremental Coding Length (ICL) measures the rate of change of perceptual entropy for each feature. This metric is used to guide the energy allocation so that the system as a whole achieves predictive coding—that is, common information causes the system to respond as little as possible, while rare information usually triggers a strong system response.

The third step is as follows:

①Divide the salient feature set SF according to the obtained incremental coding length index of each feature:

SF={i|ICL(p _i )>0} (3.1)

The partition {SF, SF} uniquely identifies the features that lead to an increase in the entropy of the overall system. And this division has a clear mathematical meaning. For a feature, only when the other is rare in the feature distribution, that is, when a new observation is made on the feature, the entropy of the overall feature distribution p will increase.

②According to the principle of predictive coding, redistribute energy between each feature, and assign weight d _i (i∈SF) to feature i in the salient feature set:

${\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; ICL</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; p</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; )</span>}{{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; \&Element;</span>\mathrm{<span\; class="notranslate">\; SF</span>}}\mathrm{<span\; class="notranslate">\; ICL</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; P</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; if\; i</span>}<span\; class="notranslate">\; \&Element;</span>\mathrm{<span\; class="notranslate">\; SF</span>}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 3.2</span>}<span\; class="notranslate">\; )</span>$

And for non-significant features, define its $d_k=0(k\&NotElement;\mathrm{SF}).$

③For a picture block x _k , its saliency is defined as m _k :

${\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; \&Element;</span>\mathrm{<span\; class="notranslate">\; SF</span>}}{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}{\mathrm{<span\; class="notranslate">\; w</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; T</span>}}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 3.3</span>}<span\; class="notranslate">\; )</span>$

④ After obtaining the saliency of each small block of the picture, the saliency map M of the whole picture is generated by reconstructing the base A:

$\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; \&Element;</span>\mathrm{<span\; class="notranslate">\; SF</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}{\mathrm{<span\; class="notranslate">\; A</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 3.4</span>}<span\; class="notranslate">\; )</span>$

where A _k represents the kth column vector of the reconstructed basis A.

It can be seen from formula (3.3) that the saliency of a small image block is not constant, but changes with time. And, because in the method of the present invention, sampling is a continuous process, and the weight of the feature will change continuously with the increase of sampling, then the sampling change can be successfully understood as the influence of the context on the weight of feature attention. The so-called "significant features" are significant because they are relative to the feature distribution of the current context.

The beneficial effects of the present invention are: (1) because the filter base that adopts is trained in advance, so when processing a new input picture, do not need to train the base function again, make processing speed very fast, efficiency is high, can do to real-time processing. (2) Since the saliency is analyzed based on the feature itself rather than the difference in the spatial distribution of the feature, the limitation on the spatial structure of the image is eliminated. In terms of processing, due to the continuous sampling, the "time slice" is eliminated, so that the data of different frames (time) can guide the processing of saliency together, which solves the problem that the saliency of different frames (time) needs to be processed independently, and realizes dynamic.

Description of drawings

Figure 1. Saliency maps for static images;

Among them: (a), (d), (g) are input pictures, (b), (e), (h) are salient maps generated by the present invention, (c), (f), (i) are marked eye movement data.

Figure 2. Saliency map of video screens (dynamic vision).

Detailed ways

The embodiments of the present invention are described in detail below in conjunction with the accompanying drawings: this embodiment is implemented on the premise of the technical solution of the present invention, and detailed implementation methods and specific operating procedures are provided, but the protection scope of the present invention is not limited to the following the described embodiment.

1. Feature construction

(1) The size of the RGB color block used is 8×8. By sampling a large number of natural pictures, 120,000 8×8 RGB color blocks are obtained. These RGB color blocks are the training data of the basis function.

(2) Use the ICA method to train the basis functions (A, W). Since 8×8 RGB color blocks are used as training samples, that is, m=8, the number of basis functions is 3×8 ² =192.

(3) For the input color picture, for example, the size of the picture is 800×640, divide it into 8000 8×8 RGB color blocks, that is, n=8000, and form a sampling matrix X=[x ₁ , x ₁ , .. ., x ₈₀₀₀ ], where x _k is the vectorized representation of the kth image block, and it is linearly transformed S _k = Wx _k = [s _{k, 1} , s _{k, 2} ,..., s _{k, 192} ], where W is the trained filtering basis function. Then S _k is the corresponding coefficient of the basis function, that is, the feature of the picture block x _k . s _{k, i} is the coefficient corresponding to the i-th basis function, which is the i-th feature.

2. Measuring Incremental Code Length (ICL) Index

(1) For each feature, calculate its activation rate p according to formula (2.1).

(2) According to the activation rate of each feature, measure its incremental encoding length index according to the formula (2.3)

3. Generate saliency map

(1) According to the incremental coding length index of each feature obtained in 2, use the formula (3.1) to divide the salient feature set SF.

(2) Using formula (3.2), redistribute the energy of each feature in the salient feature set.

(3) For the picture block x _k , according to formula (3.3), process its saliency m _k

(4) With the saliency of each small block of the input image, use the formula (3.3) to obtain the saliency map M of the input image.

Example 1: Saliency Maps for Still Images

8×8 RGB patches are used to train the basis functions (A, W), and their dimensionality is 192.

For an input picture with a size of 800×640, divide it into 8000 8×8 RGB color blocks, ie n=8000, to form a sampling matrix X=[x ₁ , x ₁ , . . . , x ₈₀₀₀ ]. And the corresponding coefficients of the basis functions, that is, the characteristics of X, are calculated by the formula S=WX.

The activation rate p of each feature is obtained by formula (2.1), and the incremental coding length index of each feature is measured according to p and formula (2.3).

Divide the salient feature set SF according to the incremental coding length index of each feature and the formula (3.1), and use the formula (3.2) to redistribute the energy of each feature in the salient feature set. Then, for the picture block x _k , according to the formula (3.3), process its saliency m _k , and finally use the formula (3.4) to generate the saliency map M of the input picture.

When sampling each small image block on a static image sequentially, the feature distribution characteristics of the image can be estimated, thereby constructing a saliency map, and the generated saliency map will be further compared with human eye movement data Comparative analysis to verify the correctness of the model. In Figure 1, (a), (d), (g) are input pictures, (b), (e), (h) are salient maps generated by the present invention, (c), (f), (i) are Annotated eye movement data. In the embodiment, the document "BRUCE N, TSOTSOS J.Saliency Basedon Information Maximization [J], Advances in Neural Information Processing Systems, 2006, 18: 155-162" (Author: BRUCE N, TSOTSOS J. Topic: Based on Information The salience degree of quantity maximization, magazine: Advanced Neural Information Processing System, 2006, No. 18, pages 155-162) provided eye movement data as a benchmark, compared the model and the traditional model, the results show that the present invention has achieved Best score.

Example 2: Salient map in video

Compared with previous similar methods, a great advantage of the method of the present invention is that it is continuous. Incremental encoding length is a process of continuous updating. The change of the distribution of the feature activation rate can be based on the space domain or the time domain. If considering that the time domain change is a Laplace distribution, assuming that p ^t is the tth frame, then p ^t can be considered as the cumulative sum of the previous characteristic responses:

${\mathrm{<span\; class="notranslate">\; p</span>}}^{\mathrm{<span\; class="notranslate">\; t</span>}}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; Z</span>}}\underset{\mathrm{<span\; class="notranslate">\; \&tau;</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\mathrm{<span\; class="notranslate">\; exp</span>}<span\; class="notranslate">\; (</span>\frac{\mathrm{<span\; class="notranslate">\; \&tau;</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; t</span>}}{\mathrm{<span\; class="notranslate">\; \&lambda;</span>}}<span\; class="notranslate">\; )</span>{\stackrel{<span\; class="notranslate">\; ^</span>}{\mathrm{<span\; class="notranslate">\; p</span>}}}^{\mathrm{<span\; class="notranslate">\; \&tau;</span>}}$

where λ is the half-life, $Z=\&Integral;{\hat{p}}^t\left(x\right)\mathrm{dx}$ is a standardized function.

When extracting visual attention from a video screen, it usually faces the problem of target movement and observation perspective movement. However, under the framework of the feature-based attention model, these problems are easily solved, because the features always move with the position of the object in the field of view.

Analyze the signal-to-noise ratio (SNR) of the image, which is defined as follows:

$\mathrm{<span\; class="notranslate">\; SNR</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; \&Element;</span>\mathrm{<span\; class="notranslate">\; f</span>}}{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; t</span>}}}{{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; \&NotElement;</span>\mathrm{<span\; class="notranslate">\; f</span>}}{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}^{\mathrm{<span\; class="notranslate">\; t</span>}}}$

where F is the manually annotated "foreground". After manually annotating 250 frames, the saliency of each frame is processed. The process is consistent with the saliency map of still pictures except that the feature activation rate p is different. Afterwards, the generated saliency map can be compared with manual annotations to analyze the signal-to-noise ratio value. Fig. 2 reflects the result, among the figure, the screenshot of the first line video, the second line reflects the signal-to-noise ratio of the present invention, and the last line is the signal-to-noise ratio of the Itti model, as can be seen from the figure, the average signal-to-noise ratio of the present invention The noise ratio is 0.4803, much better than the 0.1680 of the current mainstream Itti model.

Claims (1)

1. a feature-based dynamic visual attention region extraction method is characterized in that comprising the following steps: In the first step, the independent component analysis method is used to sparsely decompose a large number of natural images, and a set of filtering basis functions and a corresponding set of reconstruction basis functions are obtained. The input image is divided into m×m RGB small blocks, and projected to On this set of filtering basis functions, the characteristics of the graph are obtained, as follows: ① Divide the training picture into several small RGB color blocks of m×m pixel size, and vectorize each small block, and sample the natural picture to obtain a large number of m×m RGB color blocks, which are used as training samples , the value of m is 8, 16 or 32, and m is the side length of each RGB color block; ②Basic functions (A, W) are trained through the standard independent component analysis method, and the number of basic functions is m×m×3=3m ² , namely

Where w _i is the i-th filtering basis function, the number of basis function A is the same as the basis function W, 1≤i≤3m ² , A, W are the basis functions trained by the ICA method; ③For any picture X, divide it into n m×m RGB small blocks to form a sampling matrix X=[x ₁ , x ₂ ,…x _n ], where x _k is the vectorization of the kth image block Indicates that 1≤k≤n, linear transformation of x _k

Where W is the trained filtering basis function, then S _k is the corresponding coefficient of the basis function, which is the feature of the image block x _k , s _{k, i} is the corresponding coefficient of the i-th basis function, which is the value of the i-th feature , do the same process for all x _k , get the feature S=[S ₁ , S ₂ ,...,S _n ] of X, n is the number of the input picture X is divided into RGB small blocks, and its value is Determined by the size of the input image X and the value of m; In the second step, the incremental encoding length index is measured for each feature, as follows: ① For each feature i, calculate its activation rate p _i : and order

Then p is the probability density distribution of a random variable, and its entropy is H(p); ②Calculate the incremental code length ICL(p _i ) of the i-th feature: The third step is to process the saliency of each small block by redistributing the energy of each feature according to these incremental encoding length indicators, so as to finally obtain a saliency map, as follows: ① According to the obtained incremental coding length index of each feature, divide the salient feature set SF: SF＝{i|ICL(p _i )＞0} ②According to the principle of predictive coding, redistribute energy between each feature, and assign weight d _i to feature i in the salient feature set, i∈SF:

if, i ∈ SF And for non-significant features, define its weight ③Then for a small image block x _k , its saliency is defined as m _k :

where d _i is the weight assigned to the i-th feature in step 2; ④ After obtaining the saliency of each small block of the picture, the saliency map M of the whole picture is generated by reconstructing the base A:

where A _k represents the kth column vector of the reconstructed basis A.

Images