CN104079872A - Content-based Video Image Processing and Human-Computer Interaction Method - Google Patents

Abstract

The present invention discloses a content-based video image processing and human-computer interaction method, which belongs to the field of wireless communication technology. The present invention comprises the following steps: 1. extracting feature information from an input video; 2. establishing a scene model according to scene features, and performing scene-based moving object detection on this basis; 3. establishing a target model according to target content, and performing target-based detection on this basis; 4. analyzing the prominence (importance level) of the detected target by fusing scene-based and target-based detection, and generating a detection result, which includes the detected target and the importance level; 5. guiding the visual attention of monitoring personnel to important targets by using a computer vision attention selection mechanism, thereby effectively utilizing limited image display space.

Description

Content-based video image is processed and man-machine interaction method

Technical field

The present invention relates to content-based video image and process and man-machine interaction method, belong to wireless communication technology field.

Background technology

At present, China's coal bed gas is produced video monitoring still in exploitation preliminary stage, still has the problem of the following aspects:

(1) because coal bed gas field production belongs to field work, gas well quantity is many, distributional region is wide, landform more complicated, and road conditions is poor, is therefore difficult to too greatly by artificial patrol method workload the problem existing in discovery production in time.Artificial patrol method is subject to weather effect large, and running into sleety weather may can not arrive well site a few days.From the viewpoint of fail safe and following enterprise development high efficiency two, production process unattended operation, visual remote management become coal bed gas industry development inexorable trend.

(2) under the conditions of work such as the numerous and surrounding environment complexity of well head, traditional inter-frame difference video data handling procedure and traditional " windows display " man-machine interaction mode can not meet the demand to well head long-time continuous video monitoring.How studying a set of with low cost, dependable performance, be applicable to technology and the equipment of video image acquisition, transmission, storage, processing and the man-machine interaction of coal bed gas field exploitation, is urgent need to solve the problem.

Summary of the invention

The present invention is directed to the proposition of above problem, process and man-machine interaction method and develop content-based video image.

The present invention includes following steps:

1. to input video characteristic information extraction;

2. according to scene characteristic, set up model of place, and carry out on this basis detecting based on the moving object of scene;

3. according to object content, set up object module, and carry out on this basis the detection of based target;

4. by merging the detection based on scene and based target, analyzing and testing target significance (important level), generates and comprises the testing result that detects target and important level;

5. by using computer vision attention selection mechanism, monitor staff's visual attention is directed to important goal, thereby effectively utilizes limited image display space.

The principle of the invention and beneficial effect: in complex scene environment, numerous coal bed gas well heads are carried out to video monitoring in the wild, will produce massive video data.How under massive video data condition, effectively to realize man-machine interaction, be a major issue of Large Scale Video Monitoring System.As previously mentioned, coal bed gas production video monitoring system has the feature of many monitoring points, long time continuous working.Under this condition of work, monitor staff cannot focus one's attention on for a long time, the each control point of point by point scanning, thereby monitor staff's attentiveness must be attracted on abnormal picture.Research shows, human perception system mainly utilizes visual attention to select mechanism, overcomes multiple dimensioned in complicated visual scene perception and high complexity issue.Visual attention selects mechanism in computer vision system, to be paid attention to, and is applied to engineering practice as design theory.For example, in European opportunity of combat design, visual attention selection mechanism is used to refer to leads design man-machine interactive system.Fig. 1 .1 institute representation model is the application of visual attention selector built in moving object detection aspect.In this project, we will follow and utilize human attention to select mechanism, realize the efficient man-machine interaction of coal bed gas production video monitoring system.

Brief description of the drawings

Fig. 1 flow chart of the present invention.

Embodiment

1, content-based video image processing and man-machine interaction method comprises the steps:

1. to input video characteristic information extraction;

By the method for statistics with histogram, extract condition of a fire RGB triple channel color characteristic, extract the scene point-of-interest feature based on SURF;

2. according to scene characteristic, set up model of place, and carry out on this basis detecting based on the moving object of scene;

For the color characteristic (referring to step 3.) of the coal bed gas production testing target condition of a fire, extract the area-of-interest of video scene and condition of a fire color similarity, in this region, extract SURF characteristic point, according to Euler's distance the matching characteristic point of 5 × 5 neighborhoods of R passage calculated for pixel values consecutive frame SURF characteristic point, calculate the alternate position spike △ d of the corresponding SURF characteristic point of consecutive frame, the data symbol that record position is poor, utilize character check method (seeing formula (1)), if judge whether the poor variation of SURF characteristic point position meets Gaussian Profile and meet Gaussian Profile (seeing formula (2)), judge that current SURF characteristic point is as scene characteristic point, record its positional information, set up the sparse model of place based on characteristic point, otherwise, be moving object SURF characteristic point.

$f\left(\mathrm{\&Delta;d}\right)=\left\{\begin{array}{cc}+1& \mathrm{\&Delta;d}>0\\ -1& \mathrm{\&Delta;d}<0\\ 0& \mathrm{\&Delta;d}=0\end{array}\right.---\left(1\right)$

$E\left(f\left(\mathrm{\&Delta;d}\right)\right)=|\frac{\min (p(\mathrm{\&Delta;d}>0),p(\mathrm{\&Delta;d}<0))}{\max (p(\mathrm{\&Delta;d}>0),p(\mathrm{\&Delta;d}<0))}-1|<\mathrm{\&epsiv;}---\left(2\right)$

If the scene characteristic point of optimum Match detected in present frame, record according to the new feature dot position information mating with scene characteristic point in present frame the historical position message queue that scene characteristic point changed with the certain hour cycle, the renewal of positional information queue is carried out according to the principle of first in first out.Matching characteristic point in mark present frame is scene characteristic point.For the scene characteristic point that in model of place, other is not mated, carry out a character check, upgrade model of place, mark scene characteristic point and foreground features point; If the characteristic point of mating with existing scene characteristic point do not detected in present frame, the not new feature point of coupling detecting in present frame is carried out to a character check again, if new scene characteristic point detected, upgrade model of place.Otherwise, be labeled as foreground features point.

3. according to object content, set up object module, and carry out on this basis the detection of based target;

In the training stage, the region that manually produces the condition of a fire in segmented video image, utilizes the method for statistics with histogram to extract the three-channel color characteristic of condition of a fire RGB, determines and detects the color threshold threshold=200 that the condition of a fire occurs; Be partitioned into the region at condition of a fire place in training video image according to this color threshold, in this region, extract the SURF feature of the condition of a fire, calculate the center-of-mass coordinate (seeing formula (3) and formula (4)) of all SURF characteristic points in current time t video image, and the alternate position spike of consecutive frame SURF characteristic point barycenter (is shown in formula (5) and formula

(6)), these centroid positions are changed and carry out probability statistics, set up the motion model of the condition of a fire, meet $E\left(f\left({\mathrm{\&Delta;}}_x^{t_2-t_1}\right)\right),E\left(f\left({\mathrm{\&Delta;}}_y^{t_2-t_1}\right)\right)$ Condition;

$x_{\mathrm{centriod}}^t=\frac{1}{n}\left({\mathrm{\&Sigma;}}_{i=1}^n{x}_i^t\right)---\left(3\right)$

$y_{\mathrm{centriod}}^t=\frac{1}{n}\left({\mathrm{\&Sigma;}}_{i=1}^n{y}_i^t\right)---\left(4\right)$

${\mathrm{\&Delta;}}_x^{t_2-t_1}=x_{\mathrm{centriod}}^{t_2}-x_{\mathrm{centriod}}^{t_1}---\left(5\right)$

${\mathrm{\&Delta;}}_y^{t_2-t_1}=y_{\mathrm{centriod}}^{t_2}-y_{\mathrm{centriod}}^{t_1}---\left(6\right)$

At detection-phase, extract moving object SURF characteristic point according to model of place, record the poor data symbol of adjacent video scene motion object SURF characteristic point centroid position, utilize character check method, judge whether the poor variation of SURF characteristic point centroid position meets Gaussian Profile.If meet Gaussian Profile, judge that the condition of a fire occurs, otherwise the condition of a fire does not occur.

In sum, the judgment basis that the condition of a fire detects is as follows:

I(R)>threshold?∩I(G)>threshold

$\&cap;I\left(B\right)>\mathrm{threshold}\&cap;E\left(f\left({\mathrm{\&Delta;}}_x^{t_2-t_1}\right)\right)<\mathrm{\&epsiv;}\&cap;E\left(f\left({\mathrm{\&Delta;}}_y^{t_2-t_1}\right)\right)<\mathrm{\&epsiv;}$

Wherein R, G, B represents respectively the three-channel pixel value of red, green, blue.

4. by merging the detection based on scene and based target, analyzing and testing target significance, generates and comprises the testing result that detects target and important level;

On the basis of detecting in scene moving object, scene moving object SURF characteristic point centroid position change profile P in statistics certain time interval T _back, calculate the condition of a fire distribution of movement P that itself and Target Modeling training stage set up _firekL distance, determine and detect target possibility occurrence and define three important level according to KL distance, see formula (7).

$f\left(D_{\mathrm{KL}}\left(P_{\mathrm{back}}\right|\left|P_{\mathrm{fire}}\right)\right)=\left\{\begin{array}{cc}1& 0<D_{\mathrm{KL}}<{\mathrm{\&epsiv;}}_1\\ 2& {\mathrm{\&epsiv;}}_1\&le;D_{\mathrm{KL}}<{\mathrm{\&epsiv;}}_2\\ 3& D_{\mathrm{KL}}\&GreaterEqual;{\mathrm{\&epsiv;}}_3\end{array}\right.---\left(7\right)$

5. by using computer vision attention selection mechanism, monitor staff's visual attention is directed to important goal, thereby effectively utilizes limited image display space.

According to detecting target and important level thereof, the preferential high key video sequence information of display level, is directed to important goal by monitor staff's visual attention, thereby effectively utilizes limited image display space.

The english term the present invention relates to is:

SURF (Speeded-Up Robust Features): accelerate robust feature

KL (Kul lback-Leibler divergence): relative entropy distance

The above; it is only preferably embodiment of the present invention; but protection scope of the present invention is not limited to this; any be familiar with those skilled in the art the present invention disclose technical scope in; be equal to replacement or changed according to technical scheme of the present invention and inventive concept thereof, within all should being encompassed in protection scope of the present invention.

Claims (5)

1. A content-based video image processing and human-computer interaction method, characterized in that it comprises the steps of: ① Extract feature information from the input video; ②Establish a scene model according to the scene characteristics, and perform scene-based moving object detection on this basis; ③According to the target content, establish a target model, and on this basis, perform target-based detection; ④By integrating scene-based and target-based detection, analyze the saliency of the detection target, generate detection results, and generate detection results including detection targets and importance levels; ⑤ By using the computer vision attention selection mechanism, the monitor's visual attention is directed to important targets, thereby effectively utilizing the limited image display space. 2. content-based video image processing and human-computer interaction method according to claim 1, it is characterized in that: according to scene feature, set up scene model, and carry out scene-based moving object detection method on this basis as: for Coalbed methane production detects the color characteristics of the target fire, extracts the region of interest where the video scene is similar to the fire color, extracts SURF feature points in this area, calculates the position difference corresponding to the SURF feature point in adjacent frames of the video image, and records the position difference Use the sign test method to determine whether the change of the position difference of the SURF feature point satisfies the Gaussian distribution; if it satisfies the Gaussian distribution, it is determined that the current SURF feature point is a scene feature point, otherwise, it is a moving object SURF feature point. 3. content-based video image processing and human-computer interaction method according to claim 1, it is characterized in that: according to target content, set up target model, and on this basis, carry out target-based detection method as: in the training stage , aiming at the color features of the fire detection target in coalbed methane production, extract the color features of the red, green and blue channels of the fire, and determine the color threshold for detecting the occurrence of the fire; segment the fire location in the training video image according to the color threshold In this area, the SURF feature of the fire is extracted, the centroid position of all SURF feature points in the current frame video image is calculated, and the position difference of the SURF feature point centroid of the adjacent frame is calculated, and the probability statistics of the centroid position difference changes are carried out. Establish the motion model of the fire; in the detection stage, extract the SURF feature points of the moving object according to the scene model, record the data symbols of the centroid position difference of the SURF feature point of the moving object in the adjacent video scene, and use the sign inspection method to determine the centroid position difference of the SURF feature point Whether the change of satisfies the Gaussian distribution, if it satisfies the Gaussian distribution, it is determined that the fire has occurred, otherwise, the fire has not occurred. 4. content-based video image processing and human-computer interaction method according to claim 1, it is characterized in that: by fusing scene-based and target-based detection, analyzing the detection target salience, the method for generating detection results is: in the scene Based on the detection of moving objects, the position change distribution of SURF feature points of scene moving objects within a certain period of time is counted, and the KL method is used to calculate the similarity between it and the fire motion distribution established in the target modeling training stage, and the occurrence of the detection target is determined according to the similarity likelihood and define levels of importance. 5. content-based video image processing and human-computer interaction method according to claim 1, it is characterized in that: by using computer vision attention selection mechanism, the visual attention of monitoring personnel is guided to important target, thereby effectively The method of using limited image display space is: according to the detection target and its importance level, the key video information with high level is displayed preferentially, and the visual attention of the monitoring personnel is directed to the important target, so as to effectively use the limited image display space. The generated detection results include detection targets and importance levels.