CN102521582B - Human upper body detection and splitting method applied to low-contrast video - Google Patents

Abstract

The invention relates to a method for detecting and segmenting the upper body of a human body suitable for low-contrast video, which mainly consists of two processes. Firstly, the connected region representing the foreground object is extracted from the current frame through background removal technology and morphological method, and then for each foreground region, its corresponding shape feature based on the polar coordinate two-dimensional histogram is extracted as a pre-training The input of a good support vector machine-based classifier outputs a class label corresponding to the human upper body class and the non-human upper body class. In the second step, when the area that has been recognized as a human body is misjudged as a non-human area, an energy function is used to characterize the corresponding area, and the wrong contour is corrected through an energy function minimization process. Finally, the background frame is updated on the basis of obtaining the correct foreground human silhouette. The invention can process video with lower contrast and resolution in real time, and both the detection accuracy rate and segmentation results can meet the requirements of applications.

Description

A Method for Human Upper Body Detection and Segmentation Applicable to Low Contrast Video

Technical field:

The invention relates to the technical field of video processing, in particular to a method for detecting and extracting an upper body region of a human body, in particular to a method for detecting and segmenting the upper body of a human body suitable for low-contrast videos.

Background technique:

Automatic detection and segmentation of human body regions in videos are crucial steps for two different surveillance applications. Human detection methods typically find foreground objects in videos and identify them as human or non-human regions based on shape, color, and other characteristics. Background culling is a common preprocessing technique for extracting foreground regions. The other category is based on machine learning and applies many new features suitable for machine learning. Gradient-based features are most representative. These methods do not require preprocessing for background culling but come at the cost of high computational cost, thus limiting their application in real-time systems. Video segmentation methods are also based on background removal techniques while integrating probabilistic frameworks such as Bayesian theory and Markov chain Monte Carlo models.

Since many methods need to provide a relatively good background culling algorithm result, once the environment lighting changes due to lighting changes, these methods will fail. Although some improved background culling algorithms can solve the above problems, if the foreground object remains still in front of the camera for a considerable period of time, the foreground will gradually change into the background. In addition, because the quality of the CCD chips of the cameras equipped in many monitoring systems is not high, the contrast of the obtained videos is low, and the existing methods will be more difficult to process these videos.

Invention content:

(1) Foreground extraction: first specify the first frame of the video as the background frame, convert its format from RGB color space to Lab color space, and then perform color conversion in the same way for each input frame; the converted The output frame and the background frame use the method of background removal to extract the foreground object area; then, for each extracted area, use the morphological operation of dilation and erosion to filter the noise and holes, and finally use the breadth-first connected region search algorithm to search the foreground and background The area is marked to generate a foreground area mask;

(2) Shape feature extraction: First, the contour line of the foreground area is extracted and sampled through the contour detection algorithm; then a polar coordinate system is established with the center of mass of the area as the origin, and for each sampled contour point, it is mapped to a two-dimensional Finally, all sampling points form a two-dimensional histogram; finally, normalize and expand the obtained histogram to obtain a high-dimensional vector;

(3) Human upper body model training based on support vector machine: the vector obtained in the previous step is used as a sample, and the nonlinear support vector machine algorithm with the radius basis function as the kernel function is used to perform K cross-validation analysis on all training samples, and finally Generate a non-linear decision hyperplane as a classifier for human upper body regions and non-human upper body regions;

(4) Human upper body model classification based on support vector machine: also use the vector obtained in step (2) as the input of the classifier trained in step (3), and output the class label after the classifier decision mapping;

(5) Energy function minimization optimization process: For a foreground area that is initially considered to be a human body area, when the classifier detects its class label as a non-human body area during its processing, an energy function is used to process the contour curve Modeling, the correct contour curve in the previous frame is used as the initial value, and the Euler-Lagrangian method is used to solve it.

The method of the present invention mainly consists of two major processes. Firstly, the connected region representing the foreground object is extracted from the current frame through background removal technology and morphological method, and then for each foreground region, its corresponding shape feature based on the polar coordinate two-dimensional histogram is extracted as a pre-training The input of a good support vector machine-based classifier outputs a class label corresponding to the human upper body class and the non-human upper body class. In the second step, when the area that has been recognized as a human body is misjudged as a non-human body area, the present invention uses an energy function to characterize the corresponding area, and at the same time corrects the wrong contour line through an energy function minimization process. Finally, the background frame is updated on the basis of obtaining the correct foreground human silhouette. The invention can process video with lower contrast and resolution in real time, and both the detection accuracy rate and segmentation results can meet the requirements of applications.

Description of drawings:

Fig. 1 is a flowchart of the present invention.

Detailed ways:

Below according to flow chart Fig. 1 of the present invention, each part is described in detail:

1. Foreground extraction

First, specify the first frame of the video as the background frame, and convert its format from the RGB color space to the Lab color space. Then for each input frame, the color conversion is performed in the same way, and the converted output frame and background frame use the method of background removal to extract the foreground object area (that is, the method of taking the absolute value of the difference between two frames by pixel, If its value is higher than a certain threshold, it is considered to be a foreground pixel, otherwise it is a background pixel). For each region after extraction, the noise and holes are filtered using the morphological operation of dilation and erosion, and finally the breadth-first connected region search algorithm is used to mark the foreground and background regions to generate a foreground region mask.

2. Shape feature extraction

Compared with the local phased gradient histogram feature, the feature proposed by the present invention can better describe the shape of the upper body of the human body, so it has a greater degree of discrimination and a smaller computational complexity.

The contour of a person, especially the contour of the upper body, can be regarded as a convex set. If there is a point x ₀ in the set S, so that the straight line segment from x ₀ to any point in S belongs to S, then S is called a star field or a star convex set. The shape feature of the present invention is designed based on this.

For a specific foreground area, the present invention finds the centroid of the foreground area through breadth-first search, and then finds the boundary contour line of the same area through a boundary following algorithm. Then, the contour line is sampled counterclockwise on the contour line at an equal angle, that is, the centroid of the foreground area is used as the origin of a polar coordinate system, and each sampling point on the contour line can be expressed as a set of polar coordinates in this coordinate system. Coordinates (θ _i , r _i ), i=1,2,...,N, where r _i is the Euclidean distance from the area centroid to each contour point, θ _i is the polar angle of each contour point, and N is the sampling total number of points. These polar coordinate values are then projected onto a two-dimensional plane. The x-axis of the plane represents the θ value, and the y-axis represents the r value. Each dimension is quantized separately and divided into m and n parts. When a polar coordinate value (θ _i , r _i ) satisfies the following conditions:

θ _k ≤θ _i ≤θ _k+1 ,r _l ≤r _i ≤r _l+1 ,k=0,...,m-1,l=0,...,n-1

Then increase the value of the corresponding unit (k,l). When all points are traversed according to the above method, a two-dimensional histogram with a specific pattern will be formed. This particular pattern characterizes the particular shape of the corresponding contour. Finally, after expanding the value of each cell of this histogram by row and normalizing it, an m×n-dimensional vector f will be obtained. Obviously, the shape features obtained by the present invention have nothing to do with the position and size of the object.

3. Human upper body model training and detection based on support vector machine

In the training phase, a large number of human upper body images and non-human upper body images are collected, and the foreground area is manually marked to extract the shape features of the foreground. The set of high-dimensional vectors corresponding to these shape features constitutes the sample set used for training in the present invention. The present invention uses support vector machine as the algorithm of training, and its kernel function has adopted Gaussian radius base function:

K(x _i , x _j )＝exp(-γ||x _i -x _j || ² )

Where x _i , x _j are feature vectors, and γ is a normalization constant.

In order to train a classifier with the best performance, the present invention uses a K times cross-validation method to determine two parameters γ and C of the support vector machine classifier. That is, all data is divided into K sub-data, a separate sub-data is reserved as verification data, and other K-1 sub-data are used for training. The above process is repeated K times, and each time a different combination of sub-data is selected as verification Data and training data, and finally calculate the average of the obtained results. In this way, the present invention determines that the optimal classification performance parameter combination is γ=0.25, and when C=2.0, the classification accuracy rate is about 98%.

In the detection stage, for each frame, if there is a foreground area, then use the same method to extract the area contour shape feature, as the input of the pre-trained classifier, the classifier will output a Boolean value indicating whether the current area is a human body upper body area.

4. Energy function minimization optimization process

Once the upper body area of the human body cannot be recognized as a human body by the support vector machine classifier, resulting in a classification error, then the present invention will perform an energy minimization process on the wrong foreground contour line to eliminate the error caused by the expansion of the contour line due to changes in ambient light , so as to ensure the correctness of the foreground contour area. For a closed complete profile, the present invention uses an energy function E _c (s) to characterize:

E _c (s)=∮(E _int (s)+η(s)E _ext (s))ds

where E _int (s) is the internal potential energy of the contour and E _ext (s) gives the image-based external limit. η(s) is the weight corresponding to each sampling point, defined as:

$\mathrm{<span\; class="notranslate">\; \&eta;</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; the\; s</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{{<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span><span\; class="notranslate">\; \&dtri;</span>\mathrm{<span\; class="notranslate">\; I</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; the\; s</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; the\; s</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; )</span><span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}{{<span\; class="notranslate">\; \&Sum;</span>}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; N</span>}}{<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span><span\; class="notranslate">\; \&dtri;</span>\mathrm{<span\; class="notranslate">\; I</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; the\; s</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; the\; s</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; )</span><span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}$

表示图像的梯度，N为采样点的总数。优化的目标是找到使能量泛函E_c(s)最小化的曲线函数v(s)=(x(s),y(s))。本发明采用欧拉-拉格朗日乘数法把泛函式转化为偏微分方程的求解问题，然后对其离散化，最终获得一线性系统Ax=b，其中A为对角线上有且仅有五个非零元素的矩阵。可以用Cholesky分解方法解此线性系统。in

Indicates the gradient of the image, and N is the total number of sampling points. The goal of optimization is to find the curve function v(s)=(x(s),y(s)) that minimizes the energy functional E _c (s). The present invention adopts the Euler-Lagrangian multiplier method to transform the functional into a partial differential equation solution problem, then discretize it, and finally obtain a linear system Ax=b, where A is on the diagonal and A matrix with only five nonzero elements. This linear system can be solved by the Cholesky decomposition method.

5. Background area update

On the basis of obtaining the correct foreground area, the present invention uses linear interpolation to update the background area:

${\mathrm{<span\; class="notranslate">\; I</span>}}_{\mathrm{<span\; class="notranslate">\; B</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; \&alpha;</span>}{\mathrm{<span\; class="notranslate">\; I</span>}}_{\mathrm{<span\; class="notranslate">\; B</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; \&alpha;</span>}<span\; class="notranslate">\; )</span>{\mathrm{<span\; class="notranslate">\; I</span>}}_{\mathrm{<span\; class="notranslate">\; B</span>}}^{<span\; class="notranslate">\; *</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span>$

为更新前相同位置的像素值。

为对应的当前帧中属于背景区域的像素值。对于前景区域，仅简单地拷贝相应位置上的像素值

Where I _B (x, y) is the pixel value of the background frame corresponding to the updated position (x, y),

is the pixel value at the same position before the update.

is the pixel value belonging to the background area in the corresponding current frame. For the foreground area, simply copy the pixel value at the corresponding position

It should be understood that: the above-mentioned embodiments are only descriptions of the present invention, rather than limitations of the present invention, and any inventions that do not exceed the spirit of the present invention fall within the protection scope of the present invention.

Claims (5)

1. one kind is applicable to the upper half of human body detection of low contrast video and the method for cutting apart, and it is characterized in that this method may further comprise the steps:

(1) foreground extraction: first frame of designated frame as a setting at first, its form from the RGB color space conversion to the Lab color space, for each frame of input, all carry out color conversion in the same way then; The method that output frame after the conversion and background frames use background to reject is extracted the foreground object zone; To each zone after extracting, use the morphological operation of the corrosion of expanding that noise and cavity are carried out filtering then, use breadth First connected region searching algorithm that mark is carried out in preceding background area at last, generate the foreground area mask;

(2) Shape Feature Extraction: at first extract the outline line of foreground area by the profile detection algorithm and to its sampling; Be that initial point is set up a polar coordinate system with regional barycenter then, for each sampled contour point, it be mapped to a two dimensional surface, finally all sampled points have just formed a two-dimensional histogram; Histogram normalization and expansion to obtaining obtains a high dimension vector at last;

(3) based on the upper half of human body model training of support vector machine: with the vector that obtains in the previous step as sample, use with the radius basis function and as the non-linear algorithm of support vector machine of kernel function all training samples are carried out K cross validation analysis, finally generate a non-linear decision-making lineoid as the sorter of upper half of human body zone with non-upper half of human body zone;

(4) based on the upper half of human body category of model of support vector machine: equally with the input as training gained sorter in the step (3) of the vector that obtained in the step (2), the class label of output after sorter decision-making mapping;

(5) energy function minimizes optimization: for a foreground area that begins to be considered to human region, when being classified device in its processing procedure and detecting its class label and be non-human region, come contour curve is carried out modeling with an energy function, be initial value with contour curve correct in the former frame, find the solution with Euler-Lagrangian method, and upgrade the background area with last result.

2. the upper half of human body that the is applicable to the low contrast video as claimed in claim 1 method that detects and cut apart, it is as follows to it is characterized in that method that the use background described in the step (1) is rejected is extracted the process in foreground object zone: according to pixels ask the mode that takes absolute value that differs from two frames, its value is higher than certain threshold value, just think foreground pixel, otherwise be background pixel.

3. the upper half of human body that the is applicable to the low contrast video as claimed in claim 1 method that detects and cut apart is characterized in that the detailed process of step (2) is as follows:

For a specific foreground area, find the barycenter of foreground area by BFS (Breadth First Search), find the boundary contour of the same area then by the border following algorithm; Then sample counterclockwise to outline line in equal angles ground on outline line, and the barycenter of foreground area is labeled as a polar coordinate system initial point, and each sampled point on the outline line just can be expressed as one group of polar coordinates (θ under this coordinate system _i, r _i), i=1,2 ..., N, wherein r _iBe the Euclidean distance of regional barycenter to each point, θ _iBe the polar angle of each point, N is the sum of sampled point; These polar coordinates values are projected on the two dimensional surface subsequently, and the x axle on plane is represented the θ value, and the y axle is represented the r value, and each dimension is quantized respectively, are divided into m and n part; As a polar coordinates value (θ _i, r _i) when satisfying following condition:

θ _k≤θ _i≤θ _k+1,r _l≤r _i≤r _l+1,k=0,...,m-1,l=0,...,n-1

Then increase corresponding unit (k, value l); When having traveled through all points as stated above, will form a two-dimensional histogram with AD HOC, this specific pattern is characterizing the given shape of corresponding outline line; At last, carry out to obtain the vector f that a m * n ties up after the normalization by the value of this each cell of histogram of row expansion and to it.

4. the upper half of human body that the is applicable to the low contrast video as claimed in claim 1 method that detects and cut apart is characterized in that the detailed process of step (3) is as follows:

In the training stage, a large amount of upper half of human body images and non-upper half of human body image are collected, thereby extract the shape facility of prospect by the manual markings foreground area, these shape facilities the set of corresponding high dimension vector formed the sample set that the present invention is used for training, adopt support vector machine as the algorithm of training, its kernel function adopts Gauss's radius basis function:

K(x _i,x _j)=exp(-γ||x _i-x _j|| ²)

X wherein _i, x _jBe proper vector, γ is normaliztion constant;

Adopt the method for K cross validation to determine two parameter γ and the C of support vector machine classifier: all data are divided into K one's share of expenses for a joint undertaking data, a independent subdata is retained as verification msg, other K-1 one's share of expenses for a joint undertaking data are used for training, as above process is repeated inferior K time, select for use different subdata combinations as verification msg and training data at every turn, at last asking result is averaged; At detection-phase, for each frame, if there is foreground area, using the same method so extracts the region contour shape facility, as the input of the good sorter of precondition, sorter will export whether a Boolean explanation current region is the upper half of human body zone.

5. the upper half of human body that the is applicable to the low contrast video as claimed in claim 1 method that detects and cut apart is characterized in that the detailed process of step (5) is as follows:

Count E with an energy functional _c(s) characterize the integrity profile of one section closure:

E _c(s)=∮（E _int(s)+η(s)E _ext(s))ds

E wherein _Int(s) be the inside potential energy of outline line, E _Ext(s) provided outside limits based on image, η (s) is the weight corresponding to each sampled point, is defined as:

$\mathrm{\&eta;}\left(s_i\right)=\frac{{\left|\right|\&dtri;I(x\left(s_i\right),y\left(s_i\right))\left|\right|}^2}{{\&Sum;}_i^N{\left|\right|\&dtri;I(x\left(s_i\right),y\left(s_i\right))\left|\right|}^2}$

Wherein

The gradient of presentation video, N is the sum of sampled point;

Adopt Euler-lagrange's method of multipliers that the functional formula is converted into the problem of finding the solution of partial differential equation, then to its discretize, finally obtain a linear system Ax=b, wherein A is the matrix that has and only have five nonzero elements on the diagonal line; Can use this linear system of Cholesky decomposition method solution;

On the basis that obtains correct foreground area, upgrade the background area with the mode of linear interpolation:

$I_B(x,y)=\mathrm{\&alpha;}{I}_B^{\left(t\right)}(x,y)+(1-\mathrm{\&alpha;})I_B^{*}(x,y)$

I wherein _B(x, y) for upgrade the position, back (x, y) Dui Ying background frames pixel value,

Be the pixel value of same position before upgrading,

The pixel value that belongs to the background area in the present frame for correspondence; For foreground area, only copy the pixel value on the relevant position simply

Images