CN101794385A - Multi-angle multi-target fast human face tracking method used in video sequence - Google Patents

Abstract

A multi-angle and multi-target fast face tracking method for video sequences in the field of video processing technology, through fast multi-angle face detection and the establishment of color histograms for face regions, the color probability map of the image to be detected is calculated, and Camshift is used The iterative results of the algorithm update the position of the face area, and at the same time update the face motion model established by the Kalman filter. At the same time, when the target is occluded, the face prediction is performed: if there is a face occlusion, the Kalman filter predicts the face. The position of the face area, and then update the face motion model. The invention can quickly exclude non-face areas, making real-time multi-angle multi-face target tracking possible in video sequences; at the same time, by introducing a Kalman filter, when a target is occluded in a video sequence, the prediction of the Kalman filter is used As a result, updating the position of the occluded target can better overcome the tracking difficulty caused by occlusion.

Description

Multi-angle and multi-target fast face tracking method for video sequences

technical field

The invention relates to a method in the technical field of video processing, in particular to a multi-angle and multi-target fast face tracking method for video sequences.

Background technique

Face detection and tracking technology in video sequences has broad application prospects. In the fields of national security, military security and public security, intelligent access control, intelligent video surveillance, public security control, customs identity verification, driver's license verification, etc. have a wide range of applications .

After searching the prior art, it is found that the AdaBoost face detection method based on the cascade structure is currently considered to be an effective face detection scheme, such as Chinese Patent Document No. CN101350062A, published on 2009-1-21, which records a "Video-based fast face detection method", this technology uses the temporal and spatial domain features between video frames to perform face detection in the foreground area through preprocessing, and divides the face detection process into two different modes: censorship mode and tracking mode. If the face cannot be detected in the prediction area in the tracking mode or it is predicted that the face being tracked in the next frame will leave the monitoring area, the detection process will be transferred to the review mode, a comprehensive search will be performed on the foreground area, and the people in the monitoring area will be collected again. face information. This scheme uses the trained frontal face classifier to perform face detection on the foreground prediction area to achieve the purpose of improving the detection speed. However, for multiple-angle face targets in video sequences, it is difficult to meet the real-time requirements of video tracking by only relying on AdaBoost detection and foreground prediction methods, and by using face classifiers from different angles to detect each frame in the video .

Contents of the invention

Aiming at the above-mentioned deficiencies in the prior art, the present invention provides a multi-angle and multi-target fast face tracking method for video sequences, which can quickly exclude non-face areas, making real-time multi-angle multi-face target tracking of video sequences a Possibly; at the same time, by introducing the Kalman filter, when the target occlusion occurs in the video sequence, the prediction result of the Kalman filter is used to update the position of the occluded target, which can better overcome the tracking difficulties caused by occlusion.

The present invention is realized through the following technical solutions, and the present invention comprises steps:

The first step, multi-angle face rapid detection: perform skin color segmentation on the frames to be detected in the video sequence, use face classifiers from different angles to perform face detection on the skin color area, and fuse the face detection results.

The skin color segmentation refers to converting the frame to be detected from the RGB color space to the HSV color space by using the skin color feature, and then extracting the skin color area and removing the non-skin color area according to the human face skin color model.

The feature of skin color means that the color of human skin is one of the most prominent features on the surface of the human body compared to other objects, and the feature of skin color can be used to distinguish the human body from other objects.

The described human face skin color model refers to: using the characteristics that skin color will gather in a specific area in the color space under normal lighting conditions, by sampling the skin color image to establish a distribution function or find a suitable threshold for skin color distribution , the skin color area can be extracted from the background image.

Described human face detection refers to: first utilize integral graph to calculate the skin color area in candidate area, when skin color area is less than the threshold value of setting, then exclude the area corresponding to this skin color area, otherwise utilize the human face classifier that AdaBoost algorithm training obtains Detect the candidate area, if the detection result is true, the candidate area is a face area; otherwise, the candidate area is a non-face area.

The skin color area is S _skin =ii _A +ii _D -(ii _B +ii _C ), wherein: S _skin represents the skin color area in the rectangular area ABCD, and the value range is 0 to 255×the width of the rectangle ABCD×the rectangle ABCD ii _t is the integral image at point t, and its value ranges from 0 to 255×the width of the image to be detected×the height of the image to be detected.

The second step, when there is no overlap between the two face target areas, establish an H channel color histogram for each face area and calculate the H channel color histogram, and then use the Camshift iterative algorithm to calculate the face target position and update Kalman Filter face target motion model, otherwise perform the third step;

The described calculation of the H channel color histogram refers to: the color probability map of the image to be detected is calculated by the H channel color histogram, and the height of the color brightness is used as the criterion for determining the position of the centroid, and the Camshift iterative algorithm is used to calculate and search the color probability map. The location of the centroid of the region, dynamically adjust the size of the target window, and the face motion model established by the Kalman filter.

The third step, face tracking when the target is occluded: if any two face targets overlap, the position of the occluded face is predicted by the Kalman filter, and the Kalman filter parameters are updated using the face position predicted by Kalman .

Compared with the prior art, the present invention can significantly improve the detection and tracking speed, realize multi-angle and multi-face real-time tracking, and the tracking speed can reach more than 35 frames per second for 480×320 video. For multi-angle and multi-face tracking in general scenes, the accuracy rate can reach more than 95%.

Description of drawings

Fig. 1 is the flow chart of the present invention.

Figure 2 is a legend to be tested;

Among them: a is the legend of the frontal face detection; b is the legend of the left half of the face detection; c is the legend of the right half of the face detection; d is the legend of the fusion of the front and side face detection.

Figure 3 is a legend after processing;

Among them: a is the legend after skin color segmentation; b is the color probability map.

Fig. 4 is a schematic diagram of part of the tracking results in the embodiment.

Detailed ways

The embodiments of the present invention are described in detail below. This embodiment is implemented on the premise of the technical solution of the present invention, and detailed implementation methods and specific operating procedures are provided, but the protection scope of the present invention is not limited to the following implementation example.

As shown in Figure 1, this embodiment includes the following steps:

The first step, multi-angle face rapid detection: perform skin color segmentation on the frame to be detected, use the integral map to quickly eliminate non-skinned areas, perform face detection on skin-colored areas, use face classifiers from different angles to detect images, and fuse the final person Face detection area.

The second step, face tracking when there is no target occlusion: establish a color histogram for the face area, calculate the color probability map of the image to be detected, use the iterative result of the Camshift algorithm, update the position of the face area, and update the position of the face area by Kalman at the same time The face motion model built by the filter.

The third step is face prediction when the target is occluded: if there is a face occlusion, the Kalman filter is used to predict the position of the face area, and then the face motion model is updated.

The above-mentioned multi-angle face rapid detection steps are as follows:

Using the skin color feature, the image is converted from the RGB color space to the HSV color space, which overcomes the defect that the RGB color space is sensitive to changes in light brightness. According to the skin color model of the face, the skin color area is extracted, as shown in Figure 3a. Before using the multi-angle face detector to detect the image, first calculate the skin color area in the candidate area, if the skin color area is less than the set threshold, immediately exclude the area to improve the speed of detection; otherwise use the AdaBoost algorithm to train the human body The face classifier detects the candidate area. If the detection result is true, the candidate area is a face area; otherwise, the candidate area is a non-face area.

The skin color area of the candidate area can be obtained directly from the integral map S _skin =ii _A +ii _D -(ii _B +ii _C ), as shown in Figure 3a, wherein: S _skin represents the skin color area in the rectangular area ABCD, which is taken as The value range is [0, 255×the width of the rectangle ABCD×the height of the rectangle ABCD], ii _t is the integral image at point t, and the value range is [0, 255×the width of the image to be detected×the height of the image to be detected].

Finally, the face areas detected by the face detector at various angles are fused, and the detection results are shown in Figure 2. A, b, c, and d in Figure 2 represent the front, left, and right faces respectively, and the final fusion of each angle face detection results.

The specific implementation method of the above-mentioned face tracking method when no target occlusion occurs is as follows:

Judge the relative position between the human face target areas, if any two human face areas do not overlap, calculate the H channel color histogram of each human face area obtained by step 1), and calculate the image to be detected by the H channel color histogram The color probability map, as shown in Figure 3b, the higher the brightness, the closer to the color of the face. Use the Camshift iterative algorithm to calculate the centroid position of the search area in the color probability map, dynamically adjust the size of the target window, update the position of the face area, and update Facial motion model built by Kalman filter.

The implementation method of face prediction when the above target is occluded is as follows:

Judging the relative position between face target areas, if any two face areas overlap, the Kalman filter face motion model predicts the position of the occluded face area, and uses the position of the face area predicted by Kalman to update the Kalman Mann filter face motion model parameters. The prediction process is shown by the dotted line in Figure 1. The prediction results are shown in the 3rd and 20th frames of Figure 4. When two face areas overlap, the Kalman filter face motion model is used to predict (shown in the white rectangle box), even when the face is completely blocked, the Kalman filter The Mann filter face motion model can still predict the location of the face.

Claims (6)

1. a multi-angle multi-target fast human face tracking method that is used for video sequence is characterized in that, may further comprise the steps:

The first step, multi-angle people's face fast detecting: the frame to be detected in the video sequence is carried out skin color segmentation, with people's face sorter of different angles area of skin color is carried out people's face and detect, and merge people's face testing result;

Second the step, do not occur overlapping when two people's face target areas, then each is set up H passage color histogram and calculate H passage color histogram human face region, calculate people's face target location with the Camshift iterative algorithm then, upgrade Kalman filter people face target movement model, otherwise carried out for the 3rd step;

Face tracking when the 3rd step, target occlusion: overlapping if any two people's face targets occur, by the position that Kalman filter is predicted the people's face that is blocked, the Kalman filter parameter is upgraded in the people's face position that utilizes the Kalman to predict simultaneously.

2. the multi-angle multi-target fast human face tracking method that is used for video sequence according to claim 1, it is characterized in that, described skin color segmentation is meant: utilize features of skin colors, frame to be detected is transformed into the hsv color space by the RGB color space, according to the face complexion model, extract area of skin color and reject non-area of skin color then.

3. the multi-angle multi-target fast human face tracking method that is used for video sequence according to claim 2, it is characterized in that, described face complexion model is meant: by colour of skin image sampling being set up a distribution function or being sought the appropriate threshold that the colour of skin distributes, area of skin color is extracted from background image.

4. the multi-angle multi-target fast human face tracking method that is used for video sequence according to claim 1, it is characterized in that, described people's face detects and is meant: utilize the colour of skin area in the integrogram calculated candidate zone earlier, when colour of skin area less than preset threshold, the zone of then getting rid of this colour of skin area correspondence, otherwise the people's face sorter that utilizes the training of AdaBoost algorithm to obtain detects the candidate region, if testing result is true, then the candidate region is a human face region; Otherwise the candidate region is non-face zone.

5. the multi-angle multi-target fast human face tracking method that is used for video sequence according to claim 1 is characterized in that, described colour of skin area is S _Skin=ii _A+ ii _D-(ii _B+ ii _C), wherein: S _SkinColour of skin area in the expression rectangular area ABCD, span are the height of width * rectangle ABCD of 0 to 255 * rectangle ABCD, ii _tFor at an integrogram at t place, span is 0 to 255 * picture traverse to be detected * picture altitude to be detected.

6. the multi-angle multi-target fast human face tracking method that is used for video sequence according to claim 1, it is characterized in that, described calculating H passage color histogram, be meant: the color probability graph that calculates image to be detected by H passage color histogram, with the height of colour brightness as judging the centroid position criterion, in the color probability graph, calculate the centroid position of region of search with the Camshift iterative algorithm, dynamically adjust the target window size, and people's face motion model of Kalman filter foundation.

Images