JP2014093023A - Object detection device, object detection method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To accurately detect an object in many situations where shields have been caused.SOLUTION: An object detection part 101 obtains a whole body detection result and head portion detection result (detection score) of a human body to be detected. Using results obtained by the object detection part 101, a shield detecting part 102 determines whether a shield is present in the human body being detected or not. According to the state of shield, the object detection part 101 corrects the detection score. At this time, a determination is made whether an overlapping area is present in the head portion detection result of a foreground human body and the whole body detection result of the background human body. If there is an overlapping area, a shield area is calculated, and the detection score is corrected according to the calculated shield area.

Description

  In particular, the present invention relates to an object detection apparatus, an object detection method, and a program suitable for use when shielding is occurring.

  Technology that enables accurate and robust detection of even a person who is partially shielded from a single image can be applied to motion analysis and has been actively researched in recent years. . Applications of such technologies are being studied, particularly in fields such as security systems, safe driving support, and medical welfare. As described above, also in the field of surveillance cameras, vehicle-mounted cameras, and the like, a technique for robustly detecting a human body when the human body in the image is shielded is known.

  For example, in the method disclosed in Patent Document 1, first, a region where the detection score is larger than the preset detection condition of the human body is detected from the entire input image. This region is a region for detecting the human body that is present in the foreground most. Next, a peripheral search region that may be shielded is calculated based on this region. Then, by performing detection processing under a condition in which the determination condition is looser than the above condition in the peripheral search region, it is possible to detect a background person who has been blocked by the foreground human body.

  In the method disclosed in Patent Document 2, a region where a human body is partially shielded by a high-luminance object such as external light or a guardrail is detected using image contrast information. And the detected shielding area is detecting the human body robustly by calculating the score which considered that it was a shielding area.

  In addition, the method disclosed in Non-Patent Document 1 is robust even if a shield is present by explicitly incorporating an occluder indicating whether or not the shield is present in one of the parts constituting the human body. The human body is detected. By incorporating this occluder, the shielding state is determined by detecting a strong straight edge that occurs when a human body is hidden by a desk or table.

JP 2010-49435 A JP 2011-165170 A

Proceedings of the Neural Information Processing Systems (NIPS) 2011. `` Object Detection with Grammar Models '' P. Viola and M. Jones (2001). "Rapid Object Detection using a Boosted Cascade of Simple Features", IEEE Conference on Computer Vision and Pattern Recognition. Dalal, N., & Triggs, B. (2005). "Histograms of oriented gradients for human detection.", IEEE CVPR Platt, J. C. (1999) .Probabilistic Outputs for Support Vector Machines and Comparisons to Regularized Likelihood Methods.Advances in Large Margin Classifiers. Zadrozny, B., & Elkan, C. (2002) .Transforming classifier scores into accurate multiclass probability estimates.Proceedings of the Eighth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. P. Felzenszwalb, D. McAllester, D. Ramanan (2008) "A Discriminatively Trained, Multiscale, Deformable Part Model", IEEE Conference on Computer Vision and Pattern Recognition.

  However, Patent Document 1 does not explicitly make a shielding decision. For a human body that is close to the detected human body and is likely to be shielded, the detection threshold is lowered to make it easier to detect. ing. Therefore, erroneous detection increases and detection accuracy decreases.

  Further, in Patent Document 2, since the shielding region is detected only by contrast information using the characteristic that the object that shields the human body is a high-luminance object, the overlapping of human bodies that does not necessarily become high contrast. It cannot cope with shielding by.

  In Non-Patent Document 1, since the object is a straight strong edge such as a desk or table, it can be used to shield a human body by a desk or table, but the shape can be freely deformed (for example, before a detection target). It is not possible to support other people standing up).

  The present invention has been made in view of the above-described problems, and an object of the present invention is to make it possible to detect an object with high accuracy in many situations where shielding occurs.

  An object detection apparatus according to the present invention is an object detection apparatus that detects a first object located in the foreground and a second object located in the background from an input image, and is a partial region of the first and second objects. First detection means for detecting the second detection means, second detection means for detecting information indicating the postures of the first and second objects, the partial area detected by the first detection means, and the second Based on the information detected by the detection means, the determination means for determining the shielding state of the second object by the first object, and the second state according to the shielding state determined by the determination means Correction means for correcting the detection result of the object.

  According to the present invention, an object can be detected with high accuracy in many situations where shielding occurs.

It is a block diagram which shows the simple structural example of the object detection apparatus which concerns on embodiment. It is a block diagram which shows the detailed structural example of the object detection apparatus which concerns on embodiment. It is a flowchart which shows an example of the process sequence which the detection process part of the object detection apparatus which concerns on embodiment performs. It is a figure for demonstrating the outline | summary which estimates the position of a head from the result of each detector. It is a figure which shows the example of a definition of the positional relationship between the result of a whole body detector, and a head. It is a figure explaining the process which evaluates a head position estimation result using the correct reference | standard of a head. 5 is a flowchart illustrating an example of an integration processing procedure performed by an integration result output unit in the embodiment. It is a figure explaining the specific example of the process result output from an integrated result output part. It is a figure explaining a mode that the human body of a back ground is partially shielded by the human body used as a foreground. It is a figure explaining the determination method of the shielding state in FIG. 9, and the calculation method of a shielding area. It is a figure which shows an example of the state which two or more persons overlap and the shielding has arisen. It is a flowchart which shows an example of the process sequence by the shielding determination part in 1st Embodiment. It is a figure explaining the whole body detector using the parts-based detection method. It is a figure explaining the relationship between a parts-based detector and a human body axis. It is a figure explaining the example which performs shielding determination from the estimation result of a head by a plurality of parts detectors, and the estimation result of a body axis. It is a figure explaining the method of calculating the area and position of a shielding area. It is a flowchart which shows an example of the process sequence by the shielding determination part in 2nd Embodiment. It is a figure explaining a detection process in case the human body of a foreground is shielded by the human body of a foreground, and the heads of both human bodies have overlapped. It is a flowchart which shows an example of the process sequence by the shielding determination part in 3rd Embodiment. It is a figure which shows an example of the detection result of the head and body axis before an integration process.

(First embodiment)
Hereinafter, in the present embodiment, the object is divided into a plurality of partial areas, and the object-likeness of each partial area is calculated as a partial area score. And an object detection process is implemented by determining whether it is a target object based on an integrated score, integrating the partial area score of a part or all, and calculating an integrated score. Furthermore, robust detection processing is realized by calculating position and orientation information of a plurality of objects from the partial region score and the integrated score calculated in the process of detection processing, and performing occlusion determination. Although the target object detected in the present embodiment is not particularly limited, a case where the detection target object is a person will be described.

  1 and 2 are block diagrams illustrating a configuration example of the object detection apparatus 10 according to the present embodiment. Hereinafter, the configuration of the present embodiment will be described with reference to FIGS. 1 and 2. Note that the object detection apparatus 10 of the present embodiment is a control apparatus in which software (program) acquired via a network or various recording media is configured by a CPU, a memory, a storage device, an input / output device, a bus, a display device, and the like. It can be realized by executing. For the control device (not shown), a general-purpose control device may be used, or hardware optimally designed for the software of the present invention may be used.

<Configuration related to detection processing>
As illustrated in FIG. 1, the object detection apparatus 10 according to the present embodiment outputs a detection result of an object via an image input unit 100, an object detection unit 101, and a shielding determination unit 102 in the detection process. Hereinafter, these structural blocks will be described.

  The image input unit 100 is a part that inputs an image to the image processing apparatus. The image input to the image input unit 100 may be a one-frame image of a moving image obtained from a camera or the like, or may be an image stored in a storage device such as a hard disk drive.

  Hereinafter, processing for the attention area of the input image will be described. The attention area is a partial area of the input image and has the same image size as the size of an object detector described later. By sequentially setting the attention area in the input image while sliding in the image, the object can be detected from the entire image. Further, by further enlarging or reducing the input image, it is possible to detect objects captured in various sizes.

  The object detection unit 101 calculates the estimated position and score (likelihood) of the target object to be detected. Details of the object detection unit 101 will be described below with reference to FIG.

  2, the object detection unit 101 includes an image input unit 100, a first detection processing unit 211 to an nth detection processing unit 21n, and a first common part estimation unit 221 corresponding to each detection processing unit. -Nth common part estimation part 22n is provided. Furthermore, a first score correction dictionary 231 to an nth score correction dictionary 23n, a first score correction unit 241 to an nth score correction unit 24n, and an integration result output unit 250 are provided. In the following, each component will be described by taking a process for one image input to the image input unit 100 as an example.

  In the first detection processing unit 211 to the n-th detection processing unit 21n, detectors that detect different parts and states of the object are stored in advance. When a detection target is a person, a detector for a different part of the person such as a face detector, a head detector, an upper body detector, or a whole body detector is used as a different detector in each detection processing unit. be able to. Using a detector that detects different parts of a person so that the person can be detected even when part of the person is shielded from other objects or when part of the person protrudes from the image Become.

  It is desirable to prepare a detector that interpolates each of the plurality of detectors. As an example in which the detectors are interpolated with each other, for example, a combination of a head detector and a whole body detector can be considered. First, the head detector can detect a person even if the body and the lower part are shielded from other objects, and has the advantage of being able to detect a person without being affected by posture fluctuations of the body part. is there. However, since the head has few characteristic shapes, the detection performance tends to be inferior to that of the whole body detector. On the other hand, the whole body detector has an advantage that it is easy to capture the characteristics of a person and has a relatively high detection performance because it has a large target part, but it has a drawback that it is vulnerable to shielding. Therefore, by using the head detector and the whole body detector at the same time, it becomes possible to compensate for the mutual drawbacks and improve the accuracy of human detection.

  On the other hand, the face detector collects Haar-Like feature values of the face range of the learning image and enables statistical recognition of facial features by AdaBoost, as in the method disclosed in Non-Patent Document 2. Learn face detector. When learning other human parts such as the head, upper body, and whole body, the HOG feature amount described in Non-Patent Document 3 is used as the image feature. When preparing a head detector, upper body detector, or whole body detector, prepare a learning image of each part, acquire each HOG feature quantity, and discriminator such as SVM (support vector machine) or AdaBoost To learn the detector of each part. The learning result (for example, AdaBoost's weak classifier) is stored as a detector dictionary and used at the time of detection.

  Each detector calculates the likelihood of human detection as a detection score. For example, AdaBoost outputs the weighted sum of the outputs of each weak classifier as a detection score. In SVM, the distance from the identification hyperplane is calculated as a detector score. Any method other than the above may be used as long as it is a method for outputting a score representing the likelihood of an object such as likelihood. The detection score is preferably set to a value that can be compared with the detection score of each detector by converting the detection score into a probability value indicating the object. In the following description, it is assumed that the higher the detector score, the higher the output that is likely to be the human part or the human state targeted by each detector.

  Hereinafter, although this embodiment demonstrates the case where three are used as a some detector, a face detector, a head detector, and a whole body detector, the structure of the detector to be used is not this limitation.

  Next, processing by the first detection processing unit 211, the second detection processing unit 212, and the nth detection processing unit 21n will be described. FIG. 3 is a flowchart illustrating an example of a processing procedure performed by the first detection processing unit 211 to the nth detection processing unit 21n. Hereinafter, an example in which the first detection processing unit 211 performs detection processing of the whole body detector will be described.

  First, in step S301, the image feature amount of the input image is calculated. In this process, since the detector of the first detection processing unit 211 is a whole body detector, the HOG feature amount is calculated from the input image. Next, in step S302, an image feature amount at a specific position of the image to be detected is acquired. In step S303, using the detector dictionary, the object feature of the image feature quantity to be processed is determined, and a detection score is calculated.

  Next, in step S304, it is determined whether or not a detection score has been calculated for the entire input image. In order to search the entire image, it is performed on the entire image while changing the discrimination position so that the detection score is calculated at the position in each image. As a result of this determination, if the detection score is not calculated for the entire input image, the process returns to step S302. When changing the discrimination position, the image size is also changed, so that a person appearing in a different size in the image can be detected.

  On the other hand, as a result of the determination in step S304, when the detection score is calculated for the entire input image, the detection score at each position in the image is obtained. Here, all of the results may be sent to the first common part estimation unit 221. However, for the detection result of a low detection score that can be clearly determined not to be a person, the subsequent processing is omitted. The processing load can be reduced. Therefore, in the next step S305, threshold processing is performed to leave a result of a predetermined score or more, and useless detection results are deleted. As a result of the processing in step S305, the position information of the position where the detection score is higher than the predetermined value in the image and the detection score are output to the common part estimation unit 221.

  Although the processing result of one detection processing unit has been described above, the entire object detection apparatus 10 repeats the processing of this detection processing unit by the number of detection processing units.

  Next, the first common part estimation unit 221 to the nth common part estimation unit 22n will be described. In the first common part estimation unit 221 to the n-th common part estimation unit 22n, the position of the common part of the object is estimated from the result of each detector. In this embodiment, in order to integrate the results of the detectors that detect different parts, the position or range of the common part of the target object is estimated from each detector, and the detection result is based on the estimated positional relationship of the parts. Integrate.

  For example, human arms, legs, and torso are detected in each detection processing unit, and the head is estimated as a common part. It is also possible to estimate a common part from the results detected by a plurality of detection processing units and narrow down detection candidates by subsequent processing. Hereinafter, in the present embodiment, a procedure in which a person's head is defined as a common part and the first common part estimation unit 221 estimates the head position from the detection results of the respective detectors will be described. However, in this embodiment, the common part to be estimated is the head, but there is no particular limitation as long as it is a part that can be commonly estimated by each detector. For example, a body axis that connects the center of the waist from the throat of the human body can be cited as a part that can be estimated in common. When the detection target is a person, the head of the person is a part that is relatively difficult to be shielded, and thus is suitable as a common part.

  FIG. 4 is a diagram for explaining an outline of estimating the position of the head from the result of each detector. As a result of the detection processing of each detector, information on the position / range of the detection target is obtained, and in this embodiment, the position / range of the detection result is obtained by a rectangular frame surrounding the detection target. In the example shown in FIG. 4, the detection result is shown by a rectangular frame, and information on the face detection result frame 401, the head detection result frame 402, and the whole body detection result frame 403 is obtained. The coordinate X of the rectangular frame is expressed by the following equation (1) using two image coordinate points.

Here, x ₁ and y ₁ are the image coordinates of the upper left point of the rectangle, and x ₂ and y ₂ are the image coordinates of the lower right point of the rectangle. The first common part estimation unit 221 estimates a head position / range as a common part from the rectangular frame. In the example shown in FIG. 4, the head position / range estimated from the face detection result frame 401 is represented by a rectangular frame 411, and the head position / range estimated from the whole body detection result frame 403 is represented by a rectangular frame 413. . When estimating the position of the head from the detection result frame, the positional relationship between the detection result frame and the head is defined in advance, and the position of the head is converted by converting the detection result frame to the position of the head. presume.

FIG. 5 is a diagram illustrating a definition example of the positional relationship between the result of the whole body detector and the head. In the example shown in FIG. 5, the position of the head with respect to the whole body detector is 15% of the height h _B of the whole body detector, the head height h _H, and 50% of the width w _B of the whole body detector. It is defined as the head width w _H. In addition, an offset of 0.25 w _B is defined in the x-axis direction. When estimating the position of the head from the whole body detector, the head coordinates X _h are obtained from the coordinates X of the whole body result according to the definition shown in FIG. The head coordinate X _h is expressed by the following equation (2).

Here, x _h1 and y _h1 are the coordinates of the upper left point of the estimated head range, and x _h2 and y _h2 are the coordinates of the lower right point of the estimated head range. The first common part estimation unit 221 calculates head estimation coordinates X _h from the coordinates X shown in Expression (1) for each detection result obtained as a result of the processing of the first detection processing unit 211.

  Note that the definition of the head range may be designed in advance by a person inputting and designing each numerical value in advance, or may be designed from the average of head positions obtained from actual whole body detection results. When obtaining the average of the head position, it can be obtained by performing detection processing with a whole body detector on a plurality of sample images and calculating the average value of the head position in the detection result.

  In the above description, the operation of the first common part estimation unit 221 has been described using the method of estimating the head position from the whole body detector as an example. When estimating the head position from the detection results of other detectors, as in the case of the whole body detector, the positional relationship between each detection result and the head is defined, and Estimate the position. In the whole body detector, the position of the head inside the detection result is estimated, but the position to be estimated need not be inside the detection result.

  For example, a rectangular frame 411 indicating the position of the head estimated from the face detection result frame 401 shown in FIG. 4 is outside the face detection result frame 401. In the head detection result frame 402 of the head detector that detects the head itself, the processing of the common part estimation unit is omitted, and the head detection result itself is output as a result of estimation as a common part. May be.

  Next, the first score correction dictionary 231 to the nth score correction dictionary 23n and the first score correction unit 241 to the nth score correction unit 24n will be described. In the present embodiment, a plurality of different detection results are integrated using the position of the common part estimated from each detection result and each detection score. Here, the position of the common part is a result estimated from the detection result, and the estimation accuracy differs depending on the detector. In this embodiment, the head position is estimated as a common part, but the head position estimation performance is better for a detector that is close to the head position or deeply related to the head. It is done. Therefore, in order to perform integration in consideration of the difference in the estimation performance of the common part, the first score correction unit 241 to the n-th score correction unit 24n respectively include the first score correction dictionary 231 to the n-th score correction dictionary. 23n is used to correct the detection score based on the estimated performance difference of the common part. Then, by integrating the surrounding detection results using the corrected detection score, it can be expected that the position accuracy of the detection result of the object is improved.

  The first score correction unit 241 to the n-th score correction unit 24n use the information recorded in the first score correction dictionary 231 to the n-th score correction dictionary 23n for the detection score of each detector, respectively. Convert. The first score correction dictionary 231 to the n-th score correction dictionary 23n store information for correcting the detection score based on the reliability with which each detector estimates the common part.

  In score correction, a correction coefficient may be stored in each score correction dictionary for each detector, and a correction score may be calculated by multiplying the detection score by the coefficient during score correction. As an example of the correction coefficient, the correction coefficient of the head detector is 1, the correction coefficient of the face detector is 0.8, the correction coefficient of the whole body detector is 0.5, and the like. Thus, a detector close to the head (a detector with high head position estimation performance) sets a large correction coefficient, and a detector far from the head (a detector with low head position estimation performance) has a low correction coefficient. Set the correction factor. This correction coefficient can be changed depending on the posture, imaging conditions, shielding location, etc., and may be set adaptively by determining those states. In the present embodiment, a prior probability relating to detector performance is obtained in advance and the value is used. By multiplying the detection score by this correction coefficient to obtain a correction score, it is possible to obtain a correction score in consideration of the detection result of the detector and the performance of estimating the common part. The correction score is a score obtained by weighting the detection score indicating the likelihood of the target object with the likelihood of the position estimation of the common part, and indicates the target characteristic and the probability of the position together.

  The correction coefficient may be input and set by the user, but the correction coefficient is preferably set according to the correct probability of the head position estimated by each detector. Therefore, it is necessary to obtain in advance the correct probability related to the estimation of the position of the head of each detector. Hereinafter, with reference to FIG. 6, a description will be given of how to obtain the correct answer probability related to head position estimation and the correction coefficient stored in each score correction dictionary.

  First, an image sample group whose head position is known is prepared. FIG. 6A shows an example of an image in which the head position of a person in the image 600 is known, and the coordinates of the head range are recorded as a head correct answer 601. Here, it is desirable that the image 600 is an image in which only one person is shown or is cut out within the range of one person. In this way, a large number of images with known head positions are prepared.

  Next, FIG. 6B shows the result of performing face detection on the image of FIG. As a result of the face detection, the face detector detection process is sequentially performed on the entire image 600 as in the process described in the detection process. Here, attention is focused on the detection result 611 having the highest face detection detection score in the image 600. Since only one person is shown in the image 600, the detection result 611 showing the highest score is considered to be a face.

  Next, an estimation result 612 in which the position of the head is estimated from the face detection result is calculated. The head position estimation result 612 is compared with the head correct answer 601 to evaluate whether or not the head is correctly estimated. When comparing the head correct answer 601 and the head position estimation result 612, for example, if the center-to-center distance of each position is within a predetermined range, the estimation result is assumed to be correct. In addition, as another criterion, the overlap rate between the rectangular head correct answer 601 and the head position estimation result 612 may be calculated, and the estimation result may be correct when it indicates a predetermined overlap rate or higher. Good. As a method of calculating the rectangular overlap rate α, for example, it can be calculated by the following equation (3).

Here, S _b is the area of the head correct, S _e is the area of the head ranges estimated, S _BE is the area of the overlapping region of the head ranges presumed head correct. The above correct answer determination is executed for all the prepared image sample groups, and the probability that the head estimation is correct can be obtained, and the probability is set as a correction coefficient. If the detection result itself is not obtained for the image sample, the head estimation is determined as an incorrect answer.

  Similarly, for other detectors, the correct probability of head estimation may be obtained for each detector, and each correct probability may be used as a correction coefficient for each detector. In the example shown in FIG. 6D, the positional relationship between the head position 631 estimated from the detection result 630 of the whole body detector and the head correct answer 601 is evaluated. In the example shown in FIG. 6D, since the head position 631 is greatly deviated from the head correct answer 601, the estimation of the head position from the whole body detector is incorrect.

  FIG. 6C shows an example of correct answer determination of the detection result of the head detector. For the result of the head detector, the correction coefficient may be calculated by evaluating the head correct answer in the same manner as the others and evaluating the performance indicating the head position. Since the head detector does not necessarily have to estimate the position of the head, in this case, the position of the detection result itself and the head correct answer are evaluated.

  When calculating the correction coefficient using the overlap rate α, the correct probability is calculated by binary determination of correct / incorrect for each image sample. Therefore, this information may be used to perform score correction by performing Platt scaling disclosed in Non-Patent Document 4 and Isotonic Regression disclosed in Non-Patent Document 5. Moreover, it is not necessary to perform score correction at all.

  When the correction score is calculated by the above processing, the integration result output unit 250 integrates the results of these detectors, and combines the information output from the plurality of detectors into the same person. In the present embodiment, the purpose is not to collect the detection results output from the same detector around the same person.

Hereinafter, a process of combining information output from a plurality of detectors for the same person into one will be described. FIG. 7 is a flowchart illustrating an example of an integration processing procedure performed by the integration result output unit 250 in the present embodiment. In the process illustrated in FIG. 7, an example will be described in which the processes in steps S701 to S704 are performed while looping the individual output results of the whole body detector. Hereinafter, the number of the output result of the whole body detector of interest is i (i = 1,..., L), and the head estimation coordinates of that number are X _{hB, i} .

First, in step S701, it is determined whether or not the detection result of the whole body detector remains. As a result, when it remains, it progresses to step S702, and when that is not right, a process is complete | finished. Next, in step S702, the head position estimation result having the highest overlap rate with the region indicated by the coordinates X _{hB, i} is selected. At this time, the number of the selected estimation result is j (j = 1,..., M), and the estimated coordinates of the head of that number are X _{hH, j} . Here, the overlapping rate A ₀ (X _{hB, i} , X _{hH, j} ) between the area indicated by the coordinates X _{hB, i} and the area indicated by the coordinates X _{hH, j} is obtained from the following equation (4).

  Here, P (X, Y) is the area of the overlapping area between the rectangle X and the rectangle Y. S (X) and S (Y) are the areas of the rectangle X and the rectangle Y, respectively.

In step S703, the face position estimation result having the highest overlap rate with the area indicated by the coordinates X _{hB, i} is selected. At this time, the number of the selected estimation result is k (k = 1,..., N). In step S704, the vector R _i shown in the following equation (5) is output for the detection result i of each whole body detector.

Here, S _{B, i} , S _{H, j} and S _{F, k} are the correction score of the i-th whole body detector, the correction score of the j-th head detector, and the correction score of the k-th face detector, respectively. These sums are output from the integration result output unit 250 as integrated scores. In this embodiment, the score of each detector is corrected and a simple sum is taken as an integrated score. Note that correction may not be necessary depending on the type of each detector, and this can be determined by comparing the detection accuracy. Further, whether or not the score is corrected, a linear sum of the scores of the respective detectors can be taken to obtain an integrated score. The linear coefficient in this case can be obtained by learning such as SVM using the score of each detector as an input vector.

  FIG. 8 is a diagram for explaining a specific example of the processing result output from the integration result output unit 250. FIG. 8A shows a detection result at the time when it is input to the integrated result output unit 250, and shows a state in which a plurality of detection results are obtained around the person. In the example shown in FIG. 8 (A), the result of the face detector is omitted for the sake of simplicity, and only the detection result of the head detector and the detection result of the whole body detector are illustrated. Yes.

  A wavy line rectangle 801 is a detection result of the whole body detector, and a wavy line rectangle 804 is an area of the head position estimated from the whole body detector. In the example shown in FIG. 8A, the detection result of one whole body detector and the estimation result of the head are shown. Solid rectangles 802 and 803 indicate two detection results of the head detector. These are results obtained by performing a detection process while changing the search position in the image in the process of detecting the head, resulting in a plurality of detection results obtained around the person's head. The integrated result output unit 250 collects these detection results using the head position and the estimation information that are common parts.

  FIG. 8B shows the result of processing the detection result shown in FIG. 8A by the integrated result output unit 250. The head position shown in the rectangle 804 based on the whole body detector and the degree of overlap are the most. A head detection result indicated by a high rectangle 802 is selected and left as an integration result. Conversely, the detection result of the head indicated by the rectangle 803, which is considered to be a false detection of the head detector, is deleted because there is no corresponding whole body detection result.

<Shielding judgment process>
The shielding determination unit 102 determines whether there is shielding in the detection target using the result integrated in the object detection unit 101, and corrects the detection score according to the shielding state. Hereinafter, an integrated score that is an output result of the object detection unit 101 will be described as a detection score. Specifically, the shielding determination unit 102 determines shielding by sequentially referring to the whole body estimation result corresponding to the detection result of the head having a high detection score and the detection result of the head existing in the vicinity thereof. In the present embodiment, it is assumed that the human body to be detected is always upright.

  FIG. 9 is a diagram for explaining a situation in which the foreground human body 900 that is the first object is partially shielded by the rear body human body 901 that is the second object. At this time, the detection results of the head shown by solid line rectangles 910 and 911 are obtained as processing results of the object detection unit 101, and the wavy line rectangles 920 and 921 indicate the detection results of the whole body.

  FIG. 10 is a diagram for explaining the shielding state determination method and the shielding region calculation method in FIG. 9. First, in order to determine the shielding state, attention is paid to a rectangle 910 that represents the detection result of the head of the human body 900 in the foreground. Here, the subject photographed by the camera appears larger as the object in the foreground and appears smaller as the object in the back due to the influence of perspective projection. And the possibility that the front object is shielded is small. Based on this principle, all detection results output from the object detection unit 101 are sorted in descending order by detection score and detection frame size. By performing this process, it is possible to identify a human body in front of which the possibility of shielding is low and which causes the shielding. Then, shielding determination with other detection results is performed in order from the human body that is likely to cause shielding. However, in the case of a nearby human body, a significant difference in size due to perspective projection does not necessarily occur. In such a case, it is possible to determine which human body exists in the foreground in consideration of the continuity of the texture.

  Then, an intersection point or an overlap area 1002 between the rectangle 910 representing the detection result of the head of the human body 900 and the rectangle 921 representing the whole body detection result of the human body 901 is calculated. Here, when the head of the human body generally assumed to be the most foreground and the detection result of the whole body of another human body overlap, the detection score of the head corresponding to the detection result of the whole body is a threshold value. Higher than that, it can be determined that the human body is in a shielded state with a high probability. Then, the detection score is corrected by determining that a part of the human body is in the shielding state in order from the lowest head detection score corresponding to the cross-over or overlapping whole-body detection result.

Hereinafter, processing for determining the shielding state and correcting the detection result will be described.
FIG. 12 is a flowchart illustrating an example of a processing procedure performed by the shielding determination unit 102. In the process illustrated in FIG. 12, the processes in steps S <b> 1201 to S <b> 1207 are performed while looping on individual output results detected by the object detection unit 101. Here, the number of the output result of the focused whole body detector is i (i = 1,..., L).

  First, in step S1201, the detection results are sorted in descending order of detection score and in descending order of head size. As described above, this processing is based on the premise that a human body having a higher detection score and a larger head size is more likely to exist in the foreground.

  Further, when an object is present on the back side from the camera or when a child's head is detected, the detection score is high, but the size of the head tends to be small. In this case, after sorting in descending order of detection score, a certain threshold value is provided for the detection size in a situation where the system to be constructed is actually operated, and the minimum head size that can be detected is determined. Thereby, the likelihood of a detection candidate can be made highly reliable.

  Furthermore, when the subject is wearing a hat with a long handle, the detection score may be low and the size of the head may be large. In such a case, by providing the first detection processing unit 211 to the n-th detection processing unit 21n with a detector that outputs a high detection result for the head wearing a long hat, It is desirable to correct the detection score and sort again.

Next, in step S1202, the human body estimated as the foreground is selected with reference to the detection results of the whole body and head of the human body estimated as the foreground. Hereinafter, the detection result of the whole human body estimated as the foreground is A _B, and the detection result of the human head is A _H. In this process, since the sorting is completed in step S1201, the one with the highest detection score is selected.

Next, in step S1203, the detection result of the whole body and head of the human body in the background is referred to, and the detection result of the whole body that is lower than the detection score of the human body selected in step S1202 and corresponds to the detection result of the head is obtained. Select the calculated human body. Hereinafter, the detection result of the human body systemic rear ground to be selected in this process and B _B, the detection result of the human head and B _H.

Next, in step S1204, there are two or more intersections between the detection result (A _H ) of the head of the human body in the foreground selected in step S1202 and the detection result (B _B ) of the whole body of the human body in the background. Or whether there is an overlapping area (A _H ∩B _B ). As a result of the determination, if there is no intersection or overlapping region, the process proceeds to step S1207, and if it exists, the process proceeds to step S1205.

In step S1205, the foreground human body detection result (A _B ) is used to calculate the shielding area (A _B ∩B _B ) in the foreground human body. In the case of the example illustrated in FIG. 10, the shielding region 1001 is calculated using the upper left and lower right coordinate values of each detection result. First, assuming that the upper left point of the whole body detection result of the human body 901 is (x _j , y _j ) and the upper left point of the whole body detection result of the human body 900 is (x _i , y _i ), the width w _O and the high H _O (w _{O, i} , h _{O, i} ) is expressed by the following equation (6).

When the whole body of the human body 901 is detected, the width w _B and the height h _B have already been calculated. Therefore, the area S _O of the shielding region 1001 can be obtained by the following equation (7).

  Next, in step S1206, a correction value for the detection score is determined from the area of the shielding region calculated in step S1205. As in the example shown in the present embodiment, assuming that the human body to be detected is upright, the detection score decreases as the shielding area increases, so the area between the area of the shielding area and the detection score. Can be considered to have a negative correlation. Therefore, when there is shielding, the detection score is corrected by the following equation (8).

Here, δ (S _{O, i} ) is a monotonically increasing function with a range of 0 to 1. If there is no shielding, the correction coefficient is 1, and the larger the shielding area is, the larger the coefficient is designed. By applying such correction, it is possible to cancel the decrease in detection score caused by shielding.

  Similarly to the above-described method for obtaining the correct probability when estimating the position of the head, for each pattern in which shielding occurs, by obtaining the correlation between the shielding pattern and the detection score of each detector, shielding is performed. It is also possible to record the time correction coefficient as a score correction dictionary.

  In order to obtain the correction coefficient at the time of occlusion in advance, first, an image sample group whose head position and whole body position are known is prepared. Note that it is desirable to prepare images to be prepared for an image in a state where the human body is shielded and an image in a state where the human body is not shielded. Furthermore, it is also desired that the image of the human body and the image that is not so appear in the same posture and size of the human body. This is to measure what kind of change occurs in the detection score depending on the presence or absence of the shielding state.

First, the object detection unit 101 calculates a detection score S _{C, i} of a human body without a shielding state. Next, a human body detection score S _{B, i} is calculated for each shielding pattern. Then, the correlation between the shielding area and the detection score is calculated for each shielding pattern by the least square method or the like. Since the correlation between the shielding area and the detection score is different for each shielding pattern, it is desirable to store a score correction dictionary for the number of shielding patterns assuming a possible shielding state pattern in advance.

Finally, the detection score is calculated according to the following equation (9) by multiplying the detection score S _{B, i} by the correction coefficient S _{O, p} in the shielding pattern p calculated in advance according to the shielding state. . In this embodiment, since it is assumed that shielding occurs in the whole body region other than the head and face of the human body, only the detection score S _{B, i} output from the whole body detector is multiplied. On the other hand, depending on the system to be constructed, the detection score S _{F, k} output from the head detector or the correction score S _{H, j} output from the face detector may be multiplied by the correction score corresponding to the shielding state. Good.

  As described above, by preparing a score correction dictionary for each shielding pattern and correcting the detection score, it is possible to accurately detect a human body even in a shielding state.

  Next, in step S1207, it is determined whether or not shielding determination has been performed for all detection results having a detection score equal to or greater than a threshold value. As a result of this determination, if there remains a detection result that has not yet been subjected to shielding determination, the process returns to step S1202 to repeat the process. When the process is repeated, the process returns to step S1202 to reselect the detection result as the foreground, and when there is only a detection result whose detection score is less than the threshold, the process ends.

  FIG. 11 is a diagram illustrating an example of a state in which two or more persons overlap and a shield is generated. In the example shown in FIG. 11, the front human body 1100 shields the rear human body 1101, and the rear human body 1102 is shielded by the two human bodies 1100 and 1101. A shield area 1103 indicates a shield area generated in the human body 1102 due to the influence of the two human bodies 1100 and 1101.

  As described above, based on the principle of perspective projection, in step S1201, all detection results output by the object detection unit 101 are sorted in descending order of detection score and detection frame size. By this processing, it is possible to identify the human body 1100 in the near side that is unlikely to be shielded and causes the shielding. Then, shielding determination with other detection results is performed in order from the human body that is likely to cause shielding.

  The human body 1102 is in the vicinity of the human body 1101, and the head detector outputs a high detection score for the head of the human body 1102. Therefore, when the detection score of the head of the human body 1102 is larger than the detection score of the head of the human body 1101, an area where the head of the human body 1102 and the whole body of the human body 1101 are fused Output as detection result. Further, since the human body 1102 is detected as a part of the human body 1101, it is not correctly detected. When such a false detection occurs, it is determined that the shielding state is generated in the human body 1102 using the shielding state determination method described in the present embodiment. The whole body rectangle that matches the scale of the size of the head of the human body 1102 is selected, and erroneous detection is prevented by detecting the head of the human body 1101 and the whole body rectangle.

  As described above, according to the present embodiment, it is possible to detect without lowering the detection score even when the detection target is shielded as compared with the conventional method. In addition, since the detection score is corrected for each area and shielding pattern of the shielding region obtained from the inclusion relation, the accuracy of the detection result to be finally output is improved as compared with the conventional case. Furthermore, it is possible to correct the detection score without performing sophisticated and complicated calculations by directly using the output results of the head detector and the whole body detector calculated in the object detection process.

(Second Embodiment)
In the present embodiment, a shielding state determination method and a detection score correction method when a detection processing unit that detects an object by dividing it into a plurality of movable parts will be described. Also in this embodiment, an example will be described in which the detection target is a person and the common part is a person's head. However, in the first embodiment, it is assumed that the person is always in an upright state. However, in this embodiment, it is possible to cope with a forward tilted posture that is bent forward and a posture change such as squatting. The description of the same configuration and processing as those described in the first embodiment is omitted.

  The overall configuration of the object detection apparatus 10 according to the present embodiment is basically the same as that illustrated in FIGS. 1 and 2 described in the first embodiment. However, the detection targets of the first detection processing unit 211 to the nth detection processing unit 21n are different, and the processing content of the integrated result output unit 250 is also different. In addition, as a detector used in the present embodiment, an example in which a head detector and a whole body detector are used will be described. In order to perform detection corresponding to a small change in posture of an object, for example, a parts-based detection method as described in Non-Patent Document 6 is known.

  FIG. 13 is a diagram for explaining a whole-body detector using a parts-based detection method. A dotted rectangle 1302 in FIG. 13 is one part of the whole body detector, and in the example shown in FIG. 13, the whole body detector is composed of eight parts. A solid line rectangle 1301 is a whole body detection result obtained as a result of parts-based detection.

  In the example shown in FIG. 13A and the example shown in FIG. 13B, since the posture of the person is different, the positions of the parts obtained as a result of the detection are also different. In the part-based detection result, an overall detection score calculated based on the detection score and positional relationship of each part is obtained, and the position of the object and each part represented by a solid line or a broken line shown in FIG. -Range information can be obtained.

  Hereinafter, an example of estimating the position of the head from the detection result (estimating the common part) when such a parts-based detector is used will be described. First, the process of estimating the position of the head by the first common part estimation unit 221 to the nth common part estimation unit 22n from the result of the parts-based detector will be described. As a simple case, when a part whose detection target is the head is included, the part position of the head may be used as the estimation result of the head position. Further, when the head part does not match the estimated head range (for example, when there is a part targeted for detection from the head to the shoulder), as described in the first embodiment, the detection result The head position may be estimated from the head part.

On the other hand, as shown in FIG. 13, when the detector is composed of a group of parts that do not clearly indicate the head, the position of the head can be estimated using the position information of a plurality of parts. When estimating the position of the head from the position information of a plurality of parts, the position of the estimated head is obtained by linear transformation from a vector in which the coordinate information of each part is arranged. As a linear transformation formula for estimating the upper left x coordinate x _h1 of the head position from the eight parts, the head position is estimated using, for example, a formula shown in the following formula (10).

Here, X _p is obtained by adding a constant 1 to a part coordinate vector, and B _h1 is a transformation coefficient vector. Further, x _pn and y _pn are center coordinates of the n-th part, b is a conversion coefficient of each term for _obtaining x _h1 coordinates, and includes a constant term b ₀ . W and h are the width and height of the object area (solid line rectangle 1301 shown in FIG. 13), respectively. In order to obtain the estimated position X _{h of the} head, y _h1 , x _h2 , and y _h2 may be similarly obtained using different conversion coefficients.

  In the example described above, the position of the head is estimated only from the center coordinates of each part. However, the coordinate information of the object area (solid rectangle 1301 shown in FIG. 13) obtained as a result of detection is added to the part coordinate vector. May be. The conversion coefficient vector B can be obtained by the least square method from the image sample group giving the correct answer standard of the head and the detection result of the parts-based detector to the image sample group. The method for estimating the head position is not limited to the method of least squares, and can be obtained by other regression analysis using the head position as an objective variable and a plurality of parts positions as explanatory variables.

In addition, when calculating the conversion coefficient vector B of the equation (10), a group of image samples using the body axis of the human body (a straight line connecting the center of the waist from the throat) as a correct reference instead of giving the correct reference of the head. It is possible to estimate the body axis from a plurality of parts groups. For example, as shown in FIG. 14, it is possible to detect a posture when the human body is tilted slightly forward from an upright state, a posture of crouching further, and the like. In this case, the coordinate set X _U of each point of the body axis 1401 (straight line connecting the center of throat and waist) can be obtained by the following equation (11).

  When estimating the body axis 1401, the linear transformation formula shown in Formula (10) for estimating the position of the head from the eight parts indicated by the wavy rectangle 1402 in FIG. From the image sample group given the human body axis 1401 (from the throat to the center of the waist) as the correct reference instead of the correct answer reference of the head, and the detection result of the parts-based detector on the image sample group, Similar to the estimation of the body axis 1401, the body axis 1401 can be estimated by the method of least squares. The method for estimating the body axis is not limited to the method of least squares as described above, and any means suitable for the system to be constructed may be used. In addition to the head and body axis, an upper body rectangle including the head and body axis can be calculated by the same process.

  Here, the positions of the detectors of the eight parts group greatly vary depending on the inclination of the human body, as indicated by a wavy rectangle 1404 in FIG. Therefore, in order to calculate the body axis 1403 with high accuracy, it is preferable to calculate the conversion coefficient vector B by categorizing the human body according to various inclinations (postures). However, if the number of posture categories is increased, the posture identification accuracy increases, but the amount of calculation also increases. Therefore, the number of posture categories is set according to the balance with the amount of calculation. Therefore, it is necessary to determine the number of posture categories to be identified by the system to be constructed.

<Shielding judgment method and score correction method>
Next, a shielding determination method and a detection score correction method according to this embodiment will be described.
FIG. 15 is a diagram for explaining an example in which the occlusion determination is performed based on the estimation result of the head and the estimation result of the body axis by the plurality of part detectors in the present embodiment. Human bodies 1501 and 1509 are the same human body with the same posture. A human body 1501 shown in FIG. 15A is partially shielded by a foreground human body 1500, and a human body 1509 shown in FIG. 15B is partially shielded by a foreground human body 1508 bent forward. Yes.

  First, in FIG. 15A, the human head detection frames (rectangles 1502 and 1505) and the whole body detection frames (rectangles 1503 and 1506) are calculated by the method described in the first embodiment. Then, the body axes 1504 and 1507 are calculated by the method described above in this embodiment. Here, since the human body 1501 is partially shielded, the detection score is lowered and cannot be detected with the original detection score at this stage.

  FIG. 16 is a diagram for explaining a method of calculating the area and position of the shielding region 1603. The area and position of the shielding region 1603 are calculated from the eight parts detectors (dotted lines 1600-1602 in FIG. 16) shown in FIG. 14 and various detection results indicated by the rectangles 1502-1507.

  In FIG. 16A, the body axes 1504 and 1507 have an angle of less than 10 degrees with respect to the vertical upward direction. Therefore, the postures of the foreground human body and the foreground human body can be assumed to be “both standing”. In this case, the area and position of the shielding region 1603 are calculated by the method described in the first embodiment. Then, a part detector (dotted line 1600-1602 in FIG. 16A) that overlaps the calculated shielding area is identified from eight, and the detection score of the human body 1501 is corrected.

  The specific correction method is the same as the method described in the first embodiment, and inputs an image sample group without shielding and an image sample group with shielding, and which part detector is shielded. In case separation. Then, the correlation of how much the integrated score is lowered when which part detector is shielded is calculated. If the correction coefficient corresponding to the shielding state is not applied, the part detector shielded by the shielding region 1603 is specified, and the output value when the part detector is not shielded is used instead of the part detector. By making it an output value, a decrease in the integrated score is prevented. In addition, even if the detection score of parts when there is no shielding is not calculated in advance, an appropriate constant may be substituted so that the integrated score does not decrease.

  On the other hand, in the example illustrated in FIG. 15B, the human body 1508 that is inclined forward partially shields the upright human body 1509. In the case of the example shown in FIG. 15B as well, the head and whole body detection results are output and the body axes 1510 and 1511 are calculated, as in FIG. 15A.

As shown in FIG. 16B, since the body axis 1510 has an angle of 10 degrees or more when the vertical upward direction is used as a reference, the calculation method used in the first embodiment cannot be used as it is. Therefore, when the angle is 10 degrees or more, the straight line of the body axis 1510 is extended and the intersection (x _U3 , y _U3 ) with the whole body detection frame of the human body in the background is calculated. Then, a straight line obtained by extending the body axis 1510 and a region surrounded by the whole body detection frame of the foreground are calculated as a shielding region 1604.

  As for the detection score correction method, as in the correction method shown in FIG. 16A, a part detector having a region that overlaps the shielding region 1604 is specified, and the detection score of the part detector when there is no pre-calculation is calculated. Is corrected by adopting instead.

  In the parts detector according to the present embodiment, the individual position and the detection score greatly change as the posture of the human body varies. Therefore, when performing the above score correction, it is necessary to previously calculate the correlation between the movement amount of the part detector and the detection score for each posture category using the method described in Non-Patent Document 6.

  Further, the body axis 1510 is a straight line indicating the inside of the human body 1508 and does not indicate an actual shielding area. Therefore, when it is necessary to precisely estimate the shielding area, the edge information of the human body 1508 may be extracted instead of the body axis 1510 to obtain the shielding area.

FIG. 17 is a flowchart illustrating an example of a processing procedure performed by the shielding determination unit 102 according to the present embodiment.
First, the process of step S1201 is substantially the same as the process of step S1201 of FIG. 12 described in the first embodiment.

In step S1701, the human body estimated as the foreground is selected with reference to the detection results of the whole body and the head of the human body estimated as the foreground and the detection result of the body axis. Hereinafter, the detection result of the body axis is assumed to be _AU .

Next, in step S1702, with reference to the detection result of the whole body and head of the human body in the background and the detection result of the body axis, the detection result of the head is lower than the detection score of the human body selected in step S1701. A human body whose whole body detection result corresponding to is calculated is selected. Hereinafter, the detection result of the body axis is assumed to be _BU . Thus, in the present embodiment, the detection result of the body axis is referred to in addition to the detection result of the head and the detection result of the whole body.

Next, in step S1703, the human B that may be present in a linear and a rear ground to straight body axis A _U of a human body A that may be present in the foreground referenced, or the body axis A _U extended in step S1701 The intersection with the whole body detection result B _B is determined. If there is no intersection as a result of this determination, there is no object being shielded, and the process returns to step S1701.

On the other hand, the result of the determination in step S1703, if there is an intersection, in step S1704, calculates an inclination (angle) of the body axis A _U relative to the vertically upward. Then, in step S1705, the angle of the body axis A _U is equal to or less than 10 degrees. If it is determined that the angle of the body axis A _U is less than 10 degrees, foreground human body it can be determined that the posture of "standing", proceeds to the processing in step S1706. On the other hand, if the angle of the body axis A _U is not less than 10 degrees, it is determined that the foreground of the human body is the posture of the "anteversion" or "squatting" for processing in step S1707. Here, in order to simplify the explanation, the processing is changed based on the body axis angle of 10 degrees. However, depending on the system to be constructed, the body axis angle indicating the standing position may be arbitrarily set.

In step S1706, it is determined that the human body in the foreground is standing, and the position and area of the shielding area are calculated using the shielding area calculation method described in the first embodiment. On the other hand, in step S1707, the position and area of a region surrounded by the body axis A _U or a straight line obtained by extending the body axis A _U and the whole body detection result B _B are calculated.

  Next, in step S1708, which part detector is shielded among the eight part detectors is specified, and the integrated score is corrected by replacing it with the detection score of the part detector when there is no shielding. The detection score of the parts detector may be replaced with a preset constant. Step S1207 is the same as step S1207 in FIG.

  In the present embodiment, the case where the background person is standing is described as an example, but it is possible to detect occlusion by a similar process even in a posture such as forward tilt or squatting other than standing. As described above, the process of integrating the estimation frame of the head position estimated from the whole body detector is the same as that of the first embodiment. In the present embodiment, the detection score corresponding to the shielding posture can be corrected even when the posture is shielded in a posture other than the standing posture, so that the detection can be performed with high accuracy.

(Third embodiment)
In 1st and 2nd embodiment, the case where shielding had arisen in parts other than the head of a foreground was demonstrated. In the present embodiment, a method for detecting occlusion when the foreground and background human heads or common parts overlap each other will be described. Also in this embodiment, the detection target is a person, and the common part to be estimated is the head of the person. Further, in the following description, when the heads of the foreground and background people overlap each other, it is defined that the shielding flag is ON. Further, the description of the same configuration and processing as those described in the first and second embodiments is omitted.

FIG. 18 is a diagram for explaining detection processing when the foreground human body 1800 shields the foreground human body 1801 and the heads of both human bodies overlap.
In the scene shown in FIG. 18, when the detection process is performed using the object detection unit 101 described in the first or second embodiment, one of the head detection results 1802 is obtained even though there are two human bodies. Only body mass is detected. This is because the detection result of the body axis and the whole body with a low detection score is integrated into the detection result of the head, the whole body and the body axis with a high detection score as a result of the integration processing of the object detection unit 101.

  Therefore, in the present embodiment, the presence or absence of overlapping heads is detected using the detection result of the body axis before performing the integration process, and if there is overlapping between the heads, the object detection unit 101 The processing content of the integration process is changed from the head reference to the body axis reference. This is because a correct output result that cannot be obtained by the integration processing based on the head is obtained by changing the processing content to the integration processing based on the body axis.

  In addition, when the heads overlap as in the present embodiment, the respective integrated scores do not decrease so much. This is because the object detection is not particularly affected by the discontinuity of the object because the object is detected by the parts-based detector in the present embodiment. Therefore, in the present embodiment, the detection score is hardly lowered even when the head is overlapped by the overlap of the heads, so that the detection score correction process is not performed. However, when the common part is set to other than the head, the detection score may be corrected depending on the presence or absence of shielding according to the characteristics of the system to be constructed.

  FIG. 19 is a flowchart illustrating an example of a procedure for shielding determination by the shielding determination unit 102 according to the present embodiment. First, by the processing of the object detection unit 101 described in the second embodiment, the detection results 1802 of the heads of the human bodies 1800 and 1801, the detection results 1803 and 1804 of the whole body, and the body axes 1805 and 1806 are calculated temporarily. It shall be. Further, the processing in steps S1201, S1207, S1701, and S1702 is the same as the processing in FIG. 17 described in the second embodiment, and thus the description thereof is omitted.

Next, in step S1901, it calculates a straight line obtained by extending the body axis A _U of the body A as referred to in step S1701 and S1702, the intersection of the straight line obtained by extending the body axis B _U of human B. Then, it is determined whether or not an intersection exists within the rectangle of the head detection result 1802. As a result of this determination, if the intersection is not inside the rectangle, the integration process can be performed based on the result of head detection, and thus the process proceeds to step S1902 while the shielding flag remains OFF. On the other hand, if the intersection is inside the rectangle, it is determined that the heads that are paired with each body axis overlap, the shielding flag is turned on, and the process proceeds to step S1903.

  In step S1902, since it can be determined in step S1901 that the heads of human body A and human body B do not overlap, the detection score is calculated by the integration process including the head detection results described in the first and second embodiments. To do.

  In step S1903, since it can be determined that the shielding state is caused by the head of the human body A or the human body B, the head detection result is not used in the integration process, and the body described in the second embodiment is used. Performs integration processing based on the axis angle.

  FIG. 20 is a diagram illustrating an example of the detection result of the head and body axes before the integration process. 20, a head detection result candidate 2001 indicates a detection result candidate group, and corresponds to the body axes 1806 and 2002, respectively.

  When the integration process described in the first embodiment is performed, the head detection result candidate 2001 is integrated into the head detection result 1802. In the integration process of the first embodiment, the process is performed based on the detection score and the degree of overlap of the heads. Therefore, the detection result of the head of the human body 1801 whose head is shielded is obtained by the integration process. The head detection result 1802 is absorbed. In other words, when the integration process is performed with the head as a reference when the shielding flag is set, each detection result of the human body 1800 in which the detection results of the head, body axis, and whole body of the shielded human body 1801 are shielded. Will be absorbed.

  Therefore, in this embodiment, when the shielding flag is ON, the standard of the integration process is changed from the head reference to the body axis reference. As shown in FIG. 20, since a plurality of body axes 2002 are also detected in the same manner as the head detection result 1901, detection is based on the angle formed between the body axis having the highest detection score and a straight line of detection candidates for each body axis. Integrate the results. For example, comparing the body axis 1805 and the detection results of other body axes with an angle reference (for example, the threshold is 30 degrees), the body axis 2002 is integrated with the body axis 1805 because the angle formed by each body is less than 30 degrees. The body axis 1806 is not integrated because there is a difference in the angle formed by 30 degrees or more. Further, instead of the angle, the end point (the center coordinates of the throat and the waist) of the detection result of the body axis may be used to perform the integration process based on the distance between two straight lines.

  In the present embodiment, the detection result candidate that is originally detected by the detection result integration process when shielding occurs is an exception process for the purpose of preventing the detection result from being absorbed by another result. Depending on the detection result, there may be many detection candidates whose shielding flag is ON, and the sorting process is performed on the detection score values of the detection candidates whose shielding flag is ON, and a preset number (for example, 1) 2) is selected as a detection candidate. Detection results that are not selected as detection candidates are deleted by integration processing. The number to be left as detection candidates is preferably set according to the environment of the system to be constructed.

  As described above, in the integration method of the present embodiment, even if the detection results of the head overlap by utilizing the detection results of the body axis for a plurality of human bodies taking different postures before and after, the subsequent integration processing It is possible to robustly detect a human body without being deleted.

(Other embodiments)
The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed.

101 Object detection unit 102 Occlusion determination unit

Claims (16)

An object detection device that detects a first object located in the foreground and a second object located in the background from an input image,
First detection means for detecting partial areas of the first and second objects;
Second detection means for detecting information indicating postures of the first and second objects;
Determination means for determining a shielding state of the second object by the first object based on the partial area detected by the first detection means and the information detected by the second detection means;
Correction means for correcting the detection result of the second object according to the shielding state determined by the determination means;
An object detection apparatus comprising:   The object detection apparatus according to claim 1, wherein the first and second detection units detect the position and the size by dividing the object into a plurality of parts.   The object detection apparatus according to claim 1, wherein the first detection unit detects a position and a size of a human head.   The object detection apparatus according to claim 1, wherein the second detection unit detects the position and size of the whole body of the human body.   The determination unit shields whether the partial region of the first object detected by the first detection unit intersects the region of the second object detected by the second detection unit. The object detection device according to claim 4, wherein the state is determined.   The determination unit determines a shielding state based on an inclusion relationship between the partial region of the first object detected by the first detection unit and the region of the second object detected by the second detection unit. The object detection apparatus according to claim 4, wherein the shielding area is calculated.   The object detection apparatus according to claim 6, wherein the correction unit corrects a detection result of the second object according to an area of the shielding region.   The object detection apparatus according to claim 4, wherein the second detection unit further detects a body axis of the human body.   The object detection apparatus according to claim 8, wherein the determination unit switches the determination method of the shielding state according to the angle of the body axis detected by the second detection unit.   The determination unit calculates a region generated when the body axis of the first object detected by the second detection unit intersects a rectangle representing the whole body of the second object as a shielding region. The object detection device according to claim 8 or 9, wherein   The determination means is a partial region of the first object in which a straight line extending from the body axis of the first and second objects detected by the second detection means is detected by the first detection means. The object detection apparatus according to claim 8, wherein when the two intersect each other, the partial areas of the first and second objects are determined to overlap.   When the determination unit determines that the partial regions of the first and second objects overlap, the correction unit does not correct the detection result of the second object. The object detection apparatus according to claim 11.   When the determination unit determines that the partial areas of the first and second objects overlap, the determination unit changes the detection result by the first detection unit to a detection result based on the body axis. The object detection apparatus according to claim 11, wherein the object detection apparatus is an object detection apparatus.   The determination unit, when determining that the partial regions of the first and second objects overlap each other, determines an angle formed by each body axis and a detection result candidate. Item 14. The object detection device according to Item 13. An object detection method for detecting a first object located in the foreground and a second object located in the background from an input image,
A first detection step of detecting partial regions of the first and second objects;
A second detection step of detecting information indicating postures of the first and second objects;
A determination step of determining a shielding state of the second object by the first object based on the partial area detected in the first detection step and the information detected in the second detection step;
A correction step of correcting the detection result of the second object according to the shielding state determined in the determination step;
An object detection method comprising: A program for controlling an object detection device that detects a first object located in the foreground and a second object located in the background from an input image,
A first detection step of detecting partial regions of the first and second objects;
A second detection step of detecting information indicating postures of the first and second objects;
A determination step of determining a shielding state of the second object by the first object based on the partial area detected in the first detection step and the information detected in the second detection step;
A correction step of correcting the detection result of the second object according to the shielding state determined in the determination step;
A program that causes a computer to execute.

Images