JP5872395B2 - Region dividing device - Google Patents

Description

  The present invention relates to an area dividing device that divides an image obtained by capturing an object such as a person together with a background into an object area and a background area.

  For the purpose of crime prevention or the like, the occurrence of an abnormality is detected by estimating the posture of a person based on the shape of a person region extracted from a monitoring image. Since the person area in the monitoring image is relatively small, extraction errors in the person area such as background pixel contamination and person pixel loss tend to affect subsequent processing. Therefore, it is desired to improve the extraction accuracy of the person region.

  As a technique for extracting a target area such as a person area with high accuracy, a graph cut method is known in which an image is divided into a target area and a background area by modeling by linking links between pixels. . In the graph cut method, for example, a graph in which each pixel is regarded as a node is created, and a cut that divides the graph into a node group of an object region and a node group of a background region with a minimum energy is derived.

  In the technique of Non-Patent Document 1, the energy of the luminance value (hereinafter referred to as color feature) based on the likelihood of the luminance value of each pixel as an object or background is used as the energy for area division, and the position of each pixel is determined. The energy of the shape feature based on the likelihood as the object or the background is used. That is, a pixel located at a distance closer to the shape model by placing a shape model of the object on the image is more likely to be a pixel of the object, and a pixel located farther from the shape model is more likely to be a background. . As a result, it is possible to compensate for the erroneous division that is likely to occur in the portion where the color features of the object and the background are similar by the shape feature, and the accuracy of region division is improved.

  In the technique of Non-Patent Document 1, the energy of color segmentation and the energy of shape features in all pixels are added to calculate the energy for area division.

D. Freedman and T. Zhang. Interactive graph cut based segmentation with shape priors.In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), volume 1, pages 755-762, 2005.

  However, in the prior art, since two types of feature values are used in all pixels, there is a problem that the extraction accuracy is lowered at a portion where each feature value is not suitable.

  For example, when a person wearing a white shirt is walking in front of a shelf and a white wall, the color feature is not suitable for extraction near the shirt, and the extraction accuracy is reduced. Inadequate extraction accuracy decreases.

  In other words, in the vicinity of the shirt, the energy of the color feature can be reduced in both the shirt area and the wall area in addition to the boundary between the shirt and the wall. It is easy to extract a person area including.

  On the other hand, because the energy of the shape feature can be reduced near the leg edge and the edge of the shelf near the leg, a person area lacking a part of the foot is extracted or includes the area of the shelf It becomes easy to extract a person area.

  In this way, there are parts where the color is similar between the person and the background, or there is a background edge near the person, but the extraction accuracy decreases, but the color of the person varies, and the person Since the background color and edges around the person change due to the movement of, it is difficult to preset the arrangement of appropriate contribution ratios of the energy of the color features and the energy of the shape features.

  The present invention has been made in view of the above problems, and in an area dividing apparatus that divides an image into an object area and a background area based on a plurality of types of image feature amounts, different accuracy for each part of the object An object of the present invention is to provide a region extraction device that can extract a region of an object with high accuracy even if a decrease factor occurs.

  The area dividing device according to the present invention assigns each elementary area to either the object area or the background area for each elementary area composed of at least one pixel in an image obtained by imaging a predetermined object together with the background. Thus, the image is divided into regions, and for each of the elementary regions, an attribute region to which the elementary region belongs and some types of image features for evaluating the elementary region among a plurality of types of image features. In the trial setting in which the evaluation target feature is provisionally determined, the degree of attribution representing the likelihood of the evaluation target feature of each of the elementary regions belonging to the attribution region temporarily determined in the trial setting is included in the image And an evaluation value calculation unit that calculates the integrated evaluation value by summing the values, and sets the plurality of trial settings, and the integrated evaluation in each trial setting calculated by the evaluation value calculation unit Comparing, and a region division determination unit configured to determine a segmentation result attribution area in the trial set to maximize the likelihood in the entire image.

  In another area dividing device according to the present invention, the area dividing determining unit sets the evaluation target feature as the first type of image feature for the elementary region with the belonging region as the target region, In addition, the integrated evaluation value is set by setting the trial setting that uses the image feature of the second type different from the first type as the evaluation target feature for the elementary region that uses the belonging region as the background region. The above comparison is made.

  Still another area dividing apparatus according to the present invention performs area division by a graph cut method using cost as the degree of attribution and minimizing energy given by the integrated evaluation value, and the area division determining unit Sequentially sets the plurality of types of image features as the first type of image features, and minimizes the energy by an α expansion method using the image features as labels α.

  Still another area dividing apparatus according to the present invention performs area division by a graph cut method using cost as the degree of attribution and minimizing energy given by the integrated evaluation value, and the area division determining unit Sequentially sets a combination of two of the plurality of types of image features as a set of the first and second types of image features, and uses the first type of image features as a label α. The energy is minimized by an α-β exchange method in which two types of image features are labeled β.

  A preferred aspect of the present invention is an area dividing device in which the plurality of types of image features are colors and positions of the elementary areas.

  The area dividing device according to another aspect of the present invention further includes a contribution setting unit that sets a plurality of contributions that contribute the attribution of each of the plurality of types of image features to the integrated evaluation value. For each contribution, the value calculation unit calculates the integrated evaluation value by weighting the contributions of the image features with the contributions and summing the contributions in the image. The area division result is obtained for each contribution degree, and the likelihood of the area division result at each contribution degree is evaluated according to a uniform criterion independent of the contribution degree, and the area division evaluation value is calculated. The region division result having the highest value is determined as the region division result unified for the plurality of contributions.

  According to the present invention, the contribution of the feature amount is adjusted for each part even in one target object, for example, color-oriented area division is performed at the head and shape-oriented area division is performed at the leg. As a result, even if a different factor of decreasing accuracy occurs for each part of the object, the area of the object can be extracted with high accuracy.

1 is a block diagram showing a schematic configuration of an image monitoring apparatus according to an embodiment of the present invention. It is a schematic diagram of the graph used for the graph cut method in embodiment of this invention. It is a schematic diagram explaining the process by the initial region setting part. FIG. 4 is a schematic diagram illustrating an example of an object seed and a background seed set based on an initial region shown in FIG. 3, and an example of an object pixel existence probability ρ _O and a background pixel existence probability ρ _B. It is a schematic diagram which shows the other example of the presence probability (rho) _{O of} an object pixel, and the existence probability (rho) _B of a background pixel. It is a flowchart which shows the outline of the monitoring operation | movement of the image monitoring apparatus which concerns on embodiment of this invention. It is a general | schematic flowchart of a person area extraction process. It is a schematic diagram of the graph explaining the alpha expansion method. It is a schematic diagram of an image explaining an example of a pixel attribution region and a used image feature in a person region extraction process by an α extension method. It is a schematic diagram explaining the set of the background pixel adjacent to the outline pixel of a target object used for calculation of the area | region evaluation value regarding a color.

  Hereinafter, as an example of a preferred embodiment (hereinafter referred to as an embodiment) including the region dividing device of the present invention, a person region on a monitoring image is extracted by the region dividing device, and a person posture is estimated based on the shape of the person region. An image monitoring apparatus 1 that monitors the occurrence of an abnormality will be described with reference to the drawings. The area dividing device of the present invention is provided in the image monitoring apparatus 1 as the area dividing unit 41, and divides the monitoring image into a person area in which the person of interest is shown and other background areas.

[Configuration of Image Monitoring Apparatus 1]
FIG. 1 is a block diagram showing a schematic configuration of the image monitoring apparatus 1. The image monitoring apparatus 1 includes an imaging unit 2, a storage unit 3, and an output unit 5 connected to a control unit 4.

  The imaging unit 2 is a surveillance camera. The imaging unit 2 is installed to face the monitoring space in order to image a person moving in the monitoring space, and images the monitoring space at a predetermined time interval. The captured monitoring images of the monitoring space are sequentially output to the control unit 4. In the present embodiment, in order to specify the position of a person with three-dimensional coordinates, the two imaging units 2-1 and 2-2 are installed with a common visual field. These camera parameters of the imaging unit 2 are measured by a pre-calibration and stored in the storage unit 3.

  The storage unit 3 is a storage device such as a ROM (Read Only Memory) or a RAM (Random Access Memory). The storage unit 3 stores various programs and various data, and inputs / outputs such information to / from the control unit 4.

  The various data includes tracking information 30, a person shape model 31, graph information 32, and camera parameters (not shown).

  The tracking information 30 is data such as a person position that is a result of tracking a person, a person template that is generated for tracking the person and characterizes the person. The person position and person template of the person are stored in association with the person ID for each person. The coordinates of the person's head center in the three-dimensional coordinate system simulating the monitoring space are stored as the person position of the person.

  The person shape model 31 is shape data imitating the shape of a person. In the present embodiment, three-dimensional shape data in which the three parts of the head, torso, and leg of a standing person are approximated by a spheroid with the vertical axis as the rotation axis, and these are aligned in the vertical direction in order from the top. Is created and stored in advance.

  An area dividing unit 41 to be described later generates a graph as shown in FIG. 2 for the monitoring image, and α-expansion is performed to divide the graph into a human area and a background area with the minimum energy. The person region is extracted from the monitoring image by deriving by the Graph Cut method to which the method is applied.

In the graph shown in FIG. 2, the perspective view of the horizontal plane schematically represents an image that is a set of pixels. The area dividing unit 41 sets, for example, each pixel as a node as a minimum unit (elementary area) of a person area and a background area, and sets a source S and a sink T as virtual terminals on the person area side and the background area side. To do. A link (n-link) between adjacent nodes is set, and a link (t-link) is set between each node and the source and between each node and the sink. Further, the link degree of the link is set for each link. Thus, the area dividing unit 41 generates a graph for the monitoring image. In the present invention, two types of sources, ie, a color feature amount source (color source S _C ) and a color feature amount source (shape source S _S ) are provided. The degree of coupling is recorded in energy as the cost required for link disconnection for area division. Hereinafter, the value of the degree of coupling is referred to as cost.

The area dividing unit 41 sets an edge cost for cutting the n-link in accordance with the area division for each n-link. Moreover, the cost (background attributable during color cost) of the color feature when cut the t-link is assigned the node to the background area in t-link between each node and the color source S _C set cost (background attributable during shape cost) of the shape feature quantity when cut the t-link is assigned the node to the background area in t-link between each node and shape the source S _S Is set to t-link between each node and the sink T, and the cost related to the color feature amount when the t-link is cut and the node is attributed to the object region (color cost at the time of object attribution). ) And a cost (shape cost at the time of object attribution) related to the shape feature amount when the node is attributed to the object region. Since each cost is a value that increases when the division is not correct, the sum of the costs of the links that are disconnected when the monitoring image is divided into the person area side node and the background area side node is the energy of the area division. The cut that minimizes the energy is derived by the graph cut method using the α extension method. Deriving a cut that minimizes energy is equivalent to deriving a segmentation that maximizes the likelihood of belonging.

The graph information 32 is cost data that is the basis of the energy for area division. Edge cost c _E (p, q) for each combination of adjacent pixels {p (x _p , y _p ), q (x _q , y _q )} is stored and for each pixel p (x _p , y _p ) the background attributable when color cost between the color source _{S C α C · c C (} p, S C), background attributed when the shape cost between the shape source _{S S α S · c S (} p, S S ), The color cost α _C · c _C (p, T) when the object belongs to the sink T and the shape cost α _S · c _S (p, T) when the object belongs to the sink T The

Here, α _C is the degree of contribution of the energy (color energy) of the color feature quantity to the energy of area division, and α _S is the ratio of the degree of contribution of the energy (shape energy) of the shape feature quantity to the energy of area division (feature ratio). ). A value larger than 0 is set to α _C and α _S.

  The control unit 4 is configured by using an arithmetic device such as a CPU (Central Processing Unit), a DSP (Digital Signal Processor), or an MCU (Micro Control Unit), and reads out and executes a program from the storage unit 3 to execute a person tracking unit. 40, functions as an area division unit 41, an abnormal posture determination unit 42, and the like.

  The person tracking unit 40 processes the monitoring image from the imaging unit 2 to track the person position of each person shown on the monitoring image, and the monitoring image, the person position, the person ID assigned to the person, and the person The camera ID assigned in advance to the image capturing unit 2 that captured the monitoring image is output to the region dividing unit 41.

  When the monitor image and the person position of each person are input from the person tracking unit 40, the area dividing unit 41 is divided into a person area (object area) in which the person is reflected and other background areas. The result of the division is output to the abnormal posture determination unit 42.

  The area dividing unit 41 includes an initial area setting unit 410, a division cost calculating unit 411, an energy calculating unit 412, and an area determining unit 413.

  Hereinafter, each part which comprises the area | region division part 41 is demonstrated.

  The initial region setting unit 410 sets an initial region having the approximate position and approximate shape of the person region on the monitoring image as the initial value of the person region, and outputs the initial region information to the division cost calculation unit 411. The initial area is a clue for area division.

  Specifically, the initial region setting unit 410 refers to the person position and person shape model 31 of each person input from the person tracking unit 40, and places the person shape model 31 on the monitoring image based on the person position. To set the initial area. For this purpose, the initial region setting unit 410 arranges the person shape model 31 at a person position in a virtual space imitating the monitoring space, projects the arranged person shape model 31 onto the monitoring image by coordinate conversion using camera parameters, Set the projected area as the initial area. The initial region is set for each person, and when the person is captured by a plurality of imaging units 2, the initial area is set for each monitoring image captured by each imaging unit 2. The correspondence between the imaging unit 2, camera parameters, and monitoring image is specified by the camera ID.

  FIG. 3 is a schematic diagram for explaining processing by the initial region setting unit 410. FIG. 3A shows a monitoring image 100 in which the person 101 is captured. The initial region setting unit 410 receives the monitoring image 100 and the person position 112 in the XYZ coordinate system in the virtual space 110 obtained by tracking the person 101. The input person position 112 is represented by head center coordinates. FIG. 3B is a schematic perspective view of the virtual space 110 for explaining the processing for generating the initial region 121 from the person model 113, and FIG. 3C is a schematic diagram showing the processing result. The initial region setting unit 410 places the human model 113 in the virtual space 110 with its head center aligned with the human position 112, with its lower end grounded on the floor 111, and images the human model 113 using camera parameters. Projecting onto the xy coordinate system of the imaging surface 115 of the unit 2 (camera 114). As a result, an initial region 121 in which the person model 113 is projected onto the projection image 120 having the same xy coordinate system as that of the monitoring image 100 is calculated.

  The area dividing unit 41 performs area division using a plurality of types of image features having different types. For example, the area dividing unit 41 uses the color characteristics of the object and the background and the shape characteristics of the object for area division. Use multiple types of image features. For example, the shape feature is used in the part of the target where the color characteristics of the object and the background are similar and the accuracy of area division due to the color feature decreases, and the accuracy of area division based on the shape characteristic is large because there are many background edges in the vicinity. By selecting the best one type of image feature (evaluation target feature) for evaluating the likelihood of the attribution of the node for each node, such as using a color feature at the part of the target for which The region segmentation is performed with higher accuracy than when a single image feature is used. At this time, there are various relationships between the object and the background, and it is difficult to set the best relationship between the node and the evaluation target feature in advance. Therefore, the region dividing unit 41 dynamically sets the best relationship between the node and the evaluation target feature for each monitoring image and obtains a region dividing result.

In the present embodiment, the region dividing unit 41 calculates energy E defined by the following formula (1) as an integrated evaluation value. The energy E is defined as a linear sum of color energy E _C , shape energy E _S and edge energy E _E. The region dividing unit 41 derives a region dividing result that minimizes the energy E by a graph cut method to which the α expansion method is applied.

  Here, A is a label matrix in which each node belongs to either the object area or the background area, that is, which area each node belongs to. Further, i is a feature to be evaluated, C is a color feature, and S is a label corresponding to the shape feature.

That is, the first term on the right side of Equation (1) is the color cost α _C · c _C (p, S _C ) at the time of background attribution of the background pixel p whose evaluation target feature is a color feature, and the evaluation target feature is a shape feature. Shape cost α _S · c _S (p, S _S ) at the time of background attribution of the background pixel p and color cost α _C · c _C (p, T) at the time of object attribution of the person pixel p whose evaluation target feature is a color feature And the sum of the shape cost α _S · c _S (p, T) at the time of object attribution of the person pixel p whose evaluation target feature is a shape feature.

The second term on the right side of the equation (1) represents the sum of n-links cut by region division, that is, the edge cost c _E (p, q) set at the boundary between the person pixel and the background pixel.

  The division cost calculation unit 411 uses the initial region as a reference, and for each pixel of the monitoring image, it is unlikely that the image feature of the pixel belongs to each of the object region and the background region, that is, the degree of likelihood. A cost value representing the low is calculated for each image feature.

Specifically, the division cost calculation unit 411 extracts the color feature amount of the object (object color feature) and the background color feature amount (background color feature) from the monitoring image with reference to the colors inside and outside the initial region. Then, the object color feature is compared with each pixel to calculate a color cost c _C (p, T) at the time of object assignment representing the likelihood that the pixel belongs to the object region, and α _C Α _C · c _C (p, T) multiplied by is stored in the graph information 32 of the storage unit 3. Further, the pixel is calculated the background attributable during color costs representing a plausible lack of it attributable to the background area by comparing the color feature of each pixel and the background color characteristics, alpha _C · multiplied by the alpha _C to c _C (p, S _C ) is stored in the graph information 32 of the storage unit 3.

Further, the division cost calculation unit 411 sets a probability that the position of each pixel is in the object region and a probability that it is in the background region, based on the shape of the initial region. Then, the division cost calculation unit 411 represents the object belonging shape cost c _S (p, T) indicating the likelihood that the pixel belongs to the object area based on the probability that the position of each pixel is in the object area. ) And α _S · c _S (p, T) obtained by multiplying this by α _S is stored in the graph information 32 of the storage unit 3. Further, the division cost calculation unit 411 calculates a shape cost at the time of background attribution that represents the likelihood that the pixel belongs to the background region based on the probability that the position of each pixel is in the background region, and α _S Α _S · c _S (p, S _S ) multiplied by is stored in the graph information 32 of the storage unit 3.

Further, the division cost calculation unit 411 calculates an edge cost c _E (p, q) corresponding to the luminance difference between the adjacent pixels and stores the calculated edge cost c _E (p, q) in the graph information 32 of the storage unit 3.

Hereinafter, calculation of the edge cost c _E (p, q) will be described.

The division cost calculation unit 411 calculates an edge cost c _E (p, q) represented by the following expression for each n-link set between the pixel p and the adjacent pixel q.

Here, Ip represents the pixel value of the pixel p, Iq represents the pixel value of the adjacent pixel q, and dist (p, q) represents the distance between the position of the pixel p and the position of the adjacent pixel q. β is a constant for adjustment, and an appropriate value is set in advance through a preliminary experiment or the like. That is, the division cost calculation unit 411 sets an edge cost c _E (p, q) that is smaller as the pixel values are different from each other and similar to each other.

Hereinafter, the setting of the object seed and the calculation of the object belonging attribution color cost c _C (p, T) will be described.

The division cost calculation unit 411 extracts an object color feature that serves as a reference for the color feature of the object from pixel values inside the initial region in the monitoring image. In order to extract the object region with high accuracy, the object color feature is sufficiently likely to be a part of the object, and it is desirable to cover the colors constituting the object. Therefore, the division cost calculation unit 411 determines a pixel group on the central axis of the initial region as an object seed, and extracts a normalized color histogram h _O of the pixel value of the object seed as an object color feature.

  FIG. 4 illustrates an object seed 200 set on the central axis of the initial region 121 of FIG. The object seed 200 is set so as not to include the vicinity of the outline of the initial area 121 where the object area or the background object area is ambiguous.

The division cost calculation unit 411 calculates the color cost c _C (p, T) at the time of object assignment according to the following expressions (3) and (4).

Here, Ip is the pixel value of the pixel p, h _O is a normalized color histogram of the object seed, and h _O (Ip) represents the probability that the pixel value Ip is the color of the object. The value of L _C (p | оbj) decreases as the probability that the color of the pixel p is the color of the object is higher, and increases as the probability is lower. K (> 1) is a constant representing a large cost value, and a sufficiently large value is set in advance.

As described above, the division cost calculation unit 411 has an object belonging color between each pixel p and the sink T that is so low that the color of the pixel p seems to be an object, and so high that the color of the pixel p does not seem to be an object. A cost c _C (p, T) is set.

Hereinafter, setting of the background seed and calculation of the color cost at the time of background attribution c _C (p, S _C ) will be described.

The division cost calculation unit 411 extracts a background color feature that serves as a reference for the background color feature from pixel values outside the initial region in the monitoring image. In order to extract the object region with high accuracy, it is highly likely that the background seed is a part of the background, and it is desirable to cover the background color existing at the boundary with the object. Therefore, dividing the cost calculation unit 411, defined as background seed pixel group of an outer peripheral portion surrounding off the initial region by a predetermined distance, extracting the normalized color histogram h _B of the pixel values of the background seed as the background color characteristics. Specifically, the division cost calculation unit 411 expands the initial region a predetermined number of times and determines the surrounding pixels of the expanded region as the background seed. The number of expansions can be determined to be larger than the approximate error in the initial region, and can be, for example, about 10 times. In the present embodiment using the graph cut method to which the α expansion method is applied, the graph cut method is executed in two stages. In order to extract the object pixel that could not be extracted due to the color feature in the first stage in the second stage, the division cost calculation unit 411 adds the object pixel obtained in the first stage to the background seed in the second stage. .

  FIG. 4 exemplifies a background seed 201 set in the outer peripheral portion separated by 10 pixels from the outline of the initial region 121. The background seed 201 is set so as not to include the vicinity of the contour of the initial area 121 where the object area or the background object area is ambiguous.

The division cost calculation unit 411 calculates the background attribution color cost c _C (p, S _C ) according to the following formulas (5) and (6).

Here, h _B is a normalized color histogram of the background seed, and h _B (Ip) represents the probability that the pixel value Ip is the color of the background region. K and Ip are as described above. The value of L _C (p | bkg) decreases as the probability that the color of the pixel p is the background color is higher, and increases as the probability is lower.

Dividing the cost calculation unit 411 thus is between each pixel p and the color source S _C, low color of the pixel p is more likely background, higher background attributable during color cost color not like the background of the pixel p c _C (p, S _C ) is set.

Hereinafter, calculation of the object belonging shape cost c _S (p, T) will be described.

The division cost calculation unit 411 sets the existence probability ρ _O of the target pixel at each pixel position based on the position and shape of the initial region. Specifically, the division cost calculation unit 411 sets the object pixel existence probability ρ _O to 0 for pixels outside the initial region, and approaches 1 for pixels farther away from the contour of the initial region inside the initial region. Set. An example of the existence probability ρ _O of the object pixel is shown in FIG. The horizontal axis of the graph of the existence probability ρ _O shown in FIG. 4 represents the position along the straight line in the x-axis direction indicated by a dashed line in the image including the initial region 121 shown in the upper part of FIG. The vertical axis is ρ _O. In this example, ρ _O becomes 1 which is the maximum value in the object seed 200, decreases linearly toward the value 0 in the contour of the initial region 121, and becomes 0 outside the contour.

The division cost calculation unit 411 calculates the object belonging shape cost c _S (p, T) based on ρ _O according to the following expressions (7) and (8).

Here, ρ _O (p) represents a probability that the position of the pixel p in the image is within the object region. K is as described above. The value of L _S (p | оbj) decreases as the probability that the position of the pixel p is within the object region is higher, and increases as the probability is lower.

In other words, the division cost calculation unit 411 has a shape cost at the time of object attribution that is so low that the position of the pixel p between the pixels p and the sink T seems to be an object and the position of the pixel p does not seem to be an object. c _S (p, T) is set.

Hereinafter, calculation of the shape cost at the time of background attribution c _S (p, S _S ) will be described.

The division cost calculation unit 411 sets the background pixel existence probability ρ _B at each pixel position based on the position and shape of the initial region. Specifically, the division cost calculation unit 411 sets the value of the background pixel existence probability ρ _B to 0 for the pixels inside the background seed 201 and the values closer to 1 for the pixels farther away from the background seed 201 outside the background seed 201. Set. An example of the background pixel existence probability ρ _B is shown in FIG. The horizontal axis of the graph of the existence probability ρ _B shown in FIG. 4 represents the position along the straight line in the x-axis direction indicated by the alternate long and short dash line in the image including the initial region 121 shown in the upper part of FIG. the vertical axis is ρ _B. In this example [rho _B increases linearly outward from the background seeds 201.

The division cost calculation unit 411 calculates a background belonging-time shape cost c _S (p, S _S ) based on ρ _B according to the following equations (9) and (10).

Here, ρ _B (p) represents the probability that the position of the pixel p in the image is in the background area. K is as described above. The value of L _S (p | bkg) decreases as the probability that the position of the pixel p is in the background region is higher, and increases as the probability is lower.

In this way, the division cost calculation unit 411 has a background belonging shape cost c _S between each pixel p and the source S that is so low that the position of the pixel p seems to be background, and so high that the position of the pixel p does not seem to be background. (P, S _S ) is set.

FIG. 4 shows an example in which the values of the object pixel existence probability ρ _O and the background pixel existence probability ρ _B are both set to 0 around the initial region 121 and the background seed 201. As described above, ranges where ρ _O and ρ _B are larger than 0 may be set outside and inside the boundary of the initial region 121.

  In this way, the division cost calculation unit 411 sets each cost, thereby completing a graph for dividing the monitoring image into regions.

  The energy calculation unit 412 (evaluation value calculation unit) uses the cost corresponding to the setting of each pixel as the degree of attribution of the pixel in the trial setting in which the attribution region and the evaluation target feature of each pixel are provisionally determined from the storage unit 3. These are read out and summed up in the image to calculate the energy (integrated evaluation value) of area division represented by the trial setting. Specifically, the energy calculation unit 412 calculates energy E according to the equation (1).

  A plurality of trial settings are generated, and the generation is performed by the region determination unit 413 (region division determination unit). That is, the region determination unit 413 temporarily determines the belonging region setting A and the evaluation target feature type i in the equation (1) when the energy calculation unit 412 calculates the energy E. The region determination unit 413 compares the energy E in a plurality of trial settings, and detects the belonging region setting in the trial setting that minimizes the energy E, whereby the evaluation target feature of each pixel belongs to the belonging region of the pixel. A trial setting in which the likelihood of this is maximized for the entire image is detected, and the belonging area in the detected trial setting is determined as a region division result. The region determination unit 413 outputs the determined region division result to the abnormal posture determination unit 42.

  In this way, the region determination unit 413 repeats the process of calculating the energy by inputting the trial setting to the energy calculation unit 412 while changing the trial setting, and searches for the trial setting that minimizes the energy. A region division result is derived in which the likelihood of the pixel evaluation target feature and the pixel's attribution region is maximized for the entire image.

  By doing so, it is possible to divide the area by adaptively selecting the evaluation target feature for evaluating each elementary area for each monitoring image. As a result, it is possible to select image features that are unlikely to deteriorate in accuracy at each part of the object, and thus it is possible to perform high-precision area division adapted to the diversity of the relationship between the object and the background.

  The attribution region and the evaluation target feature with the minimum energy can be efficiently determined by the graph cut method to which the α extension method is applied. Details of the processing will be described later in the description of the operation.

  The abnormal posture determination unit 42 determines whether or not the shape of the person area of each person extracted by the region dividing unit 41 is an abnormal posture indicating the occurrence of an abnormal situation, and any of the person regions is determined to be an abnormal posture. A predetermined abnormality signal is output to the output unit 5. Specifically, the abnormal posture determination unit 42 calculates the degree of similarity between the shape of each person area and the abnormal posture pattern registered in advance and compares it with a preset threshold value. It is determined that the person area for which is calculated is an abnormal posture, otherwise it is determined that it is not an abnormal posture. For example, a posture shape pattern with both hands raised can be registered in advance as an abnormal posture pattern indicating the occurrence of a robbery case.

  The output unit 5 is an external output device that outputs an abnormal signal to the outside when an abnormal signal is input from the abnormal posture determination unit 42. For example, the output unit 5 includes a communication circuit connected to a security center via a telephone network or a wide area network such as the Internet, and notifies the occurrence of an abnormal situation by transmitting an abnormal signal to the security center.

[Operation of the image monitoring apparatus 1]
FIG. 6 is a flowchart showing an outline of the monitoring operation of the image monitoring apparatus 1. The operation of the image monitoring apparatus 1 will be described with reference to FIG. When an administrator who confirms that the monitoring space is unmanned turns on the apparatus, each unit and each means are initialized and start operating (S1). After the initialization, every time a new monitoring image is input from the imaging unit 2 to the control unit 4, the processes in steps S2 to S7 are repeated as a loop process.

  When a new monitoring image is input, the person tracking unit 40 of the control unit 4 tracks the person on the monitoring image and specifies the position of the person on the monitoring image (S2). The person tracking unit 40 stores the person position specified by the new monitoring image in the tracking information 30 of the storage unit 3 in association with the person ID and the camera ID.

  The control unit 4 checks whether or not a person is present on the new monitoring image, that is, whether or not the person position specified by the new monitoring image is stored in the tracking information 30 (S3). If there is no person (NO in step S3), control unit 4 skips the subsequent processes and returns the process to step S1.

  If a person exists (YES in step S3), the control unit 4 inputs the tracking information 30 obtained from the new monitoring image to the area dividing unit 41, and the area dividing unit 41 extracts the person area of each person. (S4).

  FIG. 7 is a schematic flowchart of person area extraction processing. Hereinafter, the person region extraction process in step S4 will be described with reference to FIG.

  In the graph cut method to which the α extension method is applied, a graph having a plurality of sources is divided into a graph composed of one of the plurality of sources and a sink, and energy is minimized in a stepwise manner. At this time, different types of costs are set for the t-link on the source side and the t-link on the sink side.

  In this embodiment, energy is minimized by dividing the graph shown in FIG. 2 into two graphs 400 and 401 shown in FIG. FIG. 9 is a schematic diagram of an image for explaining an example of the pixel attribution region and the evaluation target feature in the human region extraction processing by the α extension method.

  Hereinafter, this process will be described.

  First, the initial region setting unit 410 of the region dividing unit 41 reads the person shape model 31 and the camera parameter of the camera ID corresponding to the monitoring image from the storage unit 3, and uses the person's position of each person as a reference to create a virtual space. The person shape model 31 is placed therein, and the placed person shape model 31 is projected on the monitoring image by the camera parameter to set the initial area of each person (S100).

Next, the division cost calculation unit 411 of the region division unit 41 calculates the object pixel existence probability ρ _O and the background pixel existence probability ρ _B in each pixel according to the distance from the initial region of each person, respectively. It calculates as a feature and a background shape feature (S101).

Subsequently, the division cost calculation unit 411 calculates the edge cost c _E (p, q) for each combination of adjacent pixels according to the equation (2) and stores it in the graph information 32 (S102).

Further, the division cost calculation unit 411 sets the background seed of each person (S103), and calculates the color cost at the time of background attribution of each pixel with respect to the background seed for each person (S104). That is, dividing the cost calculation unit 411 extracts the normalized color histogram h _B as the background color characteristic from the periphery of the initial region of each person, when attribution background for each pixel according to the equation (5) and (6) for each person The color cost α _C · c _C (p, S _C ) is calculated and stored in the graph information 32.

Further, the division cost calculating unit 411 uses the object pixel existence probability ρ _{O for} each person calculated in step S101, and uses the object attribution shape cost α _S for each pixel according to the equations (7) and (8). C _S (p, T) is calculated and stored in the graph information 32 (S105). The processing up to this point generates the graph 400 of FIG. 8, that is, the first-stage graph 400 that includes the color source S _C and does not include the shape source S _S.

  The region determining unit 413 of the region dividing unit 41 applies the Minimum Cut / Maximum Flow algorithm to the graph 400 defined by the graph information 32, and converts the graph into the object region node and the background region node with the minimum energy. The attribute region setting A1 to be divided into two is derived (S106). That is, the region determining unit 413 minimizes the energy E by repeating the process of calculating the energy E of the equation (1) by inputting the belonging region setting temporarily determined while slightly changing the belonging region setting to the energy calculating unit 412. Search and determine the belonging area setting. In the first stage, the region determination unit 413 performs region division by setting the object belonging shape cost to the sink side t-link and the background belonging color cost to the source t-link. With this setting, the region determination unit 413 sets the evaluation feature as a color feature (corresponding to the label α in the α expansion method as the label C of the color feature) and the attribution for the pixel having the belonging region as the target region. That is, energy is compared by setting a trial setting in which a feature to be evaluated is a shape feature for a pixel whose region is a background image.

  FIG. 9A shows a state in the first stage, and the belonging area setting 500 is an example of the belonging state A1 derived in the first stage. In the subsequent step S111, the object area obtained at this stage is finally adopted as a part of the object area as it is (attributed area setting 501 in FIG. 9 (1)). This means that the object pixel is selected by setting the evaluation target feature as a color feature (evaluation target feature setting state 502 in FIG. 9A).

The division cost calculation unit 411 resets the cost of the graph information 32 and proceeds to the second stage. FIG. 9 (2) shows a state when the process moves to the second stage. The division cost calculation unit 411 sets each person's object area derived in step S106 to the background seed of the person in the second-stage graph 401 (background seed B indicated by the belonging area setting 503 in FIG. 9B). ) The attribution of the object area added to the seed does not affect the second stage and will not be changed. The object seed of each person is set (S107), and the color cost at the time of object assignment of each pixel with respect to the object seed is calculated for each person (S108). That is, the division cost calculation unit 411 extracts the normalized color histogram h _O from the center of the initial region of each person as an object color feature, and the object belonging attribute color cost for each pixel according to Expressions (3) and (4). α _C · c _C (p, T) is calculated and stored in the graph information 32.

Subsequently, the division cost calculation unit 411 uses the existence probability ρ _B of the background pixel for each person calculated in step S101, and uses the shape cost α _S · at the time of background attribution for each pixel according to the equations (9) and (10). c _S (p, S _S ) is calculated and stored in the graph information 32 (S109). Graph 401 of Figure 8 by the processing up to this point, that the second stage of the graph 401 without the shape source S colors include _S source S _C is generated.

The area determination unit 413 displays the minimum in the graph 401 defined by the graph information 32.
By applying the Cut / Maximum Flow algorithm, attributed area setting A2 is derived that divides the graph into a target area node and a background area node with a minimum energy (S110). That is, the region determining unit 413 minimizes the energy E by repeating the process of calculating the energy E of the equation (1) by inputting the belonging region setting temporarily determined while slightly changing the belonging region setting to the energy calculating unit 412. The attribution area setting A2 is searched and determined.

  FIG. 9 (3) shows a state in the second stage, and the belonging area setting 504 is an example of the belonging area setting A2 derived in the second stage. In the second stage, the region determination unit 413 performs region division by setting the color cost at the time of object assignment to the t-link on the sink side and the shape cost at the time of background attribution to the t-link on the source side. With this setting, the region determination unit 413 sets the evaluation feature as a shape feature (equivalent to setting the label α in the α expansion method as the shape feature label S) for the pixel having the belonging region as the object region, and the attribution That is, energy is compared by setting a trial setting in which the evaluation target feature is a color feature for a pixel whose region is a background image.

  The area determination unit 413 determines the final object area by connecting the object pixel obtained in step S106 and the object pixel obtained in step S110, and sets the other areas as background areas. Determine (S111).

  The belonging area setting 505 in FIG. 9 (3) is an example of the final belonging area setting A. The object pixel obtained in the second stage is selected by setting the evaluation object feature as the shape feature (state 506 in FIG. 9 (3)). It should be noted that the evaluation target characteristics of the background pixels other than the background seed obtained at this stage (pixels indicated by “S / C” in the state 506) are the object belonging color color and the object belonging time set for the pixel. It can be an image feature that gives the minimum value of the shape cost.

  In each of the above steps, the region determination unit 413 determines the evaluation target feature as the first type image feature for the elementary region having the belonging region as the target region, and the elementary region having the belonging region as the background region. Then, a trial setting that sets the evaluation target feature as a second type of image feature different from the first type is set, and the integrated evaluation values are compared. By making settings that directly contrast different image features in this way, it is possible to efficiently select the image features to be evaluated in each elementary region, so a highly accurate region adapted to the diversity of the relationship between the object and the background The division can be efficiently derived.

  In this way, the area dividing unit 41 determines an area division that minimizes the energy E while selecting any one image feature as an evaluation target feature at each node.

  When the person region of each person is extracted by the above processing, the control unit 4 advances the processing to step S5 in FIG.

  The continuation of the image monitoring process will be described with reference to FIG. 6 again.

  The abnormal posture determination unit 42 of the control unit 4 calculates the similarity between the shape of the person area of each person input from the region determination unit 413 and the abnormal posture pattern, and compares it with a preset threshold value. It is determined that the person area for which the similarity equal to or greater than the value is calculated is an abnormal posture, and otherwise, it is determined that the person region is not an abnormal posture (S5).

  If any of the person regions is determined to be in an abnormal posture (YES in step S6), the abnormal posture determination unit 42 generates a predetermined abnormal signal and outputs the signal to the output unit 5 (S7). The output unit 5 to which the abnormal signal has been input transmits the abnormal signal to the security center and makes a report. On the other hand, if none of the person regions is determined to be in an abnormal posture (NO in step S6), the abnormal output process in step S7 is skipped.

  When the above processing is completed, the control unit 4 returns the processing to step S1, and processing for the next monitoring image is performed.

[Modification]
(1) In the above embodiment, an example is shown in which region division is performed using each pixel as a raw region. However, the elementary region associated with the node may be other than the pixel. For example, pixels having similar pixel values can be segmented together in advance, and each segment can be set as a node to perform region division.

  In this case, the color cost for each segment is calculated using the representative pixel value (average value, median value, or mode value) of the segment, or the color cost for each pixel constituting the segment is calculated. The representative value (average value, median value, or maximum value) of these color costs is used as the color cost of the segment.

  In addition, the shape cost for each segment is calculated using the degree of overlap between the segment and the initial region, or the representative value of the existence probability for the pixels constituting the segment (the average value, median value, or mode value of the existence probability) Is the shape cost of the segment.

  By doing so, the number of nodes can be reduced without degrading the accuracy of area division, so that both accuracy maintenance and load reduction can be achieved.

  (2) In the above embodiment, an example in which color and shape are used as image features has been described, but other image features can also be used. For example, colors and motion feature quantities are used. In this case, the background difference process can be performed to set the background difference value of each pixel as the motion feature amount. In addition, the size of the movement vector of each pixel can be used as a motion feature amount by performing an optical flow analysis.

  (3) In the above embodiment, a region division result for minimizing energy is derived by the graph cut method. In another embodiment, the Markov Chain Monte Carlo (MCMC) method, the Belief Propagation method, and the Tree-Reweighted Message Passing (TRW) method are used instead of the graph cut method. Thus, the region segmentation result that minimizes the energy can be derived.

  (4) In the above embodiment, the graph 400 is generated in the first stage of the α expansion method and the graph 401 is generated in the second stage, and the graph cut method is performed. That is, the graph cut method may be performed by generating the graph 401 in FIG. 8 in the first stage and the graph 400 in the second stage.

  (5) In the above embodiment, the image feature is selected by the α expansion method, but an α-β exchange (αβ-swap) method can be used instead of the α expansion method. For example, the α-β exchange method can be applied with the evaluation target feature of the object region as a label α and one of the evaluation target features of the background region as a label β. In the above-described embodiment, since there are two types of image features, color features and shape features, the label α is one of them and the label β is the other of them.

  (6) The present invention can also be applied to the case where three or more types of image features are used, and in this case as well, region division results can be obtained using the α expansion method, α-β exchange method, and other methods described above. Can be derived.

  For example, when three types of image features A, B, and C are used, the α expansion method repeats the processing using the image features A, B, and C as labels α sequentially. That is, the processing is performed in two stages when there are two types of image features described above, but is performed in three stages when there are three types of image features.

  In the α-β exchange method, the process is repeated for all combinations of labels α and β that can be specified in the image features A, B, and C. For example, when the image feature A is an evaluation target feature of the object region and this is the label α, the image feature B is the background region evaluation target feature and this is the label β, and the image feature C is the background region. And a process of setting this as a label β.

  (7) In the above embodiment, the feature ratio α of each image feature is fixed and the region division is performed. However, in another embodiment, the feature ratio α is dynamically set. In this case, the feature ratio α is set in a plurality of ways, the region is divided by each setting, and the region division result in each setting is evaluated based on a uniform standard independent of the feature ratio, and the region evaluation value (region division evaluation value) And the area division result having the maximum area evaluation value can be adopted.

  As the region evaluation value, the region evaluation value V defined by the following equation can be used.

Here, 1 / V _C is a region evaluation value relating to color, and 1 / V _S is a region evaluation value relating to shape.

  The set Edge of the pixels p to be summed in Expression (12) is a set of contour pixels of the object, N (p) is a set of background pixels adjacent to the contour pixels of the object, and dist is the pixel p. And the distance between q and q. γ is a constant for adjustment, and an appropriate value is set in advance through a preliminary experiment or the like. FIG. 10 is a diagram for explaining N (p), and a schematic diagram of a partial image including a contour pixel of an object is shown on the left side of FIG. Here, the cost of n-link is calculated for four neighborhoods of each pixel as shown in FIG. On the other hand, N (p) is obtained from the vicinity of the contour pixel of the object as shown in FIG. 10, and the difference from the adjacent pixels is evaluated more than when the n-link cost is calculated. Good. By doing so, it is possible to calculate a region evaluation value that is stricter than the evaluation by the energy of the color feature in the energy calculation unit 412 described above, and it is possible to more strictly evaluate the superiority or inferiority between the region division candidates.

In Formula (13), M _λ is the number of pixels whose pixel positions match in the object region and the initial region in the region division candidate, M ₀ is the number of pixels in the initial region, and M _S is the number of pixels in the region division candidate. . The number match pixels of the initial region M _lambda is increased when 1 / V _S is higher. That is, 1 / V _S is the matching rate of the object region with respect to the initial region. The term, however 1 / M _S, only simply large object area (e.g., object region encompasses state early region) 1 / V _S in is prevented from becoming unduly high.

In this case, if the feature ratio α _C is treated as a constant and λ = α _S / α _C , the feature ratio can be controlled with one variable. That is, by changing λ, it is possible to change the ratio between image features with respect to the degree to which each image feature contributes to the region division. For example, the region dividing unit 41 is provided with a contribution ratio setting unit that sets a plurality of contribution ratios λ. Then, the energy calculation unit 412 calculates the integrated evaluation value by weighting each attribution evaluation value for the image feature by the contribution ratio λ and summing it in the image, and the region determination unit 413 converts the contribution evaluation value into the integrated evaluation value for each contribution ratio. Based on the search, the region division result is obtained. Then, an area evaluation value is calculated by evaluating the likelihood of the belonging state, which is a result of area division in each contribution ratio, according to a predetermined criterion that does not depend on the contribution ratio, and based on the area evaluation value, Select the region division result.

  In this way, by dynamically determining the feature ratio, the effect of appropriately adjusting the rate of contributing the color feature amount and the shape feature amount to the region division in accordance with the situation of the object and the background is further enhanced. Region division can be performed with higher accuracy.

  (8) In the above embodiment, the initial region is automatically set by the initial region setting unit 410. However, when the region dividing device of the present invention is applied to region dividing processing from a still image, the initial region setting unit 410 is used. It is preferable that the initial region is manually set by including a pointing device or the like.

  DESCRIPTION OF SYMBOLS 1 Image monitoring apparatus, 2 Imaging part, 3 Storage part, 4 Control part, 5 Output part, 30 Tracking information, 31 Person shape model, 32 Graph information, 40 Person tracking part, 41 Area division part, 42 Abnormal posture determination part, 100 surveillance image, 101 person, 110 virtual space, 111 floor surface, 112 person position, 113 person model, 114 camera, 115 imaging surface, 120 projected image, 121 initial region, 200 object seed, 201 background seed, 410 initial region Setting unit, 411 division cost calculation unit, 412 energy calculation unit, 413 region determination unit.

Claims (6)

In an image obtained by imaging a predetermined object together with a background, the image is divided into regions by assigning the elementary region to either the object region or the background region for each elementary region composed of at least one pixel. An area dividing device,
In the trial setting in which the attribution region to which the elementary region belongs and some types of evaluation target features for evaluating the elementary region among the plurality of types of image features are provisionally determined for each of the elementary regions, Evaluation value calculation for calculating the integrated evaluation value by summing the degree of likelihood representing the likelihood of the evaluation target feature of each area belonging to the attribution area temporarily determined in the trial setting in the image And
Setting the plurality of trial settings, comparing the integrated evaluation values in the trial settings calculated by the evaluation value calculation unit, and maximizing the likelihood of the entire image, the attribution region in the trial settings A region division determination unit for determining the region division result as
An area dividing apparatus comprising: The region division determination unit sets the evaluation target feature as the first type of image feature for the elementary region having the belonging region as the target region, and uses the belonging region as the background region. claims, characterized in that, to perform the comparison of the accumulated evaluation value by setting the trial settings to the image feature of the second type different from said first type to said evaluation feature for the region Item 2. The area dividing device according to Item 1 . The area dividing device uses a cost as the degree of attribution, and performs area division by a graph cut method that minimizes energy given by the integrated evaluation value,
The region division determination unit sequentially sets the plurality of types of image features as the first type of image features, and minimizes the energy by an α expansion method using the image features as labels α.
The area dividing device according to claim 2. The area dividing device uses a cost as the degree of attribution, and performs area division by a graph cut method that minimizes energy given by the integrated evaluation value,
The region division determination unit sequentially sets a combination of two of the plurality of types of image features as a set of the first and second types of image features, and the first type of image features is set. Minimizing the energy by an α-β exchange method with a label α and the second type of image feature as a label β;
The area dividing device according to claim 2.   5. The area dividing device according to claim 1, wherein the plurality of types of image features are colors and positions of the elementary areas. And a contribution setting unit that sets a plurality of contributions that contribute to the integrated evaluation value of the degree of attribution of each of the plurality of types of image features.
The evaluation value calculation unit calculates, for each contribution, the integrated evaluation value by weighting the contributions of the image features with the contributions and summing them up in the image,
The region division determination unit obtains the region division result for each contribution, evaluates the likelihood of the region division result at each contribution according to a uniform criterion independent of the contribution, and obtains a region division evaluation value. Calculating and determining the region division result having the highest region division evaluation value as a region division result unified for the plurality of contributions;
The area dividing device according to claim 1, wherein:

Images