TWI493477B - Method for detecting the status of a plurality of people and a computer-readable storing medium and visual monitoring device thereof - Google Patents

Description

Method for detecting multi-person status, computer readable recording medium, and view for detecting multi-person status Monitoring device

The present invention relates to video surveillance technology, and more particularly to a method for detecting a multi-person state, a computer-readable recording medium, and a video monitoring device for detecting a multi-person state.

In some places or places, it is not desirable to have more people gather. When two or more people are together, they may even think that they are abnormal and need to be detected. For example, an ATM set up in a local bank is usually open 24 hours a day, and there should be only one person in front of an ATM, that is, the customer is withdrawing money or other operating. If two or more people are listed in front of the ATM, there may be an abnormal situation. For example, the guest is held by the gangster to withdraw money from the ATM, or the customer is attacked, or the customer is robbed by the gangster.

At present, ATMs are equipped with cameras for photography, and even some people are watching the contents of these cameras. However, the manpower is limited, and it is impossible to view the contents of a large number of cameras in detail, so that even if the abnormal situation occurs, it cannot be immediately found. At present, although new video surveillance systems have been introduced, most of them are based on whether people wear masks, masks or fights, and do not identify the status of people with two or more people.

The present invention provides a method for detecting a multi-person state, which includes capturing one At least one first image of the target area; storing at least one of the first images; converting at least one of the first images into a second image of a polar coordinate image format; generating a set of multi-person states according to the second image of the polar coordinate image format The image feature is received and determined according to the image features of the plurality of people, whether the scene of the first image has a multi-person state, and an analysis result is generated. Preferably, the method further comprises receiving the analysis result and generating a corresponding one of the control signals.

In the above method of the present invention, the step of converting the at least one first image into the second image of the polar coordinate image format comprises: performing foreground detection on at least one of the first images to obtain a binarized foreground image; Converting the foreground image into a second image of the polar coordinate image format.

In the above method of the present invention, the step of generating a plurality of image features of the multi-person state according to the second image of the polar coordinate image format comprises: projecting the polar coordinate foreground object in the second image of the polar coordinate image format in the x direction Generating a projection image; and finding image features of the group of multi-person states from the projection image, wherein image characteristics of the group of multi-person states include θ ₁ , θ ₂ , θ ₃ , y1, y2, y3, θ ₁ and θ ₂ respectively, the two peak points of the projection map correspond to the angle on the x-axis, and θ ₃ is an angle between the two peak points corresponding to the x-axis, and y1, y2 and y3 are respectively The two peaks of the projected image and the valley points correspond to values on the y-axis. Preferably, the method further comprises: finding b1, b2, and b3 from the polar foreground objects in the second image of the polar coordinate image format according to the angles θ ₁ , θ _{2 ,} and θ ₃ , wherein the group of multi-person states The image features also include b1, b2, and b3, which are values at which the angles θ ₁ , θ _{2 ,} and θ ₃ correspond to the lowest point of the polar coordinate foreground.

In the above method of the present invention, the step of finding the image features of the group of multi-person states from the projection image comprises: searching from the left and right sides of the projection image along the x-axis to the intermediate point by the average value moving method. so as to find the two peaks; obtaining the two peaks respectively corresponding to the x-axis angles θ ₁ and θ _2; θ ₁ and the intermediate angle θ ₂ of the split search out along the x axis, so as to identify the inflection Obtaining the valley point corresponding to the angle θ ₃ on the x-axis; and obtaining the two peak points and the valley points respectively corresponding to the values y1, y2, and y3 on the y-axis.

The present invention also provides a computer readable recording medium, such as a disc or a non-volatile memory, that can be used to perform the above method of the present invention when the computer loads the program and executes it.

The present invention further provides a video monitoring device for detecting a multi-person state, comprising an image capturing module, a data storage module, and a shadow for implementing the above method of the present invention. Like a processing module, an image analysis module and a control module. If the image analysis module determines that the scene of the first image has a multi-person state, the control module generates a control signal to a warning device.

Compared with the prior art, the present invention can determine whether a scene of the image has a multi-person state according to an image captured by an image capturing module. When the present invention is applied to the automatic teller machine monitoring system, a remote control device is immediately notified when the multi-person state of the target area is recognized, to remind and urge the control personnel of the remote control device to handle the abnormal condition, thereby solving the past Due to lack of manpower, it is impossible to monitor the problem of each ATM one by one.

The above described features and advantages of the invention will be apparent from the following description.

1‧‧‧ places

10‧‧‧ service device

11‧‧‧

14‧‧‧ inner wall

170‧‧‧ Prospects

171‧‧‧ polar coordinates foreground

2‧‧‧Video monitoring device

20‧‧‧Image capture module

21‧‧‧Computer equipment

24a‧‧‧ Data Storage Module

24b‧‧‧Image Processing Module

24c‧‧‧Image Analysis Module

24d‧‧‧Control Module

201, 202, 203‧‧ images

204‧‧‧ foreground images

205‧‧‧ polar coordinates foreground image

206‧‧‧Projection

Steps 240~245‧‧‧Steps to determine the status of a multiplayer

Steps 242a-242b‧‧‧ steps required to convert a first image into a second image in a polar coordinate image format

Steps 243a-243c and 243b1~243b5‧‧‧ steps required to obtain a set of multi-person image features based on the second image of the polar coordinate image format

Steps 244a-244d‧‧‧ determine the multi-person state and the steps required to generate an analysis result based on the image characteristics of the multi-person state of the group

3‧‧‧Remote control equipment

4‧‧‧ wall

5‧‧‧Alarm

6‧‧‧Warning device

R1~R3‧‧‧ distance

A‧‧‧ perspective point

B‧‧·Scanning direction

P1, P2‧‧‧ peak

V‧‧‧ Valley Point

The first figure shows the case where the video monitoring device 2 of the present invention is installed in a place 1.

The second figure is a schematic diagram of an image captured by the image capturing module 20 of the video monitoring device 2 of the present invention. The video content is a situation when the location 1 is unmanned inside.

The third figure is a schematic diagram of an image captured by the image capturing module 20 of the video monitoring device 2 of the present invention, and the video content is a situation when the inside of the location 1 is present.

Fourth, a functional block diagram of a video monitoring device 2 of the present invention.

The fifth figure is a flow chart of a method for detecting a multi-person state of the present invention.

The sixth figure (a) is the first image captured by the image capturing module 20 of the present invention.

In a sixth diagram (b), the first image is processed as a schematic representation of the binarized foreground image 204.

In the seventh diagram, a schematic diagram of converting the binarized foreground image 204 into a second image in a polar coordinate image format is shown.

Figure 8 is a diagram showing the position of a perspective point of an image.

In the ninth figure, a projection map 206 projected in the x direction according to the second image of the polar coordinate image format is displayed.

The tenth figure shows a schematic diagram of three feature points b1 to b3, which are found from the second image of the polar coordinate image format.

The eleventh figure shows that the feature points b1 to b3 correspond to the distances R1 to R3 in the first image. intention.

The plan view (top view) of the first to sixth figures shows the detection of the article of the present invention. Referring first to the first figure, it is shown that the video monitoring device 2 for detecting a multi-person state of the present invention is installed in a place 1. This location 1 is essentially a cabin with a door 11 that is internally fitted with a service unit 10, such as an automated teller machine. The video monitoring device 2 includes an image capturing module 20 and a computer device 21 connected to the image capturing module 20. The computer device 21 is located near the location 1, such as behind a wall 4, or at a remote location and connected to the image capture module 20 via a network. The image capturing module 20 is a photographic device that is disposed above the site 1 to shoot a target area, for example, above the cabin, to capture at least one first image of the target area. When the target area of the location 1 is unoccupied, the scene of the first image captured by the image capturing module 20 is the same as the image 201 shown in the second figure, and only includes the service device 10, the door 11, and the location 1 ( Each inner wall 14 of the cabin) is adjacent to the floor 15. When there is one person in the target area of the site 1, the scene of the first image captured by the image capturing module 20 is the same as the image 202 shown in the third figure, including the background and the foreground, and the foreground is the person 16 . The scene of the first image captured by the image capturing module 20 is the same as the image 203 shown in FIG. 6( a ) when a state of two people is present in the target area of the location 1 . The environment arrangement of the site 1 is preferably unified and standardized, so as to improve the ease and accuracy of the video monitoring device 2 in image recognition.

Referring to the fourth figure, the computer device 21 has a data storage module 24a for storing at least one first image captured by the image capturing module 20; and an image processing module 24b for at least one of the first images. Converting into a second image of a polar coordinate image format (for example, image 205 shown in FIG. 7(b)), and generating a set of image features of a multi-person state; an image analysis module 24c receiving and according to the group of multiple people The image feature of the state determines whether the scene of the first image has a multi-person state, and generates an analysis result; a control module 24d receives the analysis result and generates a corresponding control signal; wherein the image analysis module 24c determines that the scene of the first image has a multi-person state, that is, the control module 24d generates a control signal to a warning device 6.

In an exemplary application, the video monitoring device 2 of the present invention is applied to an automatic teller machine monitoring system. In this system, please refer to the first and fourth figures. Each of the automatic teller machines is configured with a video monitoring device 2, Each video monitoring device 2 is connected to the network via a network The remote control device 3, the alert device 6 includes an alarm 5 disposed at the site 1 and a remote control device 3 located at a remote location. When the image analysis module 24c of the computer device 21 of any of the video monitoring devices 2 determines that the scene of the first image has an abnormal state of a multi-person state and generates the analysis result corresponding to the situation, the control module 24d receives and The result of the analysis immediately transmits a control signal to the alerting device 6, for example, transmitting a control signal to the remote control device 3 and/or the alarm 5. Upon receiving the control signal, the remote control device 3 sends a warning message, for example, a warning image is displayed by its display (not shown), and/or a warning is issued by its speaker (not shown), such as an alarm sound. , flashing light or voice, in order to inform the remote control device 3 control personnel, site 1 occurs in the abnormal state of multiple people, need to take corresponding measures, such as notification security personnel or police. Preferably, the computer device 21 further transmits the video captured by the image capturing module 20 to the remote control device 3 at the time when the abnormal condition occurs, so that the controller of the remote control device 3 can determine whether the video is viewed by watching the video. The aforementioned multi-person status does occur, avoiding misreporting due to erroneous warning messages. In addition, when the alarm 5 receives the control signal, it immediately issues a warning, such as an alarm sound, flashing light or voice, to intimidate the gangster or to inform the person near the place 1 that an abnormal situation requires emergency assistance.

Referring to the flowchart of FIG. 5A, the method for detecting a multi-person state of the present invention includes the following steps:

Step 240: Capture at least one first image of a target area. Specifically, the image capturing module 20 captures the first image. For convenience of explanation, it is assumed here that the first image is an image 203 as shown in the sixth diagram (a). This image 203 contains the foreground 17 caused by the juxtaposition of two people.

Step 241: Store at least one of the first images. Specifically, the first image is stored by the data storage module 24a.

Step 242: Convert at least one of the first images into a second image of a polar coordinate image format. Specifically, the above conversion is performed by the image processing module 24b. The process of the conversion is substantially as shown in FIG. BB. In step 242a, foreground detection is performed on at least one of the first images (images 203) to obtain a foreground image 204 as shown in FIG. 6(b). The black portion of the foreground image 204 is a background composed of a background pixel whose grayscale value is 255, and the white portion corresponds to the foreground 170 which is composed of a foreground pixel whose grayscale value is 0. The position of each pixel in the foreground image 204 is represented by a rectangular coordinate value (X, Y). In this embodiment, false foregrounds and shadows caused by false positives in the background or foreground, as well as light and shadow changes or other causes Have been filtered out. There are many technologies related to foreground detection, so I won't go into details.

Next, step 242b is executed. As shown in the seventh figure, the foreground image 204 is converted into a second image of the polar coordinate image format (which is a polar foreground image 205 with a perspective point A as the origin). In the seventh diagram (a), the arrow B indicates the scanning direction, the foreground image 204 is omitted from the background (not blacked out), only the foreground (i.e., the correct foreground object 170) is drawn, and the scanning line is indicated. The scan line has one every θ angle, and θ represents the angle of the scan. In this example, θ is from 0 to 360 degrees, so there are 360 scan lines, and R represents the perspective point A and the foreground. The distance between the boundaries of the image 204. However, for the sake of clarity, all the scan lines are not shown in the figure. In the seventh diagram (b), the horizontal axis is the θ angle and the 緃 axis is the distance R. The polar coordinate foreground image 205 includes a polar coordinate foreground 171 corresponding to the foreground object 170, and the position of each pixel in the polar coordinate image 204 is represented by a polar coordinate value (θ, R).

Among them, the perspective point A is the perspective point of the image (also known as the vanishing point), and its position is calculated in advance. As shown in the eighth figure, in a setting screen displayed by the computer device 21, along the two straight lines 151 (corner lines) of the image which are actually perpendicular to the ground (floor 15), the program calculates the extension line of the two straight lines 151. The position of the intersection, the position of this intersection is the perspective point A of the image 203. Preferably, the position of the perspective point A can also be calculated directly by the program using existing image technology.

Referring to FIG. 5A again, after completing the above step 242, proceeding to step 243: generating a plurality of image features of the multi-person state according to the second image (polar coordinate foreground image 205) converted to the polar coordinate image format. The image features are still generated by the image processing module 24b. The process of generating is substantially as shown in FIG. C. First, in step 243a, the polar foreground object 171 in the second image (polar coordinate foreground image 205) of the polar coordinate image format is projected in the x direction to generate A projection 206 is as shown in the ninth diagram. Next, please refer to the fifth C diagram. From step 243b, image features of the group of multi-person states are found from the projection map 206: θ ₁ , θ ₂ , θ ₃ , y1, y2, and y3. θ ₁ and θ ₂ are respectively the two peak points P1 and P2 of the projection map corresponding to the angle on the x-axis, and θ ₃ is a valley point V between the two peak points P1 and P2 corresponding to the x-axis. The angles y1, y2, and y3 are the two peak points P1, P2 of the projection map and the valley point V corresponding to the value on the y-axis, respectively. Furthermore, preferably, further comprising performing step 243c to find other image features from the polar foreground object 171 in the second image (polar coordinate foreground image 205) of the polar coordinate image format according to the angles θ ₁ , θ ₂ and θ ₃ B1, b2 and b3. Wherein, as shown in the tenth figure (the part in the figure about the background is indicated by oblique lines), b1, b2 and b3 are the values of the angles θ ₁ , θ ₂ and θ _{3 respectively} corresponding to the lowest point of the polar coordinate foreground object ( That is, the minimum y value). As shown in the eleventh figure, the foregoing b1, b2, and b3 correspond to the distances R1, R2, and R3 corresponding to the point A to the sole point in the representative image 203.

The process of finding the image features θ ₁ , θ ₂ , θ ₃ , y1 , y2 , and y3 of the group of multi-person states from the projection 26 is as shown in FIG. 5D. First, please refer to the ninth diagram, such as the steps. 243b1, searching by the mean moving method (Mean Shift) from the left and right sides of the projected image 206 along the x-axis toward the intermediate point, thereby finding two maximum values (peaks), the points indicated by the two maximum values This is the two peak points P1, P2, and then, as in step 243b2, the two peak points P1, P2 are respectively obtained to correspond to the angles θ ₁ and θ ₂ on the x-axis. Then, as in step 243b3, searching from the middle of the angles θ ₁ and θ ₂ along the x-axis to find a minimum value (valley), the minimum point is the above-mentioned valley point V, and then In step 243b4, the valley point V is obtained to correspond to the angle θ ₃ on the x-axis. Finally, in step 243b5, the two peak points P1, P2 and the valley point V are respectively obtained corresponding to the values y1, y2 and y3 on the y-axis.

After the feature extraction is completed by the above step 243, a total of nine features such as θ ₁ , θ ₂ , θ ₃ , y1, y2, y3, b1, b2, and b3 are extracted, and then, refer to the fifth A diagram, such as Step 244, receiving and determining, according to the image features of the plurality of people, whether the scene of the first image has a multi-person state, and generating an analysis result. Specifically, the image analysis module 24c performs the foregoing determination and generates the analysis result. The process is as shown in FIG. 5E. First, as shown in step 244a, the image features of the group of multi-person states are input into the pre-trained A classifier, such as a support vector machine (SVM classifier). Next, as step 244b, it is determined whether the continuous classification is successful n times. In this embodiment, n may be 3 times. When the determination result is "Yes", it indicates that the scene of the first image has been determined to have a multi-person state. Therefore, in step 244c, the analysis result corresponding to the multi-person state of the scene indicating the first image is output, and vice versa. If the result is "No", it indicates that the scene of the first image has not yet been in a multi-person state. Therefore, in step 244d, an analysis result corresponding to the scene indicating that the first image has not yet appeared in a multi-person state is output.

For the signal signal, please refer to FIG. 5A. After step 244 is completed, step 245 may be further performed, that is, the analysis result is received, and a corresponding control signal is generated. Specifically, the control module 24d receives the foregoing analysis result and generates the control signal.

The above method of the present invention can be implemented by a computer program, and the program is stored. The computer can read a recording medium, such as a disc or non-volatile memory. When the computer (such as the computer device 21 described above) loads the program and executes it, the method of the present invention can be completed to achieve a detection target. Whether the area has a multi-person status.

As can be seen from the above description, the video monitoring device 2 of the present invention can implement the present invention. The method for detecting a multi-person state is to determine whether the scene of the first image has a multi-person target according to the first image captured by the image capturing module 20. In the exemplary embodiment of the present invention, which is applied to the automatic teller machine monitoring system, the video monitoring device 2 immediately notifies the remote control device 3 when it is determined that the target area has a multi-person status, to remind and urge the remote control device 3 to control The personnel handle this abnormal situation, thus solving the problem that in the past, due to lack of manpower, it was impossible to monitor each ATM one by one.

Although the present invention has been disclosed in the above embodiments, it is not intended to limit the present invention, and any one of ordinary skill in the art can make some changes and refinements without departing from the spirit and scope of the present invention. The scope of the invention is defined by the scope of the appended claims.

2‧‧‧Video monitoring device

20‧‧‧Image capture module

21‧‧‧Computer equipment

24a‧‧‧ Data Storage Module

24b‧‧‧Image Processing Module

24c‧‧‧Image Analysis Module

24d‧‧‧Control Module

3‧‧‧Remote control equipment

5‧‧‧Alarm

6‧‧‧Warning device

Claims (13)

A video monitoring device for detecting a multi-person state includes: an image capturing module for capturing at least one first image of a target area; and a data storage module for storing at least one of the first images; The processing module converts the second image into a polar coordinate image format for at least one of the first images, and generates a plurality of image features of the multi-person state; and the image analysis module receives and according to the image characteristics of the plurality of human states Determining whether the scene of the first image has a multi-person state and generating an analysis result; a control module receiving the analysis result and generating a corresponding one of the control signals; wherein the image analysis module determines the first image The scene has a multi-person state, and the control module generates a control signal to a warning device. The video monitoring module for detecting a multi-person state according to the first aspect of the patent application, wherein the image analysis module has a pre-trained classifier, and the classifier receives and according to the image of the group of multi-person states. The feature determines whether the scene of the first image has a state in which a plurality of people are juxtaposed, and generates the analysis result. The video processing module is configured to perform foreground detection on at least one of the first images to obtain a binarized foreground image, and Converting the foreground image into a second image of the polar coordinate image format, and performing a x-direction projection on the polar coordinate foreground object in the second image of the polar coordinate image format to generate a projection image, and then finding the group from the projection image Image characteristics of a multi-person state; The image features of the multi-person state of the group include θ1, θ2, θ3, y1, y2, and y3; θ1 and θ2 are respectively the angles of the two peak points of the projection map corresponding to the x-axis, and θ3 is located at the second A valley point between the peak points corresponds to an angle on the x-axis; y1, y2, and y3 are the two peak points of the projection map and the valley points correspond to values on the y-axis, respectively. The video monitoring device for detecting a multi-person state according to claim 3, wherein the image processing module further selects a polar coordinate foreground from the second image of the polar coordinate image format according to the angles θ1, θ2, and θ3. The image features b1, b2, and b3 are found, wherein the image features of the multi-person state include b1, b2, and b3, respectively, which are angles θ1, θ2, and θ3 corresponding to the lowest point of the polar coordinate foreground object. The video monitoring device for detecting a multi-person state according to the third aspect of the patent application, wherein the image processing module is moved from the left and right sides of the projection image to the middle point along the x-axis by an average value moving method. Direction search, to find the two peaks, and search from the middle of the angles θ1 and θ2 along the x-axis to find the valley. A method for detecting a multi-person state, comprising: capturing at least one first image of a target area; storing at least one of the first images; and converting at least one of the first images into a second image of a polar coordinate image format Generating a set of multi-person image features according to the second image of the polar coordinate image format; Receiving and determining whether the scene of the first image has a multi-person state according to the image feature of the group of multi-person states, and generating an analysis result. The method for detecting a multi-person state as described in claim 6 includes receiving the analysis result and generating a corresponding one of the control signals. The method for detecting a multi-person state according to claim 6, wherein the converting the at least one first image into the second image of the polar coordinate image format comprises: at least one of the first images Performing foreground detection to obtain a binarized foreground image; and converting the foreground image into a second image of the polar coordinate image format. The method for detecting a multi-person state according to the second aspect of the patent application, wherein the step of generating a plurality of image features of the multi-person state according to the second image of the polar coordinate image format comprises: the polar coordinate image format The polar coordinate foreground object in the second image is projected in the x direction to generate a projection image; and the image features of the group of multi-person states are found from the projection image; wherein the image features of the group of multi-person states include θ ₁ , θ ₂ , θ ₃ , y1, y2 and y3; θ ₁ and θ ₂ are respectively the angles of the two peak points of the projection map corresponding to the x-axis, and θ ₃ is a valley point corresponding between the two peak points The angles to the x-axis; y1, y2, and y3 are the two peaks of the projection map and the valley points corresponding to the values on the y-axis, respectively. The method for detecting a multi-person state as described in claim 9 further includes: finding out from the polar coordinate foreground in the second image of the polar coordinate image format according to the angles θ ₁ , θ ₂ and θ ₃ B1, b2, and b3; wherein the image features of the multi-person state of the group further include b1, b2, and b3, which are values at which the angles θ ₁ , θ _{2 ,} and θ ₃ correspond to the lowest point of the polar coordinate foreground object, respectively. The method for detecting a multi-person state according to claim 9, wherein the step of finding an image feature of the group of multi-person states from the projection image comprises: moving from the projection image by an average value moving method The left and right sides search along the x-axis toward the middle point to find the two peak points; the two peak points are respectively corresponding to the angles θ ₁ and θ ₂ on the x-axis; from the middle of the angles θ ₁ and θ ₂ Searching along the x-axis to find the valley point; obtaining the valley point corresponding to the angle θ ₃ on the x-axis; and obtaining the two peak points and the valley points respectively corresponding to the values y1 and y2 on the y-axis And y3. The method for detecting a multi-person state as described in claim 6 includes pre-training a classifier, and using the trained classifier to receive and determine the image characteristics according to the group of multi-person states. Whether the scene of the first image has a multi-person state and produces the analysis result. A computer that can store a recording medium, and when the computer loads the program and executes it, the method of any one of claims 6 to 12 can be completed.