WO2017168684A1

WO2017168684A1 - Monitoring system, monitoring method, program, and computer storage medium

Info

Publication number: WO2017168684A1
Application number: PCT/JP2016/060683
Authority: WO
Inventors: 三郎山内
Original assignee: 三郎山内
Priority date: 2016-03-31
Filing date: 2016-03-31
Publication date: 2017-10-05

Abstract

This monitoring system for monitoring a predetermined region has: an imaging unit for imaging the predetermined region from above; a range finding unit for measuring the distance from the imaging unit to a measurement subject in the predetermined region; a model generation unit for generating a mesh model of the background of the predetermined region by demarking the background in the horizontal direction and the height direction with a mesh having a predetermined size, on the basis of a distance from the imaging unit to the background measured by the range finding unit; and a monitoring unit for monitoring the predetermined region by extracting a monitored subject that is in motion on the basis of image data taken in the imaging unit, applying the mesh model to the background of the predetermined region of the image data, and generating new monitoring image data obtained by superimposing the monitored subject on the mesh model.

Description

Monitoring system, monitoring method, program, and computer storage medium

The present invention relates to a monitoring system for monitoring a predetermined area, a monitoring method using the monitoring system, a program, and a computer storage medium.

Conventionally, surveillance systems using surveillance cameras have been used for crime prevention and disaster prevention purposes. In this type of monitoring system, for example, a surveillance camera is installed in a predetermined surveillance area or surveillance object such as a store, commercial facility, or public facility, and an abnormal state or dangerous event is detected based on an image captured by the surveillance camera. Monitor for occurrence of

Generally, since image data captured by a surveillance camera is two-dimensional, height information and depth information cannot be obtained. Then, for example, even if image data that captures two people with different sizes is obtained, the height of the two people is actually different, or one person is near the surveillance camera and the other person The computer cannot determine from the image data alone whether the image appears to be different because it is on the far side. That is, it is difficult to accurately grasp a person from two-dimensional image data, and therefore it is difficult to specify the person's action.

Therefore, Patent Document 1 proposes adding depth information to image data. Specifically, the imaging system described in Patent Document 1 includes an irradiation source that irradiates a laser, an imaging unit that images a subject, and a measurement unit that measures distance data from the irradiation source to the subject. Then, the imaging system associates the image data of the subject imaged by the imaging unit with the distance data measured by the measurement unit, generates composite data in which the distance data is superimposed on the image data, and further generates a three-dimensional image of the subject. Image data is generated.

Japanese Unexamined Patent Publication No. 2013-207415

However, when the imaging system described in Patent Document 1 is used as a monitoring system, in order to capture a subject, that is, a monitoring target in three dimensions, it is necessary to always measure the distance to the subject during the monitoring period. In addition, the above-described combined data of the image data and distance data and the three-dimensional image data of the subject must be generated. In such a case, the processing performed during monitoring becomes very complicated, and the amount of data to be handled becomes enormous, which places a heavy load on the computer that performs the processing. Therefore, there is room for improvement as a monitoring system.

The present invention has been made in view of such a point, and an object thereof is to monitor a predetermined area appropriately and simply.

In order to achieve the above object, the present invention provides a monitoring system for monitoring a predetermined area, which is fixedly provided at a predetermined position, and that captures the predetermined area from above, and the imaging in the predetermined area. A distance measuring unit that measures a distance from a measurement unit to a measurement target, a bottom surface of the predetermined area, and a background of the predetermined area that is permanently installed in the predetermined area during a monitoring period. Based on the distance from the imaging unit measured by the unit to the background, the background is partitioned in a horizontal direction and a height direction with a mesh of a predetermined size, and a model generation unit that generates a mesh model of the background, Based on the image data picked up by the image pickup unit, a moving monitoring target is extracted, the mesh model is applied to the background of the predetermined area of the image data, and the mesh model is applied. The monitored to generate a new monitoring image data synthesized it is characterized by having a monitoring unit that monitors the predetermined region.

In the monitoring system of the present invention, after the distance from the imaging unit to the background of the predetermined area is measured by the distance measuring unit, the mesh generation of the background of the predetermined area is generated by the model generation unit, and further the predetermined area captured by the imaging unit A mesh model is applied to the background of and a predetermined area is monitored by the monitoring unit.

Here, the mesh model of the present invention includes distance information from the imaging unit measured by the ranging unit to each mesh, that is, position information (three-dimensional information) in the horizontal direction and the height direction of each mesh. Contains. If it does so, the three-dimensional information of the said monitoring object can be acquired by grasping | ascertaining in which mesh the monitoring object in a predetermined area | region is located. In other words, once the background mesh model of the predetermined area is generated, after that, it is not necessary to always measure the distance of the monitoring target as in Patent Document 1 described above, and only by imaging the predetermined area with the imaging unit, The predetermined area can be monitored. Therefore, according to the present invention, the predetermined area can be monitored appropriately and simply.

The model generation unit may superimpose the background image in the image data captured by the imaging unit on the mesh model in the monitoring image data.

The monitoring system may further include a database in which information on objects in the predetermined area is stored, and the monitoring unit may monitor the predetermined area by specifying the object based on the database.

The model generation unit may three-dimensionally divide a space surrounded by the bottom surface of the predetermined region and the permanent object in the mesh model with the mesh of the predetermined size.

The monitoring system further includes a person setting unit that divides a person who has entered the predetermined area into a plurality of areas, and the monitoring unit is based on the rate of change of the person in each area set by the person setting unit. Then, the behavior of the person may be monitored.

The monitoring system further includes an audio detection unit that detects audio generated in the predetermined area, and the monitoring unit sets an imaging direction of the imaging unit in a direction of audio detected by the audio detection unit, You may monitor the area | region where the said sound generate | occur | produced.

Another aspect of the present invention is a monitoring method for imaging and monitoring a predetermined area with an imaging unit, and includes a bottom surface of the predetermined area and a permanent object permanently installed in the predetermined area during a monitoring period. With respect to the background of the predetermined area, a distance measuring step for measuring the distance from the imaging unit to the background with the distance measuring unit, and the background in the horizontal direction with a mesh of a predetermined size based on the distance measured in the distance measuring step And a model generation step of generating a mesh model of the background by partitioning in the height direction, an imaging step of imaging the predetermined area by the imaging unit, and a motion based on the image data captured in the imaging step Extract a certain monitoring target, apply the mesh model to the background of the predetermined area of the image data, and generate new monitoring image data by combining the monitoring target with the mesh model It is characterized by having a monitoring step of monitoring the predetermined region.

In the model generation step, the background image in the image data captured by the imaging unit may be superimposed on the mesh model in the monitoring image data.

The information on the object in the predetermined area may be stored in a database, and in the monitoring step, the object may be specified based on the database to monitor the predetermined area.

In the model generation step, in the mesh model, a space surrounded by the bottom surface of the predetermined region and the permanent object may be partitioned three-dimensionally with the mesh of the predetermined size.

In the monitoring step, a person who has entered the predetermined area may be divided into a plurality of areas, and the behavior of the person may be monitored based on the rate of change of the person in each of the divided areas.

In the imaging step, when the sound generated in the predetermined area is detected by the sound detection unit, the imaging direction of the imaging unit is set in the direction of the sound, the region where the sound is generated is imaged, and the monitoring step The area where the sound is generated may be monitored.

According to another aspect of the present invention, there is provided a program that operates on a computer for causing the computer that controls the monitoring system to function so that the monitoring method is executed by the monitoring system.

According to another aspect of the present invention, a readable computer storage medium storing the program is provided.

According to the present invention, the predetermined area can be monitored appropriately and simply.

It is explanatory drawing which shows the outline of a structure of the monitoring system concerning this Embodiment. It is explanatory drawing which shows the outline of a structure of the monitoring system concerning this Embodiment. It is a flowchart which shows the monitoring method concerning this Embodiment. It is a perspective view which shows the inside of a shop. It is explanatory drawing which shows a mode that the distance between the monitoring camera and the background of a monitoring area | region is measured. It is explanatory drawing which shows a mode that the distance between the monitoring camera and the background of a monitoring area | region is measured. It is explanatory drawing which shows a mesh model. It is explanatory drawing which shows monitoring image data. It is explanatory drawing which shows the mesh model concerning other embodiment. It is explanatory drawing which shows the outline of a structure of the monitoring system concerning other embodiment. It is explanatory drawing which shows a mode that a person is divided into a some area | region. It is explanatory drawing which shows the outline of a structure of the monitoring system concerning other embodiment. It is a flowchart which shows the monitoring method concerning other embodiment.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. In addition, in this specification and drawing, about the component which has the substantially same function structure, duplication description is abbreviate | omitted by attaching | subjecting the same code | symbol.

<1. Configuration of the monitoring system>
1 and 2 show an outline of the configuration of the monitoring system 1 according to the present embodiment. In the present embodiment, a case will be described in which the monitoring system 1 is used to monitor the inside of a store 10 such as a supermarket or a convenience store.

As shown in FIG. 1, the monitoring system 1 includes an imaging device 20 that is fixedly installed on the ceiling surface of the store 10, and a monitoring device 30 that is connected to the imaging device 20 via a network (not shown). . The network is not particularly limited as long as communication between the imaging device 20 and the monitoring device 30 can be performed. For example, the network is configured by the Internet, a wired LAN, a wireless LAN, or the like.

The imaging device 20 images the monitoring area A in the store 10 from above and measures the distance to the measurement target of the monitoring area A (the background of the monitoring area A described later). The monitoring area A coincides with the imaging area of the imaging device 20. Further, the monitoring device 30 generates a three-dimensional mesh model for the background of the monitoring region A, and applies the mesh model to the background of the monitoring region A captured by the imaging device 20 to monitor the monitoring region A. . The configurations and operations of the imaging device 20 and the monitoring device 30 will be described in detail below.

<2. Configuration of Imaging Device>
As shown in FIG. 2, the imaging device 20 has a configuration in which a transparent or semi-transparent, substantially hemispherical dome cover 22 is provided in a lower part of a housing 21. Inside the dome cover 22, a distance measuring sensor 23 as a distance measuring unit, a monitoring camera 24 as an imaging unit, and a support member 25 for hanging and supporting the monitoring camera 24 are provided. Further, inside the housing 21, a drive mechanism 26 that controls the rotation operation of the monitoring camera 24 via the support member 25, and a communication unit for transmitting data acquired by the imaging device 20 to the monitoring device 30. 27 are provided. Note that the shape of the imaging device 20 is not limited to this, and can be arbitrarily designed.

The distance measuring sensor 23 includes, for example, an irradiation source 23a for irradiating infrared rays and a light receiving element 23b for receiving reflected waves of infrared rays. For example, an LED is used as the irradiation source 23a. For example, PSD or CMOS is used for the light receiving element 23b. On the monitoring area A side of the irradiation source 23a and the light receiving element 23b, lenses (not shown) for focusing the light are provided. A plurality of irradiation sources 23a and light receiving elements 23b may be provided.

The distance measurement sensor 23 measures the distance to the measurement target by irradiating the measurement target (monitoring area A) with infrared rays from the irradiation source 23a and receiving the reflected wave of the infrared rays reflected by the measurement target with the light receiving element 23b. Is done. As a method of measuring the distance to the measurement object based on the reflected wave of infrared rays, for example, the time and phase difference from when the reflected wave is irradiated until the reflected wave returns, a light receiving element that receives the reflected wave of infrared rays There are methods for calculating from the above position, the intensity of the reflected wave of infrared rays, etc., and those skilled in the art can arbitrarily select from known methods. Then, the distance data measured by the distance measuring sensor 23 is output to the communication unit 27.

The distance measuring sensor 23 is provided in the vicinity of the surveillance camera 24 and fixed immediately below it. Therefore, the distance measured by the distance measuring sensor 23 can be regarded as the distance from the monitoring camera 24 to the measurement target.

Note that although the distance measuring sensor 23 of the present embodiment uses infrared rays to measure the distance to the measurement object, it is not limited to this, and can be arbitrarily selected, for example, an ultrasonic wave or a laser.

As the monitoring camera 24, an arbitrary camera such as a CCD camera or a CMOS camera is used. The surveillance camera 24 is supported by being suspended from a support member 25. Further, the monitoring camera 24 can be rotated in the horizontal direction (X-axis direction and Y-axis direction, pan direction) and the height direction (Z-axis direction, tilt direction) by the drive mechanism 26, and can perform a zoom operation. It is configured. As the drive mechanism 26, for example, a stepping motor or a direct drive motor is used. The monitoring camera 24 can capture an image of the monitoring area A through the dome cover 22 serving as an imaging window and acquire an image of the monitoring area A. Further, the image data captured by the monitoring camera 24 is output to the communication unit 27.

The communication unit 27 is a communication interface that mediates communication with the network, and performs data communication with an input unit 31 of the monitoring device 30 described later. Specifically, the communication unit 27 outputs the distance data measured by the distance measuring sensor 23 and the image data captured by the monitoring camera 24 to the monitoring device 30.

<3. Configuration of monitoring device>
The monitoring device 30 is configured by, for example, a computer, and is configured by, for example, a central processing unit such as a circuit (hardware) or a CPU, and a program (software) for causing them to function. The monitoring device 30 includes an input unit 31, a model generation unit 32, a monitoring unit 33, an output unit 34, a control unit 35, and a storage unit 36.

The input unit 31 is a communication interface that mediates communication with the network, and performs data communication with the communication unit 27 of the imaging device 20. Specifically, the input unit 31 receives the distance data measured by the distance measuring sensor 23 and the image data captured by the monitoring camera 24.

The model generation unit 32 generates a background mesh model of the monitoring area A based on the distance data of the input unit 31. Further, the monitoring unit 33 monitors the monitoring area A based on the image data and the mesh model. Specific operations of the model generation unit 32 and the monitoring unit 33 will be described later.

The output unit 34 outputs the monitoring result of the monitoring unit 33. The method for outputting the monitoring result is not particularly limited. For example, any method can be selected such as displaying on a display or issuing an alarm when an abnormal state or dangerous event occurs.

The control unit 35 controls each operation in the imaging device 20. That is, the control unit 35 controls, for example, the timing and position at which the distance measuring sensor 23 measures distance, and the timing and position at which the monitoring camera 24 captures an image.

The storage unit 36 stores a program for monitoring the monitoring area A by the monitoring system 1. The program may be stored in the storage unit 36 as described above, or may be a computer-readable hard disk (HD), flexible disk (FD), compact disk (CD), magnet optical desk (MO), It may be stored in a computer-readable storage medium such as various memories. Further, the program can be stored in the storage medium or the like by downloading it via a communication line network such as the Internet.

<4. Operation of the monitoring system>
Next, a monitoring method of the monitoring area A performed by the monitoring system 1 configured as described above will be described. FIG. 3 is a flowchart showing an example of main steps of the monitoring method.

In the following, a case where the inside of the store 10 shown in FIG. 4 is monitored will be described as an example. Here, the monitoring area A in the store 10 that does not move during the monitoring period is defined as the background of the monitoring area A. In the present embodiment, the floor surface 11 of the store 10 and a plurality of fixtures 12 as permanent objects permanently installed in the store 10 constitute the background of the monitoring area A. Note that the fixture 12 may be moved intentionally by changing the layout in the store 10 or the like, but it is not a monitoring period during movement. In this case, it is assumed that the monitoring period starts after the fixture 12 is installed at a new position. Therefore, in this embodiment, the fixture 12 is treated as a permanent object.

First, the distance between the monitoring camera 24 and the background of the monitoring area A is measured using the distance measuring sensor 23 of the imaging device 20 (step S1 in FIG. 3). Specifically, as shown in FIGS. 5 and 6, distance measurement is started from directly below the imaging device 20 (dotted line portion in FIG. 5), and the distance sensor 23 is rotated and the measurement direction of the imaging device 20 is changed. The distance is measured by moving from directly below to the outside (solid line portion in FIG. 5). Then, the distance between the monitoring camera 24 and the background of the monitoring area A is measured over the entire monitoring area A. The distance data measured by the distance measuring sensor 23 is output to the model generation unit 32 of the monitoring device 30 via the communication unit 27 and the input unit 31.

The model generation unit 32 generates a background mesh model of the monitoring area A based on the distance data measured by the distance measuring sensor 23 (step S2 in FIG. 3). Specifically, as shown in FIG. 5, the meshes M are stacked from directly below the imaging device 20 toward the outside. The size of the mesh M is arbitrarily set, and is set to 50 cm × 50 cm, for example. The position in the horizontal direction (X-axis direction and Y-axis direction) of each mesh M can be calculated by the number of meshes M stacked. Further, the position of each mesh M in the height direction (Z-axis direction) can be calculated from the distance data measured by the distance measuring sensor 23.

When the meshes M having the three-dimensional position information in the horizontal direction and the height direction are stacked in this way, the tertiary in which the floor 11 and the furniture 12 are reflected on the background of the monitoring area A as shown in FIG. An original mesh model D is generated. In other words, in step S <b> 2, based on the distance data measured by the distance measuring sensor 23, the background of the monitoring area A is partitioned in a three-dimensional manner by the plurality of meshes M, and the mesh model D is generated. In FIG. 7, for ease of explanation, the mesh model D of the two furniture 12 and the floor surface 11 between them is shown, but in reality, the mesh model D is generated for all the backgrounds of the monitoring area A. The In FIG. 7, the floor 11 is hatched for easy understanding.

Steps S1 and S2 so far are preparations for monitoring the monitoring area A, and actual monitoring starts here. That is, the monitoring area A is imaged using the monitoring camera 24 of the imaging device 20 (step S3 in FIG. 3). Image data captured by the monitoring camera 24 is output to the monitoring unit 33 of the monitoring device 30 via the communication unit 27 and the input unit 31.

The monitoring unit 33 analyzes the image data and extracts a person in the monitoring area A based on, for example, a background difference. This background difference is a known technique, and a moving monitoring target is extracted by taking a difference between the image data acquired by the monitoring camera 24 and the background image of the monitoring area A acquired in advance. . In this embodiment, since the store 10 is monitored, the monitoring target is described as a person, but the monitoring target is not limited to this.

On the other hand, the monitoring unit 33 applies the mesh model D generated by the model generation unit 32 to the background of the monitoring region A in the image data. That is, as shown in FIG. 8, the person P is synthesized with the mesh model D to generate new monitoring image data. Based on the monitoring image data, the monitoring area A is monitored (step S4 in FIG. 3). Note that the monitoring result of the monitoring unit 33 is output to the output unit 34.

Here, the advantage of using monitoring image data in this way will be described. For example, as shown in FIG. 8, it is assumed that two persons P1 and P2 having different sizes are displayed in the monitoring area A in the monitoring image data. Hereinafter, the side closer to the monitoring camera 24 (Y-axis negative direction side) is referred to as “front”, and the side farther from the monitoring camera 24 (Y-axis positive direction side) is referred to as “back”. In the case of conventional image data, whether the two persons are actually different in height, or whether one person P1 is in front and the other person P2 is in the back, so that it appears to be different in size, a computer Can not judge.

On the other hand, in the monitoring image data of the present embodiment, each mesh M of the mesh model D has three-dimensional information. Then, for example, by recognizing the feet of the persons P1 and P2, it is possible to grasp which mesh M of the floor surface 11 the persons P1 and P2 are located on. Then, since the lengths in the height direction of the persons P1 and P2 on the monitoring image data are known, the heights (heights) of the persons P1 and P2 can be calculated. That is, by grasping which mesh M each person P1, P2 is located on, it is possible to grasp the actual height of the persons P1, P2. Thus, the persons P1 and P2 can be grasped three-dimensionally from the monitoring image data. Even when the monitoring target is not a person, the shape of the monitoring target can be grasped three-dimensionally.

In addition, if the persons P1 and P2 can be grasped three-dimensionally in this way, for example, it can be seen from the monitoring image data that even if the movement of the person P1 in front is large, the movement is not so large. On the other hand, even if the movement of the person P2 in the back is small, it can be seen that the movement is actually large. Therefore, it is possible to accurately grasp the movements of the persons P1 and P2. Further, for example, even if the persons P1 and P2 overlap on the monitoring image data, it can be recognized that the persons P1 and P2 are not actually colliding.

Further, once the background mesh model D of the monitoring area A is generated in steps S1 and S2, the monitoring area A can be monitored by repeating steps S3 to S4. That is, in the method described in Patent Document 1, it is necessary to always measure the distance between the persons P1 and P2 when monitoring the persons P1 and P2. In this embodiment, the distance to the persons P1 and P2 is measured. do not have to. Therefore, the processing performed during monitoring is very simple and the amount of data handled is small, so that the load on the computer can be greatly reduced.

Moreover, even if the layout of the store 10 is changed or the arrangement of the imaging device 20 is changed and the background of the monitoring area A captured by the imaging device 20 changes, the steps S1 and S2 are performed. The mesh model D can be automatically generated. Therefore, the monitoring system 1 can flexibly cope with such a change in the background of the monitoring area A.

As described above, according to this embodiment, the monitoring area A can be monitored appropriately and simply.

<5. Other embodiments>
Next, another embodiment of the present invention will be described. In the following description, the description overlapping with the above embodiment is omitted.

<5-1. Other embodiments>
When generating the mesh model D in step S2 of the above embodiment, the image data of the background of the monitoring area A captured in advance by the monitoring camera 24 is superimposed on the mesh model D composed of a plurality of meshes M. Also good. The mesh model D includes a plurality of meshes M and background image data of the monitoring area A. In such a case, the monitoring area A can be monitored more appropriately in step S4.

<5-2. Other embodiments>
In the above embodiment, the storage unit 36 of the monitoring device 30 may store a database in which information on objects in the monitoring area A is stored. The object in the monitoring area A is an object assumed to exist in the monitoring area A. When the store 10 is monitored as in the present embodiment, as such an object, information such as typical product shelves, desks, chairs, and the like, which are fixtures 12, is stored in the database. In addition to such fixtures 12, information such as representative products and carts is also stored in the database.

In such a case, when the monitoring area A is monitored in step S4, the shape of the object in the monitoring area A can be grasped by using the mesh model D. Therefore, it is possible to identify what the object is by referring to the information in the database. can do. Thereby, it is possible to grasp even the specific action of the person P. For example, it can be grasped if the person P is going to enter the entry prohibition area, and can be grasped if the person P is going to shoplift a certain product from a certain product shelf. Thus, according to the present embodiment, the movement of the monitoring target can be specifically grasped, and the monitoring area A can be monitored more appropriately.

<5-3. Other embodiments>
When the mesh model D is generated in step S2 of the above embodiment, a space surrounded by the floor 11 and the fixture 12 in the monitoring area A (hereinafter referred to as monitoring space) as shown in FIG. You may partition in three dimensions. The mesh N formed in the monitoring space has the same size as the mesh M on the floor 11 and the fixture 12. And the mesh N of the monitoring space is formed in three dimensions so as to complement the space between the floor surface 11 and the mesh M of the fixture 12. In FIG. 9, the mesh M on the floor surface 11 and the furniture 12 is illustrated by dotted lines for easy understanding.

In such a case, the mesh N in the monitoring space is provided with three-dimensional position information in the horizontal direction and the height direction, similar to the mesh M of the floor surface 11 and the fixture 12. For this reason, when monitoring the monitoring area A in step S4, the person P can be more accurately and easily grasped three-dimensionally.

<5-4. Other embodiments>
In the above embodiment, as shown in FIG. 10, the monitoring device 30 may further include a person setting unit 40 that partitions the person P entering the monitoring area A into a plurality of areas. The person setting unit 40 divides the person P into a plurality of regions 50 to 52 in the height direction as shown in FIG. The upper area 50 is formed at the position of the head of the person P, the intermediate area 51 is formed at the position of the torso of the person P, and the lower area 52 is formed at the position of the legs of the person P. Specifically, the person setting unit 40 first recognizes the foot of the person P and recognizes the height of the person P from the mesh model D. Then, the rough position of the head and torso of the person P can be grasped.

In such a case, when monitoring the monitoring area A in step S4, the person P is monitored based on the rate of change of the person P in each of the areas 50 to 52. Then, for example, when the rate of change in the upper region 50 is large, it can be seen that the person P performs an operation such as swinging his / her head left and right and scolding the surroundings. In addition, for example, when the rate of change in the intermediate region 51 is large, it can be seen that the person P is performing a motion such as moving the body sideways and flirting. In addition, for example, when the rate of change in the lower region 52 is large, it can be seen that the person P is performing an operation such as waving. Thus, when monitoring the monitoring area A, the specific operation of the person P can be grasped.

<5-5. Other embodiments>
In the above embodiment, as shown in FIG. 12, the imaging device 20 further includes a sound collector 60 that collects sound generated in the monitoring area A, and the monitoring device 30 is collected by the sound collector 60. You may further have the audio | voice detection part 61 which identifies the audio | voice. The sound collector 60 is a microphone, and a plurality, for example, three are provided inside the dome cover 22.

FIG. 13 is a flowchart showing an example of main steps of a monitoring method using voice detection. In FIG. 13, steps S1 to S4 are the same steps as steps S1 to S4 shown in FIG. 3 of the above embodiment.

In parallel with the generation of the mesh model D in steps S1 and S2, specific sounds to be detected in the monitoring area A are collected and stored in the storage unit 36 of the monitoring device 30 (step T1 in FIG. 13). . The specific sound is a sound that is generated when an abnormal state or a dangerous event occurs in the monitoring area A, and is, for example, a screaming sound of a person P or a sound that breaks an object.

These steps S1 to S2 and T1 are preparations for monitoring the monitoring area A. During monitoring, for example, audio data collected by the sound collector 60 is output to the audio detection unit 61 via the communication unit 27 and the input unit 31. The voice detection unit 61 collates this voice data with a specific voice stored in the storage unit 36, and if they match, it detects that a specific voice has been generated in the monitoring area A (step T2 in FIG. 13). ).

Information that a specific sound is detected by the sound detection unit 61 is output to the monitoring unit 33. The monitoring unit 33 sets the imaging direction of the monitoring camera 24 in the direction of the specific sound detected by the sound detection unit 61 and moves the monitoring camera 24 via the control unit 35. In step S <b> 3, the monitoring camera 24 captures an area in the monitoring area A where specific sound is generated. Thereafter, in step S4, the region where the specific sound is generated is monitored.

In such a case, when an abnormal state or dangerous event occurs in the monitoring area A, it is possible to identify the sound and monitor the area where the sound is generated with particular attention. Even if the sound moves, the region where the sound is generated can be appropriately monitored by causing the monitoring camera 24 to follow the direction of the sound.

<5-6. Other embodiments>
In the above embodiment, a plurality of imaging devices 20 may be installed in the store 10. In such a case, the entire interior of the store 10 can be monitored by monitoring a plurality of monitoring areas A. Further, if one monitoring area A is monitored by a plurality of imaging devices 20, the monitoring area A can be monitored with higher accuracy.

Moreover, although the above embodiment demonstrated the case where the store 10 was monitored using the monitoring system 1, the place to monitor is not limited to this. The monitoring system 1 according to the present invention includes, for example, shopping malls, department stores, complex general facilities, commercial facilities such as exhibitions and trade fair venues, public facilities such as airports and railroads, amusement parks and stadiums. Various places such as amusement facilities such as hospitals, specialized facilities such as hospitals and health centers, offices and factories, detached houses, apartment houses, and parking lots can be monitored.

The preferred embodiments of the present invention have been described above with reference to the accompanying drawings, but the present invention is not limited to such examples. It is obvious for those skilled in the art that various changes or modifications can be conceived within the scope of the idea described in the claims, and these are naturally within the technical scope of the present invention. It is understood.

The present invention is useful, for example, when monitoring a predetermined area.

DESCRIPTION OF SYMBOLS 1 Monitoring system 10 Store 11 Floor 12 Furniture 20 Imaging device 21 Case 22 Dome cover 23 Distance sensor 23a Irradiation source 23b Light receiving element 24 Monitoring camera 25 Support member 26 Drive mechanism 27 Communication part 30 Monitoring apparatus 31 Input part 32 Model generation Unit 33 monitoring unit 34 output unit 35 control unit 36 storage unit 40 person setting unit 50 upper region 51 middle region 52 lower region 60 sound collector 61 voice detection unit A monitoring region D mesh model M mesh N mesh P (P1, P2) person

Claims

A monitoring system for monitoring a predetermined area,
An imaging unit that is fixedly provided at a predetermined position and images the predetermined area from above;
A distance measuring unit for measuring a distance from the imaging unit to a measurement object in the predetermined region;
About the background of the predetermined area composed of the bottom surface of the predetermined area and the permanent object permanently installed in the predetermined area during the monitoring period, the distance from the imaging unit to the background measured by the distance measuring unit A model generation unit that divides the background in a horizontal direction and a height direction with a mesh of a predetermined size, and generates a mesh model of the background;
A monitoring target that moves based on the image data captured by the imaging unit is extracted, the mesh model is applied to the background of the predetermined area of the image data, and the monitoring target is synthesized with the mesh model. And a monitoring unit that generates the monitoring image data and monitors the predetermined area.
The monitoring system according to claim 1, wherein the model generation unit superimposes the background image in the image data captured by the imaging unit on the mesh model in the monitoring image data.
A database in which information of objects in the predetermined area is stored;
The monitoring system according to claim 1, wherein the monitoring unit specifies the object based on the database and monitors the predetermined area.
4. The model generation unit according to claim 1, wherein in the mesh model, a space surrounded by a bottom surface of the predetermined region and the permanent object is three-dimensionally divided by the mesh of the predetermined size. A monitoring system according to claim 1.
A person setting section for dividing the person who has entered the predetermined area into a plurality of areas;
The monitoring unit according to any one of claims 1 to 4, wherein the monitoring unit monitors a behavior of the person based on a rate of change of the person in each region set by the person setting unit. Monitoring system.
A voice detector for detecting voice generated in the predetermined area;
6. The monitoring unit according to claim 1, wherein the monitoring unit sets an imaging direction of the imaging unit in a direction of the voice detected by the voice detection unit, and monitors a region where the voice is generated. The monitoring system according to one item.
A monitoring method for imaging and monitoring a predetermined area with an imaging unit,
Ranging that measures the distance from the imaging unit to the background with a ranging unit for the background of the predetermined region that is composed of a bottom surface of the predetermined region and a permanent object that is permanently installed in the predetermined region during a monitoring period Process,
Based on the distance measured in the distance measurement step, the background is partitioned in a horizontal direction and a height direction with a mesh of a predetermined size, and a model generation step for generating a mesh model of the background,
An imaging step of imaging the predetermined area by the imaging unit;
A monitoring target that moves based on the image data captured in the imaging step is extracted, the mesh model is applied to the background of the predetermined area of the image data, and the monitoring target is synthesized with the mesh model. And a monitoring step for generating the monitoring image data and monitoring the predetermined area.
The monitoring method according to claim 7, wherein, in the model generation step, the background image in the image data captured by the imaging unit is superimposed on the mesh model in the monitoring image data.
Information on objects in the predetermined area is stored in a database;
The monitoring method according to claim 7 or 8, wherein in the monitoring step, the predetermined area is monitored by specifying the object based on the database.
The method according to any one of claims 7 to 9, wherein, in the model generation step, in the mesh model, a space surrounded by a bottom surface of the predetermined region and the permanent object is three-dimensionally divided by the mesh of the predetermined size. The monitoring method according to claim 1.
In the monitoring step, the person who entered the predetermined area is divided into a plurality of areas, and the behavior of the person is monitored based on the rate of change of the person in each of the divided areas. Item 11. The monitoring method according to any one of Items 7 to 10.
In the imaging step, when the sound generated in the predetermined area is detected by the sound detection unit, the imaging direction of the imaging unit is set in the direction of the sound, and the region where the sound is generated is imaged.
The monitoring method according to any one of claims 7 to 11, wherein, in the monitoring step, a region where the sound is generated is monitored.
A program that operates on a computer that causes the computer that controls the monitoring system to function so that the monitoring method according to any one of claims 7 to 12 is executed by the monitoring system.
A readable computer storage medium storing the program according to claim 13.