JP5418072B2 - Monitoring control device and monitoring system - Google Patents

Description

  The present case relates to a monitoring control device and a monitoring system.

  In stores and homes, for the purpose of improving security, photographing is performed with a surveillance camera fixed to a wall or ceiling. Recently, techniques such as Patent Documents 1 and 2 have been proposed as techniques relating to such photographing by a monitoring camera.

  Among these, the technique of Patent Document 1 automatically changes the monitoring area in consideration of the blind spot so that the plurality of cameras cover the monitoring area widely. The technique of Patent Document 2 determines the patrol route and schedule of the patrol robot in consideration of the danger of each area from the information of the monitoring camera.

International Publication No. 2005/076761 JP 2006-72612 A

  However, the technique described in Patent Document 1 requires a large number of devices and consequently a great cost when monitoring a place where a large number of dynamic blind spots occur, such as a place where people frequently travel. There is a risk.

  Further, the technique described in Patent Document 2 is intended to warn of a person entering an unmanned space, and considers a blind spot generated in a manned space, that is, a blind spot caused by a dynamic shield (such as a person). It was n’t.

  Accordingly, the present invention has been made in view of the above problems, and an object thereof is to provide a monitoring control device and a monitoring system capable of improving the efficiency of monitoring a blind spot caused by a dynamic shielding object.

The monitoring control apparatus described in this specification includes any one of a shielding object extracting unit that extracts a shielding object from an image acquired from a fixed camera, and a plurality of management units that divide a space in which the fixed camera is provided at equal intervals. in the shield position estimating means for shielding extracted by the obstacle extracting unit estimates whether the position, based on the estimation result of the obstacle position estimation device, corresponding to the position of the blind spot of the fixed camera A blind spot calculating means for calculating a management unit; and a moving body control means for controlling the position and orientation of the moving body based on the management unit corresponding to the blind spot position of the fixed camera calculated by the blind spot calculating means. ing.

  The monitoring system described in this specification controls the position and orientation of the moving body based on a fixed camera that acquires a video, a movable body that is movable, and a video that is acquired from the fixed camera. And a monitoring control device described in this specification.

  The monitoring control device and the monitoring system described in the present specification have an effect that it is possible to improve the monitoring efficiency of a blind spot caused by a dynamic shield.

It is a figure which shows the space where the monitoring system which concerns on 1st Embodiment is installed. It is a block diagram which shows the structure of a monitoring system. It is a figure which shows the state which divided | segmented space by the some grid at equal intervals. It is a figure which shows the database of a grid list. It is a figure for demonstrating the importance of a grid. FIG. 6A is a diagram showing the situation in the space at 18:36:40, and FIG. 6B is a diagram showing the situation in the space at 18:37:10. It is a figure which shows the condition in the space at 18:39:10. It is a flowchart which shows the process for creating the database of a grid list, and the database of a blind spot list. It is a figure which shows the database of a grid list in 18 hours 36 minutes 40 seconds. FIG. 10A is a diagram showing a provisional blind spot list database at 18:36:40, and FIG. 10B is a diagram showing a blind spot list database at 18:36:40. It is a figure which shows the database of a grid list in 18:37:10. 12A is a diagram showing a provisional blind spot list database at 18:37:10, and FIG. 12B is a diagram showing a blind spot list database at 18:37:10. It is a figure which shows the database of a grid list in 18:39:10. FIG. 14A is a diagram showing a provisional blind spot list database at 18:39:10, and FIG. 14B is a diagram showing a blind spot list database at 18:39:10. It is a figure which shows the modification of 1st Embodiment. It is a block diagram which shows the structure of the monitoring system which concerns on 2nd Embodiment.

<< First Embodiment >>
Hereinafter, a first embodiment of the monitoring system will be described in detail with reference to FIGS.

  The monitoring system according to the first embodiment (hereinafter, referred to as “monitoring system 300”) captures a person who photographs the space 100 (for example, a store) shown in FIG. This is a system for providing a photographed image to a store employee or a person who performs security. It is assumed that a shelf 110 is installed in the space 100 and an important object 200 is placed on a part of the shelf 110.

  The monitoring system 300 includes a plurality of (four in FIG. 1) monitoring cameras 10 </ b> A to 10 </ b> D as fixed cameras, and the mobile robot 20 as a moving body that patrols the space 100. The monitoring system 300 includes a control device 40 (see FIG. 2) as a monitoring control device that controls the monitoring cameras 10A to 10D and the mobile robot 20.

  As shown in FIG. 1, the monitoring cameras 10 </ b> A to 10 </ b> D are fixed to a wall that partitions the space 100. FIG. 2 is a block diagram showing the configuration of the monitoring system 300. As shown in FIG. 2, each of the monitoring cameras 10 </ b> A to 10 </ b> D includes a video acquisition unit 12 that individually acquires videos in the visual fields VA to VD (see FIG. 1), and the acquired video to the control device 40. Video output means 14 for outputting. In the first embodiment, it is assumed that the visual fields VA to VD are fixed. In addition, the monitoring cameras 10A to 10D are installed at positions where the entire space 100 can be photographed without blind spots in a state where there is no person or the like in the space 100.

  As shown in FIG. 2, the mobile robot 20 includes a position / orientation detection unit 22, a video acquisition unit 24, an information notification unit 26, and an autonomous movement unit 34.

  The position / orientation detection means 22 includes an acceleration sensor, an orientation sensor (such as a geomagnetic sensor), an encoder, and the like, and detects the position and orientation of the mobile robot 20. The detection result by the position / orientation detection unit 22 is transmitted to the information notification unit 26.

  The image acquisition unit 24 includes, for example, a camera provided on the head of the mobile robot 20 and acquires an image (moving image) in a direction in which the head is facing by the camera. The video acquired by the video acquisition unit 24 is transmitted to the information notification unit 26. The information notification unit 26 outputs the detection result acquired from the position / orientation detection unit 22 and the video acquired from the video acquisition unit 24 to the control device 40.

  The autonomous moving means 34 moves in the space 100 and changes the posture based on an instruction from the control device 40. Specifically, the autonomous moving means 34 includes a wheel, a motor for driving the wheel, an actuator for changing the posture, and the like. The autonomous moving means 34 also includes an actuator that changes the direction of the head to change the shooting direction of the camera.

  The control device 40 includes a robot information acquisition unit 42, a camera image acquisition unit 44, a shielding object extraction unit 46, a shielding object position estimation unit 48, a blind spot calculation unit 50, a monitoring necessity update unit 52, and a robot target. Position / orientation determination means 54, movement instruction means 56, and notification means 58 are provided. The monitoring necessity update unit 52, the robot target position and orientation determination unit 54, and the movement instruction unit 56 are mobile body control units that control the position and orientation of the mobile robot 20 based on the positions of the blind spots of the monitoring cameras 10A to 10D. Part of it.

  The robot information acquisition unit 42 acquires information (the detection result by the position / orientation detection unit 22 and the video acquired by the video acquisition unit) input from the information notification unit 26 of the mobile robot 20 and transmits the information to the shielding object extraction unit 46. To do. The camera video acquisition unit 44 acquires the video input from the video output unit 14 of the monitoring cameras 10 </ b> A to 10 </ b> D and transmits the video to the shielding object extraction unit 46. The shielding object extraction unit 46 extracts the shielding object from the images from the monitoring cameras 10A to 10D and the image from the mobile robot 20. A background subtraction method or the like can be used to extract the shielding object. When the shielding object is extracted by the shielding object extraction unit 46, the images from the monitoring cameras 10 </ b> A to 10 </ b> D and the image from the mobile robot 20 are transmitted to the shielding object position estimation unit 48.

  The shielding object position estimating means 48 estimates the position of the shielding object from the images from the monitoring cameras 10A to 10D and the image from the mobile robot 20. For the position estimation here, for example, a stereo vision method or the like can be used. The blind spot calculating means 50 calculates a blind spot range, that is, a blind spot area, based on the estimation result of the shielding object position by the shielding object position estimating means 48. Here, for example, the blind spot area can be calculated based on the size and position of the shielding object and the imaging angles of the monitoring cameras 10A to 10D. In calculating the blind spot area, it is necessary to consider the imaging area of the mobile robot 20.

  The monitoring necessity degree update means 52 determines and updates the necessity degree of whether or not the blind spot should be monitored based on the calculation result by the blind spot calculation means 50 and the like. The robot target position / posture determination unit 54 determines the movement target position and posture of the mobile robot 20 according to the monitoring necessity updated by the monitoring necessity update unit 52. The movement instructing unit 56 issues an instruction to the autonomous moving unit 34 so that the mobile robot 20 has the target position and posture determined by the robot target position / posture determining unit 54. The reporting unit 58 includes a monitor or the like, and is for notifying the user of the movement target position and posture of the mobile robot updated by the monitoring necessity update unit 52.

  Next, a management unit (grid) used when the control device 40 manages the space 100 will be described with reference to FIG. FIG. 3 shows a state in which the space 100 is divided at equal intervals by a plurality of grids. In FIG. 3, the horizontal direction on the paper is the X-axis direction, and the vertical direction on the paper is the Y-axis direction. In addition, 23 grids are arranged in the X-axis direction, and 15 grids are arranged in the Y-axis direction. Each grid is assigned with a grid ID in order from the upper left, and these grids are managed in a grid list database as shown in FIG. This database management is performed by the monitoring necessity level update means 52 of FIG.

  In the grid list database of FIG. 4, each item of coordinates, state (current state), previous state, last update time, last confirmation time, and importance is associated with the grid ID number. Among these, in the items of “state” and “previous state”, “monitoring”, “shielding object”, or “dead angle” is recorded. “Monitoring” means a state where monitoring by the monitoring cameras 10 </ b> A to 10 </ b> D is possible, and “shielding” means a dynamic shielding, that is, a grid where a person exists. Also, “dead zone” means a grid that cannot be monitored due to the presence of a shield.

  The “last update time” item records the time when the previous state changed to the current state, and the “last confirmation time” item records the time when the last monitoring was confirmed. Is done. The “final confirmation time” specifically means the time when the image was last taken by any of the monitoring cameras 10A to 10D and the mobile robot 20. The “importance” item is an index that indicates whether the location of the grid is a place where monitoring should be prioritized. For example, “1” means that the importance is high. , “0” means that the importance is normal. In the present embodiment, as shown with double hatching in FIG. 5, the grid located around the important object 200 has a high importance level, and thus its value is set to 1. Since the importance is normal, it is assumed that the value is set to 0.

  Next, a monitoring method in the space 100 by the monitoring system 300 will be described along the flowchart of FIG. 8 and with reference to other drawings as appropriate.

  In the following description, as shown in FIG. 3, after a predetermined time has elapsed from the situation where no one is present in the space 100, a person is placed in the space 100 as shown in FIG. Suppose only one person (H1) came in. Also, assume that another person (H2) enters the space 100 as shown in FIG. 6 (b) after a predetermined time has elapsed from the situation of FIG. 6 (a). Further, after a predetermined time has elapsed from the situation of FIG. 6B, as shown in FIG. 7, one person (H2) leaves the space 100 and only one person (H1) stays in the space 100. It is assumed that the situation has become. Here, for convenience of explanation, it is assumed that the monitoring necessity update unit 52 updates the database of FIG. 4 every 30 seconds.

  Also, it is assumed that the surveillance cameras 10A to 10D finally confirmed the state of FIG. 3 at 18:36:10. Then, it was confirmed that the state of FIG. 6 (a) was reached, but it was confirmed that the state of FIG. 6 (b) was confirmed at 18:36:40 after 30 seconds. Assume that 18 seconds 37 minutes 10 seconds later. In addition, it is assumed that it was 18:39:10, two minutes later, that the state shown in FIG. 7 was confirmed.

  Under the above premise, the flowchart of FIG. 8 is executed. Hereinafter, processing at 18:36:10, 18:36:40, 18:37:10, and 18:39:10 will be described in order.

(18:36:10)
First, in step S10 of FIG. 8, the camera image acquisition unit 44 acquires the images of the monitoring cameras 10A to 10D. In the next step S12, based on the extraction result of the shielding object extracting means 46, it is determined whether or not there is a shielding object in the space 100 in the state of FIG. At this stage, since there is no shielding object, the determination in step S12 is denied and the process returns to step S10.

(18:36:40)
At 18:36:40, step S10 is performed again, so that the camera image acquisition unit 44 acquires the images of the monitoring cameras 10A to 10D, and then the process proceeds to step S12. At this stage, as shown in FIG. 6A, since the person (H1) as the shielding object exists in the space 100, the determination in step S12 is affirmed. As described above, when the determination in step S12 is affirmed, the process proceeds to step S14, and the shielding object position estimating unit 48 estimates the position of the shielding object. Here, as shown by hatching in FIG. 6A, the position of the shielding object is the coordinates of the grid (4, 1), (4, 2), (5, 1), (5, 2).

  In the next step S16, the blind spot calculating means 50 calculates the position of the blind spot area generated from the shielding object based on the position of the shielding object estimated in step S14. As a result of the processing here, the grid coordinates (6, 0), (6, 1), (6, 2), (7, 0) shown in FIG. ), (7, 1), (7, 2), (8, 0), (8, 1), (8, 2) are calculated. When calculating the position of the blind spot area, it is necessary to consider not only the images captured by the monitoring cameras 10A to 10D but also the images captured by the camera of the mobile robot 20.

  Next, in step S18, the monitoring necessity update unit 52 updates the grid list database. Here, the grid list database is updated as shown in FIG. In FIG. 9, the description regarding the grid in which the state being monitored is continued is omitted.

  Next, in step S20, the monitoring necessity degree update unit 52 executes a process of collecting the grids having “dead spots” for each continuous grid. In this case, the grid coordinates (6, 0), (6, 1), (6, 2), (7, 0), (7, 1), (7, 2), (8, 0) shown in FIG. , (8, 1), (8, 2) are all continuous in the vertical and horizontal directions. Accordingly, in step S20, all the grids whose status items are blind spots in FIG. 9 are collected, and the ID “1 ′” as shown in FIG. be registered. In addition, the item of “detection time” in FIG. 10A is the time when the blind spot is detected, and “final confirmation time” is the earliest time among the final confirmation times of the grid included in the ID “1 ′”. It is. The “importance” item is the sum of the importance of the grids included in the ID “1 ′”. Here, since the importance levels of the grid included in the ID “1 ′” are all 0 (see FIG. 9), the importance levels in FIGS. 10A and 10B are also 0.

  Next, in step S22, the monitoring necessity update unit 52 selects one blind spot area immediately before. However, at this stage, since the blind spot area did not exist until immediately before, the blind spot area cannot be selected, and the process proceeds to step S24 as it is.

  Next, in step S24, the monitoring necessity degree update unit 52 determines whether or not the blind spot area has been selected in the previous step S22. Here, since the blind spot area has not been selected, the determination is negative, and the process proceeds to step S26. In step S26, the monitoring necessity degree update means 52 newly registers the blind spot area composed of the grid collected in step S20 as the blind spot ID = “1” in the blind spot list database shown in FIG. The process of 8 is finished.

  The robot target position / posture determination means 54 determines the target position and posture of the mobile robot 20 based on the blind spot list database updated as described above. Specifically, the target position and posture of the mobile robot 20 are determined to be a position and posture where a blind spot area with a blind spot ID = “1” can be confirmed. Then, the movement instruction unit 56 controls the autonomous moving unit 34 to change the position / posture of the mobile robot 20 so that the position and posture determined by the robot target position / posture determination unit 54 are obtained. Here, the monitoring necessity degree update means 52 can detect the movement status of the mobile robot 20 by detecting the detection result of the position / orientation detection means 22 and the operation status of the autonomous movement means 34 so that the mobile robot 20 can reach the target position. It is determined whether or not. As a result of this determination, if the target position cannot be reached within a predetermined time, the monitoring necessity level update means 52 monitors the space 100 via the notification means 58 (see FIG. 2). And can warn guards. Here, for convenience of explanation, it is assumed that the mobile robot 20 has not been able to monitor the blind spot area of the blind spot ID = 1 at the stage of 18:37:10.

(18:37:10)
At 18:37:10, after the camera image acquisition unit 44 acquires the images from the surveillance cameras 10A to 10D in step S10, in step S12, the shielding object extraction unit 46 determines whether or not there is a shielding object. To do. At this stage, as shown in FIG. 6B, in addition to the person H1, the person H2 has also entered the space 100, so the determination here is affirmed and the process proceeds to step S14.

  In step S14, the shielding object position estimating means 48 estimates the position of the shielding object. Here, the positions of the shielding objects are indicated by hatching grid coordinates (4, 1), (4, 2), (5, 1), (5, 2), (10, 9), (10, 10), (10, 11), (11, 10), and (11, 11). Next, in step S16, the blind spot calculating means 50 calculates the position of the blind spot from the position of the shield. Here, the positions of the blind spots are indicated by grid coordinates (6, 0), (6, 1), (6, 2), (7, 0), (7, 0) indicated by double hatching in FIG. 1), (7, 2), (7, 9), (7, 10), (7, 11), (8, 0), (8, 1), (8, 2), (8, 9) , (8, 10), (8, 11), (9, 9), (9, 10), (9, 11).

  Next, in step S18, the monitoring necessity update unit 52 updates the grid list database based on the results of steps S14 and S16. Specifically, the grid list database is updated as shown in FIG. Here, the hatched portion in FIG. 11 is newly added to the grid list database.

  Subsequently, in step S20, the monitoring necessity degree update means 52 collects the blind spot area grids for each continuous grid. As a result of the processing in step S20, as shown in FIG. 12A, IDs “1 ′” and “2 ′” are tabulated as a temporary blind spot list database. In this case, the ID “1 ′” includes grids with grid IDs 91, 92, 93, 106, 107, 108, 121, 122, and 123, and the ID “2 ′” includes grid IDs 115, 116, 117, 130, 131, 132, 145, 146 and 147 grids are included.

  Next, in step S22, the monitoring necessity update unit 52 selects one immediately preceding blind spot area. Here, it is assumed that the blind spot area of the blind spot ID = “1” shown in FIG. 10B is selected. Next, in step S24, the monitoring necessity update unit 52 determines whether or not the blind spot has been selected. If the determination here is affirmed, the process proceeds to step S28. In step S28, the monitoring necessity level update unit 52 selects a provisional blind spot area that has the largest overlap with the selected area.

  Here, since the grid ID included in the blind spot of the blind spot ID “1” in FIG. 10B and the grid ID included in the ID “1 ′” of FIG. In step S28, the ID “1 ′” in FIG. 12A is selected.

  Next, in step S30, it is determined in the previous step S28 whether or not the monitoring necessity update unit 52 has selected the corresponding blind spot area. Since the determination here is affirmed, the process proceeds to step S34, and the monitoring necessity level update means 52 updates the previous blind spot area (dead spot ID = 1) with the content of the ID “1 ′” in FIG. To do. As described above, the reason for updating the previous blind spot area with the contents of the provisional blind spot area is to continuously follow the blind spot caused by the same shield.

  Next, in step S36, the monitoring necessity update unit 52 determines whether or not the processing for all the blind spot areas immediately before has been completed. Here, since the processing of all the blind spot areas immediately before has been completed, the determination is affirmed and the process proceeds to step S38.

  In step S38, an ID that has not been updated in the provisional blind spot list database is newly registered. That is, the blind spot of ID “2 ′” shown in FIG. 12A is registered as a new blind spot (dead spot ID = 2). In this way, the blind spot list database is updated as shown in FIG. The blind spot area of the blind spot ID “2” in FIG. This is because, as shown in FIG. 11, among the grid IDs included in the blind spot area of the blind spot ID “2”, the importance of 115 and 130 is 1, and the sum of them is 2.

The robot target position / posture determination means 54 determines the movement path of the mobile robot 20 based on the blind spot list database updated as described above. That is, the mobile robot 20 determines which blind spot to check first. Here, for example, in order to determine the confirmation order of the blind spots, the priority of each blind spot is calculated. Here, the priority P of each blind spot is the size of the blind spot (number of grid IDs included in the blind spot) a, the time b after the blind spot is reached, and the time since the last confirmation of the grid included in the blind spot. Assuming c and importance d, it can be calculated from the following equation (1).
P = a × α + b × β + c × γ + d × δ (1)

  Α, β, γ, and δ are weighting coefficients. Therefore, among the items a to d, the weighting coefficient to be integrated may be increased for an item to be emphasized, and the weighting coefficient to be integrated may be decreased for an item that is not so important. If there is an item that does not need to be considered, the weighting coefficient to be integrated may be set to zero.

  The robot target position / orientation determination means 54 determines the movement path of the mobile robot 20 so as to monitor from the one having the higher priority P as a result of calculating the priority P based on the above equation (1). Then, the movement instruction unit 56 controls the autonomous moving unit 34 to change the position / posture of the mobile robot 20 so that the position and posture determined by the robot target position / posture determination unit 54 are obtained. In the first embodiment, the mobile robot 20 monitors the blind spot with the blind spot ID = 1 at 18:37:50 and monitors the blind spot with the blind spot ID = 2 at 18:37:20. To do.

(18:39:10)
Even at 18:39:10, after step S10 is executed by the camera image acquisition unit 44 as in the past, in step S12, the shielding object extraction unit 46 determines whether there is a shielding object. At this stage, as shown in FIG. 7, since there is a person H1 as a shield, the determination here is affirmed, and the routine proceeds to step S14.

  In step S14, the shielding object position estimating means 48 estimates the position of the shielding object. Here, the position of the shielding object is estimated as grid coordinates (4, 2), (4, 3), (5, 2), (5, 3) indicated by hatching in FIG. Next, in step S16, the blind spot calculating means 50 calculates the position of the blind spot area from the position of the shield. Here, the position of the blind spot area is indicated by grid coordinates (6, 2), (6, 3), (6, 4), (7, 2), (7, 3) indicated by double hatching in FIG. , (7, 4), (8, 2), (8, 3), and (8, 4).

  Next, in step S18, the monitoring necessity update unit 52 updates the grid list database based on the results of steps S14 and S16. Specifically, the grid list database is updated as shown in FIG.

  Next, in step S20, the monitoring necessity update unit 52 collects the blind spot area grids for each successive grid and records them in the provisional blind spot list database. As a result of the processing in step S20, ID “1 ′” is recorded in the provisional blind spot list database as shown in FIG. In this case, the grid of grid IDs 93, 94, 95, 108, 109, 110, 123, 124, and 125 is included in ID “1 ′”.

  Next, in step S22, the monitoring necessity update unit 52 selects one immediately preceding blind spot area. Here, it is assumed that the blind spot area having the blind spot ID “1” shown in FIG. 12B is selected. Next, in step S24, the monitoring necessity degree update unit 52 determines whether or not the blind spot area has been selected. If the determination here is affirmed, the process proceeds to step S28. In step S28, the monitoring necessity level update unit 52 selects a provisional blind spot area that has the largest overlap with the selected area.

  In this case, the blind spot with the blind spot ID = 1 in FIG. 12B and the blind spot with the ID = “1 ′” in FIG. 14A are part of the grid ID (the grid shown with an underline in FIG. 14). ID). Accordingly, in step S28, the blind spot with ID = “1” in FIG. 14A is selected.

  Next, in step S30, in the previous step S28, the monitoring necessity degree update unit 52 determines whether or not the corresponding blind spot area has been selected. If the determination here is affirmed, the process proceeds to step S34, and the monitoring necessity degree update unit 52 sets the previous blind spot area (the blind spot ID “1” in FIG. 12B) to a new content (FIG. 14A). ) With ID “1 ′”). Next, in step S36, the monitoring necessity update unit 52 determines whether or not the processing for all of the previous blind spot areas has been completed. Here, since the process related to the blind spot area of the blind spot ID “2” in FIG. 12B remains, the determination is negative and the process proceeds to step S22.

  In step S22, the monitoring necessity update unit 52 selects one immediately preceding blind spot area. Here, it is assumed that the blind spot ID = 2 shown in FIG. 12B is selected. Next, in step S24, the monitoring necessity update unit 52 determines whether or not the blind spot has been selected. If the determination here is affirmed, the process proceeds to step S28. In step S <b> 28, the monitoring necessity level update unit 52 selects the blind spot area having the largest overlap with the selected area. However, at this stage, there is no overlapping area of the blind spot area of the blind spot ID “2” and the grid in the database of FIG.

  Next, in step S30, the monitoring necessity update unit 52 determines whether or not there is a corresponding blind spot area. Since the determination here is negative, the process proceeds to step S32. In step S32, the monitoring necessity degree update unit 52 deletes the blind spot with the blind spot ID = 2 as shown in FIG. 14B, and proceeds to step S36. In step S36, the monitoring necessity degree update means 52 determines whether or not the processing of all the blind spot areas immediately before has been completed, and if the determination here is affirmed, the process proceeds to step S38. In step S38, the monitoring necessity update unit 52 newly registers a provisional blind spot area that has not been updated. Here, there is no provisional blind spot area that has not been updated. The whole process of 8 is finished.

  The robot target position / posture determination means 54 determines the target position and posture of the mobile robot 20 to be a position and posture at which the blind spot with the blind spot ID = 1 can be confirmed based on the blind spot list database updated as described above. . Then, the movement instruction unit 56 controls the autonomous moving unit 34 to change the position / posture of the mobile robot 20 so that the position and posture determined by the robot target position / posture determination unit 54 are obtained.

  As described above in detail, according to the present embodiment, the shielding object extraction unit 46 extracts the shielding object from the images acquired from the monitoring cameras 10A to 10D, and the shielding object position estimation unit detects the shielding object. Estimate the position. The blind spot calculating means 50 calculates the blind spot position of the monitoring cameras 10A to 10D from the estimated position of the shield, and based on the blind spot position, the robot target position / posture determination means 54 and the movement instruction means 56 Since the position and orientation of the mobile robot 20 is controlled, the position of the blind spot is accurately calculated from the images taken by the monitoring cameras 10A to 10D, and then the mobile robot 20 is moved toward the blind spot and the attitude is changed. can do. Thereby, when the mobile robot 20 has a photographing function, it is possible to photograph a blind spot area that cannot be monitored by the surveillance camera. That is, according to the present embodiment, it is possible to improve the monitoring efficiency of blind spots caused by a dynamic shield such as a person.

  More specifically, the movement instruction unit 56 controls the position and orientation of the mobile robot 20 so that the camera (video acquisition unit 24) held by the mobile robot 20 captures the blind spots of the monitoring cameras 10A to 10D. The mobile robot 20 can capture a blind spot that cannot be observed with only the surveillance camera or a blind spot caused by a dynamic shield. As a result, there is no need to go directly to the places where watchers and guards are blind spots, so it is possible to save the trouble of the watchers, etc. Risk can be reduced.

  Further, according to the present embodiment, even when the movement instructing means 56 controls the position and orientation of the mobile robot 20 via the autonomous moving means 34, the reporting means 58 monitors the fact that the blind spot cannot be photographed. Alert users such as security personnel and security guards. Thereby, the supervisor or the like can easily determine whether or not the blind spot should be confirmed directly.

  Further, according to the present embodiment, the robot target position / posture determination means 54 controls the position / posture of the mobile robot 20 based on the blind spot priority P when there are a plurality of blind spots. Therefore, even when there are a plurality of blind spots, the blind spots can be monitored in an appropriate order according to the priority. In this case, by determining the priority P in consideration of the size of the blind spot, it is possible to preferentially monitor the blind spot where a large number of people are gathered and the possibility of a crime is relatively high. In addition, by determining the priority P in consideration of the period that is a blind spot, it is possible to preferentially monitor the blind spot that has a long period of the blind spot and a relatively high possibility that a crime has occurred. it can. Furthermore, by determining the priority P based on the importance specific to the place, it is possible to preferentially monitor a blind spot where crime is likely to occur, such as the vicinity of the important object 200.

  In the above embodiment, when the priority P is calculated, the situation immediately before the blind spot is taken into consideration can also be considered. For example, a place where a person exists before becoming a blind spot and a place where the person becomes a blind spot after that has a higher possibility of a crime than a place where a person did not exist before becoming a blind spot. Therefore, for example, the above equation (1) may be changed so that the priority P becomes higher when the previous state is “shielding object”.

  In the above-described embodiment, a display 60 as a display unit can be connected to the monitoring necessity update unit 52 as shown in FIG. 15 together with or instead of the notification unit 58. The display 60 may display the blind spot monitoring order determined by the monitoring necessity update unit 52. In this case, a monitor, a guard, or the like can check the monitoring order on the display 60. Moreover, it is good also as displaying on the display 60 at least one of the position of a shield (person) and the position of a blind spot with the monitoring order or instead of this. Even in this case, it is possible to improve the convenience of the supervisor.

  In the above embodiment, as shown in FIG. 15, an input interface 62 as a change acceptance unit may be connected to the monitoring necessity update unit 52. The input interface 62 is used by a monitor or the like to change the blind spot monitoring order. In this case, the monitoring order itself may be simply changed. Alternatively, the monitor or the like may change only at least one item for determining the priority, for example, the importance item. In the latter case, using the changed item, the robot target position and orientation determination unit 54 calculates the priority P again, and resets the monitoring order based on the calculation result. In addition, when the position that the monitor wants to see is specified, the mobile robot 20 may be forcibly moved to the position that the user wants to see regardless of the position of the blind spot area. By doing in this way, it is possible to realize appropriate monitoring by the mobile robot 20 reflecting the intention of a monitor or the like.

  In the above embodiment, the case where the importance level of the blind spot list database is set to the total value of the importance levels of the grids included in the same blind spot ID is described, but the present invention is not limited to this. The importance of the blind spot list database may be, for example, the highest value among the importance of the grids included in the same blind spot ID.

  In the above-described embodiment, the case where the grid ID database is updated every 30 seconds has been described. However, the present invention is not limited to this, and the update interval may be set to other intervals (for example, intervals of several seconds). In this case, the shorter the update interval, the better the monitoring performance.

  Moreover, although the said embodiment demonstrated the case where nine grids behind a person became a blind spot for convenience of explanation, it was not restricted to this. Actually, all the grids behind the person may become blind spots, so it is possible to determine the blind spot grid to cover the actual blind spots in consideration of the actual camera placement, shooting angle, etc. preferable.

<< Second Embodiment >>
Hereinafter, a second embodiment of the monitoring system will be described with reference to FIG.

  In the second embodiment, the mobile robot 20 further includes a person identifying means 28, an interaction means 30, and a suspiciousness degree calculating means 32, as shown in FIG. 15, in addition to the means shown in FIG. I have. Since other configurations are the same as those in the first embodiment, the description thereof will be omitted.

  The person identification means 28 is for identifying the presence or absence of a person from the video acquired by the video acquisition means 24. The interaction means 30 interacts with surrounding people by producing a sound or displaying an image. The suspicious degree calculating means 32 calculates the suspicious degree of the person identified by the person identifying means 28. The suspicious degree calculating means 32 holds, for example, a database in which a person's behavior pattern and its suspicious degree are associated, a database in which a person's appearance and its suspicious degree are associated, and the like. The suspicious degree of the person is calculated.

  In the second embodiment, the grid list database is updated by the same method as in the first embodiment. On the other hand, when the monitoring necessity degree update unit 52 updates the blind spot list database, the calculation result by the suspicious degree calculation unit 32 is also taken into consideration. Specifically, for the person identified by the person identifying unit 28 based on the video of the video obtaining unit 24, the suspicious degree calculating unit 32 calculates the suspicious degree and uses the calculation result for calculating the priority P. To do. In this way, it is possible to focus on monitoring the vicinity of a suspicious person.

  On the other hand, when the person identified by the person identification means 28 is in the way and cannot monitor the blind spot, the interaction means 30 instructs the person to move to another place using an image or sound. By doing so, the blind spot can be confirmed smoothly.

  As described above, according to the second embodiment, in addition to the same effects as those of the first embodiment, the mobile robot 20 can reflect the suspicious degree of the person in the calculation of the priority P. Monitoring can be performed appropriately. In addition, since the interaction means 30 can interact with a person, the person can be moved when confirming the blind spot. Thereby, it is possible to check the blind spot smoothly.

  In the second embodiment, the case where the mobile robot 20 includes both the suspicious degree calculation unit 32 and the interaction unit 30 has been described. However, the present invention is not limited to this. The mobile robot 20 may include any one of the two means.

  In each of the above-described embodiments, the case where the mobile robot 20 has the shooting function (video acquisition unit 24) has been described. However, the present invention is not limited to this. The mobile robot 20 may communicate with a person so that the blind spot area does not exist in one place for a long time, that is, the mobile robot 20 may only have the person move. Even in such a case, since the monitoring cameras 10A to 10D can capture the blind spot, the blind spot monitoring efficiency can be improved.

  Each embodiment mentioned above is an example of suitable implementation of the present invention. However, the present invention is not limited to this, and various modifications can be made without departing from the scope of the present invention.

10A-10D surveillance camera (fixed camera)
20 Mobile robot (moving body)
40 Control device (monitoring control device)
46 shielding object extraction means 48 shielding object position estimation means 50 blind spot calculation means 52 monitoring necessity degree updating means (part of moving body control means)
54 Robot target position and orientation determining means (part of moving body control means)
56 Movement instruction means (part of moving body control means)
58 Notification means 60 Display (display means)
62 Input interface (change acceptance means)
H1, H2 people (shield)

Claims (10)

Shielding object extracting means for extracting a shielding object from an image acquired from a fixed camera;
To any of the plurality of management units obtained by dividing the space in which the fixed camera is provided at regular intervals, and shield position estimating means for shielding extracted by the obstacle extracting unit estimates whether the position,
A blind spot calculating means for calculating a management unit corresponding to the position of the blind spot of the fixed camera based on the estimation result of the shielding object position estimating means;
A monitoring control apparatus comprising: moving body control means for controlling a position and orientation of a moving body based on a management unit corresponding to the position of the blind spot of the fixed camera calculated by the blind spot calculation means.   The monitoring control apparatus according to claim 1, wherein the moving body control unit controls the position and orientation of the moving body with a continuous management unit corresponding to the position of the blind spot of the fixed camera as one blind spot. . The moving body control means, as a camera in which the movable body holds photographs the blind spot of the fixed camera, monitoring and control as claimed in claim 1 or 2, characterized in that to control the position and orientation of the movable body apparatus. The monitoring control apparatus according to claim 3 , further comprising reporting means for notifying the user of the fact that the blind spot cannot be imaged even if the position and orientation of the moving body is controlled. The moving body control means, when the blind spot there are a plurality based on the priority of the blind spot, in any one of claims 1-4, characterized by controlling the position and orientation of the movable body The monitoring and control device described. The moving body control means sets the priority of the blind spot to at least one of the size of the blind spot, the period during which the blind spot state continues, the situation immediately before the blind spot, and the importance of the position at which the blind spot is located. The monitoring control device according to claim 5 , wherein the determination is based on the above. A change accepting means for accepting a change in the priority of the blind spot by the user;
The moving body control means, based on the priority after change, monitoring and control device according to claim 5 or 6, characterized in that to control the position and orientation of the movable body. Monitoring and control device according to any one of claims 1 to 7, further comprising display means for displaying at least one of the position and location of the blind spot of the shield. A fixed camera that captures video,
A mobile object that is movable,
A monitoring system comprising: the monitoring control device according to any one of claims 1 to 8 that controls a position and orientation of the moving body based on an image acquired from the fixed camera. The moving body is
A person extracting means for extracting a person from the video of the camera possessed by the self;
The monitoring device according to claim 9 , further comprising notification means for notifying the person when the person extracted by the person extraction means is in the way and cannot capture the blind spot. system.

Images