JP6972428B2 - Stay judgment device, stay judgment method and stay judgment program - Google Patents

Description

The present invention relates to a technique for determining whether or not a person is staying in a certain area from moving image data.

Patent Document 1 describes a technique for detecting a stationary person from moving image data. In Patent Document 1, the difference between the image data of the detection region and the background image data is obtained, and an attempt is made to extract a pixel region (hereinafter, human body pixel region) corresponding to the contour or region of a person from the difference image representing the difference. .. If the human body pixel region is extracted, it is determined that a person exists, and a representative position in the human body pixel region is obtained. Then, the distance at which the representative position is displaced at a predetermined time is compared with the threshold value, and the human behavior (stay, rest, movement, etc.) is determined.

Japanese Unexamined Patent Publication No. 2013-096947

In lighting control, in order to save energy, there is a demand to change the brightness depending on whether a person stays or moves without staying. For example, when a person stays, it is possible to provide the brightness necessary for work involving visual recognition, and when moving, it is possible to save energy by providing the minimum brightness necessary for moving. Be done. In order to realize this request, it is necessary to determine whether a person is staying or moving.

However, in the technique described in Patent Document 1, it should be determined that a person who is accompanied by movements such as organizing documents and putting on and taking off work clothes in an office that greatly moves the arm or body is staying. It will be determined that it is moving. In the technique described in Patent Document 1, when an action such as a large movement of an arm or a body is performed, the distance of displacement at a predetermined time becomes large and exceeds a threshold value, which is a cause of determination of movement.
An object of the present invention is to make it possible to appropriately determine whether a person is staying or moving.

The stay determination device according to the present invention is
A result acquisition unit that acquires detection results indicating the presence or absence of a stationary person and the presence or absence of a moving person for each of a plurality of partial areas in which the area included in the image data is divided.
A time measuring unit that measures the duration of the state indicated by the detection result acquired by the result acquisition unit that there is no stationary person and a moving person.
For each of the plurality of partial regions, a stay determination unit for determining whether or not a person is staying is provided based on the detection result and the duration measured by the time measurement unit.

In the present invention, a person stays based on the detection result of the presence / absence of a stationary person and the presence / absence of a moving person, and the duration of the state in which there is no stationary person and there is a moving person. It is determined whether or not it is. This makes it possible to appropriately determine whether a person is staying or moving even when there is a person who makes a movement that causes a large movement of the arm or body.

The block diagram of the stay determination apparatus 10 which concerns on Embodiment 1. FIG. The flowchart which shows the overall operation of the stay determination apparatus 10 which concerns on Embodiment 1. The flowchart of the determination process which concerns on Embodiment 1. The figure which shows the operation example of the stay determination apparatus 10 which concerns on Embodiment 1. FIG. The block diagram of the stay determination apparatus 10 which concerns on modification 1.

Embodiment 1.
*** Explanation of configuration ***
The configuration of the stay determination device 10 according to the first embodiment will be described with reference to FIG.
The stay determination device 10 is a computer.
The stay determination device 10 includes hardware of a processor 11, a storage device 12, a reception interface 13, and a transmission interface 14. The processor 11 is connected to other hardware via a signal line and controls these other hardware.

The processor 11 is an IC (Integrated Circuit) that performs processing. Specific examples of the processor 11 are a CPU (Central Processing Unit), a DSP (Digital Signal Processor), and a GPU (Graphics Processing Unit).

The storage device 12 is a device for storing data. Specific examples of the storage device 12 are a RAM (Random Access Memory), a ROM (Read Only Memory), and an HDD (Hard Disk Drive).
The storage device 12 includes an SD (registered trademark, Secure Digital) memory card, CF (CompactFlash, registered trademark), a NAND flash, a flexible disk, an optical disk, a compact disk, a Blu-ray (registered trademark) disk, and a DVD (Digital Versaille Disk). It may be a portable recording medium such as.

The receiving interface 13 and the transmitting interface 14 are interfaces for communicating with external devices such as an input device and an output device. Specific examples of the receiving interface 13 and the transmitting interface 14 are Ethernet (registered trademark), USB (Universal Serial Bus), and HDMI (registered trademark, High-Definition Multimedia Interface) ports. The receive interface 13 and the transmit interface 14 may be realized by one port.
As a specific example, the input device is a device such as an image sensor and an infrared sensor. The output device is, for example, a device such as a transmitter and a display device. The input device and the output device may be a communication chip or a NIC (Network Interface Card).

The stay determination device 10 includes a person detection unit 21 and a determination unit 22 as functional components. The determination unit 22 includes a result acquisition unit 23, a time measurement unit 24, and a stay determination unit 25 as functional components. The functions of each functional component of the stay determination device 10 are realized by software.
The storage device 12 stores a program that realizes the functions of each functional component of the stay determination device 10. This program is read by processor 11 and executed by processor 11. As a result, the functions of each functional component of the stay determination device 10 are realized.

In FIG. 1, only one processor 11 is shown. However, the number of processors 11 may be plural, and the plurality of processors 11 may execute programs that realize each function in cooperation with each other.

Information or data or signal values or variable values indicating the result of processing of each functional component are stored in the storage device 12 or the register or cache memory in the processor 11.

*** Explanation of operation ***
The operation of the stay determination device 10 according to the first embodiment will be described with reference to FIGS. 2 to 4.
The operation of the stay determination device 10 according to the first embodiment corresponds to the stay determination method according to the first embodiment. Further, the operation of the stay determination device 10 according to the first embodiment corresponds to the processing of the stay determination program according to the first embodiment.

The stay determination device 10 has a state in which a person is staying, a state in which a person is moving, or a state in which a person is absent for each of a plurality of partial areas in which an area included in the image data is divided. Determine if it is in a state.
In the first embodiment, staying means a state in which there is a stationary person in the partial area and a state in which a moving person stays for a certain period of time or longer. Moving means that a person moving in a partial area stays for less than a certain period of time and moves to another area. Absence means that there are no people in the partial area.

Therefore, the stay determination device 10 measures the duration of the state in which a moving person is present, determines that the person is staying when the duration is longer than the reference time, and determines that the person is staying when the duration is less than the reference time. Judged as a move. However, there is a possibility that a stationary person and a moving person exist at the same time in the target partial area. Therefore, the stay determination device 10 confirms that there is no stationary person when measuring the duration of the state in which there is a moving person. Therefore, the duration of the state in which there is a moving person is limited to the duration of the state in which there is no stationary person and there is a moving person.

The overall operation of the stay determination device 10 according to the first embodiment will be described with reference to FIG.
The stay determination device 10 executes processing by sequentially inputting image data of a plurality of frames constituting the moving image data. Here, it is assumed that the photographing device for photographing the moving image data is fixed, and the image data of each frame of the moving image data is obtained by photographing the same area.

(Step S1: Person detection process)
The person detection unit 21 divides the area included in the image data into a grid shape, and detects the presence / absence of a stationary person and the presence / absence of a moving person for each of the plurality of divided partial areas.
Specifically, the human detection unit 21 may detect the presence / absence of a stationary person and the presence / absence of a moving person for each of the plurality of partial regions by using the existing technology. For example, the human detection unit 21 may detect the presence or absence of a stationary person and the presence or absence of a moving person for each of a plurality of subregions by using the technique described in International Publication No. 2015-136828. It is possible.

(Step S2: Judgment process)
The determination unit 22 acquires the detection result, which is the result detected by the person detection unit 21 in step S1, and determines whether or not a person is staying in each of the plurality of partial regions.

The determination process (step S2 in FIG. 2) according to the first embodiment will be described with reference to FIG.
(Step S21: Result acquisition process)
The result acquisition unit 23 acquires the detection result, which is the result detected by the human detection unit 21 in step S1.

The processing of steps S22 to S30 is executed with each of the plurality of subregions as the target subregion.

(Step S22: Existence determination process)
The result acquisition unit 23 determines whether or not the detection result acquired in step S21 indicates that at least one of a stationary person and a moving person exists in the target partial region.
If neither the stationary person nor the moving person is present, the result acquisition unit 23 advances the process to step S23. On the other hand, the result acquisition unit 23 advances the process to step S25 when there is at least one of a stationary person and a moving person.

(Step S23: Reset process)
The time measuring unit 24 resets the duration of the target partial area to 0. Further, the time measuring unit 24 stops the measurement when the duration of the target partial region is measured. The duration indicates the duration of the state in which there is no stationary person and there is a moving person. Here, since no one is moving, the duration is reset to 0 and the duration measurement is stopped.

(Step S24: Absence processing)
The stay determination unit 25 determines that a person is absent in the target partial area.

(Step S25: Static determination process)
The result acquisition unit 23 determines whether or not the detection result acquired in step S21 indicates that there is a stationary person in the target partial region.
If there is a stationary person, the result acquisition unit 23 advances the process to step S26. On the other hand, the result acquisition unit 23 advances the process to step S28 when there is no stationary person.

(Step S26: Reset process)
The time measuring unit 24 resets the duration of the target partial area to 0. Further, the time measuring unit 24 stops the measurement when the duration of the target partial region is measured. Here, since there are no people who are stationary and no one is moving, the duration is reset to 0 and the measurement of the duration is stopped.

(Step S27: Stay processing)
The stay determination unit 25 determines that a person is staying in the target partial area.

(Step S28: Duration measurement process)
If the duration of the target partial region has not been measured, the time measurement unit 24 starts measuring the duration. That is, by the process of step S22 and the process of step S25, it is determined that the detection result indicates that there is no stationary person and there is a moving person. Therefore, the time measuring unit 24 measures the duration.
The time measurement unit 24 continues to measure the duration until the measurement is stopped by the process of step S23 or the process of step S26. Here, the processing is executed by inputting the image data of a plurality of frames constituting the moving image data in order. Therefore, by the process of step S22 and the process of step S25, the measurement of the duration is continued until the detection result does not indicate that there is no stationary person and there is a moving person.
The duration measured here is the time in the moving image data. That is, when the moving image data is 10 frames per second and the measurement of the duration is continued with the image data of 50 frames, the duration is 5 seconds.

(Step S29: Duration determination process)
The stay determination unit 25 determines whether or not the duration of the target partial area is equal to or longer than the reference time. The reference time may be a time common to all subregions or a different time for each subregion. For example, when the input moving image data is acquired by a camera or the like having a distorted lens, the way a person is reflected differs between the central portion and the edge portion of the image data. Therefore, if the time common to all the partial areas is used as the reference time, it may not be possible to accurately determine whether a person is staying or moving.
If the duration is equal to or longer than the reference time, the stay determination unit 25 advances the process to step S27. On the other hand, if the duration is less than the reference time, the stay determination unit 25 advances the process to step S30.

(Step S30: Move processing)
The stay determination unit 25 determines that a person is moving with respect to the target partial area.

An operation example of the stay determination device 10 according to the first embodiment will be described with reference to FIG.
FIG. 4 shows one example (movement example 1) determined to be moving and two examples (stay example) determined to be staying in a certain partial area of image data constituting the moving image data. 1 and 2) are shown. In FIG. 4, it is assumed that one scale is a frame acquisition cycle of moving image data, which is 200 milliseconds. That is, it is assumed that each frame of the moving image data is acquired every 200 milliseconds. Further, it is assumed that the reference time is 3000 milliseconds.

When inputting moving image data taken from a bird's-eye view from the ceiling, the human detection unit 21 divides each frame into a grid shape as shown in FIG. 4, and for each of the plurality of subregions obtained by the division. Detects the presence or absence of a stationary person and the presence or absence of a moving person.

When a person is moving, the same person does not stay in the same subregion. Therefore, as in the case of moving example 1, the duration of the state in which there is no stationary person and there is a moving person is less than the reference time, and it is determined that the person is moving.
When there is a person who is moving like carrying luggage in a narrow range, the same person remains in the same area, although the distance of displacement in time increases. Therefore, as in the case of stay 1, it is determined that the person is staying because the duration of the state in which there is no stationary person and there is a moving person is equal to or longer than the reference time. As in the case of stay 1, it is determined that the person is moving while the duration is less than the reference time, and it is determined that the person is staying when the duration is longer than the reference time.
When there is a stationary person, it is determined that the person is staying because the person is staying, as in the case of stay example 2.

The moving image data input by the stay determination device 10 may be moving image data taken with infrared rays or moving image data taken with visible light. Further, the moving image data input by the stay determination device 10 may be two-dimensionally arranged data acquired from a sensor or some device like an image.

*** Effect of Embodiment 1 ***
As described above, in the stay determination device 10 according to the first embodiment, the detection result of the presence / absence of a stationary person and the presence / absence of a moving person, and the person who is moving without a stationary person are included. It is determined whether or not a person is staying based on the duration of the state of being present. This makes it possible to appropriately determine whether a person is staying or moving even when there is a person who makes a movement that causes a large movement of the arm or body. As a result, it is appropriate to save energy by providing the brightness necessary for work involving visual recognition when a person stays and providing the minimum brightness necessary for moving when moving. It is possible to do it.

*** Other configurations ***
<Modification 1>
In the first embodiment, each functional component is realized by software. However, as a modification 1, each functional component may be realized by hardware. The difference between the first modification and the first embodiment will be described.

The configuration of the stay determination device 10 according to the first modification will be described with reference to FIG.
When each functional component is realized by hardware, the stay determination device 10 includes an electronic circuit 15 instead of the processor 11 and the storage device 12. The electronic circuit 15 is a dedicated circuit that realizes the functions of each functional component and the storage device 12.

Examples of the electronic circuit 15 include a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, a logic IC, a GA (Gate Array), an ASIC (Application Specific Integrated Circuit), and an FPGA (Field-Programmable Gate Array). is assumed.
Each functional component may be realized by one electronic circuit 15, or each functional component may be distributed and realized by a plurality of electronic circuits 15.

<Modification 2>
As a modification 2, some functional components may be realized by hardware, and other functional components may be realized by software.

The processor 11 and the electronic circuit 15 are referred to as a processing circuit. That is, the function of each functional component is realized by the processing circuit.

10 Stay determination device, 11 Processor, 12 Storage device, 13 Receive interface, 14 Transmission interface, 15 Electronic circuit, 21 Person detection unit, 22 Judgment unit, 23 Result acquisition unit, 24-hour measurement unit, 25 Stay determination unit.

Claims (6)

For each of the plurality of partial regions detected is divided to a size capable of regions included in the moving image data is human movement, acquires the detection result indicating the presence or absence of a person in motion and presence or absence of a person at rest Result acquisition department and
A time measuring unit that measures the duration of the state indicated by the detection result acquired by the result acquisition unit that there is no stationary person and a moving person.
For each of the plurality of partial areas, and the detection result, based on the has been the duration measured by the time measuring unit, a stay determination unit determines whether people are staying, stationary When the duration is longer than the reference time, which is longer than the acquisition cycle of the frames constituting the moving image data, for the partial area where the detection result indicates that there is no person who is moving and there is a person who is moving. Is a stay determination device including a stay determination unit that determines that a person is staying. The stay determination unit, the partial region indicating the detection result that the person who is stationary is moving no one is there, if the prior Symbol duration is less than the reference time, move a person The stay determination device according to claim 1. The stay determination device according to claim 1 or 2, wherein the stay determination unit determines that a person is staying in a partial area where the detection result indicates that there is a stationary person. The stay determination unit according to any one of claims 1 to 3 for determining that a person is absent in a partial area where the detection result indicates that there is no stationary person or a moving person. The described stay determination device. Result acquisition unit, for each of the plurality of partial regions detected is divided to a size capable of regions movement of people included in the moving image data and the presence or absence of a person in motion and presence or absence of a person at rest Get the detection result shown
The time measuring unit measures the duration of the state indicated by the detection result that there is no person who is stationary and there is a person who is moving.
Stay determination unit, for each of the plurality of partial regions, based on said detection result and the duration, it is determined whether or not a person is staying, who person at rest is moving no one is For the partial area indicated by the detection result, if the duration is longer than the reference time, which is longer than the acquisition cycle of the frames constituting the moving image data, it is considered that a person is staying. stay determination method to determine. For each of the plurality of partial regions detected is divided to a size capable of regions included in the moving image data is human movement, acquires the detection result indicating the presence or absence of a person in motion and presence or absence of a person at rest Result acquisition process and
A time measurement process that measures the duration of the state indicated by the detection result acquired by the result acquisition process that there is no person who is stationary and there is a person who is moving.
For each of the plurality of partial areas, and the detection result, based on the has been the duration measured by the time measuring process, a stay determination process of determining whether a person is staying, stationary When the duration is longer than the reference time, which is a time longer than the acquisition cycle of the frame constituting the moving image data, for the partial area where the detection result indicates that there is no person who is moving and there is a person who is moving. Is a stay determination program that makes a computer function as a stay determination device that performs a stay determination process for determining that a person is staying.

Images