JP2014002502A - Stretched-out hand detector, stretched-out hand detecting method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a stretched-out hand detector that analyzes distance information data based on two-dimensional distance data outputted from a distance image sensor installed above an oblique front side of an object (a merchandise display shelf or the like) front, and that detects a stretched-out hand of a person in a specified area of an object in the distance information data.SOLUTION: A stretched-out hand detector 1 analyzes distance information data based on two-dimensional distance data outputted from a distance image sensor 2, and detects a stretched-out hand of a person 4 toward an object 3. The stretched-out hand detector 1 includes: a detection area specifying means that specifies a detection area where stretched-out hand detection is performed in the distance information data; a person area calculating means that calculates a person area from the distance information data; an approach determining means that defines an area overlapped by the person area and the detection area as an approach area, whereas an area outside the approach area out of the person area is defined as a remote area; and a stretched-out hand determining means for determining the stretched-out hand of the person 4 from the approach area and the remote area.

Description

  The present invention relates to a hand extension detection technique for analyzing a two-dimensional distance data in which each pixel acquired by a single distance image sensor includes distance data and detecting a hand extension to an object such as a product shelf.

  Conventionally, behavior analysis systems that analyze consumer behavior in stores and the like have been developed. In a general behavior analysis system, a consumer in a store is photographed with a video camera or the like, and a consumer's movement or movement route is grasped by analyzing the photographed still image or moving image.

For example, Patent Document 1 discloses an invention in which a store is photographed with a camera, each consumer is identified by processing the camera image, and the movement trajectory of the identified consumer is analyzed.
However, the behavior analysis method using a camera image has a problem from the viewpoint of personal information protection because a consumer's face is reflected.

In Patent Document 2, RFID (Radio) such as a non-contact IC card and a non-contact IC tag is used.
An invention is disclosed in which a consumer's movement trajectory is analyzed by causing a consumer to hold a frequency identification), installing a detector that detects RFID in a store, and examining the detector that detects RFID in time series. Yes.

However, the behavior analysis method using RFID can recognize a consumer's approach to or contact with a product shelf or the like, but there is a problem that detailed behavior cannot be recognized.
Furthermore, in order to analyze the detailed behavior of consumers using RFID, it is necessary to install a large number of RFID detectors.

  In this regard, in Patent Document 3, using the distance image generated by converting the two-dimensional distance data output from the distance image sensor and the distance data included in the two-dimensional distance data into gradations, An invention for detecting forward bending is disclosed. Since this two-dimensional distance data is distance data from each distance image sensor to the object, personal information does not matter. In addition, since the two-dimensional distance data includes distance data measured by pixels arranged in two dimensions, the difference between the distance data included in the two-dimensional distance data is analyzed, so that the two-dimensional distance data exists before the product shelf. The consumer's area can be recognized, and detailed behavior analysis can be performed in the same manner as image processing. Furthermore, since accurate two-dimensional distance data can be obtained from one distance image sensor, it is not necessary to install a plurality of distance image sensors.

JP-A-9-330415 JP 2002-32550 A JP 2010-67079 A

  In the hand extension detection technique described in Patent Document 3, as shown in FIG. 16, the distance image sensor 2 is installed on the product shelf 3, and the person 4 (consumer) is imaged obliquely downward. And as shown in FIG. 17, when there exists a specific shape in the lower area | region of a distance image, it determines with the extension of the hand of the person 4. For this reason, even if a body part other than the hand (arm) is in the lower region, there is a problem that it is erroneously determined as a hand extension. In addition, there is a problem that it is difficult to determine where in the product shelf the hand is extended. Furthermore, since the hand extension is determined from only the consumer person area information without considering the position of the product shelf and the consumer, there is a problem that it is not known how close the hand is to the product shelf.

  The present invention has been made in order to solve such a problem, and by installing a distance image sensor on the upper front side of an object (such as a product shelf) and using distance data between the object and a person. An object of the present invention is to provide a technique for accurately detecting a person's hand extension in a specified region in a distance image of an object.

  In order to solve the above-described problem, a first invention is a hand extension detection device that analyzes distance information data based on two-dimensional distance data output from a distance image sensor and detects a person's hand extension with respect to an object. Then, in the distance information data, a detection area designating unit for designating a detection area for detecting a hand extension, a person area calculating unit for calculating a person area from the distance information data, and the person area and the detection area overlap. An approach determination unit that sets an area as an approach region, and a region other than the approach region of the person region as a remote region, and a hand extension determination unit that determines a palm of the person from the approach region and the remote region; It is a hand extension detection device characterized by comprising.

According to the first invention, it is possible to accurately detect the extension of the person's hand when the extended hand of the person overlaps the target object in the distance image, and the hand even if the body other than the hand overlaps the target object. It is possible to prevent false detections. In addition, by specifying the detection area, it is possible to accurately detect the position of the hand with respect to the object.
The “object” is an object for detecting the extension of a hand such as a product shelf or a wall.
The “distance information data” is two-dimensional distance data output from the distance image sensor or data obtained by processing the two-dimensional distance data. For example, the two-dimensional distance data is converted into image data of 256 gradations. This is a distance image.

In the first aspect of the present invention, it is preferable that the approach determination means sets a region that overlaps the detection region and has a difference in distance data smaller than a threshold among the person regions as the approach region.
This makes it possible to accurately detect the extension of the person's hand when the extension hand of the person overlaps the object in the distance information data.

In the first aspect of the present invention, it is desirable that the hand length determining means determines that the hand length is a predetermined length when a predetermined multiple of the maximum width of the approaching area is smaller than the maximum width of the remote area.
Accordingly, it is possible to accurately detect the palm of a person's hand and prevent erroneous detection of the palm of the hand even if a body other than the hand overlaps the object.

The detection area designating means in the first invention preferably designates a plurality of the detection areas in the distance information data.
Accordingly, it is possible to detect a hand extension in a plurality of regions of the object.

The hand extension detecting device according to the first aspect of the present invention further includes conversion means for converting the distance data included in the two-dimensional distance data output from the distance image sensor into a predetermined gradation to obtain the distance information data. It is desirable to have it.
Accordingly, the hand extension detection device can convert the two-dimensional distance data into distance information data that is, for example, a 256-tone distance image, and perform subsequent processing such as designation of a detection region.

The position where the distance image sensor according to the first invention is installed is preferably in front of the object, one side from the vertical center of the object and above the center in the horizontal direction. Alternatively, the position where the distance image sensor according to the first invention is installed is preferably in front of the object and outside the vertical outer end of the object and above the horizontal upper end.
Thereby, since the two-dimensional distance data can be imaged from above and obliquely in front of the object, the position of the person's hand can be accurately imaged with respect to the object.

The second invention is a hand extension detection method for analyzing the distance information data based on the two-dimensional distance data output from the distance image sensor and detecting a person's hand extension with respect to the object, wherein the distance information data includes: A detection area designating step for designating a detection area for performing a hand extension detection; a person area calculating step for calculating a person area from the distance information data; and an area where the person area and the detection area overlap with each other. An approach determining step in which an area other than the approaching area in the area is a remote area, and a hand extending determining step for determining the extension of the person's hand from the approaching area and the remote area. This is an extended detection method.
According to the second invention, when the hand extended by the person in the two-dimensional distance data overlaps with the object, the person's hand extension can be accurately detected, and even if a body other than the hand overlaps with the object It is possible to prevent false detection of a hand extension. In addition, by specifying the detection area, it is possible to accurately detect the position of the hand with respect to the object.

A third invention is a program for causing a computer to analyze a distance information data based on two-dimensional distance data output from a distance image sensor and to execute a hand extension detection method for detecting a person's hand extension with respect to an object. In the distance information data, the computer includes a detection area designating step for designating a detection area for detecting a hand extension, a person area calculating step for calculating a person area from the distance information data, the person area, An approach determination step in which an area that overlaps the detection area is an approach area, and an area other than the access area of the person area is a remote area, and a hand that determines the extension of the person's hand from the approach area and the remote area This is a program for executing the extension determination step.
By installing the third invention in a general-purpose computer, it is possible to obtain the palm detection device of the first invention and execute the palm detection method of the second invention.

  According to the present invention, a distance image sensor is installed above and obliquely in front of an object (such as a product shelf), and two-dimensional distance data between the object and a person is used to designate a specified area on the distance image of the object Therefore, it is possible to accurately detect the extension of a person's hand.

The figure which shows the example of application of a hand extension detection apparatus Schematic diagram showing distance image sensor installation status and field of view range The figure which shows the example of the two-dimensional distance data output from a distance image sensor The figure which shows the example of the distance image which imaged the person and the target object The figure explaining the hardware constitutions of a hand extension detection apparatus The flowchart which shows the pre-process of designation | designated of the detection area | region of a palm extension detection apparatus The flowchart which shows the process of hand extension determination of the hand extension detection device The flowchart which shows the detail of the division | segmentation process of the person area of a hand extension detection apparatus Schematic diagram showing the designation of the detection area of the hand extension detection device Schematic diagram showing the calculation of the human area of the hand extension detection device Schematic diagram showing examples of approaching area and remote area in person area Schematic diagram showing examples of the maximum width of the approaching area and the maximum width of the remote area in the person area The figure which shows the example of the distance image before performing a noise removal process The figure which shows the example of the distance image after the process using a spatial filter The figure which shows the example of the distance image after the process using area division The figure which shows the example of application of the conventional behavior analysis system The figure which shows the example of the distance image acquired with the conventional behavior analysis system

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
The palm detection device 1 of the present invention is widely applicable to distance information data. The distance information data is two-dimensional distance data output from the distance image sensor 2 or data obtained by processing the two-dimensional distance data. For example, the distance information data is data obtained by converting the two-dimensional distance data into image data of 256 gradations. is there. Data obtained by converting the two-dimensional distance data into image data of a predetermined gradation is called a distance image.
The distance image sensor 2 generally outputs two-dimensional distance data in which distance data to an object existing in the field angle range is two-dimensionally arranged.

  FIG. 1 is a diagram illustrating an application example of the palm detection device 1. FIG. 1 shows an object 3 (object to be extended by a person such as a product shelf or a wall), a person 4 (consumer) who extends the object 3, and an oblique front view of the object 3. The distance image sensor 2 installed on the pillar 5 or the like in the front and containing the object 3 and the person 4 within the angle of view, and the two-dimensional distance data or the two-dimensional distance data output from the distance image sensor 2 in 256 gradations The situation in the store comprised by hand extension detection device 1 which processes the distance picture generated by transforming is shown typically.

  As the distance image sensor 2, a commercially available distance image sensor can be used. The distance image sensor 2 may be, for example, a light wave ranging (Time of Flight) type such as D-Imager (Panasonic Corporation), SwissRanger (MESA Imaging), or pattern irradiation (Light) such as Kinect (Microsoft Corporation). Coding) type. The distance image sensor 2 of the light wave distance measuring type uses the time until the light irradiated from the LED of the distance image sensor is reflected on the object and returns to the distance image sensor, from the distance image sensor to the object. The distance is measured for each pixel arranged in two dimensions, and the distance information of each pixel is output at a predetermined frame rate (for example, 30 fps). The pattern irradiation type distance image sensor 2 reflects the optical pattern irradiated from the distance image sensor on an object and uses the geometric distortion of the pattern to arrange the distance from the distance image sensor to the object in two dimensions. It has a function of measuring each pixel and outputting distance information of each pixel at a predetermined frame rate (for example, 30 fps). Although the commercially available distance image sensor 2 includes a sensor having an image photographing function in addition to the distance measuring function, from the viewpoint of protecting personal information, the embodiment of the present invention uses only the distance measuring function. Do not use the image shooting function.

  FIG. 2 is a schematic diagram showing the installation state of the distance image sensor 2 and the field angle range. FIG. 2A corresponds to the view seen from the left in the horizontal direction in FIG. As shown in FIG. 2A, the distance image sensor 2 is installed above the broken line 21 indicating the horizontal center of the object 3 or above the broken line 22 indicating the upper end in the horizontal direction. The vertical angle of view of the distance image sensor 2 is, for example, 60 degrees, and the distance image sensor 2 is installed so that an area in which the extension of the hand of the object 3 is desired to be detected falls within this angle of view.

  FIG. 2B corresponds to a view seen from above in the vertical direction in FIG. As shown in FIG. 2B, the distance image sensor 2 is installed on one side of a broken line 23 indicating the center of the object 3 in the vertical direction, or outside the broken line 24 indicating the outer end in the vertical direction. The horizontal angle of view of the distance image sensor 2 is 45 degrees, for example, and the distance image sensor 2 is installed so that an area in which the extension of the hand of the object 3 is desired to be detected falls within this angle of view.

  In this way, by installing the distance image sensor 2, it is possible to accurately image the position of the hand 4 of the person 4 with respect to the object 3. In FIG. 2B, the distance image sensor 2 is installed on the right side toward the front of the object 3, but may be installed on the left side.

  FIG. 3 is a diagram illustrating an example of two-dimensional distance data output from the distance image sensor 2. In this example, the two-dimensional distance data is composed of 160 pixels in the horizontal direction × 120 pixels in the vertical direction. In each pixel, distance data from the distance image sensor 2 to the imaged object is expressed in hexadecimal. This may be an actual distance or may be converted to an actual distance by applying a predetermined proportional constant.

  FIG. 4 is an example of a distance image in which the distance image sensor 2 actually captures the object 3 and the person 4 when the distance image sensor 2 is installed as in FIGS. 1 and 2. In this distance image, each pixel is not the distance data itself but is converted into a value represented by a gradation difference of 256 gradations for each pixel for visualization. This can be generated by previously determining a distance corresponding to each gradation and converting the distance data of each pixel into a gradation.

  In the following embodiments, the hand extension detection device 1 may perform processing on the two-dimensional distance data output from the distance image sensor 2, and the two-dimensional distance data is converted to a predetermined gradation by the method described above. It is also possible to convert the converted distance image into a distance image that is image data of the image data, and perform the processing on the converted distance image.

[Hardware configuration of the palm detection device]
FIG. 5 is a hardware configuration diagram of the palm detection device 1. Note that the hardware configuration in FIG. 5 is merely an example, and various configurations can be employed depending on the application and purpose.

  A computer that implements the hand extension detection device 1 includes a control unit 11, a storage unit 12, a media input / output unit 13, a communication control unit 14, an input unit 15, a display unit 16, a peripheral device I / F unit 17, etc. Connected through.

  The control unit 11 includes a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), and the like. The CPU calls a program stored in the storage unit 12, ROM, recording medium, etc. to a work memory area on the RAM, executes it, drives and controls each device connected via the bus 18, and detects the palm detection device 1. The process to be described later is realized. The ROM is a non-volatile memory and permanently holds a computer boot program, a program such as BIOS, data, and the like. The RAM is a volatile memory, and temporarily stores programs, data, and the like loaded from the storage unit 12, ROM, recording medium, and the like, and includes a work area used by the control unit 11 for performing various processes.

  The storage unit 12 is, for example, an HDD (Hard Disk Drive), and stores a program executed by the control unit 11, data necessary for program execution, an OS (Operating System), and the like. As for the program, a control program corresponding to the OS and an application program for causing a computer to execute processing to be described later are stored. Each of these program codes is read by the control unit 11 as necessary, transferred to the RAM, and executed as various means.

  The media input / output unit 13 (drive device) inputs / outputs data, for example, media such as a CD drive (-ROM, -R, -RW, etc.), DVD drive (-ROM, -R, -RW, etc.) Has input / output devices. The communication control unit 14 includes a communication control device, a communication port, and the like, and is a communication interface that mediates communication between a computer and a network, and performs communication control between other computers via the network. The network may be wired or wireless.

  The input unit 15 inputs data and includes, for example, a keyboard, a pointing device such as a mouse, and an input device such as a numeric keypad. An operation instruction, an operation instruction, data input, and the like can be performed on the computer via the input unit 15. The display unit 16 includes a display device such as a CRT monitor and a liquid crystal panel, and a logic circuit (such as a video adapter) for realizing a video function of the computer in cooperation with the display device. The input unit 15 and the display unit 16 may be integrated like a touch panel display.

  The peripheral device I / F (interface) unit 17 is a port for connecting a peripheral device (for example, the distance image sensor 2) to the computer, and the computer is connected to the peripheral device (image) via the peripheral device I / F unit 17. Send / receive data to / from scanners, multifunction devices, etc. The peripheral device I / F unit 17 is configured by USB, IEEE 1394, RS-232C, or the like, and usually includes a plurality of peripheral devices I / F. The connection form with the peripheral device may be wired or wireless. The bus 18 is a path that mediates transmission / reception of control signals, data signals, and the like between the devices.

[Handle detection device processing]
Next, processing of the hand extension detection device 1 will be described along the flowcharts shown in FIGS. 6 to 8 with reference to FIGS. 9 to 15 as appropriate.

  First, preprocessing for designating a detection area in the palm detection device 1 will be described. This pre-processing may be performed once according to the object 3 that is the target of hand extension detection and the installation position of the distance image sensor 2. As shown in FIG. 1, the distance image sensor 2 is installed in front of the object 3 (such as a product shelf) obliquely in front of the front in advance. FIG. 6 is a flowchart showing pre-processing for designating the detection area of the palm detection device 1. FIG. 9 to be referred to as needed is a schematic diagram showing the designation of the detection area of the palm detection device 1.

  As illustrated in FIG. 6, the control unit 11 of the palm detection device 1 reads a background image from the distance image sensor 2 (S11). FIG. 9A shows an example of a background image. This background image is a distance image obtained by capturing the object 3 when the person 4 is not present. This background image may have a portion of the background 31 other than the object 3. The background image may be read from the storage unit 12 by the control unit 11 that has been captured in advance and stored in the storage unit 12.

  Next, the control unit 11 of the palm detection device 1 displays a background image on the display unit 16 (S12). This is to allow the user of the palm detection device 1 to specify the detection area 32 where the palm detection is desired in the object 3 of the background image.

  Next, the user designates a detection region 32 in which the hand extension is to be detected in the object 3 of the background image from the input unit 15 of the hand extension detection device 1 (S13). For this designation, coordinates can be input with a keyboard, an area can be designated with a mouse, or an area can be designated with a touch panel display.

FIG. 9B shows an example in which the detection area 32 (shaded portion) is designated in the background image. The detection area 32 is not particularly limited to a rectangle, and the shape is arbitrary as long as it is a closed area. Further, the user may designate a plurality of detection areas 32 for the object 3. When a plurality of detection areas 32 are designated, the following processing for the detection areas 32 is performed for each detection area 32.

Next, the control unit 11 of the palm detection device 1 creates a mask image from the background image and the designated detection area 32 (S14). Specifically, the control unit 11 leaves only the detection region 32 designated from the background image and removes the pixel values of other parts (for example, a value indicating the farthest value) as a mask image. create. FIG. 9C shows an example of a mask image.
Actually, since it is only necessary to have coordinates for determining the detection region 32 in the background image, the control unit 11 stores the coordinates of the detection region 32 in the next S15 without creating a mask image. May be.

Next, the control unit 11 of the palm detection device 1 stores the background image and the mask image in the storage unit 12 or the RAM (S15).
This is the end of the preprocessing.

  Next, a process of hand extension determination in the hand extension detection device 1 will be described. This process is performed every time the distance image sensor 2 captures a distance image. FIG. 7 is a flowchart showing the process of hand extension determination of the hand extension detection device 1. FIG. 10 to be referred to as needed is a schematic diagram showing calculation of the person area of the palm detection device 1. FIG. 11 is a schematic diagram illustrating an example of an approach area and a remote area in a person area. FIG. 12 is a schematic diagram illustrating an example of the maximum width of the approach area and the maximum width of the remote area in the person area.

  First, the control unit 11 of the palm detection device 1 reads an input image from the distance image sensor 2 (S21). As shown in the left diagram of FIG. 10, this input image is a distance image obtained by adding the person 4 to the background image (the object 3 and the background 31 other than the object 3) described above.

  Next, the control unit 11 of the palm detection device 1 reads the background image and the mask image from the storage unit 12 or the RAM (S22). These are created and stored by the preprocessing described above ((a) and (c) of FIG. 9).

  Next, the control unit 11 of the palm detection device 1 performs noise removal processing on the input image and the background image (S23). A conventional technique is used for the noise removal processing. For example, the control unit 11 executes two types of processing, that is, processing using a spatial filter and processing using region division.

  As a process using a spatial filter, for example, a smoothing process using a moving average method of 3 pixels × 3 pixels can be considered. Further, as a process using area division, for example, it is conceivable to divide an area by a 4-neighbor labeling process to remove noise whose area size is smaller than a certain value. By performing both processing using a spatial filter and processing using region division, noise can be accurately removed.

  13 to 15 are diagrams illustrating an example of noise removal processing for a distance image. FIG. 13 shows an example of a distance image before noise removal processing, FIG. 14 shows an example of a distance image after processing using a spatial filter, and FIG. 15 shows an example of a distance image after processing using region division. is there.

  Returning to the description of FIG. Next, the control unit 11 of the palm detection device 1 calculates the difference between the input image and the background image as the person region 33 (S24). Specifically, for each pixel of the input image, the control unit 11 compares the distance data with the distance data of the pixel having the same coordinates of the background image, and if they are different, the control unit 11 is a pixel constituting the person 4. Therefore, if it is the same, it is removed because it is the background (for example, a value indicating the farthest distance). Thereby, the control part 11 calculates the person area | region 33 in an input image.

  FIG. 10 is a schematic diagram of this process. If the difference in the above meaning between the input image in the left figure and the central background image is taken, the person area 33 in the right figure can be calculated.

  Returning to the description of FIG. Next, the control unit 11 of the palm detection device 1 divides the entire pixel of the person area 33 into an approach area and a remote area (S25). This is because the entire pixels of the person area 33 are within the detection area 32 and close to the object 3 (pixels included in the approach area and assumed to be “hands”) and others This is a process of dividing into (pixels included in the remote area). Details of this processing will be described with reference to FIG.

  FIG. 8 is a flowchart showing details of the person area dividing process of the palm detection device. First, the control unit 11 of the palm detection device 1 determines whether all the pixels in the person area 33 have been determined (S31). When the determination has been completed (Yes in S31), the control unit 11 ends the dividing process (S25 in FIG. 7).

  When the determination has not been completed (No in S31), the control unit 11 of the hand extension detection device 1 determines whether the coordinates (x, y) of the undetermined pixel P is within the detection area 32 of the mask image ( S32). If the control unit 11 determines that it is not within the detection area 32 (No in S32), P is set as a point constituting a remote area (S35), and the process returns to S31.

  When the control unit 11 of the hand extension detection device 1 determines that the position is within the detection region 32 (Yes in S32), the P distance data of the person region 33 and the pixel Q of the coordinate (x, y) of the mask image It is determined whether the difference from the distance data is smaller than the threshold value (S33). When the control unit 11 determines that it is not small (No in S33), P is set as a point constituting a remote area (S35), and the process returns to S31.

When the control unit 11 of the hand extension detection device 1 determines that the size is small (Yes in S33), P is set as a point constituting the approach region (S34) and the process returns to S31.
In this way, the control unit 11 divides the entire pixel of the person area 33 into the approach area and the remote area.

  FIG. 11 is a schematic diagram showing this state. As shown in FIG. 11A, it is assumed that the person 4 stands in front of the object 3 and extends his hand to the detection area 4 of the object 3. In this case, since the “hand” portion of the person area 33 is in the detection area 32 and is close to the object 3, the black-colored portion in FIG. The other part is the remote area 35.

  On the other hand, as shown in FIG. 11B, for example, it is assumed that the person 4 stands with the back facing the object 3 in front of the object 3. In this case, about half of the “body” of the person area 33 is in the detection area 32 and is close to the object 3. The other part is the remote area 35.

Returning to the description of FIG. Next, the control unit 11 of the palm detection device 1
Maximum width of approaching area x 5 <Maximum width of remote area (1)
Is determined (S26). The maximum width of the region here is the maximum value of the distance between two points included in the region when the pixel is a point constituting a two-dimensional plane. As a result, if the approach area is the “hand” part and the remote area is the “body” part, the above inequality is established, and it can be determined that the hand is stretched. The constant 5 is not limited to 5 as long as it is a value representing the ratio between the width of the hand and the height. For the sake of simplicity of processing, the maximum width may be the maximum width in the vertical direction or horizontal direction of the two-dimensional plane.

  FIG. 12 is a schematic diagram showing this state. In FIG. 12A, in the person area 33, the approach area 34 is a "hand" part and the remote area 35 is a "body" part. For large 37, the above equation (1) holds, and it is determined that the hand is open.

  On the other hand, in FIG. 12B, in the person area 33, the approach area 34 is about half of the “body”, and the remote area 35 is the other part. For the maximum width 37 of the remote area, the above equation (1) does not hold and it is not determined that the hand is open.

  Returning to the description of FIG. Next, when the control unit 11 of the palm detection device 1 determines Yes in S26, it determines “hand extension” (S27). Moreover, the control part 11 determines with "it is not a hand extension", when it is judged No at S26 (S28).

  Then, the control unit 11 of the palm detection device 1 displays the result of palm determination on the display unit 16 (S29), and the process ends.

Heretofore, the process of hand extension determination of the hand extension detection device 1 has been described.
In addition, the process of hand extension determination of the hand extension detection device 1 is not limited to this, and can be changed without departing from the spirit of the process.

  For example, it is assumed that the hand extension determination process is performed on the distance image generated from the two-dimensional distance data acquired from the distance image sensor 2 by the control unit 11, but the control unit 11 is previously acquired from the distance image sensor 2. A plurality of distance images (moving images) generated from the plurality of two-dimensional distance data are stored in the storage unit 12 or the RAM, and hand extension determination processing is performed on each distance image read from the storage unit 12 or the RAM. May be performed.

  Further, for example, in the process of dividing the entire pixel of the person area into the approach area and the remote area, in the flowchart shown in FIG. 8, all the pixels of the person area are determined in S31. The person area may be divided and the steps of S33, S34, or S35 may be performed on each pixel of the person area in the detection area.

  In addition, in the preprocessing for designating the detection area, a mask image is not created, and the control unit 11 stores the background image and the coordinates of the detection area in the storage unit 12 or the RAM in S15 of FIG. 7, the control unit 11 reads the background image and the coordinates of the detection area, and in FIG. 8, the control unit 11 uses the background image for which the coordinates of the detection area are specified when a mask image is used. It only has to be.

[Effect of the embodiment]
With the palm detection device 1 of the present embodiment, even when the person does not touch the object 3 (such as a product shelf) but appears to overlap the distance image, the palm determination can be performed accurately.
Even if a body other than the hand overlaps the object 3, it can be determined that the hand is not a hand.
In addition, by designating a portion of the object 3 that the user wants to know whether the hand has been extended as a detection region, it is possible to know which portion has been extended.

It should be noted that a sequence (moving image) that is considered to be easy to determine hand extension actually was captured, and the accuracy was confirmed by the hand extension detection device. As a situation where it is considered easy to perform hand extension judgment,
・ Hand extension to the white wall ・ Hand extension when the hand is not hidden by the body ・ Hand extension when the person and the object are clearly separated were confirmed.

  As a result, the detection accuracy correctly determined in the situation where the hand is actually extended is 86%, the detection accuracy correctly determined in the situation where the hand is not actually extended is 94%, and the detection accuracy of the entire sequence is 92%. It was a percentage.

  The present invention is an example of a behavior analysis system that detects a user's hand on a product shelf or the like, but can be widely applied to a system that detects the approach of an object to an object.

  As described above, the preferred embodiments of the palm detection device according to the present invention have been described with reference to the accompanying drawings, but the present invention is not limited to such examples. It will be apparent to those skilled in the art that various changes or modifications can be conceived within the scope of the technical idea disclosed in the present application, and these naturally belong to the technical scope of the present invention. Understood.

The distance image sensor 2 and the hand extension detection device 1 of the present invention analyze the two-dimensional distance data acquired from the distance image sensor, thereby allowing a product shelf, a sales promotion fixture, a touch panel terminal, etc. installed in a store or facility, etc. To understand consumer behavior.
As a feature of the survey using the present invention,
・ Consider consumer behavior quantitatively ・ Distance information is visualized and there is no problem of personal information ・ Consumer behavior can be shared through distance images.

1 Hand extension detection device 2 Distance image sensor 3 Object (product shelf, wall, etc.)
31 Non-target background 32 Detection area 33 Person area 34 Approach area 35 Remote area 36 Maximum width of approach area 37 Maximum width of remote area 4 Person (consumer, etc.)
5 pillars

Claims (8)

A hand extension detection device that analyzes distance information data based on two-dimensional distance data output from a distance image sensor and detects a person's hand extension with respect to an object,
In the distance information data, a detection area designating unit for designating a detection area for performing hand extension detection;
Human area calculation means for calculating a human area from the distance information data;
An approach determination means in which an area where the person area and the detection area overlap is an approach area, and an area other than the approach area of the person area is a remote area;
Hand extension determination means for determining the extension of the person's hand from the approach area and the remote area;
A hand detection device characterized by comprising: The approach determining means includes
The palm detection device according to claim 1, wherein an area that overlaps the detection area of the person area and that has a difference in distance data smaller than a threshold is defined as the approach area. The hand length determining means is:
The hand extension detection device according to claim 1, wherein the hand extension detection device determines that the hand extension is present when a predetermined multiple of a maximum width of the approach area is smaller than a maximum width of the remote area. The detection area designating means includes
4. The palm detection device according to claim 1, wherein a plurality of the detection areas are designated in the distance information data. 5. The hand extension detection device further includes:
2. The conversion device according to claim 1, further comprising conversion means for converting the distance data included in the two-dimensional distance data output from the distance image sensor into a predetermined gradation to obtain the distance information data. 4. The palm detection device according to 4. The position where the distance image sensor is installed is
6. The palm detection device according to claim 1, wherein the palm detection device is located in front of the object, on one side of the center of the object in the vertical direction and above the center of the object in the horizontal direction. A hand extension detection method for analyzing distance information data based on two-dimensional distance data output from a distance image sensor and detecting a person's hand extension with respect to an object,
In the distance information data, a detection area designating step for designating a detection area for performing hand extension detection;
A person area calculating step of calculating a person area from the distance information data;
An approach determination step in which an area where the person area and the detection area overlap is an approach area, and an area other than the approach area of the person area is a remote area;
A hand extension determination step of determining the extension of the person's hand from the approach area and the remote area;
A method for detecting a stretch of a hand, comprising: A program for causing a computer to analyze distance information data based on two-dimensional distance data output from a distance image sensor and to execute a hand extension detection method for detecting a person's hand extension with respect to an object,
In the computer,
In the distance information data, a detection area designating step for designating a detection area for performing hand extension detection;
A person area calculating step of calculating a person area from the distance information data;
An approach determination step in which an area where the person area and the detection area overlap is an approach area, and an area other than the approach area of the person area is a remote area;
A hand extension determination step of determining the extension of the person's hand from the approach area and the remote area;
A program for running