JP6382772B2 - Gaze guidance device, gaze guidance method, and gaze guidance program - Google Patents

Description

  The present invention relates to a technique for guiding a person's line of sight to a desired object.

  In recent years, telepresence robots (hereinafter referred to as robots) for communication with people have been studied in various fields. Non-Patent Document 1 describes a guidance robot as a technique for guiding a person.

  When a user needs guidance to a destination in a facility such as a hospital or hotel, the guidance robot moves in front of the user and the user walks following the guidance robot. In this way, you can reach your destination.

“Human follow-up behavior for guided robots”, Internet, <URL: https://www.jstage.jst.go.jp/article/aija/75/652/75_652_1399/_pdf>

  Non-Patent Document 1 is a robot for guiding (guidance) a user in a facility such as a hospital or a hotel where a user (person) needs guidance to a destination. That is, the guidance starts when the user recognizes that the destination to go to already exists, and the user recognizes that the robot guides the robot to the destination.

  On the other hand, when it is desired to guide a person's line of sight to an object that is not intended by the person, for example, a person who is walking on the road may want to be aware of digital signage, a signboard of a store, and the like. In this case, when the robot of Non-Patent Document 1 is applied, since the person does not recognize that the robot is a robot for performing guidance, the movement of the robot is started before the person pays attention to the robot. There is a problem that guidance is not successful.

  The present invention has been made in view of the above problems, and an object of the present invention is to guide a person's line of sight to an object that is not conscious of the person.

In order to solve the above-described problem, the present invention provides a gaze guidance device that guides a human gaze to a desired guidance destination, and the guidance destination is the gaze of the person when triggered by a gaze guidance start instruction. A first movement that moves within the range of the line of sight of the person when the destination is outside the range of the line of sight, and from within the range of the line of sight A movement control unit that performs a second movement that moves toward the guidance destination and does not perform the first movement and the second movement when the guidance destination is within the range of the line of sight. .
The present invention is a line-of-sight guidance device that guides a person's line of sight to a desired guidance destination, and moves into the range of the line of sight of the person triggered by an instruction to start guidance of the line of sight, and from within the range of the line of sight A movement control unit that moves toward the guidance destination; an imaging unit that captures an image including the guidance destination; and a communication unit that transmits the captured image to an operation terminal; and the communication unit includes a range of the captured image The start instruction transmitted by selecting the person information in the operation terminal that displays an image in which the person information existing outside is associated with a position in real space and superimposed on the captured image Receive.

The present invention is a line-of-sight guidance method performed by a line-of-sight guidance apparatus for guiding a person's line of sight to a desired guidance destination, and the guidance destination is out of the range of the person's line of sight triggered by an instruction to start guidance of the line of sight Determining whether it exists, a first moving step of moving within the line of sight of the person when the guide destination is outside the line of sight, and within the range of the line of sight of the person To the second moving step that moves toward the guidance destination, and when the guidance destination is within the range of the line of sight, the first moving step and the second moving step are not performed. .
The present invention is a line-of-sight guidance method performed by a line-of-sight guidance device that guides a person's line of sight to a desired guidance destination, an imaging step of imaging an image including the guidance destination, and transmission for transmitting the captured image to an operation terminal The person information is selected in the operation terminal that displays an image in which the person information existing outside the range of the captured image is associated with a position in real space and superimposed on the captured image. A reception step for receiving a start instruction transmitted by the step, a movement step for moving into the range of the line of sight of the person triggered by the start instruction, and moving from the range of the line of sight toward the guidance destination; ,I do.

  The present invention is a gaze guidance program that causes a computer to function as the gaze guidance device.

  According to the present invention, it is possible to guide a person's line of sight to an object that is not conscious of the person.

It is a lineblock diagram of a look guidance system in this embodiment. It is a flowchart which shows operation | movement of a gaze guidance apparatus. It is a figure which shows the gaze guidance apparatus which is outside a human gaze. It is explanatory drawing for demonstrating the moving direction of a person. It is a figure which shows the gaze guidance apparatus which moves into a gaze from the outside of a human gaze. It is a figure which shows the gaze guidance apparatus which performs the operation | movement which draws a person's attention within the range of a gaze. It is a figure which shows the gaze guidance apparatus which moves toward a guidance destination. It is a flowchart which shows operation | movement of a terminal. It is a figure which shows the positional relationship with the periphery with respect to a gaze guidance apparatus. It is a figure which shows the example of a display of a shortcut button. It is a figure which shows the example of selection of a shortcut button.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIG. 1 is a configuration diagram showing the configuration of the line-of-sight guidance system in the present embodiment. The line-of-sight guidance system includes a line-of-sight guidance device 1 and a terminal (operation terminal) 2. The line-of-sight guidance device 1 is a movement device (robot) that can be controlled to move at least, and is a device that guides the line of sight of a person (person) who wants to guide the line of sight to a desired target.

  The line-of-sight guidance device 1 illustrated includes a communication unit 11, a movement control unit 12, an operation unit 13, and an imaging unit 14.

  The communication unit 11 transmits / receives various information to / from the terminal 2. Specifically, the communication unit 11 receives a gaze guidance start instruction from the terminal 2. The communication unit 11 may transmit the captured image (camera image) captured by the imaging unit 14 and the information detected by the movement control unit 12 to the terminal 2.

  The movement control unit 12 moves within the range of the person's line of sight (field of view) upon receiving an instruction to start the line of sight guidance, and moves from the range of the line of sight toward the guidance destination. In addition, the movement control unit 12 includes various sensors, can detect its own position, and can detect the position and direction of the person to be guided and the target of the guidance destination. Moreover, the movement control part 12 shall acquire a person's gaze direction and range.

  The operation unit 13 performs an operation of drawing a person's attention within the range of the line of sight after the movement control unit 12 moves into the range of the line of sight of the person. Note that the action of drawing attention of a person includes at least one of outputting sound, emitting light, vibrating, and performing periodic motion, for example. In addition, the operation part 13 shall be provided with output devices, such as light-emitting devices, such as a speaker and a light, when outputting a sound or light.

  The imaging unit 14 includes a 360-degree digital camera having a front direction of imaging, and images an image (camera image) including a target of a guidance destination.

  For example, a telepresence robot, an unmanned aerial vehicle (drone) such as a small airplane or a helicopter may be used as the line-of-sight guidance device 1. Moreover, what is necessary is just a movement apparatus which comprises the function and mechanism mentioned above. Note that the communication path between the line-of-sight guidance device 1 and the terminal 2 may be wireless or wired.

  The terminal 2 is a device that can communicate with the line-of-sight guidance device 1, and for example, a terminal such as a smartphone, a tablet terminal, or a personal computer can be used.

  The illustrated terminal 2 is a terminal for remotely operating the line-of-sight guidance device 1, and includes a display unit 21 and a control unit 22.

  The display unit 21 includes a touch panel, and displays a captured image captured by the imaging unit 14 of the line-of-sight guidance device 1 on the touch panel. Also, a shortcut button (person information) that intuitively represents the position direction of the person (person who wants to guide) who is outside the range of the captured image and the distance to the person is displayed superimposed on the captured image.

  The control unit 22 receives an instruction from an operator who operates the terminal 2, and transmits a gaze guidance start instruction to the gaze guidance device 1.

  The above-described line-of-sight guidance device 1 and terminal 2 can use, for example, a general-purpose computer system including a CPU, a memory, an external storage device such as a hard disk, an input device, and an output device. In this computer system, each function of each unit is realized by the CPU executing a predetermined program loaded on the memory. For example, the functions of the visual guidance device 1 and the terminal 2 are executed by the CPU of the visual guidance device 1 in the case of the program for the visual guidance device 1 and by the CPU of the terminal 2 in the case of the program for the terminal 2, respectively. It is realized by doing.

  Further, the program for the line-of-sight guidance device 1 and the program for the terminal 2 can be stored in a computer-readable recording medium such as a hard disk, a flexible disk, a CD-ROM, an MO, or a DVD-ROM, or distributed via a network. You can also

  Next, operation | movement of the gaze guidance apparatus 1 of this embodiment is demonstrated.

  FIG. 2 is a flowchart showing the operation of the line-of-sight guidance device 1 of the present embodiment.

  The communication unit 11 of the line-of-sight guidance device 1 receives a line-of-sight guidance start instruction from the terminal 2 operated by the operator (S11). It is assumed that the start instruction specifies the person who wants to be guided and the target of the guidance destination.

  In the example shown in FIG. 3, the person 3 is a person who wants to guide, and the guide destination is the object 4. Here, the line of sight of the person 3 is directed in the opposite direction to the target 4 that is the guidance destination, and the line of sight of the person 3 does not include the target 4. In such a state, the line-of-sight guidance device 1 guides the line of sight (caution) of the person 3 to the target 4 to be guided.

  When the movement control unit 12 of the line-of-sight guidance device 1 receives the start instruction, the movement control unit 12 acquires the position of the person 3 and the position of the object 4, and also acquires the line-of-sight direction and range of the person (S12). The position can be acquired (measured) using various sensors such as a GPS, a position sensor, and a distance sensor.

  About a person's gaze direction and range, acquiring using the technique of the following literature 1 is considered, for example. In Document 1, it is assumed that the movement direction of a person is the line-of-sight direction, and the line-of-sight direction / range of the person is estimated using the movement direction.

Reference 1: Takahiro Matsumoto, Shunichi Seko, Ryosuke Aoki, Akihiro Miyata, Masayuki Ihara, Tomohiro Yamada, “Shoot Tag Mapping: Tag Location Map Generation Method for Indoor Location Information Services Using Smartphones,” IEICE Tech. 113, no. 470, MVE2013-64, pp. 13-18, March 2014 Assuming that the direction of movement of the person is the gaze direction, By measuring, as shown in FIG. 4, the two-dimensional position of the horizontal plane at a certain time point t is a vector P _t (X _t , Y _t ), and the previous position is a vector P _t-1 (X _{t −1} , Y _t−1 ), the movement direction of the person is expressed by the following unit vector.

図３に示すように、人の視線位置の高さをＭとすると（高さは、例えば日本人平均の視線の高さで近似）、人の視線方向は以下とおりである。

As shown in FIG. 3, when the height of the person's line-of-sight position is M (the height is approximated by, for example, the average Japanese line of sight), the line-of-sight direction of the person is as follows.

また、視線範囲は３次元位置(Ｘ_ｔ,Ｙ_ｔ,Ｍ) から最大視認距離ｒ以内の人の視線方向を中心としたθ_Ｖ・θ_Ｈで表される範囲である。なお、視線範囲θ_Ｖ・θ_Ｈは、一般的な人の視線範囲を用いる。例えば、移動時の人の視線角は、水平θ_Ｖ：８０度、垂直θ_Ｈ：６０度である。

The line-of-sight range is a range represented by θ _V · θ _H centered on the direction of the line of sight of a person within the maximum viewing distance r from the three-dimensional position (X _t , Y _t , M). Note that the visual line range θ _V · θ _H uses a general human visual line range. For example, the line-of-sight angle of a person at the time of movement is horizontal θ _V : 80 degrees and vertical θ _H : 60 degrees.

  Note that it is also possible to use the face direction based on the eye tracker result or the image processing result for the human gaze direction and range.

  Then, the movement control unit 12 uses the position of the person 3 to be guided, the line-of-sight direction and range of the person 3, and the position and direction of the guide-target 4 so that the guide-target 4 is outside the line of sight of the person 3. It is determined whether or not (S13). If the guidance target 4 is already within the range of the line of sight of the person 3, the movement control unit 12 does not need to guide, and thus ends the process without performing subsequent guidance.

  When the guidance target 4 is outside the line of sight of the person 3, the movement control unit 12 moves within the line of sight of the person 3 (S14).

  FIG. 5 is a diagram illustrating an operation in which the line-of-sight guidance apparatus 1 outside the line of sight of the person 3 illustrated in FIG. 1 moves within the range of the line of sight of the person 3. The movement control unit 12 of the line-of-sight guidance device 1 acquires the position of the line-of-sight guidance device 1 and moves from the acquired position to the range of the line of sight of the person 3 acquired in S12.

  And the movement control part 12 performs the operation | movement which draws the attention of the person 3 within the range of the line of sight of the person 3 (S15). For example, the movement control unit 12 outputs a sound for a predetermined time, emits light, vibrates up and down, and swings up and down, and swings periodically (spring motion, pendulum motion, eight motions). It is conceivable to perform at least one of character movement and the like. Thereby, the person 3 pays attention to the gaze guidance device 1 unconsciously.

  FIG. 6 is a diagram showing the line-of-sight guidance apparatus 1 performing an action (pendulum movement in the illustrated example) that draws the attention of the person 3 within the range of the line of sight.

  Then, the line-of-sight guidance device 1 moves so that the subject 4 falls within the range of the line of sight of the person 3 as the person 3 continues to look at the line-of-sight guidance device 1 (S16).

  FIG. 7 is a diagram illustrating an example of movement of the line-of-sight guidance device 1 in S16. In the example shown in the figure, a sector S1 having a predetermined angle θo centered on the person 3, the distance between the person 3 and the object 4 as a radius R1, and a line connecting the person 3 and the object 4 as one side is assumed. Then, an intersection T between a circle S2 centered on the person 3 and having a radius R2 between the person 3 and the line-of-sight guidance device 1 and the sector S1 is set as a target point.

  The movement control unit 12 of the line-of-sight guidance device 1 moves toward the target point T by moving on the circumference of a circle having a radius R2 between the person 3 and the line-of-sight guidance device 1. To do. The person 3 follows the line-of-sight guidance device 1 moving in this manner, so that the target 4 to be guided enters the field of view unconsciously (naturally). Note that the distance R2 between the person 3 and the line-of-sight guidance apparatus 1 is only a distance in the horizontal direction, and the circle S2 having a radius R2 between the person and the line-of-sight guidance apparatus 1 is horizontal to the ground. And

By moving on the circumference of a circle having a radius R2 between the person 3 and the line of sight guidance device 1, the line of sight guidance device 1 can move to the target point T without colliding with the person 3. . In addition, by setting the target point T not on the line connecting the person 3 and the object 4 but at a predetermined angle θo away from the line, the line-of-sight guidance device 1 at the target point T sees the object 4 as viewed from the person 3. It can be hidden. Thereby, the person can easily notice the target 4 to be guided. The predetermined angle θo is, for example, it is conceivable to line of sight angle horizontal theta _V smaller angles.

  Next, an example of processing in which the terminal 2 transmits a visual guidance start instruction to the visual guidance device 1 will be described. FIG. 8 is a flowchart showing the operation of the terminal 2.

  Note that the imaging unit 14 of the line-of-sight guidance apparatus 1 is in a position where the target 4 that is the guidance destination can be imaged, captures an image including the target 4, and transmits the captured image to the terminal 2 via the communication unit 11. Shall. Further, the movement control unit 12 of the line-of-sight guidance device 1 detects the position and direction of a person existing around him and transmits the detected information to the terminal 2 via the communication unit 11. And

First, the display unit 21 acquires the position direction and distance of the person 3 who wants to be detected detected by the line-of-sight guidance device 1 (S21). For example, when the person 3 is present at the position illustrated in FIG. 9 with respect to the line-of-sight guidance device 1, the rotation angle θ ₃ from the front direction of the camera of the imaging unit 14 (upward in the drawing) to the person 3 is clockwise, A distance L ₃ from the guidance device 1 to the person ₃ is acquired.

  Next, the display unit 21 generates a shortcut button that intuitively indicates the position direction of the person 3 and the distance to the person 3 using the acquired data, and displays the shortcut button superimposed on the captured image on the touch panel (S22). . Specifically, a button image (icon) representing the face is generated, the display size is changed according to the distance to the person 3, and displayed at a location corresponding to the position direction of the person 3. That is, the display unit 21 displays an image in which a shortcut button existing outside the range of the captured image is associated with a position in the real space and superimposed on the captured image.

For example, as shown in FIG. 9, the screen center of the touch panel is regarded as the current position of the line-of-sight guidance device 1 and the shortcut button SB ₃ of the person 3 is placed on the screen の in the clockwise direction θ ₃ ° with respect to the upward direction. indicate. As the button image, for example, a photographed face image of the subject may be used by reducing or enlarging it, or a prescribed face image may be used.

  Further, when there are a plurality of people detected by the line-of-sight guidance device 1, the display unit 21 superimposes and displays the shortcut buttons of the plurality of people on the captured image on the touch panel using the position direction and distance of each person. .

  When the pressing touch operation by the user is detected for the shortcut button being displayed, the display unit 21 transmits the ID of the pressed shortcut button to the control unit 22 (S23).

  The control unit 22 transmits a gaze guidance start instruction for guiding the gaze of the person 3 of the pressed shortcut button to the guidance target 4 to the gaze guidance device 1 (S24).

  Note that the gaze guidance start instruction transmission processing shown in FIG. 8 is an example, and the gaze guidance device 1 uses a different method to send a gaze guidance start instruction including the person 3 to be guided and the target 4 to be guided to the terminal. 2 may be received.

  In the present embodiment described above, the line of sight of the person is guided by the movement of the line-of-sight guidance device 1. That is, in this embodiment, the gaze guidance device 1 first enters the person's gaze in order to direct the person's gaze in the direction that he or she wants to face, and then directs the gaze toward the guidance target. Moving. As a result, the person follows the moving line-of-sight guidance apparatus 1 with his eyes, and the target of the guidance destination enters the field of view unconsciously (naturally). Therefore, in this embodiment, it is possible to smoothly guide a person's line of sight with respect to an object that the person is not conscious of.

  For example, when a person walking on the road wants to notice digital signage, a shop sign, a stadium gate, etc., the line-of-sight guidance device 1 enters the human field of view and moves toward the guidance destination. A person's line of sight and attention can be naturally directed to the destination.

  Further, in the present embodiment, the line-of-sight guidance device 1 performs an operation of drawing a person's attention after moving within the range of the line of sight of the person. Thus, even when the person does not recognize that the line-of-sight guidance device 1 is a device for performing guidance, the person naturally pays attention to the line-of-sight guidance device 1 and follows the line-of-sight guidance device 1 with his eyes. Become. Thereby, in this embodiment, a person's eyes | visual_axis can be more reliably guided to the object of a guidance destination.

  In the terminal 2 of the present embodiment, the shortcut button of the person who wants to guide is displayed on the captured image including the guide destination, and the operator touches the shortcut button, so that the start instruction of the eye guidance is given to the eye guidance device 1. Sent to. Thereby, the operator can easily input a start instruction, and the convenience for the user can be improved.

  In addition, this invention is not limited to the said embodiment, Many deformation | transformation and application are possible within the range of the summary.

DESCRIPTION OF SYMBOLS 1: Gaze guidance apparatus 11: Communication part 12: Movement control part 13: Operation | movement part 14: Imaging part 2: Terminal 21: Display part 22: Control part 3: Person who wants to guide 4: Target of guidance destination

Claims (7)

A gaze guidance device that guides a human gaze to a desired guidance destination,
In response to an instruction to start guiding the line of sight, it is determined whether or not the destination is outside the range of the line of sight of the person, and when the destination is outside the range of the line of sight, When the first movement that moves into the range of the line of sight and the second movement that moves from the range of the line of sight toward the guidance destination, and the guidance destination is within the range of the line of sight, A line-of-sight guidance device comprising a movement control unit that does not perform the first movement and the second movement . The line-of-sight guidance device according to claim 1,
An eye gaze guidance device, further comprising an operation unit that performs an action of drawing a person's attention within the gaze range after moving within the gaze range of the person. A gaze guidance device that guides a human gaze to a desired guidance destination,
In response to an instruction to start guiding the line of sight, the movement control unit moves into the range of the line of sight of the person and moves from the range of the line of sight toward the guidance destination;
An imaging unit that captures an image including a guidance destination;
A communication unit that transmits the captured image to the operation terminal ;
The communication unit displays the image superimposed on the captured image by associating the information on the person existing outside the range of the captured image with a position in real space, and the information on the person is selected. The line-of-sight guidance apparatus characterized by receiving the start instruction transmitted by A gaze guidance method performed by a gaze guidance device that guides a human gaze to a desired guidance destination,
Determining whether or not the guidance destination is outside the range of the line of sight of the person, triggered by an instruction to start guidance of the line of sight;
A first moving step of moving into the range of the person's line of sight when the guidance destination is outside the range of the line of sight; and a first movement step of moving from the range of the line of sight of the person toward the guide destination. And the second movement step is not performed when the guidance destination is within the range of the line of sight. A gaze guidance method performed by a gaze guidance device that guides a human gaze to a desired guidance destination,
An imaging step of imaging an image including the guidance destination;
A transmission step of transmitting the captured image to the operation terminal;
Sent when the person's information is selected in the operation terminal that displays an image superimposed on the captured image by associating the person's information existing outside the range of the captured image with a position in real space. A receiving step for receiving a start instruction;
Triggered by the start instruction, a movement step of moving into the range of the line of sight of the person and moving from the range of the line of sight toward the guidance destination is performed.
A gaze guidance method characterized by the above. A line-of-sight guidance program that causes a computer to function as the line-of-sight guidance device according to claim 1 . A gaze guidance program for causing a computer to function as the gaze guidance device according to claim 3.

Images