JP2014126997A - Operation device, and operation detection method - Google Patents

Abstract

An object of the present invention is to enable easy and free operation without using a hand.
A line-of-sight detection device detects a driver's gazing point and position coordinates of left and right eyes of the driver on the photographic image based on a photographed image of the driver's face from below, and the position coordinates and gazing point. Is provided to the in-vehicle device. Further, the in-vehicle device displays an operation screen on which an object such as an icon is drawn on the display device, and based on the position coordinates acquired from the line-of-sight detection device, the interval between the vertical positions of the left and right eyes on the captured image Based on this, the behavior of the driver tilting the face in the horizontal direction is detected. If the driver is gazing at the same object when the behavior is made, it is assumed that an operation corresponding to the object has been performed.
[Selection] Figure 4

Description

  The present invention relates to an operation device or the like that receives an operation from a user's line of sight.

  Conventionally, in an electronic device such as an in-vehicle device or a portable device, a button or the like is used as an interface between the device and a user. However, in recent years, many devices using a touch panel arranged on a display are seen.

  By using such a touch panel, an operation can be performed by touching an icon or the like displayed on the display, and a more intuitive operation is possible. However, the physical burden of performing the operation with the fingertip is not reduced, and there is a problem that the operation cannot be performed freely when both hands are closed, for example, during driving.

  On the other hand, Patent Document 1 describes a system that detects an operation based on a user's line of sight. The system detects the coordinates of the position on the screen that is on the user's line of sight (line-of-sight coordinates), receives a coordinate confirmation instruction from the user, and performs processing corresponding to the line-of-sight coordinate when the coordinate confirmation instruction is received Run.

Japanese Laid-Open Patent Publication No. 5-108251

  According to the system described in Patent Document 1, since an operation can be selected by line of sight, a troublesome key operation is not necessary, but an operation for confirming the selection of the operation needs to be performed by hand, and both hands are still required. It is not possible to operate freely when is closed.

  The present invention has been made in view of the above-described problems, and an object thereof is to provide an operation device and the like that can be operated easily and freely without using a hand.

  The operation device according to claim 1, which has been made to solve the above problem, is based on a display unit that displays an image on a screen of a display device, and a screen on which the user is gazing based on a captured image of the user's face. An identification means for identifying a point of sight, a detection means for detecting the behavior of moving the entire user's head based on the photographed image, and the behavior when the user's behavior is detected by the detection means. Operating means that considers that an operation corresponding to the gazing point specified by the specifying means has been performed.

  According to such a configuration, when the user moves the entire head in a predetermined manner, such as turning the face in a predetermined direction while gazing at a specific position on the screen of the display device, An operation corresponding to the gaze point is detected. For this reason, after selecting an operation by gazing at the screen, the user can confirm the selection by an extremely simple behavior of moving the entire head, and can perform the operation easily and freely without using a hand. Can do.

  In addition, as described above, an electronic device that can perform various operations by arranging a touch panel on a display and touching an object such as an icon displayed on the screen is known. According to this, a more intuitive operation is possible.

  Therefore, in the operation device according to claim 2, the display means displays an image on which one or a plurality of objects are drawn on the screen of the display device, and the operation means detects the behavior of the user by the detection means. When the gaze point specified by the specifying unit when the behavior is performed is in an area where any one object is drawn, it is considered that an operation corresponding to the object has been performed.

  According to such a configuration, when the user moves the entire head in a predetermined manner while viewing a specific object on the screen, an operation corresponding to the object is detected. For this reason, the user can confirm the selection of the object by gaze by an extremely simple behavior of moving the entire head, and can perform a simple and free operation more intuitively without using a hand. .

It is a block diagram which shows the structure of the vehicle-mounted system in 1st Embodiment. It is explanatory drawing of the arrangement position of the camera with which the gaze detection apparatus of the vehicle-mounted system in 1st Embodiment is provided. It is explanatory drawing about the process which detects direction of the face of the driver in 1st Embodiment. It is explanatory drawing about the process which confirms operation according to the operation confirmation behavior of the driver in 1st Embodiment. It is explanatory drawing about an example of the detection method of the operation confirmation behavior of the driver in 1st Embodiment. It is explanatory drawing about an example of the detection method of the operation confirmation behavior of the driver in 1st Embodiment. It is explanatory drawing about an example of the detection method of the operation confirmation behavior of the driver in 1st Embodiment. It is explanatory drawing about an example of the detection method of the operation confirmation behavior of the driver in 1st Embodiment. It is a flowchart of the operation detection process in 1st Embodiment. It is a flowchart of the operation detection process in 1st Embodiment. It is a block diagram which shows the structure of the portable apparatus in 2nd Embodiment. It is the schematic which shows the external appearance of the portable apparatus in 2nd Embodiment.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. The embodiment of the present invention is not limited to the following embodiment, and can take various forms as long as they belong to the technical scope of the present invention.

[First Embodiment]
[Description of configuration]
FIG. 1 is a block diagram illustrating a configuration of an in-vehicle system 1 according to the first embodiment. The in-vehicle system 1 includes an in-vehicle device 10 that displays various images on the display device 20, and a line-of-sight detection device 30 (eye tracker) that detects a position or the like where the driver is gazing based on a captured image of the driver's face.

  The line-of-sight detection device 30 includes a light projecting unit that projects infrared rays toward the driver's face, a camera 31 that captures the driver's face, and the like. Analyzing the eye image. Then, the line-of-sight direction of the driver is detected from the analysis result, and the coordinates (gaze point) of the position on the screen of the display device 20 being watched by the driver are detected based on the line-of-sight direction. The camera 31 is disposed at a position adjacent to the meter of the host vehicle in a state tilted upward by a predetermined angle from the horizontal direction, and images the driver's face from a viewpoint closer to the lower side than the front of the face (see FIG. 2).

  The in-vehicle device 10 includes a CPU, a ROM, a RAM, an I / O, and the like. The in-vehicle device 10 outputs video signals to the control unit 12 that performs overall control of the in-vehicle device 10 and the display device 20. And a video signal output unit 11 for displaying an image.

  The control unit 12 is configured to generate various image data, and outputs a video signal based on the generated image data to the display device 20 via the video signal output unit 11, so that the display device 20 Display various images on the screen.

[Description of operation]
(1) About operation | movement outline | summary Next, the outline | summary of operation | movement of the vehicle-mounted apparatus 10 is demonstrated.

  The in-vehicle device 10 generates image data of an operation screen for accepting an operation on the own device, and causes the display device 20 to display the operation screen. One or a plurality of objects (icons, etc.) are drawn on this operation screen, and the in-vehicle device 10 behaves in a manner that the driver inclines the face sideways while gazing at any object (in other words, The operation corresponding to the object is detected in response to the behavior of shaking the face in the horizontal direction (hereinafter, this behavior is also referred to as an operation confirmation behavior).

  Note that, for example, a behavior in which the driver tilts the face in the upward or downward direction, or a behavior in which the face moves back and forth may be considered as an operation confirmation behavior. However, there is a possibility that such a behavior is unexpectedly performed by the driver, and the intention of the driver can be more accurately reflected by setting the behavior of tilting the face in the horizontal direction as the operation confirmation behavior. Further, for example, the behavior of the entire other head may be detected as the operation confirmation behavior, such as a behavior of repeatedly shaking the entire head left and right or up and down.

  The line-of-sight detection device 30 detects the position coordinates of the left and right eyes of the driver on the photographed image and the gazing point on the operation screen based on the photographed image of the driver's face in response to an instruction from the vehicle-mounted device 10. Provided to the apparatus 10.

  The captured image of the driver's face is viewed from a position closer to the lower side than the front of the face, and is projected with the driver's face slightly looked up. Therefore, when the position coordinate of the left eye of the driver is L (lx, ly) and the position coordinate of the right eye is R (rx, ry), the vertical direction (y-axis direction) is obtained when the driver's face is facing directly in front. The difference Δd = ry−ly, which is the interval between the left and right eye positions, is a value near 0 (see FIG. 3). As the driver's face is tilted laterally, the absolute value of Δd increases.

  Therefore, the control unit 12 of the in-vehicle device 10 that acquired the position coordinates L and R calculates Δd based on this, detects the driver's face orientation based on Δd, and further determines the operation confirmation behavior based on the face orientation. Is detected.

  Of course, in addition to this, for example, it is conceivable to detect the orientation of the face using a photographed image of the driver's face from the front or above, a photographed image of the entire head, or the like. In addition to the position coordinates of the driver's eyes, for example, it is conceivable to detect feature points from the driver's face image and detect the face orientation based on the position of the feature points.

  However, by paying attention to the position of the driver's eyes, the direction of the driver's face can be detected more easily. In addition, by using an image of the driver's face taken from below and paying attention to the distance between the vertical positions of the left and right eyes, the face orientation can be detected easily and accurately. The processing load when detecting the behavior can be reduced.

  4 and 5, the control unit 12 displays the operation screen 100 on the display device 20, and then based on the position coordinates L and R, the absolute value | Δd ( t) | Then, when | Δd (t) | becomes equal to or greater than a predetermined threshold value D1, the control unit 12 sets the time at that time as Tsta. Further, the control unit 12 specifies an object on the operation screen 100 that the driver is gazing at time Tsta based on the gazing point acquired from the line-of-sight detection device 30, and highlights the object 101 on the display device 20. You may let them.

  Thereafter, the control unit 12 periodically calculates | Δd (t) |, and when | Δd (t) | becomes equal to or greater than the threshold value D2 by the time Tend (before a predetermined time elapses from the time Tsta). It is assumed that the operation confirmation behavior has been made.

  Further, during the detection of the operation confirmation behavior, the control unit 12 specifies the object on the operation screen on which the driver is gazing based on the gazing point acquired from the line-of-sight detection device 30. When the driver is gazing at the same object 101 (the highlighted object 101) until the operation confirmation behavior is detected, the control unit 12 performs an operation corresponding to the object 101. The processing corresponding to the object 101 is executed. At this time, the control unit 12 may further change the display mode of the object 101.

  In addition, the operation confirmation behavior may be detected as follows. That is, after the time Tsta, the control unit 12 periodically calculates | Δd (t) |, calculates an integrated value of | Δd (t) | starting from the time Tsta, and performs the integration until the time Tend. When the value is equal to or greater than the threshold value D3, it may be considered that the operation confirmation behavior has been made (see FIG. 6).

  Further, during the detection of the operation confirmation behavior, the control unit 12 may similarly specify an object on the operation screen that the driver is watching. Then, when the driver is gazing at the same object 101 until the operation confirmation behavior is detected, the control unit 12 may consider that an operation corresponding to the object 101 has been performed.

  In addition to this, the control unit 12 detects the behavior of tilting the face to the right (right operation confirmation behavior) and the behavior of tilting to the left (left operation confirmation behavior) as different operation confirmation behaviors. May be.

  As described above, since Δd (t) = ry−ly, when the driver tilts his face to the right, Δd (t) has a negative value, and when he tilts his face to the left , Δd (t) is a positive value. For this reason, the direction in which the driver tilted the face can be determined based on the sign of Δd (t), and the control unit 12 can detect the right and left operation confirmation behavior based on Δd (t).

  Therefore, when the right operation confirmation behavior is detected while the driver is gazing at any object, the control unit 12 detects the operation R corresponding to the object, and the left operation confirmation behavior is detected. In this case, the operation L corresponding to the object may be detected.

  More specifically, regarding the left-side operation determination behavior, when Δd (t) is equal to or greater than a predetermined threshold DL1, the control unit 12 determines the time (time Tsta) based on the gazing point acquired from the line-of-sight detection device 30. The object on the operation screen that the driver is gazing at is specified (see FIG. 7).

  Thereafter, the control unit 12 periodically calculates | Δd (t) |, and if | Δd (t) | is equal to or greater than the threshold DL2 by time Tend, the left-side operation confirmation behavior is assumed to have been performed. I reckon. If the driver is gazing at the same object during the detection of the left-side operation determination behavior, the control unit 12 regards that the operation L corresponding to the object has been performed, and corresponds to the object and the operation L. Execute the process.

  On the other hand, regarding the right operation confirmed behavior, as in the left operation confirmed behavior, when Δd (t) is equal to or less than a predetermined threshold DR1, the control unit 12 identifies the object that the driver is watching (see FIG. 7). ). Thereafter, when | Δd (t) | becomes equal to or less than the threshold value DR2 by time Tend, the control unit 12 considers that the right operation confirmation behavior has been performed.

  If the driver is gazing at the same object during the detection of the right-side operation determination behavior, the control unit 12 regards that the operation R corresponding to the object has been performed, and corresponds to the object and the operation R. Execute the process.

  Regarding the left-side operation confirmation behavior, the control unit 12 periodically calculates Δd (t) after time Tsta when Δd (t) becomes equal to or greater than the threshold value DL1, and Δd (t) starting from time Tsta. The integral value of may be calculated. Then, when the integrated value becomes equal to or greater than the threshold value DL3 by time Tend, it may be considered that the left operation confirmation behavior has been made (see FIG. 8).

  Similarly, the control unit 12 may calculate the integral value of Δd (t) starting from the time Tsta after the time Tsta when Δd (t) becomes equal to or less than the threshold value DR1 with respect to the right-side operation confirmation behavior. . Then, when the integrated value becomes equal to or less than the threshold value DR2 by time Tend, it may be considered that the left operation confirmation behavior has been performed (see FIG. 8).

  Of course, during the detection of the left or right operation confirmation behavior by the above method, the control unit 12 similarly identifies the object on the operation screen that the driver is watching, and the driver is watching the same object during the detection. In this case, it is needless to say that the operation L or R has been performed.

By detecting the operation of the driver in this manner, the driver can easily and freely operate the in-vehicle device 10 even when both hands are closed by driving.
(2) Operation detection processing Next, as an example, after time Tsta, an integrated value of | Δd (t) | is calculated, and it is determined whether or not the integrated value is equal to or greater than a threshold value D3. The operation detection process for detecting the behavior will be described with reference to the flowcharts shown in FIGS. This process is executed when the in-vehicle device 10 starts accepting an operation from a driver or the like.

In S200, the control part 12 of the vehicle-mounted apparatus 10 displays an operation screen on the display apparatus 20, and transfers a process to S205.
In S205, the control unit 12 instructs the line-of-sight detection device 30 to detect the position coordinates L and R of the left and right eyes of the driver. In the line-of-sight detection device 30, the position coordinates L and R are detected according to the instruction, and when the detection is successful, the position coordinates L and R are provided to the in-vehicle device 10.

  In subsequent S210, the control unit 12 determines whether or not the position coordinates L and R have been successfully detected and acquired. If a positive determination is obtained (S210: Yes), the control unit 12 shifts the process to S205, and if a negative determination is obtained (S210: No), the control unit 12 shifts the process to S215. .

In S215, the control unit 12 specifies the position coordinates of the left and right eyes of the driver, calculates | Δd (t) | described above based on these coordinates (S220), and shifts the processing to S225.
In S225, the control unit 12 determines whether or not | Δd (t) | is greater than or equal to the threshold value D1, and when an affirmative determination is obtained (S225: Yes), the process proceeds to S230 and a negative determination is made. Is obtained (S225: No), the process proceeds to S205.

  In S230, the control unit 12 instructs the line-of-sight detection device 30 to detect the driver's point of gaze. The line-of-sight detection device 30 detects a gazing point according to the instruction, and when the detection is successful, the gazing point is provided to the in-vehicle device 10.

  In subsequent S235, the control unit 12 determines whether or not the gaze point has been successfully detected and the gaze point has been acquired. Then, when an affirmative determination is obtained (S235: Yes), the control unit 12 proceeds to S240, and when a negative determination is obtained (S235: No), the control unit 12 proceeds to S230. .

In S240, the control unit 12 sets the current time as Tsta, determines the provided gaze point as the gaze point at time Tsta, and shifts the processing to S245.
In S <b> 245, the control unit 12 specifies an object (an object being watched by the driver) drawn in an area including the gazing point on the operation screen displayed on the display device 20, and the object is displayed on the display device 20. Is highlighted, and the process proceeds to S250. Note that if there is no corresponding object, the control unit 12 may shift the process to S205 again.

  In S250, the control unit 12 determines whether or not the current time t has reached Tend (whether or not a predetermined time has elapsed from time Tsta). If a positive determination is obtained (S250: Yes), the control unit 12 shifts the process to S255, and if a negative determination is obtained (S250: No), the control unit 12 shifts the process to S310. .

In S255, the control unit 12 instructs the line-of-sight detection device 30 to detect the driver's gaze point, and the process proceeds to S260.
In S <b> 260, the control unit 12 determines whether the gaze point has been successfully detected and the gaze point has been acquired from the line-of-sight detection device 30. When a positive determination is obtained (S260: Yes), the control unit 12 shifts the process to S265, and when a negative determination is obtained (S260: No), the control unit 12 shifts the process to S255. To do.

In S265, the control unit 12 determines the provided gazing point as the gazing point of the current driver, and the process proceeds to S270.
In S270, the control unit 12 specifies an object (an object that the driver is gazing at) drawn in an area including the above-mentioned gazing point on the operation screen, and the object and the highlighted object (at time Tsta). It is determined whether or not (objects being watched) match. Then, when an affirmative determination is obtained (S270: Yes), the control unit 12 proceeds to S275, and when a negative determination is obtained (S270: No), the control unit 12 proceeds to S310. .

In S275, the control unit 12 instructs the line-of-sight detection device 30 to detect the position coordinates L and R of the left and right eyes of the driver, and the process proceeds to S280.
In S280, the control unit 12 determines whether or not the position coordinates L and R have been successfully detected and have been acquired from the line-of-sight detection device 30. If an affirmative determination is obtained (S280: Yes), the control unit 12 shifts the process to S285, and if a negative determination is obtained (S280: No), the control unit 12 shifts the process to S275. .

  In S285, the control unit 12 specifies the position coordinates of the left and right eyes of the driver, calculates | Δd (t) | described above based on these coordinates (S290), and shifts the processing to S295.

In S295, the control unit 12 calculates an integral value of | Δd (t) | from time Tsta to the present time, and proceeds to S300.
In S300, the control unit 12 determines whether or not the integrated value of | Δd (t) | is equal to or greater than the threshold value D3. Then, when an affirmative determination is obtained (S300: Yes), the control unit 12 proceeds to S305, and when a negative determination is obtained (S300: No), the control unit 12 proceeds to S250. .

  In S305, the control unit 12 considers that an operation corresponding to the identified object (in other words, an object that was being watched by the driver while the operation confirmation behavior was performed) was performed, and responds to the operation. The process is executed and the present process is terminated.

  On the other hand, in S310, which shifts to the case where the current time t reaches Tend, the control unit 12 causes the display device 20 to release the highlighted display of the object being watched at time Tsta, and ends this processing. To do.

  Note that after the time Tsta, if it is a case where an operation confirmation behavior is detected by determining whether or not | Δd (t) | is equal to or greater than the threshold value D2, the control unit 12 determines that | Δd (t ) | Instead of determining the integral value, it may be determined whether | Δd (t) | is equal to or greater than the threshold value D2.

  If the right and left operation fixed behaviors are detected as different operation fixed behaviors, the control unit 12 calculates Δd (t) instead of | Δd (t) | in S220 and S290, In S295, instead of the integrated value of | Δd (t) |, the integrated value of Δd (t) may be calculated.

  In S225, the control unit 12 may compare Δd (t) with the threshold value DR1 or DL1 instead of comparing | Δd (t) | (specifically, Δd (t) is If it is a positive value, it is determined whether or not Δd (t) is greater than or equal to the threshold value DL1, and if it is a negative value, it may be determined whether or not Δd (t) is less than or equal to the threshold value DR1.

  In S300, the control unit 12 may compare Δd (t) with the threshold value DR2 or DL2 instead of comparing the integrated value of | Δd (t) | (specifically, Δd ( If t) is a positive value, it is determined whether or not Δd (t) is greater than or equal to the threshold DL2, and if it is a negative value, it may be determined whether or not Δd (t) is less than or equal to the threshold DR2. .

  In S300, the control unit 12 may compare the integrated value of Δd (t) with the threshold value DR3 or DL3 instead of comparing the integrated value of | Δd (t) | If the integrated value of Δd (t) is a positive value, it is determined whether the integrated value of Δd (t) is greater than or equal to the threshold value DL3. If the integrated value of Δd (t) is a negative value, the integrated value of Δd (t) is the threshold value. It may be determined whether or not DR3 or less).

[Second Embodiment]
FIG. 11 is a block diagram illustrating a configuration of a mobile device 50 (for example, a smartphone) according to the second embodiment. The portable device 50 includes a display unit 51 that displays various images, a CPU, a ROM, a RAM, an I / O, and the like. The portable device 50 is based on a control unit 52 that performs overall control of the portable device 50 and a photographed image of the user's face. A line-of-sight detection unit 53 (eye tracker) that detects a position where the user is gazing.

  As shown in FIG. 12, the display unit 51 has a substantially rectangular screen 51 a, and the screen 51 a is arranged on the surface of the portable device 50 having a rectangular plate shape, One of the short sides in the screen 51a is the upper side and the other is the lower side. The portable device 50 is configured to be used in a state where the upper short side of the screen 51a is positioned upward.

  The line-of-sight detection unit 53 includes a light projecting unit that projects infrared rays toward the user's face, a camera 53a that captures the user's face, and the like, and specifies the position of the user's eyes from the captured image. Analyzing the image of the user's eyes. And a user's gaze direction is detected from an analysis result, and the coordinate (gaze point) of the position on the screen 51a of the display part 51 which the user is gazing based on this gaze direction is detected. The camera 53a is disposed adjacent to the lower side of the screen 51a.

The control unit 12 is configured to generate various image data, and displays various images on the screen 51 a of the display unit 51.
In the portable device 50 of the second embodiment, the same operation detection process as that of the first embodiment is performed, and when the user performs an operation confirmation behavior while gazing at an object on the operation screen, an operation corresponding to the object is performed. Detected.

  That is, the control unit 52 of the portable device 50 generates image data of an operation screen for accepting an operation on the own device, and displays the operation screen on the screen 51 a of the display unit 51. Further, since the camera 53a is arranged adjacent to the lower short side of the screen 51a, the user's face that is gazing at the screen 51a is usually photographed from a lower viewpoint from the front of the face. It becomes a state to do.

  For this reason, the photographed image of the user's face by the camera 53a is projected with the face slightly looked up, as in the first embodiment, and is set to Δd (t) which is the interval between the positions of the left and right eyes in the vertical direction. Based on this, the orientation of the user's face can be detected.

  Therefore, the line-of-sight detection unit 53 detects the position coordinates L and R of the left and right eyes of the driver and the gazing point on the captured image based on the captured image of the user's face in accordance with an instruction from the control unit 52. This is provided to the control unit 52.

  On the other hand, the control unit 52 that has acquired the position coordinates L and R calculates Δd based on them in the same manner as in the first embodiment, detects the driver's face orientation based on Δd, and further determines the operation confirmation behavior. To detect. Then, when the user is gazing at the same object during the detection of the operation confirmation behavior, the control unit 52 regards that the operation corresponding to the object has been performed.

[Other Embodiments]
(1) In the operation detection process according to the first and second embodiments, when the same object on the operation screen is being watched by the driver or the like while the operation is confirmed by the driver or the user, the object is detected. An operation corresponding to is detected.

  However, the present invention is not limited to the case where such an operation screen is displayed. For example, the driver is determined in advance on the screen of the display device 20 (or the display unit 51 of the portable device 50) while the operation confirmation behavior is performed. When an area is watched, an operation corresponding to the area may be detected. Even in such a case, the same effect can be obtained.

  (2) Further, in the operation detection processing in the first and second embodiments, an object that the driver or the like is gazing at when the face orientation of the driver or the like is inclined by a predetermined amount in the lateral direction is specified. The Then, when an operation confirmation behavior is made while gazing at the object, an operation corresponding to the object is detected.

  However, the present invention is not limited to this. For example, when the control unit 12 of the in-vehicle device 10 or the control unit 52 of the portable device 50 detects that a driver or the like is gazing at a specific object, The degree of inclination of the face direction of a driver or the like in the horizontal direction may be detected. Then, when the degree of inclination of the driver or the like changes by a predetermined amount from what was initially detected while gazing at the object, these control units consider that the operation confirmation behavior has been made, and A corresponding operation may be detected. Even in such a case, the same effect can be obtained.

[Correspondence with Claims]
The correspondence between the terms used in the description of the above embodiment and the terms used in the description of the claims is shown.

The in-vehicle device 10 in the first embodiment and the portable device 50 in the second embodiment correspond to an operation device, and the display unit 51 of the portable device 50 corresponds to a display device.
Further, S200 of the operation detection process is the display means and display procedure, S230 and S255 are the specification means and specification procedure, S205 to S225 and S275 to S300 are the detection means and detection procedure, and S270 and S305 are the operation means and operation procedure. It corresponds to.

  DESCRIPTION OF SYMBOLS 1 ... In-vehicle system, 10 ... In-vehicle apparatus, 11 ... Video signal output part, 12 ... Control part, 20 ... Display apparatus, 30 ... Gaze detection apparatus, 31 ... Camera, 50 ... Portable apparatus, 51 ... Display part, 51a ... Screen 52, a control unit, 53, a line-of-sight detection unit, 53a, a camera.

Claims (7)

Display means (S200) for displaying an image on the screen of the display device (20, 51);
Identification means (S230, S255) for identifying a point of interest on the screen on which the user is gazing based on a photographed image of the user's face;
Detection means (S205 to S225, S275 to S300) for detecting the behavior of moving the entire user's head in advance based on the captured image;
When the behavior of the user is detected by the detection means, an operation means (S270, S305) that considers that an operation corresponding to the gazing point specified by the specifying means is performed when the behavior is made;
An operating device (10, 50) comprising: The operating device according to claim 1,
The display means displays an image on which one or more objects are drawn on the screen of the display device,
When the behavior of the user is detected by the detection means, the operation means is an area where any one of the objects is drawn as the gazing point specified by the specification means when the behavior is made In the case where the operation corresponding to the object has been performed,
An operation device characterized by. In the operating device according to claim 1 or 2,
The behavior is a movement that tilts the face sideways.
An operation device characterized by. The operating device according to any one of claims 1 to 3,
The detection means detects the eye position of the user based on the captured image, and detects the behavior based on the eye position;
An operation device characterized by. In the operating device according to claim 4, which refers to claim 3,
The captured image is an image of the user's face taken from below,
The detection means detects an interval between vertical positions of the left and right eyes of the user in the captured image, and determines a degree of the user tilting the face sideways based on the interval, Detecting the behavior based on the degree;
An operation device characterized by. In the operating device according to any one of claims 1 to 5,
The operating device is mounted on a vehicle;
An operation device characterized by. A display procedure (S200) for displaying an image on the screen of the display device (20, 51);
A specific procedure (S230, S255) for identifying a gaze point on the screen on which the user is gazing based on a photographed image of the user's face;
A detection procedure (S205 to S225, S275 to S300) for detecting a behavior of moving the entire head of the user based on the captured image;
When the behavior of the user is detected in the detection procedure, an operation unit (S270, S305) that considers that an operation corresponding to the gazing point specified by the specifying unit is performed when the behavior is performed;
An operation detection method comprising:

Images