JP2019156134A - Display system, display control device and display control method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technique for reducing the movement of a line of sight even when the driver operates the vehicle during driving. A sensor 50 detects a manual operation in a vehicle. The image pickup apparatus 52 captures an image of a region including a portion operated by a hand to be detected by the sensor 50. The display control device 60 outputs the image captured by the image pickup device 52 when the sensor 50 detects the operation. The display device 54 displays the image output from the display control device 60. [Selection diagram] Fig. 2

Description

  The present invention relates to display technology, and more particularly to a display system, a display control device, and a display control method in a vehicle.

  In addition to steering, buttons and dials are also installed in the car. The driver operates the steering wheel while driving, but sometimes operates a button or a dial (see, for example, Patent Document 1).

Japanese Patent Laid-Open No. 2016-216021

  When the button and dial installed in the vehicle are arranged at positions deviating from the direction of the driver's line of sight while driving, the driver deflects his line of sight from the front when operating the button and dial.

  The present invention has been made in view of such circumstances, and an object thereof is to provide a technique for reducing the movement of the line of sight even when the driver operates the vehicle while driving.

  In order to solve the above-described problem, a display system according to an aspect of the present invention includes a sensor that detects an operation by a hand in a vehicle and an area that includes a portion that is operated by a hand when the sensor detects an operation. An image pickup apparatus that picks up an image and a display device that displays an image picked up by the image pickup apparatus are provided.

  Another aspect of the present invention is a display control device. The apparatus includes a first acquisition unit that acquires a detection result of an operation performed by a hand in a vehicle from a sensor, and a second acquisition unit that acquires an image obtained by imaging an area including a portion operated by a hand from an imaging device. And an output unit that outputs the video acquired in the second acquisition unit to the display device when an operation by a hand in the vehicle is detected in the detection result acquired by the first acquisition unit.

  Yet another embodiment of the present invention is a display control method. This method includes a step of obtaining a detection result of an operation by a hand in a vehicle from a sensor, a step of obtaining an image obtained by imaging an area including a portion operated by a hand from an imaging device, and the obtained detection result And a step of outputting the acquired video to a display device when an operation by a hand in the vehicle is detected.

  An arbitrary combination of the above components, the expression of the present invention converted between an apparatus, a system, a method, a program, a recording medium recording the program, a vehicle equipped with the apparatus, and the like are also included in the present invention. It is effective as an embodiment.

  According to the present invention, it is possible to reduce the movement of the line of sight even when the driver operates during driving.

It is a figure which shows typically the interior of the vehicle which concerns on embodiment. It is a figure which shows the structure of the display system which concerns on embodiment. It is a figure which shows the image | video displayed on the display apparatus of FIG. It is a flowchart which shows the display procedure by the display system of FIG. It is a flowchart which shows another display procedure by the display system of FIG.

  Before describing the present invention in detail, an outline will be described. The present embodiment relates to a display system mounted in a vehicle. The driver may check the position of the drink holder when placing or taking a drink. When performing such confirmation during driving, the driver turns his / her line of sight from the front. While driving, it is preferable not to deflect the line of sight from the front, but if the position of the drink holder is not known without being deflected, the drink may fall over. As described above, the same applies to the operation of buttons and dials arranged at positions deviating from the direction of the driver's line of sight during driving. Therefore, it is desired to reduce the movement of the line of sight even when the driver operates during driving.

  In order to cope with this, the display system according to the present embodiment includes a sensor, an imaging device, a display control device, and a display device. A sensor detects the operation by the hand with respect to the part operated by a driver | operator, for example, a drink holder, a button, and a dial. The imaging device captures an image of a region including a portion operated by the driver. The display control device is connected to the sensor and the imaging device, and displays an image captured by the imaging device on the display device when an operation by a hand is detected in the sensor. Since the display device is arranged at a position not deviating from the direction of the driver's line of sight while driving, the driver can check the operation by hand while viewing the video displayed on the display device. Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. Each embodiment described below is an example, and the present invention is not limited to these embodiments.

  FIG. 1 schematically shows the interior of the vehicle 100 and shows a configuration of the interior of the vehicle 100 as viewed from the rear. A dashboard 10 extending left and right is disposed in front of the vehicle interior, a windshield 12 is disposed above the dashboard 10, and a ceiling 14 is disposed from above the windshield 12 toward the rear. In addition, a room mirror 16 is installed downward from a portion of the ceiling 14 on the windshield 12 side. A steering wheel 18 is disposed on the left side of the dashboard 10, and a driver seat 30 is disposed behind the steering wheel 18. A driver 34 is seated in the driver seat 30. In addition, a passenger seat 32 is disposed on the opposite side of the driver seat 30. When the steering wheel 18 is disposed on the right side portion of the dashboard 10, the positions of the driver seat 30 and the passenger seat 32 are reversed. Further, a left side mirror 22 is disposed outside the vehicle on the driver seat 30 side, and a right side mirror 24 is disposed outside the vehicle on the passenger seat 32 side.

  The center console 20 is disposed between the driver seat 30 and the passenger seat 32 from the dashboard 10. The center console 20 includes an operation unit 40 collectively referred to as a first operation unit 40a, a second operation unit 40b, a third operation unit 40c, and a fourth operation unit 40d. The first operation unit 40a is provided between the driver's seat 30 and the passenger seat 32, and includes a button and a dial. The second operation unit 40b to the fourth operation unit 40d are provided in the front portion of the center console 20, and are configured by buttons and a dial. The operation unit 40 can be operated by a driver's hand. Further, a drink holder 42 is disposed in front of the first operation unit 40a.

  Here, in order to clarify the explanation, it is assumed that the display target by the display system 200, that is, the monitoring target is the first operation unit 40a. A sensor 50 for detecting the presence of the hand or the movement of the hand is disposed in the vicinity of the first operation unit 40a. An imaging device 52 is disposed above the room mirror 16. Display light from the first display device 54a, which is a head-up display, is projected onto the windshield 12 on the steering 18 side. The display light reflected on the vehicle interior side in the first display device 54 a is visually recognized by the driver 34 seated in the driver seat 30. The driver can visually recognize the virtual image of the image projected on the first display device 54 a by overlapping the foreground of the windshield 12. The center console 20 is provided with a second display device 54b that is a center display. The second display device 54b may be a display of a navigation system, for example.

  Here, a range that does not deviate from the direction of the line of sight of the driver 34 during driving is indicated as a viewing range 300. The viewing range 300 is a range in which the driver normally distributes his / her line of sight while driving. The viewing range 300 includes the windshield 12, the room mirror 16, the left side mirror 22, the right side mirror 24, and the display device 54. However, the first operation unit 40 a to the fourth operation unit 40 d and the drink holder 42 are not included in the viewing range 300. Therefore, in order to confirm the position of the 1st operation part 40a, the driver | operator 34 needs to point a line of sight outside the visual recognition range 300. FIG.

  FIG. 2 shows the configuration of the display system 200. The display system 200 includes a sensor 50, an imaging device 52, a display device 54, and a display control device 60. The sensor 50 detects the operation by the hand in the 1st operation part 40a by making the vicinity of the 1st operation part 40a into a sensing area. As the sensor 50, for example, an infrared sensor, an ultrasonic sensor, or a proximity sensor is used. In addition, the sensor 50 may be a touch sensor or may be attached to the first operation unit 40a. Further, the sensor 50 may be integrated with the imaging device 52, and by performing an image recognition process on the video captured by the imaging device 52, an operation by a hand in the first operation unit 40a is detected. Also good. Since known techniques may be used for these, description thereof is omitted here.

  The imaging device 52 captures an image of a portion operated by a hand to be detected by the sensor 50, that is, an area including the first operation unit 40a. As described above, the first operation unit 40a is disposed at a position deviating from the line of sight of the driver 34 who is driving. The imaging device 52 outputs the captured video to the display control device 60.

  The first acquisition unit 70 of the display control device 60 acquires the detection result from the sensor 50. The 1st acquisition part 70 may acquire a detection period over the period when sensor 50 detects operation by hand in the car, and always acquires the detection result in sensor 50, and from the detection result, in the car You may extract the period when the operation by hand is made. The 1st acquisition part 70 notifies the control part 74 that the operation by the hand in the vehicle is made over the period when the operation by the hand in the vehicle is made. The second acquisition unit 72 acquires a video from the imaging device 52. The second acquisition unit 72 outputs the video to the control unit 74.

  The control unit 74 performs the second acquisition while receiving the notification from the first acquisition unit 70, that is, while the sensor 50 detects the operation or while the operation by the hand in the vehicle is detected in the detection result. The video acquired in the unit 72 is output to the output unit 76. On the other hand, the control unit 74 does not output the video to the output unit 76 unless the notification is received from the first acquisition unit 70. The output unit 76 outputs the video received from the control unit 74 to the display device 54. The display device 54 is the first display device 54a or the second display device 54b in FIG. 1 and displays the video from the output unit 76, that is, the video imaged by the imaging device 52. FIG. 3 shows an image displayed on the display device 54. For example, an image in which the first operation unit 40a is pinched by hand is displayed. Returning to FIG.

  Note that the control unit 74 may start outputting the video after a predetermined period has elapsed after receiving the notification from the first acquisition unit 70. An example of the predetermined period is 1 second. This corresponds to the start of video output after a predetermined period has elapsed since the sensor 50 detected the operation.

  This configuration can be realized by a CPU (Central Processing Unit), memory, or other LSI (Large Scale Integration) of any computer in terms of hardware, and by a program loaded in the memory in terms of software. However, here, functional blocks that are realized by their cooperation are depicted. Accordingly, those skilled in the art will understand that these functional blocks can be realized in various forms only by hardware, or by a combination of hardware and software.

  The operation of the display system 200 having the above configuration will be described. FIG. 4 is a flowchart showing a display procedure by the display system 200. If the sensor 50 detects a hand operation (Y in S10), the imaging device 52 captures an image (S12), the display device 54 displays an image (S14), and the process returns to step 10. When the sensor 50 does not detect a manual operation (N in S10) and the display device 54 displays an image (Y in S16), the display device 54 deletes the image (S18). If the display device 54 is not displaying an image (N in S16), the process is terminated.

  FIG. 5 is a flowchart showing another display procedure by the display system 200. When the sensor 50 does not detect an operation by hand (N of S50), it waits for 1 second (S52). When the sensor 50 detects a hand operation (Y in S54) or when the sensor 50 detects a hand operation (Y in S50), the process waits for one second (S56). When the sensor 50 detects a hand operation (Y in S58), the imaging device 52 captures an image (S60), the display device 54 displays an image (S62), and the process returns to step 50. If the sensor 50 does not detect a hand operation (N in S58), the process returns to step 50. When the sensor 50 does not detect a manual operation (N in S54) and the display device 54 displays an image (Y in S64), the display device 54 deletes the image (S66). If the display device 54 is not displaying video (N in S64), the process is terminated.

  According to the present embodiment, since an image is displayed when a manual operation in the vehicle is detected, the driver can check the manual operation by looking at the display device. Further, since the driver confirms the manual operation by looking at the display device, the movement of the line of sight can be reduced even when the driver operates the vehicle while driving. Further, even when the driver operates during driving, the movement of the line of sight is reduced, so that safety can be improved. Moreover, since the part arrange | positioned in the position which remove | deviated from the direction of a driver | operator's eyes | visual_axis during driving | running | working is displayed, the opportunity for a driver | operator to see the position out of the direction of eyes | visual_axis can be reduced. In addition, since the display of the video is started after a lapse of a predetermined period from the detection of the operation by the sensor, the occurrence of repeated display or deletion can be suppressed.

  The outline of one embodiment of the present invention is as follows. A display system according to an aspect of the present invention includes a sensor that detects an operation by a hand in a vehicle, an imaging device that captures an image of a region including a portion operated by a hand when the sensor detects an operation, And a display device for displaying a video imaged by the imaging device.

  According to this aspect, since an image is displayed when an operation by a hand in the vehicle is detected, the movement of the line of sight can be reduced even when the driver operates the vehicle while driving.

  The part included in the region to be imaged by the imaging device and operated by hand may be arranged at a position deviating from the direction of the driver's line of sight while driving. In this case, since the part arrange | positioned in the position remove | deviated from the direction of a driver | operator's eyes | visual_axis during driving | running | working is displayed, the opportunity for a driver | operator to see the position out of the direction of eyes | visual_axis can be reduced.

  The display device may start displaying an image after a predetermined period has elapsed since the sensor detected the operation. In this case, since the display of the video is started after a lapse of a predetermined period from the detection of the operation by the sensor, it is possible to suppress the occurrence of repeated display or deletion.

  Another aspect of the present invention is a display control device. The apparatus includes a first acquisition unit that acquires a detection result of an operation performed by a hand in a vehicle from a sensor, and a second acquisition unit that acquires an image obtained by imaging an area including a portion operated by a hand from an imaging device. And an output unit that outputs the video acquired in the second acquisition unit to the display device when an operation by a hand in the vehicle is detected in the detection result acquired by the first acquisition unit.

  According to this aspect, since an image is displayed when an operation by a hand in the vehicle is detected, the movement of the line of sight can be reduced even when the driver operates the vehicle while driving.

  Yet another embodiment of the present invention is a display control method. This method includes a step of obtaining a detection result of an operation by a hand in a vehicle from a sensor, a step of obtaining an image obtained by imaging an area including a portion operated by a hand from an imaging device, and the obtained detection result And a step of outputting the acquired video to a display device when an operation by a hand in the vehicle is detected.

  The present invention has been described based on the embodiments. It is understood by those skilled in the art that these embodiments are exemplifications, and that various modifications can be made to combinations of the respective constituent elements and processing processes, and such modifications are also within the scope of the present invention. By the way.

  In the present embodiment, the display target by the display system 200, that is, the monitoring target is the first operation unit 40a. However, the present invention is not limited to this. For example, the monitoring target may be at least one of the second operation unit 40b to the fourth operation unit 40d and the drink holder 42. In that case, the sensor 50 and the imaging device 52 should just be installed in the position according to the monitoring object. According to this modification, the degree of freedom of configuration can be improved.

  According to the present invention, it is possible to reduce the movement of the line of sight even when the driver operates during driving.

  10 Dashboard, 12 Windshield, 14 Ceiling, 16 Room Mirror, 18 Steering, 20 Center Console, 22 Left Side Mirror, 24 Right Side Mirror, 30 Driver's Seat, 32 Passenger's Seat, 34 Driver, 40 Operation Unit, 42 Drink Holder, 50 sensor, 52 imaging device, 54 display device, 60 display control device, 70 first acquisition unit, 72 second acquisition unit, 74 control unit, 76 output unit, 100 vehicle, 200 display system.

Claims (5)

A sensor for detecting a manual operation in the vehicle;
An imaging device that captures an image of a region including a portion operated by a hand to be detected by the sensor;
A display control device that outputs an image captured by the imaging device when the sensor detects an operation;
A display device for displaying video output from the display control device;
A display system comprising:   The portion that is included in a region to be imaged by the imaging device and that is operated by a hand is disposed at a position deviating from the direction of the driver's line of sight while driving. The display system according to 1.   The display system according to claim 1, wherein the display control device starts outputting an image after a predetermined period has elapsed since the sensor detected an operation. A first acquisition unit for acquiring a detection result of a manual operation in the vehicle from the sensor;
A second acquisition unit that acquires an image obtained by imaging an area including a portion operated by a hand from the imaging device;
In the detection result acquired by the first acquisition unit, when an operation by a hand in the vehicle is detected, an output unit that outputs the video acquired by the second acquisition unit to a display device;
A display control apparatus comprising: Obtaining a detection result of a manual operation in the vehicle from the sensor;
Acquiring an image obtained by imaging an area including a portion operated by a hand from the imaging device;
In the acquired detection result, when an operation by hand in the vehicle is detected, a step of outputting the acquired video to a display device;
A display control method comprising:

Images