JP2018132824A - Operation device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an operation device which enables an intuitively comprehensible input operation, based on actual shapes of input switches, with all of various input switches provided in a room as objects.SOLUTION: An operation device performs an input operation to one of predetermined apparatuses provided in a room 11 according gestures of an operator's hand. The operation device comprises a control unit 140 which makes a switch images 110 displayed on a display unit 120. Upon reception of a detection signal from a sensor unit 130, the control unit synthesizes a gesture image 111 showing a movement based on the gesture, on the switch image 110. Upon detection of a predetermined operation gesture based on the gesture image 111 performed to a desired switch image 112 desired by the operator from among the switch images 110, the control unit outputs a switch signal corresponding to the desired switch image 112, to one of the predetermined apparatuses.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an operation device that enables input by an operator's gesture operation.

  As a conventional operation device, one described in Patent Document 1 is known. That is, the operation device (in-vehicle gesture switch device) of Patent Document 1 is mounted on a vehicle and includes an image display device, a camera, a sensor, and a computer.

  The image display device is arranged at the upper center of the instrument panel of the vehicle, and displays a predetermined switch operation unit (icon) as an operation switch for the in-vehicle device. The camera is provided at the lower part of the image display device, and photographs the movement of the driver's hand (gesture operation) in a designated area (virtual operation space) set between the image display device and the driver's seat. It is like that. The sensor detects a start operation and an end operation of the gesture operation in the driver's designated area.

  When the computer determines that the gesture operation corresponding to the switch operation unit of the image display device has started based on the operation detection signal from the sensor, the computer displays a response display indicating that the switch operation unit has been operated on the image display device. At the same time, an input instruction corresponding to the switch operation unit on which the gesture operation has been performed is given to the in-vehicle device.

  As described above, in the operation device of Patent Document 1, an input operation to the in-vehicle device can be performed by a gesture operation in the air without directly touching the switch operation unit of the image display device.

JP 2010-184600 A

  However, in Patent Document 1, only an operation of a switch operation unit predetermined for the image display device can be performed, and an input operation target is limited to the switch operation unit in the image display device. Moreover, in the said patent document 1, the switch operation part is an icon displayed with an image display apparatus, does not display the shape of the actual switch mounted in a vehicle interior, and is intuitive. It is difficult to get a good feeling of operation.

  In view of the above problems, an object of the present invention is to provide an operating device that enables an intuitively easy-to-understand input operation based on the actual shape of an input switch for all input switches provided in a room. It is in.

  In order to achieve the above object, the present invention employs the following technical means.

In the present invention, in an operating device for performing an input operation on a predetermined device provided in the room (11) by a gesture of an operator's hand,
A switch image (110) reflecting the substance for the whole of various input switches (20) for a predetermined device,
A display unit (120) for displaying a switch image;
A sensor unit (130) for detecting a gesture in a predetermined region;
A switch image is displayed on the display unit, receives a detection signal from the sensor unit, synthesizes a gesture image (111) indicating movement based on the gesture with the switch image, and an operator of the switch image desires. A control unit (140) that outputs a switch signal corresponding to the desired switch image to a predetermined device when there is a predetermined operation gesture based on the gesture image with respect to the desired switch image (112).

  According to the present invention, the switch image (110) for the entirety of the various input switches (20) for the predetermined device set in the room (11) is displayed on the display unit (120). In addition, a gesture of an operator's hand performed in a predetermined area is detected by the sensor unit (130). Then, the gesture image (111) based on the gesture is synthesized and displayed on the display unit (120) by the control unit (140) with respect to the switch image (110).

  When the operator performs a predetermined operation gesture in a predetermined area on the desired switch image (112) of the switch image (110) while looking at the display unit (120), the control unit (140) Outputs a switch signal corresponding to the desired switch image (112) to a predetermined device.

  In other words, the operator covers the entire various input switches (20) without extending their hands with respect to the actual various input switches (20) or even in a region that is not actually reachable. As a result, a desired switch input operation can be performed on the screen of the display unit (120). Since the switch image (110) is an image reflecting the substance of the various input switches (20), it is possible to cope with the operation of the various familiar actual input switches (20). Easy-to-understand input operations are possible.

In addition, the code | symbol in the bracket | parenthesis of each said means shows a corresponding relationship with the specific means of embodiment description mentioned later.

It is a block diagram which shows the whole structure of the operating device in 1st Embodiment. It is explanatory drawing which shows the various input switches in a vehicle. It is a block diagram which shows the structure of an operating device. It is a flowchart which shows the control content which a control part performs. It is explanatory drawing which shows a driver | operator's gesture and the image displayed on a display part.

  A plurality of modes for carrying out the present invention will be described below with reference to the drawings. In each embodiment, parts corresponding to the matters described in the preceding embodiment may be denoted by the same reference numerals, and redundant description may be omitted. When only a part of the configuration is described in each mode, the other modes described above can be applied to the other parts of the configuration. Not only combinations of parts that clearly indicate that the combination is possible in each embodiment, but also a combination of the embodiments even if they are not clearly specified unless there is a problem with the combination. It is also possible.

(First embodiment)
The operating device 100 of 1st Embodiment is demonstrated using FIGS. 1-5. The operation device 100 according to the present embodiment is mounted on the vehicle 10 (applied to the vehicle 10), and various in-vehicle devices (predetermined) are provided by gestures of a driver (driver) who is an operator or a passenger's hand. Input operation to the device). Hereinafter, the operator will be described mainly using the driver as an example.

  As shown in FIGS. 1 to 3, the vehicle 10 is provided with various input switches (various switch groups) 20 for operating various in-vehicle devices. Examples of the various in-vehicle devices include an air conditioner device, an audio device, an electric seat device, a travel mode setting device, a parking brake device, a door lock device, a window opening / closing device in each seat door, and a sunshade device in the rear seat. .

  The various input switches 20 include an air conditioner switch 21, an audio switch 22, an electric seat switch 23, a travel mode setting switch 24, a parking brake switch 25, a door lock switch 26a, and a seat window switch 26b. Window switches 27a, 28a, 29a, sunshade switches 28b, 29b, and the like.

  The air conditioner switch 21, the audio switch 22, and the like are further divided into switches for performing individual operation settings. For example, the air conditioner switch 21 is provided with an inside / outside mode changeover switch 21a, a blowout mode changeover switch 21b, an air volume changeover switch 21c, a temperature setting switch 21d, and the like.

  When the various input switches 20 are directly operated by the driver or the passenger, the various in-vehicle devices are activated by the controller 20A (FIG. 3).

  Among the various input switches 20, an air conditioner switch 21, an audio switch 22, an electric seat switch 23, a travel mode setting switch 24, and a parking brake switch 25 are arranged at the center of the instrument panel 12 with the driver. It is provided on the opposite side. The door lock switch 26 a and the seat window switch 26 b are provided on the driver seat side door 14. Further, the window switch 27 a is provided on the passenger seat side door 15. The window switch 28a and the sunshade switch 28b are provided on the right rear seat door 16. The window switch 29a and the sunshade switch 29b are provided on the left rear seat side door 17.

  The air conditioner switch 21, audio switch 22, electric seat switch 23, door lock switch 26a, seat window switch 26b, window switches 27a, 28a, 29a, sunshade switches 28b, 29b are, for example, push type It has become a switch. The travel mode setting switch 24 is, for example, a dial type switch.

  The operating device 100 includes a switch image 110, a display unit 120, a sensor unit 130, a control unit 140, and the like.

  The switch image 110 is an image reflecting the substance for the entire various input switches 20 set in the room 11, and is stored, for example, in a storage unit (ROM) of the control unit 140 described later. Here, for example, the switch image 110 is a captured image that is captured in advance with an actual shape, and is an image familiar to the driver. The switch image 110 may be a CG (computer graphics) image created so as to conform to the substance, instead of the above-described photographed image. The switch image 110 is displayed on the display unit 120 described later while the driver's gesture is performed as described later.

  The display unit 120 is formed by, for example, a liquid crystal display, an organic EL display, or the like, and is provided at a position where the driver can easily see, for example, at the upper center of the instrument panel 12. The display unit 120 is usually configured to display map information, host vehicle position information, destination guidance information, and the like as a display unit for a navigation device, for example, but a gesture by the driver is performed. During this time, display control is performed by a control unit 140, which will be described later, and the switch image 110, the hand image 111, and the like are displayed.

  The sensor unit 130 detects a gesture of the driver's hand in a region around the steering wheel 13 (a predetermined region), and is provided, for example, on the driver's A pillar. As the sensor unit 130, for example, a camera (2D), a stereo camera (3D), or a TOF camera using a TOF (Time Of Flight) method is used, and the position and movement (gesture) of the driver's hand (finger). Is supposed to be detected. The sensor unit 130 outputs the detected detection signal to the control unit 140 described later.

  The control unit 140 is formed by a computer having a CPU as a central processing unit, a RAM for performing a calculation, a calculation program, a ROM for storing various data, and the like. Based on the detection signal from the sensor unit 130, the control unit 140 displays the switch image 110 on the display unit 120, and forms a gesture image (hand image 111) indicating movement based on the driver's gesture. The switch image 110 is synthesized.

  Further, when there is a predetermined operation gesture based on the gesture image with respect to the desired switch image 112 that the driver desires as an operation target in the switch image 110, the control unit 140 sends a switch signal corresponding to the desired switch image 112 to the vehicle. Output to the device (details will be described later).

  Note that the following gestures performed by the driver are preset and stored in the control unit 140, and the control unit 140 performs control for input operation according to the type of gesture. Yes.

  Examples of gestures performed by the driver include a start gesture, a region selection gesture, an enlarged display gesture, a predetermined operation gesture, and an end gesture.

  The start gesture is a gesture indicating that a gesture operation by the driver is started. For example, the start gesture is set as a gesture in which four fingers excluding the thumb are raised upward.

  The area selection gesture is an area of the switch group desired by the driver (the central area of the instrument panel, the area on the right rear seat side or the left rear seat side) in the switch image 110 in which the entire various input switches 20 are photographed. For example, the gesture is set as a gesture in which an area desired to be selected for the switch image 110 is drawn with a forefinger.

  The enlarged display gesture is a gesture that indicates that the driver wants to enlarge an area around the desired switch as an operation target in the desired switch group. For example, so as to aim at the desired switch with respect to the switch image 110, It is set as a gesture that moves the hand slightly forward.

  The predetermined operation gesture is a gesture indicating that an input operation similar to an actual switch operation is performed on a desired switch. For example, in the case of a push-type switch, a gesture that is pushed with an index finger, or In the case of a dial type switch, it is set as a gesture that is picked up by two fingers and rotated.

  The end gesture is a gesture indicating that the gesture operation by the driver is ended. For example, the end gesture is set as a gesture in which four fingers excluding the thumb are tilted toward the vehicle front side.

  The operation device 100 according to the present embodiment is configured as described above. Next, the contents of the input operation control executed by the control unit 140 will be described with reference to FIGS. 4 and 5.

  In the flowchart shown in FIG. 4, first, in step S100, when the control unit 140 recognizes a start gesture (with four fingers raised) by the driver based on a detection signal (hereinafter, camera image) from the sensor unit 130, Start the system. At this time, the control unit 140 displays the switch image 110 (the image of the entire switch image in the room 11 in FIG. 2) on the display unit 120 (FIG. 5 (1)).

  Next, in step S110, the control unit 140 recognizes the position and posture of the driver's hand based on the camera image from the sensor unit 130, and places the hand at a predetermined position in the switch image 110, for example, the center. The image 111 is synthesized and displayed (FIG. 5 (1)). The hand image 111 is displayed as an image according to the actual movement of the driver's hand. The hand image 111 is mainly a real image from the wrist to the fingertip side, and is transparent from the wrist to the arm side. The image is visible. The hand image 111 corresponds to the gesture image in the present invention.

  Next, in step S120, when the control unit 140 detects the driver's up / down / left / right hand gesture from the camera image, the control unit 140 displays the hand image 111 so as to move up / down / left / right in the same manner. When the driver's hand is detected for the first time, the control unit 140 sets the position coordinate of the hand at that time in the center of the switch image 110, and then follows the hand movement operation based on the position of the center. The position of the hand is relatively defined, and the hand image 111 is moved and displayed on the switch image 110.

  Next, in step S <b> 130, when the controller 140 recognizes an area selection gesture (drawing a circle with a finger) by the driver from the camera image, the controller 140 displays the switch image 110 of the area selected from the entire switch image 110 in the room 11. Display. FIG. 5B shows a state in which, for example, the area of the switch group (the switches 21 to 25) in the center of the instrument panel 12 is selected and displayed.

  Next, in step S140, when the control unit 140 recognizes an enlarged display gesture (moving the hand toward the desired switch) by the driver from the camera image, the control unit 140 selects the switch (and its surroundings) selected as the desired target. The image is enlarged at a predetermined size and displayed as a desired switch image 112 (FIG. 5 (3)).

  Next, in step S150, when the control unit 140 recognizes a predetermined operation gesture (input operation such as push or rotation) performed on the desired switch image 112 by the driver from the camera image, the control unit 140 determines that the desired switch image 112 is turned on. .

  In step S <b> 160, the control unit 140 outputs an ON command signal to the actual switch corresponding to the desired switch image 112. That is, the switch signal of the actual switch corresponding to the desired switch image 112 is output to the controller 20A and output to the corresponding in-vehicle device to operate (FIG. 5 (5)). In addition, the control unit 140 outputs a reaction sound (for example, a beeping sound) indicating that the switch is operated from a predetermined speaker or the like, and emits light around the desired switch image 112 (FIG. 5 (4)). )). That is, although it is a virtual operation on the switch image 110, it shows a reaction (sound and light) that can be felt by the driver as if the actual switch had been operated.

  In step S <b> 170, when an end gesture by the driver (falling down four fingers) is recognized from the camera image, a series of input operation control ends. If it is recognized that there is no end gesture, steps S100 to S170 are repeated.

  As described above, according to the present embodiment, the switch image 110 for the entirety of the various input switches 20 for in-vehicle devices set in the room 11 is displayed on the display unit 120. The sensor unit 130 detects a gesture of the driver's hand performed around the steering wheel 13 of the vehicle 10. Then, the control unit 140 displays the hand image 111 (gesture image) based on the gesture on the display unit 120 in combination with the switch image 110.

  When the driver performs a predetermined operation gesture around the steering wheel 13 on the desired switch image 112 in the switch image 110 while looking at the display unit 120, the control unit 140 corresponds to the desired switch image 112. The switch signal to be output is output to the in-vehicle device.

  That is, in the vehicle 10, as described with reference to FIG. 2, the various input switches 20 are provided not only on the instrument panel 12 but also on the doors 14 to 17, etc. Some things are out of reach. Therefore, in this embodiment, the driver does not extend his / her hand to the actual various input switches 20 or the entire various input switches 20 even for a switch in a region that is not actually reachable. As an object, a desired switch input operation can be performed on the screen of the display unit 120. For example, the sunshade switches 28b and 29b in the rear seat can be operated while in the driver's seat. Since the switch image 110 is an image that reflects the substance of the various input switches 20, it is possible to handle the various actual input switches 20 that are familiar to the user, and intuitive and easy-to-understand input operations are possible. It becomes.

  In addition, since the driver can perform an input operation using gestures performed around the steering wheel 13, it is not necessary to perform a gesture at a position distant from the steering wheel 13, so that driving is not hindered and safe driving is continued. can do.

  Further, in the present embodiment, the control unit 140 displays, on the display unit 120, an input switch of a selected area based on a predetermined area selection gesture performed by the driver among the various input switches 20 in the switch image 110. It is made to display. Thereby, the input operation of the input switch in the region selected by the driver can be easily performed.

  In the present embodiment, the control unit 140 enlarges and displays the desired switch image 112 on the display unit 120 based on a predetermined enlarged display gesture performed by the driver. As a result, it is possible to reliably and easily perform an input operation on a desired switch.

  In the present embodiment, the control unit 140 receives a detection signal (camera image) from the sensor unit 130 during a start gesture and an end gesture performed by a predetermined driver. As a result, during this period, an input operation that reliably reflects the driver's gesture is possible. In other words, when the time is not between the start gesture and the end gesture, the movement of the driver's hand is not accepted as a gesture, so that input operation control that is contrary to the driver's will is not performed.

  Further, in the present embodiment, when there is a predetermined operation gesture, the control unit 140 generates a reaction sound indicating that an input operation has been performed and causes the desired switch image 112 to be displayed in a light-emitting manner. Thereby, even if it is virtual operation on the switch image 110, a driver | operator can actually feel (recognize) as if the actual switch was operated.

  In the present embodiment, the control unit 140 forms the gesture image as the image 111 of the driver's actual hand. As a result, the driver can recognize the image as a reality.

(Other embodiments)
In the first embodiment, when the input operation control based on the gesture is started, the switch images 110 (images corresponding to FIG. 2) of the various input switches 20 throughout the room 11 are displayed. However, the present invention is not limited to this. However, the following may be performed.

  For example, an initial display switching unit that can be switched to a switch in a predetermined area among the various input switches 20 as an initial display of the switch image 110 on the display unit 120 based on a driver's operation is provided. . For example, the initial display switching unit is provided with an initial image setting icon (button) in the display unit 120 at the time of input operation control, and performs setting to start from displaying a switch image in a predetermined region by this icon operation. can do.

  By such an initial display switching unit, as an initial display, for example, various switches 21 to 25 arranged in the center of the instrument panel 12 frequently used by the driver can be set as the initial switch image 110. It becomes possible. In this case, step S130 described in FIG. 4 can be omitted.

  In addition, as a gesture performed by the driver, a scroll gesture for scrolling the displayed current switch image 110 (display of a switch in a predetermined area) in a desired direction is determined in advance, and scroll display is performed by the scroll gesture. You may make it make it. The scroll gesture can be, for example, a gesture that moves the index finger in the direction in which the user wants to scroll. Accordingly, it is possible to easily change from a switch display in a predetermined area to a display of a switch in another predetermined area.

  In addition, the minimum configuration of the present invention controls input operations for a predetermined operation gesture, and is at least one of a start gesture, a region selection gesture, an enlarged display gesture, an end gesture, and a scroll gesture. The processing based on this may be added as necessary.

  In the first embodiment, the input operation control is performed between the start gesture and the end gesture. However, after the start gesture, the input operation control associated with the predetermined gesture is performed for a predetermined time. It may be (accepted).

  In the first embodiment, the actual image of the driver's hand 111 is used as the gesture image. However, the present invention is not limited to this, and an image such as a robot hand imitating an actual hand may be used. Alternatively, if all switches to be input are push-type switches, an image such as an icon such as an arrow or a pointer may be used.

  In the first embodiment, the operator is mainly described as a driver. However, in the case of in-vehicle use, the operator is not limited to this, and the operator may be a passenger (passenger seat etc.).

  In the first embodiment, the operation device is described as being mounted on a vehicle. However, the operation device is not limited to this, and is not limited to this. The predetermined device (indoor illuminator, television, air conditioner, audio device) Etc.) may be applied to those operating. In this case, a predetermined area is set in the room, and the sensor unit detects an operator's gesture in this area. Also, the display unit is preferably set at a position where the operator can see the room in a predetermined area.

DESCRIPTION OF SYMBOLS 10 Vehicle 11 Indoors 13 Steering 20 Various input switches 100 Operation apparatus 110 Switch image 111 Hand image (gesture image)
112 Desired switch image 120 Display unit 130 Sensor unit 140 Control unit

Claims (9)

In an operating device for performing an input operation on a predetermined device provided in a room (11) by a gesture of an operator's hand,
A switch image (110) reflecting the substance for the entire various input switches (20) for the predetermined device,
A display unit (120) for displaying the switch image;
A sensor unit (130) for detecting the gesture in a predetermined region;
The switch image is displayed on the display unit, receives a detection signal from the sensor unit, synthesizes a gesture image (111) indicating movement based on the gesture with the switch image, and A control unit (140) that outputs a switch signal corresponding to the desired switch image to the predetermined device when there is a predetermined operation gesture by the gesture image with respect to the desired switch image (112) desired by the operator. And an operating device.   The said control part displays the input switch of the selected area | region on the said display part based on the area | region selection gesture which the said operator performs predetermined among the said various input switches in the said switch image. The operating device described.   The operation device according to claim 1, wherein the control unit enlarges and displays the desired switch image on the display unit based on a predetermined enlarged display gesture performed by the operator.   The control unit accepts the detection signal from the sensor unit for a predetermined time from a predetermined start gesture performed by the operator or during a predetermined end gesture performed by the operator from the start gesture. The operating device according to any one of claims 1 to 3.   5. The control unit according to claim 1, wherein when the predetermined operation gesture is present, the control unit generates a reaction sound indicating that an input operation has been performed, and causes the desired switch image to be lit and displayed. The operating device according to 1.   The operation device according to claim 1, wherein the control unit forms the gesture image as an image of an actual hand of the operator.   An initial display switching unit that can switch to a display of a switch in a predetermined area among the various input switches as an initial display of the switch image on the display unit based on an operation of the operator. The operating device according to any one of claims 1 to 6.   The operation device according to claim 7, wherein the control unit scrolls the display of the switch in the predetermined area based on a predetermined scroll gesture performed by the operator. Mounted on the vehicle (10),
The predetermined device is an in-vehicle device,
The operating device according to any one of claims 1 to 8, wherein the predetermined region is a region around a steering (13) of the vehicle.

Images