JP2017197015A - On-board information processing system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an on-board information processing system with which a running state of own vehicle can be intuitively controlled.SOLUTION: An on-board information processing system 10 includes: a display part 11 having at least one screen for displaying a running state of own vehicle; a touch operation part 12 for detecting contact by an operation hand of an operator; at least part of the operation hand; an imaging part 13 for imaging the touch operation part 12; and a control part 15 displaying at least part of the operation hand while overlapping on the screen based on an image filmed by the imaging part 13 and controlling the running state of own vehicle based on movement of at least part of the operation hand on the touch operation part 12.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an in-vehicle information processing system mounted on a vehicle in order to control information related to the vehicle.

  2. Description of the Related Art Conventionally, there is known a vehicle information providing apparatus that displays a surrounding situation of a host vehicle and prompts a driver to confirm appropriate safety.

  For example, the vehicular information providing apparatus disclosed in Patent Document 1 displays a driver confirmation image on an electronic mirror, and when the electronic mirror needs to be confirmed, the frequency at which the driver turns his / her line of sight toward the electronic mirror is improved. It is possible to make it.

JP2015-011457

  However, the vehicular information providing device described in Patent Document 1 displays information and notifies it as necessary. For example, the driver himself / herself cannot control the traveling state of the vehicle in automatic driving. It was. Therefore, it is desired to realize an interface that can intuitively control the traveling state of the host vehicle.

  An object of the present invention made in view of such a viewpoint is to provide an in-vehicle information processing system capable of intuitively controlling the traveling state of the own vehicle.

An in-vehicle information processing system according to a first aspect that solves the above problem is as follows.
A display unit having at least one screen for displaying a traveling state of the own vehicle;
A touch operation unit that detects contact by an operator's operator;
An imaging unit that images at least a part of the operator and the touch operation unit;
Based on the image picked up by the image pickup unit, at least a part of the operator's hand is superimposed and displayed on the screen, and based on the movement of a part of the operator's hand on the touch operation part, the own vehicle A control unit for controlling the running state of
Is provided.

In the in-vehicle information processing system according to the second aspect,
The control unit controls the traveling state of the host vehicle based on the movement of one finger in the touch operation unit.

In the in-vehicle information processing system according to the third aspect,
The control unit controls a traveling state of the own vehicle based on movements of two or more fingers in the touch operation unit.

In the vehicle information processing system according to the fourth aspect,
The display unit displays the running state of the vehicle by visual information.

In the in-vehicle information processing system according to the fifth aspect,
The traveling state of the host vehicle is a positional relationship between the host vehicle and the front vehicle.

In the vehicle information processing system according to the sixth aspect,
The traveling state of the host vehicle is the vehicle speed of the host vehicle.

In the on-vehicle information processing system according to the seventh aspect,
The traveling state of the host vehicle is a traveling position of the host vehicle in a plurality of lanes.

In the in-vehicle information processing system according to the eighth aspect,
The traveling state of the host vehicle is a planned passage route with respect to an intersection.

  According to the in-vehicle information processing system according to the first aspect, the operator can intuitively control the traveling state of the own vehicle.

  Further, according to the in-vehicle information processing system according to the second aspect, the operator can easily and intuitively control the traveling state of the own vehicle.

  Further, according to the in-vehicle information processing system according to the third aspect, the operator can intuitively control the traveling state of the own vehicle.

  Further, according to the in-vehicle information processing system according to the fourth aspect, the operator can easily visually recognize the traveling state of the own vehicle.

  Further, according to the in-vehicle information processing system according to the fifth aspect, the operator can intuitively control the inter-vehicle distance between the own vehicle and the preceding vehicle.

  Further, according to the in-vehicle information processing system according to the sixth aspect, the operator can intuitively control the vehicle speed of the own vehicle.

  Further, according to the in-vehicle information processing system according to the seventh aspect, the operator can intuitively control the traveling position of the own vehicle in a plurality of lanes.

  Further, according to the in-vehicle information processing system according to the eighth aspect, the operator can intuitively control the passage route with respect to the intersection.

It is a mimetic diagram showing the whole in-vehicle information processing system concerning this embodiment. It is a functional block diagram which shows schematic structure of the vehicle-mounted information processing system of FIG. It is the schematic diagram which showed an example of the image displayed on a display part. It is a figure which shows typically a cross section when a touch operation part is seen from a side surface direction. It is a schematic diagram when an operation part is seen from the upper surface direction. It is a schematic diagram which shows an example of the correspondence of the predetermined area | region of a touch operation part, and the operation area on the screen which comprises a display part. It is the schematic diagram which showed the mode of the image processing which an in-vehicle information processing system performs. It is an enlarged view of the display part shown in FIG. It is the schematic diagram which showed an example of control of the distance between vehicles by an operator. It is the schematic diagram which showed an example of control of the vehicle speed of the own vehicle by the operator. It is the schematic diagram which showed an example of the control of the traveling position of the own vehicle in the multiple lane by the operator. It is the schematic diagram which showed an example of the control of the scheduled passage route with respect to the intersection by the operator. It is a flowchart which mainly shows the control at the time of an operator selecting and operating a screen with an operation part. It is a flowchart at the time of an operator controlling the traveling state of the own vehicle.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a schematic diagram showing an entire in-vehicle information processing system 10 according to the present embodiment. FIG. 2 is a functional block diagram showing a schematic configuration of the in-vehicle information processing system 10 of FIG. The in-vehicle information processing system 10 includes a display unit 11, a touch operation unit 12, an imaging unit 13, an operation unit 14, a control unit 15, and a storage unit 16. FIG. 3 is a schematic diagram illustrating an example of an image displayed on the display unit 11. FIG. 3A shows an example of a menu screen, and FIG. 3B shows an example of a map screen. FIG. 4 is a diagram schematically illustrating a cross section when the touch operation unit 12 is viewed from the side surface direction. FIG. 5 is a schematic diagram when the operation unit 14 is viewed from above.

  The in-vehicle information processing system 10 associates the position coordinates in the operation area on the screen configuring the display unit 11 with the position coordinates in the predetermined area of the touch operation unit 12 and images captured by the imaging unit 13. Based on the above, the operator's operating hand is superimposed on the screen. That is, based on the touch operation by the operator on the touch operation unit 12, the operator superimposed on the screen operates the screen virtually at the corresponding position. The in-vehicle information processing system 10 causes the movement of the operator superimposed on the screen to correspond to the movement of the actual operator's operator captured by the imaging unit 13. Here, the operator is, for example, a driver driving the vehicle or a passenger sitting in the passenger seat, and the operator is, for example, a driver on the center console side or the passenger's own hand.

  As shown in FIG. 1, the display unit 11 has at least one screen. The display unit 11 may be configured by an arbitrary display device such as a liquid crystal display. When the display unit 11 is configured by a liquid crystal display, for example, the display unit 11 is disposed on an instrument panel. The display device constituting the display unit 11 may be a touch panel display or a display incapable of touch operation. In the following description, it is assumed that the display unit 11 is a display that cannot be touched.

  The in-vehicle information processing system 10 may include a so-called head-up display type device in addition to or instead of the display unit 11. In this case, the head-up display type device has a light emitting unit that generates display information as display light, reflects the generated display light toward an observer such as a driver, and the virtual image through the front windshield. Is displayed. The observer is not limited to the driver and may be a passenger sitting in the passenger seat.

  The display unit 11 displays information on the vehicle, function items for controlling the information, or a combination thereof. Hereinafter, these are collectively referred to as “display contents”. The information on the vehicle includes, for example, information such as air conditioning, car navigation, audio, an image around the vehicle by an electronic mirror, an inter-vehicle distance, a vehicle speed, a traveling position of the vehicle in a plurality of lanes, or a planned passing route with respect to an intersection. The function items for controlling the information include, for example, “return”, “forward”, “home”, “decision”, “various menus”, “temperature high / low”, “current location”, “volume high / low” ”,“ Enlargement / reduction ”,“ distance long / short ”,“ speed fast / slow ”,“ lane change ”, or“ passage selection ”. The display unit 11 may display each item as a character or an icon.

  For example, as shown to Fig.3 (a), the display part 11 displays various menus on one screen as a function item for controlling the information regarding a vehicle. Specifically, the display unit 11 displays “APPS” as an item for displaying various applications, “TEL” as an item for using a telephone, and “A / C” as an item for controlling an air conditioner. , “NAVI” is displayed as a menu for using the car navigation, and “AUDIO” is displayed as a menu for using the audio. Similarly, the display unit 11 displays “HOME” as an item for returning to the home screen, and “RETURN” as an item for returning to the previous screen.

  For example, as illustrated in FIG. 3B, the display unit 11 displays map information that is a part of the car navigation system on one screen as information about the vehicle. Further, the display unit 11 has functions such as “Destination setting”, “HOME”, and “RETURN” so as to be superimposed on the map information as a combination of information on the vehicle and functional items for controlling the information. Display items.

  The display unit 11 displays the operator's hand so as to overlap the display content. As shown in FIG. 3, the display unit 11 makes the operator's operating hand translucent and displays the above display content behind it. The display unit 11 is not limited to this, and may display an opaque operation hand so that the display content behind the operator is temporarily hidden when the operator's operation hand is superimposed.

  In the following, the degree of semi-transmission, that is, the transmittance is described as being constant without depending on the overlapping position, but is not limited thereto, and may be changed for each overlapping position. For example, at the position where the function items are displayed, the display unit 11 raises the transmittance above a predetermined value so that the operator can sufficiently recognize which item the operator should select. It may be displayed. Conversely, at a position where only the background is displayed, the display unit 11 may display the transmittance lower than a predetermined value.

  Moreover, the display part 11 may display the operator's operator hand which attached the gradation. The gradations described herein may include any step change with respect to light and darkness, color, or transmittance, or a combination thereof. The display unit 11 is preferably displayed in gradation by an arbitrary method in which the operator can easily view the display content behind. For example, the display unit 11 displays the operator by gradually increasing the brightness of the operator, gradually changing to a lighter color, or gradually increasing the transmittance as the fingertip of the operator's operator superimposed on the display approaches. May be.

  As the transmittance is lower, the reality of the operator's operator to be superimposed is improved, but it becomes difficult for the operator to visually recognize the display content behind. Conversely, the higher the transmittance, the easier it is for the operator to visually recognize the display content behind, but the reality is reduced. Therefore, it is preferable that the display unit 11 displays each display content by an arbitrary display method that allows the operator to easily view the display contents while ensuring the reality of the operator's operator to be superimposed.

  Although the display unit 11 has been described as superimposing an operator's hand in the real world on a virtual space in which the above-described display content is displayed, the present invention is not limited to this. For example, the display unit 11 may superimpose display contents and the like from the front surface of the operator's operator displayed on the screen as in a so-called mixed reality.

  As shown in FIG. 1, the touch operation part 12 is arrange | positioned at a center console, for example. The touch operation unit 12 includes a touch pad 121 and a tact switch 122 as shown in FIG. As shown in FIG. 4, the operator places his / her arm and wrist on the armrest and palmrest, respectively, and makes a part of the operation hand, for example, a finger contact the touchpad 121.

  The touch pad 121 detects contact by a contact object such as an operator's operating hand or a stylus pen at a corresponding contact position. In particular, the touch pad 121 detects contact with a part of the operator's operating hand, for example, a finger at a corresponding contact position. The operator operates information displayed on each screen constituting the display unit 11 by performing a touch operation on the touch operation unit 12, particularly the touch pad 121. The touch pad 121 is formed of, for example, transparent glass, and a touch sensor configured by an arbitrary method such as a resistive film method, a capacitance method, a surface acoustic wave method, an infrared method, or an electromagnetic induction method can be used. . In the following description, it is assumed that the touch pad 121 is a capacitive touch pad.

  As shown in FIG. 4, the tact switch 122 is disposed immediately below the touch pad 121 and supported by the substrate. When the touch pad 121 is displaced downward by the operator's pressing, the tact switch 122 is turned on by the pressing. The tact switch 122 is turned off when the push of the operator is released and the touch pad 121 returns to the original position. When the operator depresses the touch pad 121, the tact switch 122 is turned on to obtain a click feeling.

  In FIG. 4, one tact switch 122 is arranged at the central portion immediately below the touch pad 121. However, the tact switch 122 is not limited to this, and any number can be used as long as the pressure from the touch pad 121 can be detected. It may be arranged at any place. For example, one tact switch 122 may be disposed on the outer periphery of the tact switch 122 immediately below the touch pad 121, or a plurality of tact switches 122 may be disposed at dispersed positions. For example, by arranging a plurality of tact switches 122 immediately below the touch pad 121, the touch operation unit 12 is configured to be able to detect a pressure from the touch pad 121 for each predetermined area of the touch pad 121. Also good. That is, the touch operation unit 12 is configured to be able to detect at which position on the touch pad 121 the operator has pressed when a plurality of fingers of the operator are in contact with the touch pad 121 at the same time. Also good.

  The configuration unit arranged immediately below the touch pad 121 is not limited to the tact switch 122, and may be any configuration as long as the pressing from the touch pad 121 can be detected. For example, instead of the tact switch 122, a pressure sensitive sensor such as a piezoelectric element may be disposed immediately below the touch pad 121.

  In addition to the touch pad 121 and the tact switch 122, the touch operation unit 12 may include a filter in order to remove an unnecessary detection signal of the touch pad 121 due to vibration during traveling of the vehicle.

  The imaging unit 13 includes at least one camera and is disposed on, for example, a roof panel. The camera which comprises the imaging part 13 is arrange | positioned so that the inside of a vehicle may be imaged from a roof panel. More specifically, the imaging unit 13 captures an image of at least a part of an operator's operator who performs a touch operation with the touch operation unit 12 and the touch operation unit 12 from above.

  The imaging unit 13 may image at least a part of the operator's operating hand, for example, only five fingers, but it is preferable to image the entire operator's operating hand including the back of the hand. Here, the entire operating hand is the entire part including from the vicinity of the wrist of the operator to the fingertip. In this case, the imaging unit 13 is preferably arranged above the operator, such as a roof panel, in order to easily image the entire operator's operator.

  The imaging unit 13 is not limited to such an arrangement, and may be arranged at an arbitrary place as long as at least a part of the operator's operating hand can be imaged. For example, the imaging unit 13 may be arranged directly below the transparent touch pad 121 and may capture a part of the operator's operator performing a touch operation with the touch pad 121 from below. In this case, for example, by changing the palm rest portion in FIG. 4 to an arbitrary transparent support member, the imaging unit 13 can also image the entire operator's operating hand including the back of the hand.

  The imaging unit 13 is preferably composed of a camera with a wide dynamic range so that the operator's hand can be clearly imaged both in a bright state during the day and in a dark state at night. The image captured by the camera may be a black and white image or a color image.

  The imaging unit 13 is not limited to a configuration with a camera having a wide dynamic range, and may be configured with a camera capable of imaging in a bright state in the daytime. In this case, the imaging unit 13 may irradiate the operator's operating hand on the touch pad 121 with a spotlight from above in order to clearly capture the operator's operating hand even at night.

  When the vehicle performs automatic driving, it is assumed that the operator leans on the reclining seat and leans on the seat in a relaxed state. At this time, if the position of the touch operation unit 12 is fixed, the operator needs to perform the touch operation by extending his arm while leaning on the seat, and feels inconvenience. Therefore, for example, by configuring the center console on which the touch operation unit 12 is disposed so as to be lowered rearward in conjunction with the movement of the reclining seat, the operator can easily perform the touch operation without extending the arm. Can do. In the case of such a configuration, the imaging unit 13 needs to image the operator on the touch pad 121 in accordance with each position of the touch operation unit 12 that is linked to the movement of the reclining seat.

  Therefore, in addition to the above, the imaging unit 13 is preferably configured with a camera having a wide angle of view. The imaging unit 13 is not limited to this, and may be configured such that the angle of the camera itself changes in conjunction with a change in the position of the touch operation unit 12 even if the camera has a narrow angle of view. Similarly, the imaging unit 13 may be configured such that the camera itself translates in conjunction with a change in the position of the touch operation unit 12. Further, the position of the touch operation unit 12 that changes in conjunction with the movement of the reclining seat is limited to two positions, for example, a position at the time of manual operation and a position at the time of automatic operation, so as to correspond to each position. Two cameras may be arranged.

  The operation part 14 is arrange | positioned, for example on the upper surface of a palm rest, as shown in FIG. As shown in FIG. 4, the operator puts his / her arms and wrists on the armrest and palmrest, respectively, and makes a part of the operator touch the upper surface of the operation unit 14. The part of the operating hand described here is, for example, the palm side of the wrist, that is, the wrist part.

  As shown in FIG. 5, the operation unit 14 has, for example, a four-way switch such as a cross key on the upper surface thereof. The operator places a part of the operator on the upper surface of the operation unit 14, particularly the surface of the four-way switch. The operation unit 14 detects the operation of at least a part of the operator's operator touching the four-way switch. For example, when the operator pushes the four-way switch to the back side with the wrist, the operation unit 14 detects the pushing operation to the back side. Similarly, when the operator pushes the four-way switch to the near side, the left side, or the right side at the wrist, the operation unit 14 detects the push-in operation toward the near side, the left side, or the right side. That is, the operation unit 14 can detect the movements in the four directions of the operator's wrist. The operation unit 14 transmits a signal based on the detected operation to the control unit 15.

  The operation unit 14 will be described below as having a four-way switch, but is not limited thereto. For example, the operation unit 14 may have an 8-way switch. In this case, the operation unit 14 can detect at least a part of eight-direction motions of the operator's operating hand. Further, the operation unit 14 is not limited to the direction switch, and may have an arbitrary configuration as long as it can detect at least a part of the operation of the operator's operator's hand.

  The control unit 15 is a processor that controls and manages the entire in-vehicle information processing system 10 including each functional block of the in-vehicle information processing system 10. The control unit 15 includes a processor such as a CPU (Central Processing Unit) that executes a program that defines a control procedure, and the program is stored in the storage unit 16, for example.

  The control unit 15 acquires the contact information detected on the touch pad 121 from the touch operation unit 12 as an input signal. Specifically, the control unit 15 acquires detection information regarding a contact object, for example, a contact by an operator's finger and a corresponding contact position. The control unit 15 identifies accurate position coordinates on the touch pad 121 where the touch operation is performed based on the detection information regarding the corresponding contact position.

  The control unit 15 acquires a signal related to the on state or the off state of the tact switch 122 from the touch operation unit 12. Specifically, when the operator depresses the tact switch 122 via the touch pad 121, the control unit 15 acquires an on-state signal. When the operator stops pressing the touch pad 121 and releases the pressing of the tact switch 122, the control unit 15 acquires an off-state signal. The control unit 15 identifies the on state or the off state of the tact switch 122 based on the acquired signal.

  The control unit 15 selects a corresponding item on the screen constituting the display unit 11 when the touch pad 121 detects contact by a part of the operator's operating hand. At this time, the control unit 15 highlights the item. Highlighting is to display a predetermined item with emphasis. The control unit 15 provides feedback to the operator that the above item is in a selected state by highlighting. For example, as illustrated in FIG. 3A, when contact with the operator's finger is detected at a corresponding position on the touch pad 121, the control unit 15 highlights the function item “NAVI” on the screen. At this time, the control unit 15 superimposes and displays the operator's operating hand at a corresponding position based on the image captured by the imaging unit 13.

  Subsequently, when the tact switch 122 is turned on by pressing the touch pad 121 by a part of the operator's operating hand, the control unit 15 determines the selection of a predetermined item on the screen. The operation for confirming selection of a predetermined item on the screen is not limited to this, and may be an arbitrary operation such as a double tap on the touch pad 121, for example. In this case, the touch operation unit 12 may not have the tact switch 122.

  For example, in FIG. 3A, when the tact switch 122 is turned on by pressing by a part of the operator's operator's hand, the control unit 15 confirms selection of the item “NAVI” displayed on the screen. Let At this time, the control unit 15 causes the same movement to be displayed on the screen in accordance with a part of the operator's hand touching the touch pad 121, for example, pressing of the index finger or double tap.

  The control unit 15 causes the display unit 11 to display information on the vehicle, function items for controlling the information, or a combination thereof. Further, the control unit 15 superimposes and displays at least a part of the operator's operator's hand on the screen at a display magnification based on the size of the operation area on the screen constituting the display unit 11 by image processing to be described later. Let

  The control unit 15 acquires a signal detected by the operation unit 14 based on at least a part of the operator's operating hand, for example, the movement of the wrist. For example, the control unit 15 identifies the direction of the operation based on the acquired signal. The control unit 15 selects a screen for the operator to operate display contents based on the detected operation.

  The control unit 15 refers to various information stored in the storage unit 16. Specifically, the control unit 15 refers to information related to the vehicle or information related to a function item for controlling the information. The control unit 15 refers to information regarding the on state or the off state of the tact switch 122. The control unit 15 refers to the image information captured by the imaging unit 13. The control unit 15 refers to information regarding the operator's operator who has been subjected to image processing and is finally superimposed on the display unit 11.

  The storage unit 16 can be configured by a semiconductor memory, a magnetic memory, or the like, and stores various information described above, a program for operating the vehicle information processing system 10, and the like. The storage unit 16 also functions as a work memory. For example, the storage unit 16 stores information on the operator of the operator who has been subjected to image processing, which is finally superimposed on the display unit 11.

  Below, with reference to FIG.6 and FIG.7, the image processing which the vehicle-mounted information processing system 10 performs is demonstrated in detail.

  FIG. 6 is a schematic diagram illustrating an example of a correspondence relationship between a predetermined area of the touch operation unit 12 and an operation area on the screen configuring the display unit 11. FIG. 6A shows a predetermined region R1 of the touch operation unit 12. FIG. 6B shows an operation area R <b> 2 on the screen constituting the display unit 11.

  The control unit 15 sets a predetermined region R1 of the touch operation unit 12 and an operation region R2 on the screen constituting the display unit 11. The predetermined area R1 of the touch operation unit 12 is an area for the operator to perform a touch operation with a part of the operator. For example, the predetermined region R1 of the touch operation unit 12 is a part or the entire region of the touch pad 121. The operation area R2 on the screen constituting the display unit 11 is an area on the screen that can be virtually operated by the operator's operator hand superimposed on the screen. For example, the operation area R2 on the screen constituting the display unit 11 is a part or the entire area of the screen.

  As shown in FIG. 6A, the predetermined region R1 of the touch operation unit 12 is set on the back side of the touch operation unit 12 so that the operator's operator's hand is superimposed on the entire touch operation unit 12. Is preferred. The back side of the touch operation unit 12 is, for example, the back side of the touch pad 121 constituting the touch operation unit 12. That is, as shown in FIGS. 4 and 6, the back side of the touch operation unit 12 is an area on the touch pad 121 that is farthest from the wrist when the operator's arm and wrist are arranged on the armrest and palmrest, respectively. .

  The predetermined region R1 of the touch operation unit 12 is not limited to this, and may be an arbitrary partial region on the touch pad 121 or the entire region as described above. When the predetermined area R1 of the touch operation unit 12 is an arbitrary partial area on the touch pad 121, the areas on the touch pad 121 other than the predetermined area R1 are configured not to react to the touch operation. May be.

  On the other hand, the operation area R2 on the screen constituting the display unit 11 is set at the upper part of the screen so as to correspond when the predetermined area R1 is set on the back side of the touch pad 121. That is, the back side and the near side of the touch pad 121 correspond to the upper part and the lower part of the screen, respectively. Although it is most intuitive to make each correspond in this way, the correspondence relationship between the touch pad 121 and the screen constituting the display unit 11 is not limited to this. For example, the above correspondence relationship may be reversed, and the near side and the far side of the touch pad 121 may correspond to the upper part and the lower part of the screen, respectively. In this case, in order to superimpose the operator's operation hand on the entire touch operation unit 12, the operation region R2 on the screen constituting the display unit 11 is associated with the predetermined region R1 on the back side of the touch operation unit 12. , May be set at the lower part thereof.

  The control unit 15 associates the set position coordinates in the predetermined region R1 of the touch operation unit 12 with the position coordinates in the operation region R2 on the screen constituting the display unit 11. For example, when the predetermined area R1 of the touch operation unit 12 is a square area of a part or the whole of the touch pad 121, and the operation area R2 on the screen is a part of or a whole square area of the screen. think of. In this case, the control unit 15 associates the four vertices of the predetermined area R1 with the four vertices of the operation area R2. The control unit 15 can determine the correspondence between the position coordinates of each point located in the square area connecting the vertices by identifying the correspondence between the position coordinates of the four vertices.

  Such processing may be executed as calibration at an initial stage when the in-vehicle information processing system 10 is mounted on the vehicle, or may be executed as needed.

  FIG. 7 is a schematic diagram showing a state of image processing performed by the in-vehicle information processing system 10. FIG. 7A shows the state of the operator's operator performing a touch operation on the touch operation unit 12. FIG. 7B shows the state of the operator's operator hand superimposed on the screen constituting the display unit 11.

  The control unit 15 acquires image information captured by the camera from the imaging unit 13. As shown by the region R <b> 3 in FIG. 7, the captured image includes at least a part of an operator's hand performing a touch operation with the touch operation unit 12, and the touch operation unit 12, particularly the touch pad 121. That is, the imaging unit 13 images the positional relationship between the touch operation unit 12 and the operator's operating hand. In addition, as described above, the control unit 15 associates the position coordinates in the predetermined region R1 with the position coordinates in the operation region R2, so that it corresponds to the position of the operator's operator's hand in the touch operation unit 12. In addition, at least a part of the operator can be displayed in a superimposed manner on the screen.

  When the operator's operator's hand is superimposed on the display unit 11, the control unit 15 performs image processing for extracting a part or the whole of the operator's operator's hand based on the above image. That is, the control unit 15 removes image information such as an external background from the contour of the operator's operator's hand. In addition, by surrounding the periphery of the touch pad 121 with a black edge, the control unit 15 can easily extract the operator's operator based on the captured image.

  Further, when the captured image is a black and white image, the control unit 15 may or may not color the operating part by image processing. In order to further improve the reality of the operator's operator displayed on the display unit 11, the control unit 15 is preferably colored by image processing.

  On the other hand, when the captured image is a color image, the control unit 15 preferably superimposes and displays the image on the screen as it is in accordance with the actual color and brightness of the operator's operator. The control unit 15 is not limited to this, and in order to make it easier to visually recognize the display content behind, the control unit 15 eliminates the color and brightness of the operating part, and instead performs image processing that adds a predetermined color, for example. You may go. In addition, the control unit 15 may perform image processing in which the color of the operating part, brightness, and the like are excluded, and the operating part is completely colorless and transparent. In this case, the control unit 15 displays only the part near the contour of the operator on the screen.

  In the following description, it is assumed that the image captured by the imaging unit 13 is a color image, and that the control unit 15 superimposes and displays the image on the screen as it is according to the actual color and brightness of the operator's operator. That is, the description will be made assuming that the control unit 15 does not need to perform image processing relating to color, brightness, and the like.

  The control unit 15 operates the operator based on the ratio of the size between the predetermined region R1 of the touch operation unit 12 in the captured image and the operation region R2 on the screen constituting the display unit 11. Determine the hand display magnification. For example, when the predetermined area R1 of the touch operation unit 12 is a square area of a part or the whole of the touch pad 121, and the operation area R2 on the screen is a part of or a whole square area of the screen. think of. In this case, the control unit 15 calculates the ratio between the length of each side of the predetermined region R1 of the touch operation unit 12 and the length of each corresponding side of the operation region R2 on the screen. Based on the ratio, the control unit 15 determines the display magnification of the operator's operator's hand to be displayed on the display unit 11 in a superimposed manner.

  Based on the determined display magnification, the control unit 15 superimposes and displays the captured operator's operator on the display unit 11 in an enlarged or reduced state or as it is.

  The control unit 15 may make the display magnification of the operator's operator the same as the above ratio, or may be a different value based on the above ratio. The display magnification determination process may be executed simultaneously with the above-described calibration at the initial stage when the in-vehicle information processing system 10 is mounted on the vehicle, or may be executed as needed. Once the display magnification is determined, the control unit 15 may fix the magnification or change the display magnification according to the situation. For example, the control unit 15 may change the display magnification of the operator's operator between daytime and nighttime, or may appropriately change the display magnification according to the operator's setting.

  For example, when the display unit 11 is configured by a plurality of screens, the control unit 15 may change the display magnification of the operator's operator to be superimposed and displayed based on the size of the operation area R2 of each screen. Good. Without being limited thereto, the control unit 15 derives, for example, an average value based on the size of the operation region R2 of each screen, and the operator's operator who displays the average value based on the average value. The display magnification may be constant.

  The control unit 15 may change the display magnification of the operator's operator according to not only the size of the operation area R2 on the screen but also the content displayed on the screen. For example, when the display unit 11 displays a map or a function item and the operator operates each, the control unit 15 reduces the display magnification of the operator to be lower than usual so that the operator can easily operate. Then, the display unit 11 may be displayed in a superimposed manner.

  For example, the control unit 15 may change the display magnification for each operator in order to match the size of the operator displayed superimposed on the display unit 11 between operators having different sizes of the operator. Good. On the other hand, the control unit 15 makes the display magnification of the operator superimposed on the display unit 11 constant among the operators having different sizes of the operator, and adjusts it to the size of the actual operator's operator. May be displayed in a superimposed manner.

  It is preferable that the control unit 15 performs image processing within a predetermined time based on the image captured by the imaging unit 13. The predetermined time is a time delay between the timing of the operation by the operator's actual operator and the timing of the movement of the operator's operator superimposed on the screen, and the operator is not aware of it. Means a time delay. That is, it is preferable that the control unit 15 completes the image processing within a time sufficiently shorter than the time delay in which the operator feels uncomfortable with the operation due to the reaction speed and the recognition ability of the operator. For example, it is preferable that the control unit 15 limit the image processing only to the above-described extraction of the operator's operator's hand that has been imaged and adjustment of the display magnification.

  That is, the position coordinates in the predetermined region R1 of the touch operation unit 12 where the touch operation by the operator's operator is detected is not identified by image processing of the captured image by the imaging unit 13, but as described above. It is preferable that identification is performed based on detection information from the touch operation unit 12, particularly the touch pad 121.

  Although the control unit 15 is described as performing two image processes as described above, the present invention is not limited to this, and three or more image processes may be performed within a predetermined time. In this case, for example, the position coordinates in the predetermined region R1 of the touch operation unit 12 where the touch operation by the operator's operator is detected may be identified by image processing of the captured image by the imaging unit 13.

  The control unit 15 refers to information on the predetermined region R1 of the touch operation unit 12 and the operation region R2 on the screen constituting the display unit 11 from the storage unit 16 when performing the above image processing. That is, the control unit 15 refers to information regarding the position coordinates in the predetermined region R1 of the touch pad 121 corresponding to the detection information. The control unit 15 refers to information regarding the position coordinates in the operation region R2 of each screen constituting the display unit 11. The control unit 15 refers to information on the display magnification of the operator's operator's hand to be displayed in a superimposed manner determined by calibration or the like.

  Below, with reference to FIG. 8, an example of operation which an operator selects arbitrary one screen made into operation object among the several screens which comprise the display part 11 is demonstrated in detail.

  FIG. 8 is an enlarged view of the display unit 11 shown in FIG. The display unit 11 has four screens 111, 112, 113, and 114. The screen 111 includes two display layers 1111 and 1112 that are different in the depth direction. For example, the display layer 1111 of the screen 111 displays an operation screen, and the display layer 1112 displays information about the vehicle such as the inter-vehicle distance, the vehicle speed, the traveling position of the host vehicle in a plurality of lanes, or the planned passing route with respect to the intersection, that is, the host vehicle. The running state of is displayed. For example, the screens 112 and 113 display an image around the vehicle by an electronic mirror. For example, the screen 114 displays function information for controlling map information related to car navigation and information related to vehicles.

  In FIG. 8, the number of screens constituting the display unit 11 is four and the number of display layers of the screen 111 is two. However, the present invention is not limited to this. The display unit 11 may be configured by an arbitrary number of screens, and the screen 111 may be configured by an arbitrary number of display layers. Moreover, although it demonstrates as what has a display layer only in the screen 111, it is not limited to this, The other screen which comprises the display part 11 may also have a some display layer. Further, the content displayed on each screen is not limited to the above, and any screen may display any content.

  Based on the signal acquired from the operation unit 14, the control unit 15 determines the movement direction from the currently selected screen to the next selected screen, at least a part of the detected operator's operating hand (carpal portion). ) Corresponding to the direction of movement. For example, in a state where the screen 111 is selected, when the operator pushes the four-way switch to the left side with the wrist, the control unit 15 selects the screen 112 installed on the left side of the screen 111. Similarly, when the operator depresses the four-way switch to the right side with the wrist while the screen 111 is selected, the control unit 15 selects the screen 113 installed on the right side of the screen 111. In this state, when the operator pushes the four-way switch to the right again at the wrist, the control unit 15 selects the screen 114 installed on the right side of the screen 113.

  In a state where the screen 112 arranged at the left end of the display unit 11 is selected, when the operator further pushes the four-way switch to the left side, the control unit 15 may return to the right end and select the screen 114. The selection may not be changed, and the control may be performed so that the selection still remains on the screen 112. Even when the operator presses the four-way switch further to the right side with the screen 114 arranged at the right end of the display unit 11 selected, the control unit 15 may perform the same control.

  Based on the signal acquired from the operation unit 14, the control unit 15 determines the movement direction from the currently selected display layer to the next selected display layer at least a part of the detected operator's operator's hand. Correspond to the direction of movement. For example, when the display layer 1111 is selected and the operator pushes the four-way switch to the back side with the wrist, the control unit 15 selects the display layer 1112 arranged on the back side of the display layer 1111. To do. Conversely, when the display layer 1112 is selected and the operator pushes the four-way switch to the near side with the wrist, the control unit 15 moves the display layer 1111 disposed on the near side of the display layer 1112. select.

  For example, when the in-vehicle information processing system 10 includes a head-up display type device in addition to the display unit 11, the target selected by the above operation is a front that displays a virtual image in addition to each screen and each display layer. It may be a windshield. In this case, for example, when the display layer 1112 is selected and the operator pushes the four-way switch to the back side with the wrist, the control unit 15 causes the front window disposed on the back side of the display layer 1112 to be Select a shield. That is, the control unit 15 identifies the display layers 1111 and 1112 and the front windshield as a single hierarchical structure.

  In a state where the display layer 1112 or the front windshield arranged at the back end of the display unit 11 is selected, when the operator further pushes the four-way switch to the back side, the control unit 15 returns to the front end and returns to the display layer. 1111 may be selected, or the selection may not be changed, and control may be performed so that the display layer 1112 or the front windshield still remains. The control unit 15 may perform the same control even when the operator pushes the four-way switch further forward while the screen 111 arranged at the front end of the display unit 11 is selected.

  When the operator presses the four-way switch to the left side with the display layer 1112 or the front windshield selected, the control unit 15 selects the screen 112 installed on the left side. When the operator presses the four-way switch to the right side with the display layer 1112 or the front windshield selected, the control unit 15 selects the screen 113 installed on the right side.

  When the operator presses the four-way switch in the direction of selecting the screen 111 while the screen 112 or 113 is selected, the control unit 15 causes the specific display layer or the front windshield (for example, the front end of the front end) to be selected. The selection may be returned to the display layer 1111), or the selection may be returned to the display layer or front windshield selected immediately before.

  The method of selecting the screen, the display layer, or the front windshield is not limited to the above, and any method may be used as long as it corresponds to the direction of movement of at least a part of the detected operator's operator's hand. .

  The control unit 15 may or may not suppress the display level of the display layer that is not selected on the screen 111. In view of the visibility of the operator, it is preferable that the control unit 15 suppresses the display degree of the display layer that is not selected.

  Specifically, the control unit 15 may reduce the luminance of a display layer that is not selected on the screen 111. Moreover, the control part 15 may suppress the degree of display by graying out. Further, the control unit 15 may exclude all RGB of the display layer that is not selected, or may exclude only one or two elements of RGB. Further, the control unit 15 may blur the display layer that has not been selected. Furthermore, the control part 15 does not need to display the display layer which has not been selected in the first place. Without being limited to these methods, the control unit 15 performs control by any method that can relatively improve the visibility of the selected display layer by reducing the visibility of the display layer that is not selected. May be.

  Next, a method for the operator to control the traveling state of the host vehicle will be described in detail with reference to FIGS. 9 to 12. At this time, it is assumed that the control unit 15 selects the display layer 1112 of the screen 111 by a selection operation on the operation unit 14 by the operator. The control unit 15 controls the traveling state of the host vehicle based on a part of the operator's operating hand on the touch operation unit 12, in particular, the movement of the finger. Based on the image picked up by the image pickup unit 13, the control unit 15 calculates the actual finger movement on the touch operation unit 12 and the finger movement of the operator's operator's finger displayed superimposed on the display layer 1112. Make it correspond.

  The operator moves one finger on the touch operation unit 12, particularly the touch pad 121, and controls the traveling state of the own vehicle based on the movement. The movement of one finger includes a swipe operation that moves the contact point of the finger on the touch pad 121, a contact operation that presses the touch pad 121 in the vertical direction, a tap operation that taps the surface of the touch pad 121, or a twist. Operation is possible. The movement of one finger is not limited to these, and may be any movement as long as the touch pad 121 can detect the movement.

  The control unit 15 is not limited to the movement of one finger, and may control the traveling state of the host vehicle based on the movement of two or more fingers. As the movement of two or more fingers, a swipe operation, a pinch operation that expands or contracts the interval between contact points of two fingers, or a rotation operation that rotates two or more fingers simultaneously can be considered. The movement of the two fingers is not limited to these, and may be any movement as long as the touchpad 121 can detect the movement.

  Below, the traveling state of the own vehicle is the positional relationship between the own vehicle and the preceding vehicle, that is, the inter-vehicle distance, the speed of the own vehicle, the traveling position of the own vehicle in a plurality of lanes, or the planned passing route with respect to an intersection, or a combination thereof. Although described as a thing, it is not limited to these. The control unit 15 can control any traveling state of the host vehicle in any combination.

  As shown in FIGS. 9 to 12, the control unit 15 may cause the display layer 1112 to display the traveling state of the vehicle by visual information. The visual information includes any information that can be visually recognized by the operator, such as letters, numbers, figures, symbols, colors, or combinations thereof. The following display method is an example, and the control unit 15 may display any traveling state of the host vehicle in any combination by any method that can be easily visually recognized by the operator.

  FIG. 9 is a schematic diagram illustrating an example of the control of the inter-vehicle distance by the operator.

  For example, the display layer 1112 may display the positional relationship between the own vehicle and the preceding vehicle, that is, the inter-vehicle distance as characters and numbers such as “50 m” in a state where the own vehicle and the preceding vehicle are displayed in a graphic form. It may be expressed by the size of an arrow displayed between the own vehicle and the preceding vehicle. In the case of expressing with an arrow, for example, the display layer 1112 may express the arrow thicker and shorter as the inter-vehicle distance is shorter, and conversely, the arrow may be thinner and longer as the inter-vehicle distance is longer. In addition, the control unit 15 may color the arrow with a conspicuous color (for example, red) as the inter-vehicle distance is short, and conversely, the arrow is colored with a calm color (for example, blue) as the inter-vehicle distance is long. May be.

  For example, the operator touches one finger at a position on the touch pad 121 corresponding to the position of the own vehicle on the display layer 1112 and swipes toward the near side. At this time, the control unit 15 controls the host vehicle so as to increase the actual inter-vehicle distance while increasing the inter-vehicle distance displayed on the display layer 1112 according to the swipe operation of the operator.

  On the other hand, when the operator swipes to the back side, for example, the control unit 15 reduces the vehicle-to-vehicle distance displayed on the display layer 1112 and also reduces the actual vehicle-to-vehicle distance according to the operator's swipe operation. To control.

  On the other hand, in order to control the inter-vehicle distance, for example, the operator touches one finger at a position on the touch pad 121 corresponding to the position of the front vehicle on the display layer 1112, and swipes to the back side or the front side. You may let them.

  Further, the operation for performing the above control is not limited to the swipe operation with one finger, and may be any operation. For example, the operator may swipe the two fingers by touching the two positions on the touch pad 121 corresponding to both ends of the vehicle on the display layer 1112 with two fingers. At this time, the operating hand superimposed and displayed on the display layer 1112 moves the vehicle represented by a figure to the front side or the back side by sandwiching the vehicle with two fingers.

  Similarly, for example, the operator may make a pinch by bringing two fingers into contact with two positions on the touch pad 121 corresponding to the positions of the host vehicle and the front vehicle on the display layer 1112. At this time, the control unit 15 widens or narrows the inter-vehicle distance according to the pinch-out operation or the pinch-in operation.

  FIG. 10 is a schematic diagram illustrating an example of control of the vehicle speed of the host vehicle by the operator.

  For example, the display layer 1112 may display the vehicle speed as “50 km / h” in characters and numbers, or may be expressed by the size of an arrow displayed in the traveling direction of the host vehicle. In the case of expressing with an arrow, for example, the display layer 1112 may express the arrow thicker and longer as the vehicle speed increases, and conversely as the vehicle speed decreases. In addition, the control unit 15 may color the arrow with a conspicuous color (for example, red) as the vehicle speed increases, and conversely, the arrow with a calm color (for example, blue) as the vehicle speed decreases. Also good.

  For example, the operator touches one finger at a position on the touch pad 121 corresponding to the position of the own vehicle on the display layer 1112 and swipes toward the near side. At this time, the control unit 15 controls the host vehicle so as to reduce the actual vehicle speed while reducing the vehicle speed displayed on the display layer 1112 according to the swipe operation of the operator.

  On the other hand, when the operator swipes to the back side, for example, the control unit 15 controls the own vehicle so as to increase the actual vehicle speed while increasing the vehicle speed displayed on the display layer 1112 according to the swipe operation of the operator. To do.

  Further, the operation for performing the above control is not limited to the swipe operation with one finger, and may be any operation. For example, the operator may swipe the two fingers by touching the two positions on the touch pad 121 corresponding to both ends of the vehicle on the display layer 1112 with two fingers. At this time, the operating hand superimposed and displayed on the display layer 1112 moves the vehicle represented by a figure to the front side or the back side by sandwiching the vehicle with two fingers.

  Similarly, the operator may pinch at an arbitrary position on the touch pad 121, for example. At this time, the control unit 15 decreases or increases the vehicle speed according to the pinch-out operation or the pinch-in operation.

  FIG. 11 is a schematic diagram illustrating an example of control of the traveling position of the host vehicle in a plurality of lanes by the operator.

  For example, the display layer 1112 may display the traveling position of the host vehicle in a plurality of lanes as characters such as “right lane” or may be expressed by the host vehicle and an arrow displayed in the traveling direction thereof. . For example, when the display layer 1112 is represented by an arrow, the display layer 1112 represents a thick and long arrow when positioned in the right lane (passing lane), and conversely, a thin and short arrow when positioned in other lanes. May be. In addition, when the control unit 15 is located in the right lane (passing lane), the control unit 15 may represent an arrow with a conspicuous color (for example, red), and conversely, the control unit 15 is located in another lane. Sometimes, an arrow may be expressed by a calm color (for example, blue).

  In addition, the display layer 1112 may display a planned route for changing lanes, such as “change to left lane” in characters, or may be expressed by an arrow having a shape along the planned route. When expressed by an arrow, for example, the display layer 1112 may change the thickness of the arrow according to the vehicle speed when the lane is changed. That is, the display layer 1112 may be expressed as the arrow becomes thinner as the vehicle speed of the host vehicle is lower, and vice versa. In addition, the control unit 15 may color the arrow with a calm color (for example, blue) as the vehicle speed when changing the lane is low, or conversely, as the vehicle speed is high, the color is more conspicuous (for example, red). The arrow may be colored by

  For example, the operator touches one finger at a position on the touch pad 121 corresponding to the position of the own vehicle on the display layer 1112, and swipes across the displayed lane. At this time, the control unit 15 controls the own vehicle so as to change the actual traveling lane while changing the current lane displayed on the display layer 1112 according to the swipe operation of the operator.

  Further, the operation for performing the above control is not limited to the swipe operation with one finger, and may be any operation. For example, the operator may swipe the two fingers by touching the two positions on the touch pad 121 corresponding to both ends of the vehicle on the display layer 1112 with two fingers. At this time, the operator superimposed and displayed on the display layer 1112 moves the vehicle represented by the figure to a different lane by sandwiching the vehicle with two fingers.

  For example, the control unit 15 may display a dummy of the own vehicle represented by a dotted line when the operator long presses the touch pad 121 with a finger. The display layer 1112 is, for example, on a planned route for changing the lane by pinching a dummy with the operator displayed in a superimposed manner while the vehicle represented by the figure is placed at a position corresponding to the current traveling lane. May be displayed to move.

  FIG. 12 is a schematic diagram illustrating an example of the control of the scheduled passage route for the intersection by the operator.

  For example, the display layer 1112 displays the positional relationship between the intersection and the own vehicle as a graphic. In addition, the display layer 1112 may display the planned passage route for the intersection as “turn right” or the like in characters, or may be expressed by an arrow. In the case of expressing with an arrow, for example, the display layer 1112 may display an arrow having a shape along the planned passing route, or may express the planned approach speed to the intersection by the thickness of the arrow. In other words, the display layer 1112 may express the arrow thinner as the planned approach speed to the intersection of the own vehicle is smaller, and vice versa. In addition, the control unit 15 may color the arrow with a calm color (for example, blue) as the planned approach speed to the intersection is small, or conversely, as the planned approach speed is large, a conspicuous color (for example, The arrow may be colored with red.

  For example, the operator touches one finger at a position on the touch pad 121 corresponding to the position of the host vehicle on the display layer 1112, and swipes along a planned passing route at an intersection. At this time, the control unit 15 causes the display layer 1112 to display the above-described arrow in accordance with the operator's swipe operation. Thereafter, the control unit 15 controls the host vehicle so as to pass through the intersection according to the planned passing route.

  Further, the operation for performing the above control is not limited to the swipe operation with one finger, and may be any operation. For example, the operator may swipe the two fingers by touching the two positions on the touch pad 121 corresponding to both ends of the vehicle on the display layer 1112 with two fingers. At this time, the operator superimposed and displayed on the display layer 1112 moves the vehicle along the planned passage route so as to sandwich the vehicle represented by the figure with two fingers.

  For example, the control unit 15 may display a dummy of the own vehicle represented by a dotted line when the operator long presses the touch pad 121 with a finger. For example, the display layer 1112 may be displayed in such a manner that an operator who is displayed in a superimposed manner pinches a dummy and moves it on the scheduled passage route while the own vehicle represented by a graphic is placed at the current traveling position. Good.

  FIG. 13 is a flowchart showing an example of the operation of the in-vehicle information processing system 10. FIG. 13A is a flowchart mainly showing control when the operator selects and operates the screen by the operation unit 14. FIG. 13B is a flowchart when the operator controls the traveling state of the own vehicle.

  First, the control unit 15 performs calibration. That is, the control unit 15 associates the set position coordinates in the predetermined region R1 of the touch operation unit 12 with the position coordinates in the operation region R2 on the screen constituting the display unit 11 (step S10).

  Subsequently, the control unit 15 determines the display magnification of the operator's operator's hand to be displayed superimposed on the display unit 11 by calibration or the like (step S11).

  Next, the control unit 15 determines whether the operator's operating hand is superimposed on the touch operation unit 12 based on the image captured by the imaging unit 13 (step S12).

  When it is determined that the operator's operating hand is superimposed on the touch operation unit 12, the control unit 15 proceeds to step S13. When it is determined that the operator's operating hand is not superimposed on the touch operation unit 12, the control unit 15 returns to step S12 again and waits until the operator's operating hand is superimposed.

  When it is determined that the operator's operating hand is superimposed on the touch operation unit 12, the control unit 15 performs image processing for extracting a part or the whole of the operator's operating hand (step S13).

  Subsequently, the control unit 15 superimposes and displays the captured operator's hand based on the display magnification determined in step S11 (step S14).

  Subsequently, the control unit 15 determines whether or not detection information regarding at least a part of the operation of the operator's operating hand has been acquired from the operation unit 14 (step S15).

  When acquiring the detection information, the control unit 15 proceeds to step S16. When the detection information is not acquired, the control unit 15 proceeds to step S18.

  When acquiring the detection information, the control unit 15 newly selects a screen corresponding to the identified direction of operation (step S16).

  The control unit 15 displays the captured operator's operator's hand in a superimposed manner based on the display magnification adapted to the newly selected screen (step S17).

  Subsequently, the control unit 15 determines whether detection information related to the touch operation is acquired from the touch operation unit 12 (step S18).

  When acquiring the detection information, the control unit 15 proceeds to step S19. When the detection information is not acquired, the control unit 15 returns to step S18 again and waits until the detection information is acquired.

  When acquiring the detection information, the control unit 15 further determines whether the currently selected screen displays the traveling state of the host vehicle (step S19).

  When it is determined that the traveling state of the own vehicle is not displayed, the control unit 15 proceeds to step S20. When determining that the traveling state of the host vehicle is displayed, the control unit 15 proceeds to step S21 in FIG.

  When it is determined that the traveling state of the vehicle is not displayed, the control unit 15 performs an operation based on the touch operation on the touch operation unit 12 on the currently selected screen (step S20). Thereafter, the flow ends.

  On the other hand, if it is determined that the traveling state of the host vehicle is displayed, the control unit 15 determines the traveling state of the subject vehicle to be controlled in steps S21 to S24. The determination from step S21 to step S24 is not limited to this order, and may be performed in an arbitrary order or may be performed in parallel at the same time.

  Specifically, the control unit 15 determines whether or not the traveling state of the subject vehicle to be controlled is the inter-vehicle distance (step S21).

  When the control unit 15 determines that the traveling state of the subject vehicle to be controlled is not the inter-vehicle distance, the control unit 15 proceeds to step S22. If the control unit 15 determines that the distance is the inter-vehicle distance, the control unit 15 proceeds to step S25.

  Subsequently, when it is determined that the traveling state of the subject vehicle to be controlled is not the inter-vehicle distance, the control unit 15 determines whether the traveling state of the subject vehicle to be controlled is the vehicle speed (step S22).

  When the control unit 15 determines that the traveling state of the subject vehicle to be controlled is not the vehicle speed, the control unit 15 proceeds to step S23. When it is determined that the vehicle speed is the vehicle speed, the control unit 15 proceeds to step S26.

  Subsequently, when the control unit 15 determines that the traveling state of the subject vehicle to be controlled is not the vehicle speed, the control unit 15 determines whether the traveling state of the subject vehicle to be controlled is a traveling position in a plurality of lanes (step S23). ).

  When the control unit 15 determines that the traveling state of the subject vehicle to be controlled is not the traveling position, the control unit 15 proceeds to step S24. When the control unit 15 determines that the travel position is set, the process proceeds to step S27.

  Subsequently, when it is determined that the traveling state of the subject vehicle to be controlled is not a traveling position, the control unit 15 determines whether the traveling state of the subject vehicle to be controlled is a planned passing route at an intersection ( Step S24).

  When the control unit 15 determines that the traveling state of the subject vehicle to be controlled is not the planned passing route, the control unit 15 returns to FIG. 13A and the flow ends. If the control unit 15 determines that the route is a planned passing route, the process proceeds to step S28.

  When determining that the traveling state of the subject vehicle to be controlled is the inter-vehicle distance, the control unit 15 controls the positional relationship between the subject vehicle and the preceding vehicle (step S25). Thereafter, the control unit 15 returns to step S21 again.

  When it is determined that the traveling state of the subject vehicle to be controlled is the vehicle speed, the control unit 15 controls the vehicle speed of the subject vehicle (step S26). Thereafter, the control unit 15 returns to step S21 again.

  When it is determined that the traveling state of the subject vehicle to be controlled is the traveling position, the control unit 15 controls the traveling position of the own vehicle in a plurality of lanes (step S27). Thereafter, the control unit 15 returns to step S21 again.

  When determining that the traveling state of the subject vehicle to be controlled is a planned passage route, the control unit 15 controls the passage route of the subject vehicle at the intersection (step S28). Thereafter, the control unit 15 returns to step S21 again.

  As described above, in the in-vehicle information processing system 10 according to the present embodiment, the operator's operator displayed on the display unit 11 virtually operates the information on the screen. Is possible. That is, the operator can access the screen in a state closer to the actual feeling. In addition, the operator can intuitively recognize the actual position of the operator and the positional relationship on the screen. Therefore, the in-vehicle information processing system 10 can reduce the time for the operator to watch the screen as compared with a conventional device that displays a pointer or the like.

  Since the in-vehicle information processing system 10 limits the time for performing image processing, it is possible to superimpose and display the operator's hand with a minimum delay. That is, the in-vehicle information processing system 10 can reduce a temporal movement shift between the actual operator and the operator superimposed and displayed on the screen. Thereby, the operator can operate the information displayed on the screen without a more uncomfortable feeling.

  Since the in-vehicle information processing system 10 performs image processing for extracting the operator's operating hand imaged by the imaging unit 13, the operator's operating hand can be faithfully superimposed on the screen. As a result, the operator can intuitively recognize the operating hand superimposed on the screen as his / her own hand.

  The in-vehicle information processing system 10 performs image processing for changing the display magnification of the operator's operator imaged by the imaging unit 13, so that the operator's operator's hand is superimposed and displayed in an optimum size for each screen. Is possible. As a result, the operator can easily see the display content displayed behind the operator while feeling the reality of the operator superimposed on the screen.

  The in-vehicle information processing system 10 can display an operator's operator in a superimposed manner with a minimum delay compared to the case where the position coordinates in the predetermined region R1 where the touch operation is detected are identified by image processing. Is possible. The in-vehicle information processing system 10 does not indirectly identify the position coordinates based on the image captured by the imaging unit 13, but directly identifies the position coordinates by the touch operation unit 12, and thus identifies the position coordinates with high accuracy. be able to. That is, since the in-vehicle information processing system 10 detects the position where the operator is actually touching by the touch operation unit 12, when the operator selects a function item displayed on the screen, a malfunction occurs. Less likely to cause.

  In the in-vehicle information processing system 10, the imaging unit 13 captures an image of the operator's entire operating hand, so that the operator can easily recognize that the operating hand superimposed on the screen is his / her hand. Furthermore, the operator can easily recognize which part of the operator is moving based on the display on the screen. Further, since the operator accurately recognizes the relationship between the actual position of the operator and the position on the screen, the operator can easily grasp the movement amount of the operator and the movable area on the screen.

  The in-vehicle information processing system 10 can accurately identify the operator's operator on the touch pad 121 by imaging the entire operator's operator with the imaging unit 13. That is, the in-vehicle information processing system 10 can accurately recognize the operator's operating hand as a human hand. Further, the in-vehicle information processing system 10 can identify each part of the operator's operator with higher accuracy. That is, the in-vehicle information processing system 10 can accurately identify which finger corresponds to the finger in the captured image. Further, the in-vehicle information processing system 10 can accurately identify the size of the entire operating hand, the proportion of the size of each part in the entire operating hand, and the like by imaging the entire operating hand. As a result, the in-vehicle information processing system 10 accurately matches the movement amount of the operator's operator's hand on the screen and the movable area on the screen with the actual movement of the operator's hand on the touch pad 121. It is possible to identify.

  The in-vehicle information processing system 10 sets a predetermined area R1 on the back side of the touch pad 121 and sets an operation area R2 on the upper part of the screen, so that the operator inevitably moves the entire touch operation unit 12. The operator's own hand is superimposed. At this time, the in-vehicle information processing system 10 can inevitably image the entire operator's operation hand by the imaging unit 13. In addition, by making the area on the touch pad 121 other than the predetermined area R1 non-responsive to the touch operation, the operator's consciousness is concentrated on the predetermined area R1. At this time, the vehicle-mounted information processing system 10 can capture the entire operator's operating hand more reliably.

  The in-vehicle information processing system 10 can clearly recognize which finger is in contact with the touch pad 121 as visual information by highlighting an item selected by contact by the operator. In addition, the operator can easily visually recognize at which position on the screen the operator displayed in a superimposed manner or which item is selected.

  The in-vehicle information processing system 10 gives a click feeling to the operator when the tact switch 122 is turned on, so that the operator can obtain tactile feedback by his / her own operation, and is more intuitive. Operation is possible. In other words, the tact switch 122 is used to confirm the selection, so that the operator can confirm the item selected by a natural action. In addition, the operator feels more reality with respect to the operator superimposed and displayed on the screen due to the reaction force acting on the finger. That is, the operator can easily feel the illusion that his / her actual operator is touching the screen directly.

  The in-vehicle information processing system 10 makes the operator's operator's hand to be superimposed and displayed semi-transparent, so that the operator can easily visually recognize the information displayed on the screen while ensuring the reality. That is, the operator can operate information displayed on the screen more intuitively.

  The in-vehicle information processing system 10 can more easily image the entire operator's operating hand by installing the imaging unit 13 above the touch operation unit 12.

  In the in-vehicle information processing system 10, the operator operates at least a part of his / her operator on the operation unit 14, and can easily select each screen. That is, the operator can select each screen based on visual information obtained while viewing each screen of the display unit 11 without changing his / her line of sight to his / her own operator. Therefore, the in-vehicle information processing system 10 makes the operator's screen selection operation easier and improves the convenience for the operator.

  The in-vehicle information processing system 10 makes the movement direction from the currently selected screen to the next selected screen correspond to the detected direction of operation, and thus enables an intuitive screen selection operation. That is, in order to select the next screen, the operator can operate at least a part of his / her operator with a normal feeling based on the position information of each screen identified visually.

  Since the in-vehicle information processing system 10 includes a screen having a plurality of display layers different in the depth direction, more information can be displayed on one screen. That is, the in-vehicle information processing system 10 can simultaneously display different types of information on one screen by dividing the display layer.

  The in-vehicle information processing system 10 makes the movement direction from the currently selected display layer to the next selected display layer correspond to the direction of the detected operation, so that an intuitive display layer selection operation can be performed. To do. That is, in order to select the next display layer, the operator can operate at least a part of his / her operator with a normal feeling based on the position information of each display layer visually identified.

  Since the in-vehicle information processing system 10 suppresses the display degree of the display layer that is not selected, the visibility of the selected display layer can be improved.

  Since the in-vehicle information processing system 10 suppresses the display level by reducing the luminance, the selected display layer can be clearly displayed. That is, the in-vehicle information processing system 10 can relatively increase the luminance of the selected display layer by reducing the luminance of the display layer that is not selected, thereby enabling clear display.

  Since the in-vehicle information processing system 10 suppresses the degree of display of the display layer that is not selected by graying out, the visibility of the selected display layer can be relatively improved.

  The in-vehicle information processing system 10 can improve the visibility of the operator with respect to the display layer structure when displaying a display layer that has not been selected by the above-described arbitrary method as compared with the case where no display layer is displayed. That is, the operator can easily visually recognize which number of display layers are provided at which position on the screen.

  Since the in-vehicle information processing system 10 includes the operation unit 14 that is disposed on the palm rest and includes a four-way switch, the operator can select each screen or each display layer with a simple operation. That is, the operator can select each screen simply by pressing the four-way switch in the corresponding direction based on the back, near, left, and right directions regarding each screen or each display layer identified visually. Further, the in-vehicle information processing system 10 associates the back, front, left, and right directions for each screen or each display layer with the back, front, left, and right switches of the four-way switch, respectively. The operator can perform the selection operation intuitively.

  In the in-vehicle information processing system 10, the superimposed operation hand virtually controls the traveling state of the vehicle on the display layer 1112 according to the actual movement of a part of the operation hand on the touch operation unit 12. The operator can intuitively control the actual traveling state of the vehicle.

  Since the in-vehicle information processing system 10 controls the traveling state of the vehicle based on the movement of one finger on the touch operation unit 12, the operator can easily and easily perform a complicated finger operation. You can intuitively control the running state of your vehicle.

  Since the in-vehicle information processing system 10 controls the traveling state of the own vehicle based on the movement of two or more fingers on the touch operation unit 12, the operator intuitively controls the traveling state of the own vehicle. it can. Further, by increasing the number of fingers operated on the touch operation unit 12, the in-vehicle information processing system 10 can increase variations of finger operation patterns associated with control.

  Since the in-vehicle information processing system 10 displays the traveling state of the own vehicle based on various visual information, the operator can easily or intuitively visually recognize the traveling state of the own vehicle. Therefore, the in-vehicle information processing system 10 reduces the time for the operator to look at the screen, so that the time for the operator to deflect the line of sight from the traveling direction can be reduced as compared with the conventional devices around the instrument panel. Is possible.

  In the in-vehicle information processing system 10, since the inter-vehicle distance is included as the traveling state of the own vehicle, the operator can intuitively control the positional relationship between the own vehicle and the front vehicle.

  In the in-vehicle information processing system 10, since the vehicle speed is included as the traveling state of the own vehicle, the operator can intuitively control the vehicle speed of the own vehicle.

  In the in-vehicle information processing system 10, since the traveling position of the own vehicle includes the traveling positions in a plurality of lanes, the operator can intuitively control the traveling positions of the own vehicle in the plurality of lanes. That is, the operator can intuitively change the lane based on a touch operation on the touch pad 121.

  In the in-vehicle information processing system 10, since the traveling state of the vehicle includes a scheduled passage route with respect to the intersection, the operator can intuitively control the passage route with respect to the intersection. That is, the operator can intuitively control the right turn or the left turn at the intersection based on the touch operation on the touch pad 121.

  It will be apparent to those skilled in the art that the present invention can be realized in other predetermined forms other than the above-described embodiments without departing from the spirit or essential characteristics thereof. Accordingly, the above description is illustrative and not restrictive. The scope of the invention is defined by the appended claims rather than by the foregoing description. Some of all changes that fall within the equivalent scope shall be included therein.

DESCRIPTION OF SYMBOLS 10 In-vehicle information processing system 11 Display unit 111, 112, 113, 114 Screen 1111, 1112 Display layer 12 Touch operation unit 121 Touch pad 122 Tact switch 13 Imaging unit 14 Operation unit 15 Control unit 16 Storage unit R1 Predetermined area R2 operation Region R3 Region

Claims (8)

A display unit having at least one screen for displaying a traveling state of the own vehicle;
A touch operation unit that detects contact by an operator's operator;
An imaging unit that images at least a part of the operator and the touch operation unit;
Based on the image picked up by the image pickup unit, at least a part of the operator's hand is superimposed and displayed on the screen, and based on the movement of a part of the operator's hand on the touch operation part, the own vehicle A control unit for controlling the running state of
Comprising
In-vehicle information processing system. The control unit controls a traveling state of the host vehicle based on a movement of one finger in the touch operation unit.
The in-vehicle information processing system according to claim 1. The control unit controls a running state of the host vehicle based on movements of two or more fingers in the touch operation unit.
The in-vehicle information processing system according to claim 1. The display unit displays the running state of the vehicle by visual information.
The in-vehicle information processing system according to any one of claims 1 to 3. The traveling state of the host vehicle is a positional relationship between the host vehicle and the preceding vehicle.
The in-vehicle information processing system according to any one of claims 1 to 4. The traveling state of the host vehicle is the vehicle speed of the host vehicle.
The in-vehicle information processing system according to any one of claims 1 to 4. The traveling state of the host vehicle is a traveling position of the host vehicle in a plurality of lanes.
The in-vehicle information processing system according to any one of claims 1 to 4. The traveling state of the vehicle is a planned passage route with respect to an intersection.
The in-vehicle information processing system according to any one of claims 1 to 4.

Images