JP6766791B2 - Status detector, status detection system and status detection program - Google Patents

Description

The disclosure according to this specification relates to a state detection technique for detecting a driver's state in a vehicle having an automatic driving function.

Conventionally, for example, Patent Document 1 discloses a vehicle control device capable of performing a driving operation on behalf of a driver. As a handover, the vehicle control device carries out a driving change from the automatic driving state to the manual driving by the driver. Further, the vehicle control device detects whether or not the driver can return to manual driving by, for example, a driver monitor camera provided on the upper surface of the steering column, a touch sensor provided in the steering hole, or the like.

JP-A-2017-30555

By the way, even when the leg posture is inappropriate for the driving operation, it becomes difficult for the driver to cope with the driving change due to the handover. However, in the vehicle control device of Patent Document 1, whether or not the driver can respond to the driving change is determined mainly based on the state of the upper body of the driver. Therefore, even if the posture of the legs is inappropriate for the driving operation, it is difficult for the vehicle control device to determine that the driver cannot respond to the change request.

An object of the present disclosure is to provide a state detection device or the like capable of determining that a driver cannot respond to a change request when a leg posture inappropriate for a driving operation is taken.

In order to achieve the above object, one aspect disclosed is a state detection device that detects a driver's state in a vehicle (A) having an automatic driving function capable of performing a driving operation on behalf of the driver. , The information acquisition unit (31) that acquires the detection information that detects the position of the driver's leg by a configuration different from the camera, and the change request from the automatic driving function based on the state of the driver's leg indicated by the detection information. A state determination unit (32) for determining whether or not the driver can respond to the driving change based on the driving is provided , and the state determination unit includes a seat surface (110) in which at least the tip of the driver's right leg is the driver's seat (110). if it is detected from below 111), the driver are possible is judged to that state detecting device corresponding to the operation replacement.
Further, one aspect disclosed is a state detection device for detecting a driver's state in a vehicle (A) having an automatic driving function capable of performing a driving operation on behalf of the driver, and is a state detection device for a driver's leg. Whether or not the driver can respond to a driving change based on a change request from the automatic driving function based on the information acquisition unit (31) that acquires the detection information that detects the position and the state of the driver's leg indicated by the detection information. The state determination unit (32) is provided, and the state determination unit is such that the driver changes driving as the margin time (Ta) from the start of the change request to the inability to continue the automatic operation becomes longer. It is a state detection device that relaxes the permissible criteria for determining that it can be handled.
Further, one aspect disclosed is a state detection device for detecting a driver's state in a vehicle (A) having an automatic driving function capable of performing a driving operation on behalf of the driver, and is a state detection device for a driver's leg. Whether or not the driver can respond to a driving change based on a change request from the automatic driving function based on the information acquisition unit (31) that acquires the detection information that detects the position and the state of the driver's leg indicated by the detection information. The state determination unit (32) is provided with a state determination unit (32), and the state determination unit is a state detection device that tightens the permissible criteria for determining that the driver can handle the driving change after the change request.

Further, one aspect disclosed is a state detection program for detecting a driver's state in a vehicle (A) having an automatic driving function capable of performing a driving operation on behalf of the driver, and at least one processing unit. (41) is an information acquisition unit (31) that acquires detection information that detects the position of the driver's leg by a configuration different from that of the camera, and the automatic driving function based on the state of the driver's leg indicated by the detection information. It functions as a state determination unit (32) that determines whether or not the driver can respond to a driving change based on a change request, and the state determination unit has at least the tip of the driver's right leg in the driver's seat (110). If it is detected from below the seat surface (111), it is a state detection program that determines that the driver can handle the change of driving.
Further, one aspect disclosed is a state detection program for detecting a driver's state in a vehicle (A) having an automatic driving function capable of performing a driving operation on behalf of the driver, and at least one processing unit. (41) is an information acquisition unit (31) that acquires detection information that detects the position of the driver's leg, and a driving change based on a change request from the automatic driving function based on the state of the driver's leg indicated by the detection information. The state determination unit (32) functions as a state determination unit (32) for determining whether or not the driver can respond, and the state determination unit has a margin time (Ta) from the start of the replacement request until the automatic operation cannot be continued. The longer the length, the more relaxed the permissible criteria for determining that the driver can handle the change of driving .
Further, one aspect disclosed is a state detection program for detecting a driver's state in a vehicle (A) having an automatic driving function capable of performing a driving operation on behalf of the driver, and at least one processing unit. (41) is an information acquisition unit (31) that acquires detection information that detects the position of the driver's leg, and a driving change based on a change request from the automatic driving function based on the state of the driver's leg indicated by the detection information. It functions as a state determination unit (32) that determines whether or not the driver can handle the change, and the state determination unit strictly sets the permissible criteria for determining that the driver can handle the change of driving after the change request. It is said to be a state detection program.

In this aspect, the position of the driver's leg is acquired as detection information. Then, whether or not the driver can respond to the driving change based on the change request is determined based on the state of the driver's leg. Based on the above, it is possible to determine that it is not possible to respond to the replacement request from the automatic driving function when the leg posture is inappropriate for the driving operation.

The reference numbers in parentheses are merely examples of the correspondence with the specific configuration in the embodiment described later, and do not limit the technical scope at all.

It is a block diagram which shows the whole picture of the structure related to the automatic driving mounted on a vehicle. It is a time chart which shows the process of handover in chronological order. It is a figure which shows the details of the arrangement of a leg sensor and the position detection of a leg seen from the right side of a vehicle. It is a figure which shows the details of the arrangement of a leg sensor and the position detection of a leg seen from the ceiling side of a vehicle. It is a figure which shows the feature of "driving posture", "appropriate posture" and "inappropriate posture" which are judged by a state determination part in a list. It is a figure which shows the concept of the actuation in the automatic operation mode which is carried out by the cooperation of HCU and the presentation device. It is a flowchart which shows the detail of the state detection processing performed in the automatic operation mode. It is a figure which shows the details of the arrangement of the leg sensor and the position detection of a leg of the 2nd Embodiment. It is a figure which shows the details of the arrangement of the leg sensor and the position detection of a leg of the 2nd Embodiment. It is a block diagram which shows the structure of the state detection system of 3rd Embodiment. It is a figure which shows the details of the arrangement of the leg sensor and the position detection of a leg of the 3rd Embodiment. It is a figure which shows the details of the arrangement of the leg sensor and the position detection of a leg of the 3rd Embodiment. It is a figure which shows the details of the arrangement of the leg sensor and the position detection of a leg of 4th Embodiment. It is a figure which shows the details of the arrangement of the leg sensor and the position detection of a leg of the 5th Embodiment. It is a figure which shows the details of the arrangement of the leg sensor and the position detection of a leg of the 5th Embodiment. It is a figure which shows the contents organized based on the state of the hip joint and the knee joint of the right leg in a list about the classification of "appropriate posture" and "inappropriate posture".

Hereinafter, a plurality of embodiments of the present disclosure will be described with reference to the drawings. By assigning the same reference numerals to the corresponding components in each embodiment, duplicate description may be omitted. When only a part of the configuration is described in each embodiment, the configuration of the other embodiment described above can be applied to the other parts of the configuration. Further, not only the combination of the configurations specified in the description of each embodiment, but also the configurations of a plurality of embodiments can be partially combined even if the combination is not specified. Further, it is assumed that the unspecified combination of the configurations described in the plurality of embodiments and modifications is also disclosed by the following description.

(First Embodiment)
The function of the state detection device according to the first embodiment of the present disclosure is realized by the HCU (HMI (Human Machine Interface) Control Unit) 30 shown in FIG. The HCU 30 is mounted on the vehicle A together with electronic control units such as a vehicle control ECU (Electronic Control Unit) 80 and an automatic driving ECU 50. The HCU 30, the vehicle control ECU 80, and the automatic driving ECU 50 are directly or indirectly electrically connected to each other and can communicate with each other. The vehicle A has an automatic driving function by operating the vehicle control ECU 80 and the automatic driving ECU 50.

The vehicle control ECU 80 is directly or indirectly electrically connected to the vehicle-mounted actuator group 91 mounted on the vehicle A. The vehicle control ECU 80 controls the behavior of the vehicle A by integrally managing the operation of the vehicle-mounted actuator group 91. The vehicle-mounted actuator group 91 includes, for example, a throttle actuator, an injector, a brake actuator of an electronically controlled throttle, and a motor generator for driving and regeneration.

The vehicle control ECU 80 is mainly composed of a computer having a processing unit, a RAM, a memory device, an input / output interface, and the like. The vehicle control ECU 80 constructs the actuator control unit 81 as a functional block related to vehicle control by executing the vehicle control program stored in the memory device by the processing unit. The actuator control unit 81 generates a control signal output to the vehicle-mounted actuator group 91 based on at least one of the operation information based on the driver's driving operation and the autonomous driving information acquired from the automatic driving ECU 50.

The automatic driving ECU 50 is directly or indirectly electrically connected to the GNSS receiver 71, the map database 72, the autonomous sensor group 73, and the like as a configuration for acquiring information necessary for autonomous driving.

The GNSS (Global Navigation Satellite System) receiver 71 can receive positioning signals transmitted from a plurality of artificial satellites. The GNSS receiver 71 sequentially outputs the received positioning signal to the automatic driving ECU 50 as information for specifying the current position of the vehicle A.

The map database 72 is a storage device that stores a large amount of map data. The map data includes structural information such as the curvature, slope, and section length of each road, as well as non-temporary traffic regulation information such as speed limit and one-way traffic. The map database 72 provides the automatic driving ECU 50 with map data around the current position of the vehicle A and the traveling direction based on the request from the automatic driving ECU 50.

The autonomous sensor group 73 detects moving objects such as pedestrians and other vehicles, as well as stationary objects such as falling objects on the road, traffic signals, guardrails, curbs, road signs, road markings, and lane markings. The autonomous sensor group 73 includes, for example, a camera unit, a rider, a millimeter wave radar, and the like. Each of the autonomous sensor group 73 sequentially outputs the detected object information related to the moving object and the stationary object to the automatic driving ECU 50.

The automatic operation ECU 50 is mainly composed of a processing unit 61, a RAM 62, a memory device 63, and a computer having an input / output interface. The processing unit 61 is configured to include at least one such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and an FPGA (Field-Programmable Gate Array). The processing unit 61 may be provided with a dedicated processor specialized in learning and inference of AI (Artificial Intelligence).

The automatic driving ECU 50 exerts an automatic driving function capable of performing a driving operation of the vehicle A on behalf of the driver by performing acceleration / deceleration control and steering control of the vehicle A in cooperation with the vehicle control ECU 80. The automatic operation ECU 50 switches a plurality of control modes in which the control right of the operation operation is located differently. The plurality of control modes include an automatic driving mode in which the vehicle A is autonomously driven by the automatic driving function, and a manual driving mode in which the vehicle A is driven by the driving operation of the driver (see FIG. 2). Further, the control mode includes a shift request mode executed at the time of transition from the automatic driving mode to the manual driving mode, and an MRM (Minimum Risk Maneuver) mode for automatically retracting the vehicle A.

The automatic operation ECU 50 can execute the automatic operation program stored in the memory device 63 by the processing unit 61. The automatic driving program includes a program for autonomously driving the vehicle A, a program for controlling a driving change, and the like. Based on the automatic driving program, the own vehicle position specifying unit 51, the environment recognition unit 52, the plan generation unit 53, the autonomous driving control unit 54, and the like are constructed in the automatic driving ECU 50.

The own vehicle position specifying unit 51 identifies the current position of the vehicle A based on the positioning signal received by the GNSS receiver 71. The own vehicle position identification unit 51 can identify the detailed current position of the vehicle A by collating the image of the front region acquired from the camera unit of the autonomous sensor group 73 with the detailed map data acquired from the map database 72. is there.

The environment recognition unit 52 combines the position information specified by the vehicle position identification unit 51, the map data acquired from the map database 72, the object information acquired from the autonomous sensor group 73, and the like to surround the vehicle A. Recognize the driving environment. The environment recognition unit 52 recognizes the shape and moving state of the object around the vehicle A based on the integrated result of each object information, particularly within the detection range of each autonomous sensor. The environment recognition unit 52 generates a virtual space that reproduces the actual driving environment in three dimensions by combining the recognized information of surrounding objects with the position information and the map data.

The plan generation unit 53 generates a travel plan for autonomously driving the vehicle A by the automatic driving function based on the travel environment recognized by the environment recognition unit 52. As the travel plan, a long- to medium-term travel plan and a short-term travel plan are generated. In the long- to medium-term driving plan, a route for directing the vehicle A to the destination set by the driver is defined. In the short-term travel plan, the planned travel route for realizing the travel according to the long- to medium-term travel plan is defined by using the virtual space around the vehicle A generated by the environment recognition unit 52. Specifically, execution of steering for lane tracking and lane change, acceleration / deceleration for speed adjustment, and sudden braking for collision avoidance are determined based on a short-term travel plan.

Here, when the automatic driving function is switched to the driver, the system side systematically hands over the control right to the driver, and the driver acquires the control right at his / her own discretion in a highly urgent situation. There is an override to do. The plan generation unit 53 predicts the occurrence of a situation in which autonomous driving cannot be continued in the automatic driving mode based on map data in the traveling direction, information acquired from the autonomous sensor group 73, and the like. The plan generation unit 53 formulates a handover schedule (hereinafter, “authority transfer plan”) when it becomes difficult to formulate a travel plan and it is impossible to continue automatic driving.

In the authority transfer plan, the off-timing of automatic driving (time t3 in FIG. 2) is specified. Then, the execution timing (time t1 in FIG. 2) of the change request (hereinafter, “TOR: Take Over Request”) for requesting the driver to change the driving from the automatic driving function is set by calculating back from the off timing. The execution timing of the TOR is set to a time earlier than the off timing of the automatic operation by a predetermined margin time Ta (for example, 4 seconds, see FIG. 2), and the HCU 30 is notified. As described above, the plan generation unit 53 controls the handover process in the shift request mode in cooperation with the HCU 30 based on the established authority transfer plan.

In the automatic driving mode, the autonomous driving control unit 54 generates autonomous driving information instructing acceleration / deceleration and steering based on the planned traveling route determined by the plan generation unit 53. The autonomous driving control unit 54 sequentially outputs the generated autonomous driving information to the vehicle control ECU 80. The autonomous driving control unit 54 cooperates with the actuator control unit 81 to autonomously drive the vehicle A along the planned traveling route.

The HCU 30 is an electronic control unit having a function of integrally controlling the presentation of information to the driver and a function of detecting the state of the driver. The HCU 30 is directly or indirectly electrically connected to the presentation device 20, the DSM (Driver Status Monitor) 11, the leg sensor 12, and the like.

The presenting device 20 presents various information related to the vehicle A to the occupants of the vehicle A including the driver based on the presentation control signal output by the HCU 30. The presenting device 20 includes, for example, a display device that presents information by display, a speaker 21 that presents information by a notification sound, a message voice, and the like, and a tactile stimulus device 22 that presents information by vibration. The tactile stimulator 22 is provided, for example, on the seat surface 111 (see FIG. 3) or the backrest of the driver's seat 110.

The DSM 11 is composed of a near-infrared light source, a near-infrared camera, a control unit for controlling them, and the like. The DSM 11 is arranged, for example, on the upper surface of the steering column in a posture in which the near-infrared camera is directed toward the driver's seat. The DSM 11 photographs the driver's head irradiated with near-infrared light by a near-infrared light source with a near-infrared camera, and monitors the driver's condition by analyzing the captured face image. The DSM 11 sequentially outputs the driver's detection information obtained by analyzing the face image toward the HCU 30.

The leg sensor 12 shown in FIGS. 1, 3 and 4 is, for example, an optical sensor, which is an active object detection sensor that detects an object by irradiation with light. The leg sensor 12 is installed on the side surface of the center console 117 on the driver's side. The leg sensor 12 is installed in the vehicle A at a height position lower than the seat surface 111 and closer to the seat surface 111 of the driver's seat 110 than the floor surface 113. The leg sensor 12 can individually detect both legs of the driver in the detection space 120. The detection space 120 is defined in advance below the steering wheel 115 and in front of the driver's seat 110. In addition, the shape and material of the structure such as the door that divides the detection space 120 in the vehicle interior are different for each vehicle type. Therefore, it is preferable that the detection space 120 is defined for each vehicle type in consideration of the shape and material of the surrounding structure.

The leg sensor 12 has a light emitting unit 13 and a light receiving unit 14. The light projecting unit 13 is a light source that emits near infrared rays. Under the control of the HCU 30, the light projecting unit 13 transmits pulsed near-infrared rays as detection waves toward the detection space 120. Near-infrared rays are spot-irradiated from the light projecting unit 13 along the lateral direction of the vehicle A, for example. The light projecting unit 13 repeats irradiation of near infrared rays at predetermined time intervals. The directivity (irradiation angle) of near-infrared light is set to a size that exceeds the thickness of the legs and does not erroneously detect each pedal or the like.

The light receiving unit 14 is a light receiving element that converts an optical signal into an electric signal. The light receiving unit 14 detects near-infrared light reflected by an object located in the detection space 120. When the driver's legs are in the detection space 120, the near-infrared light reflected by the left leg Ll near the center console 117 and the near-infrared light reflected by the right leg Lr far from the center console 117 are emitted. , The light receiving unit 14 receives light in order. The light receiving unit 14 converts the received light into an electric signal, and sequentially transmits it to the HCU 30 as a detection signal for detecting the position of the leg.

The HCU 30 is mainly composed of a processing unit 41, a RAM 42, a memory device 43, and a computer having an input / output interface. The processing unit 41 is configured to include at least one such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and an FPGA (Field-Programmable Gate Array). The processing unit 41 may be provided with a dedicated processor specialized in learning and inference of AI (Artificial Intelligence).

The HCU 30 constitutes a state detection system 10 together with the DSM 11 and the leg sensor 12. The state detection system 10 controls the presentation of information by the presenting device 20 in the automatic operation mode, and maintains the posture of the driver in an appropriate state. The HCU 30 executes a state detection program stored in the memory device 43 by the processing unit 41 to construct a plurality of functional blocks related to the driver's state detection and posture maintenance. Specifically, the HCU 30 is constructed with an information acquisition unit 31, a state determination unit 32, a notification control unit 33, and the like.

The information acquisition unit 31 acquires the driver's detection information detected by the DSM 11. In addition, the information acquisition unit 31 controls the operation of the leg sensor 12 and acquires detection information for detecting the positions of the left and right legs of the driver. More specifically, the information acquisition unit 31 measures the difference from the time when the near-infrared light is irradiated to the time when the reflected near-infrared light is received, based on the detection signal of the leg sensor 12. Then, the information acquisition unit 31 calculates the positions of the left and right legs from the time difference, and acquires them as detection information indicating the positions of both legs of the driver.

Further, the information acquisition unit 31 acquires status information indicating the control mode and the like of the automatic operation ECU 50. For example, when the authority transfer plan is formulated by the plan generation unit 53 based on the prediction of the occurrence of the system limit of the automatic operation, the information acquisition unit 31 acquires a command for instructing the implementation of the TOR from the HCU 30.

The state determination unit 32 determines whether or not the driver can respond to a driving change within the margin time Ta (see FIG. 2) based on the TOR based on the state of the driver's upper body and legs indicated by the detection information. .. If there is no problem with the posture of the upper body, the state determination unit 32 classifies the driver's posture into one of "driving posture", "appropriate posture" and "inappropriate posture" based on the state of the driver's legs. (See FIG. 5).

The "driving posture" is the posture of the driver in the state of manual driving. If the portion of the right leg Lr beyond the ankle (hereinafter, "right foot tip") is in contact with the accelerator pedal or the brake pedal, the state determination unit 32 determines that the vehicle is in the "driving posture". ..

The "appropriate posture" is a posture capable of returning to the driving posture within the margin time Ta (see FIG. 2). On the other hand, the "inappropriate posture" is a posture in which the driver cannot return to the driving posture within the margin time Ta. In other words, when the driver's posture is classified as "appropriate posture", the state determination unit 32 determines that the driver cannot respond to the driving change within the margin time Ta, and classifies the driver as "inappropriate posture". In this case, it is determined that the driver cannot respond to the driving change within the margin time Ta.

The state determination unit 32 determines the "appropriate posture" and the "inappropriate posture" mainly based on the estimated position of the driver's right foot. When the driver's right foot tip is detected below the seat surface 111 and the right foot tip is near the pedal, the state determination unit 32 determines that the driver is in the “appropriate posture”. Even if the part of the driver's left leg Ll beyond the ankle (hereinafter, "left foot tip") is not detected from below the seat surface 111, if the right foot tip is below the seat surface 111, the state The determination unit 32 determines that the posture is "appropriate" so that the driver can take turns. On the other hand, when the driver's left foot tip is detected from below the seat surface 111 but the right foot tip is not detected below the seat surface 111, the state determination unit 32 is in an "inappropriate posture". It is determined that. If the toes of both legs are not detected, the state determination unit 32 naturally determines that the posture is "inappropriate posture".

The notification control unit 33 generates a presentation control signal to be output to the presentation device 20 based on the information acquired by the information acquisition unit 31. The notification control unit 33 controls information presentation by display, sound, vibration, or the like by outputting a presentation control signal to the presentation device 20. When the information acquisition unit 31 acquires a command instructing the execution of the TOR, the notification control unit 33 gives the driver the takeover of the driving operation at the time t1 (see FIG. 2) set in the authority transfer plan. The requested information presentation is performed using the presentation device 20.

As described above, the driver who notices the TOR notification starts the driving operation while correcting the posture after confirming the surrounding situation. At this time, the state determination unit 32 determines that the state determination unit 32 is in the "driving posture" based on the detection information of the DSM 11 and the leg sensor 12. The automatic operation ECU 50 that has acquired the determination result of the state determination unit 32 determines that the operation can be performed at time t2 (see FIG. 2), and switches the control mode from the shift request mode to the manual operation mode.

On the other hand, when the spare time Ta (see FIG. 2) elapses without the driver taking the "driving posture" after the notification of TOR, the automatic driving ECU 50 changes the control mode at the time t3 (see FIG. 2). Switch from request mode to MRM mode. As a result, the vehicle A stops at the searched evacuation area by the evacuation run.

In order to prevent such a shift to the MRM mode, the notification control unit 33 returns the driver's state to the "appropriate posture" when the state determination unit 32 determines the "inappropriate posture" in the automatic operation mode. The actuation of is continuously carried out using the presentation device 20 (see FIG. 6). For example, when the driver is sitting in a sitting position or sitting upright, the notification control unit 33 guides the driver to a state of "appropriate posture" by presenting information using the speaker 21 and the tactile stimulating device 22. Further, when the driver's right toe is placed at a position where the driver's right foot is caught in the deployment of the airbag, the notification control unit 33 presents information using the speaker 21 and the tactile stimulator 22 to "inappropriate posture". "Warning that it is, and urging the return to the" appropriate posture ". In this way, the state of the driver before the TOR is maintained so that the driving change may occur at any time.

As described above, the details of the state detection process performed by the HCU 30 in order to maintain the proper posture of the driver will be described with reference to FIG. 1 based on FIG. The state detection process shown in FIG. 7 is started by the HCU 30 based on the switching from the manual operation mode to the automatic operation mode.

In S101, the detection information by the DSM 11 and the leg sensor 12 is acquired, and the process proceeds to S102. In S102, it is determined whether or not the driver's state is the "appropriate posture" based on the detection information acquired in S101 immediately before. If it is determined in S101 that the driver's state is the "appropriate posture", the process proceeds to S106. On the other hand, if it is determined in S102 that the driver's condition is "inappropriate posture", the process proceeds to S103.

In S103, based on the state of the driver detected in S102, guidance or warning prompting the driver to return to the "appropriate posture" is performed, and the process proceeds to S104 and S105. In S104 and S105, similarly to S101 and S102, it is determined whether or not the driver has returned to the "appropriate posture" based on the detection information. If it is determined in S105 that the posture has returned to the "appropriate posture", the posture returns to S101 and the state detection process is continued. On the other hand, when the driver's posture does not return to the "appropriate posture", the information presentation prompting the correction of the sitting posture is continued by repeating the processes of S103 to S105.

On the other hand, in S106 when it is determined in S102 that the posture is "appropriate posture", it is determined whether or not the TOR notification instruction based on the limit prediction of the system is acquired from the automatic operation ECU 50. When it is determined in S106 that the start of the TOR is instructed, the process proceeds to the process of executing the TOR. On the other hand, when it is determined that the automatic operation mode is continued, the process returns to S101 and the state detection process is continued.

The HCU 30 notifies the TOR even if the limit prediction of the system occurs while the posture is "inappropriate". In this case, the HCU 30 provides the automatic driving ECU 50 with a determination result that it is impossible to return to the "driving posture" after the notification of the TOR. Based on this determination result, the automatic operation ECU 50 starts the transition to the MRM mode before the lapse of the margin time Ta (see FIG. 2).

In the first embodiment described so far, the position of the driver's leg is acquired as detection information. Then, whether or not the driver can respond to the driving change based on the TOR is determined based on the state of the driver's legs. Based on the above, it is possible to determine that TOR cannot be supported when the leg posture is inappropriate for the driving operation.

In addition, in the first embodiment, when the tip of the right foot is detected from below the seat surface 111, the state determination unit 32 determines that the posture is “appropriate”. If at least the tip of the right foot is below the seat surface 111, the driver can start operating the accelerator pedal or the brake pedal within the margin time Ta after the occurrence of TOR. Therefore, if it is determined that the TOR can be supported depending on the position of the tip of the right foot, the calculation load of the posture determination can be reduced, and the driver who cannot support the TOR can avoid the change of driving.

Further, in the first embodiment, even if the left foot tip is not below the seat surface 111, if even the right foot tip is below the seat surface 111, it is determined to be in the “appropriate posture”. Generally, the pedal operation is performed by the tip of the right foot. Therefore, it is not necessary to immediately determine "inappropriate posture" based on the fact that there is no left foot around each pedal. According to the above, the situation in which the shift to the MRM mode occurs even though the driver can start the manual operation can be avoided.

On the contrary, even if the left foot tip is below the seat surface 111, if the right foot tip is not below the seat surface 111, it is determined as "inappropriate posture". As described above, when the tip of the right foot is not around the pedal, it is difficult to start the operation quickly corresponding to TOR. Therefore, when the tip of the right foot is not below the seat surface 111, it is desirable to determine that it is not possible to handle the driving change regardless of the position of the tip of the left foot.

Further, the leg sensor 12 of the first embodiment detects the driver's leg in the detection space 120 defined below the steering wheel 115 and in front of the driver's seat 110. In this way, even if the detection space 120 of the leg sensor 12 is limited, the state determination unit 32 can determine whether or not the state of the driver's leg can correspond to TOR. Therefore, it is possible to avoid complication of the configuration of the leg sensor 12 while ensuring the accuracy of the driver's state determination.

In addition, the above-mentioned detection space 120 is a very dark space because external light is blocked by the instrument panel, the steering column, and the like. Therefore, if an active optical sensor that emits near-infrared light is used as the leg sensor 12, the information acquisition unit 31 can acquire detection information without being affected by the brightness of the surroundings of the vehicle A.

Further, in the state detection system 10 of the first embodiment, the camera is not used for detecting the state of the legs. Therefore, by matching the posture pattern taken by the driver with the image data captured at that time, the development man-hours for creating a determination device for determining the posture of the driver becomes unnecessary. Further, by not using a camera for detecting the state of the legs, it is possible to realize a state detection system 10 with high commercial value in consideration of the privacy problem of the driver.

Further, the leg sensor 12 of the first embodiment is installed at a height position closer to the seat surface 111 than the floor surface 113. For example, the position of the legs is farther from the knees, and the closer to the floor surface 113, the more likely it is to vary from front to back and left and right along the floor surface 113. On the other hand, if the height is close to the seating surface 111, the position of the legs (knees) does not change significantly. Therefore, if the leg sensor 12 is arranged at a height close to the seat surface 111, detection omission is unlikely to occur even if the irradiation range (directivity) of the near infrared light (detection wave) is narrowed. Therefore, it is possible to simplify the leg sensor 12 while ensuring the detection accuracy of the leg position.

In the first embodiment, the leg sensor 12 corresponds to the "position detection sensor", the light projecting unit 13 corresponds to the "transmitting unit", the light receiving unit 14 corresponds to the "detecting unit", and the HCU 30 is "state". Corresponds to "detector".

(Second Embodiment)
The second embodiment shown in FIGS. 8 and 9 is a modification of the first embodiment. The leg sensor 212 of the second embodiment is an optical sensor as in the first embodiment, and is installed at a position different from that of the first embodiment. The leg sensor 212 is housed in front of the seat surface portion of the driver's seat 110 at a height position closer to the seat surface 111 than the floor surface 113. The light projecting unit 213 of the leg sensor 212 irradiates near-infrared light toward the detection space 120 defined in front. In addition, the light projecting unit 213 can scan the light irradiation direction at least in the horizontal direction. The light projecting unit 213 repeats the irradiation of near infrared rays at predetermined time intervals while rotating the light irradiation direction in the horizontal direction. The directivity (irradiation angle) of near-infrared light is narrowed down to the thickness of the legs. For example, when the near-infrared light is irradiated to the left, the near-infrared light reflected by the left leg Ll is detected by the light receiving unit 214. Similarly, when the near-infrared light is irradiated to the right, the near-infrared light reflected by the right leg Lr is detected by the light receiving unit 214.

The information acquisition unit 31 (see FIG. 1) individually detects the positions of both legs of the driver in the detection space 120 based on the detection signal received from the leg sensor 212. More specifically, the information acquisition unit 31 estimates the positions of both legs based on the irradiation direction of near-infrared light by the light projecting unit 213 and the difference time from the irradiation time to the light receiving time.

The reflected light detected after a lapse of a predetermined time ts from the irradiation time of the near infrared rays is filtered as the light reflected by the structure of the vehicle A outside the region of the detection space 120. That is, the information acquisition unit 31 detects the presence / absence and the position of each leg Ll, Lr based on the reflected light received from the irradiation time to the elapse of a predetermined time ts. However, not limited to such a method, the shape and material of the structure of the vehicle A represented by the door are defined for each vehicle type. Therefore, the light receiving time and the light receiving intensity of the reflected light become known values in advance. For this reason, the light receiving time and the light receiving intensity may be set in advance according to the structure around the detection space 120, and the legs and the structure may be distinguished from each other.

Even in the second embodiment described so far, the same effect as that of the first embodiment is obtained, and it can be determined that TOR cannot be supported when the leg posture is inappropriate for the driving operation. In addition, if the configuration is such that near-infrared light is emitted from the rear of the detection space 120, the information acquisition unit 31 is based on the detection signal even if both legs are aligned in the front-rear direction of the vehicle A. You can grasp the position of both legs. In the second embodiment, the leg sensor 212 corresponds to the "position detection sensor", the light projecting unit 213 corresponds to the "transmitting unit", and the light receiving unit 214 corresponds to the "detecting unit".

(Third Embodiment)
The third embodiment shown in FIGS. 10 to 12 is another modification of the first embodiment. The state detection system 310 of the third embodiment includes the same HCU30 and DSM11 as those of the first embodiment. In addition, the state detection system 310 has a leg sensor 12 that is substantially the same as the first embodiment and a leg sensor 212 that is substantially the same as the second embodiment as a configuration for detecting the driver's leg.

The state detection system 310 uses the two leg sensors 12, 212 to detect both legs of the driver in the detection space 120 from different directions. Both the two leg sensors 12 and 212 are provided at height positions closer to the seat surface 111 than the floor surface 113. The leg sensor 12 arranged on the center console 117 is installed at a position slightly higher than the leg sensor 212 arranged on the driver's seat 110 in the height direction of the vehicle A (see FIG. 11). The information acquisition unit 31 integrates the detection signals output from the two leg sensors 12 and 212 and estimates the positions of the legs Ll and Lr in the detection space 120.

Even in the third embodiment described so far, the same effect as that of the first embodiment is obtained, and it can be determined that TOR cannot be supported when the leg posture is inappropriate for the driving operation. In addition, in the third embodiment, by using the detection information of the two leg sensors 12 and 212, the HCU 30 more accurately acquires the state of both legs of the driver and appropriately determines whether or not the driving can be changed. It can be carried out. In the third embodiment, the leg sensors 12 and 212 both correspond to the "position detection sensor".

(Fourth Embodiment)
The fourth embodiment shown in FIG. 13 is still another modification of the first embodiment. In the fourth embodiment, two leg sensors 412t and 412b are provided on the center console 117. Each of the leg sensors 412t and 412b has substantially the same configuration as the leg sensor 12 (see FIG. 1) of the first embodiment. The leg sensor 412t is provided at a position higher than the leg sensor 412b in the height direction of the vehicle. The leg sensor 412t is arranged behind the leg sensor 412b in the front-rear direction of the vehicle.

With the above configuration, the information acquisition unit 31 (see FIG. 1) can detect, for example, a state in which the legs are assembled on the knee from the detection signals of the two leg sensors 412t and 412b. More specifically, when the legs are assembled on the knee, the upper leg sensor 412t outputs a detection signal for detecting both the right leg Lr and the left leg Ll. On the other hand, the lower leg sensor 412b outputs a detection signal that detects only the right leg Lr. The information acquisition unit 31 can estimate the state of the driver's legs from the combination of these two detection signals.

Even in the fourth embodiment described so far, the same effect as that of the first embodiment can be obtained, and it can be determined that the TOR cannot be supported when the leg posture is inappropriate for the driving operation. In addition, in the fourth embodiment, by using the detection information of the two upper and lower leg sensors 412t and 412b, the HCU 30 can accurately grasp the state of the legs that are difficult to return to the driving posture, and the driver cannot respond to TOR. It is possible to avoid the implementation of the operation change of. In the fourth embodiment, the leg sensors 412t and 412b both correspond to the "position detection sensor".

(Fifth Embodiment)
The fifth embodiment shown in FIGS. 14 and 15 is still another modification of the first embodiment. In the fifth embodiment, the floor sensor 15 is provided in place of the leg sensor 12 (see FIG. 1) of the first embodiment. Like the leg sensor 12, the floor sensor 15 is a sensor that detects both legs Ll and Lr of the driver from the detection space 120 below the steering wheel 115 and in front of the driver's seat 110. The floor sensor 15 is formed in a sheet shape and is provided on the back side of the floor surface 113. The floor sensor 15 is formed to have the same size as the bottom surface of the detection space 120 so as to indirectly face the entire bottom surface of the detection space 120. The floor sensor 15 has a plurality of pressure detection units arranged in a two-dimensional manner.

For example, one pressure detection unit is provided in each region (see the alternate long and short dash line in FIG. 15) in which the floor sensor 15 is divided in a grid pattern. Each pressure detection unit detects the load due to the sole of the foot being placed on the floor surface 113. The floor sensor 15 sequentially outputs the pressure acting on each region to the HCU 30 (see FIG. 1) as a detection signal.

The information acquisition unit 31 (see FIG. 1) detects the pressure (load) acting on each region of the floor sensor 15. Based on such detection information, the state determination unit 32 (see FIG. 1) can detect the footprints of both legs Ll and Lr from the detection space 120, such as when a pressure equal to or higher than the threshold value is applied to at least two regions. Is determined to be the "appropriate posture". On the other hand, when only one foot footprint can be detected, such as when pressure above the threshold is acting on only one area, or when pressure above the threshold is not acting on any area and even one foot footprint cannot be detected. The state determination unit 32 determines that the posture is “inappropriate”.

Even with the configuration of detecting the legs using the floor sensor 15 as in the fifth embodiment described so far, the same effect as that of the first embodiment is obtained, and the posture of the legs inappropriate for the driving operation is taken. If it is, it can be determined that TOR cannot be supported.

(Sixth Embodiment)
The sixth embodiment of the present disclosure is a modification of the first embodiment shown in FIGS. 1 and 2. In the automatic operation ECU 50 of the sixth embodiment, the margin time Ta set in the authority transfer plan is variable. For example, when the traveling environment of the vehicle A is good (sunny weather, etc.) and the detection distance of the autonomous sensor group 73 is sufficiently long, the automatic driving ECU 50 grasps the distant situation and is earlier than the off timing of the automatic driving. TOR can be performed at the timing. Specifically, it is possible to formulate an authority transfer plan that secures a margin time Ta of about 10 seconds. On the other hand, when the traveling environment of the vehicle A is not good (bad weather, dark place, etc.) and the detection distance of the autonomous sensor group 73 is short, it becomes difficult for the automatic driving ECU 50 to grasp the distant situation. Therefore, the margin time Ta set in the authority transfer plan is shorter than that in the case of a good driving environment, for example, about 4 seconds.

Depending on the state of the automatic driving ECU 50 and the autonomous sensor group 73 as described above, the state determination unit 32 sets an allowable standard (see FIG. 5) for classifying the “appropriate posture” and the “inappropriate posture” as a margin time Ta. Change according to the length of. That is, if the margin time Ta becomes longer, the permissible standard is relaxed. On the contrary, if the margin time Ta becomes shorter, the permissible standard becomes stricter.

Specifically, the posture of the driver such as crossed legs and seiza on the seat surface and legs on the knees is determined to be "appropriate posture" in a driving environment where a long margin time Ta (about 10 seconds) can be secured. In a driving environment where the margin time Ta is short (about 4 seconds), it is determined to be an "inappropriate posture". On the other hand, the posture in which the right foot tip is hung on the steering wheel 115 (see FIG. 3) and the door on the driver's seat side is determined to be an "inappropriate posture" regardless of the margin time Ta.

In addition, the state determination unit 32 tightens the permissible standard as the remaining time from TOR to the off timing of automatic operation becomes shorter when the margin time Ta can be secured for a long time. For example, at the timing when the remaining time until the off timing of the automatic driving is about 4 seconds, the state determination unit 32 sets the postures such as cross-legged and seiza on the seat surface and legs on the knees as "appropriate". It is determined that the posture is not "posture" but "inappropriate posture".

Even in the sixth embodiment described so far, the same effect as that of the first embodiment is obtained, and if the leg posture is inappropriate for the driving operation, it is determined that the TOR cannot be supported. .. In addition, as in the sixth embodiment, the permissible standard for separating the "appropriate posture" and the "inappropriate posture" may be changed according to the length of the margin time Ta that can be secured in the shift request mode. According to such control, the posture allowed for the driver in the automatic driving mode is increased, so that unnecessary guidance for posture correction is reduced. Based on the above, it is possible to provide an automatic driving function that is useful for users such as drivers.

Further, as in the sixth embodiment, if the criterion for "appropriate posture" after TOR is raised as the off-timing of automatic driving approaches, information is presented to encourage the driver to return to the correct driving posture by the off-timing of automatic driving. Can be implemented.

(Other embodiments)
Although the plurality of embodiments of the present disclosure have been described above, the present disclosure is not construed as being limited to the above embodiments, and is applied to various embodiments and combinations without departing from the gist of the present disclosure. can do.

In the first modification of the sixth embodiment, the margin time set in the authority transfer plan is longer than that of the first embodiment and the like, and is a fixed length (for example, 10 seconds). Therefore, in the first modification, the change of the allowable standard in the automatic operation mode is not implemented. For example, if the detection distance and accuracy of the autonomous sensor group are sufficient, such a setting is possible. Therefore, in the automatic driving mode, postures such as cross-legged, seiza, and legs on the knees are allowed.

On the other hand, as in the sixth embodiment, the state determination unit tightens the permissible standard as the remaining time from the TOR to the off timing of the automatic operation becomes shorter. Specifically, when the remaining time until the off timing of automatic driving becomes less than a predetermined threshold value (for example, 4 seconds), the state determination unit takes a posture such as cross-legged, seiza, and legs on the knee. , It comes to judge that it is "inappropriate posture" instead of "appropriate posture". As a result, a warning is given to the driver who takes the above posture at an appropriate timing.

As in the above modification 1, even if the margin time is not variable, the state determination unit may change the permissible standard at an appropriate timing such as after TOR. Further, the specific value of the margin time may be appropriately changed according to the regulations and guidelines of the area where the vehicle is used, the speed limit of the road on which the vehicle travels, and the like.

In the above embodiment, an optical sensor, a floor sensor, or the like has been adopted as a configuration for detecting the position of the leg. However, instead of the optical sensor, a line sensor, an ultrasonic sensor, or the like can be adopted. In the ultrasonic sensor, the vibrating unit that oscillates ultrasonic waves as a detection wave corresponds to the "transmitting unit", and the receiving unit that receives the ultrasonic waves reflected by the legs corresponds to the "detecting unit". Further, not limited to these sensors, various active and passive object detection sensors can be adopted as sensors for detecting the position of the legs. Further, the above-mentioned method for distinguishing a leg from a structure can also be applied to a form using an ultrasonic sensor or the like.

More specifically, it is desirable that the sensor provided in the state detection system can detect the posture in the range of the dots shown in FIG. In FIG. 16, the posture of the driver's right leg, which is directly connected to the pedal operation, is defined by the hip joint angle θh and the knee joint angle θb, and the driver's posture when the joint angles θh and θb are changed. Is listed. The range of the dots is the range of the posture that does not require the change of the body axis until the correct driving posture is returned.

For example, the legs below the knee are classified into the "appropriate posture" because it is not necessary to change the body axis to return to the driving posture. On the other hand, when the right leg is extended and hung on the right door or window or the steering wheel 115, it is necessary to change the body axis to return to the driving posture. Similarly, if the legs are crossed on the knees, it is necessary to change the body axis. Therefore, these postures are classified as "inappropriate postures".

In the above embodiment, the "appropriate posture" and the "inappropriate posture" are determined from the state of the right leg based on the detection of the leg sensor in order to perform the above determination with a simple configuration. However, for example, by combining the detection information of the in-vehicle camera provided above the front of the driver with the detection information of the leg sensor, the above-mentioned posture in which the right leg is thrown forward can be specified. Therefore, if it is a state detection system that detects the state of the driver by combining the detection results of the upper body and the legs, it is possible to carry out highly accurate alerting.

Further, the range of the detection space by the position detection sensor may be changed as appropriate. For example, the state of the driver's hip joint located on the seat surface may be detected. The position of the position detection sensor can also be changed according to the setting range of the detection space. For example, the position detection sensor may be installed on the side surface of the center cluster, the lower surface of the steering column, the vicinity of the pedal, and the like.

Further, the configuration for monitoring the state of the driver's upper body is not limited to DSM. The state detection system may include, for example, an electrostatic sensor or a pressure sensor for detecting the grip of the steering wheel 115 (see FIG. 3), an in-vehicle camera described above, or the like, together with or in place of the DSM. Further, the configuration such as DSM may be omitted from the state detection system.

The automatic driving ECU of the above-described embodiment enables so-called eye-off automatic driving, which is responsible for monitoring the driving environment on behalf of the driver. The driver's state detection method according to the present disclosure is suitable for detecting the posture of the driver during the period during which such level 3 automatic driving is performed. In addition, the state detection method according to the present disclosure is also applicable to Level 2 autonomous driving in which the driving environment is monitored by the driver.

Then, the function of the state detection device may be realized by a hardware configuration and a software configuration different from the above-mentioned HCU. For example, the processing unit of the electronic control unit that integrally configures the automatic operation ECU and the HCU may carry out the above-mentioned state detection method. Alternatively, a plurality of electronic control devices including the HCU, the automatic driving ECU, and the vehicle control ECU may perform distributed processing of the state detection program according to the present disclosure.

Further, various non-transitory tangible storage media such as a flash memory and a hard disk can be adopted in a memory device such as an HCU as a configuration for storing a state detection program. In addition, the storage medium for storing the state detection program is not limited to the storage medium provided in the electronic control unit mounted on the vehicle, and may be an optical disk as a copy source to the storage medium, a hard disk drive of a general-purpose computer, or the like. You may.

A vehicle, Ta margin time, 10 state detection system, 12,212,412t, 412b leg sensor (position detection sensor), 13 floodlight (transmitter), 14 light receiver (detection), 30 HCU (state detection device) ), 31 Information acquisition unit, 32 Status determination unit, 41 Processing unit, 110 Driver's seat, 111 Seat surface, 113 Floor surface, 115 Steering wheel, 120 Detection space

Claims (13)

A state detection device for detecting the state of the driver in a vehicle (A) having an automatic driving function capable of performing a driving operation on behalf of the driver.
An information acquisition unit (31) that acquires detection information that detects the position of the driver's leg with a configuration different from that of the camera .
A state determination unit (32) for determining whether or not the driver can respond to a driving change based on a change request from the automatic driving function based on the state of the driver's leg indicated by the detection information is provided. ,
The state determination unit allows the driver to take turns when at least the toes of the driver's right leg are detected below the seat surface (111) of the driver's seat (110). state detecting device you determined that there. The first aspect of claim 1 , wherein the state determination unit determines that the driver can respond to a driving change even when the toes of the left leg of the driver are not detected from below the seat surface. State detector. In the state determination unit, even if the toes of the left leg of the driver are detected from below the seat surface, the toes of the right leg of the driver are not detected from below the seat surface. In this case, the state detection device according to claim 1 or 2 , wherein the driver determines that the driver cannot handle the change of driving. The state determination unit relaxes the permissible criteria for determining that the driver can handle the driving change as the margin time (Ta) from the start of the change request to the inability to continue the automatic driving becomes longer. The state detection device according to any one of claims 1 to 3 . A state detection device for detecting the state of the driver in a vehicle (A) having an automatic driving function capable of performing a driving operation on behalf of the driver.
An information acquisition unit (31) that acquires detection information that detects the position of the driver's leg, and
A state determination unit (32) for determining whether or not the driver can respond to a driving change based on a change request from the automatic driving function based on the state of the driver's leg indicated by the detection information is provided. ,
The state determination unit relaxes the permissible standard for determining that the driver can handle the driving change as the margin time (Ta) from the start of the change request to the inability to continue the automatic operation becomes longer. to that state detecting device. The state detection device according to any one of claims 1 to 5, wherein the state determination unit tightens an allowable standard for determining that the driver is capable of responding to a driving change after the change request. A state detection device for detecting the state of the driver in a vehicle (A) having an automatic driving function capable of performing a driving operation on behalf of the driver.
An information acquisition unit (31) that acquires detection information that detects the position of the driver's leg, and
A state determination unit (32) for determining whether or not the driver can respond to a driving change based on a change request from the automatic driving function based on the state of the driver's leg indicated by the detection information is provided. ,
The state determination unit, after the change request, the driver strictly be that state detecting device for determining acceptance criteria as possible to the operational replacement. The information acquisition unit acquires the detection information that detects the position of the leg in the detection space (120) that is below the steering wheel (115) and is defined in front of the driver's seat (110). The state detection device according to any one of claims 1 to 7 . The state detection device (30) according to claim 8 and
A state detection system including a position detection sensor (12,212,412t, 412b) that outputs a detection signal for detecting the position of the leg to the information acquisition unit.
The position detection sensor has a transmitting unit (13) that emits a detection wave toward the detection space, and a detection unit (14) that detects the detection wave reflected by an object located in the detection space. Detection system. The state detection system according to claim 9 , wherein the position detection sensor is installed at a height position closer to the seat surface (111) of the driver's seat (110) than the floor surface (113) in the vehicle. A state detection program for detecting the state of the driver in a vehicle (A) having an automatic driving function capable of performing a driving operation on behalf of the driver.
At least one processing unit (41)
An information acquisition unit (31) that acquires detection information that detects the position of the driver's leg by a configuration different from that of the camera .
Based on the state of the driver's leg indicated by the detection information, it functions as a state determination unit (32) for determining whether or not the driver can respond to a driving change based on a change request from the automatic driving function. ,
The state determination unit allows the driver to take turns when at least the toes of the driver's right leg are detected below the seat surface (111) of the driver's seat (110). state detection program that determined that there is. A state detection program for detecting the state of the driver in a vehicle (A) having an automatic driving function capable of performing a driving operation on behalf of the driver.
At least one processing unit (41)
An information acquisition unit (31) that acquires detection information that detects the position of the driver's leg,
Based on the state of the driver's leg indicated by the detection information, the driver is made to function as a state determination unit (32) for determining whether or not the driver can respond to a driving change based on a change request from the automatic driving function. ,
The state determination unit relaxes the permissible standard for determining that the driver can handle the driving change as the margin time (Ta) from the start of the change request to the inability to continue the automatic operation becomes longer. state detection program. A state detection program for detecting the state of the driver in a vehicle (A) having an automatic driving function capable of performing a driving operation on behalf of the driver.
At least one processing unit (41)
An information acquisition unit (31) that acquires detection information that detects the position of the driver's leg,
Based on the state of the driver's leg indicated by the detection information, the driver is made to function as a state determination unit (32) for determining whether or not the driver can respond to a driving change based on a change request from the automatic driving function. ,
The state determination unit, after the change request, the state detection program the driver you strictly determining acceptance criteria as possible to the operational replacement.

Images