WO2024224780A1 - Monitoring device for vehicle and monitoring method for vehicle - Google Patents

Abstract

An automatic driving ECU (50) has a function of a monitoring device for a vehicle, the monitoring device monitoring the status of a host vehicle (Am) that is able to travel by means of an automatic driving function. The automatic driving ECU (50) determines the approach of a suspicious person (UM), who is not expected to board the host vehicle (Am), toward the host vehicle (Am). When the approach of the suspicious person (UM) toward the host vehicle is determined, the automatic driving ECU (50) uses a vehicle exterior display (24) provided in the host vehicle (Am) to issue a vehicle exterior notification indicating that boarding is prohibited.

Description

Vehicle monitoring device and vehicle monitoring method CROSS-REFERENCE TO RELATED APPLICATIONS

This application is based on Patent Application No. 2023-072475 filed in Japan on April 26, 2023, and Patent Application No. 2023-166447 filed in Japan on September 27, 2023, and the contents of the original applications are incorporated by reference in their entirety.

This specification discloses a technology for vehicle monitoring that monitors the status of the vehicle.

Patent Document 1 discloses an in-vehicle monitoring system used in a public bus, which is an autonomous vehicle equipped with an autonomous driving function. The in-vehicle monitoring system recognizes people and luggage inside the vehicle, and issues a warning to the passengers in the vehicle if the vehicle is crowded and a seat is occupied by luggage or if a single passenger takes up multiple seats.

JP 2022-154979 A

In a vehicle that runs autonomously without a specific driver on board, such as the shared vehicle in Patent Document 1, there is a possibility that a person who is not scheduled to board (hereinafter, an alert person) may try to forcibly enter the vehicle. In order to protect passengers and the vehicle from such alert people, technology that can deal with alert people may be required.

The purpose of this disclosure is to provide a vehicle monitoring device and a vehicle monitoring method that can respond to a warning person approaching the vehicle even when the driver is not on board.

In order to achieve the above object, one disclosed embodiment is a vehicle monitoring device that monitors the status of a vehicle that can be driven by an autonomous driving function, and includes an approach determination unit that determines the approach of a person who is not scheduled to board the vehicle, and an alarm implementation unit that, when it is determined that a person who is not scheduled to board the vehicle is approaching the vehicle, issues an external alarm using an external alarm device provided in the vehicle to issue an external alarm indicating that boarding is not possible.

Another disclosed aspect is a vehicle monitoring method for monitoring the status of a vehicle that can be driven by an autonomous driving function, the method including, in processing performed by at least one processing unit, a step of determining the approach of a person who is not scheduled to board the vehicle, and, if the approach of a person who is not scheduled to board the vehicle is determined, issuing an external alarm indicating that boarding is not possible using an external alarm device provided in the vehicle.

In these aspects, an external alarm installed in the vehicle is used to notify a person approaching the vehicle that they are not allowed to board the vehicle. This external alarm can prevent a person who is not scheduled to board the vehicle from getting into the vehicle. Therefore, it becomes possible to respond to a person who is approaching the vehicle even when the driver is not in the vehicle.

Note that the reference numbers in parentheses in the claims merely indicate an example of the correspondence with the specific configurations in the embodiments described below, and do not limit the technical scope in any way. In addition, claims that are not explicitly stated in the claims may be combined together if no particular problems arise with the combination.

1 is a diagram showing an overall view of an autonomous driving system including an autonomous driving ECU according to a first embodiment of the present disclosure. FIG. FIG. 2 is a block diagram showing details of a remote system. FIG. 2 is a block diagram showing details of a vehicle-side system. 13 is a diagram showing a time series of responses in response to a determination of an approach of a cautionary person. FIG. 10 is a flowchart showing details of a cautionary person response process executed by the autonomous driving ECU; 10 is a flowchart showing details of an emergency evacuation process performed by an autonomous driving ECU. 11 is a flowchart showing details of a behavior warning process for dealing with problematic behavior by a passenger. 10 is a flowchart showing details of a theft notification process for responding to theft of a driverless vehicle. FIG. 11 is a block diagram showing details of a vehicle-side system according to a second embodiment. 10 is a flowchart showing details of a video distribution process performed by the autonomous driving ECU. 10 is a flowchart showing details of a cautionary person response process executed by the autonomous driving ECU; 5 is a flowchart showing details of a theft notification process performed by the autonomous driving ECU. 10 is a flowchart showing details of a takeover response process performed by an autonomous driving ECU. 13A to 13C are diagrams showing, in chronological order, responses in response to the determination of the approach of a person to be warned in the third embodiment. 10 is a flowchart showing details of a cautionary person response process executed by the autonomous driving ECU;

Below, several embodiments are described based on the drawings. Note that in each embodiment, corresponding components are given the same reference numerals, and duplicated descriptions may be omitted. When only a portion of the configuration is described in each embodiment, the configuration of the other embodiment described above can be applied to the other portions of the configuration. In addition to the combinations of configurations explicitly stated in the description of each embodiment, configurations of several embodiments can be partially combined together even if not explicitly stated, as long as there is no particular problem with the combination.

First Embodiment
The autonomous driving ECU (Electronic Control Unit) 50 shown in FIG. 1 is mounted on a vehicle (hereinafter, the host vehicle Am). By mounting the autonomous driving ECU 50, the host vehicle Am becomes an autonomous driving vehicle or an autonomous vehicle equipped with an autonomous driving function. The host vehicle Am can run as a driverless vehicle ADV without a driver in the driver's seat DS by using the level 4 autonomous driving function. The autonomous driving level in this disclosure is based on the standards defined by the Society of Automotive Engineers.

The autonomous driving ECU 50 functions as a vehicle monitoring device according to the first embodiment of the present disclosure. In a driverless vehicle ADV where no driver is on board, the autonomous driving ECU 50 monitors the conditions inside and outside the vehicle Am on behalf of the driver. Specifically, the autonomous driving ECU 50 performs response processing on behalf of the driver in situations where a person other than the person (user) scheduled to board the vehicle Am attempts to get on the vehicle Am.

[Specific automated operation of driverless vehicles]
The driverless vehicle ADV is used as a vehicle that provides mobility services. The operation of multiple driverless vehicles ADV including the vehicle Am is remotely monitored by a remote monitoring facility CNT. The remote monitoring facility CNT is operated by a business operator (hereinafter, a specified automated driving implementer) that provides a level 4 automated driving mobility service (hereinafter, a specified automated driving implementer). A remote monitor RO and an operation manager OM are stationed at the remote monitoring facility CNT. The remote monitor RO and the operation manager OM are employees (specified automated driving business workers) belonging to the specified automated driving implementer.

The remote monitor RO is the manager (specific automatic operation manager) who manages specific automatic operation. The remote monitoring facility CNT is staffed with multiple remote monitors RO, such as a main monitor ROm and sub-monitors ROs. The remote monitor RO monitors the operating status of the automatic operation system 100, which will be described later, and when it determines that the automatic operation system 100 is not operating normally, it will terminate the operation of the driverless vehicle ADV.

The operation manager OM manages the remote monitors RO. The operation manager OM can take over at least part of the duties of the remote monitor RO described above in cases where an abnormality occurs in the remote monitor RO or in cases where an abnormality occurs in multiple driverless vehicles ADV. Note that if the remote monitoring facility CNT is small or if multiple remote monitors RO are stationed there, the deployment of the operation manager OM may be omitted. In other words, the remote monitor RO may also function as the operation manager OM. Furthermore, there may be only one remote monitor RO deployed at the remote monitoring facility CNT.

The automated driving system 100 that enables the specific automated driving described above is composed of a remote system 130, an infrastructure system 160, and a vehicle system IVS. The details of these components will be explained in order below.

[Remote System]
The remote side system 130 shown in FIG. 1 and FIG. 2 exists remotely from the multiple driverless vehicles ADV. The remote side system 130 realizes operation management by a remote monitor RO of multiple driverless vehicles ADV including the vehicle Am. The remote side system 130 enables the remote monitor RO to remotely check the situation when an abnormality occurs in the driverless vehicle ADV. When it is difficult to continue the automatic driving of the driverless vehicle ADV, the remote side system 130 transfers the dynamic driving task from the automatic driving ECU 50 to the remote monitor RO. This allows the remote monitor RO to remotely operate the driverless vehicle ADV. The remote side system 130 is composed of a center communication device 131, multiple remote user interfaces 132, a remote control device 140, and the like.

The center communication device 131 can communicate with the infrastructure side system 160, the vehicle side system IVS, and the user terminal 170 (e.g., a smartphone, etc.) of a user who uses the mobility service via a public communication line network. The center communication device 131 receives infrastructure information collected by the infrastructure side system 160. The center communication device 131 receives vehicle information collected by the vehicle side system IVS, and transmits input information, remote driving information, monitor camera footage, etc. acquired by the remote side system 130 to the vehicle side system IVS. The center communication device 131 receives usage application information from the user terminal 170 based on input by a user who plans to use the mobility service.

A remote user interface 132 is provided for each remote user, such as a remote monitor RO and an operation manager OM, who are located at the remote monitoring facility CNT. The remote user interface 132 includes an input device 133, a remote user camera 134, a remote control unit 135, a remote side display 136, and a remote side speaker 137.

The input device 133 is a keyboard, a mouse, etc. Terminal operations by the remote person are input to the input device 133. The remote person camera 134 is installed so as to capture the face of the remote person and its surroundings. The remote person camera 134 generates a monitor camera image including the face of the remote person. The remote person camera 134 is provided with a microphone that collects the voice of the remote person. The monitor camera image is accompanied by the sound collected by the microphone.

The remote control unit 135 includes an accelerator pedal, a brake pedal, and a steering wheel. Driving operations for remotely controlling a specific driverless vehicle ADV are input to the remote control unit 135 by a remote operator. The remote side display 136 and the remote side speaker 137 present to the remote operator information related to remote monitoring of the driverless vehicle ADV, such as infrastructure information and vehicle information received by the center communication device 131.

The remote control device 140 is a computer that mainly includes a processing unit 151, RAM 152, storage unit 153, input/output interface 154, and a control circuit equipped with a bus connecting these. The processing unit 151 executes various processes (instructions) for implementing the remote management method of the present disclosure by accessing the RAM 152. The storage unit 153 stores various programs (remote management programs, etc.) executed by the processing unit 151. By executing the programs by the processing unit 151, an information acquisition unit 141, an operation acquisition unit 142, a presentation control unit 143, and an operation management unit 144, etc. are constructed in the remote control device 140 (see FIG. 2).

The information acquisition unit 141 acquires the input information of the terminal operation input to the input device 133, the monitor camera image generated by the remote person camera 134, etc. The input information and the monitor camera image acquired by the information acquisition unit 141 are transmitted to the vehicle side system IVS of the driverless vehicle ADV via the operation management unit 144 and the center communication device 131 in the event of an abnormality occurring in the driverless vehicle ADV, etc.

The operation acquisition unit 142 acquires remote driving information indicating the driving operation input to the remote operation unit 135 when the driverless vehicle ADV is no longer able to continue autonomous driving. The remote operation information includes, as an example, a plurality of driving instruction values indicating the accelerator operation amount, the brake operation amount, and the steering operation amount. The remote driving information acquired by the operation acquisition unit 142 is transmitted to the vehicle-side system IVS of the driverless vehicle ADV that is no longer able to continue autonomous driving via the operation management unit 144 and the center communication device 131.

The presentation control unit 143 presents the infrastructure information and vehicle information acquired by the center communication device 131 and the operation management unit 144 to the remote person using the remote side display 136 and remote side speaker 137. As an example, video and audio from within the cabin of the driverless vehicle ADV, as well as video and audio of the driving route of the driverless vehicle ADV, are presented to the remote person by the remote side display 136 and remote side speaker 137. The remote person understands the situation of the driverless vehicle ADV based on the information provided by the remote side display 136 and remote side speaker 137.

The operation management unit 144 cooperates with the autonomous driving ECU 50 via the center communication device 131 to manage the operation of the driverless vehicle ADV. The operation management unit 144 acquires usage application information from the user terminal 170 via the center communication device 131. The operation management unit 144 sets a destination and intermediate destinations for the driverless vehicle ADV under its management based on the usage application information, and transmits the set location information to the autonomous driving ECU 50. The operation management unit 144 may also set a driving route for the driverless vehicle ADV, and transmit the set driving route to the autonomous driving ECU 50.

The operation management unit 144 shares the usage application information acquired from the user terminal 170 with the autonomous driving ECU 50 of the driverless vehicle ADV that each user plans to use. Specifically, based on the usage application information, the operation management unit 144 transmits identification information of the user (passenger PS) who plans to board the driverless vehicle ADV at each route point to the autonomous driving ECU 50 along with location information, etc. In addition, when new usage application information is acquired, the operation management unit 144 sequentially transmits the identification information of the user to the autonomous driving ECU 50 of the driverless vehicle ADV that is suitable for use by this user (for example, the closest).

If the autonomous driving of the driverless vehicle ADV cannot be continued, the operation management unit 144 cooperates with the autonomous driving ECU 50 to switch from autonomous driving by the autonomous driving ECU 50 to remote driving by the remote monitor RO. After switching to remote driving, the operation management unit 144 cooperates with the presentation control unit 143 to continue presenting the latest status inside and outside the driverless vehicle ADV to the remote monitor, and sequentially transmits remote driving information acquired by the operation acquisition unit 142 to the vehicle-side system IVS.

[Infrastructure system]
1 includes a surveillance camera 161. The infrastructure-side system 160 is installed on the driving route of the driverless vehicle ADV. A large number of infrastructure-side systems 160 may be installed so as to capture the entire driving route of the driverless vehicle ADV without omission, or may be installed only at specific points such as stops and intersections on the driving route.

The infrastructure-side system 160 can communicate with the remote-side system 130 via a public communication line network. The infrastructure-side system 160 transmits surveillance video of the driving route captured by the surveillance camera 161 to the remote-side system 130 as infrastructure information. The infrastructure-side system 160 may be equipped with an infrastructure sensor different from the surveillance camera 161. In this embodiment, the detection information of the infrastructure sensor is transmitted from the infrastructure-side system 160 to the remote-side system 130 as infrastructure information.

[Vehicle side system]
The host vehicle Am equipped with the vehicle-side system IVS shown in Fig. 1 and Fig. 3 is operated under the management of the remote-side system 130 with a passenger PS inside the vehicle. The host vehicle Am can run autonomously under the control of the automatic driving ECU 50 even when the driver is not seated in the driver's seat DS. The automatic driving performed by the automatic driving ECU 50 may be the above-mentioned level 4 automatic driving, or may be level 3 automatic driving that is premised on constant remote monitoring of the surroundings by a remote monitor RO.

The passenger PS is a user (passenger) of the mobility service using the driverless vehicle ADV. The passenger PS uses the user terminal 170 and an application installed on the user terminal 170 to use the mobility service. The user terminal 170 transmits to the remote system 130 application information including the planned boarding location (or current location), the planned boarding time, the planned disembarking location, and identification information for identifying the passenger PS, as information necessary for using the mobility service. The identification information includes information such as the name, age, sex, and whether or not the passenger PS has a disability. The user terminal 170 is also used for personal authentication of the passenger PS when boarding (checking in) and disembarking (checking out) the driverless vehicle ADV. The fee for using the mobility service may be paid through the application of the user terminal 170.

The vehicle Am is provided with an in-vehicle operation unit 110 for manual driving by the passenger PS (hereinafter, in-vehicle driving). For example, the in-vehicle operation units 110, such as an accelerator pedal, brake pedal, and steering wheel, are provided in front of the driver's seat DS as the in-vehicle operation unit 110. In other words, in a driverless vehicle ADV, the seat closest to the in-vehicle operation unit 110 becomes the driver's seat DS. For example, if it becomes difficult to continue autonomous driving and in-vehicle driving is permitted by the autonomous driving ECU 50 or remote control device 140, the passenger PS seated in the driver's seat DS becomes the driver, and driving operations input to the in-vehicle operation unit 110 are accepted.

The vehicle-side system IVS is mainly composed of an autonomous driving ECU 50. The autonomous driving ECU 50 is communicatively connected to a communication bus of an in-vehicle network mounted in the vehicle Am. To the communication bus, an in-vehicle monitor device 29, a user reception terminal 120, a surrounding monitoring sensor 30, a locator 35, a navigation ECU 38, an in-vehicle communication device 39, a driving control ECU 40, a body ECU 43, an HMI control device 10, etc. are connected. Among these devices and ECUs, certain nodes may be directly electrically connected to each other and be able to communicate without going through the communication bus.

The in-vehicle monitor device 29 includes multiple in-vehicle cameras that capture the interior of the vehicle Am, and a control unit that controls the multiple in-vehicle cameras. The in-vehicle cameras may be visible light cameras, or may be near-infrared cameras combined with near-infrared light sources. At least one of the multiple in-vehicle cameras is installed so that it can capture an image of a passenger PS (driver) sitting in the driver's seat DS, and functions as a driver monitor. Another of the multiple in-vehicle cameras is installed near the door and is set so that it can capture an image of a user who is about to get into the vehicle Am. The in-vehicle monitor device 29 provides the automatic driving ECU 50, etc. with images captured by the in-vehicle cameras, or the results of image analysis of the captured images, as in-vehicle monitor information. The in-vehicle monitor information includes information indicating the number of passengers PS, the position of each passenger PS in the vehicle, and the state of each passenger PS, such as whether they are standing or sitting.

The user reception terminal 120 is provided near the door of the vehicle Am so that the passenger PS can hold the user terminal 170 over it when getting on and off. The user reception terminal 120 may be installed inside the vehicle cabin, on the exterior surface of the vehicle Am (for example, near the button for opening the door), or both inside and outside the vehicle. The user reception terminal 120 obtains identification information (hereinafter, boarding application) of users who plan to board the vehicle Am, in other words, users who have made reservations and whose usage application information has been obtained by the remote system 130, from the operation management unit 144 via the in-vehicle communication device 39.

The user reception terminal 120 receives identification information of a user who is about to board the vehicle Am from the user terminal 170 using near field communication (NFC) or a one-dimensional or two-dimensional code. The user reception terminal 120 compares the identification information of the user who is about to board with a boarding application previously obtained, and detects the boarding of a user who is not scheduled to board, in other words, a user who has not made a correct boarding reservation.

The perimeter monitoring sensor 30 is an autonomous sensor that recognizes the surrounding environment of the host vehicle Am. The perimeter monitoring sensor 30 includes, for example, one or more of a camera unit, a millimeter wave radar, a lidar, and a sonar. The detection range of the perimeter monitoring sensor 30 is set to include the entire perimeter of the host vehicle Am. The perimeter monitoring sensor 30 is capable of detecting moving objects and stationary objects from the detection range around the host vehicle. The perimeter monitoring sensor 30 provides detection information of objects around the host vehicle to the autonomous driving ECU 50, etc.

The locator 35 includes a GNSS (Global Navigation Satellite System) receiver and an inertial sensor. The locator 35 combines positioning signals received from multiple positioning satellites by the GNSS receiver, the measurement results of the inertial sensor, and vehicle speed information output to the communication bus, and sequentially locates the vehicle's position and traveling direction, etc., of the vehicle Am. The locator 35 sequentially outputs position information and direction information of the vehicle Am based on the positioning results to the communication bus as locator information.

The locator 35 further has a map database (hereinafter referred to as the map DB) that stores map data. The map DB is mainly composed of a large-capacity storage medium that stores a large amount of three-dimensional map data and two-dimensional map data. The three-dimensional map data is a so-called HD (High Definition) map, and includes road information necessary for autonomous driving. The locator 35 reads map data for the area around the current position from the map DB, and provides it to the autonomous driving ECU 50, etc., together with locator information.

The navigation ECU 38 sets a driving route from the current position to the destination based on location information of the current position based on the locator information and location information of the destination, intermediate points, etc. determined by the remote system 130. The navigation ECU 38 may use the driving route set by the remote system 130. The navigation ECU 38 provides route information indicating the driving route to the destination to the automatic driving ECU 50, etc.

The on-board communication device 39 is an external communication unit mounted on the vehicle Am. The on-board communication device 39 functions as a V2X (Vehicle to Everything) communication device. The on-board communication device 39 transmits and receives various information by wireless communication (V2I communication, V2V communication, etc.) between roadside devices installed on the side of the road and other vehicles around the vehicle. The on-board communication device 39 receives signal information indicating the lighting patterns of traffic signals, as well as detection information of stopped vehicles, parked vehicles, pedestrians, cyclists, etc. from the roadside devices and other vehicles.

The in-vehicle communication device 39 enables information sharing between the autonomous driving ECU 50 and the remote control device 140 by communicating with the center communication device 131 via a public communication line network (V2N communication). The in-vehicle communication device 39 transmits vehicle information indicating the state of the vehicle Am to the remote side system 130. The vehicle information includes exterior camera images captured by the camera unit of the perimeter monitoring sensor 30 and interior camera images captured by the interior monitor device 29 (interior camera). The in-vehicle communication device 39 receives input information, remote driving information, observer camera images, and usage application information (boarding application) from the remote side system 130. The in-vehicle communication device 39 provides information received from the roadside device, other vehicles, and the remote side system 130 to the HMI control device 10 and the autonomous driving ECU 50, etc.

The driving control ECU 40 is an electronic control device that mainly includes a microcontroller. The driving control ECU 40 has at least the functions of a brake control ECU, a drive control ECU, and a steering control ECU. The driving control ECU 40 is electrically connected to an operation amount detection sensor 115. The operation amount detection sensor 115 outputs in-vehicle driving information based on the driving operation input to the in-vehicle operation unit 110 to the driving control ECU 40. The driving control ECU 40 obtains from the operation amount detection sensor 115 a number of driving instruction values that indicate the accelerator operation amount, brake operation amount, and steering operation amount input to the in-vehicle operation unit 110 as in-vehicle driving information.

When the automatic driving ECU 50 is performing automatic driving, the driving control ECU 40 continuously performs brake force control for each wheel, output control of the on-board power source, and steering angle control based on the control commands generated by the automatic driving ECU 50. On the other hand, when the host vehicle Am is in a remote driving state, the driving control ECU 40 continuously performs brake force control for each wheel, output control of the on-board power source, and steering angle control according to the commands (remote driving information, driving instruction values) of the remote side system 130 input from the on-board communication device 39. Furthermore, when the passenger PS is permitted to drive inside the vehicle, the driving control ECU 40 continuously performs brake force control for each wheel, output control of the on-board power source, and steering angle control according to the on-board driving information (driving instruction values) obtained from the operation amount detection sensor 115.

The body ECU 43 is an electronic control device that mainly includes a microcontroller. The body ECU 43 controls a door lock actuator 45 installed in the host vehicle Am. The door lock actuator 45 switches the state of the lock mechanism of the doors installed in the host vehicle Am between an unlocked state and a locked state based on a command signal from the body ECU 43. The door lock actuator 45 may be capable of switching the state of the lock mechanism for the inside and outside of the vehicle separately.

The HMI (Human Machine Interface) control device 10 is a computer that mainly includes a control circuit equipped with a processing unit, RAM, a storage unit, an input/output interface, and a bus that connects these. The HMI control device 10 constitutes an in-vehicle notification system together with an in-vehicle display 21, an in-vehicle speaker 22, an external display 24, and an external speaker 25. The HMI control device 10 functions as a presentation control device, and comprehensively controls the presentation of information inside the vehicle using the in-vehicle display 21 and the in-vehicle speaker 22, etc., and the presentation of information outside the vehicle using the external display 24 and the external speaker 25, etc. Based on a request for notification obtained from the automatic driving ECU 50, the HMI control device 10 issues notifications to the inside and outside of the vehicle indicating the operating status of the driverless vehicle ADV and the operating state of the automatic driving function, etc.

The in-vehicle display 21 and the in-vehicle speaker 22 are in-vehicle alarm devices installed in the cabin of the host vehicle Am. The in-vehicle display 21 displays information to the passengers PS in the cabin. Multiple in-vehicle displays 21 are installed in the cabin of the host vehicle Am so that all passengers PS can see the in-vehicle displays 21. The in-vehicle displays 21 employ color displays capable of displaying various information in color. The in-vehicle speaker 22 plays alarm sounds or voice messages, etc. in the cabin. Similar to the in-vehicle display 21, multiple in-vehicle speakers 22 may be installed in the cabin of the host vehicle Am.

The exterior display 24 and the exterior speaker 25 are exterior alarm devices provided on the host vehicle Am. The exterior display 24 is installed on the outer surface of the host vehicle Am and displays information to the outside of the vehicle. The exterior display 24 is a display capable of displaying text, and is provided at least near the door through which the passenger PS gets in and out of the vehicle. The exterior display 24 may also be provided on the front and rear of the host vehicle Am. The exterior speaker 25 plays alarm sounds or voice messages to the outside of the vehicle. The exterior display 24 and the exterior speaker 25 notify pedestrians, cyclists, drivers of other vehicles, etc. around the host vehicle of the current operating status of the automatic driving function of the host vehicle Am.

[Autonomous driving ECU]
The autonomous driving ECU 50 is a computer mainly including a processing unit 51, a RAM 52, a storage unit 53, an input/output interface 54, and a control circuit including a bus connecting these units. The processing unit 51 executes various processes (instructions) for realizing the autonomous driving control method and the vehicle monitoring method of the present disclosure by accessing the RAM 52. The storage unit 53 stores various programs (autonomous driving control program, vehicle monitoring program, etc.) executed by the processing unit 51. By executing the programs by the processing unit 51, the autonomous driving ECU 50 is configured with an information linking unit 61, an environment recognition unit 62, an action determination unit 63, a control execution unit 64, an equipment control unit 65, etc. as multiple functional units for realizing the autonomous driving function and the vehicle monitoring function (see FIG. 3).

The information linking unit 61 provides information to the HMI control device 10 and acquires information from the HMI control device 10, the in-vehicle monitor device 29, the user reception terminal 120, etc. The information linking unit 61 has a user authentication unit 70, a situation grasping unit 71, and an alert request unit 72 as sub-functional units for linking information with the HMI control device 10, the in-vehicle monitor device 29, and the user reception terminal 120.

The user authentication unit 70 obtains information about the user (passenger PS) who will be using the vehicle Am. The user authentication unit 70 obtains the user's boarding application by sharing the usage application information provided by the remote control device 140 with the user reception terminal 120. Based on the boarding application, the user authentication unit 70 obtains information such as the number, age, and gender of users who plan to board the vehicle Am at each stopover point.

The situation grasping unit 71 grasps the situation of the passenger PS in the vehicle cabin of the vehicle Am based on the in-vehicle monitor information acquired from the in-vehicle monitor device 29. When driving inside the vehicle is permitted, the situation grasping unit 71 continuously grasps whether the passenger PS is seated in the driver's seat DS and whether the passenger PS seated in the driver's seat DS is capable of driving the vehicle.

The situation understanding unit 71 continuously monitors the interior of the vehicle cabin to detect injuries or sudden illnesses within the vehicle cabin, troubles between passengers PS within the vehicle cabin, and problematic behavior of passengers PS. As an example, the situation understanding unit 71 detects as problematic behavior the passenger PS moving to the driver's seat DS in a situation where driving in the vehicle is not permitted, and behavior such as touching a specific object (e.g., a drive key) within the vehicle cabin of the vehicle Am. Note that the behavior of the passenger PS that is considered to be problematic may be changed as appropriate.

The situation grasping unit 71 acquires operation information of operations input to the emergency stop button 117, the emergency call button 118, and the emergency evacuation button 119. A plurality of each of the buttons 117-119 are provided in the passenger compartment of the vehicle Am. Each of the buttons 117-119 is arranged near each seat where the passenger PS sits so that it can be easily operated immediately by the passenger PS. A pressing operation by the passenger PS is input to the emergency stop button 117 in a situation where it is necessary to immediately stop the traveling of the vehicle Am. A pressing operation by the passenger PS is input to the emergency call button 118 in a situation where it is necessary to report to the remote system 130 or the like. A pressing operation by the passenger PS is input to the emergency evacuation button 119 in a situation where it is necessary to evacuate the vehicle Am from its current position.

The notification request unit 72 enables the HMI control device 10 to issue a notification in synchronization with the operating state of the automatic driving function by outputting a request for notification to the HMI control device 10. The notification request unit 72 acquires control state information indicating the operating state of the automatic driving function from the action determination unit 63, and outputs an implementation request based on the control state information to the HMI control device 10. As a result, the notification request unit 72 causes the HMI control device 10 to issue a notification indicating the operating state of the automatic driving function to the inside and outside of the vehicle. In addition, the notification request unit 72 outputs a request for notification to the remote control device 140 via the in-vehicle communication device 39. The notification request unit 72 uses the remote side display 136 and the remote side speaker 137 (see FIG. 2) to notify information about the vehicle Am to the remote monitor RO and the operation manager OM linked to the remote side system 130.

The environment recognition unit 62 recognizes the driving environment of the host vehicle Am by combining the locator information and map data acquired from the locator 35 with the detection information acquired from the surrounding monitoring sensor 30. The environment recognition unit 62 can also use the detection information received by the in-vehicle communication device 39 to recognize the driving environment. The environment recognition unit 62 acquires route information from the navigation ECU 38, and provides the acquired route information to the action determination unit 63.

The environment recognition unit 62 has a target recognition unit 73 and a road recognition unit 74 as sub-functional units for recognizing the driving environment. The target recognition unit 73 recognizes the relative positions and relative speeds of dynamic targets around the vehicle, such as other vehicles traveling around the vehicle Am. The road recognition unit 74 acquires road information related to the road on which the vehicle Am is traveling or the road on which it is scheduled to travel. The environment recognition unit 62 sequentially provides the information recognized by the target recognition unit 73 and the road recognition unit 74, i.e., the recognition results of the driving environment, to the action determination unit 63.

When the autonomous driving ECU 50 has control of driving operations, the behavior determination unit 63 generates a planned driving line for the host vehicle Am to travel on, based on the results of the recognition of the driving environment by the environment recognition unit 62 and the route information generated by the navigation ECU 38. The behavior determination unit 63 outputs the generated planned driving line to the control execution unit 64, thereby causing the host vehicle Am to travel autonomously along the planned driving line. The behavior determination unit 63 has a control switching unit 77 as a sub-functional unit that complements the autonomous driving function.

The control switching unit 77 cooperates with the cruise control ECU 40 to switch the control state of the host vehicle Am between automatic driving, remote driving, and in-vehicle driving. When the remote monitor RO at the remote monitoring facility CNT determines that the host vehicle Am should be switched to remote driving, the control switching unit 77 cooperates with the remote control device 140 to transition the control state of the host vehicle Am from automatic driving to remote driving. By switching to remote driving, the cruise control ECU 40 drives the host vehicle Am based on the driving instruction values received from the remote system 130 via the in-vehicle communication device 39. When switching the control state of the host vehicle Am between automatic driving and remote driving, the control switching unit 77 always temporarily stops the host vehicle Am.

When the control state is automatic driving or remote driving, the control switching unit 77 prohibits the passenger PS from driving the vehicle by not reflecting the driving operations input to the in-vehicle operation unit 110 in the behavior of the vehicle Am in the driving control ECU 40. On the other hand, when the remote monitor RO permits the transition to in-vehicle driving, the control switching unit 77 permits the passenger PS to input driving operations to the in-vehicle operation unit 110. When the control switching unit 77 confirms that the passenger PS is seated in the driver's seat DS and is in a state (posture) in which driving operations are possible, it transitions the control state of the vehicle Am from automatic driving to in-vehicle driving. By transitioning to in-vehicle driving, the driving control ECU 40 drives the vehicle Am based on the driving instruction values obtained from the operation amount detection sensor 115.

When the autonomous driving ECU 50 has control of driving operations, the control execution unit 64 cooperates with the cruise control ECU 40 to execute acceleration/deceleration control and steering control of the host vehicle Am according to the planned driving line generated by the action determination unit 63. Specifically, the control execution unit 64 generates control commands based on the planned driving line and sequentially outputs the generated control commands to the cruise control ECU 40.

The device control unit 65 controls the operation of various in-vehicle devices mounted on the vehicle Am by outputting control commands to the body ECU 43. The device control unit 65 cooperates with the body ECU 43 and controls the door lock actuator 45 to switch the state of the door lock mechanism of the vehicle Am.

[How to deal with people approaching your vehicle]
In the driverless vehicle ADV described above, as mentioned above, there is a possibility that a person who is not scheduled to board the vehicle (hereinafter, a warning person UM, see FIG. 4) may attempt to enter the vehicle cabin when the vehicle is temporarily stopped. In such a situation, if no action is taken against the warning person UM, there is a risk of the vehicle Am being stolen and of adverse effects being caused to the passenger PS.

Therefore, when the autonomous driving ECU 50 detects the approach of a noticing person UM, it implements a series of measures to deal with the noticing person as shown in Figure 4. The measures to deal with the noticing person include at least an outside alarm to the noticing person UM, door unlocking control to enable evacuation outside the vehicle, a report to the remote monitoring facility CNT (hereinafter, "center report"), and evacuation control to move the vehicle Am away from the noticing person UM. Below, the details of the determination of the approach of the noticing person UM and the series of measures to deal with the noticing person, including the outside alarm, door unlocking control, center report, and evacuation control, are explained based on Figure 4 and with reference to Figures 1 to 3.

<Judgment of approaching person>
The approach judgment of the notable person UM is performed by the target recognition unit 73. The target recognition unit 73 judges the approach of the notable person UM who is not scheduled to board the vehicle Am to the vehicle Am based on the detection information around the vehicle. The target recognition unit 73 refers to the boarding application by sharing information with the user authentication unit 70, and acquires passenger information such as the planned boarding location where the user is scheduled to board, and the number, age, and gender of users who are scheduled to board at each planned boarding location. Based on the passenger information, the target recognition unit 73 judges whether the person approaching the vehicle Am is a user who is scheduled to board or another notable person UM.

When the vehicle Am is stopped at a location other than the planned boarding location or the planned disembarking location (hereinafter referred to as the boarding/disembarking point) due to a traffic jam or waiting at a traffic light, the landmark grasping unit 73 determines that all people approaching the vehicle Am are people to watch out for UM. The boarding/disembarking points are the destination and intermediate destinations of the driverless vehicle ADV set by the operation management unit 144 based on the usage application information. The boarding/disembarking points are virtual bus stops in automatic operation. As an example, when a person to watch out UM approaches within a few meters of the stopped vehicle Am, the landmark grasping unit 73 determines that the person to watch out UM has approached the vehicle Am.

When the vehicle Am is stopped at a boarding/alighting point and a person of an age or gender not included in the passenger information approaches the vehicle Am, the target recognition unit 73 determines that the person is a person to watch out for UM. In addition, when determining whether a person who is about to board the vehicle Am at the boarding/alighting point is a user or a person to watch out for UM, the target recognition unit 73 may use in-vehicle monitor information provided to the situation recognition unit 71 from the in-vehicle monitor device 29.

When the target object grasping unit 73 grasps the inappropriate behavior of a person attempting to board the vehicle Am, it judges this person as a person to watch out for UM. Specifically, when the target object grasping unit 73 detects a person attempting to board the vehicle Am without going through the normal boarding procedure, such as holding the user terminal 170 over the user reception terminal 120, based on the in-vehicle monitor information, it judges this person as a person to watch out for UM. Furthermore, the target object grasping unit 73 also judges a person attempting to forcefully open a closed door or a person attempting to board through the exit door, etc., as a person to watch out for UM.

<Responses to those with caution>
The outside warning in response to a noticing person is implemented by cooperation between the automatic driving ECU 50 and the HMI control device 10 based on the approach judgment of the noticing person UM. The notification request unit 72 outputs a request for implementing an outside warning indicating that boarding is not permitted to the noticing person UM (hereinafter, a boarding prohibition warning) to the HMI control device 10. As a result, the HMI control device 10 implements a boarding prohibition warning using the exterior display 24 and the exterior speaker 25. The boarding prohibition warning continues until the movement under evacuation control is terminated.

In the no-boarding warning, a message such as "No reservation has been made" or "This service is not available to customers" is displayed on the exterior display 24 located near the door (on the side or above, see Figure 1 or Figure 4). In addition, the same voice message as that displayed on the exterior display 24 is played outside the vehicle by the exterior speaker 25. Furthermore, a voice message providing usage guidance such as "If you wish to use this service, please complete the usage procedure through the app" may also be played.

Unlocking control is performed by cooperation between the device control unit 65 and the body ECU 43 after the initiation of the no-entry warning. When the device control unit 65 determines that the alert person UM is approaching the vehicle Am, it unlocks the door on the left and right sides of the vehicle Am that is opposite the side the alert person UM is approaching (hereinafter, the evacuation door) so that it can be opened from inside the vehicle. The evacuation door may be an emergency exit that is not normally used.

After the evacuation side door is unlocked by the unlocking control, the notification request unit 72 outputs to the HMI control device 10 a request to implement an in-vehicle notification indicating that the evacuation side door is unlocked. The HMI control device 10 notifies the passenger PS that they can open the evacuation side door from within the vehicle cabin and evacuate to the outside of the vehicle by issuing an in-vehicle notification (hereinafter, evacuation door guidance) using the in-vehicle display 21 and the in-vehicle speaker 22. In this evacuation door guidance, the position of the evacuation side door and how to open it may be displayed on the in-vehicle display 21.

A center report is made if the state in which it has been determined that a person UM is approaching continues for a predetermined time (e.g., about 10 seconds) or more after the initiation of the no-boarding warning. When the state in which it has been determined that a person UM is approaching continues for a predetermined time or more, the report request unit 72 works with the HMI control device 10 and uses the in-vehicle display 21 and in-vehicle speaker 22 to inquire whether or not to report the approach of the person UM to the remote monitoring facility CNT. If it is necessary to report to the remote monitoring facility CNT, the HMI control device 10 uses a display and voice message to guide the user to press the emergency report button 118. When the passenger PS presses the emergency report button 118 and the passenger PS in the vehicle cabin approves the report, the report request unit 72 makes a report to the remote monitoring facility CNT.

Evacuation control is driving control that moves the host vehicle Am from its current position. The behavior determination unit 63 determines whether the host vehicle Am can move from its current stopped position based on the approach determination of the watchful eye UM. If the behavior determination unit 63 determines that the host vehicle Am can move, it starts driving using evacuation control. Evacuation control is started, for example, after a center report. Evacuation control may be implemented only when permitted by the remote monitor RO that received the center report. Before starting driving using evacuation control, the equipment control unit 65 locks all doors so that they cannot be opened from inside or outside the vehicle.

Before starting travel under evacuation control, the target grasping unit 73 determines the possibility of contact between the watcher UM and the vehicle Am. When there is a possibility of contact with the watcher UM, the behavior judgment unit 63 suppresses the movement speed of the vehicle Am during evacuation control more than when there is no possibility of contact. Cases in which the watcher UM is already touching the vehicle Am are included in cases in which there is a possibility of contact. When the movement speed is suppressed, the behavior judgment unit 63 starts the vehicle Am to maintain a slow speed (e.g., a few kilometers per hour) while allowing the watcher UM to contact the vehicle Am. The behavior judgment unit 63 moves the vehicle Am by a few meters, and accelerates the vehicle Am normally when there is no possibility of contact with the watcher UM.

When the behavior determination unit 63 has moved the vehicle Am a predetermined distance (e.g., several meters to several tens of meters), it ends the evacuation control and temporarily stops the vehicle Am. In addition, if the user authentication unit 70 acquires a boarding application for the watchful person UM after the behavior determination unit 63 has started moving from the current position, it stops the movement of the vehicle Am. In this case, the approach determination for the watchful person UM is cancelled. The application information for the watchful person UM may be provided to the remote system 130 by the user terminal 170 of the watchful person UM, or may be provided to the remote system 130 by the user terminal 170 of the passenger PS in the vehicle cabin.

If the behavior determination unit 63 determines that the vehicle Am cannot move after determining that the watchful person UM is approaching, the device control unit 65 locks all doors of the vehicle Am so that they cannot be opened from the outside. In this case, the no-boarding warning continues. If the vehicle Am becomes movable, the behavior determination unit 63 starts movement using evacuation control while keeping all doors locked.

In addition to the normal evacuation control described above (hereinafter, normal evacuation control), the behavior determination unit 63 can also perform emergency evacuation control (hereinafter, emergency evacuation control) that immediately moves the vehicle Am. When the emergency evacuation button 119 provided in the vehicle cabin is operated by the passenger PS, the behavior determination unit 63 starts movement by emergency evacuation control based on this emergency evacuation operation.

The behavior determination unit 63, in emergency evacuation control, moves the vehicle Am to a location farther from the current position than in normal evacuation control. The behavior determination unit 63 continues moving the vehicle Am until it receives a stop command from at least one of the passenger PS in the vehicle cabin and the remote monitor RO at the remote monitoring facility CNT. Specifically, when the passenger PS inputs a pressing operation on the emergency stop button 117, or when the remote monitor RO inputs a stop command to the input device 133, the behavior determination unit 63 ends the emergency evacuation control and stops the vehicle Am.

[Handling of people with caution and emergency evacuation]
Next, the details of the not paying attention response processing and emergency evacuation processing performed by the automatic driving ECU 50 to realize the not paying attention response and emergency evacuation control described above will be further described based on Figures 5 and 6 and with reference to Figures 1 to 4. The not paying attention response processing and emergency evacuation processing are started by the automatic driving ECU 50 when the operation of the host vehicle Am by automatic driving or remote driving is started. The not paying attention response processing and emergency evacuation processing are continuously performed by the automatic driving ECU 50 until the operation of the host vehicle Am is terminated.

<Handling of people with caution>
In S11 of the watcher response processing shown in Fig. 5, the environment recognition unit 62 judges whether the host vehicle Am is stopped. If the host vehicle Am is traveling (S11: NO), the state of waiting for the host vehicle Am to stop continues. If the host vehicle Am is stopped (S11: YES), the target recognition unit 73 performs vehicle exterior monitoring in S12 and judges whether the watcher UM is approaching in S13. If the watcher UM is not approaching (S13: NO), the processing of S11 to S13 is repeated.

If it is determined that the warning person UM is approaching (S13: YES), the notification request unit 72 cooperates with the HMI control device 10 in S14 to issue an external notification indicating that boarding is not permitted. Furthermore, based on the determination that the warning person UM is approaching, the equipment control unit 65 unlocks the evacuation side door by the unlocking control described above in S15. In addition, in S15, an evacuation door guide is issued, which is an internal notification indicating that the evacuation side door is unlocked. This allows the passenger PS to understand that they can open the evacuation side door and evacuate to the outside of the vehicle.

In S16, the target recognition unit 73 determines whether the approaching state of the watcher UM has continued for a predetermined time or more. If the approaching state of the watcher UM has not continued for a predetermined time or more (S16: NO), the processes of S11 to S16 are repeated. On the other hand, if the approaching state of the watcher UM has continued for a predetermined time or more (S16: YES), the notification request unit 72 performs a process to confirm the need for a center report in S17. If the execution of a center report is approved by the passenger PS, that is, if the emergency report button 118 is operated, the notification request unit 72 executes a center report to the remote monitoring facility CNT.

The behavior determination unit 63 determines in S18 whether or not the host vehicle Am can move from its current position. If the host vehicle Am cannot move (S18: NO), the equipment control unit 65 locks all doors of the host vehicle Am in S19. Then, the target recognition unit 73 determines in S20 whether or not the watcher UM has disappeared. If the watcher UM has disappeared (S20: YES), the watcher response process is temporarily terminated. On the other hand, if the watcher UM has not disappeared and is still being detected (S20: NO), the behavior determination unit 63 waits until the host vehicle Am becomes capable of moving.

If the host vehicle Am is capable of moving (S18: YES), the target recognition unit 73 determines in S21 whether or not there is a possibility of the watcher UM coming into contact with the host vehicle Am. If there is a possibility of the watcher UM coming into contact with the host vehicle Am (S21: YES), the behavior determination unit 63 applies a setting in S22 that suppresses the movement speed of the host vehicle Am more than when there is no possibility of contact. On the other hand, if there is no possibility of the watcher UM coming into contact with the host vehicle Am (S21: NO), such a speed suppression setting is not applied. In S23, the behavior determination unit 63 continues the no-boarding warning while starting to move the host vehicle Am from its current position using normal evacuation control.

After starting the movement of the host vehicle Am, the behavior determination unit 63 determines whether the stop condition of the host vehicle Am is satisfied in S24. The behavior determination unit 63 determines that the stop condition is satisfied when the approach of the watcher UM is resolved, when the host vehicle has moved a predetermined distance, when the passenger PS inputs a pressing operation on the emergency stop button 117, or when a boarding application for the watcher UM is acquired. If the stop condition is not satisfied (S24: NO), the behavior determination unit 63 waits for the stop determination to be satisfied. On the other hand, if the stop condition is satisfied (S24: YES), the behavior determination unit 63 ends the movement of the host vehicle Am by the normal evacuation control in S25. If the watcher UM approaches the host vehicle Am again after evacuation, the automatic driving ECU 50 performs the watcher response process again to move the host vehicle Am away from the watcher UM.

<Emergency evacuation procedures>
In S41 of the emergency evacuation processing shown in Fig. 6, the situation grasping unit 71 judges whether or not an emergency evacuation operation has been performed on the emergency evacuation button 119 by the passenger PS. If there has been no emergency evacuation operation (S41: NO), the judgment of S41 is repeated, thereby maintaining a state of waiting for the input of an emergency evacuation operation. On the other hand, if there has been an emergency evacuation operation (S41: YES), emergency evacuation control is started. Emergency evacuation control is carried out with priority over normal evacuation control. In S42, the device control unit 65 locks all doors of the host vehicle Am so that they cannot be opened from inside or outside the vehicle.

In S43, the behavior determination unit 63 determines whether or not the host vehicle Am can be moved from its current position. If the host vehicle Am cannot be moved (S43: NO), the situation understanding unit 71 determines in S44 whether or not an operation to cancel emergency evacuation control has been performed. As an example, the operation of pressing the emergency stop button 117 by the passenger PS or the operation of inputting information into the input device 133 by the remote monitor RO is regarded as an operation to cancel emergency evacuation control. If a cancellation operation has been performed (S44: YES), the emergency evacuation process returns to a state waiting for input of an emergency evacuation operation. On the other hand, if a cancellation operation has not been performed (S44: NO), the behavior determination unit 63 waits for the host vehicle Am to become movable.

If the vehicle Am is capable of moving (S43: YES), the behavior determination unit 63 starts emergency movement of the vehicle Am from its current position using emergency evacuation control in S45. At this time, the notification request unit 72 may cooperate with the HMI control device 10 to provide an external notification to notify the vehicle of the start of movement. The behavior determination unit 63 determines whether a stop instruction has been acquired from at least one of the passenger PS and the remote monitor RO in S46. If a stop instruction has not been acquired (S46: NO), the determination in S46 is repeated to wait for the acquisition of a stop instruction. As a result, the behavior determination unit 63 continues traveling of the vehicle Am in a direction in which the vehicle is capable of traveling until a stop instruction is acquired from the passenger PS or the remote monitor RO. Then, when a stop instruction is acquired (S46: YES), the behavior determination unit 63 ends the movement of the vehicle Am using emergency evacuation control in S47.

[Responses to problematic behavior by passengers]
In addition to responding to the alert person UM outside the vehicle as described above, the automatic driving ECU 50 responds to problematic behavior by the passenger PS inside the vehicle cabin by continuously monitoring the interior of the vehicle. Details of the behavior warning process and the theft notification process performed by the automatic driving ECU 50 will be further described below with reference to Figures 1 to 4 as well as Figures 7 and 8. The behavior warning process and the theft notification process are started by the automatic driving ECU 50 when the host vehicle Am has started to be driven automatically or remotely. The behavior warning process and the theft notification process are started by the automatic driving ECU 50 at predetermined time intervals until the host vehicle Am has stopped being driven.

<Behavior warning processing>
The behavior warning process shown in Fig. 7 is a process for issuing an in-vehicle warning against problem behavior of the passenger PS while in the vehicle. In S61 of the behavior warning process, the situation grasping unit 71 performs in-vehicle monitoring using in-vehicle monitor information. Based on the in-vehicle monitor information acquired in S61, the situation grasping unit 71 determines in S62 whether or not problem behavior of the passenger PS has been detected by the in-vehicle monitor device 29. As described above, the in-vehicle monitor device 29 detects, as problem behavior, the movement of the passenger PS to the driver's seat DS under circumstances in which driving in the vehicle is not permitted, and the contact of the passenger PS with the drive key. If no problem behavior is detected (S62: NO), the current behavior warning process is terminated.

On the other hand, if the situation grasping unit 71 determines that problematic behavior has been detected (S62: YES), the notification request unit 72 cooperates with the HMI control device 10 in S63 to issue an in-vehicle warning associated with the detected problematic behavior. Specifically, if the movement of the passenger PS to the driver's seat DS is detected, the notification request unit 72 issues an in-vehicle warning to encourage the passenger to leave the driver's seat DS using at least one of the in-vehicle display 21 and the in-vehicle speaker 22. In this case, the notification request unit 72 may issue an in-vehicle warning to caution the passenger not to touch the in-vehicle operation unit 110. Also, if the touch of the passenger PS to the drive key is detected, the notification request unit 72 issues an in-vehicle warning to caution the passenger not to touch the drive key using at least one of the in-vehicle display 21 and the in-vehicle speaker 22.

After issuing an in-vehicle warning, the notification request unit 72 notifies the remote monitor RO of the remote monitoring facility CNT, which remotely monitors the vehicle Am, of the occurrence of problematic behavior by the passenger PS, in S64. In such a center report, the remote monitor RO is also notified of the content of the problematic behavior. Specifically, the remote monitor RO is notified by the center report of the occurrence of the passenger PS moving to the driver's seat DS, touching the drive key, etc. In this way, the remote monitor RO can determine whether the behavior of the passenger PS is not problematic behavior but is an action taken in response to an emergency by directly checking with the passenger PS.

<Theft Report Processing>
The theft notification process shown in Fig. 8 is a process for notifying the surroundings of the vehicle and the remote monitoring facility CNT of the occurrence of theft (bus jacking) of the driverless vehicle ADV. In S71 of the theft notification process, the situation grasping unit 71 determines whether or not theft has occurred. If theft has not occurred (S71: NO), the current theft notification process is terminated.

On the other hand, if it is determined that a theft has occurred (S71: YES), the notification request unit 72 notifies the remote monitoring facility CNT of the theft in S72. The notification request unit 72 may also notify the police, a security company, etc. in addition to the remote monitoring facility CNT. Furthermore, in S73, the notification request unit 72 cooperates with the HMI control device 10 to notify the surroundings of the vehicle of the theft using the exterior display 24. As an example, a text message such as "A vehicle has been stolen. Please move away from this vehicle" is flashed on the exterior display 24. The exterior notification notifying the theft of the driverless vehicle ADV continues until a predetermined stop operation is input by the remote monitor RO of the remote monitoring facility CNT.

(Summary of the first embodiment)
In the first embodiment described so far, an external notification is issued to a tormentor UM approaching the host vehicle Am, indicating that the tormentor UM is not allowed to board the host vehicle Am, using the in-vehicle display 21 and the in-vehicle speaker 22 provided in the host vehicle Am. This external notification can prevent a tormentor UM who is not scheduled to board the host vehicle Am from boarding the host vehicle Am. Therefore, even if a driver is not on board, it is possible to respond to a tormentor UM approaching the host vehicle Am. As a result, it is possible to reduce the risk of theft of a vehicle operated as a driverless vehicle ADV, the risk of adverse effects on a passenger PS boarding the driverless vehicle ADV, and the like.

In addition, in the first embodiment, when a watchful person UM approaches the host vehicle Am while the vehicle is stopped, the behavior determination unit 63 determines whether the host vehicle Am can move from its current position while the vehicle is stopped. If the behavior determination unit 63 determines that the host vehicle Am can move, it moves the host vehicle Am from its current position. By moving the host vehicle Am away from the watchful person UM through such evacuation control, the risk to the host vehicle Am caused by the watchful person UM can be reliably reduced.

In addition, in the first embodiment, when the emergency evacuation button 119 provided in the vehicle cabin is operated by the passenger PS, the behavior determination unit 63 moves the host vehicle Am to a location farther away from the current position than when the emergency evacuation button 119 is not operated. This type of emergency evacuation control makes it possible to effectively reduce the risk to the host vehicle Am caused by the alert person UM by utilizing the judgment of the passenger PS.

Furthermore, in the first embodiment, the notification request unit 72 determines the possibility of contact between the beware person UM and the vehicle Am. Then, when there is a possibility of contact with the beware person UM, the behavior determination unit 63 suppresses the movement speed more than when there is no possibility of contact. As a result, even when there is a possibility of contact with the beware person UM, it is possible to reliably move the vehicle Am away from the beware person UM while allowing contact with the beware person UM. As a result, the risk to the vehicle Am caused by the beware person UM can be reliably reduced.

In addition, in the first embodiment, if the behavior determination unit 63 determines that the host vehicle Am cannot be moved, the doors of the host vehicle Am are locked so that they cannot be opened from the outside. As a result, even in a scene in which the host vehicle Am cannot be moved away from the alert person UM, the risk of adverse effects on the passenger PS inside the vehicle cabin can be reliably reduced.

In the first embodiment, a request to board the vehicle Am is acquired by the user authentication unit 70. After the vehicle Am starts moving from its current position, if a request to board for a caution person UM is acquired by the user authentication unit 70, the behavior determination unit 63 stops the movement of the vehicle Am. As a result of the above, it is possible to avoid a situation in which a user who has no intention of harming the driverless vehicle ADV and has simply not made a request to board is continually determined to be a caution person UM.

Furthermore, in the first embodiment, the behavior determination unit 63 continues moving the host vehicle Am until it receives a stop command from at least one of the passenger PS in the vehicle cabin and the remote monitor RO that remotely monitors the host vehicle Am. As a result, it becomes possible to reliably evacuate the host vehicle Am to a position where it will not be adversely affected by the cautionary person UM.

In addition, in the first embodiment, if the state in which it has been determined that a person UM is approaching continues for a predetermined period of time or more, an inquiry is made using the in-vehicle display 21 and in-vehicle speaker 22 installed in the vehicle cabin as to whether or not to report the approach of the person UM to the remote monitoring facility CNT. Then, if the passenger PS in the vehicle cabin approves the report, the notification request unit 72 reports to the remote monitoring facility CNT. As a result of the above, when a person UM to beware that is determined to be high risk by the passenger PS approaches the vehicle Am, the remote monitor RO of the remote monitoring facility CNT can be fully aware of the approach of the person UM to be careful.

In the first embodiment, the movement of the passenger PS to the driver's seat DS, which is closest to the in-vehicle operation unit 110 provided in the vehicle Am, is detected. Then, when the notification request unit 72 determines that the passenger PS has moved to the driver's seat DS, it issues an in-vehicle warning using the in-vehicle display 21 and in-vehicle speaker 22 provided in the vehicle cabin to encourage the passenger to leave the driver's seat DS. As a result of the above, it is possible to appropriately deter problematic behavior of the passenger PS in the vehicle cabin that may lead to theft of the driverless vehicle ADV.

Furthermore, in the first embodiment, after an in-vehicle warning is issued to encourage the passenger to leave the driver's seat DS, the remote monitor RO that remotely monitors the vehicle Am is notified of the passenger PS's movement to the driver's seat DS. Therefore, the remote monitor RO can quickly and thoroughly grasp the passenger PS's behavior that may lead to theft of the driverless vehicle ADV.

In addition, in the first embodiment, it is detected that the passenger PS has touched the drive key inside the vehicle cabin of the vehicle Am. Then, when the notification request unit 72 determines that the passenger PS has touched the drive key, it issues an in-vehicle warning using the in-vehicle display 21 and in-vehicle speaker 22 provided inside the vehicle cabin to warn the passenger not to touch the drive key. This in-vehicle warning also makes it possible to appropriately deter problematic behavior by the passenger PS that could lead to theft of the driverless vehicle ADV.

In addition, in the first embodiment, when it is determined that the alert person UM is approaching the vehicle Am, the evacuation side door located on the opposite side of the vehicle Am from the side where the alert person UM is approaching is unlocked so that it can be opened from inside the vehicle cabin. Therefore, if the alert person UM is a high-risk person, the passenger PS can open the evacuation side door and evacuate to the outside of the vehicle. As a result, the risk of adverse effects on the passenger PS can be more reliably reduced.

Furthermore, in the first embodiment, after the evacuation side door is unlocked, an in-vehicle notification is issued using an in-vehicle display 21 and an in-vehicle speaker 22 provided inside the vehicle cabin to indicate that the evacuation side door is unlocked. As a result, the passenger PS can be sure that he or she can open the evacuation side door and evacuate to the outside of the vehicle. As a result, the risk of adverse effects on the passenger PS can be further reduced.

In the first embodiment, the remote monitoring facility CNT corresponds to the "report destination", the remote monitor RO corresponds to the "monitor", the in-vehicle display 21 and the in-vehicle speaker 22 correspond to the "in-vehicle alarm", and the outside display 24 and the outside speaker 25 correspond to the "exterior alarm". The autonomous driving ECU 50 corresponds to the "vehicle monitoring device", the action determination unit 63 corresponds to the "action control unit", the equipment control unit 65 corresponds to the "lock control unit", and the user authentication unit 70 corresponds to the "application acquisition unit". Furthermore, the situation grasping unit 71 corresponds to the "in-vehicle monitoring unit", the notification request unit 72 corresponds to the "notification implementation unit", the target grasping unit 73 corresponds to the "approach determination unit", and the in-vehicle operation unit 110 corresponds to the "driving operation unit".

Second Embodiment
The second embodiment of the present disclosure is a modified example of the first embodiment. The autonomous driving ECU 50 of the second embodiment shown in Fig. 9 further includes a video distribution unit 66 and a security function unit 67 by executing a program by the processing unit 51. Hereinafter, the details of the video distribution process, the warning person response process, theft notification process, and the takeover response process performed by the autonomous driving ECU 50 of the second embodiment will be described based on Figs. 10 to 13 and with reference to Figs. 1, 2, and 9.

<Video distribution processing>
The video distribution process shown in Fig. 10 is a process of distributing in-vehicle camera video from the in-vehicle monitor device 29 (in-vehicle camera) to the user terminal 170 linked to the user of the driverless vehicle ADV in approximately real time. The video distribution process is mainly performed by the video distribution unit 66. A user who is scheduled to board the driverless vehicle ADV and has made a reservation (approver) whose boarding application has been accepted by the remote system 130 requests distribution of in-vehicle camera video from an application on the user terminal 170. The video distribution unit 66 starts the video distribution process based on obtaining the distribution request from the user terminal 170.

In S201 of the video distribution process, the video distribution unit 66 identifies the user who is requesting the distribution. The video distribution unit 66 uses the usage application information provided by the remote control device 140 and the user identification information acquired together with the distribution request, etc., to identify the user. In S202, the video distribution unit 66 determines whether the user identified in S201 is an approved user whose boarding application has been accepted. If the user who requested the distribution is not an approved user (S202: NO), the video distribution unit 66 does not provide in-car camera video. In S205, the video distribution unit 66 sends an error message to the requesting user terminal 170.

If the user requesting distribution is an approved user (S202: YES), the video distribution unit 66 determines in S203 whether or not it is a distribution period during which in-vehicle camera video can be provided to the approved user. Whether or not it is a distribution period is determined based on the positional relationship between the approved user's planned boarding point and the current position of the vehicle Am, and the time relationship between the approved user's planned boarding time and the current time, etc.

The video distribution unit 66 acquires information indicating the intended boarding point of the approved user and information indicating the current location of the vehicle Am. The video distribution unit 66 determines that the distribution period is possible when the vehicle Am approaches within a predetermined distance (e.g., several hundred meters to 1 km) from the intended boarding point. The video distribution unit 66 also acquires information indicating the intended boarding time of the approved user and information indicating the current time. The video distribution unit 66 determines that the distribution period is possible when the intended boarding time is within a predetermined time (e.g., several minutes to 10 minutes). The video distribution unit 66 may permit the distribution of in-car camera video when one of the above location conditions and time conditions is met, or may permit the distribution of in-car camera video when both conditions are met.

The video distribution unit 66 continues the distribution period until a predetermined time (e.g., several minutes to 10 minutes) has elapsed after the approved user has exited the vehicle, and continues to permit distribution of the in-vehicle camera video to the user terminal 170. The video distribution unit 66 ends the distribution period when the predetermined time has elapsed after the approved user has exited the vehicle.

If the video distribution unit 66 determines that it is not a period during which distribution to an approved user is possible (S203: NO), it does not provide the in-car camera video, and in S205 sends an error message to the requesting user terminal 170. In this case, for example, a notification such as "It is outside the period during which distribution is possible" is sent to the user terminal 170 of the approved user. On the other hand, if it determines that it is a period during which distribution to an approved user is possible (S203: YES), the video distribution unit 66 starts distributing the in-car camera video from the interior camera to the user terminal 170 in S204.

When multiple in-vehicle cameras are installed in the vehicle cabin, the video distribution unit 66 switches between the in-vehicle camera images provided to the user terminal 170 in response to a request from the user terminal 170. In addition, the video distribution unit 66 may be able to change the resolution of the in-vehicle camera images provided to the user terminal 170 in response to a request from the user terminal 170, and may enable zooming in and out of the in-vehicle camera images.

Some or all of the steps of the above video distribution process may be performed by the remote control device 140. That is, a functional unit equivalent to the video distribution unit 66 may be constructed in the remote control device 140, not in the autonomous driving ECU 50. The video distribution unit 66 provided in the remote control device 140 acquires in-vehicle camera video from the driverless vehicle ADV, and provides the acquired in-vehicle camera video to a user terminal 170 linked to an approved user.

<Handling of people with caution>
The noting person response process shown in Fig. 11 is a process for responding to a noting person UM approaching the host vehicle Am at a location different from the pre-set boarding and disembarking point. The noting person response process is mainly performed by the user authentication unit 70, the notification request unit 72, the target recognition unit 73, the safety function unit 67, etc. In the noting person response process, the passenger PS in the vehicle cabin determines whether or not personal authentication of the noting person UM may be performed. As in the first embodiment, the noting person response process is started based on the start of operation of the host vehicle Am by automatic driving or remote driving, and is continuously performed until the operation of the host vehicle Am ends.

In S211 of the warning person response processing, the environment recognition unit 62 determines whether the host vehicle Am is stopped at a location other than the boarding/alighting point. If the host vehicle Am is traveling or stopped at a boarding/alighting point (S211: NO), the host vehicle continues to wait for stopping at a location other than the boarding/alighting point. On the other hand, if the host vehicle Am is stopped at a location other than the boarding/alighting point (S211: YES), the target recognition unit 73 performs outside vehicle monitoring in S212 and determines whether a warning person UM is approaching in S213. If a warning person UM is not approaching (S213: NO), the processing of S211 to S213 is repeated.

If it is determined that a person UM is approaching (S213: YES), the notification request unit 72 cooperates with the HMI control device 10 in S214, and uses the in-vehicle display 21 and the in-vehicle speaker 22 to inquire of the passenger PS in the vehicle cabin as to whether or not to perform personal authentication of the person UM. In this case, the in-vehicle inquiry displays a message such as "If you know someone outside the vehicle, please let us know" on the in-vehicle display 21. The in-vehicle display 21 also displays a touch icon such as "Allow boarding." In addition, the in-vehicle speaker 22 plays the same voice message displayed on the in-vehicle display 21 inside the vehicle cabin.

The situation grasping unit 71 determines in S215 whether or not there has been in-vehicle approval from the passenger PS. The passenger PS performs in-vehicle approval by inputting a touch operation on a touch icon, for example. The passenger PS may perform in-vehicle approval using an application on the user terminal 170. If there has been no in-vehicle approval from the passenger PS (S215: NO), the notification request unit 72 performs a center report in S216. This notifies the remote monitoring facility CNT of the approach of a watchful person UM that is not connected to the passenger PS to the vehicle Am. Furthermore, in S219, the notification request unit 72 cooperates with the HMI control device 10 to perform an outside notification using the outside display 24 and the outside speaker 25. In this case, a no-boarding warning indicating that boarding is not permitted is issued to the watchful person UM.

On the other hand, if there is in-vehicle approval by the passenger PS (S215: YES), the user authentication unit 70 performs personal authentication of the watchful UM in S217. If the watchful UM has been preregistered as a user of the mobility service, the watchful UM can complete personal authentication, for example, by holding the user terminal 170 over the user reception terminal 120. Personal authentication may be performed using an application on the user terminal 170, or may be performed by image analysis of the video from the outside-vehicle camera.

If personal authentication is completed (S217: YES), the security function unit 67 in S218 treats the alert person UM as an authenticated user and allows them to board the vehicle Am. In this case, the boarding door is unlocked through cooperation between the equipment control unit 65 and the body ECU 43. As a result, the user detected as the alert person UM is allowed to board the vehicle. On the other hand, if personal authentication is not completed (S217: NO), the security function unit 67 does not allow the alert person UM to board. In this case, in S219 as well, a no-boarding warning indicating that boarding is not permitted is issued to the alert person UM.

<Theft Report Processing>
The theft notification process shown in Fig. 12 is a process for issuing an external warning that the vehicle Am has become a stolen vehicle when the vehicle Am is stolen from a parked vehicle outside of operating hours. The theft notification process assumes that the vehicle Am is transported in a non-self-propelled state by a tow truck or a forwarding vehicle. The theft notification process is started by the security function unit 67 based on the execution of a function-on process for putting the vehicle Am into a runnable state, such as by turning on the main power supply or the ignition.

In S231 of the theft notification process, the security function unit 67 determines whether the theft detection function is on or not. The theft detection function can only be turned off by an operator with special authority, such as the operations manager OM. To switch the theft detection function off, input of a password or the like may be required. For example, when the vehicle Am is being transported by the legitimate owner, the theft detection function is turned off. If the theft detection function is off (S231: NO), the security function unit 67 determines in S235 that it is in a normal state and ends the theft notification process.

If the theft detection function is on (S231: YES), the security function unit 67 determines the current position of the vehicle Am in S232. The security function unit 67 cooperates with the environment recognition unit 62 and the locator 35 to acquire the position information of the vehicle Am determined based on the function on processing as information indicating the activation location. Furthermore, the security function unit 67 reads out information indicating the stopping location stored during the previous function off processing from the position memory unit 79 in S233. The position memory unit 79 acquires the position information of the vehicle Am based on the function off processing of the vehicle Am, such as turning off the main power supply or the ignition. The position memory unit 79 stores the acquired position information as information indicating the stopping location of the vehicle Am. The order of carrying out the above S232 and S233 may be changed as appropriate.

In S234, the security function unit 67 determines whether the location information indicating the start location and the location information indicating the stop location are consistent. If the deviation between the start location and the stop location is equivalent to the accuracy of the GNSS (for example, within a few meters), the security function unit 67 determines that the start location and the stop location are consistent. If the start location and the stop location are consistent (S234: YES), the security function unit 67 determines in S235 that theft has not occurred and that the vehicle is in a normal state.

On the other hand, if the start location and the stop location do not match (S234: NO), the security function unit 67 determines in S236 that a theft has occurred. In this case, the notification request unit 72 issues an external warning indicating that a theft has occurred. Specifically, the notification request unit 72 causes the external display 24 to display a text message such as "Your vehicle has been stolen. This vehicle is a stolen vehicle." In addition, the notification request unit 72 causes the external speaker 25 to play a voice message saying "A theft has occurred" or a loud alarm sound. Furthermore, the notification request unit 72 transmits a notification indicating that a theft has occurred together with the position information of the start location acquired in S232 to the remote monitoring facility CNT, the police, a security company, etc.

<Hookup response process>
The hijacking response process shown in Fig. 13 is a process for reporting the occurrence of theft of the vehicle Am to the remote monitoring facility CNT. In the hijacking response process, it is assumed that a thief takes over some of the functions of the vehicle-side system IVS. The hijacking response process enables reporting to the remote monitoring facility CNT by cooperating with the user terminal 170 of the passenger PS even when some of the vehicle-side system IVS is not functioning. The hijacking response process is started when the vehicle Am starts to operate by automatic driving or remote driving, and is continuously performed until the operation of the vehicle Am ends.

In S251 of the hijacking response process, the security function unit 67 determines whether or not a theft has occurred based on the situation inside the vehicle cabin ascertained by the situation ascertainment unit 71. If a theft has not occurred (S251: NO), the current hijacking response process ends. On the other hand, if it is determined that a theft has occurred (S251: YES), the security function unit 67 determines whether or not an abnormality has occurred in the vehicle-side system IVS, in other words, whether or not a function has been hijacked in connection with the theft.

Specifically, in S252, the security function unit 67 determines whether or not there is an abnormality in the external communication system mainly composed of the in-vehicle communication device 39. If there is an abnormality in the external communication system (S252: YES) and there is a possibility that a report cannot be made to a remote monitoring facility CNT or the like, the security function unit 67 cooperates with a user terminal 170 located in the vehicle cabin in S253. The user terminal 170 to be cooperated with is a user terminal 170 owned by the passenger PS, and is, for example, a user terminal 170 connected to the in-vehicle Wi-Fi (registered trademark) of the vehicle Am. The security function unit 67 may cooperate with multiple user terminals 170 located in the vehicle cabin. In S259, the security function unit 67 requests the user terminal 170 cooperated with in S253 to make a report to the center. Based on a request from the security function unit 67 in S259, the user terminal 170 transmits the current location information acquired by the terminal to the remote monitoring facility CNT, etc., along with a notification of the theft.

On the other hand, if there is no abnormality in the external vehicle communication system (S252: NO), the security function unit 67 determines in S254 whether or not there is an abnormality in the positioning system based on the locator 35. If there is an abnormality in the positioning system (S254: YES) and there is a possibility that correct position information of the vehicle Am cannot be obtained, the security function unit 67 cooperates with the user terminal 170 in the vehicle cabin in S256, as in S253. The security function unit 67 obtains the latest position information from the user terminal 170 that cooperated in S256 in S257.

If there are no abnormalities in both the exterior communication system and the positioning system (S254: NO), in S255 the security function unit 67 acquires the position information of the vehicle Am from the locator 35. In S258, the notification request unit 72 uses the in-vehicle communication device 39 to notify the center of the occurrence of the theft. In this center notification, the position information acquired in S255 or S257, which indicates the current position of the vehicle Am, is transmitted to the remote monitoring facility CNT or the like. Furthermore, in S260, the notification request unit 72 also notifies the surroundings of the vehicle of the theft using the exterior display 24 and exterior speaker 25.

(Summary of the second embodiment)
In the second embodiment described above, the outside alarm to the tormented person UM approaching the vehicle Am is also provided to prevent the tormented person UM from getting into the vehicle Am. As a result, the same effect as in the first embodiment is achieved, and it becomes possible to respond to the tormented person UM approaching the vehicle Am even when the driver is not in the vehicle.

In addition, in the second embodiment, if it is determined that a person UM approaches the vehicle Am at a location other than the preset boarding and disembarking point, an inquiry as to whether or not to perform personal authentication of the person UM is made using an in-vehicle display 21 and an in-vehicle speaker 22 provided in the vehicle cabin. Then, if the performance of personal authentication is permitted by the passenger PS in the vehicle cabin, personal authentication of the person UM is performed. Therefore, even if a person related to the passenger PS of the vehicle Am approaches the vehicle Am without going through the proper procedures, it is possible to have this person board the vehicle Am with the cooperation of the passenger PS. As a result, the convenience of the driverless vehicle ADV is further improved.

In the second embodiment, in-vehicle camera video is delivered to the user terminal 170 based on a request from the user terminal 170 linked to an approved user whose reservation for boarding the vehicle Am has been accepted. Therefore, a user who plans to use the driverless vehicle ADV can check before boarding whether there are any suspicious persons on board, whether the vehicle is excessively crowded, etc. In addition, if the driverless vehicle ADV is a shuttle vehicle for ride-hailing and is not intended for carpooling, the user can check before boarding that no other passengers have boarded by mistake.

Furthermore, in the second embodiment, when the vehicle Am approaches within a predetermined distance of the approved user's planned boarding point, the delivery of the in-car camera video to the user terminal 170 is permitted. By setting such a delivery period, the delivery destination of the in-car camera video can be limited to the range of appropriate users.

In addition, in the second embodiment, when the approved user's scheduled boarding time is within a predetermined time, the delivery of the in-car camera video to the user terminal 170 is permitted. By setting such a delivery period, the delivery destination of the in-car camera video can be limited to the appropriate range of users.

In addition, in the second embodiment, after an approved user gets off the vehicle, delivery of in-vehicle camera footage to the user terminal 170 continues to be permitted until a predetermined time has elapsed. As a result, the user can check whether or not they have left anything behind in the vehicle by viewing the in-vehicle camera footage after getting off the vehicle.

Furthermore, in the second embodiment, the stopping location of the vehicle Am is stored in the position memory unit 78 based on the function-off processing of the vehicle Am. Then, if the starting location of the vehicle Am determined based on the function-on processing is different from the stopping location, an outside-vehicle warning indicating that the vehicle Am has been stolen is implemented. As a result of the above, even if the vehicle Am is stolen after the function-off processing and taken far away, the processing leading to the discovery of the vehicle Am can be reliably executed.

In addition, in the second embodiment, if theft, such as bus hijacking, occurs on the vehicle Am while it is in operation, the location information of the user terminal 170 linked to the passenger PS inside the vehicle is notified to the remote monitoring facility CNT. Therefore, even if part of the vehicle-side system IVS becomes non-functional due to theft and it is impossible to report from the vehicle-side system IVS to the remote monitoring facility CNT, the user terminal 170 or its application functions can be used to report to the remote monitoring facility CNT. In the second embodiment, the in-vehicle monitor device 29 corresponds to the "in-vehicle camera."

Third Embodiment
The third embodiment of the present disclosure is a modified example of the second embodiment. The autonomous driving ECU 50 of the third embodiment shown in Fig. 14 is mounted on a POV (Personally Owned Vehicle) that is assumed to be owned by an individual, rather than on a vehicle that provides mobility services. The autonomous driving ECU 50 is capable of autonomously driving the host vehicle Am, which is the POV, at least in a specific area (e.g., a parking lot of a commercial facility, etc.).

The user terminal 170 of the owner OW who owns the vehicle Am is preregistered in the autonomous driving ECU 50. The user terminal 170 transmits a command to call the vehicle Am (hereinafter, the call command) to the autonomous driving ECU 50 together with the location information of the terminal. In response to a remote call from the user terminal 170 linked to the owner OW, the autonomous driving ECU 50 moves the vehicle Am to a position close to the owner OW without a driver on board. The user terminal 170 linked to the owner OW may be a general-purpose terminal such as a smartphone, or may be a dedicated terminal including the function of a FoB key.

If a notable person UM, different from the owner OW, approaches the vehicle Am during the movement period in response to the call command, the automatic driving ECU 50 performs a notable person response process corresponding to the notable person UM. The notable person response process is started by the automatic driving ECU 50 based on the acquisition of the call command by the information linking unit 61. Details of the notable person response process of the third embodiment will be described below based on FIG. 15, with reference to FIG. 14 and FIG. 9.

In S311 of the warning person response process, the environment recognition unit 62 determines whether the host vehicle Am is stopped or not. If the host vehicle Am is moving (S311: NO), the environment recognition unit 62 repeats the determination of S311 and waits for the host vehicle Am to stop. Then, if the host vehicle Am has stopped (S311: YES), the target recognition unit 73 performs outside vehicle monitoring in S312 and determines whether the warning person UM is approaching.

If the warning person UM is not approaching (S312: NO), the environment recognition unit 62 determines in S319 whether the host vehicle Am has moved to the vicinity of the owner OW, in other words, whether the summons have been completed. If the host vehicle Am has stopped near the owner OW and the summons have been completed (S319: YES), the warning person response process ends. On the other hand, if the summons have not been completed (S319: NO), the determinations of S311 and S312 are repeated.

If the watcher UM is approaching and is preventing the vehicle Am from moving to the owner OW's position (S312: YES), the notification request unit 72 notifies the user terminal 170 of the approach of the watcher UM in S313. In addition, the video distribution unit 66 transmits to the user terminal 170 in S314 the outside camera video captured by the camera unit of the perimeter monitoring sensor 30, which shows the watcher UM, to the user terminal 170.

The owner OW checks the exterior camera video delivered to the user terminal 170 and decides whether or not to allow the alert person UM to board. If the owner OW allows the alert person UM to board, he or she inputs a permission operation into the user terminal 170. In S315, the information linking unit 61 determines whether or not the owner OW has obtained permission to board. If the information linking unit 61 has obtained permission for the alert person UM to board (S315: YES), the security function unit 67 allows the alert person UM to board. As a result, in S316, the equipment control unit 65 works in cooperation with the body ECU 43 and the door lock actuator 45 to unlock the doors of the vehicle Am so that they can be opened from the outside.

On the other hand, if the information linking unit 61 does not obtain permission for the cautionary person UM to board (S315: NO), the security function unit 67 does not allow the cautionary person UM to board. In this case, the notification request unit 72 issues an exterior notification (no boarding warning) indicating that boarding is not permitted in S317 using the exterior display 24 and the exterior speaker 25. The exterior display 24 is provided, for example, on the front of the vehicle Am. In the exterior notification in S317, the spoken voice of the owner OW collected by the user terminal 170 may be played by the exterior speaker 25.

The target recognition unit 73 performs outside vehicle monitoring in S318 and determines whether the watcher UM has disappeared. If the watcher UM continues to impede the travel of the vehicle Am (S318: NO), the system waits for the owner OW to decide whether to permit the driver to board the vehicle. In this case, the outside vehicle notification may continue. On the other hand, if the watcher UM has disappeared (S318: YES), the system determines in S319 that the summons has been completed.

(Summary of the third embodiment)
In the third embodiment described above, the outside alarm to the to-be-watched person UM approaching the vehicle Am is also provided to prevent the to-be-watched person UM from getting into the vehicle Am. As a result, the same effects as those of the first and second embodiments are achieved, and it becomes possible to respond to the to-be-watched person UM approaching the vehicle Am even when the driver is not in the vehicle.

In addition, in the third embodiment, a command to summon the host vehicle Am is obtained from the user terminal 170 linked to the owner OW of the host vehicle Am. Then, if the travel of the host vehicle Am moving to the location of the owner OW is obstructed by a warning person UM, the notification request unit 72 notifies the user terminal 170. As a result of the above, even if the host vehicle Am is a privately owned vehicle, a response to a warning person UM approaching the host vehicle Am can be appropriately implemented when the owner OW is not on board.

In addition, in the third embodiment, when permission to board the vehicle is obtained for the warning person UM, the door of the vehicle Am is unlocked so that it can be opened from the outside. Therefore, even if a person related to the owner OW approaches the vehicle Am before the owner OW, the person can board the vehicle Am. As a result, the convenience of the vehicle Am can be improved. On the other hand, when permission to board the vehicle is not obtained for the warning person UM, an external alarm is issued to indicate that boarding is not possible. As a result, it is possible to encourage the warning person UM to take appropriate action. Note that in the third embodiment, the information linking unit 61 corresponds to the "command acquisition unit."

Other Embodiments
Although several embodiments of the present disclosure have been described above, the present disclosure should not be construed as being limited to the above-described embodiments, and can be applied to various embodiments and combinations within the scope not departing from the gist of the present disclosure.

The autonomous driving ECU 50 according to the first modification of the above embodiment is mounted on a POV that is assumed to be privately owned, rather than on a vehicle that provides mobility services, as in the third embodiment. The autonomous driving ECU 50 is capable of autonomously driving the vehicle Am, which is the POV, at level 4 or higher autonomous driving. As in the first modification, the vehicle mounted with the autonomous driving ECU 50 is not limited to vehicles for mobility services, but may be a general private passenger car, a rental car vehicle, a manned taxi vehicle, a freight vehicle, a bus, etc. Furthermore, the remote driving function in cooperation with the remote system 130 may be omitted as appropriate.

In the autonomous driving ECU 50 of the first modified example, when the vehicle Am is stopped at a location other than a pre-registered location such as the home, and a pre-registered user is inside the vehicle, the target recognition unit 73 determines that any person approaching the vehicle Am is a person to watch out for UM. In addition, when a person without a FoB key approaches the vehicle Am, the target recognition unit 73 may determine that a person to watch out for UM has approached.

In the second variation of the above embodiment, the no-boarding warning indicating that boarding is not permitted is issued using only one of the exterior display 24 and the exterior speaker 25. Furthermore, the exterior display 24 may issue a no-boarding warning indicating that boarding is not permitted, for example, by flashing a warning color instead of displaying a text message.

In the third variation of the above embodiment, the in-vehicle display 21 has a function as a touch panel. At least some of the emergency stop button 117, emergency call button 118, and emergency evacuation button 119 are made into touch icons displayed on the in-vehicle display 21 instead of physical switches. The autonomous driving ECU 50 and HMI control device 10 cause each of the buttons 117 to 119 to be displayed on the in-vehicle display 21 at the timing when operation is required.

The content of the evacuation control in response to a warning person in the first embodiment may be changed as appropriate. As an example, even in normal evacuation control, the behavior determination unit 63 may continue moving the host vehicle Am until a stop command is obtained from at least one of the passenger PS and the remote monitor RO. Also, in emergency evacuation control, if there is a passenger PS who is capable of driving in the driver's seat DS, driving from inside the vehicle may be permitted. Furthermore, if at least an outside vehicle alarm is implemented in response to a warning person, at least one of the unlocking control, the center notification, and the evacuation control may be omitted as appropriate. Also, the in-vehicle monitoring function and the theft alarm function may be omitted as appropriate.

The vehicle equipped with the autonomous driving ECU 50 may be a right-hand drive vehicle or a left-hand drive vehicle. Furthermore, the traffic environment in which the vehicle travels may be a traffic environment based on left-hand traffic or a traffic environment based on right-hand traffic. The positions of the doors provided on the driverless vehicle ADV, as well as the positions of the exterior display 24 and the exterior speaker 25, may be optimized as appropriate according to the road traffic laws of each country and region in which the driverless vehicle ADV is operated, as well as the position of the vehicle's steering wheel, etc.

The form of the in-vehicle operation unit 110 provided in the vehicle Am is not limited to an accelerator pedal, a brake pedal, a steering wheel, etc. For example, a controller (game pad) including a lever-shaped controller, a joystick, a cross key, etc. may be provided as the in-vehicle operation unit 110.

In the above embodiment, each process related to monitoring inside and outside the vehicle (see Figures 5 to 8, 10 to 13, and 15) was performed by the autonomous driving ECU 50 mounted on the driverless vehicle ADV. On the other hand, in Variation 4 of the above embodiment, some or all of the processes are performed by the remote control device 140. As described above, at least some of the functions of the autonomous driving ECU 50 may be implemented in the remote control device 140. In this Variation 4, the remote control device 140, together with the autonomous driving ECU 50 or alone, corresponds to the "autonomous driving control device."

Furthermore, the functions of the vehicle monitoring device may be implemented in an on-board ECU other than the autonomous driving ECU 50, such as the HMI control device 10. Furthermore, the functions of the vehicle monitoring device may be implemented in a distributed state in multiple on-board ECUs mounted on the host vehicle Am, such as a central ECU and a zone ECU. Also, a single on-board ECU having the functions of the vehicle monitoring device may be mounted on the driverless vehicle ADV. In these modified examples, the on-board ECU, either alone or in multiple units, corresponds to the "vehicle monitoring device".

In the above embodiments, each function provided by an autonomous driving ECU or the like can be provided by software and hardware that executes it, by software alone, by hardware alone, or by a combination of these. Furthermore, when such functions are provided by electronic circuits as hardware, each function can also be provided by digital circuits including a large number of logic circuits, or by analog circuits. Furthermore, the software for realizing such functions may include at least a portion of code that is automatically generated by, for example, a neural network or language model trained using camera footage of the real world.

Each processing unit in the above-described embodiments includes at least one computational core such as a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit). The processing unit may further include an FPGA (Field-Programmable Gate Array), an NPU (Neural network Processing Unit), and IP cores with other dedicated functions. The processing unit may be individually implemented on a printed circuit board, or may be implemented in an ASIC (Application Specific Integrated Circuit), a SoC (System on Chip), an FPGA, etc.

The form of the storage medium (non-transitory tangible storage medium) that stores various programs, etc. may be changed as appropriate. Furthermore, the storage medium is not limited to a configuration in which it is provided on a circuit board, but may be provided in the form of a memory card, etc., inserted into a slot portion, and electrically connected to the control circuit of the autonomous driving ECU or remote control device. The storage medium may also be an optical disk, hard disk drive, solid state drive, etc., from which the program is copied or distributed to the autonomous driving ECU or remote control device.

The control unit and the method described in the present disclosure may be realized by a dedicated computer comprising a processor programmed to execute one or more functions embodied in a computer program. Alternatively, the device and the method described in the present disclosure may be realized by a dedicated hardware logic circuit. Alternatively, the device and the method described in the present disclosure may be realized by one or more dedicated computers configured by a combination of a processor that executes a computer program and one or more hardware logic circuits. Furthermore, the computer program may be stored on a computer-readable non-transient tangible recording medium as instructions executed by the computer.

(Disclosure of technical ideas)
This specification discloses multiple technical ideas described in the following multiple dependent claims. Some of the claims may be described in a multiple dependent form, in which the subsequent claim alternatively refers to the preceding claim. Furthermore, some of the claims may be described in a multiple dependent form, in which the subsequent claim alternatively refers to the preceding claim. The claims described in these multiple dependent forms define multiple technical ideas.

(Technical Concept 1)
A vehicle monitoring device that monitors the status of a vehicle (Am) that can be driven by an automatic driving function,
An approach determination unit (73) that determines an approach of a person to be warned (UM) who is not scheduled to board the vehicle to the vehicle;
a notification implementation unit (72) that, when it is determined that the warning person is approaching the vehicle, issues an external notification indicating that boarding is not permitted, using an external alarm device (24, 25) provided in the vehicle;
A vehicle monitoring device comprising:
(Technical Concept 2)
The vehicle monitoring device described in technical idea 1 further includes an action control unit (63) that, when the person approaching the vehicle while it is stopped, determines whether the vehicle can move from its current position while it is stopped, and if it determines that the vehicle can move, moves the vehicle from its current position.
(Technical Concept 3)
The vehicle monitoring device according to Technical Idea 2, wherein when an emergency evacuation button (119) provided in the vehicle cabin is operated by a passenger (PS), the behavior control unit moves the vehicle to a location farther than the current location in a case where the emergency evacuation button is not operated.
(Technical Concept 4)
The approach determination unit determines a possibility of contact between the warning person and the host vehicle,
The vehicle monitoring device according to Technical Idea 2 or 3, wherein the behavior control unit suppresses the moving speed when there is a possibility of contact with the person to be warned, more than when there is no possibility of contact.
(Technical Concept 5)
A vehicle monitoring device as described in any one of technical ideas 2 to 4, further comprising a locking control unit (65) that locks the doors of the vehicle so that they cannot be opened from the outside when the behavior control unit determines that the vehicle is not movable.
(Technical Concept 6)
An application acquisition unit (70) for acquiring an application for boarding the vehicle,
A vehicle monitoring device as described in any one of technical ideas 2 to 5, wherein the behavior control unit stops movement of the vehicle when a boarding application for the person to be warned is acquired by the application acquisition unit after starting movement from the current location.
(Technical Concept 7)
The vehicle monitoring device described in any one of technical ideas 2 to 6, wherein the behavior control unit continues moving the vehicle until it receives a stop command from at least one of a passenger (PS) in the vehicle cabin and a monitor (RO) remotely monitoring the vehicle.
(Technical Concept 8)
The notification execution unit,
When the state in which it has been determined that the person in question is approaching continues for a predetermined time or more, an in-vehicle alarm (21, 22) provided in the vehicle cabin is used to inquire whether or not to report the approach of the person in question to a predetermined notification destination (CNT);
A vehicle monitoring device as described in any one of technical ideas 1 to 7, which executes a report to the report destination when the execution of the report is approved by a passenger (PS) in the vehicle cabin.
(Technical Concept 9)
The notification execution unit,
When it is determined that the person to be warned approaches the vehicle at a location different from a preset boarding/alighting point, an in-vehicle alarm (21, 22) provided in the vehicle cabin is used to inquire whether or not to perform personal authentication of the person to be warned;
A vehicle monitoring device as described in any one of technical ideas 1 to 8, which performs the personal authentication of the warning person when the performance of the personal authentication is permitted by a passenger (PS) in the vehicle cabin.
(Technical Concept 10)
The vehicle monitoring device described in any one of technical ideas 1 to 9 further includes a video distribution unit (66) that distributes in-vehicle camera video taken by an interior camera (29) installed in the vehicle cabin to a user terminal (170) linked to an approver whose reservation for boarding the vehicle has been accepted, based on a request from the user terminal.
(Technical Concept 11)
The video distribution unit of the vehicle monitoring device described in technical idea 10 allows the distribution of the in-vehicle camera video to the user terminal when the vehicle approaches within a predetermined distance of the approved person's intended boarding point.
(Technical Concept 12)
The vehicle monitoring device according to technical idea 10 or 11, wherein the video distribution unit permits the distribution of the in-vehicle camera video to the user terminal when a predetermined time has passed until the scheduled boarding time of the approved person.
(Technical Concept 13)
The vehicle monitoring device according to any one of technical ideas 10 to 12, wherein the video distribution unit continues to permit distribution of the in-vehicle camera video to the user terminal until a predetermined time has elapsed after the approved person has disembarked from the vehicle.
(Technical Concept 14)
The vehicle further includes an in-vehicle monitoring unit (71) that detects movement of a passenger (PS) to a driver's seat (DS) that is closest to a driving operation unit (110) provided in the vehicle itself,
A vehicle monitoring device as described in any one of technical ideas 1 to 13, in which the notification implementation unit, when it is determined that the occupant is moving to the driver's seat, uses an in-vehicle alarm (21, 22) provided in the vehicle cabin to implement an in-vehicle warning to encourage the occupant to leave the driver's seat.
(Technical Concept 15)
The vehicle monitoring device according to technical idea 14, wherein the notification implementation unit, after implementing the in-vehicle warning, notifies a monitor (RO) remotely monitoring the vehicle that the occupant has moved to the driver's seat.
(Technical Concept 16)
A vehicle interior monitoring unit (71) that detects that a passenger (PS) has touched a drive key located in a vehicle interior of the vehicle,
A vehicle monitoring device as described in any one of technical ideas 1 to 15, in which, when it is determined that the occupant has touched the drive key, the alarm implementation unit uses an in-vehicle alarm (21, 22) provided in the vehicle cabin to implement an in-vehicle warning to warn the occupant not to touch the drive key.
(Technical Concept 17)
A vehicle monitoring device as described in any one of technical ideas 1 to 16, further comprising a locking control unit (65) that, when it is determined that the warning person is approaching the vehicle, unlocks, of the doors on both the left and right sides of the vehicle, the door opposite the side from which the warning person is approaching so that it can be opened from inside the vehicle cabin.
(Technical Concept 18)
The vehicle monitoring device according to technical idea 17, wherein the notification implementation unit uses an interior alarm (21, 22) provided in the vehicle cabin to implement an interior notification indicating that the door on the opposite side is unlocked after the door on the opposite side is unlocked.
(Technical Concept 19)
A location storage unit (78) that stores a location where the vehicle is stopped based on the function-off process of the vehicle,
The vehicle monitoring device described in any one of technical ideas 1 to 18, wherein the notification implementation unit implements an outside vehicle warning indicating a theft when the start location of the vehicle, which is determined based on the function on processing of the vehicle, is different from the stopped location.
(Technical Concept 20)
The notification implementation unit is a vehicle monitoring device described in any one of technical ideas 1 to 19, which notifies a predetermined reporting destination (CNT) of the location information of a user terminal (170) linked to a passenger (PS) in the vehicle cabin when the vehicle is stolen.
(Technical Concept 21)
A command acquisition unit (61) that acquires a command to call the vehicle from a user terminal (170) associated with an owner (OW) of the vehicle,
The vehicle monitoring device described in any one of technical ideas 1 to 20, wherein the notification implementation unit notifies the user terminal when the vehicle's movement toward the owner's location is obstructed by the cautionary person.
(Technical Concept 22)
a lock control unit (65) that unlocks a door of the vehicle so that the door can be opened from the outside when the command acquisition unit acquires a boarding permission for the warning person,
The vehicle monitoring device according to technical idea 21, wherein the notification implementation unit implements the vehicle exterior notification indicating that boarding is not permitted using the vehicle exterior alarm when the boarding permission is not acquired by the command acquisition unit.
(Technical Concept 23)
A vehicle monitoring program for monitoring a status of a vehicle (Am) that can be driven by an automatic driving function,
A warning person (UM) who is not scheduled to board the vehicle is determined to be approaching the vehicle (S13, S213, S312).
When it is determined that the warning person is approaching the vehicle, an exterior alarm (24, 25) provided in the vehicle is used to issue an exterior alarm indicating that boarding is not permitted (S14, S219, S317).
A vehicle monitoring program that causes at least one processing unit (51) to execute a process including the steps of:

Claims (23)

A vehicle monitoring device that monitors the status of a vehicle (Am) that can be driven by an automatic driving function,
An approach determination unit (73) that determines an approach of a person to be warned (UM) who is not scheduled to board the vehicle to the vehicle;
a notification implementation unit (72) that, when it is determined that the warning person is approaching the vehicle, issues an external notification indicating that boarding is not permitted, using an external alarm device (24, 25) provided in the vehicle;
A vehicle monitoring device comprising: The vehicle monitoring device according to claim 1 further comprises a behavior control unit (63) that, when the person approaching the vehicle approaches the vehicle while the vehicle is stopped, determines whether the vehicle can move from its current position while the vehicle is stopped, and, when it is determined that the vehicle can move, moves the vehicle from its current position. The vehicle monitoring device according to claim 2, wherein when an emergency evacuation button (119) provided in the vehicle cabin is operated by a passenger (PS), the behavior control unit moves the vehicle to a location farther than the current location when the emergency evacuation button is not operated. The approach determination unit determines a possibility of contact between the warning person and the host vehicle,
The vehicle monitor device according to claim 2 , wherein the behavior control unit is configured to suppress a moving speed when there is a possibility of contact with the person to be warned, compared to a case where there is no possibility of contact. The vehicle monitoring device according to claim 2, further comprising a lock control unit (65) that locks the doors of the host vehicle so that they cannot be opened from the outside when the behavior control unit determines that the host vehicle is not movable. An application acquisition unit (70) for acquiring an application for boarding the vehicle,
The vehicle monitoring device according to claim 2 , wherein the behavior control unit stops the movement of the vehicle when, after starting movement from the current position, the boarding application for the alert person is acquired by the application acquisition unit. The vehicle monitoring device according to claim 2, wherein the behavior control unit continues moving the vehicle until it receives a stop command from at least one of a passenger (PS) in the vehicle cabin and a monitor (RO) remotely monitoring the vehicle. The notification execution unit,
When the state in which it has been determined that the person in question is approaching continues for a predetermined time or more, an in-vehicle alarm (21, 22) provided in the vehicle cabin is used to inquire whether or not to report the approach of the person in question to a predetermined notification destination (CNT);
The vehicle monitoring device according to claim 1 , wherein when the execution of the report is approved by a passenger (PS) in the vehicle cabin, the vehicle monitor executes the report to the report destination. The notification execution unit,
When it is determined that the person to be warned approaches the vehicle at a location different from a preset boarding/alighting point, an in-vehicle alarm (21, 22) provided in the vehicle cabin is used to inquire whether or not to perform personal authentication of the person to be warned;
The vehicle monitoring device according to claim 1 , wherein the personal authentication of the warning person is performed when the execution of the personal authentication is permitted by a passenger (PS) in the vehicle compartment. The vehicle monitoring device according to claim 1 further comprises a video distribution unit (66) that distributes in-vehicle camera video captured by an in-vehicle camera (29) installed in the vehicle cabin to a user terminal (170) linked to an approver whose reservation for boarding the vehicle has been accepted, based on a request from the user terminal. The vehicle monitoring device according to claim 10, wherein the video distribution unit permits the distribution of the in-vehicle camera video to the user terminal when the vehicle approaches within a predetermined distance from the intended boarding point of the approver. The vehicle monitoring device according to claim 10, wherein the video distribution unit permits the distribution of the in-vehicle camera video to the user terminal when the approved person's scheduled boarding time is within a predetermined time. The vehicle monitoring device according to claim 10, wherein the video distribution unit continues to permit distribution of the in-vehicle camera video to the user terminal until a predetermined time has elapsed after the approved person has exited the vehicle. The vehicle further includes an in-vehicle monitoring unit (71) that detects movement of a passenger (PS) to a driver's seat (DS) that is closest to a driving operation unit (110) provided in the vehicle itself,
The vehicle monitoring device according to claim 1, wherein when it is determined that the occupant has moved to the driver's seat, the notification implementation unit uses an interior alarm (21, 22) provided in the vehicle cabin to implement an interior warning to encourage the occupant to leave the driver's seat. The vehicle monitoring device according to claim 14, wherein the notification unit notifies a remote monitor (RO) of the movement of the passenger to the driver's seat after issuing the in-vehicle warning. A vehicle interior monitoring unit (71) that detects that a passenger (PS) has touched a drive key located in a vehicle interior of the vehicle,
The vehicle monitoring device according to claim 1, wherein when it is determined that the occupant has touched the drive key, the notification implementation unit implements an in-vehicle warning using an in-vehicle alarm (21, 22) provided in the vehicle cabin to warn the occupant not to touch the drive key. The vehicle monitoring device according to claim 1 further comprises a lock control unit (65) that unlocks, when it is determined that the warning person is approaching the vehicle, the door on the left and right sides of the vehicle that is opposite the door from which the warning person is approaching so that the door can be opened from inside the vehicle. The vehicle monitoring device according to claim 17, wherein the notification unit, after unlocking the opposite door, uses an interior alarm (21, 22) provided in the vehicle interior to notify the driver that the opposite door is unlocked. A location storage unit (78) that stores a location where the vehicle is stopped based on the function-off process of the vehicle,
The vehicle monitoring device according to claim 1 , wherein the notification unit issues an outside vehicle warning indicating that a theft has occurred when a start location of the vehicle identified based on a function-on process of the vehicle is different from the stop location. The vehicle monitoring device according to claim 1, wherein the notification unit notifies a predetermined notification destination (CNT) of location information of a user terminal (170) linked to a passenger (PS) in the vehicle cabin when the vehicle is stolen. A command acquisition unit (61) that acquires a command to call the vehicle from a user terminal (170) associated with an owner (OW) of the vehicle,
The vehicle monitor device according to claim 1 , wherein the notification unit notifies the user terminal when the vehicle moving to the location of the owner is blocked by the cautioner. a lock control unit (65) that unlocks a door of the vehicle so that the door can be opened from the outside when the command acquisition unit acquires a boarding permission for the warning person,
The vehicle monitor device according to claim 21 , wherein the notification implementation unit implements the vehicle exterior notification indicating that boarding is not permitted using the vehicle exterior alarm when the boarding permission is not acquired by the command acquisition unit. A vehicle monitoring method for monitoring a status of a vehicle (Am) capable of traveling by an automatic driving function, comprising:
A check is made to see if a person to be watched (UM) who is not scheduled to board the vehicle approaches the vehicle (S13).
When it is determined that the warning person is approaching the vehicle, an exterior alarm (24, 25) provided in the vehicle is used to issue an exterior alarm indicating that boarding is not permitted (S14).
The vehicle monitoring method includes the steps of:

Images