JP7536047B2 - Multi-factor authentication and access control in vehicle environments - Google Patents

Description

The present invention relates to multi-factor authentication and access control in a vehicle environment.

Unsecured or inadequately secured vehicles are susceptible to unauthorized access or use. Key or hardware entry systems may not prevent some forms of unauthorized access or use of the vehicle.

According to one aspect of the disclosure, a system for multi-factor entry authentication of a vehicle entry event is provided. The system may include a data processing system having a digital assistant application including an entry detection component, an entry authentication component, and an access control component. The digital assistant application may be executed within the vehicle such that the entry detection component receives at least one sensor signal acquired by a first sensor installed in the vehicle. The entry detection component may determine an entry event into the vehicle based on the at least one sensor signal. The entry event may represent a user entering the vehicle. The entry authentication component may receive a first authentication input signal and a second authentication input signal from at least a second sensor associated with the vehicle in response to the entry event into the vehicle. The entry authentication component may determine a first authentication state based on the first authentication input signal and a first authentication information of the plurality of authentication information. The entry authentication component may determine a second authentication state based on the second authentication input signal and a second authentication information of the plurality of authentication information. The access control component can identify one of a plurality of permission levels based at least in part on the first authentication state and the second authentication state. The access control component can identify a subset of the set of features available via the vehicle in response to the permission level. The digital assistant application can provide vehicular access to the subset of features and an audio output signal related to at least the subset of features.

According to one aspect of the disclosure, a method of multi-factor entry authentication of a vehicle entry event is provided. The method can include receiving, by a data processing system including a digital assistant, at least one sensor signal acquired by a first sensor installed in the vehicle. The method can include the data processing system determining an entry event into the vehicle based on the at least one sensor signal. The entry event can represent a user entering the vehicle. The method can include the data processing system receiving, in response to the entry event into the vehicle, a first authentication input signal and a second authentication input signal. The first authentication input signal and the second authentication input signal can be acquired by at least a second sensor associated with the vehicle. The method can include the data processing system determining a first authentication state based on the first authentication input signal and a first authentication information of the plurality of authentication information. The method can include the data processing system determining a second authentication state based on the second authentication input signal and a second authentication information of the plurality of authentication information. The method can include the data processing system identifying one of a plurality of access permission levels based at least in part on a first authentication state and a second authentication state. The method can include the data processing system identifying a subset of a set of functions available via the vehicle in response to the access permission level. The method can include the data processing system providing the vehicle access to the subset of functions.

These and other aspects and implementations are discussed in detail below. The preceding information and the following detailed description, including illustrative examples of the various aspects and implementations, are intended to provide an overview or framework for understanding the nature and characteristics of the claimed aspects and implementations. The drawings provide illustrations and a further understanding of the various aspects and implementations, and are incorporated in and constitute a part of this specification.

The accompanying drawings are not intended to be drawn to scale. Like reference numbers and designations in the various drawings indicate like elements. For clarity, not every component in every drawing may be labeled.

FIG. 1 is a diagram of an example system for authenticating users based on multiple access permission levels to control user access to features available via a vehicle. 2 is a top view of the vehicle shown in FIG. 1 showing sensors and electronic devices associated with the vehicle. FIG. 2 illustrates an example method for multi-factor entry authentication for a vehicle entry event using the example system shown in FIG. FIG. 1 is a block diagram of an example computer system.

Below is a more detailed description of various concepts and implementations related to methods, apparatus, and systems for multi-factor user authentication and control of user access to functionality associated with a vehicle environment. The various concepts introduced above and discussed in more detail below may be implemented in any of a number of ways.

The present disclosure is generally directed to methods and systems for entry authentication of vehicle entry events and access control in vehicle environments. Although in-vehicle features based on electronic and information technology (IT) systems can improve user experience, such in-vehicle features may also introduce other risks and technical obstacles. Availability of in-vehicle online connectivity or in-car entertainment (ICE) systems (also referred to as in-vehicle infotainment (IVI) systems) can frequently distract drivers and increase the risk of accidents. Users attempting to initiate or respond to communications (e.g., emails, messages, or calls) while driving, retrieving or accessing multimedia files for rendering on the IVI system, or initiating online actions (e.g., online shopping or online booking) can result in serious and dangerous accidents.

Moreover, given that a vehicle's connectivity to the Internet occurs primarily through wireless client devices, an increasing range of in-vehicle IT functions, including, for example, access to online data, systems or applications, can strain communication resources, such as wireless network bandwidth. Moreover, a vehicle's reliance on mobile devices for connectivity can result in degraded connectivity when the vehicle is traveling at relatively high speeds or as the vehicle moves away from a wireless network coverage area. Degraded connectivity can result in frequent interruptions to the services or data accessed by the vehicle.

In addition, because vehicles (e.g., rental cars) may be shared, for example, between family members, friends, or strangers, the availability of in-vehicle features to access data or security-sensitive data or systems may jeopardize the user's privacy and data or security. For example, the ability to remotely open a residential garage door via the vehicle may pose a serious threat or danger (e.g., crime) to the user if such features are accessible to any driver or passengers of the vehicle. Some possible vehicle features (such as automatic and remote vehicle door locking/unlocking or automatic and remote control of vehicle windows) may be of major concern if not denied, for example, to vehicle passengers. Unrestricted access to data via the vehicle also increases the risk of public exposure of that data.

In the present disclosure, a data processing system including a digital assistant application may enable hands-free control of various functions associated with a vehicle. The digital assistant application may include a conversational digital assistant that provides two-way voice-based interaction with a user in a conversational format. For example, a user may speak to the digital assistant application, which may (1) accurately interpret voice signal input from the user, (2) respond by playing a voice output signal for the user, and (3) perform tasks or actions or combinations thereof identified from the voice input signal online. For example, a driver or passenger may audibly request to render a multimedia file, open a vehicle window, lock or unlock a vehicle door, adjust the vehicle interior air conditioning, unlock or open a residential garage door, or other functions available via the vehicle. Based on the audible request, the digital assistant application may recognize and execute the requested function. By enabling hands-free user interaction, the digital assistant application may provide various in-vehicle functions without significantly increasing the potential for driver distraction.

To avoid service interruptions due to possible connectivity degradation, the digital assistant application (and data processing system) can be executed in the vehicle. Executing the digital assistant application locally in the vehicle (e.g., within one or more electronic devices of the vehicle) makes the digital assistant application available to the user in the car regardless of the vehicle's connectivity state (e.g., to the Internet or mobile devices). The digital assistant application can also store user-related data, such as playlists, multimedia files, user security credentials (e.g., login usernames, passwords, or passcodes for various services), user access permission levels for various services, or combinations thereof, and can enable access to such data locally regardless of the vehicle's connectivity state. The digital assistant application (and data processing system) can be executed in the vehicle, on a mobile device, or on a remote computer system (e.g., the cloud), or combinations thereof. For example, various instances of the digital assistant application may run on separate platforms.

Storing and accessing data locally (in a vehicle) requires reliable user authentication (or vehicle entry authentication) and reliable data security measures. The data processing system (and corresponding digital assistant application) of the present disclosure can employ multi-factor user authentication. As used herein, multi-factor authentication refers to an authentication method involving two or more authentication techniques based on two or more authentication data inputs (or authentication input signals). The data processing system can also maintain separate data accounts for individual users. The data processing system can encrypt the data accounts (or corresponding security-sensitive data) to enhance data security. The data processing system (or corresponding digital assistant application) can use multi-factor authentication and predefined access permission levels to ensure that users do not access functions or data beyond their assigned authorized access levels.

The data processing system (or corresponding digital assistant application) may use authentication techniques with user authentication that do not distract the user (e.g., do not involve manual or significant physical involvement of the user). For example, fingerprint-based (or other biometric-based) authentication may involve significant physical and mental effort by the user, which is likely to distract the user, especially the driver. Also, other authentication techniques, such as login usernames and passwords, may be inappropriate for voice-input-based authentication. Such authentication information may lose confidentiality when given by voice and may be leaked to other users in the vehicle during the authentication process. The data processing system (or corresponding digital assistant application) may use authentication techniques that do not involve significant physical action or effort by the user, such as key fob authentication, authentication based on automatic collection of mobile device identifiers, facial recognition, speaker recognition, or a combination thereof.

The data processing system that may be included in the systems and methods described herein includes a digital assistant application that receives at least one sensor signal acquired by a first sensor installed in the vehicle. The data processing system may determine an entry event into the vehicle based on the at least one sensor signal. The data processing system may receive a plurality of authentication input signals from a plurality of sensors associated with the vehicle in response to the entry event into the vehicle. The data processing system may determine a plurality of authentication states based on the plurality of authentication input signals and the plurality of authentication information. The data processing system may identify one of a plurality of access permission levels based at least in part on the plurality of identified authentication states. The data processing system may identify a subset of a set of features available via the vehicle in response to the access permission level and provide the vehicle access to the subset of features.

With regard to circumstances in which the systems discussed herein may collect or utilize personal information about a user, the user may be given the opportunity to control programs or features that may collect personal information (e.g., information about the user's social network, social activities, or social enterprises, the user's preferences, or the user's location) or to control whether or how to receive content from a content server or other data processing system that may be more appropriate for the user. In addition, certain data may be anonymized in one or more ways before being stored or used, such that personally identifiable information is removed when generating parameters. For example, the user's identity may be anonymized such that personally identifiable information about the user cannot be determined, or the user's geographic location may be generalized (such as to a city, zip code, or country level) if location information is obtained, such that the user's specific location cannot be determined. In this manner, the user may control the manner in which their information is collected and utilized by the content server.

1 illustrates an example system 100 for authenticating users based on multiple access permission levels and controlling user access to features available via a vehicle. The system 100 can include at least one remote data processing system 102, one or more vehicles 130 (e.g., manual or automated cars, trucks, motorcycles, aircraft, ships, or other transportation equipment) each associated with a respective vehicle data processing system 140, and one or more mobile client devices 150 (e.g., smartphones, tablets, or smart watches). The system 100 can include a network 148 that can communicatively couple the remote data processing system 102 to one or more vehicles 130, one or more mobile client devices 150, or a combination thereof. The system 100 can include one or more data providers 152 (or their communication devices) communicatively coupled through the network 148 to the remote data processing system 102, one or more vehicles 130, one or more mobile client devices 150, or a combination thereof.

The remote data processing system 102 may include a cloud data processing system, an enterprise data processing system, a residential data processing system, or a combination thereof. The remote data processing system 102 may include at least one server having at least one processor. For example, the remote data processing system 102 may include multiple servers located in at least one data center or server farm. The remote data processing system 102 may include multiple servers logically grouped to facilitate distributed computing techniques. A logical group of servers may be referred to as a data center, a server farm, or a machine farm. The servers may be geographically distributed. A data center or machine farm may be managed as a single entity, or a machine farm may include multiple machine farms. The servers within each machine farm may be heterogeneous, with one or more of the servers or machines operating with one or more types of operating system platforms. The remote data processing system 102 may include servers in a data center, stored in one or more high density rack systems along with associated storage systems located, for example, in an enterprise data center. Each of the components of the remote data processing system 102 may include at least one processing unit, server, virtual server, circuit, engine, agent, apparatus, or other logic device such as a programmable logic array configured to communicate with the data repository 118 and other computing devices.

The remote data processing system 102 may include an interface 104. The data processing system 102 may include a natural language processor (NLP) component 106 for parsing the voice-based input. The data processing system 102 may include an interface management component 108 for detecting and managing interfaces of other devices in the system 100. The remote data processing system 102 may include a voice signal generator component 110 for generating a voice-based signal. The remote data processing system 102 may include a direct-action application programming interface (API) 112. The remote data processing system 102 may include a response selector component 114 for selecting a response to the voice-based input signal. The remote data processing system 102 may include a validation engine (or validation component) 116 for validating the voice-based input received by the remote data processing system 102. The remote data processing system 102 may include a data repository 118, which may store parameters 120, policies 122, response data 124, and templates 126. The remote data processing system 102 may include a digital assistant application (also referred to herein as a digital assistant system) 128 for handling and managing the user's interactions with the vehicle 130 or mobile client device 150, including performing tasks or actions requested by the user.

The interface 104 may enable communication between components of the remote data processing system 102, between the remote data processing system 102 and devices or systems of the system 100, or a combination thereof. The interface 104 may be a data interface or a network interface that enables the components of the system 100 to communicate with each other. The interface 104 of the data processing system 102 may provide or transmit one or more data packets including action data structures, audio signals, or other data to the mobile client device 150 or the vehicle 130 over the network 148. For example, the data processing system 102 may provide an output signal from the data repository 118 or the audio signal generator 110 to the mobile client device 150. The data processing system 102 may also instruct the mobile client device 150 or the vehicle 130 by data packet transmission to perform a function indicated in the action data data structure. The output signal may be captured, generated, converted into one or more data packets, or transmitted as one or more data packets (or other communication protocols) from the data processing system 102 (or other computing device) to the mobile client device 150 or the vehicle 130.

The NLP component 106 can receive an input voice signal, for example, from the vehicle 130 or the mobile client device 150. The data processing system 102 can receive an input voice signal from the client device 128 or the vehicle 140. A user of the vehicle 130 or the mobile client device 150 can provide a voice input signal, which can be transmitted to the remote data processing system 102 for processing by the NLP component 106. The NLP component 106 can convert the received input voice signal into recognized text by comparing the input voice signal to a set of stored representative voice waveforms and selecting the closest match. The representative waveforms can be generated, for example, across a large set of voice signals from multiple individuals. Once the input voice signal is converted into recognized text, the NLP component 106 can match the text to words associated with actions or output voice signals, for example, by a learning phase.

The NLP component 106 may identify from the input audio signal at least one request or at least one trigger or hot keyword corresponding to a request from a user (e.g., a user of the vehicle 130 or a user of the mobile client device 150). The NLP component 106 (or other components) may convert the input audio signal into a corresponding text signal and identify the request based on the text signal. The request may indicate the intent or subject of the input audio signal. The trigger keyword may indicate the type of action that may be taken. For example, the NLP component 106 may parse the input audio signal to identify at least one request to open a vehicle window, skip to the next audio file in a music playlist, make a phone call, create and send a message (e.g., an email message or a Short Message Service (SMS) message), initiate Global Positioning System (GPS) navigation to a particular street address, access locally or remotely stored data, initiate a streaming session, initiate a client application, open or close a garage door, control home-related settings, or a combination thereof. A trigger keyword may include at least one word, phrase, root or part word, or derived word that indicates an action to be taken. For example, a trigger keyword may be "go" or "ok" or "hey."

The interface management component 108 can detect and manage interfaces of other devices in the system 100, such as interfaces of the vehicle 130 and the mobile client device 150. The response selector component 114 can generate a response to send to a user associated with an identified request. For example, when the NLP component 106 identifies a request or a trigger or hot keyword corresponding to the request, the response selector component 114 can access the data repository 118 to obtain information or content for generating a response to send to the user. For example, the response selector component 114 can obtain a phrase, a content item, or a combination thereof from response data 124 in the data repository 118.

The audio signal generator component 110 can generate or otherwise obtain an output audio signal for sending to the user as part of the response. For example, the audio signal generator component 110 can generate or select one or more audio signals associated with a phrase or content item selected or obtained, for example, by the response selector component 114. For example, when a request is fulfilled, the signal generator component 110 can generate an audio output signal corresponding to the phrase "the action is complete."

The NLP component 106 can identify a task or action (e.g., opening a vehicle window, skipping to the next audio file in a music playlist, making a phone call, creating and sending a message (e.g., an email message or a Short Message Service (SMS) message), initiating Global Positioning System (GPS) navigation to a particular street address, accessing locally or remotely stored data, initiating a streaming session, starting a client application, opening and closing a garage door, controlling home-related settings, or a combination thereof) to be performed by the remote data processing system 102, the vehicle 130, the mobile client device 150, or a combination thereof based on the received input audio signal. The instruction action API 112 of the remote data processing system 102 can generate one or more action data structures in response to the identification of the task or action. The one or more action data structures can include data or instructions for performing an action to fulfill the identified action or request. The action data structure can be a JSON-formatted data structure or an XML-formatted data structure.

The directed action API 112 can execute code or dialogue scripts that identify the parameters necessary to fulfill the request based on the identified action or task. The digital assistant application 128 can include one or more action data structures for a message to transmit to the vehicle 130, the client device 128, or the data provider 154. The digital assistant application 128 or the directed action API 112 can determine the destination of the message based on the request identified by the NLP component 106. For example, if the input voice signal includes "open a window," the NLP component 106 can identify the action word "open a window," and the digital assistant application 128 or the directed action API 112 can package the one or more action data structures in a response for transmission to the vehicle 130. The directed action API 112 can access the vehicle identifier (ID) or vehicle IP address from the response data 124 to determine the vehicle associated with the user that generated the request. Upon receiving the one or more action data structures, the vehicle 140 can process the action data structures to open a window in the vehicle. If the input audio signal corresponds to "stream the Lion Prince movie," the NLP component 106 can identify the action word "stream the movie, Lion Prince." The digital assistant application 128 or the instruction action API 112 can identify a data provider 154 that provides streaming of the requested movie and package one or more action data structures into a message to the data provider 154 or the vehicle (or mobile client device 150) to initiate a streaming session between the data provider 154 and the vehicle (or mobile client device 150).

The one or more action data structures may include information for accomplishing the request. For example, the one or more action data structures may include an XML-formatted or JSON-formatted data structure that includes attributes used to accomplish or otherwise fulfill the request. The attributes may include the location of the vehicle 130, the location of the mobile client device 150, the permission level (or authentication level) of a user associated with the vehicle 130 or the mobile client device 150, a vehicle identifier, an interface identifier, a vehicle state, or a request state. The request state may include one or more attributes that must be satisfied before the action is performed. For example, for a request of "Ok, change song," the request state may have an attribute {requester:[authenticated, passenger]} that indicates that the requester is an authorized user. The digital assistant application 128 or the validation engine 116 may decide to block transmission of the action data structure (or block execution of the corresponding action) if the permission level attribute represents an unauthorized user.

The directed action API 112 can retrieve templates 126 from the repository 118 to determine fields or attributes to include in the action data structure. The directed action API 112 can determine required parameters and package the information into the action data structure. The directed action API 112 can retrieve content from the repository 118 to obtain information about attributes of the data structure. The directed action API 112 can fill fields with data from an input audio signal. The directed action API 112 can fill fields with data from a data provider 152, a vehicle 130, a mobile client device 150, or a combination thereof. The directed action API 112 can prompt the user for additional information when filling fields. The templates 126 can be standardized for various types of actions, such as playing a media file through a vehicle head unit, responding to a message, performing a function inside the automobile, making a phone call, creating and sending a message, starting a streaming session, or a combination thereof. The remote data processing system 102 can transmit the action data structure to the vehicle data processing system 140 of the vehicle 130, and the vehicle data processing system 140 can add fields and attributes to the action data structure.

The validation engine 116 can verify or otherwise determine whether an action or function associated with an action data structure should be performed or satisfied. For example, the validation engine 116 can check the permission level (or authorization level) of the requesting user and whether the action or function is within the range of the user's permission level. The validation engine 116 can check the permission level via state attributes within the action data structure or data stored within the data repository 118. The validation engine 116 can allow the action or function to be performed if the validation is successful.

If the requester has the appropriate permission level, the verification engine 116 may also take into account other factors, such as local laws and regulations, when determining whether to allow the action to be performed. For example, in response to receiving an input audio signal representing "stream the Lion Prince movie" and determining that the movie is to be streamed to a display device in front of the driver, the verification engine 116 may refuse to accomplish the streaming action because it is inappropriate to do so or because it is unlawful under local laws or regulations to provide visual content that may be distracting to the vehicle operator. However, if the movie is to be streamed to a display device behind the front passenger seat (e.g., the driver cannot see the content being rendered on the display device), the verification engine 116 may determine that it is appropriate to stream the movie. Therefore, the verification engine 116 may allow the streaming action to be performed.

The digital assistant application 128 can handle and manage conversations with a user of the vehicle 130 or mobile client device 150, including performing tasks or actions requested by the user. The digital assistance application 128 can initiate, terminate, or coordinate actions associated with various components of the remote data processing system 102, such as the interface 104, the NLP component 106, the interface management component 108, the audio signal generator 110, the instruction action API 112, the response selector 114, the validation engine 116, the data repository 118, or combinations thereof. For example, the digital assistance application 128 can initiate (e.g., call) and terminate instances of components (or instances of combinations of these components) of the remote data processing system 102. The digital assistance application 128 can generate, monitor, and update state related to each user conversation. The interface 104, the NLP component 106, the interface management component 108, the audio signal generator 110, the instruction action API 112, the response selector 114, the validation engine 116, the data repository 118, or a combination thereof may be part of a digital assistant application 128. The remote data processing system 102 may start a separate instance of the digital assistance application 128 for each conversation, each vehicle 130, or each mobile client device 150.

The network 148 may include computer networks such as the Internet, local, wide area, metropolitan or other area networks, intranets, satellite networks, other computer networks such as voice or data cellular telephone communication networks, and combinations thereof. The network 148 may be used by the remote data processing system 102, the mobile client device 150, and the vehicle 130 to access information resources such as web pages, websites, domain names, uniform resource locators, or data providers 152. For example, the remote data processing system 102 may access, via the network 148, a data provider 152 that provides traffic data for a particular location, such as a location associated with the vehicle 130.

The network 148 may include, for example, a point-to-point network, a broadcast network, a wide area network, a local area network, a telecommunications network, a data communication network, a computer network, an ATM (asynchronous transfer mode) network, a SONET (synchronous optical network) network, an SDH (synchronous digital hierarchy) network, a wireless network, or a wired network, and combinations thereof. The network 148 may include a wireless link, such as an infrared channel or a satellite band. The topology of the network 148 may include a bus network topology, a star network topology, or a ring network topology. The network 148 may include a mobile telephone network using any protocol used for communication between mobile devices, including Advanced Mobile Phone Protocol (AMPS), Time Division Multiple Access (TDMA), Code Division Multiple Access (CDMA), Global System for Mobile Communications (GSM) system, General Packet Radio Service (GPRS), or Universal Mobile Telecommunications System (UMTS). Different types of data may be transmitted by different protocols, or the same type of data may be transmitted by different protocols.

The vehicle 130 may include one or more speakers 132, one or more display devices 134, a number of sensors 136, and a transducer 138. The vehicle 130 may include a vehicle data processing system 140. The vehicle data processing system 140 may include an entry detection component 142, an entry authentication component 144, an access control component 146, and a digital assistant application 128. The digital assistant application 128 of the vehicle data processing system 140 may be similar to (or an instance of) that of the remote data processing system 102. The vehicle data processing system 140 and each of the components may be implemented as hardware, firmware, software, or a combination thereof. The vehicle data processing system 140 and the remote data processing system 102 may have similar components. For example, the vehicle data processing system 140 or its digital assistant application 128 may include the interface 104, the NLP component 106, the interface management component 108, the audio signal generator 110, the instruction action API 112, the response selector 114, the validation engine 116, the data repository 118, a combination thereof, or a combination of instances thereof. The remote data processing system 102 may include the entry detection component 142, the entry authentication component 144, the access control component 146, or a combination of instances thereof.

One or more speakers can convert electrical signals into audible waves to render audio signals (e.g., from a radio, a multimedia device, the vehicle data processing system 140, the digital assistant application 128, or a combination thereof). The vehicle 130 can include one or more display devices 134, such as a head-up display, a display associated with a multimedia device (e.g., for rendering media content to a rear seat passenger), or a combination thereof. The transducer 136 can convert audio input (e.g., from a user or background noise within the vehicle) into a corresponding electrical signal. The vehicle data processing system 140 can process the electrical signal (or transmit the electrical signal to the remote data processing system 102 for processing), for example, to perform voice recognition, speaker recognition, or a combination thereof. The transducer 136 can convert the electrical signal into a corresponding audio, ultrasonic, or subsonic signal.

The vehicle 130 may include multiple sensors 136 that provide sensor signals used to detect entry or exit events into the vehicle 130, monitor vehicle state parameters (e.g., speed, automatic door locks, tire pressure levels, or a combination thereof), perform user authentication (or vehicle entry authentication), or a combination thereof. The various sensors 136 are discussed below with respect to FIG. 2.

Figure 2 is a top view of the vehicle 130 shown in Figure 1 and illustrates sensors and electronic devices associated with the vehicle 130. With reference to Figure 2, the interior cabin of the vehicle 130 can include a number of seats 200. Each seat 200 can include a corresponding seat sensor 202. The seat sensor 202 can sense a weight or pressure on the corresponding seat 200. The seat sensor can compare the sensed pressure or weight to a respective threshold value. The seat sensor 202 can provide a seat sensor signal representing whether the corresponding seat 200 is occupied based on a comparison of the sensed pressure or weight to a respective threshold value. The vehicle data processing system 140 can receive the seat sensor signals from the various seat sensors 202, for example, via the interface 104 of the vehicle data processing system 140.

The vehicle 130 may include one or more door sensors 204. Each door sensor 204 may provide the vehicle data processing system 140 with a door sensor signal indicative of whether a vehicle door is open or closed, whether a vehicle door is locked, or a combination thereof. The vehicle 130 may include a separate door sensor 204 for each door that allows the vehicle data processing system 140 to identify the state (e.g., open, closed, locked, or unlocked) of each door.

The vehicle 130 may include one or more motion sensors 206 that may be located at one or more locations within the vehicle 130. The motion sensors 206 may sense vehicle motion using, for example, infrared-based sensing technology, microwave-based sensing technology, ultrasonic-based sensing technology, vibration-based sensing technology, or a combination thereof. The one or more motion sensors 206 may provide a motion sensor signal to the vehicle data processing system 140 that is indicative of whether motion is sensed within the vehicle. The sensed motion may represent the vehicle being occupied by one or more users. The individual motion sensors 206 may be located or mounted in relation to (e.g., adjacent to or facing) a corresponding seat 200 of the vehicle 130. The vehicle 130 may include a pressure sensor (not shown in FIG. 2) for sensing a substantial (e.g., greater than a predetermined threshold) change in the interior air pressure of the vehicle 130. The vehicle's ignition system or a sensor thereof may provide a signal representing whether the ignition system is on or off to the vehicle data processing system 140 in response to the user turning the ignition system on or off.

The vehicle 130 may include a key fob receiver device 210 for receiving signals from a remote key fob transmitter device (not shown in FIG. 2). The key fob receiver device 210 and the key fob transmitter device form a remote keyless system (RKS). The remote key fob transmitter device may transmit a signal including an entry code for locking or unlocking the vehicle. The entry code may be encrypted. The key fob receiver device 210 may activate the locking or unlocking of the motor vehicle when the entry code in the signal matches a code stored in the key fob receiver device 210. The key fob receiver device 210 may be communicatively coupled to the vehicle data processing system 140. The key fob receiver device 210 may provide an indication (or a signal indicative) of a successful or unsuccessful event of locking or unlocking by the RKS to the vehicle data processing system 140.

The vehicle 130 may include one or more microphones 214 for capturing input audio signals and multiple speakers (or loudspeakers) 132 for providing audio output. The one or more microphones 214 may be located at one or more locations inside the vehicle 130. The vehicle data processing system 140 or the corresponding digital assistant application 128 may turn the microphones 214 on and off based on the occupancy status of the vehicle 130. For example, the vehicle data processing system 140 or the corresponding digital assistant application 128 may turn on the microphones 214 in response to occupancy (or detection of an entry event) of the vehicle 130 being detected. The microphones 214 or speakers may be automatically turned on in response to starting the vehicle. The vehicle data processing system 140 or the corresponding digital assistant application 128 may receive the input audio signals captured by the one or more microphones 214 and apply voice recognition or speaker recognition to the input audio signals (or corresponding electrical signals).

The vehicle 130 may include one or more cameras 216 for capturing digital images. Each camera 216 may be oriented toward a corresponding seat 200 of the vehicle 130 such that the face of an individual seated in the seat 200 is within the field of view of the camera 116. The vehicle data processing system 140 or the corresponding digital assistant application 128 may trigger the camera 216, for example in response to detecting an entry event, to capture one or more digital images. The vehicle data processing system 140 or the corresponding digital assistant application 128 may perform facial recognition using the captured images. The vehicle data processing system 140 or the corresponding digital assistant application 128 may trigger the camera 216 to capture one or more additional images if a previously captured image is insufficient for facial recognition (e.g., if the user's face is not clear or complete). The vehicle data processing system 140 or the corresponding digital assistant application 128 may adjust (e.g., rotate) the orientation of the camera 216. The vehicle data processing system 140 or corresponding digital assistant application 128 can provide an audio output signal through the speaker 132 requesting the user to aim or adjust the camera 216.

The vehicle 130 may include a communication transceiver 218, such as a short wavelength (e.g., 2.4-2.5 GHz) data communication transceiver, a near field communication (NFC) transceiver, or a combination thereof, for communicating with mobile client devices 150 inside the vehicle, such as mobile client devices 150a and 150b. The vehicle data processing system 140 or a corresponding digital assistant application 128 may be communicatively coupled to the communication transceiver 218. The communication transceiver 218 may communicate with the mobile client devices 150a and 150b (or generally with the mobile client devices 150 in the vehicle 130) to obtain device identifiers (IDs) of these devices. The communication transceiver 218 may provide the obtained client IDs to the vehicle data processing system 140 or the corresponding digital assistant application 128. If the communication transceiver 218 has not obtained a client ID after a predetermined period of time after an entry event or vehicle occupancy is detected, the vehicle data processing system 140 or corresponding digital assistant application 128 can instruct the user (e.g., by an audio output signal transmitted to the speaker 132) to, for example, turn on the transceiver of the respective mobile client device (e.g., mobile client device 150a or 150b) or bring the client device closer to the communication transceiver 218 (for NFC communication). The vehicle data processing system 140 or corresponding digital assistant application 128 can trigger the mobile client device 150a or 150b to transmit its respective ID by transmitting an ultrasonic or subsonic output signal through the speaker 132. The client device (e.g., mobile client device 150a or 150b) can interpret the ultrasonic or subsonic signal as a request for its respective ID and transmit its respective client ID in response to the ultrasonic or subsonic output signal.

The vehicle may include a rear seat entertainment system 212, including, for example, a multimedia device for playing multimedia content (e.g., audiovisual content, streaming content, gaming content, or a combination thereof). To avoid being visible and distracting to the driver, the rear seat entertainment system 212 may be located behind the front seat 200. The vehicle 130 may also include a head unit 220, which may execute computer code instructions related to the vehicle data processing system 140, a corresponding digital assistant application, or one or more of the components described in connection with the vehicle 130. The head unit 220 may include one or more processors for executing computer code instructions and one or more storage devices (not shown in FIG. 2) for storing computer code instructions and data, such as data related to the data repository 118, playlists, configuration data, or a combination thereof. The head unit 220 may include a display (not shown in FIG. 2) for rendering image data, such as a navigation map, playlist information, radio channel information, or a combination thereof, to a user. The head unit 220 may be communicatively coupled (e.g., by wired or wireless connection) to the sensors 136 (such as the seat sensor 202, the door sensor 204, the motion sensor 206, the air pressure sensor, the ignition system (or corresponding sensor), the key fob receiver device 210, the microphone 214, the camera 216, or the communication transceiver 218), the speaker 132, or the multimedia device of the rear entertainment system 212.

Returning to FIG. 1, the entry detection component 142, the entry authentication component 144, the access control component 146, or a combination thereof may be part of the digital assistant application 128 of the vehicle data processing system 140. The entry detection component 142 may receive at least one sensor signal acquired by one or more sensors 136 installed in the vehicle 130. For example, the entry detection component 142 may receive a key fob signal from the key fob receiver device 210, a door sensor signal from the door sensor 204, a seat sensor signal from the seat sensor 202, a motion sensor signal from the motion sensor 206, a pressure signal from an air pressure sensor, an ignition signal from the vehicle ignition system (e.g., representing an on state of the ignition system), or a combination thereof. At least one of the sensors 136 may transmit a corresponding sensor signal even when the vehicle ignition is off.

The entry detection component 142 may determine (or detect) an entry event into the vehicle 130 based on at least one sensor signal. The entry event may represent a user entering the vehicle 130. The entry detection component 142 may determine the entry event based on a sensor signal from a single type of sensor 136. For example, a sensor signal from one of the key fob receiver device 210, the door sensor 204, the seat sensor 202, the motion sensor 206, or the air pressure sensor may be sufficient for the entry detection component 142 to make a determination regarding the occurrence of an entry event. The entry detection component 142 may determine the entry event based on sensor signals from two or more types of sensors 136. For example, the entry detection component 142 may be configured to determine the occurrence of an entry event only when two or more different types of sensors (e.g., both the key fob receiver device 210 and the door sensor 204, or both the door sensor 204 and the motion sensor 206) provide corresponding sensor signals representative of vehicle occupancy by one or more individuals.

The digital assistant application 128 of the vehicle data processing system 140 can operate in a light mode prior to detection of any entry event. In the light mode, only the entry detection component 142 is awake, and other components of the digital assistant application 128 of the vehicle data processing system 140 can be in a sleep mode to conserve vehicle battery power. The entry detection component 142 can wake up other components of the digital assistant application 128 of the vehicle data processing system 140 in response to detection of an entry event into the vehicle 130.

The entry authentication component 144 can receive at least two authentication input signals in response to an entry event into the vehicle 130. The at least two authentication input signals can be acquired by at least one sensor (or electronic device) 136 associated with the vehicle 130. For example, the entry authentication component 144 can receive a first authentication input signal and a second authentication input signal acquired by at least one sensor 136 associated with the vehicle 130. The at least two authentication input signals can be of separate types acquired by at least two different types of sensors. For example, the entry authentication component 144 can receive at least two of: (1) a key fob authentication signal from the key fob receiver device 210; (2) a mobile device identifier (ID) authentication signal from a corresponding mobile client device (e.g., a smartphone, tablet, or smartwatch) 150; (3) an audio input signal from the microphone 214; or (4) a digital image from the camera 216.

The key fob authentication signal may indicate whether the key fob receiver device 210 has received a coded wireless signal corresponding to the vehicle 130 (e.g., a signal that successfully unlocked the vehicle door). The mobile device identifier (ID) authentication signal may be a signal provided by the mobile client device 150 and including a corresponding identifier ID of the mobile client device 150. As discussed above with respect to FIG. 2, the digital assistant application 128 of the vehicle data processing system 140 may cause the mobile client device 150 to transmit a corresponding mobile device identifier (ID) authentication signal, for example, by transmitting an ultrasonic or subsonic signal through the speaker 132 or by providing an output audio signal. The digital assistant application 128 may receive an audio input signal directly from the microphone 214 or from a storage device coupled to the microphone 214, and is configured to maintain the audio signal captured by the microphone 214 over a recent predefined period (e.g., the last 30 seconds, the last minute, the last two minutes). The storage device may timestamp the recorded audio signal. The storage device can separately store audio signals from the individual microphones. In response to detecting an entry event into the vehicle, an audio output signal can be provided through the speaker 132 to the digital assistant application 128 to prompt any possible user in the vehicle 130 to speak (e.g., by asking the user a specific question or simply asking him to speak). In response to detecting an entry event into the vehicle 130, the digital assistant application 128 of the vehicle data processing system 140 can trigger one or more cameras 216 to capture digital images.

The digital assistant application 128 of the vehicle data processing system 140 can generate (or update) an occupancy state of the vehicle 130 in response to an entry event into the vehicle 130. The occupancy state of the vehicle 130 can map each authenticated user to a corresponding seat 200 of the vehicle 130 (e.g., based on a sensor signal from a corresponding seat sensor or motion sensor associated with that seat 200). The digital assistant application 128 of the vehicle data processing system 140 need only trigger the microphone 214 or camera 216 associated with the occupied seat 200.

The entry authentication component 244 can determine at least two authentication states based on the at least two received authentication input signals and the plurality of authentication information. For example, the entry authentication component 244 can determine a first authentication state based on a first authentication input signal and a first authentication information of the plurality of authentication information, and can determine a second authentication state based on a second authentication input signal and a second authentication information of the plurality of authentication information. The entry authentication component 244 can perform an individual authentication for each received authentication input signal to determine a corresponding authentication state. Each authentication state can indicate whether the corresponding authentication is successful or not. If the entry authentication component 244 can identify a particular user or user category based on the authentication, the authentication can be successful. If the authentication is successful, the authentication state can represent an identified user or user category. If the authentication is successful, the authentication state can represent a probability value for each identified user or user category. If the authentication is unsuccessful, the corresponding authentication state can represent a reason for the unsuccessful (e.g., an improper authentication input signal or a lack of a match based on the corresponding authentication information).

The entry authentication component 244 may access a list of individuals (e.g., the vehicle owner, the owner's family, the owner's close friends, or a combination thereof) who may use the vehicle's keyless remote transmitter device for key fob authentication in response to receiving a key fob signal representative of successful remote unlocking of the vehicle door. The list of individuals may represent authentication information related to the key fob authentication and may be stored in a storage device (e.g., within the data repository 118) of the vehicle. The entry authentication component 244 may determine each of the individuals in the list as a possible user currently occupying the vehicle. The entry authentication component 244 may assign a probability value to each of the individuals based, for example, on statistical data of entries into the vehicle 130. For example, the owner may be assigned the highest probability, the owner's spouse may be assigned the second highest probability, and others may be assigned lower probability values.

For a mobile device identifier (ID) authentication signal, the corresponding authentication information may include a set of device IDs and corresponding device user identities stored in storage (e.g., within the data repository 118) of the vehicle 130. These signals may be encrypted. The entry authentication component 244 may compare the mobile client device ID of the received mobile device ID authentication signal to the set of stored device IDs. If a match is found, the entry authentication component 244 may identify the user associated with the matched device ID as the user occupying the vehicle 130. The authentication status may include a probability value associated with the user corresponding to the matched characteristic. If no match is found, the entry authentication component 244 may determine the authentication status corresponding to the mobile device identifier (ID) authentication signal as unsuccessful authentication. The authentication may include a reason for failure that is a lack of matching authentication information.

In the case of speaker recognition authentication, the digital assistant application 128 (or a corresponding voice processing device component) can extract one or more features representative of the speaker from the received voice input signal, and the entry authentication component 244 can compare the extracted features to a set of corresponding speaker-specific features (authentication information) stored, for example, within the data repository 118. If a match is found, the entry authentication component 244 can determine the authentication status (with respect to speaker recognition authentication) as successful. The authentication status can include a user indication associated with the matching feature in the set of corresponding speaker-specific features. The authentication status can include a probability value associated with the user corresponding to the matching feature. If no match is found, the entry authentication component 244 can determine the authentication status as unsuccessful authentication, which can include a reason for failure that matching authentication information is lacking.

In the case of facial recognition-based authentication, the digital assistant application 128 (or a corresponding image processor component) can extract one or more facial features from the received digital image, and the entry authentication component 244 can compare the extracted features to a set of corresponding features (associated with one or more known users) stored, for example, within the data repository 118. If a match is found, the entry authentication component 244 can determine the authentication status (for the facial recognition-based authentication) as successful. The authentication status can include a user indication associated with the matching facial feature in the set of corresponding facial features. The authentication status can include a probability value associated with the user corresponding to the matching facial feature. If no match is found, the entry authentication component 244 can determine the authentication status as unsuccessful authentication, which can include a reason for failure that matching authentication information is lacking. If the digital assistant application 128 (or a corresponding image processor component) cannot extract the facial features (e.g., because the face is not clearly visible in the image), the reason for failure can be defined as insufficient input.

The access control component 246 can identify one of a plurality of permission levels based at least in part on a determined authentication state, such as a first authentication state and a second authentication state. The access control component 246 can determine a particular user or one of a plurality of user categories based on the authentication state determined by the entry authentication component 244. For example, the access control component 246 can compare various authentication state information (e.g., an indication of a user or user category and corresponding probability value, if applicable) to determine a final user identity or user category associated with the individual occupying the vehicle 130. The access control component 246 can use a decision graph or a plurality of decision rules when determining a final user identity or user category associated with the individual occupying the vehicle 130. The plurality of user categories can include “owner”, “family: adult”, “family: child”, “friend”, “new user”, or a combination thereof. The access control component 246 can access a data structure (e.g., in the data repository 118) that maps each of a plurality of users or user categories to a corresponding permission level. The access control component 246 can determine a permission level associated with the determined final user identity or user category based on the accessed data structure.

The access control component 246 can identify a subset of the set of features available via the vehicle 130 in response to the access permission level. The digital assistant application 128 can place data associated with individual users or categories of users in multiple separate data accounts or multiple separate memory partitions. For a given user, the corresponding data account or memory partition can include data associated with the user, multimedia files associated with the user, access settings for the user (e.g., permission levels or accessible features), or a combination thereof. The set of functions available via the vehicle 130 may include functions or tasks that may be performed by the digital assistant application 128, such as accessing data stored on the storage device of the vehicle 130 (e.g., data associated with a data account or memory partition), accessing a locally stored playlist, controlling the rendering of a locally stored media file (or radio channel), operating a vehicle window or door, locking or unlocking a vehicle door, adjusting an air conditioning (AC) setting for the vehicle, remotely opening and closing a residential garage door, controlling or adjusting residential related settings (e.g., residential AC, home security system, sprinklers, or combinations thereof), accessing online services or mobile applications (e.g., online streaming or downloads, online shopping, or email), or combinations thereof. The digital assistant application 128 may encrypt each data account or memory partition separately using a security key that, for example, specifies the corresponding user or user category. The security key may be defined (or generated), for example, by hashing information associated with the corresponding user (e.g., a mobile device or other identifier). Digital assistant application 128 may group or combine playlists (or other data that is not security or privacy sensitive) associated with various users into a common storage area. For example, grouped playlists may be accessible to various users aboard vehicle 130.

Each permission level may be associated with a respective subset of the set of available features. The association (or mapping) between the permission levels and the corresponding subset of features may be set or adjustable by an administrator of the digital assistant application 128 (e.g., the owner of the vehicle 130). The association (or mapping) between the permission levels and the corresponding subset of features may be maintained in a second data structure (e.g., a table) stored in the data repository 118. The access control component 246 may determine the subset of features associated with the determined permission level based on the second data structure.

The access control component 246 may directly identify a subset of the set of features available via the vehicle 130 based at least in part on the determined authentication state (e.g., the first authentication state and the second authentication state) without determining a permission level. For example, the access control component 246 may identify the subset of features after identifying a user category based on the determined authentication state by accessing a data structure that maps each user category to a corresponding subset of features. Determining multiple authentication states, permission levels, subsets of features, or combinations thereof represents a multi-factor authentication process.

The digital assistant application 128 (of the vehicle data structure 140) can grant vehicular access to the determined subset of functions. When the digital assistant application 128 receives an audible request from a user in the vehicle 130, it can identify a corresponding requested action or function. For example, the instruction action API component 112 can generate an action data structure that represents the requested action or function. For each action data structure, the validation engine 116 can compare the corresponding action or function to the subset of functions mapped to the requester's permission level to determine whether to allow or deny the action (or function). When denying or blocking an action or function, the digital assistant application 128 (or the validation engine 116) can delete the corresponding action data structure and provide an audio output through the speaker 132 to indicate that the requester cannot access the requested function. Rejecting or blocking an action data structure before taking active steps in performing the corresponding function or action avoids unnecessary communication and computation, resulting in efficient use of resources (e.g., computational and bandwidth resources). If the action corresponding to the action data structure matches a function in the subset of functions accessible to the requester, the validation engine can forward the action data structure to another component for execution. The digital assistant application 128 may provide an audio output signal (e.g., through speaker 132) indicating that access to the requested function has been granted. When the function has been executed, the digital assistant application 128 may provide an audio output signal (e.g., through speaker 132) indicating that the request has been fulfilled.

When determining the access permission level or subset of allowed functions for a given user, the access control component 246 can determine, for each user occupying the vehicle, the corresponding seat 200 of the vehicle 130 in which the user is located. The access control component 246 can use seat sensor signals, door sensor signals, motion sensor signals, audio input signals from the microphone 214, digital images from the camera 216, or a combination thereof to map each user in the vehicle to a corresponding seat 200. Such a mapping can represent the occupancy state of the vehicle 130, which can later facilitate distinguishing between audible requests received from individual users in the vehicle. For example, when the digital assistant application 128 receives an audible request to open a vehicle window or unlock a vehicle door, the digital assistant application 128 can determine whether the audible request is from an adult in a front passenger seat or a person in a back passenger seat, for example, based on recorded instances of audible requests associated with individual microphones 214 in the vehicle. The digital assistant application 128 can compare the amplitude of the recorded instances of an audible request or the delay time between these instances to identify the microphone 214 (and therefore the associated seat) closest to the user making the audible request. An instance of an audible request (e.g., a corresponding electrical signal) recorded by the microphone 214 closest to the user making the audible request may arrive earlier and have a larger amplitude than other instances associated with other microphones 214 that are farther away from the requesting user. By identifying the microphone 214 closest to the requester, the digital assistant application 128 can use the mapping between the seat 200 and the user to identify the requester.

The digital assistant application 128 (or the entry detection component 142) may use the received sensor signals (e.g., a key fob signal, at least two separate seat sensor signals, at least two door sensor signals, at least two motion sensor signals, an ignition system signal, an air pressure sensor signal, or a combination thereof) to detect two entry events representing a first user and a second user entering the vehicle. The two entry events may be associated with simultaneous occupancy of the vehicle by the first user and the second user. The entry authentication component 144 may receive a first plurality of authentication input signals (e.g., associated with the first user) and a second plurality of authentication input signals (e.g., associated with the second user) in response to the two entry events. For example, the first plurality of authentication input signals and the second plurality of authentication input signals may include at least two input audio signals (e.g., from separate microphones 214), at least two digital images (e.g., from separate cameras 216), at least two mobile device ID authentication signals (e.g., from separate mobile client devices 150), a key fob signal, or a combination thereof.

The entry authentication component 144 can determine a first plurality of authentication states based on the first plurality of authentication input signals and a first set of authentication information from the plurality of authentication information. The entry authentication component 144 can check each authentication input signal of the first plurality of authentication input signals (associated with a first user) against corresponding authentication information from the plurality of authentication information (e.g., as described above with respect to various types of authentication input signals) to determine the first plurality of authentication states. The entry authentication component 144 can check each authentication input signal of the second plurality of authentication input signals (associated with a second user) against corresponding authentication information from the plurality of authentication information to determine the second plurality of authentication states.

The access control component 246 can identify a first permission level (e.g., for a first user) based at least in part on the first plurality of authentication states, and can identify a first subset of the set of features available via the vehicle 130 in response to the first permission level. The access control component 246 can identify a first user or a corresponding user category based on the first plurality of authentication states, and can identify the first permission level based on a data structure that maps the permission level to a corresponding subset of features. Similarly, the access control component 246 can identify a second permission level (e.g., for a second user) based at least in part on the second plurality of authentication states, and can identify a second subset of the set of features available via the vehicle 130 in response to the second permission level. The access control component 246 can directly identify the first and second subsets of the set of features based at least in part on the determined authentication states, without determining the first and second permission levels. For example, the access control component 246 may identify a first user category and a second user category based on the determined authentication status, and then access a data structure that maps each user category to a corresponding subset of capabilities to identify the first subset and the second subset.

The digital assistant application 128 can grant the first user vehicle access to a first subset of features and can grant the second user vehicle access to a second subset of features. Granting the first user access to the first subset of features can include the validation engine 116 intercepting an action data structure associated with the request by the first user to check whether the corresponding action (or requested function) matches any of the functions in the first subset. If no match is found, the validation engine 116 can delete the action data structure to block the corresponding action or requested function, and the digital assistant application 128 can provide an audio output to inform the first user of the denial of access. Otherwise, the digital assistant application 128 can perform the function and can provide an audio output representing the grant of access or the completion of the requested function. Similarly, giving the second user access to the second subset of functions may include the validation engine 116 intercepting an action data structure associated with the function requested by the second user and comparing the corresponding action (or requested function) of the action data structure with the functions of the second subset. Based on the comparison, the digital assistant application 128 can determine whether to perform or block the function requested by the second user.

The digital assistant application 128 may monitor data accounts or memory partitions (e.g., stored in the data repository 118 or the storage device of the vehicle 130). The data accounts or memory partitions may be temporary, as the digital assistant application 128 may maintain each of the data accounts or memory partitions for a period of time not exceeding a predetermined period from the last entry event (or last exit event) of the corresponding user. The digital assistant application 128 may maintain an individual time counter for each data account or memory partition. The digital assistant application 128 may reset the time counter for a given data account in response to detecting an entry event (or exit event) by a user associated with the data account. If the time counter exceeds a predetermined period (e.g., a week, a few weeks, a month, or several months), the digital assistant application 128 may delete the corresponding data account or memory partition (or its data). Because the data accounts (or memory partitions) are stored locally, the use of temporary data accounts allows for efficient use of the vehicle's memory resources.

The digital assistant application 128 (or the entry detection component 142) can detect an exit event representing a user's exit from the vehicle 130. For example, the entry detection component 142 can receive one or more additional sensor signals (e.g., an ignition system signal, a seat sensor signal, a motion detection signal, an air pressure signal, a door sensor signal, a key fob signal, or a combination thereof). The one or more additional sensor signals can represent an ignition system turning off, a previously occupied seat 200 being vacant, a reduction in motion detected inside the vehicle 130, a substantial change in air pressure inside the vehicle 130 (e.g., when a door is opened or closed), a door being opened or closed, a door being remotely locked, or a combination thereof. The entry detection component 142 can determine that a user is exiting the vehicle 130 based on the one or more additional sensor signals.

The digital assistant application 128 can update the occupancy status of the vehicle 130 in response to an exit event. For example, when the digital assistant 128 determines that a particular user has left the vehicle 130 (e.g., based on additional sensor signals), the digital assistant application 128 can update the occupancy status of the vehicle by removing the user from a list of users mapped to seats 200 in the vehicle 130, setting the corresponding seats 200 as vacant, or a combination thereof. In response to the occupancy update, the digital assistant application 128 can reset a time counter associated with the data account of the exiting user. The digital account assistant can encrypt a temporary data account (or memory partition) associated with the exiting user. The digital assistant application 128 can set the temporary data account associated with the user who left the vehicle 130 as inaccessible until the same user is authenticated again in the vehicle 130 (after a new entry event by that user).

Mobile client device 150 may include a corresponding data processing system (neither shown in FIG. 1) that includes a corresponding instance of digital assistant application 128. The data processing system and digital assistant application 128 associated with mobile client device 150 may include components similar to those discussed with respect to remote data processing system 102 and vehicle data processing system 140.

The processes described above with respect to the vehicle data processing system 140 (e.g., processes related to multi-factor authentication and access control to features available via the vehicle 130) may be performed by the remote data processing system 102 or a data processing system of the client device 150. For example, the vehicle data processing system 140 (or a corresponding digital assistant application 128) may receive and transmit (e.g., via the client device 150) a sensor signal, an authentication input signal, or a combination thereof to the remote data processing system 102 or to a data processing system associated with the client device 150. Detection of an entry event, multi-factor authentication, or determination of access permission levels or accessible features may be performed by the remote data processing system 102 or a data processing system associated with the client device 150.

3 illustrates an example method 300 of multi-factor entry authentication of a vehicle entry event. The method 300 may include receiving at least one sensor signal acquired by a first sensor installed in the vehicle (block 302). The method 300 may include determining an entry event into the vehicle based on the at least one sensor signal (block 304). The method 300 may include receiving a first authentication input signal and a second authentication input signal in response to the entry event into the vehicle (block 306). The method 300 may include determining a first authentication state based on the first authentication input signal and a first authentication information of the plurality of authentication information (block 308). The method 300 may include determining a second authentication state based on the second authentication input signal and a second authentication information of the plurality of authentication information (block 310). Method 300 may include identifying a permission level of a plurality of permission levels based at least in part on the first authentication state and the second authentication state (block 312). Method 300 may include identifying a subset of a set of functions available via the vehicle in response to the permission level (block 314). Method 300 may include the data processing system providing the vehicle access to the subset of functions (block 316). In general, method 300 may include any of the steps discussed with respect to the components of FIGS. 1 and 2.

Referring to FIGS. 1-3, the method 300 can include a step (block 302) in which a data processing system including the digital assistant receives at least one sensor signal acquired by a first sensor installed in the vehicle. The data processing system can include a remote data processing system 102 and a data processing system associated with the vehicle data processing system 140 or the mobile client device 150. The data processing system can receive, for example, a key fob signal from the key fob receiver device 210, a door sensor signal from the door sensor 204, a seat sensor signal from the seat sensor 202, a motion sensor signal from the motion sensor 206, a pressure signal from an air pressure sensor, an ignition signal from a vehicle ignition system (e.g., indicative of an on state of the ignition system), or a combination thereof.

The method 300 may include a step (block 304) of the data processing system determining an entry event into the vehicle 130 based on at least one sensor signal. The entry event may represent a user entering the vehicle 130. The data processing system may determine the entry event based on a sensor signal from a single type of sensor 136, or may determine the entry event based on sensor signals from two or more types of sensors 136 (e.g., an ignition signal and a seat sensor signal). The data processing system may determine the entry event using one or more decision rules or decision graphs. The decision rule or decision graph may represent a combination of sensor signals that signifies the occurrence of an entry event.

The method 300 may include a step (block 306) of the data processing system receiving a first authentication input signal and a second authentication input signal in response to an entry event into the vehicle. The first authentication input signal and the second authentication input signal may be obtained by at least one other sensor (e.g., the key fob receiver device 210, the microphone 314, the camera 216, or the communication transceiver 218) associated with the vehicle 130. The data processing system may receive two or more authentication input signals. The two or more authentication input signals may be of separate types obtained by at least two different types of sensors. For example, the entry authentication component 144 may receive at least two of: (1) a key fob authentication signal from the key fob receiver device 210, (2) a mobile device identifier (ID) authentication signal from a corresponding mobile client device (e.g., a smartphone, tablet, or smartwatch) 150, (3) an audio input signal from the microphone 214, or (4) a digital image from the camera 216. The data processing system may trigger two or more processors (or electronic devices) associated with the vehicle 130 to provide corresponding authentication input signals in response to an entry event (as discussed above with respect to FIGS. 1 and 2).

The method 300 may include a step (block 308) of the data processing system determining a first authentication state based on the first authentication input signal and the first authentication information of the plurality of authentication information, and a step (block 310) of determining a second authentication state based on the second authentication input signal and the second authentication information of the plurality of authentication information. The step of determining both the first authentication state and the second authentication state may be considered as a multi-factor authentication process. The data processing system may determine more than two authentication states depending on the number of authentication input signals received. The data processing system may perform a separate authentication for each of the received authentication input signals to determine a corresponding authentication state. Each authentication state may represent whether the corresponding authentication is successful or not. If the data processing system can identify a particular user or user category based on the authentication, the authentication may be successful. If the authentication is successful, the authentication state may represent an identified user or user category. If the authentication is successful, the authentication state may represent a probability value for each identified user or user category. If the authentication is unsuccessful, the corresponding authentication status may indicate the reason for the failure (e.g., an improper authentication input signal or lack of matching authentication information). The process of determining the authentication status for various types of authentication input signals is described above with respect to Figures 1 and 2.

The method 300 may include the data processing system identifying a permission level of a plurality of permission levels based at least in part on the first authentication state and the second authentication state (block 312). The method 300 may include the data processing system identifying a subset of a set of functions available via the vehicle in response to the permission level (block 314). The data processing system may identify a particular user or a respective user category of a plurality of user categories based on the determined authentication state. For example, the data processing system may compare information (e.g., an indication of a user or user category and corresponding probability value, if applicable) of the various authentication states to identify a final user identity or user category associated with an individual occupying the vehicle 130. The plurality of user categories may include, for example, “Owner”, “Family: Adult”, “Family: Child”, “Friend”, “New User”, or combinations thereof. The data processing system may determine a permission level associated with the determined final user identity or user category based on a data structure mapping each user or user category to a corresponding permission level. The data processing system can identify the subset of capabilities based on another data structure that maps each access permission level to a corresponding subset of capabilities.

The set of functions available via the vehicle 130 may include functions or tasks that may be performed by the digital assistant application 128 of the data processing system, such as accessing data stored on the storage device of the vehicle 130 (e.g., data associated with a data account or memory partition), accessing a locally stored playlist, controlling the rendering of a locally stored media file (or radio channel), operating a vehicle window or door, locking or unlocking a vehicle door, adjusting an air conditioning (AC) setting on the vehicle, remotely opening and closing a residential garage door, controlling or adjusting a residential related setting (e.g., residential AC, home security system, sprinklers, or combinations thereof), accessing online services or mobile applications (e.g., online streaming or downloading, online shopping, or email), or combinations thereof. Each of the acts in blocks 312 and 314 may be combined into a single step of determining a subset of functions based on the first authentication state and the second authentication state (e.g., without a step of determining an access permission level).

Method 300 may include a step (block 316) of the data processing system providing vehicle access to the subset of functions. The data processing system may receive an audible request from a user of the vehicle 130 and identify a corresponding requested action or function. The data processing system generates an action data structure representing the requested function and compares the requested function to the subset of functions mapped to the requester's access permission level to determine whether the requested function should be allowed or denied. The data processing system may perform or prevent the requested function based on the comparison (as discussed above with respect to Figures 1 and 2).

As described above with respect to Figures 1 and 2, the data processing system may place data associated with individual users or user categories into multiple separate temporary data accounts (or temporary memory partitions) and delete any data accounts (or memory partitions) that do not have a corresponding entry event for a predetermined period of time. The data processing system may separately encrypt each data account or memory partition, for example, with a security key that specifies the corresponding user or user category. The security key may be defined (or generated), for example, by hashing information associated with the corresponding user (e.g., a mobile device or other identifier). The data processing system may detect an exit event based on additional sensor signals and update the occupancy status of the vehicle based on the exit event.

The data processing system may detect two (or more) entry events representing two (or more) users entering the vehicle. The detected entry events may result in simultaneous occupancy of the vehicle 130 by the two (or more) users. The data processing system may receive an authentication input signal for each of the users and perform an individual multi-factor authentication (with the identified multiple authentication states) for each user, as discussed above with respect to Figures 1 and 2. The data processing system may determine an individual access permission level and individual subset of accessible functionality for each user. The data processing system may then grant each user access to a corresponding subset of functionality.

4 is a block diagram of an example computer system 400. The computer system or device 400 may include or be used to implement system 100 or a component of system 100, such as a data processing system associated with remote data processing system 102, vehicle data processing system 140, or mobile client device 150. The computer system 400 includes a bus 405 or other communication component for communicating information and a processor 410 or processing circuitry coupled to the bus 405 for processing information. The computer system 400 may also include one or more processors 410 or processing circuitry coupled to the bus for processing information. The computer system 400 also includes a main memory 415, such as a random access memory (RAM) or other dynamic storage device, coupled to the bus 405 for storing information and instructions executed by the processor 410. The main memory 415 may be or include the data repository 118. The main memory 415 may also be used by the processor 410 to store location information, temporary variables, or other intermediate information during execution of instructions. The computer system 400 may further include a read-only memory (ROM) 420 or other static storage device coupled to the bus 405 for storing static information and instructions for the processor 410. A storage device 425, such as a solid-state device, magnetic disk, or optical disk, may be coupled to the bus 405 for persistently storing information and instructions. The storage device 425 may include or be part of the data repository 118.

The computer system 400 may be coupled via a bus 405 to a display 435, such as a liquid crystal display or active matrix display, for displaying information to a user. An input device 430, such as a keyboard including alphanumeric and other keys, may be coupled to the bus 405 for communicating information and command selections to the processor 410. The input device 430 may include a touch screen display 435. The input device 430 may also include a cursor control, such as a mouse, trackball, or cursor direction keys, for communicating instructional information and command selections to the processor 410 and for controlling cursor movement on the display 435. The display 435 may be part of, for example, the data processing system 102, the client computing device 150, the head unit 220 of the vehicle 130, or other components of FIGS. 1 and 2.

The processes, systems, and methods described herein may be implemented by computer system 400 in response to processor 410 executing sequences of instructions stored in main memory 415. Such instructions may be read into main memory 415 from another computer-readable medium, such as storage device 425. Execution of the sequences of instructions stored in main memory 415 causes computer system 400 to perform the exemplary processes described herein. One or more processors in a multiprocessing arrangement may also be employed to execute the instructions stored in main memory 415. Hardwired circuitry may be used in place of or in combination with software instructions with the systems and methods described herein. The systems and methods described herein are not limited to any particular combination of hardware circuitry and software.

Although an example computer system is illustrated in FIG. 4, the subject matter including the operations described herein may be implemented in other types of digital electronic circuitry, or in computer software, firmware, or hardware including the structures disclosed herein and their structural equivalents, or in combinations of one or more of them.

The subject matter and operations described herein may be implemented in digital electronic circuitry, or in computer software, firmware, or hardware including the structures disclosed herein and structural equivalents thereof, or in combinations of one or more of them. The subject matter described herein may be implemented as one or more computer programs, e.g., circuits of one or more computer program instructions encoded on one or more computer storage media for execution by or to control the operation of a data processing device. Alternatively, or in addition, the program instructions may be encoded on an artificially generated propagated signal, e.g., a machine-generated electrical, optical, or electromagnetic signal generated to encode information for transmission to a suitable receiver device for execution by the data processing device. The computer storage medium may be, or may be included in, a computer-readable storage device, a computer-readable storage substrate, a random-access or serial-access memory array or device, or a combination of one or more of them. Although the computer storage medium is not a propagating signal, it may be a source or destination of computer program instructions encoded in an artificially generated propagating signal. A computer storage medium may also be or be included in one or more separate components or media (e.g., multiple CDs, disks, or other storage devices). Operations described herein may be implemented as operations performed by a data processing apparatus based on data stored in one or more computer-readable storage devices or data received from other sources.

The terms "data processing system", "computer device", "component", or "data processing apparatus" encompass a variety of apparatuses, devices, and machines for processing data, including, for example, a programmable processor, a computer, a system on a chip, or a combination of the foregoing. Such apparatuses may include special purpose logic circuitry, such as a field programmable gate array (FPGA) or an application specific integrated circuit (ASIC). In addition to hardware, such apparatuses may also include code that creates an execution environment for the computer program in question, such as code that constitutes a processor firmware, a protocol stack, a database management system, an operating system, a platform-wide runtime environment, a virtual machine, or one or more combinations thereof. Such apparatuses and execution environments may provide infrastructure for a variety of different computing models, such as infrastructures for web services, distributed computing, and grid computing. The components of system 100 may include or share one or more data processing apparatuses, systems, computer devices, or processors.

A computer program (also known as a program, software, software application, app, script, or code) may be written in any type of programming language, including compiled or interpreted, declarative or procedural, and may be deployed in any form, including as a stand-alone program or module, or as a component, subroutine, object, or other unit suitable for use in a computing environment. A computer program may correspond to a file in a file system. A computer program may be stored as part of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple cooperating files (e.g., files that store one or more modules, subprograms, or portions of code). A computer program may be deployed to be executed on one computer, or on multiple computers located at one site or distributed across multiple sites and interconnected by a communications network.

The process and logic flows described herein may be performed by one or more programmable processors executing one or more computer programs (e.g., components of a data processing system associated with the remote data processing system 102, the vehicle data processing system 140, or the mobile client device 150) that perform actions by operating on input data to generate output. The process and logic flows described above may also be performed by, and each apparatus may be implemented as, special purpose logic circuitry, such as, for example, an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit). Devices suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and storage devices, including, by way of example, semiconductor memory elements, such as, for example, EPROM, EEPROM, flash memory elements, magnetic disks, such as, for example, internal hard disks, removable disks, magneto-optical disks, and CD ROM and DVD-ROM disks. The processor and memory may be supplemented by, or incorporated in, special purpose logic circuitry.

The subject matter described herein may be implemented in a computer system including a back-end component, e.g., as a data server, or a middleware component, e.g., an application server, or a front-end component, e.g., a client computer having a graphical user interface or web browser that allows a user to interact with an implementation of the subject matter described herein, or a combination of one or more of such back-end, middleware, or front-end components. The components of the system may be interconnected by any form of medium of digital data communication, e.g., a communications network. Examples of communications networks include a local area network ("LAN"), a wide area network ("WAN"), an interconnected network (e.g., the Internet), and a peer-to-peer network (e.g., an ad-hoc peer-to-peer network).

A computer system such as system 100 or system 400 may include clients and servers. The clients and servers are typically remote from one another and typically interact through a communications network (e.g., network 148). The relationship of client and server arises by virtue of computer programs running on the respective computers with the client-server relationship to one another. In some implementations, the server transmits data (e.g., data packets representing content items) to the client mobile device 150 or head unit 220 of the vehicle 130 (e.g., to display data to a user interacting with the client device and receive user input). Data generated at the mobile client device 150 or vehicle data processing system 140 (e.g., a result of a user interaction) may be received by the server (e.g., received from the mobile client device 150 or vehicle 130 by data processing system 102).

Although the figures depict operations in a particular order, such operations need not be performed in the particular order shown, or sequentially, nor are all of the operations shown required to be performed. Actions described herein may be performed in different orders.

The separation of various system components does not require separation in all implementations, and the described program components may be included in a single hardware or software product. For example, the NLP component 106 and the validation engine 116 may be part of a single component, app, or program, or a logical device having one or more processing circuits, or one or more servers of the remote data processing system 102 or the vehicle data processing system 140.

Although several example implementations have been described, it is understood that the foregoing are presented by way of example and are illustrative and not limiting. In particular, while many of the examples presented herein involve particular combinations of method acts or system elements, those acts and those elements may be combined in other manners to achieve the same purpose. Acts, elements, and features discussed in connection with one implementation are not intended to be excluded from a similar role in other implementations.

The phraseology and terminology used herein is for the purposes of description and should not be considered as limiting. The use herein of "including," "comprising," "having," "holding," "containing," "characterized by," "characterized by," and variations thereof are meant to encompass the items listed thereafter, equivalents thereof, and additional items, as well as alternative implementations consisting of the items listed thereafter exclusively. In one implementation, the systems and methods described herein consist of one, a combination of each of more than one, or all of the described elements, acts, or components.

Any singular reference to implementations or elements or acts of the systems and methods herein may also encompass implementations that include a plurality of those elements, and any plural reference to implementations or elements or acts herein may encompass implementations that include only a single element. The singular or plural references are not intended to limit the presently disclosed systems or methods, their components, acts, or elements to a single configuration or multiple configurations. A reference to any act or element that is based on any information, act, or element may include implementations in which the act or element is based at least in part on any information, act, or element.

Any implementation disclosed herein may be combined with any other implementation or embodiment, and references to "an implementation," "several implementations," "one implementation," etc. are not necessarily mutually exclusive and are intended to indicate that a particular feature, structure, or characteristic described in connection with an implementation may be included in at least one implementation or embodiment. Such terms as used herein do not necessarily all refer to the same implementation. Any implementation may be combined with any other implementation, either inclusively or exclusively, in any manner not inconsistent with the aspects and implementations disclosed herein.

References to "or" may be construed as inclusive, such that any term described using "or" may refer to any of "one," "two or more," and "all" of the described terms. A reference to "at least one of A and B" can include "A only," "B only," and "both A and B." Such references used in conjunction with "comprising" or other open-ended terms can include additional items.

When a reference sign follows a technical feature in a drawing, a description, or any claim, the reference sign is included to improve the intelligibility of the drawing, the description, and the claims. Therefore, the presence or absence of a reference sign does not have a limiting effect on the scope of the claim element.

The systems and methods described herein may be embodied in other specific forms without departing from their characteristics. The foregoing implementations are illustrative rather than limiting of the described systems and methods. Accordingly, the scope of the systems and methods described herein is indicated by the appended claims, rather than the foregoing description, and all modifications that come within the meaning and range of equivalence of the claims are intended to be embraced therein.

100 Systems
102 Remote Data Processing System
104 Interface
106 Natural Language Processor (NLP) Components
108 Interface Management Components
110 Audio signal generator, audio signal generator component
112 Directed Action Application Programming Interface (API)
114 Response Selector, Response Selector Component
116 Verification Engine
118 Data Repositories
120 Parameters
122 Policy
124 Response Data
126 Templates
128 Digital Assistant Applications
130 Vehicles
132 Speaker
134 Display Devices
136 Sensors
138 Transducer
140 Vehicle Data Processing System
142 Entry Detection Component
144 Entry Authentication Component
146 Access Control Components
148 Network
150 mobile client devices
150a Mobile Client Devices
150b Mobile Client Devices
152 Data Providers
154 Data Providers
200 sheets
202 Sheet Sensor
204 Door Sensor
206 Motion Sensor
210 Key Fob Receiver Device
212 Entertainment System
214 Microphone
216 Camera
218 Communication Transceiver
220 Head Unit
244 Entry Authentication Component
246 Access Control Component
400 Computer Systems
405 Bus
410 Processor
415 Main Memory
420 Read-Only Memory (ROM)
425 Storage Device
430 Input Devices
435 Display

Claims (15)

1. A method implemented by one or more processors, comprising:
receiving a plurality of sensor signals from a plurality of sensors disposed in the vehicle;
generating an occupancy state of the vehicle based on one or more of the plurality of sensor signals, the occupancy state of the vehicle representing a mapping of one or more users located within the vehicle to a plurality of corresponding seats of the vehicle;
receiving an audible request to execute a vehicle function available via the vehicle from one or more microphones among a plurality of microphones located on the vehicle that are activated based on the occupancy status;
identifying a permission level for a given one of the one or more users that provided the audible request based on the occupancy status of the vehicle;
determining from which seat within the vehicle the audible request originated based on the occupancy state and acoustic information associated with the audible request;
identifying a user mapped to the determined seat as the given user who provided the audible request;
determining whether the given user is authorized to cause the vehicle function to be performed based on the authorization level associated with the given user;
and in response to determining that the given user is authorized to cause the vehicle function to be performed, causing the vehicle function to be performed via the vehicle. The method of claim 1, wherein generating the occupancy state of the vehicle is responsive to detecting an entry event into the vehicle. Detecting the entry event into the vehicle includes:
receiving a plurality of additional sensor signals from the plurality of sensors located in the vehicle;
and determining that one or more of the users have entered the vehicle based on one or more of the plurality of additional sensor signals. The method of claim 1 , wherein the given user is an operator of the vehicle, and the operator of the vehicle is authorized to cause the vehicle function to be performed. The method of claim 1 , wherein the given user is a passenger in the vehicle, and the passenger in the vehicle is not authorized to cause the vehicle function to be performed. the acoustic information associated with the audible request;
2. The method of claim 1, further comprising: detecting one or more of: corresponding amplitudes of multiple instances of the audible request generated by each of the one or more turned on microphones; or corresponding times of arrival of multiple instances of the audible request generated by each of the one or more turned on microphones. identifying the permission level for the given user who provided the audible request,
7. The method of claim 6, comprising one or more of the steps of: comparing the corresponding amplitudes of the multiple instances of the audible request to identify a proximate user to a given one of the turned on microphones as the given user who provided the audible request; or comparing the corresponding times of arrival of the multiple instances of the audible request to identify a proximate user to a given one of the turned on microphones as the given user who provided the audible request. The plurality of sensor signals
a key fob signal generated by a key fob receiver located in said vehicle;
a seat sensor signal generated by one or more seat sensors disposed in the vehicle;
a door sensor signal generated by one or more door sensors disposed in the vehicle;
The method of claim 1 , comprising one or more of a motion sensor signal generated by one or more motion sensors disposed on the vehicle , or a digital image signal generated by one or more cameras disposed on the vehicle. The method of claim 1, further comprising: ceasing to cause the vehicle function to be performed via the vehicle in response to determining that the given user is not authorized to cause the vehicle function to be performed. 1. A system comprising:
At least one processor;
and a memory storing instructions that, when executed, cause the at least one processor to:
receiving a plurality of sensor signals from a plurality of sensors disposed in the vehicle;
generating an occupancy state of the vehicle based on one or more of the plurality of sensor signals, the occupancy state of the vehicle representing a mapping of one or more users located within the vehicle to a plurality of corresponding seats of the vehicle;
receiving an audible request to execute a vehicle function available via the vehicle from one or more microphones among a plurality of microphones located on the vehicle that are activated based on the occupancy status;
identifying a permission level for a given one of the one or more users that provided the audible request based on the occupancy status of the vehicle;
determining from which seat within the vehicle the audible request originates based on the occupancy state and acoustic information associated with the audible request;
identifying a user mapped to the determined seat as the given user who provided the audible request;
determining whether the given user is authorized to cause the vehicle function to be performed based on the permission level associated with the given user;
and in response to determining that the given user is authorized to cause the vehicle function to be performed, causing the vehicle function to be performed via the vehicle. The system of claim 10 , wherein the instructions for creating the occupancy state of the vehicle are executed in response to detecting an entry event into the vehicle. instructions for detecting the entry event into the vehicle,
instructions for receiving a plurality of additional sensor signals from the plurality of sensors located in the vehicle;
and instructions for determining that one or more of the users have entered the vehicle based on one or more of the plurality of additional sensor signals. 11. The system of claim 10 , wherein the given user is an operator of the vehicle, and the operator of the vehicle is authorized to cause the vehicle function to be performed. 11. The system of claim 10 , wherein the given user is a passenger in the vehicle, and the passenger in the vehicle is not authorized to cause the vehicle function to be performed. 1. A non-transitory computer-readable storage medium having instructions recorded thereon, comprising:
The instructions, when executed, cause at least one processor to:
receiving a plurality of sensor signals from a plurality of sensors disposed in the vehicle;
generating an occupancy state of the vehicle based on one or more of the plurality of sensor signals, the occupancy state of the vehicle representing a mapping of one or more users located within the vehicle to a plurality of corresponding seats of the vehicle;
receiving an audible request to execute a vehicle function available via the vehicle from one or more microphones among a plurality of microphones located on the vehicle that are activated based on the occupancy status;
identifying a permission level for a given one of the one or more users that provided the audible request based on the occupancy status of the vehicle;
determining from which seat within the vehicle the audible request originates based on the occupancy state and acoustic information associated with the audible request;
identifying a user mapped to the determined seat as the given user who provided the audible request;
determining whether the given user is authorized to cause the vehicle function to be performed based on the permission level associated with the given user;
and in response to determining that the given user is authorized to cause the vehicle function to be performed, causing the vehicle function to be performed via the vehicle.

Images