CN115631550A - User feedback method and system - Google Patents

Abstract

The embodiment of this specification discloses a method and system for user feedback, the method comprising: acquiring status information, the status information including at least one of vehicle status and user status; sending a first reminder based on the status information Send information to the user terminal; obtain feedback information on the driving condition of the vehicle input by the user in response to the first reminder information on the user terminal, and the vehicle is the vehicle on which the user rides.

Description

Method and system for user feedback

technical field

The embodiments of this specification relate to the technical field of vehicle control, and in particular to a method and system for user feedback.

Background technique

With the continuous development of computer technology, intelligent travel (for example, shared travel or automatic driving, etc.) has also been developed rapidly. Taking autonomous driving as an example, autonomous vehicles can refer to vehicles that can achieve a certain level of driving automation. For example, self-driving vehicles can be controlled by the system (eg, background remote control) to achieve automatic driving. Self-driving vehicles can be used as taxis or public transportation. While self-driving vehicles provide services for humans, how to collect user feedback on the vehicle in a timely manner to optimize the automatic control of the vehicle and better meet the different needs of users is an urgent problem to be solved.

Contents of the invention

An aspect of the embodiments of this specification provides a method for user feedback, including: acquiring status information, the status information including at least one of vehicle status and user status; based on the status information, sending first reminder information to the user A terminal; obtaining the feedback information on the driving condition of the vehicle input by the user in response to the first reminder information on the user terminal, where the vehicle is the vehicle on which the user rides.

An aspect of the embodiment of this specification provides a system for user feedback, including: a first acquisition module, configured to acquire status information, the status information including at least one of vehicle status and user status; a sending module, configured to The state information is to send the first reminder information to the user terminal; the second acquisition module is used to obtain the feedback information on the driving condition of the vehicle input by the user in response to the first reminder information on the user terminal, the The vehicle is a vehicle on which the user rides.

An aspect of the embodiments of this specification provides a device for user feedback, the device includes at least one processor and at least one memory; the at least one memory is used to store computer instructions; the at least one processor is used to execute the computer At least part of the instructions are instructions to implement the operations corresponding to the user feedback method described in any one of the preceding items.

One aspect of the embodiments of this specification provides a computer-readable storage medium, the storage medium stores computer instructions, and when the computer instructions are executed by a processor, the operation corresponding to the user feedback method described in any one of the preceding items is realized .

Description of drawings

This specification will be further described in terms of exemplary embodiments, which will be described in detail with the accompanying drawings. These examples are non-limiting, and in these examples, the same number indicates the same structure, wherein:

Fig. 1 is a schematic diagram of an application scenario of a user feedback system according to some embodiments of this specification;

Fig. 2 is a flow chart of a user feedback method according to some embodiments of the present specification;

Fig. 3 is a flow chart of determining state information according to some embodiments of the present specification;

Fig. 4 is a flow chart of sending first reminder information according to some embodiments of this specification;

Fig. 5 is a flowchart of predicting a target position according to some embodiments of the present specification;

6A-6D are user interface diagrams according to some embodiments of the present description.

Detailed ways

In order to more clearly illustrate the technical solutions of the embodiments of the present specification, the following briefly introduces the drawings that need to be used in the description of the embodiments. Apparently, the accompanying drawings in the following description are only some examples or embodiments of this specification, and those skilled in the art can also apply this specification to other similar scenarios. Unless otherwise apparent from context or otherwise indicated, like reference numerals in the figures represent like structures or operations.

It should be understood that "system", "device", "unit" and/or "module" used in this specification is a method for distinguishing different components, elements, parts, parts or assemblies of different levels. However, the words may be replaced by other expressions if other words can achieve the same purpose.

As indicated in the specification and claims, the terms "a", "an", "an" and/or "the" are not specific to the singular and may include the plural unless the context clearly indicates an exception. Generally speaking, the terms "comprising" and "comprising" only suggest the inclusion of clearly identified steps and elements, and these steps and elements do not constitute an exclusive list, and the method or device may also contain other steps or elements.

The flowchart is used in this specification to illustrate the operations performed by the system according to the embodiment of this specification. It should be understood that the preceding or following operations are not necessarily performed in the exact order. Instead, various steps may be processed in reverse order or simultaneously. At the same time, other operations can be added to these procedures, or a certain step or steps can be removed from these procedures.

Fig. 1 is a schematic diagram of an application scenario of a user feedback system according to some embodiments of the present specification. In some embodiments, the user feedback system 100 may include a server 110 , a network 120 , a user terminal 130 , a database 140 and a vehicle 150 . The server 110 may include a processing device 112 .

In some embodiments, server 110 may be used to process information and/or data related to user feedback. Server 110 may be an individual server or a group of servers. The server group can be centralized or distributed (eg, server 110 can be a distributed system). In some embodiments, the server 110 may be regional or remote. For example, the server 110 can access information and/or data stored in the user terminal 130 , the database 140 , the vehicle 150 , and the detection unit 152 through the network 120 . In some embodiments, the server 110 can be directly connected to the user terminal 130, the database 140, the vehicle 150, and the detection unit 152 to access the information and/or data stored therein. In some embodiments, server 110 may execute on a cloud platform.

In some embodiments, server 110 may include processing device 112 . The processing device 112 may process data and/or information related to user feedback to perform one or more functions described in the specification. For example, processing device 112 may obtain status information. For another example, the processing device 112 may send the first reminder information to the user terminal based on the state information, so as to obtain feedback information on the driving condition of the vehicle input by the user in the user terminal in response to the first reminder information.

In some embodiments, the processing device 112 may include one or more sub-processing devices (eg, a single-core processing device or a multi-core, multi-core processing device). By way of example only, processing device 112 may include a central processing unit (CPU), an application specific integrated circuit (ASIC), an application specific instruction processor (ASIP), a graphics processing unit (GPU), a physical processing unit (PPU), a digital signal processor ( DSP), field programmable gate array (FPGA), programmable logic circuit (PLD), controller, microcontroller unit, reduced instruction set computer (RISC), microprocessor, etc. or any combination of the above. In some embodiments, processing device 122 may be integrated into vehicle 150 and/or user terminal 130 .

Network 120 may facilitate the exchange of data and/or information. In some embodiments, one or more components in system 100 (eg, server 110 , user terminal 130 , database 140 , vehicle 150 , detection unit 152 ) can send data and/or information to other components via network 120 . In some embodiments, network 120 may be any type of wired or wireless network. For example, network 120 may include a cable network, a wired network, a fiber optic network, a telecommunications network, an intranet, the Internet, an area network (LAN), a wide area network (WAN), a wireless area network (WLAN), a metropolitan area network (MAN) , Public Switched Telephone Network (PSTN), Bluetooth network, ZigBee network, Near Field Communication (NFC) network, etc. or any combination of the above. In some embodiments, network 120 may include one or more network access points. For example, network 120 may include wired or wireless network access points, such as base stations and/or internetwork switching points 120-1, 120-2, ..., through which one or more components of system 100 may be connected to network 120 to exchange data and/or information.

In some embodiments, the user terminal 130 may be a terminal for the user to input feedback information on the driving condition of the vehicle. In some embodiments, a user may be a service consumer. For example, service users may include passengers of online car-hailing platforms, passengers of self-driving vehicles, navigation service users, transportation service users, etc. In some embodiments, the user terminal 130 may include a mobile device 130-1, a tablet computer 130-2, a notebook computer 130-3, an in-vehicle device (not shown), a vehicle built-in device (not shown), a wearable One or any combination of devices (not shown) etc. In some embodiments, the mobile device 130-1 may include a smart home device, a smart mobile device, a virtual reality device, an augmented reality device, etc., or any combination thereof. Smart mobile devices may include smartphones, personal digital assistants (PDAs), gaming devices, navigation devices, point-of-sale (POS) devices, etc., or any combination thereof. The virtual reality device or the augmented reality device may include a virtual reality helmet, virtual reality glasses, virtual reality goggles, augmented reality helmet, augmented reality glasses, augmented reality goggles, etc. or any combination thereof. In some embodiments, automotive built-in devices may include on-board computers, on-board televisions, and the like. In some embodiments, wearable devices may include smart bracelets, smart footwear, smart glasses, smart helmets, smart watches, smart clothing, smart backpacks, smart accessories, etc., or any combination thereof. In some embodiments, the user terminal 130 may be a device with positioning technology for locating the location of the user terminal 130 .

Database 140 may store data and/or instructions. In some embodiments, the database 140 can store data obtained from the user terminal 130 , the detection unit 152 of the vehicle 150 , the processing device 112 and other components. In some embodiments, database 140 may store information and/or instructions for execution or use by server 110 to perform the exemplary methods described herein. For example, the database 140 may store the current vehicle state (eg, bumping condition, speed, acceleration, etc.) collected by the detection unit 152 . For another example, the database 140 may store feedback information input by the user at the user terminal 130 . For another example, the database 140 may also store the target position determined by the processing device 112 and the like. In some embodiments, the database 140 may include mass memory, removable memory, volatile read-write memory (eg, random access memory RAM), read-only memory (ROM), or any combination thereof. In some embodiments, database 140 may be implemented on a cloud platform. For example, the cloud platform may include private cloud, public cloud, hybrid cloud, community cloud, decentralized cloud, internal cloud, etc. or any combination thereof.

In some embodiments, database 140 may be connected to network 120 to communicate with one or more components of system 100 (eg, server 110, user terminal 130, vehicle 150, detection unit 152, etc.). One or more components of system 100 may access data or instructions stored in database 140 via network 120 . For example, the server 110 can obtain the user status from the database 140 and perform corresponding processing. In some embodiments, the database 140 can directly connect or communicate with one or more components in the system 100 (eg, the server 110, the user terminal 130). In some embodiments, database 140 may be part of server 110 . In some embodiments, database 140 may be integrated into vehicle 110 .

The vehicle 150 may be any type of driving vehicle, for example, a self-driving vehicle, a vehicle driven by an online car-hailing driver, and the like. In some embodiments, vehicle 150 may be a vehicle in which a user rides. As used herein, an autonomous vehicle may refer to a vehicle capable of achieving some level of driving automation. Exemplarily, the driving automation level may include the first level, that is, the vehicle is mainly supervised by humans and has specific autonomous functions (for example, autonomous steering or acceleration), and the second level, that is, the vehicle has one or more advanced driver assistance systems (ADAS, For example, adaptive cruise control systems, lane keeping systems), which can control braking, steering and/or accelerating the vehicle, level three, the vehicle can drive automatically when one or more certain conditions are met, level four, the vehicle can Operate without human input or omission, but still be subject to certain restrictions (e.g. restricted to a certain area), level 5 i.e. the vehicle can operate autonomously in all situations, etc. or any combination thereof.

In some embodiments, the vehicle 150 may have an equivalent structure that enables the vehicle 150 to move or fly. For example, vehicle 150 may include structures of a conventional vehicle, such as a chassis, suspension, steering device (eg, steering wheel), braking device (eg, brake pedal), accelerator, and the like. As another example, the vehicle 150 may have a body and at least one wheel. The body may be any type of body, such as a sports vehicle, coupe, sedan, pick-up truck, station wagon, sport utility vehicle (SUV), minivan, or convertible. At least one wheel may be configured as all wheel drive (AWD), front wheel drive (FWR), rear wheel drive (RWD), or the like. In some embodiments, it is contemplated that vehicle 150 may be an electric vehicle, a fuel cell vehicle, a hybrid vehicle, a conventional internal combustion engine vehicle, or the like.

In some embodiments, the vehicle 150 is capable of sensing its environment and driving through the use of one or more detection units 152 . The at least two detection units 152 may include sensor devices (eg, radar (eg, lidar devices)), global positioning system (GPS) modules, inertial measurement units (IMUs), cameras, etc., or any combination thereof. Radar (eg, a lidar device) may be configured to scan the surroundings of the vehicle 150 and generate corresponding data. A GPS module may refer to a device capable of receiving geolocation and time information from GPS satellites and determining the geographic location of the device. An IMU can refer to an electronic device that uses various inertial sensors to measure and provide a vehicle's specific forces, angular rates, and sometimes magnetic fields around the vehicle. In some embodiments, various inertial sensors may include acceleration sensors (eg, piezoelectric sensors), speed sensors (eg, Hall sensors), distance sensors (eg, radar, infrared sensors), steering angle sensors (eg, tilt sensors), traction-related sensors (eg, force sensors), etc. The camera may be configured to acquire one or more images related to an object (eg, a person, animal, tree, barricade, building, or vehicle) within range of the camera.

Those of ordinary skill in the art will appreciate that when an element (or component) of the user feedback system 100 is implemented, the element may be implemented by electrical and/or electromagnetic signals. For example, when the user terminal 130 sends feedback information to the server 110, the processor of the user terminal 130 may generate an electrical signal encoding the feedback information. The processor of the user terminal 130 may then send the electrical signal to the output port. If the user terminal 130 communicates with the server 110 via a wired network, the output port may be physically connected to a cable, and the cable may also transmit electrical signals to the input port of the server 110 . If the user terminal 130 communicates with the server 110 via a wireless network, the output port of the user terminal 130 may be one or more antennas, which convert electrical signals into electromagnetic signals. In an electronic device such as the user terminal 130 and/or the server 110, when its processor processes instructions, issues instructions and/or performs actions, the instructions and/or actions are performed through electrical signals. For example, when a processor retrieves or saves data from a storage medium (eg, database 140), it can send electrical signals to the storage medium's read/write device, which can read or write structured data in the storage medium. The structure data can be transmitted to the processor in the form of electrical signals through the bus of the electronic device. Here, an electrical signal may refer to one electrical signal, a series of electrical signals and/or at least two discontinuous electrical signals.

In some embodiments, a processing device (eg, processing device 112 ) may include a first acquisition module, a sending module, and a second acquisition module.

The first acquiring module may be used to acquire status information, and the status information includes at least one of vehicle status and user status. For specific details about the state information, reference may be made to step 210 and its related descriptions, which will not be repeated here.

The sending module may be configured to send first reminder information to the user terminal based on the state information.

In some embodiments, the sending module can be further used to determine a feedback reminder method, and the feedback reminder method is related to at least one of user status, user terminal status, driving environment, and user association information; and generate a feedback reminder method based on the feedback reminder method. the first reminder information, and send the first reminder information to the user terminal. For specific details about sending the first reminder information, reference may be made to FIG. 4 and its related descriptions, which will not be repeated here.

The second obtaining module may be used to obtain feedback information on the driving condition of the vehicle input by the user on the user terminal in response to the first reminder information.

It should be noted that the above description of the user feedback system and its modules is only for the convenience of description, and does not limit this specification to the scope of the examples. It can be understood that for those skilled in the art, after understanding the principle of the system, it is possible to combine various modules arbitrarily, or form a subsystem to connect with other modules without departing from this principle. In some embodiments, the disclosed first obtaining module, sending module and second obtaining module may be different modules in one system, or one module may realize the functions of the above two or more modules. For example, each module may share one storage module, or each module may have its own storage module. Such deformations are within the protection scope of this specification.

Fig. 2 is a flowchart of a user feedback method according to some embodiments of the present specification. As shown in FIG. 2 , the process 200 may include the following steps 210 , 220 , and 230 . In some embodiments, process 200 may be performed by a processing device (eg, processing device 112).

Step 210, acquiring status information, the status information includes at least one of vehicle status and user status. In some embodiments, step 210 may be performed by the first obtaining module.

In some embodiments, the vehicle state is related to the driving conditions of the vehicle. For specific details about the driving conditions of the vehicle, reference may be made to step 230 and its related descriptions, which will not be repeated here. The vehicle status may include information such as driving parameters, driving status, driving environment, driving section, driving location, and driving route.

In some embodiments, the vehicle status may include at least one of the current vehicle status, the vehicle status after a preset time, and the vehicle status at the target location.

In some embodiments, the current vehicle status may reflect part or all of the current driving conditions of the vehicle. For example, it reflects the navigation route that the vehicle is using, the road section or location where the vehicle is currently located, the current driving state of the vehicle, the current driving parameters of the vehicle, and the like. For specific details about the driving conditions, reference may be made to step 230 and related descriptions, which will not be repeated here.

In some embodiments, different data on the current vehicle state can be obtained in different ways. For example, the current driving parameters can be acquired through sensors installed in the vehicle (eg, speed sensor and acceleration sensor). For another example, the current driving state can be obtained through a monitoring system installed in the vehicle. For another example, the current weather environment and the current time environment are obtained from the weather service platform. For another example, the road condition environment may be reported by users of other vehicles, or obtained from a map or navigation service platform. The current driving section and the current driving position can be obtained through positioning technology, etc.

The vehicle state after the preset time may reflect part or all of the driving conditions of the vehicle after the preset time. The state of the vehicle at the target location may reflect part or all of the driving conditions of the vehicle at the target location. For specific details about the vehicle state after the preset time and the vehicle state at the target location, refer to step 220 and its related descriptions, which will not be repeated here.

In some embodiments, the user status is related to one or more of the following information: the user's vehicle usage, the user's availability, and the user's mood. In some embodiments, the user status may include the current user status, the user status after a preset time, and the user status at the target location.

The user's vehicle usage can reflect the state relationship between the user and the vehicle. For example, the user has already boarded the vehicle, the user is waiting for the vehicle to pick him up, the user is about to end using the vehicle (or the user will end using the vehicle after a certain time), etc.

A user's availability can at least reflect whether the user has time for feedback. For example, the availability status of the user may include very busy, relatively busy, free and so on. The busy level can be set as required. For example, if the user is on the phone, it can be considered very busy. If the user is using the mobile phone for entertainment (such as chatting, playing games, etc.), it can be regarded as relatively busy.

The user's emotion may reflect the user's feeling about the driving condition of the vehicle. For example, the user's emotion may include whether to be angry, uncomfortable, panicked, happy, helpless, and depressed.

In some embodiments, the processing device may obtain the current state information based on the image information of the user and/or the audio information of the vehicle. For example, get the current user state. For specific details about obtaining status information, reference may be made to FIG. 3 and its related descriptions, which will not be repeated here.

In some embodiments, the processing device may determine the user's emotion based on data collected by the wearable device (eg, heartbeat, heart rate, pulse, and blood pressure). For example, the number of heartbeats at a preset time exceeds a preset threshold, and the user's emotion may be nervous. In some embodiments, the processing device may determine the current user status based on the status of the user terminal. For example, determine the current user state and user sentiment according to the web pages browsed by the user or the information viewed.

The approaches of different embodiments can be combined. For example, in some embodiments, the processing device can obtain the emotional state of the user based on the data monitored by the wearable device, and obtain the vehicle state based on the storage device; in some embodiments, the processing device can obtain the vehicle state based on image data and/or audio data, Obtain the user's vehicle usage or idle status, and obtain driving parameters based on sensors.

Step 220, based on the status information, send first reminder information to the user terminal. In some embodiments, step 220 may be performed by a sending module.

The first reminder information may be information that guides the user terminal to give feedback on the driving condition. The first reminder information includes the content and manner of the first reminder information.

In some embodiments, the content of the first reminder message can be preset. In some embodiments, the content of the first reminder information may be determined based on the driving condition targeted by the feedback. For example, if the sensor of the vehicle detects bumps, the content of the first reminder message may include "please feedback whether the vehicle is stable". For another example, if the target location Q is a turning section, the content of the first reminder message may include "please give feedback on whether the turning at location Q is smooth". In some embodiments, the content of the first reminder information may be determined based on the user status. For example, if the acquired user's state is body shaking, the content of the first reminder message may include "please feedback whether the vehicle brakes suddenly". In some embodiments, the content of the first reminder information can be determined in combination with the user status and the driving condition for which the feedback is directed. For example, if no abnormality of the vehicle is found, but the user is angry, the content of the first reminder message may include "please give feedback on whether the current driving condition of the vehicle is satisfactory" and so on. The content of the corresponding first reminder information can be set for different user states and/or driving conditions targeted by the feedback, and the processing device can directly obtain the corresponding content after acquiring the user status and/or the driving conditions targeted by the feedback.

The first reminder information can be expressed in multiple ways. For example, the first reminder information may be voice, text, image, video, vibration and so on. For example, the user terminal may display an icon of "Tap Down" or "Like", as the first reminder information.

In some embodiments, the processing device may determine the presentation manner of the first reminder information according to at least one of user state, user terminal state, driving environment, and user related information. For details, refer to FIG. 4 and related descriptions.

In some embodiments, the processing device may send the first reminder information to the user terminal based on a preset rule. The preset rules can be specifically set according to the actual situation.

In some embodiments, the preset rule may include sending the first reminder information at preset time intervals. For example, the first reminder message is sent every 5 minutes. The preset time can be set according to the actual situation, so as to avoid the poor ride experience of the user caused by frequent sending of reminder information, or the incomplete collection of feedback information due to untimely reminders.

In some embodiments, the preset rule may include sending the first reminder information at intervals of a preset driving distance. For example, the first reminder information is sent every 50m driving distance. The preset rule may also include sending the first reminder information at a preset location. For example, wait at red, yellow and green lights. In some embodiments, the preset rules can be adjusted and optimized according to driving conditions. For example, if the weather is rainy or snowy, the preset time or preset driving distance may be shortened.

The approaches of different embodiments can be combined. For example, in some embodiments, the processing device may send the content of the first reminder information related to the vehicle speed to the wearable device worn by the user, and the wearable device will prompt in a vibration manner. In some embodiments, the processing device may send the first reminder message "Please give feedback on the current driving condition" to the user terminal at preset time intervals, and each time the first reminder message is sent in a different way, namely voice, image, text etc.

In some embodiments, the processing device may send the first reminder information to the user terminal based on the user portrait. User portraits can reflect the interests and needs of users. User portraits can be determined based on user association information. For more information about user association information, please refer to the related description below. For example, when the user profile reflects that the user is interested in the speed of the vehicle, the processing device may send first reminder information related to the speed of the vehicle. Through the user portrait, it is possible to send reminder information that the user is more interested in, to increase the user's interest in feedback, and to obtain more feedback information.

In some embodiments, the processing device may determine whether to send the first reminder information to the user terminal based on the vehicle status and/or the user status, or/and determine the content of the sent first reminder information. In some embodiments, the processing device may determine whether to send the first reminder message based on the current status information (including the current vehicle status and/or the current user status), or/and determine the content of the sent first reminder message.

In some embodiments, the processing device may determine whether to send the first reminder message and its content based on the current vehicle state. The processing device may determine whether to send the first reminder information and its content based on one type of information of the current vehicle state. For example, the processing device may determine whether to send the first reminder information based on whether the current vehicle state is an abnormal driving state in which a driving event occurs. Specifically, if the current vehicle state is an abnormal driving state, the first reminder information is sent. The processing device may determine the content of the first reminder information according to the driving event type of the abnormal driving state. For example, if a driving event "bump" occurs in the current vehicle state, the processing device may send a first reminder message "please feedback whether the vehicle is bumpy". The processing device may determine whether to send the first reminder information and its content based on the current driving section of the vehicle. For example, if the vehicle is on a turning section, the first reminder message "Please feedback whether the vehicle is turning smoothly" is sent. The processing device may determine whether to send the first reminder information and its content based on the current driving environment of the vehicle. For example, when the road condition environment of the current vehicle is congested, the processing device may send the first reminder message "Please feedback whether the vehicle frequently brakes suddenly".

In some embodiments, the processing device may determine whether to send the first reminder information and the content of the first reminder information based on various information of the current vehicle state. For example, the processing device may determine whether to send the first reminder information based on preset rules and in combination with various information of the current vehicle state. The preset rule can be to score various information of the current vehicle state separately, and then fuse the scores corresponding to the various information (such as averaging, weighted sum, weighted average, etc.), and determine whether to Sending first reminder information (for example, if the fused score is greater than a certain threshold, the first reminder information is sent). The content of the first reminder information may be determined based on the information with a score greater than a threshold among various types of information.

In some embodiments, the processing device may process various information of the current vehicle state based on the trained first reminder model, and determine whether to send the first reminder information and the content of the first reminder information. The processing device inputs a variety of information into the first reminder model, and outputs whether to send the first reminder information and the content subject of the first reminder information. Before various information is input into the model, the processing device can preprocess various information, including screening, vector representation, vector normalization, etc. The content theme refers to the driving situation that the user feedback needs to target. Content topics can include: bumps, speeding, sudden braking, etc. The content of the corresponding first reminder information can be preset based on the content theme.

The trained first reminder model can be obtained based on a plurality of first training samples. The first training samples include sample vehicle status, and the tags include whether to send the first reminder information and the content subject of the first reminder information. The first reminder model may include but not limited to linear regression model, neural network model, decision tree, support vector machine, naive Bayesian and so on.

In some embodiments, the processing device may determine whether to send the first reminder information and/or determine the content of the first reminder information based on the current user status. In some embodiments, the processing device may determine whether to send the first reminder information and its content based on a type of current user state information. For example, when the user's idle status reflects that the user is idle, the processing device may send a first reminder message "Please evaluate the automatic driving technology of the vehicle". For another example, if the user's emotion reflects that the user's emotion is panic, the processing device may send the first reminder information "is the vehicle stable" or "is the vehicle suddenly braking?" and so on. For another example, the user's vehicle usage status reflects that the user has gotten into the vehicle, and the processing device may send the first reminder message "whether the vehicle starts smoothly" or the like.

In some embodiments, the processing device may determine whether to send the first reminder information and the content of the first reminder information based on various information of the current user status. For example, the processing device may determine whether to send the first reminder information and its content based on preset rules and in combination with various information of the current user status. For example, the user's vehicle usage status reflects that the user has already boarded the vehicle and the user is busy, and the processing device may not need to send the first reminder information. The user's mood is panic, but the user is relatively busy, and the processing device may send the first reminder message.

In some embodiments, the processing device may determine whether to send the first reminder information and its content based on one or more types of information in the current user state and one or more types of information in the current vehicle state. In some embodiments, the processing device may use preset rules to determine whether to send the first reminder information and the content of the first reminder information based on the current user state and the current vehicle state. Preset rules can be set according to requirements. For example, if the current driving section of the vehicle is a turning section, and the user's emotion is panic, the processing device may send the first reminder message "is the current turning smooth". For another example, if the current driving state of the vehicle is speeding, and the user is in a bad mood, the processing device may send the first reminder message "Is the current speed too fast?". For another example, if the user is in a bad mood and the current interior temperature of the vehicle is relatively high, the processing device may send a first reminder message "whether the current interior temperature is too high".

In some embodiments, the processing device may use the second reminder model to determine whether to send the first reminder information and its content based on the current user status and the current vehicle status. The second reminder model is similar to the first reminder model.

In some embodiments, compared with the first reminder model, the content topics output by the second reminder model may also be related to user emotions. Correspondingly, the content of the first reminder information may include content matching the user's emotion. For example, if the user's emotion is angry, the content of the first reminder message may include content to appease the user.

By comprehensively considering the current vehicle state and the current user state to determine whether to send the first reminder information, a more accurate assessment of the current driving condition of the vehicle can be ensured. This enables the user to be reminded in time when user feedback is required, and provides a basis for subsequent driving optimization. Also, disturbance to users can be avoided.

In some embodiments, the processing device may determine whether to send the first reminder information based on the user's feedback intention. For example, when the user's willingness to give feedback reflects that the user is not willing to give feedback, the processing device does not send the first reminder information to the user terminal. The user's willingness to feedback may be determined according to user association information. The user-associated information refers to information related to the user. User association information includes: user basic information and user historical behavior information. User basic information includes user credit rating, user gender, age, occupation, hobbies, user registration platform duration, user feedback intention and degree of feedback intention, etc. User historical behavior information includes: user historical order information, user historical evaluation information, and user historical feedback information. In some embodiments, the processing device may acquire user association information from a storage device or a platform (for example, an online car-hailing platform).

The approaches of different embodiments can be combined. For example, in some embodiments, the processing device may determine whether to send the first reminder information and the content of the first reminder information based on the vehicle state and the user profile. For example, the user profile reflects that the user is sensitive to speed, and when the speed of the vehicle is detected as exceeding the speed limit, the processing device may send the first reminder information.

The approaches of different embodiments can be combined. For example, in some embodiments, the processing device may send the first reminder message to the vehicle terminal in the form of text based on the vehicle status; in some embodiments, the processing device may send the first reminder message in the form of vibration based on the user status A wearable device for the user; in some embodiments, the processing device can send the first reminder information to the user's mobile phone in the form of an image based on the vehicle status and the user status. After the user receives the first reminder information, click the icon way of feedback.

In some embodiments, the processing device may determine whether to send the first reminder information, and the content and/or manner of the first reminder information based on the subsequent state information. The subsequent status information includes vehicle status after a preset time, user status after a preset time, vehicle status at the target location and/or user status at the target location.

The vehicle state after the preset time may be the vehicle state after the preset time starting from the current time. There can be one or more preset times. The preset time can be preset, for example, 10 minutes, 20 minutes and so on. The preset time can be determined according to driving conditions, user status and/or user association information, and the like. For example, if the current road is congested, the preset time can be extended. For another example, if it is determined according to the user association information that the user is not positive about the feedback, the preset time may be extended.

In some embodiments, the vehicle state can be obtained predictively after a preset time. According to the current vehicle state and the preset time, the vehicle state after the preset time is predicted. For example, predicting whether there will be a bumpy driving event after the preset time may be determined based on the possible driving position of the vehicle after the preset time. The predicted driving position after the preset time can be determined based on the current driving speed and driving route.

In some embodiments, the prediction may be achieved by a first vehicle state prediction model. The first vehicle state prediction model can process the current vehicle state and the preset time to obtain the vehicle state after the preset time.

The second training samples used in the training of the first vehicle state prediction model include samples of current vehicle state and sample preset time, and the label represents the state of the vehicle after the preset time. The first vehicle state prediction model may include but not limited to linear regression model, neural network model, decision tree, support vector machine, naive Bayesian and so on.

In some embodiments, in addition to the preset time and the current vehicle state, the input of the first vehicle state prediction model may also include the navigation route (ie, the first navigation route) used by the vehicle currently driving, etc., correspondingly, the first The second training samples also include sample first navigation routes.

In some embodiments, the processing device may also optimize or update the state of the vehicle after the preset time by using the historical vehicle driving conditions at the location where the vehicle arrives after the preset time. For example, the processing device predicts that the vehicle arrives at position X after a preset time, and the vehicle state after the preset time predicted by the processing device includes an abnormal driving state of "bumping", and the historical vehicle driving status at position X that exceeds a threshold does not include "bumping". ", the vehicle status can be updated to "no bumps" after the preset time. Through the historical vehicle driving conditions at the position where the vehicle arrives after the preset time, the vehicle status after the preset time is optimized or updated, and the driving conditions of multiple historical vehicles can be used to obtain the after-the-preset time that is more suitable for the actual situation. Vehicle status, to improve the prediction accuracy of vehicle status after a preset time.

The state of the user after the preset time is similar to the state of the vehicle after the preset time, and will not be repeated here. In some embodiments, predicting the user state after a preset time can be realized by using a first user state prediction model. The first user state prediction model can process the current user state and the preset time to obtain the user state after the preset time. The first user state prediction model can also process the user state, preset time and user related information to obtain the user state after the preset time. For user association information, please refer to the relevant description above. The first user state prediction model is similar to the first vehicle state prediction model, and will not be repeated here.

The target position in the vehicle state at the target position or in the user state at the target position may be a preset position. For example, a preset or determined location of a user terminal, a driving terminal, or a processing device. The target location can include: a point, an area or a road segment.

There can be one or more target locations. In some embodiments, the target location can be set based on preset rules. The preset rule may be to determine a target position every preset distance. The preset rule may also take the location where the driving event may occur as the target location. For example, a turning position, a section or position with pedestrian congestion (such as a school, etc.) is used as the target position. The processing device, the user terminal or the driving terminal may obtain the navigation route (ie, the first navigation route) that the vehicle is currently driving, and determine the location where the driving event may occur as the target location from the navigation route.

In some embodiments, the target position can be predicted based on feedback information, historical information, etc. For specific details about the predicted target position, refer to step 520 and related descriptions, which will not be repeated here.

In some embodiments, the state of the vehicle at the target location can be obtained predictively. According to the current vehicle state and the target position, the vehicle state at the target position is predicted. In some embodiments, the prediction is achieved by a second vehicle state prediction model. The second vehicle state prediction model can process the current vehicle state and the target position to obtain the vehicle state at the target position. The third training sample used for training the second vehicle state prediction model includes a sample current vehicle state and a sample target position, and the label is used to represent the vehicle state at the target position. In some embodiments, the second vehicle state prediction model may include but not limited to linear regression model, neural network model, decision tree, support vector machine, naive Bayesian and so on.

In some embodiments, the processing device may also determine the driving conditions at the target location based on historical driving conditions at the target location. For example, the historical driving conditions at the target location, the current vehicle state and the target location are input into the second vehicle state prediction model to determine the driving conditions at the target location.

In some embodiments, the processing device can also calculate the preset time for the vehicle to travel to the target position based on the target position and the current driving parameters, so as to obtain the target position based on the preset time and the current state of the vehicle using the first vehicle state prediction model vehicle status.

In some embodiments, the prediction of the user state at the target location can be realized by a second user state prediction model. The second user state prediction model can process the current user state and the target location to obtain the user state at the target location. The second user state prediction model is similar to the second vehicle state prediction model, and will not be repeated here.

In some embodiments, similar to determining whether the first reminder message occurs based on the current vehicle state, the processing device may determine whether to send the first reminder message and its content based on the vehicle state after a preset time. For example, the processing device may determine whether a driving event occurs based on the state of the vehicle after a preset time, and if so, may send the first reminder information.

In some embodiments, similar to determining whether to send the first reminder information based on the current user status, the processing device may determine whether to send the first reminder information and its content based on the user status after a preset time.

In some embodiments, similar to determining whether to send the first reminder information based on the current user status and the current vehicle status, the processing device may determine whether to send the first reminder information and based on the vehicle status after a preset time and the user status after a preset time. its content.

In some embodiments, similar to determining whether to send the first reminder message based on the current vehicle state, the processing device may determine whether to send the first reminder message and its content based on the vehicle state at the target location.

In some embodiments, similar to determining whether to send the first reminder information based on the current user status, the processing device may determine whether to send the first reminder information and its content based on the user status at the target location.

In some embodiments, similar to determining whether to send the first reminder information based on the current user status and the current vehicle status, the processing device may determine whether to send the first reminder information and its content based on the vehicle status at the target location and the user status at the target location .

The first reminder information determined based on the subsequent state information is early reminder information sent to the user, that is, the user is reminded in advance to give feedback on the subsequent driving conditions. At this time, the time for sending the first reminder information to the user terminal may be adjusted according to actual conditions. For example, it is possible to send at a preset distance from the target location. For another example, it may be sent at a certain time before reaching the preset time. For another example, the sending time may be determined according to user state, user terminal state, driving environment, or/and user related information. Exemplarily, before the preset time or the target location is reached, the user state or the user terminal state is idle and the state is sent.

In some embodiments, the processing device may determine the manner of the first reminder information based on the subsequent user state, the subsequent user terminal state, the subsequent driving environment, and user association information. For details, reference may be made to FIG. 4 and related descriptions thereof, and details are not repeated here.

In some embodiments, the subsequent state of the user terminal may include the state of the user terminal after a preset time and the state of the user terminal at the target location. Regarding the state of the user terminal, refer to FIG. 4 and related descriptions.

In some embodiments, the state of the user terminal or the state of the user terminal at the target location can be predicted after a preset time. For example, it is possible to identify chat content and predict whether the chat will end after a preset time. In some embodiments, the processing device may predict a subsequent user terminal state according to the current user terminal state, user association information and/or current user state.

The approaches of different embodiments can be combined. For example, in some embodiments, the subsequent user terminal state can be predicted based on user emotion and current user terminal state. For example, if the user emotion is "happy" and the current user terminal state is playing games, then the predicted subsequent user terminal state is playing games. In some embodiments, the subsequent state of the user terminal may be predicted based on user idle conditions, user emotions, and current user terminal state.

In some embodiments, the subsequent driving environment may include the driving environment after a preset time and the driving environment at the target location. In some embodiments, the driving environment and the driving environment at the target location can be obtained from a service platform (for example, map service or weather service, etc.) after a preset time.

The approaches of different embodiments can be combined. For example, in some embodiments, the processing device can send the first reminder information to the user terminal in real time based on the status information; in some embodiments, the processing device can send the first reminder information to the user terminal in real time in the form of an image based on the current status information. The user terminal, and based on the subsequent state information, sends the first reminder information to the user terminal by voice at a certain time before reaching the preset time or the target position.

In some embodiments, the first reminder information determined based on the subsequent state information may be directly displayed on the navigation interface of the user terminal. In this way, the user can know in advance what needs feedback and where to give feedback by viewing the navigation route.

As shown in Figures 6A-D, Figure 6B is the driving interface of the vehicle. As shown in Figure 6B, the vehicle is traveling from the current position A to the position B1, and the processing device can display on the user terminal interface that the vehicle is traveling from the current position to the next position. Estimated time at a location to prompt the user for feedback when they arrive at the next location. For example, the interface of the user terminal may display "Going to B1, estimated time for feedback: 20 minutes".

In some embodiments, the processing device can display the road conditions at the driving position corresponding to the subsequent vehicle state (ie, the vehicle state after a preset time and the vehicle state at the target position) on the current position feedback interface and the vehicle traveling interface. Taking FIG. 6A or 6B as an example, the road segment icons from position A to position B1 displayed on the user terminal interface indicate that the road segment is a delayed driving segment. For example, road sections that require longer driving than usual due to muddy roads or construction; road section icons from position B2 to position B3 indicate that the road section has a school; road section icons from position B3 to position B4 indicate that the road section is a road section with many traffic lights .

In some embodiments, the processing device may also display road conditions at the current location of the vehicle on the vehicle's traveling interface. For example, as shown in FIG. 6B , the floating window on the right side in FIG. 6B may display that the current road of the vehicle is muddy/under construction.

In some embodiments, the processing device may dynamically adjust the current position feedback interface and the vehicle's traveling interface based on the user's preset instruction. The preset instruction may be an instruction generated after the user gives feedback on the driving condition at the current location. Still taking FIG. 6A as an example, if the user clicks the emoticon icon to give feedback on the driving conditions at the current location, the processing device may switch the current location feedback interface to the vehicle's driving interface.

In some embodiments, the user can zoom in on the current position feedback interface and the vehicle's traveling interface on the user terminal interface. For example, taking FIG. 6A or FIG. 6B as an example, the user can click the "-" icon and the "+" icon displayed on the current position feedback interface and the vehicle's traveling interface to zoom the interface.

In the user interface shown in Figure 6A, the current vehicle position is position A, and the vehicle positions after the preset time are B ₁ , B ₂ and B ₃ , where B ₁ , B ₂ and B ₃ are the vehicle positions after 10 minutes, The vehicle position after 20 minutes and the vehicle position after 30 minutes. On the current user interface, the first reminder message for the feedback of the driving situation in 10 minutes is displayed, "Estimated to arrive at position B ₁ in 10 minutes, please prepare to give feedback on the driving situation in 10 minutes." The user can click on the three expressions on the interface Icons for feedback, among which, from left to right, the three emoji icons represent very satisfied, generally satisfied and dissatisfied respectively. After 10 minutes, the user interface is updated to Figure 6B, the current vehicle position is updated to position B ₁ , B ₂ and B ₃ are the vehicle position after 10 minutes and the vehicle position after 20 minutes, respectively, and the first reminder message on the current interface is updated to "Estimated 10 minutes Arrived at position B ₂ , please prepare to give feedback on the driving conditions after 10 minutes."

In the user interface shown in Figure 6C, the current vehicle position is position A, and the target positions include positions C ₁ , C ₂ , C ₃ , C ₄ , C ₅ and C ₆ , where C ₁ , C ₂ , C ₃ , C _{4. C5} and _C6 are turning position, turning position, school section, turning position, traffic light position and turning position respectively. On the current user interface, the first reminder message "Please prepare to give feedback on whether the turn at _C1 is smooth" is displayed for the feedback of the driving condition at position _C1 , and the user can click the three emoticon icons on the interface to give feedback. After the vehicle travels to _C1 , the user gives feedback on the driving conditions at this location (specifically whether the turning is smooth or not). And, the user interface enters Figure 6D, the current vehicle position is updated to C ₁ , the target position is updated to positions C ₂ , C ₃ , C ₄ , C ₅ and C ₆ , and the first reminder message on the current interface is updated to "Please prepare for Feedback on whether the turn at C ₂ is smooth."

It can be seen that the vehicle state after the preset time and the vehicle state at the target position are used to reflect the future driving conditions of the vehicle. By sending the first reminder information based on the future driving conditions, the user can be reminded in advance before reaching the preset time or the target position Feedback is performed to avoid the situation that the user does not send feedback information when the vehicle has passed the position at the preset time or the target position due to the delay in sending the first reminder information.

Step 230, acquiring the feedback information on the driving condition of the vehicle input by the user on the user terminal in response to the first reminder information. In some embodiments, step 230 may be performed by the second obtaining module.

User means a passenger traveling in a vehicle. The vehicle may be a vehicle used in smart travel, that is, a vehicle on which a user rides. For example, the vehicle can be a shared vehicle in online car-hailing. For another example, the vehicle may be an automatic driving vehicle or the like. For specific details about the self-driving vehicle, refer to the relevant description above, and will not repeat them here.

A user terminal is a terminal for a user to perform operations or receive information. In some embodiments, the user terminal may be a terminal that provides online services for users. For example, the user terminal may be a terminal that provides online car-hailing services for users. For another example, the user terminal may be a terminal that provides automatic driving services for users. User terminals may include, but are not limited to, one or more combinations of notebooks, tablet computers, mobile phones, personal digital assistants (PDAs), in-vehicle camera equipment, dedicated touch devices, and wearable devices. In some embodiments, the user terminal may also be a device terminal inside the vehicle for collecting information or providing information, for example, a vehicle terminal, an electronic device, or a remote control device.

The running condition of the vehicle may include information related to the running of the vehicle. The driving condition may include: one or a combination of driving route, driving location, driving road section, driving environment, driving state, and driving parameters.

The travel route may be a route along which the vehicle travels from the departure point to the destination. In some embodiments, the driving route may be at least one navigation route automatically generated by the processing device based on the departure place and the destination. The origin and destination can be obtained in various ways. For example, it can be acquired by the user inputting on the user terminal.

The driving location may be a location where the vehicle is driving. Locations can include points, areas, or segments. In some embodiments, the driving location may be a location that the vehicle has passed, a current location, or a location that is about to pass. The driving location may be a track point in the driving route. For example, the driving location may include a departure place, a destination, a current location, and the like.

A road section may be a line formed by at least two location points on the road network. At least two location points may be adjacent or non-adjacent. For example, a road segment can be part or all of a road.

In some embodiments, the driving section may be a section that the vehicle travels through. In some embodiments, the driving section may be a section that the vehicle has passed, a section where the current location is located, or a section that is about to pass. In some embodiments, the driving section may also be a preset section included in the driving route. The preset road section can be specifically set according to actual needs. For example, the preset road section can include at least one of a turning road section, a roundabout road section, a no-stopping road section, a one-way driving road section, and an accident-prone road section.

The driving environment may be environmental information related to the vehicle or the driving of the vehicle. The driving environment may include the environment inside the vehicle when the vehicle is running and the external environment where the vehicle is running. In some embodiments, the driving environment may include the air quality in the car, the humidity in the car, the temperature in the car, the number of passengers in the car, whether the car is clean, the light in the car, and the appearance of the passengers in the car. In some embodiments, the driving environment may include at least one of weather environment, road condition environment and time environment.

In some embodiments, the weather environment may reflect air temperature, air pressure, humidity, visibility, and the like. For example, weather conditions may include raining, snowing, sunny, and the like.

In some embodiments, the road condition environment may reflect the smoothness of the vehicle's driving. For example, the road condition environment can reflect the steepness of the road, the number of lanes, the degree of road congestion, the situation of road closure, the situation of road construction and the situation of traffic control, etc. For example, the road condition environment may include traffic flow, the number of traffic lights, and construction ahead.

In some embodiments, the time environment may reflect time information of vehicle travel. For example, the time environment may include whether it is a holiday, whether it is a working day, whether it is a rush hour, and so on.

The driving state may be state information of the vehicle during the driving process of the vehicle. In some embodiments, the driving state may include: a normal driving state or an abnormal driving state, and the like. The abnormal driving state may be driving in which a driving event occurs, and the normal driving state may be driving in which no driving event occurs. The driving event may be an event related to vehicle operation and an event that affects the user's riding experience that occurs during the driving of the vehicle. In some embodiments, the driving events may include events such as starting, stopping, accelerating, decelerating, rapid acceleration, overspeed, sudden braking, turning and bumping of the vehicle.

The driving parameter refers to a parameter reflecting the driving condition of the vehicle. In some embodiments, the driving parameters include but are not limited to: speed parameters, acceleration parameters and the like.

The feedback information can reflect the user's attitude towards the driving condition of the vehicle. In some embodiments, the feedback information may include whether the driving condition of the vehicle is satisfactory and/or the degree of satisfaction (for example, very satisfied, generally satisfied, dissatisfied, etc.) and the like. In some embodiments, the feedback information may also reflect other information. For example, reasons for satisfaction or dissatisfaction, suggestions for the driving condition of the vehicle, etc.

In some embodiments, the user may input feedback information on the user terminal according to his own wish or demand. For example, the user inputs feedback information on the user terminal according to the driving condition of the vehicle (for example, the vehicle speed is too fast or the vehicle brakes suddenly). For another example, the user inputs feedback information on the user terminal at any time.

In some embodiments, the user may input feedback information based on the first reminder information sent by the processing device.

The approaches of different embodiments can be combined. For example, in some embodiments, the user can input feedback information on the driving parameters of the vehicle in the mobile phone; in some embodiments, the processing device can determine whether to send the first reminder information based on the vehicle state after a preset time, and the user can After receiving the first reminder information, input the feedback information on the driving route of the vehicle into the vehicle-mounted terminal at an appropriate time.

In some embodiments, the user may input feedback information in various ways. For example, users can send feedback information by clicking icons or text buttons, sending text or voice, or making gestures or facial movements. For example, the "like" and "dislike" icons displayed on the user terminal interface, the user sends feedback information representing dissatisfaction by clicking the "dislike" icon. The user faces the camera of the terminal and makes a preset gesture action corresponding to "satisfactory" as the feedback information sent.

The approaches of different embodiments can be combined. For example, in some embodiments, the user can click an icon on the mobile phone interface to input feedback information on the driving event; in some embodiments, the processing device can determine whether to send the first reminder information based on the vehicle status at the target location, and if sent, then The processing device sends the first reminder information to the user's wearable device when it is at a preset distance from the target position, and the user can send a voice through the wearable device as feedback information on the driving route after receiving the first reminder information.

After the user inputs the feedback information, the feedback information may be stored in the storage device or directly sent to the processing device. The processing device (for example, the second obtaining module) may directly obtain the feedback information from the user terminal, or may read the feedback information from the storage device.

In some embodiments, after the processing device acquires the feedback information, it may process the feedback information to determine the attitude reflected in the feedback information. For example, the processing device may determine the attitude represented by the icon clicked by the user on the user terminal according to the preset relationship between the icon and the attitude. For another example, the processing device may recognize (for example, through a model or algorithm, etc.) the text, picture, video or voice sent by the user terminal, and determine the reflected attitude.

The approaches of different embodiments can be combined. For example, in some embodiments, the processing device may determine whether to send the first reminder message based on the current user state (for example, the user's current emotion), and the user may input the Feedback information; in some embodiments, the processing device can determine whether to send the first reminder message based on the vehicle status after the preset time, and if so, send the first reminder message to the mobile phone at a certain moment before the preset time, and the user Feedback information can be input into the mobile terminal based on the first reminder information received by the mobile terminal, and when the feedback information reflects that the user is not satisfied with the navigation route, the navigation route can be updated.

In some embodiments of this specification, the first reminder information is sent to the user terminal in a timely manner by acquiring status information, so that the user responds to the first reminder information and inputs feedback information on the status of the vehicle at the user terminal, which facilitates subsequent processing of the vehicle by the processing device. The driving system is operated to improve the driving situation corresponding to the unsatisfactory user feedback and improve the user's riding experience; at the same time, through the user's timely feedback during the travel process, it can avoid that the travel control automatically set by the self-driving vehicle cannot meet the user's actual situation. To further enhance the user's ride experience.

Fig. 3 is a flow chart of determining state information according to some embodiments of the present specification. As shown in FIG. 3 , the process 300 may include the following steps 310 to 340 . In some embodiments, process 300 may be performed by a processing device (eg, processing device 112). In some embodiments, the process 300 may be executed by a first acquiring module.

Step 310, acquiring image information of the user from the first terminal.

The first terminal may be a terminal device for capturing images. For example, a camera device, a monitoring device, a user terminal with a camera, or a vehicle-mounted device with a camera, etc. The image information of the user may be an image of the user captured by the first terminal or an image frame included in a video of the user captured by the first terminal.

In some embodiments, the first terminal can acquire the image information of the user in real time and upload it to the database. The processing device may acquire the image information of the user from the first terminal in real time. The processing device can also obtain the image information of the user from the database.

Step 320, acquiring vehicle-related audio information from the second terminal.

In some embodiments, the second terminal may be a terminal device for acquiring audio information. For example, a recording device, a camera device, a user terminal with a recording function, or a vehicle-mounted device with a recording function, etc. The second terminal and the first terminal may be the same or different.

The vehicle-related audio information refers to audio data emitted by the vehicle or emitted in the environment inside the vehicle. In some embodiments, the audio information related to the vehicle may include: the audio generated by the driving of the vehicle (such as the sound of the engine, the sound of brakes, etc.), the audio generated or received by the user using the user terminal, or the audio information between the user and the driver of the vehicle. communication audio etc.

In some embodiments, the second terminal can acquire vehicle-related audio information in real time and upload it to the database. The processing device can acquire vehicle-related audio information from the second terminal in real time. The processing device may also obtain vehicle-related audio information from the database.

Step 330, extracting the image features of the image information and the audio features of the audio information.

In some embodiments, image features may include color features, texture features, shape features, spatial features, and the like. In some embodiments, the processing device may extract image features through an image feature extraction algorithm or an image feature extraction model. For example, the image feature extraction model may include a convolutional neural network model and the like.

In some embodiments, audio features may include sampling frequency, bit rate, number of channels, frame rate, zero-crossing rate, short-term energy, short-term autocorrelation coefficient, and the like. In some embodiments, the processing device may extract audio features through an audio feature extraction algorithm. Audio feature extraction algorithms may include, but are not limited to, linear predictive analysis (Linear Prediction Coefficients, LPC), perceptual linear predictive coefficients (Perceptual Linear Predictive, PLP), linear predictive cepstral coefficients (Linear Predictive Cepstral Coefficient, LPCC) and Mel frequency cepstrum Number (Mel-Frequency Cepstrum Coefficient, MFCC) and so on.

In some embodiments, the processing device pre-processes the image information or audio information before extracting the audio features or image features. The preprocessing of image information includes: smoothing, noise elimination, edge enhancement, edge feature extraction, etc. The preprocessing of audio information includes: pre-emphasis, framing, windowing, etc.

Step 340, based on the processing of image features and audio features, determine status information.

In some embodiments, processing may be performed by a pre-trained feature processing model. Pre-trained feature processing models can determine state information based on image features and audio features. In some embodiments, the fourth training sample of the feature processing model includes sample image features and sample audio features of the sample vehicle, and the label is used to represent the state information of the sample vehicle. When the determined status information is the status of the vehicle, the tag represents the status of the vehicle, for example, the tag represents the driving event of the vehicle: starting, stopping, accelerating or turning, etc. When the determined status information is the user status, the tag represents the user status, for example, the tag represents the user's emotion: angry, uncomfortable, panicked or happy, etc. The processing device can iteratively update the parameters of the initial feature processing model based on a plurality of training samples, so that the loss function of the model satisfies a preset condition, for example, the loss function converges, or the loss function value is smaller than a preset value. When the loss function meets the preset conditions, the model training is completed, and the trained feature processing model is obtained.

In some embodiments, the image feature extraction model and feature processing model can be obtained through end-to-end training. Specifically, the sample image information of the sample user is input into the initial image feature extraction model, the sample audio feature is input into the initial feature processing model, and the loss function is constructed based on the label representing the state information of the sample vehicle and the output result of the initial feature processing model, and based on the loss The function iteratively updates the parameters of the initial feature processing model until the preset conditions are met.

In some embodiments of this specification, the current user status and the current vehicle status can be determined in real time through the user image information and vehicle audio information acquired in real time, so as to facilitate subsequent real-time acquisition of current status information and sending of the first reminder message.

Fig. 4 is a flowchart of sending first reminder information according to some embodiments of the present specification. As shown in FIG. 4 , the process 400 may include the following steps 410 and 420 . In some embodiments, process 400 may be performed by a processing device (eg, processing device 112).

Step 410, determine the feedback reminder mode. In some embodiments, step 610 may be performed by a sending module.

In some embodiments, the feedback reminder method may be used to determine the representation method of the first reminder information. For example, voice, text, image, video, vibration, etc. The feedback reminder method may be related to at least one of user status, user terminal status, driving environment and user associated information. For specific details about the user status, reference may be made to the above step 210 and related descriptions, which will not be repeated here. For specific details of the driving environment, refer to the above-mentioned step 230 and related descriptions, which will not be repeated here.

The user terminal status can reflect the terminal power, the terminal standby time, the terminal use time, whether the terminal is being used, the current use mode of the terminal (watching videos, listening to songs, etc.), etc.

In some embodiments, the way of reminding the feedback can be determined based on the status of the user.

The approaches of different embodiments can be combined. For example, in some embodiments, when the user's idle situation reflects that the user is reading a book, the feedback reminder method can be voice; in some embodiments, when the user's idle situation reflects that the user is listening to music, the feedback reminder method can be text display or image; in some embodiments, when the user's emotion is anger, the feedback reminder method can be voice.

In some embodiments, the way of reminding the feedback can be determined based on the state of the user terminal.

The approaches of different embodiments can be combined. For example, in some embodiments, when the user terminal state reflects that the user is using a speaker to talk or play music, etc., the feedback reminder method can be image display or text; When the power is on or in standby state, the feedback reminder method can be carried out through the on-board device or wearable device, for example, through the on-board device or smart watch voice display, image display or text display.

In some embodiments, the feedback reminder method can be determined based on the driving environment.

The approaches of different embodiments can be combined. For example, in some embodiments, when the environmental information in the car reflects that the environment is noisy and it is inconvenient to listen to the audio, the feedback reminder method can be through image display or text; when the environmental information in the car reflects that the light in the car is dim or the light is too bright, The feedback reminder method can be voice; in some embodiments, when the road condition environment is a congested road section, the vehicle brakes frequently at this time, and looking at the mobile phone screen for a long time may cause user discomfort, and the feedback reminder method can be voice or vibration.

In some embodiments, the feedback reminder method may also be determined based on user association information. For example, if the user association information reflects that the user's age exceeds the preset age, the text and sound in the feedback reminder method can be amplified. For another example, the user information reflects that the user's usual language is English, and the text and sound in the feedback reminder method can be switched to English. For another example, if the user information reflects that the user is Chinese, the text and sound in the feedback reminder method can be switched to Chinese.

In some embodiments, the feedback reminder method can also be based on various determinations in user status, user terminal status, driving environment and user associated information. The processing device can determine a model based on a preset rule or a trained feedback and reminder method, process various information in user status, user terminal status, driving environment and user related information, and determine a feedback reminder method.

For example, as mentioned above, each of user status, user terminal status, driving environment and user related information can contain multiple types, and each type can preset a corresponding feedback reminder method, and the preset rule can be a user status , user terminal status, driving environment, and user-related information are scored separately, and the scores of the same type of feedback reminder methods are fused (such as averaging, weighted sum, weighted average, etc.), based on different types The final feedback reminder method is determined by the fused score of the feedback reminder method (for example, the feedback reminder method with the largest fusion score is selected).

For another example, the processing device may simultaneously process multiple information in the user state, user terminal state, driving environment, and user related information based on preset rules or a trained feedback reminder method determination model, and determine multiple initial feedback reminder methods; Based on preset screening rules, a final feedback reminder method is determined from multiple initial feedback reminder methods. The preset filtering rules can be random filtering or set according to requirements.

In some embodiments, the feedback reminder method can be changed in real time based on the above actual situation. For example, when the user is on the phone, the feedback reminder method can be changed from the original voice display to text display or image display.

Step 420: Generate first reminder information based on the feedback reminder method, and send the first reminder information to the user terminal. In some embodiments, step 420 may be performed by a sending module.

For specific details about generating the first reminder information and sending it to the user terminal, reference may be made to the above step 220 and related descriptions, which will not be repeated here. In some embodiments, the processing device may send the first reminder information to the user terminal through a transmission method such as a short message, a network, or Bluetooth.

In some embodiments of this specification, based on various information (for example, user status, user terminal status, driving environment and user related information, etc.), the corresponding feedback reminder method is intelligently and dynamically determined to avoid the impact of fixed feedback reminder methods on The user is disturbed, and the user's riding experience is improved.

Fig. 5 is a flow chart of predicting a target position according to some embodiments of the present specification. As shown in FIG. 5 , the process 500 may include the following steps 510 and 520 . In some embodiments, process 500 may be performed by a processing device (eg, processing device 112). In some embodiments, the process 500 may be executed by the first acquiring module.

Step 510, acquiring historical feedback information related to at least one of the track point on the first navigation route and the user, and historical driving conditions corresponding to the historical feedback information.

In some embodiments, the first navigation route may be a navigation route that the vehicle is using. As described in step 230, the driving route may include at least one navigation route. In some embodiments, the first navigation route may be a navigation route selected by the user from the driving routes, or may be automatically selected by the processing device from the driving routes. The optimal navigation route (for example, the navigation route with the shortest travel time, the navigation route with the shortest time spent, or the navigation route with the least consumption amount, etc.).

In some embodiments, historical feedback information of historical users on historical driving conditions of historical vehicles may be stored in a storage device (eg, database 140 ) or a platform. It can be understood that there is at least one piece of historical feedback information corresponding to the historical driving conditions of the historical vehicles. For example, the correspondence between historical users, historical vehicles, historical driving conditions and historical feedback information may be stored in the storage device.

The track points on the first navigation route may be position points included on the first navigation route. In some embodiments, the acquiring module can acquire historical feedback information related to track points on the first navigation route. For example, the obtaining module may obtain historical feedback information corresponding to track points on the first navigation route and historical driving conditions corresponding to the historical feedback information from a storage device or a platform.

In some embodiments, the acquiring module can acquire historical feedback information related to the user. For example, the acquisition module may acquire the historical feedback information of the historical user as the user from the storage device or the platform, and the historical driving conditions corresponding to the historical feedback information.

In some embodiments, the acquiring module can acquire historical feedback information related to the user and the track points on the first navigation route. For example, the obtaining module obtains the historical feedback information of the historical user on the track points on the first navigation route from the storage device or the platform, and the historical driving conditions corresponding to the historical feedback information, wherein the historical user is the user.

Step 520, based on historical feedback information and historical driving conditions, predict the target position on the first navigation route.

In some embodiments, the processing device may predict the target position on the first navigation route based on historical feedback information (which may be referred to as "first historical feedback information") related to track points and/or users on the first navigation route . For example, the processing device may determine a track point corresponding to negative feedback (eg, unsatisfactory feedback) in the first historical feedback information as the target position. For example, if the user or other users were dissatisfied with past feedback information on the historical driving conditions of track point 1 and track point 2 included in the first navigation route, the processing device may determine track point 1 and track point 2 as target positions. For another example, the processing device may determine, as the target position, a track point whose number of times of negative feedback included in the first historical feedback information is greater than a threshold. The preset number of times can be specifically set according to actual needs, for example, 3 times or 5 times.

In some embodiments, the processing device may predict the target position on the first navigation route based on the historical driving conditions corresponding to the first historical feedback information (may be referred to as "first historical driving conditions"). For example, the processing device may determine a track point reflecting a driving event in the historical driving conditions as the target position. For example, if the driving event "turn" is reflected at track points 2 and 3, then track points 2 and 3 may be determined as target positions.

In some embodiments, the processing device may predict the target position on the first navigation route based on the first historical feedback information and the first historical driving condition. In some embodiments, the processing device may determine, as the target position, a track point that reflects a driving event in the first historical driving condition and whose corresponding first historical feedback information is negative feedback. Still taking the above example as an example, track point 2 may be determined as the target position.

In some embodiments, the processing device may predict the target position on the first navigation route based on the trained target position prediction model. Specifically, the target position prediction model may process the first historical feedback information and the first historical driving condition, and output the target position. The target position prediction model may include, but not limited to, a graph neural network model, a linear regression model, a neural network model, a decision tree, a support vector machine, and naive Bayesian.

When the target position prediction model is a graph neural network model, the nodes of the graph neural network model are track points on the first navigation route, and the characteristics of the nodes may at least include: the first historical feedback information, the first historical driving conditions, and the attributes of the track points . The attribute of the track point may reflect the type of the track point, for example, a turning road section, a congested road section, near a school, and the like. The edge of the graph neural network model is the relationship between track points, and the features of the edge can include distance relationship, whether they appear on the same navigation route and/or whether they are on the same road segment, etc. The characteristics of the edges and the characteristics of the nodes are input into the graph neural network model, and the nodes of the graph neural network model output whether the corresponding track point is the target position.

As mentioned above, the first historical feedback information can be divided into multiple categories: feedback information of the user on the track points on the first navigation route in the past, feedback information of other users on the track points on the first navigation route in the past, and user past feedback information on the track points on the first navigation route. Feedback information for other locations (that is, locations other than track points on the first navigation route). In some embodiments, when determining the target position based on historical feedback information, weights may be set for different historical feedback information. The weight can be set according to the situation. For example, the user's feedback information on the track point has the highest weight in the past, so that the determined target position can be more in line with the user's needs.

In some embodiments of this specification, by determining the target position through historical feedback information and historical driving conditions, the track points on the navigation route being used by the vehicle that have been historically fed back or historically determined to have driving events can be determined as the target position, and then collected The user's feedback on the target position facilitates subsequent optimization of vehicle control to enhance the user's riding experience.

The embodiment of this specification also provides a user feedback device, the device includes at least one processor and at least one memory; the at least one memory is used to store computer instructions; the at least one processor is used to execute the computer instructions At least part of the instructions to implement the operations corresponding to the user feedback method described in any one of the preceding items.

The embodiment of this specification also provides a computer-readable storage medium, the storage medium stores computer instructions, and when the computer instructions are executed by a processor, the operation corresponding to the user feedback method as described in any one of the preceding claims is realized.

The basic concept has been described above, obviously, for those skilled in the art, the above detailed disclosure is only an example, and does not constitute a limitation to this description. Although not expressly stated here, those skilled in the art may make various modifications, improvements and corrections to this description. Such modifications, improvements and corrections are suggested in this specification, so such modifications, improvements and corrections still belong to the spirit and scope of the exemplary embodiments of this specification.

Meanwhile, this specification uses specific words to describe the embodiments of this specification. For example, "one embodiment", "an embodiment", and/or "some embodiments" refer to a certain feature, structure or characteristic related to at least one embodiment of this specification. Therefore, it should be emphasized and noted that two or more references to "an embodiment" or "an embodiment" or "an alternative embodiment" in different places in this specification do not necessarily refer to the same embodiment . In addition, certain features, structures or characteristics in one or more embodiments of this specification may be properly combined.

In addition, those skilled in the art will understand that various aspects of this specification can be illustrated and described by several patentable categories or situations, including any new and useful process, machine, product or combination of substances, or any combination of them Any new and useful improvements. Correspondingly, various aspects of this specification may be entirely executed by hardware, may be entirely executed by software (including firmware, resident software, microcode, etc.), or may be executed by a combination of hardware and software. The above hardware or software may be referred to as "block", "module", "engine", "unit", "component" or "system". Additionally, aspects of this specification may be embodied as a computer product comprising computer readable program code on one or more computer readable media.

A computer storage medium may contain a propagated data signal embodying a computer program code, for example, in baseband or as part of a carrier wave. The propagated signal may have various manifestations, including electromagnetic form, optical form, etc., or a suitable combination. A computer storage medium may be any computer-readable medium, other than a computer-readable storage medium, that can be used to communicate, propagate, or transfer a program for use by being coupled to an instruction execution system, apparatus, or device. Program code residing on a computer storage medium may be transmitted over any suitable medium, including radio, electrical cable, fiber optic cable, RF, or the like, or combinations of any of the foregoing.

The computer program codes required for the operation of each part of this manual can be written in any one or more programming languages, including object-oriented programming languages such as Java, Scala, Smalltalk, Eiffel, JADE, Emerald, C++, C#, VB.NET, Python etc., conventional programming languages such as C language, Visual Basic, Fortran2003, Perl, COBOL2002, PHP, ABAP, dynamic programming languages such as Python, Ruby and Groovy, or other programming languages. The program code may run entirely on the user's computer, or as a stand-alone software package, or run partly on the user's computer and partly on a remote computer, or entirely on the remote computer or processing device. In the latter case, the remote computer can be connected to the user computer through any form of network, such as a local area network (LAN) or wide area network (WAN), or to an external computer (such as through the Internet), or in a cloud computing environment, or as a service Use software as a service (SaaS).

In addition, unless explicitly stated in the claims, the order of processing elements and sequences described in this specification, the use of numbers and letters, or the use of other names are not used to limit the sequence of processes and methods in this specification. While the foregoing disclosure has discussed by way of various examples some embodiments of the invention that are presently believed to be useful, it should be understood that such detail is for illustrative purposes only and that the appended claims are not limited to the disclosed embodiments, but rather, the claims The claims are intended to cover all modifications and equivalent combinations that fall within the spirit and scope of the embodiments of this specification. For example, while the system components described above may be implemented as hardware devices, they may also be implemented as a software-only solution, such as installing the described system on an existing processing device or mobile device.

In the same way, it should be noted that in order to simplify the expression disclosed in this specification and help the understanding of one or more embodiments of the invention, in the foregoing description of the embodiments of this specification, sometimes multiple features are combined into one embodiment, drawings or descriptions thereof. This method of disclosure does not, however, imply that the subject matter of the specification requires more features than are recited in the claims. Indeed, embodiment features are less than all features of a single foregoing disclosed embodiment.

In some embodiments, numbers describing the quantity of components and attributes are used. It should be understood that such numbers used in the description of the embodiments use the modifiers "about", "approximately" or "substantially" in some examples. grooming. Unless otherwise stated, "about", "approximately" or "substantially" indicates that the stated figure allows for a variation of ±20%. Accordingly, in some embodiments, the numerical parameters used in the specification and claims are approximations that can vary depending upon the desired characteristics of individual embodiments. In some embodiments, numerical parameters should take into account the specified significant digits and adopt the general digit reservation method. Although the numerical ranges and parameters used in some embodiments of this specification to confirm the breadth of the range are approximations, in specific embodiments, such numerical values are set as precisely as practicable.

Each patent, patent application, patent application publication, and other material, such as article, book, specification, publication, document, etc., cited in this specification is hereby incorporated by reference in its entirety. Application history documents that are inconsistent with or conflict with the content of this specification are excluded, and documents (currently or later appended to this specification) that limit the broadest scope of the claims of this specification are also excluded. It should be noted that if there is any inconsistency or conflict between the descriptions, definitions, and/or terms used in the accompanying materials of this manual and the contents of this manual, the descriptions, definitions and/or terms used in this manual shall prevail .

Finally, it should be understood that the embodiments described in this specification are only used to illustrate the principles of the embodiments of this specification. Other modifications are also possible within the scope of this description. Therefore, by way of example and not limitation, alternative configurations of the embodiments of this specification may be considered consistent with the teachings of this specification. Accordingly, the embodiments of this specification are not limited to the embodiments explicitly introduced and described in this specification.

Claims (10)

1. A method of user feedback, comprising:

acquiring state information, wherein the state information comprises at least one of a vehicle state and a user state;

based on the state information, sending first reminding information to the user terminal;

and acquiring feedback information of the driving condition of the vehicle, which is input by the user at the user terminal in response to the first reminding information, wherein the vehicle is taken by the user.

2. The method of claim 1, the user status relating to one or more of the following information: vehicle usage by the user, idle status by the user, and mood by the user.

3. The method of claim 1, the vehicle state comprising at least one of a current vehicle state, a vehicle state after a preset time, and a vehicle state at a target location; the vehicle state is related to a running condition of the vehicle.

4. The method of claim 3, the driving condition comprising: at least one of a driving route, a driving location, a driving section, a driving environment, a driving state, and a driving parameter.

5. The method of claim 3, the target location being obtained by a prediction process comprising:

obtaining historical feedback information related to at least one of a track point on a first navigation route and the user and historical driving conditions corresponding to the historical feedback information; and

predicting the target location on the first navigation route based on the historical feedback information and the historical driving conditions, wherein,

the first navigation route is a navigation route being used by the vehicle.

6. The method of claim 1, wherein the sending the first alert message to the ue comprises:

determining a feedback reminding mode, wherein the feedback reminding mode is related to at least one of a user state, a user terminal state, a driving environment and user association information; and

and generating the first reminding information based on the feedback reminding mode, and sending the first reminding information to the user terminal.

7. The method of claim 1, the vehicle being an autonomous vehicle.

8. A system of user feedback, comprising:

the system comprises a first acquisition module, a second acquisition module and a control module, wherein the first acquisition module is used for acquiring state information, and the state information comprises at least one of a vehicle state and a user state;

the sending module is used for sending first reminding information to the user terminal based on the state information;

and the second acquisition module is used for acquiring feedback information of the driving condition of the vehicle, which is input by the user terminal in response to the first reminding information.

9. An apparatus for user feedback, the apparatus comprising at least one processor and at least one memory;

the at least one memory is for storing computer instructions;

the at least one processor is configured to execute at least some of the computer instructions to implement the method of any of claims 1 to 7.

10. A computer readable storage medium storing computer instructions which, when executed by a processor, implement the method of any one of claims 1 to 7.

Images